Betting on Results

Posted: November 12th, 2015 | Author: | | 1 Comment »

Alex pointed me to this on 538:

Today in the Proceedings of the National Academy of Sciences, Nosek and an international team of researchers present a tool for doing that — betting. They found that compared to simply asking experts to predict the likelihood that studies will be reproduced, asking them to bet money on the outcomes improved the accuracy of the guesses.

The researchers began by selecting some studies slated for replication in the Reproducibility Project: Psychology — a project that aimed to reproduce 100 studies published in three high-profile psychology journals in 2008. They then recruited psychology researchers to take part in two prediction markets. These are the same types of markets that people use to bet on who’s going to be president. In this case, though, researchers were betting on whether a study would replicate or not.

Before each prediction market began, participants (47 actively took part in the first market, 45 traded in the second) were asked two questions: How likely do you think it is that each hypothesis in this market will be replicated, and how well do you know this topic?

They were then given points worth a total of $100 to bet on whether the studies in their prediction market would replicate. A replication was considered successful if it produced a result, with a p-value of less than 0.05, in the same direction as the original result. Players entered the market with 10,000 points each and could buy and sell contracts for each hypothesis. If a replication succeeded, then its share paid 100, but if the replication failed, then it paid nothing. “If you believe the result will be replicated, you buy the contract, which increases the price,” said the study’s lead author, Anna Dreber, an economist at the Stockholm School of Economics. “If you don’t believe in a study, then you can short-sell it.”

….The prediction market correctly called nearly three-quarters (71 percent) of the attempted replications, 39 percent of which succeeded in the reproducibility project. By comparison, the survey conducted before the market began correctly predicted the result of only 58 percent of the replication studies. The prediction market anticipated a finding’s reproducibility better than asking the same bunch of experts to put their best guesses in a hat.

Our team has sometimes done that at Bridge. For example, we’ve had a pool for over a year to predict our kids’ KCPE scores. Not sure yet how we did!

“The beauty of the market is that we allow people to be Bayesian,” Dreber said. People come in with some prior belief, but they can also follow prices to see what other people believe and may update their beliefs accordingly. While the survey required everyone to provide an estimate for every study, participants in the market could focus their bets on the studies they felt most sure of, and as a result, rough guesses didn’t skew the averages as much.

Finally, putting money at stake motivated people to try harder to find the right answer and reveal what they really think. “It’s really putting your money where your mouth is,” Dreber said. “You want to see what people do, not what they say.”

I’m not sure if our “winner gets to pick restaurant, everyone else takes him/her out” is particularly motivating. But I do think we changed our picks to a degree as we noticed what others said, and were allowed to ask their rationale.

Better Turn And Talks (from Doug)

Posted: October 20th, 2015 | Author: | | No Comments »

A turn-and-talk is when the teacher has kids talk to each other in a small group. At Bridge, the most common version is the 3 kids who sit together on the same bench. It’s very hard to build a productive culture where the conversations kids have are valuable. We’ve struggled there. Our kids come from their old schools where they sit silently while the teacher talks; they typically only speak in “choral response.”

Doug Lemov profiles a teacher who is particularly good at turn-and-talk. Erin Krafft in Houston.

He writes:

The first thing we noticed was how beautifully she built a positive, productive and efficient procedure for Turn and Talk. In this clip you can see her essentially installing the Turn and Talk system she will use for the duration of the year. It’s a brilliant system that we hope you’ll steal! The system included guidance on how to agree or disagree within one’s partner within the Turn and Talk, and it fostered an incredibly powerful Culture of Error (discussed later) that enables more rigorous and productive student-to-student discussion.

Some key points:

Erin’s Turn and Talk began with clear “Managed Turns.” “Door partners” were instructed to speak first, and Erin checked with a show of hands to be sure that students were clear on which partner is “door partner.” Then it would be the “window partner’s turn.” This small detail is critical since the goal of Turn and Talk is to involve all students. In many classrooms students are sent to a series of Turn and Talks where some kids talk constantly and others talk not at all. Managing turns makes sure everyone talks. Read the rest of this entry »

Great News From the Developing World

Posted: October 1st, 2015 | Author: | | 3 Comments »

In the NY Times, Nick Kristof writes (hat tip Whitney Tilson):

• The number of extremely poor people (defined as those earning less than $1 or $1.25 a day, depending on who’s counting) rose inexorably until the middle of the 20th century, then roughly stabilized for a few decades. Since the 1990s, the number of poor has plummeted.

• In 1990, more than 12 million children died before the age of 5; this toll has since dropped by more than half.

• More kids than ever are becoming educated, especially girls. In the 1980s, only half of girls in developing countries completed elementary school; now, 80 percent do.

The most important thing going on in the world today is something we almost never cover: a rapid decline in poverty, illiteracy and disease.

Granted, some 16,000 children still die unnecessarily each day. It’s maddening in my travels to watch children dying simply because they were born in the wrong place at the wrong time.

But one reason for our current complacency is a feeling that poverty is inevitable — and that’s unwarranted.

The world’s best-kept secret is that we live at a historic inflection point when extreme poverty is retreating.

Read the whole thing here.

Cato Institute adds:

But, where did all the progress that Kristof talks about come from?

The Homo sapiens has been on this earth for 200,000 years. For 99.9% of that time, we lived in ignorance, poverty and misery. What has changed? Reading the NYT, the reader is left with the impression that “good stuff,” like manna from heaven, suddenly was conjured up out of thin air.

Not so. The key to the improvements in the lives of ordinary people over the last 200 years were industrialization and trade, which generated historically unprecedented rates of growth. And the importance of growth cannot be overemphasized. There is not a single example of a country emerging from widespread poverty without sustained economic growth. As University of Oxford Professor Paul Collier writes, “Growth is not a cure-all, but lack of growth is a kill-all.”

As a reader of Kristof’s column, I was not left with the impression that this progress was conjured up out of thin air.

But I agree with Cato that industrialization and trade are important drivers here.

I think private philanthropy has helped, particularly in health, and it would seem that Cato and Kristof agree here too.

I think getting more girls to attend school is perhaps less driven by industrialization. Some progress on supply-side issues and demand-side issues (like social norms). World Bank, maybe even the oft-maligned (by me included) UN, has likely had some role here influencing policy.

Unlike progress in school enrollment, however, I don’t think there is much good news on actual learning. I.e., more kids are spending more days in school (particularly girls) — good! But grimmer news on how much they learn (boys and girls) once in school (bad).

Just like in the USA, where policy can’t easily pierce schools to the day-to-day classroom reality of kids and teachers, so too in poor nations.

The 3 key limiting factors from inside a typical school in the developing world:

a. Strong strong strong pedagogical preference for “teacher lectures and writes some stuff on the board, while kids sit still, and copy notes.”

We’ve made progress on changing this at Bridge.

b. Virtually no reading of good fiction (by any definition) or non-textbook non-fiction. Related: If they do read, they typically can’t read and summarize the key ideas in their own words…it’s not even what they’re instinctively trying to do (to form meaning as they read, which until 2013 I thought was an intuitive thing to try to do, but I no longer think that).

We’re in early phases of trying to create some breakthroughs here.

c. Kids frequently don’t know how to form meaningful questions of their own, let alone ask them out loud.

This puzzle needs a lot more work. I’m interested in anyone who has seen breakthroughs here. Our early efforts have “bounced off” (unlike “a” above, where we’ve had success). We’re gearing up (resources permitting) for a new dive here in 2016.

Teach For India

Posted: September 4th, 2015 | Author: | | No Comments »

From Wendy Kopp:

In Delhi, Teach For India fellow Pooja Chopra taught 30 fourth-grade girls in a poor settlement. As she visited with her students and their families, she learned that, far too often, girls in the community were not expected to reach the highest levels of education.

Pooja oriented her teaching toward helping her students realize their dreams through education. She set out to foster their personal leadership and to ensure they would always be able to learn, regardless of circumstances. Pooja broke the class up into groups, allowing each group to choose its leader and set its rules. Within each group, every girl was assigned a buddy, whose learning she was also responsible for. Pooja guided the students through their lessons, made sure that no group or pair fell behind, and gave them the agency to debate questions and determine their answers independently.

As Pooja developed relationships with the girls’ parents, she discovered that many of the mothers had no financial means to support their daughters’ educations. To enable them to partner in their daughters’ schooling, she helped them launch a small tailoring business. Inside and outside of her classroom, Pooja’s leadership is setting her students on new life paths. By the end of the school year, her students showed an incredible 1.7 years of growth and were exhibiting new confidence, initiative, and independence.

You can read the whole thing here.

Teach For India, just like Teach For America, has at least 2 big effects:

a. The direct effect on kids when a teacher like Pooja helps 30 kids.

b. The larger effect when lots of Teach For India alumni create an education reform movement that is focused on school quality — the outcomes (what kids learn), rather than the inputs (what is spent, or what philosophy is used).

With Bridge now recruiting for a team to launch our first schools in India, it’s easy to see what a powerful force the Teach For India alumni are — they’re results-driven, humble.

High-Dosage Tutoring, Kenya-Style

Posted: February 22nd, 2015 | Author: | | 2 Comments »

No Match Corps available here. Sixth graders tutoring 1st graders. Every day for 40 minutes, in math.


From Snow To Hailstones

Posted: February 17th, 2015 | Author: | | No Comments »

On Saturday, I was on one of the last flights out of Boston, escaping the latest blizzard. My kids stuck around and played in the snow. image2

Meanwhile, it’s rainy season here in Nairobi. But I didn’t expect the hailstones we got yesterday!



Kids were preparing for midterm exams. Geordie (who took the photos) and I spent some time visiting with teachers, trying to gauge how it was going. Came away with a long list of notes on little things we can do better in terms of curriculum.


Little things can add up to big things!


Omitted photo: Back at the house, my colleague Carol had left out 2 bananas by the window sill, to ripen. They were stolen by a passing monkey — no joke. Meanwhile, I stole some bottled water from another colleague. Monkey see, monkey do.

Bridge in the News

Posted: February 3rd, 2015 | Author: | | No Comments »

Hi folks –

A few news stories about Bridge.

1. CNBC video here about our co-founders.

Accompanying article

2. Fast Company here about:

The Girl Effect Accelerator, a partnership between the Nike Foundation and Unreasonable Group, has a singular mission: to scale up projects in the developing world that benefit girls in poverty. Launched in November 2014, the two-week program offered mentorship, financing, and networking to 10 for-profit projects, with the ultimate goal of rapidly scaling them up.

The Girl Effect Accelerator is an investor in Bridge.

3. Last, the Bill and Melinda Gates 2015 Annual Letter has a photo of Bridge (they’re investors). It’s worth reading for a broad view of progress and challenges in the developing world (as well as USA ed reform).

Tracking In India

Posted: January 23rd, 2015 | Author: | | 2 Comments »

In my first post on this blog back in July 2013, I wrote about meeting MIT economist Esther Duflo.

I noticed today she co-wrote an Op-Ed for the Indian Express, with colleague Rukmini Banerji. It’s called Let’s Remake The Classroom.

The basic idea?

Take all the kids in Grades 3, 4, 5; reorganize them into reading groups and math groups for some of the school day; teach lessons slated to the “right level” of the kids.

That way, a top 3rd grader is with other strong pupils getting “the right level” of challenge. Similarly, a struggling 5th grader would have an hour to be with other strugglers, and therefore also “at the right level.”

In the US context, this is called tracking, and Read the rest of this entry »

The Problem of Using SDs to Compare Effect Sizes

Posted: January 14th, 2015 | Author: | | 2 Comments »

Ryan suggested yesterday that I check out this blog by Abhijeet Singh. It’s over at World Bank blog.

Title? “How standard is a standard deviation? A cautionary note on using SDs to compare across impact evaluations in education.”

C’mon. If that doesn’t get you excited, then you’re not geeky enough. Read the rest of this entry »

Measuring Student Growth In India

Posted: January 13th, 2015 | Author: | | 2 Comments »

Michael Dell made a fortune on computers. He and his wife Susan set up a foundation, MSDF, to give away a ton of dough to ed reform causes.

They just released a report. It’s about Assessment in India. Good stuff.

MSDF has donated more than $130 million in India. But do their investments pay off for Read the rest of this entry »