Unconventional but effective strategies for public education can provide significant advances in student achievement nationwide.
TODAY I want to talk about inequality in the 21st century, in particular on the decline in the significance of discrimination and the increase in the significance of human capital.
Let me start with some basic facts about the achievement gap in America. If you listen to NPR or tune into 60 Minutes, you probably get a sense that the United States is lagging behind other countries in student achievement and that there is a disturbing difference in the performance of racial groups.
For example, on average 44% of all students, regardless of race, are proficient in math or reading in 8th grade. That’s disheartening, but far from the worst news. In Detroit, 3% of black 8th graders are considered proficient in math—that’s 3%. In some places, such as Cleveland, the achievement gap between white and black students is relatively small, but the reason is that the white students are not doing well either. In the District of Columbia, roughly 80% of white 8th graders, but only 8% of their black classmates, are proficient in math.
Many people will object that test scores do not measure the whole child. That’s true, but I will argue that they are important.
My early training and research in economics was not linked to education, but I was asked in 2003 to explore the reasons for the social inequality in the United States. I began by looking at the National Longitudinal Survey of Youth, focusing on people who were then 40 years old. Compared to their white contemporaries, blacks earned 28% less, were 27% less likely to have attended college, were 190% more likely to be unemployed, and 141% more likely to have been on public assistance. These grim statistics are well known and are often used to illustrate the power of racial bias in U.S. society.
I decided to trace back through the lives of this cohort to try to identify the source of these disparities. One obvious place to look was educational achievement. I went back to the test scores of this cohort when they were in 8th grade and did some calculations. If one compared blacks and whites who had the same test scores in 8th grade, the picture at age 40 was dramatically different. The difference in wages was 0.6%, the difference in unemployment was 90%, the difference in public assistance was 33%, and blacks were actually 137% more likely to have attended college.
That was easy. In two weeks I reported back that achievement gaps that were evident at an early age correlated with many of the social disparities that appeared later in life. I thought I was done. But the logical follow-up question was how to explain the achievement gap that was apparent in 8th grade. I’ve been working on that question for the past 10 years.
I am certainly not going to tell you that discrimination has been purged from U.S. culture, but I do believe that these data suggest that differences in student achievement are a critical factor in explaining many of the black-white disparities in our society. It is no longer news that the United States is a lackluster performer on international comparisons of student achievement, ranking about 20th in the world. But the position of U.S. black students is truly alarming. If they were to be considered a country, they would rank just below Mexico in last place among all Organization of Economic Cooperation and Development countries.
How did it get this way? When do U.S. black students start falling behind? It turns out that development psychologists can begin assessing cognitive capacity of children when they are only nine months old with the Bayley Scale of Infant Development. We examined data that had been collected on a representative sample of 11,000 children and could find no difference in performance of racial groups. But by age two, one can detect a gap opening, which becomes larger with each passing year. By age five, black children trail their white peers by 8 months in cognitive performance, and by eighth grade the gap has widened to twelve months
Remember, Horace Mann told us that public education was going to be the great equalizer; it was going to compensate for the inequality caused by differences in income across zip codes. That was the dream.
Unfortunately, what happens is that the inequality that exists when children begin school becomes even greater during schooling. The gap grows not only across schools, but within the same school, even with the same teacher. This means that even for children from the same neighborhood, the same school, and the same teachers, academic performance diverges each year in school.
I spent two or three years trying to figure out what factors could explain this predicament. I looked at whether or not teachers were biased against some kids or groups. I looked at whether or not kids lost ground during the summer. I looked at various measures of school quality. I looked at the results of numerous different types of standardized test. None of these could explain why certain groups, blacks in particular, were losing ground to their peers.
TO REINFORCE HIGH EXPECTATIONS, WE AIMED TO CREATE AN ENVIRONMENT THAT REFLECTED SERIOUSNESS. WE ELIMINATED GRAFFITI AND REMOVED THE BARBED WIRE THAT SURROUNDED SOME OF THE SCHOOLS. WE REGULARLY REPEATED THE GOALS THAT WE EXPECTED STUDENTS TO ACHIEVE.
When I was presenting this finding at a meeting, a woman challenged me to stop focusing on our failures and to let audiences know what works. I said “OK, but what works?” She said more education for teachers, increased funding, smaller class size. I recognized this as the conventional wisdom, but I thought I better examine the data that demonstrate that these strategies are effective.
I discovered that we have actually implemented this approach for many decades. The percentage of teachers with a master’s degree increased from 23% in 1961 to 62% in 2006. The average class size has declined from 22 to 16 students since 1970. Per pupil annual spending grew from $5,000 in 1970 to $12,000 in 2008 in constant dollars. In spite of applying this apparently sound advice, overall student academic achievement has remained essentially flat. Clearly, we need to try something else.
As befits an arrogant economist, my first thought was that this will be easy: We just have to change the incentives. Let’s apply a rational agent model and examine the calculation we are asking students to make. Society is telling them that they will be rewarded for their efforts in school in 15 years when they enter the labor market. As an economist I know that no one has a discount rate that would justify waiting 15 years for a payoff. My solution was to propose that we pay them incentives now to reward good school performance.
Oh my gosh, I wish someone had warned me. No one told me this was going to be so incredibly unpopular. People were picketing me outside my house saying I would destroy students’ love of learning, that I was the worst thing for black people since the Tuskegee experiments. Really? Experimenting with incentives when nothing else seems to work is the equivalent of injecting people with syphilis without informing them?
We decided to try the experiment and raised about $10 million. We provided incentives in Dallas, Houston, Washington, DC, New York, and Chicago. We also, just for fun, added a large experiment with teacher incentives just to cover all our bases, to make sure that we had paid everybody for everything.
The question for us was, first of all, could incentives increase achievement? Second, what should we pay for and how should we structure the incentives? The conventional economic theory is that we should pay for outputs. It follows from that—don’t laugh—that kids should borrow money based on their expected future earnings to pay for tutors or make other investments in their learning to improve their performance. We took a more direct approach, conducting randomized trials that primarily paid for inputs.
In Dallas we paid kids $2 for each book they read. They had to take a test to verify that they actually did the reading. In Houston we paid kids to do their math homework. In Washington we paid kids to attend school, complete homework, score well on tests, and avoid activities such as fighting. We also tried incentives for outputs. In New York we paid kids for good test scores so that the emphasis was completely on outputs. In Chicago we paid ninth graders half the money for attendance and the second half for graduation. The amounts were generous for poor kids. A Washington middle schooler could earn as much as $2,000 per year. In New York, fourth grades could make up to $250 and seventh graders up to $500.
Throughout the experiment we were bombarded with complaints from adults, particularly those who did not have children in the experiment. We never had a kid complain. Well, once we did. I came to one Washington school to participate in a ceremony at which checks were distributed. Before the event started, one kid came up to me and said, “Professor, I don’t think we should be paid to come to school. I think we should pay to come to school because school is such a valuable resource. You should not pay us. We should pay you.”
I was blown away by this. I thought this kid really gets it. About 20 minutes later I was distributing checks in the cafeteria. Kids names were called, and they ran or danced to the front of the room. I called the kid’s name, and he came up. I put his check in my pocket. He said, “What are you doing?” I told him that just 20 minutes earlier he had told me that he should pay me for the privilege of coming to school. He looked at me in a way that only an 11-year-old can and said, “I never said that.”
We found that incentives, if designed correctly, can have a positive return on investment. However, they are not going to close the big gaps that exist between blacks and whites. We did learn that it is more effective to provide kids with incentives for inputs rather than outputs. This contradicts what I learned in my economics training, but it was very clear when I actually talked to the kids. I asked one kid in Chicago, where they were paid for outputs, Did you stay after school and ask your teacher for extra time? No. Did you borrow against your expected income and hire a tutor? No. What did you do? Basically, I came. I tried harder. School was still hard. At some point, I gave up.
The reality is that most of these kids do not know how to get from point A to point B. The assumption that economists make when designing incentives is that people know how to produce the desired output, that they know the “production function.” When they don’t know that, designing incentives is incredibly difficult.
What we learned through this $10 million and a lot of negative press and angry citizens is that kids will respond to incentives—and that incentives to teachers do not have a significant effect on student achievement. They will do exactly what you want them to do. By the way, they don’t do anything extra either. I had this idea that they were going to discover that school is great and to try harder in all of their subjects, even those that do not provide incentives. No. You offer $2 to read a book, and they read a book. They are going to do exactly what you want them to do. That showed me the power, and the limitations, of incentives for kids. I saw that if you really squinted and designed them perfectly, incentives would have a high return on investment because they are so cheap, but they were never going to close the gap.
Something new and different
At the same time I was writing up my incentives paper, I started doing the analysis of Geoffrey Canada’s work in the Harlem Children’s Zone. This changed my entire research trajectory.
With the help of large philanthropic contributions, Canada had developed a creative and ambitious approach to education. A group of Harlem students were randomly selected to attend Canada’s charter schools beginning in 6th grade. A couple of things are important here. One, the lottery winners and losers were, if anything, slightly below the New York City average. This is significant because the students that enroll in charter schools are often above-average achievers from the start.
The evidence of improvement can be seen in the first year, and the gains are even better in the second year. By year three, these students have essentially reached the level of the average white New York student.
Now, I haven’t controlled for anything. If I were to include factors such as eligibility for free lunch, the black students would be slightly outperforming the white students. Their performance in reading improved but not nearly as much as it did in math. I would summarize the results in these simple terms: After three years in Canada’s Promise Academy Charter Schools, the students were able to erase the achievement gap in math and to cut it by a third in reading.
I had never seen results that came close to this. When I first saw the numbers, I thought my research assistant had made a coding error. This was a reason to get excited about the possibility of make a big difference in children’s lives.
Further research into public charter schools enabled me to see that this not just about the Harlem Children’s Zone. Although the average charter school is statistically no better than the average regular public school, there are a number of charter schools achieving the type of results we found in the Harlem Children’s Zone. The research challenge is to identify what they are doing that works.
Let me stop for a story. My grandmother makes a fabulous coconut cake, so I asked her for the recipe. She told me what she does with a finger full of this and palm full of that. When I tried it, the result was a cement block, so I decided that the only way to learn the recipe was to watch her make it. When she grabbed a palm of coconut flakes, I made her put it in a measuring cup. For your future reference, a grandmother’s palm is equal to a quarter cup. It took a long time and annoyed my grandmother, but now I have a recipe I can use and pass down to my children.
If you ask Geoffrey Canada what’s in his secret education sauce, he will say a little bit of this, a little bit of that. You will be moved by his powerfully inspirational speeches, but you will not learn how to build a better school. You’ll just wish that you were also a genius.
To help the rest of us who are not geniuses, we assembled a research team that spent two years examining in detail what was happening at charter schools, some good and some not so good. We hung around. We used video cameras. We interviewed the kids. We interviewed the teachers. We interviewed the principals. We spent hours in these schools trying to figure out what the good ones did and what the not-so-good ones didn’t do.
We found a number of practices that were clearly correlated with better student performance. For teachers, it is important that they receive reliable feedback on their classroom performance and that they rigorously apply what they learn from assessments of their students to what they do in the curriculum and the classroom.
Even low-performing schools know that data are important. When I visited a middling school, they would be eager to show me their data room. What I typically found was wall charts with an array of green, yellow, and red stickers that represented high-, mid-, and low-performing students, respectively. And when I asked what has this led you to do for red kids, they would say that they hadn’t reached that step yet, but at least they knew how many there are.
When I asked the same question in the data rooms of high-performing schools, they would say that they have their teaching calibrated for the three blocks. They would not only identify which students were trailing behind, but would identify the pattern of specific deficiencies and then provide remediation for two or three days on the problem areas. They would also note the need to approach these areas more diligently in future editions of the course.
The third effective practice was what I call tutoring, but which those in the know call small learning communities. It is tutoring. Basically what they do is work with kids in groups of six or fewer at least four days per year.
The fourth ingredient was instructional time. Simple. Effective schools just spent more time on tasks. I think of it as the basic physics of education. If your students are falling behind, you have two choices: spend more time in school or convince the high-performing schools to give their kids four-day weekends. The key is to change the ratio.
The icing on the cake was that effective schools had very, very high expectations of achievement regardless of their social or economic background. My father went to prison when I was a kid. I didn’t meet my mother until I was in my twenties. Fortunately, I had a grandmother who didn’t know the meaning of the word excuse. A high school counselor who was aware of my situation tried to help me by saying that I could be part of a special program that would require only a half day of school and reduce my work load. I knew my grandmother wouldn’t buy that, so I refused.
The essential finding is that kids will live up or down to our expectations. Of course they are dealing with poverty. Of course 90% of the kids have single female head of households. They all have that. That wasn’t news. The question is how are we going to educate them?
We met incredible educators who not only understood the big picture but sweated all the details. One principal had developed a very clever and efficient method for distributing worksheets, exams, and other handouts in class. I’ve never worried about that, so I asked what was the point. She said that every teacher does this in every class many times a day. If we can save 30 seconds each time, we will add several days of productive class time over the course of a year, and these kids need every minute we can give them.
Testing the thesis
I believe there is real value in analyzing the data that provides the evidence that these five strategies work, but there is nothing very surprising or counterintuitive in the findings. The question is why so few schools are implementing these practices.
We set out to discover if there was any reason that public schools could not implement these practices and achieve the expected results. We approached a number of school districts to ask if we could conduct an experiment applying these techniques in some of their schools. I won’t belabor all the reasons we heard for why it was impossible, but suffice it to say that we were not welcomed with open arms. Apparently, it is not practical to increase time in school, provide tutoring, give teachers regular feedback and guidance, use data to inform instructional practice, and increase expectations.
We did eventually find a willing partner in the Houston school district, where the superintendent and the school board were willing to give it a try. We began to work in 20 schools, including four high schools, with a total of 16,000 students. These are traditional public schools. There is no waiting list. There is no sign up. There is no Superman. Nothing complicated. These are just ordinary neighborhood public schools.
All of the schools were performing below expectations and were in line to be taken over by the state. They qualified for the federal dollars for turning schools around. As part of that program, all of the principals and about half the teachers were replaced.
We increased the school day by one hour. We lengthened the school year by two weeks. We also cut down on some curious non-instructional activities. We discovered, for example, that 20 minutes is set aside each day for bathroom breaks. For no additional cost you can increase instructional time just by making kids pee more quickly. How cool is that?
I SAW THAT IF YOU REALLY SQUINTED AND DESIGNED THEM PERFECTLY, INCENTIVES WOULD HAVE A HIGH RETURN ON INVESTMENT BECAUSE THEY ARE SO CHEAP, BUT THEY WERE NEVER GOING TO CLOSE THE GAP.
Second, small group tutoring. We hired more than 400 full-time tutors. They worked with ten kids a day, two at a time during five of the day’s six periods. We offered a $20,000 salary even though we were told that no one would do the job for that amount. In five weeks, we had 1,200 applications. Some were young Teach for America types. Others were retirees from the Johnson Space Center. We decided to focus on math tutoring in what we had found were the critical fourth, sixth, and ninth grades.
For data-driven instruction, we worked with the existing requirements for the Houston schools. For example, Houston sets 212 objectives that fifth graders are expected to achieve. We designed a schedule that would make it possible to reach all the objectives while also including remediation for students and professional development for teachers. A feedback system was designed that resulted in teachers receiving ten times as much feedback as teachers in other Houston schools.
To reinforce high expectations, we aimed to create an environment that reflected seriousness. We eliminated graffiti and removed the barbed wire that surrounded some of the schools. We regularly repeated the goals that we expected students to achieve.
The experiment had a couple of potential fault lines. One, we were taking best practices out of charter schools and trying to implement them in traditional public schools. It could be that those best practices work only with a set of highly motivated teachers and parents. We weren’t sure about that. Second, we had to face all the political realities of a traditional public school. During the three-year experiment, I aged about 24 years. I will never be the same.
But the results made it worth the effort. When we began, the black/white achievement gap in the elementary schools was about 0.4 standard deviations, which is equivalent to about 5 months. Over the three years, our elementary schools essentially eliminated the gap in math and made some progress in reading. In secondary schools, math scores rose at a rate that would close the gap in in roughly four to five years, but there was no improvement in reading. One other significant result was that 100% of the high school graduates were accepted to a two- or four-year college.
Let me put it in context for you. The improvement in student achievement in the Houston schools where we worked was roughly equivalent to the results in the Harlem Children’s Zone and in the average KIPP charter school. But we did this with 16,000 kids in traditional public schools. We are now repeating the experiment in Denver, Colorado, and Springfield, Massachusetts. We actually do know what to do, especially for math. The question is whether or not we have the courage to do it.
The last thing I will show you is a return on investment calculation for a variety of interventions. We calculated what a given level of improvement in achievement would mean for a student’s lifetime earnings and what that would mean for government income tax revenue. Reducing class size costs about $3,500 per kid and results in an ROI of about 6.2%, which is better than the long-term stock market return of about 5%. Expanded early childhood education has an ROI of 7.6%, an even better investment.
“No excuses” charter schools cost about $2,500 per kid and have an ROI of 18.5%. Using the same methodology, we calculated that the investment in our Houston schools had an ROI of 13.4% in the secondary schools and 26.7% in the elementary schools. But that was based on the implementation cost, which I raised from private sources. Houston did not spend anything more per student, so its ROI was infinite.
My journey into education has been similar to that of many other people. I was frustrated with the data, frustrated that we didn’t know which of the scores of innovations were most effective. We took the simple approach of looking closely at the schools that were producing the results we all want to see.
We found five actions that explain roughly 50% of the variation among charter schools. We then conducted an experiment to see if those same five actions would have the same result in a typical urban public school system. The results are truly encouraging. In three years these public school students made remarkable progress in math achievement and some improvement in reading. That’s not everything, but it is far more than what was achieved in decades with the conventional wisdom of smaller classes, more teacher certification, and increased spending.
It is not rocket science. It is not magic. There is nothing special about it. When the film Waiting for Superman came out, people complained that the nation is undersupplied with supermen. But an ordinary nerd like me was able to uncover a simple and readily repeated recipe for progress. Anyone can do this stuff.
One last story. During the experiment in Houston, an education commissioner from another state came to tour Robinson elementary school, one of the toughest in the city. He knew Houston and was familiar with Robinson. At the end of the tour, he pulled me aside. He had one question: “Where did you move the kids who used to go to school here?” I said that these are all the same kids, but they behave a lot differently when we do our jobs properly. They are listening. They are learning. They will live up to the expectations that we have for them.
I was a kid who went to broken schools. Thanks to my grandmother and some good luck, I beat the odds. But one success story is not what we want. What we want are rigorously evaluated, replicable, systematic educational practices that will change the odds.
Roland Fryer is Robert M. Beren Professor of Economics and faculty director of the Education Innovation Laboratory at Harvard University. This article is adapted from the Henry and Bryna David Lecture, which he delivered at the National Academy of Sciences on April 29, 2014.