In a National Bureau of Economics Research (NBER) white paper from March on 鈥淭aking Teacher Evaluation to Scale,鈥� five researchers offer a bottom line on the teacher-evaluation push that loomed so large in the Obama era. They , with high statistical confidence, that the effort had no meaningful impact on student outcomes (regardless of the specific program design features, relevant student characteristics, or the local context).
For those who recall Race to the Top, federal dollars and directives, the Gates Foundation鈥檚 intense Measures of Effective Teaching push, grandiose state plans, the L.A. Times鈥� massive name-by-name look at teacher value-added scores, and the intense teacher-evaluation fights of the late aughts and early 2010s, the whole thing is a cautionary tale. Of course, none of this should be a surprise by now. After all, Brown University鈥檚 Matt Kraft (one of the co-authors of the new paper) has previously shown that nothing of import actually changed as a result of new teacher-evaluation laws. And RAND鈥檚 extensive evaluation of the Gates Foundation鈥檚 half-billion dollar effort on teacher evaluation registered a .
In the new NBER paper, Josh Bleiberg and his colleagues offer some thoughts as to the familiar factors that help explain what happened鈥攊ncluding political opposition and the U.S.鈥檚 decentralized system of public education. Of course, none of that stuff should be at all surprising. Indeed, these challenges and the problem of taking reform to scale is an old one (see, for instance, Dick Elmore鈥檚 classic 1996 ).
This well-worn frustration is responsible for what鈥檚 been a recurring theme of this blog for 13 years: the conviction that it鈥檚 crucial to challenge the heedless enthusiasm, moral certitude, and blind confidence that looms so large in the DNA of school improvement.
As I observed several years ago, in , 鈥淧olicy can make people do things but it can鈥檛 make them do them well. Policy is a blunt tool that works best when making people do things is enough.鈥� Education policies are most likely to deliver the hoped-for results when dealing with 鈥渕usts鈥� and 鈥渕ust nots,鈥� as with things like compulsory attendance, required annual assessments, class-size limits, and graduation requirements.
Unfortunately, as I noted in Letters, 鈥淧olicy is far less effective when it comes to complex endeavors where how things are done matters more than whether they鈥檙e done. This is because policy can鈥檛 make schools or systems adopt reforms wisely or well.鈥� That鈥檚 why advocates promoting social-emotional learning requirements, 鈥渞estorative鈥� disciplinary policies, career and technical education directives, educational savings accounts鈥攐r teacher-evaluation systems鈥攏eed to be prepared for teeth-rattling bumps.
I want to be clear: The bumps (usually) aren鈥檛 due to ill intent on anybody鈥檚 part but to a series of banal factors. Educators in a given school or system may not be that invested in the effort. They may not know how to do it. Any training they receive may be slapdash, mediocre, or insufficient. Students or families in some locales may not like the measures. And, as the NBER paper authors note, proposals will encounter opposition (shocker!) or may flounder amid the byways of our decentralized system.
When improvement efforts don鈥檛 work out, those who pushed the change have the unlovely habit of acting as if no one could have anticipated the challenges that bedevil them鈥攕ounding a lot like a kid who leaves his new bike outside and unlocked and then gets furious when it鈥檚 stolen. Frustrated would-be reformers proceed to blame their frustrations on everyone else: parents, politicians, textbook publishers, educators, bike thieves. You name it.
There鈥檚 a tendency to insist that their idea was swell and that any issues are just 鈥渋mplementation problems.鈥� Calling something an 鈥渋mplementation problem鈥� is how those who dreamed up an improvement scheme let themselves off the hook. It鈥檚 a fancy way to avoid acknowledging their failure to anticipate predictable problems.
The upshot is that they didn鈥檛 realize how their idea would work in practice, when adopted by lots of real people in lots of real schools ... and it turned out worse than they鈥檇 hoped.
I鈥檝e said it many times before and I鈥檒l say it again: There鈥檚 no such thing as an 鈥渋mplementation problem.鈥� What matters in schooling is what actually happens to 50 million kids in 100,000 schools. That鈥檚 all implementation.
Responsible advocates and change agents prepare accordingly. They know that the measure of their idea is not how promising it seems in theory but how it works in practice. That鈥檚 a test that would-be reformers have too often failed. Going forward, whether we鈥檙e talking about SEL or education savings accounts, we need to do better. On that count, the teacher-evaluation boomlet has valuable lessons to teach.
