Guest post by John Thompson.
Most states accepted the offer they couldn鈥檛 refuse and adopted laws based of the Gates Foundation鈥檚 untested 鈥渢eacher quality鈥� theory. Two recent responses to that top-down mandate are illuminating. The more literary statement was issued by economist , the head of the Gates Measures for Effective Teaching (MET) project. A scholarly analysis, 鈥� was published by Morgan Polikoff and Andrew Porter.
In contrast to Kane鈥檚 eccentric diagnosis of education鈥檚 shortcomings, Polikoff鈥檚 and Porter鈥檚 peer reviewed study is one of many social science studies that are giving a fuller picture of the many problems with the Gates鈥� and Kane鈥檚 policy gamble.
As he often does, Kane begins will a clueless metaphor. He said that previous, incremental school reforms were like 鈥渟hooting a bottle rocket to the moon.鈥� Because educators have never been empowered in ways that would produce transformational change, Kane believes that he and other corporate reformers should take over and design an educational Race to the Moon.
A better metaphor for the Kane blueprint could be borrowed from the 1950s television show, 鈥淭he Honeymooners,鈥� as when Ralph Kramden proclaims, 鈥淎lice, Straight to the Moon!鈥� Like Jackie Gleason, the corporate response to dissent is 鈥淏ANG, ZOOM!,鈥� and a punch to the kisser. Because teachers, armed with bottle rockets, did not produce the education equivalent of a Moon landing in a decade, Kane demands that high-stakes testing be used as a club to get our attention and to propel our schools into outer space.
Kane ignores the obvious flaw in his logic. When NASA planned a flight to the moon, he argues, they realized that 鈥渋ncremental improvements in rocket design would not provide enough lift鈥� to get astronauts into orbit. But Kane ignores what it would take to lay the educational foundations for such a grand effort. He鈥檚 silent on the policies - such as high-quality early education and a well-designed campaign to teach reading for comprehension by 3rd grade - that have the potential for propelling our schools into new levels of excellence. He forgets about what it takes to launch a rocket, as he seeks to be the Ground Control inventor of systems that he pretends are already capable of flight.
Then, Kane鈥檚 hypotheses get downright weird. He acknowledges that each of the MET policies that he promotes is under-powered. Firing bad teachers, better professional development, and better standards of instruction each get anemic results. They will only work as a package. So, educators have to accept them all!
In other words, Kane selected the wrong metaphor. He鈥檚 like the old-time merchant who loses money on everything he sells - but makes it up on volume!
In contrast, as explained by 91制片厂视频 Week鈥檚 Holly Yettick, Polikoff and Porter:
Found no association between value-added results and other widely accepted measures of teaching quality, such as the degree to which instruction is aligned with state standards or the contents of assessments. Nor did the study find associations between "multiple measure" ratings, which combine value-added measures with observations and other factors.
The study also raises questions about the cornerstone of reformers鈥� theories by casting doubt on whether a focus on standards is the proper lever for improving instruction. That issue deserves a full analysis beyond the scope of this post.
To the Gates Foundation鈥檚 credit, it supported the Polikoff and Porter study with a $125,000 grant. The research drew on a subset of MET data from 327 4th and 8th grade teachers in six school districts.
Yettick reports that 鈥淢r. Polikoff suggested that the study鈥檚 findings could represent something like the worst-case scenario for the correlation between value-added scores and other measures of instructional quality.鈥� Proponents of the Gates-favored evaluations should be alarmed. Its correlations are 鈥渮ero or near-zero.鈥�
The MET, for reasons that still escape me, investigated whether, on the average, there are correlations between value-added and other indicators of effective teaching. But, the policies spawned by the MET seek to fire individual teachers for so-called 鈥渋苍别蹿蹿别肠迟颈惫别苍别蝉蝉.鈥� (Emphasis mine) Nowhere does the MET address the policy issue of whether its estimate of ineffectiveness is valid for determining whether the individual is a good teacher or not.
Polikoff and Porter use a MET subsample and methodology to drill down to the policy questions that should have been addressed before risky new teacher evaluations experiments were imposed. They find, 鈥淚n some places, [value-added measures] and observational scores will be correlated, and in some places they won鈥檛.鈥� He wrote in an email, 鈥淭hese correlations will depend on things like the content and quality of the test, the type of [measures] used, the properties of the observation system ...鈥�
Yettick then summarizes other peer-refereed studies that reach similar conclusions. Douglass Harris, in the American 91制片厂视频al Research Journal, reports that principals鈥� ratings and their value-added ratings of teachers were 鈥渙nly weakly correlated.鈥� Harris formulates an insightful metaphor to explain why 鈥渋t is no surprise鈥� the observations and value-added reach different outcomes, 鈥淵ou can think of principal evaluations and value-added as measuring two different elements of 鈥榪uality instruction鈥� in the same way that temperature and humidity are two key elements of quality weather.鈥�
Also in the explain why value-added models fall short, even when they try to account for nonrandom classroom assignment. Real world, in contrast to the rarified experimental conditions in the MET process, supporters of value-added 鈥渢ypically fail to consider behavior, personality, and other factors that profoundly influenced the classroom-assignment decisions.鈥� That weakness can bias value-added results.
Another forthcoming study in the peer-refereed Economics of 91制片厂视频 Review, led by Brigham Young鈥檚 Scott Condie, draws on reading and math scores from a massive sample of 1.3 million students over six years. Condie et. al 鈥渇ound that between 15 percent and 25 percent of teachers misranked.鈥� Moreover, it sounds like the study documents a dynamic where, metaphorically, MET supporters would evaluate (and fire) teachers based on humidity metrics when practitioners would prefer to use the temperature, or vica versa. Condie explains, "[A] teacher who is fired may generate more social value than one who is retained.鈥�
Yettick also reviews the work of Marianne P. Bitler of the University of California. Bitler et. al discover an even better metaphor. Holding teachers accountable for test score growth, as estimated by value-added models, is like holding educators accountability about how much taller their students grow during the school year. They 鈥渇ound that teachers鈥� one-year 鈥榚ffects鈥� on student height were nearly as large as their effects on reading and math.鈥�
So it is up to the Gates Foundation to decide whether it defends its quest for a better bottle rocket to drive school reform, or to heed the research of independent scholars. It can continue to remain supportive of Tom Kane鈥檚 continuing search for a metaphor to justify their overreach, or start to consider evidence-driven school improvement policies.
What do you think? When Gates-funded, peer reviewed research contradicts the opinions the opinions it favors, will they adjust? Will the Gates Foundation explicitly support the efforts of educators to back off from value-added evaluations? And which side has the better metaphors?
John Thompson was an award winning historian, with a doctorate from Rutgers, and a legislative lobbyist when crack and gangs hit his neighborhood, and he became an inner city teacher. He blogs for This Week in 91制片厂视频, the Huffington Post and other sites. After 18 years in the classroom, he is writing his book, Getting Schooled: Battles Inside and Outside the Urban Classroom.