The figures are resoundingly familiar.
In Michigan, 98 percent of teachers were rated effective or better under new teacher-evaluation systems recently put in place. In Florida, 97 percent of teachers were deemed effective or better .
Principals in Tennessee judged 98 percent of teachers to be 鈥渁t expectations鈥� or better last school year, while evaluators in Georgia gave good reviews to 94 percent of teachers taking part in a pilot evaluation program.
Those results, among the first trickling out from states鈥� newly revamped yardsticks, paint a picture of a K-12 system that remains hesitant to differentiate between the best and the weakest performers鈥攁s well as among all those in the middle doing a solid job who still have room to improve.
The data are also raising new questions about the observation components of the systems, which tended to produce the highest scores.
鈥淢ost of these districts are trying really hard to think about teacher evaluation in a good way and to use it developmentally, but there鈥檚 still some cultural challenges,鈥� Sarah W. Lenhoff, the assistant director of policy and research at the 91制片厂视频 Trust-Midwest, said about the Michigan results. The Royal Oak, Mich.-based advocacy group has issued several reports on the overhaul of evaluation in that state.
鈥淎lthough teachers may be getting more feedback and talking about their practice more, it hasn鈥檛 trickled down to variations in rating,鈥� Ms. Lenhoff continued. 鈥淎nd that鈥檚 going to take time.鈥�
Some teachers鈥� unions, though, see the data as an affirmation of teachers鈥� hard work.
鈥淒espite all the rhetoric blaming teachers for all the problems in education, most teachers are doing a good job, given their limited resources,鈥� said Doug Pratt, a spokesman for the Michigan 91制片厂视频 Association.
More than half of states will be implementing revised teacher-evaluation systems in the next three years.
Some early adopters have already begun to make or propose changes to key aspects, such as the weight given to quantitative measures, based on early results and feedback from teachers.
SOURCE: 91制片厂视频 Week
While only a handful of states already have a year or more of evaluation data, a host of others are now in the midst of full-scale implementation and will release initial results later this year.
Dozens of states have taken steps in recent years to overhaul their teacher-evaluation systems, often in response to federal incentives. Such changes have also been promoted by an influential lineup of organizations that calls for greater accountability in the teaching profession. The states hope to use the systems to strengthen teaching practices and dismiss poorly performing teachers.
Translating Results
In Florida, where every district was required to implement a new teacher-evaluation system in 2011-12, data released in December show that 97 percent of teachers received one of the top ratings. That figure, while high, is still lower than the 99.9 percent from before the revisions, state officials noted.
鈥淲e know that in the first year, most districts exercised an abundance of caution,鈥� said Kathy Hebda, the state chancellor for educator quality. 鈥淲e said upfront that our plan was to start together, and to get better every year. We do think it was a really good start, considering how big we are, and how much work there was to do.鈥�
Evaluation data in Georgia, also released in December, have so far been limited to 26 districts participating in the state鈥檚 federal Race to the Top grant. In 2012, administrators there essentially had to navigate two evaluation systems: the pilot program and the one required by the terms of existing laws and board policies.
鈥淚t鈥檚 not likely that many principals are going to rate teachers differently on the pilot system than the system they鈥檙e using for [human resources] purposes,鈥� said Martha Ann Todd, the state鈥檚 associate superintendent for teacher and leader effectiveness.
Michigan鈥檚 high numbers, released in November, could point to uncertainty in that state about the process, which remains somewhat in flux. A council working to outline a model state system for districts has not yet made its recommendations to the legislature.
Scores High
The early results offer several possible interpretations. As scholars have pointed out, there is no consensus about the percentage of teachers who should be identified as underperforming or superior in any given year.
鈥淚 do think we are still in a space of trying to do the research ... as these systems are being implemented, making sure that we are following up on things like alignment between the different measures,鈥� said Laura Goe, a research scientist in the Learning and Teacher Research Center of the N.J.-based 91制片厂视频al Testing Service.
鈥淲e just don鈥檛 have enough large-scale research studies yet to say that this is the right way to do it at the school level, the district level, or the state level,鈥� she said.
One of the key areas of congruence throughout the state data from Florida, Tennessee, and Georgia is the generally high scores given to teachers during classroom observations, a finding that comes right as new research is revealing clues about the properties of such observations and how they are shaped by the norms within schools.
A 2010 study on a pilot system in Chicago found, for instance, that when using the high end of the scale, principals often inflated their ratings compared to other observers, in part because of cultural expectations. And researchers have also found that having several different people observe teachers helped make their judgments more consistent over time and mitigate such bias.
Some districts say they worked deliberately to reach more nuanced findings. Florida鈥檚 Lee County showed a broader range of scores than most other districts in the state. Under its system, about 1.5 percent of teachers received unsatisfactory ratings, compared with 0.2 percent statewide.
Most Lee County teachers鈥� instruction was deemed solid, but the district reserved the highest category for only the top 10 percent of teachers. (Statewide, 23 percent of teachers earned that 鈥渉ighly effective鈥� rating.)
Officials for the school district say they worked jointly with the local teachers鈥� union to agree on how state-generated test-score data would be interpreted, and took pains to make the evaluation instrument clear, to help make those finer distinctions in performance.
鈥淲e鈥檙e still working on developing training. It鈥檚 huge, and very important, because our goal is to bring about the real consistency in our schools about how we evaluate,鈥� said Greg Adkins, the chief negotiator for the 85,000-student Lee County district.
Multiple Measures
The introduction of quantitative, supposedly objective data also might help ensure a broader range of scores.
鈥淲ith value-added in particular, you are essentially ranking results for teachers, so ... you have some who are necessarily going to be closer to the bottom. Whereas with observations you can have all the teachers on the top,鈥� said Ms. Goe, who also advises the Great Teachers and Leaders Center, a federally funded technical-assistance provider housed at the American Institutes for Research.
鈥淰alue added鈥� is a statistical method of estimating the effect of a teacher鈥檚 instruction on his or her students鈥� test scores.
Tennessee鈥檚 data released last summer show, for instance, that observers gave only 0.2 percent of teachers the lowest score, compared to quantitative measures that put 16.5 percent of teachers in that category.
Georgia officials are still examining the quantitative portion of the pilot data, but preliminary reports on 鈥渟tudent learning objectives"鈥攄istrict-determined common growth measures鈥攕howed more variability than did observations.
Disparities between different measures ought to invite more investigation, Ms. Goe of the ETS said. It could mean that 鈥渢here is a mismatch between what we can see the teacher doing and how the learning is taking place,鈥� she said, 鈥渙r that there are other factors entering into the situation.鈥�
Tennessee has already begun such analyses. This school year, the state sent 鈥渃oaches鈥� to 73 schools that had high teacher ratings and low levels of student-growth in 2011-12. The goal was to help support and retrain evaluators on how to document teacher practice.
鈥淲e鈥檝e committed to a process of continuous improvement for this evaluation system, and when we saw this need, we acted on it,鈥� said Kelli Gauthier, a spokeswoman for the Tennessee education department.
Midyear data suggest that results from those schools are likely to be distributed more broadly, although a majority of teachers will still pass muster.
Georgia officials are also redoubling efforts to improve consistency across raters. So far, the state has trained about 3,400 evaluators in face-to-face or online sessions, Ms. Todd, the associate superintendent, said. Some districts are also creating libraries of teaching videos to help refresh evaluators鈥� memories of exemplary performance at each level. The challenges for ensuring inter-rater reliability are somewhat greater for states that have given more discretion to their districts regarding training.
鈥淚 think there鈥檚 a huge incentive for the state to invest in principal training,鈥� said Ms. Lenhoff said of Michigan, 鈥渁nd make sure every principal understands and has practiced how to observe teacher behavior, how to take notes, how to give feedback to teachers.鈥�
