The concept sounds appealing: Measure the effectiveness of schools and teachers based on the amount of academic progress their students make from one year to the next. Often known as 鈥渧alue added鈥� measures because they track the 鈥渧alue鈥� that schools add to individual students鈥� learning over time, such methods are increasingly popular with educators and policymakers.
Some view the methods as an antidote to accountability systems that focus solely on getting children to a specified achievement level on a state test, regardless of where they start. Others view them as a way to isolate the effects of teachers and schools on learning, separate from such background characteristics as race and poverty.
Read the accompanying story,
Three national conferences on the topic took place last month alone. And this week, the Washington-based Council of Chief State School Officers planned to host a meeting on their use.
鈥淰alue-added measurement is a very active area today,鈥� Nancy S. Grasmick, the state superintendent of education in Maryland, said during a conference at the University of Maryland College Park last month. 鈥淲e know there鈥檚 controversy surrounding this,鈥� she added. 鈥淲e need to ferret out all of the factors and not just jump into this without a strong research base.鈥�
Indeed, as policymakers and practitioners rush to take up value-added methods, researchers continue to debate their merits and how the existing models can be improved.
While value-added assessments are well past their infancy, noted Robert Lissitz, the director of the Maryland Assessment Research Center for Student Success at the University of Maryland, 鈥渢he practical applications of value-added models are complex, difficult, and often controversial.鈥�
Fairer Measurement?
That hasn鈥檛 stopped the momentum, which has gained steam in part because of the federal No Child Left Behind Act. The law requires states to test every student annually in reading and mathematics in grades 3-8 and at least once in high school.
That mandate has opened up the possibility of tracking individual student growth from grade to grade in far more states, a prerequisite for value-added modeling. At the same time, concerns that the law鈥檚 accountability provisions are unfair to schools has sent people scrambling for alternatives.
Sixteen state schools chiefs wrote to U.S. Secretary of 91制片厂视频 Rod Paige earlier this year requesting the flexibility to use value-added or growth measures to meet the accountability requirements. States such as Ohio and Pennsylvania are now working to incorporate such models into their state accountability systems, joining existing ventures in Arizona, Florida, North Carolina, and Tennessee. And many other states, including Arkansas, California, Colorado, Louisiana and Minnesota, are considering adding value-added assessments.
One of the big attractions for educators is that value-added methods could provide a fairer way to measure school and teacher effectiveness than existing accountability systems.
The NCLB law, for example, judges schools primarily on the percentage of children who perform at the 鈥減roficient鈥� level on state tests. Schools don鈥檛 get credit for students who make lots of growth in a given year but still fail to reach the proficiency bar, or for advanced students who continue to progress.
Schools also are judged by comparing the performance of cohorts of students in successive years鈥攆or example, the performance of this year鈥檚 3rd graders vs. last year鈥檚 3rd graders鈥攅ven though the two groups may be quite different. In contrast, value-added methods track the growth of each student in a school over time from the child鈥檚 starting point.
Such methods also can provide schools with diagnostic information about the rate at which individual children are learning in particular subjects and classrooms, making it possible to target assistance to students or teachers or areas of the curriculum that need help.
In 2002, the Pennsylvania education department invited districts that were already testing in grades 3-8 to participate in a pilot value-added project, using the model that William L. Sanders developed for Tennessee in 1992. The plan is to take the project statewide next school year.
鈥楢 Great Diagnostic Tool鈥�
The 4,500-student DuBois Area School District, about 100 miles from the Ohio border, signed up immediately.
鈥淭here are people who are really worried about this concept and want it to be perfect before we say yes,鈥� said Sharon Kirk, the superintendent of the district. She spoke last month at a conference in Columbus, Ohio, sponsored by Battelle for Kids, a nonprofit there that is working with about 80 Ohio districts on a value-added pilot using the Sanders method. 鈥淚 can鈥檛 imagine why we would not absolutely embrace information that is going to make us better.鈥�
One of the first things Ms. Kirk did was ask each principal to predict which group of students his or her school was serving best. Daniel Hawkins, the principal of the DuBois Middle School, said he鈥檇 been confident the school was doing a fine job educating its most academically advanced students. When the data came back, it showed that in both math and reading, those students were making less progress over the course of a year than similarly high-performing students in other schools.
鈥淚 was wrong,鈥� he said, 鈥渙bviously wrong.鈥�
Amy Short, an algebra teacher at the school, said educators realized they were spending too much time reviewing material at the start of each school year and needed to accelerate instruction.
The school set up four different levels of algebra and provided additional periods of math practice for students with the lowest math scores who also were falling behind their peers. Each week, teachers in the same grade and subject sat down to decide what they would teach in the coming week, and crafted nine-week assessments to track students鈥� progress.
By 2003, DuBois Middle School students were demonstrating significantly more growth over the course of the year than similarly performing students elsewhere.
鈥淚 really like this because I think it鈥檚 a great diagnostic tool for me,鈥� said Ms. Short, who uses the data on individual students to tell whether they need additional support or enrichment. 鈥淚 thought I was teaching my kids better.鈥�
Research by Mr. Sanders and others in the field has found that the variability in effectiveness between classrooms within schools is three to four times greater than the variability across schools. Moreover, students assigned to highly effective teachers for several years running experience much more academic growth than students assigned to a string of particularly ineffective teachers, although the precise size of those effects and how long they persist are unclear.
Based on such findings, said Daniel Fallon, the chairman of the education division at the Carnegie Corporation of New York, people have come to recognize that the effects of good teaching 鈥渁re profound and appear to be cumulative.鈥�
High Stakes?
Most people appear comfortable using value-added information as a powerful school improvement tool. The bigger question is whether states are ready to use such methods in high-stakes situations.
So far, the U.S. Department of 91制片厂视频 has not permitted any state to use a value-added model to meet the requirements for adequate yearly progress under the No Child Left Behind law. And it鈥檚 not certain the department has the authority to do so without changing the statute.
Celia H. Sims, a special assistant in the department鈥檚 office of elementary and secondary education, said at the time states submitted their accountability plans to the federal government, most didn鈥檛 have in place the grades 3-8 testing or student-information systems that would permit them to track individual student gains over time.
鈥淰alue-added can certainly be used even right now as an additional academic indicator by the state,鈥� she noted, although no state has made that choice. In part, that鈥檚 because additional academic indicators can only serve to increase the number of schools potentially identified for improvement under the federal law.
鈥淪tates are still looking at how growth can fit within No Child Left Behind,鈥� Ms. Sims said. She does not know of any value-added model that specifies how much growth students must make each year, so that all students perform at the proficient level by 2013-14, as the law requires. 鈥淭hat鈥檚 the non-negotiable,鈥� she said.
Researchers in at least three organizations鈥攖he Dover, N.H.-based Center for Assessment, the Portland, Ore.-based Northwest Evaluation Association and the Washington-based American Institutes for Research鈥攈ave been working on models to combine value-added analyses with absolute measures of student performance, so that students would be on track to achieve proficiency by a specified point.
鈥淭his, to me, is a central issue with value-added,鈥� said Mitchell D. Chester, the assistant state superintendent for policy and accountability in the Ohio education department. By state law, the department must incorporate Mr. Sanders鈥� value-added method into the accountability system by 2007-08. 鈥淗ow do you combine looking at progress with still trying to ensure youngsters in Ohio end up graduating with the skills and knowledge that they need to succeed beyond high schools?鈥�
Some policymakers also are eager to use value-added models as part of teacher evaluation or compensation systems. But while many researchers and educators said value-added results might, eventually, be used as one component of such systems, they should not be the only measure.
鈥淚 think that really puts too much of a burden on value-added measures,鈥� said Henry I. Braun, a statistician with the Princeton, N.J.-based 91制片厂视频al Testing Service.
In general, such measures can distinguish between highly effective and ineffective teachers, based on the amount of growth their students make, researchers say, but they have a hard time distinguishing between the vast majority of teachers whose performance hovers around average.
Moreover, while value-added models can identify schools or teachers that appear more effective than others, they cannot identify what those teachers do that makes them more effective.
鈥淚n the earliest years of implementing a value-added assessment system, it鈥檚 probably smart to lower the stakes,鈥� said Dale Ballou, a professor of education at Vanderbilt University in Nashville, Tenn.
It鈥檚 also unclear how such measures would work for teachers whose subjects are not measured by state tests.
Both the Ohio Federation of Teachers and the Ohio 91制片厂视频 Association have supported the use of a growth measure as part of Ohio鈥檚 accountability system.
鈥淲e felt there were a lot of hard-working people out there who were not getting adequate credit for moving kids along the way they do,鈥� said Debbie Tully, an official with the OFT, an affiliate of the American Federation of Teachers.
But while the union is 鈥渕ore than open鈥� to using such measures as one component in teacher evaluation, Ms. Tully added, it鈥檚 far too early to tell if it can be used as an evaluation tool.
鈥楿nder the Hood鈥�
Yet for all the criticism of value-added methods, said Mr. Braun of the ETS, 鈥渨e have to confront the logic behind the enthusiasm that we see out there in the world for value-added measures.鈥�
The key, he said, is for policymakers to 鈥渓ook under the hood,鈥� and not just take such measures at face value.
鈥淚 think the fact that people are taking this stuff seriously now is focusing people on the right questions,鈥� said Vanderbilt鈥檚 Mr. Ballou.
While value-added models may eventually run up against insurmountable limitations, they鈥檙e not there yet.
鈥淎ll the other methods are also flawed,鈥� Mr. Ballou noted, 鈥渟o if you鈥檙e not going to use this one, what鈥檚 the alternative?鈥�
