91制片厂视频

In March 1996, the nation鈥檚 governors met in Palisades, N.Y., and called for an 鈥渆xternal, independent, nongovernmental effort鈥� to measure and report on each state鈥檚 annual progress in raising student achievement and improving the public schools.

91制片厂视频 Week responded by launching the first edition of Quality Counts in January 1997, with the goal of producing an annual report card on public education in the 50 states and the District of Columbia.

At the time, standards-based education was in its infancy, spawned largely by frustration with a fragmented public education system whose governance was spread across more than 14,000 school districts, 50 states, and the federal government, and by low levels of student achievement, especially among poor and minority students.

The concept of standards-based education, as described by researchers, was to invite policy coherence where none existed. States would set clear and challenging content standards for what students should know and be able to do, ensure teachers were prepared to teach to those standards, provide the resources necessary to achieve the goals, and then develop tests and accountability systems closely aligned with the standards to measure progress and to prod and push schools to reach higher.

The coverage and research in Quality Counts largely mirrored this standards-based framework, including its assumption that the primary and constitutional authority for public education resided in the states.

A decade later, it鈥檚 fair to ask: Has student achievement improved? To what extent have states put in place the pieces of standards-based education? And is there any evidence that the two are related?

To answer those questions, this 10th edition of Quality Counts first examines scores on the National Assessment of 91制片厂视频al Progress, often known as 鈥渢he nation鈥檚 report card,鈥� from 1992, near the beginning of the standards movement, through 2005, both nationally and state by state. In addition to analyzing changes in average student achievement on NAEP, Quality Counts looks at progress for those students all too often left behind: poor, black, and Hispanic young people. And it asks whether some states have moved further or faster than others. The 91制片厂视频al Testing Service, of Princeton, N.J., conducted the NAEP analyses for Quality Counts.

Second, the report examines the progress states have made on some of the core indicators that Quality Counts has tracked over the past decade in the areas of standards, assessments, accountability, and efforts to improve teacher quality. (鈥淎 Decade of Policy Indicators鈥�, this report).

Finally, the 91制片厂视频 Research Center conducted a series of analyses for this report to examine the relationship between states鈥� increasing embrace of standards-based education and gains in student achievement. Profiles of the experiences in a handful of states expand on those analyses.

Such exploratory analyses have their limits.

First, gains on NAEP may fall short of measuring changes in student achievement at the state level. That鈥檚 in part because NAEP tests are given only to a sample of students in each state, do not carry any stakes for individual students or schools, and may not completely match state content standards and priorities.

Second, as researcher Henry I. Braun and his colleagues from the ETS point out, state responses to the call for improving education have been far from uniform. States vary in the specific policies they employ, how consistently and coherently they carry them out, when they put them into effect, and the degree to which those policies are actually picked up and put into action by districts and schools. Any longitudinal analysis also must be approached with care because of the many incremental changes to Quality Counts鈥� indicators over time, although the analysis tries to account for that evolution by focusing on a subset of indicators that have remained consistent over the 10-year period. Finally, from a statistical perspective, the analysis relies on a small sample, the 50 states. That fact provides another reason for caution in interpreting the results.

Despite those caveats, the time and energy put into standards-based education over the past decade suggest it鈥檚 appropriate to examine the effects of state policies as a way to help inform state practices in the future.

The conclusions are at once heartening and sobering. They鈥檙e heartening because when looked at over this span from 1992 to 2005, student achievement has gotten better, particularly in mathematics and particularly for those students who started furthest behind. Meanwhile, an increasing number of states have embraced a standards-based-education framework, with some of the earliest and most ardent adopters of standards-based accountability systems making some of the most progress in raising achievement, as highlighted by the case studies on Delaware, Massachusetts, New York, North Carolina, and Texas in this report.

A Mixed Picture

Standards-based education, explains Charlotte Ciancio, the superintendent of the 6,000-student Mapleton, Colo., public schools, has provided 鈥渢he leverage that we鈥檝e been able to use to make the changes that have been necessary in Mapleton. All of our reform work has been based on a standards-based design.鈥�

At the same time, it would be hard to ignore the fact that progress has not come nearly far or fast enough. That鈥檚 particularly true in reading, where average scores nationally have barely budged since 1992. It鈥檚 also true that, despite the solid gains of poor, African-American, and Hispanic students during this period, the achievement gaps between those students and their more affluent and white peers remain disturbingly deep鈥攁t least 20 points in both grades and subjects, or the equivalent of two grade levels or more.

After widening a bit during the mid-1990s, those gaps have begun to close again. But in many cases, the gaps now mirror what they were in the early 鈥�90s, and progress in closing them has been less dramatic since 2003.

An EPE Research Center analysis found a positive relationship between strengthening implementation of standards-based education over the past decade and gains in student achievement on NAEP math tests, but a slight negative relationship for reading. A closer examination found a negative relationship between state efforts to improve teacher quality, as measured by Quality Counts鈥� indicators, and gains in student achievement. After eliminating the teacher-quality indicators from the analyses, the relationship between state implementation of standards, assessments, and accountability systems and gains on NAEP became positive in both reading and math. The implication is that some elements of standards-based reform are more effective (or more effectively implemented) than others.

The good news is that, when examined from 1992 to 2005, the nation has made gains in student learning, particularly in math.

Student Achievement

Nationally, NAEP scores on 4th grade math have increased by 18.5 points on a 500-point scale, or nearly two grade levels, since 1992. Even more heartening are the gains for black and Hispanic 4th graders: 27.7 points and 24.2 points, respectively, during that same period.

One way to think about those math gains is that if the scores for white students had not also improved, the advances would have been enough to shrink the black-white achievement gap that existed in 1992 by 80 percent, and the gap between Hispanic 4th graders and their non-Hispanic white peers by 94 percent, virtually closing the gap between those groups. The scores for low-income students, which NAEP began reporting in 1996, basically mirror the average national increase for the 4th grade.

Trends in Student Achievement on NAEP

Since 1992, average student achievement on the National Assessment of 91制片厂视频al Progress has gotten better in mathematics, but results in reading are discouraging.

Note: Trends between 1992 and 2005 reflect statistically significant increases for math in grades 4 and 8 and reading in grade 8. Data from 1996 to 2005 reflect the use of accommodations for students with disabilities and English-language learners. Accommodations were not permitted in 1992 and 1994.

SOURCE: 91制片厂视频 Research Center, 2006

In math, improvements have come both at the 鈥渂asic鈥� and 鈥減roficient鈥� levels on NAEP, both for students overall and for black, Hispanic, and low-income youngsters. But, particularly when it comes to poor and minority students, improvements have been greater at the basic than at the proficient level. Those gains are mirrored across most states. Of the 41 states and the District of Columbia with NAEP data, all have made significant progress in 4th grade math achievement, and all but two have made significant progress in 8th grade math achievement, since 1992.

It鈥檚 hard to be as sanguine about reading. The national average in reading barely budged from 1992 to 2005, inching up just 2 points in both grades 4 and 8.

But, even here, somewhat better news lies beneath the surface. The reading scores for black, Hispanic, and low-income 4th graders increased at nearly triple the national average, or about two-thirds of a grade level. The scores for Hispanic and low-income 8th graders also increased, but not significantly.

As Marshall S. Smith, who served as the acting U.S. deputy secretary of education during the Clinton administration, points out in a paper, 鈥淭he Future of Systemic School Reform,鈥� the results are more solid when one considers how much the composition of U.S. public schools has changed over that time. Most notably, the number of Hispanic students increased by 55 percent between the 1993-94 and 2002-03 school years. Their proportion of the total K-12 population rose from 12.6 percent to 17.7 percent. Thus, the scores of Hispanic students, who typically perform less well than non-Hispanic whites, weigh more heavily in computing the national average in 2005 than they did in 1992.

鈥淗ad the proportion of all groups remained constant over the decade, the average national gain would have been greater than 10 points鈥� in reading in grade 8, writes Smith, who is now the director of education programs at the William and Flora Hewlett Foundation, in Menlo Park, Calif.

Of course, it鈥檚 precisely the change in the student population that makes the need to improve the performance of traditionally underserved groups more important than ever. Given that, what can be said about the nation鈥檚 progress in closing achievement gaps between poor and minority students and their more affluent and white peers?

Dissecting the Gap

Mapping Out Reading Achievement

Beneath a modest national improvement in 4th grade reading, an analysis reveals considerable variation in patterns of change for individual states.

Gain Patterns in 4th Grade Reading (1992-2005)

Achievement Gap Patterns in 4th Grade Reading (1992-2005)

Note: Accommodations were not permitted for students with disabilities and English-language learners in 1992. Gap analysis is based on average scale scores and examines poor-nonpoor, white-black, and white-Hispanic differences.

SOURCE: 91制片厂视频al Testing Service analysis of U.S. Department of 91制片厂视频, Institute of 91制片厂视频 Sciences, National Center for 91制片厂视频 Statistics, National Assessment of 91制片厂视频al Progress, 1992, 1998, and 2005 Reading Assessments

Nationally, the achievement gaps have narrowed somewhat for all groups in both reading and math and at both the 4th and 8th grade levels鈥攅xcept between black and white 8th graders in reading. The gap-closing was significant between black and white students in math at both grades, and between Hispanic and non-Hispanic white students in 4th grade math.

Progress is even more mixed at the state level, although the picture is complicated by the fact that many states either did not participate in the state-level NAEP during the periods examined, or did not have enough poor or minority students generated by the NAEP samples to permit comparisons of change over time.

鈥淚 think that the bigger picture is that we were losing ground over the course of the 1990s, and we鈥檝e turned that around,鈥� says Ross Wiener, the policy director of the Washington-based 91制片厂视频 Trust, which works to raise the achievement of poor and minority students. But he adds that the recent slowdown underscores the need for vigilance.

An analysis of 8th grade math performance on NAEP by Braun of the ETS and his colleagues found 鈥減rofound, persistent, and pervasive鈥� achievement gaps between black and white students even within schools serving predominantly nonpoor students.

What鈥檚 most striking, in looking at the achievement picture, is how some states stand out.

Some States Stand Out

For example, Delaware gained 12.9 points in 4th grade reading between 1992 and 2005, including gains of 17.1 points for blacks and 25.7 points for poor children. In 8th grade, Delaware students gained 12.3 points in reading, including an 18.1-point gain for African-American students between 1998 and 2005.

Florida鈥檚 black 4th graders gained 17.8 points in reading between 1992 and 2005, and 35 points in math. New York state saw the average reading scores of its 4th graders jump 24.2 points for Hispanic students and 13.9 points for low-income students. And in Texas, black and Hispanic youngsters gained more than 25 points in math at grade 4 and in excess of 20 points in math at grade 8.

Performance in the Southern states, which typically started well below the rest of the country, has surged. 鈥淭he South鈥檚 progress has been greater than the nation鈥檚 and needed to be,鈥� says Mark Musick, the president emeritus of the Atlanta-based Southern Regional 91制片厂视频 Board. Last year, for the first time, 62 percent of 4th graders in the South scored at or above 鈥渂asic鈥� in reading on NAEP, the same proportion as in the country as a whole.

鈥淭wenty-five years ago,鈥� says Musick, 鈥渋f you said the South would help pull up the national average, people would have thought you were joking, or smoking something.鈥�

Yet those gains on NAEP have not been reflected in the South鈥檚 staggeringly low high school graduation rates. In states such as Florida, Georgia, and South Carolina, just over half of high school freshmen graduate four years later, a rate that is worse for African-American and Hispanic teenagers. Nationally, about seven in 10 students graduate four years after starting high school.

The more controversial question is whether state policies adopted over the past decade have made any difference in raising student achievement or closing achievement gaps.

What鈥檚 Going On?

As the chart on Page 10 illustrates, Quality Counts found a steady increase in the number of states embracing state content standards and tests linked to those standards over the course of the past decade. Fewer states adopted all of the accountability and teacher-quality indicators tracked for this report during the same period.

To examine the relationship between standards-based education and gains on NAEP, the EPE Research Center conducted a series of regression analyses for Quality Counts 2006. The independent variable was the changes in the strength of states鈥� standards-based policies between 1997 and 2005. The dependent variable was the changes in NAEP math achievement between 1996 and 2005, and in NAEP reading achievement between 1998 and 2005. The center conducted separate analyses for math and reading in grades 4 and 8. To avoid biasing the results, the analyses controlled for states鈥� initial NAEP performance at the start of the period, and for the initial strength of their standards-based policies.

One way in which the research center鈥檚 analysis differs from other studies is that it considered the overall effect of states鈥� standards-based policies鈥攁cross the areas of standards, assessments, accountability, and efforts to improve teacher quality鈥攁nd looked at which of those policy areas, relatively speaking, independently contributed the most to changes in achievement.

Initially, the analyses found a moderate positive relationship between states鈥� overall embrace of standards-based education and gains in student math achievement. A slight negative relationship was observed for reading.

Further analyses exploring the relative contributions of the four major policy components revealed that the implementation of policies related to teacher quality was negatively related to achievement gains in both reading and math. In a second analysis, the teacher-quality policies were eliminated from the overall measure of standards-based-reform implementation in order to focus specifically on the contributions of policies related to standards, accountability, and assessment. Once teacher quality was taken out of the picture, the relationship between states鈥� efforts to carry out standards-based improvements and their gains in student achievement became much stronger. Improvement for math in both grades 4 and 8 became statistically significant, while more modest but positive effects emerged for reading. (Regression Analysis, this report)

Preliminary analyses also found no relationship between state resource and equity indicators and student-achievement gains, after controlling for states鈥� initial NAEP performance.

The findings from the exploratory analyses for Quality Counts generally agree with results from other studies.

A Blunt Instrument

In a forthcoming ETS study, 鈥淭he Black-White Achievement Gap: Do State Policies Matter?,鈥� Braun and his colleagues find a positive, but modest, relationship between states鈥� overall policy rankings in five areas鈥攇overnance, education finance, curriculum and standards, teacher quality, and assessment and accountability鈥攁nd their success in raising black students鈥� NAEP math scores in grade 8. While no single policy component accounted for the differences in outcomes among the 10 states studied, efforts to improve teacher quality and to institute assessments and accountability appeared to matter.

鈥淪tate policy is a blunt tool, but a tool nonetheless,鈥� concludes Braun.

Perhaps the bluntest of those tools, but the one that also has gotten the most attention, are states鈥� efforts to hold schools accountable for results in student achievement.

In 2000, RAND Corp. researcher David W. Grissmer and his colleagues suggested that states such as North Carolina and Texas that had implemented standards and standards-based tests in the early 1990s were showing large positive gains in NAEP math scores between 1992 and 1996. But the authors cautioned that more research across all states was needed before drawing any firm conclusions about the cause of the gains.

Since then, a number of scholars have weighed in with studies that have found a positive relationship between states that adopted strong accountability systems for schools and gains on NAEP. A study by Martin Carnoy and Susanna Loeb of Stanford University in the winter of 2002, for example, found that stronger accountability systems were associated with considerable increases in the percent of students scoring at the basic level or better on NAEP math tests in grade 8, with the largest gains by black and Hispanic students.

In 2004, a study by Eric A. Hanushek and Margaret E. Raymond of the Hoover Institution, also based at Stanford, found that accountability systems introduced during the 1990s had a clear positive impact on student achievement on NAEP. But they concluded that strong accountability systems alone did not lead to any closing of the black-white achievement gap.

And in a 2005 study, economist John H. Bishop of Cornell University found a similarly positive relationship between states that had implemented strong school-level accountability systems between 1997 and 2003, based on Quality Counts鈥� ratings, and gains on 8th grade NAEP math scores. Bishop also found a positive relationship between states that adopted high school graduation or end-of-course tests and gains on NAEP.

Smith of the Hewlett Foundation also observes that most of the gains on NAEP occurred between the mid-1990s and 2002, consistent with the emerging maturity of state standards-based reform efforts. The greater improvements in math than in reading scores, he argues, are in line with research suggesting that math scores tend to be more sensitive to and reliant upon what is taught in schools. 鈥淭hese fragments of evidence by no means create an ironclad case, but they are suggestive of the reforms鈥� having a positive force with respect to improving test scores,鈥� he concludes.

The gains in math also may reflect a national push for improvement in that subject, starting with the release of voluntary national standards by the National Council of Teachers of Mathematics in 1989 and promoted by such federal investments as the National Science Foundation鈥檚 systemic initiative in math and science, starting in 1990.

Not everyone has found a positive relationship between state accountability systems and test-score gains.

In a 2002 study, researchers Audrey L. Amrein and David C. Berliner at Arizona State University suggested that state efforts to tie student test results to serious consequences for students and schools had yielded no real achievement gains on NAEP. They also found dropout rates increased after states made passing tests a requirement for high school graduation, though other researchers have disputed that finding. Also, a growing body of evidence suggests that test-based accountability is encouraging schools to focus on tested subjects at the expense of nontested subjects.

10-Year Trend in High School Graduation Rates

The overall national graduation rate has improved slightly in recent years after decline and stagnation during the 1990s, but large gaps between groups remain.

SOURCE: EPE Research Center, 2006

Quality Counts found that high school graduation rates remained relatively stable from 1992-93 to 2001-02. They also remain abysmally low, particularly for poor and minority students.

While state accountability systems appear to be linked with gains in student achievement, according to Hanushek of Stanford University, 鈥渢he only way we are going to get the substantial improvements in student performance that we want is through upgrading the quality of teaching and the quality of teachers that we have in our schools. And that is a much more difficult policy issue than accountability.鈥�

Do Teacher Policies Matter?

Quality Counts鈥� indicators for efforts to improve teacher quality have changed markedly since 1997, in part to reflect the policy debate. Specifically, the report has added more details about teacher education and testing requirements, alternative routes into the profession, and accountability for teacher quality. During that period, there鈥檚 been an increase in the number of states that require prospective teachers to demonstrate knowledge of the subjects they will teach, either by holding a subject-matter major or by passing subject-matter tests.

But the EPE Research Center鈥檚 exploratory analysis did not find any relationship between state efforts to strengthen teacher quality and gains in student achievement on NAEP.

One problem, says Hanushek, is that any policy 鈥渉as to allow for the fact that we have this huge overhang of existing teachers, and you cannot think of simply changing the entire stock of teachers overnight. So to me, that suggests you have to have a much longer planning horizon.鈥�

Another problem, though, is legitimate disagreement about how to measure teacher quality. Hanushek, for example, is convinced that current licensing requirements are almost totally unrelated to teachers鈥� actual performance in the classroom. 鈥淎s long as we resist making judgments about who is doing a good job and who isn鈥檛,鈥� he says, 鈥渋t鈥檚 going to be very hard to think of getting any changes in achievement.鈥�

A report released last year by the American 91制片厂视频al Research Association highlighted the lack of research on the impact of teacher-testing, -accreditation, and -certification polices on pupils鈥� learning. State levers for improving the quality of the teaching force are 鈥減retty weak,鈥� says Robert B. Schwartz, a professor of education at Harvard University. 鈥淭he action is really much more at the district level here.鈥�

Given what the United States has learned about standards-based education, where does that leave policymakers and educators?

What鈥檚 Next?

鈥淲e shouldn鈥檛 give up on standards-based reform,鈥� says Smith, the Hewlett Foundation and former Clinton administration official. 鈥淲e should stay the course; we should make it better, improve it.鈥�

To some, that means moving from 50 sets of state standards to one set of national standards.

For many people, that means getting beyond state policy to efforts that actually make a difference to teaching and learning in classrooms.

鈥淎ny initiative or any goal to reduce the achievement gap substantially must do something to affect what goes on inside the school,鈥� Braun of the ETS says. 鈥淎nd that鈥檚 hard to do from the state level all the way down.鈥�

To others, that means conceding standards-based education has its limits, and that legislators need to look beyond the traditional system of public schools if they are serious about raising achievement, particularly at the high school level.