Ask teachers whether the drive to raise academic standards in the nation鈥檚 schools is a 鈥渕ove in the right direction,鈥� and the overwhelming majority answer yes. But ask teachers whether instruction focuses too much on the tests used to measure those standards, whether they have enough time to teach all the expected knowledge and skills, or whether they have enough materials and training to do the job, and their responses are more critical.
After more than a decade, standards-based initiatives to improve schools are, indeed, reaching into classrooms. While some studies suggest the results have been positive--including a more demanding curriculum and better test scores--evidence also suggests that teachers are being pushed to place too much emphasis on tests and test-preparation activities.
This fifth edition of Quality Counts tries to provide a classroom-eye view of how standards-based education is playing out. It is based on a new, nationally representative survey of public school teachers conducted for 91制片厂视频 Week.
The report also includes the most comprehensive survey to date of what the 50 states are doing to set academic standards, create testing and accountability systems linked to those standards, and provide support for students and educators to reach the higher expectations.
Quality Counts 2001 comes at a crucial juncture. The vast majority of states have been working diligently on policies related to standards-based changes. Almost all now have standards for what students should know in core subjects, tests to measure student learning, and at least the beginnings of an accountability system to hold schools responsible for results. But some large pieces of the puzzle are still missing.
In particular, the assessments now being used to judge whether students and schools have met state standards leave much to be desired. In too many states, the tests still focus too much on low-level, multiple-choice questions and are poorly aligned with the standards they are designed to measure. In too many states, students鈥� academic fates rest too heavily on performance on a single test. And, in some states, the passing scores for students may be set so high that thousands of youngsters risk repeating a grade or failing to earn a diploma.
Moreover, while states have begun to provide materials, training, and other standards-related help to teachers, students, and schools that fare poorly on state exams, those efforts have only scratched the surface.
The conclusion is simple and one voiced repeatedly by classroom teachers: If states really want to improve teaching and learning, they must find a better balance among standards, tests, and the support needed to do the job.
鈥淓verything that we do in our classroom has to revolve around the standards,鈥� says Solange M. Pohlman, a 7th grade teacher at Brentwood Middle School in Charleston, S.C. 鈥淚t鈥檚 given us a good guideline, a good basis to go by.鈥�
But, she adds, 鈥淚 don鈥檛 feel like there鈥檚 been enough training. I don鈥檛 feel like we have the materials--the hands-on things--to do it.鈥�
Conversation Shifts
The amount of ground traveled by states since the early 1980s is striking. At that time, most states were focused on minimum-competency tests for students to ensure that they had mastered basic reading and math skills before graduating from high school. Few states were committed to significantly higher expectations for all students--a goal that has since become a mantra of the school improvement movement.
鈥淚 think the most powerful, positive effect of standards-based reform has been that it鈥檚 focused attention on student learning and shifted the conversation from one about different groups of students鈥� meeting different standards to all students鈥� being taught to meet high standards,鈥� says Warren Simmons, the director of the Annenberg Institute for School Reform. 鈥淭hat鈥檚 a fundamental sea change in the goals of American education.鈥�
The push to improve education based on more rigorous academic standards dates back at least to the late 1980s. In 1989, the nation鈥檚 governors and then-President George Bush agreed to set ambitious education goals for the nation. With the goals Bush unveiled the following year, the leaders pledged that by 2000, among other objectives, American students would leave grades 4, 8, and 12 having demonstrated competency in challenging subject matter in English, mathematics, science, history, and geography.
A panel of experts convened by Congress concluded in 1992 that to measure progress toward the goals in a meaningful way, Americans should consider the creation of national education standards and a voluntary system of assessments to measure whether students had met those standards. 鈥淚n the absence of well-defined and demanding standards, education in the United States has gravitated toward de facto national minimum expectations,鈥� warned the National Council on 91制片厂视频 Standards and Testing, 鈥渨ith curricula focusing on low-level reading and arithmetic skills and on small amounts of factual material in other content areas.鈥�
Today, the idea of a single set of national standards and tests for all remains mired in controversy, making both unlikely to get off the ground. And the national education goals are already fading into memory, a year after their target date passed with the goals unmet.
But states have embraced the broad feature of standards-based education with astonishing alacrity.
Forty-nine states now have statewide academic standard for what students should know and be able to do in at least some subjects; 50 states test how well their students are learning; and 27 hold schools accountable for results, either by rating the performance of all their schools or identifying low-performing ones.
All of which poses critical questions: Is standards-based reform working? Are schools actually changing in response to state standards, and are students learning more? Have states adopted the right balance of policies to arrive at their desired goals?
Critics assert that the whole premise of standards-based school improvement is fundamentally misguided because it shifts the locus of control from individual schools, teachers, and parents to distant state bureaucrats. They also contend that the tests being used to drive higher expectations are, ironically, squeezing out high-quality instruction, and that the standards are producing mindless and undesirable standardization in schools. Equally important, they say, states are highlighting the wrong problem. If they really wanted to improve learning, critics suggest, states would pay more attention to the quality of teaching, inequities in school financing, and the excessive size of many high schools.
Conversely, supporters of the standards movement point to rising test scores in some states and districts as proof that standards are on the right track. They argue that the emphasis on results has caused educators to zero in on academics and to provide more support for low-performing students and schools. And, they maintain, the focus on standards has encouraged states to increase funding for education and to devote greater attention to teacher quality.
Trying to find a firm footing across such a philosophical divide is difficult, particularly because the details of state policies and practices vary so widely.
I think [standards] are making all teachers more aware of what needs to be taught.鈥�
Some states set standards in only a few subjects; rely primarily on commercially developed, multiple-choice tests to measure whether students are learning; and do not reward or punish schools based on performance. Other states set standards in English, math, science, social studies, and more; test every student every year in grades 3-10; and use test results to determine whether students graduate or schools are labeled as failing. While some states have been working on a set of standards-related policies for more than a decade, others have barely begun. All of which make it hard to generalize about the positive or negative effects of the effort as a whole.
The Rumblings of Change
Even so, some patterns are beginning to emerge. In particular, some of the states making the greatest gains in reading and mathematics on the National Assessment of 91制片厂视频al Progress in the 1990s, such as Connecticut, Kentucky, North Carolina, and Texas, also were early and consistent supporters of state standards and assessments. Studies by economist John Bishop at Cornell University have found that students in states with a combination of high school exit tests and end-of-course exams perform better on national tests than students in states without such exams.
A report released last fall by the American Federation of Teachers highlighted 11 urban districts where students have made academic progress, based on tests, and sustained it for at least three years--a trend that was not evident at the beginning of the 1990s. Many of those districts, including Boston, Philadelphia, and Corpus Christi, Texas--have placed strong standards and aligned assessments at the centerpiece of their school improvement efforts. Experts warn that some of those test-score gains could be inflated by teachers鈥� spending too much time preparing students to pass the tests.
But there is also a growing body of evidence that suggests schools serving high proportions of poor and minority children can achieve at high levels, based on strong leadership, rigorous academic expectations, a close attention to data, and a strong, shared sense of professional responsibility. (See related story, Page 19.)
In addition, surveys and case studies of schools and teachers in Illinois, Kentucky, Michigan, New Jersey, New York, Vermont, and Washington state have found that districts, as well as individual schools, are trying to revise their curricula to match state expectations. Teachers reported that they were adding new topics to their lessons to reflect what鈥檚 on state tests. In some cases, they were changing their teaching strategies.
Many of the 1,019 teachers surveyed for Quality Counts similarly reported that improvements are occurring in their schools. A majority said that, compared with three years ago, the curriculum is more demanding of students, teacher expectations are higher, teachers in their schools are collaborating more, and students are writing more. Nearly half also reported that students are reading more. Of those who said the curriculum is more demanding, more than six in 10 attributed the change to statewide academic standards. Just over half said their students are working harder because of the new standards.
鈥淚 think [standards] are making all teachers more aware of what needs to be taught,鈥� says Kenneth H. Knowles, an English teacher at Carmel High School in Carmel, Ind., in a sentiment echoed by many teachers.
鈥淔or the first time in the 29 years since I鈥檝e been teaching, I鈥檝e seen charts published by the state department of education on those standards and what each student should know by what grade.鈥�
A large majority of those surveyed also said they had modified their curricula to reflect state standards, and that they had adopted or created modules, curriculum units, or lesson plans linked to state expectations.
鈥淭eachers report teaching a much more balanced curriculum today than they were reporting 10 years ago,鈥� says Andrew C. Porter, the director of the Wisconsin Center for 91制片厂视频 Research at the University of Wisconsin- Madison. 鈥淏y balance, I mean a better balance across various kinds of cognitive skills: memorizing facts, computational skills, problem-solving, being able to communicate about ideas in science and mathematics.鈥�
But, in most cases, the research suggests that such changes do not run deep. Although standards are beginning to find their way into classrooms, those classrooms still look much as they did 10 years ago.
鈥淧eople are doing more of the content that鈥檚 being tested,鈥� says William A Firestone, a professor of educational policy at Rutgers University, who is studying the influence of state standards and assessments on math and science teaching in New Jersey classrooms. 鈥淲hat鈥檚 harder to do is get teachers to change the ways they鈥檙e teaching, so that it really encourages students to think mathematically or think scientifically.鈥�
A survey he and his colleagues conducted of some 245 4th grade teachers in New Jersey found that many reported using hands-on materials, such as geometric solids, to teach math. 鈥淏ut when we go into the classroom and watch teachers use the manipulatives with kids,鈥� he says, 鈥渕y impression is there鈥檚 a lot of basically drill and practice.鈥�
In Illinois, a longitudinal study of how districts are implementing state standards 鈥渄efinitely saw changes from last year to this year,鈥� says Lizanne DeStefano, a professor of educational psychology at the University of Illinois at Urbana-Champaign. Principals and teachers are now more aware of the standards, she says, and making initial efforts to match them to their curricula. But, she adds, 鈥渨e have not seen a lot of indications of standards moving more deeply to influence instruction.
鈥淚 think principals and teachers are struggling with what to do next,鈥� she says.
The Quality Counts survey found that elementary teachers were more likely than middle or high school teachers to report changes in curriculum and expectations. Teachers in states where students must pass exit tests to graduate, not surprisingly, also reported more changes in response to state standards and assessments than those in states where the stakes are lower.
Too Much Focus on Tests
Research suggests that high-stakes testing can be a powerful tool to change what happens in schools and classrooms, but not always for the better. Indeed, no aspect of the standards-based agenda has generated more debate--or stirred more dismay among teachers-than the heavy reliance on state tests to measure student learning and to dole out rewards and penalties to schools and students.
鈥淲hen we talk. about standards-based reform in Chicago, and it鈥檚 actually true everywhere, don鈥檛 show me the standards documents. Show me what you test,鈥� says Anthony S. Bryk, a professor of education and sociology at the University of Chicago, 鈥渂ecause the load-bearing wall in all of this is not the standards documents, it鈥檚 the assessments.鈥�
Forty-five states now compile report cards on schools, and 27 rate school performance, primarily on the basis of test scores.
Studies suggest that teacher are changing their instruction to align it with what鈥檚 tested. Teachers may increase their attention to specific topics, shift instructional time to concentrate more on the subjects that are tested, devise exercises that mirror test formats and expectations, and work with their students on such test-taking skills as filling in the bubbles on multiple-choice questions. (See related story, Page 23.)
In Quality Counts鈥� survey of public school teachers, about 30 percent said statewide academic standards have led to just the right amount of teaching that parallels the content of state tests. But nearly seven in 10 said instruction stresses tests 鈥渇ar鈥� or 鈥渟omewhat鈥� too much. Sixty-six percent also said state assessments were forcing them to concentrate too much on what鈥檚 tested to the detriment of other important topics.
鈥淚 think that schools are changing their curriculum to teach to the test, and I think that鈥檚 unfortunate for children and teachers because it doesn鈥檛 allow for any creativity,鈥� says Lori S. Teague, a science teacher at Parry McCluer Middle School in Buena Vista, Va. For example, this year Teague jettisoned a popular math and science unit on sharks because it didn鈥檛 match the Virginia assessments, which eventually will determine something would be amiss. But measurement experts caution that any one test samples only a narrow range of what students should be learning. If teachers concentrate on the test--rather than on the broader content undergirding the exams--it could lead to a bump in test results that does not reflect real learning gains.
Teachers and parents also worry that the strong focus on tests may be reducing students鈥� learning enjoyment.
鈥淎s a parent of an Indiana 2nd grader whose curriculum seems to be an almost exclusive regimen of test-taking practice ... I am becoming increasingly disgruntled with the standardized-testing explosion,鈥� says David Ander on of Evansville, Ind. 鈥淭he net effect seems to be a 1st and 2nd grade filled with math and language drilling that sends the message that learning is a chore, not something fun and exciting.鈥�
鈥楾oo Much To Teach, Not Enough Time鈥�
State standards and tests have improved, but they are not as good as they should be. Though state standards have become clearer and more specific since the early 1990s, experts say there are still too many for teachers to cover them all. And, in some states, they remain so vague that teacher naturally turn to the tests for guidance.
In New York state, says Patricia A. Johnson, a math teacher at Scotia Glenville High School in Scotia, N.Y., 鈥渨e thought when the new state exam came in, we鈥檇 be able to do things in more depth because there would be fewer topics. And, instead, there are just as many topics.
鈥淭here鈥檚 just too much to teach, not enough time, so you don鈥檛 get to do anything in depth anymore,鈥� she says.
That view is widely shared. Seven in 10 of the teachers surveyed for Quality Counts said there is 鈥渟omewhat too little鈥� or 鈥渇ar too little鈥� time in the school year to cover what the state academic standards require, a view that was more prominent among elementary and middle school teachers than among high school teachers.
Moreover, in many states, the tests may not adequately reflect the standards. That raises fundamental issues of fairness, because schools and students may be judged on tests that don鈥檛 mirror their curricula.
Forty states report that they have so-called criterion-referenced tests aligned with their standards in English in elementary, middle, and high school; 34 states report having such criterion-referenced tests in math. But studies by Achieve, a nonprofit group based in Cambridge, Mass., that promotes standards-based school improvement, have found that state tests do not sufficiently match their standards. For example, states may test some standards to the exclusion of others. And they tend to stress the less demanding knowledge and skills, rather than the more ambitious content spelled out in the standards. (See related story, Page 33.)
鈥淓veryone always says that their tests are aligned with their standards,鈥� remarks Lauren B. Resnick, the director of the Learning, Research, and Development Center at the University of Pittsburgh, which worked with Achieve on the alignment studies. 鈥淲ell, enough! We now know that they鈥檙e not.鈥�
Moreover, while most states no longer rely solely on multiple-choice questions to assess students鈥� knowledge and skills, relatively few ask students to write extensive essays, conduct experiments, or engage in other, more expansive projects to demonstrate what they know. Only seven states have performance assessments in subjects other than English. And only two--Kentucky and Vermont--use portfolios, or compilations of students鈥� classroom work, to help measure student progress.
In addition, states rarely provide teacher and students with the feedback necessary to use tests to help improve instruction.
What鈥檚 Realistic?
Meanwhile, some states may have rushed far too quickly to attach high stakes to their assessments or may be setting goals that are unrealistic. (See related story, Page 53.)
鈥淢ost states that I look at set performance goals without any reference to what students actually do, or what research says about how much improvement is feasible,鈥� says Daniel M. Koretz, a senior social scientist with the RAND Corp.
Only 17 percent of teachers surveyed for Quality Counts said the academic standards in their states were 鈥渧ery much鈥� or 鈥渟omewhat鈥� too high. Teachers in high-poverty schools were just as likely to say the standards were 鈥渁bout right鈥� as teachers in other schools. But as the poverty rate in schools increases, teachers are less likely to say their students are currently meeting the standards for their grade levels.
Teachers are particularly worried that states expect schools serving large proportions of low-income children to perform just as well as other schools on state tests without ensuring a level playing field.
鈥淚 happen to be in an upper-middle-class environment, so it鈥檚 easier for me to reach the standards with those children,鈥� says Reynold S. Forman, an English teacher at Colts Neck High School in Colts Neck, N.J. 鈥淚 don鈥檛 think it鈥檚 fair to say, across the nation, no matter what socioeconomic area you鈥檙e in, we鈥檙e all going to be on the same plateau.鈥�
Eighteen states--California, Kentucky, and North Carolina among them--have tried to address that equity issue by evaluating how much a school鈥檚 performance improves over time; 19 require schools to meet an absolute standard. Some do both.
Limited Help
The question of how high to set the bar is particularly crucial when tests are being used to make decisions about whether students can graduate or be promoted to the next grade. Eighteen states require students to pass exit exams to earn a high school diploma, and six more plan to do so in the future. Three states require students to pass state tests to be promoted to the next level in specified grades, a number that will rise to seven by 2003.
In 1999, for example, only 67 percent of 10th graders in Massachusetts passed a state English test and 44 percent passed a state math test that will be required for graduation beginning in 2003. The high failure rates led state Commissioner of 91制片厂视频 David P. Driscoll to propose that students who passed their courses, but failed the exams, be allowed to receive a state-endorsed local certificate. In Arizona, officials delayed until 2004 a requirement that students pass a math test to earn a diploma and are considering delaying the diploma requirements for other state tests, following initial high failure rates.
We now run the risk of unfairness, in terms of this first wave of older students.鈥�
But he worries that because policymaker underestimated how long it would take to get standards into classrooms, 鈥渨e now run the risk of unfairness, in terms of this first wave of older students. I just think that鈥檚 an issue that states really need to pause and think through.鈥�
Of the 18 states that require students to pass tests to receive their diplomas, 15 mandate that they get extra help if they fail the tests. But only nine of those states help subsidize remedial instruction for students who fail the exams the first time around. Louisiana, New Mexico, and North Carolina base student promotion in some grades on test results. But only Louisiana and North Carolina help pay for remedial study for students who fail the exams.
The issue is particularly troubling for students with special needs and those whose primary language is not English. Such students typically perform less well on state exams and are most likely to require special accommodations that may not be available or that may raise questions about the validity of test results.
In addition, little evidence exists to show that forcing students to repeat a grade pays off academically. A study in Chicago, for example, found that 3rd and 6th graders who were held back in 1997, under the city鈥檚 then-new promotion policy, did not perform any better--and sometimes did worse--than students who were 鈥渟ocially promoted鈥� before the policy took effect. Even worse, the researchers found, one-third of students who failed 8th grade dropped out. Other research suggests that high-stakes testing in Texas also has contributed to increased dropout and retention rates, although those findings are under dispute.
Both education and measurement experts caution against using a single test score to make a high-stakes decision about a student, such as graduation or promotion. Yet, in many states, tests are acting as gatekeepers.
In general, surveys indicate that teachers are far less supportive of using state tests to evaluate schools or determine whether students graduate than are parents or members of the general public.
A survey conducted last year for the Business Roundtable found that more than six in 10 parents and members of the public believed that students should have to pass state tests to graduate from high school, even if they have passing grades in their classes. By comparison, fewer than four in 10 teachers surveyed for Quality Counts agreed that graduation should hinge on test scores.
But the evidence from opinion surveys suggests the public might be more comfortable with an approach that does not depend solely on test scores. More of those surveyed by the Business Roundtable thought grades and teacher evaluations were 鈥渆xcellent鈥� or 鈥済ood鈥� guides to promotion than said standardized tests were. A 2000 survey by Public Agenda found that more than seven in 10 parents 鈥渟trongly鈥� or 鈥渟omewhat鈥� agreed that it鈥檚 wrong to use the results of one test to decide whether a student graduates or is promoted. Nearly eight in 10 said schools should use test scores along with teacher evaluations to make such decisions.
A Richer Picture
The solution, according to many experts, is to provide a richer picture of student and school performance by heeding other measures in addition to test scores. 鈥淚 do not believe that in any foreseeable future, you can really drive reform through high-stakes testing systems,鈥� says Bryk of the University of Chicago. 鈥淚 think that the assessment systems are just too easily corruptible, and you can鈥檛 really measure fully everything you value.
鈥淲hile this is a component,鈥� he continues, 鈥渋t鈥檚 got to be balanced by other components.鈥�
Experts suggest paying more attention to the quality of teaching and learning in schools, the amount of time spent on various subjects, and the rigor of the curriculum. That might be accomplished by examining samples of student work or by sending teams of outside reviewers to visit schools. In England, such 鈥渟chool inspections鈥� are the primary tool used to identify strengths and weaknesses and help schools improve.
Of the 27 states that currently rate schools, 11 do so based entirely on test scores. Sixteen include other measures in their rating systems--such as attendance and dropout rates--but they rarely carry enough weight to alter a school鈥檚 standing.
鈥淚 think where we鈥檝e gone overboard is the notion that if you set clear targets, and you have some way of measuring them, you can then step out of the picture and somehow it will be the responsibility of local professionals to figure out how to get there,鈥� says Brian M. Stecher, a senior social scientist with RAND. 鈥淲hile we were probably wrong two decades ago to be only focusing on accreditation, process-types of measures and not outcomes, I think we鈥檝e probably gone overboard now in the other direction.鈥�
Others suggest that states should engage in much more limited testing, in part, to leave more room for ongoing classroom assessments by teachers. They argue that individual schools and districts should make the ultimate decision about who graduates. Schools and districts also would be responsible for providing their communities with more detailed information about performance, based on local goals and standards. Under such a scenario, the state would intervene when a school or district was not fulfilling its accountability role.
But exactly what 鈥渕ultiple measures鈥� states should include in their accountability systems, or how that would work in practice, is far from clear.
Support Not There
In any case, better tests and accountability systems alone will not produce standards-based classrooms. States have yet to provide access to the professional development and high-quality teaching, curriculum, and other materials needed for schools and students to reach state standards. (See related story, Page 43.)
鈥淢y personal bias is that if you want to change day-to-day practice, you have to give people materials to work with,鈥� says Hilda Borko, a professor of education at the University of Colorado at Boulder. 鈥淵ou can鈥檛 just depend on the assessments.鈥�
David K. Cohen, a professor of education at the University of Michigan, and his research colleagues have found that when teachers participate in professional development in the subjects they teach and practice with materials they will use with students, instructional practices change and student achievement rises.
But, in general, states are only beginning to address those issues. Although 38 states, for example, require teachers to participate in professional development to renew their teaching licenses, only seven require that at least some of that training be in the teachers鈥� subjects or 鈥渆ndorsement鈥� areas. And while 42 states pay for some professional development for teachers, only 24 provide such money for every district or school. Moreover, the quality of that training varies widely.
Of the teachers surveyed for Quality Counts, fewer than half said they had 鈥減lenty鈥� of access to textbooks and other materials, lesson plans, and curriculum guides that match state standards. Fewer than half also reported having 鈥減lenty鈥� of access to training in using the standards or interpreting test results. In fact, in the past year, a majority of teachers had five hours or less of training in each of those topics.
鈥淭he professional development is not there,鈥� says Andrew Jervis, a social studies teacher at Babcock Middle School in Westerly, R.I. 鈥淪chool systems are not doing all that they can to make sure that all staff members have received proper training on how to use the standards.鈥�
The survey found that teachers who had received more training in the use of state standards and assessments were more likely than colleagues with less training to report that they were using lesson plans linked to the standards, had significantly modified their curricula to align with state standards, and were using test results to diagnose individual student needs.
鈥楢n Honest Review鈥�
To their credit, many states are adjusting and modifying their approaches to standards-based changes as they learn more about how to make such systems work.
In Ohio, for example, the governor has appointed a commission to rethink the state鈥檚 testing and accountability system. States such as Indiana, Nevada, and New Mexico have revised their academic standards for students to make them clearer and more precise. Maryland has delayed requiring new high school exit tests for graduation until the state can ensure the supports for students are in place.
Such adjustments, observers argue, are not only desirable but essential, if standards-based reform is to live up to its potential. 鈥淚t鈥檚 time for us to really do an honest review of where we are, what鈥檚 happening, identify different weaknesses and problems, and correct them,鈥� says Bob Chase, the president of the National 91制片厂视频 Association, 鈥渁nd to do it in a very open and honest way with all the stakeholders involved.鈥�
Whether that potential will ultimately be realized is an open question.
鈥淭hose who have been pushing the standards movement, and the rhetoric of it, are now becoming aware that we鈥檙e not going to get the results,鈥� warns John I. Goodlad, a professor of education at the University of Washington. 鈥淎nd so there鈥檚 a big cloud moving in.鈥�
But others caution that it鈥檚 simply too early to tell. 鈥淚 can cite you a very long list of concerns that I鈥檝e got about almost every aspect of this set of reforms,鈥� says Achieve鈥檚 Schwartz. 鈥淏ut if you ask me, 鈥楬as my basic belief that this strategy is the right long-term strategy been altered by the experience to date?鈥� the answer would be no.鈥�
