The average scores of graduating teacher-candidates on state-required licensing exams are uniformly higher, often significantly, than the passing scores states set for such exams, according to an 91制片厂视频 Week analysis of preliminary data from a half-dozen states.
The pattern appears across subjects, grade levels, and test instruments supplied by a variety of vendors, the new data show, raising questions about the rigor and utility of current licensing tests.
There are, in essence, two main ways to interpret the findings. Some observers say the data suggest most states set low passing marks, screening out only candidates with the very lowest level of content knowledge.
鈥淚f there鈥檚 not a lot of variation in the performance of graduates by institution, it could mean that education seems to set a lower bar for institutions than other professions,鈥� said Dan Goldhaber, a research professor at the University of Washington Bothell, who has studied teacher-licensing tests.
The other point of view holds that most current teacher examinations are better thought of as a minimum screen, and should be considered alongside professional-entry mechanisms as a whole.
The average scaled score of 2009-10 teacher education program completers from four states on the Praxis II Mathematics Content Knowledge exam were higher than the national average.
SOURCES: 91制片厂视频al Testing Service; 91制片厂视频 Week
The tests 鈥渁re not designed to assess anything a candidate learns in a credential program,鈥� said Teri Clark, the director of professional services for the California Commission on Teacher Credentialing, the state鈥檚 teacher-standards board.
She noted that California requires prospective educators to pass the tests typically before they complete their pedagogy work. 鈥淚t doesn鈥檛 make a lot of sense to prepare a person to teach the subject matter if they don鈥檛 already have the subject-matter knowledge,鈥� she said.
The battery of tests varies by state. Some require basic-skills exams before entering a program; others require educators to pass tests measuring content or pedagogy knowledge before completing their preparation, as in California; still others require them before recommending a candidate for a license.
Where to Cut Off
The 2008 rewrite of the federal Higher 91制片厂视频 Act establishes much more detailed licensure test reporting requirements than its predecessor.
Among other requirements, Title II of the law compels states to include both passing rates on the tests and candidates鈥� average scaled scores. Scaled scores are, in essence, candidate performance translated from the raw number of correct answers to the scale on which the test is graded.
Though the U.S. Department of 91制片厂视频 has not yet posted the state-generated 鈥渞eport cards鈥� containing that information, 91制片厂视频 Week obtained preliminary data from higher education institutions鈥� reports and directly from state officials.
The states reviewed, selected at random, were California, Georgia, Indiana, Maryland, South Carolina, and Virginia.
States鈥� average scaled scores for program completers in 2009-10 were compared with passing scores for each test obtained from test vendors and state websites.
Among the findings:
鈥� Virginia鈥檚 teacher graduates鈥� average score on an elementary-content test, at 172, was nearly 30 points higher than the state cutoff score of 143.
鈥� Among the tests California requires for licensure, only one consistently seemed to pose any significant challenge for graduates: the Reading Instruction Competence Assessment, required of elementary and special education teachers and updated in 2009.
鈥� Of Georgia鈥檚 tests, an infant/early-childhood teacher test had the smallest disparity between program completers鈥� average scaled score and the state passing mark, at 13 points. By contrast, a music test and a Spanish test in Georgia show average scaled scores that were 41 and 38 points higher, respectively, than the cutoff scores on those tests.
鈥� The cutoff score on a popular secondary math test in South Carolina and Indiana is set well below both the average scaled score of the state鈥檚 program completers and the national average of all U.S. candidates taking the exam. (See chart, right.)
Data in Context
High overall passing rates on licensing tests have been well documented since the federal government started collecting information on the tests in 1998.
According to the 91制片厂视频 Department, the overall passing rate on licensing exams by graduates of traditional teacher-preparation programs in 2008-09 was 96 percent. The high rates overall are, in part, a reflection that many states have made passing the tests an institutional requirement.
Some technical differences exist between the Praxis series, developed by the Princeton, N.J.-based 91制片厂视频al Testing Service and used in many states, and licensing tests developed exclusively for states, as is the case in California and Georgia.
Still, regardless of the test, the overall pattern indicates that states have set the passing bar significantly below the mean in many cases鈥攅ven for those teaching at the high school level, where teachers on average have stronger academic qualifications than their colleagues in the lower grades.
There is no objectively correct passing mark on such tests; determinations are made by state panels that weigh demographic, educational, and political factors. Where the state ultimately sets the cutoff point matters because it helps determine the supply of teachers.
Overall, the data echo a 2010 report from the ETS that, in general, found a wide disparity in mean performance on licensing tests between teacher-candidates who passed the exams and the small proportion who failed them.
The Title II data do have limitations. For one, it is hard to tell from the preliminary data just how 鈥渄ifficult鈥� it is, relatively speaking, to pass the tests. Researchers typically express that relationship in terms of standard deviations from the average scaled score. To get at that information requires the spread of scores on the test, which states are not required to include in their report cards.
Another wrinkle: Most states permit teachers to take certification tests multiple times. It is not clear from the state-generated data how that policy affects the scaled scores.
The reports also show, overall, that teachers at the beginning of their preparation programs had lower average scores and passed the tests less often than those who had completed all program requirements. But even among those test-takers, average passing rates tended to be above 80 percent.
Value Debated
Researchers continue to debate the utility of current teacher-licensing exams and whether they adequately measure the range of skills that new teachers should master. The University of Washington鈥檚 Mr. Goldhaber, the author of a 2007 paper on such exams, found that teachers鈥� performance on them bore a weak or inconsistent relationship to how their students performed in class.
鈥淚 think [the Title II] results don鈥檛 tell you much about the workforce as a whole,鈥� Mr. Goldhaber said.
But he added that such information does raise questions about the cost-effectiveness of the current teacher-testing systems. 鈥淚f studying for these exams doesn鈥檛 make you a better teacher, you have a very expensive system to weed out a tiny percentage of the workforce,鈥� he said.
At their most basic level, the results appear to stand in stark contrast to those of other professions that include tests as one part of licensing, such as medicine or law. Consider, for example, the California bar examination, which in its most recent administration was passed by only 54 percent of candidates.
Yet experts note that merely raising cutoff scores may not lead to a more highly trained workforce that is better able to teach.
鈥淲hat gets tested on the teacher-licensure tests has little to do with teacher education programs at all. They鈥檙e not the materials you learn for teaching,鈥� said Drew H. Gitomer, a professor at the Rutgers Graduate School of 91制片厂视频, in New Brunswick, N.J., and a former researcher at ETS. 鈥淭he bigger question is what are the right measures and quality controls we want to have at licensure, and also beyond licensure.鈥�
South Carolina鈥檚 deputy superintendent for school effectiveness, Mark A. Bounds, said the state education department is considering recommending that cutoff scores on roughly a dozen teacher-licensing assessments be raised. But overall, he said, the state is working to improve teaching quality through a number of initiatives, such as linking 鈥渧alue added鈥� data on students back to the programs that produced their teachers.
Internal analyses show that only 77 of every 100 new teachers in the state advance from an initial to a professional license, which they must do through an induction program within their first three years on the job, he said.
鈥淚 don鈥檛 want to diminish testing, but I don鈥檛 want people to believe that if we only raised the Praxis scores, our teaching force would significantly improve,鈥� Mr. Bounds said. 鈥淚t鈥檚 one gateway to move through, as are induction and formal evaluation.鈥�
Ms. Clark of California鈥檚 professional-standards board said that performance assessments of student-teachers, which are also required in the Golden State, are better aligned with teacher-preparation courses. Those tests, she said, are selected and scored locally by institutions, and programs use the information to counsel struggling individuals out of the profession. But the results are not reported as part of Title II.
Twenty-four states are at various stages of piloting and implementing a similar test that proponents hope will prove to be a more valid assessment of teaching ability.
Still other critics believe that teacher-licensing exams should be strengthened in ways other than raising the passing scores or integrating performance tasks.
鈥淒irect questions should be raised about how the public is to be assured that any teacher subject test is at the college level,鈥� said Sandra Stotsky, a professor of education reform at the University of Arkansas. 鈥淐ut scores can be raised, but the test can still be a weak test.鈥�
