Are charter school networks really improving their students鈥� academic achievement?
That鈥檚 the central question to be addressed in what researchers involved believe is the first comprehensive study of the efficacy of charter-management organizations, or CMOs, which is scheduled to be launched later this year.
CMOs are nonprofit organizations that operate and replicate clusters of like-minded charter schools; among the better-known CMOs are the Los Angeles-based Green Dot Public Schools, the San Diego-based High Tech High, and the New Haven, Conn.-based Achievement First.
鈥淭here is a big black box between a student being affiliated with CMOs and that student鈥檚 learning,鈥� said Paul T. Hill, the study鈥檚 principal investigator and the director of the Center on Reinventing Public 91制片厂视频, which is based at the University of Washington in Seattle. 鈥淲e鈥檙e trying to fill in the box.鈥�
The three-year longitudinal National Study of CMO Effectiveness, by the San Francisco-based NewSchools Venture Fund entrepreneurial philanthropy and the Seattle-based Bill & Melinda Gates Foundation, will cover almost 200 schools within 33 CMOs in 12 states.
鈥淎lthough many studies have been done on the effectiveness of charter schools, this is the first expansive study on CMO effectiveness,鈥� Vicki L. Phillips, the Gates Foundation鈥檚 education chief, said in an e-mail.
Starting next fall, investigators will research in earnest what鈥檚 projected to be a three-pronged look at the efficacy of as many nonprofit charter networks as will consent to be studied. The Gates Foundation expects that the study will cost more than $3 million.
The study will focus on the academic impact of CMOs, their best practices, and how CMOs affect and are affected by their regulatory, competitive, and other contexts, said Brian P. Gill, a senior social scientist at Mathematica Policy Research Inc., of Princeton, N.J., who will co-lead the study.
According to Mr. Gill, the study will use both experimental and what he called 鈥渟tate-of-the-art quasi-experimental鈥� data.
For example, in studying the effectiveness of a CMO in a given area, the study will make use of comparative achievement data from students who gained admission to a given charter school vs. students who did not, in cases where an oversubscribed school employed an admissions lottery.
鈥淭he problem, though, is that not every charter school is oversubscribed, so we can鈥檛 do this everywhere鈥攚e鈥檙e going to do it opportunistically,鈥� Mr. Gill said.
Ultimately, he said, achievement will be measured student by student, using respective scores on state standardized tests.
Criticism Seen
Mr. Hill acknowledged the challenges of comparing student and CMO achievement in different ways.
鈥淲e think we鈥檙e brave as hell,鈥� he said. 鈥淓very one of these comparisons is imperfect in some way鈥攜ou have to be very clear about that.鈥� But he said that the comparisons will 鈥渂e Granny Smiths-to-Galas, not apples to oranges.鈥�
Marc Dean Millot, the editor of the Alexandria, Va.-based online newsletter School Improvement Industry, who previously worked with the Center on Reinventing Public 91制片厂视频 and Mr. Gill, said he would construct the study differently. 鈥淚 think you鈥檇 want to look at similarly situated traditional schools,鈥� in addition to comparing CMO-affiliated charters with non- CMO charters, said Mr. Millot, who also maintains the independent blog on edweek.org.
He also questioned why foundation research on effectiveness was only being initiated now, given the philanthropic money that鈥檚 been poured into CMOs over the years. The Gates Foundation reports having invested more than $105 million in CMOs since 2000.
鈥淭his kind of research should have been going on from day one,鈥� Mr. Millot said. 鈥淭here is no excuse from a social-purpose standpoint that, 鈥楬ere鈥檚 all this money, but we鈥檙e not going to open the stove until 10 years into the cooking process.鈥欌��
Joanne Weiss, a NewSchools Venture Fund partner and its chief operating officer, said her organization, which will oversee the study, has been gathering achievement, cost-effectiveness, and other data all along.
鈥淏ut that鈥檚 really, really different from a research study,鈥� she said, adding that to do a valid research study, the study鈥檚 sample size needs to be big enough.
