91制片厂视频

Classroom Technology

Researchers Sort Out Data-Analysis Software

By Lynn Olson 鈥 January 14, 2004 6 min read
  • Save to favorites
  • Print
Email Copy URL

Schools districts can choose from a dizzying array of software products to analyze student-achievement data. Now, three researchers have published a paper to help them make an educated choice.

The paper鈥攂y Jeffrey C. Wayman and Samuel Stringfield of Johns Hopkins University in Baltimore and Mary Yakomowski of that city鈥檚 public school system鈥攔eviews the range of commercially available software products. In addition, the authors have set up a Web site to provide ongoing updates of software reviews.

Although schools now gather vast quantities of data, the authors note, the information is typically not stored in ways that are useful to teachers and principals. But that鈥檚 changing rapidly with the advent of easy-to-use and efficient software programs.

The report, is available from the . See also the researchers鈥 for updates of software reviews.

鈥淭o improve achievement, we鈥檙e going to need to get data down to the classroom level, and we feel that means getting data into the hands of teachers,鈥 said Mr. Wayman, an associate research scientist at Johns Hopkins鈥 Center for the Social Organization of Schools.

鈥淭eachers can do this,鈥 he added. 鈥淭hese things are user-friendly. If you can order a book off of amazon.com, you can look at your students鈥 data.鈥

Selecting from among the commercial packages is not easy, though, because they offer a wide range of features. While some offer strengths in data efficiency, for example, others specialize in data presentation or graphics.

鈥淣o one piece of software accomplishes everything,鈥 the researchers write, 鈥渁nd some features are more common than others.鈥

For instance, most companies offer already-formatted reports of student data that can be generated with the click of a mouse. Fewer companies permit users to perform a customized query of the data and then save that query for future use. Only two of the 13 companies reviewed offer storage and retrieval of student work samples or electronically accessible portfolios of such work.

鈥楳essier Than You Think鈥

Above is an example from a commercial software package that schools might use to see if they meet federal education goals.

The researchers identified existing software through literature and Web searches and conversations with knowledgeable individuals, including directly communicating with people in the industry.

They included software in their evaluation if it enabled many different users to analyze existing student data. The researchers excluded software that permitted only district personnel to view such information. They also eliminated software that focused solely on school management issues, that was developed to generate assessment data, or that required ongoing data entry by teachers. Companies were included only if they could provide interactive demonstrations of their products.

Although the paper describes three well-respected, district-generated initiatives鈥攊n Broward County, Fla., Cleveland, and Houston鈥攖he authors believe that most districts would be wiser to rely on commercial vendors than to take the do-it- yourself route.

鈥淪chool data is always, always messier than you think it is,鈥 said Mr. Wayman. The authors argue that unless a district is certain it has the expertise to deal with data problems quickly and efficiently in-house, the experience outside vendors bring to the process is well worth the cost.

Phyllis Chasser, a senior data-warehouse analyst for the Broward County schools, said the district developed its own data-gathering and -analysis tools for several reasons, beginning with the fact that others were not available when the Florida district launched its warehouse in 1996.

Commercial products, she believes, can鈥檛 handle the volume or the specialized needs of large districts. 鈥淪ome of the software that is available is built for small districts鈥 that don鈥檛 have the expertise to build their own, Ms. Chasser said. 鈥淪ome are smaller than the size of one of our high schools.鈥

Any software Broward might buy, she added, would have to be customized, which would likely end up being more costly.

All the programs reviewed are Web-based, so that users can access them from any Internet connection. Each of the programs also can produce reports on student performance by subgroup, such as low-income or minority students, as mandated in the federal No Child Left Behind Act. All also offer some form of ongoing technical support.

Each evaluation starts with an overview of the company and the focus of its software. The reviewers then describe the preformatted reports available, as well as tools teachers can use to pose questions of their own and an assessment of their ease of use.

In addition, the researchers address whether the software meets data-exchange standards that permit software packages to communicate with each other, a feature known as the Schools Interoperability Framework, or SIF. The Baltimore-based group advises schools to ensure that the software they use can communicate with other products, as a way of protecting their investment. Finally, the reviews address any special features of the software, such as the ability to present and analyze data longitudinally.

In general, the researchers advise, good software for student-data analysis should be user-friendly, have a relatively rapid response time, offer query tools for less sophisticated users, and enable users to 鈥渄rill down鈥 to the level of individual grades, classrooms, and students.

What鈥檚 more, they assert, users should be able to access the software from home or the workplace and have the information available through a variety of means, including quick snapshots, query tools, and preformatted reports.

Making a Choice

In choosing a vendor, Mr. Wayman, Mr. Stringfield, and Ms. Yakomowski suggest that districts consider a variety of factors.

For example, officials should inventory the data currently stored by their district, assess the quality of that information, and estimate the cost of correcting or 鈥渃leaning鈥 the data. Schools also need to evaluate their data and analytical needs. And they must weigh how much outside or third-party help they require to set up the system, including inventorying the initial data, cleaning the data, merging data sets, and housing the data over time. Districts also need to consider how long it will take to get the system up and running, as well as its costs.

Although costs depend on many variables, including the size of the district and the features it wants, they range from as low as $2 per student per year to more than $10, the authors note, sometimes with a higher cost in the first year of implementation.

鈥淪chools and districts would be well advised to contact other schools using the software, query them thoroughly, and visit as many as possible to get practical feedback on the types of products and levels of services each vendor offers,鈥 the authors say.

Given the rapidly changing market, they add, schools also need to consider the long-term viability of the companies.

Denis P. Doyle, the co-founder of SchoolNet, one of the companies whose software is reviewed in the paper, said such independent, third-party evaluations are important to both school people and vendors.

鈥淪uperintendents, in particular, I think almost without exception feel their ability to make informed judgments is limited in this market because it鈥檚 so technical and so complex,鈥 he said. 鈥淭hey鈥檙e scared to death of making the wrong decisions, so they really welcome some kind of independent verification.鈥

Coverage of technology is supported in part by the William and Flora Hewlett Foundation.

Related Tags:

A version of this article appeared in the January 14, 2004 edition of 91制片厂视频 Week as Researchers Sort Out Data-Analysis Software

Events

Recruitment & Retention Webinar Keep Talented Teachers and Improve Student Outcomes
Keep talented teachers and unlock student success with strategic planning based on insights from Apple 91制片厂视频 and educational leaders.鈥
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 91制片厂视频 Week's editorial staff.
Sponsor
Families & the Community Webinar
Family Engagement: The Foundation for a Strong School Year
Learn how family engagement promotes student success with insights from National PTA, AASA鈥痑nd leading districts and schools.鈥
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 91制片厂视频 Week's editorial staff.
Sponsor
Special 91制片厂视频 Webinar
How Early Adopters of Remote Therapy are Improving IEPs
Learn how schools are using remote therapy to improve IEP compliance & scalability while delivering outcomes comparable to onsite providers.
Content provided by 

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide 鈥 elementary, middle, high school and more.
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.

Read Next

Classroom Technology What Schools Can Do to Make Teens Smarter Users of AI
Teens who have talked about AI in school are more likely to use it responsibly.
2 min read
Illustration of three educators in hard hats lifting up a very large letter "I" next to a large letter A.
DigitalVision Vectors
Classroom Technology Spotlight Spotlight on Media Literacy
This Spotlight will provide you with strategies to spot AI manipulation, review how media literacy has evolved in schools, and more.


Classroom Technology Q&A Don't Buy the AI Hype, Learning Expert Warns
Benjamin Riley, founder and CEO of Cognitive Resonance, explains his concerns about using AI in education.
5 min read
AI Skeptic 1244482154
metamorworks/iStock/Getty
Classroom Technology Then & Now How the 'Sexting' Panic Previewed Today's Debate About Kids' Cellphone Use
As technology evolves, one axiom stays the same: Schools must help students weigh how to responsibly fit it into their lives.
7 min read
Illustration
F. Sheehan for 91制片厂视频 Week + Getty