91制片厂视频

Special Report
Data

Push for 鈥楲earner Profiles鈥 Stymied by Barriers

By Benjamin Herold 鈥 October 20, 2014 12 min read
  • Save to favorites
  • Print
Email Copy URL

For many proponents of personalized learning, the notion of a digital 鈥渓earner profile鈥 is something of a Holy Grail鈥攖antalizingly powerful to imagine, but stubbornly difficult to actually find.

Just ask Jose P. Ferreira, the CEO of Knewton, a New York City-based company that is currently amassing hundreds of millions of bits of data on the learning histories of about 2 million American schoolchildren.

The goal is to generate ever-more comprehensive portraits of each student鈥檚 strengths, weaknesses, and preferences in order to provide them with customized academic content.

But to date, efforts by Knewton and other vendors to build a truly holistic portrait of any one child have been stymied. The long list of barriers includes technical hurdles, turf battles among educational publishers, lack of clear demand from schools, concerns about data privacy, and criticism that their work is based on a fundamental misunderstanding of how children learn.

鈥淥ne day the world鈥檚 going to wake up and say, 鈥榃ell, of course I鈥檝e got a learner profile that鈥檚 hosted in the cloud and goes from book to book and school to school and country to country and gets smarter and smarter the more I use it,鈥 Mr. Ferreira said. 鈥淲e know that鈥檚 obviously going to happen. But it鈥檚 not obvious to anybody outside of Knewton. So we鈥檝e got our work cut out for us.鈥

Harvesting Student Data

Although far from a mainstream idea, the concept of digital learner profiles has been around for years. Its clearest roots are in the health-care sector, where electronic patient records have in recent years become the norm.

The basic idea is to harvest all the data students generate while interacting with the dizzying range of software and apps now found in many classrooms in the United States鈥攅very minute spent on a given problem, every answer to every question, even every click and hover鈥攖hen store them in a single place.

From there, the data can be analyzed for signs that students have mastered a given standard or concept, as well as for evidence of the optimal ways in which each student learns.

In theory, the resulting profile can then be used by teachers to target instruction based on individual students鈥 needs and preferences. Software can also analyze the information in order to recommend the specific piece of content in the exact format at the precise time that algorithms determine is most likely to help an individual student master the specific concepts at hand. Some evidence suggests that such analyses may yield new and unexpected insights into how different topics are best taught.

鈥淚t鈥檚 the type of information that parents and teachers want to know, and frankly, should know,鈥 said John Bailey, the executive director of the Tallahassee, Fla.-based nonprofit Digital Learning Now, who co-authored the 2012 report 鈥淒ata Backpacks: Portable Records & Learner Profiles.鈥

Online adaptive tests, now in widespread use, rely on similar principles, but on the much smaller scale of an individual assessment.

Likewise, private-sector developers of educational software such as Amplify, Dreambox, and Khan Academy now use proprietary personalized learning platforms to generate robust profiles of individual students, but those portraits are generally confined to a particular course or suite of content.

And makers of K-12 learning management systems are increasingly trying to integrate the student-learning data generated from a variety of sources, but progress has been halting, and most LMS software lacks the sophisticated predictive algorithms that actually put all that data to work.

So for now, the ideal learner profile鈥攁 digital dossier that stitches together everything a student does across multiple textbooks, inside school and out, over the course of multiple years, and feeds software that uses the data to make recommendations for individual students鈥攍ives mostly in the imaginations of those who believe the concept has the power to transform education, for better or worse.

Few companies are searching as aggressively as Knewton for a way to make the vision a reality.

鈥淚f we have not just a semester鈥檚 worth of your math [learning] history, but every semester, and we can tie that to your science history, we can tell you things like, 鈥榊ou鈥檙e struggling in physics because of your math skills,鈥 鈥 Mr. Ferreira said. 鈥淚n the whole history of education up until now, those have been invisible failures.鈥

Critiquing the Vision

But for Yong Zhao, such grand pronouncements would mostly be annoying, if they weren鈥檛 so dangerous.

The professor and presidential chair in the college of education at the University of Oregon, in Eugene, is among those who believe that the notions underlying Knewton鈥檚 work鈥攖hat the bulk of what students should learn in school amounts to a series of discrete concepts; that mastery of those concepts is best 鈥渁cquired鈥 in a linear, orderly fashion; and that algorithms can analyze digital data to predict the optimal path for students to follow鈥攁re, at best, unsupported by evidence.

鈥淐hildren have feelings. They鈥檙e social beings. And they learn best by interacting with others,鈥 Mr. Zhao said. 鈥淏y putting too much blind faith in this data, we risk treating learning as a mechanical process and not a human endeavor.鈥

School Districts Wrestle With 'Learner Profile' Idea

A handful of school districts in the United States are wrestling with the idea of incorporating comprehensive digital profiles of students into their personalized learning strategies.

But they鈥檝e yet to find the right software or other tools to help make such 鈥渓earner profiles鈥 possible, said Eileen P. Harrity, a program officer for the Bill & Melinda Gates Foundation, which for the past year has provided grant funding to six districts to think about how 鈥渘ext generation鈥 software systems can support personalized learning. (91制片厂视频 Week receives support from the Gates Foundation for coverage of college- and career-ready standards.)

鈥淚 think [the districts] were hoping there would be one tool to do this, and they鈥檝e looked at options among both learning management systems and student information systems,鈥 Ms. Harrity said. 鈥淚t鈥檚 a definite need in the marketplace.鈥

The districts receiving the aid from the Gates Foundation are in Riverside, Calif.; Denver; Lake and Pinellas counties, Fla.; Henry County, Ga.; and Dallas. In general, Ms. Harrity said, officials in those school systems believe learner profiles should include information on what students know and can do, down to the granular level of individual standards and concepts, as well as information about their learning preferences and interests. The profiles should be constantly growing and evolving, most believe, in order to capture new information so as to paint a more holistic portrait of each learner and allow both teachers and software to provide more customized instructional resources and teaching.

David A. Irwin, a managing partner for the K-12 education practice of Stamford, Conn.-based Gartner Consulting, which is helping support the Gates grantees, likened the concept to the popular digital-music service Pandora. That service constantly captures data on the types of music that listeners prefer and uses that information to suggest new songs.

Technical Barriers

But there are barriers in the ed-tech market. Existing software systems tend to work only for specific bands of information that are contained within the limited universe of a particular vendor鈥檚 offerings. To be comprehensive, any learner-profile tool would need to be integrated with a wide range of existing learning management systems, student information systems, and educational software, a task made enormously difficult by the lack of agreed-upon technical standards in the sector, as well as by turf battles between vendors.

And Mr. Irwin said district officials aren鈥檛 sure exactly what such a service would look like in the K-12 context鈥攁n uncertainty that exacerbates the disconnect between educators and ed-tech providers.

鈥淚 can鈥檛 say we鈥檙e at a point where we know what functions [schools] want. We just know that they鈥檙e not currently out there,鈥 he said. 鈥淚 think it will happen when a district or school system can clearly articulate, 鈥楾his is what my new teaching and learning model looks like, and this is what I need to run that new model.鈥 鈥

The potential costs, he said, include inaccurate or misguided judgments about children, a lack of appreciation for how mistakes and mental detours can actually aid the learning process, and time and money diverted from 鈥渢he things that parents and educators actually care about,鈥 such as small class size.

That鈥檚 probably the highest-order critique of the kind of comprehensive learner profile for which Knewton and others advocate. But it鈥檚 far from the only one.

Even some proponents of big data in K-12 schools say that many software companies purporting to personalize education by profiling students are going about it in the wrong way.

鈥淭here are Silicon-era snake-oil vendors that slot people into very crude pre-existing categories instead of learning from patterns that emerge in the data,鈥 said Viktor Mayer-Sch枚nberger, a professor of Internet governance and regulation at England鈥檚 Oxford University.

And the most vocal concerns about learner profiles come from those worried about the privacy and security of students鈥 data鈥攅specially when it comes to vendors like Knewton, which boasts about collecting orders of magnitude more data about its users than consumer companies such as Google and Facebook.

鈥淒ata collection is just out of control in the education space,鈥 said Khaliah Barnes, a lawyer with the Electronic Privacy Information Center, or EPIC, a Washington-based advocacy group. 鈥淐hildren and families are really left out of the process and have no control over their information or how it will be used.鈥

Part of the problem, Ms. Barnes said, is the sheer complexity and opacity of what a company like Knewton actually does.

When trying to visualize what the company鈥檚 learner profiles actually look like, it鈥檚 easy to imagine a spreadsheet or a Facebook page or a manila folder with a paper dossier.

But in reality, the profiles that Knewton currently builds amount to constantly updating, mind-bogglingly long streams of alphanumeric code that are tied to an anonymous identifier. Much of that data is not directly about the individual student, but instead pertains to how his or her experiences contribute to a virtually endless series of probability calculations that also consider the experiences of millions of other students.

Even if it were possible to hit a button and receive the complete data record Knewton has amassed on an individual child鈥攃ompany officials say it鈥檚 not鈥攖he resulting information would be so voluminous and come in such a format that it would be unreadable by a human being.

Knewton鈥檚 data only becomes accessible to real people, company officials say, after it is whittled down and the relevant bits are presented through a dashboard created by one of its publishing partners. (Knewton doesn鈥檛 produce content or sell anything directly to schools, but gets paid by publishers to 鈥減ower鈥 their content with its learning-analytics technology.)

At no point in this process does Knewton hold any personally identifiable information on a child, company officials stressed. The company鈥檚 anonymized data are only connected back to specific children by publishers or schools, they said.

In the future, however, the company hopes to allow students and parents to create accounts directly with Knewton, opting to tie their identities to their learning data in order to gain access to a portable learner profile they could choose to 鈥渃arry鈥 with them from school to school or district to district.

Ms. Barnes of EPIC is skeptical.

If the hundreds of millions of records Knewton collects and the analyses it generates are at some point being tied back to an individual student, she said, then that student鈥檚 privacy is at stake鈥攅specially if the information is being used as the basis for important decisions about what the student knows, how he or she should be taught, and what his or her educational trajectory should look like.

And that student鈥檚 privacy is not being adequately protected, Ms. Barnes maintains, if he or she will never be able to access or contest the legitimacy of the information behind the analyses, will never be able to see or understand the algorithms performing the analyses, and will never know for sure what all that information he or she generated is ultimately being used for.

While such big-data-driven technologies are now widely accepted in the consumer world, she said, they deserve harsher scrutiny inside schools.

鈥淚f Amazon recommends a book you don鈥檛 want, there鈥檚 no lasting impact on your life,鈥 Ms. Barnes said.

鈥淏ut if a company says, 鈥榊ou鈥檙e not good at math鈥 or 鈥楲et me steer you away from this subject,鈥 that can be crippling to a student.鈥

Making matters worse, many in the ed-tech community believe current student-data-privacy laws are not up to the task of resolving other knotty questions presented by companies like Knewton, such as how long a third-party vendor should be allowed to maintain the information it gathers on students.

The constellation of concerns around data privacy were enough to topple the Atlanta-based nonprofit inBloom earlier this year, abruptly ending the most ambitious attempt to date to construct the type of data repository that might make comprehensive learner profiles possible on a large scale.

Beneath his hyperbole, Mr. Ferreira acknowledges鈥攁nd even agrees with鈥攕ome of the concerns that critics raise.

On the issue of data privacy, he said, it would be 鈥渟uicide鈥 for a vendor like Knewton to attempt to sell students鈥 sensitive information. 鈥淲e sleep well at night,鈥 he said, because the company does not gather any information that it can tie directly to an individual student.

But Mr. Ferreira also acknowledged that many school districts continue to do a terrible job of writing strong protections into their contracts with vendors who will be harvesting students鈥 data. And he conceded that parents often have little insight or say into the vast amounts of information on their children that is already being shared with third parties such as Knewton.

Both issues, he said, need to be resolved in a manner that gives parents more control over their children鈥檚 information.

Recognizing Limitations

And when it comes to the fundamental nature of schooling, Mr. Ferreira鈥檚 philosophy is actually quite similar to that of many of his critics.

鈥淭here鈥檚 too much testing, and too much teaching to the test,鈥 he said. 鈥淪chools are reacting to [accountability-driven education policies] by drilling students on basic skills, and it鈥檚 just ruining education.鈥

In his view, Knewton should be allowed to take over that menial work, freeing educators to do the kinds of teaching that he believes really matters: building students鈥 critical-thinking skills, spurring their imaginations, and helping them understand difficult concepts, make sense of ambiguous information, and solve big problems.

Algorithm-powered adaptive software, he said, 鈥渄oesn鈥檛 work if there isn鈥檛 a right answer鈥 and an effective means of measuring student proficiency. Knewton doesn鈥檛 even attempt to 鈥減ower鈥 courses in fields that involve a high degree of subjectivity, complexity, and creativity, such as art, philosophy, and advanced math and science.

鈥淲e鈥檙e very careful to limit ourselves,鈥 Mr. Ferreira said. 鈥淲e know exactly where the borders are, where what we do doesn鈥檛 work anymore.鈥

Business Barriers

Conceptual and policy problems aside, Knewton鈥攚hich has a significant presence in the U.S. higher education and international K-12 sectors鈥攈as been comparatively slow to grow its footprint inside K-12 schools in the United States for business reasons.

The company has partnerships with Triumph Learning, a small educational publisher based in New York City, as well as two of the largest educational publishers working in the United States, Houghton Mifflin Harcourt and Pearson.

But to date, Knewton is powering only very limited slices of each group鈥檚 content offerings. Getting its partners to focus on the business and technical hurdles associated with stitching together Knewton鈥檚 learner profiles across multiple textbooks, subjects, and years has mostly been a nonstarter.

Such companies also have a tremendous incentive to not provide a third party such as Knewton with the means to build learner profiles that cut across content from multiple publishers.

鈥淭his is a land grab, with many different commercial stakeholders going into the fray and trying to get a piece鈥 of the growing market for big-data solutions in education, said Mr. Mayer-Sch枚nberger, the Oxford professor.

For the time being, at least, the resulting balkanization in the marketplace will likely be reflected in schools, where unified, comprehensive learner profiles are likely to remain an aspiration, while smaller, segmented profiles for individual courses or publishers become an increasingly widespread reality.

Ultimately, only demand from parents and schools is likely to change that equation.

While there are stirrings in that direction among a handful of districts, the complexity of the technology, concerns around data privacy, and wariness over the direction of K-12 schooling still stand in the way.

Mr. Ferreira, however, remains undeterred in his conviction that the Holy Grail will eventually be found.

鈥淩ight now, the market doesn鈥檛 understand it,鈥 he said. 鈥淏ut we鈥檝e got to be ready for when the day comes when people do.鈥

Related Tags:

Coverage of personalized learning and systems leadership in 91制片厂视频 Week and its special reports is supported in part by the Eli and Edythe Broad Foundation. 91制片厂视频 Week retains sole editorial control over the content of this coverage.
A version of this article appeared in the October 22, 2014 edition of 91制片厂视频 Week as Push for 鈥楲earner Profiles鈥 Stymied by Barriers

Events

Recruitment & Retention Webinar Keep Talented Teachers and Improve Student Outcomes
Keep talented teachers and unlock student success with strategic planning based on insights from Apple 91制片厂视频 and educational leaders.鈥
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 91制片厂视频 Week's editorial staff.
Sponsor
Families & the Community Webinar
Family Engagement: The Foundation for a Strong School Year
Learn how family engagement promotes student success with insights from National PTA, AASA鈥痑nd leading districts and schools.鈥
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 91制片厂视频 Week's editorial staff.
Sponsor
Special 91制片厂视频 Webinar
How Early Adopters of Remote Therapy are Improving IEPs
Learn how schools are using remote therapy to improve IEP compliance & scalability while delivering outcomes comparable to onsite providers.
Content provided by 

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide 鈥 elementary, middle, high school and more.
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.

Read Next

Data Q&A College-Success Algorithms Often Get It Wrong for Students of Color
New research shows the models used to predict higher education achievement could hurt students on both sides of the equation.
3 min read
Illustration of pop up windows and notifications of different programs and applications
iStock/Getty
Data A New Digital Divide? Low-Income Students See More Ads in the Tech Their Schools Use
Students from the lowest-income families are the most likely to attend schools that do not systematically vet their education technology.
4 min read
Group of Students in IT Class
iStock
Data What the Research Says What Does 'Evidence-Based' Mean? A Study Finds Wide Variation.
Fewer than 1 in 3 education interventions get consistent judgments on their evidence base from reviewers.
5 min read
photograph of a magnifying glass on an open book
Valiantsin Suprunovich/iStock
Data 'Hidden Homeless': A Key Measure of Homelessness Excludes Most Students
Federal agencies differ in how they measure homelessness鈥攁nd many vulnerable students are left out.
3 min read
Photograph of a low angle view of children with backpacks climbing the school staircase.
E+/Getty