Testing and the Law of Unintended Personalization

When unexpected things happen despite — or even because of — our best intentions, we call that the Law of Unintended Consequences. Unintended consequences might be good, bad, or indifferent; they just weren’t intended. I’m beginning to the think that one of the unintended consequences of NCLB is the end of the traditional role of the teacher.

NCLB is of course based on bringing the idea of accountability to the school system. Rigorous and regular testing will sort out winners from losers — good school districts from bad, good schools from bad, good teachers from bad.

Whether NCLB is a good idea or a bad one is not my issue here. What interests me is what might unintentionally happen from this explosion of, and faith in, testing. This massive increase in testing has been enabled by the growth and development of database functionality. The power of database applications, software that stores information for easy retrieval and analysis, has exploded these last twenty years, creating extraordinarily powerful new functionality. Ironically, aggregating data on a group of people inevitably results in personalization, because you can see quite clearly how the individual differs from the norm.

You can see an interesting example of database functionality with collaborative filtering on Amazon.com: “Customers who bought this book also bought….†Amazon, in real time, compares everyone who bought that book with other things they've bought and takes the five most common matches and displays them dynamically. But that's the easy stuff.

What Amazon also does is compare your buying choices with everyone else's, lumps you into a particular category, then recommends other products you haven't bought but people like you have. In this way, they give you personalized recommendations.

Beyond e-commerce, the same trends are happening in advertising. “Targeted†and “accountable†are the biggest buzzwords in advertising now. The idea is to send ads to a person online and perhaps even TV that you already know that person is interested in, then use database functionality to make sure I’ve actually seen the ad. Inevitably, this same thinking will be — is being — brought into the classroom. The focus on building data on student performance will lead to personalized reports on individual students. An individual student’s statewide test results will be crunched and a report generated to the teacher detailing the student’s strength and weaknesses, and recommending various resources to improve those areas. Prescriptive analyzes is actually quite complicated to program, but once the data's been collected, you can bet that someone will be eager to crunch it.

Few teachers have the ability to do a prescriptive analysis of each of their students — and if they did have the ability, they certainly wouldn't have the time. If those reports are handed to them on a regular basis, with the expectation that they will act on that information, how can a teacher possibly design or even execute personalized learning strategies for even a significant minority of her students?

Teachers may end up focusing almost exclusively on being the interface between the student and the educational system rather than an active and engaged teacher. Or perhaps “teachers†will give narrow and specific lessons to every changing micro-groups of students who’ve been evaluated as having the same learning issue, while a number of “grade administrators†will be in charge of making sure students complete their assignments, get tested, and go to the correct teacher for specific reinforcement.

More likely, I’m not even close and some totally different structure will be formed. Was this the intention of the NCLB program? Not at all, but something like it could very well be its legacy.

Email: Craig Ullman