Specifically, Koretz outlines an example of the Cobra Effect known as Campbell’s Law in which social scientist Donald T. Campbell asserts “the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." Insufficient measures generate distortions, and Koretz argues that in education, standardized testing is the insufficient measure.
There are too many for a short review but Koretz does a good job organizing the basic misunderstandings, unintended consequences, and defeat devices associated with standardized testing. From the widening gaps among the intended, taught, and learned curricula… to the people-sorting nature of standardized tests… to the incentivizing of instructional behaviors that run counter to how people learn, Koretz reviews the specifics of how standardize testing is an incomplete instrument for evaluating teaching and learning.
More importantly, in using Campbell’s Law, Koretz includes education in a broader discussion about social change processes, cleverly applying in his thesis what he recommends for education evaluation; broaden the discussion, increase balance in the evaluation process, recognize that there are tradeoffs in managing complex systems, and stop trying to quantify everything.
I won’t throw a bowl of popcorn into the air the next time I see a standardized test, but I do fear the real snakes in the grass, the unintended consequences of misusing standardized tests. The charade, as Koretz calls it, will eventually come back to bite us.