Nancy Flanagan is a retired teacher with decades of experience. In this post, she remembers when she used to take standardized test scores seriously. Then she went to a state board meeting in Michigan, where the topic of discussion was setting cut scores. Cut scores are the lines that determine whether students scored “advanced,” “proficient,” “basic,” or “below basic.” 

What she learned was that the cut scores are arbitrary. There is no science involved in setting the cut scores. It’s guesswork. The cut scores can be moved up or down to produce good news or bad news. 

She writes: 

Here’s the (incendiary) headline: Test Scores Show Dramatic Declines!

Here’s the truth: this set of test scores tells us nothing for certain. The data are apples-to-oranges-to bowling balls muddled. If anything, if you still believe test scores give us valuable information, the data might be mildly encouraging, considering what students have encountered over the past 18 months…

The problem is this: You can’t talk about good schools or good teachers or even “lost learning”any more, without a mountain of numbers. Which can be inscrutable to nearly everyone, including those making policies impacting millions of children. When it comes to standardized test score analysis, we are collectively illiterate. And this year’s data? It’s meaningless.

Bridge Magazine (headline: Test Scores Slump) provides up/down testing data for every school district in Michigan. The accompanying article includes plenty of expert opinion on how suspect and incomplete the numbers are, but starts out with sky-is-falling paragraphs: In English, the share of third-graders considered “proficient” or higher dropped from 45.1 percent to 42.8 percent; in sixth-grade math, from 35.1 percent to 28.6 percent; in eighth-grade social studies, from 28 percent to 25.9 percent.

These are, of course, aggregated statewide numbers. Down a few percent, pretty much across the board. Unsurprising, given the conditions under which most elementary and middle school students were learning. Down the most for students of color and those in poverty—again, unsurprising. Still, there’s also immense score variance, school to school, even grade to grade. The aggregate numbers don’t tell the whole story–or even the right story.

The media seemed to prefer a bad-news advertising campaign for the alarming idea that our kids are falling further behind. Behind whom, is what I want to know? Aren’t we all in this together? Is a two-point-something score drop while a virus rages reason to clutch your academic pearls?…

It’s time to end our national love affair with testing, to make all Americans understand that educational testing is a sham that’s harmed many children. Testing hasn’t ever worked to improve public education outcomes, and it’s especially wasteful and subject to misinterpretation right now