The Sham of Standardized Tests

…or, How Much Math Does Your Child Know?

I raised that question in an earlier post, and I’d like to add a few thoughts on the subject.  And some of them will likely surprise you.

If you’re reading this, you probably agree that it’s important for kids to learn math.  If for no other reason, to avoid the College Math Tax, which I discussed earlier.

So, how can you find out how much math your child knows?

Seems like a reasonable question doesn’t it?  After all, they teach math in school, and they do a lot of testing.

It makes sense that somewhere in all that testing, they must have the answer to that question.

But, if you thought that, you’d be wrong!

Sounds kind of crazy doesn’t it?  But I assure you it’s true.  Your child’s school cannot answer that question for you.

Let’s start with the “high-stakes” tests you hear so much about.  For decades, these tests were developed by individual states and administered to students in most grades.

Now, these tests are being replaced in some states (fewer every day fortunately) by either the SBAC or PARCC tests that are designed to assess whether students have met the Common Core Math Standards.

In some states, students must pass some of these tests to move on to the next grade.  That’s one reason they call them “high-stakes” tests.  The main reason, however, is that it determines how much money the schools get.  But enough of that…

These high-stakes tests are also referred to as “standardized” tests.  (Sounds impressive, doesn’t it.) What it means, technically, has to do with how the test is created, and the kind of information it gives you.

I won’t bore you with the details, but I can tell you they’re not the kind of tests a teacher makes and gives to the class.  They’re created by people with backgrounds in Psychology and Statistics, and they’re often referred to as “psychometric tests.”

A psychometric test is used to measure something you can’t see, touch, hear, smell, or otherwise identify as an actual physical thing – like how much math someone knows.

Because a standardized test measures these “psychological” things, the test builders have to create a “measurement scale” that measures whatever it is the test is designed to measure.  In order to do that, they have to make it conform to some specific statistical requirements so the data can be “interpreted.”

For one thing, that means the items on the test must never be revealed to the public, including teachers.  If word got out, it would destroy the “validity” of the test.  In other words, because people would cheat!

So one thing you will never get from these tests is anything specific about what kinds of topics your child answered correctly on and what kind they answered incorrectly.  But isn’t that what you want to know?

So what do these tests tell you?   Wait for it…

They rank order students.  That’s it.

The score they report is called a “scale score” because it’s the score on the “measurement scale” they created.  But, surprisingly, the one thing they can’t tell you is what that scale measures, except some vague notion like “math ability” or “math achievement” or something else made to sound like its related to math.

An even scarier thing is that many of these tests tell you more about the parents’ income level or the child’s reading level than it does about anything to do with math.  But that’s a story for another day.

Your child’s “scale score” tells you one thing, and one thing only – how your child ranked among all the students who took the test.  If the norm (the average score) was 400 and your child scored 450, you know your child was above the norm.

Unfortunately, what you don’t know is how much math the “norm” student knows.  Your child could be well above the norm and still not have learned much math.  That’s the hidden fallacy in this kind of testing.

So here’s the skinny on standardized tests.

They rank order students, but you don’t really know what they’re being rank ordered on, and you can’t tell what it means if you did!

A little disappointing, huh?

So what about other tests?  Well, there are benchmark tests that most School Districts now administer about 3-4 times a year.  They use these to assess how much the students have learned on particular topics.  But again, there are problems.

First, the details of the results of these tests are almost never given to parents.  And even if they were, they don’t tell the whole story.  The reason?

Perhaps the most important information you need to have access to is what your child knows about topics taught in prior grades.  Gaps in knowledge from prior grades has an enormous impact on how much your child is able to learn in the current grade.  But once these gaps develop, they’re gone from sight for good.

But benchmark tests (or any others) don’t tell you whether the gaps are getting worse, staying the same, or getting better.  Unfortunately, without specific, individualized attention, they almost never get better.  Instead, they usually get worse, and the impact can be devastating.

And that’s the subject of a later post: the hidden problem in learning math.