Should we get rid of standardized testing? - Arlo Kempf
The first standardized tests
that we know of
were administered in China
over 2,000 years ago
during the Han dynasty.
Chinese officials used them to determine
aptitude for various government posts.
The subject matter included philosophy,
farming,
and even military tactics.
Standardized tests continued to be used
around the world for the next two millennia,
and today, they're used for everything
from evaluating stair climbs
for firefighters in France
to language examinations
for diplomats in Canada
to students in schools.
Some standardized tests measure scores
only in relation to the results
of other test takers.
Others measure performances on how well
test takers meet predetermined criteria.
So the stair climb for the firefighter
could be measured by comparing
the time of the climb
to that of all other firefighters.
This might be expressed in what
many call a bell curve.
Or it could be evaluated with reference
to set criteria,
such as carrying a certain amount
of weight a certain distance
up a certain number of stairs.
Similarly, the diplomat might be measured
against other test-taking diplomats,
or against a set of fixed criteria,
which demonstrate different levels
of language proficiency.
And all of these results can be expressed
using something called a percentile.
If a diplomat is in the 70th percentile,
70% of test takers scored below her.
If she scored in the 30th percentile,
70% of test takers scored above her.
Although standardized tests
are sometimes controversial,
they're simply a tool.
As a thought experiment,
think of a standardized test as a ruler.
A ruler's usefulness
depends on two things.
First, the job we ask it to do.
Our ruler can't measure
the temperature outside
or how loud someone is singing.
Second, the ruler's usefulness depends
on its design.
Say you need to measure the circumference
of an orange.
Our ruler measures length,
which is the right quantity,
but it hasn't been designed with the
flexibility required for the task at hand.
So, if standardized tests are given
the wrong job,
or aren't designed properly,
they may end up measuring
the wrong things.
In the case of schools,
students with test anxiety may have
trouble performing their best
on a standardized test,
not because they don't know the answers,
but because they're feeling too nervous
to share what they've learned.
Students with reading challenges
may struggle with the wording
of a math problem,
so their test results may better reflect
their literacy
rather than numeracy skills.
And students who were confused by examples
on tests that contain
unfamiliar cultural references
may do poorly,
telling us more about the test taker's
cultural familiarity
than their academic learning.
In these cases, the tests may need
to be designed differently.
Standardized tests can also
have a hard time
measuring abstract
characteristics or skills,
such as creativity, critical thinking,
and collaboration.
If we design a test poorly,
or ask it to do the wrong job,
or a job it's not very good at,
the results may not be reliable or valid.
Reliability and validity
are two critical ideas
for understanding standardized tests.
To understand the difference between them,
we can use the metaphor
of two broken thermometers.
An unreliable thermometer
gives you a different reading
each time you take your temperature,
and the reliable but invalid thermometer
is consistently ten degrees too hot.
Validity also depends on accurate
interpretations of results.
If people say results of a test
mean something they don't,
that test may have a validity problem.
Just as we wouldn't expect a ruler
to tell us how much an elephant weighs,
or what it had for breakfast,
we can't expect standardized tests alone
to reliably tell us how smart someone is,
how diplomats will handle
a tough situation,
or how brave a firefighter
might turn out to be.
So standardized tests may help us learn
a little about a lot of people
in a short time,
but they usually can't tell us a lot
about a single person.
Many social scientists worry about
test scores resulting in sweeping
and often negative changes
for test takers,
sometimes with long-term
life consequences.
We can't blame the tests, though.
It's up to us to use the right tests
for the right jobs,
and to interpret results appropriately.
