Teaching Tips to Try Newsletter

February 21, 2010

From Linda The Math Coach

Go Beyond the Theory. Apply the Research. Make it work!

WHEN DATA LIES

How We Abuse Data and How to Set it
Right.
Data charts.
Data chats.
Data walls.
Data. Data. Data.
Data is everywhere.
Know your data.
Post your data.
Analyze your data.
Use your data.
We are sold on the idea that data should drive instruction, and it does.
We are told that “data doesn’t lie,” and it does.
So are we using data or abusing data? Let’s look at a few ways we have let data
go bad, and more importantly, how to make it right.

Looking At the Wrong Data

The first mistake we make with data is looking at the wrong data. We want our
students to do well on high stakes end of year testing. We huddle up during PD
time and analyze data. Question time: Are you looking at data on last year 8th
graders taught by last year teachers to set goals for this year 8th graders taught
by this year teachers? Hmm…. How would you like your doctor to hand you a
prescription for your migraine based on his diagnosis of his previous patient who
came to him with a fertility problem they wanted resolved?
Let’s not compare apples to oranges. If you want good results for this year 8th
graders, you need to consider where THEY are. Look at THEIR scores from the
previous year when they were 7th graders. Assess them to determine where they
are now in regard to specific skills and knowledge they will encounter on end of
year test. Use this more relevant data to inform lesson planning and instruction.

Looking at Wrong Growth Indicators

Another mistake we make surrounds measuring growth with pre and post testing.
Are you using pre and post test data to measure student growth of understanding
on every unit you teach? You know, the same question before-and–after picture
of student success. Your students quickly discover that they should just guess
on the pre test because it doesn’t matter. Then, when they take the real test,
they will look really smart and like they learned a lot compared to the nothing
they knew before. I have known teachers (under pressure to perform at the local
dog and pony data shows) who even led their students to believe this.
So are we really just wasting time with this kind of pre testing? Yes, you need to
know where students are and what they have already mastered. This is part of
informed instruction, but there are better ways to get this information untainted
by the incentive to demonstrate growth at the expense of integrity. I recommend
a general pre test of the major course-specific knowledge and skills given at the
beginning of the course. You can repeat this test several times throughout the
year to see how students are progressing. This is also the kind of test that you
will want to pull out and give to new students on their first day in class. Besides
giving them something useful and meaningful to do, it gives you and them an
immediate understanding of what will happen next.
It is also appropriate and useful to pre-assess arithmetic and calculator skills that
your students will need to deal successfully with the course. (Be prepared with a
plan to address any make up skills that need to be taught that will not punish or
hold students back from attaining the current curriculum.) The best such tests I
recommend are still teacher-made. You know what your students have to do
mentally, with pencil and paper, and with a calculator. If you are not sure, make it
a point to think about this and find out. Collaborate with other teachers and find
out their thoughts on this matter. Then devise a short easy to grade assessment
that will give you the information you need on an individual student basis. Give
this assessment to your new students as well.
Now, back to those unit pre tests. Why not turn your unit pre testing into a more
formative measure? Assess specific prerequisite knowledge and skills to
determine the entry level of each student into a mapped out progression of the
unit. If students know how to solve equations using the addition property, move
them forward to the next skill or sideways for enrichment. Continue to formatively
assess students as they work their way through the flow of the unit. Stop and
check for understanding of what you have taught and check for retention of what
students have practiced.
Looking at the Wrong Average
When you monitor success, are you looking at averages? Often, we lump
together groups of students by class or teacher and calculate an arithmetic
average. This is not a very useful statistic in that it doesn’t tell you how many
students are hitting the mark and certainly not who is hitting the mark. How are
you defining “average?” Where are you setting the bar for what you call
average? To simply state that 70% of the students need to be at 80% mastery
could put too much emphasis on test scores and ignore what really matters, what
do students know? What are they able to do? How are they developing problem
solving skills? How are individuals progressing?
If you use averages, start off with a clear definition of what you are averaging and
how you will determine an appropriate measure of that average. I recommend
you consider “average” to mean the mastery level of specific skills and
knowledge that students must have to pass successfully to the next unit of study.
An appropriate measure of such an average would determine the mastery level
of each skill and understanding of the content. A blanket score of 70% doesn’t tell
you what the student knows. It doesn’t even tell you what 30% they don’t know.
The 30% they are missing might be the critical piece that connects to a really
major skill or concept in the very next unit. While blanket averages may be
necessary for political decisions, they don’t offer useful information to a
classroom teacher.
Summing It Up
The cold hard fact about data? It lies. It lies when we want it to. It lies when we
are ignorant to where it comes from and how it is calculated. We would like to
think that data tells it like it is. It doesn’t. Ask any math teacher who has had
even one stats class. Data is an observational tool that can be used or abused.
If your task is to use data for analyzing results and informing instruction, keep the
following points in mind:
• Clearly define the student learning that is worth measuring.
• Administer the appropriate assessment to measure the level or degree of
student learning.
• Identify the kind of data that appropriately represents that learning.
• Base instructional decisions upon what the data tells you about individual
students, not trends of the group.

Next week read how to Use that Awful Data to Motivate Students to Action!
Your Coach,
Linda Cordes

Click Here To SUSCRIBE. You can have Teaching Tips To Try

Newsletter Delivered to you Weekly for FREE!

Quotes About Assessment and Data

Stiggins, Richard J. 2001. “The Principal’s Leadership Role in Assessment.”

NASSP Bulletin (January 2001): 13–26.

“To assess student achievement accurately, teachers and administrators must

understand the achievement targets their students are to master. They cannot
assess (let alone teach) achievement that has not been defined”.

42.7% of all statistics are made up on the spot. –Unknown

He uses statistics as a drunken man uses lamp-posts—for support rather than

illumination.—Andrew Lang

I don’t divide the world into the weak and the strong, or the successes and the
failures, those who make it or those who don’t. I divide the world into learners
and non-learners. –Benjamin Barber

If the brain were simple enough for us to understand it, we would be too simple
to understand it.—Ken Hill

The normal curve is a distribution most appropriate to chance and random

activity. Education is a purposeful activity and we seek to have students learn
what we would teach. Therefore, if we are effective, the distribution of grades will
be anything but a normal curve. In fact, a normal curve is evidence of our failure
to teach.—Benjamin Bloom