Fighting Data with Data:
The educational and political benefits of disaggregated
Supplying student demographic data on test answer sheets can be a pain. "You mean we have to look up all this stuff in each studentís file?" we ask ourselves.
that the information being requested is the foundation for understanding what is going on, achievement wise, at a school and that the same information is the basis for "fighting data with data." By fighting data with data I mean responding to external criticisms of our test scores with additional data of our own.
Disaggregated Data and the Political Arena
Consider this scenario:
The local gazette reports your schoolís test scores alongside all the other schools in the district. Your reading scores are well below those from the norm group and the lower third of those in the district. Some rather inflammatory statements are made in the paper and your school is called on the carpet for being "less than what we expect from our school."
What the article did not say!
Your schoolís limited English population is 24 percent. The percent Limited English in the norm group was 3 percent. The test publisher stated, in writing, "Any student who is limited English and takes this test is taking it under handicapping conditions." If 3 percent of the norm group took the test under handicapping conditions and 24 percent of your students took the test under handicapping conditions, itís hardly a fair comparison!
Extracting and reporting the scores of those students who are not handicapped when taking the test, at least from a language perspective, provides a far more meaningful comparative base.
Responding to the media
You respond to the mediaís indictment by saying something like, "Yes, we were at the 38th percentile in reading. However, we had a large percentage of our students who, according to the test publisher, were handicapped when taking the test because of their limited understanding of English. When you consider only those students who were not handicapped by language, our reading scores were at the 54th percentile. This group is really the group that is most comparable the national norm."
I recommend that when a situation like the above occurs, you acknowledge what the data say, then you rebut the critics with a more valid set of data that comes from using a subgroup of students who tell a more accurate story.
The capability of a school being able to stand tall is totally dependent upon the accuracy of the identifying information on the studentsí answer sheets. There are several demographic variables over which you have no control--such as language proficiency or mobility--that can account, in part, for your test scores being as they are. Reasonable people, those who donít play "gotcha" to promote their own agendas, will reasonably acknowledge this situation.
Disaggregated Data and the Educational Arena
Consider this scenario:
Your Grade 6, 7, and 8 test data show a consistent pattern of girls outscoring the boys in each grade. This pattern has existed over the past two years. The need for content mastery by all students must dispel the myth that itís okay for results like the above because "thatís the way it is. Girls are simply better readers than boys." In this case, we reject the notion and set out to determine if there are inequities in the instructional program we provide to girls and the one we provide to boys, even though in our eyes, itís exactly the same program. Once, in our judgment, causal factors have been identified, we set out to do something about them.
So often we consider test scores to be solely a political issue. Such is not the case. Focusing on educational implications can go far to assist us in our work. Give test scores a chance to work for you. You'll find disaggregated data far more meaningful than total group data when describing and explaining test scores. Generating accurate demographic data is essential and never to be taken lightly, educationally or politically.
Other Columns by Jim Cox: