teacherken's musings on everything

(this diary is cross-posted at dailykos
Here)

A new report issued this week, entitled The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing provides clear evidence that the added emphasis on high stakes testing in the federal education law known as No Child Left Behind (NCLB) is causing major distortions in the data being recorded because of the pressures on success ons cores imposed by the law.

Commissioned by the Great Lakes Center, and produced by Sharon Nichols of University of Texas at Arlington and David Berliner of Arizona State, the report begins with a reference to a well-known axiom of social science research, Campbell's law, which states

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

.

After the citation above, the authors note

Applying this principle, this study finds that the over-reliance on high-stakes testing has serious negative repercussions that are present at every level of the public school system.

I have pasted below the entire executive summary, which, along with the report in its entirely may be downloaded in pdf form  from the Great Lakes Center .  

Before you get to the Executive Summary, let me note that most of the problems identified with the imposition of high stakes testing have been well known for year.  There is an extensive body of research that demonstrates if sufficient weight is put on the results of certain measurements, those being measured will adjust their behavior in such a way as to give distorted results.  That is, the measurement becomes one of he response to being measured, not the underlying attribute you really want to measure.

We can see this in things like Florida now quietly negotiating to lower its standards of what would be non-compliance with NCLB, as just one example.

I offer the complete executive summary because of the importance of this subject.   I hope that at least a few peopl will take the time to read the complete report, as have I.  Here I can assure readers that -- unlike the case in much educational research  -- the executive summary IS supported by the underlying data (not like many of the reports released in press conferences by some on the right of the educational spectrum).

WARNING -  Berliner is known to be skeptical of much of the testing mania and of the assumptions underlying it.  He is a co-author of a book, along with Bruce J. Biddle, entitled The Manufactured Crisis, which attacked the the assumptions of the report issued by Reagan's Education Department, A Nation At Risk, which began the entire push to "save our schools" some two decades ago. I happen to think he is write, but it is possible that some will, because of his previous work, ignore or attack anything he has to say.

The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing

Sharon Nichols, University of Texas at San Antonio
and  David C. Berliner, Arizona State University

Executive Summary

     This research provides lengthy proof of a principle of social science known as Campbell's law:  "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." (1)  Applying this principle, this study finds that the over-reliance on high-stakes testing has serious negative repercussions that are present at every level of the public school system.  
     Standardized-test scores and the variables that lead to a mean score for school districts have become corruptible indicators because of the high stakes attached to them: employment status and bonus pay of teachers and administrators, promotion of a student to a higher grade, achievement of a high school degree, reconstitution or classification of a school, and federal and state funding that a school or school district receives.  
     Evidence of Campbell's law at work was found in hundreds of news stories across America, and almost all were written in the last few years.  The stories were gathered using LexisNexis, Inbox Robot, Google News Alerts, The New York Times, and Ed Week Online.  In addition to news stories, traditional research, studies, and stories told by educators about the effects of high-stakes testing are also part of the data.  The data fell into 10 categories.  Taken together these data unveil a striking picture of the corrupting effects of high-stakes testing:

1.    Administrator and Teacher Cheating:  In Texas, an administrator gave students who performed poorly on past standardized tests incorrect ID numbers to ensure their scores would not count toward the district average.

2.    Student Cheating:  Nearly half of 2,000 students in an online Gallop poll admitted they have cheated at least once on an exam or test.  Some students said they were surprised that the percentage was not higher.

3.    Exclusion of Low-Performance Students From Testing: In Tampa, a student who had a low GPA and failed portions of the state's standardized exam received a letter from the school encouraging him to drop out even though he was eligible to stay, take more courses to bring up his GPA, and retake the standardized exam.

4.    Misrepresentation of Student Dropouts:  In New York, thousands of students were counseled to leave high school and to try their hand at high school equivalency programs.  Students who enrolled in equivalency programs did not count as dropouts and did not have to pass the Regents' exams necessary for a high-school diploma.

5.    Teaching to the Test:  Teachers are forced to cut creative elements of their curriculum like art, creative writing, and hands-on activities to prepare students for the standardized tests.  In some cases, when standardized tests focus on math and reading skills, teachers abandon traditional subjects like social studies and science to drill students on test-taking skills.

6.    Narrowing the Curriculum:  In Florida, a fourth-grade teacher showed her students how to navigate through a 45-minute essay portion of the state's standardized exam.  The lesson was helpful for the test, but detrimental to emerging writers because it diluted their creativity and forced them to write in a rigid format.

7.    Conflicting Accountability Ratings:  In North Carolina, 32 schools rated excellent by the state failed to make federally mandated progress.

8.    Questions about the Meaning of Proficiency:  After raising achievement benchmarks, Maine considered lowering them over concerns that higher standards will hurt the state when it comes to No Child Left Behind.

9.    Declining Teacher Morale: A South Carolina sixth-grade teacher felt the pressure of standardized tests because she said her career was in the hands of 12-year-old students.
10.    Score Reporting Errors:  Harcourt Educational Measurement was hit with a $1.1 million fine for incorrectly grading 440,000 tests in California, accounting for more than 10 percent of the tests taken in the state that year.

High-stakes tests cannot be trusted - they are corrupted and distorted.  To avoid exhaustive investigations into these tests that turn educators into police, this research supports building a new indicator system that is not subject to the distortions of high-stakes testing.

FOOTNOTE:
(1)  Campbell, D. T. (1975). Assessing the impact of planned social change. In G. Lyons (Ed.), Social research and public policies: The Dartmouth/OECD Conference. (Chapter 1, pp 3-45). Hanover, NH: Dartmouth College, The Public Affairs Center. (p. 35)

Comments, suggestions and even rude remarks are welcomed!
Email accepted at "kber at earthlink dot net"
Preface email messages with "teacherken" so I know they are not spam.

Those who can, DO.
Those who can do more, TEACH!