teacherken's musings on everything

crossposted at dailykos, myleftwing, teacherken.blogspot.com

Tests dominate American education. Even before NCLB insisted on evidence that all children in grades 3-8 be tested every year in Math and Reading (and starting shortly in Science) and that schools demonstrate that all disaggregated groups be making Annual Year Progress towards the unattainable goal of 100% proficiency by all groups in all grades in all subjects by 2014, we had a heavy reliance upon tests. As a result of the attack on American public schools in the inaccurate “A Nation at Risk” most states had already moved towards high stakes tests for graduation in in some cases for promotion. And our incessant desire for rankings and competition had led to Jay Mathews of the Washington Post creating his Challenge Index of the ratio of AP tests divided by graduating seniors in American High Schools.

Mathews has started something of a firestorm with a piece he recently wrote that appeared on the op-ed page of the Post in which he encouraged teaching to the test. Our current Secretary of Education has justified teaching to the test as improving of education. I had until today refrained from commenting on the controversy. I have decided that I need to offer my voice.

The immediate reason for my deciding to comment is a piece in response to Mathews by Colman McCarthy that appears on today’s Washington Post op; ed page. I will discuss it further down. I first want to explore Jay’s original piece. My use of his first name requires me to disclose that we are friends, that I have been quoted in his columns more than a dozen times and that we frequently have non-public dialogs on education. Jay knows that I disagree with his emphasis on tests, and thus yesterday he gave me a ‘heads up” on McCarthy’s piece. After reading it early this morning I thought I ought to comment on the issue from the perspective of a teacher who must confront not only Jay’s favored AP tests (I have 77 students in AP US Government) but also mandatory state tests as well (for 65 of those 77 - some have previously sat for the state test - and 74 more).

Jay’s original piece appeared February 20. Entitled Let’s Teach to The Test In it he argues that “teaching to the test” is

the most deceptive phrase in education today

After a discussion of what he means by the term, which he insists is not mere drill and kill, which he says strong teachers are able to resist, he attempts to reframe the issue as addressing any course which requires assessment of students by tests prepared by outsiders. He then offers this paragraph:

Yet if you asked the thousands of educators who have written the questions for the state tests that allegedly produce all these terrible classroom practices, they would tell you their objective is the same as the classroom teacher's: to help kids learn. And if you watched the best teachers at work, as I have many times, you would see them treating the state test as nothing more than another useful guide and motivator, with no significant change in the way they present their lessons.

There is so much in just this paragraph that is a problem. First, many tests items (questions) are NOT produced by classroom teachers. Even if classroom teachers may write the individual items, that does not mean the items are necessarily good measures of knowledge or skill in the domain being assessed, even if the distribution of the items fairly samples the domain. And while Jay has never actually seen me test, he has talked with students I have taught and their parents and knows that I am considered a pretty good teacher. AND YET - even I have to acknowledge that how I teach IS affected by the fact that my students will sit for such outside tests that bear significant consequences. I have to adjust my planning to give my students practice at the kinds of questions they will face. I have to teach them on multiple choice items to read all possible answers, eliminating those they know are wrong and picking from what is left whatever is least inaccurate. That’s because sometimes the way the question is phrased it has no correct answer, or perhaps it has more than one answer that technically could be considered correct.

I also have to devote some class time to practice sessions, so that students have some ability to learn how to pace themselves. For the AP Government exam, I have to teach my students how to do a kind of writing which is NOT good writing. Bear with me. They will have to do 4 “free response questions” (FRQs) in 100 minutes, 25 minutes each. And yet the way those questions are scored taking time for either a proper topic sentence or an effective conclusion is hurting the student, because it is rare that the way a question is scored gives credit for either item. I have to use class time, homework time, and my time for correcting their efforts, to teach them a kind of writing that has little purpose outside of this particular test.

Were I not to make this effort, which is clearly a significant change to how I would otherwise teach, my students would not be able to demonstrate how effectively they understand the content and the concepts.

Jays’ next paragraph is equally problematic:

hose who complain are not really talking about teaching to the state test. Unless teachers sneak into the counseling office and steal a copy, which can get them fired, they don't know what's on the test. They are teaching not to the test but to the state standards -- a long list of things students are supposed to learn in each subject area, as approved by the state school board.

Actually that is very much of an oversimplification on both points. First, for most tests there is a universe of released items and released forms (examples of tests -- and here it is important to note that on many tests, such as our high-stakes High School Assessments in Maryland, not all students take the same tests -- it is not merely that questions or sections may be in different order, as is that case on the SAT, but that students actually answer different questions, and in some cases different distribution of the types -- long essays, short constructed responses, multiple choice questions - within the test). Teachers - and students - turn to these released questions and forms as a means of deconstructing the test itself, of seeing the kinds of topics that are addressed, at how the test-makers attempt to mislead students (and remember, wrong answers are technically called “distractors”!). One does not have to know the specific items that will appear to be able to determine much about the forthcoming test.

The final sentence about state standards demonstrates a real part of the problem. First, even Jay acknowledges the idea of “long lists” of items. I might note that in other countries with which we are often (inaccurately and erroneously, in my opinion) are compared students are not required to “cover” as much. Such coverage often leads to a surface knowledge which is addressed by racing through ideas without much exploration in depth. In those countries fewer ideas are covered in greater depth so that students develop the ability to apply concepts and skill more broadly on their own.

Such long lists of standards that students must meet often mean that not all domains among the standards can be assessed, and/or that what assessment which does occur is limited to selecting one choice out of four or five. That a required content matter appears as a distractor (wrong answer) does not mean that you have truly addressed whether the student has learned that content.

Multiple choice items can be quickly and cheaply scored, so we tend to rely on them far too much. I have had students who know how to address such questions (by process of elimination) who do successfully but who could never provide the answers on their own. Have they really learned the content being assessed> Are their scores an accurate measurement of their underlying knowledge? I think not. Conversely, I have students who truly understand the domain being assessed who get frustrated because they can recognize the existence of more than one technically correct answer or the erroneous framing of the stem (question) such that there is no correct answer. Does that shock of recognition negatively impact their performance? I am unaware of any serious exploration of this issue in the research literature.

Finally, before leaving this paragraph, let me note that a large number of advocates of our testing approach are quite willing to argue that the tests SHOULD drive the instruction. Heck, even if (a) the standards were appropriate, and (b) the tests were an accurate measurement of the learning that has occurred, and (c) the tests were properly aligned with the standards both as to content and as to distribution, should not the tests be measuring the quality of the learning demonstrated by the students (not all of which is an artifice of the instruction received from the teachers) than attempting to drive the instruction itself?

if there is any doubt on this last point, all one need to do is consider how often those who actually produce the test (that is, the corporations which sell the tests) also sell curricula, test prep materials, and the like. One cannot help but wonder about the real motives of many involved in strong advocacy for such tests.

I cannot go through Jay’s entire piece in the same fashion. Let me acknowledge one thing -- he does provide an opportunity for those who disagree with him to respond. And times he has opened his on-line column, Class Struggles (which was the title of a book he wrote on education a number of years ago which featured the high school I had attended which is what led me to first contacting him) as a place where people can offer extended responses, or in which opposing points of view can be expressed in an interchange between advocates of both sides. This past week (Tuesday, March 14) Jay used his column to to present a number of responses to his print op-ed. If you are interested, you can read that column, Backlash to “Let’s Teach to the Test”.

As noted (far too many words ago) Colman McCarthy, who used to be a Post columnist, and who has been an advocate and exemplar of teaching Peace Studies in high schools (which has also been the subject of some controversy -- see here and here ), today has an op ed in response, entitled ’Teach to the Test?’ What Test?. He begins by taking Mathews to task for writing

"in 23 years of visiting classrooms I have yet to see any teacher preparing kids for exams in ways that were not careful, sensible and likely to produce more learning."

by noting in response

On Mathews's visit to my classroom four years ago -- at School Without Walls, where I have been volunteering since 1982 -- he must not have noticed that not only was I not preparing my 28 students for tests but that I regard tests as educational insults. At School Without Walls and two other high schools where I am a guest teacher -- Wilson High School in the District and Bethesda-Chevy Chase High School in lower Montgomery County -- I have never given a test. I respect my students too much to demean them with exercises in fake knowledge.

Now in fairness, McCarthy's response on this point is a bit unfair - his class is an elective for which he has almost total control as to content, methods of instruction, and methods of assessment, and thus would not fairly be included in what Jay was attempting to address.

I should also disclose that while I am not a “friend” of McCarthy, Colman presented a session on teaching peace studies to a group of those who either taught in Quaker Schools (most of the attendees) or were themselves Quakers who taught in public or other non-Quaker schools (one other person besides me). I am also in orientation very much in agreement with the point of his second paragraph, especially its final sentence:

Tests represent fear-based learning, the opposite of learning based on desire. Frightened and fretting with pre-test jitters, students stuff their minds with information they disgorge on exam sheets and sweat out the results. I know of no meaningful evidence that acing tests has anything to do with students' character development or whether their natural instincts for idealism or altruism are nurtured.

McCarthy used Mathews’ column as a basis of exploring with his students their own attitude towards cheating. What should not surprise most readers is that the vast majority of students acknowledged having cheated, and that most would, if they could be reasonably assured of not being detected, cheat again.

McCarthy has a different concern, one which is not often discussed in our various conversations on education:

Standardized tests measure braininess and memory skills. American society has plenty of people who were academic whizzes in high school but were so driven by the lure of a high grade-point average that their spiritual lives remained stunted. I worry about students who make too many A's. What parts of their inner lives are they sacrificing to conform to someone else's notion that doing well in tests means doing well in life? Is any time left over from mastering theoretical knowledge for gaining the kind of experiential knowledge found in community service or volunteering in programs such as Special Olympics or DC Reads?

Most of us who teach could not take the approach he offers in his penultimate paragraph:

To compensate for my no-testing policy, I assign tons of homework. The assignments? Tell someone you love him or her. Do a favor for someone who won't know you did it. Say a kind word to the workers at the school: the people who clean the toilets, cook the food, drive the buses and heat the buildings. And a warning: If you don't do the homework, you'll fail. You'll fail your better self, you'll fail to make the world better, you'll fail at being a peacemaker.

And let’s be frank - not enough of our schools provide an opportunity for students to explore how to be a peacemaker. We may have opportunities for peer mediation, but even those are not universally available. And perhaps many parents, and administrators, and elected officials would be comfortable with assignments such as those McCarthy describes, even though he is but speaking of common courtesy writ large, something which is nowadays rarely displayed by our political leaders who seek the largest political and personal advantage from almost every incident and situation, courtesy or the other person (or nation) be damned.

I know many young people - current and former students alike - who would offer strong agreement for McCarthy’s final paragraph:

or 25 years of testing the waters by not testing, I've been telling my students not to worry about answering questions. Be braver and bolder: Question the answers. Which answers? To start, the ones from anyone who champions classroom get-aheadism based on test scores. Throw off your chains, students. You have nothing to lose but your backpacks.

This is already a very long posting. But I owe those who have come so far already some insight into my own thinking on this subject. I teach in a public school, and most often the courses I teach are in some fashion required for my students. Thus I must accept that people will want some independent evaluation of the “effectiveness” of what my students encounter in my classroom and with my assigned work. I have taught electives -- in Social Issues and in Comparative Religion - in which I gave no tests, and thus was able to explore issues in far more depth. Students who have taken both my Government classes (required) and one or both of the electives have often told me that they learned more - about themselves, about society, about learning and thinking - in those electives than they did even in my often quite challenging and provocative government classes.

Above I noted that the existence of external tests inevitably influences what occurs in my classroom. I cannot avoid my responsibility for preparing my students to do well on those tests. That takes time away from other things I might want to explore. it limits my ability to respond to events in the world and in the lives of my students that might be far more meaningful in connecting them with the domain. To be fair, I have never had a principal tell me HOW to teach my students. I have at times been questioned, challenged to justify a specific approach, although even this is rare. I must acknowledge that one big reason for the flexibility I have is because my students do quite well on the external tests, and that the only discrepancy between the grades I award for the work I assign and the scores on the tests is the aforementioned example of those who do not do as well for me as they do for the external tests. In my case I benefit in some measure from the existence of the scores on external tests.

But I still do not like them. They are incomplete and gross measures of what my students can know or do. Unlike my tests, they do not provide meaningful feedback either to the students or to me -- the scores are not provided ina fashion timely enough to shape ongoing instruction, to demonstrate my need to reteach a sub-domain because far too many students misunderstood, or for them to realize a need to reexamine the subject, perhaps to obtain extra help (which I do offer, and which is also available in the form of tutoring by National Honor Society members as part of their service requirement).

I worry that we are cheating our young people of their right to true education by our increasing emphasis on tests. I think there may now be enough evidence of the counterproductivity of the approach we have been taking. “A Nation at Risk” came out in 1983. Since then we have ramped up our emphasis on standards and high stakes tests. And yet now we are constantly told that things are even worse. Im will not in examine in detail some of the worst examples of this, such as John Stossel’’s recent atrocity on 20-20, which like far too many such exemplars was apparently based on false or selective reading of a limited subset of information in order to reach a predetermined position.

There are things that need to be addressed in how we do education in this country. Often my posting in electronic fora such as those in which this piece appears has to be in response to issues such as this, rather than on things I think far more important. This issue is prominent enough that I felt I must respond, even though I doubt that I will able to persuade a sufficient number of people to change the structure of our debate on education. B ut perhaps as a result of reading this I will be able to inspire a few people to make a difference. Perhaps a parent will challenge the conventional wisdom on tests with a schoolboard member or a state legislator. Perhaps a teacher will decide to take the risk to focusing on more important aspects of learning and trust that the entire class does not have to be focused on performance on an external test.

I do not know what responses will be engendered. I look forward to any offered here in the form of comments, or in off-line responses to me, or in extended independent postings.

Now I must return to my primary task for the day, which inevitably includes preparing my students to do well on external tests.

Peace.

Comments, suggestions and even rude remarks are welcomed!
Email accepted at "kber at earthlink dot net"
Preface email messages with "teacherken" so I know they are not spam.