from a public HS teacher (Gov't, Religion, Soc. Issues), who is eclectic (Dem-leaning) politically and Quaker (& open) on everything else. Hope you enjoy what you find here.

Monday, February 07, 2005

What is a "well-designed" testing program? 

This blog is a copy of a comment I posted on dailykos today here in response to this

EXCEPT that I have gone through and cleaned up the typos!!!

Define "Well-Designed" testing program :

because what I have seen teaching in Maryland and Virginia is hardly what I would call well-designed, and what I have tracked around the country while I was doing doctoral work was hardly well-designed.

Tests can provide one piece of information, but in isolation, since usually a test is but a sampling of a domain, regardless of where you set test scores you will get errors  -- either false positive, that is, people who pass that administration because of an artificially high score when the cut score is set too low [to eliminate false negative];   or false negatives, when a person scores artificially low on that particular administration and the cut score is set too high [to elmiinate false positives],   --  or a mixture of both because the cut score is set somewhere in the middle.

Most of the tests that are being used are NOT designed to measure a minimum competence level, but to sort people.  That's one problem.   Another is that a test that allows valid inferences to be drawn for one purpose, that is, how well the student is performing, is almost certainly not designed to allow the drawing of valid inferences for other purposes, that is, how well the school is performing.   The two are not the same.  And the problem is even worse when using only multiple choice items that allow for only one correct answer in domains where another answer may be partially right  -- this binary scoring fails to give meaningful information even about what the student knows or does not know.  And that assumes that the question is well designed, which I assure you is quite often NOT the case.

You want to argue first things first.  NCLB is not just the testing, it is how the tests are being misused.  The entire way Annual Yearly Progress is being measured, incluidng in subgroups,  pretty much guarantees that most schools are going to be found not to measure up  .. and that is clearly the intent, regardless of what is stated.

We are taking guidance on testing from people from Texas.  May I remind you that the so-called Houston Miracle under Rod Paige was FAKE  -- students were tested in 10th grade, but minorities were held back in 9th grade for two or three years 'til they dropped out.  But they didn't count as dropouts because they would be coded as intending to get a GED.  You had high schools in Houston claiming dropout rates of less than 5% when the system as a whole ws graduating less than 40% of those who entered in 7th grade.  Walt Haney of Boston Colelge took all this apart BEFORE Congress passed the monstrosity of NCLB.

First things first  -- people learn to read by reading, and learn to do math by doing math.  The real purpose of tests should be to inform both the student and the teacher of what the student does not understand and for which s/he needs help.  But these tests serve no such purpose.  And the measuring of this year's group of 10th graders against next year's tells you nothing  --  you have no idea how the characteristics [prior knowledge, socioeconomics, reading or math ability, etc.] compare between the two cohorts.  Uncontrolled variance in the cohorts may account for as much as 90% of the difference in performance.

We have been down this road many times before.   This bill is designed to bash public schools, transfer money to testing companies   [gee, Harcourt is based in Texas,  and Harold McGraw is a close personal friend of the Bush family], try to shift funds to charters [did you know the Jeb Bush forced the Florida Teachers' pension -- for public schools teachers  -- to buy the controlling interest in Edison Schools, that disaster of a for-profit business founded by Chris Whittle and touted by the likes of Bill Bennett and Lamar Alexander] and tutoring companies [Kaplan must be loving this, and remember, they are the MOST profitable division of the company that owns the Washington Post and Newsweek].

I tried to persaude people in Congress that this program would be a disaster.  I exchanged emails with George Miller.  I lobbied my own Congressman and talked with his point person on education.  Democrats went along because they thought they would get more Federal money for public schools.   Hah.   They got more mandates, which more than  exceed in cost any additional money, and some of the money they already had will be transfered away from them.

NCLB is a sham.  That is why Republican legislatures are objecting.  The only reason the Utah legislature did not formally oppose before the election is that they were warned if they did so they might find a military base or two suddenly being closed.  

NCLB has done more damage to public schools than  did that atrocity and inaccurate report entitled A Nation at Risk  (whose executive summary did not even agree with its own contents, and whose main assumption, that we were going to suffer disastrously in economics to places like Japan, has clearly been demonstrated to be false), and that is saying something.

Hey, Democrats are as much to blame.  Mark Warner here in Virginia has bought into the idea of testing.  Bill Clinton was head of NGA when they helped produce the atrocious GOALS 2000 during the Bush I presidency  [remember, we were going to be First in the World in Math and Science by 200  --  so what happened?   is it possible tht the changes we made then DAMAGED our public schools?  Have we learned NOTHING??].

Sorry, on this one I not only disagree, I do so vehemently, not just from my experience as a classroom teacher, but as one who is published on the subject, who has done extenseive reading in the literature on educational policy and the effects in particular of various testing regimes.

FEEL FREE TO CONTACT ME OFFLINE at kber@earthlink.net Comments, suggestions and even rude remarks are welcomed! Preface any messages with "teacherken" so I know they are not spam.
I agree with you on this. Testing is not the way to go, but just to be devil's advocate, the reason it's done is to hold incompetent teachers accountable. Now I know you and I are not those kinds of teachers, but think about it. How many people in your school should not be in education? If the teachers unions could break out of the age-old tenure and lifelong jobs rut, we might be able to weed out the weak teachers and make teaching what it should be. Come to think of it, if we could weed out some of those teachers at the pre-service level, we'd be even better off!
Testing measuring what students know at one particular moment in time does not give an accurate measure even of what they learned in that particular class. Even pre- and post-testing does not necessarily control for factors outside of the class contributing to the results of that teacher.

As far as your question, in my department of 17 teachers, there is no one who should not be in teaching, and only two I can think of who may be overmatched by the rest of us.

I believe in teachers' unions, and served as building rep, because I have observed far too many systems where administrators and school board members were the real problem and where teachers were denied due process. The proper function of teachers' unions is quite off the topic of my post, and really belngs on a separate thread.

As does the issue of "weeding out" at a preservice level. Certainly there is much we can do to improve the preparation of those aspiring to be teachers. I do not start with the presumption that most need to be weeded out. I rather think that the prepartion eneds significant alteration, as does the support with which new teachers are provided.

But again, as interesting as these issues are, they are separate from the issue of my post, about the nature of a "well-desinged" testing program, an expression that I quite frankly view as an oxymoron. And on thjis I will end with what Jay Mathews of the Washington Post once called a teacher's credo, quoting me as remarking that I will not object to my students having to sit for outside tests provided you do not attempt to mandate how I prepare them for those tests. I will not teach to the test, I will not follow your scope and sequence within the course, and I sure as heck won't follow your pacing guide closely, because I am going to be paying attention to what my students are grasping, and with what they a re still struggling. And in the end, my test scores will exceed those of most if not all of those who followed your strict guidelines. And so long as my students "perform" well, I am left alone to teach as I think best those who pass through my classroom.
I certainly see your point, and like I said, agree with you. I think that the 'standardized' tests as given are a joke. They do not account for gaps in what is perceived as 'common' knowledge from either geographical or econonmic differences.

I agree with you, as well, that teacher unions serve an important function protecting teachers from abusive administrators, but I feel that at times they also get in the way and allow lazy teachers an excuse to not do what is best for students. I have seen it both ways, having taught in both Illinois (strong unions) and Texas (right to work), but like you said, that is an entirely different post.
seen your blog. great work.
when you have time, pay a visit to my site here:

global domains
Post a Comment

This page is powered by Blogger. Isn't yours?