teacherken's musings on everything

originally posted at Dailykos

Educational software, a $2 billion-a-year industry that has become the darling of school systems across the country, has no significant impact on student performance, according to a study by the U.S. Department of Education.

The long-awaited report amounts to a rebuke of educational technology, a business whose growth has been spurred by schools desperate for ways to meet the testing mandates of President Bush's No Child Left Behind law.

Note the driver - schools desperate to meet the testing mandates of NCLB. And note the results - no significant impact

Come along with me for some commentary.

The quotes are from an article that appeared on the top left - above the fold - of the April 5 Washington Post entitled Software's Benefits On Tests In Doubt and subtitled Study Says Tools Don't Raise Scores. And it might seem amazing that the Bush Education Department would release a study which seemingly undercuts the profits of many who have been feeding at the NCLB trough. After all, Nekl Bush is in the business, and this Education department has been known to suppress other reports that undercut its asserted (should we say theologically based) educational positions. So let me offer one real caution from the article that I think most people miss, and please note the added emphasis:

"We are concerned that the technology that we have today isn't being utilized as effectively as it can be to raise student achievement," said Katherine McLane, spokeswoman for the Department of Education.

In other words, it is not the DOE is saying that there is a problem with or a waste of funds expended on such software. Want to bet that eventually it will all be blamed on teachers who don't use the software properly? Well, all we need to do is to read a bit further,so let me offer two snips from the industry:

Industry officials played down the study and attributed most of the problems to poor training and execution of the programs in classrooms.

and

"This may sound flip or like we're making excuses, but the fact is that technology is only one part of it, and the implementation of the technology is critical to success," said Schneiderman, whose group represents 150 companies that produce educational software.

This was a Congressionally mandated study. Let me quote again, adding emphasis:

The study, mandated by Congress when it passed No Child Left Behind in 2002, evaluated 15 reading and math products used by 9,424 students in 132 schools across the country during the 2004-05 school year. It is the largest study that has compared students who received the technology with those who did not, as measured by their scores on standardized tests. There were no statistically significant differences between students who used software and those who did not.

Let me note that the immediate past superintendent of the district in which I teach, Prince George's County MD, purchased one of the products evaluated, made by Leapfrog SchoolHouse, for over a million dollars, in a transaction in which (a) the superintendent was living with a saleswoman for the company, (b) when the company investigated she and the saleswoman who got credit for the sale were fired, and (c) he has been indicted for corruption for demanding kickbacks. Let me also note that one explanation for the high visibility placement of the article is buried well down in the article (after the jump from Page A01 to Page A07) and again I will offer some emphasis:

To persuade companies to participate in the study, researchers promised not to report the performances of particular programs. Among the businesses whose products were in the study were LeapFrog SchoolHouse, PLATO Learning, Scholastic Inc. and Pearson. (The Washington Post Co. owns Kaplan, a test preparation company that sells education software. Kaplan applied to be in the study but was not included.)

Folks, we are talking big bucks - Los Angeles spent over 50 million on software. And the explanations, from the DOE and the vendors, is that it is an issue of implementation, of training, of how the software is used.

I will not burden you with more examples of prose that supports the statement I just made - I have given you the link to the article, and you can decide whether or not my representation is fair.

Let me offer some commentary. Technology is a tool, never a silver bullet. It can be used in some cases to provide an inexpensive way of drill, and giving immediate feedback and correction to students, that can be useful IF THE SOFTWARE IS WELL DESIGNED. Sorry for the screaming, but I spent 20 years of my life in the field of data processing, and I am often quite cynical as a result. First, I have seen far too many products that are not all that well designed. Second, just as machine-scored bubble in test force a convergent thinking pattern that may not be an accurate measure of what students know and can do, many programs force one to thinki in a particular fashion, and unlike a well-trained human cannot make adjustments to the unpredicted. I am sure many of you with experience in the microcomputer world can remember many such occasions in your use of software.

Further, I have seen many efforts at technology tha were supposed to be magic solutions in my student days in high schools and colleges. They were almost inevitably oversold and had an unfortunate equal tendency of underperforming. To put it in terms that perhaps even Republicans can understand - they were NOT cost effective methods of supporting instruction.

What worries me is some of the rhetoric that is coming into play. Remember, the current atrocity of a Federal education policy was sold by the idea of leaving no child behind, even though the negative impact of NCLB has fallen most heavily on the schools and students it was purported to be designed to help, minorities, from lower socio-economic status, etc. Thus when one reads a statement from a University professor who is an advocate of such technology, who wants to discount the study because it only followed the students for one year, perhaps one should take note:

"This is the last thing that we need now," he said. "It is the poor kids who will suffer, because it is their schools who will not get technology because of this study."

I should inform those readers who don't know that many in universities are in bed with for-profit ventures. Heck, in many cases they create them. We have seen this in other fields, in the distortion of corporately funded scientific research, and it should not be surprising to also find it in Schools and Departments of Education. We have seen clear evidence of this in recent examinations of the corruption in the Reading First Initiative, especially with certain personalities at the University of Oregon.

I am NOT making specific accusations against the professors quoted in the article. It would be interesting to ask the reporter how we wound up talking with those particular professors. I wish I had the time for such a dialog.

Technology has its uses. I am certainly not a technophobe, as my 20 years in a variety of roles in DP should indicate. I make use of a variety of technologies in my teaching, in my classroom when I can, in various labs around the school. But I worry that this is indicative of an insidious trend. We want to do things in "rigorous" manners that diminish human variability. We want to replace human effort and the expenses associated with same by using technology instead. There are even well-developed trends towards computerized scoring of essays. I do not think these can all be justified as either more efficient or ultimately more productive, unless our only goals are maximizing profits for the vendors, reducing costs spent on personnel, creating more conformist school graduates, and so on. We might well get higher test scores on tests that are similary narrow and reductionist in their tendencies. We will be losing the amazing and ultimately beneficial variability among our students.

Perhaps it is that I was trained as a musician, that I am a humanist by nature, and not an engineer, a technician, or a businessman. Perhaps that is why I have the visceral reaction that I do. But to me if we are to leave no child behind, that means that we give every child the opportunity to fully explore her own potentiality, even if it is not easily supported by a computer, even if it requires the interaction with a human mentor, you know, that creature we call a teacher.

And that's my rant for the day.