consultants are sandburs

Saturday, March 14, 2015

The level of sheer stupidity involved in taking standardized testing to the extreme is apparent in one online item, and another aptly criticizes "Management by Pinheads." And we have them. In our legislature.

Exhibit A.

Exhibit B.

Explain to me what is the measure of an educated person. Winning a Nobel Prize? Few do. Making a Bill Gates/Warren Buffet fortune? Few do. Writing a Pynchon novel is something only Pynchon has done. Without having to take a multiple choice test about novel writing.

Scoring in the 99th percentile on the LSAT? Is that a measure of an educated person? It may help get you into a law school, but will you have the talent in pressing circumstances to fashion an acquital on, "If the glove don't fit, you've got to acquit?"

Of those legislators pushing for standardized testing, how many will publish their own SAT scores? 

Financial genius Nienow? Suppose he did score highly. That proves what? That the SBA and taxpayers should mop up his personal fiscal bad-judgment mess?

These are bozos leading a bozo parade, union busting being the actual aim, and some should know better.

Yes there truly are problems with winnowing talent as students move upstream, yes along the way the majority really do get left behind, but remember Medtronic was the result of a first class post-graduate medical school program at the Twin Cities campus, and the entrepreneurial sparks for succeeding generations do not come from children left behind, but from those not held back from where their capabilities and tenacity can allow them to reach.

And yes, Bill Gates scored big time three sigma or so off the high end from the middle of the SAT bell curve, and yes, he scored big cash/power accumulation while he can demonstrate an exceptional SAT score, but that is irrelevant to preparing an entire population to think semi-analytically in order to be capable of productively participating in voting, holding a steady job, seeking and holding office, or getting a PhD in grad school at MIT.

And yes, there is dead wood in any bureaucracy, in Boeing, in Airbus, they are big enough that some will be in and over their heads. It is the same with teaching. But those parroting, "Give standardized tests, it's Nirvana," should note that neither Boeing nor Airbus do that, and they produce very safe airplanes which we value in large part because of the successful judgment of the engineers and cost accountants who know where to shave a buck, and where the risk of dogmatic buck-shaving is too high. Show me one good multiple choice test that uncovers Boeing-grade management and engineering judgment skills.

There isn't any.

Have you any idea why they don't manage the medical profession by standardized testing and success rates of medical procedures? Fire the doctors who lose patients, never mind that some come to hospitals with incurable conditions, and never mind that everybody dies sometime and most likely the majority die while under medical attention.

If that testing stupidity were suggested for the medical profession everyone would laugh, because they could see through it. Why dump it on the teachers then? The job is thankless enough. Yes, they have a cohesive union. But so do the cops, and the weasels are not ripping at their flesh.

Go figure that one out.

In closing, Diane Ravitch thinks, while too many in the legislature decline to. Posturing is easier, and after all, path of least resistance is how the waters of the earth flow.

Arguably anecdotal reporting has value. But that possibility is apart from why bust a union, or get in the way of busting a union, when the facts are in such clear disrepair?

There exists no reliable data that standardized testing is effective for anything. My personal experience is that the SAT, the GSAT, general and for chemistry grad school purposes, and the LSAT scores were incredibly correlated with one another - ETS had exceptional internal test-to-test score consistency, but I could not see much link between test scores and academic or professional success. The senior lawyer where I worked was a better lawyer than I was, and his test scores had been mediocre. And the evil - training to the test - is both real and counterproductive to producing educated, inquisitive, analytical minds.

It is a bunch of hooey.

___________FURTHER UPDATE__________
Let us consider best evidence. People unquestionably brighter and more learned than your average legislator bent on union busting; people with no bias one way or the other on whether the teacher's union is too powerful and/or too Democratic for their liking, have gone on record, and if you read nothing else this month, read this. It debunks VAM.

Read it to know what VAM stands for, but that's revealed early in the brief seven pages, so read the entire thing. At p7 of seven pages, this is well noted

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.

The VAM scores themselves have large standard errors, even when calculated using several years of data. These large standard errors make rankings unstable, even under the best scenarios for modeling. Combining VAMs across multiple years decreases the standard error of VAM scores. Multiple years of data, however, do not help problems caused when a model systematically undervalues teachers who work in specific contexts or with specific types of students, since that systematic undervaluation would be present in every year of data.

A VAM score may provide teachers and administrators with information on their students’ performance and identify areas where improvement is needed, but it does not provide information on how to improve the teaching. The models, however, may be used to evaluate effects of policies or teacher training programs by comparing the average VAM scores of teachers from different programs.

This is from statisticians, whose livelihoods and well-being are related to uses of statistics - in this context student scores in standardized testing - and they are noting how ill-clothed is the emperor of using student test scores in hire-fire teacher retention decision making. They caution against stupid use of their methodologies, which they caution, are not well used in conniving by those having an ax to grind against union collective bargaining outcomes. The assault on the unions is a centralized and orchestrated thing, and only true fools or con men, moving unthinkingly or with ill-motive, would go along with it.

In the public high school in suburban St. Louis that I attended, the sophomore English fast track class was constructed around the notion of the epic in literature over the ages. We read the Iliad, Aeneid, and Paradise Lost, and a common thread was to consider the changing view of the hero, in literature. I cannot remember now whether we did Faust. In the junior year the theme was coming of age as to education in preparation for adult life. We read Portrait of the Artist as a Young Man, The Education of Henry Adams, and Look Homeward Angel. Two years of Latin were offered.

How do you wrap multiple-choice calipers around that kind of thought out process of having the brighter of students wrestle with things that at the time they were only partially equipped to comprehend. It avoided the boredom that TV then offered and offers still. There was some teaching to the test, but it was a limited thing, not something biased by employment security worry being mixed badly into the process of education.

So, again, the question too often swept under the rug, the square one question - what is the measure of a teacher's effectiveness? If you cannot cogently answer that in terms of where standardized testing can prove useful - aggregate evaluation of narrow things not key to becoming educated but which are easily measured by test-taking, variation in test results over an aggregate grouping, depending on socioeconomic variables mainly, AND you recognize these limitations for what they are; then the process is a limited part of things and not perverted for politically motivated aims and ends for which the statisticians themselves say it is inappropriate as a decision making tool.

How it is and you had best believe it, if you want educated children, on the whole, to take over in their adulthood and to not drive the nation into one disaster or another but to be even keeled and collectively able to do best.

In the context of the VAM debate, Ravitch has aptly written

As many education researchers have explained–including a joint statement by the American Educational Research Association and the National Academy of Education– the VAM ratings of those who teach children with disabilities and English language learners will be low, because these children have greater learning challenges than their peers, as will the ratings of those who teach gifted students, because the latter group has already reached a ceiling. Those two groups, like the ASA agreed that test scores are affected by many factors besides the teacher, not only the family, but the school’s leadership, its resources, class size, curriculum, as well as the student’s motivation, attendance, and health. Yet the Obama administration and most of our states are holding teachers alone accountable for student test scores.

The ASA warns that the current heavy reliance on VAMs for high-stakes testing and their simplistic interpretation may have negative effects on the quality of education. There will surely be unintended consequences, such as a diminishment in the number of people willing to become teachers in an environment where “quality” is so crudely measured.

Ask yourself, would you as a recent college graduate be more or less willing to dedicate your adult life to something "where 'quality' is so crudely measured"?

Were you to say "more willing, absolutely," I would have to question how well educated you are. How fit for the purpose you would be.

The evidence is what it is, and teaches what it does, and the ASA knows more about its area of expertise and is less biased as to inferring political directions from such expertise, than is ALEC.

Know that, or be ignorant. Uneducated. Ill-educated.

___________FINAL UPDATE____________
It intrigues me how in the legislature and Congress seniority is such a factor, witness Richard Russell and Orin Hatch - Tip O'Neill and Mitch McConnell. Yet the legislature, with seniority arguably too rampant a determinant, arguably counterproductive but the way things are done, is where the attack on teacher seniority is in full swing.

There is a saying about do as I say, not as I do.

1 comment:

Wes Volkenant said...

Nice follow-up to your questions raised at the Town Hall today.

Am pleased that former-teacher Mark Dayton has taken a stand against over-use of standardized testing.