Tests, Tests, Tests and More Tests – Results??? Who Knows….

http://www.sfgate.com/cgi-bin/article/article?f=/c/a/2009/09/15/MNOU19N6G8.DTL

update: http://www.nytimes.com/2010/08/16/nyregion/16gap.html?_r=1&hpw

http://www.nytimes.com/2010/09/05/business/economy/05view.html?hpw

This blog was started three weeks ago when I first captured the headlines.  I needed time to process what it was about the article that was necessarily disturbing, discomforting and revealing of how little the data in education seems to point in a meaningful direction.

At the same time this article was written, I had just begun reading the book The Black Swan by Nassim Nicholas Taleb, ISBN 978-0-1410-3459-1.  What I began to realize is that the test  results, while based on PAST notions of what children need to learn to be successful, are not predictive in a linear fashion of student success or even a reasonable measure of progress towards college.

If, as is pointed out in The Black Swan,  this type of data that is based on bell curve scenarios is not a quality predictive indicator, why do we persist in using it and what do we hope to obtain from the data?  If we are looking for a predictive indicator, we will most certainly be let down.  If we are looking for a measure of individual success, we will be even more let down as student learning is not linear, rather it occurs in bursts as the brain acquires enough experiences to process the over all schema.  If we are looking for qualitative measures to substantiate what we do as educators, we are applying narrative, retrospective distortion and Platonifying (over evaluating  factual information) and we will find exactly what we are seeking through interpretation.

This, in my mind, does not mean we should immediately stop all testing of students or using tests as an evaluative technique, but it should not be the exacting measure of all success or failure as indicated in the case example of Malcolm X Academy in San Francisco.   ‘

What I am understanding about myself is that I most definitely believe in Black Swans, know they exist and although we try to plan around them, realize we are victims of the perception that we can avoid what we don’t know is possible.  It is this very part of myself that quite possibly drives others crazy, much in the same way Mr. Taleb explains it in his book.

If tests are the end all of predictive value, there needs to be more than a correlation in the evidence and yet, that is all there is at this time.  I have yet to see conclusive evidence (even by the most hard core believers out there – charter schools such as KIPP and Aspire) which supports test scores translating into something such as the ability to complete/graduate college.  There are so many black swans for students who come from poverty that even the best education can not guarantee success in college – nor should that be the only outcome of an education (thank you Bill Gates). No, Bill Gates was not poor, however, he did not finish college.

There are no studies which definitively indicate a college degree will help you obtain more money during your life time, be rich, be famous, be popular.  The studies I have read indicate there is greater potentiality/possibility for some one to earn more money over their lifetime by having higher education (the higher you go, potentially the more money you can obtain).   All of this is in naught as I have close friends with a PhD who do not have the earning potential they should right now as as it is cheaper to hire a lecturer than a bona fide PhD person to place on tenure track.  I have friends who have taken a royal bath with the fall out of Wall Street even though they have an Ivy Education, including MBA degrees.  There are other friends of mine who were or had been doing moderately well except for the housing mortgage meltdown.  Most of the friends who were ensnared in this debacle would have been fine if they could wait out 10 or 15 years for the economy to right itself and housing to regain momentum instead of moving for a job.  Each item I wrote about in the last four sentences was a Black Swan none of us saw coming when we were undergrads or graduate students.

Which means, all those great grades we  (the people talked about above – and they know who they are if they are reading this blog) obtained in elementary, middle and high school, the SAT’s, GRE’s, etc. were never predictive of our success, rather all those grades and scores were predictive of our future potential.

So, my question remains, what do the results mean?  How should we use these test results to improve education? How should we deliver tests (multiple choice/written, etc.) to obtain results with more predictive value?  Can testing provide predictive value?  The questions are endless.  All I know is education has become something completely counterintuitive to what we know from Piaget, Montessori, etc.   If we really want results, we need to be more longitudinal in our thinking and cope up to the Black Swans out there which will always change the penultimate outcome of our best written and delivered lessons.

Advertisements

One thought on “Tests, Tests, Tests and More Tests – Results??? Who Knows….

  1. Though I agree with overall testing to see what a student has learned in school, what I find, overall, is that school overlook an student’s sociology. If you apply even just basic anthropology and look at these tests and results, what you find is, basically, american schools (and many others basing their education system on this nation) have screwed up. In the words of the fictional Dr. Thorne, “How can you disign ofr people if you don’t know the [sociology]? You can’t. Because your mathematical formulas may be perfect, but the people [aren’t]. And if that happens, it means YOU screwed it up.” I find much blame in poor placement based on test scoring: it’s the teachers, it’s the parents, it’s the system, it’s youth today. We need an overhaul.
    Testing, as you remarked, is a matter of finding out a student’s learning potential for future circumstances. However, how do you explain the straight-A highschool student with top SAT scores flunking out of college? How do you explain the C-average highschooler who blooms in college to the Magna-cum-laude position? The answer: we are not taking sociological events into account when we offer advice on post-grade placement. No written test will ever offer us clear-cut meaningful predictive values. If you cannot change the thing, you must change how you think about a thing. This means that we must yes, accept the test values, and no, not predict any human’s success by a number on paper.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s