In doing research for some contract work, I stumbled upon the above article. It was kismet as I had spent some time researching web sites for education department professors at various ed schools and the most recent corporate education (both for profit and not for profit) websites relating to K-12 and early college. I was not looking at data per se, I was looking for philosophical bends (Perennialism, Essentialism, Progressivism and the like) and how these were represented, or not, on various web sites. Most of all, I was looking to see if these philosophical differences related to materials professional educators select for lessons, teaching, student concept development and how these ideals might play out across text books, etc. Essentially the relationships between how one believes education should be manifested and the reality in the classroom are merely grad school explorations.
What I realized through my research is most everything in education has been made quantifiable (except for the folks who prefer Waldorf and Montessori style education). The quantification of so much data means less in education as noted in the article above. Reasonable people can see past the curtain to OZ – if they choose. The rest will allow for the numbers to gain traction as the marketing departments use fancy tools to market the numbers.
No matter the discipline or industry, the rise of the quants tends to happen in four stages. Stage one is what you might call pre-disruption, and it’s generally best visible in hindsight.
Next comes stage two, disruption.
After disruption, though, there comes at least some version of stage three: overshoot. The most common problem is that all these new systems—metrics, algorithms, automated decision making processes—result in humans gaming the system in rational but often unpredictable ways.
It is entirely possible to believe everything in education could be based on data. While the data may appear mesmerizing, it is arbitrary and representational. It looks great on paper, or at least in some instances. Data can be manipulated to be beautiful without having true substance to the topic at hand, a technique sometimes employed by the folks at Wired Magazine. Wired Magazine will explore a graphic/chart to make us look at the mundane or something crazy interesting in a wholly new manner – all the while not using the situation for people to make important decisions. Examples include the following: http://www.wired.com/design/2013/10/26-amazing-food-infographics/ or http://omnilligence.net/wired-infographics/wired-infographic-showing-the-color-location-world-30427/ This is highbrow art and interpretation is something to explore for fun.
At the end of the day, exploring the data presented at http://aspirepublicschools.org/approach/results/ or http://www.kipp.org/reportcard would make you wish to believe charter schools, or at least the two largest organizations, have it in the bag and ready to deliver. Drill down and you find out empirically the charter school movement is inconsistent, students leaving these schools and completing college is slightly above the average ‘marginal’ numbers of any other public schools. Taken in aggregate, the numbers look ‘pretty’ although the distortion is in many instances misleading.
Sociologist Donald T. Campbell noted this dynamic back in the ’70s, when he articulated what’s come to be known as Campbell’s law: “The more any quantitative social indicator is used for social decision-making,” he wrote, “the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
On a managerial level, once the quants come into an industry and disrupt it, they often don’t know when to stop. They tend not to have decades of institutional knowledge about the field in which they have found themselves. And once they’re empowered, quants tend to create systems that favor something pretty close to cheating.
The quantification has changed over time, most noticeably with Aspire Public Schools. They have gone from showing graphs with by school data to giving global data. They no longer show the stats for students graduating college or how many students get in, something they used to pride themselves on.
It is clear the organizations mentioned are looking for a way in which to appeal to the public to one up each other, all the while not clearly demonstrating a genuine improvement in the system or social dividends -previous blog: https://whereiskatima.wordpress.com/?s=social+dividends
As soon as managers pick a numerical metric as a way to measure whether they’re achieving their desired outcome, everybody starts maximizing that metric rather than doing the rest of their job—just as Campbell’s law predicts.
That’s why they arrive at stage four: synthesis—the practice of marrying quantitative insights with old-fashioned subjective experience.
It would be great if data was used to demonstrate student learning in the larger picture – going to college and/or trade school, community shifts including lessening poverty rates, better health. When I think about the purpose of a good public education, it was never about test scores, rather it was always about something else – tilting the fulcrum. When I see info graphics with distortion, and not for nuanced and humorous effect, I wonder what went wrong along the way. Who is being played by the
That’s what a good synthesis of big data and human intuition tends to look like. As long as the humans are in control, and understand what it is they’re controlling, we’re fine. It’s when they become slaves to the numbers that trouble breaks out. So let’s celebrate the value of disruption by data—but let’s not forget that data isn’t everything.