Observations – Parity Across the States (NAEP)

The past four months provided a very interesting job opportunity in a way I was not expecting.  I was hired by Westat to help administer the ‘national’ exam.   While this position requires a great deal of organizational capacity, it also taught me so much about those who ‘know’ and don’t really know what is involved in educational reform.   In my mind, NAEP is a giant scientific experiment extremely well thought out as well as planned and executed for optimum good statistical analysis.  NAEP is the real deal of data.

My first observation was how many teachers and administrators in public schools (including charter) did not know what NAEP was about.

Example:  Spoke with a principal who, when asked by another principal why his school was participating in NAEP answered he ‘guessed’ he drew the short stick. (I completely understood the humor, however, I was sad he truly did not know what it was about.) We talked on the phone and I asked him why he thought the school was participating (aha – a bit of Socratic methodology) and he answered the following three items (1) his school improved test scores by 17 points (2) his school has the youngest teachers in the district/area (3) this had something to do with spring testing but he was not sure. 

 I replied that while I like his answers, the reason behind national testing was to have some test scores to provide parity  between the various states which have different state standards and different ways of measuring educational success.

Parity in sports is defined as attempting to make an equal playing field for all participants, specifically with regard to financial issues. When parity in a sports league is achieved, all participating teams enjoy roughly equivalent levels of talent. In such a league, the “best” team is not significantly better than the “worst” team. This leads to more competitive contests where the winner cannot be easily predicted in advance. Such games are more entertaining and captivating for the spectators. The opposite condition, which could be considered “disparity” between teams, is a condition where the elite teams are so much more talented that the lesser teams are hopelessly outmatched. – Wikipedia/ January 2011

 I further explained that schools which received federal funds w  ere required to participate if the necessary sample of student attributes was at their school.  We talked for a few more minutes so I could answer some paperwork questions and we each went off on our own separate journey through the day.

In a different phone call with another principal the day before, I was asked about how the school would access these test scores so they could use them to compare with their API and AYP since other schools in the district were not taking this special test. I had to explain the scores were not disaggregated down to the district level.  This particular issue kept cropping up with teachers as well, especially on assessment day. Teachers asked if they would have the scores to use by Spring……

In  the above scenarios, I was talking to people who could speak clearly about certain aspects of educational reform, albeit only those minimal measures which had been drummed into them through some grad school program/administrative credentialling program/school district.   My shock was that these were ‘good’ schools in so called ‘good’ districts so how could these administrators not have run across NAEP?  I actually asked a couple administrators where they attended graduate school.

Along the way there were also some funny stories – a school which is in a very wealthy area had a substitute teacher’s aide show up with alcohol on their breath and the principal had to deal with that issue; another school had some students order a pizza via their cell phone, except when it was delivered, the office staff realized no ‘individual’ student would order a 2 L bottle of coke.  I had a colleague talk to a custodian in Spanish, only to have the principal state the man understands and speaks English…..

In my mind I was surprised as I have known about NAEP since I was a child – I went through at least one of the testing sessions in Grade 4, possibly Grade 8.  I read about NAEP and went to a lecture regarding The Nation’s Report Card when I was in graduate school.  NAEP was the organization where  the National Science Standards were related so people could discuss trends in science ed ucation.  I was beginning to feel as if I had entered some alternate universe where educational reform happened on a different planet on an alternate flat plane.

On a more personal note, I noticed (part of the script I read requires me to ask a few questions) there is not a category for people(s) of Middle East origin.  I am not sure if this was intentional, as in who really cares what those students do (even though we seem to care about Asians) or some one with far more wisdom then myself decided these people are, well, white.  Since I do not know specifically what NAEP is looking for, I can only speculate on a ‘forced’ selection of race/ethnicity.  One question asks students to delineate Latino/Hispanic and then the next question is everything else.  I feel bad for the Philipino’s who actually know their history as they are Latino (Spain) and Asian, not either or.

I have never looked at the test questions as I continue to have a teaching credential and this, in my mind is inappropriate.  I have looked at the release questions published in booklets for parents and/or teachers and administrators who may have questions.  Not much was gleened from this process as I do not support the efficacy of multiple choice exams since there is always an inherent 25% of accuracy by randon choice on a four answer question.   As is the case with SAT prep, it is not about the right answer so much as the ability to use your mind to reason ‘out’ what are the wrong answers.  The SAT is in no way indicative of much, my favorite examples being people who bucked the system and did not complete college, such as Bill Gates or people who did poorly on the SAT and succeeded far more than anyone would have guessed, Timothy Ferriss. 

NAEP allows educators and statisticians to peer into the minds of students to take a peak at how various curriculums play out across the U.S.  Our ‘Nation’s Report Card’  is just that, a report.  In a broad way we are able to see where education seems to have traction (typically, in places with low socioeconomic distress) and where no amount of money seems to change the consequences of childhood poverty. 

I have been to schools with views which certainly must prevent even a lax daydreamer from focusing and I have been to schools which remind me all too often of things I have seen in Peace Corps and traveling various third world countries.  This previous sentence is a different kind of parity – until we have PARITY, we will not change education in any formidable manner.  It takes an abundant amount of community involvement, parental education and literacy resources (notice I did not say monetary resources) to overcome poverty.  No amount of well constructed testing is needed to prove this out – rather, we just need to travel outside our own familiar community.


A Pure Nugget of Truth

Most Educators Do Not Understand Importance Of Critical Thinking, Author Says. The Des Moines Register (5/15) features an interview with Tony Wagner, the co-director of the Change in Leadership Group at the Harvard Graduate School of Education and author of the book, The Global Achievement Gap: Why Even Our Best Schools Don’t Teach The New Survival Skills Our Children Need – And What We Can Do About It. In the book, Wagner asserts that “the seven survival skills…all students should master are critical thinking and problem solving; collaboration across networks and leading by influence; agility and adaptability; initiative and entrepreneurship; effective oral and written communication; accessing and analyzing information; and curiosity and imagination.” Those skills, he says will help teach “students how to think.” When asked why the push to teach critical-thinking skills has not “taken hold” in schools, Wagner said, that “most teachers, parents, and employers don’t understand the importance of critical thinking.” Some contend “that critical thinking is too fuzzy. Therefore, it’s not tested. If it’s not tested, it’s not going to be taught.”

Most educators with professional expertise/credentials recognize ‘Bloom’s Taxonomy’.   Unfortunately, as wonderful as Mr. Bloom was in pointing out the obvious, most teachers are not ‘allowed’ (I am using this term loosely as allowing some one to do something, according to Merriam-Webster means everything from “to give consideration to circumstances or contingencies” to “forbear or neglect to restrain or prevent”) to reach out to students and expect the top levels of the taxonomy as, indeed, as noted above – it is not tested, or more clearly, not easy to test in a multiple choice version of a test.  Due to the labor hours required to correct written out, thoughtful and dynamic answers, this methodololgy of instruction is rarely used in classrooms, well up to AP classes and sometimes in college.

It is not that long ago I remember  different principals telling me there was this ‘test producing software’ I could use to make my life easier and it is also easier to score/grade.  The questions are in a bank of questions from various sources (often, the same people who put poorly worded garbage in text books) and you pick out the ones you want to use, ‘align’ the questions to the standards at hand and viola’, a quiz, test, etc. that is scientifically proven.  I usually shrugged my shoulders, said thank you for the advice and went back to creating quizzes and tests where students had to not only answer the questions at hand but use justification or reasoning as to why they selected their answer. It took forever to grade, however, I was sure at the end if a student did or did not understand something.

The problem with what I was doing was a  triple-fold threat: (1) It was difficult to explain to parents in a conference why their child did not understand concept X or Y (the parents themselves were often not educated enough to understand the concept – anywhere from soccer mom without college degree to people of extreme poverty and equally lacking in formal education) when the teacher across the hall could show similar students understood concept X or Y on a multiple choice test. I became the horrible teacher because I would not settle for the ease of multiple choice tests or the symbolism of a 1/4 right answer just by luck (25% of the time a student randomly penciling in a,b,c,d would ‘get’ the right answer) and expected students to THINK.  (2) I was the exhausted teacher as I made it a personal goal to myself to grade quizzes/tests the day they were taken for quick  and appropriate feed back. This often meant staying up well through the early morning hours with coffee to grade upwards of 90-120 papers to turn them back to students the next day if I was so unfortunate as to have not given a test on a Friday.  (3)   I was the shamed teacher because other teachers did not want to put in the effort so the best they could do was bully me (typical in a school setting where tenured and/or teachers with more longevity set the expectations, not the new, young upstarts) and make me feel bad about actually sticking to Bloom’s Taxonomy and the value of critical thinking. I became the teacher few principals would  support as I also made their lives difficult expecting more of myself, my students and in effect competing with my colleagues.  I was the teacher others valued and feard for I would do what I knew was correct, without shortcuts or excuses (KIPP schools should value this, however, they use the multiple choice testing software also – not sure the excuse….convenience?).

While I asserted in my own way that children needed in -depth learning, I was chastised by the very people that should have supported me. In a few situations I ran across like minded teachers, we banded together and spent many, many hours after school perfecting our craft by sharing lesson planning ideas, strategizing creative ways for students to express their knowledge, improving/scripting/writing out better lessons, sharing material resources and grading – using rubrics to make sure we as individuals were grading our students fairly (we sometimes graded each others student work we had personal misgivings about our own grading).  This created horrific problems with the union as we were ‘breaking’ rank and making other teachers look bad. We were ‘ruining the data’ at non-union charter schools because how on earth could you compare a multiple choice test result to a written out test result. We were ‘insulting’ our students by expecting the appropriate, grade level work out of them and being willing to help every step of the way.

Teaching was a labor of love – it was about having all the awesome  aha! moments along with the magic of seeing a student take an idea to the next level. 

It was the reason I left teaching.   After being told indirectly (so know one would get caught knowing it was actually what we did) to teach to the test so the scores could go up, being denied the time within a subject for depth instead of  merely breadth of concepts on the test, being asked to ‘follow’ what the other teachers were doing and most of all, being exhausted to the point of not being able to even enjoy what little vacation time I had (charter schools go almost all year – by professional development expectations and teaching days), I decided the best I could do was walk away proudly and know in my heart, one day the tide would change.

I keep seeing this word innovation bantered about.  I have found in Judy Estrin’s book Closing The Innovation Gap a common thread with what I have believed all these years.  I want to hug her for saying there is truth to what I have thought and practiced.  Most of all I want to thank her for planting the seed that will hopefully grown to a beautiful tree which will change the way education is ‘performed’ in America.