The past four months provided a very interesting job opportunity in a way I was not expecting. I was hired by Westat to help administer the ‘national’ exam. While this position requires a great deal of organizational capacity, it also taught me so much about those who ‘know’ and don’t really know what is involved in educational reform. In my mind, NAEP is a giant scientific experiment extremely well thought out as well as planned and executed for optimum good statistical analysis. NAEP is the real deal of data.
My first observation was how many teachers and administrators in public schools (including charter) did not know what NAEP was about.
Example: Spoke with a principal who, when asked by another principal why his school was participating in NAEP answered he ‘guessed’ he drew the short stick. (I completely understood the humor, however, I was sad he truly did not know what it was about.) We talked on the phone and I asked him why he thought the school was participating (aha – a bit of Socratic methodology) and he answered the following three items (1) his school improved test scores by 17 points (2) his school has the youngest teachers in the district/area (3) this had something to do with spring testing but he was not sure.
I replied that while I like his answers, the reason behind national testing was to have some test scores to provide parity between the various states which have different state standards and different ways of measuring educational success.
Parity in sports is defined as attempting to make an equal playing field for all participants, specifically with regard to financial issues. When parity in a sports league is achieved, all participating teams enjoy roughly equivalent levels of talent. In such a league, the “best” team is not significantly better than the “worst” team. This leads to more competitive contests where the winner cannot be easily predicted in advance. Such games are more entertaining and captivating for the spectators. The opposite condition, which could be considered “disparity” between teams, is a condition where the elite teams are so much more talented that the lesser teams are hopelessly outmatched. – Wikipedia/ January 2011
I further explained that schools which received federal funds w ere required to participate if the necessary sample of student attributes was at their school. We talked for a few more minutes so I could answer some paperwork questions and we each went off on our own separate journey through the day.
In a different phone call with another principal the day before, I was asked about how the school would access these test scores so they could use them to compare with their API and AYP since other schools in the district were not taking this special test. I had to explain the scores were not disaggregated down to the district level. This particular issue kept cropping up with teachers as well, especially on assessment day. Teachers asked if they would have the scores to use by Spring……
In the above scenarios, I was talking to people who could speak clearly about certain aspects of educational reform, albeit only those minimal measures which had been drummed into them through some grad school program/administrative credentialling program/school district. My shock was that these were ‘good’ schools in so called ‘good’ districts so how could these administrators not have run across NAEP? I actually asked a couple administrators where they attended graduate school.
Along the way there were also some funny stories – a school which is in a very wealthy area had a substitute teacher’s aide show up with alcohol on their breath and the principal had to deal with that issue; another school had some students order a pizza via their cell phone, except when it was delivered, the office staff realized no ‘individual’ student would order a 2 L bottle of coke. I had a colleague talk to a custodian in Spanish, only to have the principal state the man understands and speaks English…..
In my mind I was surprised as I have known about NAEP since I was a child – I went through at least one of the testing sessions in Grade 4, possibly Grade 8. I read about NAEP and went to a lecture regarding The Nation’s Report Card when I was in graduate school. NAEP was the organization where the National Science Standards were related so people could discuss trends in science ed ucation. I was beginning to feel as if I had entered some alternate universe where educational reform happened on a different planet on an alternate flat plane.
On a more personal note, I noticed (part of the script I read requires me to ask a few questions) there is not a category for people(s) of Middle East origin. I am not sure if this was intentional, as in who really cares what those students do (even though we seem to care about Asians) or some one with far more wisdom then myself decided these people are, well, white. Since I do not know specifically what NAEP is looking for, I can only speculate on a ‘forced’ selection of race/ethnicity. One question asks students to delineate Latino/Hispanic and then the next question is everything else. I feel bad for the Philipino’s who actually know their history as they are Latino (Spain) and Asian, not either or.
I have never looked at the test questions as I continue to have a teaching credential and this, in my mind is inappropriate. I have looked at the release questions published in booklets for parents and/or teachers and administrators who may have questions. Not much was gleened from this process as I do not support the efficacy of multiple choice exams since there is always an inherent 25% of accuracy by randon choice on a four answer question. As is the case with SAT prep, it is not about the right answer so much as the ability to use your mind to reason ‘out’ what are the wrong answers. The SAT is in no way indicative of much, my favorite examples being people who bucked the system and did not complete college, such as Bill Gates or people who did poorly on the SAT and succeeded far more than anyone would have guessed, Timothy Ferriss.
NAEP allows educators and statisticians to peer into the minds of students to take a peak at how various curriculums play out across the U.S. Our ‘Nation’s Report Card’ is just that, a report. In a broad way we are able to see where education seems to have traction (typically, in places with low socioeconomic distress) and where no amount of money seems to change the consequences of childhood poverty.
I have been to schools with views which certainly must prevent even a lax daydreamer from focusing and I have been to schools which remind me all too often of things I have seen in Peace Corps and traveling various third world countries. This previous sentence is a different kind of parity – until we have PARITY, we will not change education in any formidable manner. It takes an abundant amount of community involvement, parental education and literacy resources (notice I did not say monetary resources) to overcome poverty. No amount of well constructed testing is needed to prove this out – rather, we just need to travel outside our own familiar community.