What鈥檚 in a name? A lot, apparently.
Members of the panel that sets policy for the National Assessment of Educational Progress鈥攂etter known as the Nation鈥檚 Report Card鈥攐n Saturday approved to the test鈥檚 description of what constitutes 鈥渁dvanced,鈥� 鈥減roficient,鈥� and 鈥渂asic鈥� performance.
From now on, they鈥檒l be preceded by the word NAEP, as in 鈥淣AEP advanced,鈥� 鈥淣AEP proficient,鈥� and 鈥淣AEP basic,鈥� and references to performance in a grade will be stricken and replaced with performance on the NAEP assessment.
The impetus behind these revisions is to improve public understanding about what NAEP鈥檚 achievement levels mean; educators, parents, and the media often misunderstand NAEP proficiency to mean grade-level work, when it鈥檚 generally considered somewhat more difficult than that.
The rewording may seem awfully minor to the uninitiated. But there鈥檚 a deeper subtext behind the changes, and that鈥檚 why this is worth noting.
For nearly 30 years, some advocates have criticized the NAEP proficient bar as too high and therefore misleading for the public, and this latest go-around has opened up that debate once again.
In the public comments that accompanied this change, some commenters urged the board to overhaul the achievement level system entirely and others commended the National Assessment Governing Board for sticking to its guns.
A History of Achievement Levels
To fully appreciate this wording change, you need a little history lesson first. NAEP originally put the achievement levels and descriptors into place around 1990. Before that, the exam results were reported as a big list of scale scores, and there was no attempt to translate them for the lay person.
This was obviously difficult for most people to engage with, a problem because at that time people were hungry for more school-performance information. 鈥淎 Nation At Risk,鈥� the report that sounded a national alarm over education, had come out just seven years earlier, and the first push of 鈥渞eform鈥� efforts were well underway.
The creation of the achievement levels was quite controversial. Both the wording of the achievement level descriptions, and also the technical 鈥渟tandards setting鈥� itself鈥攖he process of setting the cutoff points for 鈥渂asic,鈥� 鈥減roficient,鈥� and 鈥渁dvanced鈥� performance鈥攃reated a lot of consternation and were variously attacked and defended.
While never universally embraced, the achievement levels eventually gained currency through the 1990s and became a keystone of NAEP鈥攅specially among the media (cough) which naturally found reporting the results using achievement levels to be easier for readers than reporting out scale scores.
Then we come to 2002 and the advent of the No Child Left Behind Act. Under that law, every state had to develop its own tests and cutoff scores and passing levels. Many of them chose to use the word 鈥減roficient鈥� in their own systems.
I鈥檓 pretty sure you know what I鈥檓 going to say next: People began to get all of these terms, tests, and policies hopelessly confused.
Sometimes, people who confuse NAEP with grade-level performance are making innocent mistakes, and . (Read this story for background on all the depressing ways people tend to misuse NAEP results).
In any case, from the beginning some edu-folks have argued that the NAEP proficient bar is really more aspirational than realistic. They say it鈥檚 actively misleading parents and the public and warping education policy unproductively. The group currently leading this charge is the under NAEP鈥檚 current levels.
The Roundtable has lobbied for NAGB to describe the proficient level on NAEP as 鈥渆xtremely demanding鈥� and the basic level as 鈥渞oughly analogous to grade level.鈥�
Studies of NAEP and state tests find that NAEP鈥檚 idea of proficiency does tend to be higher than the states'鈥�
New Questions Ahead?
That brings us to the current action.
The change approved by NAGB at its Nov. 17 meeting is actually part of a larger document laying out its policy for examining and reviewing the cutoff scores for the achievement levels. (The board is required to do this by federal law.) NAGB wanted to refresh the document to reflect technical evolution in standards setting.
That document was out for public comment, and from the 70-plus responses that came back, it was clear that some observers are worried that the door could now be open for the board to lower the bar, while others think doing so would paint a better picture of student achievement.
To its credit, the board also hosted a panel discussion with various perspectives on this topic. Marc Tucker, who has long studied workforce preparedness, for example, said he thought NAEP proficiency should be aligned to the level of work students need to succeed in the first year of community college鈥攁 lower threshold than the current one. On the other hand, David Driscoll, a former Massachusetts superintendent, attributed that state鈥檚 work to align its expectations to NAEP as a factor in its improved student performance.
But let鈥檚 make one thing really clear: There are no current proposals to change NAEP standards. While the board will begin revising its reading and math test frameworks over the next few years, the board鈥檚 intent is to maintain the test鈥檚 current cut scores and the preserve the 鈥渢rend line,鈥� said Andrew Ho, a testing expert at Harvard University and the chair of NAGB鈥檚 committee on standards, design, and methodology.
The board could do more to try to make the achievement levels easier for the public to understand what proficient performance looks like, beyond the it currently uses, he said.
鈥淭he most common question we get asked is, 鈥榃hat does proficient mean?鈥� and the answer is this paragraph. But the paragraph is somehow deeply unsatisfying,鈥� Ho noted in remarks to the board. 鈥淚t鈥檚 also overwhelming and intimidating and dense.鈥�
And that, it appears, is what NAEP will try to do: contextualize the test scores better, possibly by examining the landscape of other tests or perhaps student work samples.
To conclude, I鈥檒l leave you with a sampling of the public comments that accompanied NAGB鈥檚 draft.
The Council of the Great City Schools: 鈥淔or decades ... NAEP has been, and should remain, the standard for these terms. Application of these terms from assessment-to-assessment have been made relative to NAEP definitions鈥攅ven if they have not been faithfully applied. Changing the terminology suggests that NAEP should no longer be the standard upon which we understand student achievement.鈥�
The Education Trust: 鈥淚f the revision of the Achievement Level Policy results in lower expectations for what it means to be 鈥榩roficient鈥� or 鈥榓dvanced鈥� without solid justification for these changes, it could harm students across the country, with the highest risks for students who are already underserved in our schools.鈥�
AASA, The School Superintendents Association: 鈥淭he original achievement levels were developed in a rushed process, and resulted in levels that continue to confuse educators, citizens, and policymakers. The levels have been described as 鈥榳ishful thinking鈥� more than 鈥榬easonable鈥� or 鈥榗ommon sense,鈥� and the latest research linking NAEP鈥檚 benchmarks to international assessments reveals that the majority of students in most nations cannot clear NAEP鈥檚 proficiency bar.鈥�
The Superintendent鈥檚 Roundtable: 鈥淚n the Roundtable鈥檚 judgement, the modifications are in no way responsive to the major criticisms that have been leveled at the NAEP benchmarks over the years. To retreat behind the claim that the proficient benchmark is an aspirational standard is deceptive and evasive.鈥�
Emily Maurek, teacher: 鈥淎merica鈥檚 children have been made out to be 鈥榝ailing鈥� when they score below Proficient, when in reality the passing mark is out of reach and always will be. The National Center for Education Statistics has clearly stated that 鈥榩roficient鈥� is not synonymous with grade level performance. But when a metric is so clearly misused, misunderstood, and abused, it is clearly time for an immediate restructuring. That time is now.鈥�
Jack Jennings, former congressional aide and Center on Education Policy president: 鈥淭hose levels, instead, have led to confusion in the news media and among teachers, parents, and the general public. It appears that very high aspirations ruled their development, instead of realistic conclusions based on sound data.鈥�