The governing board that oversees the test known as the 鈥渘ation鈥檚 report card鈥� has adopted a new framework for designing the reading assessment, one that will provide more granular information about student performance by socioeconomic status and race and test students鈥� ability to read across disciplinary contexts.
But even after a unanimous vote to approve the new framework last week, some members of the panel tapped to develop the document have lamented what they see as missed opportunities for a fairer test鈥攖he after-effect of a heated back-and-forth over equity in assessment during the development process over the past several years.
The National Assessment of Educational Progress, the NAEP, is given to a nationally representative sample of U.S. students to measure what they know and can do across subjects.
The National Assessment Governing Board supervises the NAEP, and leads the process for updating the frameworks that guide how the test is constructed. The reading framework was last revised in 2009. The new changes will go into effect for the 2026 test administration.
A key consideration in updating the framework is maintaining NAEP鈥檚 long-term trend line, the ability to compare results from upcoming years to past scores, so as to draw conclusions about whether students are improving or not. (The National Center for Education Statistics, which conducts and analyzes NAEP tests, has said that the new adopted framework is likely to maintain trend.)
Understanding what the trends are is especially important now, said Lesley Muldoon, the NAGB鈥檚 executive director, to evaluate the effect that COVID-19 has had on student achievement 鈥渟o that people can have a trusted baseline that they can use going forward.鈥�
The framework development process has always included a diversity of perspectives, with varying factions working to hammer out their differences to develop a consensus document. But tensions ran especially high this time.
The debate raised questions central to the construct of reading itself: What does 鈥渞eal-world reading鈥� actually look like? And how much of it is influenced by readers鈥� cultural backgrounds and the social contexts in which they learn?
At the same time, these conversations were taking place in the middle of a national conversation on race that has pushed educational organizations to consider how teaching, learning, and assessment can better support students of color.
Framework offers more data on students鈥� reading across disciplines
There are significant changes in the consensus document鈥攃hanges that advocates on both sides of the framework debate said, in interviews with 澳门跑狗论坛, would make NAEP a richer source of data on students鈥� reading ability.
The new framework calls for more detailed reporting on NAEP subgroups. Scores won鈥檛 just be disaggregated by race, ethnicity, and English-language learner status, but also differentiated by socioeconomic status within race and ethnicity. So, going forward, it would be possible to see the differences in scores between Black students from high-income families and Black students from low-income families, for example.
Students will also be tested on their ability to read informational text in social studies and science. This isn鈥檛 meant to evaluate students鈥� content knowledge鈥�"this is not a test about whether they know the causes of the American Revolution,鈥� Muldoon said鈥攂ut rather that students can use discipline-specific reading skills in genres they鈥檒l encounter in the classroom and real world.
And the framework adds a new 鈥渃omprehension target,鈥� or tested component of reading comprehension ability. Previously, the framework included three: 1) locate and recall information, 2) integrate and interpret information, and 3) analyze and evaluate information.
Now, students will also be expected to 鈥渦se and apply鈥� what they read, to solve problems or create something new. For example, after reading a series of opinion pieces on a subject, a student might be asked to write a blog that synthesizes the different positions or offers their own argument.
鈥淭his is not just your mother鈥檚 and father鈥檚 鈥榝ind the main idea,鈥欌�� said David Steiner, a professor of education at Johns Hopkins University and the executive director of its Institute for Education Policy. (Steiner was not involved in the drafting of the framework, but has commented publicly on the process.)
Other updates to the framework formalize changes that have already been made to the NAEP, following its shift to digital, rather than paper, administration. These include updates such as incorporating more digitally native text鈥攕uch as what might be read on websites鈥攁nd virtual 鈥渃haracters鈥� that simulate a classroom environment or group work.
One new feature added to this list: Test-takers will also have examples of student responses to questions, to better illustrate what a strong response looks like.
鈥榃hat kind of reading do we want to draw inferences about?鈥�
At last week鈥檚 board meeting, held both in person in McLean, Va., and streamed online, members praised the consensus process that resulted in the framework adoption.
Still, some members of the development panel felt that the final version diverged too far from the initial drafts鈥攁nd that commitments made to equity were stripped at the 11th hour by a vocal minority of NAGB鈥檚 main board.
At the heart of this disagreement were two interconnected questions: How to define reading comprehension and what constitutes 鈥渞eal-world鈥� reading.
Early versions of the framework, written by the NAGB-appointed development board, put forth a sociocultural model of reading comprehension. The model argues that reading is in part about what鈥檚 going on inside a student鈥檚 head鈥攖he cognitive processes鈥攂ut that comprehension is greatly influenced by social and cultural contexts like home, school, and community.
These early drafts also broadened the use of 鈥渋nformational universal design elements,鈥� text introductions, pop-ups, and videos that give students some background knowledge about the passages that they are about to read. This change was suggested because research has shown that reading comprehension ability is greatly influenced by readers鈥� background knowledge on the topic. (Students will probably have an easier time reading Animal Farm, for example, if they have some understanding of the Russian Revolution.)
Gina Cervetti, an associate professor of literacy at the University of Michigan School of Education, and a member of the framework development panel, said that beefing up these knowledge scaffolds would have made NAEP a truer test of students鈥� reading comprehension ability. It would test their knowledge of text structures, or their skills in analyzing information, rather than their content knowledge, she said. It would level the playing field for students who come to the test with different stores of knowledge.
When this version of the framework was put out for public comment, though, it brought forth harsh criticism from some corners of the education world. 鈥淭his came to be seen as an attempt to inflate the scores of traditionally underperforming students,鈥� Cervetti said. 鈥淎nd nothing could be further from the truth.鈥�
But Steiner, , said that providing all that supporting information would have created conditions on the NAEP that don鈥檛 exist in real-world reading. Take a word like yacht, he said. 鈥淵ou could argue, and this is argued in many state assessments, you can鈥檛 use a word like yacht, because less-affluent students have not grown up in a world of yachts.鈥�
But 鈥測acht,鈥� Steiner said, is a word that regularly shows up in works that students might be expected to read as adults: news, magazines, novels. It鈥檚 part of a broad public vocabulary that students would be expected to know, and that teachers could reasonably be expected to make sure students know, he said.
Testing whether students are prepared for reading in college and career should include testing whether they can read and make sense of texts that include that word, he argued鈥攁nd not testing this could mask indicators that students might have trouble with reading later on.
The draft framework was released for public comment last summer, and the development panel incorporated changes resulting from that feedback. But in May, when the revised framework was presented to the full board, some members thought the changes didn鈥檛 go far enough.
Grover (Russ) Whitehurst, a NAGB board member and former director of the Institue of Education Sciences, conducted his own, further revision of the document, striking most of the references to sociocultural frameworks and toning down the use of informational UDEs, to the alarm of many members of the original development panel.
鈥淭he goal ... is to handle background knowledge in ways that strengthen the validity of the assessment, rather than trying to define it out of existence as a factor in reading comprehension,鈥� Whitehurst wrote at the time.
To hammer out these differences and create a consensus document, NAGB鈥檚 chair, Haley Barbour, assembled a smaller, cross-committee working group which put forth the final framework as adopted.
Informational UDEs are still in the framework, but they play a much smaller role. This concerns Cervetti, who maintains that a more robust set of informational UDEs would make the NAEP more like 鈥渞eal鈥� reading, not less.
鈥淚n the real world, outside of a standardized assessment, we rarely read completely unfamiliar texts in isolation,鈥� she said. If a student reads a word they didn鈥檛 know, they can look it up. 鈥淲e all have phones, and computers, and people [around us], and dictionaries,鈥� Cervetti said.
鈥淲hat constitutes real reading is, I think, a real bone of contention. And it makes a huge difference,鈥� said P. David Pearson, a professor emeritus at the University of California, Berkeley鈥檚 Graduate School of Education, and the chair of NAGB鈥檚 development panel. 鈥淏ut the question is, what kind of reading do we want to draw inferences about?鈥�
Possible changes to framework development process on the horizon
Pearson said the final framework is 鈥渟omething to be celebrated,鈥� but also that he would want to see more work done鈥攊n defining reading in more of a sociocultural context, which he said would bring NAEP in line with other national and international assessments, and in gathering more data about students鈥� school and community environments. And he questioned the framework development process, which requires that NAGB approve new developments through consensus.
鈥淚 think that鈥檚 a great tradition, but if things get controversial, and if there are ideological and theoretical differences, then I鈥檓 not convinced that consensus is the only way to make important decisions,鈥� he said. 鈥淭he other thing about consensus is that it鈥檚 another name for minority rule, just as the filibuster in the Senate is another name for blocking the majority.鈥�
The majority of the framework development committee supported the version of the document put forth in earlier drafts, Pearson said, and changes were introduced by a small group of dissenters in the full NAGB board.
But Whitehurst, one of these dissenters, said that his does not represent a minority view. He argued that many in the reading education community鈥攊ncluding researchers and school-level educators alike鈥攚ould endorse a model of reading that put more emphasis on the cognitive processes than sociocultural contexts. But, he said, this diversity of viewpoints wasn鈥檛 included on the framework development panel.
鈥淭hose of us on the board who sort of had to take that position would not have had to if there were greater diversity in the views of those who developed the document,鈥� he said.
After a drawn-out public battle over the reading framework, the framework development process itself is up for review this September by the NAGB board鈥攊n part, so the team can 鈥渉ave an easier time with framework development in the future,鈥� said Sharyn Rosenberg, NAGB鈥檚 assistant director for assessment development, in the board meeting last week.
Ideally, Whitehurst said, the framework development process going forward would produce documents in which 鈥渢he tensions are already worked out.鈥�
