澳门跑狗论坛

Opinion
Federal Opinion

Is PISA a Victim of Its Own Success? IES Head Calls for Change

By Mark Schneider 鈥 January 31, 2019 5 min read
BRIC ARCHIVE
  • Save to favorites
  • Print
Email Copy URL

Data collection for the 2018 Program for International Student Assessment has just ended. When PISA scores are released later this year, they will attract attention across the globe. However, there are several flaws in PISA that need attending to. Getting PISA right is important because it is likely the world鈥檚 best-known education assessment.

Coordinated by the Organization for Economic Cooperation and Development, PISA has grown from testing students in 32 countries in the year 2000 to 80 countries and subnational 鈥渆ntities鈥 (including some Chinese provinces) in 2018. Designed to monitor national education systems through regular assessments employing a common framework, PISA provides benchmarks against which countries can compare their own performance. The United States has supported PISA since its inception, but we in the U.S. Department of Education are now concerned that PISA is in danger of becoming a victim of its own success.

The United States sees two specific challenges potentially undermining the quality of PISA:

  • First is PISA鈥檚 lack of investment in the research and development needed to ensure the quality of such a complex, wide-ranging, and high-profile assessment program.
  • Second is PISA鈥檚 short, three-year testing cycle, which made sense in 2000 when PISA started, but, given new testing technologies, is expensive and unnecessary.

We believe that action is needed now to safeguard the future of PISA.

Every large-scale assessment program requires extensive, sustained research and development. For a program with the international visibility of PISA, R&D must be more than basic validation and testing; it should encompass a strategic approach that places R&D at the heart of future planning and creates an infrastructure for validating ideas.

See also: If Students Aren鈥檛 Trying on International Tests, Can We Still Compare Countries鈥 Results?

Rather than recognizing how much R&D is needed to get things right, PISA has been trying to roll out a new 鈥渋nnovative鈥 domain every three years. The sad lesson from PISA鈥檚 2018 鈥済lobal competence鈥 assessment is that there is great risk in rushing to prepare an assessment for a brand new construct every three years without extensive research conducted over a reasonable time period. Indeed, the global-competence assessment was of such low quality that 40 of the countries taking the digitally based version of PISA simply refused to administer it鈥攖he United States among them.

We believe that action is needed now to safeguard the future of PISA."

A fully functioning R&D operation could help ensure quality, by conducting the work needed to validate or modify proposed changes to PISA鈥檚 scale, scope, and design. It also could help turn investments in exploratory, one-time assessments of innovative domains into better-thought-out assessments. Clearly, this R&D function would have to have a large degree of independence from the OECD to resist political and competitive pressures鈥攊ncluding the lure of innovation鈥攖hat now buffet PISA.

Development in PISA is not limited to the innovative domains. The three major domains of math, reading, and science literacy require constant research and updating. PISA essentially has two tasks: measure evolving forms of literacy to help assess the mastery of skills needed to prosper in the changing world and produce trend data on levels of skills across time. The first goal requires an assessment framework that constantly evolves; the second requires reliable measurement over time. Without sufficient development work, PISA is in danger of losing the capacity to address either task adequately.

With proper R&D, PISA could measure trends against past performance, while letting the assessment framework evolve so that any changes in the nature of literacies needed could be taken into account. But meeting that challenge will require careful research and development conducted over more time than PISA now allows.

The OECD also needs to reconsider its three-year testing cycle. This cycle made sense in the early days of PISA when countries were first learning about where their education systems stood relative to others, when there were only three subjects tested, and, perhaps most critically, when accurately measuring all subjects was not feasible on every cycle.

Initially, PISA was designed to assess 15-year-old students in-depth in one of three core subjects鈥攎athematics literacy, reading literacy, and scientific literacy鈥攚ith one of the three administered as the 鈥渕ain鈥 domain and assessed much more accurately than the others. This meant that in-depth data on student performance in any single domain could be reported only every nine years, which was deemed adequate for countries to monitor their education systems.

Assessment technologies have improved markedly since the launch of PISA in 2000. These improvements mean that PISA will be able to provide in-depth data for each of its core subjects after each administration, rather than waiting nine years.

Today, a five-year testing cycle would produce the information countries need almost twice as often as originally envisioned in PISA, but with a lower financial burden and less strain on each country鈥檚 capacity to administer the tests. Most importantly, a longer cycle would allow more resources to be devoted to test development and analysis rather than test administration, allow more time to ensure that PISA reporting and the assessments themselves adhere to the highest quality standards, and give OECD more time to vet innovations.

The PISA program is currently spending too much time and too much money on secondary innovative domains and a three-year survey cycle. Those resources should be redirected into research that extends the existing PISA program to investigate system-level educational interventions that can increase our understanding of what policies are associated with improvements in learning outcomes.

OECD members and other PISA participants have long called for . It is time for the OECD to acknowledge these calls and to recognize that it is time for a systematic reconsideration of PISA鈥檚 design, periodicity, and quality control.

Related Tags:

A version of this article appeared in the February 13, 2019 edition of 澳门跑狗论坛 as Is PISA a Victim of Its Own Success?

Events

Artificial Intelligence K-12 Essentials Forum Big AI Questions for Schools. How They Should Respond鈥
Join this free virtual event to unpack some of the big questions around the use of AI in K-12 education.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 澳门跑狗论坛's editorial staff.
Sponsor
School & District Management Webinar
Harnessing AI to Address Chronic Absenteeism in Schools
Learn how AI can help your district improve student attendance and boost academic outcomes.
Content provided by 
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of 澳门跑狗论坛's editorial staff.
Sponsor
Science Webinar
Spark Minds, Reignite Students & Teachers: STEM鈥檚 Role in Supporting Presence and Engagement
Is your district struggling with chronic absenteeism? Discover how STEM can reignite students' and teachers' passion for learning.
Content provided by 

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide 鈥 elementary, middle, high school and more.
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.

Read Next

Federal White House Starts Scrapping Pending Regulations on Transgender Athletes, Student Debt
The Biden administration plans to jettison pending regulations to prevent President-elect Trump from retooling them to achieve his own aims.
6 min read
President Joe Biden delivers remarks on lowering prices for American families during an event at the YMCA Allard Center on March 11, 2024, in Goffstown, N.H.
President Joe Biden delivers remarks on lowering prices for American families during an event at the YMCA Allard Center on March 11, 2024, in Goffstown, N.H. His administration is withdrawing proposed regulations that would provide some protections for transgender student<ins data-user-label="Matt聽Stone" data-time="12/26/2024 12:37:29 PM" data-user-id="00000185-c5a3-d6ff-a38d-d7a32f6d0001" data-target-id="">-</ins>athletes and cancel student loans for more than 38 million Americans.
Evan Vucci/AP
Federal Then & Now Will RFK Jr. Reheat the School Lunch Wars?
Trump's ally has said he wants to remove processed foods from school meals. That's not as easy as it sounds.
6 min read
Image of school lunch - Then and now
Liz Yap/澳门跑狗论坛 with iStock/Getty and Canva
Federal 3 Ways Trump Can Weaken the Education Department Without Eliminating It
Trump's team can seek to whittle down the department's workforce, scrap guidance documents, and close offices.
4 min read
Then-Republican presidential nominee former President Donald Trump smiles at an election night watch party at the Palm Beach Convention Center, Nov. 6, 2024, in West Palm Beach, Fla.
President-elect Donald Trump smiles at an election night watch party at the Palm Beach Convention Center on Nov. 6, 2024, in West Palm Beach, Fla. Trump pledged during the campaign to eliminate the U.S. Department of Education. A more plausible path could involve weakening the agency.
Evan Vucci/AP
Federal How Trump Can Hobble the Education Department Without Abolishing It
There is plenty the incoming administration can do to kneecap the main federal agency responsible for K-12 schools.
9 min read
Former President Donald Trump speaks as he arrives in New York on April 15, 2024.
President-elect Donald Trump speaks as he arrives in New York on April 15, 2024. Trump pledged on the campaign trail to eliminate the U.S. Department of Education in his second term.
Jabin Botsford/The Washington Post via AP