A prominent state education group has recalled an online report that presented the number and percentage of schools that states have identified as needing improvement under the No Child Left Behind Act.
Saying that journalists had used the data to make 鈥渦nwarranted negative assessments鈥� of the quality of states鈥� schools, the Education Commission of the States said in a widely distributed e-mail on March 31 that it had removed the six-page document from its Web site and wouldn鈥檛 distribute it further.
The document warned against judging states based on the number and percentage of schools failing to make adequate yearly progress, or AYP, under the federal law. But the disclaimer didn鈥檛 explain specifically that judging the quality of state school systems by comparing the data isn鈥檛 valid because each state sets its own AYP standards.
Though ECS officials said the data were accurate, they withdrew the report because of the way the information was being interpreted.
鈥淲hat I felt was not there, was the correct interpretation of what [the numbers] meant,鈥� Piedad F. Robertson, the president of the bipartisan, Denver-based clearinghouse on state education policies said in an interview last week.
鈥楤ottom-Feeder Ranking鈥�
The incident is the latest example of the way newly accessible data are changing how the public views schools. It also demonstrates why state officials are wary of efforts that produce state-by-state data on student achievement.
For example, a new Web site produced by Standard & Poor鈥檚 and state officials allows users to compare schools鈥� student-achievement data within their own states. That kind of comparison is valid, state officials say, because those schools administer the same tests and are expected to hold students to the same standards. (鈥淥nline Tools For Sizing Up Schools Debut,鈥� March 30, 2005.)
But if the site鈥檚 users want to compare a state鈥檚 student achievement with another state鈥檚, they are able to look only at scores on the National Assessment of Educational Progress and college-entrance exams鈥攖he only testing programs given in every state.
The ECS report, which was released online last month, documented the number and percentage of schools in every state that are in various stages of school improvement under the federal law. Those stages range from the mandate that schools in the first year of an improvement plan offer students a transfer to another public school, to the implementation of a state-approved school improvement plan in the fifth year.
The data were presented in a state-by-state table at the end of the report.
The report explained that the numbers and percentages of schools not making AYP vary between states because of a variety of differences in the ways states are implementing the 3-year-old law, a reauthorization of the Elementary and Secondary Education Act. It said states鈥� standards differ, as do the difficulty of their tests and their short-term goals for making adequate progress.
But some journalists glossed over such warnings in their reporting, ECS officials said.
In Hawaii, for example, the Honolulu Star-Bulletin ran a story headlined: 鈥淚sle Schools Fare the Worst.鈥� The story highlighted the ECS report鈥檚 finding that 10 percent of Hawaii鈥檚 schools were in their fifth and last year of needing improvement鈥攖he highest percentage in the nation.
Hawaii has 28 of its 284 schools at that stage because it has set ambitious AYP targets, a state spokesman said. It was unfair to label the state鈥檚 schools as the worst in the nation because those schools might actually have higher student achievement than in states with lower AYP goals, said Greg Knudsen, the spokesman for the state department of education.
State-to-state comparisons of AYP numbers are unfair 鈥渂ecause states have such different standards and different timetables,鈥� Mr. Knudsen said.
In the only other news story cited by ECS officials, a March 29 story in the Arkansas Democrat-Gazette said Arkansas 鈥渉as some of the highest percentages in the nation of public schools categorized as academically troubled.鈥�
Two days later, an editorial said the report gives 鈥淎rkansas its usual, bottom-feeder ranking鈥� on school quality.
The information collected does include worthwhile data, Mr. Knudsen of the Hawaii education department and Todd Ziebarth, the author of the report, agreed.
Useful Information
State officials could look at the number of schools each state is identifying as failing to achieve AYP, said Mr. Ziebarth, a policy analyst for Augenblick, Palaich and Associates, a Denver-based consulting firm that conducted the research for the ECS.
In doing so, he suggested, they can determine if they are labeling too many or too few schools as not reaching their goals. He said states also could find other states that have had experience with the interventions available for schools needing help, such as restructuring of those schools鈥� programs and staffs. 鈥淚t helps to provoke states to think about how their system compares to others,鈥� he said.
But Ms. Robertson said that while such information is useful, the ECS plans to produce reports that highlight states鈥� progress in complying with the law.
鈥淲hat we need to concentrate on is working with every individual state,鈥� said Ms. Robertson, a former president of Santa Monica College near Los Angeles who became president of the ECS Feb. 1.
