Corrected: This article presented data on student test scores incorrectly. The study found that 4th grade students in a nationally representative sample of charter schools scored 4.2 points behind students in regular public schools on the National Assessment of Educational Progress in reading, and 4.7 points behind regular public-school students in mathematics, when various student characteristics were considered. Those score differences were measured on the NAEP achievement scale of zero to 500.
A federal reanalysis of 2003 test-score data from the National Assessment of Educational Progress finds that charter schools trailed regular public schools that year in student achievement in both reading and mathematics.
The average reading score for the 150 charter schools examined, taking into account a range of background characteristics of students and schools, was 4.2 percentage points lower than in a pool of more than 6,700 regular public schools, according to the report released last week by the National Center for Education Statistics, an arm of the U.S. Department of Education. In mathematics, the charters scored 4.7 percentage points lower.
is available from the . Learn more about the .
The study used data from the 2003 administration of the 4th grade NAEP, which included a special oversampling of charter schools to allow for comparisons between regular and charter public schools. The 150 charters are a nationally representative sample.
Many researchers and charter advocates urge caution in using data from NAEP鈥攖he congressionally required assessment of samplings of students in key subjects known as 鈥渢he nation鈥檚 report card鈥濃�攖o examine charter schools. They say, for instance, that the data represent a snapshot at one point in time, with no consideration of students鈥� prior academic achievement and no way of discerning what effect a charter school has actually had on students.
Using a statistical technique called hierarchical linear modeling, or HLM, the study sought to go further than a similar 2004 NCES comparison of charter and regular public school NAEP scores by considering multiple student and school characteristics simultaneously.
In the new analysis, the differences between charters and regular public schools in the unadjusted data were about 1 percentage point higher in both reading and math than after the student and school characteristics were taken into account.
While the earlier analysis studied student-level scores, the new study focused on the average scores of schools, said Henry I. Braun, a researcher at the Educational Testing Service, of Princeton, N.J., who was the study鈥檚 lead author.
Overall, the report鈥檚 findings are 鈥渘umerically very close鈥� to the earlier NCES study, Mr. Braun said. 鈥淭he statistical significance is more extreme here,鈥� he said, 鈥渂ut in terms of the sort of policy results, there鈥檚 not much change after doing this adjustment.鈥�
Results Consistent
During an Aug. 22 conference call with reporters, Mark S. Schneider, the commissioner of the NCES, pointed to a range of caveats about the data, however. 鈥淭he report was based on 2003 data and may have limited applicability to today鈥檚 world of charters,鈥� he said. 鈥淎bout half of the [charter] schools were less than 5 years old.鈥�
Mr. Schneider also believes that the NCES should not undertake studies such as this one, begun under his predecessor, that entail what he sees as subjective methods.
U.S. Secretary of Education Margaret Spellings also emphasized that many charter schools were 鈥渞elatively new鈥� in an Aug. 22 statement on the study that reflected the Bush administration鈥檚 general support of the independent but publicly financed schools. 鈥淲e need to examine how they improve student performance over time for a better picture of how they compare to traditional public schools,鈥� she said.
Other charter supporters stressed that the study looks at only one year of data.
鈥淭he real question about charter school effectiveness is 鈥楢re kids doing better than they would be doing in other schools?鈥� 鈥� said Paul T. Hill, who heads the National Charter School Research Project at the University of Washington, in Seattle. 鈥淭his kind of snapshot study really doesn鈥檛 let you answer that question. You can鈥檛 sort out what the kid learned in the charter school.鈥�
Nelson Smith, the president of the Washington-based National Alliance for Public Charter Schools, suggested the report is recycling old data.
鈥淭his is the fourth time the public has been greeted with essentially the same 鈥榥ews鈥� derived from a source that is singularly ill-suited to the measurement of charter schools鈥� performance,鈥� he said.
But an official at the American Federation of Teachers, which has become increasingly critical of charter schools and issued its own widely discussed report on the same data two years ago, argues that the new study is very telling.
鈥淭hey don鈥檛 like the results, so they鈥檙e dismissing the methodology,鈥� Nancy Van Meter, an expert on charters at the AFT, said of charter proponents. 鈥淚 think that the preponderance of evidence is coming in saying charter schools are not outperforming public schools on a consistent basis.鈥�
