A new study suggests that the methods most states are using to judge schools鈥� academic progress may produce an incomplete鈥攁nd possibly misleading鈥攑icture of the jobs the schools are doing.
Read from the . (Requires free registration.)
Based on two years of test-score data for 230,000 students in 22 states, the study was released last week by the Northwest Evaluation Association, a nonprofit group that provides testing help to 1,200 member school districts.
The findings are noteworthy, researchers said, because they come as states are gearing up to comply with the federal testing requirements outlined in the No Child Left Behind Act. Under the law, states are required to use test scores to determine whether schools are making 鈥渁dequate yearly progress鈥� and to hold schools accountable when students鈥� scores fail to rise.
To meet that requirement, most states are using simple test-score averages, said G. Gage Kingsbury, the director of research for the Portland, Ore.-based group. The problem with that method, he said, is that it might not reveal much about how much students are learning鈥攊n other words, their academic growth over time.
For example, two schools might have identical test-score averages and thus earn similar ratings under the federal law. What the numbers might not reveal is that students at one school started out farther behind students at the other school.
鈥淭here鈥檚 not too much difference between these schools in terms of NCLB,鈥� said Mr. Kingsbury. 鈥淏ut if I was a parent, I鈥檇 want my child in the school that鈥檚 causing more growth in students鈥� learning.鈥�
Similar, But Different
To explore how such a growth-oriented evaluation approach might play out, the Oregon researchers analyzed scores for students in grades 3-8 from 723 of the association鈥檚 member districts. Students took the tests in the spring of 2002 and 2003.
The goal was to gauge the amount of unexpected score gains that students made from one year to the next.
As they had predicted, the researchers found that schools with similar performance ratings actually varied widely in the test-score gains their students were making.
More than 20 percent of the schools that were rated high-performing by their states fell into the bottom quarter when measured by the researchers鈥� test-score-growth index.
The reverse was also true: Several schools with low average test scores had score gains as large as those of the highest-performing schools. The differences between similarly ranked schools sometimes added up to as much as two-thirds of a year in a students鈥� academic growth.
鈥淚f a low-status school started out with students with low status and yet moved them dramatically forward, I would argue that might actually be a success story,鈥� Mr. Kingsbury said.
Under provisions of the federal law that allow students to transfer out of failing schools, though, that school could lose students, Mr. Kingsbury said. The students who transfer could also conceivably end up at higher-ranked schools where they might make less academic progress than they did at the schools they left behind.
Likewise, Mr. Kingsbury argued, high- performing schools that seemed to be doing little more than nudging students鈥� learning along have no incentive to work harder under the evaluation methods that states are using now.
Students at high-performing schools didn鈥檛 鈥渢op out鈥� on the tests because the tests were given on computers that adapted the test questions to students鈥� actual test performance, he added. In other words, the programs assigned progressively harder questions to students as they answered more questions correctly on particular topics.
The drawback to such growth or 鈥渧alue added鈥� measurement techniques, Mr. Kingsbury said, is that tracking students鈥� progress is difficult when students move from school to school.
To address that problem, he said, states should assign students identification numbers that can be tracked electronically through their testing systems.
Still, he said, combining the growth measure with a measure of average test scores might give a clearer picture of the kinds of jobs that schools are doing in enhancing students鈥� academic progress than the methods that states rely on now.
Currently, Mr. Kingsbury said, most states turn to academic-growth measures only after a school has been labeled failing.
