U.S. Secretary of Education Margaret Spellings has indicated she may be open to new ways of measuring school progress, known as 鈥済rowth鈥� models, under the No Child Left Behind Act. But state officials are still unsure what the federal government might find acceptable.
In a speech last month, Ms. Spellings said the federal Department of Education would convene a panel of experts to consider ways of allowing states to incorporate that kind of gauge into their accountability systems. (鈥淪tates to Get New Options on NCLB Law,鈥� April 13, 2005.)
Unlike the current setup, which requires states to set and schools to meet annual targets for the percent of students who are rated 鈥減roficient鈥� or higher on state reading and math tests each year, growth models give schools and districts credit for increasing student achievement, even if they have not yet reached the proficiency goals.
Defining what鈥檚 meant by 鈥済rowth鈥� and how to measure it, though, is far from clear.
鈥淭here have been enough different versions of what that might be that I don鈥檛 think there鈥檚 yet a coherent view,鈥� said Brian Gong, the executive director of the Center for Assessment, a nonprofit group in Dover, N.H., that provides research and consulting services on assessment and accountability systems.
While a handful of states have proposed using some kind of growth model, said Kerri L. Briggs, a senior policy adviser in the U.S. Department of Education, Ms. Spellings and Raymond J. Simon, the acting deputy secretary of education, want to hear from researchers and experts in the field before approving any of the current proposals. The department has yet to name any members to the working group.
鈥淚t鈥檚 really too soon to get into any details,鈥� Ms. Briggs said last week. 鈥淚 think they want to know what鈥檚 possible, given the statutory parameters. I think they welcome the idea in the big picture.鈥�
The Massachusetts Model
Massachusetts has already won federal approval for one such model. The state uses a performance index to rate schools that awards 100 points for every student who scores at the proficient level or higher on state tests, but also gives schools reduced credit for students at lower performance levels.
Under the existing NCLB rules, schools and districts must meet steadily increasing targets both for their total enrollments and for specific groups of students鈥攕uch as those who are poor or minority鈥攗ntil all youngsters score at the proficient level in 2013-14.
See the related item,
To make adequate yearly progress in Massachusetts, schools must meet the state鈥檚 current target on the performance index for the total student population and for each subgroup. Schools or subgroups within a school that are far below the target can still make AYP, however, if their rate of improvement on the index, based on where they started, is steep enough that all students would reach proficiency by 2014 if it continued.
鈥淲e ought to distinguish between those schools down there that are making great progress versus those that aren鈥檛, and they ought to get some credit for it,鈥� said David P. Driscoll, Massachusetts鈥� commissioner of education.
Other states鈥攊ncluding Minnesota and Oklahoma鈥攗se a similar method under the federal law鈥檚 鈥渟afe harbor鈥� provision, designed to provide a second look at schools that don鈥檛 make AYP initially. It permits schools to make adequate progress if they reduce the percent of non-proficient students in the subgroup that missed its target by 10 percent from the previous year. But many consider such a jump too big to be reasonable for many schools.
Like most states, Massachusetts compares the performance of different cohorts of students over time to determine if a school is progressing. Because a school鈥檚 student body and the profile of a particular grade can vary by year, critics contend that鈥檚 like comparing apples and oranges.
In addition, because test scores are strongly correlated with such student characteristics as poverty, researchers argue that such 鈥渟tatus鈥� models say more about who attends a school than about how much its educators contribute to student learning.
Value-Added Models
In contrast, Tennessee has proposed using a 鈥渧alue added鈥� system to measure its growth under the federal law.
Value-added models track the performance of individual students over time and judge schools based on how much academic growth each student makes from year to year.
鈥淪ome people think that by following individual student growth, it will reveal good performance that is not shown in the status and improvement models that are incorporated in NCLB,鈥� said Mr. Gong. Whether that will prove true is unclear, he added.
So far, federal officials have been leery of value-added models largely because they are not designed to ensure that all students reach the proficient level by 2014. Instead, such models typically ask whether schools are making more or less progress with their students than comparison schools.
Tennessee has tried to address that issue. Under its proposal to the federal Education Department, a school that did not meet the state鈥檚 proficiency targets could still make adequate progress as long as the percent of students on track to pass the state鈥檚 high school exit test by graduation had increased by 10 percent over the previous year.
鈥淚 feel it鈥檚 another option to show progress in those areas where no progress is apparent,鈥� said Connie J. Smith, who directs the office of innovation, improvement, and accountability in the Tennessee education department. The state based the proposal on a dozen years of tracking individual student data in grades 3-8.
鈥淲e鈥檝e got a technique that we think shows progress over time and improvement toward meeting the objective for high school graduation,鈥� Ms. Smith said. 鈥淲e think we can be a model for the United States.鈥�
She鈥檚 been talking with other states, including Arkansas, California, and Pennsylvania, about Tennessee鈥檚 plan, which had not been approved by the federal government as of last week.
Minnesota received approval to use a growth model similar to that in Massachusetts for its accountability plan. But the state would like to incorporate an additional growth calculation, subject to federal approval, once it is testing students in each of grades 3-8.
Patricia D. Olson, the assistant commissioner for accountability and improvement in the Minnesota education department, said the state would like to use a value-added model starting with its spring 2007 tests. The state will be conducting pilot studies of different growth models next year.
鈥淚t gives us a way to use multiple measures to make a determination,鈥� Ms. Olson said, 鈥渁nd it should be fairer to everybody.鈥�
鈥淲e do have schools that are doing a wonderful job at pulling their kids up, but they can鈥檛 quite hit that bar yet,鈥� she said. 鈥淵ou want to be able to acknowledge that.鈥�
Wait and See
Other states are interested in using growth models to satisfy the demands of the NCLB law, but are waiting to see what happens.
鈥淚 know that we support, in concept, the idea that states should have the option to use growth models,鈥� said Pete Bylsma, the director of research, evaluation, and accountability for the Washington state education department. 鈥淲e鈥檙e not there yet.鈥�
Pennsylvania officials did some calculations to see what would happen if the state used an index similar to Massachusetts鈥�. While a number of schools would have made AYP, an equal number that made adequate progress under the law鈥檚 existing safe-harbor provision would not have done so.
鈥淪o at that point, we decided that we were not going to pursue that for this year,鈥� said John Weiss, the acting chief of the division of performance analysis and reporting in the Pennsylvania education department. The state also is piloting a value-added model that tracks individual students鈥� progress.
In Louisiana, Assistant Superintendent of Education Robin Jarvis said the state鈥檚 accountability commission is interested in value-added analyses, but wants to wait until the state has several years of data from new state tests before making a decision. 鈥淚 don鈥檛 expect them to come back to that until spring 2007,鈥� she said, 鈥渂ut they may surprise me.鈥�
How Much Is Enough?
One of the biggest issues, said Mr. Gong of the Center for Assessment, is what rate of growth should be expected and how to determine it.
Should the same rate apply to all students, for example, or should different trajectories be set for students with disabilities or those who speak limited English? While some people think the latter is more realistic, others argue that setting different expectations for students based on their characteristics is precisely what the federal law was trying to get away from.
It鈥檚 unlikely that federal officials would approve growth models that set such low or minimal growth expectations that most youngsters wouldn鈥檛 reach proficiency by 2014.
Sandy Kress, a former adviser to President Bush who helped craft the federal law, said: 鈥淢argaret [Spellings] is opening the door, but I think the key word is 鈥榩roficiency.鈥� I think there may be an openness to growth models that show an expectation of proficiency, in which students who are not now proficient but are on a clear path to proficiency can be credited for the growth.
鈥淚 do not see people getting growth models approved under NCLB language that are not what I call proficiency-rooted or proficiency-based,鈥� he said.
At a meeting on the use of such longitudinal-data systems at the Washington-based Urban Institute this month, researchers also raised cautions about the technical issues associated with value-added models. Those concerns included the quality of the tests, whether to adjust for student and school characteristics, and what to do when some data on individual students are missing.
What鈥檚 clear is that support is growing for at least exploring value-added and other growth measures.
鈥淚 think we have to proceed very carefully,鈥� said Steven Rivkin, an economist at Amherst College in Massachusetts, who has done value-added analyses using Texas data. But, he added, the way schools are now judged 鈥渋s just clearly wrong, and my sense is using the value-added or growth models is going to get you closer to the right answer.鈥�
Rob Meyer, who directs a center on value-added research at the University of Wisconsin-Madison, agreed: 鈥淭he attainment model is so obviously flawed that we ought to work really hard to figure out whether we can get value-added to work.鈥�
