While other ideas for revamping the No Child Left Behind Act are taking center stage, a quiet debate is unfolding over proposals to tinker with the law鈥檚 definition of what constitutes 鈥渟cientifically based research鈥� in education.
The phrase is one of the most oft-repeated in the lengthy text of the nearly 6-year-old law. Sprinkled through the federal education statute more than 100 times, the references to 鈥渟cientifically based research鈥� require educators to rely on such studies in choosing everything from approaches to reading instruction to anti-drug programs for students. And that鈥檚 not to mention the law鈥檚 use of such related terms as 鈥渆vidence-based鈥� research.
But the legislative definition of 鈥渟cientifically based research,鈥� which favors randomized or experimental studies over other kinds of research in determining what works in schools, has also been criticized for promoting a narrow view of educational scholarship.
Leaders of the House Education and Labor Committee, in a circulating since late summer, would tone down that emphasis on scientific experiments by stipulating that studies aimed at determining whether an educational program or practice works may include鈥攂ut are not limited to鈥攔andom-assignment experiments.
The proposed change has raised concerns among some federal education research officials and research groups.
鈥淚鈥檓 not thrilled with that provision,鈥� said Grover J. 鈥淩uss鈥� Whitehurst, the director of the Institute of Education Sciences, the research arm of the U.S. Department of Education. He has been spearheading the department鈥檚 efforts to promote rigorous education research.
鈥淚 don鈥檛 know of any credible expert who doesn鈥檛 think a randomized research design doesn鈥檛 give you the best possible answer to causal questions,鈥� he added.
The Gold Standard
The Knowledge Alliance, a Washington group representing a mix of research and research-and-development centers, also voiced opposition to the draft bill鈥檚 proposed change to members of Congress.
But other Washington-based education groups, such as the American Educational Research Association and the Software & Information Industry Association, say the new language would be an improvement over current law.
鈥淔or a bill like NCLB, we think it reflects a much more appropriate understanding of research methodology,鈥� Gerald E. Sroufe, the director of government relations of the 25,000-member AERA, said of the proposed change. 鈥淢y opinion about RCTs [randomized-control trials] is that they are underutilized in educational research and overemphasized in political discussions.鈥�
Known as the 鈥済old standard鈥� for answering cause-and-effect questions, randomized field trials are those in which participants are randomly assigned to either experimental or control groups. Such studies are rare in education, partly because educators balk at giving an intervention to one group of students but not another. While such experiments can determine what works, scholars say, they can鈥檛 explain why.
The House draft bill鈥檚 proposed definition of 鈥渟cientifically valid鈥� research is modeled on one used in the , which created the IES. But the draft legislation would omit the preference for randomized studies that is now expressed in both the NCLB and the ESRA laws.
Scientifically valid research, the House committee鈥檚 proposal states, would include studies that are 鈥渞igorous, systematic, and objective鈥� and that present findings and make claims that are 鈥渁ppropriate鈥� to the methods used. The discussion draft also gives a nod to examining the weight of evidence鈥攊n other words, the consistency of findings across multiple studies or sites鈥攁nd presenting clear, detailed data so that findings can be replicated.
鈥淚 think there鈥檚 a general movement away from the more rigorous definition of the so-called 鈥榞old bar,鈥欌�� said Charles Blaschke, the president of Education TURNKEY Systems Inc. of Falls Church, Va., which tracks federal education legislation for publishers.
James W. Kohlmoos, the president of the Knowledge Alliance, said the apparent backlash against the current law鈥檚 references to 鈥渟cientifically based research鈥� on Capitol Hill may stem from bad publicity over the Education Department鈥檚 management of the Reading First program. The program requires states to use practices grounded in 鈥渟cientifically based research.鈥�
In a series of reports issued since fall 2006, the department鈥檚 inspector general said program administrators may have steered the grant-application process for the $1 billion-a-year initiative to ensure that particular reading programs and instructional approaches were widely used by participating schools, and that others were left out鈥攊ncluding some that had solid research track records.
鈥淩eading First didn鈥檛 help promote the legitimate implementation of 鈥榮cientifically based research鈥� and caused a lot of confusion about the concept,鈥� Mr. Kohlmoos said.
鈥楻igorous鈥� and 鈥楢ppropriate鈥�
Meanwhile, a pair of bills introduced in the Senate last month offer yet another view of what 鈥渟cientifically based research鈥� should look like. Sponsored by Sen. Richard G. Lugar, R-Ind., and Sen. Jeff Bingaman, D-N.M., the proposed would grant schools that use 鈥渞esearch-proven鈥� programs in Reading First a competitive preference of 10 points. A companion bill, the , would authorize up to $100 million for the development and evaluation of such programs.
Both bills say 鈥渞esearch-proven鈥� programs are those evaluated by at least two studies that use academically equivalent control and treatment groups, include sample sizes of five or more classes or 125-plus students, and use standardized assessments rather than developer-created measures, among other criteria.
鈥淭he hope is that some of what鈥檚 in these bills could get folded into NCLB,鈥� said Robert E. Slavin, a co-founder of the Success For All Foundation, whose allegations that the Reading First program was being mismanaged launched the inspector general鈥檚 inquiries. Mr. Slavin helped write the proposed definition in the Senate bills.
In the end, though, said Mr. Kohlmoos, lawmakers need to be consistent within NCLB and across other education laws.
鈥淵ou should have clarity and consistency or the effort to promote rigor goes nowhere,鈥� he said.
