In a quiet corner of the University of Chicago, Shazia R. Miller is putting the finishing touches on what she calls her 鈥渓ittle people.鈥� The small, human- shaped figures on a bar chart represent students as they go through high school. Those at risk of academic failure are orange, while those who graduate are green. Dropouts are red.
The simple but striking illustration, created by sifting through a mountain of data on student mobility and performance, shows what actually occurred to students from a real Chicago school. Here at the Consortium on Chicago School Research, producing high-quality studies to meet the needs of local educators and policymakers is the guiding creed.
鈥淵ou could send something out to a journal and reach a small sphere of people,鈥� said Ms. Miller, an associate director with the consortium. 鈥淏ut it鈥檚 not the same as having this kind of impact.鈥�
Amid a new push in education for more scholarship focused on the immediate concerns of practitioners, the work of the consortium represents an unusual example of 鈥渦sable research鈥� focused on one place. In a steady stream of reports, its researchers dissect Chicago district policy and paint detailed pictures of the conditions inside schools.
From teacher professional development to magnet schools, little that鈥檚 been done in the name of school improvement in the city has escaped the consortium鈥檚 scrutiny. Researchers have pointed out the shallowness of the content taught in many Chicago classrooms. And they鈥檝e shown the important link between students鈥� academic success in their freshman year and whether they graduate from high school.
鈥淭hey are not ivory-tower researchers,鈥� said Arne Duncan, the chief executive officer of the 437,000-student Chicago district. 鈥淭hese are people who roll up their sleeves and get out to the schools and conduct research that is applicable to real situations.鈥�
Accomplishing that goal has meant casting aside the usual relationship between researchers and their subjects. For the consortium, the local school community is both the object of analysis and its closest partner. Its members regularly cull the advice of education advocates and school officials in deciding what to investigate. And they keep them in the loop throughout the study process.
At times, it鈥檚 an uneasy marriage. The education leaders who help shape the consortium鈥檚 research agenda don鈥檛 always agree with the results. Its studies have fueled heated debate on a host of questions, such as how the city鈥檚 dropout rates should be calculated, and the wisdom of Chicago鈥檚 policy of requiring students to repeat grades if they don鈥檛 meet certain standards.
During the administration of former CEO Paul G. Vallas, who is now the head of the Philadelphia public schools, tensions ran so high that some feared a breakup with the consortium.
Anthony S. Bryk, the University of Chicago education professor who led the push to form the consortium, called that friction 鈥渢he price of relevance.鈥� What matters to him, he said, is not whether everyone agrees about the consortium鈥檚 conclusions, but that its studies raise the debate on school improvement in the city by grounding it in hard evidence.
鈥淲e鈥檙e informing the reform conversation,鈥� said Mr. Bryk, who is now on sabbatical at Stanford University. 鈥淲e鈥檙e constantly thinking about the big ideas that people can take away from our work that would make them think better about what they do.鈥�
Setting the Agenda
In large measure, the Consortium on Chicago School Research owes its existence to the 1988 state law that sought to greatly decentralize the governance of the city schools. The measure gave broad new authority to decide issues of personnel, budget, and instructional programs to a local council of parents and educators at each school.
To many observers, it seemed obvious that such dramatic change warranted a major research effort to gauge its effects. Such a project might help schools use their new powers wisely, the thinking went, by providing new data on what worked and what didn鈥檛.
The challenge was to find a way to give various stakeholders in the city the chance to help shape and inform the research agenda, while still producing rigorous, credible studies that didn鈥檛 appear biased toward any group鈥檚 perspective.
The answer was the consortium鈥檚 steering committee, forged in a series of meetings that Mr. Bryk, at the request of district officials, convened in 1990 with local school leaders, education advocates, and professors from area universities.
With 24 members, the panel includes academics, district leaders, union officers, and representatives of many of the city鈥檚 school-related interest groups. The body reflects a wide array of views.
Donald R. Moore, the executive director of the group Designs for Change, for instance, has long criticized the city鈥檚 efforts to end 鈥渟ocial promotion鈥� by holding back students based on test scores.
Meanwhile, Barbara Sizemore, a retired dean of DePaul University鈥檚 education school, has been one of the strongest supporters of the approach.
鈥淭here have been times when passions have been very high,鈥� said Alfred G. Hess Jr., an education professor at Northwestern University who also sits on the panel. 鈥淏ut the steering committee has weathered those.鈥�
While members of the panel offer advice on which studies to carry out, and even on how, the researchers retain ultimate control over the reports they produce.
The committee also doesn鈥檛 hold the consortium鈥檚 purse strings. With a staff of about 20, the consortium gets most of its $1.8 million annual budget from local philanthropies, primarily the John D. and Catherine T. MacArthur, Joyce, and Spencer foundations.
Still, the steering committee plays a crucial role in making sure the research staff stays focused on what matters most to Chicago鈥檚 education community, said Penny Bender Sebring, the consortium鈥檚 most veteran research associate.
鈥淚t ensures that the research we do is going to be a lot more useful, and that it鈥檚 going to be some answer to questions that people locally have,鈥� she said. 鈥淚t just keeps our feet on the floor.鈥�
Mining Tools
Teaming up with the district gives consortium researchers access to an unusually rich source of information. To date, the group has amassed records on some 1.4 million current and former Chicago public school students, including test scores, attendance rates, and grade transcripts. That trove of data is what allows the consortium to follow what happens to students as they go through the system.
Dorothy Shipps, a former consortium researcher who is now a professor at Teachers College, Columbia University, points to a consortium study on truancy to explain why keeping close track of students is important. Mining the extensive database showed that, in fact, many students considered truant at one school had actually moved to another. So part of the truancy problem was really a mobility one.
鈥淲e came up with very interesting findings you could never get if you didn鈥檛 have this longitudinal database,鈥� Ms. Shipps said. 鈥淜ids were moving among schools within a two- or three-mile radius.鈥�
Some of the consortium鈥檚 most powerful findings have emerged by linking Chicago students with the district鈥檚 own survey results. Since 1991, the research group has carried out periodic polls of Chicago principals, parents, teachers, and students. Items on the questionnaires are meant to probe the quality of a school鈥檚 climate, leadership, staff training, and instruction.
Those surveys are what allow the researchers to track how major education policies play out in the schools. For instance, three years after the 1988 decentralization law, a consortium report showed that some 40 percent of the city鈥檚 elementary schools had well-conceived plans to raise student performance. But many others had just tacked on an assortment of unrelated programs鈥攚hat the study memorably called 鈥淐hristmas tree鈥� schools.
Mr. Hess of Northwestern University says the snapshot made two important points: Despite some earlier predictions to the contrary, large numbers of schools were making good use of the new authority granted to them. Others clearly needed more guidance.
鈥淗ere was something that said some schools are doing better, and if some are, why aren鈥檛 the rest?鈥� Mr. Hess said. 鈥淚t kind of created the possibility for improvement.鈥�
To help individual schools see what they most need to work on, the consortium has also created a kind of report card based on its surveys. Each school that returns enough questionnaires to make for a statistically valid sample gets a document that includes tallies of its own answers.
From the responses, the consortium calculates a series of indices meant to show the school鈥檚 strengths and weaknesses in certain critical areas, such as parental involvement, student discipline, and professional development.
The reports aren鈥檛 used for accountability purposes. In fact, to ensure candid responses, they aren鈥檛 even given the district鈥檚 central office. Instead, they鈥檙e used by school leaders and staff members to pinpoint problems.
Barbara J. Martin, the principal at Holmes Elementary School, for instance, said the survey responses helped her see the need for improving communication among her staff. When she received her first consortium report, she said, 鈥渨e were just shocked to find that we didn鈥檛 do a lot of talking to each other.鈥�
鈥業n the Middle of It鈥�
By holding a mirror up to the district and its schools, however, the consortium often finds itself at the center of controversy. Said John Q. Easton, who was named executive director after Mr. Bryk left for Stanford: 鈥淲e鈥檙e sort of in the middle of it, all the time.鈥�
Especially contentious has been the consortium鈥檚 work focusing on the accountability measures put in place under Mr. Vallas, the district鈥檚 former chief executive.
Sent to run the schools by Mayor Richard M. Daley after the state gave the mayor control of the school system in 1995, Mr. Vallas established cutoff scores on standardized tests to determine whether students could go on to the next grade. The policy required students in danger of being held back to attend summer school, after which they鈥檇 be retained if they still didn鈥檛 meet the standard.
In general, the consortium鈥檚 studies have shown that the strategy has had mixed results. Although the summer school program has helped many students meet the requirements for promotion, the boost in academic achievement appears short-lived. And while the dropout rates of students who were held back haven鈥檛 spiked鈥攁s some critics of the effort had said would happen鈥攖hey haven鈥檛 plummeted.
The findings contributed to a public debate on whether Mr. Vallas鈥� highly publicized plan was working, straining relations between the district and the research group, according to many members of the consortium鈥檚 staff and steering committee. Some even worried that the district might cut the consortium off from the student data that is its lifeblood. But it didn鈥檛.
鈥淭here were concerns about the relationship breaking off,鈥� said Paul D. Goren, the acting president of the Spencer Foundation. 鈥淭here were efforts internally and externally to patch things up.鈥�
Mr. Vallas now downplays the tension. Often, he said, his dispute was more with what he saw as a particular spin in the consortium鈥檚 press releases and the commentary in its reports鈥攏ot with its findings. In fact, he has plans to replicate the consortium in his new district, Philadelphia, by starting a similar independent, privately supported research group dedicated to studying that city鈥檚 schools.
鈥淚 always viewed the consortium as an asset more than a liability,鈥� Mr. Vallas said recently, 鈥渆ven during our own version of the 鈥榙ays of rage.鈥� 鈥�
There is no doubt that the consortium enjoys good relations with the Chicago schools鈥� current leadership. Many of the policy initiatives of the current district chief, Mr. Duncan, appear to address concerns raised by the research group.
Mr. Duncan, for example, has broadened the criteria used to hold schools accountable for their results, adding factors meant to gauge students鈥� improvement, not just their performance.
And when he unveiled a major report last summer outlining new policy directions for the school system, it stressed many consortium findings, such as the importance of instructional 鈥減rogram coherence鈥� to ensure that students aren鈥檛 taught the same things grade after grade.
The Chicago consortium is a rare example of scholarship that has been able to shape policy while also studying it, said Michael W. Kirst, a professor at Stanford who has observed the research group from afar.
鈥淭hey鈥檝e got to be critical and influential,鈥� he said. 鈥淎nd they鈥檙e able to do that because of the depth of the research they do.鈥�
Coverage of research is underwritten in part by a grant from the Spencer Foundation.
