Corrected: In the print version of this story, the non-profit group overseeing the Teacher Advancement Project was misidentified. The correct group is the National Institute for Excellence in Teaching.
A variety of federally financed grants based on performance pay are providing insights into how districts and teachers can collaborate to implement sustainable programs designed to improve teaching and learning.
But the question of whether those will yield measurably higher student achievement, applicant pools with better-qualified teachers and principals, and improved retention of effective teachers so far remains unanswered, say researchers, district administrators, and federal officials.
鈥淲e are learning a good deal about the implementation barriers that TIF grantees are encountering and their strategies to overcome these obstacles,鈥� said Matthew G. Springer, the director of the at Vanderbilt University in Nashville, Tenn. 鈥淸But] it is still debatable whether the incentive plans designed and implemented as part of TIF will actually realize their intended purpose.鈥�
The 34 grantees taking part in the Teacher Incentive Fund, which has received more than $196 million since its inception two years ago, are still in the early stages of collecting and tracking data. Instead, they have focused on practical matters, including the creation of systems to link teacher and student data, the provision of professional-development offerings for teachers and school leaders, and the refinement of techniques for determining pay bonuses.
Each grantee has made at least one round of bonus payouts, according to federal officials. Several sites have faced setbacks.
One district, Lakewood County, Fla., dropped out in June because of a disagreement between the superintendent and the school board over whether to use district funds to include schools outside the grant鈥檚 scope in the first year.
Still, a deputy assistant secretary for elementary and secondary education at the U.S. Department of Education underscores the value of the information gained from both successes and problems in implementation.
鈥淲ill we have advanced the field [of performance pay]? Absolutely. Does that mean everything these grantees are doing is perfect? Absolutely not,鈥� said Amanda Farris. 鈥淭he legacy of what TIF has been able to do is this informing of the field, perhaps more than any other program aside from Reading First.鈥�
As the first federal funding stream exclusively dedicated to performance pay, also known as merit pay, the incentive fund is likely to be scrutinized on Capitol Hill. So far, appropriators have funded only continuation grants, but a possibility of a boost for fiscal 2009 exists. A House appropriations subcommittee has put forth a preliminary figure of $112 million.
Staff-Development Focus
Conceived by the Bush administration and established by a 2006 appropriations bill, the program has among its goals increasing the number of effective teachers in low-performing schools and improving student achievement. Projected five-year grant awards range from $1.1 million for a Massachusetts charter school to $34 million for a South Carolina program in 23 schools.
Early reports from grantees stress the importance of a startup year to build professional development and collaboration among teachers, principals, and union leaders, Ms. Farris said.
Officials of the Hillsborough County district in Florida spent the bulk of their first year explaining to teachers and principals how the program鈥檚 bonus plans operate. They also focused on instituting opportunities for members of the instructional staff to master the performance standards to which the bonuses are tied.
Such opportunities are crucial, said Rob Weil, the deputy director of educational issues for the 1.4 million-member American Federation of Teachers.
鈥淎 lot of [performance-pay] programs are built through a recognition system,鈥� Mr. Weil said. 鈥淓ighteen months after a teacher has done something, she gets an award. You ask the teacher what she did to improve student learning, and she鈥檒l say, 鈥業 have no clue.鈥� That doesn鈥檛 translate into system improvement.鈥�
Hillsborough County explicitly linked new training opportunities to the bonuses.
The program hinges on 12 professional-development courses open only to teachers and administrators in the district鈥檚 TIF schools. They include workshops for analyzing student-achievement data and establishing professional-learning communities.
鈥淵ou don鈥檛 want to just say [to teachers], 鈥楧o better,鈥� 鈥� said Tanly Cabrera, the TIF project manager of the 182,000-student district. 鈥淪chools here have an opportunity for their teachers and administrators to attend quality staff development for their professional growth. In turn, we鈥檙e hoping for improved learning gains.鈥�
Scaling Back in Philadelphia
Of the lessons the incentive fund has taught so far, one of the more painful is that even a well-intentioned plan with a strong professional-development component can begin to unravel because of financial pressures, turnover in core leadership, and a breakdown in communications among stakeholders.
Philadelphia鈥檚 original plan under its $20.5 million grant was to offer the Teacher Advancement Project model in 20 public schools. TAP, an initiative overseen by the Santa Monica, Calif.-based nonprofit , offers a four-pronged approach to merit pay that includes instructionally focused professional development, 鈥渕aster鈥� and 鈥渕entor鈥� teaching positions, and standards-based evaluations.
Initially, the project received support from the Philadelphia Federation of Teachers, according to Tim Field, the deputy of the district鈥檚 office of innovation and new initiatives. But negotiations broke down.
鈥淭he plan did not include any of our suggestions shared at a number of meetings prior to the submission,鈥� said Jerry T. Jordan, the president of the union, an AFT affiliate.
Mr. Field also attributed the failure to reach agreement to an untimely deficit in the district鈥檚 budget and to the looming departure of Paul G. Vallas, who had announced his resignation as the school system鈥檚 chief executive officer in early 2007.
The program has shifted to low-income charter schools with the highest teacher-turnover rates, officials said. The 167,000-student district hopes to reopen negotiations with the union this year.
Midcourse Change
Collaboration and communication between teachers and district were crucial to getting Houston鈥檚 grant back on track after a rocky beginning.
Both Mr. Weil of the AFT and Ms. Farris of the Education Department pointed to the district鈥檚 $12 million grant as one that underwent retooling at the behest of teachers and emerged a significantly better program.
The grant supplemented a new initiative designed to help teachers shift from focusing on the static 鈥減roficient鈥� achievement level on statewide standardized tests to examining growth at all levels of student achievement. The district settled on an in-house model for analyzing student growth and awarding bonuses to high-performing teachers.
But when the first bonuses were announced in January 2007, schools that fared well differed markedly from those that had received awards under a previous initiative. And principals did not feel that bonuses were awarded to the most effective teachers in a school building, said Carla Stevens, Houston鈥檚 assistant superintendent for research and accountability.
鈥淚t was a very big deal,鈥� Ms. Stevens said. 鈥淭here were a lot of people who were very angry. They couldn鈥檛 see the results. They didn鈥檛 understand our analyses.鈥�
The 200,000-student district reconstituted a teacher-advisory committee to gain feedback for improving the system. Among the panel鈥檚 first recommendations was to switch to a more statistically robust, nationally recognized methodology for examining teachers鈥� effect on students鈥� academic growth, Ms. Stevens said.
Houston now uses the value-added model pioneered by William L. Sanders, a senior research manager at the Cary, N.C.-based SAS Institute, a vendor of business-intelligence software.
Ms. Stevens said the switch allowed the district to institute a Web-based delivery system offering finer-grained achievement reports at the building level, grade level, and subject level, and to connect pay bonuses to broader school-improvement efforts.
The changes appear to have resulted in improved acceptance among Houston teachers. Only eight spoke out against the program at February and March school board meetings compared with hundreds last year, Ms. Stevens said.
But, she added, 鈥渢here are still a lot of people who don鈥檛 understand [student-achievement] growth, who don鈥檛 like one teacher getting more money than another. It鈥檚 going to take a while to move everyone in this direction.鈥�
For all the new information gleaned about how to make performance pay work, hard data on the TIF grants鈥� effect on student achievement, recruitment pools, and turnover rates are scarce.
District officials report some positive trends. In Houston, officials attribute student test-score improvements this year to the district鈥檚 rebranded program, Ms. Stevens said.
鈥淗aving the dollars gets teachers to focus on growth data, but the conversation turns from, 鈥楬ow much money am I getting?鈥� to 鈥榃ow, I made a lot of progress with these kids here, but not those,鈥� 鈥� Ms. Stevens said.
Susan Ostrich, the project director for the TIF charter schools in Philadelphia, said she has seen effects on recruitment.
鈥淚鈥檓 getting feedback that being part of the TIF where you can earn performance bonuses is attractive,鈥� she said. 鈥淲e鈥檙e also finding that people who may have opted to go someplace else are talking about staying, but it鈥檚 too soon for us to know how [TIF] is impacting retention.鈥�
Researchers who study performance-pay programs are eager for the data.
鈥淲ithout evaluation, we could experience another two decades of debate on whether or not [performance pay] is a good policy,鈥� Mr. Springer said.
To reflect the different approaches taken, the Department of Education has permitted each grantee to design its own evaluation plan and research questions. Although the agency is planning a comprehensive study, that leeway could complicate efforts to draw large-scale conclusions about the incentive fund鈥檚 effects.
Of the site-based evaluations, only one grantee is using a randomized controlled trial鈥攖he 鈥済old standard鈥� favored by the Bush administration for education research. Two others are using quasi-experimental techniques. Such methodologies can attribute changes in test scores or turnover rates to the programs.
Daniel Goldhaber, a research professor at the , at the University of Washington in Seattle, expressed concern that the evaluations would not address questions about whether the grants, over time, change the composition of the teacher and principal workforces.
鈥淚t might be [a change] that takes longer to realize,鈥� he said.
Coverage of policy efforts to improve the teaching profession is supported by a grant from the Joyce Foundation.
