澳门跑狗论坛

Academic tracking in secondary education appears to confound an increasingly common method for gauging differences in teacher quality, according to two recently released studies.

Failing to account for how students are sorted into more- or less-rigorous classes鈥攁s well as the effect different tracks have on student learning鈥攃an lead to biased 鈥渧alue added鈥� estimates of middle and high school teachers鈥� ability to boost their students鈥� standardized-test scores, the papers conclude.

鈥淚 think it suggests that we鈥檙e making even more errors than we need to鈥攁nd probably pretty large errors鈥攚hen we鈥檙e applying value-added to the middle school level,鈥� said Douglas N. Harris, an associate professor of economics at Tulane University in New Orleans, whose study examines the application of a value-added approach to middle school math scores.

High-school-level findings from a separate second study, by C. Kirabo Jackson, an associate professor of human development and social policy at Northwestern University in Evanston, Ill., complement Mr. Harris鈥� paper.

鈥淎t the elementary level, [value-added] is a pretty reliable measure, in terms of predicting how teachers will perform the following year,鈥� Mr. Jackson said. 鈥淎t the high school level, it is quite a bit less reliable, so the scope for using this to improve student outcomes is much more limited.鈥�

Expanded Use

Value-added is a form of statistical modeling that measures students鈥� learning gains over the course of a year, while controlling for factors that could skew the estimates. Teachers鈥� impact on those test scores is estimated as the difference between students鈥� projected growth and their actual performance.

Once primarily used to analyze large sets of education data, the technique has expanded into policy.

States and districts are increasingly coupling value-added data with other sources of information, such as surveys of students and observations of teachers鈥� instructional skill, to inform teacher evaluations鈥攁nd in some cases, to identify teachers for performance bonuses or promotions.

The technique has been dogged with controversy, though, especially after newspapers in Los Angeles and New York City released reports on individual teachers based on the calculations, which critics deemed flawed and misleading.

More than 30 states now require evaluations of teachers to include student academic performance, according to the National Council on Teacher Quality, a research and advocacy group in Washington.

Value-added is only one of the measures being used, and its use is limited to certain subjects and grades. But its influence appears to be growing.

Tracking Bias

Mr. Harris鈥� paper will be presented at the annual conference of the Association for Public Policy and Management, to be held in Baltimore next month. Mr. Jackson鈥檚 study was published in working form by the National Bureau for Economic Research in January and updated several times since.

The researchers used similar methodologies to probe the question of how academic tracking might affect the teacher-performance estimates.

Mr. Harris and a University of Wisconsin graduate student, Andrew A. Anderson, examined several years of Florida achievement data. Their question: Is the population of students that takes each set of classes skewed in such a way that it biases the teacher calculations?

Each course in the state鈥檚 database was coded into one of three types: remedial, midlevel, and advanced. Then, the scholars analyzed whether the value-added estimates for teachers changed when they introduced a control for tracking鈥攊n essence, by subtracting out the average achievement difference among students in the relevant tracks.

They found that the estimates differed markedly as a result. After the authors accounted for tracking, between 30 percent and 70 percent of teachers in the sample were placed in a different performance quartile. (The degree of miscategorization depended on which quartile each teacher began in, and which track they predominantly taught.)

The scholars鈥� analysis also showed that teachers who taught more remedial classes tended to have lower value-added scores, on average, than those teachers who taught mainly higher-level classes.

That phenomenon was not due to the best teachers鈥� disproportionately teaching the more-rigorous classes, as is often asserted. Instead, the paper shows, even those teachers who taught courses at more than one level of rigor did better when their performance teaching the upper-level classes was compared against that from the lower-level classes.

The idea of tracking is a common-sense scenario for any parent who has faced the politics of middle school, Mr. Harris said. 鈥淚t鈥檚 not that surprising when you think about how tracking works,鈥� he said. 鈥淧art of it is based on whether your parents are the ones who are more savvy about this and are going to call the counselor and lobby for you to be in these higher courses.鈥�

But if such bias is not accounted for in policy, a teacher could, in effect, boost his or her value-addedscore simply by teaching all higher-level courses, the paper notes.

Value-added is less commonly applied at the high school level, and Mr. Jackson鈥檚 paper questions whether it makes sense to expand its use there.

He based his study on a set of end-of-course test-score data in English 1 and Algebra 1 from North Carolina. Mr. Jackson wanted to determine whether the content of specific tracks鈥攆or instance, taking several Advanced Placement courses, participating in a program that teaches study skills, or experiencing a sequence of higher-level science courses鈥攁ffects the formulas.

鈥淚t seems perfectly reasonable and plausible that when you learn Newtonian physics, you鈥檙e learning skills that make it easier to learn math,鈥� he said. 鈥淪o if you鈥檙e comparing two algebra teachers, you would want to see the students don鈥檛 differ in their exposure to mathematical concepts outside of algebra classes.鈥�

To find out, he separated students into tracks defined by whether they took a popular sequence of 10 courses, took remedial or advanced English and Algebra 1, and attended the same school.

Poor Predictors

Mr. Jackson found that, after controlling for such tracks, the value-added method was not able to determine top from bottom teachers in Algebra 1 with much precision. In English 1, the estimates picked up almost no meaningful differences between high- and low-performing teachers.

In addition, Mr. Jackson concluded that the estimates were generally poor at predicting how successive classes of students taught by those same teachers would do, calling into question whether policies based on the estimates would have much effect overall in helping to retain the best teachers.

鈥淲e know there are other ways in which we could be spending our energy to improve student outcomes,鈥� he said. 鈥淢y takeaway is that this is not it.鈥�

Both papers stand somewhat in contrast with other studies that use value-added methodology, most of which have been conducted on elementary-level data.

The Bill & Melinda Gates Foundation鈥檚 $45 million Measures of Effective Teaching study, for example, found that value-added estimates, over time, tended to be more predictive of future teacher performance than measures such as observations performed by principals or students鈥� impressions of their teachers. The value-added estimates were somewhat volatile, though, from year to year. (The foundation also provides grant support for 澳门跑狗论坛鈥榮 coverage of business and innovation.)

Concerns that selection bias may skew value-added calculations have been raised at the elementary level, too, but other researchers have concluded that using additional years of achievement data helps to mitigate that problem.

Addressing the issue at the middle and high school levels may be a complex matter, Mr. Harris said, because it isn鈥檛 yet clear whether adding controls, as the scholars have done in their research, accounts for all the ways tracking could introduce bias.

鈥淚t鈥檚 not clear that simply adding that indicator [of tracking] solves the problem,鈥� Mr. Harris said. 鈥淥ne reason is that how well we measure tracks, and even what鈥檚 happening in those different tracks, is going to vary across schools.鈥�

For instance, the factors that determine how students are assigned to tracks may differ from school to school, making it more difficult to properly control for them, he said.