澳门跑狗论坛

The 53 teachers gathered around tables here have been called to a new kind of jury duty. But they won鈥檛 be deciding whether a fellow citizen is guilty of a crime: Their charge is to pass judgment on stacks of instructional materials.

Amid papers and coffee cups, they pore over a 90-page curricular unit on constitutional freedoms. In Socratic rounds of discussion, they explore the high school unit from dozens of angles, looking for fidelity to the common core.

How clearly does the unit state its purpose? Does it expect students to read texts that are rich and complex enough? Does it offer sufficient support for students who are struggling? Does it provide good, clear ways to assess how well students are learning as they go along?

These teachers are trying to answer one of the most vexing questions in the age of common-core instruction: Which materials fully reflect the new standards for English/language arts and mathematics? They鈥檝e come to this suburb of the nation鈥檚 capital from more than 20 states to learn and practice a new rating system for lessons and units that purport to be 鈥渇ully aligned鈥� with the Common Core State Standards.

The new system, called EQuIP, represents one way that teachers are trying to make sense of the flood of curricular offerings that鈥檚 been unleashed by the nearly nationwide adoption of the common standards.

There are other tools or processes, too, that are designed to evaluate instructional materials for common-core alignment. The two national teachers鈥� unions have launched free portals where teachers can post and comment on lessons. Student Achievement Partners, whose founders led the writing of the common standards, has a set of free online tools that can be used to judge the fidelity of instructional materials to the standards.

Reviewing Materials

For-profit groups鈥攍ike the Austin, Texas-based Learning List, which uses panels of judges to size up instructional materials鈥攁re also wading into the alignment-evaluation business. The Business Roundtable is talking with partners about creating a group to do 鈥�Consumer Reports-type reviews鈥� of common-core materials.

And a group of experts led by Maria M. Klawe, the president of Harvey Mudd College, is creating a nonprofit to review the most widely used common-core math materials.

The EQuIP program was created by Achieve, a Washington-based nonprofit organization that played a key role in launching the common-standards initiative in 2009. Achieve has worked for many years with states on academic expectations and accountability.

The approach for EQuIP originated with the work of three states鈥擬assachusetts, New York, and Rhode Island鈥攖hat designed a set of criteria to use in judging lessons and units for alignment with the standards.

The Tri-State Rubric, as it came to be known, created a buzz as other states tried it out and sought training to spread its use among teachers and curriculum writers. Achieve renamed the project EQuIP鈥攆or Educators Evaluating Quality Instructional Products鈥攁nd began training for states, districts, teachers鈥� unions, colleges of education, and others interested in using it.

Achieve selected its first round of teacher-jurors last June. It chose a second group in November. Both groups came together late last month for training and practice using the rubrics鈥攃hecklists of criteria鈥攖o evaluate math and English/language arts lessons and units that had been submitted by states, districts, groups of teachers, and nonprofit curriculum developers.

Jurors have reviewed about 40 of the 125-plus lessons and units that have been submitted so far, and of those, only nine have been deemed sufficiently aligned to be posted as resources on the EQuIP website, according to Alissa Peltzman, a vice president of Achieve who leads the project.

Curricular materials produced by large publishing houses鈥攚hich dominate tens of thousands of classrooms鈥攈aven鈥檛 been submitted to EQuIP for review, Ms. Peltzman said. EQuIP doesn鈥檛 anticipate evaluating much from that sector, either, in part because of licensing restrictions that limit those materials鈥� use, she said, and in part because it lacks the capacity to analyze such a big volume of materials.

At the session here in Arlington, Va., reaching consensus on whether a lesson was aligned wasn鈥檛 easy or quick. It took all day to evaluate and rate the constitutional-freedoms unit, with rounds of detailed analysis and areas of disagreement.

One table of teachers, for instance, was deeply divided on whether the unit lived up to a key criterion in the rubric: stating a 鈥渃lear and explicit purpose for instruction.鈥� The teachers also disagreed about whether the unit met a criterion that required materials to demand a good deal of writing based on evidence in a text.

One middle school teacher criticized the unit for providing 鈥渟caffolding,鈥� or support, for students in the assignments but not in the readings. A group of elementary school teachers at a nearby table got into an animated debate about whether the unit鈥檚 reading material from primary and secondary sources was challenging enough, yet still accessible to students.

Terri King Hunt, a teacher from Atlanta, told the group that she thought the unit fell short on that criterion.

鈥淵ou could say you threw a lot of material at them, but what did they get out of it?鈥� she said.

Next to Ms. Hunt, Kay Dugan, an assistant superintendent from Bensenville, Ill., pressed her colleagues to 鈥渂e hard鈥� in sizing up how well the unit provides ways to assess student learning day to day. 鈥淲e need to get better at giving students feedback through formative assessment,鈥� she told them.

Cautionary Notes

Since this was a sample exercise, votes on rating each 鈥渄imension鈥� of the materials were taken by a show of hands. Collectively, the teachers voting showed they thought the unit needed significant revision.

Had it been a real evaluation, three or more reviewers would have graded each dimension on a scale of 0 to 3, and written explicit feedback to guide developers in revising the materials. Then a 鈥渓ead reviewer鈥� would have written one evaluation, summarizing the feedback and assigning an overall rating.

The materials rated 鈥渆xemplar鈥� or 鈥渆xemplar if improved鈥� would be posted on EQuIP鈥檚 website. Those needing more revision, or not yet ready for review, would not be posted, but feedback would be returned to the developers.

Guiding curriculum developers is a central aim of the EQuIP project. In fact, a section of the training was devoted to giving effective feedback.

鈥淚t鈥檚 easy to get cranky, like 鈥榃hy didn鈥檛 you do this?鈥� and 鈥業f I were teaching this, I鈥檇 do it this way,鈥� 鈥� Judson Odell, one of the facilitators of the training, told the participants. 鈥淭ry to stay positive.鈥�

The first round of EQuIP judging returned individual reviewers鈥� comments to developers. That feedback could be 鈥渃onflicted and confusing鈥� because of the differences in each reviewer鈥檚 comments, Ms. Peltzman of Achieve said. So another layer鈥攖he lead review鈥攈as been added to the process in an attempt to send clear, overall messages about the revisions needed.

Evaluating instructional materials for quality or alignment is inherently thorny and subjective, and reaching consensus can be tricky. Assembling juries of experts to make those choices is hardly new.

In 19 states, panels comb through submitted materials and decide what to include on an 鈥渁doption list鈥� from which districts must choose if they purchase materials with state funds. Elsewhere, districts are left to shop for themselves.

Even those who welcome the evaluations by EQuIP and other organizations voice cautionary notes about the process.

Sandy Hayes, the immediate past president of the National Council of Teachers of English, said the EQuIP guidelines are 鈥渢errific tools for conversation鈥� as teachers write lessons together in their schools. But she worries that a rating system could subtly work to narrow the concept of 鈥済ood鈥� materials.

鈥淚n talking about what鈥檚 good, sometimes people can be silenced, or there鈥檚 this 鈥榞roupthink鈥� that happens, when you find yourself compromising to get consensus,鈥� Ms. Hayes said. 鈥淚 just wonder what鈥檚 lost in that process.鈥�

Another risk in creating panels to evaluate materials is that their findings can be viewed as silver bullets.

Jim Burke, an English teacher at Burlingame High School in California, said it鈥檚 important to avoid mistaking any panel鈥檚 approved materials for a 鈥渢otal curriculum solution.鈥�

鈥淚t does give you some assurance of quality about the 鈥榳hat,鈥� but it doesn鈥檛 deal with the heart of the work鈥攖he 鈥榟ow'鈥攈ow you are teaching your students,鈥� he said.

Educators who had come to the Virginia training and EQuIP jurors were eager, however, to see how the process could help them, and cadres of colleagues back home, move forward with the common standards. They seemed particularly interested in its value as a tool for creating their own instructional materials.

In Idaho, a small team of literacy coaches at the state department of education will use the EQuIP process this spring to help 250 teachers write model units, said Christopher Butts, one of the state coaches, who is serving as an EQuIP juror.

Suzanne Snider, a curriculum coordinator, said many of the teachers in her districts in California鈥檚 San Bernardino County 鈥渉aven鈥檛 had a lot of direction鈥� in figuring out what constitutes real alignment to the common core. 鈥淭eachers have been going on Pinterest, for goodness鈥� sake,鈥� she said. 鈥淲e really need this.鈥�

Different Uses

States, districts, and vendors have been using the EQuIP tools in a variety of ways. The Maryland education department submitted sample lessons and units, and found the reviewers鈥� feedback 鈥渆xplicit and valuable,鈥� said Ava B. Spencer, the state鈥檚 English/language arts coordinator. Teachers are working on revisions based on that feedback, she said.

In North Carolina, the state education department has used the EQuIP rubrics in training to help teachers think about how to design their own curriculum materials, said Julie Joslin, who oversees English/language arts. Eight regional programs in the fall of 2012 proved so popular that the department added seven more the next winter and spring, she said.

鈥淲e would get phone calls all the time, 鈥業s this a common-core-aligned lesson?鈥� 鈥� Ms. Joslin said. 鈥淭eachers were struggling to understand what that is, and the rubric does help with that.鈥�

Washington state has used EQuIP鈥檚 evaluation criteria as it searches for open education resources to build a digital library that the state legislature mandated in 2012. But 鈥渙ne of the big questions people had about open education resources was 鈥業t鈥檚 free, but is it good?鈥� 鈥� said Barbara Soots, the state鈥檚 program manager for such resources.

In scouring instructional materials from such sources as the Library of Congress, the National Endowment for the Humanities, and those developed by other states, including New York and Utah, Washington state鈥檚 panel of reviewers is employing a handful of evaluation tools as guidance, Ms. Soots said.

Along with EQuIP, which evaluates lessons and units, it is also using Student Achievement Partners鈥� Instructional Materials Evaluation Tool, which is more focused on full-course content, she said.

鈥楢 Wild West Moment鈥�

Expeditionary Learning has been both a consumer and a subject of the EQuIP process. The New York City-based nonprofit company, which runs 170 regular and charter schools, used the criteria to shape an English/language arts curriculum for grades 3-8 in New York state, and then submitted the curriculum to EQuIP reviewers for evaluation.

Scott Hartl, its chief executive officer, said he viewed participation in the evaluation process as important because of the way the curricular landscape is evolving in the common-core era.

鈥淚t鈥檚 a Wild West moment, with lots of people saying their materials are aligned to these new expectations,鈥� he said. 鈥淲e wanted our curriculum to go through the rigors of smart folks with a common vision looking at our stuff in relationship to what was out there.鈥�

No curriculum developer can rightfully draw conclusions yet about what鈥檚 good, Mr. Hartl said. That will be left to time and experience.

鈥淭here has been a tremendous wave of innovation and new-product creation that eventually will get sorted out by real-life market forces,鈥� he said. 鈥淭hat鈥檚 what will show us the results.鈥�