There鈥檚 a war of sorts going on within the normally staid assessment industry, and it鈥檚 a war over the definition of a type of assessment that many educators understand in only the sketchiest fashion.
Formative assessments, also known as 鈥渃lassroom assessments,鈥� are in some ways easier to define by what they are not. They鈥檙e not like the long, year-end, state-administered, standardized, No Child Left Behind Act-required exams that testing professionals call 鈥渟ummative.鈥� Nor are they like the shorter, middle-of-the-year assessments referred to as 鈥渂enchmark鈥� or 鈥渋nterim鈥� assessments.
Or they shouldn鈥檛 be, at least according to experts inside and outside the testing industry, who believe that truly 鈥渇ormative鈥� assessments must blend seamlessly into classroom instruction itself.
鈥淚t makes me want to scream and run out of the room,鈥� said Ray Wilson, the executive director of assessment and accountability for the 33,000-student Poway Unified School District in Poway, Calif., referring to off-the-shelf commercial products labeled 鈥渇ormative assessment鈥� that major test-makers try to sell him. 鈥淚 still contend that so long as a teacher doesn鈥檛 have primary control [over assessment content],鈥� he added, 鈥測ou will never have truly formative assessment.鈥�
鈥淚 had not heard, frankly, that there was some resistance to companies selling formative assessment,鈥� said Robert Block, the associate vice president for K-12 learning and development at the Princeton, N.J.-based Educational Testing Service, which sells test questions in bulk to schools as 鈥渇ormative assessment item banks.鈥�
But he pointed out that school districts鈥� requests for proposals often ask specifically for formative assessments.
鈥淚t has become the standard,鈥� he said of the testing industry鈥檚 practice of labeling some assessment products as 鈥渇ormative.鈥�
鈥淚鈥檓 not sure if it鈥檚 good or bad鈥攊t鈥檚 just what the market is looking for.鈥�
鈥楢蝉蝉别蝉蝉惭颈苍迟蝉鈥�
The schism was in full view at the Council of Chief State School Officers鈥� National Conference on Student Assessment in Orlando last June. In the main exhibit hall, lined with competing assessment-makers, Peter D. Hofman, the vice president of marketing and business development at the Dover, N.H.-based testing company Measured Progress, handed out boxes of mints labeled 鈥淎ssessMints鈥� to strolling psychometricians and state-assessment directors.
鈥淭hese are the only legitimate formative-assessment products,鈥� declared Mr. Hofman, whose company does not sell formative-assessment materials.
He left unsaid his implicit message: The room鈥檚 rival exam firms鈥攍ike almost every other testing company in the country鈥攕ell something that鈥檚 not what it purports to be.
It鈥檚 not just semantic consistency that鈥檚 at stake: Formative assessments are a more than half-billion-dollar business in the United States, according to the latest analysis by Outsell Inc., a Burlingame, Calif.-based research and advisory firm for the information, education, and publishing industries.
That total, including tests, test delivery, scoring, scoring analysis, professional development, and other services, but not such bundled products as textbooks, accounted for about 30 percent of the $2.1 billion in overall assessment revenue generated in the United States in the 2006-07 academic year鈥攖he most recent year for which statistics are available.
What鈥檚 more, said Laurence Bloom, an affiliate analyst for Outsell, the latest estimates project that formative-assessment revenue will climb at a rate of 10 percent to 20 percent per year through 2010, making it the fastest-growing segment in the assessment market.
That鈥檚 a lot of money being spent on something that experts say can鈥檛 really be sold鈥攐nly practiced.
More Than Word Choice
Measured Progress is one of the few large testing companies taking a pass on the lucrative market. That鈥檚 mostly because Stuart R. Kahl, the company鈥檚 president and chief executive officer, takes his cues, as many other testing experts do, from a 1998 research-literature review of Paul Black and Dylan Wiliam, then at King鈥檚 College, University of London. The , which predated the formative-assessment market, concluded that the research to date showed achievement gains using formative-assessment strategies that were 鈥渁mong the largest ever reported for educational interventions.鈥�
鈥淲e use the general term 鈥榓ssessment鈥� to refer to all those activities undertaken by teachers鈥攁nd by their students in assessing themselves鈥攖hat provide information to be used as feedback to modify teaching and learning activities,鈥� they wrote. 鈥淪uch assessment becomes formative assessment when the evidence is actually used to adapt the teaching to meet student needs.鈥�
Product or Approach?
鈥淲hat they鈥檙e talking about is not what people are selling; it鈥檚 what teachers are doing day in and day out,鈥� said Mr. Kahl, whose company produces state standardized tests used by Kentucky, Massachusetts, Montana, New Hampshire, Rhode Island, and Vermont.
鈥淧eople say, 鈥榊ou鈥檙e just being picky about a word choice.鈥� I think it鈥檚 more serious than that,鈥� Mr. Kahl said. 鈥淭he consumers out there ... are being told [in advertisements and pamphlets] that these [products] satisfy their formative needs. They believe that and purchase it and don鈥檛 think they have to do anything more for formative assessment.鈥�
With so much money at stake for products sold under the label 鈥渇ormative,鈥� said W. James Popham, a professor emeritus of education at the University of California, Los Angeles, and the author of the 2008 book Transformative Assessment, 鈥淭hey鈥檇 call it 鈥楥lytemnestra鈥� if they thought it would sell,鈥� referring to the mythological Greek figure best known for helping kill her husband.
When test companies sell things and call them formative, he said, 鈥渢hese vendors are being disingenuous鈥攚e used to call it lying.鈥�
Calls seeking comment from the Maple Grove, Minn.-based Data Recognition Corp. and from Pearson Education, a division of the global media giant with headquarters in London and New York City, were not returned.
In response to criticism of commercial 鈥渇ormative assessment鈥� products, Hillary Michaels, a senior research scientist for Monterey, Calif.-based CTB/McGraw-Hill, said at a National Conference on Student Assessment session that 鈥渨hen it comes to formative assessment, we need to rethink some of the ways we鈥檙e accustomed to thinking of testing.鈥�
Avoiding the Term
Other experts acknowledge that the lack of widespread understanding about the nature of formative assessment makes the term vulnerable to appropriation, but don鈥檛 necessarily agree that testing companies are deliberately obfuscating the distinction.
鈥淚 would say that the interests of any business in maintaining their revenues is such that when the world around them changes, they redescribe what they have and repurpose what they have,鈥� said Neal M. Kingston, an associate professor of the Program in Research, Evaluation, Measurement, and Statistics at the University of Kansas in Lawrence who has held high-ranking positions at Measured Progress, CTB/McGraw-Hill, and the ETS. He now helps create the state鈥檚 annual standardized Kansas Assessment Program exam.
鈥淚t鈥檚 not that there鈥檚 complete agreement on what makes something 鈥榝ormative鈥欌�攊t鈥檚 all how you鈥檙e using it,鈥� Mr. Kingston said.
Richard J. Stiggins, the executive director of the Portland, Ore.-based Assessment Training Institute, mostly stays away from the term. 鈥淭here鈥檚 not universal agreement on what it means鈥擨鈥檝e actually stopped using the word,鈥� said Mr. Stiggins. He said that he now usually refers to formative assessment as 鈥渁ssessment for learning鈥� to cut down on ambiguity.
Mr. Stiggins said he doesn鈥檛 believe testing companies are shading the truth, but he added, 鈥淢erely selling people a test on the assumption that it will be used formatively won鈥檛 cut it.鈥�
In-House Differences
No one whom 澳门跑狗论坛 asked would identify a specific product or company that he or she thought was misusing the term. Many experts were leery of pointing to a company, in part because there are often differences of opinion within a company itself.
鈥淚t鈥檚 really hard to do that because even within companies, there鈥檚 a wide range,鈥� said Mr. Kingston. His own university, for example, uses the word formative when referring to an assessment its staff created.
Another example is Mr. Stiggins. Although he has made the assertion 鈥渇ormative assessment isn鈥檛 something you buy鈥攊t鈥檚 something you practice鈥� something of a mantra in the many talks he gives to educators around the country, his company is owned by the ETS, which sells item banks under the label 鈥渇ormative.鈥�
Within any testing company, said Mr. Kingston, 鈥測ou have your curriculum experts, you have your psychometric experts, you have your marketing and business people, and they all approach the needs that are out there in different ways.鈥�
Mr. Kahl, the CEO of Measured Progress, said he sympathizes鈥攖o a point. 鈥淚 know of individuals within those companies that feel [ambivalent about some products],鈥� he said. 鈥淥n the the other hand, you look at the [product] catalogs and you see things in there labeled 鈥榝ormative assessment,鈥� and you鈥檝e got to wonder who鈥檚 driving the truck in those big companies.鈥�
