For Jeannine Ugalde and her middle school students, using assessment data to inform classroom instruction is a regular part of the school day.
Drawing on the results of computer-adaptive tests given periodically throughout the year, the 7th and 8th grade humanities teacher and her students at Oak Valley Middle School in San Diego set classroom goals that target the areas the youngsters struggle with the most.
鈥淚t鈥檚 hard work coming up with goals, so they feel a lot more ownership of their education,鈥� said Ms. Ugalde, whose 1,050-student school is part of California鈥檚 Poway Unified School District. 鈥淎nd now they specifically say, 鈥楻emember, you said we were going to work on this.鈥� 鈥�
Jerry Chen, one of Ms. Ugalde鈥檚 7th graders, explained.
鈥淭he data can help me make goals because I learn where my weaknesses in school are, so I know what to make my goal,鈥� he said. 鈥淸The information] helps me find where I am and helps me set goals according to my needs.鈥�
As more schools gain the technological knowledge and hardware to implement computer-based tests, districts are showing a growing interest in computer-adaptive testing, which supporters say can help guide instruction, increase student motivation, and determine the best use of resources for districts.
A computer-adaptive assessment is one that uses the information it receives during the test to determine which question to present the test-taker with next. For example, when a student answers a question correctly, he or she will be presented with a harder question, but if the answer is wrong, the next question will be easier. Consequently, a 5th grader taking the test could answer questions at the 6th or 3rd grade level, depending on his or her responses.
This method of testing shortens the test by not asking high-achieving students questions that are too easy for them, and likewise not giving struggling students questions that are too hard.
In essence, 鈥渆ach student gets questions that are appropriate just for them,鈥� said David J. Weiss, a professor of psychology at the University of Minnesota-Twin Cities and an expert on computer-adaptive assessments.
With a computer-adaptive assessment, the questions adjust in difficulty based on the student鈥檚 previous answers. Here is how the assessment might work to determine a middle or high school student鈥檚 proficiency in basic grammar and usage.
SOURCE: Northwest Evaluation Association
鈥淸In a fixed-form test], low-ability students are going to get questions that are too difficult, and they鈥檙e going to be frustrated,鈥� he said. 鈥淸With an adaptive test], everybody will be equally challenged.鈥�
In addition to shortening the length of the test, the approach creates a fairer psychological test-taking environment for each student, said Mr. Weiss, who has studied computer-adaptive tests since the 1970s.
鈥楰id-Centric鈥� Assessments
The Northwest Evaluation Association, a nonprofit organization based in Lake Oswego, Ore., that partners with school districts to provide testing services, has created one of the most widely used computer-adaptive assessments. Called the Measures of Academic Progress, or MAP, it is used by more than 2,340 school districts in the United States, including the Poway Unified schools, and in 61 other countries.
By adjusting in difficulty based on the student鈥檚 performance, MAP makes assessment 鈥渒id-centric,鈥� said Matt Chapman, the president and chief executive officer of the NWEA. And after administering the assessment, the teacher can see not only if the student passed a certain benchmark, but also pinpoint exactly where the student鈥檚 achievement level is鈥攚hether below, at, or above grade level.
鈥淚t increases the information you learn about the student, the student鈥檚 growth, and how the class is doing,鈥� Mr. Chapman said of the method.
Teachers, then, can use the information 鈥渢o inform instruction at the classroom level,鈥� said Ginger Hopkins, the vice president of partner relations for the NWEA. 鈥淚t allows teachers to adjust whole-group instruction and create flexible grouping,鈥� for students at similar achievement levels, she said.
Some schools, such as the 150-student K-8 Trinity Lutheran School in Portland, Ore., have used the test to regroup classrooms based on the most efficient use of resources. Facing declining enrollment and budget constraints, the school has used MAP to reconfigure students into multiage classes, said Principal Jim Riedl.
The Chicago International Charter Schools鈥攁 group of 12 charter schools teaching 7,500 students鈥攊s using the assessment in part to evaluate the efficacy of each of the four educational management organizations, or emos, the network employs.
鈥淚t鈥檚 very high-stakes for the emo providers, and I鈥檓 not saying for our schools that are underperforming, it鈥檚 not scary,鈥� said Beth D. Purvis, the executive director of the group of schools. 鈥淏ut frankly, if you鈥檙e an underperforming [school], you should be scared. It is a fair way of saying, 鈥楪et your act together.鈥� 鈥�
Engaging Learners
Although many teachers were skeptical about the assessments when they were introduced in Poway Unified in 2001, 鈥渁s students鈥� performance on the state test provided us with evidence that it was working, the schools came on board one by one,鈥� said Ray Wilson, the recently retired executive director of assessment and accountability for the 33,000-student district, who oversaw the school system鈥檚 adoption of MAP.
The assessments have especially helped students on the upper and lower ends of the performance spectrum, said Linda Foote, the instructional technology specialist for the district.
In fixed-form tests, 鈥渟tudents who are high-performing could look good all year without much effort,鈥� she said, 鈥渁nd the struggling students could work and grow dramatically鈥� but still appear to be underperforming, she added.
With MAP, 鈥渨e don鈥檛 honor students who aren鈥檛 working and dishonor students who are,鈥� Ms. Foote said.
The assessments also play a key role in increasing student motivation, she said.
鈥淥ne of the most stunning pieces is that the kids, because they finally have the connection between the work that they do and the progress they make,鈥� Ms. Foote said, are 鈥渕uch more willing ... to do the work.鈥�
The approach makes students active participants in their learning, she said, and 鈥渢akes the mystery out of education.鈥�
NCLB Constraints
Still, only one state鈥擮regon鈥攗ses a computer-adaptive assessment for reporting purposes under the federal No Child Left Behind Act.
Because NCLB requires that students be tested on grade level, most computer-adaptive tests, which may present students with questions above or below grade level depending on how they answer previous questions, are not allowed for accountability purposes.
However, Oregon鈥檚 test鈥攖he Oregon Assessment of Knowledge and Skills, or OAKS鈥攊s an adaptive test that stays on grade level, said Anthony Alpert, the director of assessment and accountability for the state department of education.
The test was piloted in 2001, the year the NCLB legislation passed Congress, and has since grown in popularity among school districts in the state, said Mr. Alpert. Today, almost all of Oregon鈥檚 state testing is through the computer-adaptive online assessment, he said.
鈥淭he real brilliance of the founders of the online system in Oregon was making it an optional approach and letting districts be innovative in making decisions in how they鈥檙e able to participate,鈥� he said. As an incentive to encourage districts to use OAKS, each student is allowed to take the assessment three times a year, and the state records only the student鈥檚 highest score.
Although no other state uses computer-adaptive assessments for reporting purposes under NCLB, Utah Governor Jon M. Huntsman Jr., a Republican, last month signed into law a bill that allows a pilot test of such assessments in his state.
This school year, three districts are participating in the pilot, said Patti Harrington, the Utah superintendent of public instruction. The decision to use computer-adaptive testing came from recommendations from a panel on assessment that included superintendents, legislators, parents, and state- and district-level administrators, as well as Ms. Harrington.
鈥淲e like it because it does so much more than just say whether or not a student reached a certain point,鈥� she said. 鈥淚t emphasizes growth and can be used throughout the year to inform instruction. And because it鈥檚 computer-based, teachers have the results immediately.鈥�
Addressing the Drawbacks
But computer-adaptive assessments are not the best way to evaluate students in every situation, experts point out.
鈥淲e think that [a computer-adaptive assessment] offers certain benefits, and it has certain drawbacks,鈥� said Scott Marion, the vice president of the National Center for the Improvement of Educational Assessment, based in Dover, N.H. 鈥淟ike all testing, it really depends on what you鈥檙e trying to do, and what you鈥檙e trying to learn.鈥�
Computer-adaptive tests don鈥檛 provide as detailed diagnostic information as standards-based assessments, said Mr. Marion.
鈥淎 score is not diagnostic. 鈥榊ou鈥檙e pretty good in this area鈥� is not really diagnostic,鈥� he said. 鈥淎nd in the standards-based world, that鈥檚 where we really run into trouble.鈥�
Like Mr. Marion, Neal M. Kingston, an associate professor at the University of Kansas, in Lawrence, and a co-director of the Center for Educational Testing and Evaluation, also has concerns about computer-adaptive assessments.
鈥淚tem-response theory, or the mechanism used to determine which items are easier or harder, ... assumes there鈥檚 a universal definition of hard and easy,鈥� he said. In some subjects鈥攔eading, for instance鈥攖hat assumption may hold, he said, but for other subjects鈥攕uch as high school math, which may combine algebra and geometry questions鈥攖hat assumption isn鈥檛 always correct.
鈥淭he adaptive-testing model assumes that everyone has taken [courses] or learned [subjects] in the same way,鈥� which is not always the case, Mr. Kingston said.
Subjects such as social studies, in which the curriculum varies greatly from place to place, present a particularly difficult challenge for computer-adaptive tests, which are often created on a national or state level, he said.
鈥淚 think that the companies that provide these kinds of tests have an obligation,鈥� Mr. Kingston said, 鈥渢o provide more technical information than I have seen them do in the past about the appropriateness of the models that they鈥檙e using to determine whether it鈥檚 the same test in rural Kansas as it is in the center of New York City.鈥�
Both Mr. Marion and Mr. Kingston say they recognize the potential of computer-adaptive testing, even as they voice caution.
鈥淚t鈥檚 a pretty nice framework for making certain types of tests for certain purposes,鈥� said Mr. Marion, 鈥渂ut the promises鈥攆rom what I鈥檝e seen鈥攆ar exceed the practice.鈥�
