Widespread technical failures and interruptions of recent online testing in a number of states have shaken the confidence of educators and policymakers in high-tech assessment methods and raised serious concerns about schools鈥� technological readiness for the coming common-core online tests.
The glitches arose as many districts in the 46 states that have signed on to the Common Core State Standards are trying to ramp up their technological infrastructure to prepare for the requirement that students take online assessments starting in 2014-15.
Disruptions of testing were reported across Indiana, Kentucky, Minnesota, and Oklahoma and were linked to the states鈥� assessment providers: CTB/McGraw-Hill, in Indiana and Oklahoma; ACT Inc., in Kentucky; and the American Institutes for Research, in Minnesota.
Thousands of students experienced slow loading times of test questions, students were closed out of testing in mid-answer, and some were unable to log in to the tests. Hundreds, if not thousands, of tests may be invalidated.
The difficulties prompted all four states鈥� education departments to extend testing windows, made some state lawmakers and policymakers reconsider the idea of online testing, and sent district officials into a tailspin.
The testing problems were 鈥渁bsolutely horrible, in terms of kids being anxious,鈥� said Eric F. Hileman, the executive director of information technology services for the 43,000-student Oklahoma City schools. Some high school students were taking Oklahoma鈥檚 high-stakes tests, which require that students pass four out of seven end-of-instruction tests to graduate.
鈥淚t was heartbreaking to watch them,鈥� Mr. Hileman said. 鈥淪ome of them were almost in tears.鈥�
System Overload
The problems in Oklahoma and Indiana began on April 29.
Several states experienced major mishaps with their statewide assessments recently, breakdowns that caused delays and disruption for teachers and students.
Indiana
April 29: Over 30,000 test sessions were interrupted as students began taking state tests. The state department of education extended the testing window by three days.
April 30: Test interruptions spiked to 8 percent of test-takers, and the department suspended testing for the rest of the day.
May 1: The education department instructed districts to continue testing students, but to reduce the number of tests they plan to give daily by 50 percent.
May 2: The department extended the testing window an additional two days, for a total of five extra testing days.
Kentucky
May 1: School systems were ordered to suspend online end-of-course tests after dropped and slow connections were reported in about 25 districts throughout the state.
May 2: State officials said the problem was caused when its testing vendor, ACT Inc., reported that its system became overloaded. Company officials told the state that the capacity of the system would be increased, but the state department of education also said it would work with local districts to help them 鈥渕aximize the testing system鈥檚 capacity鈥� and avoid other problems. Online testing was scheduled to resume by May 8. State officials say they will provide districts that give online tests paper exams as an alternative.
Minnesota
April 16: Schools reported widespread problems with online testing. Up to 5,000 students鈥� tests were disrupted.
April 17: Testing resumed.
April 23: Test interruptions resurfaced. About 48 districts reported slow loading times or other problems.
April 24: A handful of districts reported persistent testing interruptions. About 60 students were affected.
May 1: Minnesota added one day to its testing cycle to allow districts to catch up.
Oklahoma
April 29: Students were taking state tests online when problems began at 9 a.m.
April 30: Testing began at 7 a.m.; problems arose again around 10 a.m. About 3,000 out of 300,000 students statewide experienced some test irregularities.
May 1: The state extended its testing window by two days to allow districts to catch up.
In Oklahoma鈥攚here roughly 300,000 students were using online tests and about 3,000 experienced problems鈥攊t was the end of the testing window for grades 3-8 and the middle of the testing window for high school students, said Tricia Pemberton, a spokeswoman for the state department of education.
At 9 a.m., students鈥� testing sessions were disrupted by the glitches, and some were unable to restart or to proceed, she said. The problems continued the following day, when the system crashed at 10 a.m.
Indiana experienced similar problems on those days, and districts were instructed later in the week by the state education department to reduce the number of tests being given by half to proceed with the state assessments.
Ms. Pemberton said the simple explanation from Monterey, Calif.-based CTB/McGraw-Hill was that computer servers could not handle the testing load.
In a statement, CTB/McGraw-Hill officials said that earlier practice simulations 鈥渄id not fully anticipate the patterns of live student testing, and as a result, our system configuration experienced service interruptions that impacted the testing process.鈥�
Kentucky officials were forced to suspend online end-of-course exams last week after problems, including slow or dropped connections, were reported in about 25 of the state鈥檚 school districts, said department of education spokeswoman Nancy Rodriguez. About 60 percent of Kentucky鈥檚 district鈥檚 deliver those state-mandated exams online, while the rest use paper-and-pencil assessments.
State officials issued a statement saying that the test vendor, ACT Inc., indicated the problem occurred when its system became overloaded, and that capacity for it had been subsequently increased. Online testing was scheduled to resume early this week, and state officials said they would provide districts with guidance on how to 鈥渕aximize the testing system鈥檚 capacity鈥� to avoid additional breakdowns.
In Minnesota, problems with online testing began on April 16 and were experienced in multiple districts across the state, affecting up to 5,000 students, said Charlene Briner, a spokeswoman for the state education department.
A week later, on April 23, 48 districts reported disruptions in online testing, and the following day, a handful of districts experienced further problems, she said.
Jon Cohen, an executive vice president of the AIR, the Washington-based not-for-profit research and assessment organization that provided the online tests in Minnesota, said the tests are designed to allow students to pause and log back in later if they鈥檙e experiencing technical problems.
According to AIR data, he said, about 3,000 students out of 15,000 being tested at the time had slower load times of more than 30 seconds. He said AIR servers were overloaded not by the number of test-takers, but by the large amount of diagnostic data the organization was collecting.
About 95 percent of students in Minnesota take the math portion of the state tests online, but only about 30 percent take the reading portion online, Ms. Briner said. That will change under the common-core standards, which Minnesota has adopted for English/language arts but not for math. All students will eventually have to take ELA tests online.
Ms. Briner said that in light of the recent online testing problems, however, the state is evaluating 鈥渨hether or not a paper option is better for accountability testing.鈥�
鈥淲e believe in moving to a next-generation set of assessments,鈥� she said, 鈥渂ut we鈥檙e also believers in making sure people have confidence in the accuracy of the information we report.鈥�
On 鈥楶ause鈥�
Others were also worried about the future, particularly when it comes to common-core testing.
Coincidentally, just before the testing problems arose in Indiana, the state legislature passed and sent to the governor there.
Glenda Ritz, Indiana鈥檚 superintendent of public instruction, told 澳门跑狗论坛 that the state might pull out of the Partnership for Assessment of Readiness for College and Careers, or PARCC, one of two consortia developing common-core tests.
She called the testing problems the state experienced last month 鈥渦nacceptable.鈥�
Wendy Y. Robinson, the superintendent of the 30,900-student Fort Wayne, Ind., community schools, said she doesn鈥檛 know how students, parents, or educators can now have confidence using online testing for the common core.
鈥淭eacher pay, school evaluations, student grades 鈥� are all going to be tied to a system that none of us have any faith in anymore,鈥� she said.
In Oklahoma, lawmakers such as Rep. Curtis McDaniel, a Democrat, called for a moratorium on online testing. 鈥淚f we can鈥檛 get this little piece of the puzzle working in the right direction, how are we going to get it right for the whole state or the country?鈥� he said.
鈥淐ommon core has some good values,鈥� he said, 鈥渂ut we need to re-evaluate what we鈥檙e doing.鈥�
But Chad Colby, a spokesman for PARCC, said despite the problems, the advantages of computer-based testing remain, especially when it comes to evaluating student knowledge, offering more interactive testing, and maintaining test security.
He acknowledged, however, that the tests must be reliable.
鈥淭he benefits of computer-based assessments for students and teachers vastly outweigh the growing pains and issues in a few states,鈥� he said. PARCC will work to solve any technical problems before the common-core online tests are rolled out, he added.
Anticipating Glitches
Joe Willhoft, the executive director of Smarter Balanced Assessment Consortium, the other group developing common-core tests, said a number of states in his consortium have already been using online testing with all their students for years without major incidents. He said he was confident any kinks could be worked out before common-core tests were launched.
But educators around the country remain concerned about their technical preparedness for common-core online testing, said Keith R. Krueger, the CEO of the Washington-based Consortium for School Networking, or CoSN.
A survey that CoSN released in March found that preparing for the online tests ranked second among the top priorities for the group鈥檚 members.
Though districts have primarily been worried about their own infrastructure and testing capacity, now they are realizing that even if they are well prepared, some problems are out of their control, Mr. Krueger said.
鈥淚 would think of this as the canary in the coal mine,鈥� he said of the recent testing problems. 鈥淭hese things are not easy to pull off on a statewide basis. We need to do it in a careful way and plan for the unexpected.鈥�
Mr. Cohen of the AIR, which is working with Smarter Balanced to deliver adaptive pilot tests that adjust the difficulty of questions based on how well a student is answering them, said his organization鈥檚 assessments are designed in a way that allows for glitches and gets students back on track when they occur.
鈥淭he tests need to be designed as online tests and not as paper tests,鈥� he said. 鈥淵ou recognize that the technology is going to fail somewhere, sometime, and you build the test to be robust.鈥�
Douglas Levin, the executive director of the State Educational Technology Directors Association, or SETDA, based in Glen Burnie, Md., added that there can be risks with paper tests, too, such as when floods washed out warehouses of tests in the past year, he said.
Even so, Mr. Levin鈥攍ike Ms. Ritz in Indiana鈥攕aid the recent problems with online testing were unacceptable.
While there鈥檚 been an emphasis on getting districts technologically ready to administer online testing for the common core, he said, 鈥渢here is a need for the assessment industry to ensure that it has the capacity to serve these larger numbers of kids with a quality of service that really has to be very high.鈥�
鈥楧oubt and Uncertainty鈥�
In the meantime, districts were just trying to deal with the logistical and emotional fallout from the online testing problems.
John Althardt, a spokesman for the 30,000-student Indianapolis public schools, said students in 50 buildings experienced testing disruptions, and the district was just focusing on getting through the testing cycle before thinking about how to proceed in the future.
鈥淪ome of our folks would say they鈥檙e ready to go back and use stone tablets at this point,鈥� Mr. Althardt said.
In Oklahoma, Superintendent Keith Ballard of the 42,000-student Tulsa public schools, said in a statement that the testing problems were 鈥渘othing short of disastrous,鈥� adding that the district would be forced to invalidate at least 460 tests.
Ms. Pemberton of the Oklahoma education department said all students will have an opportunity to retake the tests if they want to. Those who scored enough to receive a proficient grade on the tests do not need to retake them, even if they did not finish.
But the logistics of extending the testing window and retesting students are significant.
In the Oklahoma City system, for instance, one middle school had bused its students to a local university because the school lacked enough devices for students to take the online tests.
Education observers will be watching whether the spate of problems helps prepare both districts and testing companies for the online common-core tests.
Mr. Levin of SETDA said the testing problems will provide an additional argument for opponents of the common core.
鈥淭here are people who, for all sorts of reasons, are looking for ammunition to spread fear and doubt and uncertainty about the implementation of common core overall,鈥� Mr. Levin said. 鈥淚 wouldn鈥檛 be surprised if this ends up among the arrows in their quiver.鈥�
