What makes one intervention work in a school when another seemingly similar one falls flat?
Increasingly detailed computer models of student behavior and learning may help researchers avoid such setbacks by better pinpointing interventions before taking them to schools.
鈥淚n education research, I get a great idea, apply for funding, 鈥� then I spend a few months in schools taking time from students and teachers, and often find out it doesn鈥檛 work,鈥� said Richard L. Lamb, an assistant professor of science education and educational measurement at Washington State University in Pullman.
鈥淭hat鈥檚 great that we have that data,鈥� he said, 鈥渂ut it鈥檚 not the most efficient way to do [research and development].鈥�
Instead, Mr. Lamb and colleagues are working to pair education technology and neuroscience to mimic how students learn in a classroom and provide an additional means of testing and honing interventions.
A Washington State University project is developing a virtual student pool for researchers to test interventions by tracking real students performing cognitive tasks in video games.
Students play video games that include tasks shown to require specific cognitive functions.
鈫�
The artificial intelligence records what a student does and learns to approach a problem as the student would.
鈫�
The system can be used to simulate hundreds or thousands of students.
鈫�
Researchers can use the simulations to supplement school-based field testing of an intervention.
SOURCE: 澳门跑狗论坛
The Student Task and Cognition Model, or STAC-M, is an 鈥渁rtificial neural network,鈥� a type of artificial intelligence system that mimics human learning and pattern recognition.
The project so far has collected data on more than 2,000 high school students ages 14 to 18.
While the main group is nationally representative, Mr. Lamb said the researchers are also planning to sample specific student groups, such as students of different language backgrounds.
The Washington State University project is part of a trend of increasing use of online and live student-data collections during early phases of education research.
Kenneth R. Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon University, in Pittsburgh, who supervises a separate student-data-analysis initiative but who is not part of the Washington State University project, said education watchers are seeing a steady 鈥渢rickle鈥� of new projects creating more dynamic uses of education data.
He believes artificial learning networks will be a normal part of education research in a decade or so.
鈥淚 think that will increasingly be how we can and should evaluate new approaches to instruction,鈥� Mr. Koedinger said.
Teaching a Machine 鈥楤rain鈥�
For example, a model like STAC-M, which can simulate the effects of an intervention on 100,000 students, could supplement field-testing of a new program in real schools to produce a larger overall sample and strengthen the reliability of the study鈥檚 findings.
In the first STAC-M pilot, which is depicted in the 2014 study, Mr. Lamb and his colleagues asked 645 high school students from a Mid-Atlantic traditional public high school to play science-based video games in the lab.
Several of the game鈥檚 tasks required critical science reasoning and understanding of concepts like conservation of mass, volume, liquids, and numbers.
In previous studies using functional magnetic resonance imaging, the tasks had been associated with activation in specific parts of the brain involved with reasoning and executive function.
Fifteen hours of play created more than 450,000 data points.
鈥淜ids do lots of things in games, and with game technology, you can get lots of background: what they click on, what they answer, what they don鈥檛 answer, what tools they use,鈥� Mr. Lamb said.
Each student鈥檚 responses 鈥渢aught鈥� a part of the system to approach and solve problems in the way the student would.
Just as adaptive tutoring systems 鈥渓earn鈥� how to teach the next lesson from a student鈥檚 current responses, the STAC-M creates a population of virtual students who respond in the same way live students would.
Not only can it respond to changes in similar science tasks, but also to other problems that would use the same type of cognitive skills.
For Mr. Koedinger, the artificial-student system starts off learning much more slowly than real students do, but as it progresses, it tends to learn a bit faster than live students. In part, he thinks that might be because the artificial intelligence is 鈥渕uch more logical鈥� than a student. 鈥淎 human can consider and reconsider uncertainties鈥攖ake certain things as uncertain when they really are certain,鈥� Mr. Koedinger said. 鈥淭he [artificial] system takes some things for granted more than the human brain does.鈥�
But that鈥檚 where students help teach the system.
A computer learning how to do a science task won鈥檛 second-guess itself or forget what it previously learned because it is worried that someone else won鈥檛 think it is good at that particular subject鈥攂ut it can learn to replicate that slowdown in function when modeling hundreds of students over time.
鈥淭he data gets you to a place you wouldn鈥檛 get to otherwise; it鈥檚 already discovering elements of real student learning that have implications for educational design,鈥� Mr. Koedinger said.
Tweaking the Testing Pool
After collecting the data, the researchers were able to simulate a separate intervention of 100,000 students, varying the curriculum and students鈥� levels of initial background knowledge to test different ways of teaching critical thinking as either an independent skill or one that is taught as part of a specific subject.
Mr. Lamb and his colleagues suggested that pairing the system with future brain-imaging studies could help researchers pinpoint the ways in which students respond to very similar-seeming interventions differently at the neurological level.
鈥淚鈥檓 not saying we never have to go into the classroom again, but we can run the model to target before we go into the classroom,鈥� Mr. Lamb said.
Challenges in Systems
However, Cynthia Coburn, a professor of education and social policy at Northwestern University, in Evanston, Ill., and a veteran of long term field-testing in schools, warned that data simulations should never take the place of full-scale experimental trials in schools, because interventions often succeed or fail for reasons that have nothing to do with how students learn.
鈥淭he history of education research is littered with really wonderful lab experiments and interventions created outside the classroom that, for whatever reason, didn鈥檛 work in the classroom,鈥� Ms. Coburn said.
鈥淚 think, bottom line, what we鈥檙e trying to understand are phenomena of student learning, teacher learning,鈥� she added. 鈥淭hose processes fundamentally occur in schools, so you鈥檙e never going to get anywhere if you don鈥檛 look at them in schools.鈥�
Mr. Koedinger of Carnegie-Mellon agreed: Rather than artificial student models replacing live testing in schools, he said he envisions trained teachers and researchers in every school eventually using ongoing computer modeling and data analysis to set up natural experiments during the regular course of school operation.
鈥淭he false premise is that before we do an intervention, we have to be sure鈥� it works, Mr. Koedinger said, but, 鈥渢here are experiments going on in schools all the time鈥攙ariation in the textbooks, teachers, etcetera鈥攂ut there鈥檚 not monitoring of those choices, no observations to see on a regular basis whether these things work. That鈥檚 what we need.鈥�
