An AI-powered tutoring assistant increased human tutors鈥� capacity to help students through math problems and improved students鈥� performance in math, according to .
The digital tool, Tutor CoPilot, was created by Stanford researchers to guide tutors, especially novices, in their interactions with students.
The study is the first randomized controlled trial to examine a human-AI partnership in live tutoring, according to the researchers. The study examines whether the tool is effective for improving tutors鈥� skills and students鈥� math learning.
It comes as tutoring has become a key learning-recovery tool. Schools, however, have run into challenges in scaling and sustaining tutoring programs because they require a lot of human tutors, time, and money.
In an interview with 澳门跑狗论坛, Susanna Loeb, an education professor at Stanford and one of the study鈥檚 authors, discussed the creation of the tool, the trial findings, and its implications for schools.
This interview has been edited for brevity and clarity.
How did Tutor CoPilot come about?
School districts have bought into tutoring and have worked to provide this kind of support for students. It鈥檚 not easy, because you have to fit it into the school schedule. It takes a lot of effort recruiting so many tutors.
One of the benefits of virtual tutoring is that you can draw tutors from a broader geographic area. Many of those tutors are novice tutors, and the good programs provide them training and materials, but they still have a learning curve.
Tutor CoPilot came out of this idea that maybe we can support tutors, not only with this upfront training and coaching every now and then, but with in-the-moment support.
How was Tutor CoPilot trained?
We partnered with FEV [a virtual tutoring company]. We did a first study using the data that they have to really understand the potential of generative AI for providing this information. We tried first with just the basic generative AI that鈥檚 available to all of us, things like ChatGPT. You can feed in the problem that the student is having: What is 10 minus 7? The student said 5. You can ask ChatGPT how an expert tutor would respond to the student鈥檚 mistake and it would just say 10 minus 7 is 3.
So we worked with a bunch of expert math teachers to understand the process they go through when they鈥檙e responding to students鈥� mathematical mistakes, trying to understand the reason for them, and thinking about different strategies. We tried a series of different prompts [for the chatbot]. One of them that worked quite well was, instead of asking how it would respond, we say, 鈥渞espond to this problem with this strategy.鈥� There were seven strategies that we had it respond to, and then the tutor had the option of choosing which strategy [to use].
What are the big takeaways from the results?
What we found was the students who were working with the [tutors] in the treatment group, who had the access to [Tutor CoPilot], did better. They were more likely to demonstrate mastery by 4 [percentage points]. Generally, about 62 percent of the time students demonstrate mastery, so this moved it up to 66 percent of the time that students were demonstrating mastery. That鈥檚 a strong effect.
What was more exciting was that if you looked at the heterogeneity, the effect is far stronger on the weaker [tutors]. [They experienced a 9 percentage point increase in students鈥� demonstrating mastery]. They started at a lower rate, so that鈥檚 a meaningful increase in the proportion of time students are demonstrating mastery. It was most effective for the novice teachers. So that鈥檚 where you really saw the positive effect.
Another finding that was exciting is the language difference. The tutors using Tutor CoPilot were more likely to do the good things, to prompt students to explain, to ask guiding questions, to focus on what students were doing really well, and much less likely to do the generic encouragement, but also significantly less likely to give away the solution or to answer the question directly.
What are the implications for schools and districts?
Tutors, educators, and many adults can really be helped by these small supports that happen in the moment. They can be much more helpful than big books and saying, 鈥淥h, just learn this,鈥� or a one-time class. The potential of this support that is light-touch but continuous is a very important thing to think about when we鈥檙e trying to help adults change behavior and to learn.
When people think about generative AI and its potential for improving education, they often go to generative AI tutors, where students sit in front of the computer, ask it a question, and it gives them an answer. That is one role for generative AI.
But there are lots of benefits of the relationships that are inherent in the kind of education that we do in tutoring and in classrooms. Those relationships are so important for engaging and motivating students, and generative AI can support those, as well鈥攖his is an example of that.
By providing generative AI to the educator instead of directly to the student, you may get some of the benefits of this immediate feedback while still maintaining the really important parts of the in-person, relationship-based educational approaches that we鈥檝e taken traditionally.
