Although Carnegie Learning鈥檚 is the only blended learning curriculum Mary Brierley has ever taught, she trusts its quality.
鈥淚t鈥檚 made me change the way I question students, and it鈥檚 improved the way that we teach the course,鈥� said Ms. Brierley, who teaches two block sections of Algebra 1 and one of Algebra 2 at Severna Park High School, located about a half-hour south of Baltimore. 鈥淚 don鈥檛 know how, it just seems to force you to ask the question that makes the student think about how to get to the answer.鈥�
In many respects, the curriculum used in the Maryland teacher鈥檚 classroom bears similarities to many blended learning programs, instructional approaches that combine technology-based and traditional classroom lessons.
While testimony such as Ms. Brierley鈥檚 has been enough to persuade hundreds of districts to use some form of the Cognitive Tutor program since the late 1990s, the curriculum is unusual in that it also now has the backing of independent research, stemming from a $6 million study financed by the U.S. Department of Education carried out by the Santa Monica, Calif.-based
鈥攚hich included 25,000 students across middle and high schools鈥攆ound that in the second year of the curriculum鈥檚 implementation, high school students using the curriculum鈥檚 combination of self-paced software and class-paced textbooks made statistically significant additional learning gains compared with students using a traditional curriculum. Middle school students were also found to make gains in the second year, though not to a statistically significant degree.
What is less clear is why the program works, which of its components are most effective, and whether its apparent success is a greater victory for the Pittsburgh-based Carnegie Learning鈥檚 corporate future, or the entire field of blended learning, which has seldom had a chance to prove itself in research of this magnitude.
鈥淚 do think that it has value in the blended learning community as [a measure of] proof that a strategy that incorporates technology in a blended fashion produces positive results,鈥� said John Pane, a senior scientist at RAND and the study鈥檚 lead author. 鈥淏ut generalizing from that to other blended learning curricula is a risky move. And while many people may be tempted to do that, the devil is in the details of any blended learning curriculum.鈥�
Software-Textbook Connections
When it comes to Cognitive Tutor, those details are not all that revolutionary, at least within the scope of the field of blended learning, which has gained most of its traction in the past half-dozen years.
Ms. Brierley鈥檚 Algebra 1 classroom, and many others that use the program, functions squarely within the commonly used 鈥渟tation rotation鈥� blended learning model, which is seen more often in the elementary and middle grades.
After a brief pencil-and-paper warm-up, her second-period class divides into two groups of about a dozen students each. One group of students turns to a problem from a textbook, with clusters of students working together at desks, while members of the other group migrate to the laptop cart in the classroom鈥檚 corner, take a device back to their desk, log in to their Cognitive Tutor software accounts, and tackle problems tailored to each student鈥檚 learning progress. After 35 minutes or so, the groups switch tasks.
鈥淚t does free [teachers] up to be more of a troubleshooter than anything,鈥� said Ms. Brierley, an 18-year teaching veteran who has spent the last third of her career working with Cognitive Tutor. 鈥淚t gives [students] an opportunity to be independent and work through things and sometimes work things out in their head without us telling them what they should be doing.鈥�
A study found that the math scores of high school students using Cognitive Tutor improved significantly during the second year that schools implemented the program. That growth was equivalent to moving a student from the 50th percentile to the 58th percentile of performance.
Source: RAND Corp.
But Cognitive Tutor has some notable nuances for a station-rotation model. Among them, both the print text and the software come from the same provider. So while some students may reach concepts in print first, and others first encounter them online, the terminology and theory behind teaching concepts remains constant.
Both branches of the curriculum also stress the manipulation of numbers and variables. The text features perforated tearaway pages so students scribble in or alongside charts and equations rather than on separate scrap paper. (This also means a district implementing the curriculum has the added expense of purchasing new textbooks every year.) The software requires students to set their own bounds for graphs and tables and type key information from paragraph-length word problems into charts before answering a series of questions all based on the same scenario.
Perhaps most importantly, the curriculum has undergone an endless evolution since its launch as Carnegie Learning鈥檚 first product in 1998 and is modeled around research on human cognition conducted by Carnegie Mellon University, in Pittsburgh, which at one time owned a stake in the company.
鈥淚t is built on a very well-considered theory of cognition and has been assembled over decades into what it is now,鈥� Mr. Pane said. 鈥淭hat鈥檚 quite in contrast with the great vast set of education technology/blended learning interventions that are out there that are ad hoc.鈥�
Academic Improvements
It鈥檚 worth noting that RAND restricted Carnegie Learning鈥檚 employees in their interaction with schools in the study, to ensure schools did not receive an unusual level of assistance in implementing the curriculum. In other words, the study suggests those second-year results stem from a range of real-world conditions, where some schools and some teachers are implementing the program in more effective ways than others.
That鈥檚 essential in a district like the 78,000-student Anne Arundel County, where every Algebra 1 classroom has used Cognitive Tutor since 2009, but not every teacher is as enamored with the program as Ms. Brierley.
While the expectation is for students to spend 30 to 40 minutes of class time twice a week working on the curriculum鈥檚 computer component, some teachers barely meet the minimum time requirement, said Amy Smith, the district鈥檚 coordinator of secondary mathematics. Others, meanwhile, allow more class time and encourage students to use the program during study halls or after school, she said.
Teachers鈥� responses range from 鈥渆xcellent fidelity,鈥� Ms. Smith said, to 鈥溾�嗏�業鈥檓 checking the box鈥攜ep, I put them on鈥欌�嗏�� the computer.
Inevitably, challenges related to classroom management or instructional-time allotment鈥攅specially in shorter-length middle school periods鈥攆orce teachers to lock up laptop carts more than they鈥檇 like.
鈥淚t鈥檚 not perfect by any means,鈥� said Ms. Smith.
Still, she insists Cognitive Tutor Algebra 1, first used as a supplemental tool by the district, is driving recent improvements on the Maryland High School Assessment for that subject. Between 90 percent and 93 percent of the county鈥檚 high school students have passed the graduation-requirement exam during each of the past six years, and during the past five, the proportion scoring 鈥渁dvanced鈥� has risen steadily from 34.2 percent in 2009 to 41.6 percent in 2013. (No Anne Arundel County schools were included in RAND鈥檚 research.)
Paper vs. Computer
Julia Freeland, an education fellow at the Cambridge, Mass.-based Clayton Christensen Institute, which has led the effort to catalog and categorize blended learning, agrees that results both from the RAND study and real-world settings, such as in Anne Arundel County, are encouraging. But she said the field is still lacking granular research that shows the reasons that programs like Cognitive Tutor are effective鈥攔esearch that could help educators decide whether implementing the program is worth the investment.
Although the federal Education Department paid for the sizeable study, it may be more difficult to find a funder for follow-up work that examines those kinds of questions.
Given that reality, John Watson, the founder of Evergreen Education Group, a Colorado-based consulting organization that tracks online and blended learning trends, said he hopes the study is considered more broadly than just in terms of Carnegie Learning鈥檚 credibility.
鈥淭here are literally dozens of [blended] math programs,鈥� he noted. 鈥淭here鈥檚 no way that we鈥檙e going to have $6 million studies done about dozens of math programs individually. And so, while this is certainly a good thing for Carnegie and it鈥檚 a good thing broadly for the field, I want to make sure that it鈥檚 seen as part of the validation of the entire field as much, or more than being seen as suggesting a particular specific product.鈥�
Ms. Smith is confident that the program is helping the 2,700 Algebra 1 students across the Anne Arundel County school system. Yet even six years after it was introduced, there are always new challenges, such as the first year of implementing the Common Core State Standards, which she fears could drag down achievement results.
Perhaps sensing that uneasy transition, Ms. Brierley鈥檚 students鈥攎ost new to blended learning鈥攁ppear mixed on whether they prefer doing algebra on a screen or in a book.
Celeste Davis, a freshman in Ms. Brierley鈥檚 first-period class, actually would like to see less of the laptops.
鈥淚 like working out on the paper and being able to figure the problems out,鈥� she said. 鈥淚 think the computer just doesn鈥檛 explain it the way that I read it.鈥�
Antony Rutherford, a freshman in Ms. Brierley鈥檚 second-period class, said doing the work on a computer screen can be a bit more cumbersome at times.
鈥淚鈥檓 kind of an all-around person,鈥� he said. 鈥淚 do it a little bit faster on paper. But on the computer, it鈥檚 fine.鈥�
