Back in 2009, Joel Rose and his team used spreadsheets to manually produce customized 鈥減laylists鈥� for 70 New York City middle school students each day. The goal was to always give each child the best possible content, delivered in the optimal manner, at the best possible time.
Figuring out how to make that happen took 11 hours every night.
Now, Rose鈥檚 New York-based nonprofit organization, , performs those same functions each day for 11,000 students in 38 district, charter, and independent schools spread across 10 states and the District of Columbia.
Thanks to complex algorithms, the process is now complete by 4:30 p.m. each day.
鈥淭he basic concept of using data to match each student with the best possible lesson every day has remained consistent,鈥� Rose, a former middle school math teacher, said. 鈥淏ut on the tech side, we鈥檝e made quantum leaps.鈥�
Belief in the potential of 鈥渃urriculum playlists鈥� is central to the K-12 personalized-learning movement. The premise is that in any given math classroom, for example, some students will need help with basic arithmetic, while others should be practicing solving equations. And some students might need to work with a teacher, while others might be better off practicing online. Even the best teachers can鈥檛 consistently differentiate at that level for 30 students every single day. Technology can help.
The concept has led to both new opportunities and new challenges for schools.
Scores of publishers and educational technology companies now purport to use algorithms to deliver personalized content to students. As a result, educators looking for new digital classroom resources must not only sift though a growing menu of curricular options, but also determine how customizable each lesson truly is and how well vendors actually match students with the right material.
The stakes are even higher with groups such as New Classrooms, which does not operate its own schools, but sells an instructional model known as Teach to One. Along with California charter network Summit Public Schools and the Bay Area startup AltSchool, New Classrooms takes a more comprehensive approach to curriculum playlists, treating them as a catalyst for rethinking the basic organization of school.
The Teach to One model has garnered considerable attention, including plaudits from Microsoft founder and philanthropist Bill Gates and the U.S. Department of Education.
But some have found that implementing a playlist-based instructional model is harder鈥攁nd more expensive鈥攖han it looks: On top of facilities renovations and technology upgrades, yearly software licenses cost $225 per student, and schools typically pay New Classrooms between $90,000 and $160,000 per year for other supports. More than a quarter of the 53 schools that have started with Teach to One: Math are no longer using the program.
Research on Teach to One鈥檚 impact, meanwhile, is mixed. One major study found no significant effects on achievement, with many students saying they felt they learned more when they worked directly with a teacher in a more traditional manner.
That combination of hype and uncertainty has led some observers to voice concern that algorithm-driven playlists are just another technology that K-12 schools are embracing without adequate scrutiny or regard for possible unintended consequences.
鈥淧eople don鈥檛 really demand evidence when technology and algorithms are involved, because they鈥檙e bewildering,鈥� said Cathy O鈥橬eil, a data scientist and activist and the author of Weapons of Math Destruction.
鈥淏ut you don鈥檛 need to understand what鈥檚 in the black box to know if something works,鈥� O鈥橬eil said.
Some innovators want to use playlists to link young people to their passions, to each other, and to opportunities to make an impact in the world.
Curriculum playlists are often thought of as algorithm-driven systems for matching students with content and learning experiences that can best help them master prescribed learning standards as efficiently as possible.
But proponents of the 鈥渃onnected learning鈥� movement are taking playlists in a different direction: They want to connect young people to their passions, to each other, and to opportunities to make a real impact on the world.
鈥淢y concern about the current approach to algorithms and personalized learning in schools is that we鈥檙e still so tied to traditional notions of curriculum and what content needs to be taught,鈥� said Constance Yowell, the CEO of , a nonprofit organization spun off by the MacArthur Foundation in 2015.
鈥淚f you put four healthy foods in front of a young person, but they hate all of them, which one they choose is not all that interesting,鈥� Yowell said.
Collective Shift鈥檚 biggest project is LRNG, a platform intended to provide students with tailored opportunities to pursue their own interests through a wide variety of often-informal learning experiences, ranging from museum trips to wearable-technology projects to online coding tutorials.
It鈥檚 still early days for the project. Some of the challenges Yowell and her team are wrestling with:
Who gets to design the playlists?
Right now, the answer is mostly adults and experts at various groups that LRNG partners with, including a new set of grantees financed to develop playlists on such topics as digital literacy and career preparation. But eventually, Yowell said, the goal is for young people to be able to create and share their own playlists.
How do you make playlists matter?
The connected-learning movement is heavily focused on out-of-school and informal-learning opportunities. Proponents are generally not interested in measuring effectiveness with standardized-test results. So what exactly do young people get out of completing an LRNG playlist? Right now, the answer is mostly digital badges, with some additional real-world opportunities, such as an internship with a local partner organization, sprinkled in.
How do you get young people to access and use the playlist platform?
Currently, it happens mostly through mentors, such as teachers or program staff at a local Boys & Girls Club. A mobile app is on the horizon. But even after LRNG manages to draw young people in, questions remain. One biggie: how to strike the right balance between recommending learning experiences to users versus letting them discover such experiences for themselves?
LRNG is still in beta, and there鈥檚 a long way to go.
But Yowell said the vision is clear.
鈥淲e want LRNG to be an infrastructure that enables in-school and out-of-school learning to work together,鈥� she said.
Remaking the Classroom
Peek into a 6th grade math classroom at Nathan Hale Middle School in Norwalk, Conn., and you might see as many as 67 children in a single room, each pursuing his or her own Teach to One playlist.
Some might be conducting a 鈥渓ive investigation,鈥� working with a teacher to solve a problem. Many will be using Chromebooks, working independently through exercises that algorithms have selected specifically for them. Part of the room will look almost like a traditional classroom, with another teacher leading students through a group lesson.
鈥淲hen I first heard about Teach to One, I was taken aback. I didn鈥檛 see how it could work,鈥� said Albert Sackey, the school鈥檚 third-year principal. 鈥淏ut when I saw it in action, I thought it was incredible.鈥�
Undoubtedly, switching to a playlist-driven instructional model has been a major lift at Hale. Sheetrock had to be torn down, so that multiple classrooms could be converted into one large learning space. The school鈥檚 wifi network had to be upgraded. Sackey and his team had to rework Hale鈥檚 entire bell schedule鈥攖eachers in the Teach to One program can鈥檛 plan ahead, because each day鈥檚 lessons are driven by algorithmic recommendations created the previous evening.
Still, eight months into the transition, Sackey said the upheaval has been worth it.
鈥淕ood teaching is always good teaching,鈥� he said. 鈥淭each to One is just a tool that helps teachers be more targeted.鈥�
Personalizing Learning
Here鈥檚 how the technology works.
At the beginning of the school year, each student in the New Classrooms network takes a series of diagnostic assessments. Based on the results, and with the help of technology, the group then maps out the specific skills and concepts that each student needs to learn over the year. For 2016-17, that meant 11,000 different skills maps for 11,000 different students.
From there, New Classrooms鈥� algorithms begin identifying the optimum learning sequence for each student, so that the skills he or she encounters in the playlist will build on each other in a natural, coherent way.
Then the algorithms start identifying the specific lessons or curricular content that will ostensibly best help each student master the next skill they encounter. New Classrooms staff members have vetted and approved roughly 9,000 lessons from more than two-dozen content partners.
During extended class periods that last 80 to 90 minutes, students work through the material their playlist serves up to them. Three students sitting side by side in the 鈥渋ndependent learning zone鈥� might work on three entirely different skills, using three different problems, from three separate vendors.
At the end of each day, each student completes a short quiz known as an 鈥渆xit ticket.鈥� The results go to New Classrooms, where the algorithms go to work.
What skills did students master? Are they struggling with solving equations because they don鈥檛 understand variables or because they don鈥檛 know how to multiply? Would the gaps in each student鈥檚 understanding be best addressed via a live investigation or with some solo practice using the free online math website Khan Academy? What do each student鈥檚 past experiences say about what type of content will be most engaging now?
After figuring out what each student needs, the algorithms then work to figure out what鈥檚 most feasible for the whole classroom. Maybe there鈥檚 not a critical mass of students who need help with dividing fractions, so a small-group lesson isn鈥檛 an option.
The algorithms鈥� final daily output consists of a 鈥減review schedule鈥� that fits all the pieces of the puzzle together. New Classrooms鈥� human scheduling team reviews those algorithmic recommendations, makes tweaks as necessary, and sends each school suggestions on how its classrooms should be organized the following day.
Teachers on the ground get final say鈥攚hat if the algorithms want to put two feuding students into the same small group?
When students walk back into the classroom the next day, their playlists have already been updated, and large flat-screen TVs point them to the section of the room where they鈥檒l spend the period.
Algorithmic Bias?
Teach to One has come a long way, but there鈥檚 still plenty of room to grow, said Rose, the New Classrooms co-founder and CEO.
On the technology side, other companies, such as New York City-based Knewton, are already leveraging the power of machine learning to help make their own playlist-creation systems smarter.
Logistical challenges also remain. When the District of Columbia school system tried Teach to One at one of its middle schools, for example, experienced teachers ended up feeling limited by the technology, because they wanted the freedom to plan more than one day ahead.
鈥淲hen the rubber met the road, there were just so many little practical challenges to making it work,鈥� said John Rice, the district鈥檚 director of educational technology, who helped make the decision to drop the program last year.
Then there are the growing number of questions about the role algorithms play in society more broadly.
In fields such as criminal justice, concerns have been raised about 鈥渁lgorithmic bias.鈥� Formulas used to help determine criminal sentences, for example, have been found to produce unfair outcomes for African-American defendants.
One fear for K-12 schools is that as the algorithms behind playlists become 鈥渟marter鈥� and more efficient, students with special needs鈥攕ay, dyslexia鈥攎ight find that their atypical mental processes are not reflected in the software that shapes their instruction.
鈥淎lgorithms are inherently optimized for some people,鈥� said O鈥橬eil, the author and activist.
For his part, Rose turns the question around: Students with learning disabilities are often poorly served in traditional classrooms, he said, and internal metrics show the 600 or so special-needs students in Teach to One classrooms have made significant academic growth.
But for the time being, at least, New Classrooms does not have any built-in checks to ensure that its technology is free of bias.
Figuring out how to ensure public trust by making that happen is just one of the hurdles the group will have to overcome if it wants to take its curriculum playlists from the margins to the mainstream.
鈥淥ur long-term vision is we鈥檇 love to see personalized learning be the way kids learn,鈥� Rose said. 鈥淎nd we鈥檇 like to be one of the models.鈥�
