Education leaders in North Carolina鈥檚 Charlotte-Mecklenburg school district are scrutinizing the habits and grades of elementary school students to determine who may fall off track and fail to graduate from high school a decade or more from now.
They don鈥檛 need a crystal ball to make predictions. Officials in the 141,000-student district are relying on a 鈥渞isk-factor scorecard鈥� to help them spot children who are in jeopardy of becoming dropouts and then deploy resources to help them change course.
Using high-tech data analytics to examine grades, attendance, course failures, declines in grade point average, and disciplinary incidents, Charlotte-Mecklenburg鈥檚 scorecard system, which was put in place during the 2010-11 school year, predicts even after the first few months of kindergarten which students are at risk.
District leaders, principals, and classroom teachers are using the information to make decisions about how to deploy resources all across the district.
鈥淭his information is very powerful,鈥� says Scott Muri, the district鈥檚 chief information officer. 鈥淭his helps to inform our decisionmaking process about children, budget processes, and human resources. Decisions at every level can be impacted by this.鈥�
From the moment children enter kindergarten, school districts begin collecting information about them. And for years, many districts have tried to build data systems that organize that information and make sense of it. For some districts鈥攁nd some states鈥攖hose systems are finally mature enough to look into the future, by using complex data analytics to predict which indicators mean students may go off track down the line.
Some districts use such predictive analysis as an early-warning system for who is at risk of failing to graduate. Others view it through the lens of higher education to determine which students are unlikely to be college-ready by graduation.
Sixteen states now produce early-warning systems that flag students not on track to graduate from high school and relay that information to districts, and 18 others have plans to institute similar systems, according to a November 2011 study by Civic Enterprises, a Washington-based policy firm, and the Everyone Graduates Center at Johns Hopkins University, in Baltimore.
鈥淪chools are kind of on overload when it comes to collecting data and talking about data,鈥� says Mindee O鈥機ummings, a senior research analyst for the Washington-based American Institutes for Research and co-team leader for the National High School Center鈥檚 early warning system, which is working with several states and districts to implement the center鈥檚 free early-warning system for high school students and is also developing one for middle school students.
鈥淏ut when they can really apply that knowledge to make a difference,鈥� she says, 鈥淚 see a kind of rejuvenation of energy around using data.鈥�
鈥榃ork Smarter, Not Harder鈥�
Many of the predictive models start with 9th graders, but others, like the one used in the Charlotte-Mecklenburg district, start in the earliest grades, says Leora Itzhaki, an academic facilitator for the district鈥檚 Elizabeth Lane Elementary School.
1. Determine whether your state already has an 鈥渆arly warning system鈥� for using data to predict future student performance. Make use of that system if one is available.
2. Consult research-based risk-factor criteria shown to accurately predict whether a student is on track to graduate. Examples include the criteria available from the free early-warning system at the National High School Center.
3. Make sure teachers and school leaders get predictive data on a regular basis, understand how to interpret the information, and then use it to come up with intervention strategies.
4. If you鈥檙e already using a predictive-data system for high school students, determine whether such a program can be expanded to middle and elementary schools.
Itzhaki says the risk-factor scorecard helps teachers take the initiative in preventing students from falling further behind. When a new student comes into the district, for example, the system will automatically let the teacher know how many times the student has transferred schools, or if the student鈥檚 family doesn鈥檛 speak English at home, or if the child is much younger or older than peers in the same grade鈥攁ll deemed risk factors by the district data system.
鈥淭he scorecard puts that data out there with the click of a button and makes it really clear,鈥� Itzhaki says. 鈥淎 lot of this is common sense, but having it grouped together helps teachers work smarter, not harder. You don鈥檛 have to dig in a folder for all of these bits of information.鈥�
A strong set of existing research has defined many risk factors for students. For example, researchers say looking closely at credits earned in 9th grade and course grades can accurately predict whether a student is on track to graduate. Absences during the first year of high school are also a criterion that many such systems use.
While many districts use such already-identified factors to build their predictive data systems, others add to them or craft their own.
In the 144,000-student Montgomery County, Md., school district, officials have been using predictive data analysis since 2009, when the district rolled out its 鈥淪even Keys to College Readiness.鈥� The 鈥渒eys鈥� the district has identified include meeting reading targets in kindergarten and 2nd grade, doing 6th grade math in 5th grade, and having a C or higher in Algebra 1 by 8th grade and an Advanced Placement exam score of 3 or higher by 12th grade.
The keys were developed, says Adrian Talley, the associate superintendent for the office of shared accountability, by looking at district students who were college graduates and then searching back through their district histories for common factors.
Defending Decisions
Patte Barth, the director of the Center for Public Education at the National School Boards Association, in Alexandria, Va., who is working on predictive data analysis through a grant from the Seattle-based Bill & Melinda Gates Foundation, says school boards have reported that the information gleaned from such analysis 鈥渢akes the stress out of decisionmaking.鈥�
鈥淭he power in this data is that it makes it much easier to defend decisions and give confidence that districts will get a return on their investment,鈥� Barth says. 鈥淚t helps them identify where the needs are and to align the resources to those needs.鈥�
That鈥檚 what happened in the 6,700-student Washington Local schools, based in Toledo, Ohio, which is adopting the National High School Center鈥檚 early-warning tool. The district collected risk-factor information from 8th graders and then gathered additional data on grades and attendance for the same students as they started 9th grade.
The system flagged students who had missed three or more days in the first 30 days of school. Teachers who worked with those students were sent a script designed to help them hit specific talking points when approaching students about concerns over their absences.
鈥淚t was to let them know that we were paying attention,鈥� says James Nino, the special education co-chair at Washington Local鈥檚 Whitmer High School, who is helping to oversee the project on high school risk factors there.
In addition, Whitmer High is starting a mentoring program focused on students the early-warning system has highlighted as being at risk, Nino says. It was helpful to know exactly which students to concentrate on, he says.
鈥淲e鈥檙e getting a better idea of who we need to service, and trying to make sure we鈥檙e not jumping into a mentoring program that is very resource-intensive for all students, when that may not be what every kid needs,鈥� he says.
But it鈥檚 important not to stop there, says Marcy Lauck, the manager of continuous improvement for California鈥檚 32,000-student San Jose Unified district, which is close to launching a particularly detailed predictor model that takes into account the more typical factors such as test scores and attendance, but also looks at students鈥� physical-fitness levels, other health issues, and socioeconomic standing.
Lauck says districts working to predict how students will do must also collect data on the interventions used to move those students back on track.
鈥淭he biggest challenge is to understand which interventions are successful for which kids and to collect data on that, too,鈥� she says. 鈥淲e鈥檙e really looking at how to capture that data and quantify it to let us know if we鈥檙e being successful.鈥�
