Educational technology companies trying to do business with schools often have only a vague sense of district officials鈥� specific buying needs, according to a new study that delves into vendors鈥� and K-12 leaders鈥� frustrations with the procurement process.
The study, based on surveys and interviews of district leaders and ed-tech company officials, finds that many business officials regard the district buying process as cumbersome and confusing.
Districts administrators, for their part, say they鈥檙e often overwhelmed by the vast number of ed-tech products peddled to them, which they lack the time and resources to evaluate thoroughly, the study says.
The overall findings are likely to resonate for many executives in the K-12 industry who have long regarded the K-12 procurement as bureaucratic and particularly unwelcoming to startup companies with potentially cutting-edge products and ideas.
But the report also provides details on which aspects of the buying process cause the most difficulty for vendors and K-12 administrators鈥攁nd it offers suggestions for how the process could be improved to help buyers and sellers, and, presumably, students and schools.
The study found that ed-tech vendors, on the whole, are much more frustrated by the procurement process than are K-12 officials such as superintendents, curriculum directors, chief technology officers, and school business officials.
There鈥檚 a price to be paid when companies lose faith in districts鈥� procurement processes, said Karen Cator, the president and CEO of Digital Promise, a Washington- and California-based nonprofit that was one of the organizations to sponsor the procurement study.
She said entrepreneurs and others offering digital products and tools that could help schools may end up avoiding K-12 systems, and instead look to markets they regard as easier to work in and more lucrative.
鈥淓ducation is not necessarily the market they鈥檙e going to jump into,鈥� said Ms. Cator, a former director of the U.S. Department of Education鈥檚 office of educational technology. 鈥淭he consequence is a lessening of innovation and new ideas.鈥�
Vague Priorities
The study was conducted by researchers at Johns Hopkins University, and was commissioned by Digital Promise, a congressionally authorized organization that focuses on improving education through technology and research, and the Education Industry Association, a Vienna, Va.-based group with more than 200 members, including small and large technology companies. Both Digital Promise and the EIA helped design the study.
The university researchers used a mixed-methods approach, based on surveys of nearly 300 district officials who have some connection to procurement, including superintendents, curriculum directors, business officials, technology directors, and principals. The researchers supplemented that data with interviews with survey participants.
The survey offers myriad examples of shortcomings that vendors and school officials see in the procurement process. Among the findings:
鈥� A majority of ed-tech vendors, 66 percent, were dissatisfied or very dissatisfied with district procurement procedures. By contrast, most district officials鈥攂etween 60 and nearly 80 percent, depending on their job titles鈥攕aid they were satisfied or very satisfied with the buying process, the survey found. (Superintendents were more likely to be content with the procurement process than other K-12 administrators, such as school business officers and principals.)
鈥� Nearly all districts acquire ed-tech products based on some kind of 鈥渘eeds assessment.鈥� But those assessments are often informal and don鈥檛 provide vendors with information on the specific educational challenges that districts hope to conquer.
鈥� Districts struggle with 鈥渄iscovery,鈥� or trying to find the best ed-tech products to meet their needs within the crowded and competitive landscape of companies touting their goods. As a result, school officials tend to regard the buying of tools and systems as a 鈥渉it-or-miss鈥� enterprise.
鈥� Similarly, many companies, especially less-proven ones, report that they don鈥檛 have the capacity to market their products to large numbers of districts, and so their ed-tech goods remain unknown to many potential K-12 buyers.
鈥� District officials see 鈥渃ooperative purchasing鈥� with other K-12 systems as an appealing but often unrealistic strategy, because they regard districts as having specialized needs.
鈥� Superintendents and other central-office administrators see decentralized, or school-based purchasing, as undesirable, because it leads to a loss of quality control and fragmented instructional practices. School principals, as one might expect, are much warmer to the concept.
鈥� With limited, reliable information on what to buy, district officials tend to rely on recommendations from peers in other districts. They also rely on pilot studies鈥攖hough such pilot studies tend to be conducted using very different standards, which may not provide useful information about the strengths and shortcomings of products, the study found.
鈥� Principals and teachers are typically only moderately involved in district needs assessments in determining which educational technologies to buy, despite those employees鈥� key roles in making digital approaches work in classrooms.
The study was funded by the Bill and Melinda Gates Foundation, which also provides support for 澳门跑狗论坛鈥榮 coverage college- and career-ready standards. 澳门跑狗论坛 retains sole editorial control over the content of this coverage.
As it now stands, many district needs assessments are often so vague that developers are often unclear if their ed-tech product are what those districts covet, said Steven M. Ross, a professor of educational psychology and a senior research scientist at Johns Hopkins, who is a co-author of the study.
For instance, in requests for proposals and other public statements districts will lay out their broad goals for increasing student achievement in math or language arts without laying out what types of ed-tech interventions or instructional strategies most appeal to them, Mr. Ross said.
Simply saying 鈥溾�榳e need a math program for high school,鈥欌�� without providing other relevant details, he observed, is 鈥渟ort of like saying, 鈥榃e need a car with four doors.鈥欌��
Teachers鈥� lack of involvement in determining districts鈥� ed-tech needs is especially troubling, Mr. Ross said. The complex nature of many ed-tech products, and the uncertainty about how students will respond to them, means they need to be vetted with educators in ways that textbooks and other printed materials do not, he said.
Mr. Ross was not surprised by districts鈥� reliance on pilot tests of ed-tech products. But he pointed out that pilot research often means different things to different districts.
In some districts, a pilot is treated as more of an informal 鈥渄emo鈥� to get teachers and students exposed to the tech tool, he noted. A better approach is to have groups of teachers testing out ed-tech tools under 鈥渞eal conditions鈥� and 鈥渁ctively engaging with products,鈥� Mr. Ross said, yet 鈥渧ery few districts are doing that.鈥�
Marketplace 鈥楴oise鈥�
Steven Pines, the executive director of the Education Industry Association, said the study results echo the complaints he often hears from vendors, many of whom struggle to make it in the K-12 market and tell him they feel 鈥渋nvisible,鈥� with little hope that districts will ever become aware of their products.
The survey made it clear districts are flummoxed by several parts of the procurement process, too.
Both sides believe 鈥渢here鈥檚 noise in the marketplace, and that they have difficulty navigating through that clutter,鈥� Mr. Pines said.
John Musso, the executive director of the Association of School Business Officials International, in Reston, Va., said the study offered valuable perspective on school and company officials鈥� views of procurement.
But he was skeptical of the notion that large numbers of promising ed-tech startups are being shut out of the school market. Entrepreneurs are securing a foothold in K-12 systems all the time, he pointed out. The problem is that many companies don鈥檛 deliver on marketing promises, which makes K-12 leaders reluctant to test new ones.
鈥淪chools have a really hard time distinguishing the good, the bad, and the ugly,鈥� Mr. Musso said.
The study didn鈥檛 account for other factors that can disrupt K-12 procurement of educational technology, he added, such as the turnover in district leadership. 鈥淲hen you get a superintendent forced out because of low test scores, clearly the new person will come in with different [priorities]鈥� in buying, he said.
Mr. Musso agreed with the study鈥檚 conclusion that vendors could benefit from getting better information about districts鈥� specific buying needs.
But he said districts face very different challenges in gauging their needs鈥攁nd they evaluate their needs in different ways, as a result. It鈥檚 typically much easier for smaller districts to get a sense of teachers鈥� and schools鈥� ed-tech demands than it is for larger school systems, he said.
鈥淭here鈥檚 a difference between the vendor鈥檚 view of helpful and the district鈥檚 view of helpful,鈥� Mr. Musso said.
The study鈥檚 authors put forward a number of recommendations for improving the buying process. Some of that advice is meant to increase the flow of information about ed-tech products and districts鈥� demands.
One recommendation is to build an online 鈥淓d Tech Product Information Exchange鈥� that would list and describe digital tools, report research on their effectiveness and consumer reaction to them, and allow for sharing of that information across K-12 districts. Mr. Musso said he liked that idea鈥攁s long as that online resource was feeding K-12 officials independent, reliable information, not merely spin provided by companies.
Another idea is to establish guidelines for conducting assessments of district needs, for staging pilot tests of products that can help better align tests to the real needs of teachers, produce reliable information on school officials鈥� satisfaction and student outcomes, and allow for the analyzing of data for product evaluation and development.
Pilots should be a tool 鈥渇or discovery and evaluation,鈥� the authors say. When they are structured and rigorous, pilots 鈥済enerate evidence [about digital tools] that is not only useful locally, but also to other districts.鈥�
