Over the past two decades, the role of data in international development in higher education has expanded rapidly. Policymakers, implementers, funders, and researchers are increasingly using data to make important decisions around resources and priorities. This coincides with the shift from a focus on outputs, such as Millennium Development Goal 2 of universal education attendance, to outcomes, such as Sustainable Development Goal (SDG) 4’s emphasis on the importance of learning. This focus on outcomes, coupled with increases in data literacy, is bringing a renewed focus to demonstrating and measuring the achievement of positive outcomes for the beneficiaries of social programs around the world
At the Center for Universal Education (CUE), we have identified four types of data necessary for achieving outcomes, including data on the cost of action and inaction, final results data, and data along the way—or real-time performance data. If we are to achieve meaningful progress toward SDG 4 on education, real-time data are essential, and we must build systems of continuous measurement to ensure systematic collection, scrutiny, and usage of data.
The randomized controlled trial (RCT) movement, which gained momentum in the early 2000s, pushed for an increase in rigorous final results data for understanding what works (and doesn’t), but RCTs often tell very little about why. In the international development community, there is a widespread and growing recognition that end-term evaluations are insufficient, and efforts are underway to raise the profile of ongoing monitoring and evaluation that can lead to course adjustment and improved outcomes through adaptive management.
In contrast to final evaluation data, real-time performance data are gathered and analyzed quickly for timely decisionmaking and adaptation. In multiyear government- or donor-funded programs or policies, meaningful results for beneficiary populations hinge on understanding what is and isn’t working—and for whom—throughout the cycle of implementation, rather than simply ex post after the program or policy cycle has ended.
While organizations have long collected program monitoring data, monitoring is largely focused on implementation fidelity and measuring program activities and outputs, rather than ongoing measurement of outcomes. While these data are important, they depend on critical assumptions underlying theories of change about the links between activities and outputs and final outcomes and results. Real-time performance data focuses on the ongoing tracking of program results for beneficiaries, in order to better inform adaptations to program inputs, activities, and outputs along the way.
WHY REAL-TIME DATA—HOW TO GET IT?
In the education sector, the availability of—and access to—real-time data benefits all stakeholders. Administrators (such as school or government leaders or program facilitators) can use information on performance to adaptively manage resources and tailor programs according to the needs of students and teachers. Similarly, teachers can use student data to ensure that they are teaching at the right level and to customize their instruction based on individual learning patterns and needs.
Collecting data in real time is inherently challenging. Paper-based systems of data collection historically used are slow, administratively burdensome, and prone to human error. Digital technology offers more efficient collection and analysis of real-time data, allowing for more flexibility and customizability, as well as functionalities such as automatically generated visualizations and ongoing recommendations. This complements the successful use of digital tools in other areas of education—including recent booms in ed-tech-based instruction and electronic data collection for formal program evaluations—both figuratively and literally, through integrated data systems.
Digital tools for real-time data collection can come in different forms—whether they be education-specific data tools, adaptive learning tools, or survey tools for data collection. For example, the digital platform Tangerine, developed by RTI International, includes a number of different functionalities, including applications for student assessment, as well as Tangerine Coach for classroom observation and continuous teacher professional development. Promise3, a tool being developed by Vera Solutions, will track indicators such as educational access, quality, and equity/equality.
At CUE, we are seeking to better understand how digital tools for real-time data are contributing to data-informed decisionmaking in education in low- and middle-income countries. We have developed and launched a survey to learn more about the development and functionalities of such tools, such as their usability and adaptability, data collection and analysis processes, and key stakeholders. We look forward to sharing our findings as we analyze the different aspects of these tools.
If you are interested in learning more about our research or work with a real-time data collection tool that would be valuable for this research, please reach out.