The 2030 Agenda for Sustainable development underscores the importance of meeting development goals and targets for all people, especially, “...reach first those who are furthest behind.” Vulnerable groups such as refugees, ultra-poor, migrants, persons with disabilities often have the lowest share of development benefits, and are most at risk from any economic, health, or climate change shocks. Yet, reliable and disaggregated data is rarely available to improve our understanding of service gaps, or to target interventions relevant to vulnerable populations.

Government officials are used to collecting and meticulously entering large volumes of administrative data for reporting. The rapid expansion of low-cost devices and internet has boosted digitization efforts of governments, equipping frontline workers with handheld devices for capturing data at the point of service delivery. Despite these measures, who gets excluded from public services is not systematically tracked. It is salient that the UN World Data Forum in October this year highlights “Leaving No one Behind” as one of the thematic areas. It is a timely reminder that digitization efforts ought to enable equitable benefits for all.

Digitization has increased timely access to administrative data, but not its use for correct identification and targeting of resources. For data to be useful, it is important that the right data is collected, is without errors, and is comprehensive and available for timely analysis. Efforts are needed to build trust in data and statistics, as data reliability is a necessary and integral component to data use.

Most digitization processes lack systematic measures to ensure comprehensiveness of coverage of eligible population and reliability of data generated. The absence of one or both aspects can lead to suboptimal decisions, where vulnerable populations continue to remain excluded from development benefits. We discuss potential measures that can be adopted by governments to improve data completeness and quality, thereby expanding its use for better service delivery and improved outcomes.

Administrative data, typically collected at the time of transactions, such as ration delivery at the fair price shop or health services delivered at a clinic, tend to exclude a subset of potential beneficiaries as they record information for only those individuals who show up at least once to avail the benefit. The people who tend to get left out may often be from the most vulnerable and marginalized sections of society such as migrants or socially disadvantaged groups. In the absence of data on the universe of potential beneficiaries, coverage of programs can be overestimated leading implementers to believe that programs have reached the targeted population. Such errors can exacerbate already existing inequalities.

Ideally, there needs to be a mechanism to pre-populate the universe of potential eligible beneficiaries from which records can be updated at the time of service delivery. Mapping and census exercises (wherein each household within a defined geographic unit is enumerated) are used by researchers to develop a sampling frame to generate samples representative of the underlying population. This practice can be adopted by governments to create the universe of potential beneficiaries who would be eligible for different services and benefits. Frontline workers often carry out an annual census/mapping exercise of the villages to identify households that need services for microplanning purposes, but such data is rarely digitized. As a result, it is difficult to even know if anyone is being left out, and the extent and patterns of exclusion. As many governments are providing handheld devices to frontline workers for their work, it is now possible to capture such census information digitally, to improve tracking of eligible beneficiaries.

Measurement errors are commonplace during data collection. Errors can occur due to software issues, capacity constraints of front-line workers and willful misreporting. Use of inaccurate data, especially relating to key outcomes can lead to unwarranted changes to programs.

In order to ensure data reliability, a measure adopted during large-scale surveys is independent quality audits, or back checks. These are undertaken by persons independent of the survey team, who revisit a subset of respondents within a few days of the original survey to re-administer small, objectively verifiable parts of the same questionnaire. Reconciliation of both the data sets provides an estimate of the error rate, or the discrepancy in the responses collected by the two different enumerators from the same respondent. An analysis of the reasons for such discrepancies helps in taking corrective steps in the short term to improve quality of data, and over time, helps users determine which fields of the dataset are reliable. A policy of independent pollution audits adopted by the Government of Gujarat not only led to improvements in data quality but also incentivized plants to pollute less.

A related measure are beneficiary validation surveys, where information on whether eligible beneficiaries received the benefits, the timeliness and quality of service delivery helps identify any patterns of systematic exclusion of certain population groups. Such validation surveys can lead to improvement in implementation as experienced by farmers who were beneficiaries of a cash transfer program especially benefiting farmers with the smallest landholdings. Can governments adopt and sustain such practices at scale to improve the usability and reliability of their administrative data? The Government of Haryana is now engaged in a long-term partnership with J-PAL’s IDEA Lab to strengthen the use of health systems data in decisions. This followed a collaboration on a large-scale evaluation where independent audits done by the research team of immunization data collected by the Auxiliary Nurse Midwives (ANMs) and continuous feedback helped government efforts to strengthen data collection practices.

A strong frontline worker data collection system, with measures to improve comprehensiveness and reliability, is within the reach of any government that has the commitment, persistence, and dedicated resources. Efforts such as the UN World Data Forum 2020 can influence wider adoption of practices which can reinforce people’s and policymakers' trust in data, facilitating its use for decisions, stronger service delivery, and truly ensure that no one is left behind.

The Abdul Latif Jameel Poverty Action Lab (J-PAL) was established in 2003 as a research centre at MIT's Department of Economics. Since then, it has grown into a global network of researchers who use randomized evaluations to answer critical policy questions to fight against poverty. J-PAL affiliates have been working on evaluations in South Asia for over a decade with more than 190 ongoing or completed evaluations in the region. J-PAL South Asia is based at the Institute for Financial Management and Research (IFMR) Chennai.

For more information please visit the J-PAL South Asia website.