When the data stop: Five lessons for data-driven decision-making in crises

The Sustainable Development Goals presented us with an important mandate: to achieve measurable progress, leave no one behind, and leverage the latest innovations to achieve the global goals through data and evidence-driven policies. The COVID-19 global crisis has highlighted data’s crucial and transformative potential. Governments, private sector, civil society, and international institutions are striving to collect and analyze data to understand and contain the pandemic; and address the socio-economic effects of pandemic response.

But amidst this renewed emphasis and demand for hyper-local, real-time data and evidence, we’re also faced with an acute challenge: What happens when the data... stop?

The COVID-19 crisis has acutely impacted the data value chain, because pandemic response and mitigation measures effectively “lock down” traditional social development data sources. In many countries, in-person surveys and census preparations were (or remain) paused and postponed. From the UN Statistical Division and World Bank’s Global COVID-19 survey of National Statistical Offices, over 50% of censuses and nearly 40% of planned surveys have been delayed.

Further, over 56% of survey respondents cited “moderate or severe” impediments to administrative data processing. Administrative data come primarily from civil registration and vital statistics systems, and sectoral information systems, and are often the go-to source for public planning and management. Administrative data are often collected at points of service delivery – many of which have either closed to the public (such as schools) or have seen a significant change in the composition of citizens being served (such as health centers) due to the pandemic.

Yet decisions are (and will continue to be) made, regardless of data availability. So how can we make the best decisions with the data we do have?

First, information still exists, even if it’s not in the format we’re used to. In a number of countries, local government staff (health workers, social workers) have been re-trained and are assisting with pandemic response. These and other subnational actors arguably know best about real citizen needs, and can serve as a key information resource.

Importantly, this data collection should seek to be responsive, not extractive, in nature; and must avoid over-burdening subnational staff or compromising service delivery. The Principles for Subnational Data Use provide a starting point for learning about good practices for engaging with stakeholders.

Second, surveys carried out using alternative data collection methods can help us understand and address the effects of this crisis. These include UN-led Socio-Economic Impact Assessments; mobile phone surveys led by the World Bank, Innovations for Poverty Action, and IDinsight; and innovative applications of existing spatial data and predictive modeling.

Data collection has also included targeted, problem-driven data collection efforts. For example, a consortium of South African universities are conducting ongoing rapid mobile surveys on the economic and health impacts of lockdown differentiated by gender; Development Gateway partnered with IFDC and AFAP to gather rapid data on COVID-19’s impact on agriculture input supply chains; and UNICEF has used of U-Report to gather perceptual data and provide needs-based services (i.e. mental health and psychosocial support).

We can also still see generalizable trends from the data we do have. For example, through surveillance data in the United States public health experts attributed a decline in health-seeking behavior to COVID concerns. And, even without perfect information, there are matters about which we know enough to act. For example, we know lockdown and/or crisis response conditions increase the risk of violence against women and children, and decision-makers can implement evidence-based prevention and mitigation strategies.

However, the above should be taken with two important caveats.

First, the methods mentioned above – perceptual data, alternative data sources, rapid surveys, general trends – are not without limitations. In non-crisis times, no data are perfect; in present pandemic times, we must be particularly aware of limitations in data representativeness.

Tools such as mobile surveys, qualitative information, and community feedback may exclude society’s most vulnerable: women, ethnic minorities, people with disabilities, rural populations, etc. We should not dismiss “good enough” data, but we must remain conscious of the potential biases and omissions in the information we have – particularly in light of our mandate to “leave no one behind”.

Second, as emphasized by the OECD, crises should not be used as a rationale for unduly relaxing legislation or good practices regarding data privacy, ethics, and protection. In a number of instances, the crisis has brought to light the lack of such frameworks and policies, and contributed to government responses that fail to protect citizens’ rights, result in data privacy breaches, and generally contribute to public mistrust. These examples underscore the importance of having these frameworks in place before a crisis occurs, and clear communications with public, private, and government stakeholders.

I’m looking forward to discussing these and other topics at the 2020 Virtual UN World Data Forum on 19-21 October 2020, and connecting with others keen to share solutions and thinking in support of the 2030 Agenda for Sustainable Development and monitoring and recovery from COVID-19.