My first experience of data collection was running a survey in a small, isolated village in Northern Mozambique in the late 1990s as part of my academic research. I collected and analysed the data, and when I had finished with it, I put the 400 or so paper surveys in boxes (which I still have!) and put them in a cupboard. That was fairly typical for the times -- a world where the collectors and users of data were the same, and where data sharing and re-use were the exception, not the rule.
Everything has changed since then. We live in a world where a huge amount of data is now collected by machines -- and where the collectors of the data, the mobile phone companies or owners of satellites, are often not the users of the data they collect. And where data has most value when it is open, and can be re-used over and over again, or combined with other data through the magic of digital programing.
My most recent experience of data collection was being part of a team that created a global survey run by the United Nations Development Programme. We collected responses from 10 million people on paper, online and by mobile phones -- the data is all available online, and has been used by thousands of academics, governments and civil society organisations and combined with hundreds of other datasets -- including household surveys and data from twitter -- to produce more insights and impact than could possibly have been achieved by the small team running the project.
This shift, as I have learned in my working life, means that it is literally impossible to take up the new opportunities of data without finding new ways to bring the very different producers and the users of data together.
The data revolution is all about connections. I see what these connections can do every day through the work of the Global Partnership for Sustainable Development Data. When I sit round a table in Dakar, Senegal with government officials, private sector partners and civil society, and hear how they are working together to ensure that all the benefits of the data revolution are brought to bear to deliver sustainable development in that country. When I sit in a café in London and hear how a data scientist is developing a model to predict malaria outbreaks in Zambia. And when I hear, on a Skype call from Bogotá, Colombia, how mobile phone companies, programmers and developers are working with governments, academics and civil society organisations across Latin America to gather insights from over 1,000 different datasets that can help to monitor and achieve the Sustainable Development Goals.
But I am also very aware that making this happen is not easy. In this new landscape the producers, analysts and users of data come from different sectors -- private and public, civil society and academic. They operate within different boundaries -- national governments, international agencies, global companies, city authorities and district level NGOs. They have different incentives -- political, commercial, social and professional.
We know what is possible now, and thanks to the work of innovators all over the world, are starting to understand the benefits that the data revolution can deliver. But making it happen, worldwide, and making sure that everyone can benefit, is as much a political as a technical challenge. Working out how to combine datasets to produce insights that are methodologically rigorous and usable in different contexts is one thing. Creating the legal, regulatory and political frameworks, and making the investments in human and technical resources that make it possible to use those insights at scale is quite another.
The United Nations World Data Forum is coming at exactly the right moment. The technical possibilities are dazzling. But the political and economic questions that need to be answered if essential collaborations are to happen, are only just beginning. A UN forum, bringing together governments, companies, UN agencies and global networks like the GPSDD, is just the place to begin those conversations. I, for one, cannot wait.