Whereas the statistical community has made good progress on using Big Data, many questions and challenges remain. Statistical Offices still need to make further progress in delivering timelier, more frequent and more granular data; the private sector remains a formidable competitor; more engagement and collaboration with private sector and other partners is necessary to combine and benefit from each other's strengths.
This means that additional steps have to be taken: results from the work of the existing Task Teams should become usable for the statistical community; Big Data, administrative data and traditional statistical data sources should really be treated together via a multi-source approach; the challenge is not to overcome problems of using Big Data in itself, but to achieve trusted data collaboration; and in that respect the UN Global Working Group on Big Data needs a collaborative Global Platform for the overall organization of its work.
In short, the strength of the community of official statistics is trusted data, and the real need of the community is close collaboration with private sector, academia, research community and civil society.
Data collaboratives are a new challenge and new opportunity for the community of official statistics – in relation to Big Data, to the SDGs, to the sharing of data, services, technologies, and know how.
These and similar questions will be addressed at the 4th UN Conference on Big Data in Bogota, Colombia from November 8th to 10th – organised in collaboration of the UN Global Working Group, DANE and the Colombian Ministry for ICT.
The Cape Town Global Action Plan [Annex I] for Sustainable Development Data and the regional action plans for the tansformative agenda of official statistics both emphasize strengthening specific strategic areas: (a) coordination and leadership by the national statistical office; (b) innovation and modernization of national statistical systems; (c) integrated data systems for the production of statistics; (d) multi-stakeholder partnerships; (e) dissemination and communication of data; and (f) capacity building and resource mobilization.
The GWG works on all those aspects, but specifically on innovation and modernization of national statistical systems. This strategic area addresses the access and integration of traditional, administrative, Big and geospatial data sources, the transformation of the statistical production processes and information architecture, the legislative reform around security and privacy and the adoption new methods, tools and technologies.
The statistical office are increasingly playing a leading role in the integration of statistics into national planning and in the provision of society with relevant information through Open Data standards while upholding privacy and confidentiality. Advancing open data standards will break down data silos and increase the shared value of data by creating an ecosystem in which data consumers can interoperate with data producers in a way that is far more powerful than currently possible, enabling more applications to make sense of a broader set of data. This theme addresses the policies, technologies, processes that enable the free access, use, reuse, modify and redistribution of data by anyone.
While the societal benefits from the access and integration of multiple sources of microdata, including from statistical and administrative sources, are undisputed, many challenges have to resolved. The national statistical offices (NSOs) must consider the legal issues around microdata control and access, confidentiality/privacy concerns, and the costs associated with preparing public, restricted or scientific use files, as well as the cost to both the researcher and the NSO of maintaining secure data enclaves. Providing transnational microdata access expands the list of legal and organisational considerations and may increase the public’s concern regarding the confidentiality of data collected by the NSOs. Also technical considerations are to be addressed in providing standardization of metadata through DDIs and SDMX to support the discovery of micro and macrodata files and their content, in assessing the need and eligibility for access to restricted data as well as in operationalising access and confidentiality constraints. NSOs should consider advancing international standards such as DDI for metadata and standards for data and technology infrastructure and apply them within national data centers around the world, and notably in those which provide remote and transnational access.
With the advent of information society, the national statistical systems have to adopt existing new working relations with existing partners and open new collaborations with new partners. New data producers increased significantly including many from the private sector. The new demands from the rapid evolvement of the data and technology infrastructure, data security, data interoperability and other control mechanisms make collaboration with private partners in technology sector inevitable. Moreover, the complexities of data preparation, estimation and analysis and dissemination require data scientists, who are mostly linked with research and academic institutes.
This drive for innovation and modernization of national statistical systems calls for a rethinking of the partnerships of the community of official statistics with private sector, academia and civil society through an interconnected ecosystem of open data and technology collaborations and platforms at national, regional and global level. In this context, “Trusted Data Collaboratives” are a new way of working together with proper definition of the interests of the various stakeholders, proper definition of responsibilities and access, and appropriate protocols to safeguard confidentiality.
On 25 September 2015 world leaders committed to the 2030 Agenda for Sustainable Development, including many ambitious goals and targets to be achieved by 2030. The statistical community was charged with defining appropriate indicators to monitor the progress towards these targets. It was stressed that differentiation by population groups, sub-national location and smaller time intervals would be make the information base more useful for policy decisions.
As indicated in the introduction of the Global Action Plan, high quality and timely data are vital for enabling governments, international organisations, civil society, private sector and the general public to make informed decisions and to ensure the accountability of representative bodies. Effective planning, follow-up and review of the implementation of the 2030 Agenda for Sustainable Development requires the collection, processing, analysis and dissemination of an unprecedented amount of data and statistics at local, national, regional and global levels and by multiple stakeholders.
The 2030 Agenda explicitly calls for enhancing capacity building to support national plans to implement the sustainable development goals (SDGs). National statistical systems (NSS) face the urgent need to adapt and develop to meet the widening, increasing and evolving needs of data users, including data for all SDG indicators.
"Leaving no one behind" was an often-repeated wish by the politicians, and statisticians needed to heed the call by disaggregating the necessary information. Big Data and other non-traditional data sources should help in this endeavor.
Along with the SDG monitoring, the underlying microdata should also become accessible through national, regional and international open data platforms and become discoverable through standard metadata documentation. These open data platforms should also apply open data protocols for the creation and use of interoperable APIs based on ISO standards to allow for trusted data collaboratives, comparability and interoperability, improved governance and citizens engagement and inclusive development and innovation (see open data charter)
Due to limited space at the Conference venue and due to the fact that this Conference is organized within the framework of a UN intergovernmental process, registration for this event depends on the profile of your institute and is coordinated through the organizing committee. If your institute is interested in participating in this Conference, please email firstname.lastname@example.org with copy to email@example.com. Please provide information on the nature of the work of your institute and how this relates to Big Data and official statistics.