Statistical Data and Metadata Exchange - SDMX

What is SDMX?

SDMX - Statistical Data and Metadata Exchange - is an initiative sponsored by the Bank for International Settlements (BIS), the European Central Bank (ECB), Eurostat, the International Monetary Fund (IMF), the Organization for Economic Co-operation and Development (OECD), United Nations Statistics Division (UNSD) and the World Bank aimed at fostering more efficient processes for exchange and sharing of statistical data and metadata among international organizations, their constituencies and users. Detailed information on the SDMX standards and organization are available on http://www.sdmx.org

What are the goals of SDMX?

SDMX tries to take full advantage of Internet technology for collecting and disseminating statistics, with the full awareness that this involves developing common standards for the exchange of statistical data and metadata. Moreover, in order to benefit both producers and users of statistics, SDMX standards hopefully can be applied in due course to all stages of the statistical production chain. SDMX standards aim to ensure that data are always accompanied by appropriate metadata, helping to ensure the coherence of data and making it immediately useable. Application of SDMX standards and practices can improve data quality measured in terms of timeliness, accessibility, ease of interpretation and coherence, while reducing costs.

How SDMX changes data exchange?

SDMX-ML - the XML format for the exchange of SDMX-structured data and metadata. SDMX-ML is a data definition format for the electronic communication of statistical data which is set to modernize statistical data collection and dissemination around the world. It provides major benefits in the preparation, analysis and communication of statistical information. It offers cost savings, greater efficiency and improved accuracy and reliability to all those involved in supplying or using statistical data. It is an open, free of licence fees standard.
Common Metadata Vocabulary - A common terminology for the exchange of statistical information that takes into account developments on two fronts: (a) on “content” issues, relating to the statistical meaning of metadata items (e.g. definitions, how statistical data are collected, compiled, processed and disseminated), and (b) on technical means of managing and transferring the information, i.e. IT tools and related models of metadata flows. The two aspects are both essential for a complete solution to the problem.

What are the potential uses of SDMX-ML?

SDMX-ML can be applied to a very wide range of statistical. Among other things, it can handle: Collecting information from data providers in a standardized format; Exchange data between internal database systems; Exchange information between systems of departments or between other institutions, such as statistical offices or international agencies; Disseminate information for public use in a standardized manner; Long term storage (archiving) data in a well described textual format.

What are the benefits using SDMX-ML?

SDMX-ML increases the usability of statistical information. With the adoption of SDMX-ML, agencies can automate data collection. The need to re-key statistical data from paper based questionnaires can be eliminated. For example, data from different statistical offices with different systems can be assembled quickly, cheaply and efficiently. Once data is gathered in SDMX-ML, different types of outputs using varying subsets of the data can be produced with minimum effort. Not only can data handling be automated, removing time-consuming, error-prone processes, but the data can be checked by software for accuracy.

How to transfer data using SDMX-ML?

There are a number of ways to transfer data using SDMX-ML:
SDMX-ML aware toolkits are becoming available which will support the export of data in SDMX-ML form. These tools allow users to map charts of accounts and other structures to SDMX-ML tags. Statistical data can be mapped into SDMX-ML using SDMX-ML software tools designed for this purpose. Applications can transform data in particular formats into SDMX-ML. For example, web sites are in operation to transform currency exchange rates into SDMX-ML, providing more efficient access to specific data. The route which an individual agency may take will depend on its requirements and software and systems it currently uses, among other factors.

What kind of database should be used with SDMX-ML?

SDMX-ML is a format for exchanging information between applications. Therefore each application will store data in whatever form is most effective for its own requirements and import and export information in SDMX-ML format so that it can be readily imported or exported in turn by other applications. In some applications, for example, the SDMX-ML formatted information being used may be mostly tabular numeric information, hence easily manipulated in a relational database. In other applications, the SDMX-ML information may consist of narrative document-like structures with a lot of text, so that a native XML database may be more appropriate. There is no mandatory relationship between SDMX-ML and any particular database or other processing or storage architecture.

What are the differences between HTML, XML and SDMX-ML?

HTML (Hypertext Markup Language) is a standard way of marking up a document so it can be published on the World Wide Web and viewed in a browser. It provides a set of pre-defined tags describe on how content appears in a browser. For example, it describes the font and color of text. It gives little information on meaning or context. XML (Extensible Markup Language) uses tags to identify the meaning, context and structure of data.
XML is a data definition language which is maintained by the World Wide Web Consortium (W3C). XML does not replace HTML; it is a different markup language that is platform independent, allowing XML data to be rendered on any device such as a computer, cell phone, PDA or tablet device. It enables rich, structured data to be delivered in a standard, consistent way. Whereas HTML offers a fixed, pre-defined number of tags, XML neither defines nor limits tags. Instead, XML provides a framework for defining tags (i.e. taxonomy) and the relationship between them (i.e. schema).
SDMX-ML is an XML-based schema that focuses specifically on the requirements of statistical reporting. SDMX-ML builds upon XML, allowing statisticians to identify items that are unique to their reporting environment. The SDMX-ML schema defines how to create SDMX-ML documents and SDMX-ML taxonomies, providing users with a set of information tags that allows users to identify information in a consistent way. SDMX-ML is also extensible in that users are able to create their own SDMX-ML taxonomies that define and describe tags unique to a given environment.