Scanner Data

Task Team of the UN Committee of Experts on Big Data and Data Science for Official Statistics

Introduction

The statistical community has the obligation of exploring the use of new data sources, such as scanner and web scraped data, to meet the expectation of society for enhanced products and improved and more efficient ways of working. The Task Team on Scanner Data was set up in 2017 with the aim of increasing the use of scanner data in national statistical systems. Scanner data has been used for the calculation of consumer price indices alongside other alternative data sources such as web scraped data and administrative data. The Task Team has been through several iterations since 2017. The current aim for the Task Team (as refreshed in 2024) is to enable and expand the practical usage of these data in consumer price statistics. Future work will look to develop the wider usage of these data (for example, in household expenditure statistics). In line with the overall mandate of the UNCEBD, the Task Team will achieve this through expanding available guidance materials and promoting training and sharing of experience.

Objectives

To guide, develop, and promote the practical usage of alternative data sources (including scanner, web scraped and administrative data) in consumer price statistics, the specific objectives of the Task Team are as follows:

Specific Objectives:

  • Repository of information. Provide guidance on the end-to-end methods and processes of using these data. The guidance should reference existing literature where possible.
  • New guidance material. Provide methodological guidance for those topics where existing literature does not exist or does not fully capture the different range of possible options for consideration.
  • Practical implementation. Provide practical guidance on the implementation of these methods and processes in a live production setting.
  • Training. Provide a full suite of training materials to supplement the guidance produced above.

These objectives will be supported by two underlying principles:

  • Timeliness. Ensure that any content produced by the Task Team in line with the above objectives remain up to date with current discussions in the field. For this objective, the Task Team will be supported by a Secretariat function which will be responsible for maintaining the existing published content.
  • Informed choice. Provide a range of options for these topics, so we should not recommend a single “best” approach - instead colleagues will be encouraged to review the options and decide what is best for their particular situation.

Scope of Work

  1. Standards and frameworks: Publish and maintain guidance on utilising new data sources in the production of consumer price statistics. Guidance to be structured in a form most applicable to support NSOs, including but not limited to an evergreen handbook, approaches to reproducibility, example code, etc.
  2. Capacity building: Publish and maintain training content aligned to the handbook guidance, covering the end-to-end process of using these new data sources in the production of consumer price statistics.
  3. Research and innovation: Identify gaps in the existing literature and publish additional guidance as appropriate.
  4. Collect feedback from the community: Present the work of the Task Team via established networks and conduct surveys to help shape guidance and support the planning of future workstreams.

Events

Eighteenth meeting of the Ottawa Group on Price Statistics
Hybrid   13-15 May 2024
The Task Team presented an update on the work covering the last 12 months to May 2024 (Session 5). We also ran a workshop session (workshop on use of alternative data sources for CPI) which covered topics such as the newly published handbook and updated guidance on classification methods.

Meeting of the Group of Experts on Consumer Price Indices
Hybrid   7-9 Jun 2023
The Task Team presented an update on the work covering the last 12 months to June 2023 (Session 1). We also ran two workshops (workshops on scanner data 1 and 2) which covered topics such as acquisition of scanner data and classification methods.

Seventeenth meeting of the Ottawa Group on Price Statistics
Hybrid   7-10 Jun 2022
The Task Team presented an update on the work covering the last 12 months to June 2022 (Session 4).

Task Team members

Countries

  • Australia
  • Austria
  • Belgium
  • Brazil
  • Canada
  • Denmark
  • Finland
  • Germany
  • Indonesia
  • Italy
  • Luxembourg
  • Malaysia
  • Mexico
  • Netherlands
  • New Zealand
  • Norway
  • Poland
  • Switzerland
  • Thailand
  • Turkey
  • United Kingdom
  • United States

Agencies

  • Eurostat
  • IMF
  • UNSD
  • World Bank

Universities

  • University of Graz, Austria