Census Knowledge Base » Browse » CENSUS DATA PROCESSING

Report on the Results of a Survey on Census Methods used by Countries in the 2010 Census Round

admin — Fri, 16 Dec 2011 10:40:48 GMT

By United Nations Statistics Division, 2011.

In 2009 and early 2010, the United Nations conducted a survey to collect information on methods used by countries or areas in the planning and implementation of their 2010 round population and housing censuses. The survey also collected information on challenges that countries or areas faced or expect to face in the implementation of their censuses for the 2010 round as well as on aspects of the census process in which they may have expertise. The questionnaire used for conducting the survey had 28 questions seeking information on: (i) source of population and housing census data; (ii) cartography; (iii) method(s) of enumeration; (iv) census evaluation; (v) data processing; (vi) data dissemination; (vii) census budget and source of funding; and (viii) technical assistance required and areas of expertise. This report presents the main results of this survey.

[Working paper: UNSD/DSSB/1]

Handbook on Population and Housing Census Editing, Rev. 1

Jan Beise — Wed, 26 May 2010 12:09:17 GMT

By United Nations Statistics Division, DESA, 2010.

The purpose of the present publication (the first revision of the Handbook published first 2001) is to provide countries with a broad overview of census and survey data editing methodology and to provide information for concerned officials on the use of various approaches to census editing. It is also intended to encourage countries to retain a history of their editing experiences, promote communication among subject matter and data processing specialists, and document the activities undertaken in the current census or survey in order to avoid duplicating effort during the next census or survey.

The Handbook reviews the advantages and disadvantages of manual and computer-assisted editing. In large censuses, manual correction is rarely economically feasible. The conditions for such corrections are usually specified in specially-designed computer programs that perform automatic error scrutiny and imputation based on other information for that person or household or for other persons or households. The bulk of the Handbook deals with the automatic correction of data. Computer edits play an important role in error detection and correction. At the computer editing stage, detailed consistency checks can be established in consultation with subject matter specialists. The errors detected can be corrected either by reference to original schedules or automatically. While automatic editing speeds up data processing, it demands careful control over the quality of incoming data.

The publication is divided into an introduction and five chapters. The introduction describes the census process and the various types of errors that occur in a census. Chapter I covers the basics of census editing. Chapters II to V present procedures and techniques for editing census data at various stages of processing. Technical considerations, particularly those pertinent to programming, are covered in the annexes.

Handbook on Population and Housing Census Editing, Revision 1, Series F, No.82 (United Nations Publication, Sales No. E.09.XVII.11), 2010.

CANCEIS Experiments of Edit and Imputation with 2006 Census Test Data

Jan Beise — Tue, 27 Apr 2010 09:27:19 GMT

By U.S. Census Bureau, 2007.

In this report, we demonstrate the CANCEIS (CANadian Census Edit and Imputation System) experiments
of edit and imputation with the 2006 test data. The major effort is to translate the if-then-else
rules of current edit and imputation system of the decennial census into the decision logic tables (DLT)
of CANCEIS. We also formulate the input files that are needed to run the CANCEIS software. The advantages
of using DLT are that it is easy to understand the edit rules; and DLTs are input, not part of
the software, making it easier to change when edit rules are changed. We also compare the imputation
results between the CANCEIS experiments and the 2006 Census Edited File. The comparison is for our
curiosity because the constructed DLTs are not identical to the edit rules specified in the 2006 edit specs.
Although the edit rules are not identical, the comparison still shows some similarities between the CEF
and CANCEIS results.

Principles and Recommendations for Population and Housing Censuses, Rev. 2

Jan Beise — Wed, 06 Jan 2010 06:05:21 GMT

By United Nations Statistics Division, 2008.

Since its earliest years, the United Nations has issued a series of international principles and recommendations on population and housing censuses to assist national statistical offices and census officials, throughout the world, in planning and carrying out improved and cost-effective censuses.

The Principles and Recommendations for Population and Housing Censuses, Revision 2 is divided into three parts: Part One - Operational aspects of population and housing censuses – covers mainly issues of census planning and management; Part Two – Topics for population and housing censuses – includes topics as well as their definitions and classification for both population and housing; and Part Three – Census products and data utilizations – is aimed at enhancing the dissemination and utilization of census data. The publication also contains a set of recommended tabulations on population and housing that every country is should produce at least once during the 2010 census round. There are also additional tabulations that countries may produce depending on their needs.

Principles and Recommendations for Population and Housing Censuses, Revision 2 (United Nations publication, Sales No. E.07.XVII.8), 2008.

[available in English, Arabic, Russian]

Botswana 2011 Population and Housing Census Project Document

Jan Beise — Wed, 06 Jan 2010 05:06:10 GMT

By Central Statistics Office, Botswana, 2009.

The CSO in collaboration and support from UNFPA embarked on the development
of a comprehensive Census Project Document that will also serve as source for
support and resource mobilization for all processes of the 2011 Census undertaking.
Botswana has a tradition of preparing project documents for the census. Census
project documents have been prepared for the 2001, 1991 and 1981 censuses. It is
hoped that the document will assist in resource mobilisation especially from
development partners to participate in this massive project during the current global
economic challenges.
This Document covers the evaluation of the 2001 census processes, the expected
structure of the 2011 National Census Secretariat, roles and responsibilities of
different census committees, the institutional capacity assessment of the CSO for a
successful conduct of the census in order to find response to needs’ requirements
and to proactively fill the gaps. The project document also provides different
strategies as census implementation guide and the needed Technical Assistance.
The Document provides recommendations to be implemented for the success of the
2011 Census.

Methodology of the Hong Kong 2006 Population By-census

Jan Beise — Mon, 14 Dec 2009 06:10:36 GMT

By Census and Statistics Department, Hong Kong, 2007.

The 2006 Population By-census wasconducted in July to August 2006. The By-censusprovides a wealth of data on the socio-economiccharacteristics of the population in Hong Kong, thusallowing studies to be undertaken on various facets ofthe community.

This report provides an overview on how the2006 Population By-census was conducted withdetailed descriptions on various aspects fromplanning, to data collection, processing anddissemination. It describes the design, data collection method,operation procedures and techniques of the By-census. It also includesan assessment of the data quality.

This document is Volume II of the two part "Main Report".Volume I presents detailed analysis and interpretation of the resultsof the 2006 Population By-census.

[In Chinese and English]

Census Data Capture Methodology

Jan Beise — Wed, 07 Oct 2009 11:00:00 GMT

By UN Statistics Division, 2009.

Data capture in census is the system used to convert the information obtained in the census to aformat that can be interpreted by a computer. Whilst it is acknowledged that data capture is only onesmall part of a national census project it is deemed to be one of the most critical, costly and timeconsuming activities of a population and housing census. Rapid advances in data-capture technology,especially optical, have greatly increased the speed and reliability of producing census databases inan accurate and timely manner. Nevertheless in the recent past many countries have faceddifficulties in mastering these technologies, sometimes by lack of preparation or sufficient knowledgeto avoid the numerous pitfalls.

In order to build knowledge on the vast amount of information presented and collected during theseworkshops, this technical report has been prepared to help countries in their planning of their nextpopulation and housing census.

The largest part of the budget for undertaking a national census is used in acquiring the temporarylabour necessary to run the census, with the data capture and information technology (IT) systemsbeing a much smaller part. Due consideration needs to be given to both personnel and IT systems tocompliment each other and ensure the smooth delivery of the census results. The data capture partof a census should not be viewed in isolation as it and other systems before and after it haveinterdependencies.

This report is intended to give the reader an insight into the various technical options available fordata capture and how they apply to each method. It should be pointed out combinations of thesedescribed methods have been used to good effect by some countries; therefore each method is notnecessarily mutually exclusive. The circumstances relating to any method/s chosen relates directly toeach nation’s specific needs and requirements.

The evaluation of the capturing system using scanning technology: Challenges in measuring completeness and quality of Census data processing in South Africa

Jan Beise — Wed, 26 Aug 2009 12:49:00 GMT

By Statistics South Africa, 2009.

Statistics South Africa has used scanning technology in the last 2001 Census as well as the subsequentsurveys as means of capturing the large-volume data. The implementation of the scanning technology hasallowed Stats SA to draw some important lesson in the implementation of an innovative approach incapturing system. Thereafter, corrective measures were implemented with success in the large survey calledthe community survey. Stats SA has acknowledged that the lack of proper assessment, planning, testing anddevelopment of a full complete system at pilot phase has resulted in some weaknesses of 2001 Census dataprocessing. The common challenges in the developing countries are the absence of good local knowledgebasein information technology, the lack of thoroughly tested systems, lack of good management skills of thesystems and the unavailability of enough financial resources supporting the information technology locally.

Paper presented at the 57th Session of the International Statistical Institute (ISI), Durban (South Africa), August 2009.

Challenges in Census data processing using contemporary technologies: The case of Malawi

Jan Beise — Wed, 26 Aug 2009 12:17:00 GMT

By National Statistical Office, Malawi, 2009.

National Statistical offices, as census implementing agencies, are faced with the challenge of releasing census results timely after census enumeration. This among other things entails shortening the data capture period mostly through more improved systems such as scanning. Malawi Statistical Office has been using scanning as a data capture method for small scale surveys since 2002. However, scanning at a large scale was used for the first time with the 2008 Population and housing census. This paper describes the preparations and the actual procedure of the automatic data capturing including the problems encountered and the lessons learned.

Paper presented at the 57th Session of the International Statistical Institute (ISI), Durban (South Africa), August 2009.

Data Processing Experience of the 2000 Census in Zambia

Jan Beise — Thu, 02 Apr 2009 11:55:00 GMT

By Central Statistical Office, Zambia, 2008.

This paper describes the data processing methodology adopted for Zambia’s Population and Housing Census 2000. It was planned that the 2000 Census data would be captured using Optical Mark Reading (OMR) Technology. The paper discusses several aspects of the process including designing of the OMR scanable census questionnaires, outsourcing of resources for scanning and gives an overview of the problems encountered and constraints identified.

Paper presented at Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, Dar es Salaam, Tanzania, 2008.

Data Capture and Processing in the 2006 Population and Housing Census in Nigeria

Jan Beise — Thu, 02 Apr 2009 11:46:00 GMT

By National Population Commission, Nigeria, 2008.

This paper describes the data processing methodology adopted for Nigeria’s Population and Housing Census 2006. It particularly focuses on data capture and first level editing carried out at the Data Processing Centres (DPCs) using Intelligent Character Recognition (ICR) and Optical Character Recognition (OCR) technology. It briefly touches on second level editing at the Data Validation Unit (DVU) based at National Population Commission (NPopC) headquarters, Abuja.

Paper presented at Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, Dar es Salaam, Tanzania, 2008.

Census Data Processing in Kenya

Jan Beise — Thu, 02 Apr 2009 11:34:00 GMT

By Kenya National Bureau of Statistics, 2008.

The processing of data collected in a census constitutes one of the most important and challenging activities that have to be undertaken efficiently and expeditiously in order to justify the immense resources invested in a census. This activity entailed several processes: manual editing of the questionnaires after enumeration, data capture, data cleaning and validation, and finally tabulation.Government’s commitment to provide provisional results within six months after enumeration and final basic results within another six months greatly influenced the strategies and actions adopted. This paper describes the experiences made and the lessons learned during the processing of the census data.

Paper presented at Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, Dar es Salaam, Tanzania, 2008.

Census Data Capture with OCR Technology: Ghana’s Experience.

Jan Beise — Thu, 02 Apr 2009 11:22:00 GMT

By Ghana Statistical Service, 2008.

This document describes of the Ghana Statistical Service in using Optical character recognition (OCR) technology for the data capture during the 2000 population and housing census. It discusses the reasons for choosing the OCR technology, the various working steps in capturing and editing the data, and difficulties and challenges encountered.

Paper presented at Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, Dar es Salaam, Tanzania, 2008.

Using OCR for Large Volume Census Data Capture in China

Jan Beise — Thu, 02 Apr 2009 10:45:00 GMT

By National Bureau of Statistics of China, 2008.

National Bureau of Statistics of China (NBSC) has used OCR technology in two cases of large-volume census data capture; one is the fifth national population census, the other is the second national agricultural census. The document describes the experiences of the NBSC in using these technologies during those two major statistical undertakings.

Paper presented at UNSD-ESCAP Regional Workshop on Census data processing: contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving; Bangkok, Thailand, 2008.

Brunei Darussalam experience in 2001 Census Data Processing

Jan Beise — Thu, 02 Apr 2009 10:30:00 GMT

By Department of Statistics, Brunei Darussalam, 2008.

This document describes shortly the main features of the 2001 Population and Housing Census in Brunei Darussalam. It focuses in particular on the data capture and data processing, and gives an outlook for the 2011 census.

An overview of Editing and Imputation methods for the next Italian Censuses

Jan Beise — Fri, 23 May 2008 09:51:00 GMT

Paper by National Institute of Statistics of Italy

The National Institute of Statistics of Italy is preparing the next Population and Housing Censuses that will be held on 2011. In this paper, a short description of the main editing procedures for the 2011 Census is given. The impact of some likely census innovations on the editing and validation is also reviewed.

Paper presented at the Joint UNECE/Eurostat Meeting on Population and Housing Censuses, Geneva, 13-15 May 2008.

[Paper also available in French and Russian]

Data Processing of the 1999 Population Census in Viet Nam

Margaret Mbogoni — Wed, 19 Mar 2008 10:25:00 GMT

The paper, prepared by the General Statistics Office of Vietnam, presents the experience of Viet Nam in the processing of data collected during the 1999 census. Issues discussed relate to the choice of software; distribution of data processing facilities; data entry, editing and tabulations operations; and data dissemination. Also presented are plans for the 2009 census.

Paper prepared for the ESCAP Expert Group Meeting on Effective Use of IT in Population Censuses, Bangkok, 10-12 December 2007

Broad Issues of Editing And Imputation For Population Censuses

Jan Beise — Wed, 21 Nov 2007 10:00:00 GMT

This paper by Mike Bankier of Statistics Canada explores the issues of editing and imputation (E&I) in the context of population census data. E&I should eliminate non-response and inconsistent responses in an effective but conservative manner. It should be possible to specify edits easily and then either do minimum change donor imputation or specify deterministic imputation actions. In addition, the user should be able to evaluate the results easily during Census production to determine if the imputation should be redone. Because usually not all census variables can be processed simultaneously, the variables should be partitioned into groups and then these groups should be processed sequentially or in parallel.

Somewhat different imputation techniques are often required for households with total non-response compared to those with partial non-response. Also, sampled households that received a long questionnaire with additional questions should be processed carefully to avoid biasing estimates based on these samples.

The above issues will be illustrated by describing the techniques used in the 2006Canadian Census of Population to carry out E&I.

Paper presented at the Satellite Meeting on Innovative Methodologies for Censuses in the New Millennium of the 56th Session of the ISI, Lisboa, 31 August - 2 September 2007.

Using emerging technologies: Getting the best results from paper based data capture

Jan Beise — Wed, 21 Nov 2007 05:54:00 GMT

This paper by Andy Tye and Mike Smethurst of DRS Data Services Limited reviews the tried and tested techniques of Census “Data collection/processing” via booklets/paper and also considers the latest emerging technology which is now available to census organisation to capture the data from paper based census forms. Although it is acknowledged that other non paper based data collection methods are being used in census, such as the use of Personal Digital Assistant (PDA), Internet, Telephone, etc., this paper specifically focuses on the collection and processing of Census data from the paper medium. There are four methods of traditional data collection from booklet/paper. New emergingtechnology is able to combine key features of these traditional techniques. This paper discusses some projects where these combined techniques are being used.This paper makes the following assumptions: The paper based forms are designed for purpose. A Control procedure is in place for the receiving of reports from the field and an initial Quality procedure is undertaken. All batches should havea fully auditable batch control sheet associated with them.

Paper presented at the Satellite Meeting on Innovative Methodologies for Censuses in the New Millennium of the 56th Session of the ISI, Lisboa, 31 August - 2 September 2007.

Conference of European Statisticians Recommendations for the 2010 Censuses of Population and Housing

Margaret Mbogoni — Mon, 29 Oct 2007 09:01:00 GMT

The main objectives of the CES Recommendations for the 2010 Round of Population and Housing Censuses are: ( i) to provide guidance and assistance to countries in the planning and conducting of their population and housing census; ( ii) to facilitate and improve the comparability of the data at regional level through the selection of a core set of census topics1 and the harmonization of definitions and classifications.

The first part of the publication (Part One) presents two chapters on census methodology and technology. The objective is not to recommend a specific method or a technology, but rather to present the different approaches with their advantages and disadvantages and guide countries to make the best choice that fits their national circumstances. These chapters are complemented by additional material presented in various appendices. The second and third parts of the publication present respectively population topics (Part Two) and housing topics (Part Three). The Recommendations aim to describe the different census topics that are relevant for the region presenting not only definitions and standards but also analyzing their relevance and their comparative advantage in relation to other census topics and other data collection activities outside the census. The topics are presented using a structure by theme (demographic, migration, ethno-cultural characteristics, etc.) to facilitate a broader view in the description of the content of a census.

Conference of European Statisticians Recommendations for the 2010 Censuses of Population and Housing (United Nations publication, ECE/CES/STAT/NONE/2006/4), New York and Geneva, 2006

[Also available in French and Russian]

Population and Housing Census in Thailand

Jan Beise — Wed, 12 Sep 2007 06:22:00 GMT

This paper by Jirawan Boonperm of the National Statistical Office, Bangkok, Thailand, presents experience from the 2000 census of Thailand, including problems encountered.

Paper presented at the ASEAN Meeting on the 2010 round of Population and Housing Census, Siem Reap (Cambodia), 31 July - 2 August 2006.

Population and Housing Census in Viet Nam

Jan Beise — Wed, 12 Sep 2007 06:20:00 GMT

The paper presents experience from the 1999 census of Viet Nam, including strong and weak points.

Paper presented at the ASEAN Meeting on the 2010 round of Population and Housing Census, Siem Reap (Cambodia), 31 July - 2 August 2006.

Handbook on Census Management for Population and Housing Censuses

Jan Beise — Wed, 12 Sep 2007 06:03:00 GMT

This publication is a reference document on management aspects of conducting a population and housing census. It provides guidance on how to

(i) develop a structure able to effectively manage the census planning and operational processes;
(ii) plan all the process that need to be considered in establishing a census; and
(iii) design control and monitoring processes.

The structure of the handbook reflects as closely as possible the census cycle. The initial chapters discuss management aspects concerning the planning and preparatory stages, followed by discussions on the operational stages (i.e. field operations and processing), dissemination of census results, and finally evaluation. The publication is also useful to those who need to plan surveys given the detailed description of the overall process of collecting, processing, and disseminating the data.

Handbook on Census Management for Population and Housing Censuses, Series F, No.83/Rev. 1 (United Nations Publication, Sales No. E.00.XVII.15), 2001.

[online also available in Arabic]

The 2011 Census: A Design for England and Wales

Jan Beise — Wed, 12 Sep 2007 04:55:00 GMT

This paper by the United Kingdom Office for National Statistics discusses the proposed design for a 2011 Census in England and Wales, drawing on the experiences and lessons learned from the 2001 Census, current world best practice, and the opportunities arising from electronic data capture and other changes in available technology. The paper also examines potential benefits and risks of the proposed design.

Paper presented at the United Nations Symposium on Population and Housing Censuses, New York, 13-14 September 2004.

Population and Housing Census in Malaysia

Jan Beise — Tue, 11 Sep 2007 07:01:00 GMT

The paper by the Department of Statistics Malaysia (DOSM) presents the experience from the 2000 census in Malaysia as well the preparatory work undertaken for the 2010 Census.

Paper presented at the ASEAN Meeting on the 2010 round of Population and Housing Census, Siem Reap (Cambodia), 31 July - 2 August 2006. as preparations for the 2010 census.