Symposium 2001/45

26 July 2001

 

                                                                                                           English only

 

Symposium on Global Review of 2000 Round of

Population and Housing Censuses: 

Mid-Decade Assessment and Future Prospects

Statistics Division

Department of Economic and Social Affairs

United Nations Secretariat

New York, 7-10 August 2001

 

 

 

 

 

 

 

 

 

 

 

 

Adapting new technologies to census operations *

Jack Arthur**


CONTENTS

 

B. Information technology environment: the track record. 1

C. Working with the commercial sector: outsourcing. 3

D. Applying effective technologies. 4

E. Collection and dissemination technologies. 5

1. Collection. 5

2. Dissemination. 6

F. Geographic information systems and related spatial technologies. 6

G. Ubiquitous electronic access: the internet 7

H. Conclusions, lessons and best practices. 8

I. Questions and suggestions for further discussion. 8

References. 9

 


A. Introduction

1.                  Public policies requiring population censuses have always been formulated with implicit assumptions concerning available technologies, both intellectual and material. Newly available information technologies (IT) have continually coloured census requirements, costs and expectations on the use of census products. To a great extent, this is typical of many public policy roles in which governments have been assigned a role to manage and maintain information for the public good. Weather forecasting and tracking, environmental quality monitoring, and economic forecasting are government functions with similar characteristics that immediately come to mind.

 

2.                  Censuses, however, have generally been in place longer, have more ponderous periodic requirements for consistency (such as the mid-decade assessment being discussed at this conference), and are very expensive (about $50 billion worldwide) to complete. They are generally more sensitive to the price/performance and response time characteristics of IT than many public policy applications of IT. While cost “savings” drive a good deal of the application of new technology in censuses, there are further implications for the management and structure of the census itself when new technologies are applied.  Whether IT is being adapted to streamline the data-collection process, disseminate the products more efficiently or facilitate better use of the census information, the characteristics and cost of the technology being applied have a material effect on all aspects of the census. Technology cannot be applied without careful consideration to the resulting changes to the census process.

 

3.                  The concept of IT in this paper applies not just to faster and more powerful computers and software, but also to the application of powerful new methods of organizing and managing information itself. Technologies for storing, organizing, combining, searching, correlating and presenting information have undergone a revolution in the past few years—particularly with the now ubiquitous availability of the Internet and related technologies.  Census data can be captured in new ways, correlations and presentations of census products have spawned whole new industries, and the “information age” has changed the entire intellectual process of how we see the world and predict the future.

 

4.                  At the same time, IT projects, particularly in the public sector, often result in colossal failures.  There is strong evidence that these failures occur more often than not, so that the great opportunities in adapting new technologies to census operations are also fraught with pitfalls.  These opportunities surely cannot be ignored, because there is too much at stake, both in reducing direct costs and in improving the richness of the census.

 

5.                  This paper will cover both developments and trends in IT in relation to census management, and focus on how managers who are not IT professionals can better manage IT. It will also cover what these managers need to know about IT, when they should rely on others and how they can assess how well they can do.

B. Information technology environment: the track record

6.                  Information technology offers such phenomenal opportunities in the gathering, processing, and disseminating of information that it must be an integral part of the census process.  Increasingly, demands for changed, improved, and more cost-effective products require the adaptation of IT to every aspect of census activity. Often, in these projects, return on investment in direct cost savings amounts to 20 per cent and more. When combining these savings with the increased effectiveness of the census process, the return on investment is often several times the initial cost for the investment. These opportunities are simply too great, the demands too strong, and the results too compelling for responsible organizations and managers to ignore them.

 

7.                  However, successfully managing IT projects and realizing the expected results are very difficult. It is important for census managers to be aware of the difficulties, and to equip themselves to successfully cope with the challenges described here. To gain perspective, it is useful to look closely at success and failure in large IT projects.  These projects can be assessed through several factors:

 

·        Budget: Did the system meet spending estimates?

·        Timeliness: Was the project delivered on time?

·        Usability or quality: Does the product deliver as planned?

 

8.                  Success and failure are therefore ultimately related to delivery of the final product, and in principle failure can be measured as percentage deviations from initial budget projections, deadlines and design requirements. However, major IT projects, which are an integral part of a larger business process, such as a population census, are not so simple to identify and track separately. These projects are dynamic—they change while they are implemented and as the business process evolves. At the same time, a project that exceeds the budget and is not delivered on time can be a substantial success from the point of view of the business process.

 

9.                  Additionally, a project that meets design requirements and is delivered on time and at the projected price can still be a failure in practice because of lack of training of the users and lack of necessary organizational changes to effectively implement the system. In order to be successful, an IT project must be part of the wider business process and related organizational changes rather than an end in itself. Census managers are in charge of the census process, not the development and implementation of IT projects. This is a focus that often gets lost as a large IT project develops.

 

10.              There is no systematic comparative survey of the scale and scope of success and failure of large public IT systems in worldwide census activities. However, it is instructive to look at large IT systems in general, because they have many characteristics in common with census IT work. These surveys and anecdotes generally indicate that failure in implementing large IT systems is an almost global phenomenon.

 

11.              For example, the Standish Group has undertaken surveys of success and failure rates in the private companies and public organizations in the USA since 1995.

 

12.              Project realization is assessed according to three dimensions of success: delivery of usable projects on budget, and on time. On this basis, the Standish Group operates with three groups of projects:

 

·        Successful projects, which are completed on time and on budget with all features and functions as initially specified;

·        Challenged projects, which are eventually operational but are delivered over-budget or over the time estimate or which offer fewer features and functions than originally specified; and

·        Impaired projects, which are cancelled at some point during the development cycle.

 

The results of the surveys in 1995 and 2001 are depicted below.

 

 

Sources:       1) Standish Group, www. Standish.com

                   2) Organisation for Economic and Cooperative Development (OECD), 2001.

 

13.              Though outputs have improved over the last five years, only 28 per cent of IT projects in 2000 succeeded according to this survey, 23 per cent were cancelled and almost half were impaired. Clearly the risk is large in these projects, and the ability to count on originally projected results is low.

 

14.              Because the results are so poor in the implementation of these large IT projects, a great deal of effort has been applied lately in identifying good project management practices (OECD, 2000). These can be summarized as:

 

·        Involving top-level management;

·        Employing effective risk analysis in guiding direction;

·        Avoiding leading-edge technologies, opting for small projects; and

·        Involving end users in project formulation and management.

 

15.              The preparation of a detailed IT business case is now common practice for invoking the necessary discipline to ensure that these practices are followed. The business case and investment process is designed to analyse return on investment, identify risks and mitigation strategies, ensure modular development to avoid large-scale failures, and provide for oversight and review at crucial steps along the way. No sizeable IT project should be undertaken without this management process being followed. In the United States, the process is specified in detail in several laws governing IT budgeting and procurement[1]. This direction describes three major phases of the investment and control process for IT projects[2]:

 

·                    Selecting (screening, evaluating risks and return, and mission mix);

·                    Controlling (monitoring against costs, schedule and performance); and

·                    Evaluating (post-implementation reviews, adjustments and lessons).

 

 

investments process diagram

This management discipline has been largely responsible for improving the success of larger IT projects in the United States government over the past five years.

 

C. Working with the commercial sector: outsourcing

16.              One of the greatest opportunities, but also the greatest challenges, in adapting IT to census work, is the use of “commercial practices”, or outsourcing.  Organizations generally receive best value using commercial products, if they can be adapted to the mission requirements. The common reasons for considering outsourcing are:

 

·        “Standard” or “commodity” products are less expensive due to competition;

·        Vendors often have specialized skills that are difficult to maintain “in-house”;

·        The agency can concentrate on the core mission (which is not IT project development);

·        Competition can bring innovation to bear on a mission requirement;

·        Risks can be shared with a vendor if the rewards are great enough; and

·        Proprietary or unique software or systems may be available only from a vendor.

 

17.              Standard IT products are those having high market penetration, such as the personal computer, which have industry standard characteristics, and are available as commodities from a variety of sources. This category also includes some classes of software and network service products, such as computer operating systems, office software suites, some statistical packages, network portal services, and web hosting. These are the most likely candidates for outsourcing not only the product itself, but often the maintenance, update and replacement.

 

18.              Vendors are often used to outsource IT project development and implementation or provide specialized skills (such as network management, proprietary database knowledge, etc.). This is done where cost-effective and the designated tasks fit an organizational strategy for in-house versus out-house skills and responsibilities. The requirements for this category are the most diverse and vary not only from country to country, but also from organization to organization.  Past performance and the experience of other similar organizations can be the most effective aid in deciding the level, source and scope of outsourcing IT project development and support.

 

19.              The accomplishment of a mission requirement itself can often be competed through outsourcing, rather than simply competing individual pieces of the project. With careful construction of “proper government function” and protection of the information being handled, vendors can be given the incentive to accomplish an end result by innovative methods.  Recent examples include the collection of delinquent taxes by commercial vendors, using a for-profit business model to govern the contract. It is becoming more feasible to outsource or contract larger individual segments of government functions.

 

20.              Outsourcing can often allow the risks to be shared with a vendor, along with the rewards. Often a vendor can share in the benefits from implementing a project, either through rights to data and/or products developed, or from revenues from services, such as dissemination. This arrangement can give a significant incentive to a vendor to be successful and receive a return on his investment in the project. Improved contracting rules and regulations have changed this area in many countries over the past few years.

 

21.              A vendor can offer unique products, such as market-leading software that is effective enough to outstrip the competition.  Careful market analysis is needed to be sure the acquisition of the product is justified and does not present excessive risks in its application. In more specialized fields this is often a requirement because of the dominance of a single or a few vendors. Geographic Information Systems (GIS) are an example of a market where proprietary systems with widely varying information formats and standards are the norm rather than the exception. This type of situation requires care not to invent something that is much more cost-effectively available in the market, recognizing that the commercial products may be unique and/or proprietary. Attempts at government “standardization” in these markets have often met with failure, or have led to the creation of a sub-optimal solution to a problem best left to the creativity and competition of the private sector.

 

22.              Finally, one of the most vexing problems in outsourcing is maintaining a healthy, productive and ethical business relationship with a commercial supplier.  Too often, government contracts provide significant penalties and “protections” for the government to impose but fail to recognize how to provide significant incentives for the vendor.  Good practice is to decide in advance what success should be for a vendor and then include that consideration in the design of the contract. Managing the business relationship in an adversarial way almost always leads to project failure at some point in larger IT projects. Conversely, most reputable vendors are in business for the long run and understand that the overall success of a project is the most important objective. Finding or recruiting managers in the census organization with the skill and experience to effectively manage an outsourced function is often as difficult as finding specialized technical skills to carry out the work internally.

D. Applying effective technologies

23.              What things does a census manager face in applying effective technologies?

 

·        Continual availability of new and different IT capabilities and products;

·        Claims by marketing organizations of amazing results in applying their products;

·        Lobbying by internal staff who have become advocates of a product or specific technology in which they have become skilled and knowledgeable; and

·        Continual pressure to reduce costs and improve delivery of products.

 

24.              The alternatives for making choices on these IT solutions are difficult, the technology is changing rapidly and the stakes are high, but census managers are not alone in this environment. The discipline brought by an effective plan and requiring a comprehensive IT business case are the best management tools to ensure success. 

 

25.              Keeping an eye on how well commercial products are faring in the marketplace is critical to avoid ending up in a “blind alley” with an orphaned product. The larger vendors generally provide more stability and support of products over the long term, while many of the smaller vendors, who rely on a narrow product line, generally have more volatile business prospects.

 

26.              When you have a solid technology management process in place in your organization, it is still a challenge to choose appropriate technologies to leverage the mission.  Generally, it is best to:

 

·        Avoid new or “immature” technologies;

·        Assess results of others, who have already applied the product;

·        Decide if the product is compatible with the existing information/technology architecture of the organization; and

·        Assess the risks and price/performance of the product.

 

27.              New and emerging technologies are notoriously difficult to assess in terms of when, or if, they will mature and become mainstream products. In the 1980s it appeared that imaging technology for document storage and retrieval was just around the corner, and several large IT companies bet their business futures on its emergence. They all lost, and that market still struggles below most of the market projections for the technology.  Geographic information systems similarly struggled in the 1980s and early 1990s, although spatial information products now seem to be achieving larger market penetration. Of course, the ubiquitous access provided by the Internet, and the associated explosion of the technology beyond almost any projections before 1995, is potent testimony to the power of a technology whose time has come.

 

28.              Learning from the experience of others with similar business issues is also a powerful tool. Visiting other countries that have adapted technology for similar functions and comparing their results can be very helpful. It is often difficult, however, to account for differences in application and operating environment from one organization to another.  It is critical to be assured that when applied successfully in one organization, the technology can be similarly applied in another. Carefully inspecting the adaptations necessary to operate in another environment can make the difference between success and failure during implementation.

 

29.              Finally, before choosing to adopt a new technology, it is critical to assess the risks and determine, conservatively, the price/performance. Remembering that the track record for such implementations is notoriously poor, it is crucial that managers be convinced that the business case is complete and solidly demonstrates a successful implementation strategy.

 

30.              The major technologies that are being adapted in population census work process are:

 

·        Collection and dissemination innovations;

·        Spatial and geographic information systems; and

·        Ubiquitous electronic access through the Internet.

 

31.              Each of these technologies will be discussed in more detail in the sections that follow.

E. Collection and dissemination technologies

1. Collection

                 

32.              One of the challenges of the IT industry has been in collecting and capturing information effectively.  Because it has been more challenging to develop technology to successfully capture the way humans communicate, whether in written form or speech, a good deal of technology has been developed to collect information more effectively with computer systems. 

 

33.              Bar-coding is the most common example of automated information collection and tracking technology.  In this case, it is relatively easy to get scanners to recognize bar codes (after all, that is what they were developed for in the first place).  The technology allows the application of a label to an object (a paper, a commodity product in a store, etc.), and subsequently the scanning at some other point (checkout counter, processing desk in paper processing, etc.).  This technology has been used effectively in tracking various forms and objects in census work, where an identification code is associated with a specific form or survey item, and its subsequent return and processing are tracked by scanning.  The commercial availability and wide application of bar-coding technology make it a good candidate for further application in census work.

 

34.              Character recognition is another technology that has been successfully applied to a wide range of collection and input processes.  This can most effectively be accomplished with forms specifically designed for scanning. It can make use of the recognition of special marks, as well as numeric, alphabetic and specially defined characters.  However, the more complex the character recognition task, the higher the error rate, and the higher the cost of human intervention for interpretation and correction. A good deal of experience has been gained over the past 10 years in applying this technology in census work (United Nations, 2001), and it is an obvious candidate for further implementation in the mid-decade assessment being discussed at this conference. At the same time, it is important to keep this application in line with the larger application of character recognition and related recognition products in the IT industry, and to carefully review more complex applications which have been custom-developed for census work. The probability of being left in a blind alley is more likely as the industry matures, and as the wider application of general character-recognition products emerges.

 

35.              The technology of information collection and capture is still not a mature field and is due to change significantly over the next five to ten years.  Managers would do well to carefully scrutinize the investments and proposals for new application of this technology over the next several years. Careful planning and risk assessment—especially the comparison of the labour-intensive but relatively straightforward manual interpretation and coding process—are considered an alternative to large-scale character-recognition technology.

 

36.              The use of the Internet to collect information directly from respondents is a technology that also holds great promise.  Survey forms, particularly in the commercial environment, have been widely implemented on the Internet and already have significantly reduced costs where the respondent is given an incentive to cope with the technology.  However, there are a number of difficult problems, including authenticity, equal access, fraud, and general information security that will have to be understood and addressed more fully over the next several years.

 

2. Dissemination

 

37.              A decade or so ago, the primary means of census dissemination was through printed publications.  The entire publishing process was an industry in itself.  The requirements for document preparation, proofing, phototypesetting, printing, and distribution were major cost and schedule components.  The development in the IT industry, enabling “desktop publishing”, has revolutionized the process.  However, the need to maintain adequate review and control over information that is officially disseminated has not actually changed much during that time.  One of the dangers of this increase in efficiency is to lose the quality control and coherence brought by more careful management that was included in several steps of the old publishing process.  If a “publication” of information is still necessary, adequate management of the presentation and content must still be maintained, while the labour and number of steps to be “product ready” have (and should have) at the same time been significantly reduced.

 

38.              Of even more importance to the dissemination of this type of information is the direct electronic access to the census information, whether in finished “published” form, or in a more complex database form with multiple ready-made views and direct access methods provided.  This type of storage and dissemination is generally more complex to manage because the provider does not necessarily know what the recipient intends or will interpret from the information.  Paper publishing generally required a fairly labour-intensive process just to capture the published data from tables, whereas general electronic publishing allows the dissemination of much more detailed and voluminous data without the costs and tedious process which existed in the paper forms.  Additionally, and this is well known but poorly managed, concerns with the privacy of raw or granular data and the potential inference of private information from this type of dissemination process are legion.  It is a delicate balance to provide detailed electronic information products, which can be analysed and correlated with other diverse information, and adequate protection for the privacy and security of the granular sources.

 

39.              In spite of these complications, the creation of “information portals”, which provide a complex data view to be put forward electronically, on demand, for subsequent combination and presentation, are a requirement which must be met by any modern population census.  Whether disseminated in a hard form, such as a CD-ROM, or in soft form on the Internet, the comprehensive organization and standards for the information content must be carefully developed and managed, or a disaster will surely follow.  The same care and professional standards that were so dutifully followed and understood in the paper publishing business must be re-created in the electronic dissemination business.  However, because of the newness and fast-changing nature of the IT environment, far less discipline and management experience is available to cope with the problem.  It is incumbent on census managers to apply their experience in this rapidly changing landscape.

 

40.              The categorization process and metadata practices for census information are comprehensive fields in themselves, but these disciplines must be carried over into the development of storage and dissemination products.  Most information technology suppliers and system developers are not experts in this field, so managing the contractual relationship between census subject-matter experts and IT suppliers and developers is more complex than in many other classes of IT projects.  At the same time, many of the niche commercial suppliers of this type of census software may not be keeping up with (or able to afford to keep up with) industry trends that provide powerful interfaces to complex information bases.  The census manager must also be wary of deviating from emerging larger-scale industry standards for storage and dissemination. 

F. Geographic information systems and related spatial technologies

41.              The past decade has seen the rapid emergence of a variety of powerful geographic information systems (GIS) and related spatial technologies. The ability to integrate a variety of information sources—from road maps to remotely sensed satellite imagery to census data—has made a new generation of commercial products widely available in the marketplace.  Several of these developments in census GIS processes are being discussed in other tracks during this conference.

 

42.              The greatest difficulties in implementing GIS applications are generally coping with the large volumes of data needed to depict geographic information and choosing effective storage and data-manipulation formats and standards.  The industry is still changing algorithms and technologies for storing and manipulating large-scale GIS data.   Integrating tabular and spatial data into a comprehensive data structure is also a problem, both in terms of intellectual structures (choosing appropriate grids or spatial units for tabular data) and in storing and presenting them comprehensively.  Some standards for data interchange have been successfully deployed, but large-scale acceptance and market penetration have not progressed as far as had been hoped a few years ago.

 

43.              During this same period, the availability of large volumes of remotely-sensed satellite imagery, with great detail, in a wide electromagnetic spectrum of sensed values, has made a wealth of new information available for both correlation with census data, and for the classification of existing structures, boundaries, terrestrial features, and environmental conditions. These developments alone have made the demand for spatially-oriented census products much larger than they have ever been in the past, and they have driven the need to categorize and present census information in literally a whole new dimension.

 

44.              Managing census data in spatial terms has also presented another dilemma for census managers—that is, considering the scope of the census work itself.  How much work in correlating base map information, physical structures, road networks, and the like is appropriate for a statistical/census organization, and how much should be left to others?  There is a vigorous industry for value-added resellers of census data in some countries, and aside from questions of statistical accuracy and quality control, how far should products and dissemination technologies go in making more information available?

G. Ubiquitous electronic access: the internet

45.              Of all the IT developments of the past 20 years, the emergence of the Internet since about 1995 has been the most astounding.  Computers have gotten faster and more powerful, personal computers have gotten into many homes, but the ubiquitous electronic access to information has revolutionized many businesses and the information industry more than any other development to date.  The effective connection to remote computers and resources around the world has made the concept of intellectual discovery and discourse change in fundamental ways.

 

46.              Access to live information, when you need it, extracted and formatted for purposes you decide at the time, is rapidly changing the way people around the world relate to information such as a population census.  Why shouldn’t I have information, which is collected on such an important topic, as it is available, in forms that I can define and use, as I need them?  If I can buy millions of products on demand 24 hours a day, view remotely-sensed weather images as they are received or review detailed financial data on the markets as they occur, why should I wait months and months for a stodgy and careful analysis of population census information?  These are questions raised by the widespread implementation of creative new products and the development of whole industries, which have become available during the Internet age.

 

47.              The ability to reach millions of people around the world via the Internet and collect information directly from them without human intervention is also too powerful to ignore. Several census organizations have already been testing such data-collection processes, but they also have to cope with the challenges presented (United Nations, 1999).  How do you ensure that bias is not introduced because of uneven access to Internet technology?  How do you reach people properly, and what is their address and identity on the Internet? How do you properly establish the authenticity of the respondent? 

 

48.              Over the next few years, census managers will be challenged to address these questions and to provide new adaptations of Internet technology to supply answers.  Additionally, challenges that emerge, such as the security and the integrity of live data on the Internet, must also be addressed.  Rampant federal government computer break-ins, denials of service, and general disruption of computer services have been causing increasing concern about whether these services can continue to operate and grow effectively unless further actions are taken to mitigate the risks.  Increasing direction for compliance can be expected in this area over the next few years, so that the capability to adapt systems to these new requirements will also have to be considered as new Internet applications are developed. 

 

49.              The interdependency of live applications on the Internet is also a concern.  If, for example, a census dissemination product requires access to other forms of economic, administrative or remotely-sensed information to function properly, who is responsible for the integrity of the entire system?  If the integrity of one data source is compromised, is the effective functioning of the others also at stake?  The reliability of all sources must be considered in one form or another when dependency on a “data portal” such as this is implemented.  At the same time, the power of such arrangements cannot be denied, so it is up to managers in charge of these processes to decide the appropriate level of risk and the strategies to balance those risks against the value of the products.

 

50.              Powerful new methods of organizing and searching through information on the Internet are being developed (Berners-Lee et al., 2001).  Several groups have been formed to develop definitions of key terms and meanings to relate information across the Internet in a dynamic way.  The census community will need to participate further in the application of technologies such as Extensible Markup Language (XML) and the Resource Definition Framework (RDF), which are key technologies in the expansion of Internet technology to knowledge management.  These technologies offer tremendous long-term opportunities to make comprehensive census information available across the Internet.

 

51.              Internet technologies represent the greatest opportunity to increase the efficiency and value of a population census, but they also present the greatest challenges.  The contractor and vendor community is also the most volatile in this area; as the market matures these sources of help can also be expected to improve.  In this rapidly emerging environment the choices are very compelling, but mistakes in judgement are very unforgiving.

H. Conclusions, lessons and best practices

52.              The adaptation of new technology to census work is a study in contrasts.  There are new opportunities to provide improved products and insights into the meaning of a population census, but there are also questions of the proper role and scope of a census organization.  At the same time, the track record of successfully implementing large IT projects in government and industry is not good.  Recent experience has shown that the most successful IT projects follow these management practices:

 

·        Involve top-level management;

·        Employ effective risk analysis in guiding direction;

·        Avoid-leading edge technologies, opting for small projects; and

·        Involve end users in project formulation and management.

 

53.              As new technologies are developed and become available, the balance needed to appropriately match the IT industry and commercial off-the-shelf products with the proper role and function of the census organizations must come from census managers as they make difficult decisions on these issues.  Most technical staffs already have a bias to implement what they know and see as opportunities in IT. Managers know the programme and its proper role, and it is the effective consideration of a well-developed business case that will best guide the adaptation of new technologies to the business of census functions.

 

I. Questions and suggestions for further discussion

1)            When considering new technologies to adapt for census work, what are the guidelines and principles for deciding the proper role and function of the census itself? 

 

2)            Where is the line for leaving further interpretation and analysis to others, including commercial sources and resellers?

 

3)            What has been the best experience with outsourcing IT and what were the success factors?

 

4)            When do you abandon a long-standing in-house software product for a newer, more generic commercial offering?  How do you cut your losses and keep the staff committed?

 

5)            How do you balance the need to have enough internal IT knowledge and skill to effectively judge the opportunities for outsourcing? 

 

6)            Can you find staff with the necessary IT technical skill and experience, who can perform effectively in a job whose role it is to “provide an environment for others (contractors) to perform in”?

 

7)            Has the Internet forever changed the role of census information products, or is it just providing more tools for interpretation?

 


References

Berners-Lee, Tim, James Hendler and Ora Lassila (2001). The semantic web. Scientific American, vol. 284, No. 5, May.

 

Organisation for Economic Cooperation and Development (2000). Management of large public information technology projects. OECD Country Reports, United States.

 

_____ (2001). The hidden threat to E-Government. Paris: OECD.

 

United Nations (1999). Conference of European Statisticians, Meeting on the Management of Statistical Information Technology, 15-18 February.

 

­­_____ (2001). ESCAP Workshop on Population Data Analysis, Storage and Dissemination Technologies, Bangkok, 27-30 March.

 

 



*       This document was reproduced without formal editing.

**     Stevensville, USA. The views expressed in the paper are those of the author and do not imply the expression of any opinion on the part of the United Nations Secretariat.

[1] United States Public Laws 103-62, 104-13, and 104-106.

[2] United States Office of Management and Budget Memorandum M-97-02 and additional guidance.