BUILDING INSTITUTIONAL CAPACITY TO MAINTAIN THE INTEGRITY AND LEGITIMACY OF THE NATIONAL STATISTICAL SERVICE
Australian Bureau of Statistics (ABS)
Public confidence in the integrity and legitimacy of the National Statistical Service is one of the foremost challenges for the Director of the NSO. Unless they have such confidence, they have not really succeeded in their endeavours.
Furthermore, the credibility of the statistical system plays a fundamental role in the value to the users of their statistical outputs. This is because few users can validate directly the data released by statistical offices. Instead they rely on the reputation of the provider of the information. The value of its statistical outputs therefore depend directly on the credibility of the statistical system.
In Australia, we believe the ABS has achieved that respect as evidenced by the following quotes by senior commentators.
"There are many public sector employers and institutions deserving of praise ... The Australian Bureau of Statistics ... is one of the great governmental organisations in Australia".
"The ABS data has a deserved reputation of being of high integrity. This does not mean that they are always error free, and it does not mean that the statistics should be used in an unquestioned way, but it does mean that the user knows that they have been collected and processed professionally and that the statistics have not been fudged for expediency ... Integrity is one quality that the ABS must continue to treasure."
The trust in our work is a great asset to us. However it is an ongoing task to maintain that trust. It cannot be taken for granted. It can be easily lost. Complacency may be a cause. That can make you more vulnerable to forgetting the importance of core values, or being at risk of a major incident (eg security breach, releasing confidential data, or significant errors in key statistics). These incidents are likely to lead to a poorer profile and a deteriorating relationship with the media. We should also not forget the opinions of key stakeholders, including those at the political level, can be influenced very much by the media.
Good financial management is also fundamental to trust. Poor financial management, particularly on major projects, can lead to a lack of respect for the NSO.
In this short paper, I have addressed some of the key elements in having the institutional capacity to maintain the integrity and legitimacy of the national statistical service. They are in turn (a) legislation, (b) profile and broad community support, (c) maintenance of core values, (d) sound governance arrangements, (e) a good planning system (f) good resource management, (g) the trust and confidence of respondents, and (h) professionalism.
A lot of the comments in the paper refer to the practices of the Australian Bureau of Statistics, but hopefully much is relevant to the NSOs of the ESCAP region. Nevertheless, it may be useful to rewrite in terms that are more relevant to the situation of developing countries.
The common aspirations of national statistical offices are embodied in a set of principles adopted by the United Nations Statistical Commission in 1994. (See Appendix 1). The aim of these principles is to engender trust and credibility in the statistics agency.
The United Nations Statistical Office has published model legislation for those who wish to develop or redevelop statistical legislation. This will often be a good starting point.
Sound legislation is crucial to a good statistical system.
· It provides legitimacy and transparency to statistical operations.
· It is necessary to provide public confidence in the statistical system - however, it is not sufficient; that depends on the actions of Ministers and senior people in the statistical system;
· It provides for continuity of arrangements as key personnel change, often with different ideals and backgrounds.
What are some key elements in statistics legislation?
(a) the authority to collect statistical data including powers to compel respondents to provide data under certain circumstances;
(b) access to administrative data (eg customs, births and deaths, tax) for statistical purposes to ensure the statistical benefits of such data are given due credence;
(c) confidentiality provisions to help maintain the trust and confidence of respondents;
(d) outlining those circumstances where non-standard confidentiality practices may occur (eg support for research) - these should not exceed what the public regards as acceptable;
(e) the governance arrangements for the national statistical system;
(f) arrangements for ensuring the independence of the NSO and the authority of the head of the NSO;
(g) the appointment and removal arrangements for the head of the NSO so that they cannot be removed at the whim of the government;
(h) the functions of the NSO, particularly in the context of a national statistical system (ie statistical coordination responsibilities), also the specific responsibilities of the head of the NSO;
(i) any specific responsibilities of the Minister - these should not be extensive if the NSO is to operate with a reasonable degree of independence; and
(j) a legal requirement to publish outputs from statistical collections.
Ideally statistics legislation should cover each of these points. It should be emphasised that, to be effective, breaches of the legislation should not be ignored. They should be followed through in some way - how will depend on the circumstances.
3. Profile and Broad Community Support
The relationship with the media is crucial. They can have a very significant influence on key opinion leaders. Furthermore, most of the community find out about statistics through the media - newspapers, radio and television.
At the ABS, we have found it to be to our significant advantage to help the media to do their job. Media releases and access to statistical publications are important. We have also found it to be very advantageous to be open to the media - to provide access to statistical experts and to encourage statistical experts to talk to the media. It is very useful to train these staff on how to deal with the media. Training the media in basic statistical concepts (eg sample errors, seasonal adjustment) can also be useful.
Criticism in the media is taken seriously. If the criticism is unfair, we will respond (eg letter to the Editor). If it is valid, we try to respond by changing what we are doing.
Consistent branding of the national statistical office is important for profile. There are many opportunities - through publications; by insisting on citation by the media when using NSO statistics; at the time of high profile collections such as the Population Census; presentations at Conferences; media presence by senior staff; etc.
Senior staff have a strong presence at the relevant Conferences to facilitate communication and networking particularly with statistical users. A special bilateral relationship should be established with the key users, especially the policy agencies. We refer to them as "lifeline" clients because of the influence they can have on government attitudes and support for the statistical agency. In fact, working through them and their committees may be the most effective way of influencing governments on statistical matters.
Last but not least, you must be a well managed and trusted organisation. If not, you may have a profile but not the type of profile you are seeking!
4. Maintenance of Core Values
A key part of the ABS culture is that it is value driven. The strength of this is reinforced by management consultants that work with us. We believe values are extremely important in influencing the behaviours of statistical staff.
Core values cannot be just statements hanging on the wall. They have to be understood. They have to be reflected in behaviours, particularly by the leaders of our organisations.
More than anything, core values distinguish us from other statistical providers in Australia. The core values are:
· Relevance - regular contact with those with policy influence, good statistical planning, which requires a keen understanding of the current and future needs for statistics, are essential, as is the need for statistics to be timely and relatable to other statistics.
· Integrity - our data, analysis and interpretation should always be objective and we should publish statistics from all collections. Our statistical system is open to scrutiny, based on sound statistical principles and practices.
· Access for all - our statistics are for the benefit of all Australians and we ensure that equal opportunity of access to statistics is enjoyed by all users.
· Professionalism - the integrity of our statistics is built on our professional and ethical standards. We exercise the highest professional standards in all aspects of ABS statistics.
· Trust of providers - we have a compact with respondents; they are to encourage us to provide accurate information and we ensure that the confidentiality of the data provided is strictly protected. We keep the load and intrusion on respondents to a minimum, consistent with meeting justified statistical requirements.
Similar core values may be appropriate for other NSOs.
5. Sound Governance Arrangements
This is relevant to both external and internal governance arrangements. Externally, arrangements should be established for the whole statistical system not just the NSO. It is inevitable that there will be a range of coordination and other issues that need to be resolved across the whole statistical system. Hopefully these can be resolved by cooperation but, if not, it will be necessary to put some dispute resolution process in place. A National Statistics Council can be effective as part of the institutional arrangements.
Internally, good governance is essential for good management. Some key elements are:
· An Executive that will focus (perhaps at different times) on strategy, accountability, policy and financial issues and make/disseminate the key decisions resulting from these discussions;
· Senior management committees that focus on the key issues (eg security, human resources, IT resources, publishing);
· Regular review of processes (with some external representation from time to time);
· A good planning system that covers both the short term and long term;
· Regular reporting usage of finances and other key resources; and
· A framework for ensuring good management of projects.
Within Australia, the ABS is regarded as a well managed organisation. I am asked from time to time for the reasons. I provide the following response.
· Legislation which supports the operations of the ABS, its responsibilities and statutory independence;
· Continual interaction with the users of our statistics;
· An effective Advisory Council;
· Internal governance arrangements which are well understood and address strategic, accountability and tactical issues;
· Effective planning systems;
· Service charters for our dealings with clients and business respondents;
· Continuing review of our performance and its cost-effectiveness; and
· Significant devolved responsibility for personal and financial management.
6. A Good Planning System
A good planning system must set the long term directions for the organisation. This will help determine priorities for capacity building. It may also assist with "government buy-in" particularly when seeking budget assistance. They are more likely to be supportive of the long term directions of the NSO if they know what they are.
Many statistical offices are reliant on funding from donors to undertake some of their activities. One criticism you hear is that the statistical activities that donors want to fund are not necessarily compatible with national priorities. The existence of a long term plan for the NSO makes it easier to negotiate with donors on outcomes that are mutually beneficial.
One important element of a long term plan is the organisational mission. The ABS's mission is:
"We assist and encourage informed decision-making research and discussion within governments and the community, by providing a high quality, objective and responsive national statistical service."
This was established 15 years ago. The key changes to our behaviour at that time were the emphasis on "informed decision making" and that "community" was a key target group for the national statistical service. Informed decision making, among other things, meant we put more emphasis on analysis, dissemination of statistics and ensuring statistics were used effectively. The focus on community resulted in putting special emphasis in getting information to the community through the media, public libraries, schools and more recently through the web site. The discussions leading up to the development of the mission statement did have a big influence on our work.
The long term plan should also identify the strategic issues that the organisation should focus on - these will depend on the state of the development of the NSO and may usefully be reviewed every three or four years whereas you would expect the mission statement to change much less frequently.
For the short term, a good planning system will record the resource allocations by program over the next few years. Workforce planning will also be required to ensure people with the right skills will be available. The system should also record the planned outputs from the budget allocations whether they be statistical outputs or some other form of output (eg completion of a research program, a major system enhancement).
7. Good Resource Management
The resources of a NSO are many but include people, finance and technology. Without doubt people are the most important resource. It is essential to have staff who know what has to be done, have the required skills, the equipment and facilities to complete the task and the incentives to perform well.
There are many books written about good people management so I will not even attempt to summarise in this paper.
I will just make one point. It is essential that all staff, but particularly the senior staff, understand the mission and core values, and understand the reasons why they exist. It is very difficult to be an effective contributor unless you understand the role of your NSO and believe in its importance. These aspects are continually reinforced at the ABS commencing with the induction programs for new staff.
Similarly, it would not be appropriate to summarise what makes good financial management in a short paper like this. Sufficient to say that it is necessary to have good financial reports that cover both operating and capital expenditure so that you understand what is happening and receive signals as to future problems.
Technology is a great enabler to do new things and improved productivity, but
· it can be very expensive;
· IT projects are subject to very high failure rate; and
· technology can increase risks (eg security).
The following is a brief summary of what we have found to be good management practices with respect to technology.
· Involve top management - include the Chief Information Officer in the management team.
· Ensure IT staff have strong alignment with the activities of the NSO.
· Be business like in considering IT investment proposals. Only proceed if the benefits justify the costs, after taking account of the risks.
· Have an agreed enterprise architecture to guide acquisitions and developments. Buy rather than build wherever possible.
· Stay with mainstream products. Avoid leading edge (non-proven) technologies. For example, some NSOs have had difficulties using imaging technology without adequate support.
8. The Trust and Confidence of Respondents
An official statistical agency must maintain good relations with respondents if it wants them to co-operate and provide high quality data. The ABS approach includes - explaining the importance of the data to government policy, business decisions and public debate; a policy of thoroughly testing all forms before they are used in an actual survey; obtaining the support of key stakeholders; minimising the load placed on respondents particularly by using administrative data where possible; and carefully protecting privacy and confidentiality.
The ABS monitors and manages the load it imposes on both households and businesses; we have developed 'respondent charters' for both groups. As well, a Statistical Clearing House has been set up within the ABS to coordinate surveys of businesses across government agencies (including the ABS), to reduce duplication and to ensure that statistics of reasonable quality are produced.
All ABS forms and collection methods are tested to ensure that the data we seek are available at reasonable cost, and the best available ways are used to collect them. For business surveys, our units model, classifications and data items, are designed to be as consistent as possible with the way businesses operate. This now corresponds closely with reporting for taxation purposes, making it easier to integrate survey data with data collected for taxation purposes. For household surveys, the extensive use of cognitive testing tools within the ABS, and the establishment of a questionnaire testing laboratory, have helped to improve quality and to reduce respondent load. Standards for form design and form evaluation are set out in manuals and are promoted and supported by experts in form design.
The ABS uses efficient survey designs to minimise sample sizes and hence total reporting load; we also control selection across collections to spread the load more equitably. To take advantage of the current reforms of the Australia taxation system, the ABS is seeking every opportunity to improve the efficiency of our sample designs, and to use taxation data as a substitute for some of the data now gathered through direct collections. As mentioned above we have started the process of simplifying the business unit structure used in our surveys to make it consistent with the structure used for taxation purposes.
The Fundamental Principles of Official Statistics underpin the practices of statistics agencies including: decisions on the range of statistical outputs that constitute the agency's work program; the way in which these statistics are collected and compiled; and the manner in which they are disseminated. In determining the nature and scope of the statistics to be produced, agencies need to be conscious of the policy process but at the same time independent of it. They must be objective and impartial. Their statistics must be relevant to the key policy questions and contemporary issues in society and they must be able to respond to emerging information needs of clients if they are to properly inform discussion and debate, and in so doing contribute to the democratic process. My predecessor, Bill McLennan, once observed
"... an indicator of how well a democracy is working is shown by the degree of independence the official statistical agencies are accorded, and by their performance in producing relevant and timely statistics".
The credibility and reputation of a national statistics agency rests on the production of high quality statistics that are acknowledged as such by users. To achieve this an agency needs to go about its business in a professional way. It must use sound methodologies and statistical practices which achieve a balance between cost, quality and timeliness. It should be involved in developing and applying statistical standards, both national and international and ensure that statistics are compiled in an objective and impartial manner. Concepts and procedures used should be described and documented so they are open to scrutiny. Limitations of any of the methods used should be acknowledged. Information on the quality of the statistics (both sampling errors and other sources of error) should be provided.
The manner in which statistics are disseminated is equally important because it is through this process that statistics are converted into information. First and foremost statistics must be presented in a professional manner and accessible to all under equal conditions. Statistical agencies should adopt dissemination practices which inform and explain without advocating a particular position. They should provide analysis and interpretation to assist in understanding the statistics. In addition, they should describe the context surrounding the issues which the statistics address.
It can be a fine line between explanation and advocacy. A consciousness of this on the part of statistical agencies coupled with a desire to ensure that statements are properly qualified to guard against misinterpretation can result in a convoluted and/or boring way of writing about the statistics. However this does not have to be the case. Social trends publications produced by several agencies are proof of this, although criticism of lack of objectivity are more likely in these types of publications.
Finally in disseminating their statistics national statistical agencies have a role in commenting on erroneous interpretations and misuse. This may take a number of forms including special briefings or seminars for particular users; modifying the presentation in publications to minimise the scope for misinterpretation; providing additional documentation or analysis; preparing special articles; writing letters to the editor.
10. Issues for Discussion
This paper is a brief summary of my views of the key elements of the institutional capacity required to maintain the integrity and legitimacy of the National Statistical System. The following issues may be worth discussing.
1. Do you agree that all the elements mentioned above are important? Is there anything missing?
2. What can the international agencies, particularly those that operate in the region, do to help you strengthen your institutional capacity? Which of the key elements are most in need of attention?
3. Is there a role for peer reviews to help you identify the strengths and weaknesses of your institutional arrangements? Will they assist where it is necessary to persuade governments to change policy on statistical matters.
4. Are there any particular integrity and legitimacy issues from your country you would like to discuss?
FUNDAMENTAL PRINCIPLES OF OFFICIAL STATISTICS
(Adopted by the UN Statistical Commission at its meeting 11-15 April 1994)
1. Official statistics provide an indispensable element in the information system of a democratic society, servicing the government, the economy, and the public with data about the economic, demographic, social, and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens' entitlement to public information.
2. To retain trust in official statistics, the statistical agencies need to decide, according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage, and presentation of statistical data.
3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods, and procedures of the statistics.
4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.
5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to chose the source with regard to quality, timeliness, costs, and the burden on respondents.
6. Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.
7. The laws, regulations, and measures under which the statistical systems operate are to be made public.
8. Co-ordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.
9. The use by statistical agencies in each country of international concepts, classifications, and methods promotes the consistency and efficiency of statistical systems at all official levels.
10. Bilateral and multilateral co-operation in statistics contributes to the improvement of systems of official statistics in all countries.