Data Quality:

Census Bureau Needs to Accelerate Efforts to Develop and Implement Data Quality Review Standards

GAO-05-86: Published: Nov 17, 2004. Publicly Released: Dec 17, 2004.

Additional Materials:

Contact:

J. Christopher Mihm
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Data from the decennial census are used to apportion and redistrict seats in the House of Representatives, distribute billions of dollars of federal funds, and guide the planning and investment decisions of the public and private sectors. Given the importance of these data, it is essential that they meet high quality standards before they are distributed to the public. After questions arose about the quality of certain data from the 2000 Census, the requesters asked GAO to review U.S. Census Bureau (Bureau) standards on the quality of data disseminated to the public.

The Bureau did not have detailed agencywide standards for the review of data from the 2000 Census to determine if the data were of sufficient quality for public dissemination. Instead, analysts and managers in different parts of the Bureau primarily used their own judgment and unwritten, program-specific guidance to decide when and whether data should be released and what supporting information should accompany the data. The lack of sufficient data quality review standards led to a variety of problems, including missed opportunities for correcting data before release, inconsistent decisions on disseminating data with similar quality issues, and inadequate communication to users about the reasons for dissemination decisions. As a result, some users of data from the 2000 Census lost confidence in the quality of the data and in the Bureau's review procedures. In the 4 years since the 2000 Census, the Bureau has publicly issued general information quality guidelines, including eight performance principles, and one new standard that allows individuals to request correction of certain errors in data disseminated by the Bureau. Both of these documents resulted from the enactment of the Information Quality Act in 2000 and the subsequent guidelines issued by the Office of Management and Budget in 2002. However, except for the one standard, the Bureau did not provide any specific guidelines or procedures on the implementation of the general guidelines. The Bureau also began work on other standards, including one on minimal information that must be provided with data and another on discussion of errors in data released to the public. Neither has been issued in final form. In response to our earlier recommendations, the Bureau created an interdirectorate working group charged with developing and publicly issuing Bureau-wide standards for quality in data releases. The working group has taken some steps, but the Bureau has not provided information on the scope or the time frame for its efforts to develop these standards. The standards that the Bureau has under development and the activities of the working group are encouraging. However, it will be important for the Bureau to proceed with greater urgency to ensure that fully tested standards are in place for the 2010 Census. Until spring 2004, no additional resources were provided to support the working group, and over a year after it began, it has not issued any new standards or said when it will be ready to do so. A comprehensive, Bureau-wide data quality framework, with interrelated standards, and specific implementing procedures could help ensure consistent decisions about the quality of the data from the next decennial census and conditions under which the data will be disseminated. Moreover, the benefits the Bureau can achieve by developing and effectively implementing comprehensive data quality standards would not be limited to the decennial census. Because they would apply to all data disseminated by the Bureau, it will be important for any new standards to be developed promptly, implemented across the Bureau, and released to the public.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: To address this recommendation, the Bureau conducted an in-depth review of the efforts that its counterparts, such as Statistics Canada, are undertaking to establish and implement a comprehensive data quality framework. This effort led to a benchmarking study of the Bureau's quality standards, for the purpose of assessing weaknesses in its data release standards and developing standards to address these weaknesses. The benchmarking report has not been finalized; however, the Bureau decided, in September 2005, to use the draft report to inform a decision to develop a comprehensive set of data quality standards. The Bureau expects to publish a comprehensive set of data quality standards on a flow basis from this effort.

    Recommendation: To ensure that the 2010 Census, the ACS, and other Census data products will provide public data users with more complete, accurate, and useful information, the Secretary of Commerce should direct the Director of the U.S. Census Bureau to accelerate the Bureau's effort to establish comprehensive data quality standards by developing and making public a detailed plan, including interim milestones, for developing such standards and procedures.

    Agency Affected: Department of Commerce

  2. Status: Closed - Implemented

    Comments: In May 2005 the Bureau released the standard on "Distribution of Census and Survey Data Products." This standard, which requires that all published data and statistical statements meet specified levels of statistical rigor, is available on the Census Bureau's Internet site. Specifically, the standard notes that the Census Bureau has the responsibility to inform its data users of the sampling and nonsampling errors in its surveys. Moreover, the Standard accommodates varying types of Census Bureau data products and the nature and extent of the information available on the types of errors that affect the published estimates. It also lays out the various offices within the Census Bureau that are responsible for implementing the standard, and where the public can submit inquiries.

    Recommendation: To ensure that the 2010 Census, the ACS, and other Census data products will provide public data users with more complete, accurate, and useful information, the Secretary of Commerce should direct the Director of the U.S. Census Bureau to include the implementation of the data quality review standards in the Bureau's plans for the 2010 Census, and test new draft guidelines on data quality review using the annual ACS test program and other surveys.

    Agency Affected: Department of Commerce

 

Explore the full database of GAO's Open Recommendations »

Oct 27, 2014

Oct 24, 2014

Oct 20, 2014

Oct 9, 2014

Oct 8, 2014

Oct 2, 2014

Sep 30, 2014

Sep 26, 2014

Sep 25, 2014

Looking for more? Browse all our products here