2010 Census:

Follow-up Should Reduce Coverage Errors, but Effects on Demographic Groups Need to Be Determined

GAO-11-154: Published: Dec 14, 2010. Publicly Released: Dec 14, 2010.

Additional Materials:

Contact:

Robert N. Goldenkoff
(202) 512-2757
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The U.S. Census Bureau (Bureau) puts forth tremendous effort to conduct a complete and accurate count of the nation's population and housing; yet some degree of error in the form of persons missed, duplicated, or counted in the wrong place is inevitable due to the complexity in counting a large and diverse population. The Bureau designed two operations, Coverage Follow-up (CFU) and Field Verification (FV), to reduce certain types of counting, or coverage, errors in the 2010 Census. GAO was asked to assess (1) the extent to which the Bureau completed CFU and FV on schedule and within estimated cost and (2) the implications of their key design elements for improving coverage. GAO reviewed Bureau evaluations, planning, and other documents on CFU and FV, and prior GAO work, and interviewed Bureau officials.

The Bureau completed CFU and FV on schedule and within budget. FV cost $21 million (about 38 percent less than estimated) and CFU cost about $267 million (about 2 percent less than estimated). These operations followed up on potential errors on census returns or lists of addresses after census data had been initially collected. Their completion provided follow-up data used by subsequent data processing that removed errors from the official census tabulations. Three of the Bureau's key CFU design elements will likely improve overall census accuracy, but their effect on undercounts of different demographic groups is not clear. One key design element increased the number and types of follow-up cases. The Bureau expanded the scope of CFU from about 2 million households in the 2000 Census to more than 7 million in 2010. It also added 20 different types of households for potential follow-up. New types included households that reported members temporarily residing elsewhere, such as at college, in nursing homes, or in jail. According to the Bureau, the 2010 CFU operation should remove more than 2.7 million coverage errors from the census. Another key design element of CFU prioritized follow-up cases based on their likelihood to result in a census correction, which was a reasonable attempt to leverage the resources for the operation. However, the Bureau's evaluation plans, based on considerations of what may best reduce cost or increase accuracy in the future, do not link the demographic characteristics of households to how they responded to the additional questions or CFU results for those households. Therefore, it is unclear whether the prioritized follow-up will help reduce differences in the accuracy of census counts across demographic groups. Finally, CFU's design relied on a telephone-only approach to complete follow-up rather than personal visits. This limited costs, resulting in more follow-up and likely more coverage errors being removed from the census. But the telephone-only decision excluded about 700,000 households from CFU that could not be contacted by telephone. Prior Bureau experience indicates that some historically undercounted groups were less likely to be reachable by telephone, and more recent independent research suggests that trends in telecommunication usage may also make it harder to reach some demographic groups this way in the future. Yet the Bureau's evaluation plans do not include an assessment of either the usefulness of the telephone numbers it collected in reaching specific groups or the effect of these trends. Greater understanding of how best to reach different groups as well as the influence of trends on the effectiveness of CFU could help to control costs while working to further reduce differential undercounts. GAO recommends that the Secretary of Commerce direct the Bureau to assess (1) how well questions to help identify miscounted people on census forms helped reduce differences in the undercounts between demographic groups; (2) the degree to which telephone numbers led to completed contacts for households of various demographic characteristics; and (3) how trends in telecommunication usage and new technology may influence the effectiveness of CFU. The Secretary of Commerce concurred with our recommendations.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In September 2013, the Bureau provided GAO with a draft research study describing the relationship between responses to specific probes and the characteristics of the responding households. This commitment to understanding how the Bureau's design of the operation affected census coverage of different demographic groups will help the Bureau as it designs more cost-effective operations for 2020 Census.

    Recommendation: To improve the Bureau's planning for the 2020 Census, and to help the Bureau decide which coverage probes, if any, to use and prioritize for future follow-up efforts, the Secretary of Commerce should require the Director of the U.S. Census Bureau to assess the extent to which historically overcounted and undercounted demographic groups responded to the probes the Bureau followed up on and determine the effectiveness of specific probes in reducing differential undercounts.

    Agency Affected: Department of Commerce

  2. Status: Closed - Implemented

    Comments: The Bureau requested additional data from the Decennial Response Integration System contractor to support the recommended analyses. As a result, in August 2011 the Bureau provided us with the results of an assessment of the usability and success of the telephone numbers it used. This commitment to understanding how the Bureau's design of the operation worked to achieve Bureau goals will help the Bureau as it designs more effective operations for 2020 Census.

    Recommendation: To improve the Bureau's planning for the 2020 Census, and to support the Bureau's efforts to control costs while improving census accuracy, the Secretary of Commerce should require the Director of the U.S. Census Bureau to determine the demographic characteristics of the households for which it did and did not obtain telephone numbers and, to the extent feasible, assess the degree to which the telephone numbers were usable and led to completed contacts for households of various follow-up groups and demographic characteristics.

    Agency Affected: Department of Commerce

  3. Status: Closed - Implemented

    Comments: In January 2013, the Census Bureau 2020 Research and Testing Contact Frame Team conducted a test to understand the coverage and completeness of telephone numbers and e-mail addresses acquired through commercial sources. Included in the test was use of a service that indicated whether a telephone number was a landline or a cell number. Preliminary results indicate 30% of the calls completed were to cell phone numbers. In addition, the team plans additional analysis of the call contact history of the 2010 Census Coverage Followup Operation. The Bureau is in the process of obtaining the necessary files to determine phone classifications (cell or landline) and plans to do further analysis of the phone numbers associated with the 2010 Census Coverage Followup Operation. In addition, the Bureau will be able to look at what cell phone numbers it has been able to obtain through its administrative record third party data providers with a 2010 vintage and determine how many of the CFU cases might have benefited from the use of cell phone numbers. The project will also be reviewing respondent reported telephone numbers from the 2010 Census and the last three years of the ACS to determine the rates at which respondents have provided cell phones. In September 2013 the Bureau provided GAO with a draft research study analyzing cases where a phone number was used from a commercial database or from the original census respondent, looking at demographic distributions and other characteristics of the household and informing what effects different trends in telecommunication, such as cell phone usage, might have on census coverage of various demographic groups. This commitment to understanding the effect that changing technology and its use can have on census operations and census coverage will help the Bureau as it designs more effective operations for 2020 Census.

    Recommendation: To improve the Bureau's planning for the 2020 Census, and to ensure that the design of future follow-up efforts is effective in improving census coverage, the Secretary of Commerce should require the Director of the U.S. Census Bureau to assess the implications that trends in landline and wireless usage and other modes of communication and new technology may have both on the design decisions for future CFU-like operations and on their effectiveness in improving census coverage in terms of both overall census accuracy and differential undercounts.

    Agency Affected: Department of Commerce

 

Explore the full database of GAO's Open Recommendations »

Oct 24, 2014

Oct 20, 2014

Oct 9, 2014

Oct 8, 2014

Oct 2, 2014

Sep 30, 2014

Sep 26, 2014

Sep 25, 2014

Sep 19, 2014

Looking for more? Browse all our products here