2020 CENSUS: Additional Actions Could Strengthen Field Data Collection Efforts
Fast Facts
If you don't return your Census form, a Census taker will interview you in person using a mobile device.
To prepare for the 2020 Census, the Census Bureau tested how well this interview process works. Our review of their 2016 tests found many non-interviews—cases where no data or insufficient data were collected. The Bureau is researching potential causes.
We also found some cases where better procedures or more training may improve the process. For example, Census takers could not access recently closed incomplete cases on their mobile devices—and were therefore unable to interview a household when people returned home.
A Non-response Follow-Up Visit
A census worker in a doorway using a mobile device to collect data from a household member
Highlights
What GAO Found
The Census Bureau (Bureau) recently completed its 2016 Census Test in Los Angeles County, California, and Harris County, Texas. One primary focus of the test was to assess the methodology for non-response follow-up (NRFU), where enumerators personally visit households that do not self-respond to the census. GAO found that during the 2016 Census Test, NRFU generally proceeded according to the Bureau's operational plan. However, data at both test sites indicate that the Bureau experienced a large number of non-interviews. Non-interviews are cases where either no data or insufficient data are collected. Bureau officials are not certain why there were so many non-interviews for the 2016 Census Test and are researching potential causes. Going forward, it will be important for the Bureau to better understand the factors that contributed to the non-interview rate because of its relationship to the cost and quality of the census.
GAO also found that refining certain enumeration procedures and training enumerators better could produce additional efficiencies by enabling the Bureau to be more responsive to situations enumerators encounter on the ground. For example, enumerators, by design, were unable to access on the mobile device recently closed, incomplete cases. Bureau officials acknowledged that closing cases in this fashion represented a missed opportunity and plan to test greater flexibilities as part of the 2018 End-to-End Test. Programming some flexibility into the mobile device—if accompanied with adequate training on how and when to use it—should permit enumerators to complete some interviews and reduce the cost of follow-up attempts. Further, enumerators did not always understand procedures for visiting property managers in multi-unit buildings. Specifically, the 2016 Census Test demonstrated that vacant units could quickly be removed from the NRFU workload where a property manager was readily available to provide that information; however, in other cases the procedures confused enumerators and they did not understand how to proceed. Without the knowledge of which units were vacant, enumerators may have unnecessarily visited some vacant units and thereby increased the cost of NRFU.
During GAO's field visits, GAO encountered several instances where enumerators learned that returning at a specific time on a later date would improve their chance of obtaining an interview from either a household respondent or a property manager. However, the Bureau's 2016 Census Test and automated case management system did not have an efficient way to leverage that information. Attempting contact at non-responding households at times respondents are expected to be available increases the completion rate and reduces the need to return.
Why GAO Did This Study
With a life-cycle cost of about $12.3 billion, the 2010 Census was the most expensive enumeration in U.S. history. To help control costs and maintain accuracy, the 2020 Census design includes new procedures and technology that have not been used extensively in earlier decennials, if at all. While these innovations show promise for a more cost-effective head count, they also introduce risks. As a result, it will be important to thoroughly test the operations planned for 2020.
The objective of this report is to assess key NRFU operations performed during the 2016 Census Test to identify any lessons learned that could have a potential impact on pending design decisions for the 2020 Census. To assess NRFU operations GAO visited both test locations and observed enumerators conducting NRFU interviews, and reviewed relevant documents including the test plan and enumerator training manuals.
Recommendations
GAO recommends the Secretary of Commerce direct the Bureau to: (1) determine causes for non-interviews, and revise and test what, if any, changes need to be made to operational procedures and training; (2) revise and test procedures and training on accessing closed cases, (3) revise and test procedures and training for initial property manager visits; and (4) revise and test procedures and training for how to use enumerator-collected data on the best time or day to conduct an interview. The Department of Commerce agreed with GAO's recommendations, and the Bureau provided technical comments that were incorporated, as appropriate.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce |
Priority Rec.
The Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to determine the cause(s) for non-interviews experienced during the non-response follow-up operation and revise and test what, if any, changes need to be made to operational procedures, training, or both, including making contact with proxy respondents. |
Commerce agreed with this recommendation. In the Bureau's March 2017 action plan, it reported that analysis of the 2016 Census Test data was underway and that the 2016 Census Test experiences and the ongoing 2016 Census Test data analysis would inform the Bureau's evolution of operational methods and development of revised training content. The Census Bureau anticipated that changes to the non-response follow-up operation in the 2018 End-to-End Test would result in more completed interviews. As of January 2019, Bureau officials told us that the 2018 End-to-End Test also experienced high non-interview rates and that officials during their review identified several contributing factors including insufficient enumerator ownership over completing cases with full data collection, insufficient management tools for implementing late-operation data collection, and confusion over proxy interview procedures. Officials provided us with a list of changes to be made to training, procedures, and business rules for following up on non-respondents to the census, which they expect to help reduce non-interview rates. The Bureau's analysis of the 2018 test and these related planned actions should help mitigate the high non-interview rates and improve the quality of the census.
|
| Department of Commerce | The Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to revise and test operational procedures for accessing incomplete closed cases and revise and test training material to reflect when this flexibility to access incomplete closed cases should be used by the enumerator. |
Commerce agreed with this recommendation. In the Bureau's March 2017 action plan, it reported that the Bureau had identified enhancements to methods to address this recommendation and would consider how procedures can provide flexibility, and how the enumerator training can evolve to communicate the availability of this flexibility, and when the capability should be leveraged. It reported that how the software solutions will reflect these enhancements will inform the development of the procedures and training used by enumerators for the 2018 Census End-to-End Census Test. It reported a target completion of December 2018 following completion and analyses of the nonresponse follow-up operation for the 2018 End-to-End Census Test. In January 2018, the Bureau told us they had developed new procedures for the 2018 End-to-End Test that will allow the enumerator to open closed cases and would provide us with those procedures once finalized. During the 2018 Test, Bureau officials described to us the flexibility that enumerators had for accessing these types of cases, yet we found that not all enumerators we observed were consistently aware of how to do so. In order to fully implement this recommendation, as of June 2019 we await the Bureau providing additional information on how it will ensure that enumerators know how to use this flexibility. In July 2019, the Bureau provided us with copies of training material clearly describing this functionality, including an illustrative circumstance much as we had originally observed. The Bureau's provision to enumerators the flexibility to access recently closed but incomplete cases at times such as when encountering a recently-missed respondent will help the Bureau complete its non-response follow-up workload more quickly and at lower cost.
|
| Department of Commerce | The Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to revise and test operational procedures and relevant training materials for initial property manager visits to ensure procedures and training material are communicated to and understood by enumerators and their supervisors. |
Commerce agreed with this recommendation. In the Bureau's March 2017 action plan, it reported that based on its 2016 Census Test experiences, it had identified modifications to the system capabilities and procedures for initial multi-unit manager visits and that it would implement and further study these changes in future tests, including the 2018 End-to-End Census Test. It reported a target completion in December 2018, following completion and analyses of the nonresponse follow-up operation for the 2018 End-to-End Census Test. In January 2018, Bureau officials told us they were developing new procedures for multi-unit manager visits. We observed related issues continuing during the 2018 Test. As of June 2019, Bureau documents indicate that the Bureau is taking steps it believes will reduce the number of multi-unit manager visits required. In order to fully implement this recommendation, the Bureau needs to take steps to better ensure that procedures and training material are communicated to and understood by enumerators and their supervisors. In July 2019 the Bureau provided us with training materials for 2020 Census that clearly describe how a manager visit is to be conducted, including a small role-playing exercise. Taking steps to ensure that enumerators understand how to conduct these potentially time-saving visits during non-response follow up will help the Bureau control the cost of the 2020 Census.
|
| Department of Commerce | The Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to revise and test procedures on how to better leverage enumerator-collected information on the best time or day to conduct interviews, and ensure enumerators are properly trained on these procedures. |
Commerce agreed with this recommendation. In January 2018, Bureau officials told us that leveraging enumerator collected information on the best time to conduct an interview would not be a part of questionnaire design's functionality for 2020 and will most likely rely on the system optimizer to determine the best time to contact a household. Bureau officials indicated they may push this recommendation out to 2030. During the 2018 Test, we continued to observe that enumerator notes were not being systematically reviewed by supervisors and managers, such that information such as office hours for apartment managers among other information we reported on was not being used by the operation. In June 2019, the Bureau informed us that it had added functionality to the enumerator device for enumerators to alert their supervisors (CFS) of case notes of "high importance" and was revising training to explain its use. Additionally, in December 2019, the Bureau shared documentation that reiterated the importance of enumerators reviewing prior case notes but clarified that enumerators should not expect their supervisors to regularly review those case notes. However, in April 2020, Bureau officials indicated that it was not likely that the Bureau would revise CFS training to systematically review case notes for the 2020 cycle, and in March 2021 we reported on the Bureau's use of the optimizer to assign and route cases using an algorithm that identifies the best time of day to enumerate. As of March 2024, we await Bureau officials to conclude analysis related to how field data collection will be managed for the 2030 Census, expected by the end of September 2024. To fully implement this recommendation for future fieldwork later in the decade, the Bureau needs to make better use of the information collected by enumerators during interview attempts about when to make additional attempts, such as during the reported working hours of property managers for large multi-unit structures that house a large number of non-respondents and revise CFS training to require that CFS review case notes.
|