2020 Census: Census Bureau Needs to Assess Data Quality Concerns Stemming from Recent Design Changes
Fast Facts
In response to the COVID-19 pandemic, the Census Bureau changed the design of the census. After multiple timeline changes, the Bureau ultimately shortened the period for door-to-door follow-up work by 2 weeks and reduced its response processing time from 153 days to 77 days.
We note that these changes raise concerns for the quality of the data collected. For example, a shorter follow-up period could affect population counts, and shorter processing time could affect the data in other ways. The Bureau is working to assess the quality of the 2020 Census. We recommended the Bureau address data quality concerns we identified as part of that effort.
Highlights
What GAO Found
The U.S. Census Bureau (Bureau) responded to COVID-19 in multiple phases. The Bureau first suspended field operations in March 2020 for two successive 2-week periods to promote the safety of its workforce and the public. In April 2020, the Bureau extended this suspension to a total of 3 months for Non-response Follow-up (NRFU), the most labor-intensive decennial field operation that involves hundreds of thousands of enumerators going door-to-door to collect census data from households that have not yet responded to the census. At that time, the Department of Commerce also requested from Congress a 120-day extension to statutory deadlines providing census data for congressional apportionment and redistricting purposes, and the Bureau developed and implemented plans to deliver the population counts by those requested deadlines.
The Bureau implemented NRFU in multiple waves between July 16 and August 9, 2020, to ensure that operational systems and procedures were ready for nationwide use. The Bureau considered COVID-19 case trends, the availability of personal protective equipment, and the availability of staff in deciding which areas to start NRFU first.
On August 3, 2020, the Bureau announced that, as directed by the Secretary of Commerce, it would accelerate its operational timeframes to deliver population counts by the original statutory deadlines. The U.S. District Court for the Northern District of California in September 2020 issued an injunction that reversed the Secretary's August 2020 directions for design changes and the Bureau's adherence to the statutory deadlines, but the Supreme Court ultimately stayed this injunction in October 2020 and allowed the Bureau to proceed with its August 2020 design changes. As a result, the Bureau shortened NRFU by over 2 weeks and reduced the time allotted for response processing after NRFU from 153 days to 77 days.
GAO has previously noted that late design changes create increased risk for a quality census. The Bureau is examining ways to share quality indicators of the census in the near term and has a series of planned operational assessments, coverage measurement exercises, and data quality teams that are positioned to retrospectively study the effects of design changes made in the response to COVID-19 on census data quality. The Bureau is still in the process of updating its plans for these efforts to examine the range of operational modifications made in response to COVID-19, including the August 2020 and later changes.
As part of the Bureau's assessments, it will be important to address a number of concerns GAO identified about how late changes to the census design could affect data quality. These concerns range from how the altered time frames have affected population counts during field data collection to what effects, if any, compressed and streamlined post-data collection processing of census data may have on the Bureau's ability to detect and fully address processing or other errors before releasing the apportionment and redistricting tabulations. Addressing these concerns as part of the overall 2020 assessment will help the Bureau ensure public confidence in the 2020 Census and inform future census planning efforts.
Why GAO Did This Study
As the Bureau was mailing out invitations to respond to the decennial census and was preparing for fieldwork to count nonresponding households, much of the nation began closing down to contain the COVID-19 pandemic.
In response to the pandemic, the Bureau has made a series of changes to the design of the census. Understanding the chronology of events and the Bureau's decisions, along with the factors and information sources that it considered, can help to shed light on the implications and tradeoffs of the Bureau's response.
This report, the first in a series of retrospective reviews on the 2020 Census, examines the key changes that the Bureau made in response to the COVID-19 outbreak and how those changes affect the cost and quality of the census. GAO performed its work under the authority of the Comptroller General to conduct evaluations on the 2020 Census to assist Congress with its oversight responsibilities. GAO reviewed Bureau decision memos, interviewed Bureau officials, and consulted contemporaneous COVID-19 case data for context on the Bureau's COVID-19 response.
Recommendations
GAO is recommending that the Bureau update and implement its assessments to address data quality concerns identified in this report, as well as any operational benefits. In its comments, the Department of Commerce agreed with GAO's findings and recommendation. The Bureau also provided technical comments, which GAO incorporated as appropriate.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce |
Priority Rec.
The Secretary of Commerce and the Director of the U.S. Census Bureau should update and implement assessments, evaluations, and coverage measurement efforts to address the effects of the Bureau's response to COVID-19 that we identified, including data quality concerns and potential operational benefits from innovations. (Recommendation 1) |
Commerce agreed with our recommendation. In May 2021, agency officials provided us with updates of their template and guidance for the Bureau's planned operational assessments demonstrating that authors of the assessments were to include descriptions of changes to operations made due to the COVID-19 pandemic, such as schedule changes. The updates provided concern only the operational assessments, and call for itemized operational changes, but do not include any analysis of the effects of those changes on data quality. In September 2021, Bureau officials indicated that they would be preparing a thematic report on implementation of late design changes once key operational assessments are complete. In July 2023 officials provided us an outline and proof of concept for how this report would be written. In another step demonstrating partial implementation of and commitment to the recommendation, in November 2023 the Bureau demonstrated how it is accounting for COVID-19-related changes to its Demographic Analysis and Post-Enumeration Survey releases, in addition to its evaluations and experiments. In April 2024, the Bureau publicly released its thematic report on the COVID-19-related changes, complete with operation-specific considerations for data quality, and provided to us examples of potential innovations it is researching for 2030 based on the changes they made for 2020. In so doing, the Bureau has taken important steps to be transparent about its assessment of operational changes made in response to the pandemic as well as to build on those changes for future decennials where appropriate.
|