This is the accessible text file for GAO report number GAO-13-53 
entitled '2020 Census: Initial Research Milestones Generally Met but 
Plans Needed to Mitigate Highest Risks' which was released on December 
7, 2012. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

November 2012: 

2020 Census: 

Initial Research Milestones Generally Met but Plans Needed to Mitigate 
Highest Risks: 

GAO-13-53: 

GAO Highlights: 

Highlights of GAO-13-53, a report to congressional requesters. 

Why GAO Did This Study: 

GAO’s prior work has shown that it will be important for the Bureau to 
re-examine the design of the census in order to ensure a cost 
effective census in 2020. As requested, this report evaluates the Bureau
’s efforts to improve the cost-effectiveness of the enumeration, 
paying particular attention to the following three key efforts: (1) 
leveraging the Internet to increase self-response; (2) improving how 
the Bureau builds its address and mapping databases, including a 
possible move to targeted address canvassing, and use of private-
sector geographic data; (3) and using administrative records to reduce 
nonresponse follow-up costs. This report (1) identifies what 
opportunities and risks, if any, the Bureau might need to consider for 
these efforts going forward and (2) examines to what extent these 
three efforts are on track with respect to scheduling, resources, and 
other performance metrics. To meet these objectives, GAO reviewed 
Bureau documents and interviewed officials. 

What GAO Found: 

According to U.S. Census Bureau (Bureau) officials, to inform timely 
design decisions, research on new methods to improve the cost 
effectiveness of the 2020 Census must be accomplished early enough in 
the decade to confirm their likely impact on both cost and quality. 
Three key efforts—-(1) the use of the Internet as a response option, 
(2) a potential move towards targeted address canvassing, and (3) the 
possible use of administrative records to replace data collected 
during census field operations-—present the Bureau with potential 
opportunities to reduce costs while maintaining quality. The Bureau’s 
2020 Research and Testing Program has 14 fiscal year 2012 projects 
focused on informing design decisions related to the three key 
efforts. Bureau officials are also aware that the changes they are 
testing come with many risks, and for each project the Bureau has 
identified a number of risks and prioritized them from high to low. 
However, the Bureau has not developed mitigation or contingency plans 
for these project risks. For example, there are several risks, 
including tight time frames and accurate cost information, without 
mitigation and contingency plans. Additionally, GAO found that the 
Bureau had not developed cost estimates for any of its 2020 research 
and testing projects as required in guidance provided to project 
teams. Unreliable cost estimates was one of the reasons the 2010 
Census was placed on GAO’s high-risk list. Without timely cost 
estimates, it will be difficult for the Bureau to ensure that 
resources are adequate to support the research and testing program. 

The Bureau met its internal deadline for submitting each of its 14 
research project’s plans and charters. However, not all the project 
plans were complete. For example, some project teams did not fully 
document the types of skills needed or perform a skills gap assessment 
to determine the resources needed to carry out their respective 
projects as required in the Bureau’s planning template. According to 
Bureau officials, they are working to document these skills sets, as 
well as any gaps in skills. Completing this analysis is important to 
ensuring sufficient resources are available for conducting the 
research and testing projects. Additionally, performance metric 
documentation for several projects was incomplete. According to Bureau 
guidance, project teams were to provide performance metrics for 
measuring progress and for determining the project’s final outcome. 
However, one team did not provide either of these required performance 
metrics, while six other teams did not include performance metrics 
that could be used to monitor research and testing progress. Absent 
these metrics, the Bureau does not have the assurance that it will be 
able to avoid potential schedule slips or the certainty as to whether 
project outcomes will be adequate for making decisions. 

What GAO Recommends: 

GAO recommends that the Acting Census Director take a number of 
actions to improve the Bureau’s research and testing for the 2020 
Census, such as developing risk mitigation plans, contingency plans 
and cost estimates for each project, and performance metrics and skill 
sets for those projects that do not have them. The Department of 
Commerce concurred with GAO’s findings and recommendations and 
provided one clarification, which was included in the final report. 
Also, the Bureau in its comments noted that it has begun to address 
GAO’s recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-13-53]. For more 
information, contact Robert Goldenkoff at (202) 512-2757 or 
goldenkoffr@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Key Efforts Present the Bureau with Both Opportunities and Risks: 

Research and Testing Project Plans Were Delivered on Schedule, but Not 
All Project-Level Resources and Performance Metrics Have Been 
Identified: 

Conclusion: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Fourteen Research Projects Will Inform Design Decisions 
on Key Efforts (Text for Interactive Figure 3): 

Appendix III: Census Mapping and Address Contracts: 

Appendix IV: Comments from the Department of Commerce: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Project Teams Involved in Development of Internet Response 
Option: 

Table 2: Project Teams Contributing to Decision to Use Administrative 
Records: 

Table 3: Project Teams Contributing to Decision to Implement Targeted 
Address Canvassing: 

Figures: 

Figure 1: 2020 Census Design Alternatives Show Varying Degrees of 
Change, Risk, and Potential Cost Savings: 

Figure 2: The Life Cycle of the 2020 Census Has Five Phases: 

Figure 3: Research Projects Will Inform Design Decisions on Key 
Efforts: 

Abbreviations: 

ACS: American Community Survey: 

Bureau: U.S. Census Bureau: 

GSS: Geographic Support Systems: 

LUCA: Local Update of Census Addresses: 

MAF: Master Address File: 

NRFU: Nonresponse follow-up: 

TIGER®: Topologically Integrated Geographic Encoding and Referencing: 

USPS: United State Postal Service: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

November 7, 2012: 

Congressional Requesters: 

The 2010 Census, at around $13 billion, was the most expensive 
headcount in our nation's history. Meanwhile, the cost of conducting 
the census has, on average, nearly doubled each decade since 1970 in 
constant 2010 dollars. If that growth rate continues, the 2020 Census 
could potentially cost around $25 billion. The U.S. Census Bureau 
(Bureau) has recognized the need for more cost-effective approaches to 
conducting the enumeration and is committed to identifying and 
implementing innovations and improvements as necessary to conduct the 
2020 Census at a lower cost per housing unit than the approximately 
$100 per housing unit cost of the 2010 Census (in constant 2010 
dollars) while still maintaining high quality. Between fiscal years 
2012 and 2014, the Bureau will conduct its research and testing phase 
for the 2020 Census in order to develop an initial design by September 
2014. The decisions made in the decade's first years shape the design 
and set the framework for later decisions that will affect how the 
census will be ultimately conducted. 

As requested, this report evaluates the Bureau's research efforts to 
improve the cost-effectiveness of the 2020 Census enumeration, paying 
particular attention to the following three key efforts that past 
Bureau research has shown could result in substantial cost savings: 

* leveraging the Internet to increase self-response; 

* improving how the Bureau builds its address and mapping databases, 
including a possible move to targeted address canvassing (as opposed 
to the full address canvassing used in earlier censuses, where field 
staff walked nearly every street in the country to verify the address 
list), and use of private-sector geographic data; and: 

* using administrative records,[Footnote 1] such as tax data, to 
reduce nonresponse follow-up costs. 

Our objectives were to: (1) identify what opportunities and risks, if 
any, the Bureau might need to consider for these research efforts 
going forward and (2) examine to what extent these three research 
efforts are on track with respect to scheduling, resources, and other 
performance metrics. 

To meet these objectives, we reviewed documentary evidence, including 
2020 Census research and testing plans, strategic framework documents, 
and our prior reports on the subject; and conducted interviews with 
cognizant Bureau officials, as well as knowledgeable stakeholders, 
such as those at the National Academy of Sciences, to determine 
opportunities and risks related to the Internet, administrative 
records and the address and mapping databases. We also obtained and 
compared Bureau-developed research planning requirements to research 
deliverables in order to identify gaps in the Bureau's research and 
testing efforts as it relates to schedule, resources, and performance 
metrics. Additionally, we interviewed agency officials to identify the 
Bureau's 2020 research and testing projects, timelines, and 
benchmarks. We obtained and reviewed contract-related documents 
related to geographic data and services, as well as interviewed Bureau 
officials to determine the Bureau's plan for using the private sector 
to assist with the Bureau's address and mapping needs. For more 
information on our scope and methodology see appendix I. 

We conducted this performance audit from October 2011 to September 
2012 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Background: 

The Bureau's 2020 Census early research agenda is focused on 
conducting the 2020 Census at a lower cost than the 2010 Census 
[Footnote 2] while maintaining high quality results. In order to do 
this the Bureau has developed a range of design alternatives for the 
2020 Census. The Bureau's 2020 design alternatives have potential for 
containing costs but also have the potential to increase risks. The 
design alternatives focus on options to use the Internet and other 
social media to increase response rate, target address canvassing, and 
expand the use of administrative records. Figure 1 shows the current 
range of six 2020 Census design alternatives currently being 
considered and how the three key efforts fit into them. According to 
the Bureau, the final 2020 design is likely to incorporate both 
existing approaches as well as design alternatives that have never 
been used in the decennial census. Greater changes to the overall 
design will result in greater potential for cost savings. However, 
greater design changes also could result in greater risk, and testing 
will be needed to identify the risks, costs, and benefits of any new 
approaches. According to the Bureau, alternative one has the lowest 
risk because it most closely mirrors the 2010 Census design (mailout 
of census forms, and in-person follow-up of non respondents) and is 
not dependent on implementing innovations such as increased use of 
administrative records and targeted address canvassing. 

Figure 1: 2020 Census Design Alternatives Show Varying Degrees of 
Change, Risk, and Potential Cost Savings: 

[Refer to PDF for image: illustration] 

Alternatives are listed as follows: 
Least change from 2010 design; Lowest risk; Less cost savings; 
to: 
Greater change from 2010 design; Higher risk; Potential for greater 
cost savings. 

Canvassing and continuous address frame updating: 
Alternative 1: Full address canvassing (similar to 2010 design); 
Alternative 2: Targeted address canvassing; 
Alternative 3: Targeted address canvassing; 
Alternative 4: Targeted address canvassing; 
Alternative 5: No address canvassing; 
Alternative 6: No address canvassing. 

Type of enumeration: 
Alternative 1: Traditional mailout of paper survey plus Internet; 
Alternative 2: Multi-mode contact (e-mail, text, social networking,  
etc.); 
Alternative 3: Visit household then use administrative records; 
Alternative 4: Use administrative records then visit household; 
Alternative 5: Near paperless; 
Alternative 6: Administrative records based. 

Source, U.S. Census Bureau. 

[End of figure] 

The Bureau formally kicked off initial planning for the 2020 Census on 
October 3, 2008, with the 2020 Census Planning Summit. At that 
meeting, decennial-census managers and subject matter experts 
discussed challenges, strengths, and weaknesses that would need to be 
considered during the 2020 Census strategic decision-making process. 
From 2009 to 2011, during the options analysis phase of 2020 Census 
planning, Bureau officials went through a research topic winnowing 
process where they: identified cost and quality drivers, conducted 
brainstorming sessions, received expert input (e.g., National Academy 
of Sciences), cataloged over 600 ideas, clustered those ideas into 
over 100 projects with research questions, and then prioritized them 
against the Bureau's Guiding Principles[Footnote 3] for the 2020 
Census. The Bureau identified 26 research projects to inform 
preliminary design decisions with the goal of conducting the 2020 
Census at a lower cost than the 2010 Census, while maintaining high 
quality. 

Using the 26 research projects, Bureau officials plan to validate the 
feasibility of the various strategies for reducing costs between 
fiscal years 2012 and 2014. Bureau officials stated that they are 
planning to narrow down the design alternatives by September 2014, and 
that research would continue into the next phase. Bureau officials 
believe that conducting research earlier in the decade will better 
position them to make more effective changes to the 2020 Census 
design. Figure 2 shows the five phases of the life cycle for the 2020 
Census. 

Figure 2: The Life Cycle of the 2020 Census Has Five Phases: 

[Refer to PDF for image: illustration] 

2009-2011: Options analysis. 

2012-2014: Early research and testing. 

2015-2018: Supplemental research and testing; Operational development 
and systems testing. 

2019-2023: Readiness testing, execution and close out. 

Source: U.S. Census Bureau. 

[End of figure] 

The Bureau is now in the "Early Research and Testing" phase and is 
currently refining and developing its 26 project plans. These plans 
include, among other requirements, project goals, objectives, scope, 
limitations, and skills required, and had a set of required iterative 
research and testing deliverables due 30, 60, 90, and 120 days after 
project kickoff. According to Bureau documentation, this process 
encouraged the development and proper consideration of ideas. The 
Bureau selected 17 of the 26 projects to start in fiscal year 2012. 
The remaining projects are scheduled to begin in fiscal years 2013 and 
2014. Of the 17 research projects the Bureau is conducting in 2012, 14 
projects are to inform design decisions related to the Internet, 
administrative records, and targeted address canvassing. See figure 3 
for a list of these 14 research projects and their objectives. 
According to the Bureau's timeline, project teams are to complete 
their work on these 14 projects by September 2014.[Footnote 4] 

Figure 3: Research Projects Will Inform Design Decisions on Key 
Efforts: 

[Refer to PDF for image: Interactive graphic] 

Move mouse over each activity to see more information afterwards Click 
'X' to close the open pop-up information display panel. 

A full text version of this graphic is available in appendix II. 

Source: GAO analysis of U.S.Census Bureau information. 

[End of figure] 

Key Efforts Present the Bureau with Both Opportunities and Risks: 

Research on new methods likely to result in a more cost-effective 2020 
Census must be accomplished early enough in the decade to confirm 
their likely impact on both cost and quality and to inform timely 
design decisions. The use of the Internet as a response option, a 
potential move towards targeted address canvassing, and the possible 
use of administrative records to replace data collected during census 
field operations present the Bureau with opportunities to reduce costs 
while maintaining a high quality census. As part of the 2020 Census 
planning process the Bureau has identified both program-level and 
project-level risks associated with these options. According to the 
Bureau, program-level risks span the entire 2020 research and testing 
phase and if not managed properly could jeopardize the phase's goals 
and objectives. In contrast, project-level risks pertain to the 
successful completion of specific projects. However, the Bureau has 
not developed mitigation and contingency plans for the project-level 
risks. Also, the Bureau has not developed cost estimates for any of 
its 2020 research and testing projects as required in guidance 
provided to project teams. Finally, some of the project plans we 
reviewed were incomplete. For example, not all project plans 
identified the necessary staff resources, while other project plans 
failed to provide performance metrics for measuring progress and for 
determining the project's final outcome. 

All Three Key Efforts Present the Bureau with Cost Saving 
Opportunities: 

Internet: 

During the 2000 Census, the Bureau piloted an Internet response option 
that had a limited number of respondents. They considered building on 
this 2000 experience for the 2010 Census, but the Bureau decided in 
July 2006 not to include the Internet option in the design for the 
2010 Census because test results indicated that the Internet option 
did not increase the overall response rate, including response among 
hard-to-count population groups, and they had underestimated the costs 
of the contract that included developing the Internet option.[Footnote 
5] 

However, more recent tests conducted in 2011 showed that adding an 
Internet response option increased the overall response rate. The 2011 
test results, coupled with the increased prevalence and accessibility 
of the Internet, led Bureau officials to commit to providing an 
Internet response option for the 2020 Census. With regard to the cost 
for the 2020 Census, if this option can help achieve an overall 
increase in the response rate it can save money, since Bureau field 
staff would need to visit fewer households during nonresponse follow-
up (NRFU), which is the largest and most costly census field 
operation.[Footnote 6] Furthermore, testing has shown that the cost of 
an Internet survey is low compared to a mail survey, which incurs 
printing and postage costs. Moreover, web survey responses are 
generally available more quickly and are of better quality than 
responses from a mail survey because there is no lag time as the 
responses are captured in real time and there are reminders to prompt 
the respondent if a question is unanswered. Quicker and complete 
responses can also help reduce the amount of time and money spent on 
following up on late or incomplete census forms.[Footnote 7] 

In April and November 2011, the Bureau tested an Internet response 
option in its American Community Survey (ACS),[Footnote 8] and 
beginning in January 2013, the Bureau plans to offer it as a response 
option. Bureau officials have stated that they intend to build on this 
existing information technology infrastructure in order to reduce the 
cost of implementing an Internet option for the 2020 Census. 

Given the Bureau's decision to move forward with an Internet design 
option, six research and testing projects are focused on how best to 
implement an Internet response option in concert with other self-
response options. The project team with primary responsibility for 
coordinating the design of the Internet response option is tasked with 
answering the following research question: 

* How does the Bureau leverage technology and new response modes 
(including the Internet) to increase self-response, improve 
nonresponse follow-up data collection strategies, while maintaining 
overall quality? 

One objective of this project is to determine what contact strategy 
should be used to inform the public that census responses can be filed 
via the Internet in order to increase self-response (i.e., Bureau use 
of email, social networks or phone for initial contact or for 
reminders). Contact strategy testing will examine the impact of 
various Bureau instructions to respondents, as well as the timing of 
the delivery of the mailing pieces, on such factors as response rates, 
return rates, percentage of Internet returns, and speed of returns. 
This project will also determine what mode (paper, Internet, or phone) 
is best for each demographic, geographic, and language group, as well 
as determine the effect a lack of access to technology has on each 
demographic group. Additionally this project will refine and test 
security features. The other five projects will cover such areas as 
types of languages that will be offered by response mode, public 
perception and potential concerns about an Internet response, and 
management of the workload for multiple modes of response. 

Targeted Address Canvassing and Use of the Private Sector for 
Geographic Data Services: 

In the 2010 and earlier censuses, the Bureau mounted a full address 
canvassing operation, where field staff verified every housing unit in 
the nation to update the Master Address File (MAF) and the associated 
mapping system called TIGER® (Topologically Integrated Geographic 
Encoding and Referencing). This labor-intensive effort was one of the 
more expensive components of the 2010 Census: the 2010 Address 
Canvassing Operation required 140,000 temporary workers to verify 145 
million addresses (by going door-to-door) at a cost of $444 million, 
or 3 percent of the $13 billion total cost of the 2010 Census. One of 
the reasons the Bureau conducted a full address canvassing for the 
2010 Census was because it wanted to capture hidden and hard-to-
capture addresses, such as housing units in converted garages and 
large storage sheds. For the 2020 Census the Bureau would like to 
reduce the amount of field work and cost of address and map updates by 
targeting the address canvassing operation. According to Bureau 
planning documents, through early research and testing the Bureau 
plans to attempt to determine the quality and stability of its address 
file in terms of the number of changes (adds, deletes, duplicates, and 
corrected addresses) that took place to each census block in 2010. 
Depending on that outcome, the Bureau would target for address 
canvassing only those blocks where the quality of the addresses was 
determined to be inadequate. 

To determine whether to target address canvassing for the 2020 Census, 
the Bureau has two ongoing efforts to improve the Bureau's map and 
address databases. First, the Bureau's Geography Division is working 
with the United States Postal Service (USPS) and other federal 
agencies on a new program called the Geographic Support Systems (GSS) 
Initiative, which allows government agencies at all levels to 
regularly share and continuously update their address lists with the 
Bureau. According to current plans, USPS will continue providing the 
Bureau with address updates. Likewise, tribal, state, and local 
governments, which maintain address lists for purposes such as 
emergency response and property assessment, will also have the 
opportunity to share addresses with the Bureau throughout the decade, 
rather than solely 2 years prior to the decennial, as had been the 
case in prior decennial censuses. Additionally, the GSS Initiative is 
working to develop, and in some cases improve, methods for identifying 
and capturing hidden and hard-to-capture addresses. 

As part of this first effort, the Bureau is investigating the role and 
possible contributions the private sector can make in improving its 
address and mapping databases. For the 2010 Census the Bureau relied 
heavily on the private sector to update addresses and maps. 
Specifically, in June 2002, the Bureau awarded an 8-year contract of 
about $200 million to improve the accuracy of its mapping database and 
investigate options and methods of updating its address database. The 
contract, among other tasks, corrected in TIGER® the location of 
streets, boundaries, and other map features so that coordinates would 
be aligned with their true geographic locations. According to Bureau 
officials, continued reliance on the private sector is necessary in 
order to maintain the major upgrades that were made to its address and 
mapping databases last decennial. The Geography Division has nine 
contracts in place, totaling about $90 million dollars, to help it 
manage its mapping and address database (see app. III for a list of 
those contracts). Some of the work being done by the private sector 
includes a service contract to provide system upgrades and maintenance 
to the MAF/TIGER® database and a data entry services contract to 
update the MAF/TIGER® database with address information provided by 
state and local governments. 

Moreover, to ensure the most current address and mapping data are 
available for the 2020 Census, the Bureau, in April 2012, began 
conducting market research with three private-sector firms to assess 
how those firms might assist the Bureau with its address and mapping 
needs. Possibilities to be explored include how to best maintain 
address and map data, understand private-sector innovation and 
technology, and options for address and map data-sharing methods. 
According to Bureau officials, negotiations concerning the scope, 
tasks, and activities to be completed with the first contractor have 
taken place. However, negotiations with the remaining two firms have 
not begun. Use of the private sector for mapping is an area we have 
previously reported needs to be explored. Specifically, in December 
2010, we listed a number of reexamination areas which have particular 
implications for controlling costs and improving accuracy and raised 
the question: To what extent can private-sector and other sources of 
information such as maps, address lists, and geographic databases be 
employed to help support the census?[Footnote 9] 

The second effort to improve the Bureau's address and mapping database 
is centered on the 2020 Research and Testing Program. There are three 
research projects designed to inform the Bureau's decision about the 
extent to which it might be able to conduct a targeted address 
canvassing operation. Two projects will use modeling to predict where 
coverage errors (e.g., missed or duplicate addresses) occur in the 
MAF. These models will be used to assess the quality of data sources, 
types of errors, and what information, such as a missing map location 
for an address, is not in the MAF. If the Bureau decides to conduct 
targeted address canvassing, this information will be used to 
determine which areas meet acceptable quality standards, as well as 
where targeted address canvassing would be effective. According to the 
Bureau, collectively the two projects are to answer the following 
question: 

* As the related GSS Initiative proceeds, how will the Bureau 
determine the required level of quality needed in the address frame to 
conduct an accurate census and then measure the quality of the 
continually updated MAF for that purpose? 

The third research project in this effort, the Local Update of Census 
Addresses (LUCA) Program Improvement project,[Footnote 10] will 
examine how to modify the Bureau's existing partnership program in 
consultation with the GSS Initiative. The LUCA program is a 
partnership program that provides an opportunity for local and tribal 
governments to review and update individual address information or 
block-by-block address counts from the MAF and associated geographic 
information in the TIGER® database just prior to the decennial. The 
goal of LUCA is to improve the completeness and accuracy of address 
information. This research project's objective is to determine the 
best way to modify and improve LUCA procedures to ensure compatibility 
with continual updating of address information throughout the decade. 
The project will also determine whether it is necessary for local and 
tribal governments to actively participate in LUCA, a once-a-decade 
operation, if they have been submitting addresses and mapping data 
continuously throughout the decade as part of the GSS Initiative. 

Administrative Records: 

Administrative records are a growing source of information on 
individuals and households. For purposes of the decennial census, the 
Bureau is considering administrative records from government agencies, 
including tax data and Medicare records, as well as commercial sources 
to identify persons associated with a particular household address. 
[Footnote 11] During the 2010 Census, the Bureau made limited use of 
administrative records. For example, the Bureau used USPS files to 
update its address list, and federal agency records (such as those 
from the Department of Defense) were used to count military and 
federal civilian employees stationed outside of the United States. 

For 2020, in its first attempt to estimate potential costs of various 
design alternatives for the 2020 Census, the Bureau estimated that by 
expanding the use of administrative records it could save up to $2 
billion by reducing the workload for several operations, including 
NRFU, address building, and quality assurance.[Footnote 12] Depending 
on the results of the research and testing projects, Bureau officials 
plan to build a composite of quality administrative records from 
various sources (i.e., federal agencies, state and local governments, 
and commercial sources) that it can use to reduce or replace costly 
field work. Thus, using administrative records could allow the Bureau 
to reduce the scale of its NRFU operation and reduce the need for 
field office space and staff. The 2012 research and testing projects 
are also focused on identifying the best available administrative 
records to use for address frame building. Administrative data from 
other sources would be combined with USPS data, thus allowing the 
Bureau to continuously update its MAF throughout the decade. Finally, 
administrative records could reduce the cost of quality control, 
according to Bureau officials. In order to ensure the accuracy of the 
field work conducted during census field operations, such as NRFU, the 
Bureau sends quality control staff into the field to verify the work 
completed by a random sample of enumerators. Bureau officials are 
determining for 2020 whether administrative records can be used as a 
quality control check, rather than sending staff into the field. 

There are nine research and testing projects that will inform the 
Bureau's decision making on the extent to which the Bureau can expand 
the use of administrative records in order to reduce costs. These 
research and testing efforts are focused on determining whether the 
quality of administrative records is sufficient to be used for this 
purpose. Initial testing is to be completed and decisions on these 
matters are to be made by September 2014. Of the nine, there are two 
primary administrative records research projects. They are tasked with 
answering the following research questions: 

* How can the Bureau best develop and maintain an independent 
collection of administrative records and assess the quality of those 
records (best sources and methods)? 

* How can the Bureau leverage administrative records (including 
commercial files) to significantly reduce decennial census cost while 
maintaining quality? 

The objective of one of these projects is to acquire, process, and 
analyze administrative records from federal, state, and commercial 
sources to assess their utility for the 2020 Census. The other project 
will research and test methods to enhance NRFU operations with 
administrative records, such as replacing phone or in-person collected 
data with administrative data. 

The Bureau Has Identified 2020 Census Program-Level and Project-Level 
Risks but Has Not Developed Project-Level Mitigation and Contingency 
Plans: 

Bureau officials are aware that the changes they want to make to the 
decennial census come with many risks. The Bureau has identified and 
prioritized from high to low both program-level and project-level 
risks. According to the Bureau, program-level risks span the entire 
2020 research and testing phase and if not managed properly could 
jeopardize the phase's goals and objectives. In contrast, project-
level risks pertain to the successful completion of projects. In 
accordance with best risk management practices,[Footnote 13] the 
Bureau has identified and drafted mitigation plans for all of the 14 
program-level risks it has identified. In May 2012 the Bureau 
designated two of the 14 program-level risks as high risk: 

* Timely research and testing results: The 2020 research and testing 
strategy involves "many small field tests to research design 
alternatives with an accelerated, agile, and informed decision-making 
process for incorporating changes to the 2020 Census design." 
According to the Bureau, if the research and testing results are late, 
then decisions will not be made on time and the program may not be 
ready to move out of the research and testing phase on schedule. 
According to the Bureau's 2020 Census Risk Management Mitigation 
Plans, officials have begun implementing a mitigation strategy for 
this risk, including documenting clear roles and responsibilities and 
communicating them to staff; establishing a schedule that contains 
decision milestones and tracking interdependencies of research results 
and testing projects; and reviewing ongoing project status including 
updates on dependencies, risks, and metrics. 

* Significant budget cuts: The Bureau reported that if its funding for 
2020 Census planning is significantly reduced for fiscal year 2013, 
key projects will have to be delayed and as a result the Bureau could 
have major technical and operational difficulties making preliminary 
design decisions. According to Bureau documents, to mitigate this 
risk, Bureau officials have briefed staff from House of 
Representatives Subcommittee for Commerce, Justice, Science and 
Related Agencies, Committee on Appropriations, and the Office of 
Management and Budget on the implications of reduced funding and plan 
to brief oversight committee staff. In addition, Bureau officials 
reported they have analyzed budgetary needs for fiscal year 2013 and 
prioritized projects to manage limited resources. 

Related to these risks, a June 2012 congressional hearing held by the 
Joint Economic Committee discussed the impact of eliminating or 
reducing funding to the ACS. According to Bureau officials, such 
action would have a significant impact on the 2020 Census research and 
testing program because ACS is one of its testing platforms. This risk 
is not listed as a program-level risk, or part of the current risk 
mitigation plans, but Bureau managers told us they are strategizing on 
what they will do if ACS is not funded and will update the risk 
mitigation plan accordingly. 

The Bureau has identified lack of support from external stakeholders 
as a medium program-level risk. Bureau officials are concerned that if 
external stakeholders, such as Congress, the Office of Management and 
Budget, and the National Academy of Sciences, do not support research 
on alternative design options, then the research and testing agenda 
will have to be redesigned. To manage this risk, Bureau officials have 
communicated the alternative design options to a wide range of 
external stakeholders, but are concerned that support may diminish as 
time passes and stakeholders change. To mitigate this risk, the Bureau 
is engaging external stakeholders in communications regarding 2020 
research and testing efforts and seeking feedback on Bureau plans. 
According to Bureau documents, the Bureau has met with external 
stakeholders, including congressional staff on its House of 
Representatives and Senate oversight committees, to discuss 2020 
Census planning, and is working on a strategy for keeping stakeholders 
informed as plans progress. 

Unlike the program-level risks, risk mitigation and contingency plans 
have not been drafted to address the project-specific risks. 
Specifically, each of the Bureau's research and testing projects has 
its own risk register[Footnote 14] that identifies and prioritizes the 
risks associated with that project, but no risk mitigation or 
contingency plans have been developed for these project-level risks. 
Several risks appear in multiple project team risk registers, 
including tight time frames and accurate cost information. For 
example, one Internet-related project listed that there is a risk that 
if tests are not timed properly, and if time frames are too tight then 
they may not be able to apply what they have learned from one test to 
the next. Similarly, one of the administrative records project teams 
was concerned that one year to integrate data, build a model, and 
conduct adequate analysis of administrative records was "not 
realistic." In another example concerning cost data, two teams stated 
that without accurate cost data at the project level, research results 
will not be sufficient to inform the design of the 2020 Census. 

Project teams also identified project-level risks directly related to 
the use of administrative records in place of data collected by 
enumerators during field visits. Among these risks is access to 
administrative records to build a high quality compilation of 
administrative records for each household across the country. To 
accomplish this task, the Bureau needs timely access to data collected 
by the federal government and, as previously mentioned, is currently 
compiling administrative records from various federal agencies 
including the Departments of Housing and Urban Development and Health 
and Human Services. However, not all agencies are authorized to 
provide their data to the Bureau thus limiting, and in some cases 
preventing, the Bureau's use of that data. Bureau officials provided 
us with two examples of agency administrative record sources that they 
are not authorized to access: records maintained under the Family 
Educational Rights and Privacy Act and the National Directory of New 
Hires.[Footnote 15] Bureau officials said they would like to have 
access to these and other data sources for statistical purposes in 
order to further increase the number and improve the quality of 
available administrative records. According to Bureau officials, 
reviewing and determining whether they have access to administrative 
records with every agency is a time consuming process and the Bureau 
would like to have the ability to fully access and use agency 
administrative data, especially since the confidentiality of census 
data are protected by Title 13.[Footnote 16] To move this forward, 
according to a senior Bureau official, legislation might need to be 
enacted that would compel federal agencies to provide the Bureau with 
access to administrative data for the decennial census. However, the 
enactment of such legislation is likely to be a time consuming process 
that raises serious policy concerns including whether the Bureau has 
made a sufficiently strong case that it needs access to all federally 
collected data and the resulting impact on personal privacy 
protections. 

According to Bureau officials, the project teams did not submit risk 
mitigation and contingency plans for the project-level risks as 
required at the 90-day checkpoint because after reviewing all of the 
project-level risk registers the project leaders first wanted to 
unduplicate similar risks across projects in order to prevent 
redundancies. Bureau officials were not able to provide a date for 
when risk mitigation and contingency plans would be available. 
However, they stated they are committed to managing risk in order to 
avoid the missteps encountered in 2010 when the Bureau was on our high-
risk list due to weaknesses in managing IT, operational planning, and 
cost estimating.[Footnote 17] In our prior work, we reported that risk 
mitigation involves identifying, analyzing, prioritizing, and 
documenting risks, and ideally more than one alternative should be 
assessed.[Footnote 18] The Department of Commerce's Inspector General 
has recommended that risk management activities begin from the outset 
of the current decennial census life cycle, rather than just before 
field operations. Without mitigation plans the 2020 Census planning 
team may not be able to fully manage risks associated with these 
projects. 

Project-Level Cost Estimates Have Not Been Developed: 

The Bureau did not develop cost estimates for any of its 2020 research 
and testing projects by the 90-day checkpoint as required in guidance 
provided to project teams. The importance of reliable cost estimates 
is underscored by the Bureau's experience leading up to the 2010 
Census, where we found that the Bureau's cost estimates lacked 
detailed documentation on data sources and significant assumptions and 
was not comprehensive because it did not include all costs.[Footnote 
19] Partly as a result, some operations had substantial variances 
between their initial cost estimates and their actual costs. For 
example, the 2010 address canvassing operation cost $88 million more 
than the original estimate of $356 million, an overrun of about 25 
percent; and the Bureau's 2010 NRFU operation cost $1.6 billion, about 
$660 million, or 29 percent, less than the Bureau initially estimated. 

According to 2020 Census planning documents, at the 90-day checkpoint 
project teams were to: (1) prepare and submit cost estimate requests 
needed for the project; (2) document estimates for completing the work 
for the project; (3) identify and document funds needed for additional 
resources and requirements, and (4) prepare and submit requests for 
additional funds. According to a senior Bureau official, cost 
estimates have not yet been completed because more work needs to be 
done to determine project costs and they are in the process of 
formulating cost estimates for each project. 

According to our cost estimate guide,[Footnote 20] a cost estimate 
provides an assessment of the costs most likely to be incurred and 
should include the requirements, resources, and tasks that must be 
accomplished. As the Bureau moves forward with its 2020 research and 
testing program, project cost estimates will allow the Bureau to plan 
for the resources required to conduct necessary field testing, ensure 
there are sufficient funds, and adjust testing accordingly. In 
addition, the Bureau could use these cost estimates to help determine 
how budget cuts might hinder the census research and testing program's 
progress or effectiveness. Thus, if the Bureau does not develop cost 
estimates in a timely manner it will be difficult for the Bureau to 
ensure that resources are adequate to support research and testing 
projects. 

Research and Testing Project Plans Were Delivered on Schedule, but Not 
All Project-Level Resources and Performance Metrics Have Been 
Identified: 

The Bureau's early research and testing is currently refining the 
project plans associated with the14 projects related to the Internet, 
administrative records, and targeted address canvassing. Each of the 
research and testing projects has a set of required deliverables due 
30, 60, 90, and 120 days after project kickoff. We determined that 
each project team had provided the 120-day required deliverables on 
schedule to Bureau managers in June 2012. The research and testing 
documents required by the 2020 Census planning team and delivered by 
the 14 project teams describe management and technical plans for each 
project, and include a team charter, project plan, resources, 
preliminary schedule, and field tests. These documents are important 
because they serve as the baseline for each of the research and 
testing projects. 

However, not all resource requirements were documented. For example, 
some project teams did not fully document the types of skills needed, 
or perform a skills gap assessment to determine the resources needed 
to carry out their respective projects as required in the Bureau's 
planning template (the template lays out in detail what is to be 
included in the deliverables). Bureau officials plan to use this 
skills information to allocate staff to ensure teams have the skills 
necessary to complete their projects. While 11 project teams provided 
all the required information, 2 project teams conducting research on 
issues that will inform decisions on targeted address canvassing and 
expanded use of administrative records, respectively, did not provide 
documentation that a skills gap analysis was conducted and another 
project researching expanded use of administrative records did not 
provide the necessary skill sets.[Footnote 21] According to Bureau 
officials, they are working to document these skills sets, as well as 
any gaps. Completing this analysis is important to ensuring sufficient 
resources are available for conducting the research and testing 
projects. 

We also found incomplete performance metric documentation for several 
projects. According to Bureau guidance, project teams were to provide 
performance metrics (including the methodology for monitoring and 
evaluating project performance), as well as exit criteria for 
determining each project's final outcome. However, the "Enhancing 
Demographic Analysis" team that is researching the use of 
administrative records to assess the accuracy of census counts at the 
national level did not provide any of the required performance 
metrics, while six teams[Footnote 22] failed to include performance 
metrics that could be used to monitor research and testing progress. 
According to Bureau officials, not all teams followed the project 
templates, and those teams have been asked to adhere to the 
requirements and provide all necessary information. Project 
performance metrics are a key element of effective project planning. 
As we have noted in prior work on project planning, measurable 
performance goals should be identified and performance data should be 
gathered to determine progress and whether desired outcomes have been 
achieved.[Footnote 23] Absent such metrics, the Bureau does not have 
the assurance that it will be able to avoid potential slips in the 
research and testing schedule, or certainty as to whether project 
outcomes will be adequate for making decisions. 

Conclusion: 

Just over 7 years remain until Census Day 2020. While this might seem 
like an ample amount of time to shore up the Bureau's planning process 
and take steps to control costs, past experience has shown that the 
chain of interrelated preparations that need to occur at specific 
times and in the right sequence leave little room for delay or 
missteps. According to the Bureau, if 2010 operations are repeated in 
2020, continued growth in population size and complexity will likely 
lead to an unsustainable increase in census cost. Further, traditional 
enumeration methods may no longer effectively produce a high quality 
census. To contain costs and maintain quality, bold innovations in 
both planning for and conducting the 2020 Census will be required. 

The early research and testing phase represents a critical stage in 
preparing for the 2020 Census. At this time Bureau management is 
shaping the next decennial census as it determines what new operations 
will be a part of the 2020 Census design, which operations need to be 
revised, what risks remain, and how those risks will be mitigated. We 
have highlighted some of the cost savings that can be attained through 
three new operational changes being considered--the use of the 
Internet as a self-response option, replacing enumerator collected 
data with administrative records, and targeting only certain addresses 
for field verification. 

However, these innovations also come with risks that need to be 
managed. Specifically, while the Bureau has identified project-level 
risks for the 14 projects we reviewed, it has not developed risk 
mitigation plans for them. Given the number of changes the Bureau 
would like to make for 2020 it is imperative that risks are 
sufficiently managed and that mitigation plans be in place. For 
example, the Bureau needs to manage the risk for accessing federally 
collected administrative data that it will use for the 2020 Census. 
While it currently has access to some federally collected data, the 
Bureau would like access to all federally collected data, which 
according to the Bureau would require a statutory change. However, 
such a change raises serious policy concerns including the impact on 
personal privacy protections, and most likely would be a time 
consuming process. Therefore, it is imperative that risk mitigation 
and contingency plans be in place if the Bureau is unable to 
successfully gain access to all federally collected data. Other areas 
that need attention include ensuring that all research and testing 
projects have reliable cost estimates, so as to avoid research and 
funding shortfalls, that performance metrics are identified to ensure 
project goals are monitored and met, and that skills are identified to 
ensure the selection and assignment of appropriate staff to each 
project. 

Recommendations for Executive Action: 

We recommend that the Acting Secretary of Commerce require the Under 
Secretary for Economic Affairs who oversees the Economics and 
Statistics Administration, as well as the Acting Director of the U.S. 
Census Bureau, to take the following three actions to improve the 
Bureau's Research and Testing for the 2020 Census, and thus better 
position the Bureau to carry out a cost-effective decennial census: 

* Develop risk mitigation and contingency plans for all projects to 
ensure that risks are adequately managed to minimize their effect on 
the project. 

* Develop cost estimates for each project. 

* Ensure documentation for projects are complete, including specifying 
the performance metrics that will be the basis for determining that 
each of the projects has completed its work and identifying skills 
needed to inform the selection and assignment of appropriate staff to 
each research project. 

Agency Comments and Our Evaluation: 

The Acting Secretary of Commerce provided written comments on a draft 
of this report on October 22, 2012. The comments are reprinted in 
appendix IV. The Department of Commerce agreed with the assessments 
and recommendations of the report. In addition, the Acting Secretary 
of Commerce provided a technical comment and suggestion where 
additional context might be needed, and we revised the report to 
reflect this comment. 

The Bureau in its comments stated that our report accurately 
represented the extensive work that has been completed during the 
early research and testing phase for the 2020 Census. To address our 
concerns, the Bureau noted that it is now focusing on managing risks 
at the project level and will begin obtaining more specific cost 
estimation details for each project. The Bureau also agrees with our 
recommendations to ensure that performance metrics and skill sets are 
identified for all projects and teams. Finally, the Bureau stated that 
these efforts are either already underway or are planned as a major 
focus during fiscal year 2013. 

As arranged with your offices, unless you publicly announce the 
contents of this report earlier, we plan no further distribution until 
30 days after the date of this report. At that time, we will send 
copies of this report to the Acting Secretary of Commerce, the Under 
Secretary of Economic Affairs, the Acting Director of the U.S. Census 
Bureau, and interested congressional committees. The report also is 
available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov]. 

If you have any questions on matters discussed in this report, please 
contact me at (202) 512-2757 or by e-mail at goldenkoffr@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Key contributors 
to this report are located in appendix V. 

Signed by: 

Robert Goldenkoff: 
Director: 
Strategic Issues: 

List of Requesters: 

The Honorable Thomas R. Carper: 
Chairman: 
The Honorable Scott Brown: 
Ranking Member: 
Subcommittee on Federal Financial Management, Government Information, 
Federal Services, and International Security: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Danny K. Davis: 
Ranking Member: 
Subcommittee on Health Care, District of Columbia, Census, and the 
National Archives: 
Committee on Oversight and Government Reform: 
House of Representatives: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

This report evaluates the U.S. Census Bureau's (Bureau) efforts to 
improve the cost-effectiveness of the enumeration, paying particular 
attention to the following three key efforts that past Bureau research 
has shown could result in substantial cost savings: (1) leveraging the 
Internet to increase self-response; (2) improving how the Bureau 
builds its address and mapping databases, including a possible move to 
targeted address canvassing, and use of private-sector geographic 
data; and (3) using administrative records, such as tax data, to 
reduce nonresponse follow-up costs. To meet our objectives of 
identifying what opportunities and risks, if any, the Bureau might 
need to consider for these efforts going forward, and examining to 
what extent these three efforts are on track with respect to 
scheduling, resources, and other performance metrics, we reviewed 
Bureau documents pertaining to the early planning of the 2020 Census. 
These generally consisted of the Bureau's operational plans, strategic 
framework documents, strategies and planning memorandums, timelines, 
and benchmarks. Many of these documents were considered draft, but 
Bureau officials said they were sufficiently developed for purposes of 
our review. To describe the Bureau's efforts to use private-sector 
firms to assist with its address and mapping needs, we spoke to agency 
officials and reviewed the Bureau's current mapping and address 
contracts and its request for information to solicit input from 
private-sector firms. We interviewed Bureau officials responsible for 
the early planning of the 2020 Census and inquired about their process 
for designing their research and testing program, as well as 
opportunities and risks presented by these three key efforts. 

We reviewed our prior reports on and other reports on leading 
practices to identify gaps in the Bureau's research and testing 
efforts as it relates to schedule, resources, and performance metrics. 
We also reviewed reports from and spoke to knowledgeable stakeholders, 
such as officials at the National Academy of Sciences to determine 
opportunities and risks related to the Internet, administrative 
records, and the address and mapping databases. In addition, we spoke 
with Bureau officials and reviewed our prior work to identify 
obstacles the Bureau may need to address moving forward; for example, 
legal barriers to the Bureau's access to administrative records. 

To assess the Bureau's progress in meeting its stated goals for the 
research and testing program, we evaluated whether each of the 14 
research projects the Bureau initiated in 2012 had provided the 
required deliverables: the 120 day project plan, the project schedule, 
the project charter, and the project risk register. We also performed 
a content analysis of Bureau planning documents. For this analysis we 
examined the deliverables for each of the 14 research projects. We 
developed a data collection instrument to record the presence or 
absence of 11 pieces of information, or elements, in the deliverables 
for each project. The 11 elements were identified through a review of 
the Bureau's project plan template which outlines information that 
each project was expected to report by the end of the 120-day 
initiation phase. A coding dictionary was created to provide a uniform 
set of coding criteria for each element coded. Each element could be 
coded as present, partially present, or absent. During the coding 
process, two analysts independently coded documents for each project 
using the coding dictionary as a guide. In the next phase, the two 
coders met to reconcile any discrepancies between their coding 
decisions and then made a final determination for that element. In 
cases of disagreement, the engagement methodologist was consulted as a 
tiebreaker. Following the coding process, the team analyzed the number 
of present and absent elements to identify patterns for each element 
across the 14 projects, as well as the completeness of documentation 
for each project. 

This engagement did not examine the Bureau's information technology 
policies, procedures, and information security in depth because our 
information technology team is working on two engagements that cover 
these issues. 

[End of section] 

Appendix II: Fourteen Research Projects Will Inform Design Decisions 
on Key Efforts (Text for Interactive Figure 3): 

Table 1: Project Teams Involved in Development of Internet Response 
Option: 

Project team: Optimizing Self-Response; 
Objectives in Internet development: Develop requirements for the 
Internet response option and coordinate the relationship between 
different response modes (i.e., Internet, paper, telephone). 

Project team: Questionnaire Content, Design, and Mode Study; 
Objectives in Internet development: Determine the number of languages 
that will be used in the Internet response option. 

Project team: Workload Management; 
Objectives in Internet development: Develop the infrastructure that 
will support the Internet response option. 

Project team: Coding, Editing and Imputation Study; 
Objectives in Internet development: Help determine what prompts to 
include in an Internet response option to improve data quality and 
respondents' ease of use. 

Project team: Privacy and Confidentiality Study; 
Objectives in Internet development: Identify public perception and 
concerns about responding to the Census via the Internet. 

Project team: Contact Frame; 
Objectives in Internet development: Provide alternate contact methods, 
such as e-mail addresses. 

Source: GAO analysis of U.S. Census Bureau information. 

[End of table] 

Table 2: Project Teams Contributing to Decision to Use Administrative 
Records: 

Project team: Alternative Administrative Records Composite; 
Objectives in administrative records development: Acquire, process, 
and analyze administrative records from federal, state, and commercial 
sources to assess their utility for the 2020 Census. 

Project team: Supplementing and Supporting Nonresponse Follow-up 
(NRFU) with Administrative Records; 
Objectives in administrative records development: Research and test 
methods to replace or supplement NRFU data collected in person with 
administrative data. 

Project team: Local Update of Census Addresses (LUCA) Program 
Improvement; 
Objectives in administrative records development: Explore the use of 
administrative records to validate new addresses. 

Project team: Coding, Editing and Imputation Study; 
Objectives in administrative records development: Research the use of 
administrative records as a source to obtain missing household and 
address information. 

Project team: Enhancing Demographic Analysis; 
Objectives in administrative records development: Research the use of 
administrative records to assess the accuracy of Census counts at the 
national level. 

Project team: Improving Quality Control; 
Objectives in administrative records development: Examine the use of 
administrative records to supplement or replace field work in quality 
control operations. 

Project team: Privacy and Confidentiality Study; 
Objectives in administrative records development: Identify public 
perceptions related to the use of administrative records for 
enumeration purposes. 

Project team: Matching Process Improvement; 
Objectives in administrative records development: Examine methods to 
ensure that the Census Bureau can accurately match administrative 
records to individuals and housing units. 

Project team: Contact Frame; 
Objectives in administrative records development: Acquire and process 
administrative records for the purpose of using them for alternate 
contact information. 

Source: GAO analysis of U.S. Census Bureau information. 

[End of table] 

Table 3: Project Teams Contributing to Decision to Implement Targeted 
Address Canvassing: 

Project team: Master Address File (MAF) Error Model; 
Objectives in targeted address canvassing development: Develop a 
statistical model of errors (e.g., missing or duplicate addresses) in 
the MAF. 

Project team: Independent MAF Quality Assessment; 
Objectives in targeted address canvassing development: Use the MAF 
Error Model to assess the quality of the MAF and determine whether 
targeted address canvassing would be effective, and, if so, where it 
should be performed. 

Project team: Local Update of Census Addresses (LUCA) Program 
Improvement; 
Objectives in targeted address canvassing development: Examine what 
modifications to the Bureau's existing LUCA program will be required 
to support targeted address canvassing. 

Source: GAO analysis of U.S. Census Bureau information. 

[End of table] 

[End of section] 

Appendix III: Census Mapping and Address Contracts: 

Contractor: Acquis Inc; 
Description of services: Develop, implement, and support a system 
which allows the Census Bureau (Bureau) to update its Master Address 
File (MAF) with TIGER® (Topologically Integrated Geographic Encoding 
and Referencing) data, the U.S. mapping data, as the data is received; 
Contract value: $1,361,780. 

Contractor: ASRC Research and Technology Solutions; 
Description of services: Analyze mapping data as it is provided by 
state and local governments and other agencies, and upload changes 
into TIGER®; 
Contract value: $9,998,400. 

Contractor: Caliper Corporation; 
Description of services: Customize and maintain software so that over 
40,000 state, local, and tribal governments can update and verify the 
Bureau's address and mapping databases; 
Contract value: $2,049,675. 

Contractor: ERDAS, Inc; 
Description of services: Supply digital map files and digital aerial 
imagery to maintain and update the MAF/TIGER® database; 
Contract value: $26,015[A]. 

Contractor: ESRI, Inc; 
Description of services: Customize, install, and provide technical 
support for software to process and edit files to outline geographic 
areas, and support mapping applications, including the use of handheld 
computers to collect and edit data; 
Contract value: $1,091,468. 

Contractor: Group 1 Software, Inc; 
Description of services: Provide software to match addresses from 
surveys and other agencies to the MAF/TIGER® database; 
Contract value: $14,016[A]. 

Contractor: Gunnison Consulting Group, Inc; 
Description of services: Develop and maintain software which 
automatically places text on Bureau maps so all map features are 
clearly identified; 
Contract value: $2,103,727. 

Contractor: Oracle America, Inc; 
Description of services: Update and consolidate the infrastructure, 
servers and database, of the MAF/TIGER® system; 
Contract value: $9,973,265. 

Contractor: Sabre Systems, Inc; 
Description of services: Provide services for the modernization of the 
Bureau's MAF/TIGER® system, including expansion, design, programming, 
operation and maintenance of the infrastructure supporting the 
MAF/TIGER® system; 
Contract value: $62,927,907. 

Total Contract value: $89,546,253. 

Source: GAO analysis of U.S. Census Bureau information. 

[A] Cost of the most recent purchase. 

[End of table] 

[End of section] 

Appendix IV: Comments from the Department of Commerce: 

United States Department of Commerce: 
The Secretary of Commerce: 
Washington, D.C. 20230: 

October 22, 2012: 

Mr. Robert Goldenkoff: 
Director, Strategic Issues: 
U.S. Government Accountability Office: 
Washington, DC 20548: 

Dear Mr. Goldenkoff: 

The U.S. Department of Commerce appreciates the opportunity to comment 
on the U.S. Government Accountability Office's draft report entitled, 
"2020 Census: initial Research Milestones Generally Met But Plans 
Needed to Mitigate Highest Risks" (GA0-13-53) The Department of 
Commerce's comments on this report are enclosed. 

Sincerely, 

Signed by: 

Rebecca M. Blank: 
Acting Secretary of Commerce: 

Enclosure: 

U.S. Department of Commerce Comments on the U.S. Government 
Accountability Office Draft Report Entitled "2020 Census: Initial 
Research Milestones Generally Met But Plans Needed to Mitigate Highest 
Risks" (GAO-13-53), October 2012: 

The U,S. Census Bureau appreciates the opportunity to comment on this 
draft U.S. Government Accountability Office (GAO) report. We believe 
that this report accurately represents the extensive work that has 
been completed during the early research and testing phase for the 2020
Census. The Census Bureau also agrees with the report's assessments 
and recommendations about additional efforts that are needed to 
develop risk mitigation plans, contingency plans, and cost estimates 
at the project level. 

Recognizing that these are areas that need to be strengthened based on 
the 2010 Census experience, work up to now has been focused on 
developing a strong risk register for the 2020 Research and Testing 
Program at the program level. That risk register is now being expanded 
to address program level risks for the entire 2020 Census Lifecycle. 
We will also he focusing on managing risks at the project level. In 
addition, cost estimation work has been done for the 2020 Program with 
the initial development of a Rough Order of Magnitude costing out 
potential design alternatives. A team has been formed to mature this 
cost estimation work and to develop a lifecycle plan for the 202(1 
Census to ensure that the research and testing phase will lead us to 
the timely determination of optimal design decisions that will reduce 
the cost of the Census while maintaining data quality and minimizing 
risk. Although we had basic cost estimates for the work of each early 
research project, we will now focus on obtaining more specific cost 
estimation details at the project level for each project. In addition, 
the Census Bureau agrees with recommendations to ensure that 
performance metrics and skill sets are identified for all projects and 
teams. These efforts are underway or planned as a major focus of the 
program during Fiscal Year 2013. 

The Census Bureau has one substantive comment on this draft report. On 
page 13, the 2nd paragraph states that, "For 2020, the Bureau 
estimates that by expanding the use of administrative records it could 
save up to $2 billion by reducing the workload for several operations, 
including NRFU, address building, and quality assurance." As written, 
this statement does not reflect the preliminary nature of this 
estimate or the likelihood that the estimate will change as we learn 
from our research and refine our cost estimation efforts. Hence, the 
Census Bureau requests the wording he modified to something like: "In 
its first attempt to estimate potential costs of various design 
options for the 2020 Census, the Census Bureau estimated that it could 
save up to $2 billion by expanding the use of administrative records." 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov: 

Staff Acknowledgments: 

Other key contributors to this report include Lisa Pearson, Assistant 
Director; David Bobruff; Robert Gebhart; Richard Hung; Kirsten B. 
Lauber; Andrea Levine; and Timothy Wexler. 

[End of section] 

Related GAO Products: 

2020 Census: Sustaining Current Reform Efforts Will Be Key to a More 
Cost-Effective Enumeration. [hyperlink, 
http://www.gao.gov/products/GAO-12-905T]. Washington, D.C.: July 18, 
2012. 

2020 Census: Additional Steps Are Needed to Build on Early Planning. 
[hyperlink, http://www.gao.gov/products/GAO-12-626]. Washington, D.C.: 
May 17, 2012. 

Decennial Census: Additional Actions Could Improve the Census Bureau's 
Ability to Control Costs for the 2020 Census. [hyperlink, 
http://www.gao.gov/products/GAO-12-80]. Washington, D.C.: January 24, 
2012. 

2010 Census: Preliminary Lessons Learned Highlight the Need for 
Fundamental Reforms. [hyperlink, 
http://www.gao.gov/products/GAO-11-496T]. Washington, D.C.: April 6, 
2011. 

2010 Census: Data Collection Operations Were Generally Completed as 
Planned, but Long-standing Challenges Suggest Need for Fundamental 
Reforms. [hyperlink, http://www.gao.gov/products/GAO-11-193]. 
Washington, D.C.: December 14, 2010. 

GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs. [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP]. Washington D.C.: March 2, 
2009. 

2010 Census: Census Bureau Should Take Action to Improve the 
Credibility and Accuracy of Its Cost Estimate for the Decennial 
Census. [hyperlink, http://www.gao.gov/products/GAO-08-554]. 
Washington, D.C.: June 16, 2008. 

Information Technology: Significant Problems of Critical Automation 
Program Contribute to Risks Facing 2010 Census. [hyperlink, 
http://www.gao.gov/products/GAO-08-550T]. Washington, D.C.: March 5, 
2008. 

2010 Census: Cost and Design Issues Need to Be Addressed Soon. 
[hyperlink, http://www.gao.gov/products/GAO-04-37]. Washington, D.C.: 
January 15, 2004. 

High-Risk Series: Quick Reference Guide. [hyperlink, 
http://www.gao.gov/products/GAO/HR-97-2]. Washington, D.C.: February 
1997. 

[End of section] 

Footnotes: 

[1] Administrative records refer to data records contained in files 
collected and maintained by administrative or program agencies and 
commercial entities. Government and commercial entities maintain these 
files for the purpose of administering programs and providing services. 

[2] Per housing unit, on an inflation-adjusted basis. 

[3] The Bureau's Guiding Principles for the 2020 Census are: Honor 
Privacy and Confidentiality; Be Objective; Commit to Quality; Contain 
Costs Wherever Possible; Leverage Assets, Methods, Knowledge, and 
Processes across the Bureau; and Be Responsive and Transparent to 
Partners and Stakeholders. 

[4] A Bureau official stated that they are assessing the impact of the 
fiscal year 2013 Continuing Resolution on their ability to complete 
all of their work by September 2014. 

[5] Census Bureau, Rationale for the Decision to Eliminate the 
Internet Option for the 2010 Decennial Census, 2010 Decennial Census 
Program Decision Memorandum Series No. 14 (Washington, D.C.: July 19, 
2006). 

[6] During NRFU the Bureau sends enumerators to collect data from 
households that did not mail back their census forms. NRFU procedures 
instruct enumerators to make up to six attempts to contact a 
household. The 2010 Census NRFU operation cost $1.6 billion. 

[7] We have a separate review underway examining the Bureau's 
information technology security issues. 

[8] ACS is an ongoing survey that provides data every year. 
Information from the survey generates data that help determine how 
federal and state funds are distributed each year. 

[9] GAO, 2010 Census: Data Collection Operations Were Generally 
Completed as Planned, but Long-standing Challenges Suggest Need for 
Fundamental Reforms, [hyperlink, 
http://www.gao.gov/products/GAO-11-193] (Washington, D.C.: Dec. 14, 
2010). 

[10] The LUCA Program was established by the Census Address List 
Improvement Act of 1994 to improve the accuracy of census' address 
list by exchanging information with tribal, state, and local 
governments. 

[11] The Bureau's access to and use of administrative records is 
governed by agency-specific statutes. For example, the Bureau has 
access to tax data under 26 U.S.C. § 6103(j)(1) "for the purpose of, 
but only to the extent necessary in, the structuring of censuses ... 
and conducting related statistical activities." 

[12] The amount of and quality of administrative records the Bureau is 
able to collect will impact the amount of cost savings the Bureau is 
able to realize. 

[13] Broadly defined, risk management is a strategic process for 
helping policymakers make decisions about assessing risk, allocating 
finite resources, and taking actions under conditions of uncertainty. 

[14] A risk register contains the results of a projects risk analysis 
and includes risk descriptions, probability of occurrence, cost 
impact, schedule impact, and identifies the Bureau official with 
ownership of the risk. 

[15] The Family Educational Rights and Privacy Act protects the 
privacy of student education records including personally identifiable 
information (name, address, social security number, etc.) from those 
records. The National Directory of New Hires contains wage and 
employment information that is used by state agencies to enforce child 
support payments. 

[16] 13 U.S.C. § 9. 

[17] GAO, Information Technology: Significant Problems of Critical 
Automation Program Contribute to Risks Facing 2010 Census, [hyperlink, 
http://www.gao.gov/products/GAO-08-550T] (Washington, D.C.: Mar. 5, 
2008). 

[18] GAO, 2010 Census: Cost and Design Issues Need to Be Addressed 
Soon, [hyperlink, http://www.gao.gov/products/GAO-04-37] (Washington, 
D.C.: Jan. 15, 2004). 

[19] GAO, 2010 Census: Census Bureau Should Take Action to Improve the 
Credibility and Accuracy of Its Cost Estimate for the Decennial 
Census, [hyperlink, http://www.gao.gov/products/GAO-08-554] 
(Washington, D.C.: Jun. 16, 2008). 

[20] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington D.C.: Mar. 2, 
2009). 

[21] The LUCA Program Improvement and Supplementing and Supporting 
NRFU with Administrative Records project teams did not provide 
documentation that a skills gap analysis was conducted. The Enhancing 
Demographic Analysis project team did not provide the necessary skill 
sets. 

[22] The six project teams were: Master Address File (MAF) Error 
Model, Independent MAF Quality Assessment, LUCA Program Improvement, 
Workload Management, Matching Process Improvement, and Contact Frame. 

[23] [hyperlink, http://www.gao.gov/products/GAO-04-37]. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]