This is the accessible text file for GAO report number GAO-12-80 
entitled 'Decennial Census: Additional Actions Could Improve the 
Census Bureau's Ability to Control Costs for the 2020 Census' which 
was released on February 23, 2012.  

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

January 2012: 

Decennial Census: 

Additional Actions Could Improve the Census Bureau's Ability to 
Control Costs for the 2020 Census: 

GAO-12-80: 

GAO Highlights: 

Highlights of GAO-12-80, a report to congressional requesters. 

Why GAO Did This Study: 

A complete count of the nation’s population is an enormous challenge 
requiring the U.S. Census Bureau (Bureau) to balance requirements for 
accuracy with the need to control escalating costs. The 2010 Census 
was the costliest U.S. Census in history at about $13 billion, and was 
about 56 percent more costly than the $8 billion cost of the 2000 
Census (in 2010 dollars). The fundamental challenge facing the Bureau 
going forward is cost effectively counting a population that is 
growing steadily larger, more diverse and becoming increasingly 
difficult to enumerate. As requested, this report assesses (1) the key 
factors affecting cost growth from the 2000 Census to the 2010 Census; 
(2) the Bureau’s plans for controlling costs for the 2020 Census and 
what additional steps, if any, could be taken; and (3) the extent to 
which the Bureau’s plans for developing life cycle cost estimates for 
2020 are consistent with best practices. The report is based on GAO’s 
analysis of Bureau data and documents as well as interviews with 
Bureau officials. 

What GAO Found: 

The average cost to count each housing unit rose from $70 in 2000 to 
$97 in 2010 (in constant 2010 dollars). While the U.S. Census Bureau 
(Bureau) made changes to its budget structure from 2000 to 2010, they 
did not document the changes that would facilitate comparisons over 
time and cannot identify specific drivers of this cost growth. 
According to GAO’s Cost Estimating and Assessment Guide, an agency can 
strengthen its ability to control costs by using available cost data 
to make comparisons over time and identify and quantify trends. The 
Bureau faces the fundamental challenge of striking a balance between 
how best to control costs without compromising accuracy. However, the 
Bureau’s inability to identify specific actionable factors affecting 
past growth will make it difficult for the Bureau to focus its efforts 
to control costs for the 2020 Census. 

The Bureau developed several design alternatives for the 2020 Census 
that could help reduce costs, but has not identified decision points 
when executives would review progress and decide whether the Bureau is 
prepared to move forward from one project phase to another. Office of 
Management and Budget guidance and previous GAO work support the use 
of these practices to strengthen an agency’s decision making on large-
scale projects. Incorporating these practices in its 2020 planning 
could help the Bureau improve its ability to manage risk to achieve 
desired cost, schedule and performance outcomes. 

The Bureau is taking steps to strengthen its life cycle cost 
estimates. However, the Bureau has not yet established guidance for 
developing cost estimates. The Bureau is scheduled to begin work on 
the 2020 Census estimate in fiscal year 2013 but has limited time to 
develop guidance. By finalizing such guidance, the Bureau can better 
ensure that it is developing comprehensive, accurate, and credible 
estimates for the 2020 Census. 

Figure: The Average Cost of Counting Each Housing Unit Escalated Each 
Decade While Mail Response Rates Declined: 

[Refer to PDF for image: combined vertical bar and line graph] 

Year: 1970; 
Average cost per housing unit (in constant 2010 dollars): $16; 
Mail response rate: 78%. 

Year: 1980; 
Average cost per housing unit (in constant 2010 dollars): $30; 
Mail response rate: 75%. 

Year: 1990; 
Average cost per housing unit (in constant 2010 dollars): $39; 
Mail response rate: 66%. 

Year: 2000; 
Average cost per housing unit (in constant 2010 dollars): $70; 
Mail response rate: 66%. 

Year: 2010; 
Average cost per housing unit (in constant 2010 dollars): $97 
(estimated); 
Mail response rate: 63%. 

Source: GAO analysis of U.S. Census Bureau data. 

[End of figure] 

What GAO Recommends: 

GAO recommends that the Census Director develop a method to identify 
and address specific factors that contribute to cost increases, 
identify decision points, and finalize guidance for the 2020 life 
cycle cost estimate. The Department of Commerce expressed broad 
agreement with the overall theme of the report but did not directly 
comment on the recommendations. It raised concerns about specific 
aspects of the summary of findings which GAO addressed as appropriate. 

View [hyperlink, http://www.gao.gov/products/GAO-12-80]. For more 
information, contact Robert Goldenkoff at (202) 512-2757 or 
goldenkoffr@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

The Bureau Cannot Identify Specific Sources of Cost Growth from 2000 
to 2010: 

The Bureau Is Assessing Options for Controlling Field Costs, but 
Additional Steps Could Be Taken: 

Without a Process in Place for Developing 2020 Census Life Cycle Cost 
Estimates, the Bureau Is at Risk of Not Following Best Practices: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objective, Scope, and Methodology: 

Appendix II: Comments from the Department of Commerce: 

Appendix III: 2010 Census Costs Were Concentrated in Certain Budget 
Categories and Projects: 

Appendix IV: The 12 Steps of a High-Quality Cost Estimating Process: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Table: 

Table 1: Field Data Collection Costs Accounted for the Largest Growth 
from 2000 to 2010: 

Figures: 

Figure 1: The Average Cost of Counting Each Housing Unit Escalated 
Each Decade While Mail Response Rates Declined: 

Figure 2: 2010 Census Costs Were Concentrated in Certain Budget 
Categories and Projects: 

Figure 3: Cost per Housing Unit Increased at a Faster Rate Than 
Workload from 2000 to 2010: 

Figure 4: 2020 Census Design Alternatives Show Varying Degrees of 
Change, Risk, and Potential Cost Savings: 

Figure 5: Estimated Marginal Cost of Checking in NRFU Questionnaires 
Dramatically Rose in the Final Weeks of the Operation: 

Figure 6: Executive-Level Review at Decision Points Could Improve 2020 
Census Planning: 

Abbreviations: 

CBS: Commerce Business System: 

CPEX: Census Program for Evaluations and Experiments: 

IT: information technology: 

LCO: local census office: 

LUCA: Local Update of Census Addresses: 

NRFU: nonresponse follow-up: 

OMB: Office of Management and Budget: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

January 24, 2012: 

Congressional Requesters: 

Although the 2020 Census is still approximately 8 years away, the U.S. 
Census Bureau (Bureau) has already begun critical research and testing 
activities. The cost of the decennial census has steadily increased 
during the past 40 years, in part because of increasing difficulty in 
accurately counting a population that is growing larger, more diverse, 
and increasingly difficult to enumerate. For example, at about $13 
billion, the 2010 Census was the costliest U.S. census in history, and 
was 56 percent more costly than the $8.1 billion 2000 Census (in 
constant 2010 dollars). Further, based on past trends, if the growth 
rate continues unchecked the census could cost approximately $25 
billion in 2020. Thus, as the Bureau plans for the 2020 Census, it 
faces the fundamental challenge of striking a balance between how best 
to control costs without compromising accuracy. The Bureau is well 
aware of this, and its business plan, which describes its efforts for 
the early research and testing phase of the 2020 Census, notes that it 
is committed to planning and implementing a 2020 Census that costs 
less than the approximately $100 per housing unit that was spent on 
the 2010 Census.[Footnote 1] 

In response to your request that we review the Bureau’s cost control 
efforts for the 2020 Census, this report assesses (1) the key factors 
affecting cost growth from the 2000 Census to the 2010 Census; (2) the 
Bureau’s plans for controlling costs for the 2020 Census and what 
additional steps, if any, could be taken; and (3) the extent to which 
the Bureau’s plans for developing life cycle cost estimates for 2020 
are consistent with best practices. 

In reviewing key factors affecting cost growth, we reviewed and 
analyzed actual cost data from the Bureau’s financial management 
system for the 2000 Census life cycle and the 2010 Census life cycle. 
[Footnote 2] We adjusted the data for inflation and compared broad 
budget categories from the 2000 and 2010 Censuses; however, 
limitations in cost data made it impossible to conduct comprehensive 
comparisons below the category level. In addition, we reviewed Bureau 
documentation and interviewed officials to identify factors that could 
have driven cost growth. To assess the Bureau’s cost control efforts 
for the 2020 Census and its plans for developing life cycle cost 
estimates, we reviewed available planning documents for the 2020 
Census and interviewed Bureau officials involved with the planning 
process. We used GAO’s Cost Estimating and Assessment Guide to 
identify best practices for cost estimation.[Footnote 3] For more 
information on our scope and methodology (see appendix I). 

We conducted this performance audit from December 2010 through January 
2012 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

On January 6, 2012, the Secretary of Commerce provided written 
comments on a draft of this report (see appendix II). 

Background: 

The cost of the census, in terms of cost for counting each housing 
unit, has been escalating over the last several censuses. The average 
cost for counting a housing unit increased from about $16 in 1970 to 
around $97 in 2010 constant dollars (see figure 1). Meanwhile, the 
return of census questionnaires by mail (the primary mode of data 
collection) declined over this period from 78 percent in 1970 to 63 
percent in 2010. Declining mail response rates are significant and 
have led to higher costs because the mail response rate directly 
dictates the number of housing units in the nonresponse follow-up 
(NRFU) universe. NRFU, where the Bureau attempts to contact households 
that did not mail back questionnaires, was the largest and most costly 
Bureau field operation in 2000 and 2010 and has had an impact on 
overall census costs. 

Figure 1: The Average Cost of Counting Each Housing Unit Escalated 
Each Decade While Mail Response Rates Declined: 

[Refer to PDF for image: combined vertical bar and line graph] 

Year: 1970; 
Average cost per housing unit (in constant 2010 dollars): $16; 
Mail response rate: 78%. 

Year: 1980; 
Average cost per housing unit (in constant 2010 dollars): $30; 
Mail response rate: 75%. 

Year: 1990; 
Average cost per housing unit (in constant 2010 dollars): $39; 
Mail response rate: 66%. 

Year: 2000; 
Average cost per housing unit (in constant 2010 dollars): $70; 
Mail response rate: 66%. 

Year: 2010; 
Average cost per housing unit (in constant 2010 dollars): $97 
(estimated)[A]; 
Mail response rate: 63%[B]. 

Source: GAO analysis of U.S. Census Bureau data. 

[A] The 2010 life cycle runs from 2002 through 2013, so costs for the 
2010 Census are not yet final. 

[B] In the 2010 Census, the Bureau used only a short-form 
questionnaire. For this report, we use the 1990 and 2000 Census short-
form mail response rate when comparing 1990, 2000, and 2010 mail-back 
response rates. Census short-form mail response rates are unavailable 
for 1970 and 1980, so we use the overall response rate. 

[End of figure] 

Over the past several censuses, the Bureau has attempted to address 
the competing goals of containing costs and improving the quality of 
census information, but costs continued to rise in part because 
external factors, such as a growing and increasingly diverse 
population, required the Bureau to devote more resources in order to 
ensure a complete count. The Bureau is assessing various measures of 
the quality of the 2010 Census. This effort, combined with a better 
understanding of the specific sources of cost growth, could help 
managers make cost control decisions. 

Within its financial management system, the Bureau classifies census 
costs into eight broad categories and hundreds of projects (see figure 
2). These broad categories are further subdivided into individual 
projects that may be discrete, such as the NRFU operation, which has 
its costs captured in a single project line, or several project lines 
may be combined, sometimes from multiple categories, to reflect the 
total cost of an operation, as is the case with the Local Update of 
Census Addresses (LUCA) operation.[Footnote 4] The 2010 Census costs 
were concentrated in few categories. 

Figure 2: 2010 Census Costs Were Concentrated in Certain Budget 
Categories and Projects: 

[Refer to PDF for image: interactive graphic] 

Directions: 
Click each tab below to see characteristics of the highest cost 
projects within each Decennial Census 2010 budget category. 

Click to make view needed visible. In the “Print” dialog box, choose 
“Current page,” then “OK.” Repeat for each view. 

A print version of this graphic is also available in appendix III. 

Source: GAO analysis of U.S. Census Bureau data. 

Notes: See appendix III for a noninteractive breakout of all the data. 
Some projects have the same name, but these costs do not overlap with 
one another. The money was spent in different time periods for each 
project. All costs were adjusted to 2010 dollars using the gross 
domestic product deflator. The Bureau identified one project that did 
not fit into any regular category, the project labeled "adjustments." 
Changes were made to project descriptions to increase clarity. For 
example, abbreviations were expanded into full words. 

[End of figure] 

Planning for the 2020 Census is divided into five phases: (1) options 
analysis; (2) research and testing; (3) operational development and 
systems testing; (4) supplemental research and testing; and (5) 
readiness testing, execution, and closeout. The Bureau has identified 
a range of design alternatives for the 2020 Census and will narrow 
this range over the census life cycle. During fiscal year 2012, the 
Bureau will enter the research and testing phase and intends to 
develop a preliminary design that when adjusted for inflation will 
cost less than the $97 per housing unit cost of the 2010 Census but 
will also maintain quality. During the research and testing phase, the 
Bureau plans to execute at least 35 research projects to explore how 
design areas could be modified to control costs or improve quality. 
For example, the Bureau will examine the feasibility of using 
administrative records, such as Internal Revenue Service tax records, 
to collect information from nonresponders and thus reduce the 
fieldwork. Other research areas include new response options, such as 
the Internet and social networking sites. 

The Bureau uses life cycle cost estimates as a starting point for 
annual budget formulation and revises the estimates based on 
appropriations and updated budget information. As noted in our Cost 
Estimating and Assessment Guide, a life cycle cost estimate can be 
thought of as a "cradle to grave" approach to managing a program 
throughout its duration.[Footnote 5] However, in our past work, we 
found that the Bureau's 2010 Census life cycle cost estimate was not 
reliable because it lacked adequate documentation and was not 
comprehensive, accurate, or credible.[Footnote 6] The Bureau may 
continue to be challenged in developing reliable life cycle cost 
estimates for a program as large, costly, and complex as the census. 
As part of its planning for 2020, the Bureau has developed an early 
life cycle cost estimate based on existing information and plans to 
release a full range of life cycle cost estimates in the budget 
submission for fiscal year 2015. 

The Bureau Cannot Identify Specific Sources of Cost Growth from 2000 
to 2010: 

Field Data Collection Drove Cost Increases, but More Details Are 
Needed to Help Control Future Costs: 

Of the Bureau's eight broad budget categories, field data collection 
and its associated support systems accounted for $3.5 billion of the 
$4.6 billion life cycle cost increase, or 77 percent of the overall 
cost growth from 2000 to 2010 (see table 1). This represents a 64 
percent growth in the field data collection category from its 2000 
totals, which was the largest percentage increase of all budget 
categories. Field data collection costs include training, labor, and 
mileage for temporary workers, as well as the support systems needed 
to run operations, including rental space and office equipment for 
local census offices (LCO). We previously reported that the field data 
collection budget category was also the largest contributor to cost 
growth from the 1990 Census to the 2000 Census.[Footnote 7] 

The remaining seven budget categories accounted for less than 25 
percent of overall cost growth. The automated data collection category 
experienced the second largest growth, accounting for 12 percent of 
overall cost growth from 2000 to 2010. Expenses in this category were 
$547 million more than in 2000, a 42 percent increase. This category 
includes data processing activities and related information technology 
(IT) system costs. Smaller categories experienced cost growth as well, 
including content, questionnaires, and products; census design, 
methodology, and evaluation; and census test and dress rehearsal. 
Other categories actually experienced cost decreases, including 
program development and management. 

Table 1: Field Data Collection Costs Accounted for the Largest Growth 
from 2000 to 2010: 

2010 budget category: 1. Field data collection and support systems; 
2000 Census costs[A]: $5.509 billion; 
2010 Census costs: $9.052 billion; 
Absolute change[B]: $3.543 billion; 
Percentage change within category: 64%; 
Percentage of total cost growth by category: 77%. 

2010 budget category: 2. Automated data collection, systems, and data 
capture; 
2000 Census costs[A]: $1.298 billion; 
2010 Census costs: $1.844 billion; 
Absolute change[B]: $547 million; 
Percentage change within category: 42%; 
Percentage of total cost growth by category: 12%. 

2010 budget category: 3. Content, questionnaires, and products; 
2000 Census costs[A]: $670 million; 
2010 Census costs: $1.045 billion; 
Absolute change[B]: $375 million; 
Percentage change within category: 56%; 
Percentage of total cost growth by category: 8%. 

2010 budget category: 4. Census design, methodology, and evaluation; 
2000 Census costs[A]: $372 million; 
2010 Census costs: $531 million; 
Absolute change[B]: $159 million; 
Percentage change within category: 43%; 
Percentage of total cost growth by category: 3%. 

2010 budget category: 5. Program development and management[C]; 
2000 Census costs[A]: $192 million; 
2010 Census costs: $70 million; 
Absolute change[B]: -$122 million; 
Percentage change within category: -64%; 
Percentage of total cost growth by category: Did not contribute to 
cost growth[D]. 

2010 budget category: 6. Census test and dress rehearsal 
implementation; 
2000 Census costs[A]: $90 million; 
2010 Census costs: $108 million; 
Absolute change[B]: $18 million; 
Percentage change within category: 20%; 
Percentage of total cost growth by category: Did not contribute to 
cost growth[D]. 

2010 budget category: 7. Contingency[E]; 
2000 Census costs[A]: Did not exist; 
2010 Census costs: $78 million; 
Absolute change[B]: $78 million; 
Percentage change within category: N/A; 
Percentage of total cost growth by category: 2%. 

2010 budget category: 8. Uncategorized expenditures; 
2000 Census costs[A]: $31 million; 
2010 Census costs: $28 million; 
Absolute change[B]: -$4 million; 
Percentage change within category: -12%; 
Percentage of total cost growth by category: Did not contribute to 
cost growth[D]. 

2010 budget category: Total; 
2000 Census costs[A]: $8.161 billion; 
2010 Census costs: $12.754 billion; 
Absolute change[B]: $4.593 billion; 
Percentage change within category: 56%; 
Percentage of total cost growth by category: 100%. 

Source: GAO analysis of U.S. Census Bureau data. 

Note: All costs are adjusted to 2010 dollars using a gross domestic 
product price deflator. 

[A] The 2000 Census budget categories were different from 2010 budget 
categories. The Bureau remapped 2000 Census costs from 2000 budget 
categories into 2010 budget categories to facilitate comparison. 
Remapping was not possible at the more detailed budget project level. 

[B] Numbers may not subtract correctly because of rounding. 

[C] Program development and management costs decreased from 2000 to 
2010 because the Bureau shifted the cost of project managers' pay out 
of the program development and management category and into the other 
categories that contained the actual projects. 

[D] Categories that did not contribute to cost growth either accounted 
for a 0 (#6) or a negative percentage (#5 and $8) of total cost growth 
by category. 

[E] The contingency category was established as part of risk 
mitigation planning by the Bureau in 2010 to prepare for unforeseen 
events, such as an unexpected drop in mail response rates. This 
category also included funding for fingerprinting activities. 

[End of table] 

According to the Bureau, an increased workload--a larger number of 
housing units to count--is one of the factors driving up census costs. 
This, however, does not fully explain (1) why the cost to count each 
housing unit grew at a faster pace than the workload (39 percent 
increase to count each housing unit compared to 12 percent increase in 
workload) or (2) why component costs, such as data capture systems, 
experienced cost increases (see figure 3). 

To more fully understand what is driving up census costs aside from an 
increase in workload, it will be important for the Bureau to analyze 
cost growth below the category level to determine the specific reasons 
why cost per housing unit continues to grow at a faster pace than 
workload. Key questions in this regard include, for example, (1) to 
what extent did increased labor and gasoline costs contribute to 
overall increases in field data collection costs, (2) how did 
additional use of technology contribute to field data collection 
costs, (3) how did increased investments in non-field-related IT 
systems affect cost growth, and (4) to what extent did the weak 
economy in 2010 help the Bureau reduce costs for field operations. 
While some cost increases, such as rising gasoline prices, might have 
been outside of the Bureau's direct control, better information on the 
sources of census cost growth could enable the Bureau to develop work-
arounds and alternatives that could mitigate their impact. 

Figure 3: Cost per Housing Unit Increased at a Faster Rate Than 
Workload from 2000 to 2010: 

[Refer to PDF for image: 3 vertical bar graphs] 

Total cost per census: 
2000 Census: $8.2 billion; 
2010 Census: $12.8 billion; 
Percent change, 2000-2010 censuses[A]: 56%. 

Total number of housing units: 
2000 Census: 117.3 million; 
2010 Census: 131.7 million; 
Percent change, 2000-2010 censuses[A]: 12%. 

Total cost per housing unit: 
2000 Census: $70; 
2010 Census: $97; 
Percent change, 2000-2010 censuses[A]: 39%. 

Breakdown of cost per housing unit by budget category: 

Field data collection and support systems: 
2000 Census: $47; 
2010 Census: $69; 
Percent change, 2000-2010 censuses[A]: 46%. 

Automated data collection, systems, and data capture: 
2000 Census: $11; 
2010 Census: $14; 
Percent change, 2000-2010 censuses[A]: 27%; 

Content, questionnaires, and products: 
2000 Census: $6; 
2010 Census: $8; 
Percent change, 2000-2010 censuses[A]: 39%. 

Remaining categories: 
2000 Census: $6; 
2010 Census: $6; 
Percent change, 2000-2010 censuses[A]: 6%. 

Source: GAO analysis of U.S. Census Bureau data. 

Note: All costs are adjusted to 2010 dollars using a gross domestic 
product price deflator. 

[A] Percentage growth may not exactly match numbers in figure because 
of rounding. 

[End of figure] 

Additional Documentation Could Position the Bureau to Better Identify 
Specific Sources of Cost Changes: 

Best practices in GAO's Cost Estimating and Assessment Guide 
illustrate how an agency can strengthen its ability to control costs 
by using available cost data to make comparisons over time and 
identify and quantify trends.[Footnote 8] However, the Bureau cannot 
identify specific sources of cost growth below broad budget categories 
from 2000 to 2010 because the Bureau changed the way it defines 
projects without creating a crosswalk that documents the changes over 
time. As a result, the Bureau cannot specifically determine where 
costs are growing. 

While it is reasonable for the Bureau to modify its budget structure 
to accommodate changes from one decennial to the next, a crosswalk 
would have enabled officials to compare costs for specific projects. 
For example, for the 2000 Census, 236 projects were identified in the 
budget. For the 2010 Census, the Bureau changed its budget structure 
to more precisely capture costs, and as a result, the number of 
projects listed in the budget increased by almost 400 percent to 1,175 
projects. However, the Bureau created no documentation to facilitate 
comparison for most projects in the budget from 2000 to 2010. For 
example, costs for LUCA are combined into one project in 2000 data 
while in 2010 data, LUCA activities were identified in 11 separate 
projects (for example, LUCA processing and LUCA testing). Without 
documentation explaining what costs were included in LUCA for the 2000 
Census, it is impossible to accurately compare costs for LUCA between 
the two decennials and determine where any cost growth might have 
occurred. 

Further, the Bureau cannot accurately calculate the growth in field 
infrastructure costs, if any, from 2000 through 2010 because of a 
similar lack of documentation. Although the $2 billion the Bureau 
spent on its field infrastructure in 2010--including 12 regional 
census centers and almost 500 LCOs used to support field activities--
represented a major investment, the Bureau lacks the information 
needed to accurately compare the costs of specific components from one 
decennial to the next. Such information would enable the Bureau to 
more accurately determine where any significant cost increases 
occurred and thus better focus its cost control efforts for the 2020 
Census, as well as allow the Bureau to more precisely determine the 
potential cost savings of any operational changes. 

Although these structural changes are recent, the absence of 
documentation has been a challenge in the past as well. In a prior 
report comparing costs for the 1990 and 2000 censuses, we were unable 
to compare costs at the project level because of limitations in the 
available data and documentation.[Footnote 9] For the 1990 Census, the 
Bureau provided limited cost data by activity and project, so we were 
not able to attempt detailed cost comparisons. Moving forward, it will 
be important for the Bureau to put a process in place to enhance its 
ability to identify potential factors affecting cost growth and, if 
necessary, target cost control efforts appropriately. 

Cost Factors Identified by the Bureau Are of Limited Use: 

Although the Bureau identified five broad factors affecting cost 
growth, their ability to help the agency pinpoint and control future 
costs is limited because they mainly focus on high-level, generic 
management challenges rather than specific census-taking activities on 
which the Bureau can assess and take action as appropriate. 
Additionally, the Bureau has no data to support how much these factors 
contributed to cost growth. The five factors include: 

1. the increasing diversity of the population; 

2. the demand for the Census Bureau to strive for improving accuracy 
over previous censuses; 

3. the lack of full public participation in the self-response phase of 
the census, requiring the hiring of a large field staff for NRFU; 

4. the failure or challenges with linking major acquisitions, the 
schedule, and the budget; and: 

5. substantial investments in major national updating of the address 
frame just prior to enumeration (2009).[Footnote 10] 

The Bureau plans to use these factors to guide 2020 Census planning 
and research efforts. For example, the forthcoming research and 
testing phase will focus on the decreasing self-response rate; the 
linkage of acquisitions, schedule, and budget; and updates to the 
address frame. While these factors, which the Bureau developed through 
management experience, likely affected the cost of the census, 
evaluating the extent to which specific operations and activities 
drove up census costs would provide the Bureau with more actionable 
information. As one example, the Bureau identified the demand for 
improved accuracy as a factor, but this effort to improve accuracy 
involved a number of operations aimed at producing a more complete 
count, ranging from advertising in different languages to special 
enumeration programs aimed at hard-to-count populations. What is not 
clear, and will be important for the Bureau to determine, is how the 
cost of the special enumeration programs compared to those for 2000, 
the extent to which they contributed to the cost of the 2010 Census, 
and whether they produced the desired results. 

The Bureau Is Assessing Options for Controlling Field Costs, but 
Additional Steps Could Be Taken: 

The Bureau Is Considering Design Alternatives to Potentially Reduce 
Costs for the 2020 Census: 

The Bureau has developed a range of design alternatives for the 2020 
Census aimed at counting each housing unit at a lower cost than in 
2010. The Bureau estimated that if it repeated the design of the 2010 
Census, and assuming real costs grow at the same rate they did between 
1990 and 2010, it would cost $151 to count each housing unit--more 
than a 55 percent increase, compared to 2010. The challenge for the 
Bureau, as recognized in its 2020 Census business plan, is striking a 
balance between an accurate census, on the one hand, and reducing 
costs and managing risks, on the other.[Footnote 11] 

The Bureau's 2020 design alternatives have potential for containing 
costs but at varying degrees of risk for meeting cost, schedule, and 
performance goals. The design alternatives focus on options to target 
address canvassing, using the Internet and other social media to 
increase response rates, and reengineer field and IT infrastructures. 
[Footnote 12] Figure 4 shows the current range of 2020 Census design 
alternatives. According to the Bureau, the final 2020 design is likely 
to incorporate both existing approaches as well as activities that 
have never been used in the decennial census, such as a near paperless 
NRFU. 

Figure 4: 2020 Census Design Alternatives Show Varying Degrees of 
Change, Risk, and Potential Cost Savings: 

[Refer to PDF for image: illustration] 

Address canvassing and continuous frame updating: 
Alternative 1: Full address canvassing (similar to 2010 design); 
Alternative 2: Targeted address canvassing; 
Alternative 3: Targeted address canvassing; 
Alternative 4: Targeted address canvassing; 
Alternative 5: No address canvassing; 
Alternative 6: No address canvassing. 

Enumeration: 
Alternative 1: Traditional plus Internet; 
Alternative 2: Multimode contact (e-mail, text, social networking, 
etc.); 
Alternative 3: Visit then use administrative records; 
Alternative 4: Use administrative records then visit; 
Alternative 5: Near paperless; 
Alternative 6: Administrative records based. 

Field Infrastructure: 
Alternative 1: Decentralized infrastructure; 
Alternative 2: Decentralized infrastructure; 
Alternative 3: Hybrid infrastructure; 
Alternative 4: Hybrid infrastructure; 
Alternative 5: Centralized infrastructure; 
Alternative 6: Centralized infrastructure. 
               
Alternatives 1-6:            
Least change from 2010 design; Lowest risk; Less cost savings; 
to: 
Greater change from 2010 design; Higher risk; Potential for greater 
cost savings. 

Source: GAO analysis of U.S. Census Bureau information. 

[End of figure] 

According to the Bureau, the greater the change to the overall design, 
the greater the potential for cost savings. However, greater design 
changes also incur greater risk, and further testing will be needed to 
identify the risks, costs, and benefits of any new approaches. 
According to the Bureau, alternative one has the lowest risk, as it 
most closely mirrors the 2010 Census design and is not dependent on 
implementing innovations such as administrative records and targeted 
address canvassing. The remaining alternatives incorporate varying 
degrees of centralized infrastructure; address canvassing; and use of 
administrative records, the Internet, and social networks. For 
example, most of the new design options use administrative records, 
which could save money by reducing labor-intensive and costly field 
operations. Yet, the Bureau has not previously used administrative 
records to supplement respondent data on a national level, so the new 
approach will present a certain degree of risk as such things as data 
quality and access to records must first be resolved. 

Additional Analysis of 2010 Data Could Improve the Bureau's Ability to 
Make Cost Decisions about 2020: 

The Bureau collects data on the costs of its field operations that are 
a potentially valuable source of information to help guide future cost 
quality trade-off decisions during the planning process. However, it 
could make better use of this information in gaining an understanding 
of return on investment for costly census-taking activities, such as 
address building and NRFU. According to Office of Management and 
Budget (OMB) guidance on benefit-cost analysis, agencies should have a 
plan for periodic, results-oriented evaluation of the effectiveness of 
federal programs. The guidance also notes that retrospective studies 
can be valuable in determining if any corrections need to be made to 
existing programs and to improve future estimates of other federal 
programs.[Footnote 13] In addition, our Cost Estimating and Assessment 
Guide suggests that agencies should seek the best value solution by 
gathering data on alternatives that inform agencies on cost and 
performance trade-offs.[Footnote 14] One way agencies can improve 
their ability to evaluate benefits and costs is to examine the 
marginal cost of activities, or the incremental cost of producing one 
more unit of output. For the Bureau, this means mining its performance 
and cost data to evaluate the effectiveness of its operations and to 
identify potential opportunities for improvement. 

Although the Bureau has a number of efforts under way within two 
initiatives to help guide 2020 planning, only a handful are aimed at 
producing return on investment information that enhances its ability 
to make decisions on cost quality trade-offs. These initiatives are 
the 2010 Census Program for Evaluations and Experiments (CPEX) looks 
back at 2010 operations, and the research and testing phase looks 
ahead at potential design alternatives for 2020. According to Bureau 
officials, of the more than 100 planned evaluations, assessments, 
experiments, and quality profiles in CPEX, a few are designed to 
produce information describing the return on investment of census-
taking activities, which can help the Bureau make decisions about cost-
quality trade-offs. For example, 2 planned evaluations will examine 
potential cost savings for address canvassing--one looks at the 
potential cost reduction associated with targeted address canvassing 
and the other looks at potential cost savings associated with 
automated field data collection of address canvassing results. 
Moreover, of planned 2010 CPEX evaluations for which we have a 
description, the vast majority will measure aspects of accuracy or 
coverage. 

The Bureau may be missing opportunities to mine performance data for 
information that could help officials increase the efficiency of 
costly field operations and could help inform difficult decisions for 
controlling costs and maintaining quality. As part of CPEX, the Bureau 
has planned about 50 assessments of specific enumeration activities 
and operations, such as address canvassing and NRFU. These assessments 
include an analysis of cost that would be of limited usefulness for 
informing return on investment decisions. For example, the assessments 
will compare budgeted and actual costs and indicate why an operation 
was over or under budget, but will not determine the marginal return 
for different enumeration or address-building operations. Information 
on the marginal returns on investments could, for example, help the 
Bureau determine where to focus cost control efforts. As one example, 
based on our analysis of operational data provided by the Bureau for 
NRFU, we determined that the marginal cost per questionnaire checked 
into LCOs was approximately $1,045 in the final weeks of the operation 
(see figure 5). During this time, the Bureau completed a little over 
2,300 questionnaires or roughly .005 percent of the entire NRFU 
universe of over 47 million housing units. This estimate is roughly a 
$1,000 increase per questionnaire compared to the first few weeks of 
the operation, which began on May 1, when the Bureau completed 
approximately 39 percent of the NRFU universe. Thus, it cost the 
Bureau approximately 17 times more per questionnaire in the final 
weeks of NRFU to attempt to obtain information from nonresponding 
housing units, units that may have been contacted as many as six times 
in person or by phone. More extensive analyses of these data could 
help the Bureau determine the extent to which specific activities 
contributed to cost growth and help it target control cost effects 
without compromising accuracy. 

Figure 5: Estimated Marginal Cost of Checking in NRFU Questionnaires 
Dramatically Rose in the Final Weeks of the Operation: 

[Refer to PDF for image: vertical bar graph] 

Time period when questionnaire was completed: May 1 through May 25; 
Estimated marginal cost per questionnaire: $61. 

Time period when questionnaire was completed: May 26 through June 15; 
Estimated marginal cost per questionnaire: $15; 

Time period when questionnaire was completed: June 16 through July 6; 
Estimated marginal cost per questionnaire: $30; 

Time period when questionnaire was completed: July 7 through July 31; 
Estimated marginal cost per questionnaire: $1,045. 

Source: GAO analysis of U.S. Census Bureau data. 

[End of figure] 

As the Bureau enters the research and testing phase, several planned 
projects will yield information that will improve its ability to make 
decisions balancing the competing goals of cost and quality. According 
to the Bureau, it is essential to conduct research and testing of 
multiple design alternatives prior to deciding upon a final census 
design and technical solution to ensure that the final census design 
is effective and works within the 2020 Census environment. Our review 
of Bureau planning documents identified 8 of 35 projects scheduled in 
the early part of the decade that will include analyses of costs and 
benefits. For example, a project on reducing and improving in-person 
follow-up operations is designed to examine the costs and benefits of 
different contact strategies and whether these will achieve the goals 
of the operation. However, most projects examine the accuracy and 
quality implications of conducting enumeration and not cost 
implications. Without gathering data on cost during this phase, 
specifically the potential cost savings that could be realized with 
certain alternatives, the Bureau could be making decisions based on 
incomplete information on the design alternatives. 

The lack of emphasis on cost analyses is consistent with our previous 
reports that fundamental reforms will be needed to ensure that the 
Bureau's management, culture, and business practices are aligned with 
cost-effective enumeration.[Footnote 15] According to Bureau 
officials, previous decisions about operational changes were based on 
a priority to improve quality and were sometimes made without much 
complete knowledge of cost implications. As we reported in 2009, the 
Bureau has not always used available information to determine the 
value added of the operation. For example, the Bureau has the 
information but has not determined which of its 11 operations for 
building its address list[Footnote 16] provide the best return on 
investment in terms of contributing to accuracy and coverage.[Footnote 
17] 

Decision Points Could Help the Bureau Control Costs and Avoid Delays: 

The Bureau's planning documents have not clearly identified and 
defined decision points that can help avoid cost overruns and schedule 
delays. OMB guidance for large projects suggests that agencies develop 
a schedule with defined phases, decision points, and an identified 
decision authority to evaluate whether an agency should proceed to the 
next phase in the investment life cycle. In addition, our previous 
body of work on acquisition policies in high-performing organizations 
includes the best practice of identifying critical junctures, also 
known as knowledge or decision points, in the acquisition cycle and 
requiring executive-level oversight at critical junctures.[Footnote 
18] Agencies can use decision points to determine whether a particular 
investment is ready to proceed to the next phase. For example, when 
moving out of an early phase agencies must determine if resources--
that is, technology and funding--and needs are matched. The 2020 
Census is a complex, costly project with immutable deadlines. Decision 
points at key phases of the planning process could improve the 
Bureau's ability to manage risks as well as achieve desired cost, 
schedule, and performance outcomes for the decennial. 

The Bureau's 2020 business plan has a high-level preliminary schedule 
for the major phases of the decennial that includes, for example, a 
yearlong activity at the end of the research and testing phase to 
determine and refine initial operational designs. However, the 
schedule has no decision points at the end of research and testing or 
any phase, as best practices suggest, to determine whether progress 
was made and ensure that the agency's needs for quality and accuracy 
match the available resources--that is, technology, design, time, and 
funding. In addition, there is no identified executive-level review at 
any point in the schedule. Since the research and testing phase is 
intended to develop a preliminary design from a range of alternatives, 
a decision point at the end of this phase could help the Bureau 
determine if it has enough information to support the increased 
investment necessary to move to the next phase of development and 
testing (see figure 6). At subsequent stages in the process, decision 
points could be used to determine that the design was stable enough to 
meet operational requirements. Later decision points could also be 
used to determine whether a particular design alternative could be 
implemented within cost and schedule constraints while meeting quality 
targets and maintaining reliability. Absent such an approach at each 
phase, the Bureau lacks assurance that it has obtained the critical 
technological and design knowledge that best practices call for to 
avoid cost overruns, schedule slips, and performance shortfalls going 
forward. 

Figure 6: Executive-Level Review at Decision Points Could Improve 2020 
Census Planning: 

[Refer to PDF for image: illustration] 

2020 planning activities: 

Beginning of 2020 life cycle: FY 2012. 

Select and baseline the operational design: FY 2015. 

Update and re-baseline the operational design: FY 2017. 

Census Day: April 1, 2020. 

2020 Census phases: Best practices model: 
 
Research and testing: FY 2012 through FY 2014. 

Key decision review points where executive-level review is needed: FY 
2015. 

Supplemental research and testing: FY 2015 through FY 2018. 

Key decision review points where executive-level review is needed: FY 
2017. 
  
Operational development and systems testing: FY 2015 through FY 2019. 

Key decision review points where executive-level review is needed: FY 
2019. 

Readiness testing, execution, and closeout (to end of life cycle 
2023): FY 2019 through FY 2023. 

Source: GAO analysis of U.S. Census Bureau information. 

[End of figure] 

The Bureau Could Further Develop Cost Criteria for Selecting among 
2020 Design Alternatives: 

According to one Bureau planning memo, cost is one of four categories 
of criteria that will be used to evaluate design options for the 2020 
Census.[Footnote 19] However, the memo does not describe specifically 
how the cost criterion will be used to select among design 
alternatives.[Footnote 20] For example, the criterion for cost can be 
expressed by ranking costs (i.e., least costly to most costly), 
weighting costs for different elements, or specifying that costs fall 
within a range. We have previously reported that criteria should be 
clearly defined, well documented, transparent, and consistently 
applied.[Footnote 21] Neither the Bureau's strategic plan nor its 
early business plan, which outlines and guides the early development 
of the 2020 Census, describes criteria or identifies when criteria 
would be used to select the design of the 2020 Census. 

Bureau officials said they have not established when they will develop 
specific evaluation criteria for cost. Further, they acknowledged that 
selecting among design alternatives may take place during the research 
and testing phase, which begins in fiscal year 2012. In addition, 
Bureau officials told us that not all 2020 Census planning memos will 
be updated throughout the course of 2020 planning. Therefore, it is 
unclear how updates to criteria will be made to this planning memo. As 
a result, the Bureau may make decisions to eliminate design 
alternatives before clearly documenting how cost criteria will be 
applied, as well as how the alternatives will be considered along with 
the other criteria. 

The Bureau's early cost estimates range from $12.8 billion to $18 
billion for four of the six design alternatives.[Footnote 22] Because 
of the wide range of 2020 cost estimates, documenting and consistently 
using cost as a criterion when deciding among design alternatives can 
help the Bureau control costs. It is important for the Bureau to apply 
cost in decision making because the Bureau has not achieved previous 
goals for containing costs and made late design changes that proved 
costly in previous censuses. 

Without a Process in Place for Developing 2020 Census Life Cycle Cost 
Estimates, the Bureau Is at Risk of Not Following Best Practices: 

The Bureau has not yet established policies, procedures, or guidance 
for developing the 2020 Census life cycle cost estimate and is at risk 
of not following related best practices. The Bureau uses the life 
cycle cost estimate as the starting point for the annual budget 
formulation process and, according to our Cost Estimating and 
Assessment Guide, a reliable cost estimating process is necessary to 
ensure that cost estimates--particularly for large, complex projects 
like the 2020 Census--are comprehensive, well documented, accurate, 
and credible.[Footnote 23] Put another way, reliable cost estimates 
are essential for a successful census because they help ensure that 
the Bureau has adequate funds and that Congress, the administration, 
and the Bureau itself can have reliable information on which to base 
decisions. 

Our guide identifies 12 steps of a high-quality cost estimation 
process, including, among other things, determining the estimate's 
purpose; defining the program's characteristics; clearly defining 
ground rules and assumptions; conducting sensitivity, risk, and 
uncertainty analyses; and documenting all steps used to develop the 
estimate. These best practices, if followed correctly, should produce 
reliable estimates that management can use for making informed 
decisions (see appendix IV). 

To date, the Bureau has developed a rough-order-of-magnitude estimate, 
which covers the four 2020 Census design alternatives that are the 
most similar to the 2010 Census design.[Footnote 24] Bureau officials 
stated that this was not an official estimate, but rather a starting 
point that will be revised and improved as the Bureau gathers more 
data in the research and testing phase. As the Bureau goes forward in 
its 2020 planning, it will be important for it to have reliable and 
accurate cost estimates as it narrows down design alternatives and 
moves closer to a final design. 

The Bureau's early 2020 planning documents note that the Bureau 
intends to use our cost guide as it develops cost estimates for 2020, 
and Bureau officials have stated that its cost estimators would follow 
best practices wherever practicable. Nevertheless, the Bureau has not 
yet documented how it plans to conduct its cost estimates; and, while 
officials stated that they plan on developing more detailed 
documentation in the future, they could not provide a specific time 
when such documents would be finalized. 

Although the 2020 Census is still a number of years away, the timeline 
for the Bureau to develop a cost estimation process is growing short. 
The Bureau plans to begin work on an official life cycle cost estimate 
in fiscal year 2013, and plans to include its initial life cycle cost 
estimate in its fiscal year 2015 budget submission covering 
initiatives from 2015 through 2018. As a result, the Bureau has about 
a year to establish and finalize a process for preparing high-quality 
life cycle cost estimates. 

The importance of reliable cost estimates is underscored by the 
Bureau's experience leading up to the 2010 Census, where we found that 
the Bureau's cost estimate lacked detailed documentation on data 
sources and significant assumptions and was not comprehensive because 
it did not include all costs. Among other weaknesses, we noted that 
the Bureau had insufficient policies and procedures for conducting 
high-quality cost estimation. Partly as a result, some operations had 
substantial variances between their initial cost estimates and their 
actual costs. Until the Bureau finalizes its cost estimating policies, 
procedures, and guidance, the Bureau runs the risk of again developing 
unreliable cost estimates for 2020. 

Conclusions: 

For the Bureau to improve its ability to control the costs of future 
censuses without sacrificing accuracy, it will be critical for it to 
have a better understanding of the factors affecting cost increases 
from prior decennials, as well as how various census-taking activities 
contributed to the overall quality of the count. Although the Bureau 
will gain valuable insights from its evaluations of the 2010 Census as 
well as from research and testing for 2020, this information will only 
be of limited use in helping the Bureau develop a complete picture of 
the key drivers of census costs and the steps needed to control costs 
going forward. Therefore, to improve its capacity to identify cost 
drivers and effectively target cost control efforts, it will be 
important for the Bureau to develop a way to compare costs for key 
activities across censuses and assess the marginal returns of each. 

The Bureau has set a clear goal for controlling costs while 
maintaining accuracy for the 2020 Census, and has developed a range of 
design alternatives aimed at achieving that goal. Given the number of 
design alternatives the Bureau is evaluating for the 2020 Census, it 
will be important for the Bureau to set explicit decision points for 
executive-level review at the end of individual phases to reduce the 
risk of cost, schedule, and performance shortfalls. Without clearly 
defined decision points in its 2020 planning phases, the Bureau may 
not be able to determine that it is on track or make the necessary 
adjustments in its design approach to achieve a more cost-effective 
census. Moreover, decision points would allow the Bureau to determine 
its readiness to move on to the next phase in 2020 planning. In 
conjunction with scheduled decision points, it will be critical for 
the Bureau to finalize evaluation criteria that are transparent, 
thoroughly documented, and consistently applied to maximize its 
ability to control costs for the 2020 Census. Without specifying 
explicit cost evaluation criteria for selecting among design 
alternatives, the Bureau and stakeholders, such as Congress, cannot 
accurately consider costs and may not have assurance that they are on 
the path to a more efficient census in 2020. 

Cost estimates are necessary tools for major programs because they 
help in developing budget requests and efficiently allocating scarce 
resources. In a time of constrained budgets, these tools become even 
more important. However, cost estimates are technically complex and 
cost estimators face challenges in developing estimates for complex 
programs such as the 2020 Census. Previously, the Bureau had 
insufficient policies and procedures for developing reliable and high-
quality cost estimates. Without clear guidance in place, there is no 
assurance that the Bureau will develop life cycle cost estimates for 
2020 that are reliable and high-quality and follow best practices. 

Recommendations for Executive Action: 

To improve the Bureau's ability to control costs for the 2020 
decennial and balance cost and quality, we recommend that the 
Secretary of Commerce direct the Under Secretary of the Economics and 
Statistics Administration, as well as the Director of the U.S. Census 
Bureau, to take the following four actions: 

* Develop and document a method to compare costs in 2010 to those in 
future decennials, for example, around major activities or 
investments, to allow the Bureau to identify and address factors that 
contribute to cost increases. 

* Analyze data from key census-taking activities to determine their 
marginal costs and benefits, and use this information to inform 
decisions on developing more cost-effective methods. 

* Identify decision points at the end of each planning phase and 
assign decision-making authority at the executive level, as well as 
consider adding decision points within phases to determine progress 
and readiness to proceed to the next phase. 

* Finalize how the Bureau will apply cost as an evaluation criterion 
for choosing among design alternatives for 2020 and ensure that all 
criteria are transparent, well documented, and consistently applied 
before alternatives are eliminated. 

We have previously recommended that the Secretary of Commerce direct 
the Bureau to establish guidance, policies, and procedures for cost 
estimation that would meet best practice criteria.[Footnote 25] To 
help ensure that the Bureau produces a reliable and high-quality cost 
estimate for the 2020 Census, we recommend that the Bureau take the 
following action: 

* Finalize guidance, policies, and procedures for cost estimation in 
accordance with best practices prior to developing the Bureau's 
initial 2020 life cycle cost estimate. 

Agency Comments and Our Evaluation: 

The Secretary of Commerce provided written comments from the Bureau on 
a draft of this report on January 6, 2012. The comments are reprinted 
in appendix II. The Department of Commerce expressed broad agreement 
with the overall theme of the report but did not directly comment on 
the recommendations. It raised concerns about specific aspects of the 
summary of findings which GAO addressed as appropriate. 

Specifically, the Bureau agrees with the importance of using analysis, 
such as assessing marginal returns to help with decision making on 
balancing the need to control costs while maintaining accuracy. Moving 
forward, it will be important for the Bureau to also recognize that 
more in-depth understanding of the growth in costs from prior censuses 
can, in fact, strengthen its decision-making ability and help it more 
effectively target cost control efforts in the future. The Bureau said 
understanding the growth in costs from 2000 through 2010 in depth has 
not been its highest-priority area for investment of scarce resources. 
We are sensitive to existing budget constraints. The fiscal issues 
facing federal agencies make it even more imperative for Bureau 
decision makers to develop and use actionable information, such as 
data on the extent to which specific operations and activities drove 
up costs, to pinpoint problem areas and target cost control efforts 
accordingly. 

The Bureau expressed concern that the summary of findings and 
conclusions on the highlights page seemed premature and unsupported by 
discussions in the full report. In commenting on the first paragraph, 
the Bureau stated that it does not believe its inability to identify 
specific factors affecting past growth will make it difficult to 
control costs for the 2020 Census. However, our report concludes that 
the Bureau's inability to identify specific actionable factors will 
make it difficult for the Bureau to focus its cost control efforts for 
the 2020 Census. To help pinpoint problem areas for controlling census 
costs, it is important for the Bureau to have a better understanding 
of the specific sources of cost growth. This requires analysis of 
costs below the broad category level, focusing on projects that tie 
directly to major operations and investments. We believe understanding 
how the cost of these programs compared to 2000 and the extent to 
which they contributed to the cost of the 2010 Census, and whether 
they produced the desired results can help with decision making on 
areas where there are trade-offs in cost and accuracy. We added 
language to the highlights page to reflect the need for analyses to 
more effectively target future cost control efforts. 

In commenting on the second paragraph of the highlights page, the 
Bureau stated that it had not yet received any appropriated funds or 
had the opportunity to develop program management efforts for the 2020 
Census that would allow the agency to establish formal guidance for 
developing cost estimates. However, the paragraph discusses practices 
for strengthening agency decision making for large projects rather 
than establishing formal guidance for developing cost estimates. OMB 
guidance for large projects and our previous body of work on 
acquisitions policies in high-performing organizations suggest setting 
explicit decision points for executive-level review at the end of 
individual planning phases can help reduce the risk of cost, schedule, 
and performance shortfalls. By clearly defining decision points in its 
2020 planning phases, the Bureau could better ensure that it is on 
track or make adjustments in its design approach earlier in the 2020 
planning process to achieve a more cost-effective census. On the issue 
of having appropriated funds for planning purposes, we agree that 
funding for the 2020 Census life cycle did not officially begin until 
fiscal year 2012. However, the Bureau includes costs of early planning 
for the next census in the final years of the previous census life 
cycle (i.e., 2010 appropriations pay for 2020 planning). During our 
audit, we interviewed individuals who were planning and developing the 
2020 Census. We also reviewed informational memos, such as the 
strategic plan and the business plan issued in 2009 and 2010, 
respectively, as guiding documents for the 2020 Census planning 
effort. Moreover, during the audit, the Bureau released newly 
developed and revised planning documents, such as the updated business 
plan and rough-order-of-magnitude estimates. While we made changes to 
the second paragraph of the highlights page, these were not made in 
response to this comment. 

The Bureau commented that it was unclear why the graphic in our 
highlights page focused on costs and mail response rate data over 
time. We selected mail response rates because, as the Bureau notes, 
declining mail response rates are significant and have led to higher 
costs. For example, the mail response rate directly dictates the 
number of housing units in the NRFU universe. NRFU is the largest and 
most costly Bureau field operation and has an impact on overall census 
costs. We agree with the Bureau that the declining mail response rate 
is only one factor leading to higher census costs. Our report 
acknowledges other factors that contribute to higher costs. As such, 
we made no change to the highlights page in response to this comment. 

The Bureau made a number of technical comments on the body of the 
report. The Bureau commented that our report implies that the Bureau 
attributed all cost growth over the decades only to population growth. 
In fact, our draft report has a section dedicated to the five broad 
factors the Bureau identified as affecting cost growth. However, we 
added clarifying language to the discussion on workload and census 
costs to note workload is one of the factors driving up census costs. 

The Bureau commented that the statement that the Bureau cannot 
determine areas of cost growth is a sweeping and premature conclusion 
given that the 2020 research and testing program just began. The 
Bureau stated that its primary focus is to study ways to reduce the 
cost of the next census while maintaining quality. While we 
acknowledge that the research and testing effort may help identify 
ways to reduce costs, coupling that information with specific factors 
of past cost growth could strengthen the Bureau's ability to target 
cost reduction efforts in the future. We made no change to the report 
to address this comment. 

The Bureau noted that statements in two areas seemed to be based on an 
assessment of how well the Bureau documented and analyzed costs 
relative to our Cost Estimating and Assessment Guide. The Bureau 
states that the guide was not issued until March 2009 and that the 
Bureau has not fully incorporated all those practices into a program 
as large, costly, and complex as the census. While it is true that we 
published the guide in March 2009, we issued an exposure draft in 2007 
and shared a copy of our guide with the Bureau during our October 2006-
June 2008 audit of Bureau cost estimating practices (GAO-08-554). In 
fact, the cost guide is based on long-standing industry and government 
best practices on cost estimation followed long before GAO published 
them in a concise form in 2009. Moreover, in its June 2008 action plan 
to address GAO recommendations, the Bureau noted its plan to use the 
guide, particularly the 12 steps of a high-quality cost estimating 
process. In less than 1 year from now, the Bureau plans to begin work 
on its official life cycle cost estimate for the 2020 Census. By not 
establishing policies, procedures, or guidance for developing life 
cycle cost estimates, the Bureau again runs the risk of developing 
cost estimates that are not comprehensive, accurate, or credible. We 
made no change to the report in response to this comment. 

The Bureau commented that it was unsure why we presented the NRFU 
analysis of marginal costs as it was a small percentage of the entire 
budget. However, we used this as an example of how such an analysis 
may help point to areas for targeting cost reduction efforts or for 
modifying the Bureau's approach to data collection. The analysis does 
not imply that the Bureau should ignore the remaining households at 
the end of NRFU as the Bureau's comment states. Instead, it highlights 
the importance of considering alternative approaches in order to 
ensure a complete and cost-effective enumeration. The more important 
point is that it highlights the increasing marginal costs of 
contacting certain households at the tail end of the enumeration. We 
agree that the Bureau cannot ignore hard-to-contact households. By 
mining performance data on the NRFU operation, the Bureau may be in a 
better position to identify alternative approaches for the hardest-to-
contact households that have the greatest potential to reduce costs 
without compromising accuracy. We made no change to the report to 
address this comment. 

Finally, the Bureau commented that our reported costs for local census 
operations were incomplete, so we corrected the number based on 
information provided by the Bureau. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to other 
interested congressional committees, the Secretary of Commerce, and 
the Director of the U.S. Census Bureau. In addition, the report will 
be available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. GAO staff who made key 
contributions to this report are listed in appendix V. 

Signed by: 

Robert Goldenkoff: 
Director Strategic Issues: 

List of Requesters: 

The Honorable Thomas R. Carper: 
Chairman: 
The Honorable Scott P. Brown: 
Ranking Member: 
Subcommittee on Federal Financial Management, Government Information, 
Federal Services, and International Security: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The Honorable Darrell E. Issa: 
Chairman: 
The Honorable Elijah E. Cummings: 
Ranking Member: 
Committee on Oversight and Government Reform: 
House of Representatives: 

The Honorable Trey Gowdy: 
Chairman: 
The Honorable Danny Davis: 
Ranking Member: 
Subcommittee on Health Care, District of Columbia, Census and the 
National Archives: 
Committee on Oversight and Government Reform: 
House of Representatives: 

The Honorable John S. McCain: 
United States Senate: 

The Honorable William Lacy Clay: 
House of Representatives: 

The Honorable Patrick T. McHenry: 
House of Representatives: 

[End of section] 

Appendix I: Objective, Scope, and Methodology: 

To identify the key factors affecting cost growth from the 2000 Census 
to the 2010 Census, we reviewed U.S. Census Bureau (Bureau) strategic 
planning documents for 2000 and 2010, Bureau operational and systems 
plans for 2000 and 2010, Bureau assessments and evaluations of past 
census operations, National Academy of Sciences work on decennial 
census costs, and our prior work on implementation of 2000 and 2010 
census operations. We assessed the Bureau's approach to determine 
trends in cost data using best practices for cost estimation in GAO's 
Cost Estimating and Assessment Guide.[Footnote 26] The guide 
illustrates the importance of using cost data to understand trends and 
drivers. In addition, we interviewed Bureau officials and reviewed 
agency documentation on actions taken to determine sources of cost 
growth between decennials. 

To identify sources of cost growth from the 2000 Census and the 2010 
Census, we reviewed and analyzed expenditure data on 2000 Census and 
2010 Census life cycle costs from the Bureau's Commerce Business 
System (CBS). CBS is the Bureau's financial management system and the 
official system of record for expenditures.[Footnote 27] CBS data 
contained information at two levels of aggregation for the census: 
budget categories, which are broad groupings of related items, and 
budget projects, which are the lowest level of cost information. To 
determine the level of cost growth from the 2000 Census and the 2010 
Census, we developed life cycle totals for each census and life cycle 
totals for each of the budget categories within those censuses, 
comparing their absolute and percentage growth. We adjusted all 
monetary data for inflation using the gross domestic product implicit 
price deflator. All costs were adjusted to fiscal year 2010 dollars. 
In addition, we compared costs after adjusting for the number of 
housing units for each census. 

We assessed the reliability of the Bureau's CBS data by reviewing 
relevant documentation, interviewing knowledgeable agency officials, 
and conducting comparisons with other data sources. We reviewed 
previous GAO, Department of Commerce Inspector General, and other 
Department of Commerce reports covering the system. We conducted 
interviews with Bureau officials who maintain the system at the Bureau 
level and its primary users within the Decennial Management Division. 
After receiving the cost data covering the 2000 and 2010 censuses, we 
compared them to financial management reports provided by Bureau 
officials to determine data consistency. We determined that these data 
are sufficiently reliable for the purposes of this report. 

Our review was subject to some limitations. The budget categories and 
budget projects for 2000 and 2010 varied from census to census. We 
requested and the Bureau provided recategorized cost data to 
facilitate comparison of the 2000 budget categories with 2010 budget 
categories. We requested and the Bureau was unable to provide 
recategorized cost data to facilitate comparison of 2000 budget 
projects with 2010 budget projects. We attempted to compare costs at 
the budget project level from 2000 to 2010 but were unable to do so 
for the following reasons: (1) the Bureau's budget projects were not 
consistent from 2000 to 2010, making it impossible to match projects 
directly using project descriptions or project codes;[Footnote 28] (2) 
the number of projects increased substantially from 236 in 2000 to 
1175 in 2010; and (3) the Bureau was unable to provide us with any 
documentation tracking similar projects from the 2000 Census to the 
2010 Census.[Footnote 29] We attempted to group similar projects in 
2000 and 2010 for comparison, but the available project descriptions 
did not provide enough information to group 2000 costs with the same 
precision as 2010. Therefore, we could not conduct a comparison of 
groups of projects. 

To assess the Bureau's plans for controlling costs for the 2020 Census 
and what additional steps, if any, could be taken, we reviewed 
available documentation on 2020 Census planning and 2010 Census 
evaluations and assessments, such as 2010 evaluation study plans. We 
consulted with GAO staff with expertise in economics to determine the 
potential for leveraging available Bureau cost data to better support 
the Bureau's ability to make cost-quality trade-offs. We reviewed 
Office of Management and Budget guidance on major acquisitions as well 
as GAO work on acquisition best practices to determine whether the use 
of decision points could help the Bureau make more informed decisions 
about census design that could relate to cost control. Further, we 
reviewed Bureau documentation on criteria for selecting among 2020 
design alternatives. 

We analyzed the marginal cost of conducting nonresponse follow-up 
(NRFU)--the costliest field operation in the 2010 Census--to determine 
how the Bureau might be able to further use its cost data in planning 
for 2020. We used Bureau cost and progress data from the 2010 Census 
to identify the marginal costs of the NRFU operation in 3-week 
intervals. This analysis compared the cost of the operation for that 
period with the number of questionnaires checked in to identify return 
on investment. We assessed the reliability of the Bureau's 2010 cost 
and progress data by consulting with the Bureau about variables we 
used and reviewing past GAO data reliability work that used cost and 
progress data. The cost and progress system is a daily management tool 
used by Bureau officials to track the work completed of various census 
operations. It includes measures of cost (such as field hours or 
mileage costs) and measures of work completed (such as questionnaires 
checked in). Our estimate of the marginal costs of checking in NRFU 
questionnaires in the early weeks of the operation may be somewhat 
overstated because, for instance, we included training costs as well 
as fieldwork costs because training costs were incurred in the early 
part of NRFU and those costs were not spread over the life of the 
operation. As a result, costs for the early weeks of the operation 
could be lower than presented in the graphic. 

After developing the marginal costs methodology, we followed up with 
agency officials knowledgeable about the data when we had questions 
about potential errors or inconsistencies. In addition, we reviewed 
prior GAO data reliability work on cost and progress data that 
examined the accuracy and completeness of the entry and processing of 
data. Based on this work, we determined that the data were 
sufficiently reliable for gauging the approximate marginal cost 
increase per questionnaire checked in during the final weeks of the 
NRFU operation. 

To assess the extent to which the Bureau's plans for developing life 
cycle cost estimates for 2020 are consistent with best practices, we 
reviewed available Bureau documentation on the Bureau's life cycle 
cost estimation processes and procedures. For example, we reviewed 
documentation from the Bureau's rough-order-of-magnitude estimate--an 
early high-level estimate developed from limited data. We reviewed the 
guidance contained in our Cost Estimating and Assessment Guide and our 
previous work on census life cycle cost estimates. We also conducted 
interviews with knowledgeable Bureau officials and contractor staff 
and received a demonstration of new capabilities in the Bureau's 
budgeting tool that will be used for 2020 Census cost estimation. 

We conducted this performance audit from December 2010 through January 
2012 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Commerce: 

United States Department of Commerce: 
The Secretary of Commerce: 
Washington, D.C. 20230: 

January 6, 2012:  

Mr. Robert Goldenkoff: 
Director: 
Strategic Issues: 
U.S. Government Accountability Office: 
Washington, DC 20548:  

Dear Mr. Goldenkoff: 

The U.S. Department of Commerce appreciates the opportunity to comment 
on the U.S. Government Accountability Office's draft report titled, 
Decennial Census: Additional Actions Could Improve the Census Bureau's 
Ability to Control Costs for the 2020 Census (GAO-12-80). The 
Department of Commerce's comments on this report are enclosed.  

Sincerely, 

Signed by: 
 
John E. Bryson:  

Enclosure: 

[End of letter] 

U.S. Department of Commerce Comments on the United States Government 
Accountability Office Draft Report titled: Decennial Census: 
Additional Actions Could Improve the Census Bureau's Ability to 
Control Costs for the 2020 Census (GAO-12-80), December 2011: 

The U.S. Census Bureau (Bureau) would like to thank the U.S. 
Government Accountability Office (GAO) for this opportunity to comment 
on their draft report. We appreciate the hard work of the GAO staff 
who devoted 12 months to auditing the costs of the 2010 Census 
beginning in December of 2010, when the apportionment numbers were 
delivered to the President. They continued their work throughout 2011, 
while the remaining 2010 operations were winding down to their current 
low level. The Bureau appreciates the observations regarding areas 
where costs increased between 2000 and 2010. The report correctly 
notes that the primary drivers of the cost increases were the 
nonresponse follow up field operations and the data processing 
activities. 

The Bureau is completely committed to controlling costs and risks for 
the 2020 Census. Although the Bureau was only recently appropriated 
funding to begin work on 2020, as you note, it has already developed a 
series of life cycle Rough Order of Magnitude (ROM) cost estimates and 
a 3 year research program that has as its goal reducing the costs of 
taking the 2020 census without sacrificing levels of coverage and 
accuracy acceptable to the Congress and the Nation. This is where we 
are focusing our attention and resources. 

We expect many design elements in 2020 to be significantly different 
than what was achievable in 2000. For that reason, understanding the 
growth in costs between 2000 and 2010 in depth has not been our 
highest priority area for investment of scarce resources. While we 
need some understanding of the past, we are looking ahead. That look 
ahead includes using best practices for cost estimation and evaluating 
design alternatives against cost and accuracy criteria. 

We agree completely with GAO that it is important to look at the 
marginal returns on various field operations. It is very well known to 
the public and private data collection organizations that marginal 
costs increase as the field operations proceed, due to the increasing 
difficulty of getting a response from the most reluctant responders. 
The report does a good job of describing this phenomenon. We fully 
intend to use whatever data we have to look at these and other similar 
areas where there are significant tradeoffs in costs, accuracy, 
coverage, and the willingness of the Congress to accept those 
tradeoffs. 

While data that we have from 2010 will he useful, we intend to make 
heavy use of the American Community Survey as a test bed for elements 
of the 2020 design. This will allow us to gather just the type of 
information we need to do the types of cost estimating and analyses of 
tradeoffs that you are recommending. The Bureau welcomes your input as 
we proceed to develop our plans. 

Although there are some specific comments on the body of the report 
below, the Bureau is most concerned that the overall summary of 
findings presented on the Highlights page consists of several broad 
conclusions that seem both premature arid unsupported by the lengthier 
discussions, context, and details presented in the report itself. For 
example: 

* The first paragraph of the Highlights page concludes that the Census 
Bureau is unable to identify specific factors affecting past cost 
growth, and so will find it difficult to control costs for the 2020 
Census. On page I, the GAO notes—and the Census Bureau agrees—that the 
real difficulty in controlling costs for any decennial census relates 
to balancing expenditures against accuracy. While we largely agree 
with the GAO's summary and statements on page 1 (and on pp. 27-30), we 
do not believe they support the broad conclusion in the opening 
paragraph of the Highlights page. 

* The second paragraph of the Highlights page states the Census Bureau 
has not established formal guidance for developing costs estimates and 
has not incorporated certain OMB and GAO best practices in the way it 
will make decisions about the 2020 Census. GAO then concludes that the 
Census Bureau will be limited in the ability to manage costs, risks, 
schedules, and performance for the 2020 Census. This conclusion seems 
very sweeping and premature, given that the Census Bureau had not yet 
received any appropriated funds to plan the 2020 research and testing 
phase nor had any opportunity to develop program management efforts 
for the 2020 Census at the time of the audit. 

* The Census Bureau is unclear why the chart presented at the bottom 
of the Highlights page (and later in the report) focuses on costs and 
mail response rates over time. In light of the report's statement on 
page 1 that the key difficulty is balancing costs and coverage (as 
agreed by the Census Bureau), the Census Bureau believes a more 
meaningful chart would show census costs and census coverage over 
time. Declining mail response rates are significant, and have led to 
higher costs for census and survey work in both the public and private 
sector, but this decline is only one factor in that cost growth. 

The Census Bureau also has the following specific comments on the 
details of this report: 

* Page 10, first full paragraph: This statement implies that the 
Census Bureau has attributed all cost growth over the decades only to 
population growth. While Census has stated that it costs more to count 
more people, it has never stated or implied that population growth was 
the only driver of census costs. This statement also ignores factors 
acknowledged earlier in the report. Specifically, population diversity 
and declining mail response rate also are significant factors in cost 
increase. Also, the reference to a 12 percent increase in workload 
appears to be based on the overall growth in the number of housing 
units. However, the size of the field data collection workload also 
depends on such things as the mail response rate. 

* Page 12, last sentence of paragraph at top of page: The statement 
"the Bureau cannot determine where it is saving money or where costs 
are growing" is a very sweeping conclusion and seems premature given 
(a) the formal 2020 research and testing program only recently came 
into existence, and (b) that program's primary focus is to study ways 
to reduce the cost of the next census, while maintaining quality. 

* Page 14, first full paragraph, and Page 17, first full paragraph: 
These statements seem mostly based on an assessment of how well the 
Census Bureau has documented and analyzed costs relative to GAO's Cost 
Estimating and Assessment Guide. GAO did not issue that guide until 
March 2009, and since then, Census has taken a number of steps to 
begin to implement best practices. Concluding that the Census Bureau 
will he unable to control costs in the future should not be based on 
not having followed those practices in previous decades or for not 
having yet incorporated all those best practices into a program as 
large, costly, and complex as the decennial census. We fully intend to 
follow best practices for the 2020 Census. 

* Page 19, analysis of NRFU costs: The NRFU operation for the 2010 
Census cost about $1.6 billion dollars. The NRFU cost of the final 
weeks that GAO refers to was about $2-3 million. Therefore, the 
calculated high marginal cost for this phase accounts for about 0.2% 
of the entire budget. It does not seem remarkable or notable that the 
marginal cost is higher for enumerating the last few, and likely 
hardest to count or, at least, hardest to contact households. Yet, in 
trying to count everyone as accurately as possible, we cannot just 
ignore these households. Therefore, we are unsure why GAO has presented
this analysis. We note that we do have research planned to study the 
balance between quality (count accuracy) and cost (such as the number 
of attempts made to count these households). 

* Appendix III (page 38): The Local Census Operations (T.00) cost of 
$502.3M is incomplete. The LCO Office cost in FY 10 was split into two 
projects (5352042 and 5312042). The $502.3M cost shown in the table is 
the amount for project 5352042, but there was additional cost of 
$189.9M in project 5312042. 

[End of section] 

Appendix III: 2010 Census Costs Were Concentrated in Certain Budget 
Categories and Projects: 

[Refer to PDF for image: illustrated table containing a pie-chart and 
horizontal bar graph for each budget category] 

Budget category and total cost: Field data collection and support 
systems; 
Total category cost: $9.0515 billion; 
Most costly life cycle budget categories and projects: 31% of total: 
Nonresponse follow-up (NRFU) operation: $1.6459 billion (59%); 
Local census office operations: $699.6 million (25%);  
Address canvassing operation: $441.7 (16%). 

Budget category and total cost: Automated data collection, systems, 
and data capture; 
Total category cost $1.8441 billion; 
Most costly life cycle budget categories and projects: 44% of total: 
Decennial Response Integration System contract: $498.7 million (62%); 
Decennial Response Integration System contract: $230.5 million (29%); 
Decennial Response Integration System contract: $75.8 million (9%). 

Budget category and total cost: Content, questionnaires, and products; 
Total category cost: $10445 billion; 
Most costly life cycle budget categories and projects: 46% of total: 
Questionnaire postage: $230.5 million (48%); 
American Community Survey mail: $138.1 million (29%); 
Questionnaire printing: $110.1 million (23%). 

Budget category and total cost: Census design, methodology, and 
evaluation; 
Total category cost $531.2 million; 
Most costly life cycle budget categories and projects: 16% of total: 
Coverage measurement system and process: $28.8 million (34%); 
American Community Survey methods panel: $28.8 million (34%); 
Evaluations planning and coordination: $26.8 million (32%). 

Budget category and total cost: Program development and management; 
Total category cost: $69.5 million; 
Most costly life cycle budget categories and projects: 37% of total; 
Program design and development: $11.5 million (44%); 
American Community Survey project management: $8.3 million (32%); 
Program design and development: $6.2 million (24%). 

Budget category and total cost: Contingency; 
Total category cost: $77.9 million; 
Most costly life cycle budget categories and projects: 83% of total; 
Fingerprinting — Field and National Processing Center: $27.3 million 
(42%); 
Fingerprinting — NRFU: $20.7 million (32%); 
Contingency — Decennial Response Integration System contract: $17.0 
million (26%). 

Budget category and total cost: Census test and dress rehearsal 
implementation; 
Total category cost: $107.9 million; 
Most costly life cycle budget categories and projects: 22% of total; 
Census test — Local census offices: $9.5 million (40%); 
Census test — Regional office support: $7.5 million (32%); 
Special purpose tests: $6.6 million (28%). 

Budget category and total cost: Uncategorized expenditures; 
Total category cost: $27.6 million; 
Most costly life cycle budget categories and projects: 100% of total; 
Adjustments: $27.6 million (100%). 

Source: GAO analysis of U.S. Census Bureau data. 

Notes: This graphic is a static breakdown of the data presented in the 
background section of the report. Some projects have the same name, 
but these costs do not overlap with one another. The money was spent 
in different time periods for each separate project. All costs were 
adjusted to 2010 dollars using the gross domestic product deflator. 
The Bureau identified one project that did not fit into any regular 
category, the project labeled "adjustments." Changes were made to 
project descriptions to increase clarity. For example, acronyms were 
spelled out and abbreviations were expanded into full words. 

[End of figure] 

[End of section] 

Appendix IV: The 12 Steps of a High-Quality Cost Estimating Process: 

Step: 1; 
Description: Define estimate's purpose; 
Associated tasks: 
* Determine estimate's purpose, required level of detail, and overall 
scope; 
* Determine who will receive the estimate. 

Step: 2; 
Description: Develop estimating plan; 
Associated tasks: 
* Determine the cost estimating team and develop its master schedule; 
* Determine who will do the independent cost estimate; 
* Outline the cost estimating approach; 
* Develop the estimate timeline. 

Step: 3; 
Description: Define program characteristics; 
Associated tasks: 
* In a technical baseline description document, identify the program's 
purpose and its system and performance characteristics and all system 
configurations; 
* Any technology implications; 
* Its program acquisition schedule and acquisition strategy; 
* Its relationship to other existing systems, including predecessor or 
similar legacy systems; 
* Support (manpower, training, etc.) and security needs and risk items; 
* System quantities for development, test, and production; 
* Development and maintenance plans. 

Step: 4; 
Description: Determine estimating structure; 
Associated tasks: 
* Define a work breakdown structure (WBS) and describe each element in 
a WBS dictionary (a major automated information system may have only a 
cost element structure); 
* Choose the best estimating method for each WBS element; 
* Identify potential cross-checks for likely cost and schedule drivers; 
* Develop a cost estimating checklist. 

Step: 5; 
Description: Identify ground rules and assumptions; 
Associated tasks: 
* Clearly define what the estimate includes and excludes; 
* Identify global and program-specific assumptions, such as the 
estimate's base year, including time phasing and life cycle; 
* Identify program schedule information by phase and program 
acquisition strategy; 
* Identify any schedule or budget constraints, inflation assumptions, 
and travel costs; 
* Specify equipment the government is to furnish as well as the use of 
existing facilities or new modification or development; 
* Identify prime contractor and major subcontractors; 
* Determine technology refresh cycles, technology assumptions, and new 
technology to be developed; 
* Define commonality with legacy systems and assumed heritage savings; 
* Describe effects of new ways of doing business. 

Step: 6; 
Description: Obtain data; 
Associated tasks: 
* Create a data collection plan with emphasis on collecting current 
and relevant technical, programmatic, cost, and risk data; 
* Investigate possible data sources; 
* Collect data and normalize them for cost accounting, inflation, 
learning, and quantity adjustments; 
* Analyze the data for cost drivers, trends, and outliers and compare 
results against rules of thumb and standard factors derived from 
historical data; 
* Interview data sources and document all pertinent information, 
including an assessment of data reliability and accuracy; 
* Store data for future estimates. 

Step: 7; 
Description: Develop point estimate and compare it to an independent 
cost estimate; 
Associated tasks: 
* Develop the cost model, estimating each WBS element, using the best 
methodology from the data collected,[A] and including all estimating 
assumptions; 
* Express costs in constant year dollars; 
* Time-phase the results by spreading costs in the years they are 
expected to occur, based on the program schedule; 
* Sum the WBS elements to develop the overall point estimate; 
* Validate the estimate by looking for errors like double counting and 
omitted costs; 
* Compare estimate against the independent cost estimate and examine 
where and why there are differences; 
* Perform cross-checks on cost drivers to see if results are similar; 
* Update the model as more data become available or as changes occur 
and compare results against previous estimates. 

Step: 8; 
Description: Conduct sensitivity analysis; 
Associated tasks: 
* Test the sensitivity of cost elements to changes in estimating input 
values and key assumptions; 
* Identify effects on the overall estimate of changing the program 
schedule or quantities; 
* Determine which assumptions are key cost drivers and which cost 
elements are affected most by changes. 

Step: 9; 
Description: Conduct risk and uncertainty analysis; 
Associated tasks: 
* Determine and discuss with technical experts the level of cost, 
schedule, and technical risk associated with each WBS element; 
* Analyze each risk for its severity and probability; 
* Develop minimum, most likely, and maximum ranges for each risk 
element; 
* Determine type of risk distributions and reason for their use; 
* Ensure that risks are correlated; 
* Use an acceptable statistical analysis method (e.g., Monte Carlo 
simulation) to develop a confidence interval around the point estimate; 
* Identify the confidence level of the point estimate; 
* Identify the amount of contingency funding and add this to the point 
estimate to determine the risk-adjusted cost estimate; 
* Recommend that the project or program office develop a risk 
management plan to track and mitigate risks. 

Step: 10; 
Description: Document the estimate; 
Associated tasks: 
* Document all steps used to develop the estimate so that a cost 
analyst unfamiliar with the program can re-create it quickly and 
produce the same result; 
* Document the purpose of the estimate, the team that prepared it, and 
who approved the estimate and on what date; 
* Describe the program, its schedule, and the technical baseline used 
to create the estimate; 
* Present the program's time-phased life cycle cost; 
* Discuss all ground rules and assumptions; 
* Include auditable and traceable data sources for each cost element 
and document for all data sources how the data were normalized; 
* Describe in detail the estimating methodology and rationale used to 
derive each WBS element's cost (prefer more detail over less); 
* Describe the results of the risk, uncertainty, and sensitivity 
analyses and whether any contingency funds were identified; 
* Document how the estimate compares to the funding profile; 
* Track how this estimate compares to any previous estimates. 

Step: 11; 
Description: Present estimate to management for approval; 
Associated tasks: 
* Develop a briefing that presents the documented life cycle cost 
estimate; 
* Include an explanation of the technical an programmatic baseline and 
any uncertainties; 
* Compare the estimate to an independent cost estimate (ICE) and 
explain any differences; 
* Compare the estimate (life cycle cost estimate (LCCE)) or 
independent cost estimate to the budget with enough detail to easily 
defend it by showing how it is accurate, complete, and high in quality; 
* Focus in on logical manner on the largest cost elements and cost 
drivers; 
* Make the content clear and complete so that those how are unfamiliar 
with it can easily comprehend the competence that underlies the 
estimate results; 
* Make backup slides available for more probe questions; 
* Act on and document feedback from management; 
* Request acceptance of the estimate. 

Step: 12; 
Description: Update the estimate to reflect actual costs and changes; 
Associated tasks: 
* Update the estimate to reflect changes in technical or program 
assumptions or keep it current as the program passes through new 
phases or milestones; 
* Replace estimates with earned value management (EVM) estimate at 
completion (EAC) and independent EAC from the integrated EVM system; 
* Report progress on meeting cost and schedule estimates; 
* Perform a postmortem and document lessons learned for elements whose 
actual costs or schedules differ from the estimate; 
* Document all changes to the program and how they affect the cost 
estimate. 

Source: GAO-09-3SP. 

[A] In a data-rich environment, the estimating approach should precede 
the investigation of data sources; in reality, a lack of data often 
determines the approach. 

[End of table] 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Signora May, Assistant 
Director; Tom Beall; Tim Carr; Eric Charles; Sara Daleski; Dewi 
Djunaidy; Ron Fecso; Robert Gebhart; Rich Hung; Kirsten Lauber; Jason 
Lee; Andrea Levine; Donna Miller; Stacey Steele; and Jack Wang made 
key contributions to this report. 

[End of section] 

Footnotes: 

[1] U.S. Census Bureau, Business Plan for the 2020 Census in Support 
of the FY 2012 Budget Initiative (June 13, 2011). 

[2] Fiscal years 2011 through 2013 costs are projected for the 2010 
life cycle. 

[3] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[4] The LUCA Program enables state, local, and tribal governments to 
review and update the list of addresses and maps the Bureau uses to 
deliver questionnaires within those communities. 

[5] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[6] GAO, 2010 Census: Census Bureau Should Take Action to Improve the 
Credibility and Accuracy of Its Cost Estimate for the Decennial 
Census, [hyperlink, http://www.gao.gov/products/GAO-08-554] 
(Washington, D.C.: June 16, 2008). 

[7] GAO, 2000 Census: Significant Increase in Cost Per Housing Unit 
Compared to 1990 Census, [hyperlink, 
http://www.gao.gov/products/GAO-02-31] (Washington, D.C.: Dec. 11, 
2001). 

[8] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[9] [hyperlink, http://www.gao.gov/products/GAO-02-31]. 

[10] The updates to the address frame here refer to activities in 2009 
to update the address frame, such as address canvassing where census 
employees check addresses in the field. 

[11] U.S. Census Bureau, Business Plan for the 2020 Census. 

[12] In the 2010 and earlier censuses, the Bureau conducted full 
address canvassing, where census workers generally went door-to-door 
and attempted to verify every address in the country. Targeted address 
canvassing would limit this operation to areas in which the Bureau 
believes more work is needed to develop an accurate and complete 
address list. 

[13] OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost 
Analysis of Federal Programs (Oct. 29, 1992). 

[14] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[15] GAO, 2010 Census: Data Collection Operations Were Generally 
Completed as Planned, but Long-standing Challenges Suggest Need for 
Fundamental Reforms, [hyperlink, 
http://www.gao.gov/products/GAO-11-193] (Washington, D.C.: Dec. 14, 
2010), and 2010 Census: Preliminary Lessons Learned Highlight the Need 
for Fundamental Reforms, [hyperlink, 
http://www.gao.gov/products/GAO-11-496T] (Washington, D.C.: Apr. 6, 
2011). 

[16] According to the 2010 Census Operations and Systems Plan, address-
building operations include LUCA, the New Construction Operation, the 
Address Canvassing Operation, and the Group Quarters Validation 
Operation, among others, and serve as the building blocks for an 
accurate mailing list. 

[17] GAO, 2010 Census: Efforts to Build an Accurate Address List Are 
Making Progress, but Face Software and Other Challenges, [hyperlink, 
http://www.gao.gov/products/GAO-10-140T] (Washington, D.C.: Oct. 21, 
2009). 

[18] For example, see GAO, Best Practices: Capturing Design and 
Manufacturing Knowledge Early Improves Acquisition Outcomes, 
[hyperlink, http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: 
July 15, 2002), and Best Practices: Better Matching of Needs and 
Resources Will Lead to Better Weapon System Outcomes, [hyperlink, 
http://www.gao.gov/products/GAO-01-288] (Washington, D.C.: Mar. 8, 
2001). 

[19] The other three criteria are business and technical requirements 
and risk. 

[20] U.S. Census Bureau, Plan for Developing 2020 Census Alternative, 
Memorandum No. 6 (Jan. 15, 2010). 

[21] GAO, A Model of Strategic Human Capital Management, [hyperlink, 
http://www.gao.gov/products/GAO-02-373SP] (Washington, D.C.: Mar. 15, 
2002). 

[22] The Bureau did not complete approximations for the remaining 
alternatives, those that would maximize the use of administrative 
records and therefore potentially realize the most costs savings, but 
plans to do so after gathering information from the research and 
testing phase. 

[23] The definition of a "large, complex project" varies by agency 
depending on the size and value of the assets it manages. 

[24] A rough-order-of-magnitude estimate is a quick, high-level 
estimate that generally involves less time and effort than a budget-
quality estimate. The Bureau chose not to do estimates for design 
alternatives five and six, which are more radical changes, because 
Bureau officials did not feel they yet had sufficient information to 
develop reliable estimates. 

[25] [hyperlink, http://www.gao.gov/products/GAO-08-554]. 

[26] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[27] The system is referred to in older GAO reports as the Commerce 
Administrative Management System (CAMS). It is the same system, but 
the name changed over time. 

[28] There were a small number of exceptions that could be directly 
matched based on project descriptions, such as the nonresponse follow-
up operation. 

[29] The Bureau said it has no such crosswalk at the project level and 
said that such comparison is inherently difficult because of the 
changing nature of the census. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select “E-
mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548.