2020 Census: Census Bureau Needs to Improve Its Life-Cycle Cost Estimating Process
Highlights
What GAO Found
Since 2012, the U.S. Census Bureau (Bureau) has taken significant steps to improve its capacity to carry out an effective cost estimate; however, its October 2015 cost estimate for the 2020 Census does not fully reflect characteristics of a high-quality estimate and cannot be considered reliable. To reflect these characteristics, an organization must meet or substantially meet four best practices. Overall, GAO found the cost estimate partially met the characteristics of two best practices (comprehensive and accurate) and minimally met the other two (well-documented and credible). One reason why GAO's overall assessment is low is because the estimate is not well-documented. Improving cost estimation practices will increase the reliability of the Bureau's cost estimate, which will in turn help improve decision making, budget formulation, progress measurement, course correction when warranted, and accountability for results.
Best practices state a risk and uncertainty analysis should be performed to determine the level of risk associated with the cost estimate. The Bureau carried out such an analysis only for a portion of estimated costs for fiscal years 2018 to 2020. According to Bureau officials, they scoped the analysis narrowly to those 3 years when most of the census costs occur. GAO found that, as a result, the Bureau's risk and uncertainty analysis (modeled costs) covered $4.6 billion, only about 37 percent of the $12.5 billion total estimated life-cycle cost, and less than one-half of the total estimated cost of the census during future fiscal years.
Components of Total 2020 Census Life-Cycle Cost Estimate
Note: All figures are in constant 2020 dollars.
The Bureau has risk identification processes, which identify a broad range of risks that could affect the cost of the 2020 Census. Yet this awareness of risk is not leveraged in the Bureau's cost estimation. The cost estimation team did not consult risk registers or examine specific risks directly for inclusion in the cost model or risk and uncertainty analysis. It was not known what risks, if any, had been accounted for in other data in the cost model. As a result, neither the Bureau nor GAO are able to determine with confidence what risks the Bureau is prepared to mitigate or address within its $12.5 billion cost estimate. Improving control over how risk is accounted for will improve confidence that the Bureau's budgeted contingencies are at appropriate levels.
Why GAO Did This Study
In October 2015, the Bureau estimated that with its new approach it can conduct the 2020 Census for $12.5 billion, $5 billion less than the $17.8 billion it estimated it would cost to repeat the design and methods of the 2010 Census. Reliable cost estimates can help an agency manage large complex activities like the 2020 Census, as well as help Congress make funding decisions and provide oversight.
GAO was asked to evaluate the reliability of the Bureau's life-cycle cost estimate. Among other objectives, this report assesses the extent to which (1) the Bureau's life-cycle cost estimate met GAO's best practices for cost estimation and (2) the Bureau identified and accounted for key risks facing the 2020 Census. To meet these objectives, GAO reviewed documentary and testimonial evidence from Bureau officials responsible for developing the 2020 Census cost estimate. GAO used its cost assessment guide ( GAO-09-3SP ) and Standards for Internal Control in the Federal Government ( GAO-14-704G ) as criteria.
Recommendations
GAO is making three recommendations including that the Secretary of Commerce direct the Bureau to take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate and improve control over how risk and uncertainty are accounted for in cost estimation. The Department of Commerce agreed with GAO's recommendations and provided additional context that was incorporated, as appropriate.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce |
Priority Rec.
To help ensure the Bureau produces a reliable cost estimate for the 2020 Census, the Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to take the following steps to meet the characteristics of a high-quality estimate: (1) Comprehensive--among other practices, ensure the estimate includes all life-cycle costs and documents all cost-influencing assumptions. (2) Well-documented--among other practices, ensure that its planned documentation plan captures the source data used; contains the calculations performed and the estimating methodologies used for each element; and describes step by step how the estimate was developed. (3) Accurate--among other practices, ensure the estimating technique for each cost element is used appropriately and that variances between planned and actual cost are documented, explained, and reviewed. (4) Credible--among other practices, ensure the estimate includes a sensitivity analysis, major cost elements are cross-checked to see whether results are similar, and an independent cost estimate is conducted to determine whether other estimating methods produce similar results. |
In August 2019 the Bureau provided an update to the 2020 Census cost estimate, which we found to be sufficiently reliable as it now meets the four characteristics of a reliable cost estimate. The Bureau's 2017 cost estimate had met or substantially met three out of the four characteristics of a reliable cost estimate (comprehensive, accurate, and credible), while it only partially met the characteristic for being well-documented. We assessed the Bureau's 2019 update to the cost-estimate and additional provided documentation and found that the estimate now substantially meets the well-documented characteristic. Specifically, the Bureau now substantially meets the best practice for discussing the technical baseline description and ensuring that data in the baseline is consistent with the estimate. Moreover, the Bureau has also implemented a system to track and report variances between actual and expected cost elements. Tools to track these variances are important because it enables management to measure progress against planned outcomes.
|
| Department of Commerce |
Priority Rec.
To further ensure the credibility of data used in cost estimation, the Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to establish clear guidance on when information for cost assumptions can and should be changed as well as the procedures for documenting such changes and traceable sources for information being used. |
Commerce agreed and has taken action to implement this recommendation. On May 4, 2018, in response to GAO's recommendation, Bureau officials provided GAO with their Life Cycle Cost Estimate Version Control Plan document. Our review of the Bureau's 2017 life cycle cost estimate determined that the Bureau substantially met GAO criteria for documenting cost-influencing ground rules and assumptions. Specifically, documentation of the Bureau's approach included guidance on cost estimate updates, changes and releases and procedures for providing traceable sources of information for cost assumptions. Such changes support the overall reliability of cost estimates that management can use for making informed decisions.
|
| Department of Commerce |
Priority Rec.
To ensure Bureau and congressional confidence that the Bureau's budgeted contingencies are at appropriate levels, the Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to improve control over how risk and uncertainty are accounted for and communicated with the Bureau's decennial cost estimation process, such as by implementing and institutionalizing processes or methods for doing so with clear guidance. |
Commerce agreed with this recommendation. Specifically, we reported that the Bureau should ensure that its budget for contingencies for the 2020 Census reflect an accurate accounting of risk and uncertainty. In doing this, the Bureau should improve controls over risk and uncertainty in the cost estimate process, and institutionalize these controls by providing clear methods for their use. We reviewed documentation to support the updated October 2017 cost estimate of $15.6 billion and in August 2018 reported that the Bureau performed uncertainty and sensitivity analysis on the estimate and appropriately added funding into the cost estimate to reflect inherent uncertainty and to account for specific risks. While the Bureau was able to better quantify risk; we also reported in August 2018 that the Secretary of Commerce included a contingency amount of about $1.2 billion in the 2017 cost estimate to account for what the Bureau refers to as "unknown unknowns." According to Bureau documentation, these include such risks as natural disasters or cyber attacks. The Bureau provided a description of how the risk contingency for "unknown unknowns" is calculated; however, this description did not clearly link calculated amounts to the risks themselves. Thus, only $14.4 billion of the Bureau's $15.6 billion cost estimate had justification. In April 2019, we reported that according to Bureau officials, while the cost estimate remained at $15.6 billion, they were managing the 2020 Census at a lower level of funding and, at that time, did not request contingency funds for unknown unknowns. Bureau officials told us instead of requesting funding for these contingencies upfront the Bureau had planned to work with OMB and Commerce to request additional funds, if needed. Although, the Bureau did not request additional Secretarial-directed contingency funds for FY 2018 - 2021, Congress appropriated funding for the Secretarial contingency fund in each of those years except for 2019. While the Bureau was not able to link specific risks to the $1.2 billion in contingency funds, we found that the Bureau did improve controls over risk for the remaining cost estimate and then actively managed to a cost estimate without the $1.2 billion secretarial general contingency fund. Because of these actions we consider this recommendation as implemented.
|