Actions Needed to Evaluate the Impact of Efforts to Estimate Costs of Reports and Studies
GAO-12-480R: Published: May 10, 2012. Publicly Released: May 10, 2012.
- Accessible Text:
What GAO Found
DOD is estimating and publishing approximate costs for selected types of internally and externally required reports, but in some cases its approach is not fully consistent with relevant cost estimating best practices and cost accounting standards. Specifically, DOD entities have been directed to use the cost estimating tool to capture marginal costs of activities associated with completing a report or study that would not have been performed otherwise. These costs consist of certain manpower costs (such as the prorated salaries of military and civilian personnel based on the time they spent) and nonlabor costs (such as contract services, travel, or printing). In comparing DODs approach to (1) GAOs Cost Estimating and Assessment Guidewhich states that high-quality, reliable cost estimates should be comprehensive, well documented, and accurateand (2) relevant accounting standards, we found the following.
Comprehensive. GAOs cost guide states that cost estimates should include all costs, but allows flexibility for the estimates to exclude costs where information is limited as long as steps are taken to clearly define and document what costs are included or excluded. DODs guidance on using the tool defines and documents decisions to include certain manpower and all nonlabor costs in its calculation. In addition, it documents the decision to exclude other manpower costs, such as those for health care and training expenses. It also provides broad examples of some types of activities to consider in estimating costs, but leaves it up to the discretion of the individuals generating the cost estimate to decide which types of activities to include or exclude. As a result, based on the nine reports we reviewed, we found inconsistencies in the types of activities individuals decided to include when developing their cost estimates. For example, we found that for six of the nine reports we reviewed, some individuals calculated the manpower costs for activities associated with coordinating the report through the final review while others did not. In addition, according to relevant accounting standards, appropriate procedures and practices should be established to enable, among other things, the interpretation and communication of cost information. However, when presenting the cost estimate on the front cover of a report or study, the language DOD uses does not provide information on what costs are included or excluded. Specifically, it does not indicate that the estimate reflects only certain marginal manpower and nonlabor costs. Without further explanation, this wording could be subject to misinterpretation such that recipients of the reports or studies may assume that all costs are included in the estimate when that is not the case.
Well documented. GAOs Cost Estimating and Assessment Guide states that cost estimates should be easily traceable to source documents. While DODs guidance on using the tool states that individuals should be prepared to explain how they developed the cost estimate, it does not include a requirement for individuals to retain source documentation. In practice, we found that of the nine reports we reviewed, individuals were able to easily retrieve documentation for the estimates prepared for three reports, but not for the other six. As a result, DOD may not have the information needed so that others can understand how a cost estimate was derived and readily replicate it.
Accurate. GAOs Cost Estimating and Assessment Guide further states that cost estimates should have all cost inputs checked to verify that they are accurate. However, we found that DODs guidance on using the tool has no requirement to independently verify the costs used to generate the cost estimate. In practice, the individual generating the cost estimate is responsible for ensuring its accuracy. As a result, without independent verification, DOD cannot ensure that cost estimates are accurate and reliable.
DOD currently lacks a means to ensure that organizations are developing cost estimates for all required types of reports and studies, but is taking steps to enhance its ability to monitor the preparation of reports and studies to satisfy reporting requirements, including those for which a cost estimate is required to be generated. DODs guidance on using the tool identified 10 specific types of reports or studies that require a cost estimate. The guidance states that the DOD component preparing a report or study is responsible for ensuring that a cost estimate, if required, is included. However, the guidance does not include any process or requirement to track whether organizations are developing required cost estimates. According to CAPE officials who developed the tool, they were not directed to ensure that cost estimates have been developed for all required reports and studies, and they added that the cost estimating tool was not designed to track reports and studies that require a cost estimate. These same officials noted that the magnitude of DODs reporting requirements makes it challenging to identify the universe of these requirements and to track the completion of reports and studies to meet them, including whether cost estimates have been generated. In the past year, DOD has initiated efforts to improve its visibility over its internal and external reporting requirements. In March 2011, the Secretary of Defense issued a memorandum requiring the Assistant Secretary of Defense for Legislative Affairs to enhance that offices database capability and further automate the process of tracking external reports to Congress and required the Office of the Director of Administration and Management to track internal reports. The Office of Legislative Affairs enhanced its Congressional Hearings and Reporting Requirements Tracking System by adding a data field to record the estimated costs of reports or studies generated by DOD components. To gain greater visibility of internal reporting requirements, the Office of the Director of Administration and Management tasked Washington Headquarters Services with developing a repository for tracking internal reports. According to a Washington Headquarters Services official, the repository has a field to capture whether an internal report or study has a cost estimate. Currently, the repository is in development and has been populated with reports that date back 3 years.
DOD has not evaluated and currently does not plan to evaluate the usefulness of its efforts to estimate the costs of selected reports and studies as one of the means for achieving the Secretary of Defenses intended purpose of increasing the transparency of costs, reducing or eliminating reporting requirements, and instilling a culture of cost consciousness across DOD. According to GAOs Standards for Internal Control in the Federal Government, activities such as assessments need to be established to monitor performance, and managers need to compare actual performance against targets and then analyze any significant differences so that appropriate action can be taken to ensure that organizational goals are met. However, under current guidance, no requirement exists for DOD to evaluate the impact of its cost estimating efforts, such as whether these efforts have prompted internal or external decision makers to consider cost as a factor in deciding whether to establish a reporting requirement for DOD. Without evaluating the impact of its efforts, including seeking the views of internal and external decision makers, DOD does not have the information it needs to assess whether the time and effort it is investing to develop cost estimates is having the desired effect of achieving greater transparency, reducing reporting requirements, and raising cost consciousness, as intended by the Secretary.
Why GAO Did This Study
Citing long-term fiscal challenges affecting the federal government, in May 2010, the Secretary of Defense directed the Department of Defense (DOD) to undertake a departmentwide initiative to assess how the department is staffed, organized, and operated with the goal of reducing excess overhead costs and reinvesting these savings in sustaining DODs current force structure and modernizing its weapons portfolio. The Secretarys initiative targeted both shorter- and longer-term improvements and set specific goals and targets for achieving cost savings and efficiencies. The initiative was organized along four tracks, each of which had a different focus. The fourth track focused on specific areas where DOD could take immediate action to reduce inefficiencies and overhead, in particular, to reduce headquarters and support bureaucracies and to instill a culture of cost consciousness and restraint in the department. As part of the fourth track, the Secretary of Defense announced a number of specific initiatives, including actions intended to address the need to reduce or eliminate reporting requirements for DOD reports and studies. For example, in his August 9, 2010, speech announcing the overall efficiency initiative, the Secretary of Defense stated that the department is awash in taskings for reports and studies and directed several specific actions that, according to the press release accompanying the announcement, were intended to combat the enormous amounts of taskings for reports and studies. In his remarks, the Secretary noted that there is little basis to determine whether the value gained is worth the considerable time and resources expended to generate reports and studies. With respect to specific actions, the Secretary directed DOD and its components to track the approximate cost of preparing DOD reports and studies and publish the cost on the front cover of each report or study. He also called for a comprehensive review of all oversight reports and use of the results to reduce the volume of reports and studies generated internally while engaging with Congress on ways to meet their needs while working together to reduce the number of reports.
In September 2010, the Secretary of Defense tasked DODs Cost Assessment and Program Evaluation (CAPE) office with implementing the Secretarys requirement to publish the cost of preparing DOD reports and studies. By November 2010, the CAPE office had developed a cost estimating tool and corresponding guidance on using the tool, which described the categories of costs to be estimated and identified the types of reports and studies for which costs were to be estimated. The following month, the Secretary of Defense issued a memorandum requiring the use of the tool beginning on February 1, 2011.
In House Report Number 112-78, which accompanied a bill for the National Defense Authorization Act for Fiscal Year 2012, the House Committee on Armed Services commended DOD for implementing a process for collecting an estimate of resources required to generate internally and externally required reports and agreed that additional transparency would be useful for decision makers when determining the utility of various reporting requirements. The committee also observed that any tool used to collect costs was only as useful as the inputs received and directed GAO to conduct an assessment of DODs methodology and tools for collecting cost data on both internally and externally required reports and to submit a report, including any recommendations needed to improve the data collection, transparency, and utility of the tool. Specifically, we evaluated DODs approach to estimate the costs of selected reports and studies, including (1) whether DOD entities are capturing and presenting costs in a manner consistent with relevant cost estimating best practices; (2) the status of DODs efforts to track whether organizations are developing cost estimates for all required types of reports and studies; and (3) whether DOD has evaluated the usefulness of its efforts to estimate the costs of reports and studies as one of the means for achieving the Secretarys intended purpose of increasing the transparency of costs, reducing or eliminating reporting requirements, and instilling a culture of cost consciousness across DOD.
What GAO Recommends
We are making a recommendation to the Secretary of Defense to take steps to evaluate DODs effort to estimate costs to determine whether that effort is having the desired effect of achieving greater transparency, reducing or eliminating reporting requirements, and raising cost awareness. Based on the results of this evaluation, if DOD plans to continue the effort to estimate costs for selected reports and studies, we recommend that the Secretary modify the current guidance or otherwise take steps to improve DODs cost estimating approach. In commenting on a draft of our report, DOD partially concurred with our recommendations, except it disagreed with one of the steps we recommended to improve its cost estimating approach.
For more information, please contact Sharon L. Pickup at (202) 512-9619 or firstname.lastname@example.org.
- Review Pending
- Closed - implemented
- Closed - not implemented
Recommendations for Executive Action
Recommendation: To determine whether its effort to estimate costs is having the desired effect of achieving greater transparency, reducing or eliminating reporting requirements, and raising cost awareness, the Secretary of Defense should take steps to evaluate the effort, including reaching out to internal and external decision makers to obtain their views on how it has affected their decisions to establish internal and external reporting requirements and whether they have a common understanding of what types of costs are being reported.
Agency Affected: Department of Defense
Comments: On August 14, 2013, a Cost Assessment and Program Evaluation (CAPE) official provided anecdotal information stating they they had received feedback from internal and external stakeholders about the cost estimating tool. He stated that they had efforts ongoing prior to the completion of the GAO study and has since continued those efforts but have not established, and do not intend to establish, a formal, systematic process to gather views on the impact the tool has had on establishing reporting requirements or whether there is a common understanding of the types of cost that are captured in the cost estimate. The CAPE official stated that no other action is planned at this time.
Recommendation: Based on the results of this evaluation, if DOD plans to continue the effort to estimate costs for selected reports and studies, the Secretary of Defense should modify the current guidance or otherwise take steps to improve DOD's cost estimating approach, including (1) clarifying those activities that should be included in the estimate as they are incurred, including time spent at every level to review and approve the report or study; (2) requiring that the language on the front cover of reports and studies make clear that the estimate is based on certain marginal manpower and nonlabor costs; (3) requiring that source documentation used to develop the cost estimate is retained and easily accessible for review purposes; and (4) establishing and implementing a verification process to provide reasonable assurance of consistency and completeness of cost inputs used to develop the cost estimate.
Agency Affected: Department of Defense
Comments: CAPE's Studies and Reports Guidance Document was updated in October 2012 and partially addresses our recommendation that it clarify those activities that should be included in the estimate as they are incurred, including time spent at every level to review and approve the report or study. In response to the GAO recommendation, guidance was updated so that oversight activities could be captured in the manpower portion of the cost estimate. However, the guidance fails to specify what these activities include,such as reviewing and approving the report or study, and it continues to provide broad examples of some types of activities to consider in estimating costs. Further, the guidance continues to allow individuals to use their best judgment in determining what costs to include. This could potentially lead to inconsistencies in the types of activities that individuals decide to capture in their cost estimate. In response to our recommendation to clarify the language on the front cover of reports and studies to make clear that the estimate is based on certain marginal manpower and nonlabor costs, DOD has taken action that meets the intent of this recommendation. Specifically, the CAPE Studies and Reports Guidance Document was updated in October 2012 to change the label on the front cover from displaying: "Preparation of this report/study cost the Department of Defense a total of approximately $________ in Fiscal Year _____" to "The estimated costs of report or study for the Department of Defense is approximately $_________ for the ________Fiscal Year. This includes $_____ in expenses and $________ for labor." DOD nonconcurred with our recommendations to require that source documentation used to develop the cost estimated is retained and easily accessible for review purpose and to establish and implement a verification process to provide reasonable assurance of consistency and completeness of cost inputs used to develop the cost estimate. According to a CAPE official no action has been or will be taken in response to these recommendations.