Defense Management: Actions Needed to Evaluate the Impact of Efforts to Estimate Costs of Reports and Studies

What GAO Found

DOD is estimating and publishing approximate costs for selected types of internally and externally required reports, but in some cases its approach is not fully consistent with relevant cost estimating best practices and cost accounting standards. Specifically, DOD entities have been directed to use the cost estimating tool to capture marginal costs of activities associated with completing a report or study that would not have been performed otherwise. These costs consist of certain manpower costs (such as the prorated salaries of military and civilian personnel based on the time they spent) and nonlabor costs (such as contract services, travel, or printing). In comparing DOD’s approach to (1) GAO’s Cost Estimating and Assessment Guide—which states that high-quality, reliable cost estimates should be comprehensive, well documented, and accurate—and (2) relevant accounting standards, we found the following.

Comprehensive. GAO’s cost guide states that cost estimates should include all costs, but allows flexibility for the estimates to exclude costs where information is limited as long as steps are taken to clearly define and document what costs are included or excluded. DOD’s guidance on using the tool defines and documents decisions to include certain manpower and all nonlabor costs in its calculation. In addition, it documents the decision to exclude other manpower costs, such as those for health care and training expenses. It also provides broad examples of some types of activities to consider in estimating costs, but leaves it up to the discretion of the individuals generating the cost estimate to decide which types of activities to include or exclude. As a result, based on the nine reports we reviewed, we found inconsistencies in the types of activities individuals decided to include when developing their cost estimates. For example, we found that for six of the nine reports we reviewed, some individuals calculated the manpower costs for activities associated with coordinating the report through the final review while others did not. In addition, according to relevant accounting standards, appropriate procedures and practices should be established to enable, among other things, the interpretation and communication of cost information. However, when presenting the cost estimate on the front cover of a report or study, the language DOD uses does not provide information on what costs are included or excluded. Specifically, it does not indicate that the estimate reflects only certain marginal manpower and nonlabor costs. Without further explanation, this wording could be subject to misinterpretation such that recipients of the reports or studies may assume that all costs are included in the estimate when that is not the case.

Well documented. GAO’s Cost Estimating and Assessment Guide states that cost estimates should be easily traceable to source documents. While DOD’s guidance on using the tool states that individuals should be prepared to explain how they developed the cost estimate, it does not include a requirement for individuals to retain source documentation. In practice, we found that of the nine reports we reviewed, individuals were able to easily retrieve documentation for the estimates prepared for three reports, but not for the other six. As a result, DOD may not have the information needed so that others can understand how a cost estimate was derived and readily replicate it.

Accurate. GAO’s Cost Estimating and Assessment Guide further states that cost estimates should have all cost inputs checked to verify that they are accurate. However, we found that DOD’s guidance on using the tool has no requirement to independently verify the costs used to generate the cost estimate. In practice, the individual generating the cost estimate is responsible for ensuring its accuracy. As a result, without independent verification, DOD cannot ensure that cost estimates are accurate and reliable.

DOD currently lacks a means to ensure that organizations are developing cost estimates for all required types of reports and studies, but is taking steps to enhance its ability to monitor the preparation of reports and studies to satisfy reporting requirements, including those for which a cost estimate is required to be generated. DOD’s guidance on using the tool identified 10 specific types of reports or studies that require a cost estimate. The guidance states that the DOD component preparing a report or study is responsible for ensuring that a cost estimate, if required, is included. However, the guidance does not include any process or requirement to track whether organizations are developing required cost estimates. According to CAPE officials who developed the tool, they were not directed to ensure that cost estimates have been developed for all required reports and studies, and they added that the cost estimating tool was not designed to track reports and studies that require a cost estimate. These same officials noted that the magnitude of DOD’s reporting requirements makes it challenging to identify the universe of these requirements and to track the completion of reports and studies to meet them, including whether cost estimates have been generated. In the past year, DOD has initiated efforts to improve its visibility over its internal and external reporting requirements. In March 2011, the Secretary of Defense issued a memorandum requiring the Assistant Secretary of Defense for Legislative Affairs to enhance that office’s database capability and further automate the process of tracking external reports to Congress and required the Office of the Director of Administration and Management to track internal reports. The Office of Legislative Affairs enhanced its Congressional Hearings and Reporting Requirements Tracking System by adding a data field to record the estimated costs of reports or studies generated by DOD components. To gain greater visibility of internal reporting requirements, the Office of the Director of Administration and Management tasked Washington Headquarters Services with developing a repository for tracking internal reports. According to a Washington Headquarters Services official, the repository has a field to capture whether an internal report or study has a cost estimate. Currently, the repository is in development and has been populated with reports that date back 3 years.

DOD has not evaluated and currently does not plan to evaluate the usefulness of its efforts to estimate the costs of selected reports and studies as one of the means for achieving the Secretary of Defense’s intended purpose of increasing the transparency of costs, reducing or eliminating reporting requirements, and instilling a culture of cost consciousness across DOD. According to GAO’s Standards for Internal Control in the Federal Government, activities such as assessments need to be established to monitor performance, and managers need to compare actual performance against targets and then analyze any significant differences so that appropriate action can be taken to ensure that organizational goals are met. However, under current guidance, no requirement exists for DOD to evaluate the impact of its cost estimating efforts, such as whether these efforts have prompted internal or external decision makers to consider cost as a factor in deciding whether to establish a reporting requirement for DOD. Without evaluating the impact of its efforts, including seeking the views of internal and external decision makers, DOD does not have the information it needs to assess whether the time and effort it is investing to develop cost estimates is having the desired effect of achieving greater transparency, reducing reporting requirements, and raising cost consciousness, as intended by the Secretary.

Why GAO Did This Study

Citing long-term fiscal challenges affecting the federal government, in May 2010, the Secretary of Defense directed the Department of Defense (DOD) to undertake a departmentwide initiative to assess how the department is staffed, organized, and operated with the goal of reducing excess overhead costs and reinvesting these savings in sustaining DOD’s current force structure and modernizing its weapons portfolio. The Secretary’s initiative targeted both shorter- and longer-term improvements and set specific goals and targets for achieving cost savings and efficiencies. The initiative was organized along four tracks, each of which had a different focus. The fourth track focused on specific areas where DOD could take immediate action to reduce inefficiencies and overhead, in particular, to reduce headquarters and support bureaucracies and to instill a culture of cost consciousness and restraint in the department. As part of the fourth track, the Secretary of Defense announced a number of specific initiatives, including actions intended to address the need to reduce or eliminate reporting requirements for DOD reports and studies. For example, in his August 9, 2010, speech announcing the overall efficiency initiative, the Secretary of Defense stated that the department is “awash in taskings for reports and studies” and directed several specific actions that, according to the press release accompanying the announcement, were intended to “combat the enormous amounts of taskings for reports and studies.” In his remarks, the Secretary noted that there is little basis to determine whether the value gained is worth the considerable time and resources expended to generate reports and studies. With respect to specific actions, the Secretary directed DOD and its components to track the approximate cost of preparing DOD reports and studies and publish the cost on the front cover of each report or study. He also called for a comprehensive review of all oversight reports and use of the results to reduce the volume of reports and studies generated internally while engaging with Congress on ways to meet their needs while working together to reduce the number of reports.

In September 2010, the Secretary of Defense tasked DOD’s Cost Assessment and Program Evaluation (CAPE) office with implementing the Secretary’s requirement to publish the cost of preparing DOD reports and studies. By November 2010, the CAPE office had developed a cost estimating tool and corresponding guidance on using the tool, which described the categories of costs to be estimated and identified the types of reports and studies for which costs were to be estimated. The following month, the Secretary of Defense issued a memorandum requiring the use of the tool beginning on February 1, 2011.

In House Report Number 112-78, which accompanied a bill for the National Defense Authorization Act for Fiscal Year 2012, the House Committee on Armed Services commended DOD for implementing a process for collecting an estimate of resources required to generate internally and externally required reports and agreed that additional transparency would be useful for decision makers when determining the utility of various reporting requirements. The committee also observed that any tool used to collect costs was only as useful as the inputs received and directed GAO to conduct an assessment of DOD’s methodology and tools for collecting cost data on both internally and externally required reports and to submit a report, including any recommendations needed to improve the data collection, transparency, and utility of the tool. Specifically, we evaluated DOD’s approach to estimate the costs of selected reports and studies, including (1) whether DOD entities are capturing and presenting costs in a manner consistent with relevant cost estimating best practices; (2) the status of DOD’s efforts to track whether organizations are developing cost estimates for all required types of reports and studies; and (3) whether DOD has evaluated the usefulness of its efforts to estimate the costs of reports and studies as one of the means for achieving the Secretary’s intended purpose of increasing the transparency of costs, reducing or eliminating reporting requirements, and instilling a culture of cost consciousness across DOD.