Defense Acquisitions: Better Approach Needed to Account for Number, Cost, and Performance of Non-Major Programs
Highlights
What GAO Found
The Department of Defense (DOD) could not provide sufficiently reliable data for GAO to determine the number, total cost, or performance of DOD's current acquisition category (ACAT) II and III programs. These non-major programs range from a multibillion dollar aircraft radar modernization program to soldier clothing and protective equipment programs in the tens of millions of dollars. GAO found that the accuracy, completeness, and consistency of DOD's data on these programs were undermined by widespread data entry issues, missing data, and inconsistent identification of current ACAT II and III programs. See the figure below for selected data reliability issues GAO identified.
Two Most Frequent Reliability Issues Identified by GAO in DOD-Reported Data for Acquisition Category (ACAT) II and III Programs
DOD components are taking steps to improve ACAT II and III data, but these steps do not fully address the problems GAO identified. For example, the components have not established systematic processes to perform data quality tests and assess the results to help identify problems for further review. These types of tests and assessments can be an important step in determining whether data can be used for its intended purposes. Additionally, DOD lacks metrics to assess ACAT II and III cost and schedule performance trends across programs and in some cases was missing baseline cost and schedule data to measure performance. Having timely and reliable cost, schedule, and performance data on smaller acquisition programs is critical to ensuring that DOD and its components can account for how they are spending their money and how well they are spending it. Reliable data are also essential for effective oversight and bringing the right oversight resources to bear when programs approach the cost threshold to become a major defense acquisition program due to cost growth.
Thirteen of the 15 ACAT II or III programs GAO reviewed in-depth had exceeded their original cost or schedule targets. Program officials from ACAT II and III programs GAO reviewed cited changing performance requirements, testing issues, quantity changes, and flaws in original cost estimates, among other factors, as the reasons for cost and schedule growth. GAO has previously found that similar factors affect the performance of major acquisition programs.
Why GAO Did This Study
DOD requested $168 billion in fiscal year 2014 to develop, test, and acquire weapon systems and other products and equipment. About 40 percent of that total is for major defense acquisition programs or ACAT I programs. DOD also invests in other, non-major ACAT II and III programs that are generally less costly at the individual program level. These programs typically have fewer reporting requirements and are overseen at lower organizational levels than ACAT I programs, although they may have annual funding needs that are just as significant.
GAO was asked to examine ACAT II and III programs. This report addresses, among other issues, (1) the extent to which information is available on the number, cost, and performance of ACAT II and III programs and (2) factors that affected the performance of selected ACAT II and III programs. GAO collected program and cost data on current ACAT II and III programs from five DOD components. GAO also selected a non-generalizable sample of 15 programs based on program cost and other criteria and reviewed documentation and interviewed officials about program performance.
Recommendations
GAO recommends that DOD establish guidelines on what constitutes a current ACAT II and III program, take steps to improve data reliability, and determine how to measure cost and schedule performance. DOD partially concurred with the recommendations and described actions it plans to take. However, as discussed in the report, DOD's planned actions may not fully address the issues that GAO identified.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To improve DOD's ability to collect and maintain reliable data on its acquisitions, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology, and Logistics, in consultation with DOD components, to establish guidelines on what constitutes a "current" ACAT II or III program for reporting purposes; the types of programs, if any, that do not require ACAT designations; and whether the rules for identifying current MDAPs would be appropriate for ACAT II and III programs. |
DOD partially concurred with our recommendation and stated that it planned to review existing policy to see if revisions were needed. As of August 2023, DOD has yet to establish department-wide guidelines as we recommended. Starting in September 2018, DOD began providing the military departments with a capability to identify ACAT II and III programs using the Defense Acquisition Visibility Environment (DAVE) system. The DAVE system is now considered to be a trusted source for ACAT II and III program data. DOD, in consultation with the military departments, established standard data elements for collection across ACAT II and III programs for inclusion in DAVE. However, the military departments determine individually what constitutes a "current" program and the types of programs that do not require ACAT designations. In August 2023, an official from the Office of the Under Secretary of Defense for Acquisition and Sustainment told us that a memo addressing military department reporting on ACAT II and III programs, among other topics, was awaiting final signature. We will review this memo when we receive it to determine if it addresses our recommendation.
|
| Department of Defense | To improve DOD's ability to collect and maintain reliable data on its acquisitions, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology, and Logistics, in consultation with DOD components, to determine what metrics should be used and what data should be collected on ACAT II and III programs to measure cost and schedule performance; and whether the use of DAMIR and the MDAP selected acquisition report format may be appropriate for collecting data on ACAT II and III programs. |
DOD partially concurred with our recommendation and in September 2018, April 2019, and September 2023, took actions to implement it. Specifically, in September 2018, DOD provided the military departments with the capability to identify ACAT II and III programs in DAVE/DAMIR and made the DAVE/DAMIR Acquisition Program Baseline (APB) workflow tool for cost and schedule data collection available for components' use in April 2019. In September 2023, the Under Secretary of Defense for Acquisition and Sustainment (USD(A&S)) signed a memo requiring military services and components to provide program data listed in the Acquisition Visibility Data Framework to USD(A&S) for all acquisition categories (ACAT), to include ACAT II and III programs. The Acquisition Visibility Data Framework documents common standards for core program data needed for effective management and oversight of defense acquisitions. If implemented consistently, this memo should improve the collection and maintenance of reliable data on ACAT II and III programs at DOD.
|
| Department of Defense | To improve DOD's ability to collect and maintain reliable data on its acquisitions, the Secretary of Defense should direct the Secretaries of the Air Force, Army, and Navy and the Commander of SOCOM to assess the reliability of data collected on ACAT II and III programs and work with PEOs to develop a strategy to improve procedures for the entry and maintenance of data. |
In October 2020, the Office of the Under Secretary of Defense for Acquisition and Sustainment issued guidance promulgating a department-wide framework for acquisition and sustainment data, referred to as the Acquisition Visibility Data Framework (AVDF). The AVDF, which applies to all acquisition categories in the major capability acquisition pathway, provides common business rules across DOD for collecting acquisition and sustainment data. The AVDF was developed in coordination with the military departments to promote a common understanding and the efficient management of acquisition data as an asset that can be shared across the Department. This effort followed a September 2015 tasking that directed the military departments and DOD components to assess the reliability of ACAT II and III data. If implemented consistently, the AVDF should provide a valuable tool to improve the reliability of information on acquisition category II and III programs across DOD.
|
| Department of Defense | To improve DOD's ability to collect and maintain reliable data on its acquisitions, the Secretary of Defense should direct the Secretaries of the Air Force, Army, and Navy and the Commander of SOCOM to develop implementation plans to coordinate and execute component initiatives to improve data on ACAT II and III programs. |
In October 2020, the Office of the Under Secretary of Defense for Acquisition and Sustainment issued guidance promulgating a department-wide framework for acquisition and sustainment data, referred to as the Acquisition Visibility Data Framework (AVDF). The AVDF applies to all acquisition categories in the major capability acquisition pathway. The October 2020 guidance established an expectation for the military departments to modernize and adapt their acquisition databases to align with the data framework. These efforts are coordinated across the military departments through two working groups - the Acquisition Visibility Steering Group and the Acquisition Visibility Working Group, which were established to provide a mechanism for collaborating and coordinating across the department on policies, guidance, standards, and capabilities related to provision and use of acquisition data.
|
| Department of Defense | To help ensure compliance with relevant provisions of DOD acquisition policy with the purpose of improving DOD's ability to provide oversight for ACAT II and III programs, including those programs that may become MDAPs, the Secretary of Defense should direct the Secretary of the Air Force and Commander of SOCOM to establish a mechanism to ensure compliance with APB requirements in DOD policy. |
DOD has taken some steps to improve compliance with APB requirements, but the Secretary of Defense has not directed the Secretary of the Air Force and Commander of SOCOM to establish a mechanism to ensure compliance with APB requirements in DOD policy as we recommended. DOD partially concurred with our recommendation and in September 2015, the Office of the Assistant Secretary of Defense for Acquisition requested that DOD components review their mechanisms for establishing and enforcing the APB requirements for all ACAT II and III programs. In July 2018, the Office of the Under Secretary of Defense for Acquisition & Sustainment confirmed that, based on the results of these reviews, it did not plan to take any action to implement this recommendation. However, in 2019, DOD made its DAVE/DAMIR APB workflow tool available for military department use, and the Air Force elected to use the tool to create and track APBs for ACAT II and III programs. Additionally, in 2020, the Air Force required its Program Executive Officers to submit memorandums to the Office of the Assistant Secretary of the Air Force for Acquisition, Technology & Logistics validating that all applicable ACAT programs within their purview had an approved APB.
|
| Department of Defense | To help ensure compliance with relevant provisions of DOD acquisition policy with the purpose of improving DOD's ability to provide oversight for ACAT II and III programs, including those programs that may become MDAPs, the Secretary of Defense should direct the Secretaries of the Air Force, Army, and Navy to improve component procedures for notifying the Defense Acquisition Executive of programs with a cost estimate within 10 percent of ACAT I cost thresholds. |
The Air Force and Navy have taken some steps to improve component procedures for notifying the Defense Acquisition Executive of programs with a cost estimate within 10 percent of ACAT I cost thresholds, but DOD has not implemented our recommendation to direct the Secretaries of the Air Force, Army, and Navy to improve component notification procedures. DOD partially concurred with our recommendation, but. the Office of the Under Secretary of Defense for Acquisition & Sustainment confirmed in July 2018 and June 2021 that it does not plan to take additional actions to implement this recommendation. However, in 2019, the Air Force updated its procedures for its semi-annual review of Air Force acquisition data to improve its ability to identify programs approaching the ACAT I cost threshold. Additionally, as part of an in-depth review that the Navy concluded in April 2020, the Navy identified all programs in its database that were within 10 percent of the ACAT I cost threshold and required Navy Program Executive Officers to identify actions that had or would be taken to address related notification requirements. The Navy also developed an automated warning report in its acquisition reporting system to flag programs approaching or exceeding the cost threshold for the next ACAT level.
|