Major Automated Information Systems:
Selected Defense Programs Need to Implement Key Acquisition Practices
GAO-13-311: Published: Mar 28, 2013. Publicly Released: Mar 28, 2013.
What GAO Found
Of the 14 selected Department of Defense (DOD) major automated information system (MAIS) programs, 9 had stayed within their planned cost estimates, while 5 did not (with cost increases ranging from 3 to 578 percent); 5 programs remained on schedule, while 9 experienced delays (ranging from 6 months to 10 years); and 8 programs met their system performance targets, while 5 did not fully meet their targets, and 1 did not have system performance data available. Looking at these areas collectively, 3 programs stayed within their planned cost and schedule estimates and met their system performance targets, and 2 programs experienced shortcomings in all of the areas--cost, schedule, and performance.
The three selected programs demonstrated mixed results in effectively defining and managing risks of various levels. Specifically, Navy's Consolidated Afloat Networks and Enterprise Services had implemented key practices for risk management, including identifying risks that could negatively affect work efforts. In contrast, the Air Force's Defense Enterprise Accounting and Management System's risk reports were out of date and not regularly updated to include the current status of mitigation actions. To its credit, the program had recently taken steps to improve its risk management process, such as establishing a risk and issues working group. These recent steps should help the program effectively identify and manage program risks going forward. Finally, Global Combat Support System-Army had developed program risks and mitigation plans, but the program was using multiple risk management systems that contained inconsistent data. Until the program establishes a risk management system that includes a comprehensive and up-to-date log of all current threats to the program, it will lack assurance that it is appropriately mitigating all identified risks.
The three selected programs demonstrated mixed progress in implementing key requirements management and project monitoring and control best practices. Specifically, the Navy and Army programs had implemented key requirements management best practices. However, while the Air Force program had also implemented selected practices, it had not traced all of its lower-level requirements to its desired higher-level system capabilities--which is inconsistent with requirements management best practices. Program officials stated that they expect this mapping to be completed by the fourth quarter of fiscal year 2013. Regarding project monitoring and control practices, the Navy program had implemented key best practices, while the Air Force and Army programs lacked certain practices. For example, while the Air Force program regularly communicated with its stakeholders, it had not ensured stable leadership--having four program managers in the past 4 years. DOD commented that it supports tenure agreements, with the first two program managers each completing 3-year terms. While the third and fourth program managers did not complete 3-year tenures, DOD stated that it expects the current program manager to do so. Further, while the Army program also met with stakeholders, it did not effectively use its independent verification and validation (IV&V) function to monitor its program. Until the Army program specifies the roles and responsibilities of the IV&V agent to ensure that it maintains its independence from the risk management processes that it reviews, the program jeopardizes its ability to fully monitor and control the program.
Why GAO Did This Study
In 2011, DOD allotted at least $5.6 billion for designated MAIS programs, which are intended to help the department sustain its key operations. The National Defense Authorization Act for Fiscal Year 2012 mandated that GAO select and assess DOD MAIS programs annually through March 2018. This report discusses the results of GAOs first annual assessment. The act directed GAO to (1) describe the extent to which selected MAIS programs have stayed within planned cost and schedule estimates and met performance targets, (2) assess selected MAIS programs actions to manage risks, and (3) assess the extent to which selected MAIS programs used key information technology acquisition best practices.
To do so, GAO selected 14 of the 48 DOD MAIS programs based on several factors, including size of total life-cycle costs, and summarized the results of analyses of cost, schedule, and performance across the programs. Further, GAO selected 3 of the 14 programs (one Army, one Air Force, and one Navy) and analyzed their risk management actions and assessed them against best practices for requirements management and project monitoring and control.
What GAO Recommends
GAO recommends that DOD direct the Army program to address weaknesses in its risk management and IV&V practices. DOD concurred with these two recommendations and provided additional information that removed the need for a third potential recommendation regarding leadership on the Air Force program.
For more information, contact Valerie C. Melvin at (202) 512-6304 or firstname.lastname@example.org.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: The Army has established a comprehensive risk log that includes an aggregation of all up-to-date risks and associated mitigation plans related to the GCSS-Army program. Specifically, the October 2014 log includes risks and mitigation plans from both the government and the program's prime contractor. As a result, the Army should be better assured that the program office is appropriately mitigating all identified risks.
Recommendation: To better ensure that Global Combat Support System-Army (GCSS-Army) implements effective risk management and project monitoring and control practices, the Secretary of Defense should direct the Secretary of the Army to direct the GCSS-Army program office to establish a comprehensive risk log that maintains an aggregation of all up-to-date risks and associated mitigation plans.
Agency Affected: Department of Defense
Comments: In November 2013, the Army's Program Executive Office for Enterprise Information Systems (PEO EIS) engaged an independent verification and validation (IV&V) agent to act as a third party to ensure that PEO EIS programs, including GCSS-Army, were executed effectively. However, according to Army officials, the IV&V agent is not yet verifying and validating GCSS-Army's risks. Army officials stated that PEO EIS is developing policy and procedures that define the roles and responsibilities of the IV&V agent to enable the agent to verify and validate program risks. We will continue to follow-up with the program and provide an update when PEO EIS has specified the IV&V agent's roles and responsibilities for validating and verifying GCSS-Army's risks.
Recommendation: To better ensure that GCSS-Army implements effective risk management and project monitoring and control practices, the Secretary of Defense should direct the Secretary of the Army to direct the GCSS-Army program office to specify the roles and responsibilities of the IV&V agent to ensure that it acts as a third party that validates and verifies the risks and mitigation plans developed by the program office and system integrator.
Agency Affected: Department of Defense