Major Automated Information Systems:

Selected Defense Programs Need to Implement Key Acquisition Practices

GAO-14-309: Published: Mar 27, 2014. Publicly Released: Mar 27, 2014.

Additional Materials:

Contact:

Carol R. Cha
(202) 512-4456
chac@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Of the 15 selected Department of Defense (DOD) major automated information system (MAIS) programs, 13 had cost information available (2 did not, due to revisions to requirements and changes in scope). Of these 13 programs, 11 experienced changes in their cost estimates, including 7 that experienced increases ranging from 4 to 2,233 percent and 4 that experienced decreases ranging from 4 to 86 percent. Two programs remained unchanged in their cost goals. Additionally, of 14 programs that had schedule information available (1 did not due to revisions to requirements), 13 experienced schedule changes—including 12 that had slippages ranging from a few months to 6 years, and 1 that accelerated its schedule. One program remained on schedule. Further, of 11 programs that had system performance data available, 3 programs met their system performance targets, while 8 did not fully meet their targets.

The three programs selected for analysis of risk management demonstrated mixed progress in effectively defining and managing risks. Specifically, the Defense Health Agency's (DHA) Theater Medical Information Program – Joint Increment 2 had implemented key risk management practices. While the Navy's Global Combat Support System – Marine Corps program did not, among other things, update its risk tracking log during a 5-month period in 2013, the program recently updated its risks and mitigation plans, which should help the program to more effectively manage risks going forward. The Defense Logistics Agency's (DLA) Defense Agencies Initiative program had taken steps to implement selected risk management practices, including establishing a risk management board. However, the program was still in the early stages of identifying risks and had not yet identified a comprehensive set of program risks, nor consistently evaluated and categorized its risks. Until this program maintains a complete risk log and mitigation plans, and accurately evaluates and categorizes its risks, it will lack assurance that it is appropriately mitigating all identified risks.

The three programs also demonstrated mixed progress in implementing key requirements management and project monitoring and control best practices. Specifically, the Navy program had implemented all key requirements management best practices and the DLA program had recently taken steps to do so. However, while the DHA program had implemented many requirements management best practices, it had not maintained complete traceability between its requirements and work products. Additionally, the DHA program had not updated its capabilities baseline to reflect program scope changes. Until DHA implements these requirements management best practices, stakeholders will lack assurance that the system will have all intended functionality to meet users' needs. Regarding project monitoring and control, each of the programs lacked key practices. For instance, the DLA program had not tracked significant deviations in performance. Additionally, the Navy program did not always take timely corrective actions to address issues. Further, the DHA program did not use earned value management to track contractor performance in meeting planned cost targets, even though these data were being collected and certain contracts met DOD's threshold for its use; as such, the program had not effectively determined progress against the plan. Until the three programs implement these project monitoring best practices, they will be limited in their ability to manage the programs.

Why GAO Did This Study

The National Defense Authorization Act for Fiscal Year 2012 mandated that GAO select and assess DOD MAIS programs annually through March 2018. This report discusses the results of GAO's second annual assessment. Based on the act's requirements, GAO's objectives were to (1) describe the extent to which selected MAIS programs have changed their planned cost and schedule estimates and met performance targets; (2) assess selected MAIS programs' actions to manage risks; and (3) assess the extent to which selected MAIS programs used key information technology acquisition best practices.

To do so, GAO selected 15 of the 42 DOD MAIS programs based on several factors, including representation from multiple DOD components, and summarized the results of analyses of cost, schedule, and performance across the programs. Further, GAO selected 3 of the 15 programs (1 each from DHA, DLA, and Navy) and assessed them against best practices for risk management, requirements management, and project monitoring and control.

What GAO Recommends

GAO recommends that DOD direct the programs to address respective weaknesses in their risk management, requirements management, and project monitoring and control practices. DOD concurred with six of GAO's recommendations and partially concurred with the remaining two. GAO maintains that it is important that the DHA program trace all capabilities to their associated requirements and update its capabilities baseline to reflect program scope changes.

For more information, contact Carol R. Cha at (202) 512-4456 or chac@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: The Assistant Secretary of Defense (Acquisition) stated that the risks are evaluated and categorized consistent with DLA Instruction 6601, Program Level Risk Management within the Acquisition Life Cycle. However, it is unclear if the Department is maintaining a comprehensive risk log that includes all up-to-date risks with evaluations and categorizations that comply with DLA's defined parameters and associated mitigation plans. We will continue to follow-up with the program about these issues.

    Recommendation: To better ensure that the Defense Agencies Initiative (DAI) implements effective risk management and information technology (IT) acquisition best practices, the Secretary of Defense should direct the Director of the Defense Logistics Agency to direct the DAI program office to establish a comprehensive risk log that includes all up-to-date risks with evaluations and categorizations that comply with DLA's defined parameters; and associated mitigation plans.

    Agency Affected: Department of Defense

  2. Status: Open

    Comments: The Assistant Secretary of Defense (Acquisition) stated that during weekly meetings, the Defense Agencies Initiative (DAI) program reviews the integrated master schedule and cost information to identify and document significant cost and schedule deviations from the DAI program plan. The program has not reported any cost or schedule deviations to date. We will continue to follow-up with the program regarding this recommendation.

    Recommendation: To better ensure that DAI implements effective risk management and IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Logistics Agency to direct the DAI program office to identify and document significant cost and schedule deviations from the program's plan.

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: Marine Corps officials stated that the program has not experienced any critical issues since our report was issued in March 2014, and that GCSS-MC/LCM Increment 1 declared Full Deployment in December 2015 and is now in sustainment. However, we have not received evidence that demonstrates that the program did not experience any critical issues. We will continue to follow-up with GCSS-MC officials.

    Recommendation: To help ensure that the Global Combat Support System-Marine Corps (GCSS-MC) implements effective project monitoring and control best practices, the Secretary of Defense should direct the Secretary of the Navy to direct the GCSS-MC program office to include corrective actions and time frames in future analyses of critical issues and monitor actions taken against those time frames.

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: The Department stated that while there is no benefit in updating the requirements documentation for capabilities that have already been built, tested, and deployed, the TMIP-J program will ensure that all future program capabilities are traced to their associated requirements in the appropriate requirements traceability matrices and the program office will update the program's capabilities baseline to reflect program scope changes. However, the Department has not provided documentation detailing these updates. We will continue to follow-up with the program regarding this recommendation.

    Recommendation: To improve the Theater Medical Information Program - Joint Increment 2 (TMIP-J) program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to direct the TMIP-J program office to update the program's capabilities baseline to reflect program scope changes.

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: The Department stated that while there is no benefit in updating the requirements documentation for capabilities that have already been built, tested, and deployed, the TMIP-J program will ensure that all future program capabilities are traced to their associated requirements in the appropriate requirements traceability matrices. However, the Department has not provided documentation tracing these capabilities. We will continue to follow-up with the program regarding this recommendation.

    Recommendation: To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to trace all capabilities to their associated requirements in the requirements traceability matrix.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: Officials stated that 6 of 7 senior program staff have fully completed required EVM courses and the remaining staff member was on track to complete the required training. We will continue to follow-up with the program regarding this recommendation.

    Recommendation: To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to implement earned value management in accordance with DOD's policy and train management staff with project oversight responsibilities on the proper use of earned value management.

    Agency Affected: Department of Defense

  7. Status: Open

    Comments: The Department stated that the TMIP-J has consistently maintained an authoritative list of all interfaces currently included in the scope of the program. However, they have not provided evidence of this list. The TMIP-J Increment 2, Release 2, Stem View 6 schematic reflects a total of 112 interfaces. Department officials stated that given that the schematic is a living document, the number of interfaces is subject to change. We will continue to follow-up with the program regarding an authoritative listing of the program's interfaces.

    Recommendation: To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to develop an authoritative listing of all interfaces currently included in the scope of the program.

    Agency Affected: Department of Defense

  8. Status: Open

    Comments: Department officials stated that the TMIP system does not provide any automatic capability to track the number of users or locations that employ TMIP capabilities. Therefore, the program manager has implemented a manual process involving monthly data requests to identify the operational platforms and locations. Additionally, Department officials stated that TMIP directly supports military operations and all metrics will change with the dynamic mission environment. We will continue to follow-up with the program regarding this recommendation.

    Recommendation: To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to report to stakeholders the number of current and planned system users and sites and provide updates as needed.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Dec 1, 2016

Nov 30, 2016

Nov 28, 2016

Nov 21, 2016

Nov 17, 2016

Nov 14, 2016

Nov 1, 2016

Oct 31, 2016

Looking for more? Browse all our products here