DOD Business Systems Modernization: Key Marine Corps System Acquisition Needs to Be Better Justified, Defined, and Managed
GAO-08-822
Published: Jul 28, 2008. Publicly Released: Jul 28, 2008.
Skip to Highlights
Highlights
GAO has designated the Department of Defense's (DOD) business systems modernization as a high-risk program because, among other things, it has been challenged in implementing key information technology (IT) management controls on its thousands of business systems. The Global Combat Support System-Marine Corps program is one such system. Initiated in 2003, the program is to modernize the Marine Corps logistics systems. The first increment is to cost about $442 million and be deployed in fiscal year 2010. GAO was asked to determine whether the Department of the Navy is effectively implementing IT management controls on this program. To accomplish this, GAO analyzed the program's implementation of several key IT management disciplines, including economic justification, earned value management, risk management, and system quality measurement.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To ensure that each Global Combat Support System-Marine Corps (GCSS-MC) system increment is economically justified on the basis of a full and reliable understanding of costs, benefits, and risks, the Secretary of Defense should direct the Secretary of the Navy to ensure that investment in the next acquisition phase of the program's first increment is conditional upon fully disclosing to program oversight and approval entities the steps under way or planned to address each of the risks discussed in this report, including the risk of not being architecturally compliant and being duplicative of related programs, not producing expected mission benefits commensurate with reliably estimated costs, not effectively implementing earned value management (EVM), not mitigating known program risks, and not knowing whether the system is becoming more or less mature and stable. We further recommend that investment in all future GCSS-MC increments be limited if the management control weaknesses that are the source of these risks, and which are discussed in this report, have not been fully addressed. |
DOD has addressed the intent of our recommendation. Specifically, the department conducted an Enterprise Risk Assessment Methodology (ERAM)-based review in February 2009, to support the May 2010 Milestone C decision to begin the program's Production and Deployment phase, and reported in March 2011 that all of the risks except one identified in the review are closed. Since then, the department reported that it closed the remaining issue, related to having adequate staff to support multiple software releases. The ERAM process identified seven risk areas, some of which relate to risks discussed in our report, such as not having a well-developed schedule. The assessment included a review of the program's risk management database and policies, and the results were presented at a May 2009 investment review board meeting. In addition, DOD developed metrics related to system maturity and stability, such as metrics to track help desk tickets and change requests, and is generating monthly trend analyses of each. Finally, the department has made progress in addressing weaknesses in its management control practices, including cost estimating.
|
| Department of Defense | To improve the accuracy of the GCSS-MC cost estimate, as well as other cost estimates for the department's enterprise resource planning (ERP) programs, the Secretary of Defense should direct the appropriate organization within DOD to collaborate with relevant organizations to standardize the cost element structure for the department's ERP programs and to use this standard structure to maintain cost data for its ERP programs, including GCSS-MC, and to use this cost data in developing future cost estimates. |
DOD has largely implemented the recommendation, as intended. Specifically, the department established in October 2011 a standard practice for creating work breakdown structures, which includes common elements for ERP programs such as GCSS-MC, and reported that it plans to continue to establish common practices for ERP programs in the area of cost. Further, DOD reported that it expects the benefits of increased accuracy in cost estimating to be realized in the future when the standard WBS practice is implemented cross the department and additional data is collected in the cost elements.
|
| Department of Defense | To improve the credibility of the GCSS-MC cost estimate, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the program's current economic analysis is adjusted to reflect the risks associated with it not reflecting cost data for comparable ERP programs, and otherwise not having been derived according to other key cost estimating practices, and that future updates to the GCSS-MC economic analysis similarly do so. |
The department took steps to improve the credibility of its cost estimate in accordance with our recommendation. For example, while it has yet to use the cost estimate for its economic analysis, the department has applied risk and uncertainty to hardware refresh and software license costs based on the variability in prices experienced in other programs. According to officials, an updated economic analysis is planned to be completed for the program's Full Deployment Decision in December 2012.
|
| Department of Defense | To enhance GCSS-MC's use of EVM, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the program office (1) monitors the actual start and completion dates of work activities performed so that the impact of deviations on downstream scheduled work can be proactively addressed; (2) allocates resources, such as labor hours and material, to all key activities on the schedule; (3) integrates key activities and supporting tasks and subtasks; (4) identifies and allocates the amount of float time needed for key activities to account for potential problems that might occur along or near the schedule's critical path; (5) performs a schedule risk analysis to determine the level of confidence in meeting the program's activities and completion date; (6) allocates schedule reserve for high-risk activities on the critical path; and (7) discloses the inherent risks and limitations associated with any future use of the program's EVM reports until the schedule has been risk-adjusted. |
As of September 2012, the Department of Defense has made progress addressing our recommendation, but work remains. For example, the GCSS-MC program office reported that it has begun to monitor and report planned versus actual start and complete dates of work activities, and assigns resources to the schedule to project the level of effort needed to accomplish detailed program activities. In addition, the department has disclosed the risk associated with its schedule in using EVM reports, in accordance with our recommendation. However, key activities in program sub-schedules have yet to be integrated, which has led to schedule anomalies and inconsistency between the sub-schedules, and an inability to identify a critical path. Further, while the schedule includes risk-related activities, the program has yet to conduct a quantitative schedule risk analysis or allocate schedule risk reserve for high-risk activities. By not performing these practices the schedule does not provide a reliable basis for performing EVM.
|
| Department of Defense | To improve GCSS-MC management of program risks, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the program office (1) adds each of the risks discussed in this report to its active inventory of risks, (2) tracks and evaluates the implementation of mitigation plans for all risks, (3) discloses to appropriate program oversight and approval authorities whether mitigation plans have been fully executed and have produced the intended outcome(s), and (4) only closes a risk if its mitigation plan has been fully executed and produced the intended outcome(s). |
DOD has taken a number of actions to strengthen risk management, in addition to revising its Risk Management Plan in March 2009 to reflect new processes and policies. First, DOD reported that the program office inserted all risks reported by GAO, as well as risks identified through DOD's Enterprise Risk Assessment Methodology, into its risk database. Second, the Risk Board, chaired by the program manager and including senior program leadership, reviews on a monthly basis, new program risks, the status of existing high risks, and management of risk mitigation activities. Third, the program meets monthly with the Program Executive Office for Enterprise Information Systems, and quarterly with the Assistant Secretary of the Navy (Research Development and Acquisition), to discuss program risks. Also, the program reports risks to other program oversight bodies, such as the Weapons Systems Lifecycle Management and Materiel Supply and Services Management Investment Review Board, and the Defense Business Systems Management Committee. Finally, risks are not closed without risk board approval.
|
| Department of Defense | To strengthen GCSS-MC system quality measurement, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the program office (1) collects the data needed to develop trends in unresolved system defects and change requests according to their priority and severity and (2) discloses these trends to appropriate program oversight and approval authorities. |
This recommendation has largely been implemented as intended. The program office is now collecting data according to their priority and severity and developing trends over time in open and closed help desk tickets and change requests. These trends are reported to the III Marine Expeditionary Force; Director, Operational Test and Evaluation; Program Executive Officer--Enterprise Information Systems and the Marine Corps Installation and Logistics Director.
|
Full Report
Public Inquiries
Topics
Cost analysisCost overrunsDefense capabilitiesDefense cost controlDefense procurementEarned value management systemsInformation technologyInternal controlsIT acquisitionsLogisticsProgram evaluationProgram managementRisk managementSchedule slippagesSystems conversionsSystems designBenefit-cost trackingBusiness transformationCost estimates