DOD Business Transformation:

Improved Management Oversight of Business System Modernization Efforts Needed

GAO-11-53: Published: Oct 7, 2010. Publicly Released: Oct 7, 2010.

Additional Materials:

Contact:

Asif A. Khan
(202) 512-9869
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Defense (DOD) invests billions of dollars annually to modernize its business systems, which have been on GAO's high-risk list since 1995. DOD is in the process of implementing nine enterprise resource planning (ERP) efforts which perform business-related tasks such as general ledger accounting and supply chain management. These efforts are essential to transforming DOD's business operations. GAO was asked to (1) provide the status of the ERPs as of December 31, 2009; (2) determine whether selected ERPs followed schedule and cost best practices; and (3) determine if DOD has defined the performance measures to assess whether the ERPs will meet their intended business capabilities. To accomplish these objectives, GAO reviewed data on the status of each ERP from the program management officers and interviewed the DOD and military departments' chief management officers.

Based upon the data provided by DOD, six of the nine ERPs have experienced schedule delays ranging from 2 to 12 years and five have incurred cost increases ranging from $530 million to $2.4 billion. DOD has stated that the ERPs will replace over 500 legacy systems that cost hundreds of millions of dollars to operate annually. However, delays in implementing the ERPs require DOD to fund the legacy systems longer than anticipated, thereby reducing the funds available for other DOD priorities. In 2007, 2008, and 2009, GAO made 19 recommendations to improve the management of DOD's ERP efforts. While DOD agreed with the recommendations, 14 have not yet been fully implemented. GAO analyzed four of the nine ERPs to determine whether scheduling and cost estimating best practices were being followed. In addition to reiterating its existing recommendations, GAO is making eight recommendations to the Secretary of Defense aimed at improving schedule and cost practices and the development of performance measures to evaluate whether the ERPs' intended goals are being accomplished. DOD concurred with our recommendations and plans to take action to implement them.Regarding scheduling practices, GAO found that none of the programs had developed a fully integrated master schedule as an effective tool to help in the management of the programs. A reliable schedule is crucial to estimating the overall schedule and cost of a program. Without a reliable schedule, DOD is unable to predict, with any degree of confidence, if the estimated completion dates are realistic. Regarding the cost estimates, GAO found that although the four ERPs' cost estimates generally met the criteria for three of the four best practices--well-documented, accurate, and comprehensive--three ERPs did not fully meet the credibility criteria because potential limitations were not discussed. More specifically, the three ERPs lacked a sensitivity analysis or a risk and uncertainty analysis as stipulated in GAO, Office of Management and Budget, and DOD guidance, thus diminishing the credibility of the estimates. While the ERPs are critical to transforming DOD's business operations, DOD lacks a comprehensive set of performance measures to assess these systems and their contribution to transforming business operations. Management needs to define what constitutes a successful implementation in terms that can be used to assess whether the system is (1) being used as expected and (2) providing the intended benefits. Accordingly, the actual measures used to accomplish these objectives will differ depending on the system. For example, measures for a logistical system may focus on reducing inventory levels, while those for a financial system may focus on reducing prompt payment penalties. Without performance measures to evaluate how well the ERPs are accomplishing their intended goals, DOD decision makers do not have all the information they need to determine whether DOD investments are accomplishing their desired goals, and program managers do not have the information they need to ensure that their individual program is helping DOD to achieve business transformation and thereby improve upon its primary mission of supporting the warfighter. In addition to reiterating its existing recommendations, GAO is making eight recommendations to the Secretary of Defense aimed at improving schedule and cost practices and the development of performance measures to evaluate whether the ERPs' intended goals are being accomplished. DOD concurred with our recommendations and plans to take action to implement them.

Recommendations for Executive Action

  1. Status: Open

    Comments: In commenting on the draft report, DOD concurred with the recommendation and cited that the appropriate military department Chief Management Officer will direct Program Managers to implement this recommendation and have oversight over implementation of GFEBS. We will follow-up with Army officials to (1) obtain and evaluate documentation and (2) determine whether the Army has taken sufficient action to address this recommendation. There has been no updated information provided in the past fiscal year.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the Secretary of the Army to ensure that the Chief Management Officer of the Army directs the program management offices (PMO) for the General Fund Enterprise Business System (GFEBS) to develop an integrated master schedule (IMS) that fully incorporates best practices. The schedule should 1) sequence all activities, 2) assign resources to all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, 5) identify float between activities, 6) conduct a schedule risk analysis, and 7) update schedule using logic and durations to determine dates.

    Agency Affected: Department of Defense

  2. Status: Open

    Comments: In commenting on the draft report, DOD concurred with the recommendation and cited that the appropriate military department Chief Management Officer will direct Program Managers to implement this recommendation and have oversight over implementation. To support Congress's continuing oversight of DOD?s progress in implementing its ERP systems, we were asked to review the current schedule for selected DOD ERP systems. In accordance with this request, our review determined the extent to which the schedule for GCSS-Army were prepared consistent with GAO's Schedule and Cost Guides. We reviewed the most current GCSS-Army schedule and cost estimates available at the time of our review, which supported DOD's December 2012 full deployment decision While the Army made some improvements to the schedule that supported the full deployment decision, the Army did not fully meet best practices in developing a reliable schedule for the GCSS-Army program. The Army made progress in incorporating schedule best practices, such as The Army?s December 2012 GCSS-Army schedule used to support the full deployment decision addressed several of the best practices that were an issue in our prior report, including capturing all activities, sequencing all activities, and integrating activities horizontally and vertically. However, we continued to identify several best practices that were not yet fully addressed and also identified several new areas where the 2012 schedule did not incorporate best practices, such as activity durations and baseline schedule. Although GCSS-Army is in full deployment, without fully addressing best practices for scheduling, program managers will not have the best information available to make decisions related to issues such as the sequencing of activities and the flexibility of the schedule according to available resources. It is critical to correct the deficiencies identified with the schedule and cost estimates to help ensure that the projected spending for this program is being used in the most efficient and effective manner. By incorporating best practices for developing reliable schedule and cost estimates, DOD would increase the probability of GCSS-Army successfully achieving full deployment by the fourth quarter of fiscal year 2017 to provide needed functionality for financial improvement and audit readiness.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should directs the Secretary of the Army to ensure that the Chief Management Officer of the Army direct the PMO for GCSS-Army to develop an IMS that fully incorporates best practices. The schedule should 1) capture all activities, 2) sequence all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, and 5) conduct a schedule risk analysis.

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: In commenting on the draft report, DOD concurred with the recommendation and cited that the appropriate military department Chief Management Officer will direct Program Managers to implement this recommendation and have oversight over implementation of DEAMS. We will update the status on this recommendation after completion of our current follow-up audit on DEAMS. The current audit on DEAMS is ongoing.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should Direct the Secretary of the Air Force to ensure that the Chief Management Officer of the Air Force directs the PMO for Defense Enterprise Accounting and Management System (DEAMS) to develop an IMS that fully incorporates best practices. The schedule should 1) capture all activities, 2) sequence all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, 5) identify float between activities, 6) conduct a schedule risk analysis, and 7) update schedule using logic and durations to determine dates.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: In 2011, after failing to achieve a full deployment decision within 5 years after funds were first obligated for ECSS, DOD initiated a program assessment to improve the performance of the program. After this assessment, DOD implemented actions to improve the ECSS program. These actions included establishing performance-based success criteria to measure program progress and addressing concerns that relate directly to our recommendation. While these actions occurred too late to provide significant improvement in program performance, however, they did provide DOD officials critical information to evaluate the overall ECSS program. As a result, in December 2012, DOD terminated development and implementation of the ECSS program. DOD officials said they cancelled ECSS because (1) it was going to require an additional $1 billion in excess of the $1 billion cost already expended, (2) it was not going to provide significant military capability; and (3) the deployment date would be delayed until fiscal year 2020. Also, consistent with our recommendation to develop an integrated master schedule that fully incorporates best practices, ECSS program officials stated that the lack of a master schedule contributed to the ECSS termination. As a result, the intent of this recommendation was addressed.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should Direct the Secretary of the Air Force to ensure that the Chief Management Officer of the Air Force directs the PMO for Expeditionary Combat Support System (ECSS) to develop an IMS that fully incorporates best practices. The schedule should 1) sequence all activities, 2) assign resources to all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, 5) identify float between activities, 6) conduct a schedule risk analysis, and 7) update schedule using logic and durations to determine dates.

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: In commenting on the draft report, DOD concurred with the recommendation and cited that the appropriate military department Chief Management Officer will direct Program Managers to implement this recommendation and have oversight over implementation. We will follow-up with Army officials to (1) obtain and evaluate documentation and (2) determine whether the Army has taken sufficient action to address this recommendation. There has been no updated information provided on GFEBS in the past fiscal year.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the Secretary of the Army to ensure that the Chief Management Officer of the Army directs the PMO for GFEBS to update the cost estimates by preparing sensitivity and risk and uncertainty analyses using best practices.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: In commenting on the draft report, DOD concurred with the recommendation and cited that the appropriate military department Chief Management Officer will direct Program Managers to implement this recommendation and have oversight over implementation. We will update the status on this recommendation after completion of our current follow-up audit on GCSS-Army. The current audit on GCSS-Army is ongoing.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the Secretary of the Army to ensure that the Chief Management Officer of the Army directs the PMO for GCSS-Army to update the cost estimates by using actual cost and preparing a sensitivity analysis using best practices.

    Agency Affected: Department of Defense

  7. Status: Closed - Implemented

    Comments: In 2011, after failing to achieve a full deployment decision within 5 years after funds were first obligated for ECSS, DOD initiated a program assessment to improve the performance of the program. After this assessment, DOD implemented actions to improve the ECSS program. These actions included establishing performance-based success criteria to measure program progress and addressing concerns that relate directly to our recommendation. While these actions occurred too late to provide significant improvement in program performance, however, they did provide DOD officials critical information to evaluate the overall ECSS program. As a result, in December 2012, DOD terminated development and implementation of the ECSS program. DOD officials said they cancelled ECSS because (1) it was going to require an additional $1 billion in excess of the $1 billion cost already expended, (2) it was not going to provide significant military capability; and (3) the deployment date would be delayed until fiscal year 2020. Also, consistent with our recommendation to develop an integrated master schedule that fully incorporates best practices, ECSS program officials stated that the lack of a master schedule contributed to the ECSS termination. As a result, the intent of this recommendation was addressed.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should Direct the Secretary of the Air Force to ensure that the Chief Management Officer of the Air Force directs the PMO for ECSS to update the cost estimates by preparing a sensitivity analysis using best practices.

    Agency Affected: Department of Defense

  8. Status: Open

    Comments: In commenting on the draft report, DOD concurred with the recommendation and cited that the Department of Defense (DOD) will issue guidance requiring its business systems to include performance measures which can be used to assess expected business benefits. Also, the Department added that these measures will also be incorporated into the Business Enterprise Architecture. We will follow-up with DOD officials to (1) obtain and evaluate documentation and (2) determine whether the department has taken sufficient action to address this recommendation. There has been no updated information provided in the past fiscal year.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the department's Chief Management Officer and the chief management officers of the military departments to establish performance measures based on quantitative data that will enable the department to assess whether each respective military service's ERP efforts are providing the intended business capabilities to the system users.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Sep 3, 2015

Aug 27, 2015

Aug 26, 2015

Aug 12, 2015

Aug 11, 2015

Jul 31, 2015

Jul 30, 2015

Jul 27, 2015

Jul 23, 2015

Jul 21, 2015

Looking for more? Browse all our products here