DOD Business Transformation:

Improved Management Oversight of Business System Modernization Efforts Needed

GAO-11-53: Published: Oct 7, 2010. Publicly Released: Oct 7, 2010.

Additional Materials:

Contact:

Asif A. Khan
(202) 512-9869
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Defense (DOD) invests billions of dollars annually to modernize its business systems, which have been on GAO's high-risk list since 1995. DOD is in the process of implementing nine enterprise resource planning (ERP) efforts which perform business-related tasks such as general ledger accounting and supply chain management. These efforts are essential to transforming DOD's business operations. GAO was asked to (1) provide the status of the ERPs as of December 31, 2009; (2) determine whether selected ERPs followed schedule and cost best practices; and (3) determine if DOD has defined the performance measures to assess whether the ERPs will meet their intended business capabilities. To accomplish these objectives, GAO reviewed data on the status of each ERP from the program management officers and interviewed the DOD and military departments' chief management officers.

Based upon the data provided by DOD, six of the nine ERPs have experienced schedule delays ranging from 2 to 12 years and five have incurred cost increases ranging from $530 million to $2.4 billion. DOD has stated that the ERPs will replace over 500 legacy systems that cost hundreds of millions of dollars to operate annually. However, delays in implementing the ERPs require DOD to fund the legacy systems longer than anticipated, thereby reducing the funds available for other DOD priorities. In 2007, 2008, and 2009, GAO made 19 recommendations to improve the management of DOD's ERP efforts. While DOD agreed with the recommendations, 14 have not yet been fully implemented. GAO analyzed four of the nine ERPs to determine whether scheduling and cost estimating best practices were being followed. In addition to reiterating its existing recommendations, GAO is making eight recommendations to the Secretary of Defense aimed at improving schedule and cost practices and the development of performance measures to evaluate whether the ERPs' intended goals are being accomplished. DOD concurred with our recommendations and plans to take action to implement them.Regarding scheduling practices, GAO found that none of the programs had developed a fully integrated master schedule as an effective tool to help in the management of the programs. A reliable schedule is crucial to estimating the overall schedule and cost of a program. Without a reliable schedule, DOD is unable to predict, with any degree of confidence, if the estimated completion dates are realistic. Regarding the cost estimates, GAO found that although the four ERPs' cost estimates generally met the criteria for three of the four best practices--well-documented, accurate, and comprehensive--three ERPs did not fully meet the credibility criteria because potential limitations were not discussed. More specifically, the three ERPs lacked a sensitivity analysis or a risk and uncertainty analysis as stipulated in GAO, Office of Management and Budget, and DOD guidance, thus diminishing the credibility of the estimates. While the ERPs are critical to transforming DOD's business operations, DOD lacks a comprehensive set of performance measures to assess these systems and their contribution to transforming business operations. Management needs to define what constitutes a successful implementation in terms that can be used to assess whether the system is (1) being used as expected and (2) providing the intended benefits. Accordingly, the actual measures used to accomplish these objectives will differ depending on the system. For example, measures for a logistical system may focus on reducing inventory levels, while those for a financial system may focus on reducing prompt payment penalties. Without performance measures to evaluate how well the ERPs are accomplishing their intended goals, DOD decision makers do not have all the information they need to determine whether DOD investments are accomplishing their desired goals, and program managers do not have the information they need to ensure that their individual program is helping DOD to achieve business transformation and thereby improve upon its primary mission of supporting the warfighter. In addition to reiterating its existing recommendations, GAO is making eight recommendations to the Secretary of Defense aimed at improving schedule and cost practices and the development of performance measures to evaluate whether the ERPs' intended goals are being accomplished. DOD concurred with our recommendations and plans to take action to implement them.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: In our October 2010 report, we recommended that the Army develop a reliable integrated master schedule for the General Fund Enterprise Business System (GFEBS) program that fully incorporated best practices, such as sequencing all activities, assigning resources to all activities, integrating schedule activities horizontally and vertically, establishing the critical path for all activities, identifying float between activities, conducting a schedule risk analysis, and updating schedule using logic and durations to determine dates. GFEBS is the Army's General Fund general ledger and became the Army's system of record at the beginning of Fiscal Year 2013. On July 1, 2012, the Army completed GFEBS deployment to more than 53,000 users at 227 locations. In June 2015, we obtained and reviewed the April 2015 GFEBS integrated master schedule and evaluated the schedule using the criteria described in GAO's Schedule Assessment Guide. We determined the extent to which the schedule was prepared in accordance with the above best practices. The Army's April 2015 GFEBS schedule addressed one best practice, sequencing all activities. However, we continued to identify that the remaining best practices were not yet fully addressed. Therefore, the Army did not fully meet best practices in developing a reliable schedule for the GFEBS program. The GFEBS IMS has an end date of December 2015 and is no longer relevant and we do not reasonably expect DOD to take any further actions to address this recommendations. Therefore we are closing this recommendation not implemented.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the Secretary of the Army to ensure that the Chief Management Officer of the Army directs the program management offices (PMO) for the General Fund Enterprise Business System (GFEBS) to develop an integrated master schedule (IMS) that fully incorporates best practices. The schedule should 1) sequence all activities, 2) assign resources to all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, 5) identify float between activities, 6) conduct a schedule risk analysis, and 7) update schedule using logic and durations to determine dates.

    Agency Affected: Department of Defense

  2. Status: Open

    Comments: In our October 2010 report, we recommended that the Army develop a reliable integrated master schedule for the Global Combat Support System-Army (GCSS-Army) program that fully incorporated best practices, such as capturing all activities, sequencing all activities, integrating schedule activities horizontally and vertically, establishing the critical path for all activities, and conducting a schedule risk analysis. In our September 2014 report, we found that although the Army made some improvements to the schedule that supported the December 2012 full deployment decision, the Army did not fully meet best practices in developing a reliable schedule for the GCSS-Army program. The GCSS-Army's December 2012 schedule addressed several of the best practices that were an issue in our prior report, including capturing all activities, sequencing all activities, and integrating activities horizontally and vertically. However, we continued to identify several best practices that were not yet fully addressed and also identified several new areas where the 2012 schedule did not incorporate best practices, such as activity durations and baseline schedule. Although GCSS-Army is in full deployment, without fully addressing best practices for scheduling, program managers will not have the best information available to make management decisions. It is critical to correct the deficiencies identified with the schedule to help ensure that the projected spending for this program is being used in the most efficient and effective manner. By incorporating best practices for developing a reliable schedule, DOD would increase the probability of GCSS-Army successfully achieving full deployment by the fourth quarter of fiscal year 2017 to provide needed functionality for financial improvement and audit readiness. We are following up with Army officials to obtain documentation for the GCSS-Army IMS and cost estimate to determine if DOD has taken corrective actions to address this recommendation.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should directs the Secretary of the Army to ensure that the Chief Management Officer of the Army direct the PMO for GCSS-Army to develop an IMS that fully incorporates best practices. The schedule should 1) capture all activities, 2) sequence all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, and 5) conduct a schedule risk analysis.

    Agency Affected: Department of Defense

  3. Status: Closed - Not Implemented

    Comments: In our October 2010 report, we recommended that the Air Force develop a reliable integrated master schedule for the Defense Enterprise Accounting and Management System (DEAMS) program that fully incorporated best practices, such as capturing all activities, sequencing all activities, integrating schedule activities horizontally and vertically, establishing the critical path for all activities, identifying float between activities, conducting a schedule risk analysis, and updating schedule using logic and durations to determine dates. In our February 2014 report, we found that the Air Force's schedule that supported the February 2012 Milestone B decision for DEAMS did not fully meet best practices in developing a reliable schedule for the DEAMS program. GAO found that the schedule supporting the Air Force's decision to invest in DEAMS, partially or minimally met the four characteristics for developing a high-quality and reliable schedule. For example, the schedule did not reflect all government and contractor activities, and resources were not assigned to specific activities. The schedule also lacked a valid critical path, preventing management from focusing on the activities most likely to cause critical program delays if they are not completed as planned. In addition, a schedule risk analysis was not conducted to predict a level of confidence in meeting the program's completion date. The results of our analyses of the DEAMS schedule reflected similar weaknesses to those we reported in October 2010. We have reviewed the DEAMS integrated master schedule (IMS) three times since 2012 and the most recent October 2015 IMS. There has not been any significant improvement in the quality of the schedules over this timeframe. Based on the limited amount of improvement on the DEAMS IMS since 2012, we do not reasonably expect DOD to take more actions to address the recommendation. Therefore we are closing this recommendation not implemented.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should Direct the Secretary of the Air Force to ensure that the Chief Management Officer of the Air Force directs the PMO for Defense Enterprise Accounting and Management System (DEAMS) to develop an IMS that fully incorporates best practices. The schedule should 1) capture all activities, 2) sequence all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, 5) identify float between activities, 6) conduct a schedule risk analysis, and 7) update schedule using logic and durations to determine dates.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: In 2011, after failing to achieve a full deployment decision within 5 years after funds were first obligated for ECSS, DOD initiated a program assessment to improve the performance of the program. After this assessment, DOD implemented actions to improve the ECSS program. These actions included establishing performance-based success criteria to measure program progress and addressing concerns that relate directly to our recommendation. While these actions occurred too late to provide significant improvement in program performance, however, they did provide DOD officials critical information to evaluate the overall ECSS program. As a result, in December 2012, DOD terminated development and implementation of the ECSS program. DOD officials said they cancelled ECSS because (1) it was going to require an additional $1 billion in excess of the $1 billion cost already expended, (2) it was not going to provide significant military capability, and (3) the deployment date would be delayed until fiscal year 2020. Also, consistent with our recommendation to develop an integrated master schedule that fully incorporates best practices, ECSS program officials stated that the lack of a master schedule contributed to the ECSS termination. As a result, the intent of this recommendation was addressed.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should Direct the Secretary of the Air Force to ensure that the Chief Management Officer of the Air Force directs the PMO for Expeditionary Combat Support System (ECSS) to develop an IMS that fully incorporates best practices. The schedule should 1) sequence all activities, 2) assign resources to all activities, 3) integrate schedule activities horizontally and vertically, 4) establish the critical path for all activities, 5) identify float between activities, 6) conduct a schedule risk analysis, and 7) update schedule using logic and durations to determine dates.

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: In our October 2010 report, we recommended that the Army update the cost estimate for the General Fund Enterprise Business System (GFEBS) by preparing sensitivity and risk and uncertainty analyses using best practices. In April 2015, Army Program Management Office (PMO) officials stated that the GFEBS cost estimate for sustainment has not been officially updated subsequent to the GAO October 2010 report. GFEBS has consistently managed to the Full Deployment Decision Review Army Cost Position approved in 2009. GFEBS is currently in the process of updating the cost estimate for sustainment. Sustainment costs have been estimated in a Project Office Estimate (POE). This POE is currently being reviewed by the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE) for sufficiency. Army PMO officials stated that the POE as modified by ODASA-CE will reflect best practices with respect to sensitivity and risk and uncertainty analyses prior to being submitted as an official updated Army Cost Position. GFEBS is the Army's General Fund general ledger and became the Army's system of record at the beginning of Fiscal Year 2013. On July 1, 2012, the Army completed GFEBS deployment to more than 53,000 users at 227 locations. We are following up with Army officials to obtain documentation for the GFEBS cost estimate to determine if DOD has taken corrective actions to address this recommendation.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the Secretary of the Army to ensure that the Chief Management Officer of the Army directs the PMO for GFEBS to update the cost estimates by preparing sensitivity and risk and uncertainty analyses using best practices.

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: In our October 2010 report, we recommended that the Army update the cost estimate for the Global Combat Support System-Army (GCSS-Army) by using actual cost and preparing a sensitivity analysis using best practices. In our September 2014 report, we found that while the Army made some improvements to the cost estimate that supported the December 2012 full deployment decision, the Army did not fully meet best practices in developing a reliable cost estimate for the GCSS-Army program. For the 2012 cost estimate, we found that the Army had made progress, but we continued to identify deficiencies in documentation related to the sensitivity analysis, risk and uncertainty analysis, and cross-checking of major cost elements for reasonableness. Therefore, the Army did not provide the information needed to support the full deployment decision for the GCSS-Army program. A reliable cost estimate is critical to the success of any program and is updated continually throughout its life cycle. Such an estimate provides the basis for informed investment decision making, realistic budget formulation and program resourcing, meaningful progress measurement, proactive course correction when warranted, and accountability for results. By incorporating best practices for developing a reliable cost estimate, DOD would increase the probability of GCSS-Army successfully achieving full deployment by the fourth quarter of fiscal year 2017 to provide needed functionality for financial improvement and audit readiness. We are following up with Army officials to obtain documentation for GCSS-Army's IMS and cost estimate to determine if DOD has taken corrective actions to address this recommendation.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the Secretary of the Army to ensure that the Chief Management Officer of the Army directs the PMO for GCSS-Army to update the cost estimates by using actual cost and preparing a sensitivity analysis using best practices.

    Agency Affected: Department of Defense

  7. Status: Closed - Implemented

    Comments: In 2011, after failing to achieve a full deployment decision within 5 years after funds were first obligated for ECSS, DOD initiated a program assessment to improve the performance of the program. After this assessment, DOD implemented actions to improve the ECSS program. These actions included establishing performance-based success criteria to measure program progress and addressing concerns that relate directly to our recommendation. While these actions occurred too late to provide significant improvement in program performance, however, they did provide DOD officials critical information to evaluate the overall ECSS program. As a result, in December 2012, DOD terminated development and implementation of the ECSS program. DOD officials said they cancelled ECSS because (1) it was going to require an additional $1 billion in excess of the $1 billion cost already expended, (2) it was not going to provide significant military capability; and (3) the deployment date would be delayed until fiscal year 2020. Also, consistent with our recommendation to develop an integrated master schedule that fully incorporates best practices, ECSS program officials stated that the lack of a master schedule contributed to the ECSS termination. As a result, the intent of this recommendation was addressed.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should Direct the Secretary of the Air Force to ensure that the Chief Management Officer of the Air Force directs the PMO for ECSS to update the cost estimates by preparing a sensitivity analysis using best practices.

    Agency Affected: Department of Defense

  8. Status: Open

    Comments: In our October 2010 report, we recommended that the military departments establish performance measures based on quantitative data that will enable the department to assess whether the military service's ERP efforts are providing the intended business capabilities to the system users. DOD concurred with the recommendation and cited that the Department of Defense (DOD) would issue guidance requiring its business systems to include performance measures which can be used to assess expected business benefits. Also, the Department added that these measures would also be incorporated into the Business Enterprise Architecture. The Office of the Deputy Chief Management Officer (DCMO) stated that the DCMO has put in place performance measures and metrics through the issuance of Acquisition Decision Memorandums addressed to the Department Secretaries and Chief Management Officers. However, documentation to support this action was not provided. We are following up with DCMO officials to obtain documentation and determine if the actions taken address the recommendation.

    Recommendation: To strengthen DOD's management oversight and accountability over business system investments and help provide for the successful implementation of the ERPs, the Secretary of Defense should direct the department's Chief Management Officer and the chief management officers of the military departments to establish performance measures based on quantitative data that will enable the department to assess whether each respective military service's ERP efforts are providing the intended business capabilities to the system users.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Sep 22, 2016

Sep 21, 2016

Sep 19, 2016

Sep 12, 2016

Sep 8, 2016

Sep 7, 2016

Sep 6, 2016

Aug 25, 2016

Looking for more? Browse all our products here