DOD Business Systems Modernization:
Navy ERP Adherence to Best Business Practices Critical to Avoid Past Failures
GAO-05-858: Published: Sep 29, 2005. Publicly Released: Oct 31, 2005.
The Department of Defense's (DOD) difficulty in implementing business systems that are efficient and effective continues despite the billions of dollars that it invests each year. For a decade now--since 1995--we have designated DOD's business systems modernization as "high-risk." GAO was asked to (1) provide a historical perspective on the planning and costs of the Navy's four Enterprise Resource Planning (ERP) pilot projects, and the decision to merge them into one program; (2) determine if the Navy has identified lessons from the pilots, how the lessons are being used, and challenges that remain; and (3) determine if there are additional best business practices that could be used to improve management oversight of the Navy ERP.
The Navy invested approximately $1 billion in four ERP pilots without marked improvement in its day-to-day operations. The planning for the pilots started in 1998, with implementation beginning in fiscal year 2000. The four pilots were limited in scope and were not intended to be corporate solutions for any of the Navy's long-standing financial and business management problems. Furthermore, because of the various inconsistencies in the design and implementation of the pilots, they were not interoperable, even though they performed many of the same business functions. In short, the efforts were failures and $1 billion was largely wasted. Because the pilots would not meet its overall requirements, the Navy decided to start over and develop a new ERP system, under the leadership of a central program office. Using the lessons learned from the pilots, the current Navy ERP program office has so far been committed to the disciplined processes necessary to manage this effort. GAO found that, unlike other systems projects it has reviewed at DOD and other agencies, Navy ERP management is following an effective process for identifying and documenting requirements. The strong emphasis on requirements management, which was lacking in the previous efforts, is critical since requirements represent the essential blueprint that system developers and program managers use to design, develop, test, and implement a system and are key factors in projects that are considered successful. While the Navy ERP has the potential to address some of the Navy's financial management weaknesses, as currently planned, it will not provide an all-inclusive end-to-end corporate solution for the Navy. For example, the current scope of the ERP does not include the activities of the aviation and shipyard depots. Further, there are still significant challenges and risks ahead as the project moves forward, such as developing and implementing 44 system interfaces with other Navy and DOD systems and converting data from legacy systems into the ERP system. The project is in its early phases, with a current estimated completion date of 2011 at an estimated cost of $800 million. These estimates are subject to, and very likely will, change. Broader challenges, such as alignment with DOD's business enterprise architecture, which is not fully defined, also present a significant risk. Given DOD's past inability to implement business systems that provide the promised capability, continued close management oversight--by the Navy and DOD--will be critical. In this regard, the Navy does not have in place the structure to capture quantitative data that can be used to assess the effectiveness of the overall effort. Also, the Navy has not established an independent verification and validation (IV&V) function. Rather, the Navy is using in-house subject matter experts and others within the project. Industry best practices indicate that the IV&V activity should be independent of the project and report directly to agency management in order to provide added assurance that reported results on the project's status are unbiased.
Recommendations for Executive Action
Comments: In comments to our report, the Secretary of Defense agreed with our recommendation and stated that the Navy ERP Direct Reporting Program Manager is activating quantitative metrics related to configuration control, development progress, earned valued, and quality. According to the Navy ERP Program Manager, the Navy ERP Program has implemented a resource-loaded Integrated Master Schedule and a set of metrics to track system development, testing, and deployment. The metrics are to be used to track program progress. We recently reported (GAO-08-896) that the program has not effectively implemented several key acquisition management controls which has resulted in schedule delays and cost overruns. In the course of our our recent job (code 310666, report GAO-09-841)we requested the Program Metrics Plan. In DOD's May 2009 response to our request followup, they reported that the plan was never finalized and they are instead using their Integrated Master Schedule (IMS). GAO reported problems with their schedule in GAO-08-896, and DOD recently reported in response to recommendations in the report that they are revising the IMS.
Recommendation: To improve the Navy's and DOD's oversight of the Navy ERP effort, the Secretary of Defense should direct the Secretary of the Navy to require that the Navy ERP Program Management Office develop and implement the quantitative metrics needed to evaluate project performance and risks and use the quantitative metrics to assess progress and compliance with disciplined processes.
Agency Affected: Department of Defense
Comments: In comments on our report, the Secretary of Defense agreed with our recommendation and stated that the Direct Program Management Office is establishing an independent verification and validation team within its organization which will report directly to the Program Manager (PM). Reports were to be provided directly to the Navy ERP PM and the Navy Component Acquisition Executive. The Component Acquisition Executive and/or the Navy ERP PM was responsible for the informing Business Transformation Office of any significant results which it believes will result in a breach of the established program baseline. We recently reported (GAO-08-896) that the program has not effectively implemented several key acquisition management controls which has resulted in schedule delays and cost overruns. Although the IV&V function was established, we recently reported (GAO-09-841) that the Navy has not effectively managed its IV&V function. The program oversight aspect of the recommendation has not been fully implemented, as IV&V only reports directly and solely to the program manager.
Recommendation: To improve the Navy's and DOD's oversight of the Navy ERP effort, the Secretary of Defense should direct the Secretary of the Navy to require that the Navy ERP Program Management Office establish an IV&V function and direct that all IV&V reports be provided to Navy management and to the appropriate DOD investment review board, as well as the project management.
Agency Affected: Department of Defense
Comments: In comments on our report, the Secretary of Defense partially agreed with our recommendation and stated that the Defense Business Systems Management Committee (DBSMC) meetings will include transformation briefings from each of the components that contain their overall efforts, initiatives, and systems. The Secretary also stated that the DBSMC is conducting reviews of the DOD enterprise level business programs.
Recommendation: Furthermore, given the uncertainty of the DOD business enterprise architecture, the Secretary of Defense should direct the Defense Business Systems Management Committee to institute semiannual reviews of the Navy ERP to ensure that the project continues to follows the disciplined processes and meets its intended costs, schedule, and performance goals. Particular attention should be directed towards system testing, data conversion, and development of the numerous system interfaces with the other Navy and DOD systems.
Agency Affected: Department of Defense