Business Systems Modernization:
Scope and Content of DOD's Congressional Report and Executive Oversight of Investments Need to Improve
GAO-10-663: Published: May 24, 2010. Publicly Released: May 24, 2010.
Since 1995, GAO has designated the Department of Defense's (DOD) multibillion dollar business systems modernization program as high risk, and it continues to do so today. To assist in addressing DOD's modernization challenges, the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (the act) requires the department to, among other things, report specific information about business system investments, including (1) milestones and actual performance against specified measures and any revisions and (2) actions taken to certify that a modernization investment involving more than $1 million meets defined conditions before obligating funds. The act also directs GAO to review each report. As agreed, GAO focused on the fiscal year 2010 report's compliance with, among other things, these provisions of the act. To do so, GAO compared DOD's report to the act's reporting requirements, interviewed DOD officials, analyzed relevant documentation, and leveraged prior GAO reports.
DOD's fiscal year 2010 report to Congress on its business systems modernization program complies with key provisions in the act, but its scope and content are nevertheless limited. Specifically, (1) The report includes milestones, performance against milestones, and milestone revisions for specific investments. However, other important performance measures, such as measures of progress against program cost, capability, and benefit commitments are not included in the report. DOD officials attributed the missing performance-related information to various factors, including that most of the investments addressed in the report have not progressed far enough in their life cycles to measure cost, capability, and benefit performance. However, the report also cites a number of investments that have produced business improvements and cost savings, and thus it follows that performance-related information about investment costs incurred, capabilities delivered, and benefits realized is available and can be reported relative to program expectations. Moreover, programs that have not yet delivered capabilities or realized benefits have nevertheless incurred costs, which DOD can report relative to expected costs. (2) The report identifies certification actions associated with 116 business system modernization investments. However, the report omits certification actions for 40 other investments. According to DOD officials, the omitted actions are not new certifications, but rather are recertifications that were intentionally excluded from the report. However, certification memoranda show this is not the case for four of the actions and DOD guidance defines a recertification as a type of certification action. Further, the underlying bases for a number of reported actions are limited because program weaknesses that GAO's prior work has raised, such as the reliability of the systems' economic analyses and the sufficiency of business enterprise architecture compliance determinations, are not reflected in the reported certification actions. DOD's guidance does not require that certification submissions disclose program weaknesses that GAO has raised, and DOD officials stated that reviews are limited to the information that is submitted. As a result, DOD's annual report does not provide the full range of information that is needed to permit meaningful and informed congressional oversight of the department's business systems modernization efforts. Moreover, the bases for some certification actions exclude relevant information about known investment weaknesses, and thus these actions may not be sufficiently justified.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: In response to our recommendation, the department included a list of business system certification actions taken in the previous year in its 2014 annual report to Congress. More specifically, the 2014 annual report provided information about, among other things, investments that were certified, certified with conditions, and not certified. In addition, the fiscal year 2014 report stated that DOD had begun working to collect and report information on cost, capability and performance. For example, the report stated that the fiscal year 2014 organizational execution plans were required to contain information on cost drivers. Further, according to officials from the office of the Deputy Chief Management Officer (DCMO), each organizational execution plan was to include performance measures for its portfolio. Our review of those plans indicated that they had begun to include such information. According to the DCMO officials, these preliminary steps helped the Department begin to understand the costs that drive operations and their impact on performance. DCMO officials also stated that the effort also provided insight into the challenges the Department will face to gather cost, capability and benefit performance measures for each business process and the information technology that enables those processes. Given that the National Defense Authorization Act for Fiscal Year 2012 significantly increased the number of business systems subject to the annual certification and approval requirement, the department's planned portfolio-based approach to monitoring cost, capability, and performance is appropriate. Accordingly, due to progress made in reporting on certification actions and planned actions for capturing cost, capability, and performance information from a portfolio perspective, we determined this recommendation has been fully implemented.
Recommendation: To facilitate congressional oversight and promote departmental accountability, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that the scope and content of future DOD annual reports to Congress on compliance with section 332 of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, as amended, be expanded to include (1) cost, capability, and benefit performance measures for each business system modernization investment and actual performance against these measures; and (2) all certification actions, as defined in DOD guidance, which were taken in the previous year by the department on its business system modernization investments.
Agency Affected: Department of Defense
Status: Closed - Implemented
Comments: In response to our recommendation, the Office of the Deputy Chief Management Officer updated its investment management guidance to require precertification authorities to include information about any open GAO recommendations as well as a status update on addressing these open recommendations as part of the certification requests. More specifically, the guidance for fiscal year 2014 and 2015 business system certification submissions requires precertification authorities to include any open GAO recommendations for program weaknesses as well as the status of actions to address them in the precertification memorandums.
Recommendation: To ensure that Investment Review Board (IRB) certification actions are better informed and justified, the Secretary should direct the Deputy Secretary to ensure that DOD guidance be revised to include provisions that require IRB certification submissions disclose program weaknesses raised by GAO and the status of actions to address our recommendations to correct the weaknesses.
Agency Affected: Department of Defense