Business Systems Modernization:

DOD Needs to Fully Define Policies and Procedures for Institutionally Managing Investments

GAO-07-538: Published: May 11, 2007. Publicly Released: May 11, 2007.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6256
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In 1995, GAO first designated the Department of Defense's (DOD) business systems modernization program as "high-risk," and continues to do so today. In 2004, Congress passed legislation reflecting prior GAO recommendations for DOD to adopt a corporate approach to information technology (IT) business system investment management. To support GAO's legislative mandate to review DOD's efforts, GAO assessed whether the department's corporate investment management approach comports with relevant federal guidance. In doing so, GAO applied its IT Investment Management framework and associated methodology, focusing on the framework's stages related to the investment management provisions of the Clinger-Cohen Act of 1996.

DOD has established the management structures needed to effectively manage its business system investments, but it has not fully defined many of the related policies and procedures that GAO's IT Investment Management framework defines. Specifically, the department has defined four of nine practices that call for project-level policies and procedures, and one of the five practices that call for portfolio-level policies and procedures. For example, DOD has established an enterprisewide IT investment board responsible for defining and implementing its business system investment governance process, documented policies and procedures for ensuring that systems support ongoing and future business needs, developed procedures for identifying and collecting information about these systems to support investment selection and control, and assigned responsibility to an individual or a group for managing the development and modification of the business system portfolio selection criteria. However, DOD has not fully documented business system investment policies and procedures for directing investment board operations, selecting new investments, reselecting ongoing investments, integrating the investment funding and the investment selection processes, and developing and maintaining a complete business system investment portfolio(s). Regarding project-level investment management practices, DOD officials said that these are performed at the component level, and that departmental policies and procedures established for overseeing components' execution of these practices are sufficient. For portfolio-level practices, however, these officials stated that they intend to improve departmental policies and procedures for business system investments by, for example, establishing a single governance structure, but plans or time frames for doing so have not been established. Until DOD fully defines departmentwide policies and procedures for both individual projects and portfolios of projects, it risks selecting and controlling these business system investments in an inconsistent, incomplete, and ad hoc manner, which in turn reduces the chances that these investments will meet mission needs in the most cost-effective manner.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: DOD has taken steps to address this recommendation. Specifically, DOD has instituted investment review boards (IRB) and assigned the boards with the responsibility, authority, and accountability for programs throughout the investment life cycle, including those in operations and maintenance. In addition, DOD's business capability life cycle (BCL) acquisition policy and guidance outlines aspects of how the business investment review processes are to be coordinated with other DOD decision-support processes, such as the Joint Capabilities Integration Development System (JCIDS), the Defense Acquisition System (DAS), and the Planning, Programming, Budgeting, and Execution (PPBE) process. For example, the BCL policy and guidance integrates the DOD business investment review process with JCIDS and DAS by establishing a single governance framework that is intended to provide for a more rapid delivery of system capability. To facilitate rapid and responsive development and implementation, the BCL policy and guidance requires that a defense business system should be divided into discrete, fully funded, and manageable 12 to18 month increments. To integrate the investment review process with PPBE, the BCL policy and guidance requires funding to be identified based on the increments, and will serve as input for DOD budget requests. Further, according to the guidance, the department is able to more closely monitor the defense business system cost (planned, programmed, and budgeted), schedule, and performance based on each increment, as opposed to a large-scale system implementation. As a result of these collective actions, we consider this recommendation to be implemented.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address instituting the investment boards, including assigning the investment boards responsibility, authority, and accountability for programs throughout the investment life cycle and specifying how the business investment management system is coordinated with Joint Capabilities Integration and Development System (JCIDS), planning, programming, budgeting, and execution (PPBE), and Defense Acquisition System (DAS).

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: The department has recently taken steps to update its policies and procedures to address this investment recommendation. For example, Air Force's 2008 IT investment review guidance calls for the review of all business systems, to include those in operations and maintenance, and specifies how these business investments are to be prioritized using factors such as mission and strategic value and risk.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address selecting new investments, including specifying how cost, schedule, and benefit data are to be used in making certification decisions; defining the criteria used to select investments as enterprisewide; and establishing consistent and effective guidance for business enterprise architecture (BEA) compliance.

    Agency Affected: Department of Defense

  3. Status: Closed - Not Implemented

    Comments: DOD has not fully established policies and procedures that address reselecting ongoing investments, including specifying how cost, schedule, and performance data are to be used in the annual review process and providing for the reselection of investments that are in operations and maintenance. Specifically, the 2009 IRB guidance states that the IRBs will consider cost, schedule, and benefit data in performing annual reviews but it does not define how the boards are to consider these factors and make consistent, repeatable reselection decisions across all investments. Further, the corporate investment boards only focus on reselecting those investments that are in the development/modernization stage or investments in operations and maintenance that were previously certified by the IRBs. Moreover, while components are required by policy to annually review all business systems, IRB officials stated in March 2010 that they have little insight into components' adherence to cost, schedule, and performance investment selection and control criteria as part of their reviews. In May 2011, DOD officials stated that they intend to update the IRB guidance in fiscal year 2012.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address reselecting ongoing investments, including specifying how cost, schedule, and performance data are to be used in the annual review process and providing for the reselection of investments that are in operations and maintenance.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: DOD has taken steps to address this recommendation. For example, as part of the process of selecting an investment, the 2009 IRB guidance and BCL acquisition policy calls for investments involving more than $1 million to be reviewed and approved by the IRB and DBSMC before funds are obligated. Further, the guidance and policy requires funding information to be considered by the IRBs and the DBSMC in carrying out decisions on system certifications and approvals. Moreover, the policy calls for defense business system capabilities to be delivered within 12 to 18 month increments and funding to be identified for each capability, and used as input for DOD component budget requests. As a result of these collective actions, we consider this recommendation to be implemented.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address integrating funding with the process of selecting an investment, including specifying how the DBSMC and the Investment Review Boards (IRB) use funding information in carrying out decisions on system certification and approvals.

    Agency Affected: Department of Defense

  5. Status: Closed - Not Implemented

    Comments: DOD has not fully established policies and procedures that address overseeing IT projects and systems, including providing sufficient oversight and visibility into component-level investment management activities. Specifically, the department's January 2009 IRB guidance requires components to annually review all business systems and submit an end-of-the-fiscal-year report listing those systems that have been reviewed. However, in March 2010, IRB officials stated that the reports provide little insight into components' adherence to cost, schedule, and risk investment selection and control criteria. In May 2011, DOD officials stated that they intend to update the IRB guidance in fiscal year 2012 to provide additional details about criteria for IRB and component reviews.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the Defense Business Systems Management Committee (DBSMC), to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. At a minimum, this should include project-level management policies and procedures that address overseeing IT projects and systems, including providing sufficient oversight and visibility into component-level investment management activities.

    Agency Affected: Department of Defense

  6. Status: Closed - Not Implemented

    Comments: DOD has not fully established policies and procedures that address creating and modifying IT portfolio selection criteria for business system investments. To its credit, the department has assigned responsibility to its certification authorities for defining the criteria to be used for making portfolio selection decisions, and creating and modifying portfolio IT selection criteria for business system investments. However, these authorities have yet to fulfill these responsibilities. In May 2011, DOD officials stated that they intend to update the IRB guidance to include policies and procedures for defining the portfolio selection criteria by fiscal year 2012.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the DBSMC, to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. These well-defined and disciplined business system investment management policies and procedures should also include portfolio-level management policies and procedures that address creating and modifying IT portfolio selection criteria for business system investments.

    Agency Affected: Department of Defense

  7. Status: Closed - Not Implemented

    Comments: DOD has not fully established policies and procedures that address analyzing, selecting, and maintaining business system investment portfolios. To its credit, the department has assigned responsibility to its certification authorities for defining the criteria to be used for making portfolio selection decisions, and analyzing, selecting, and maintaining business system investment portfolios. However, these authorities have yet to fulfill these responsibilities. In May 2011, DOD officials stated that they intend to update the IRB guidance to include policies and procedures for defining the portfolio selection criteria by fiscal year 2012.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the DBSMC, to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. These well-defined and disciplined business system investment management policies and procedures should also include portfolio-level management policies and procedures that address analyzing, selecting, and maintaining business system investment portfolios.

    Agency Affected: Department of Defense

  8. Status: Closed - Not Implemented

    Comments: DOD has not fully established policies and procedures that address reviewing, evaluating, and improving the performance of its portfolios. Specifically, the department assigned responsibility in January 2009 to its certification authorities for defining the criteria to be used for making portfolio selection decisions, and reviewing, evaluating, and improving the performance of portfolio investments. However, these authorities have yet to fulfill these responsibilities. In May 2011, DOD officials stated that they intend to update the IRB guidance to include policies and procedures for defining the portfolio performance criteria by fiscal year 2012.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the DBSMC, to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. These well-defined and disciplined business system investment management policies and procedures should also include portfolio-level management policies and procedures that address reviewing, evaluating, and improving the performance of its portfolio(s) by using project indicators, such as cost, schedule, and risk.

    Agency Affected: Department of Defense

  9. Status: Closed - Implemented

    Comments: DOD has taken steps that address this recommendation. Specifically, the department's January 2009 IRB guidance provides for performing post implementation reviews (called a close-out annual review) on investments subject to DOD IRB annual review. This close-out review is to be performed when a development or modernization is completed and no additional development or modernization funding is to be requested. According to the guidance, this review is to function as a post implementation review for IRB-certified systems and is to provide the IRBs with lessons learned and metrics about completed investment efforts. As a result of these collective actions, we consider this recommendation to be implemented.

    Recommendation: To strengthen DOD's business system investment management capability and address the weaknesses discussed in this report, the Secretary of Defense should direct the Deputy Secretary of Defense, as the chair of the DBSMC, to ensure that well-defined and disciplined business system investment management policies and procedures are developed and issued. These well-defined and disciplined business system investment management policies and procedures should also include portfolio-level management policies and procedures that address conducting postimplementation reviews for all investment tiers and directing the investment boards, which are accountable for corporate business system investments, to consider the information gathered and to develop lessons learned from these reviews.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Jun 10, 2014

May 22, 2014

May 12, 2014

May 8, 2014

May 7, 2014

Apr 2, 2014

Feb 26, 2014

Feb 12, 2014

Looking for more? Browse all our products here