Best Practices:

An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD's Acquisition Outcomes

GAO-07-388: Published: Mar 30, 2007. Publicly Released: Mar 30, 2007.

Additional Materials:

Contact:

Michael J. Sullivan
(937) 258-7915
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Over the next several years, the Department of Defense (DOD) plans to invest $1.4 trillion in major weapons programs. While DOD produces superior weapons, GAO has found that the department has failed to deliver weapon systems on time, within budget, and with desired capabilities. While recent changes to DOD's acquisition policy held the potential to improve outcomes, programs continue to experience significant cost and schedule overruns. GAO was asked to examine how DOD's processes for determining needs and allocating resources can better support weapon system program stability. Specifically, GAO compared DOD's processes for investing in weapon systems to the best practices that successful commercial companies use to achieve a balanced mix of new products, and identified areas where DOD can do better. In conducting its work, GAO identified the best practices of: Caterpillar, Eli Lilly, IBM, Motorola, and Procter and Gamble.

To achieve a balanced mix of executable development programs and ensure a good return on their investments, the successful commercial companies GAO reviewed take an integrated, portfolio management approach to product development. Through this approach, companies assess product investments collectively from an enterprise level, rather than as independent and unrelated initiatives. They weigh the relative costs, benefits, and risks of proposed products using established criteria and methods, and select those products that can exploit promising market opportunities within resource constraints and move the company toward meeting its strategic goals and objectives. Investment decisions are frequently revisited, and if a product falls short of expectations, companies make tough go/no-go decisions. The companies GAO reviewed have found that effective portfolio management requires a governance structure with committed leadership, clearly aligned roles and responsibilities, portfolio managers who are empowered to make investment decisions, and accountability at all levels of the organization. In contrast, DOD approves proposed programs with much less consideration of its overall portfolio and commits to them earlier and with less knowledge of cost and feasibility. Although the military services fight together on the battlefield as a joint force, they identify needs and allocate resources separately, using fragmented decision-making processes that do not allow for an integrated, portfolio management approach like that used by successful commercial companies. Consequently, DOD has less assurance that its investment decisions address the right mix of warfighting needs, and, as seen in the figure below, it starts more programs than current and likely future resources can support, a practice that has created a fiscal bow wave. If this trend goes unchecked, Congress will be faced with a difficult choice: pull dollars from other high-priority federal programs to fund DOD's acquisitions or accept gaps in warfighting capabilities.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To ensure commercial best practices are incorporated, the Secretary of Defense should hold officials at all levels accountable for achieving and maintaining a balanced, joint portfolio of weapon system investments that meet the needs of the warfighter within resource constraints.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: Since the issuance of our report DOD's leadership has taken several actions that indicate their commitment to holding programs in general and program officials specifically accountable for achieving good outcomes. On April 6, 2009 the Secretary of Defense testified before Congress and outlined sweeping changes aimed at reducing or eliminating poor performing programs in DOD's weapon system acquisition portfolio. The proposed changes included halting production of the Air Force's F-22 fighter aircraft at 187 planes, eliminating funding for the new presidential helicopter program, and drastically restructuring the Army's Future Combat Systems program into smaller, more manageable elements. Following these actions, the Secretary of Defense releaved the Joint Strike Fighter program manager of his duties because of poor program performance, and directed a program restructuring. On June 28, 2010 the Undersecretary of Defense (Acquisition, Technology, and Logistics) issued a memorandum to the defense acquisition community that expresses the intent of the department's leadership to gain efficiency and eliminate unneeded programs and activities to generate cost savings and better balance DOD's overall investment portfolio. These actions by the current department leadership address the intent of our recommendation.

    Recommendation: To ensure commercial best practices are incorporated, the Secretary of Defense should establish portfolio managers who are empowered to prioritize needs, make early go/no-go decisions about alternative solutions, and allocate resources within fiscal constraints.

    Agency Affected: Department of Defense

    Status: Closed - Not Implemented

    Comments: On September 25, 2008 the Department of Defense issued DOD Directive 7045.20, which established policy and assigned responsibilities for the use of capability portfolio management to advise senior DOD leadership on making capability investments. The policy establishes Capability Portfolio Managers (CPMs) and emphasizes the need to view weapon system investments from a portfolio perspective. CPMs do not have independent decision-making authority and must operate within DOD's existing framework for planning and acquiring weapon systems, but they do advise the Deputy's Advisory Working Group (DAWG) which is the senior DOD proponent charged with integrating, synchronizing, and coordinating portfolio content. In addition, CPM's have access to the Joint Requirements Oversight Council, Defense Acquisition Board, and other established Component forums to raise portfolio related issues and influence resource allocation decisions at early decision points. While this is a step in the right direction, this influence does not empower the CPMs to make program or resource allocation decisions as we recommended. Therefore, we are closing this recommendation as not-implemented.

    Recommendation: To ensure commercial best practices are incorporated, the Secretary of Defense should require increasingly precise cost, schedule, and performance information for each alternative that meets specified levels of confidence and allowable deviations at each decision point leading up to the start of product development.

    Agency Affected: Department of Defense

    Status: Closed - Not Implemented

    Comments: The Weapon System Acquisition Reform Act of 2009 and DOD's revised acquisition policy-the DOD 5000 series issued on December 2, 2008-emphasize the importance of assessing program costs early and establishing high-confidence cost estimates before initiating system development. The 2009 Act specifically states that the "Director of Cost Assessment and Program Evaluation, and the Secretary of the military department concerned or the head of the Defense Agency concerned shall each disclose the confidence level used in establishing a cost estimate for a major defense acquisition program or major automated information system program, the rationale for selecting such a confidence level, and, if such confidence level is less than 80 percent, the justification for selecting a confidence level of less than 80 percent." DOD's revised acquisition policy requires initial cost estimates to be done prior to Milestone A and refined during technology development. If, during technology development, the cost estimate established at Milestone A increases by 25 percent or more the program manager is required to notify the milestone decision authority, who will, in consultation with the requirements community, determine whether the magnitude of the need justifies the required level of resources. Requiring programs now to conduct a technology development phase should help programs attain the requisite knowledge needed to estimate costs. While these actions are a step in the right direction, DOD does not require programs to provide a range of potential costs around a point estimate, that is information about allowable deviations. According to DOD cost analysts within the Office of the Director of Cost Analysis and Program Evaluation, DOD does not now nor does it plan to require programs to provide range estimates as a matter of policy. Therefore, we are closing this recommendation as not implemented.

    Recommendation: To ensure commercial best practices are incorporated, the Secretary of Defense should prioritize programs based on the relative costs, benefits, and risks of each investment to ensure a balanced portfolio.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: On April 6, 2009 the Secretary of Defense testified before Congress and outlined sweeping changes aimed at reducing or eliminating poor performing programs in DOD's weapon system acquisition portfolio. The proposed changes included halting production of the Air Force's F-22 fighter aircraft at 187 planes, eliminating funding for the new presidential helicopter program, and drastically restructuring the Army's Future Combat Systems program into smaller, more manageable elements. Following these major program changes, the Undersecretary of Defense (Acquisition, Technology, and Logistics) issued a memorandum to the defense acquisition community on June 28, 2010 that expresses the intent of the department's leadership to gain efficiency and eliminate unneeded programs and activities to generate cost savings and better balance DOD's overall investment portfolio. These actions by the current department leadership address the intent of our recommendation.

    Recommendation: To ensure commercial best practices are incorporated, the Secretary of Defense should implement a review process in which needs and resources are integrated early and in which resources are committed incrementally based on the achievement of specific levels of knowledge at established decision points.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: On December 2, 2008, DOD issued a revised version of the defense acquisition policy-the DOD 5000 series-containing several requirements that, in concert with the previously enacted legislation, could provide a better foundation for integrating needs and resources early on, before programs are initiated. According to Title 10 section 2366b, major acquisition programs beginning after March 2008 may not receive Milestone A approval to begin a technology development program until the milestone decision authority (MDA) certifies to Congress that, among other things, a valid need is being addressed, the right people are addressing the need, and a cost estimate for the system has been submitted. DOD's revised policy echoes these requirements, noting that DOD Components shall conduct an Analysis of Alternatives that integrates needs with resources prior to Milestone A, and submit a cost estimate for proposed solutions at Milestone A. The policy further notes that if the cost estimate established at Milestone A increases by 25 percent or more prior to Milestone B, the program manager must notify the MDA. The MDA, in consultation with the requirements community, shall then determine whether the capability need is truly worth the estimated level of resources. Additionally, DOD is also now conducting preliminary design review earlier to balance requirements with resources. Some DOD programs also appear to be embracing incremental processes. Because of these actions, we are closing this recommendation as implemented and we will continue to monitor DOD's efforts in GAO's ongoing work in systems acquisitions.

    Recommendation: The Secretary of Defense should implement an enterprise-wide portfolio management approach to making weapon system investments that integrates the assessment and determination of warfighting needs with available resources and cuts across the services by functional or capability area. To ensure the success of such an approach, the Secretary should establish a single point of accountability at the department level with the authority, responsibility, and tools to ensure that portfolio management for weapon system investments is effectively implemented across the department.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: On September 25, 2008 DOD issued Directive 7045.20 "Capability Portfolio Management" establishing an enterprise-wide portfolio management policy. Specifically the policy states that "the Department of Defense shall use capability portfolio management to advise the Deputy Secretary of Defense and the Heads of the DOD components on how to optimize capability investments across the defense enterprise (both material and non-material) and minimize risk in meeting the Department's capability needs in support of strategy." The policy further notes that "capability portfolios shall be managed by civilian and military Capability Portfolio Managers (CPM) co-leads designated by the Deputy Secretary of Defense." The CPMs are intended to evaluate capability demand (both warfighting and non-warfighting) against resource constraints, identify and assess risks, and suggest capability trade-offs within their capability portfolio. As currently structured, they provide advice to the Deputy's Advisory Working Group (DAWG) which is the senior DOD proponent charged with integrating, synchronizing, and coordinating portfolio content to ensure alignment to strategic priorities and capability demand. CPMs also have access to the Joint Requirements Oversight Council, Defense Acquisition Board, and other established Component forums to raise portfolio related issues and influence resource allocation decisions.

    Recommendation: The Secretary of Defense should take steps to support department-level decision makers and portfolio managers by developing a stronger joint analytical capability to assess and prioritize warfighting needs.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In the Weapon System Acquisition Reform act of 2009, the Congress, among other things, created the position of Director of Cost Assessment and Program Evaluation (CAPE). In general the law established the Director as the principle official within the senior management of the Department of Defense for overseeing, reviewing, and in some cases conducting program funding, cost, schedule, and performance analyses to provide advice to the Secretary of Defense to inform the prioritization of weapon system investments. The Act identified "leading the development of improved analytical skills and competencies within the cost assessment and program evaluation workforce of the Department of Defense and improved tools, data, and methods to promote performance, economy, and efficiency in analyzing national security planning and the allocation of defense resources," as one of the Director's specific responsibilities. Bringing these various responsibilities together under a single joint organization is likely to create synergy and thereby strengthen DOD's analytical capability, but it is still to soon to determine its full impact. In addition to the creation of CAPE, DOD is in the process of studying ways to improve its Joint Capabilities Integration and Development System, which validates warfighting needs. These actions represent significant movement towards the implementation of this recommendation and we are, therefore, closing it as implemented. GAO will continue to monitor DOD's efforts in ongoing work in systems acquisitions.

    Jul 31, 2014

    Jul 30, 2014

    Jul 28, 2014

    Jul 17, 2014

    Jul 14, 2014

    Looking for more? Browse all our products here