Skip to main content

Weapon System Requirements: Detailed Systems Engineering Prior to Product Development Positions Programs for Success

GAO-17-77 Published: Nov 17, 2016. Publicly Released: Nov 17, 2016.
Jump To:

Fast Facts

The Department of Defense expects to spend more than $1.4 trillion to develop and procure its current portfolio of major weapon system programs, such as the F-35 Joint Strike Fighter and the KC-46A refueling tanker. DOD's major programs have historically come in over cost and behind schedule.

Our latest look at 9 programs found that delays and overruns might be averted through early, detailed systems engineering—a disciplined process for ensuring that program requirements can be met with available resources.

We recommended DOD submit systems engineering plans to Congress when requesting new program funding.

F-35 Fighter Jet 

Picture of an F-35 fighter jet in flight.

Picture of an F-35 fighter jet in flight.

Skip to Highlights

Highlights

What GAO Found

GAO's analysis of nine case studies identified four factors that frame the challenge posed by a given weapon system's requirements: acquisition approach, technology status, design maturity, and system interdependency. Systems engineering is the primary means for determining whether and how that challenge can be met. It is a disciplined learning process that translates requirements into specific design features and thus identifies key risks to be resolved. GAO's prior best practices work has found that if detailed systems engineering is done early, a program can resolve such risks through trade-offs and additional investments before a program starts. A key point in systems engineering where this match can be assessed is the preliminary design. As shown below, establishing a preliminary design through early detailed systems engineering portends better program outcomes than doing so after program start.

Timing of Systems Engineering for Problematic and Successful Programs Timing of Systems Engineering for Problematic and Successful Programs

GAO's analysis of selected Department of Defense (DOD) programs illustrates the relationship among the four factors, systems engineering, and program outcomes. Programs with modest requirements and early detailed systems engineering had better outcomes. For example, the Small Diameter Bomb Increment I program, which delivered within cost and schedule estimates, had an incremental approach, mature technologies, a derivative design, and detailed systems engineering before development began. Programs that began with more challenging requirements and insufficient systems engineering reported worse outcomes. For example, the F-35 Lightning II, which has encountered significant cost and schedule problems, began development with a single-step approach, a highly complex design, immature technologies, and little systems engineering.

Understanding the dynamic between a program's requirements, risks, and the requisite systems engineering effort has important implications for oversight. A particular challenge is that Congress often must consider requests to authorize and fund a new program in advance of the start of product development, when the business case would be better established. DOD policy requires that DOD decision makers have information about a proposed program's risk factors and systems engineering status, in a systems engineering plan, at the start of a new program.  However, it is not clear whether Congress also has this information at that time. A systems engineering plan could provide more robust information to Congress when considering a budget request to start a new program. In commenting on a draft of this report DOD disagreed.

Why GAO Did This Study

Cost and schedule growth in DOD major defense acquisition programs persist, and some acquisition reform proponents believe such growth is due to unplanned changes in program requirements (commonly referred to as "requirements creep"). GAO found in June 2015 that cost and schedule growth are often more directly related to a lack of systems engineering, which, if done, would reduce risk by introducing discipline and rigor into the process of defining and understanding a program's initial requirements.

House Armed Services Committee Report 114-102 contained a provision for GAO to review the DOD requirements process. This report (1) identifies a framework for assessing the challenge posed by weapon system requirements and the extent of systems engineering done before product development begins; (2) illustrates the relationship between systems engineering and program outcomes; and (3) assesses implications for program oversight. GAO analyzed a non-generalizable sample of nine case studies. GAO assessed the extent to which systems engineering was conducted before development by reviewing program requirements and analyzing cost and schedule documentation for each case study. GAO also reviewed prior GAO work and interviewed DOD officials.

Recommendations

To support oversight and inform budget decisions, Congress should consider requiring DOD to report on systems engineering status of each major acquisition program in the department's annual budget request.

Matter for Congressional Consideration

Matter Status Comments
To enhance program oversight and provide more robust input to budget deliberations, Congress should consider requiring DOD to report on each major acquisition program's systems engineering status in the department's annual budget request, beginning with the budget requesting funds to start development. The information could be presented on a simple timeline--as done for the case studies in this report--and at a minimum should reflect the status of a program's functional and allocated baselines as contained in the most current version of the program's systems engineering plan.
Open
GAO conducted work examining recent congressionally mandated changes in DOD's acquisition and requirements processes. Congress took action in Section 805 of the NDAA for FY 2022, in part requiring DOD to develop a new approach to report program acquisition data. In June 2023, DOD's Office of the Under Secretary of Defense for Acquisition and Sustainment issued an Implementation Plan for the Modernized Selected Acquisition Report Process in part to respond to Section 805. The plan notes that DOD expects to report key engineering milestones, but did not indicate whether the department will also report on the status of systems engineering baselines as we recommended. In November 2023, DOD issued its guidance for preparing the new reports, which does not require programs to report on the status of systems engineering baselines.

Full Report

Office of Public Affairs

Topics

Systems designProduct developmentDefense acquisition programsCost and scheduleWeapon systemsBudget requestsBest practicesCost estimatesEngineeringCongressional oversightCost controlCost overrunsDefense budgetsDefense procurementReporting requirementsRisk assessmentSystems acquisitionSystems developmentExecutive agency oversightCost estimatesCost and schedule performanceAcquisition costs