Weapon System Requirements: Detailed Systems Engineering Prior to Product Development Positions Programs for Success

What GAO Found

GAO's analysis of nine case studies identified four factors that frame the challenge posed by a given weapon system's requirements: acquisition approach, technology status, design maturity, and system interdependency. Systems engineering is the primary means for determining whether and how that challenge can be met. It is a disciplined learning process that translates requirements into specific design features and thus identifies key risks to be resolved. GAO's prior best practices work has found that if detailed systems engineering is done early, a program can resolve such risks through trade-offs and additional investments before a program starts. A key point in systems engineering where this match can be assessed is the preliminary design. As shown below, establishing a preliminary design through early detailed systems engineering portends better program outcomes than doing so after program start.

Timing of Systems Engineering for Problematic and Successful Programs

GAO's analysis of selected Department of Defense (DOD) programs illustrates the relationship among the four factors, systems engineering, and program outcomes. Programs with modest requirements and early detailed systems engineering had better outcomes. For example, the Small Diameter Bomb Increment I program, which delivered within cost and schedule estimates, had an incremental approach, mature technologies, a derivative design, and detailed systems engineering before development began. Programs that began with more challenging requirements and insufficient systems engineering reported worse outcomes. For example, the F-35 Lightning II, which has encountered significant cost and schedule problems, began development with a single-step approach, a highly complex design, immature technologies, and little systems engineering.

Understanding the dynamic between a program's requirements, risks, and the requisite systems engineering effort has important implications for oversight. A particular challenge is that Congress often must consider requests to authorize and fund a new program in advance of the start of product development, when the business case would be better established. DOD policy requires that DOD decision makers have information about a proposed program's risk factors and systems engineering status, in a systems engineering plan, at the start of a new program.  However, it is not clear whether Congress also has this information at that time. A systems engineering plan could provide more robust information to Congress when considering a budget request to start a new program. In commenting on a draft of this report DOD disagreed.

Why GAO Did This Study

Cost and schedule growth in DOD major defense acquisition programs persist, and some acquisition reform proponents believe such growth is due to unplanned changes in program requirements (commonly referred to as "requirements creep"). GAO found in June 2015 that cost and schedule growth are often more directly related to a lack of systems engineering, which, if done, would reduce risk by introducing discipline and rigor into the process of defining and understanding a program's initial requirements.

House Armed Services Committee Report 114-102 contained a provision for GAO to review the DOD requirements process. This report (1) identifies a framework for assessing the challenge posed by weapon system requirements and the extent of systems engineering done before product development begins; (2) illustrates the relationship between systems engineering and program outcomes; and (3) assesses implications for program oversight. GAO analyzed a non-generalizable sample of nine case studies. GAO assessed the extent to which systems engineering was conducted before development by reviewing program requirements and analyzing cost and schedule documentation for each case study. GAO also reviewed prior GAO work and interviewed DOD officials.