Skip to main content

Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes

GAO-01-288 Published: Mar 08, 2001. Publicly Released: Mar 08, 2001.
Jump To:
Skip to Highlights

Highlights

This report examines how best practices offer improvements to the way the Department of Defense defines and matches weapon system requirements to available resources such as cost, schedule, and mature technologies. GAO identified three factors that were key to matching needs and resources before product development began. First, developers employed the technique of systems engineering to identify gaps between resources and customer needs before committing to a new product development. Second, customers and developers were flexible. Leeway existed to reduce or defer customer needs to future programs or for the developer to make an investment to increase knowledge about a technology or design feature before beginning product development. Third, the roles and responsibilities of the customer and the product developer were matched, with the product developer being able to determine or significantly influence product requirements. In cases where these factors were not present at program launch, product development began without a match between requirements and resources. Invariably, this imbalance favored meeting customer needs by adding resources, which resulted in increased costs and later deliveries.

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Defense To realign the mechanics of the requirements setting and program approval processes to bring more knowledge into the process of setting requirements, the Secretary of Defense should require that the systems engineering needed to evaluate the sufficiency of available resources--knowledge, time, money, and capacity--be conducted in time to help identify and make the critical trade-offs that precede the formalization of requirements. One option is to allow the award of well-defined systems engineering contracts to prospective product developers--contractors--before the system development and demonstration phase.
Closed – Implemented
DOD has taken steps to insure that product requirements are better informed by the systems engineering process. It recently revised its guidance for setting requirements. The guidance states before program approval, the requirements described by the customer should depict a technologically mature and affordable product increment that was demonstrated in an operationally relevant environment. As a part of its revised acquisition policy, the 5000 series, DOD has emphasized the use of Advanced Concept Technology Demonstrations and Advanced Technology Demonstrations as a means to better transition innovative product concepts to the warfighter. The expanded use of techniques like Advanced Concept Technology Demonstrations and Advanced Technology Demonstrations could help provide a better match between requirements and resources prior to program launch. DOD has stated that they would like demonstrations that allow systems engineering to identify and close gaps between customer wants and achievable resources prior to program launch to be the norm instead of the exception. This has been emphasized with the realignment and refinement of the acquisition phases leading up to program approval.
Department of Defense To realign the incentives of the requirements setting and program approval processes with the need to match available resources, the Secretary of Defense should reduce the pressures put on user representatives to set requirements high to win the competition for program approval. One way to reduce these pressures, drawing on the experiences of the Tactical Unmanned Aerial Vehicle and Global Hawk, is to have higher level officials in the services and DOD decide on the type of weapon system that is needed to meet a valid need before the requirements setters begin detailed work on framing a specific solution. Making such a decision earlier in the process would ease the pressure to set overly demanding and inflexible requirements that will crush alternatives and win program approval.
Closed – Implemented
DOD, in its revised acquisition policy, has made the use of evolutionary acquisitions, the preferred product development approach. This approach allows for a better matching of requirements and resources providing an opportunity to defer challenging requirements until technologies are ready. The initial product should provide the user only the minimum acceptable capability with the expectation of future capabilities. The deferred capabilities can be incorporated into later iterations of the product. This reduces the pressures put on user representatives to set requirements high, forcing the developer to develop a "quantum leap" solution. GAO found evolutionary product development to be a best practice used by commercial firms.
Department of Defense To realign the incentives of the requirements setting and program approval processes with the need to match available resources, the Secretary of Defense should require, as a condition for starting the system development and demonstration phase for a weapon system--program launch--that sufficient evidence exists to show that there is a match between a weapon system's requirements and the resources the program manager has to develop that weapon. Based on GAO's current and past work on the best practices of leading commercial firms, there is a key tool the Secretary can use to define what resources the Department is willing to apply--establishing limits on the time its takes to complete system development, such as not to exceed five years. Further, having a formal agreement among the requirement setters, program managers, and resource providers on development and delivery of the required product would emulate the best practice of establishing accountability for subsequent actions that stray from the agreement.
Closed – Implemented
DOD has taken concrete actions to both lessen the incentives for authors to aim too high in writing requirements, and to strengthen the emphasis on the ability of the program manager to employ mature technology in matching a new system's requirements. Specifically, the revised 5000 series of acquisition regulations calls for evolutionary approaches to developing new weapons and requiring technology maturity before starting new programs. Also, new guidance on preparing requirements calls for establishing time-phased requirements to better align with the evolutionary acquisition approach. While DOD has not set specific time limits on product development, they have emphasized the reduction of product development cycle times from the 10 to 15 historical averages. DOD has taken steps that should help them to accomplish this. DOD's revised acquisition policy emphasizes the use of evolutionary development approaches that develops products in increments, delivering product developments in much shorter time periods. For example, the Air Force has set a goal of delivering a new product every 2 to 4 years. Each increment should provide a useful capability to the user. Evolutionary development should be anchored in a formal agreement among the requirements setter, program manager, and resource providers on the specific plan to deliver the products.

Full Report

GAO Contacts

Michael J. Sullivan
Director
Contracting and National Security Acquisitions

Media Inquiries

Sarah Kaczmarek
Managing Director
Office of Public Affairs

Topics

Best practicesConcurrencyDefense capabilitiesDefense cost controlDefense procurementPrivate sector practicesProcurement planningWeapons research and developmentWeapons systemsComparative benchmarking products