Best Practices:

Increased Focus on Requirements and Oversight Needed to Improve DOD's Acquisition Environment and Weapon System Quality

GAO-08-294: Published: Feb 1, 2008. Publicly Released: Feb 1, 2008.

Additional Materials:

Contact:

Michael J. Sullivan
(937) 258-7915
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

A Senate report related to the National Defense Authorization Act for Fiscal Year 2007 asked GAO to compare quality management practices used by the Department of Defense (DOD) and its contractors to those used by leading commercial companies and make suggestions for improvement. To do this, GAO (1) determined the impact of quality problems on selected weapon systems and prime contractor practices that contributed to the problems; (2) identified commercial practices that can be used to improve DOD weapon systems; (3) identified problems that DOD must overcome; and (4) identified recent DOD initiatives that could improve quality. GAO examined 11 DOD weapon systems with known quality problems and met with quality officials from DOD, defense prime contractors, and five leading commercial companies that produce complex products and/or are recognized for quality products.

Problems related to quality have resulted in major impacts to the 11 DOD weapon systems GAO reviewed--billions in cost overruns, years-long delays, and decreased capabilities for the warfighter. For example, quality problems with the Expeditionary Fighting Vehicle program were so significant that DOD extended development 4 years at a cost of $750 million. The F-22A fighter aircraft experienced cracks in the plane's canopy that grounded the flight test aircraft, and initial operating capability for the Wideband Global SATCOM satellite was delayed 18 months because a supplier installed some fasteners incorrectly. GAO's analysis of 11 DOD weapon systems illustrates that defense contractors' poor practices for systems engineering activities as well as manufacturing and supplier quality problems contributed to these outcomes. Reliance on immature designs, inadequate testing, defective parts, and inadequate manufacturing controls are some of the quality problems that GAO found. Senior prime contractor officials GAO met with generally agreed with GAO's assessment of the causes of the quality problems. In contrast, leading commercial companies GAO contacted use more disciplined systems engineering, manufacturing, and supplier quality practices. For example, rather than wait to discover defects after the fact, Boeing Commercial Airplanes tries to design parts that can be assembled only one way. Effective use of many systems engineering practices has helped Space Systems/Loral, a satellite producer, improve overall quality, for example, by allowing the company to operate its satellites for more than 80 million consecutive hours in orbit with just one failure. Companies also put significant effort into validating product design and production processes to catch problems early on, when problems are less costly to fix. They conduct regular audits of their suppliers and hold them accountable for quality problems. DOD faces its own set of challenges--setting achievable requirements for systems development and providing effective oversight during the development process. In conducting systems development, DOD generally pays the allowable costs incurred for the contractor's best efforts. These conditions contribute to an acquisition environment that is not conducive for incentivizing contractors to build high-quality weapon systems and DOD, which typically uses cost-reimbursement contracts to develop weapon systems, assumes most of the risks and pays contractors to fix most of the problems. DOD has taken steps to improve its acquisition practices by experimenting with a new concept decision review practice, selecting different acquisition approaches according to expected fielding times, and establishing panels to review weapon system configuration changes that could adversely affect program cost and schedule. None of these initiatives focus exclusively on quality issues, and none specifically address problems with defense contractors' practices.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: Defense Contract Management Agency (DCMA) is in various stages of developing three web-based tools that will provide metrics for measuring the effectiveness of contractors' quality management systems for DOD programs. This includes the development of several indicators that measure contractor quality, through tracking of corrective action requests, quality defects, and the effectiveness of supplier control procedures, among other things. Initial software releases for each of the tools have been completed. Additional functionality will be added throughout 2011 and early 2012.

    Recommendation: To better assess the quality of weapon system programs and prime contractor performance, DOD needs to obtain and analyze more comprehensive data regarding prime contractors and their key suppliers. Therefore, the Secretary of Defense should direct the Defense Contract Management Agency and the military services to identify and collect data that provides metrics about the effectiveness of prime contractors' quality management system and processes by weapon system and business area over time.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: The Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 requires the Secretary of Defense to establish configuration steering boards for all major defense acquisition programs, thus expanding the use of the configuration steering board initiative. In March 2010 we reported that 7 of 42 programs we assessed held configuration steering board meetings in 2009 and in July 2011, we reported that 74 of 96 programs held these meetings in 2010.

    Recommendation: To ensure that the department is taking steps to improve the quality of weapon systems, the Secretary of Defense should establish measures to gauge the success of the concept decision reviews, time-defined acquisition, and configuration steering board initiatives and properly support and expand these initiatives where appropriate, when related to recent initiatives highlighted in DOD's Defense Acquisition Transformation and Program Manager Empowerment and Accountability reports to Congress to improve its focus on setting achievable requirements and oversight.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: DoD partially agreed with GAO's recommendation. In July 2008, the Under Secretary of Defense (Acquisition, Technology and Logistics) issued a policy that directs the military services and defense agencies to establish a reliability improvement acquisition policy to address the problem of inadequate system reliability, availability, and maintainability. As part of this effort, each policy is to include practices that ensure development contracts evaluate reliability, availability, and maintainability during system design and to evaluate the appropriate use of contract incentives to achieve these objectives. This action by DOD is broader than what we recommended, however, the requirements of the policy meet the intent of our recommendation. The House of Representatives is also considering acquisition reform legislation that would require the inclusion of provisions related to systems engineering and reliability growth in requests for proposals and developmental test and evaluation.

    Recommendation: To ensure that the department is taking steps to improve the quality of weapon systems, the Secretary of Defense should, as a part of the concept decision review initiative, have contractors perform more detailed systems engineering analysis to develop sound requirements before DOD selects a prime contractor for the systems development contract, which would help ensure that weapon system requirements, including those for reliability, are achievable with given resources, related to recent initiatives highlighted in DOD's Defense Acquisition Transformation and Program Manager Empowerment and Accountability reports to Congress to improve its focus on setting achievable requirements and oversight.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: Defense Contract Management Agency (DCMA) is in various stages of developing three web-based tools that will provide metrics for measuring the effectiveness of contractors' quality management systems for DOD programs. One tool uses a supplier's past performance to help predict their ability to provide quality products and will also be used to assist the agency in determining the type and frequency of surveillance that is required at a supplier's facility. Inital tool functionality was deployed in March 2011 and upgraded capabilities are expected to be deployed through the end of fiscal year 2011.

    Recommendation: To better assess the quality of weapon system programs and prime contractor performance, DOD needs to obtain and analyze more comprehensive data regarding prime contractors and their key suppliers. Therefore, the Secretary of Defense should direct the Defense Contract Management Agency and the military services to develop evaluation criteria that would allow DOD to score the performance of prime contractors' quality management systems based on actual past performance, which could be used to improve quality and better inform DOD acquisition decision makers.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Oct 30, 2014

Oct 16, 2014

Oct 10, 2014

Oct 8, 2014

Sep 30, 2014

Sep 25, 2014

Sep 23, 2014

Sep 19, 2014

Sep 18, 2014

Looking for more? Browse all our products here