Best Practices:

DOD Can Achieve Better Outcomes by Standardizing the Way Manufacturing Risks Are Managed

GAO-10-439: Published: Apr 22, 2010. Publicly Released: Apr 22, 2010.

Additional Materials:

Contact:

Michael J. Sullivan
(937) 258-7915
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Cost growth and schedule delays are prevalent problems in acquiring defense weapon systems. Manufacturing systems has proven difficult, particularly as programs transition to production. In December 2008, the Department of Defense (DOD) issued an updated version of its acquisition policy that reflects earlier consideration of manufacturing risks. A joint defense and industry group developed manufacturing readiness levels (MRL) to support assessments of manufacturing risks. Use of MRLs on all weapon acquisition programs has been proposed. In response to a congressional request, this report assesses the manufacturing problems faced by DOD, how MRLs can address manufacturing problems, how MRLs compare to manufacturing best practices of leading commercial firms, and challenges and barriers to implementing MRLs at DOD. In conducting our work, we contacted DOD, military services, and contractors; held interviews with leading commercial firms; reviewed program documents and policy proposals; and spoke with manufacturing experts.

DOD faces problems in manufacturing weapon systems--systems cost far more and take much longer to build than estimated. Billions of dollars in cost growth occur as programs transition from development to production, and unit-cost increases are common after production begins. Several factors contribute to these problems including inattention to manufacturing during planning and design, poor supplier management, and a deficit in manufacturing knowledge among the acquisition workforce. Essentially, programs did not identify and resolve manufacturing risks early in development, but carried risks into production where they emerged as significant problems. MRLs have been proposed as new criteria for improving the way DOD identifies and manages manufacturing risks and readiness. Introduced to the defense community in 2005, MRLs were developed from an extensive body of manufacturing knowledge that includes defense, industry, and academic sources. An analysis of DOD's technical reviews that assesses how programs are progressing show that MRLs address many gaps in core manufacturing-related areas, particularly during the early acquisition phases. Several Army and Air Force centers that piloted MRLs report these metrics contributed to substantial cost benefits on a variety of technologies and major defense acquisition programs. To develop and manufacture products, the commercial firms we visited use a disciplined, gated process that emphasizes manufacturing criteria early in development. The practices they employ focus on gathering sufficient knowledge about the producibility of their products to lower risks, and include stringent manufacturing readiness criteria to measure whether the product is sufficiently mature to move forward in development. These criteria are similar to DOD's proposed MRLs in that commercial firms (1) assess producibility at each gate using clearly defined manufacturing criteria to gain knowledge about manufacturing early, (2) demonstrate manufacturing processes in a production-relevant environment, and (3) emphasize relationships with critical suppliers. However, a key difference is that commercial firms, prior to starting production, require their manufacturing processes to be in control--that is, critical processes are repeatable, sustainable, and consistently producing parts within the quality standards. DOD's proposed MRL criteria do not require that processes be in control until later. Acceptance of MRLs has grown among some industry and DOD components. Yet, DOD has been slow to adopt a policy that would require MRLs across DOD. Concerns raised by the military services have centered on when and how the MRL assessments would be used. While a joint DOD and industry group has sought to address concerns and disseminate information on benefits, a consensus has not been reached. If adopted, DOD will need to address gaps in workforce knowledge, given the decrease in the number of staff in the production and manufacturing career fields.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In April 2010, GAO issued a report identifying commercial best practices on the use of analytical models and tools that could support assessments of manufacturing readiness. To enhance the use of manufacturing assessments, GAO recommended that the Department of Defense assess the need for such models and tools in supporting assessments of manufacturing readiness. The Department agreed to initiate efforts to capture this knowledge and provide tools for use in conducting assessments. The Department also updated its Defense Acquisition Guidebook in July 2011 that provide references to tools and resources that can be found in the Manufacturing Readiness Level (MRL) Deskbook. The May 2011 MRL Deskbook references a collaborative website (http://www.dodmrl.com/) that has links to numerous tools and resources from industry, academia and DOD for sharing knowledge and to support manufacturing assessments.

    Recommendation: To ensure that DOD is taking steps to strengthen and improve the producibility and manufacturing readiness of technologies, weapon systems, subsystems, or manufacturing processes, the Secretary of Defense should direct the Office of the Director, Defense Research and Engineering to assess the need for analytical models and tools to support MRL assessments.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: In April 2010, GAO found that the Department of Defense required their manufacturing processes and procedures to be in control after the production decision (milestone C) whereas commercial best practice requires that manufacturing processes be in control prior to the production decision. GAO recommended that the criteria--to include the process capability and control of critical components and/or interfaces--be strengthened for making the initial production decision. The Department of Defense concurred with this recommendation and subsequently revised and strengthened its production decision criteria. The Department's deskbook that contains the strengthened manufacturing criteria on process and procedures was updated and approved in July 2010.

    Recommendation: To ensure that DOD is taking steps to strengthen and improve the producibility and manufacturing readiness of technologies, weapon systems, subsystems, or manufacturing processes, the Secretary of Defense should direct the Office of the Director, Defense Research and Engineering to examine strengthening the MRL criteria related to the process capability and control of critical components and/or interfaces prior to milestone C, or equivalent, for low-rate initial production decision.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: In April 2010, GAO found that the Department of Defense was lacking manufacturing knowledge at key decision points and that, although there have been a variety of evaluations performed for years, there was no uniform way to measure and communicate manufacturing risk and readiness. GAO recommended that defense programs be assessed using manufacturing readiness level (MRL) criteria for assessing, reporting, and communicating manufacturing readiness risk on programs. In July 2011, the Department updated its Defense Acquisition Guidebook which provides that the MRL Deskbook be used as a tool and resource to help defense programs with manufacturing related reporting and oversight.

    Recommendation: To ensure that DOD is taking steps to strengthen and improve the producibility and manufacturing readiness of technologies, weapon systems, subsystems, or manufacturing processes, the Secretary of Defense should require the assessment of manufacturing readiness across DOD programs using consistent MRL criteria as basis for measuring, assessing, reporting, and communicating manufacturing readiness and risk on science and technology transition projects and acquisition programs.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: GAO issued a report identifying an overall decline of manufacturing expertise in the workforce across the Department of Defense and the industrial base to conduct the work and perform manufacturing readiness level (MRL) assessments. To address concerns about the Department's diminishing manufacturing workforce that could hamper efforts to implement MRL assessments, GAO recommended the Department of Defense to assess its manufacturing workforce knowledge and skills across the services and agencies and develop a plan to address current and future workforce gaps. In April 2010, the Department reported that each of the services and other defense components were planning and deploying initiatives that support the workforce growth strategy, which includes hiring and retention efforts to mitigate the potential loss in experienced senior-level manufacturing talent and increase the size of the manufacturing workforce. The Secretary established manufacturing career growth targets and the Department reports that growth trends are underway.

    Recommendation: To ensure that DOD is taking steps to strengthen and improve the producibility and manufacturing readiness of technologies, weapon systems, subsystems, or manufacturing processes, the Secretary of Defense should assess the adequacy of the manufacturing workforce knowledge and skills base across the military services and defense agencies and develop a plan to address current and future workforce gaps.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Nov 20, 2014

Nov 19, 2014

Nov 18, 2014

Oct 30, 2014

Oct 16, 2014

Oct 10, 2014

Oct 8, 2014

Sep 30, 2014

Looking for more? Browse all our products here