Best Practices:

Stronger Practices Needed to Improve DOD Technology Transition Processes

GAO-06-883: Published: Sep 14, 2006. Publicly Released: Sep 14, 2006.

Additional Materials:

Contact:

Michael J. Sullivan
(937) 258-7915
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Defense (DOD) relies on its science and technology community to develop innovative technologies for weapon systems, spending $13 billion on basic, applied, and advanced technology research. Several GAO reports have addressed problems in transitioning technologies to the acquisition community. This report, which was prepared under the Comptroller General's authority to conduct evaluations, compares DOD's technology transition processes with commercial best practices. Specifically, GAO identifies technology transition techniques used by leading companies and assesses the extent to which DOD uses the techniques

Leading commercial companies use three key techniques for successfully developing and transitioning technologies, with the basic premise being that technologies must be mature before transitioning to the product line side. (1) Strategic planning at the corporate level: Strategic planning precedes technology development so managers can gauge market needs, identify the most desirable technologies, and prioritize resources. (2) Gated management reviews: A rigorous process is used to ensure a technology's relevancy and feasibility and enlist product line commitment to use the technologies once the labs are finished maturing them. (3)Corroborating tools: To secure commitment, technology transition agreements solidify and document specific cost, schedule, and performance metrics labs need to meet for transition to occur. Relationship managers address transition issues within the labs and product line teams and across both communities. Meaningful metrics gauge project progress and process effectiveness. Not only does DOD lack the breadth and depth of these techniques, the department routinely accepts high levels of technology risk at the start of major weapon acquisition programs. The acquisition community works with technologies before they are ready to be transitioned and takes on responsibility for technology development and product development concurrently. A defined phase for technology transition is not evident. These shortcomings contribute significantly to DOD's poor cost and schedule outcomes. A stark contrast exists between DOD's and private industry's environments for developing technology. The numerous examples of DOD programs that have incurred cost overruns, schedule delays, and reduced performance serve as reminders that inserting a few best practices and changing the mechanics of technology transition processes without changing the environment that determines incentives may not produce better outcomes.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: We believe greater use of tools, such as technology transition agreements, relationship managers, and metrics, could help the department improve its ability to deliver mature technologies when needed, address transition issues more quickly, and gauge the impact of their science and technology investments and lab processes. Therefore, the Secretary of Defense should adopt additional process-oriented metrics, such as the percentage of advanced technologies that--once past milestone A of the acquisition process--transitioned into a weapons program or were fielded and the cycle time from milestone A to milestone B as a way to measure the effectiveness of S&T processes and the impact of science and technology investments.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Air Force has 3 metrics to evaluate how well it is transition technologies: percentage of technology transition plans that are signed within 6 months of initiation; percentage of completed or ongoing advanced technology demonstrations against all that were initiated, and percentage of advanced technology demonstrations that transitioned or are in transition to the end user. The Navy has been tracking process-oriented metrics since fiscal year 2005 for its Future Naval Capabilities program. After fiscal year 2006, the Navy had an independent body review the status of successful S&T transitions from previous years,which a Navy official said addressed our recommendation for more process oriented metrics. The Army has initiated a Lean Six Sigma Black Belt project to establish transition metrics and identify an appropriate frequency of review.

    Recommendation: We believe greater use of tools, such as technology transition agreements, relationship managers, and metrics, could help the department improve its ability to deliver mature technologies when needed, address transition issues more quickly, and gauge the impact of their science and technology investments and lab processes. Therefore, the Secretary of Defense should expand the use of relationship managers by designating people at various levels in both the S&T and acquisition communities to address systemic transition issues and those related to specific weapon system programs, and also define responsibilities for each level of relationship manager.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Air Force has developed a Technology Development and Transition Strategy Guidebook that highlights the best practices program managers and science and technology officials should use to develop and transition technologies to weapons programs. One of those practices is the use of a team of subject-matter experts to collaborate across the development time frame to ensure that activities do not fall through the cracks, and that technologies are eventually transitioned onto an acquisition program. The guidebook points out that an integrated process team consisting of the program manager, technology manager, and additional representatives from acquisition, lab, logistics, contracting, and other communities should be established to collaborate on technology transition efforts. The Navy is continuing to use its science and technology program officers and acquisition program managers to maintain open lines of communication for the majority of its technology projects. Navy officials believe most transition failures result from reprioritization of acquisition funds rather than a breakdown in communications. The Navy has however established a set of deal brokers for its Rapid Technology Transition and Technology Insertion Program for Savings programs. In March 2008 the Army implemented a Lean Six Sigma Black Belt project on technology transition to further increase the efficiency and effectiveness of these processes.

    Recommendation: We believe greater use of tools, such as technology transition agreements, relationship managers, and metrics, could help the department improve its ability to deliver mature technologies when needed, address transition issues more quickly, and gauge the impact of their science and technology investments and lab processes. Therefore, the Secretary of Defense should include additional metrics in technology transition agreements to provide science and technology (S&T) and acquisition program managers demonstrated knowledge about the manufacturing readiness, producibility, other benefits, and risks of including the technology on a weapons program.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Manufacturing Readiness Level Deskbook, issued July 2010, calls for including manufacturing readiness level target in technology transition agreements. Furthermore. the Air Force is developing criteria for assessing technology maturity, including technical performance, manufacturing readiness, integrability, testability, and supportability that can be used to assess and mitigate the risk of problems related to immature technology. It has also identified questions to augment existing technology readiness assessment criteria, identified lessons learned from past assessments, and defined an initial training strategy. The Navy's technology transition agreements have evolved to now include exit criteria, integration strategy, risk mitigation, and estimates of the technology readiness level at transition. When appropriate, a cost target is also included. The Army's agreements include specific technical performance metrics agreed to between the S&T provider and the acquisition program manager.

    Recommendation: We believe greater use of tools, such as technology transition agreements, relationship managers, and metrics, could help the department improve its ability to deliver mature technologies when needed, address transition issues more quickly, and gauge the impact of their science and technology investments and lab processes. Therefore, the Secretary of Defense should expand the use of technology transition agreements to applied and advanced development projects.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In July 2010, DOD issued the Manufacturing Readiness Level Deskbook, as a resource for program officials to use to increase the capability and reliability of weapons systems. The Deskbook states that consideration should be given to manufacturing risks and issues early in technology development and intensify as technologies mature so that manufacturing maturity is sufficient at the time of transition. The Deskbook encourages technology project managers to work with transition customers to identify target manufacturing readiness levels for transition and to include this in a technology transition agreement.

    Recommendation: DOD should take steps to improve its transition of technologies to more efficiently deliver capabilities to its warfighters. DOD's current process lacks specific decision points with "go/no go" decisions. As such, the Secretary of Defense should include specific criteria to support continued funding of specific projects in that process.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Air Force has developed the Technology Development and Transition Strategy Guidebook to provide best practices for technology development and transition to its program managers and science and technology officials. The guidebook identifies 4 stage gates, along with tasks that should be completed for each stage and exit criteria that serve as "go/no go" decisions. The guidebook is available on the Defense Acquisition University's web site. The Navy continues to use its Future Naval Capabilities program as a primary tool for transitioning technology. The process used to oversee technology development and transition, as well as the criteria used to support continued funding, is very similar to the approach used at the time of our review. The Army is awaiting the release of the new modified DODI 5000.2, which will include criteria for specific gates.

    Recommendation: DOD should take steps to improve its transition of technologies to more efficiently deliver capabilities to its warfighters. DOD's current process lacks specific decision points with "go/no go" decisions. As such, the Secretary of Defense should develop a gated process for developing and transitioning technologies that establishes a transition phase and defines activities that should occur during this phase.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Air Force has developed a Technology Development and Transition Strategy Guidebook that highlights the best practices program managers and science and technology officials should use to develop and transition technologies to weapons programs. The guidebook identifies 4 stage gates for technology development, including gates for proof of concept, refinement breadboard, development brassboard, and prototype in relevant environment. It also includes the tasks necessary to complete each stage. The guidebook is available on the Defense Acquisition University website is implementing a gated process for developing and transitioning technologies that will provide stakeholders a common, consistent, and repeatable technology transition process. This includes using scheduled decision point events to assess technology maturity throughout the entire life cycle process of a weapon system. The Navy continues to use its Future Naval Capabilities program as a primary tool for transitioning technology. The process used to oversee technology development and transition is very similar to the approach used at the time of our review. The Army is awaiting the release of the new modified DODI 5000. which is establishing a gated process for developing and transitioning technologies with specific criteria and activities that take place during the transition phases.

    Recommendation: Commercial companies fund technology development and transition activities in their labs and hold the labs accountable for delivering mature technology to their product lines. As such, the Secretary of Defense should set aside a portion of advanced component development and prototype funds for the S&T community to manage the transition of technologies to acquisition programs. For this funding to be used effectively, it will require the discipline provided by corporate leadership in defining priorities, processes, and metrics.

    Agency Affected: Department of Defense

    Status: Closed - Not Implemented

    Comments: None of the services have taken steps to implement this recommendation.

    Jul 31, 2014

    Jul 30, 2014

    Jul 28, 2014

    Jul 17, 2014

    Jul 14, 2014

    Looking for more? Browse all our products here