Skip to main content

Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices, but More Controls Are Needed

GAO-04-53 Published: Nov 10, 2003. Publicly Released: Nov 10, 2003.
Jump To:
Skip to Highlights

Highlights

The Department of Defense's (DOD) investment in new weapon systems is expected to exceed $1 trillion from fiscal years 2003 to 2009. To reduce the risk of cost and schedule overruns, DOD revamped its acquisition policy in May 2003. The policy provides detailed guidance on how weapon systems acquisitions should be managed. The Senate report accompanying the National Defense Authorization Act for Fiscal Year 2004 required GAO to determine whether DOD's policy supports knowledge-based, evolutionary acquisitions and whether the policy provides the necessary controls for DOD to ensure successful outcomes, such as meeting cost and schedule goals. The report also required GAO to assess whether the policy is responsive to certain requirements in the Bob Stump National Defense Authorization Act for Fiscal Year 2003 concerning DOD's management of the acquisition process.

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for program launch (milestone B) should ensure the capture of knowledge about the cost and schedule estimates based on knowledge from a preliminary design using systems engineering tools.
Closed – Implemented
In response to GAO's report, DOD stated that it agreed in principle with the advantages of using knowledge-based controls at key points in the acquisition process to assess risk and ensure readiness to proceed into the next phase of the acquisition process. Title 10 U.S.C. section 2366a, enacted by sec. 801 of the National Defense Authorization Act for fiscal year 2006 (P.L. 109-163, January 2006) and Title 10 section 2366b, enacted by sec. 943 of the National Defense Authorization Act for fiscal year 2008, establish controls to ensure appropriate knowledge is captured prior to beginning system development. Both sections of the law require the Secretary of Defense to certify that major defense acquisition programs have certain knowledge based deliverables and meet certain knowledge based criteria prior to the start of system development at milestone B (section 2366a establishes controls for milestone B, while section 2366b establishes controls for milestone A). We believe that the these certification requirements establish controls that meet the intent of our recommendation. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the completion of 90 percent of engineering drawings.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB)to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will be able to complete 90 percent of their engineering drawings by the design readiness review. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the completion of subsystem and system design reviews.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will capture knowledge from sub-system and system design reviews before transitioning into the system demonstration phase. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the agreement from all stakeholders that drawings are complete and the design is producible.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will be able to obtain agreement from all stakeholders that drawings are complete and the design is producible before transitioning into the system demonstration phase. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the completion of failure modes and effects analysis.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will capture knowledge about failure modes and effects before transitioning into the system demonstration phase. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the identification of key system characteristics.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will capture knowledge about key system characteristics before transitioning into the system demonstration phase. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the identification of critical manufacturing processes.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will identify critical manufacturing processes before transitioning into the system demonstration phase. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for transitioning from system integration to system demonstration (design readiness review) should ensure the capture of knowledge about the reliability targets and a reliability growth plan based on demonstrated reliability rates of components and subsystems.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that programs will capture knowledge system reliability before transitioning into the system demonstration phase. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for the production commitment (milestone C) should ensure the capture of knowledge about the completion of production representative prototypes.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that DOD will be in a position to complete development of production representative prototypes before committing to production - particularly given the department's current prototyping initiative. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for the production commitment (milestone C) should ensure the capture of knowledge about the availability of production representative prototypes to achieve reliability goal and demonstrate the product in an operational environment.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that DOD will be in a position to complete development of production representative prototypes, demonstrate those prototypes in an operational environment, and capture knowledge about system reliability prior to committing to production - particularly given the department's current prototyping initiative. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for the production commitment (milestone C) should ensure the capture of knowledge about the collection of statistical process control data.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L) issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that DOD will capture knowledge about manufacturing processes before committing to production - particularly given the stated intent of department's current prototyping initiative. Therefore, we are closing this recommendation as implemented.
Department of Defense DOD must design and implement necessary controls to ensure that appropriate knowledge is captured and used at critical junctures to make decisions about moving a program forward and investing more money. The Secretary of Defense should require additional controls for capturing knowledge at three key points--program launch, design readiness review for transitioning from system integration to system demonstration, and production commitment. The additional controls for the production commitment (milestone C) should ensure the capture of knowledge about the demonstration that critical manufacturing processes are capable and in statistical control.
Closed – Implemented
DOD has several acquisition reform initiatives and pilot programs that we believe address the intent of our recommendation. In July 2007, the Under Secretary of Acquisition, Technology, and Logistics issued a policy memorandum directing the military departments to establish Configuration Steering Boards (CSB) to control changes in program requirements and technical content after milestone B. In September 2007, the USD (AT&L)issued another policy memorandum that requires all pending and future acquisition programs to have acquisition strategies and funding to provide for two or more competing teams to produce prototypes through milestone B. The stated intent of this prototyping policy is to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. In addition to these two policy memos, DOD is also in the process of establishing metrics that would be used in a knowledge-based acquisition system. In a briefing presented to OMB, DOD identified process metrics and controls, and also identified the information these metrics are expected to produce. Given our discussions with pertinent DOD officials, it is highly likely that these metrics will be implemented into the upcoming revision of DOD's acquisition policy, the DOD 5000 series. We believe that these new policies address the intent of our recommendation by increasing the likelihood that DOD will capture knowledge about manufacturing processes before committing to production - particularly given the stated intent of department's current prototyping initiative. Therefore, we are closing this recommendation as implemented.
Department of Defense Because knowledge about technology, design, and manufacturing at critical junctures can lower DOD's investment risk, decisions that do not satisfy knowledge-based criteria should be visible and justified. Therefore, the Secretary of Defense should document the rationale for any decision to move a program to the next stage of development without meeting the knowledge-based criteria, including those listed in the first recommendation. The responsible milestone decision authority should justify the decision in the program's acquisition decision memorandum and in a report to Congress.
Closed – Implemented
The Department of Defense agreed that they should be accountable for program decisions and appropriately record the considerations in moving from one stage of development to the next. Milestone decision authorities are to employ the decision memorandums to document decisions and their rationale. Title 10 U.S.C. section 2366a, enacted by sec. 801 of the National Defense Authorization Act for fiscal year 2006 (P.L. 109-163, January 2006) and Title 10 section 2366b, enacted by sec. 943 of the National Defense Authorization Act for fiscal year 2008, establish controls to ensure appropriate knowledge is captured prior to beginning system development. Both sections of the law require the Secretary of Defense to certify that major defense acquisition programs have certain knowledge based deliverables and meet certain knowledge based criteria prior to the start of system development at milestone B (section 2366a establishes controls for milestone B, while section 2366b establishes controls for milestone A). We believe that the these certification requirements establish controls that meet the intent of our recommendation. Therefore, we are closing this recommendation as implemented.

Full Report

GAO Contacts

Office of Public Affairs

Topics

Best practicesDecision makingDefense procurementInternal controlsPolicy evaluationProcurement policyReporting requirementsWeapons research and developmentSpace acquisition programsDefense Acquisition