Weapon Systems Annual Assessment: Knowledge Gaps Pose Risks to Sustaining Recent Positive Trends
Fast Facts
We report annually on the programs DOD uses to buy its 86 major weapon systems—which are worth $1.66 trillion. We looked at changes to DOD's weapon system portfolio since our 2017 report, including DOD's progress implementing purchasing reforms.
This year, the outlook is mixed. While newer defense weapon systems have done a better job of staying within budget estimates, many are proceeding without key pieces of knowledge essential to good cost and schedule outcomes. Does this indicate reforms might not be working the way Congress intended? Read more to find out.
Four images show a missile being launched, a Presidential Helicopter, Joint Strike Fighters, and an Air Force control system.
Highlights
This special report, GAO's 16th annual assessment of the Department of Defense's (DOD) $1.66 trillion portfolio of 86 major weapon systems acquisition programs, examines changes in the portfolio since 2016, including DOD's progress implementing acquisition reforms. Drawing from questionnaire data, this report also offers a quick look at the cost, schedule, and performance of 57 individual weapon programs.
This report provides observations on:
1. cost and schedule performance for DOD's 2017 portfolio of 86 programs that provide annual acquisition reports to congress;
2. implementation of acquisition reforms among 57 individual weapon programs not in serial production or with new capabilites; and
3. knowledge that these 57 programs attained at key points in the acquisition process. GAO also makes observations specific to two sets of programs: those initiated since 2010 and before 2010.
Since DOD began to implement acquisition reforms 8 years ago, new defense weapon systems programs have done a better job staying within budget estimates than their predecessors. However, most programs continue to proceed without the key knowledge essential to good acquisition outcomes. As the figure shows, DOD's major acquisition programs proceed through three phases--technology development, system development, and production--that align with three key points for demonstrating knowledge.
Department of Defense (DOD) Acquisition Process
DOD's 2017 portfolio of major weapon programs has grown in cost and size. GAO's analysis shows that programs initiated since 2010 had better cost performance between 2016 and 2017 than the rest of the portfolio-- an estimated $5.6 billion decrease versus a $60.3 billion increase. It is too early to say whether this performance will continue and curb future cost growth. Future cost outcomes hinge on how these programs perform once they enter production, when cost growth is most prevalent. (See figure.)
Programs that implemented acquisition strategies to promote competition, including competitive award of contracts, reported decreases in total acquisition cost estimates as compared to others. In 2010, DOD implemented reforms including some aimed at increasing competition to introduce greater affordability and efficiency. Subsequently, GAO observed that individual programs have taken steps to implement acquisition strategies that promote competition. Of the programs in this year's assessment that awarded development, test, or production contracts, 61 percent did so competitively.
While programs initiated since 2010-- when sweeping acquisition reforms were implemented--have stayed within their cost estimates better than earlier programs, most continue to proceed without the key knowledge essential to good acquisition outcomes. Historically, this has translated to schedule delays, cost growth, and other inefficiencies that have beset DOD programs for years.
As in previous assessments, DOD programs continue to not fully implement knowledge-based acquisition practices. GAO observed that most of the 45 current programs have proceeded into system development, through critical design reviews, and into production without completing key knowledge-based practices associated with each of these three points. (See table.) Further, almost all of the 12 future programs GAO reviewed, not yet in DOD's portfolio, reported that they do not currently plan to fully meet all applicable practices when starting system development.
|
Practices Associated with the Three Key Knowledge Points (KP) |
Thirty-Seven programs GAO previously assessed that had completed the KP |
Eight programs GAO assessed in 2018 that recently completed the KP |
|
Demonstrate all critical technologies are very close to final form, fit, and function within a relevant environment
|
◐ | ● |
|
Demonstrate all critical technologies are in form, fit, and, function within a realistic environment
|
◯ | ● |
|
Completed preliminary design review before system development start
|
◐ | ◯ |
|
Release at least 90 percent of design drawings to manufacturing
|
◯ | ● |
| Test a system-level integrated prototype | ◯ | ◯ |
|
Demonstrate critical manufacturing processes are in statistical control
|
◯ | ◯ |
| Demonstrate critical processes on a pilot production line | ◐ | ● |
|
Test a production-representative prototype in its intended environment
|
◯ | ◐ |
Programs completing each best practice ● 75 - 100 percent ◐ 50 - 74 percent ◯ 0 - 49 percent
This lack of knowledge and the effects it can have throughout a program's acquisition life cycle can increase the risk of undesirable cost and schedule outcomes. Based on GAO's exploratory statistical analysis of 15 programs in production, the major DOD acquisition programs that completed one or more of three specific knowledge-based acquisition practices, among eight key practices GAO evaluated, had significantly lower cost and schedule growth than those that did not. These three practices were (1) demonstration that all critical technologies were very close to final form, fit, and function, within a relevant environment, before starting development; (2) completion of a preliminary design review prior to starting development; and (3) release of at least 90 percent of design drawings by critical design review.