Defense Software Acquisitions: Changes to Requirements, Oversight, and Tools Needed for Weapon Programs
Fast Facts
The Department of Defense's ability to respond to evolving threats is increasingly determined by its ability to rapidly develop and deploy software. One way it is doing this is by trying to follow Agile development principles, which deliver working software in less than a year and add capability after that based on user feedback.
The iterative and fast-paced nature of Agile software development relies on flexible requirements, regular user engagement, and an understanding of the value of what has been delivered. DOD, however, has not incorporated these principles across all of its acquisition paths. We recommended it do so.
Highlights
What GAO Found
The Department of Defense's (DOD) ability to respond to evolving threats is increasingly determined by its ability to rapidly develop and deploy software. Efforts to modernize DOD's software acquisition include a transition to Agile development principles from a traditional waterfall approach. Agile is intended to deliver working software to users in less than a year and add capability iteratively based on user needs. In contrast, the waterfall approach could take over 10 years to deliver software and involves greater risk.
Comparison of Waterfall and Agile Methods for Developing Software
In 2020, DOD established six acquisition pathways—or sets of policy and guidance—that are tailored to the type of capabilities being acquired. DOD requires programs on its software pathway to use requirements processes tailored to support Agile development. For example, software pathway programs are to use streamlined requirements documents and develop user agreements, which help ensure programs iteratively develop software aligned with user needs. However, DOD requirements processes used by weapon programs developing software on a different pathway generally do not incorporate Agile principles. By not incorporating Agile principles into requirements processes, these programs risk developing capabilities that may not reflect changing user needs or threats.
DOD developed policies and guidance for oversight of programs on the software pathway, including using metrics and value assessments that measure Agile development outcomes. However, DOD has not issued corresponding policies or guidance for weapon programs using Agile software development on other pathways. As a result, programs on other acquisition pathways, such as those developing new aircraft or ships, may not be positioned to conduct effective oversight of iteratively delivered software capabilities.
The iterative and fast-paced nature of Agile relies on the use of modern engineering tools to help identify needs, develop capabilities, and understand outcomes. DOD, however, has not developed a plan or identified resources to enable the adoption of modern engineering tools across all programs. The absence of a plan prevents DOD from fully realizing the benefits of Agile.
Why GAO Did This Study
Congress included a provision in statute for GAO to examine DOD's software modernization efforts. This report assesses the extent to which DOD has (1) policy and guidance that establish requirements processes to support the use of Agile software development in weapon programs; (2) policy and guidance to provide direction for overseeing Agile software development in weapon programs; and (3) enabled program adoption of modern engineering tools for Agile.
GAO reviewed DOD's software acquisition and requirements policies and guidance, and GAO's leading practices in Agile and iterative development. GAO selected a nongeneralizable sample of five weapon programs based on their use of Agile, and other factors. GAO also interviewed relevant DOD officials, program officials, and user representatives.
Recommendations
GAO is making three recommendations to DOD, including that for all programs using Agile for software development, DOD incorporate Agile principles into policies and guidance used for requirements processes as well as oversight; and that DOD establish an overarching plan—and identify resources—to enable the adoption of modern engineering tools across programs. DOD partially concurred with all three recommendations and identified some planned actions that would address elements of the recommendations. However, GAO noted additional steps DOD needs to take to fully address the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment and the Chairman of the Joint Chiefs of Staff collaborate to incorporate Agile principles into requirements policy and guidance for all programs using Agile for software development. This should include a Capability Needs Statement and User Agreement. (Recommendation 1) |
DOD partially agreed with this recommendation, stating that it would clarify requirements policy to provide guidance on using a Capability Needs Statement and User Agreement for development of software embedded in an already validated requirements document without needing additional requirements validation. In June 2024, DOD noted that it planned to update its requirements policy for embedded software development by December 2024. DOD added that for standalone software development, the Software Initial Capabilities Document provides the requirements validation process for use of Agile software development, and that for programs with minimal hardware development, the Information Systems initial Capabilities Document or Information Systems Capability Development Document provide a requirements process with delegated authority for rapid software development. However, we continue to believe that the requirements processes for such programs would benefit from further incorporation of Agile principles.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment incorporate oversight of Agile development of software into acquisition policy and guidance for all programs using Agile. This should include use of metrics, including outcome-based metrics, and continually assessing the value of capability delivered to support iterative software development. (Recommendation 2) |
DOD partially agreed with this recommendation, stating that it would consider changes within each acquisition pathway to amplify tailoring considerations for programs developing software, and would address any necessary changes through standard processes for updating DOD policy. In September 2023, DOD issued guidance on acquisition data for managing and overseeing acquisition programs, which directed that program performance measures be shared to provide insight into the performance of acquisition programs. In June 2024, DOD noted that the Under Secretary of Defense for Acquisition and Sustainment had directed an assessment of the overall adaptive acquisition framework, which would shape the timing and character of updates to pathway policies. DOD added that it expected to publish policy and guidance updates for the pathways by March 2025.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Research and Engineering, with the input of the military departments, establishes an overarching plan—which identifies associated resources—to enable the adoption of modern engineering tools, across all programs. This should include (1) mission engineering, (2) systems engineering, and (3) software engineering. (Recommendation 3) |
DOD partially agreed with this recommendation, stating that it would develop an overarching plan in conjunction with the military departments, and with the Under Secretary of Defense for Research and Engineering actively supporting implementation efforts through working groups and relevant policy and guidance development. In June 2024, DOD noted that it was beginning the approval process of the draft overarching plan, after adjudication of comments from the military departments, with approval planned for 2024.
|