Information Technology: Agencies Need to Establish and Implement Incremental Development Policies
Highlights
What GAO Found
All five agencies in GAO's review—the Departments of Defense (Defense), Health and Human Services (HHS), Homeland Security (DHS), Transportation (Transportation), and Veterans Affairs (VA)—have established policies that address incremental development; however, the policies usually did not fully address three key components for implementing the Office of Management and Budget's (OMB) guidance (see table). Specifically, only VA fully addressed the three components. Among other things, agencies cited the following reasons that contributed to these weaknesses: (1) the guidance was not feasible because not all types of investments should deliver functionality in 6 months, and (2) the guidance did not identify what agencies' policies are to include or time frames for completion. GAO agrees these concerns have merit. Until OMB issues realistic and clear guidance and agencies address the weaknesses in their incremental development policies, it will be difficult to deliver project capability more rapidly.
Assessment of Selected Agencies' Incremental Development Policies
|
Component |
Defense |
HHS |
DHS |
Transportation |
VA |
|
Require delivery of functionality every 6 months |
○ |
○ |
○ |
○ |
● |
|
Define functionality |
◐ |
○ |
○ |
○ |
● |
|
Define a process for enforcing compliance |
◐ |
◐ |
○ |
○ |
● |
Key ●=Fully met ◐=Partially met ○=Not met
Source: GAO analysis of agency documentation.
The weaknesses in agency policies have enabled inconsistent implementation of incremental development approaches: almost three-quarters of the selected investments did not plan to deliver functionality every 6 months, and less than half planned to deliver functionality in 12-month cycles (see table). Without consistent use of incremental development approaches, information technology (IT) expenditures are more likely to continue producing disappointing results.
Number of Selected Investments Planning to Incrementally Deliver Functionality
|
Agency |
Total number of selected investments |
Investments planning to deliver functionality every 6 months |
Investments planning to deliver functionality every 12 months |
|
Defense |
37 |
1 |
11 |
|
HHS |
14 |
9 |
11 |
|
DHS |
12 |
2 |
6 |
|
Transportation |
20 |
5 |
7 |
|
VA |
6 |
6 |
6 |
|
Totals |
89 |
23 |
41 |
Source: GAO analysis of agency data.
Why GAO Did This Study
Federal agencies plan to spend at least $82 billion on IT in fiscal year 2014. However, prior IT expenditures have often produced disappointing results. Thus, OMB has called for agencies to deliver investments in smaller parts or increments. In 2010, it called for IT investments to deliver capabilities every 12 months and now requires investments to deliver capabilities every 6 months. GAO was asked to review agencies' incremental development approaches. Among other things, this report (1) assesses whether selected agencies have established policies for incremental IT development; and (2) determines whether selected agencies are using incremental development approaches to manage their IT investments. To do so, GAO selected five agencies—Defense, HHS, DHS, Transportation, and VA—and 89 total investments at these agencies. GAO then reviewed the agencies' incremental development policies and plans.
Recommendations
Among other things, GAO recommends that OMB develop and issue realistic and clear guidance on incremental development and that the selected agencies update and implement their incremental development policies to reflect OMB's guidance. OMB partially disagreed, believing its guidance is realistic. Four agencies generally agreed with the report or had no comments, and one agency did not agree that its recommendations should be dependent on OMB first taking action. GAO continues to believe that its recommendations are valid, as discussed in this report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | The Director of the Office of Management and Budget should direct the Federal Chief Information Officer to update, and clearly and explicitly issue incremental development guidance that addresses the following three components: (1) requires projects associated with major IT investments to deliver incremental functionality at least every 12 months, with the exception of the three types of investments identified in this report; (2) specifies how agencies are to define the project functionality that is to be delivered; and (3) requires agencies to define a process for enforcing compliance with incremental functionality delivery, such as the use of TechStat sessions. |
The Office of Management and Budget (OMB) has taken steps to address our recommendation. Specifically, in June 2014, OMB updated its information technology (IT) budget guidance to, among other things, define project functionality as any changes to an IT system that primarily provide new or improved capability to the end user. Additionally, in June 2015, OMB issued its guidance on how agencies are to implement December 2014 federal IT acquisition reform legislation. As part of that guidance, OMB required CIOs to ensure that all acquisition strategies and acquisition plans that include IT apply adequate incremental development principles. Lastly, in August 2017, OMB issued its fiscal year 2019 IT budget guidance, which included language that clearly and explicitly laid out expectations for agencies to utilize incremental development practices for their investments. Specifically, OMB's guidance: (1) requires agencies to structure major planning and acquisition into useful segments with a narrow scope and brief duration, which must be approved by the agency CIO; (2) includes requirements for agencies to provide information related to the project's use of iterative (incremental) development techniques; and (3) requires agency CIOs to certify the adequate use of incremental development as part of submitted IT resource statements. While this new guidance does not address a requirement for projects to deliver functionality every twelve months, as we recommended, it does provide additional structure and requirements around agencies' use of incremental development. In strengthening its guidance related to incremental development in the areas we identified, OMB has helped ensure that agency investments deliver project functionality more rapidly and created mechanisms to measure delivered functionality and enforce compliance with its guidance.
|
| Office of Management and Budget | The Director of the Office of Management and Budget should direct the Federal Chief Information Officer to require agencies to clearly identify on exhibit 300 submissions whether, for each project, functionality will be delivered within the time frames called for by this incremental development guidance, and to provide justification for projects that do not plan to do so. |
The Office of Management and Budget (OMB) agreed with, and has taken steps to implement, this recommendation. Specifically, since June 2014, OMB has required as part of its annual information technology (IT) budget guidance that agencies identify on exhibit 300 submissions whether each project associated with major IT investments will deliver functionality every 6 months. OMB has also made this information publicly available on the IT Dashboard. Additionally, OMB's guidance calls for agencies to provide a justification for projects that do not plan to deliver functionality every 6 months in their exhibit 300 submissions. As a result, OMB should be better able to oversee the extent to which projects and investments are implementing its incremental development guidance.
|
| Department of Homeland Security | The Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should modify, finalize, and implement their agencies' policies governing incremental development to ensure that those policies comply with OMB's guidance, once that guidance is made available. |
The Department of Homeland Security (DHS) agreed with our recommendation and stated that it would finalize its guidance governing incremental development to ensure it complied with OMB guidance. In April 2016, DHS finalized its guidance on Agile Development and Delivery for Information Technology, which included time frames for when a project should deliver useable functionality, a definition of useable functionality, and a process for enforcing compliance with incremental delivery through its acquisition review process. By finalizing its incremental development guidance to include these three components, DHS is better positioned to deliver functionality more rapidly, measure how often projects are delivering functionality, and enforce compliance with the time frames identified in their guidance.
|
| Department of Defense | The Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should modify, finalize, and implement their agencies' policies governing incremental development to ensure that those policies comply with OMB's guidance, once that guidance is made available. |
The Department of Defense (Defense) has taken steps to address our recommendation. Specifically, in January 2015, Defense finalized Department of Defense Instruction 5000.02: Operation of the Defense Acquisition System, which calls for information technology (IT) investments to use incremental development approaches. Subsequently, in August 2016, the department issued its Fiscal Year 2018 IT Budget Guidance, which included a definition of useable functionality, time frames for delivery, and a process for enforcing compliance with incremental delivery through its process for Chief Information Officer certification of an investment's adequate incremental development. In addition, the department issued its Department of Defense Instruction 5000.75: Business System Requirements and Acquisition in February 2017, which addresses the use of incremental development for IT business systems. By finalizing its incremental development guidance to include these three components, Defense is better positioned to deliver functionality more rapidly, measure how often projects are delivering functionality, and enforce compliance with the time frames identified in their guidance.
|
| Department of Health and Human Services | The Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should modify, finalize, and implement their agencies' policies governing incremental development to ensure that those policies comply with OMB's guidance, once that guidance is made available. |
The Department of Health and Human Services (HHS) has taken steps to address our recommendation. Specifically, in December 2016, HHS finalized its Policy for Information Technology (IT) Enterprise Performance Life Cycle, which includes a definition of useable functionality, time frames for delivery at least every 6 months, and a process for enforcing compliance by performing investment reviews at appropriate stages as well as periodic audits of investments to ensure investments are adhering to the guidance. By finalizing its incremental development guidance to include these three components, HHS is better positioned to deliver functionality more rapidly, measure how often projects are delivering functionality, and enforce compliance with the time frames identified in their guidance.
|
| Department of Transportation | The Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should modify, finalize, and implement their agencies' policies governing incremental development to ensure that those policies comply with OMB's guidance, once that guidance is made available. |
The Department of Transportation (Transportation) has taken steps to address our recommendation. Specifically, in June 2016, the department issued revised guidance on incremental development. Transportation's Enterprise Program Management Review Framework includes time frames for when a project should deliver useable functionality, a definition of useable functionality, and a process for enforcing compliance with incremental delivery through its investment management process. By finalizing its incremental development guidance to include these three components, Transportation is better positioned to deliver functionality more rapidly, measure how often projects are delivering functionality, and enforce compliance with the time frames identified in their guidance.
|
| Department of Homeland Security | When updating their policies, the Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should consider the factors identified in this report as enabling and inhibiting incremental development. |
The Department of Homeland Security (DHS) agreed with our recommendation and stated that it would update its guidance to encourage the factors identified in the report as enabling incremental development and would include strategies to minimize factors inhibiting this development. In April 2016, DHS finalized its guidance on Agile Development and Delivery for Information Technology, which addresses the factors identified in our report. For example, DHS's revised policy establishes Agile development as the preferred development approach for all IT acquisitions, requires active engagement with all stakeholders throughout the acquisition process, and ensures staff have the necessary skills and experience in Agile development and contracting. By finalizing its incremental development guidance to address these factors, DHS is better positioned to address the challenges the department faces in acquiring IT investments faster and more efficiently.
|
| Department of Defense | When updating their policies, the Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should consider the factors identified in this report as enabling and inhibiting incremental development. |
The Department of Defense (Defense) has taken steps to address our recommendation. Specifically, in February 2017, Defense finalized Department of Defense Instruction 5000.75: Business System Requirements and Acquisition, which addresses the factors identified in our report. For example, Defense's guidance addresses the use of incremental development for IT business systems, lays out roles and responsibilities for the development, management, and testing of requirements to ensure they are stable and aligned to business needs, and requires active engagement with all stakeholders throughout the process. In addition, the guidance encourages program managers to apply commercial best practices and lessons learned with incremental development to the development and deployment of these systems. By finalizing its guidance to address these factors, Defense is better positioned to address the challenges the department faces in acquiring IT investments faster and more efficiently.
|
| Department of Health and Human Services | When updating their policies, the Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should consider the factors identified in this report as enabling and inhibiting incremental development. |
The Department of Health and Human Services (HHS) has taken steps to address our recommendation. Specifically, in December 2016, HHS finalized its Policy for Information Technology (IT) Enterprise Performance Life Cycle, which addresses factors identified in our report. For example, HHS's guidance addresses the use of incremental development for IT projects, lays out roles and responsibilities for active participation with stakeholders throughout the process, and promotes the development and sharing of cost and schedule estimating best practices. In addition, the guidance encourages program managers to apply commercial best practices and lessons learned with incremental development to the development and deployment of these systems. By finalizing its guidance to address these factors, HHS is better positioned to address the challenges the department faces in acquiring IT investments faster and more efficiently.
|
| Department of Transportation | When updating their policies, the Secretaries of Defense, Health and Human Services, Homeland Security, and Transportation should consider the factors identified in this report as enabling and inhibiting incremental development. |
The Department of Transportation (Transportation) has taken steps to address our recommendation. Specifically, the department issued two sets of guidance on incremental development in June 2016. Transportation's Enterprise Program Management Review Framework and Investment Management Process Guidance address the factors identified in our report. For example, Transportation's revised guidance requires all projects to use incremental development such as Agile or provide justification for an alternate approach, requires active engagement with all stakeholders throughout its investment review process, and establishes an annual operational analysis to measure investment performance against established cost, schedule, and performance parameters and share best practices departmentwide. By finalizing its incremental development guidance to address these factors, Transportation is better positioned to address the challenges the department faces in acquiring IT investments faster and more efficiently.
|
| Department of Veterans Affairs | The Secretary of Veterans Affairs should consider incorporating the factors identified in this report as enabling and inhibiting incremental development in the department's related policy. |
The Department of Veterans Affairs (VA) agreed with our recommendation and stated that it would review its existing policy for incremental development and consider incorporating the factors we identified. Subsequently, in June 2014, VA revised its incremental development guidance to address all of the factors identified in our report. For example, VA's revised policy requires information technology (IT) programs to use contracting strategies that increase flexibility, use key technologies to accelerate development work, and implement risk management best practices. By updating its incremental development policy to address these factors, VA is better positioned to address the challenges the department faces in acquiring IT investments faster and more efficiently.
|