Information Technology Reform: Agencies Need to Increase Their Use of Incremental Development Practices
GAO-16-469
Published: Aug 16, 2016. Publicly Released: Sep 15, 2016.
Skip to Highlights
Highlights
What GAO Found
For fiscal year 2016, 22 agencies reported 64 percent of their software development projects would deliver useable functionality every 6 months on the Information Technology (IT) Dashboard, as required by the Office of Management and Budget (OMB). However, shortcomings with OMB's guidance—the lack of clarity regarding the types of projects where incremental development would not apply and how the status of these non-software projects should be reported—have affected the accuracy of the data on the IT Dashboard.
GAO reviewed 7 departments' software projects and found approximately half of these projects reported delivery of functionality every 6 months. However, there are significant differences in the data reported to GAO and on the IT Dashboard (see figure below) due to inconsistencies in reporting non-software projects, the timing of reporting, and a lack of support for reported delivery, which affects the accuracy of reported rates. Department officials also reported that management and organizational challenges and project complexity and uniqueness impact their ability to deliver incrementally. It is critical that departments continue to improve their use of incremental development to deliver functionality and reduce the risk that these projects will not meet cost, schedule, and performance goals.
Comparison of Software Development Projects' Percentage of Planned Delivery Every Six Months Reported on IT Dashboard and to GAO for Fiscal Year 2016
ªDefense did not provide the requested information in time to verify the information reported to GAO.
Although OMB's requirement has been in place since June 2015, only three departments (Department of Commerce, Department of Homeland Security, and the Department of Transportation) had policies and processes to ensure that the chief information officer would certify IT investments are adequately implementing incremental development. Officials from three departments reported they were updating their existing policies to address certification, but had not yet finalized these efforts, and one department stated its current processes were sufficient.
Why GAO Did This Study
Federal agencies plan to spend more than $89 billion on IT in fiscal year 2017. Historically, long-term system development projects have often produced disappointing results. To help address these issues, OMB now requires agencies to deliver useable functionality every 6 months to reduce risk and deliver capabilities more quickly. GAO's objectives were to (1) describe the number of major IT investments primarily in development identified on the IT Dashboard as reporting the delivery of functionality every 6 months; (2) assess selected departments' and investments' delivery of incremental functionality and determine the factors affecting delivery rates; and (3) assess the quality and completeness of selected departments' plans to employ incremental development practices. GAO analyzed information on the IT Dashboard, analyzed reported project data and policies of 7 selected departments with the most investments primarily in development, and interviewed OMB and department officials.
Recommendations
GAO is making 12 recommendations to 8 agencies to improve reporting of incremental data on the IT Dashboard and policies for CIO certification of adequate incremental development. Five departments agreed with our recommendations, the Department of Defense partially agreed with one and disagreed with another, OMB did not agree or disagree, and the Department of the Treasury did not comment on the recommendations. GAO continues to believe its recommendations are appropriate.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | In order to improve the accuracy of IT Dashboard incremental development data, the Director of OMB should direct the Federal Chief Information Officer (CIO) to clarify existing guidance regarding what IT investments are and are not subject to requirements on the use of incremental development and how CIOs should report the status of projects that are not subject to these requirements. |
The Office of Management and Budget (OMB) has taken initial steps to implement our recommendation. Specifically, OMB's August 2017 annual capital planning guidance for fiscal year 2019 included instructions on what types of investments were required to adhere to incremental development requirements related to the delivery of usable functionality and how CIOs are to report the status of nonsoftware projects. Specifically, the guidance stated that all software development projects must produce usable functionality at intervals of no more than six months and projects that do not involve software development are not required to leverage iterative/agile methodologies. In addition, the guidance noted that all projects within investments are required to use modular development principles. Lastly, the guidance included language in the requirement to report the project's software development life cycle methodology that identified which methodologies were considered software or systems development and also included an option which allowed agencies to indicate the project was not a software development or systems development project. By implementing our recommendation, OMB has helped ensure agencies more accurately report the status of nonsoftware development investment data on the Dashboard.
|
| Department of Homeland Security | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of Homeland Security (DHS) has taken steps to address our recommendation. In April 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, DHS has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of Education | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of Education (Education) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Education has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of Commerce | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of Commerce (Commerce) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Commerce has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of Defense | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of Defense (Defense) partially concurred with our recommendation. In its written response to our draft report, the department stated that it was taking action to update IT Dashboard data as appropriate, but the department asserted that it considered other factors and information from key department stakeholders in rating investments on the IT Dashboard. In addition, the department stated that it does not require that each investment be developed incrementally and provide capability every 6 months. We reported that there was a significant discrepancy in the incremental data reported on the Dashboard versus the data reported to us, noting that on the Dashboard, Defense reported that only 8 percent of its projects delivered functionality in fiscal year 2016 while the projects that we reviewed reported a 63 percent delivery rate to us-a 55 percentage point difference. We also noted that OMB's guidance, clearly states that all projects, regardless of whether they use incremental development principles, must produce usable functionality at least every 6 months. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Defense has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of Health and Human Services | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of Health and Human Services (HHS) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, HHS has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of Transportation | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of Transportation (Transportation) has taken steps to address our recommendation. In June 2018, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key area identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Transportation has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of the Treasury | To improve the quality of the seven departments' information on project incremental delivery reported to the IT Dashboard, the Secretaries of Commerce, Defense, Education, Health and Human Services, Homeland Security, Transportation, and the Treasury should direct their CIOs to review major IT investment project data reported on the IT Dashboard and update the information as appropriate in the following areas: (1) whether the project is in-progress or complete; (2) whether the project is a software development project or not; and (3) the status of the delivery of functionality every 6 months, ensuring that these data are consistent across all reporting channels. |
The Department of the Treasury (Treasury) has taken steps to address our recommendation. In May 2019, a review of the IT Dashboard found that the department had updated its major IT investment project data to address the key areas identified in our report. For example, the IT projects on the IT Dashboard now include whether the project is a software development project and provide information on the status of the project's delivery of incremental functionality. By implementing our recommendation, Treasury has helped to ensure that OMB and other key stakeholders have the most accurate and current information about the department's investments in order to make decisions.
|
| Department of Education | To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act. |
The Department of Education (Education) concurred with, and has taken steps to address, our recommendation. Specifically, in December 2018, Education established a department policy, IT Governance and Investment Management, which describes the Chief Information Officer's (CIO) process for reviewing each major investment in order to certify the adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA. In particular, the department has established an enterprise program management review process in which the CIO reviews every major investment during each phase of the investment lifecycle to ensure investments are adequately using incremental development and producing functionality at intervals of no more than six months. By implementing our recommendation, Education will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of Defense | To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act. |
The Department of Defense (Defense) did not concur with our recommendation. In its written response to our draft report, the department stated that its existing acquisition system includes policy and processes defined in Defense Directive 5000.01 and Defense Instruction 5000.02 regarding the use of incremental development, which was documented in the department's approved FITARA implementation plan. We reported that OMB's guidance clearly states that each department should define processes and policies to ensure that the CIO certifies the use of adequate incremental development for its IT investments, and that both of the department's directives were issued prior to OMB's guidance. In November 2017, Defense issued its financial management regulation 7000.14-R, Volume 2B, Budget Formulation and Presentation (Chapters 4-19), which requires each component CIO to certify that IT investments were adequately implementing incremental development. In addition, the guidance states that the component CIOs are to document the certification in a memorandum using their agency's letterhead and include explanations for any investments which do not conform to department budget guidance. These memos will be submitted to the Defense CIO on an annual basis. By implementing our recommendation, Defense will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of Health and Human Services | To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act. |
The Department of Health and Human Services (HHS) concurred with our recommendation and has taken steps to address it. Specifically, in September 2021, the department established a department policy, HHS Policy for Information Technology Portfolio Management, which describes the Chief Information Officer's (CIO) process for reviewing each major investment in order to certify the adequate use of incremental development, in accordance with OMB's guidance on the implementation of FITARA. In particular, the department established a process for the CIO to review all IT investments during the annual IT portfolio review meeting to ensure that investments are adequately using incremental development. In addition, the HHS CIO will certify the use of adequate incremental development in the IT resource statements submitted to OMB as part of the annual budget submission. By implementing our recommendation, HHS will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
| Department of the Treasury | To improve the certification of adequate incremental development, the Secretaries of Defense, Education, Health and Human Services, and the Treasury should direct their CIOs to establish a department policy and process for the certification of major IT investments' adequate use of incremental development, in accordance with OMB's guidance on the implementation of the Federal Information Technology Acquisition Reform Act. |
The Department of the Treasury (Treasury) has taken steps to address our recommendation. Specifically, in February 2021, the department provided documentation showing that Treasury had established guidance that designated the department CIO with responsibility for meeting the requirements under FITARA, including the CIO's certification of adequate incremental development. In particular, the department CIO reviews project investment information related to use of incremental development on a monthly and annual basis and documents certification in the annual IT resource statement submitted to OMB. By implementing our recommendation, Treasury will be able to help ensure that the department is adequately implementing and benefiting from incremental development practices.
|
Full Report
GAO Contacts
David A. Powner
Topics
Chief information officersSoftwareInformation technologyIT investmentsSoftware developmentProgram evaluationRegulatory agenciesReporting requirementsIT acquisitionsData collectionInvestmentsCompliance oversight