Information Technology: Consistently Applying Best Practices Could Help IRS Improve the Reliability of Reported Cost and Schedule Information
Highlights
What GAO Found
According to the Internal Revenue Service (IRS), 12 of its 20 major information technology (IT) investments were within 10 percent of cost and schedule estimates or significantly below cost between October 2011 and October 2012. For the remaining 8 investments, 3 were reported as being significantly over cost and 5 were reported as being significantly behind schedule. Reported reasons for these significant variances include unplanned work activities, procurement delays, and additional costs associated with terminating an investment that was being replaced.
The reliability of the reported variance information--which is dependent upon having (1) a reliable cost estimate and a well-constructed and controlled schedule estimate, and (2) a process for determining variances using estimates and comparing them to actual or projected amounts--varied for the seven investments reviewed. Specifically, the cost estimates for CADE 2 and IRDM, and the schedule estimates for CADE 2, IRDM, and ACA were more favorable than for the remaining investments because they were more consistent with best practices.
In addition, regarding IRS's process for determining variances, the agency generally determined investment cost and schedule variances for completed activities with actual amounts--although in about 25 percent of the cases it did not do so within the 60-day time frame specified in the Department of Treasury's guidance. Further, while IRS determined variances using projected cost and schedule for in-process activities, the guidance for doing so did not specify how projected amounts should be determined. This introduces the risk that projected amounts will not consistently reflect best practices and these amounts will therefore remain questionable.
Why GAO Did This Study
IRS relies extensively on IT systems to annually collect more than $2 trillion in taxes, distribute more than $300 billion in refunds, and carry out its mission of providing service to America's taxpayers in meeting their tax obligations. In fiscal year 2012, the agency spent about $2.5 billion for IT. Given the size and significance of IRS's IT investments, and the challenges inherent in successfully delivering these complex IT systems, it is important that Congress be provided ongoing, accurate, and objective information on the progress toward completion and the risks facing these projects.
Accordingly, GAO's objectives, among other things, were to (1) summarize the reported cost and schedule performance for IRS's major IT investments, and (2) for selected investments, determine the reliability of reported cost and schedule variances.
Recommendations
GAO recommends that IRS improve the reliability of reported cost and schedule information by addressing weaknesses in future updates of cost and schedule estimates. GAO also recommends that IRS ensure projects consistently follow guidance for updating performance information 60 days after completion of an activity and develop and implement guidance that specifies best practices to consider when determining projected amounts. IRS agreed with three of GAO's four recommendations and partially disagreed with the fourth recommendation related to guidance on projecting cost and schedule amounts. GAO continues to believe this recommendation is warranted.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Internal Revenue Service | To improve the reliability of reported cost and schedule variance information for the seven major investments we reviewed, the Acting Commissioner of IRS should direct the Chief Technology Officer to improve the reliability of cost estimates by addressing the weaknesses we identified in this report so that each investment at least substantially meets each of the characteristics of a reliable cost estimate. |
We followed up on the status of IRS's actions to address this recommendation for the Customer Account Data Engine (CADE) 2, and the Return Review Program (RRP), the two investments with significant planned expenditures for development in fiscal year 2017, according to data reported on the Federal IT dashboard in June 2017 (the remaining five investments in our 2013 review were primarily in operations and maintenance based on the same IT dashboard data and additional documentation provided by IRS). We selected CADE 2 and RRP because they would benefit most from improvements to cost estimates given their life cycle stage. In the Summer of 2017, IRS provided documentation supporting the CADE 2 Transition State 2 October 2015 Basis of Estimate and the September 15, 2015 RRP Phase 1 independent cost estimate as evidence of actions taken to address the weaknesses we had identified with the CADE 2 and RRP cost estimates. Our assessment showed that CADE 2 at least substantially met each of the two characteristics that we selected for our review. For RRP, our assessment showed that the estimate did not substantially meet the characteristics of a reliable cost estimate as we recommended. Given the fact that RRP Phase I has since been completed, the agency has no plans to implement our recommendation.
|
| Internal Revenue Service | To improve the reliability of reported cost and schedule variance information for the seven major investments we reviewed, the Acting Commissioner of IRS should direct the Chief Technology Officer to improve the extent to which schedules are well-constructed and controlled by addressing the weaknesses we identified in this report so that each investment at least substantially meets each of these characteristics. |
We followed up on the status of IRS's actions to address this recommendation for the Customer Account Data Engine (CADE) 2, and the Return Review Program (RRP), the two investments with significant planned expenditures for development in fiscal year 2017, according to data reported on the Federal IT dashboard in June 2017 (the remaining five investments in our 2013 review were primarily in operations and maintenance based on the same IT dashboard data and additional documentation provided by IRS). We selected CADE 2 and RRP because they would benefit most from improvements to cost estimates given their life cycle stage. In the Summer of 2017, IRS provided documentation supporting the schedule for the Release 3 schedule of CADE 2 and RRP Infrastructure Replacement project schedule as evidence of actions taken to address the weaknesses we had identified with the CADE 2 and RRP schedule estimates. Our assessment showed that CADE 2 at least substantially met each of the two characteristics of a reliable schedule estimate that we selected for our review. For RRP, we shared the results of our assessment with IRS, and, in response, the agency provided an updated estimate. However, our assessment of the updated estimate showed that it did not substantially meet the characteristic of being "controlled" as we recommended. Given the fact that the RRP Infrastructure Project has since been completed, the agency has no plans to implement our recommendation.
|
| Internal Revenue Service | To improve the reliability of reported cost and schedule variance information for the seven major investments we reviewed, the Acting Commissioner of IRS should direct the Chief Technology Officer to ensure projects consistently follow guidelines for updating performance information 60 days after completion of an activity. |
In February 2015, we reported two actions that IRS had taken to address this recommendation (see GAO-15-297). First, the department of Treasury modified the monthly reporting system to automatically calculate a schedule variance based on the current date for any activity where the planned completion date had passed and investment staff had not provided an actual figure within 45 days. This action helped ensure schedule variances were appropriately updated. Second, IRS increased coordination between its Strategy and Planning group (responsible for overseeing monthly variance reporting) and investment staff to ensure cost performance information is appropriately updated. We verified that selected investments had updated performance information in 86 percent of the cases, which was an improvement from the 77 percent we had previously reported. IRS also began disclosing when it was not able to provide timely updates to cost and schedule information within the 60-day time frame established by Treasury in its reports to Congress. The agency began these making these disclosures in the fiscal year 2015 quarterly report dated April 2015.
|
| Internal Revenue Service | To improve the reliability of reported cost and schedule variance information for the seven major investments we reviewed, the Acting Commissioner of IRS should direct the Chief Technology Officer to develop and implement guidance that specifies best practices--such as including evaluating critical path (for projected schedule), using earned value management data, evaluating the performance of completed work and comparing it to the remaining budget, assessing commitment values for material needed to complete remaining work, and estimating future conditions--to consider when determining projected cost and schedule amounts. |
In June 2016, we reported on IRS's development and implementation of its Investment Performance Tool for tracking and improving the reliability of cost, schedule and scope metrics for its IT investments. The October 2015 guidance for the tool shows (which was still in effect as of June 2018) specifies best practices, such as including projected schedule, using earned value management data, evaluating performance of completed work in comparison with budget, assessing commitment values for remaining work and estimating future conditions for projected cost and schedule amounts. We confirmed IRS' use of these practices in our 2016 and 2018 reviews of selected IT investments (see GAO-16-545 and GAO-18-298).
|