IRS Management: Cost Estimate for New Information Reporting System Needs to be Made More Reliable
Highlights
What GAO Found
The 2011 Information Reporting and Document Matching (IRDM) cost estimate, used to justify the program’s projected budgets of $115 million for fiscal years 2012 through 2016, generally does not meet best practices for reliability. The cost estimate did not fully meet any of the four best practices for a reliable cost estimate.
For example, the cost estimate minimally meets best practices for a well documented estimate because the Internal Revenue Service (IRS) did not provide detailed support for staff resources, and the cost estimate documentation only justified about 6 out of the 86 requested full time equivalent staff for IRDM, among other things. If documentation does not provide source data or cannotexplain the calculations underlying the cost elements, the estimate’s credibility may suffer. Although IRS has an independent office of cost estimators that can develop and update cost estimates using cost modeling software that generally follows GAO’s best practices, this office did not develop the 2011 IRDM cost estimate. IRS policy does not require project teams to work with the office to update cost estimates. Additionally, IRS’s cost estimation guidance for project managers is inconsistent regarding how cost estimates should be related to a budget, an inconsistency that could lead to budget requests that do not accurately estimate program funding needs.
The IRDM program’s earned value management (EVM) data did not meet data reliability criteria in the areas GAO reviewed. For example, the IRDM project schedule was not properly sequenced—meaning activities were not properly linked in the order in which they are to be carried out. In addition, surveillance was not conducted on IRDM’s EVM system, as required by the Office of Management and Budget and the Department of the Treasury. Surveillance involves having qualified staff review an EVM system. Because IRDM’s 2011 cost estimate is based on unreliable EVM data, it does not provide adequate support for IRDM’s budget requests. Until IRS addresses deficiencies in the EVM data, it cannot provide a reliable cost estimate for IRDM.
Why GAO Did This Study
The Internal Revenue Service (IRS) began developing the Information Reporting and Document Matching (IRDM) program in fiscal year 2009 to enhance IRS’s ability to automatically compare different sources of tax information and thus improve its capacity to identify and address taxpayer noncompliance. GAO’s May 2011 report recommended that IRS follow best practices from the GAO’s Cost Estimating and Assessment Guide if IRS updated the cost estimate for building IRDM systems. IRS provided a new cost estimate for IRDM in August 2011. In this report, GAO assessed the extent to which (1) the IRDM funding request is supported by a reliable cost estimate and, if not reliably supported, why not; and (2) IRS’s practices for capturing data on IRDM’s actual costs and comparing them to estimated costs—known as earned value management (EVM)—generate reliable performance data. GAO compared IRS’s 2011 IRDM cost estimate to criteria in GAO’s cost guide and analyzed IRDM’s earned value management data.
Recommendations
GAO recommends that IRS ensure that IRDM has a reliable cost estimate, require certain project teams to work with its Estimation Program Office, improve cost estimation guidance, and improve the reliability of IRDM’s EVM data. IRS agreed with one, partially agreed with one, and disagreed with two of GAO’s recommendations. GAO generally disagrees with IRS’s concerns, and still believes the recommendations have merit.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Internal Revenue Service | To improve the quality of cost and budget information for IRS IT projects, the Commissioner of Internal Revenue should ensure that the IRDM life-cycle cost estimate is reliable and that budget requests are justified by a reliable cost estimate that follows best practices. |
Internal Revenue Service (IRS) officials said that the Information Reporting Document Matching (IRDM) cost estimate was not updated because the program was scheduled to exit the development phase shortly after of the issuance GAO's recommendation. As of June 2014, the last cost estimate that IRS's Estimation Program Office prepared for IRDM was an estimate for IRDM release two, completed in August 2012. In an April 2013 report, GAO found that the IRDM release two estimate met three of the four characteristics of a reliable cost estimate, see GAO-13-401, Information Technology: Consistently Applying Best Practices Could Help IRS Improve the Reliability of Reported Cost and Schedule Information.
|
| Internal Revenue Service | To improve the quality of cost and budget information for IRS IT projects, the Commissioner of Internal Revenue should require project managers to consult with Estimation Program Office (EPO) to determine if projects could benefit from EPO assistance in updating cost estimates for programs that exceed thresholds recommended by IRS's Modernization and Information Technology Services' (MITS) "Estimation Procedures" document. For those projects where EPO does not update the cost estimate, IRS should require that the decision and rationale be documented. EPO should use the information from updated cost estimates to develop a historical repository of cost estimation data. |
In June 2013, IRS released guidance requiring the lead estimator or manager for a project to submit planned and actual costs for most projects to the newly developed Estimate Repository for Estimated and Actual Results. At that time, IRS also released an aggregate analysis of the scope, size, effort, schedule and cost characteristics for 16 completed information technology programs and projects. The analysis contained a compendium, based on the repository, of specific program and project results to aid in budgetary and other resource planning. Further, IRS's Estimation Procedure document specifies that a Lead Estimator from IRS's Estimation Program Office will be assigned to major new development/modernization/enhancement investment proposals and major releases.
|
| Internal Revenue Service | To improve the quality of cost and budget information for IRS IT projects, the Commissioner of Internal Revenue should review all guidance applicable to cost estimates and take steps to ensure that they are consistent. As a first step, IRS guidance should require the use of current and reliable project cost estimates to inform budget requests, in accordance with the "Estimator's Reference Guide." |
The Internal Revenue Service (IRS) updated their cost estimation guidance and, in fiscal year 2016, developed additional guidance documents. The documents provide improved consistency and provide additional guidance on cost estimates for budget requests and legislative proposals. The guidance describes an estimation technique, which GAO's Cost Estimating and Assessment Guide states is best used for programs early in the life cycle--that can help IRS develop reasonable cost estimates for budget requests, while making efficient use of IRS estimation staff. In addition, IRS developed a calculator to convert an estimate of expenditures and disbursements, from the project's cost estimate, to also include commitments and obligations, which are needed for budget submissions. This will allow cost estimates to include the higher costs from commitments and obligations that accrue once a project has begun. Providing managers with a tool to incorporate new cost information into budget requests will help requests to better reflect current and accurate funding needs for a project.
|
| Internal Revenue Service | To improve the quality of cost and budget information for IRS IT projects, the Commissioner of Internal Revenue should improve the reliability of IRDM's EVM data, specifically: (1) address work breakdown structures (WBS) issues by developing an EVM baseline for IRDM that reflects the same WBS as the detailed schedule and IRDM cost estimate; (2) address sequencing issues and enable the development of a time-phased budget baseline by creating a single integrated master schedule for IRDM that is properly sequenced and resource-loaded so that effective and meaningful EVM data can be obtained to better manage the program; (3) conduct an independent baseline validation for the IRDM EVM baseline; and (4) conduct independent surveillance of EVM systems to ensure that data are reliable. |
The Internal Revenue Service (IRS) disagreed with this recommendation in our original report. As of August 2015, IRS does not have plans to implement this recommendation.
|