Accuracy Has Improved, and Additional Efforts Are Under Way to Better Inform Decision Making
GAO-12-210: Published: Nov 7, 2011. Publicly Released: Dec 7, 2011.
Each year the federal government spends billions of dollars on information technology (IT) investments. Given the importance of program oversight, the Office of Management and Budget (OMB) established a public website, referred to as the IT Dashboard, that provides detailed information on about 800 federal IT investments, including assessments of actual performance against cost and schedule targets (referred to as ratings). According to OMB, these data are intended to provide both a near-real-time and historical perspective of performance. In the third of a series of Dashboard reviews, GAO was asked to examine the accuracy of the Dashboard's cost and schedule performance ratings. To do so, GAO compared the performance of eight major investments undergoing development from four agencies with large IT budgets (the Departments of Commerce, the Interior, and State, as well as the General Services Administration) against the corresponding ratings on the Dashboard, and interviewed OMB and agency officials.
Since GAO's first report in July 2010, the accuracy of investment ratings has improved because of OMB's refinement of the Dashboard's cost and schedule calculations. Most of the Dashboard's cost and schedule ratings for the eight selected investments were accurate; however, they did not sufficiently emphasize recent performance for informed oversight and decision making. (1) Cost ratings were accurate for four of the investments that GAO reviewed, and schedule ratings were accurate for seven. In general, the number of discrepancies found in GAO's reviews has decreased. In each case where GAO found rating discrepancies, the Dashboard's ratings showed poorer performance than GAO's assessment. Reasons for inaccurate Dashboard ratings included missing or incomplete agency data submissions, erroneous data submissions, and inconsistent investment baseline information. In all cases, the selected agencies found and corrected these inaccuracies in subsequent Dashboard data submissions. Such continued diligence by agencies to report complete and timely data will help ensure that the Dashboard's performance ratings are accurate. In the case of the General Services Administration, officials did not disclose that performance data on the Dashboard were unreliable for one investment because of an ongoing baseline change. Without proper disclosure of pending baseline changes, OMB and other external oversight bodies may not have the appropriate information needed to make informed decisions. (2) While the Dashboard's cost and schedule ratings provide a cumulative view of performance, they did not emphasize current performance--which is needed to meet OMB's goal of reporting near-real-time performance. GAO's past work has shown cost and schedule performance information from the most recent 6 months to be a reliable benchmark for providing a near-real-time perspective on investment status. By combining recent and historical performance, the Dashboard's ratings may mask the current status of the investment, especially for lengthy acquisitions. GAO found that this discrepancy between cumulative and current performance ratings was reflected in two of the selected investments. For example, a Department of the Interior investment's Dashboard cost rating indicated normal performance from December 2010 through March 2011, whereas GAO's analysis of current performance showed that cost performance needed attention for those months. If fully implemented, OMB's recent and ongoing changes to the Dashboard, including new cost and schedule rating calculations and updated investment baseline reporting, should address this issue. These Dashboard changes could be important steps toward improving insight into current performance and the utility of the Dashboard for effective executive oversight. GAO plans to evaluate the new version of the Dashboard once it is publicly available in 2012. GAO is recommending that the General Services Administration disclose on the Dashboard when one of its investments is in the process of a rebaseline. Since GAO previously recommended that OMB improve how it rates investments relative to current performance, it is not making further recommendations. The General Services Administration agreed with the recommendation. OMB provided technical comments, which GAO incorporated as appropriate.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: The General Services Administration (GSA) agreed with our recommendation and GSA's Administrator stated that the agency would take appropriate action to implement it. Subsequently, as we recommended, GSA complied with the Office of Management and Budget's (OMB) guidance related to submissions on the information technology (IT) Dashboard. Specifically, GSA now updates the chief information officer (CIO) ratings and CIO comments field for each major GSA investment on a monthly basis and as new information becomes available that affects the CIO's assessment. For example, in April 2012, GSA updated the CIO comments section for the Human Capital Information Technology Services investment to state that the investment was in the process of a rebaseline. Additionally, in July 2012, GSA updated the CIO comments for the Regional Business Application investment to state that a baseline change request for the investment was submitted in June 2012. Further, between January 2013 and July 2013, GSA updated its investment ratings every 30 days, on average. By updating its CIO ratings in accordance with OMB requirements, GSA has increased the transparency of the performance of the agency's investments and, in doing so, allows users of the IT Dashboard to hold GSA accountable for investment results and progress.
Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Administrator of the General Services Administration (GSA) should direct its Chief Information Officer (CIO) to comply with OMB's guidance related to Dashboard data submissions by updating the CIO rating for a given GSA investment as soon as new information becomes available that affects the assessment, including when an investment is in the process of a rebaseline.
Agency Affected: General Services Administration