Information Technology:

OMB Has Made Improvements to Its Dashboard, but Further Work Is Needed by Agencies and OMB to Ensure Data Accuracy

GAO-11-262, Mar 15, 2011

Additional Materials:

Contact:

David A. Powner
(202) 512-9286
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Each year the federal government spends billions of dollars on information technology (IT) investments. Given the importance of oversight, the Office of Management and Budget (OMB) established a public Web site, referred to as the IT Dashboard, that provides detailed information on about 800 federal IT investments, including assessments of actual performance against cost and schedule targets (referred to as ratings). In the second of a series of Dashboard reviews, GAO was asked to (1) determine OMB's efforts to improve the Dashboard and how it is using data from the Dashboard, and (2) examine the accuracy of the Dashboard's cost and schedule performance ratings. To do so, GAO analyzed documentation on OMB oversight efforts and Dashboard improvement plans, compared the performance of 10 major investments from five agencies with large IT budgets against the ratings on the Dashboard, and interviewed OMB and agency officials.

Since GAO's first review, in July 2010, OMB has initiated several efforts to increase the Dashboard's value as an oversight tool, and has used the Dashboard's data to improve federal IT management. These efforts include streamlining key OMB investment reporting tools, eliminating manual monthly submissions, coordinating with agencies to improve data, and improving the Dashboard's user interface. Recent changes provide new views of historical data and rating changes. OMB anticipates that these efforts will increase the reliability of the data on the Dashboard. To improve IT management, OMB analysts use Dashboard data to track investment changes and identify issues with performance. OMB officials stated that they use these data to identify poorly performing IT investments for review sessions by OMB and agency leadership. OMB reported that these sessions and other management reviews have resulted in a $3 billion reduction in life-cycle costs, as of December 2010. While the efforts above as well as initial actions taken to address issues GAO identified in its prior review--such as OMB's updated ratings calculations to factor in ongoing milestones to better reflect current status--have contributed to data quality improvements, performance data inaccuracies remain. The ratings of selected IT investments on the Dashboard did not always accurately reflect current performance, which is counter to the Web site's purpose of reporting near real-time performance. Specifically, GAO found that cost ratings were inaccurate for six of the investments that GAO reviewed and schedule ratings were inaccurate for nine. For example, the Dashboard rating for a Department of Homeland Security investment reported significant cost variances for 3 months in 2010; however, GAO's analysis showed lesser variances from cost targets for the same months. Conversely, a Department of Transportation investment was reported as on schedule on the Dashboard, which does not reflect the significant delays GAO has identified in recent work. These inaccuracies can be attributed to weaknesses in how agencies report data to the Dashboard, such as providing erroneous data submissions, as well as limitations in how OMB calculates the ratings. Until the selected agencies and OMB resolve these issues, ratings will continue to often be inaccurate and may not reflect current program performance. GAO is recommending that selected agencies take steps to improve the accuracy and reliability of Dashboard information and OMB improve how it rates investments relative to current performance and schedule variance. Agencies generally concurred with the recommendations; OMB did not concur with the first recommendation but concurred with the second. GAO maintains that until OMB implements both, performance may continue to be inaccurately represented on the Dashboard.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Homeland Security should direct the CIO to ensure that investment data submissions include complete and accurate investment information for all required fields.

    Agency Affected: Department of Homeland Security

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Homeland Security should direct the CIO to comply with OMB's guidance on updating the CIO rating as soon as new information becomes available that affects the assessment of a given investment, including when an investment is in the process of a rebaseline.

    Agency Affected: Department of Homeland Security

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Homeland Security should direct the CIO to work with C4ISR officials to comply with OMB's guidance on updating investment cost and schedule data on the Dashboard at least monthly.

    Agency Affected: Department of Homeland Security

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Transportation should direct the CIO to work with Automatic Dependent Surveillance-Broadcast officials to comply with OMB's guidance on updating investment cost and schedule data on the Dashboard at least monthly.

    Agency Affected: Department of Transportation

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of the Treasury should direct the CIO to comply with OMB's guidance on updating the CIO rating as soon as new information becomes available that affects the assessment of a given investment, including when an investment is in the process of a rebaseline.

    Agency Affected: Department of the Treasury

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of the Treasury should direct the CIO to work with Modernized e-File officials to report accurate actual percent complete data for each of the investment's activities.

    Agency Affected: Department of the Treasury

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of the Treasury should direct the CIO to work with Payment Application Modernization officials to disclose the extent of this investment's data reliability issues in the CIO rating assessment on the Dashboard.

    Agency Affected: Department of the Treasury

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Veterans Affairs should direct the CIO to comply with OMB's guidance on updating the CIO rating as soon as new information becomes available that affects the assessment of a given investment, including when an investment is in the process of a rebaseline.

    Agency Affected: Department of Veterans Affairs

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Veterans Affairs direct the CIO to work with Medical Legacy officials to comply with OMB's guidance on updating investment cost and schedule data on the Dashboard at least monthly.

    Agency Affected: Department of Veterans Affairs

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Secretary of the Department of Veterans Affairs direct the CIO to ensure Medical Legacy investment data submitted to the Dashboard are consistent with the investment's internal performance information.

    Agency Affected: Department of Veterans Affairs

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides accurate cost and schedule performance ratings, the Commissioner of the Social Security Administration should direct the CIO to ensure that data submissions to the Dashboard include accurate investment information for all required fields.

    Agency Affected: Social Security Administration

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides meaningful ratings and reliable investment data, the Director of OMB should direct the Federal CIO to develop cost and schedule rating calculations that better reflect current investment performance.

    Agency Affected: Executive Office of the President: Office of Management and Budget

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To better ensure that the Dashboard provides meaningful ratings and reliable investment data, the Director of OMB should direct the Federal CIO to update the Dashboard's schedule calculation for in-progress activities to more accurately represent the variance of ongoing, overdue activities.

    Agency Affected: Executive Office of the President: Office of Management and Budget

    Status: Review Pending

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.