IT Dashboard: Agencies Need to Fully Consider Risks When Rating Their Major Investments
GAO-16-494
Published: Jun 02, 2016. Publicly Released: Jun 02, 2016.
Skip to Highlights
Highlights
What GAO Found
Agencies determined investments' Chief Information Officer (CIO) ratings using a variety of processes, which included the Office of Management and Budget's (OMB) six suggested factors (including risk management, requirements management, and historical performance). Specifically, all 17 selected agencies incorporated at least two of OMB's factors into their risk rating processes and 9 used all of the factors. However, agencies' interpretations of these factors varied. For example, most agencies considered active risks, such as funding cuts or staffing changes, when rating investments, but others only evaluated compliance with the agency's risk management processes. Further, 13 agencies required monthly updates to CIO ratings as does OMB (as of June 2015), 1 agency scheduled its reviews based on risk, and 3 agencies required updates less often than on a monthly basis.
GAO's assessments generally showed more risk than the associated CIO ratings. In particular, of the 95 investments assessed, GAO's assessments matched the CIO ratings 22 times, showed more risk 60 times, and showed less risk 13 times (see graphic).
Comparison of Selected Investments' Chief Information Officer Ratings to GAO Assessments
Aside from the inherent judgmental nature of risk ratings, three issues contributed to these differences:
Forty of the 95 CIO ratings were not updated during the month GAO reviewed, which led to more differences between GAO's assessments and the CIOs' ratings. This underscores the importance of frequent rating updates, which help to ensure that the information on the Dashboard is timely and accurately reflects recent changes to investment status.
Three agencies' rating processes span longer than 1 month. Longer processes mean that CIO ratings are based upon older data and may not reflect the current level of investment risk.
Seven agencies' rating processes did not focus on active risks. According to OMB's guidance, CIO ratings should reflect the CIO's assessment of the risk and the investment's ability to accomplish its goals. CIO ratings that do not incorporate active risks increase the chance that ratings overstate the likelihood of investment success.
Why GAO Did This Study
Although the government spends more than $80 billion in information technology (IT) annually, many of the investments have failed or have been troubled. In December 2014, provisions commonly referred to as the Federal Information Technology Acquisition Reform Act (FITARA) were enacted. Among other things, FITARA states that OMB shall make available to the public a list of each major IT investment including data on cost, schedule, and performance. OMB does so via the Federal IT Dashboard—its public website that reports on major IT investments, including ratings from CIOs which should reflect the level of risk facing an investment.
GAO's objectives were to (1) describe agencies' processes for determining CIO risk ratings for major federal IT investments primarily in development and (2) assess the risk of federal IT investments and analyze any differences with the investments' CIO risk ratings. To do so, GAO selected major IT investments with at least 80 percent of their fiscal year 2015 budget allocated to development (resulting in 95 investments across 15 agencies) and compared CIO rating processes to OMB guidance. GAO also analyzed data on those investments to create its own risk assessments.
Recommendations
GAO is making 25 recommendations to 15 agencies to improve the quality and frequency of CIO ratings. Twelve agencies generally agreed with or did not comment on the recommendations and three agencies disagreed, stating their CIO ratings were adequate. GAO continues to believe these recommendations are valid.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Agriculture | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of Agriculture (Agriculture) agreed with the recommendation and, subsequently, in November 2017, implemented new CIO rating scoring criteria that incorporate active risks. Specifically, Agriculture's updated scoring criteria include an evaluation of the management and risk exposure scores of active risks. As a result, Agriculture's CIO ratings on the Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of Education | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of Education (Education) agreed with the recommendation and, in September 2019, the department's Office of the Chief Information Officer (CIO) issued updated guidance that incorporates active risks into the CIO ratings process. Specifically, the CIO's guidance states that Education assesses individual active risks in order to determine how investments are ultimately reviewed and scored. For instance, if the department determines that an investment is "inherently risky," the investment is subject to a more stringent CIO rating review process. As a result, Education's CIO ratings on the IT Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of Energy | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of Energy (Energy) agreed with the recommendation and, subsequently, updated its OMB IT Dashboard Standard Operating Procedure in December 2016 to incorporate active risks. Specifically, this updated procedure revised the Department's CIO rating criteria to include evaluations of the management and the risk exposure scores of active risks. As a result, Energy's CIO ratings on the Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of Health and Human Services | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of Health and Human Services (HHS) agreed with the recommendation and, in a written response, stated that it updated its CIO evaluation methodology to measure active risks. In October 2023, HHS submitted updated CIO rating guidance showing that the probability and impact scores of active risks are considered as part of the CIO rating process. As a result, HHS's CIO ratings on the Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of the Interior | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of the Interior (Interior) agreed with our recommendation and, subsequently, in March 2017, implemented an updated CIO risk process that standardizes ratings criteria and incorporates active risks. Specifically, the process calls for the Office of the CIO to consider the probability and impact of investment risks and factor those calculations into CIO ratings. As a result, Interior's CIO ratings on the Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of Veterans Affairs | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of Veterans Affairs (VA) agreed with the recommendation and, in a written response, stated that the department was amending its CIO rating review process to ensure that active risks are factored into its IT Dashboard CIO ratings. In September 2021, VA submitted documentation describing this new process, which incorporates active risks into its investments' CIO ratings. Specifically, the process uses a formula to determine an average investment risk probability and impact score, which is then factored into the overall CIO rating. As a result, VA's CIO ratings on the IT Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of State | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Department of State (State) agreed with the recommendation, and, subsequently, in December 2021, State released its "CIO IT Evaluation Framework," which provides guidance for evaluating and reporting on investment risk. Specifically, this Framework directs the department to conduct monthly risk assessments that score investments on multiple evaluation factors related to different types of active risks. The resulting scores are then sent to the CIO to help determine the risk ratings uploaded to the IT Dashboard. As a result of considering active risks in its evaluation process, State's CIO ratings on the Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Office of Personnel Management | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Education, Energy, Health and Human Services, the Interior, State, and Veterans Affairs; and the Director of the Office of Personnel Management should direct their CIOs to factor active risks into their IT Dashboard CIO ratings. |
The Office of Personnel Management (OPM) agreed with our recommendation and, in response to our report, implemented a risk rating process that incorporates active risks. Specifically, this process, which OPM implemented in May 2016, factors both the severity of the active risks and whether they are monitored by the investment team. As a result, OPM's CIO ratings on the Dashboard should better reflect its investments' true likelihood of success to OMB, other oversight bodies, and the public.
|
| Department of Homeland Security | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Defense, Education, and Homeland Security; and the Commissioner of the Social Security Administration should direct their CIOs to update their CIO ratings at least as frequently as required in OMB's guidance. |
The Department of Homeland Security (DHS) disagreed with this recommendation. In its written responses, the department stated that its risk-based process complied with OMB's "Fiscal Year 2017 IT Budget-Capital Planning Guidance" with regard to the frequency of its CIO rating updates. However, as noted in the report, we maintained that the guidance required at least monthly updates, and DHS rated its investments either monthly, quarterly, or semi-annually, depending on investment risk. DHS also noted that its process would be supported by the release of subsequent OMB guidance. After the publication of our report in June 2016, OMB issued its "Fiscal Year 2018 IT Budget-Capital Planning Guidance." This guidance removed the mandatory reporting frequency, but stated that OMB expected that the CIOs would evaluate and rate their investments at specific times, including when the investment business cases were submitted to OMB in the agency budget request and when the business cases were prepared for the President's Budget release. In light of this new guidance, we analyzed the department's update frequency for its 97 investments (as listed on the IT Dashboard in June 2017). From June 2016 through May 2017, we found that 82 investments had updates posted during at least two separate months. For the other 15 investments, 11 were updated during one month, and 4 investments were not updated at all. This analysis shows that, for the majority of its investments, DHS is meeting OMB's expectations for at least semi-annual updates. By meeting these expectations, the department will help ensure that the information on the Dashboard is timely and accurately reflects recent changes.
|
| Department of Education | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Defense, Education, and Homeland Security; and the Commissioner of the Social Security Administration should direct their CIOs to update their CIO ratings at least as frequently as required in OMB's guidance. |
The Department of Education (Education) initially partially concurred with the recommendation. In its comments published in the report, Education stated that OMB's "Fiscal Year 2017 IT Budget-Capital Planning Guidance" addressed the required frequency of updates in several places and that the section specific to CIO evaluations only required agencies to update their ratings as soon as new information became available. In response, we maintained that the requirement for monthly updates was explicitly stated in the guidance and was confirmed by OMB staff. In comments from July 2016, Education stated that it agreed with the recommendation to update its CIO ratings at least as frequently as required in OMB's guidance. It also noted that its process complied with current OMB guidance. Indeed, OMB issued its "Fiscal Year 2018 IT Budget-Capital Planning Guidance" after our report was issued in June 2016. This guidance removed the mandatory reporting frequency, but stated that OMB expected that the CIOs would evaluate and rate their investments at specific times, including when the investment business cases were submitted to OMB in the agency budget request and when the business cases were prepared for the President's Budget release. In light of this new guidance, we analyzed Education's update frequency for its 30 investments (as listed on the IT Dashboard in June 2017). From June 2016 through May 2017, we found that 29 investments had updates posted during at least two separate months. The remaining investment was updated during one month. This analysis shows that, for the majority of its investments, Education is meeting OMB's expectations for at least semi-annual updates. By meeting these expectations, the Department will help ensure that the information on the Dashboard is timely and accurately reflects recent changes.
|
| Department of Defense | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Defense, Education, and Homeland Security; and the Commissioner of the Social Security Administration should direct their CIOs to update their CIO ratings at least as frequently as required in OMB's guidance. |
The Department of Defense (DOD) did not concur with this recommendation. In its written response, the Department noted that its semi-annual reporting is consistent with FITARA requirements and is documented in its OMB-approved FITARA Implementation Plan. After the publication of our report in June 2016, OMB issued its "Fiscal Year 2018 IT Budget-Capital Planning Guidance." This guidance removes the mandatory reporting frequency, but states that OMB expects that the CIOs would evaluate and rate their investments at specific times, including when the investment business cases are submitted to OMB in the agency budget request and when the business cases are prepared for the President's Budget release. In light of this guidance, we analyzed how frequently the Department updated the CIO ratings for its 26 major investments listed on the IT Dashboard in May 2018. From May 2017 through April 2018, we found that 21 of the 26 investments' ratings were updated in at least two separate months. Of the remaining 5 investments, 2 were updated in one month and 3 were not updated at all. This analysis shows that, for the majority of its investments, DOD is meeting OMB's expectations for at least semi-annual updates. By meeting these expectations, the Department will help ensure that the information on the Dashboard is timely and accurately reflects recent changes.
|
| Social Security Administration | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Defense, Education, and Homeland Security; and the Commissioner of the Social Security Administration should direct their CIOs to update their CIO ratings at least as frequently as required in OMB's guidance. |
The Social Security Administration (SSA) agreed with the recommendation. In its written comments, SSA stated that its investment evaluations now occur on a more frequent basis, consistent with OMB's "Fiscal Year 2018 IT Budget-Capital Planning Guidance." OMB released this new version of their Capital Planning Guidance following the publication of our report in June 2016. This guidance removed the mandatory reporting frequency, but stated that OMB expected that the CIOs would evaluate and rate their investments at specific times, including when the investment business cases were submitted to OMB in the agency budget request and when the business cases were prepared for the President's Budget release. In light of this new guidance, we analyzed SSA's update frequency for its 11 investments (as listed on the IT Dashboard in June 2017). From June 2016 through May 2017, we found that all 11 investments had CIO rating updates posted during at least two separate months. This analysis shows that SSA is meeting OMB's expectations for at least semi-annual updates. By meeting these expectations, SSA will help ensure that the information on the Dashboard is timely and accurately reflects recent changes.
|
| Department of Homeland Security | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Homeland Security (DHS) agreed with our recommendation, and, subsequently, in April 2017, implemented updated guidelines for scoring CIO risk ratings. Specifically, these updated guidelines increase the focus on investment risk and call for the consideration of risk exposure calculations when determining CIO ratings. As a result, DHS's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the Department's IT investments receive appropriate levels of oversight.
|
| Department of Agriculture | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Agriculture (Agriculture) agreed with our recommendation and, subsequently, issued updated rating criteria in November 2017 that change how Agriculture assesses investment risk during its CIO rating process. Specifically, these updated rating criteria require an evaluation of investment risk registers and the adequacy of the mitigation strategy for each risk. As a result, Agriculture's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the Department's IT investments receive appropriate levels of oversight.
|
| Department of Education | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Education (Education) agreed with the recommendation, and, in September 2019, issued guidance for its updated CIO rating process. Specifically, this guidance states that the department uses an assessment of individual active risks in order to determine how investments are ultimately reviewed and scored. As a result, Education's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight..
|
| Department of Commerce | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Commerce (Commerce) agreed with our recommendation and, subsequently, issued updated guidance for the department's CIO ratings process in August 2020. Specifically, this guidance states that the CIO review process includes an assessment of investments' high-profile risks that require special attention. Further, a Frequently Asked Questions document from March 2018 specifies that Commerce's CIO ratings are based, in part, on an evaluation of investments' risk registers. As a result, Commerce's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight.
|
| Department of Defense | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Defense (DOD) partially concurred with this recommendation. While DOD agreed that CIO ratings need to reflect risk, the Department stated that its then-current CIO rating process already fulfilled this recommendation. Nevertheless, DOD subsequently implemented an updated CIO rating process in November 2016 that requires the DOD CIO to review additional sources of risk information before making CIO rating determinations. These additional sources include the Defense Acquisition Executive Summary (DAES), a primary source for identifying acquisition program risks. As a result, DOD's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the Department's IT investments receive appropriate levels of oversight.
|
| Department of Energy | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Energy (Energy) agreed with our recommendation and, subsequently, updated its OMB IT Dashboard Standard Operating Procedure in December 2016. Specifically, this updated procedure revised the Department?s CIO rating criteria to increase the focus on investment risk and provide additional guidance on the assessment of CIO evaluation factors such as risk management, requirements management, and contractor oversight. As a result, Energy's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the Department's IT investments receive appropriate levels of oversight.
|
| Department of Health and Human Services | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Health and Human Services (HHS) agreed with the recommendation and, in a written response, stated that it updated its CIO evaluation methodology to measure active risks. In October 2023, HHS submitted updated guidance that directs HHS operating divisions to perform self-assessments of their investments as part of the CIO rating process. These self-assessments incorporate risk management factors, including the probability and impact scores of active investment risks. As a result, HHS's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight.
|
| Social Security Administration | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Social Security Administration (SSA) partially agreed with our recommendation and, in March 2019, issued a new CIO rating policy that stated that the CIO is to consult with appropriate agency officials for a quantitative and qualitative risk assessment as part of the rating process. In January 2020, SSA provided clarifying documentation stating that the CIO rating process requires an evaluation of whether there are unmitigated risks requiring executive attention. As a result, SSA's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight.
|
| Department of Transportation | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Transportation (Transportation) agreed with our recommendation and, subsequently, updated its CIO rating process in its June 2016 Investment Management Process Guidance. This guidance specifies that IT Dashboard rating recommendations should be based on risks associated with cost, schedule, and performance against planned goals and objectives. As a result, Transportation's CIO ratings should better reflect the level of risk facing its investments, thereby helping to ensure that the Department's IT investments receive appropriate levels of oversight.
|
| Department of the Treasury | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of the Treasury (Treasury) did not comment on this recommendation. Nevertheless, Treasury subsequently updated its CIO rating process in July 2016 to focus more on investment risk. Specifically, the new process includes separate risk calculations for project and operational risks that, when combined, account for half of the final CIO rating. As a result, Treasury's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the Department's IT investments receive appropriate levels of oversight.
|
| Department of Veterans Affairs | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of Veterans Affairs (VA) agreed with the recommendation and, in a written response, stated that it will ensure that CIO ratings reflect the level of risk facing its investments. In September 2021, VA submitted documentation describing its new rating process, which incorporates active risks into its investments' CIO ratings. Specifically, the process uses a formula to determine an average investment risk probability and impact score, which is then factored into the overall CIO rating. As a result, VA's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight.
|
| Department of State | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Department of State (State) agreed with the recommendation and, subsequently, in December 2021 released its "CIO IT Evaluation Framework," which provides guidance for evaluating and reporting on investment risk. Specifically, this Framework directs the department to conduct monthly risk assessments that identify and address investment risks. The resulting scores from these assessments are then used to inform the CIO's IT Dashboard risk ratings. As a result of this process, State's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight.
|
| Environmental Protection Agency | To better ensure that the Dashboard ratings more accurately reflect risk, the Secretaries of the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, State, Transportation, the Treasury, Veterans Affairs; the Administrator of the Environmental Protection Agency; and the Commissioner of the Social Security Administration should direct their CIOs to ensure that their CIO ratings reflect the level of risk facing an investment relative to that investment's ability to accomplish its goals. |
The Environmental Protection Agency (EPA) disagreed with the recommendation; however, in June 2020, the agency submitted its 2020 IT Dashboard investment rating tool, which provides an overview of how EPA uses risk data to determine its CIO ratings. Specifically, EPA officials are to, among other things, identify and provide information about multiple active risks and determine whether the risk information is consistent with the information included in management briefings or agency status reviews. As a result, EPA's CIO ratings on the Dashboard should better reflect the level of risk facing its investments, thereby helping to ensure that the department's IT investments receive appropriate levels of oversight.
|
Full Report
GAO Contacts
David A. Powner
Public Inquiries
Topics
Chief information officersCommand control communications systemsEvaluation criteriaInformation technologyManagement information systemsRegulatory agenciesReporting requirementsRisk managementStrategic planningSystems analysisSystems conversions