This is the accessible text file for GAO report number GAO-13-98 
entitled 'Information Technology Dashboard: Opportunities Exist to 
Improve Transparency and Oversight of Investment Risk at Select 
Agencies' which was released on November 15, 2012. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

October 2012: 

Information Technology Dashboard: 

Opportunities Exist to Improve Transparency and Oversight of 
Investment Risk at Select Agencies: 

GAO-13-98: 

GAO Highlights: 

Highlights of GAO-13-98, a report to congressional requesters. 

Why GAO Did This Study: 

In June 2009, OMB launched the federal IT Dashboard, a public website 
that reports performance data for over 700 major IT investments that 
represent about $40 billion of the estimated $80 billion budgeted for 
IT in fiscal year 2012. The Dashboard is to provide transparency for 
these investments to aid public monitoring of government operations. 
It does so by reporting, among other things, how agency CIOs rate 
investment risk. GAO was asked to (1) characterize the CIO ratings for 
selected federal agencies’ IT investments as reported over time on the 
Dashboard, (2) determine how agencies' approaches for assigning and 
updating CIO ratings vary, and (3) describe the benefits and 
challenges associated with agencies’ approaches to the CIO rating. 

To do so, GAO selected six agencies spanning a range of 2011 IT 
spending levels and analyzed data reported for each of their 
investments on the Dashboard. GAO also interviewed agency officials 
and analyzed related documentation and written responses to questions 
about ratings and evaluation approaches, as well as agency views on 
the benefits and challenges related to the CIO rating. 

What GAO Found: 

Chief Information Officers (CIO) at six federal agencies rated the 
majority of their information technology (IT) investments as low risk, 
and many ratings remained constant over time. Specifically, CIOs at 
the selected agencies rated a majority of investments listed on the 
federal IT Dashboard as low risk or moderately low risk from June 2009 
through March 2012; at five of these agencies, these risk levels 
accounted for at least 66 percent of investments. These agencies also 
rated no more than 12 percent of their investments as high or 
moderately high risk, and two agencies (Department of Defense (DOD) 
and the National Science Foundation (NSF)) rated no investments at 
these risk levels (see table). Over time, about 47 percent of the 
agencies’ Dashboard investments received the same rating in every 
rating period. For ratings that changed, the Department of Homeland 
Security (DHS) and Office of Personnel Management (OPM) reported more 
investments with reduced risk when initial ratings were compared with 
those in March 2012; the other four agencies reported more investments 
with increased risk. In the past, the Office of Management and Budget 
(OMB) reported trends for risky IT investments needing management 
attention as part of its annual budget submission, but discontinued 
this reporting in fiscal year 2010. 

Table: Average Composition of CIO Ratings for Agencies’ Major IT 
Investments, June 2009 through March 2012: 

High risk and moderately high risk investments: 
Agency: 
DOD: 0%; 
DHS: 11%; 
HHS: 5%; 
DOI: 4%; 
OPM: 12%; 
NSF: 0%. 

Medium risk investments: 
Agency: 
DOD: 15%; 
DHS: 38%; 
HHS: 21%; 
DOI: 13%; 
OPM: 22%; 
NSF: 0%. 

Low risk and moderately low risk investments: 
Agency: 
DOD: 85%; 
DHS: 51%; 
HHS: 74%; 
DOI: 83%; 
OPM: 66%; 
NSF: 100%. 

Range in the number of investments during the period: 
Agency: 
DOD: 49-87; 
DHS: 63-83; 
HHS: 55-81; 
DOI: 33-48; 
OPM: 6-9; 
NSF: 3-5. 

Source: GAO analysis of data downloaded from OMB’s IT Dashboard. 

Note: Table does not include downgraded or eliminated investments. HHS 
is the Department of Health and Human Services. DOI is the Department 
of the Interior. 

[End of table] 

Agencies generally followed OMB’s instructions for assigning CIO 
ratings, which included considering stakeholder input, updating 
ratings when new data become available, and applying OMB’s six 
evaluation factors. DOD’s ratings were unique in reflecting additional 
considerations, such as the likelihood of OMB review, and consequently 
DOD did not rate any of its investments as high risk. However, in 
selected cases, these ratings did not appropriately reflect 
significant cost, schedule, and performance issues reported by GAO and 
others. Moreover, DOD did not apply its own risk management guidance 
to the ratings, which reduces their value for investment management 
and oversight. 

Various benefits were associated with producing and reporting CIO 
ratings. Most agencies reported (1) increased quality of their 
performance data, (2) greater transparency and visibility of 
investments, and (3) increased focus on project management practices. 
Agencies also noted challenges, such as (1) the effort required to 
gather, validate, and gain internal approval for CIO ratings; and (2) 
obtaining information from OMB to execute required changes to the 
Dashboard. OMB has taken steps to improve its communications with 
agencies. 

What GAO Recommends: 

GAO is recommending that OMB analyze agencies’ investment risk over 
time as reflected in the Dashboard’s CIO ratings and present its 
analysis with the President’s annual budget submission, and that DOD 
ensure that its CIO ratings reflect available investment performance 
assessments and its risk management guidance. Both OMB and DOD 
concurred with our recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-13-98]. For more 
information, contact David A.Powner at (202) 512-9286 or 
pownerd@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

CIOs Rated Most IT Investments as Low Risk or Moderately Low Risk; 
Agencies Had Mixed Results in Reducing Higher Risk Levels: 

Most Agencies Established Approaches for CIO Ratings That Follow OMB's 
Instructions: 

CIO Ratings Present Benefits and Challenges for Agencies' Investment 
Management: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Risk Levels for Investments at Selected Agencies, as of 
March 2012: 

Appendix III: CIO Ratings and Net Changes for Major IT Investments at 
Selected Agencies, June 2009 through March 2012: 

Appendix IV: Comments from the Department of Defense: 

Appendix V: GAO Contacts and Staff Acknowledgments: 

Tables: 

Table 1: Dashboard Variance and Rating Colors: 

Table 2: Investment Evaluation Factors Identified by OMB for Assigning 
CIO Ratings: 

Table 3: IT Dashboard CIO Rating Colors, Based on a Five-Point Scale 
for CIO Ratings: 

Table 4: Average Composition of CIO Ratings for Agencies' Major IT 
Investments from June 2009 through March 2012: 

Table 5: Number of Major IT Investments with CIO Ratings That Changed 
Compared to Those That Remained Constant, Initial Rating through March 
2012: 

Table 6: Net Changes in Investment Risk, Based on Comparison of 
Initial Rating to March 2012: 

Table 7: Data Sources and Derivation of CIO Ratings by Selected 
Agencies: 

Table 8: CIO Ratings and Budget Totals for Selected Agencies' Major IT 
Investments as of March 2012: 

Figures: 

Figure 1: Example of an Agency Portfolio Page as Reported on OMB's IT 
Dashboard Website, July 2012: 

Figure 2: Example of an Agency's IT Investment Page from the IT 
Dashboard Website, July 2012: 

Figure 3: CIO Ratings and Associated IT Budgets for Fiscal Year 2012 
Dashboard Investments from Selected Agencies, as of March 2012: 

Figure 4: Percentages of Major IT Investments at Selected Agencies 
with CIO Ratings That Changed Compared to Ratings That Remained 
Constant, Initial Rating through March 2012: 

Figure 5: Percentages of Major IT Investments at Selected Agencies 
with Changes in CIO Ratings, Initial Rating Compared to March 2012: 

Figure 6: CIO Ratings for Major IT Investments at the Department of 
Defense, as Reported on the IT Dashboard from July 2009 through March 
2012: 

Figure 7: Changes to Risk Levels for Major IT Investments at the 
Department of Defense, Initial CIO Rating through March 2012: 

Figure 8: CIO Ratings for Major IT Investments at the Department of 
Homeland Security, as Reported on the IT Dashboard from July 2009 
through March 2012: 

Figure 9: Changes to Risk Levels for Major IT Investments at the 
Department of Homeland Security, Initial CIO Rating through March 2012: 

Figure 10: CIO Ratings for Major IT Investments at the Department of 
Health and Human Services, as Reported on the IT Dashboard from July 
2009 through March 2012: 

Figure 11: Changes to Risk Levels for Major IT Investments at the 
Department of Health and Human Services, Initial CIO Rating through 
March 2012: 

Figure 12: CIO Ratings for Major IT Investments at the Department of 
the Interior, as Reported on the IT Dashboard from July 2009 through 
March 2012: 

Figure 13: Changes to Risk Levels for Major IT Investments at the 
Department of the Interior, Initial CIO Rating through March 2012: 

Figure 14: CIO Ratings for Major IT Investments at the National 
Science Foundation, as Reported on the IT Dashboard from June 2009 
through March 2012: 

Figure 15: Changes to Risk Levels for Major IT Investments at the 
National Science Foundation, Initial CIO Rating through March 2012: 

Figure 16: CIO Ratings for Major IT Investments at the Office of 
Personnel Management, as Reported on the IT Dashboard from July 2009 
through March 2012: 

Figure 17: Changes to Risk Levels for Major IT Investments at the 
Office of Personnel Management, Initial CIO Rating through March 2012: 

Abbreviations: 

CIO: chief information officer: 

DEAMS: Defense Enterprise Accounting and Management System: 

DOD: Department of Defense: 

DOI: Department of the Interior: 

DHS: Department of Homeland Security: 

GCSS-Army: Global Combat Support System-Army: 

GFEBS: General Fund Enterprise Business System: 

HHS: Department of Health and Human Services: 

IT: information technology: 

NSF: National Science Foundation: 

OMB: Office of Management and Budget: 

OPM: Office of Personnel Management: 

TechStat: TechStat Accountability Sessions: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

October 16, 2012: 

The Honorable Thomas R. Carper: 
Chairman: 
The Honorable Scott P. Brown: 
Ranking Member: 
Subcommittee on Federal Financial Management, Government Information, 
Federal Services, and International Security: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

Spending on information technology (IT) represents a significant 
portion of the federal budget--estimated at $80 billion for fiscal 
year 2012.[Footnote 1] More than 700 major investments account for 
approximately $40 billion of this IT spending. The Clinger-Cohen Act 
of 1996 charges the Director of the Office of Management and Budget 
(OMB) with responsibility for analyzing, tracking, and evaluating the 
risks and results of all major IT investments as part of the federal 
budget process, and reporting to Congress on the performance benefits 
achieved by these investments.[Footnote 2] The act also places 
responsibility for managing investments with the heads of agencies and 
establishes chief information officers (CIOs) to advise and assist 
agency heads in carrying out this responsibility. 

OMB launched the Federal IT Dashboard in June 2009 as a public website 
that reports performance and supporting data for the major IT 
investments. The Dashboard is to provide transparency for these 
investments in order to facilitate public monitoring of government 
operations and accountability for investment performance by the 
federal CIOs who oversee them. In January 2010, OMB began using the 
Dashboard as one of several tools to identify troubled investments. 
These investments became the focus of joint OMB-agency TechStat 
Accountability Sessions (TechStats)--evidence-based reviews intended 
to improve investment performance through concrete actions. In 
December 2010, OMB reported that these sessions resulted in $3 billion 
in reduced life-cycle costs and subsequently incorporated the TechStat 
model into its 25-point plan for reforming federal IT management. 
[Footnote 3] With this plan, agency CIOs became responsible for 
leading TechStat sessions at the department level, analyzing 
investments using data from the Dashboard, and terminating or turning 
around at least one-third of underperforming IT projects within 18 
months. OMB reported its progress on the plan, improvements to the 
Dashboard, and results of TechStat sessions in the analytical 
perspectives it provided for the President's 2012 and 2013 budget 
submissions.[Footnote 4] 

In response to your request, our objectives for this review were to 
(1) characterize the CIO ratings for selected federal agencies' IT 
investments as reported over time on the Dashboard, (2) determine how 
agencies' approaches for assigning and updating CIO ratings vary, and 
(3) describe the benefits and challenges associated with agencies' 
approaches to the CIO rating. 

To establish the scope of our review, we selected six agencies that 
spanned a range of IT spending for fiscal year 2011, including the 
three highest spending agencies, two of the lowest, and an agency in 
the middle. Collectively, these agencies accounted for approximately 
$51 billion, or 65 percent, of 2011 spending on IT investments. The 
six agencies are the Department of Defense (DOD), Department of 
Homeland Security (DHS), Department of Health and Human Services 
(HHS), Department of the Interior (DOI), National Science Foundation 
(NSF), and Office of Personnel Management (OPM). 

To address our objectives, we downloaded CIO ratings and related data 
reported for investments on the Dashboard and analyzed these data for 
the period June 2009 to March 2012.[Footnote 5] We did not 
independently evaluate the ratings as reported by the agencies, but 
determined that they were sufficiently complete and accurate for our 
analyses. We interviewed agency officials, including CIOs where 
possible, and obtained written responses and supporting documents, 
related agency policies, procedures, reported data, artifacts, as well 
as agency views on the benefits and challenges associated with 
performing these ratings and reporting them to the Dashboard. We also 
utilized recent GAO and DOD Inspector General reviews of DOD's major 
IT investments and compared findings in these reports to the CIO 
ratings that the department submitted to the Dashboard. In addition, 
we analyzed OMB documentation and interviewed OMB staff to update our 
information on how the Dashboard has evolved, identify the guidance 
agencies received about CIO ratings, determine the efforts OMB has 
under way to improve the Dashboard, and describe the ways in which OMB 
is using the data to improve IT management. 

We conducted this performance audit from January 2012 through 
September 2012 in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. Further details of our objectives, scope, and methodology 
are provided in appendix I. 

Background: 

The Clinger-Cohen Act of 1996 requires OMB to establish processes to 
analyze, track, and evaluate the risks and results of major capital 
investments in information systems made by federal agencies and report 
to Congress on the net program performance benefits achieved as a 
result of these IT investments.[Footnote 6] Further, the act places 
responsibility for managing investments with the heads of agencies and 
establishes CIOs to advise and assist agency heads in carrying out 
this responsibility. 

OMB established the Management Watch List in 2003 to help carry out 
its oversight role. The Management Watch List included mission-
critical projects that needed to improve performance measures, project 
management, IT security, or overall justification for inclusion in the 
President's budget submission. Further, in August 2005, OMB 
established a High-Risk List, which consisted of projects identified 
by federal agencies, with OMB's input, as requiring special attention 
from oversight authorities and the highest levels of agency 
management. Between 2005 and 2009, OMB described its efforts to 
monitor and manage risky federal IT investments in the annual budget 
submission. 

Over the past several years, we have reported and testified on OMB's 
initiatives to highlight troubled IT projects, justify investments, 
and use project management tools.[Footnote 7] For instance, in 2006 we 
recommended that OMB develop a single aggregated list of high-risk 
projects and their deficiencies and use that list to report to 
Congress on progress made in correcting high-risk problems.[Footnote 
8] As a result, OMB started publicly releasing aggregate data on its 
Management Watch List and disclosing the projects' deficiencies. 
Moreover, between 2007 and 2009, the President's budget submission 
included an overview of investment performance over several budget 
years, including the number of federal IT projects in need of 
management attention. Such information helped Congress stay better 
informed of high-risk projects and make related funding decisions. 
[Footnote 9] With the advent of its IT Dashboard in 2009, OMB 
discontinued this type of reporting in the fiscal year 2010 budget 
submission. 

OMB's Dashboard Provides Visibility into the Performance of Federal IT 
Investments: 

In June 2009, OMB deployed a public website to further improve the 
transparency and oversight of agencies' IT investments, replacing the 
Management Watch List and High-Risk List. Known as the IT 
Dashboard,[Footnote 10] this site displays federal agencies' cost, 
schedule, and performance data for over 700 major federal IT 
investments at 27 federal agencies that are responsible for about $40 
billion of the federal budget. According to OMB, these data are 
intended to provide a near-real-time perspective on the performance of 
these investments, as well as a historical perspective. Further, the 
public display of these data is intended to allow OMB; other oversight 
bodies, including Congress; and the general public to hold the 
government agencies accountable for progress and results. 

OMB reported on plans and implementation progress for this management 
tool in the "Analytical Perspectives" section of the President's 
budget submissions for fiscal years 2012 and 2013,[Footnote 11] 
including planned updates to the Dashboard during 2012 to support 
closer executive oversight and intervention to prevent schedule 
delays, cost overruns, and failures in delivering key functionality 
needed by federal programs. For example, it reported using the 
Dashboard to identify investments for TechStat reviews. 

The Dashboard visually presents performance ratings for agencies 
overall and for individual investments using metrics that OMB has 
defined--cost, schedule, and CIO evaluation. The website also provides 
the capability to download certain data. Figure 1 is an example of an 
agency's (OPM) portfolio page as recently depicted on the Dashboard. 

Figure 1: Example of an Agency Portfolio Page as Reported on OMB's IT 
Dashboard Website, July 2012: 

[Refer to PDF for image: illustration] 

Depicted on the page: 

Current and Trends for: 
Investment Evaluation by Agency CIO; 
Project Cost; 
Project Schedule. 

Source: OPM's Portfolio page as reported on OMB's Dashboard on July 
18, 2012. 

[End of figure] 

The Dashboard's data spans the period from June 2009 to the present, 
and is based, in part, on each agency's exhibit 53 and exhibit 300 
submissions[Footnote 12] to OMB, as well as on agency assessments and 
supporting information on each investment. Over the life of the 
Dashboard, OMB has issued guidance to agencies on, among other things, 
what data to report, how those data need to be structured and 
formatted for upload to the Dashboard, and procedures for using the 
Dashboard's submission tools. For instance, OMB instructed agencies to 
update and submit investment cost and schedule data monthly. OMB has 
made various changes to the organization, available data, and features 
of the Dashboard over time, including: 

* improvements to Dashboard calculations to incorporate the variance 
of "in progress" milestones rather than just "completed" milestones; 

* web pages containing data on historical ratings and rebaselines of 
eliminated and downgraded investments; 

* added data on awarded contracts, with links to USAspending.gov; 

* release of IT Dashboard source code and documentation to an open 
source hosting provider; 

* enhancements to baseline history, which give users the ability to 
see field-by-field changes for each rebaseline; 

* a mechanism for OMB analysts to provide feedback to agencies on 
investment submissions; and: 

* mobile-friendly formatting of Dashboard displays. 

Once OMB has received agency-reported investment data, it converts 
these into investment performance ratings for display on the dashboard 
according to calculations and protocols described on its website. OMB 
assigns cost and schedule performance ratings by using data submitted 
by agencies to calculate variances between the planned cost or 
schedule targets and the actual or projected cost or schedule values. 
OMB converts these variances to percentages, and assigns the ratings 
to be presented on the Dashboard within three ranges, red, yellow, and 
green, as shown in table 1. 

Table 1: Dashboard Variance and Rating Colors: 

Project level cost and schedule variance rating: greater than or equal 
to 30%; 
Rating color: Red. 

Project level cost and schedule variance rating: greater than or equal 
to 10% and less than 30%; 
Rating color: Yellow. 

Project level cost and schedule variance rating: less than 10%; 
Rating color: Green. 

Source: OMB's IT Dashboard. 

[End of table] 

Although the thresholds for assigning cost and schedule variance 
ratings has remained constant over the life of the Dashboard, the cost 
and schedule data agencies are required to submit have changed in 
several ways, as have the variance calculations. For example, in 
response to our recommendations (further discussed in the next 
section), OMB changed how the Dashboard calculates the cost and 
schedule ratings in July 2010, to include "in progress" milestones 
rather than just "completed" ones for more accurate reflection of 
current investment status. 

GAO Has Previously Reported on the Dashboard's Value, Data Quality, 
and Recent Improvements: 

We have previously reported that OMB has taken significant steps to 
enhance the oversight, transparency, and accountability of federal IT 
investments by creating its IT Dashboard, and by improving the 
accuracy of investment ratings. We also found issues with the accuracy 
and data reliability of cost and schedule data, and recommended steps 
that OMB should take to improve these data. 

In July 2010, we reported[Footnote 13] that the cost and schedule 
ratings on OMB's Dashboard were not always accurate for the 
investments we reviewed, because these ratings did not take into 
consideration current performance. As a result, the ratings were based 
on outdated information. We recommended that OMB report on its planned 
changes to the Dashboard to improve the accuracy of performance 
information and provide guidance to agencies to standardize milestone 
reporting. OMB agreed with our recommendations and, as a result, 
updated the Dashboard's cost and schedule calculations to include both 
ongoing and completed activities. Similarly, in March 2011, we 
reported[Footnote 14] that OMB had initiated several efforts to 
increase the Dashboard's value as an oversight tool, and had used its 
data to improve federal IT management. We also reported, however, that 
agency practices and the Dashboard's calculations contributed to 
inaccuracies in the reported investment performance data. For 
instance, we found missing data submissions or erroneous data at each 
of the five agencies we reviewed, along with instances of inconsistent 
program baselines and unreliable source data. As a result, we 
recommended that the agencies take steps to improve the accuracy and 
reliability of their Dashboard information, and that OMB improve how 
it rates investments relative to current performance and schedule 
variance. Most agencies generally concurred with our recommendations; 
OMB agreed with our recommendation for improving ratings for schedule 
variance. It disagreed with our recommendation to improve how it 
reflects current performance in cost and schedule ratings, but more 
recently made changes to Dashboard calculations to address this while 
also noting challenges in comprehensively evaluating cost and schedule 
data for these investments. 

More recently, in November 2011, we reported[Footnote 15] that the 
accuracy of investment cost and schedule ratings had improved since 
our July 2010 report because OMB had refined the Dashboard's cost and 
schedule calculations. Most of the ratings for the eight investments 
we reviewed were accurate, although we noted that more could be done 
to inform oversight and decision making by emphasizing recent 
performance in the ratings. We recommended that the General Services 
Administration comply with OMB's guidance for updating its ratings 
when new information becomes available (including when investments are 
rebaselined) and the agency concurred. Since we previously recommended 
that OMB improve how it rates investments, we did not make any further 
recommendations. 

CIO Ratings Are Important for OMB's IT Reform and Management 
Initiatives: 

Unlike the Dashboard's cost and schedule ratings, which are derived by 
OMB based on agency-submitted data, the "Investment Evaluation by 
Agency CIO" (also called the CIO rating) is determined by agency 
officials; OMB translates the agency's numerical assignment for an 
investment into a color for depiction on the Dashboard. An OMB staff 
member from the Office of E-Government and Information Technology 
noted that the CIO rating should be a current assessment of future 
performance based on historical results and is the only Dashboard 
performance indicator that has been defined and produced the same way 
since the Dashboard's inception. According to OMB's instructions, a 
CIO rating should reflect the level of risk facing an investment on a 
scale from 1 (high risk) to 5 (low risk) relative to that investment's 
ability to accomplish its goals. Each agency CIO is to assess their IT 
investments against a set of six preestablished evaluation factors 
identified by OMB (shown in table 2) and then assign a rating of 1 to 
5 based on his or her best judgment of the level of risk facing the 
investment. According to an OMB staff member, agency CIOs are 
responsible for determining appropriate thresholds for the risk levels 
and for applying them to investments when assigning CIO ratings. 

Table 2: Investment Evaluation Factors Identified by OMB for Assigning 
CIO Ratings: 

Evaluation factor: Risk management; 
Supporting examples: Risk management strategy exists; 
Risks are well understood by senior leadership; 
Risk log is current and complete; 
Risks are clearly prioritized; 
Mitigation plans are in place to address risks. 

Evaluation factor: Requirements management; 
Supporting examples: 
Investment objectives are clear and scope is controlled; 
Requirements are complete, clear and validated; 
Appropriate stakeholders are involved in requirements definition. 

Evaluation factor: Contractor oversight; 
Supporting examples: Acquisition strategy is defined and managed via 
an Integrated Program Team; 
Agency receives key reports, such as earned value reports, current 
status, and risk logs; 
Agency is providing appropriate management of contractors such that 
the government is monitoring, controlling, and mitigating the impact 
of any adverse contract performance. 

Evaluation factor: Historical performance; 
Supporting examples: 
No significant deviations from planned cost and schedule; Lessons 
learned and best practices are incorporated and adopted. 

Evaluation factor: Human capital; 
Supporting examples: 
Qualified management and execution team for the IT investments and/or 
contracts supporting the investment; Low turnover rate. 

Evaluation factor: Other; 
Supporting examples: 
Other factors that the CIO deems important to forecasting future 
success. 

Source: OMB's IT Dashboard. 

[End of table] 

OMB recommends that CIOs consult with appropriate stakeholders in 
making their evaluation, including Chief Acquisition Officers, program 
managers, and other interested parties. Ultimately, CIO ratings are 
assigned colors for presentation on the Dashboard, according to the 
five-point rating scale, as illustrated in table 3. 

Table 3: IT Dashboard CIO Rating Colors, Based on a Five-Point Scale 
for CIO Ratings: 

Rating (by agency CIO): 5-Low risk; 
Color: Green. 

Rating (by agency CIO): 4-Moderately low risk; 
Color: Green. 

Rating (by agency CIO): 3-Medium risk; 
Color: Yellow. 

Rating (by agency CIO): 2-Moderately high risk; 
Color: Red. 

Rating (by agency CIO): 1-High risk; 
Color: Red. 

Source: OMB's IT Dashboard. 

[End of table] 

OMB has made the CIO's evaluation and rating a key component of its 
larger IT Reform Initiative and 25 Point Plan. In its plan, OMB 
reported that it used agencies' CIO ratings to select investments for 
the TechStat review sessions it conducted between 2010 and 2011. These 
sessions are data-driven assessments of IT investments by agency 
leaders that are intended to result in concrete action to improve 
performance. OMB reported that the TechStats it conducted on selected 
investments resulted in approximately $3 billion in reduced costs. 
Building on the results of those sessions, the plan articulates a 
strategy for strengthening IT governance, in part, through the 
adoption of the TechStat model by federal agencies. In conducting 
TechStats, agencies are to rely, in part, on CIO ratings from the IT 
Dashboard. The TechStat Toolkit, developed by OMB and a task force of 
agency leads, provides sample questions regarding an investment's CIO 
rating and associated risks for use in TechStat sessions. 

Furthermore, OMB issued guidance in August 2011[Footnote 16] that 
stated, among other things, that agency CIOs shall be held accountable 
for the performance of IT program managers based on their governance 
process and the data reported on the IT Dashboard, which includes the 
CIO rating. According to OMB, the addition of CIO names and photos on 
Dashboard investments is intended to highlight this accountability and 
link it to the Dashboard's reporting on investment performance. Figure 
2 illustrates the CIO rating information presented on the Dashboard 
for an example IT investment. 

Figure 2: Example of an Agency's IT Investment Page from the IT 
Dashboard Website, July 2012: 

[Refer to PDF for image: illustration] 

Source: CBP - Automated Commercial Environment I International Trade 
Data System (ACE/ITDS) page as reported on OMB's IT Dashboard on July 
18, 2012. 

[End of figure] 

CIOs Rated Most IT Investments as Low Risk or Moderately Low Risk; 
Agencies Had Mixed Results in Reducing Higher Risk Levels: 

As of March 2012, CIO ratings for most investments listed on the 
Dashboard for the six agencies we reviewed indicated either low risk 
or moderately low risk (223 out of 313 investments across all the 
selected agencies). High risk or moderately high risk ratings were 
assigned to fewer investments (12 out of 313 investments across all 
the selected agencies). Figure 3 presents the total number of IT 
investments rated on the Dashboard for each of the selected agencies 
according to their risk levels, as of March 2012, and illustrates the 
predominance of low risk investments for the agencies in our review. 
The figure also reports agencies' budgets for their major IT 
investments for fiscal year 2012, as presented on the Dashboard. 

Figure 3: CIO Ratings and Associated IT Budgets for Fiscal Year 2012 
Dashboard Investments from Selected Agencies, as of March 2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the dashboard: 

Agency: DOD; 
Low risk and moderately low risk investments: 74; 
Medium risk investments: 13; 
High risk and moderately high risk investments: 0; 
Budget: $13.3 billion. 

Agency: DHS; 
Low risk and moderately low risk investments: 45; 
Medium risk investments: 35; 
High risk and moderately high risk investments: 3; 
Budget: $4.4 billion. 

Agency: HHS; 
Low risk and moderately low risk investments: 59; 
Medium risk investments: 19; 
High risk and moderately high risk investments: 3; 
Budget: $2.5 billion. 

Agency: DOI; 
Low risk and moderately low risk investments: 34; 
Medium risk investments: 8; 
High risk and moderately high risk investments: 6; 
Budget: $865.6 million. 

Agency: OPM; 
Low risk and moderately low risk investments: 6; 
Medium risk investments: 3; 
High risk and moderately high risk investments: 0; 
Budget: $46.3 million. 

Agency: NSF; 
Low risk and moderately low risk investments: 5; 
Medium risk investments: 0; 
High risk and moderately high risk investments: 0; 
Budget: $85.5 million. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 
Budget figures represent spending on major IT investments for fiscal 
year 2012. 

[End of figure] 

Historically, over the life of the Dashboard from June 2009 to March 
2012, low or moderately low risk ratings accounted for at least 66 
percent of all ratings at five of the six agencies (the exception is 
DHS with 51 percent). Medium risk ratings accounted for between 0 to 
38 percent of all reported ratings across agencies during this period. 
The maximum percentage of ratings in the high risk or moderately high 
risk categories for any agency during this 34-month period was 12 
percent, with two agencies--DOD and NSF--reporting no high risk 
investments.[Footnote 17] DOD stated in written comments that this was 
because they did not deem any of their investments to be high risk. 
(DOD's investment risks are further discussed in the next section.) An 
NSF official from the Division of Information Systems stated that 
there were no high risk investments because most of their investments 
were in the operations and maintenance phase. Table 4 presents the 
average composition of ratings for each agency during the reporting 
period of June 2009 to March 2012. Appendix III depicts each agency's 
CIO ratings by risk level on a monthly basis during the reporting 
period. 

Table 4: Average Composition of CIO Ratings for Agencies' Major IT 
Investments from June 2009 through March 2012: 

High risk and moderately high risk investments: 
Agency: 
DOD: 0%; 
DHS: 11%; 
HHS: 5%; 
DOI: 4%; 
OPM: 12%; 
NSF: 0%. 

Medium risk investments: 
Agency: 
DOD: 15%; 
DHS: 38%; 
HHS: 21%; 
DOI: 13%; 
OPM: 22%; 
NSF: 0%. 

Low risk and moderately low risk investments: 
Agency: 
DOD: 85%; 
DHS: 51%; 
HHS: 74%; 
DOI: 83%; 
OPM: 66%; 
NSF: 100%. 

Range in the number of investments during the period: 
Agency: 
DOD: 49-87; 
DHS: 63-83; 
HHS: 55-81; 
DOI: 33-48; 
OPM: 6-9; 
NSF: 3-5. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Table does not include downgraded or eliminated investments. 

[End of table] 

Overall, the CIO rating remained constant for 147 of 313 investments 
that were active as of March 2012 (about 47 percent of the investments 
we reviewed). These investments were rated at the same risk level 
during every rating period (see figure 4). 

Figure 4: Percentages of Major IT Investments at Selected Agencies 
with CIO Ratings That Changed Compared to Ratings That Remained 
Constant, Initial Rating through March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level changed: 53% (166); 
Risk level constant: 47% (147). 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 

[End of figure] 

Four of the six agencies did not change the CIO rating for a majority 
of their investments (excluding any investments that were downgraded 
or eliminated) during the time frame we examined. In contrast, the 
other two agencies--OPM and HHS--changed the CIO rating for more than 
70 percent of their investments at least once between the investment's 
initial rating and the rating reported as of March 2012. Table 5 lists 
the number of each agency's investments whose ratings were constant 
and changed over time. 

Table 5: Number of Major IT Investments with CIO Ratings That Changed 
Compared to Those That Remained Constant, Initial Rating through March 
2012: 

Remained constant: 
DOD: 51; 
DHS: 42; 
HHS: 17; 
DOI: 31; 
OPM: 2; 
NSF: 4. 

Changed: 
DOD: 36; 
DHS: 41; 
HHS: 64; 
DOI: 17; 
OPM: 7; 
NSF: 1. 

Total: 
DOD: 87; 
DHS: 83; 
HHS: 81; 
DOI: 48; 
OPM: 9; 
NSF: 5. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Table does not include downgraded or eliminated investments. 

[End of table] 

Agencies offered several reasons for why many investments had no 
changes in their CIO ratings during their entire time on the 
Dashboard. Five of the six selected agencies indicated that many 
investments were in a steady-state or operations and maintenance phase 
with no new development. One agency reported that their investments' 
CIO ratings remained constant because the investments consistently met 
all requirements and deadlines and were using project management best 
practices. 

The agencies we reviewed showed mixed results in reducing the number 
of higher risk investments during the rating period.[Footnote 18] For 
investments whose rating changed at least once during the period, 40 
percent (67 investments) received a lower risk rating in March 2012 
than they received initially, 41 percent of investments (68 
investments) received a higher risk rating, and the remaining 19 
percent (31 investments) received the same rating in March 2012 as 
they had initially received, despite whatever interim changes may have 
occurred (i.e., there was no "net" change to their reported risk 
levels). (See figure 5.) 

Figure 5: Percentages of Major IT Investments at Selected Agencies 
with Changes in CIO Ratings, Initial Rating Compared to March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level increased: 41% (68); 
Risk level reduced: 40% (67); 
No net change in risk level: 19% (31). 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 
Percentage represents initial CIO rating compared to the CIO rating as 
of March 2012 and does not represent any fluctuations that may have 
occurred in between that time frame. 

[End of figure] 

Two agencies--DHS and OPM--reported more investments with reduced risk 
in March 2012, as compared with initial ratings. The other four 
agencies reported more investments with increased risk. Table 6 
presents net changes in risk levels at each of the selected agencies 
(among investments that were not downgraded or eliminated). Appendix 
III graphically summarizes these data for all six agencies. 

Table 6: Net Changes in Investment Risk, Based on Comparison of 
Initial Rating to March 2012: 

Risk level reduced:
DOD: 15; 
DHS: 26; 
HHS: 17; 
DOI: 4; 
OPM: 5; 
NSF: 0. 

Risk level increased: 
DOD: 18; 
DHS: 12; 
HHS: 25; 
DOI: 11; 
OPM: 1; 
NSF: 1. 

No net change: 
DOD: 3; 
DHS: 3; 
HHS: 22; 
DOI: 2; 
OPM: 1; 
NSF: 0. 

Total number of changed investments: 
DOD: 36; 
DHS: 41; 
HHS: 64; 
DOI: 17; 
OPM: 7; 
NSF: 1. 

Source: GAO analysis of data downloaded from OMB's IT Dashboard. 

Note: Table does not include downgraded or eliminated investments. 
Table represents investments' initial CIO rating compared to the CIO 
rating as of March 2012 and does not represent any fluctuations that 
may have occurred in between that time frame. 

[End of table] 

Agencies most commonly cited additional oversight or program reviews 
as factors that contributed to decreased risk levels. Specifically, 
agencies commented that the CIO ratings and Dashboard reporting had 
spurred improved program management and risk mitigation. For example, 
one agency's officials commented that the CIO now closely monitors the 
monthly performance and risk data generated by their investments, and 
that the additional oversight has brought about strengthened processes 
and more focused attention to issues. 

In contrast, several agencies cited generally poor risk management at 
the investment level, the introduction of new investment/programs 
risks, as well as instances of poor project management as factors 
contributing to increased risk for investments. For example, one 
agency responded that internal review findings revealed new risks that 
caused an investment's risk level to increase. Another agency's 
officials reported that various technical issues caused one of their 
investments to fall behind schedule, thus increasing risk. 

Both OMB and several agencies suggested caution in interpreting 
changing risk levels for investments. They noted that an increase in 
an investment's risk level can sometimes indicate better management by 
the program or CIO because previously unidentified risks have been 
assessed and included in the CIO evaluation. Conversely, a decrease in 
an investment's risk level may not indicate improved management if the 
data and analysis on which the CIO rating are based is incomplete, 
inconsistent, or outdated. 

Further analysis of the characteristics and causes of Dashboard's CIO 
ratings, and reporting on the patterns of risk within and among 
agencies, could provide Congress and the public with additional 
perspectives on federal IT investment risk over time. However, for the 
past four budget submissions, OMB has not summarized the extent of 
risk represented by major federal IT investments in the analysis it 
prepares annually for the President's budget submission, as it did 
prior to the fiscal year 2010 submission. As a result, OMB is missing 
an opportunity to integrate such risk assessments into its evaluation 
of major capital investments in reporting to Congress. 

Most Agencies Established Approaches for CIO Ratings That Follow OMB's 
Instructions: 

OMB has provided agencies with instructions for assigning CIO ratings 
for the major IT investments reported on the Dashboard. Specifically, 
OMB's instructions state that agency CIOs should rate each investment 
based on his/her best judgment and should: 

* include input from stakeholders, such as Chief Acquisition Officers, 
program managers, and others; 

* update the rating as soon as new information becomes available that 
might affect the assessment of a given investment; and: 

* utilize OMB's investment rating factors, including: risk management, 
requirements management, contractor oversight, historical performance, 
and human capital, as well as any other factors deemed relevant by the 
CIO. 

Despite differences in the specific inputs and processes used, 
agencies generally followed OMB's instructions for assigning CIO 
ratings. However, DOD's ratings reflected additional considerations 
beyond OMB's instructions and did not reflect available information 
about significant risks for certain investments. The sections that 
follow describe how each agency addressed OMB's instructions. 

Include input from stakeholders. Each of the six agencies we reviewed 
relied on stakeholder input, at least in part, when assigning CIO 
ratings. Agencies also cited a variety of review boards, data from 
program and financial systems, and other investment assessments as 
inputs to the rating. Table 7 describes the data and processes that 
agencies reported using when they derived their CIO ratings. 

Table 7: Data Sources and Derivation of CIO Ratings by Selected 
Agencies: 

Agency: DOD; 
Data sources used in CIO rating: Input and recommendations from a 
variety of stakeholders, including functional leads, acquisition 
community, component CIOs, and cost and schedule experts; 
How rating is derived: The DOD CIO reviews investment performance 
information and recommendations from stakeholders and makes a 
subjective evaluation of the investment based upon those inputs. 

Agency: DHS; 
Data sources used in CIO rating: Data from internal investment 
management systems and the department's periodic reporting system, 
program management reviews, CIO portfolio reviews, and input from the 
executive steering committees; 
How rating is derived: The DHS CIO determines the rating based upon 
information and recommendation gathered by an analysis team from the 
department's Enterprise Business Management Office. 

Agency: HHS; 
Data sources used in CIO rating: Cost and schedule variance data, 
exhibit 300 Quality Review, number of rebaselines, and investment 
manager self assessment; 
How rating is derived: The HHS CIO determines the CIO ratings using a 
score calculated from the data. 

Agency: DOI; 
Data sources used in CIO rating: Bureau-proposed rating of investment, 
Electronic Capital Planning Investment Control Technology[A] data, 
iStatb performance reviews, applicable GAO or Office of Inspector 
General findings; 
How rating is derived: One of the processes the agency follows begins 
with the bureau responsible for the investment completing a template 
to document its proposed bureau rating of the investment and submits 
the proposed rating to the department CIO through the Capital Planning 
and Investment Control office. The office reviews and analyzes the 
data, compares the data against known information, such as iStat 
performance reviews or applicable GAO or Office of Inspector General 
findings regarding the current state of the investment, and provides a 
recommendation to the CIO, who determines the rating. Additionally, 
the CIO can make the decision to change the rating outside of this 
process. 

Agency: NSF; 
Data sources used in CIO rating: IT portfolio management tool; 
program manager and stakeholder input from monthly reviews where 
budget, schedule, and risks are reviewed; 
How rating is derived: NSF staff meets monthly with the CIO to discuss 
investment data and review evaluations using OMB's guidance. The staff 
assesses and ranks the investments. The CIO issues the rating based 
upon this information. 

Agency: OPM; 
Data sources used in CIO rating: IT security rating; enterprise 
architecture rating; earned value management program manager rating; 
risk management rating; Electronic Capital Planning Investment Control 
Technology; cost, schedule, and overall variance rating, IT Dashboard 
cost and schedule variance; and program managers' self-assessments; 
How rating is derived: The IT Investment Management group reports to 
the CIO on collected information regarding the investments. The group 
produces a chart that tracks 20 different metrics; each metric is 
presented as either a red, yellow, or green rating. The CIO issues the 
rating based upon this information. 

Source: GAO analysis of agency-reported data. 

[A] Electronic Capital Planning Investment Control Technology is a 
government-owned technology system that is designed to help federal 
agencies in the management and control of their initiatives, 
portfolios, and investment priorities. This web-based application 
assists managers and staff involved in IT planning in assessing IT 
investments in terms of their costs, risks, and expected returns. 

[B] iStat is DOI's internal investment review process, which was 
modeled after OMB's TechStat process. 

[End of table] 

Update CIO ratings. All six agencies established guidelines for 
periodically reviewing and updating their CIO ratings. Specifically, 
HHS, NSF, DOI, and OPM reported that they update CIO ratings on a 
monthly basis. DOD has adopted a quarterly update cycle, although an 
official noted that the actual process of collecting information and 
evaluating investments for the ratings takes slightly longer than 3 
months. DHS officials with the Office of the CIO stated that the 
frequency of its updates varies based on the risk level of an 
investment's previous rating: investments with a previous CIO rating 
of green are to be reviewed semiannually; yellow investments are to be 
reviewed quarterly; and red investments are to be reviewed monthly. 

Utilize OMB's investment rating factors. Most of the selected agencies 
use OMB's investment rating factors when evaluating their investments. 
Only one agency (HHS) does not use all of them. Specifically, an HHS 
official from the Office of the CIO told us that human capital issues 
are not explicitly covered in their CIO rating criteria because 
investment owners are to provide adequate IT human capital, and that 
these owners will reflect any issues that arise when providing input 
for the CIO rating. 

Among the agencies we reviewed, DOD was unique in that its ratings 
reflected additional considerations beyond OMB's instructions. For 
example, briefing slides prepared for DOD's 2011 CIO rating exercise 
identified the need to "balance" CIO ratings, and advised that yellow 
or red ratings could lead to an OMB review.[Footnote 19] In addition, 
DOD officials explained that the department rated investments green 
(or low risk) if the risk of the investment not meeting its 
performance goals is low; yellow (or medium risk) if the investment is 
facing difficulty; and red (high risk) only if the department planned 
to restructure or cancel the investment, or had already done so. DOD 
officials further stated that their CIO ratings provide a measured 
assessment of how DOD believes an investment will perform in the 
future. 

Although the CIO ratings submitted by DOD to the Dashboard are 
consistent with their ratings approach, they do not reflect other 
available information about the risk of these investments. As we 
previously noted, none of DOD's investments that were active in March 
2012 were rated as high risk, and approximately 85 percent were rated 
as either low risk or moderately low risk throughout their time on the 
Dashboard. However, these ratings did not always reflect significant 
schedule delays, cost increases, and other weaknesses identified for 
certain investments in our recent reviews, or problems with those 
investments identified in a recent report by the DOD Inspector 
General.[Footnote 20] 

Based on the department's long-standing difficulties with such 
programs, we designated DOD business systems modernization as a high-
risk area in 1995 and it remains a high-risk area today. More 
recently, we reported weaknesses in several of the department's 
business system investments.[Footnote 21] Specifically, we reported 
that the department had not effectively ensured that these systems 
would deliver capabilities on time and within budget; that acquisition 
delays required extended funding for duplicative legacy systems; that 
delays and cost overruns were likely to erode the cost savings these 
systems were to provide; and that, ultimately, DOD's management of 
these investments was putting the department's transformation of 
business operations at risk. 

Although the following selected examples of DOD investments 
experienced significant performance problems and were included with 
those considered to be high-risk business system investments in our 
recent reviews of those systems, they were all rated low risk or 
moderately low risk by the DOD CIO. 

* Air Force's Defense Enterprise Accounting and Management System 
(DEAMS): DEAMS is the Air Force's target accounting system designed to 
provide accurate, reliable, and timely financial information. In early 
2012, GAO reported that DEAMS faced a 2-year deployment delay, an 
estimated cost increase of about $500 million for an original life-
cycle cost estimate of $1.1 billion (an increase of approximately 45 
percent), and that assessments by DOD users had identified operational 
problems with the system, such as data accuracy issues, an inability 
to generate auditable financial reports, and the need for manual 
workarounds.[Footnote 22] In July 2012, the DOD Inspector General 
reported that the DEAMS' schedule delays were likely to diminish the 
cost savings it was to provide, and would jeopardize the department's 
goals for attaining an auditable financial statement. DOD's CIO rated 
DEAMS low risk or moderately low risk from July 2009 through March 
2012. 

* Army's General Fund Enterprise Business System (GFEBS): GFEBS is an 
Army financial management system intended to improve the timeliness 
and reliability of financial information and to support the 
department's auditability goals. In early 2012, we reported that GFEBS 
faced a 10-month implementation delay, and that DOD users reported 
operational problems, including deficiencies in data accuracy and an 
inability to generate auditable financial reports.[Footnote 23] These 
concerns were reiterated by the DOD Inspector General in July 2012. 
DOD's CIO rated GFEBS as moderately low risk from July 2009 through 
March 2012. 

* Army's Global Combat Support System-Army (GCSS-Army): GCSS-Army is 
intended to improve the Army's supply chain management capabilities 
and provide accurate equipment readiness status reports, among other 
things. In March 2012, we reported that GCSS-Army was experiencing a 
cost overrun of approximately $300 million on an original life-cycle 
cost estimate of $3.9 billion (an increase of approximately 8 percent) 
and a deployment delay of approximately 2 years.[Footnote 24] DOD 
rated GCSS-Army as low or moderately low risk from July 2009 through 
March 2012. 

Explanations submitted by DOD with the CIO ratings for these 
investments did not provide meaningful insight for why they were rated 
at the lowest risk levels in the face of known issues.[Footnote 25] 
DOD officials told us that they rated these investments as low risk 
because, in their view, the cost and schedule variances listed above 
did not constitute significant risks. Officials explained that: (1) 
the cost variances were not that large compared to DOD's overall size 
and large amount of IT spending; (2) the schedule variance needed to 
be understood in the context that the average DOD large-scale IT 
program takes 7 years (or 84 months) to implement; and (3) that each 
of those programs had risk mitigation plans in place. However, the 
first two reasons are inconsistent with DOD's own risk management 
guidance,[Footnote 26] which recommends that risks be assessed against 
the program's own cost and schedule estimates, not other department 
investments. In addition, completing risk mitigation plans does not 
necessarily lower investment risk. DOD's guidance calls for 
implementing the mitigation plan and then reassessing resulting 
changes to the risk. Even if the department adopts these elements of 
its own guidance, the CIO's evaluation will be incomplete unless it 
also reflects the assessments of investment performance and risks 
identified by us and others. Until the department does so, CIO ratings 
for DOD's Dashboard investments may not be sufficiently accurate or 
useful for its TechStat sessions or OMB's management and oversight. 

CIO Ratings Present Benefits and Challenges for Agencies' Investment 
Management: 

Selected agencies identified various benefits associated with 
performing CIO ratings and Dashboard reporting in general. Almost all 
of the agencies (five of six) reported the following three benefits. 

* Increased quality of investment performance data. For example, one 
agency also reported that the Dashboard has made information about 
investments more understandable. 

* Greater transparency and visibility for CIOs and their staff into 
investment-and program-level performance data. One agency reported 
that its CIO was better able to conduct reviews with actual investment 
numbers, as opposed to self-reported data presented by the 
investment's program managers. Agencies could also compare their 
investments' ratings to those of other agencies and departments. 

* Increased focus on project management practices. Two agencies 
reported improved investment performance as a direct result of their 
Dashboard rating and reporting activities; another stated that 
Dashboard reporting supported and reinforced their existing IT 
governance, capital planning, and program management processes. 

Some of these benefits were interrelated. Several agencies viewed the 
improved data quality as a by-product of greater scrutiny brought 
about by having to report such data to the Dashboard on a regular 
basis. One agency response noted that their program managers were 
surprised to see the extent to which investment data were visible to 
the public, and that this visibility motivated their staff to provide 
accurate and timely data (which has improved data quality). Another 
agency noted that the visibility of the IT Dashboard has increased 
awareness among investment and project managers about the need to 
improve the planning of project activities and the definition of 
operational performance metrics (which support program management). 

Nevertheless, agencies also identified challenges associated with 
producing and reporting CIO ratings. First, three agencies reported a 
challenge associated with the time and effort required to gather, 
validate, and gain internal approval for CIO ratings and other data 
reported to the Dashboard. For example, one agency reported that, due 
to the number of organizations involved and the number of investments 
being evaluated, it generally takes 90 to 120 days to develop and 
update its CIO ratings. The agency further reported that this effort 
was separate from (and in addition to) time it already spends on its 
own internal processes for managing and overseeing acquisition 
programs. 

Second, four of the six agencies identified challenges with the number 
of changes OMB has made to the Dashboard, as well as with the 
timeliness and clarity of OMB's communication regarding those changes. 
For example, officials at one agency commented that the frequency of 
changes has actually hindered their efforts to improve data quality, 
since errors sometimes resulted when it adapted to changes required by 
OMB. Officials at another agency stated that OMB allowed insufficient 
time for agencies to test their systems' interfaces with the Dashboard 
when changes were made, which they said resulted in data errors and 
challenges for staff. These officials also noted that OMB's guidance 
for agency submissions has, at times, not matched the technical data 
schemas implemented by OMB, impeding agencies' efforts to successfully 
upload their data. An OMB staff member commented that their office 
releases changes to the Dashboard as early in the fiscal year as 
possible to give agencies time to adjust and that OMB announces 
planned changes to agencies before they are implemented via the 
Dashboard's interagency web portal. OMB has recently held meetings 
with agency officials to discuss these issues and determine ways to 
better communicate going forward. 

Finally, one agency responded that while monthly updates to the 
Dashboard have increased investment and project managers' attention to 
the performance of their investments and projects, this regular 
scrutiny could encourage investment and project managers to "perform 
to the test" rather than concentrate on effective investment and 
project management. However, based on the interrelationships of the 
benefits of CIO ratings identified by some agencies, the process of 
generating and reporting CIO ratings does not have to be just a 
grading exercise. As previously noted, the benefit of improved 
investment performance data for the CIO's investment evaluation can 
lead to more effective management, which could, in turn, improve 
investment performance. Executives and staff who can envision these 
results from the Dashboard's CIO evaluations may be less likely to 
view the additional time and effort required to generate the CIO 
ratings as a challenge, but as an opportunity for more efficient and 
effective management. 

Conclusions: 

Since its inception in 2009, the Federal IT Dashboard has increased 
the transparency of the performance of major federal IT investments. 
Its CIO ratings, in particular, have improved visibility into changes 
in the risk levels of agencies' investments over time. Determining 
whether such changes represent improvements or deficiencies in 
management and oversight can be difficult without additional 
information on investment performance and the rating process, but 
analyzing and reporting the ratings for investments and agencies over 
time for the President's budget submission could help OMB ensure that 
risk is accurately assessed and that patterns of risk deserving of 
special management attention are identified. 

DOD demonstrated one such pattern of interest in its CIO ratings. 
During the 34-month life of the Dashboard, none of the 87 investments 
that were active as of March 2012 were rated high risk or moderately 
high risk, and approximately 85 percent of ratings were low risk or 
moderately low risk. Although DOD implemented OMB's broad instructions 
for producing CIO ratings, it also considered how the ratings might 
increase the likelihood of an OMB review of an investment and 
minimized the effects of significant schedule delays and cost 
increases, which were identified in our reviews and those of DOD's 
Inspector General. As a result, DOD is masking significant investment 
risks, has not employed its own risk management guidance, and has not 
delivered the transparency intended by the Dashboard. By incorporating 
the results of external reviews into its evaluations, DOD can further 
improve the quality of the information on which investment risk 
ratings are based. 

Beyond the transparency they promote, CIO ratings present an 
opportunity to improve the data and processes agencies use to assess 
investment risk. Some agencies have already experienced collateral 
benefits and management results from their risk evaluations. 
Continuing focus from OMB and agencies on how to accurately portray 
and derive value from the ratings and the associated processes could 
enable agencies to experience such benefits. 

Recommendations for Executive Action: 

To ensure that OMB's preparation of the President's budget submission 
accurately reflects the risks associated with all major IT 
investments, we are recommending that the Federal CIO analyze agency 
trends reflected in Dashboard CIO ratings, and present the results of 
this analysis with the President's annual budget submission. 

To ensure that DOD's CIO evaluations of investment risk for its major 
IT Dashboard investments reflect all available performance assessments 
and are consistent with the department's own guidance for managing 
risk, we are recommending that the Secretary of Defense direct the 
department's CIO to reassess the department's considerations for 
assigning CIO risk levels for Dashboard investments, including 
assessments of investment performance and risk from outside the 
programs, and apply the appropriate elements of the department's risk 
management guidance to OMB's evaluation factors in determining CIO 
ratings. 

Agency Comments and Our Evaluation: 

We provided a draft of our report to the six agencies selected for our 
review and to OMB. In oral comments, staff from OMB's Office of E-
Government & Information Technology stated that OMB concurred with our 
recommendation that the Federal CIO analyze agency trends reflected in 
Dashboard CIO ratings and present the results of this analysis with 
the President's annual budget submission. OMB staff also provided 
technical comments, which we incorporated as appropriate. In a written 
response, DOD's Deputy Chief Information Officer for Information 
Enterprise agreed with our recommendation that the department's CIO 
reassess considerations for assigning CIO risk levels for Dashboard 
investments, and committed to updating the department's CIO ratings 
process to better report risk and improve the timeliness and 
transparency of reporting. DOD's written response is reprinted in 
Appendix IV. Officials at DOI provided technical comments, which we 
incorporated as appropriate. The remaining agencies had no comment on 
the draft report. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to interested 
congressional committees; the Secretaries of Defense, Interior, 
Homeland Security, Health and Human Services, the Director of the 
National Science Foundation, the Director of the Office of Personnel 
Management, the Director of the Office of Management and Budget; and 
other interested parties. In addition, the report will be available at 
no charge on the GAO website at [hyperlink, http://www.gao.gov]. 

If you or your staffs have any questions on the matters discussed in 
this report, please contact David A. Powner at (202) 512-9286 or by e-
mail at pownerd@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this report. GAO staff who made major contributions to this 
report are listed in appendix IV. 

Signed by: 

David A. Powner: 
Director, Information Technology Management Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objectives were to (1) characterize the Chief Information Officer 
(CIO) ratings for selected federal agencies' information technology 
(IT) investments as reported over time on the federal IT Dashboard; 
(2) determine how agencies' approaches for assigning and updating CIO 
ratings vary; and (3) describe the benefits and challenges associated 
with agencies' approaches to the CIO rating. 

To establish the scope of our review, we downloaded and examined data 
on total IT spending for fiscal year 2011 for the 27 agencies reported 
on the IT Dashboard. (The Office of Management and Budget (OMB) 
extracts these data based on exhibit 300 forms submitted by each 
agency.) We then selected six agencies that spanned a range of IT 
spending for fiscal year 2011, including the three highest spending 
agencies, two of the lowest, and an agency in the middle. 
Collectively, these agencies accounted for approximately $51 billion, 
or 65 percent, of 2011 spending on IT investments. The six agencies 
are the Department of Defense, Department of Homeland Security, 
Department of Health and Human Services, Department of the Interior, 
National Science Foundation, and Office of Personnel Management. The 
results in this report represent only these agencies. 

To address the first objective, we downloaded and examined the 
Dashboard's CIO ratings for all investments at the six agencies we 
selected (a total of approximately 308 investments reported by these 
agencies).[Footnote 27] To characterize the numbers and percentages of 
major IT investments at each risk level at each of our subject 
agencies, we analyzed, summarized, and--where appropriate--graphically 
depicted average CIO ratings for investments by agencies over time 
during the period from June 2009 to March 2012. Specifically, we 
compared the CIO ratings in June 2009 (or whenever an individual 
investment was first rated) up through and including each investment's 
rating as of March 2012 and summarized the data by agency. To describe 
whether CIO ratings indicated higher or lower investment risk over 
time, we calculated the numbers and percentages of investments (by 
agency and collectively for all the agencies) that maintained a 
constant rating over the entire performance period, and those that 
experienced a change to their CIO rating in at least one rating 
period. Then we analyzed the subset of investments that experienced at 
least one changed rating and compared the first CIO rating with the 
latest CIO rating (no later than March 2012) to determine the numbers 
and percentages of investments (by agency and collectively for all the 
agencies) that experienced a net rating increase, a net rating 
decrease, or no net change. We also examined the comments provided 
with the ratings to determine whether such comments were useful in 
understanding the ratings. We presented our results to each agency and 
OMB and solicited their input, explanations for the results, and 
additional corroborating documentation, where appropriate. 

To address our second objective, we reviewed available documentation, 
obtained written responses to questions we posed to all agencies, and 
interviewed OMB and agency officials to determine their policies and 
practices related to assigning and updating the CIO ratings and 
related data for the Dashboard. Specifically, we gathered descriptions 
about the data, participants, and processes used to generate CIO 
ratings for investments; when and under what circumstances each agency 
updates its ratings; the specific factors agencies used in assigning 
their ratings; and the reason(s) for their approaches to assigning and 
reporting the ratings. We reviewed our results with agency officials 
to ensure that our presentation of their approach was accurate. In 
addition, we utilized our prior work and a report by the Department of 
Defense's Office of the Inspector General related to the department's 
major IT investments. We compared the findings in these reports to the 
CIO ratings the department submitted to the Dashboard for investments 
that had been rated consistently low or moderately low risk, and 
discussed our results with department officials. 

To address our third objective, we reviewed written and oral 
descriptions of the benefits and challenges that agencies and OMB have 
experienced in developing, submitting, updating, and utilizing CIO 
ratings. We sought specific examples, corroborating documentation, and 
causal factors, where available. After obtaining this information from 
individual agencies, we compared their responses to identify benefits 
and challenges common to multiple agencies and applied our judgment in 
determining whether any additional benefits or challenges were present. 

We conducted this performance audit from January 2012 to September 
2012 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Risk Levels for Investments at Selected Agencies, as of 
March 2012: 

The table below lists the total number of major information technology 
(IT) investments rated on the federal IT Dashboard as of March 2012 
for each agency selected for this review, with the numbers of 
investments rated at each of the risk levels specified by the Office 
of Management and Budget (OMB) for the chief information officer (CIO) 
rating. The last line in the table reports each agency's total budget 
for fiscal year 2012 for their major IT investments, as also reported 
on the Dashboard in March 2012. 

Table 8: CIO Ratings and Budget Totals for Selected Agencies' Major IT 
Investments as of March 2012: 

High risk and moderately high risk: 
DOD: 0; 
DHS: 3; 
HHS: 3; 
DOI: 6; 
OPM: 0; 
NSF: 0. 

Medium risk: 
DOD: 13; 
DHS: 35; 
HHS: 19; 
DOI: 8; 
OPM: 3; 
NSF: 0. 

Low risk and moderately low risk: 
DOD: 74; 
DHS: 45; 
HHS: 59; 
DOI: 34; 
OPM: 6; 
NSF: 5. 

Total number of investments rated on the Dashboard: 
DOD: 87; 
DHS: 83; 
HHS: 81; 
DOI: 48; 
OPM: 9; 
NSF: 5. 

Budget for major IT investments (FY 2012): 
DOD: $13.3 billion; 
DHS: $4.4 billion; 
HHS: $2.8 billion; 
DOI: $865.6 million; 
OPM: $46.3 million; 
NSF: $85.5 million. 

Source: GAO analysis of data downloaded from OMB's IT Dashboard. 

Note: Table does not include downgraded or eliminated investments. 

[End of table] 

[End of section] 

Appendix III: CIO Ratings and Net Changes for Major IT Investments at 
Selected Agencies, June 2009 through March 2012: 

This appendix provides additional information about chief information 
officer (CIO) ratings for major IT information technology (IT) 
investments at each of the agencies selected for this review. The 
first figure for each agency depicts the number of investments at each 
rating level for the end of each month, as reported on the federal IT 
Dashboard.[Footnote 28] The second figure depicts the number of 
investments whose risk level demonstrated a net increase, net 
decrease, no net change, or remained constant during the investment's 
entire time on the Dashboard.[Footnote 29] 

Department of Defense: 

Figure 6: CIO Ratings for Major IT Investments at the Department of 
Defense, as Reported on the IT Dashboard from July 2009 through March 
2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the Dashboard: 

Month: July 2009; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2009; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2009; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2009; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2009; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2009; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2010; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2010; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2010; 
Low risk: 26; 
Moderately low risk: 17; 
Medium risk: 5; 
Moderately high risk: 0; 
High risk: 0. 

Month: April 2010; 
Low risk: 31; 
Moderately low risk: 13; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: May 2010; 
Low risk: 31; 
Moderately low risk: 13; 
Medium risk: 6; 
Moderately high risk: 0; 
High risk: 0. 

Month: June 2010; 
Low risk: 30; 
Moderately low risk: 13; 
Medium risk: 7; 
Moderately high risk: 0; 
High risk: 0. 

Month: July 2010; 
Low risk: 30; 
Moderately low risk: 13; 
Medium risk: 7; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2010; 
Low risk: 30; 
Moderately low risk: 12; 
Medium risk: 8; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2010; 
Low risk: 30; 
Moderately low risk: 12; 
Medium risk: 8; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2010; 
Low risk: 30; 
Moderately low risk: 12; 
Medium risk: 8; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2010; 
Low risk: 40; 
Moderately low risk: 21; 
Medium risk: 11; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2010; 
Low risk: 40; 
Moderately low risk: 21; 
Medium risk: 11; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2011; 
Low risk: 40; 
Moderately low risk: 21; 
Medium risk: 11; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2011; 
Low risk: 40; 
Moderately low risk: 21; 
Medium risk: 11; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2011; 
Low risk: 41; 
Moderately low risk: 15; 
Medium risk: 16; 
Moderately high risk: 0; 
High risk: 0. 

Month: April 2011; 
Low risk: 41; 
Moderately low risk: 15; 
Medium risk: 16; 
Moderately high risk: 0; 
High risk: 0. 

Month: May 2011; 
Low risk: 41; 
Moderately low risk: 15; 
Medium risk: 16; 
Moderately high risk: 0; 
High risk: 0. 

Month: June 2011; 
Low risk: 35; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: July 2011; 
Low risk: 35; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2011; 
Low risk: 35; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2011; 
Low risk: 35; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2011; 
Low risk: 50; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2011; 
Low risk: 50; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2011; 
Low risk: 50; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2012; 
Low risk: 50; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2012; 
Low risk: 50; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2012; 
Low risk: 50; 
Moderately low risk: 24; 
Medium risk: 13; 
Moderately high risk: 0; 
High risk: 0. 

Source: GAO analysis of data from OMB's IT Dashboard. 

[End of figure] 

Note: Figure does not include downgraded or eliminated investments. 

Figure 7: Changes to Risk Levels for Major IT Investments at the 
Department of Defense, Initial CIO Rating through March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level constant: 51; 
Risk level increased: 18; 
Risk level reduced: 15; 
No net change in risk level: 3. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. An 
investment's CIO rating is constant when the rating remained unchanged 
from the initial rating through March 2012. All other categories of 
risk in the chart compare the initial CIO rating to the CIO rating as 
of March 2012, ignoring any interim changes that may have occurred in 
between those dates. 

[End of figure] 

Department of Homeland Security: 

Figure 8: CIO Ratings for Major IT Investments at the Department of 
Homeland Security, as Reported on the IT Dashboard from July 2009 
through March 2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the Dashboard: 

Month: July 2009; 
Low risk: 4; 
Moderately low risk: 27; 
Medium risk: 23; 
Moderately high risk: 7; 
High risk: 2. 

Month: August 2009; 
Low risk: 4; 
Moderately low risk: 28; 
Medium risk: 23; 
Moderately high risk: 6; 
High risk: 2. 

Month: September 2009; 
Low risk: 4; 
Moderately low risk: 28; 
Medium risk: 23; 
Moderately high risk: 6; 
High risk: 2. 

Month: October 2009; 
Low risk: 4; 
Moderately low risk: 28; 
Medium risk: 23; 
Moderately high risk: 6; 
High risk: 2. 

Month: November 2009; 
Low risk: 4; 
Moderately low risk: 28; 
Medium risk: 23; 
Moderately high risk: 6; 
High risk: 2. 

Month: December 2009; 
Low risk: 4; 
Moderately low risk: 28; 
Medium risk: 23; 
Moderately high risk: 6; 
High risk: 2. 

Month: January 2010; 
Low risk: 4; 
Moderately low risk: 28; 
Medium risk: 23; 
Moderately high risk: 6; 
High risk: 2. 

Month: February 2010; 
Low risk: 5; 
Moderately low risk: 26; 
Medium risk: 22; 
Moderately high risk: 8; 
High risk: 2. 

Month: March 2010; 
Low risk: 5; 
Moderately low risk: 26; 
Medium risk: 22; 
Moderately high risk: 8; 
High risk: 2. 

Month: April 2010; 
Low risk: 3; 
Moderately low risk: 33; 
Medium risk: 23; 
Moderately high risk: 9; 
High risk: 0. 

Month: May 2010; 
Low risk: 5; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 9; 
High risk: 0. 

Month: June 2010; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 26; 
Moderately high risk: 9; 
High risk: 0. 

Month: July 2010; 
Low risk: 6; 
Moderately low risk: 29; 
Medium risk: 26; 
Moderately high risk: 10; 
High risk: 0. 

Month: August 2010; 
Low risk: 6; 
Moderately low risk: 29; 
Medium risk: 26; 
Moderately high risk: 9; 
High risk: 1. 

Month: September 2010; 
Low risk: 6; 
Moderately low risk: 29; 
Medium risk: 26; 
Moderately high risk: 9; 
High risk: 1. 

Month: October 2010; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 8; 
High risk: 1. 

Month: November 2010; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 8; 
High risk: 1. 

Month: December 2010; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 8; 
High risk: 1. 

Month: January 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 8; 
High risk: 1. 

Month: February 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 8; 
High risk: 1. 

Month: March 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 27; 
Moderately high risk: 8; 
High risk: 1. 

Month: April 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 28; 
Moderately high risk: 7; 
High risk: 1. 

Month: May 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 28; 
Moderately high risk: 7; 
High risk: 1. 

Month: June 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 28; 
Moderately high risk: 7; 
High risk: 1. 

Month: July 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 28; 
Moderately high risk: 7; 
High risk: 1. 

Month: August 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 28; 
Moderately high risk: 7; 
High risk: 1. 

Month: September 2011; 
Low risk: 6; 
Moderately low risk: 30; 
Medium risk: 33; 
Moderately high risk: 7; 
High risk: 1. 

Month: October 2011; 
Low risk: 7; 
Moderately low risk: 37; 
Medium risk: 37; 
Moderately high risk: 1; 
High risk: 1. 

Month: November 2011; 
Low risk: 7; 
Moderately low risk: 38; 
Medium risk: 35; 
Moderately high risk: 2; 
High risk: 1. 

Month: December 2011; 
Low risk: 7; 
Moderately low risk: 38; 
Medium risk: 35; 
Moderately high risk: 2; 
High risk: 1. 

Month: January 2012; 
Low risk: 7; 
Moderately low risk: 38; 
Medium risk: 35; 
Moderately high risk: 2; 
High risk: 1. 

Month: February 2012; 
Low risk: 7; 
Moderately low risk: 38; 
Medium risk: 35; 
Moderately high risk: 3; 
High risk: 0. 

Month: March 2012; 
Low risk: 7; 
Moderately low risk: 38; 
Medium risk: 35; 
Moderately high risk: 3; 
High risk: 0. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 

[End of figure] 

Figure 9: Changes to Risk Levels for Major IT Investments at the 
Department of Homeland Security, Initial CIO Rating through March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level constant: 42; 
Risk level increased: 12; 
Risk level reduced: 26; 
No net change in risk level: 3. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. An 
investment's CIO rating is constant when the rating remained unchanged 
from the initial rating through March 2012. All other categories of 
risk in the chart compare the initial CIO rating to the CIO rating as 
of March 2012, ignoring any interim changes that may have occurred in 
between those dates. 

[End of figure] 

Department of Health and Human Services: 

Figure 10: CIO Ratings for Major IT Investments at the Department of 
Health and Human Services, as Reported on the IT Dashboard from July 
2009 through March 2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the Dashboard: 

Month: July 2009; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2009; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2009; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2009; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2009; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2009; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2010; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2010; 
Low risk: 19; 
Moderately low risk: 26; 
Medium risk: 10; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2010; 
Low risk: 12; 
Moderately low risk: 36; 
Medium risk: 15; 
Moderately high risk: 2; 
High risk: 0. 

Month: April 2010; 
Low risk: 15; 
Moderately low risk: 34; 
Medium risk: 11; 
Moderately high risk: 5; 
High risk: 0. 

Month: May 2010; 
Low risk: 14; 
Moderately low risk: 35; 
Medium risk: 11; 
Moderately high risk: 5; 
High risk: 0. 

Month: June 2010; 
Low risk: 8; 
Moderately low risk: 39; 
Medium risk: 11; 
Moderately high risk: 5; 
High risk: 2. 

Month: July 2010; 
Low risk: 12; 
Moderately low risk: 32; 
Medium risk: 14; 
Moderately high risk: 5; 
High risk: 2. 

Month: August 2010; 
Low risk: 13; 
Moderately low risk: 27; 
Medium risk: 17; 
Moderately high risk: 6; 
High risk: 2. 

Month: September 2010; 
Low risk: 13; 
Moderately low risk: 25; 
Medium risk: 19; 
Moderately high risk: 6; 
High risk: 2. 

Month: October 2010; 
Low risk: 14; 
Moderately low risk: 30; 
Medium risk: 14; 
Moderately high risk: 7; 
High risk: 8. 

Month: November 2010; 
Low risk: 13; 
Moderately low risk: 30; 
Medium risk: 15; 
Moderately high risk: 7; 
High risk: 0. 

Month: December 2010; 
Low risk: 14; 
Moderately low risk: 29; 
Medium risk: 15; 
Moderately high risk: 7; 
High risk: 0. 

Month: January 2011; 
Low risk: 13; 
Moderately low risk: 30; 
Medium risk: 16; 
Moderately high risk: 6; 
High risk: 0. 

Month: February 2011; 
Low risk: 13; 
Moderately low risk: 35; 
Medium risk: 13; 
Moderately high risk: 4; 
High risk: 0. 

Month: March 2011; 
Low risk: 11; 
Moderately low risk: 38; 
Medium risk: 12; 
Moderately high risk: 4; 
High risk: 0. 

Month: April 2011; 
Low risk: 11; 
Moderately low risk: 38; 
Medium risk: 13; 
Moderately high risk: 3; 
High risk: 0. 

Month: May 2011; 
Low risk: 10; 
Moderately low risk: 41; 
Medium risk: 11; 
Moderately high risk: 3; 
High risk: 0. 

Month: June 2011; 
Low risk: 10; 
Moderately low risk: 41; 
Medium risk: 11; 
Moderately high risk: 3; 
High risk: 0. 

Month: July 2011; 
Low risk: 10; 
Moderately low risk: 41; 
Medium risk: 16; 
Moderately high risk: 3; 
High risk: 0. 

Month: August 2011; 
Low risk: 10; 
Moderately low risk: 41; 
Medium risk: 16; 
Moderately high risk: 3; 
High risk: 0. 

Month: September 2011; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Month: October 2011; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Month: November 2011; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Month: December 2011; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Month: January 2012; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Month: February 2012; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Month: March 2012; 
Low risk: 10; 
Moderately low risk: 49; 
Medium risk: 19; 
Moderately high risk: 3; 
High risk: 0. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 

[End of figure] 

Figure 11: Changes to Risk Levels for Major IT Investments at the 
Department of Health and Human Services, Initial CIO Rating through 
March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level constant: 17; 
Risk level increased: 25; 
Risk level reduced: 17; 
No net change in risk level: 22. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. An 
investment's CIO rating is constant when the rating remained unchanged 
from the initial rating through March 2012. All other categories of 
risk in the chart compare the initial CIO rating to the CIO rating as 
of March 2012, ignoring any interim changes that may have occurred in 
between those dates. 

[End of figure] 

Department of the Interior: 

Figure 12: CIO Ratings for Major IT Investments at the Department of 
the Interior, as Reported on the IT Dashboard from July 2009 through 
March 2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the Dashboard: 

Month: July 2009; 
Low risk: 10; 
Moderately low risk: 18; 
Medium risk: 5; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2009; 
Low risk: 10; 
Moderately low risk: 18; 
Medium risk: 5; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2009; 
Low risk: 10; 
Moderately low risk: 19; 
Medium risk: 4; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2009; 
Low risk: 10; 
Moderately low risk: 19; 
Medium risk: 4; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2009; 
Low risk: 10; 
Moderately low risk: 19; 
Medium risk: 4; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2009; 
Low risk: 10; 
Moderately low risk: 20; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2010; 
Low risk: 10; 
Moderately low risk: 20; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2010; 
Low risk: 10; 
Moderately low risk: 21; 
Medium risk: 2; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2010; 
Low risk: 10; 
Moderately low risk: 21; 
Medium risk: 2; 
Moderately high risk: 0; 
High risk: 0. 

Month: April 2010; 
Low risk: 10; 
Moderately low risk: 21; 
Medium risk: 1; 
Moderately high risk: 1; 
High risk: 0. 

Month: May 2010; 
Low risk: 14; 
Moderately low risk: 22; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: June 2010; 
Low risk: 14; 
Moderately low risk: 22; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: July 2010; 
Low risk: 14; 
Moderately low risk: 22; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: August 2010; 
Low risk: 15; 
Moderately low risk: 21; 
Medium risk: 1; 
Moderately high risk: 1; 
High risk: 0. 

Month: September 2010; 
Low risk: 15; 
Moderately low risk: 21; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: October 2010; 
Low risk: 15; 
Moderately low risk: 23; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: November 2010; 
Low risk: 15; 
Moderately low risk: 23; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: December 2010; 
Low risk: 15; 
Moderately low risk: 23; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: January 2011; 
Low risk: 15; 
Moderately low risk: 23; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: February 2011; 
Low risk: 15; 
Moderately low risk: 23; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: March 2011; 
Low risk: 14; 
Moderately low risk: 24; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: April 2011; 
Low risk: 14; 
Moderately low risk: 24; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: May 2011; 
Low risk: 14; 
Moderately low risk: 24; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: June 2011; 
Low risk: 14; 
Moderately low risk: 24; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: July 2011; 
Low risk: 14; 
Moderately low risk: 24; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: August 2011; 
Low risk: 14; 
Moderately low risk: 24; 
Medium risk: 9; 
Moderately high risk: 1; 
High risk: 0. 

Month: September 2011; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Month: October 2011; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Month: November 2011; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Month: December 2011; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Month: January 2012; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Month: February 2012; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Month: March 2012; 
Low risk: 13; 
Moderately low risk: 21; 
Medium risk: 8; 
Moderately high risk: 6; 
High risk: 0. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 

[End of figure] 

Figure 13: Changes to Risk Levels for Major IT Investments at the 
Department of the Interior, Initial CIO Rating through March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level constant: 31; 
Risk level increased: 11; 
Risk level reduced: 4; 
No net change in risk level: 2. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. An 
investment's CIO rating is constant when the rating remained unchanged 
from the initial rating through March 2012. All other categories of 
risk in the chart compare the initial CIO rating to the CIO rating as 
of March 2012, ignoring any interim changes that may have occurred in 
between those dates. 

[End of figure] 

National Science Foundation: 

Figure 14: CIO Ratings for Major IT Investments at the National 
Science Foundation, as Reported on the IT Dashboard from June 2009 
through March 2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the Dashboard: 

Month: July 2009; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2009; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2009; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2009; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2009; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2009; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2010; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2010; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2010; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: April 2010; 
Low risk: 2; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: May 2010; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: June 2010; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: July 2010; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2010; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2010; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2010; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2010; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2010; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: April 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: May 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: June 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: July 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: August 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: September 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: October 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2011; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2012; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2012; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2012; 
Low risk: 3; 
Moderately low risk: 2; 
Medium risk: 0; 
Moderately high risk: 0; 
High risk: 0. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 

[End of figure] 

Figure 15: Changes to Risk Levels for Major IT Investments at the 
National Science Foundation, Initial CIO Rating through March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level constant: 4; 
Risk level reduced: 1. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. An 
investment's CIO rating is constant when the rating remained unchanged 
from the initial rating through March 2012. All other categories of 
risk in the chart compare the initial CIO rating to the CIO rating as 
of March 2012, ignoring any interim changes that may have occurred in 
between those dates. 

[End of figure] 

Office of Personnel Management: 

Figure 16: CIO Ratings for Major IT Investments at the Office of 
Personnel Management, as Reported on the IT Dashboard from July 2009 
through March 2012: 

[Refer to PDF for image: stacked vertical bar graph] 

Number of investments on the Dashboard: 

Month: July 2009; 
Low risk: 1; 
Moderately low risk: 3; 
Medium risk: 1; 
Moderately high risk: 1; 
High risk: 0. 

Month: August 2009; 
Low risk: 2; 
Moderately low risk: 2; 
Medium risk: 1; 
Moderately high risk: 1; 
High risk: 0. 

Month: September 2009; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 1; 
Moderately high risk: 1; 
High risk: 0. 

Month: October 2009; 
Low risk: 3; 
Moderately low risk: 1; 
Medium risk: 1; 
Moderately high risk: 1; 
High risk: 0. 

Month: November 2009; 
Low risk: 2; 
Moderately low risk: 3; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: December 2009; 
Low risk: 2; 
Moderately low risk: 3; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: January 2010; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: February 2010; 
Low risk: 4; 
Moderately low risk: 1; 
Medium risk: 0; 
Moderately high risk: 1; 
High risk: 0. 

Month: March 2010; 
Low risk: 4; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 1; 
High risk: 0. 

Month: April 2010; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: May 2010; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: June 2010; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: July 2010; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: August 2010; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: September 2010; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: October 2010; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: November 2010; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 1; 
Moderately high risk: 2; 
High risk: 0. 

Month: December 2010; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 1; 
Moderately high risk: 2; 
High risk: 0. 

Month: January 2011; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 2; 
High risk: 0. 

Month: February 2011; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 2; 
High risk: 0. 

Month: March 2011; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 1; 
High risk: 0. 

Month: April 2011; 
Low risk: 5; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 1; 
High risk: 0. 

Month: May 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: June 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: July 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: August 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: September 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 2; 
Moderately high risk: 1; 
High risk: 0. 

Month: October 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: November 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: December 2011; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: January 2012; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: February 2012; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Month: March 2012; 
Low risk: 6; 
Moderately low risk: 0; 
Medium risk: 3; 
Moderately high risk: 0; 
High risk: 0. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. 

[End of figure] 

Figure 17: Changes to Risk Levels for Major IT Investments at the 
Office of Personnel Management, Initial CIO Rating through March 2012: 

[Refer to PDF for image: pie-chart] 

Risk level constant: 2; 
Risk level increased: 1; 
Risk level reduced: 5; 
No net change in risk level: 1. 

Source: GAO analysis of data from OMB's IT Dashboard. 

Note: Figure does not include downgraded or eliminated investments. An 
investment's CIO rating is constant when the rating remained unchanged 
from the initial rating through March 2012. All other categories of 
risk in the chart compare the initial CIO rating to the CIO rating as 
of March 2012, ignoring any interim changes that may have occurred in 
between those dates. 

[End of figure] 

[End of section] 

Appendix IV: Comments from the Department of Defense: 

Department of Defense: 
Chief Information Officer: 
6000 Defense Pentagon: 
Washington, D.C. 20301-6000: 

September 13, 2012: 

Mr. David A. Powner: 
Director, Information Technology Management: 
U.S. Government Accountability Office: 
441 G Street, NW, Washington, DC 20548: 

Dear Mr. Powner, 

This is the Department of Defense (DoD) response to the GAO draft 
report, GAO-12-923, 'Information Technology Dashboard: Opportunities 
Exist to Improve Transparency and Oversight of Investment Risk at 
Select Agencies,' dated August 16, 2012 (GAO Code 311264). 

Our comments on the draft report are attached. My point of contact is 
Mr. Kevin Garrison, 571-372-4473, Kevin.aarrison@osd.mil. 

Sincerely, 

Signed by: 

David L. DeVries: 
Deputy Chief Information Officer for Information Enterprise: 

[End of letter] 

GAO Draft Report Dated August 2012: 
GAO-12-923 (GAO Code 311264): 

"Information Technology Dashboard: Opportunities Exist to Improve 
Transparency and Oversight of Investment Risk at Select Agencies" 

Department of Defense Comments to the GAO Recommendations: 

Recommendation 1: To ensure that DOD's CIO evaluations of investment 
risk for its major IT Dashboard investments reflect all available 
performance assessments and are consistent with the departments' own 
guidance for managing risk, we are recommending that the Secretary of 
Defense direct the department's' CIO to reassess the department's 
considerations for assigning CIO risk levels for Dashboard 
investments, including assessments of investment performance and risk 
from outside the programs and apply the appropriate elements of the 
department's' risk management guidance to OMB's evaluation factors in 
determining CIO ratings. 

DoD Response: DoD concurs with the recommendation and will update our 
CIO ratings process to better report risk and improve timeliness and 
transparency. 

[End of section] 

Appendix V: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

David A. Powner, (202) 512-9286 or pownerd@gao.gov: 

Staff Acknowledgments: 

In addition to the contact name above, the following staff also made 
key contributions to this report: Paula Moore (Assistant Director), 
Neil Doherty, Lynn Espedido, Rebecca Eyler, Kate Feild, Dan Gordon, 
Andrew Stavisky, Sonya Vartivarian, Shawn Ward, Kevin Walsh, Jessica 
Waselkow, and Monique Williams. 

[End of section] 

Footnotes: 

[1] As reported by agencies to the Office of Management and Budget. 

[2] 40 U.S.C. § 11302(c). 

[3] U.S. Chief Information Officer, 25 Point Implementation Plan to 
Reform Federal Information Technology Management, The White House 
(Washington, D.C.: Dec. 9, 2010). 

[4] Office of Management and Budget, Budget of the United States 
Government, Analytical Perspectives. Fiscal Years 2012, 2013. 

[5] [hyperlink, http://www.itdashboard.gov]. 

[6] 40 U.S.C. § 11302(c). 

[7] [hyperlink, http://www.gao.gov/products/GAO-09-624T]; GAO, 
Information Technology: Treasury Needs to Better Define and Implement 
Its Earned Value Management Policy, [hyperlink, 
http://www.gao.gov/products/GAO-08-951] (Washington, D.C.: Sept. 22, 
2008); and Air Traffic Control: FAA Uses Earned Value Techniques to 
Help Manage Information Technology Acquisitions, but Needs to Clarify 
Policy and Strengthen Oversight, [hyperlink, 
http://www.gao.gov/products/GAO-08-756] (Washington, D.C.: July 18, 
2008). 

[8] GAO, Information Technology: Agencies and OMB Should Strengthen 
Processes for Identifying and Overseeing High Risk Projects, 
[hyperlink, http://www.gao.gov/products/GAO-06-647] (Washington, D.C.: 
June 15, 2006). 

[9] Office of Management and Budget, Budget of the United States 
Government, Analytical Perspectives. Fiscal Years 2005, 2006, 2007, 
2008, 2009. 

[10] [hyperlink, http://www.itdashboard.gov]. 

[11] Office of Management and Budget, Budget of the United States 
Government, Analytical Perspectives. Fiscal Years 2012, 2013. 

[12] Exhibit 53s list all of the IT projects and their associated 
costs within a federal organization. An exhibit 300 is also called the 
Capital Asset Plan and Business Case. It is used to justify resource 
requests for major IT investments and is intended to enable an agency 
to demonstrate to its own management, as well as to OMB, that a major 
project is well planned. 

[13] [hyperlink, http://www.gao.gov/products/GAO-10-701]. 

[14] [hyperlink, http://www.gao.gov/products/GAO-11-262]. 

[15] [hyperlink, http://www.gao.gov/products/GAO-12-210]. 

[16] Chief Information Officer Authorities, Memorandum for Heads of 
Executive Departments and Agencies, M-11-29 (Washington, D.C.: Aug. 8, 
2011). 

[17] CIO ratings for investments that were downgraded (no longer 
defined as a major investment) or eliminated during this period are 
not included in these percentages. 

[18] A reduction in risk level is indicated by a higher CIO rating. An 
increase in risk level is indicated by a lower CIO rating. 

[19] Office of the DOD CIO, "Summary of June 2011 Recommended Rating 
Changes" (including Backup Slides), June 2011 Update. 

[20] Department of Defense Office of Inspector General, Enterprise 
Resource Planning Systems Schedule Delays and Reengineering Weaknesses 
Increase Risks to DOD's Auditability Goals. DODIG-2012-111 
(Alexandria, Va.: July 13, 2012). 

[21] GAO, DOD Business Transformation: Improved Management Oversight 
of Business System Modernization Efforts Needed, [hyperlink, 
http://www.gao.gov/products/GAO-11-53] (Washington, D.C.: Oct. 7, 
2010) and DOD Financial Management: Reported Status of Department of 
Defense's Enterprise Resource Planning Systems, [hyperlink, 
http://www.gao.gov/products/GAO-12-565R] (Washington, D.C: Mar. 30, 
2012). 

[22] GAO, DOD Financial Management: Reported Status of Department of 
Defense's Enterprise Resource Planning Systems, [hyperlink, 
http://www.gao.gov/products/GAO-12-565R] (Washington, D.C.: Mar. 30, 
2012) and DOD Financial Management: Implementation Weaknesses in Army 
and Air Force Business Systems Could Jeopardize DOD's Auditability 
Goals, [hyperlink, http://www.gao.gov/products/GAO-12-134] 
(Washington, D.C.: Feb. 28, 2012). 

[23] [hyperlink, http://www.gao.gov/products/GAO-12-565R] and 
[hyperlink, http://www.gao.gov/products/GAO-12-134]. 

[24] [hyperlink, http://www.gao.gov/products/GAO-12-565R]. 

[25] Explanations included "program is being closely monitored" 
(Defense Enterprise Accounting and Management System), "program 
schedule being closely monitored" (General Fund Enterprise Business 
System), and "no outstanding issues or concerns" (Global Combat 
Support System-Army). 

[26] Department of Defense, Risk Management Guide for DOD Acquisition, 
Sixth Edition (Version 1.0), August 2006. 

[27] We did not independently evaluate the ratings reported by 
agencies. However, we determined that they were sufficiently reliable 
for the purposes of our objectives by confirming with each agency that 
the ratings that we downloaded from the IT Dashboard were complete, 
accurate, and reflected the data they had reported to OMB. 

[28] The Office of Management and Budget (OMB) directs agency CIOs to 
evaluate investments and assign ratings according to a five-point 
scale. The risk levels are: 5-low risk (green), 4-moderately low risk 
(green), 3-medium risk (yellow), 2-moderately high risk (red), and 1-
high risk (red). 

[29] An investment's risk level increased when it received a lower CIO 
rating in March 2012 compared with its initial Dashboard rating. An 
investment's risk level was reduced when it received a higher CIO 
rating in March 2012 compared with its initial Dashboard rating. An 
investment's CIO rating exhibited no net change when it received the 
same rating in March 2012 as its initial rating, despite any interim 
changes that may have occurred. An investment's CIO rating was 
constant when the rating remained unchanged from the initial CIO 
rating through March 2012. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548. 

[End of document]