This is the accessible text file for GAO report number GAO-13-87 
entitled 'Information Technology: Agencies Need to Strengthen 
Oversight of Billions of Dollars in Operations and Maintenance 
Investments' which was released on November 15, 2012. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to the Subcommittee on Federal Financial Management, Government 
Information, Federal Services, and International Security, Committee on 
Homeland Security and Governmental Affairs, U.S. Senate: 

October 2012: 

Information Technology: 

Agencies Need to Strengthen Oversight of Billions of Dollars in 
Operations and Maintenance Investments: 

GAO-13-87: 

GAO Highlights: 

Highlights of GAO-13-87, a report to the Subcommittee on Federal 
Financial Management, Government Information, Federal Services, and 
International Security, Committee on Homeland Security and 
Governmental Affairs, U.S. Senate. 

Why GAO Did This Study: 

Of the $79 billion federal agencies budgeted for IT in 2011, $54 
billion (about 69 percent) was reported to have been spent on the 
operations and maintenance of existing legacy IT systems-—commonly 
referred to as steady state investments. Given the size and magnitude 
of these investments, it is essential that agencies effectively manage 
them to ensure they continue to meet agency needs. As such, OMB 
directs agencies to periodically examine the performance of such 
investments against, among other things, established cost, schedule, 
and performance goals by performing annual OAs. 

GAO was asked to determine the extent to which federal agencies 
analyze the performance of steady state investments in accordance with 
OMB guidance. To do so, GAO (1) selected five agencies, DOD, HHS, DHS, 
Treasury, and VA, which reported spending $4.6 billion annually on 
major steady state investments; and (2) and compared their fiscal year 
2011 OAs to OMB criteria. GAO also analyzed documents and interviewed 
agency officials regarding any variances as well as their causes. 

What GAO Found: 

Federal agency assessments of the performance of information 
technology (IT) investments in operations and maintenance (O&M)—-
commonly referred to as operational analyses (OAs)-—vary 
significantly. Office of Management and Budget (OMB) guidance calls 
for agencies to develop an OA policy and perform such analyses 
annually to ensure steady state investments continue to meet agency 
needs. The guidance also includes 17 key factors (addressing areas 
such as cost, schedule, customer satisfaction, and innovation) that 
are to be assessed. The five agencies GAO reviewed varied in the 
extent to which they carried out these tasks. 

The Departments of Homeland Security (DHS) and Health and Human 
Services (HHS) developed a policy which included all OMB assessment 
factors and performed OAs. However, they did not include all 
investments and key factors. In particular, DHS analyzed 16 of its 44 
steady state investments, meaning 28 investments with annual budgets 
totaling $1 billion were not analyzed; HHS analyzed 7 of its 8 steady 
state investments. For OAs performed by DHS and HHS, both fully 
addressed approximately half of the key factors. With regard to the 
DHS and HHS investments that did not undergo an analysis or were not 
fully assessed against key factors, agency officials said this was due 
in part to program officials inconsistently applying OMB and agency 
guidance in conducting OAs and that OAs were not a priority. DHS and 
HHS have recently begun to take action to make OAs a priority and 
improve consistency. For example, DHS’s chief information officer 
recently issued a directive requiring all steady state IT investments 
to conduct analyses annually and plans to assign staff in the office 
of the chief information officer to review them to ensure they are 
complete. 

The Departments of Defense (DOD), the Treasury (Treasury), and 
Veterans Affairs (VA) did not develop a policy and did not perform 
analyses on their 23 major steady state investments with annual 
budgets totaling $2.1 billion. DOD and VA officials said that they did 
not have a policy or perform analyses because they measure the 
performance of steady state investments via development of plans and 
business cases submitted to OMB (called exhibit 300s) as part of the 
budget process. While these can be helpful in managing performance and 
do address aspects of the 17 key factors, they do not address 11 of 
the key factors. For example, the exhibit 300 does not address 
reviewing strategic business results and making recommendations to 
modify or terminate an investment. Treasury officials stated that they 
did not to perform OAs in 2011 and instead decided to use the time to 
develop a policy. However, the officials stated that they did not 
anticipate the policy to be completed until the end of this calendar 
year. 

Overall, these five agencies have steady state investments with a 
fiscal year 2011 budget of over $3 billion which have not undergone 
needed analyses. While OMB requires agencies to perform OAs, its 
existing guidance does not provide mechanisms that ensure the OAs are 
completed and allow public transparency into the results of the 
assessments. Until agencies address these shortcomings, there is 
increased risk that these agencies will not know whether the 
multibillion dollar investments fully meet their intended objectives. 

What GAO Recommends: 

GAO is recommending that DOD, Treasury, and VA develop an OA policy 
and conduct annual OAs; and that DHS and HHS ensure OAs are being 
performed for all investments and that all factors are fully assessed. 
GAO is also recommending that OMB revise its guidance to incorporate 
mechanisms to ensure OAs are completed and provide for increased 
transparency. In commenting on a draft of this report, OMB and the 
five agencies GAO reviewed agreed with its content and recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-13-87]. For more 
information, contact David A. Powner at (202) 512-9286 or 
pownerd@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Federal Agencies' Assessments of Major IT Steady State Investments Vary 
Significantly: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objective, Scope, and Methodology: 

Appendix II: Extent to Which DHS's and HHS's OAs Addressed Key Factors: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: Comments from the Department of Homeland Security: 

Appendix V: Comments from the Department of the Treasury: 

Appendix VI: Comments from the Department of Veterans Affairs: 

Appendix VII: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Total Reported Federal IT Spending by the 26 Key Agencies for 
Fiscal Year 2011 (in millions), in Descending Order of O&M Spending: 

Table 2: Total Steady State IT Investments, and Number of Investments 
for Five Agencies That Had OAs and Did Not Have OAs with Cost (in 
millions): 

Table 3: Extent to Which DHS Steady State IT Investments Underwent OAs 
in Fiscal Year 2011 (by Component and Cost in Millions): 

Table 4: Extent to Which HHS Steady State IT Investments Underwent OAs 
in Fiscal Year 2011 (by Component and Cost in Millions): 

Table 5: U.S. Customs and Border Protection: Analysis of Extent to 
Which Infrastructure's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 6: Office of the Chief Information Officer: Analysis of Extent to 
Which Homeland Secure Data Network's Fiscal Year 2011 OA Addressed OMB 
Key Factors: 

Table 7: Federal Emergency Management Agency: Analysis of Extent to 
Which Disaster Management E-Government Initiative's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Table 8: Federal Emergency Management Agency: Analysis of Extent to 
Which Integrated Financial Management Information System's Fiscal Year 
2011 OA Addressed OMB Key Factors: 

Table 9: U.S. Immigration and Customs Enforcement: Analysis of Extent 
to Which Intelligence Fusion System's Fiscal Year 2011 OA Addressed OMB 
Key Factors: 

Table 10: National Protection and Programs Directorate: Analysis of 
Extent to Which National Security and Emergency Preparedness Priority 
Telecommunication Services's Fiscal Year 2011 OA Addressed OMB Key 
Factors: 

Table 11: Transportation Security Administration: Analysis of Extent to 
Which Federal Air Marshal Service Mission Scheduling and Notification 
System's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 12: Transportation Security Administration: Analysis of Extent to 
Which Federal Air Marshal Service Network's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Table 13: Transportation Security Administration: Analysis of Extent to 
Which Hazardous Materials Endorsement Threat Assessment Program's 
Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 14: Transportation Security Administration: Analysis of Extent to 
Which Information Technology Infrastructure Program's Fiscal Year 2011 
OA Addressed OMB Key Factors: 

Table 15: Transportation Security Administration: Analysis of Extent to 
Which Secure Flight's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 16: U.S. Coast Guard: Analysis of Extent to Which Coast Guard 
Business Intelligence's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 17: U.S. Coast Guard: Analysis of Extent to Which Core Accounting 
System Suite's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 18: U.S. Coast Guard: Analysis of Extent to Which Direct Access's 
Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 19: U.S. Coast Guard: Analysis of Extent to Which Marine 
Information for Safety and Law Enforcement's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Table 20: U.S. Citizenship and Immigration Services: Analysis of Extent 
to Which Immigration Computer Linked Application Information Management 
System's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 21: Centers for Disease Control and Prevention: Analysis of 
Extent to Which National Select Agency Registry's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Table 22: Centers for Medicare and Medicaid Services: Analysis of 
Extent to Which Beneficiary e-Services's Fiscal Year 2011 OA Addressed 
OMB Key Factors: 

Table 23: Centers for Medicare and Medicaid Services: Analysis of 
Extent to Which Health Care Quality Improvement and Evaluation System's 
Fiscal Year 2011 OA Addressed OMB Key Factors: 

Table 24: Health Resources and Services Administration: Analysis of 
Extent to Which Electronic Handbooks Program Management Office's Fiscal 
Year 2011 OA Addressed OMB Key Factors: 

Table 25: Health Resources and Services Administration: Analysis of 
Extent to Which National Practitioner Data Bank's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Table 26: Indian Health Service: Analysis of Extent to Which 
Infrastructure, Office Automation, and Telecommunications's Fiscal 
Year 2011 OA Addressed OMB Key Factors: 

Table 27: National Institutes of Health: Analysis of Extent to Which 
Business Intelligence System's Fiscal Year 2011 OA Addressed OMB Key 
Factors: 

Figure: 

Figure 1: Percentages of Total IT Spending for Fiscal Years 2011 and 
2012 for the 26 Key Federal Agencies: 

Abbreviations: 

CIO: chief information officer: 

DOD: Department of Defense: 

DHS: Department of Homeland Security: 

HHS: Department of Health and Human Services: 

IT: information technology: 

NASA: National Aeronautics and Space Administration: 

OA: operational analysis: 

O&M: operations and maintenance: 

OMB: Office of Management and Budget: 

Treasury: Department of the Treasury: 

VA: Department of Veterans Affairs: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

October 16, 2012: 

The Honorable Thomas R. Carper: 
Chairman: 
The Honorable Scott P. Brown: 
Ranking Member: 
Subcommittee on Federal Financial Management, Government 
Information, Federal Services, and International Security: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

In fiscal year 2011, 26 key federal agencies reported spending 
approximately $79 billion on information technology (IT) systems to the 
Office of Management and Budget (OMB).[Footnote 1] Of the $79 billion, 
$54 billion was reported by the agencies to be spent on operations and 
maintenance (O&M), which consists of existing legacy systems (i.e., 
steady state) and systems that are in both development and O&M (known 
as mixed life cycle). Given the size and magnitude of these 
investments, it is important that agencies effectively manage the 
operations and maintenance of existing investments to ensure they (1) 
continue to meet agency needs, (2) deliver value, and (3) do not 
unnecessarily duplicate or overlap with other investments. 

OMB directs agencies to periodically examine the performance of these 
investments against, among other things, established cost, schedule, 
and performance goals. Specifically, OMB calls for agencies to perform 
annual operational analyses (OA), which is a key method for examining 
the performance of such investments in O&M. 

As requested, our objective was to determine the extent to which 
federal agencies assess the performance of steady state IT investments 
in accordance with this OMB guidance. To do so, we selected five 
agencies, the Departments of Defense (DOD), Health and Human Services 
(HHS), Homeland Security (DHS), the Treasury (Treasury), and Veterans 
Affairs (VA), which have the largest budgets for major steady state IT 
investments, accounting for approximately $37 billion annually or about 
70 percent of all reported O&M spending in fiscal year 2011. In doing 
this we focused on these agencies' 75 major IT investments valued at 
$4.6 billion annually that were strictly in the O&M phase (i.e., 
excluded mixed cycle investments). We determined whether the agencies 
developed OA policies in accordance with OMB guidance. We also 
determined whether these agencies were conducting OAs to manage these 
investments. More specifically, we reviewed all of these agencies' OAs 
performed during fiscal year 2011 and compared them to OMB and related 
criteria. 

We conducted this performance audit from October 2011 to September 2012 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objective. Details on our objective, 
scope, and methodology are contained in appendix I. 

Background: 

In fiscal year 2011, the 26 key federal agencies that report to OMB on 
their IT investments reported spending approximately $79 billion on a 
wide variety of IT systems. Of this amount, agencies reported spending 
$54 billion on O&M for existing steady state investments; they plan on 
spending about $53 billion in fiscal year 2012. As shown in figure 1, 
these amounts represent a significant majority (i.e., 69 and 71 
percent) of the overall reported IT spending in 2011 ($79 billion) and 
that planned for 2012 ($75 billion), respectively. 

Figure 1: Percentages of Total IT Spending for Fiscal Years 2011 and 
2012 for the 26 Key Federal Agencies: 

[Refer to PDF for image: 2 pie-charts] 

2011: 
O&M: 69%; 
Development: 31%. 

2012: 
O&M: 71%; 
Development: 29%. 

Source: GAO based on OMB data. 

[End of figure] 

Although O&M spending governmentwide is about 70 percent of total IT 
spending, the amount spent by each agency varies from a high of 98 
percent to a low of 45 percent (as shown in the table below). 

Table 1: Total Reported Federal IT Spending by the 26 Key Agencies for 
Fiscal Year 2011 (in millions), in Descending Order of O&M Spending: 

Agency: Department of Defense[A]; 
Total: $37,120; 
Development: $13,896; 
O&M: $23,224; 
O&M percentage of total spending: 62.6%. 

Agency: Department of Health and Human Services; 
Total: $7,030; 
Development: $1,721; 
O&M: $5,309; 
O&M percentage of total spending: 75.5%. 

Agency: Department of Homeland Security; 
Total: $5,987; 
Development: $1,631; 
O&M: $4,356; 
O&M percentage of total spending: 72.8%. 

Agency: Department of the Treasury; 
Total: $3,419; 
Development: $553; 
O&M: $2,866; 
O&M percentage of total spending: 83.8%. 

Agency: Department of Veterans Affairs; 
Total: $3,193; 
Development: $980; 
O&M: $2,213; 
O&M percentage of total spending: 69.3%. 

Agency: Department of Agriculture; 
Total: $2,504; 
Development: $409; 
O&M: $2,095; 
O&M percentage of total spending: 83.7%. 

Agency: Department of Justice; 
Total: $2,991; 
Development: $978; 
O&M: $2,013; 
O&M percentage of total spending: 67.3%. 

Agency: Department of Energy; 
Total: $1,998; 
Development: $266; 
O&M: $1,732; 
O&M percentage of total spending: 86.7%. 

Agency: National Aeronautics and Space Administration; 
Total: $1,777; 
Development: $173; 
O&M: $1,604; 
O&M percentage of total spending: 90.2%. 

Agency: Department of Commerce; 
Total: $2,337; 
Development: $840; 
O&M: $1,496; 
O&M percentage of total spending: 64.0%. 

Agency: Department of Transportation; 
Total: $2,926; 
Development: $1,613; 
O&M: $1,313; 
O&M percentage of total spending: 44.9%. 

Agency: Department of State; 
Total: $1,400; 
Development: $94; 
O&M: $1,306; 
O&M percentage of total spending: 93.3%. 

Agency: Department of the Interior; 
Total: $1,028; 
Development: $132; 
O&M: $896; 
O&M percentage of total spending: 87.2%. 

Agency: Social Security Administration; 
Total: $1,471; 
Development: $727; 
O&M: $743; 
O&M percentage of total spending: 50.5%. 

Agency: Department of Labor; 
Total: $607; 
Development: $113; 
O&M: $495; 
O&M percentage of total spending: 81.5%. 

Agency: General Services Administration; 
Total: $599; 
Development: $106; 
O&M: $493; 
O&M percentage of total spending: 82.4%. 

Agency: Department of Education; 
Total: $580; 
Development: $97; 
O&M: $483; 
O&M percentage of total spending: 83.3%. 

Agency: Environmental Protection Agency; 
Total: $467; 
Development: $76; 
O&M: $391; 
O&M percentage of total spending: 83.7%. 

Agency: Department of Housing and Urban Development; 
Total: $546; 
Development: $210; 
O&M: $336; 
O&M percentage of total spending: 61.6%. 

Agency: Nuclear Regulatory Commission; 
Total: $153; 
Development: $18; 
O&M: $136; 
O&M percentage of total spending: 88.5%. 

Agency: U.S. Agency for International Development; 
Total: $145; 
Development: $31; 
O&M: $115; 
O&M percentage of total spending: 78.9%. 

Agency: Small Business Administration; 
Total: $122; 
Development: $19; 
O&M: $103; 
O&M percentage of total spending: 84.2%. 

Agency: National Science Foundation; 
Total: $95; 
Development: $12; 
O&M: $82; 
O&M percentage of total spending: 86.9%. 

Agency: National Archives and Records Administration; 
Total: $143; 
Development: $70; 
O&M: $73; 
O&M percentage of total spending: 51.1%. 

Agency: Office of Personnel Management; 
Total: $79; 
Development: $10; 
O&M: $69; 
O&M percentage of total spending: 86.8%. 

Agency: Smithsonian Institution; 
Total: $66; 
Development: $4; 
O&M: $62; 
O&M percentage of total spending: 94.1%. 

Agency: Total; 
Total: $78,784; 
Development: $24,779; 
O&M: $54,005; 
O&M percentage of total spending: 68.5%. 

Source: GAO analysis based on OMB data. 

[A] While the Army Corps of Engineers submits information on its IT 
investments to OMB separate from the Department of Defense's 
submission, we have included it here as part of the Department of 
Defense. 

[End of table] 

The National Aeronautics and Space Administration (NASA) reported 
spending approximately 90 percent of its total IT spending on O&M with 
the remaining 10 percent going to new investments. The reason for this 
mix of spending, according to NASA officials, is due to NASA's mission 
(i.e., the space shuttle mission) which relies heavily on legacy 
systems. By contrast, the Department of Transportation reported 
spending approximately 45 percent of its total IT costs on O&M with the 
other 55 percent going to new investments. According to department 
officials, this mix of spending is largely due to the fact that the 
department has a number of IT development investments underway that 
involve large financial commitments relative to O&M investments. 

OMB's Roles and Responsibilities: 

for Overseeing IT Investments, 

Including Operations and Maintenance: 

To assist agencies in managing their investments, Congress enacted the 
Clinger-Cohen Act of 1996, which requires OMB to establish processes to 
analyze, track, and evaluate the risks and results of major capital 
investments in information systems made by federal agencies and report 
to Congress on the net program performance benefits achieved as a 
result of these investments.[Footnote 2] Further, the act places 
responsibility for managing investments with the heads of agencies and 
establishes chief information officers to advise and assist agency 
heads in carrying out this responsibility. 

In carrying out its responsibilities, OMB uses several data collection 
mechanisms to oversee federal IT spending during the annual budget 
formulation process. Specifically, OMB requires 26 key federal 
departments and agencies to provide information to it related to their 
IT investments (called exhibit 53s) and capital asset plans and 
business cases (called exhibit 300s).[Footnote 3] 

* Exhibit 53. The purpose of the exhibit 53 is to identify all IT 
investments--both major and nonmajor[Footnote 4]--and their associated 
costs within a federal organization. Information included on agency 
exhibit 53s is designed, in part, to help OMB better understand what 
agencies are spending on IT investments. The information also supports 
cost analyses prescribed by the Clinger-Cohen Act. As part of the 
annual budget, OMB publishes a report on IT spending for the federal 
government representing a compilation of exhibit 53 data submitted by 
the 26 agencies. 

* Exhibit 300. The purpose of the exhibit 300 is to provide a business 
case for each major IT investment and to allow OMB to monitor IT 
investments once they are funded. Agencies are required to provide 
information on each major investment's cost, schedule, and performance. 

In addition, in June 2009, to further improve the transparency into and 
oversight of agencies' IT investments, OMB publicly deployed a website, 
known as the IT Dashboard (Dashboard), which replaced its Management 
Watch List and High-Risk List. As part of this effort, OMB issued 
guidance directing federal agencies to report, via the Dashboard, the 
performance of their IT investments. Currently, the Dashboard publicly 
displays information on the cost, schedule, and performance of over 700 
major federal IT investments at 26 federal agencies. In addition, the 
Dashboard allows users to download exhibit 53 data, which include 
information on both major and nonmajor investments. According to OMB, 
these data are intended to provide a near real-time perspective of the 
performance of these investments, as well as a historical perspective. 
Further, the public display of these data is intended to allow OMB, 
other oversight bodies, and the general public to hold the government 
agencies accountable for results and progress. Since the Dashboard has 
been implemented, we have reported and made recommendations to improve 
the data accuracy and reliability. In 2010 and 2011, we reported on the 
progress of the Dashboard and made recommendations to further improve 
how it rates investments relative to current performance.[Footnote 5] 

Further, OMB has developed guidance that calls for agencies to develop 
an OA policy for examining the ongoing performance of existing IT 
investments to measure, among other things, that the investment is 
continuing to meet business and customer needs and is contributing to 
meeting the agency's strategic goals.[Footnote 6] This guidance calls 
for the policy to provide for an annual OA of each investment that 
addresses the following: cost, schedule, customer satisfaction, 
strategic and business results, financial goals, and innovation. To 
address these areas, the guidance specifies the following 17 key 
factors that are to be addressed: 

* assessment of current costs against life-cycle costs; 

* a structured schedule assessment (i.e., measuring the performance of 
the investment against its established schedule); 

* a structured assessment of performance goals (i.e., measuring the 
performance of the investment against established goals); 

* identification of whether the investment supports customer processes 
as designed and is delivering goods and services it was designed to 
deliver; 

* a measure of the effect the investment has on the performing 
organization itself; 

* a measure of how well the investment contributes to achieving the 
organization's business needs and strategic goals; 

* a comparison of current performance with a pre-established cost 
baseline and estimates; 

* areas for innovation in the areas of customer satisfaction, strategic 
and business results, and financial performance; 

* indication if the agency revisited alternative methods for achieving 
the same mission needs and strategic goals; 

* consideration of issues, such as greater utilization of technology or 
consolidation of investments to better meet organizational goals; 

* an ongoing review of the status of the risks identified in the 
investment's planning and acquisition phases; 

* identification of whether there is a need to redesign, modify, or 
terminate the investment; 

* an analysis on the need for improved methodology (i.e., better ways 
for the investment to meet cost and performance goals); 

* lessons learned; 

* cost or schedule variances; 

* recommendations to redesign or modify an asset in advance of 
potential problems; and: 

* overlap with other investments. 

With regard to overseeing the agencies' development of policies and 
annual performance, OMB officials responsible for governmentwide OA 
policy stated that they expect agencies to perform all the steps 
specified in the guidance and to be prepared to show documentation as 
evidence of compliance with the guidance should OMB decide to check. 

Federal Agencies' Assessments of Major IT Steady State Investments Vary 
Significantly: 

Although OMB guidance calls for agencies to develop an OA policy and 
perform such analyses annually, the extent to which the five federal 
agencies we reviewed carried out these tasks varied significantly. 
Specifically, DHS and HHS developed a policy and conducted OAs, but in 
doing so, they excluded key investments and assessment factors. DOD, 
Treasury, and VA did not develop a policy or conduct OAs. The following 
table shows the total number of steady state investments for each 
agency, and provides the number and budgeted amount for those 
investments that underwent an assessment and those that did not. 

Table 2: Total Steady State IT Investments, and Number of Investments 
for Five Agencies That Had OAs and Did Not Have OAs with Cost (in 
millions): 

Agency (total investments in steady state): DOD (4); 
Total investments with an OA: 0; 
Fiscal year 2011 cost: [Empty]; 
Total investments without an OA: 4; 
Fiscal year 2011 cost: $381. 

Agency (total investments in steady state): DHS (44); 
Total investments with an OA: 16; 
Fiscal year 2011 cost: $1,175; 
Total investments without an OA: 28; 
Fiscal year 2011 cost: $1,011. 

Agency (total investments in steady state): HHS ( 8); 
Total investments with an OA: 7; 
Fiscal year 2011 cost: $207; 
Total investments without an OA: 1; 
Fiscal year 2011 cost: $77. 

Agency (total investments in steady state): Treasury (16); 
Total investments with an OA: 0; 
Fiscal year 2011 cost: [Empty]; 
Total investments without an OA: 16; 
Fiscal year 2011 cost: $152. 

Agency (total investments in steady state): VA (3); 
Total investments with an OA: 0; 
Fiscal year 2011 cost: [Empty]; 
Total investments without an OA: 3; 
Fiscal year 2011 cost: $1,600. 

Agency (total investments in steady state): Total (75); 
Total investments with an OA: 23; 
Fiscal year 2011 cost: $1,400; 
Total investments without an OA: 52; 
Fiscal year 2011 cost: $3,200. 

Source: GAO analysis based on OMB data. 

[End of table] 

Until agencies more completely address their policy and performance 
shortcomings, there is increased risk that existing multibillion dollar 
investments will continue to be funded although it is not fully known 
whether they meet their intended objectives. 

DHS and HHS Developed an OA Policy and Performed OAs, but Did Not 
Address All Investments and Key Factors: 

DHS and HHS had developed policies, which contained all performance 
factors identified in OMB's guidance. Specifically, 

* In 2008, DHS issued its policy called "Operational Analysis 
Guidance."[Footnote 7] The guidance states that OAs should be performed 
on an annual basis to evaluate the operational results of agency steady 
state investments. In addition, the guidance provides a report template 
which includes sections that should be contained and reported on in it. 
DHS's policy addressed all of the key factors in the OMB guidance, 
including, for example, assessing current costs against life-cycle 
costs and a detailed schedule assessment. 

* In 2008, HHS issued its policy called "Practices Guide: Annual 
Operational Analysis."[Footnote 8] The guide states OAs are required to 
be performed on an annual basis. Further, the guide includes a template 
and a checklist for conducting them. In addition, agencies within the 
department have issued their own policy. For example, in 2011, The 
Centers for Disease Control and Prevention issued its "Operational 
Analysis Guide" and in 2010, the National Institutes of Health issued 
its framework, "A How-to Guide."[Footnote 9] These policies contained 
all of the key factors identified in the OMB policy, such as measuring 
the effect the investment has on the performing organization itself and 
identifying any areas for innovation. 

* Further, DHS and HHS performed OAs on some of their steady state 
investments, but not for all. Specifically, of their 52 total steady 
state investments, DHS and HHS conducted analyses on 23 with total 
budgets of $1.4 billion and did not conduct analyses on 29 investments 
with total budgets of $1.1 billion. More specifically, 

* Of DHS's 44 steady state investments, the department conducted OAs 
on16 of them, which have an annual budget of $1.2 billion; it did not 
perform analyses on the other 28, which have an annual budget of almost 
$1 billion. 

* Of HHS's 8 steady state investments, the department conducted 
analyses on 7 of them, which have an annual budget of $207 million; it 
did not perform an OA on the remaining investment, which has an annual 
budget of $77 million. 

Tables 3 and 4 show DHS's and HHS's steady state investments by 
component agency and whether OAs were performed on these investments in 
fiscal year 2011. (Details of our analysis of all the analyses and a 
brief description of the investments are included in appendix II.) 

Table 3: Extent to Which DHS Steady State IT Investments Underwent OAs 
in Fiscal Year 2011 (by Component and Cost in Millions): 

Component: U.S. Customs and Border Protection; 

Investment: 1. Advance Passenger Information System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $4.78. 

Investment: 2. Automated Targeting System Maintenance; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $33.89. 

Investment: 3. SBInet Block 1; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $27.65. 

Investment: 4. Infrastructure; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $556.69. 

Investment: 5. Land Border Integration; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $75.78. 

Investment: 6. Non-Intrusive Inspection Systems Program; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $110.49. 

Investment: 7. Systems, Applications, and Products; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $18.21. 

Component: DHS, Office of the Chief Information Officer; 

Investment: 8. Homeland Secure Data Network; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $47.66. 

Component: Federal Emergency Management Agency; 

Investment: 9. Disaster Management E-Government Initiative; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $12.27. 

Investment: 10. Infrastructure; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $56.14. 

Investment: 11. Integrated Financial Management Information System; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $4.30. 

Investment: 12. National Flood Insurance Program Technology Systems 
and Services; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $10.83. 

Component: U.S. Immigration and Customs Enforcement; 

Investment: 13. Federal Financial Management System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $24.69. 

Investment: 14. Intelligence Fusion System; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $7.36. 

Component: National Protection and Programs Directorate; 

Investment: 15. Information Systems Security Line of Business; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $3.02. 

Investment: 16. National Security and Emergency Preparedness Priority 
Telecommunication Services; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $46.05. 

Investment: 17. Visitor and Immigrant Status Indicator Technology 
Arrival and Departure Information System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $20.65. 

Investment: 18. Visitor and Immigrant Status Indicator Technology 
Automated Biometric Identification System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $104.36. 

Component: Office of Health Affairs; 

Investment: 19. BioSurveillance Common Operating Network; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $1.49. 

Component: Transportation Security Administration; 

Investment: 20. Crew Vetting; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $11.60. 

Investment: 21. Federal Air Marshal Service Mission Scheduling and 
Investment underwent OAs: Notification System; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $14.13. 

Investment: 22. Federal Air Marshal Service Network; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $41.79. 

Investment: 23. Hazardous Materials Endorsement Threat Assessment 
Program; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $12.00. 

Investment: 24. Information Technology Infrastructure Program; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $284.20. 

Investment: 25. Performance Management Information System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $10.64. 

Investment: 26. Secure Flight; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $84.19. 

Investment: 27. Transportation Worker Identification Credential; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $9.20. 

Investment: 28. Transportation Security Administration Operating 
Platform; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $121.80. 

Component: U.S. Coast Guard; 

Investment: 29. Asset Logistics Management Information System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $8.15. 

Investment: 30. Coast Guard Business Intelligence; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $6.29. 

Investment: 31. Core Accounting System Suite; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $32.13. 

Investment: 32. Direct Access; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $9.64. 

Investment: 33. Infrastructure Standard Workstation Infrastructure 
Recapitalization and Sustainment; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $62.93. 

Investment: 34. Infrastructure Coast Guard One; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $25.00. 

Investment: 35. Marine Information for Safety and Law Enforcement; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $10.39. 

Investment: 36. Ports and Waterways Safety System IT; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $38.40. 

Investment: 37. Vessel Logistics System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $4.85. 

Component: United States Citizenship and Immigration Services; 

Investment: 38. Customer Service Web Portal; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $20.89. 

Investment: 39. Immigration Computer Linked Application Information 
Management System 3; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $5.99. 

Investment: 40. Infrastructure (End User Support); 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $111.42. 

Investment: 41. Integrated Document Production; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $48.14. 

Investment: 42. Naturalization Computer Linked Application Information 
Management System 4; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $2.42. 

Component: United States Secret Service; 

Investment: 43. Enterprise Financial Management System; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $5.67. 

Investment: 44. Information Technology Infrastructure; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $38.03. 

Total; 
Investment: 44; 
Investment underwent OAs: Yes: 16; 
Investment underwent OAs: No: 28; 
Cost for fiscal year 2011: $2,186.20. 

Source: GAO analysis based on OMB data. 

[End of table] 

Table 4: Extent to Which HHS Steady State IT Investments Underwent OAs 
in Fiscal Year 2011 (by Component and Cost in Millions): 

Component: Centers for Disease Control and Prevention; 

Investment: 1. Information Technology Infrastructure; 
Yes: [Empty]; 
Investment underwent OAs: No; 
Cost for fiscal year 2011: $77.09. 

Investment: 2. National Select Agency Registry; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $5.05. 

Component: Centers for Medicare and Medicaid Services; 

Investment: 3. Beneficiary e-Services; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $60.82. 

Investment: 4. Health Care Quality Improvement and Evaluation System; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $21.20. 

Component: Health Resources and Services Administration; 

Investment: 5. Electronic Handbooks Program Management Office; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $23.69. 

Investment: 6. National Practitioner Data Bank; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $23.51. 

Component: Indian Health Service; 

Investment: 7. Infrastructure, Office Automation, and 
Telecommunications; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $55.31. 

Component: National Institutes of Health; 

Investment: 8. Business Intelligence System; 
Investment underwent OAs: Yes; 
Cost for fiscal year 2011: $17.60. 

Component: Total; 
Investment: 8; 
Investment underwent OAs: Yes: 7; 
Investment underwent OAs: No: 1; 
Cost for fiscal year 2011: $284.27. 

Source: GAO analysis based on OMB data. 

[End of table] 

In addition, although DHS and HHS performed analyses, the agencies did 
not address all key factors in conducting them. Specifically, 

* of DHS's 16 OAs, which were to include a total 272 key factors, DHS: 

* addressed 145 (or 53 percent), 

* partially addressed 20 (or 7 percent), and: 

* did not address 107 (or 39 percent); and: 

* of HHS's 7 OAs, which were to include a total of 119 key factors, 
HHS: 

* addressed 66 (or 55 percent), 

* partially addressed 6 (or 5 percent), and: 

* did not address 47 (or 39 percent) factors. 

The following provides key examples by component agency to illustrate 
how factors were fully addressed, partially addressed, or not addressed 
at all. 

* In its operational analysis of its U.S. Coast Guard Business 
Intelligence investment, the U.S. Coast Guard fully addressed five key 
factors (see table 16 in appendix II). For example, on the factor 
regarding 
whether the investment supports customer processes as designed and is 
delivering the goods and services it was designed to deliver, the 
component measured (via surveys) customer satisfaction, usage trends, 
system trends, and feedback, and used this information to implement 
system improvements. U.S. Coast Guard partially addressed three 
factors. For example, in assessing performance goals, the component 
identified two major goals of the investment, but did not include how 
or when these goals were to be achieved. U.S. Coast Guard did not 
address nine key factors, including those on identifying lessons 
learned and reviewing the status of risk versus cost, schedule, and 
performance. These factors are important because they provide 
management with key information on why problems occurred and how they 
can be avoided in the future, as well as whether the investment is 
worth pursing given anticipated costs, benefits, and associated risks. 

* In assessing the Information Technology Infrastructure Program, 
Transportation Security Administration addressed eight key factors (see 
table 14 in appendix II). For example, on the factor calling for 
performance of a structured schedule assessment, the component analyzed 
a detailed list of task descriptions, start and end dates, and planned 
versus actual costs to ensure the investment is performing against an 
established schedule which can minimize costs over the life cycle of an 
investment. The component partially addressed one key factor; 
specifically, the factor calling for identifying whether the investment 
supports customer processes and is delivering the goods and services 
intended. In assessing this factor, Transportation Security 
Administration conducted surveys to measure customer satisfaction, but 
in doing so did not include measures to assess whether the investment 
was delivering the goods and services it was designed to deliver. The 
component did not address eight key factors. For example, it did not 
identify any areas for innovation or whether the investment overlapped 
with other systems. These latter steps are essential to identifying 
investment improvements, increasing value and reducing costs, and 
eliminating duplicate systems and the costs associated with them. 

* For the U.S. Immigration and Customs Enforcement's Intelligence 
Fusion System, the component fully addressed nine key factors (see 
table 9 in appendix II). These factors included analyzing current costs 
against life-cycle costs and whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver, through measures such as customer surveys and help 
desk metrics. The component partially addressed the factor on 
identifying areas (e.g., business results and customer satisfaction, 
financial performance) for innovation. Specifically, it identified two 
areas for innovation, namely strategic and business results and 
customer satisfaction, but did not address financial performance. U.S. 
Immigration and Customs Enforcement did not address seven key factors; 
for example, it did not identify lessons learned or assess whether to 
modify or terminate the investment. Fully addressing these factors is 
crucial to agencies in determining whether to continue an investment 
that is not performing as required. 

* For its Infrastructure, Office Automation, and Telecommunications 
investment, Indian Health Service fully addressed 14 key factors (see 
table 26 in appendix II). For example, in addressing the factor on 
assessing performance goals, it analyzed the investment's performance 
goals against the results to date for each goal. The component 
partially addressed the factor on the status of risks versus cost, 
schedule, and performance. Specifically, it analyzed cost and schedule 
progress, but did not include an assessment of risks. Indian Health 
Service did not address two key factors. For example, it did not 
identify lessons learned and whether the investment overlapped with 
other systems. Addressing these factors is important because they help 
agencies to, among other things, identify where cost-effective 
improvements can be made. 

* HHS's Health Resources and Services Administration fully addressed 15 
key factors in its operational analysis of its Electronic Handbooks 
Program Management Office (see table 24 in appendix II). For example, 
it conducted a structured assessment of performance goals, including a 
detailed list of goals, and how and when they were addressed. Health 
Resources and Services Administration only partially addressed the key 
factor on providing a structured schedule assessment. Specifically, the 
component identified certain parts of the investment schedule, such as 
standard and unscheduled maintenance efforts, but other schedule 
elements, such as completion dates and goals, were not identified. 
Health Resources and Services Administration did not address one 
factor. For example, it did not assess current costs against life-cycle 
costs. This factor is important because it can, among other things, 
provide information to agency decision makers with answers to whether 
annual operating and maintenance costs are comparable to the estimated 
costs developed during the development phase. 

* In its analysis of the Business Intelligence System, National 
Institutes of Health fully assessed six key factors (see table 27 in 
appendix II). For example, on the factor calling for identifying 
whether the investment supports customer processes as designed and is 
delivering the goods and services it was designed to deliver, the 
component analyzed user and customer assessments that showed 
improvement in this area. The component partially addressed the key 
factor on measuring the effect an investment has on the performing 
organization itself. For example, National Institutes of Health 
identified that the investment was in line with the appropriate 
component enterprise architecture, but did not identify the effect the 
investment had on other aspects of the department such as its mission 
and business processes. National Institutes of Health did not address 
10 factors, including lessons learned and determining whether to modify 
or terminate the investment. These factors are critical to whether to 
continue an investment that is not performing as required. 

With regard to why analyses were not performed on all investments and 
why those that were conducted did not fully address all factors, DHS 
and HHS attributed these shortfalls to the following: 

* Officials from DHS's Office of the CIO who are responsible for 
overseeing the performance of OAs department-wide told us the 
components only performed 16 of the 44 analyses and did not address all 
key factors (in the 16 OAs that were performed) because they were not 
consistently implementing department and OMB policy as they should have 
because it was not a priority. To illustrate their point, the officials 
told us that while most components strive to perform annual analyses, 
other components do not require them to be conducted on an annual basis 
citing other tasks as taking precedence. To address these shortfalls, 
the department recently took steps to make OAs a priority and to ensure 
consistent application of department and OMB policy. Specifically, in 
May 2012, DHS's CIO issued a memorandum stating that all steady state 
IT investments are required to have an annual OA completed no later 
than June of each fiscal year and that component CIOs are to work with 
program managers to implement and ensure compliance with DHS OA 
requirements. Further, as part of this initiative, DHS's CIO plans to 
assign resources and responsibility to CIO office staff to review and 
ensure compliance with DHS's policy. These are steps in the right 
direction; however, the DHS CIO officials told us that these 
initiatives will not be fully implemented until sometime in fiscal year 
2013. 

* HHS officials from the Office of the CIO said their shortfalls (i.e., 
one component did not perform an OA and those that were performed did 
not address all factors) were due, in part, to inconsistent 
implementation of department and OMB policy across the components due 
to analyses not being a priority. As an example of this, officials from 
the office of the HHS CIO who are responsible for overseeing the 
department OA program cited how they had planned to implement an 
initiative to annually review all analyses performed by the components 
to ensure consistency and quality but have not been able to do so due 
to limited CIO staff being assigned to other initiatives. 

Although DHS and HHS had 23 investments--with collective annual budgets 
of $1.4 billion--that underwent OAs, these investments were not 
thoroughly assessed against all key factors. Until these agencies 
assess all steady state investments and ensure that they are fully 
assessed against factors, there is increased risk that these agencies 
will not know whether the multibillion dollar investments fully meet 
their intended objectives. 

DOD, Treasury, and VA Did Not Develop Policies or Perform OAs: 

DOD, Treasury, and VA had not developed a policy for performing OAs and 
did not conduct OAs for their 23 steady state investments that have 
combined annual budgets of $2.1 billion. Specifically, 

* DOD did not conduct analyses for its 4 major investments that have 
annual budgets totaling $381 million, 

* Treasury did not conduct such analyses for its16 major investments 
that have annual budgets totaling $152 million, and: 

* VA did not conduct OAs for its 3 major investments that have annual 
budgets totaling $1.6 billion. 

Regarding why DOD and VA had not developed policies and are not 
performing analyses, officials from those agencies stated that in lieu 
of conducting OAs, they assess the performance of steady state 
investments as part of developing their annual exhibit 300 submissions 
to OMB. While we have previously reported that using the exhibit 300 
process can be a tool to manage investment performance,[Footnote 10] 
our analysis shows that the process does not fully address 11 of the 17 
factors. For example, the exhibit 300 process does not fully provide 
for addressing the following factors: 

* identifying alternative methods for achieving the same mission needs 
and strategic goals. Doing this is important because it helps agencies 
assess whether they are using the most cost effective solution to 
achieving agency goals; 

* addressing greater utilization of technology or consolidation of 
investments will better meet organizational goals. It is also critical 
to helping agencies ensure that their investments are meeting 
performance goals in the most cost-effective manner; 

* identifying lessons learned, why problems occurred, or how savings 
were realized, which is essential to avoid repeating the same mistakes, 
which helps saving resources; and: 

* identifying where the agency needs to redesign, modify, or terminate 
the investment, which is a means to achieving solutions that return the 
greatest benefit for funds invested. 

Further, OMB officials told us that the exhibit 300 process is not a 
substitute for conducting OAs. Although OMB requires OAs for all steady 
state systems, its guidance does not provide a mechanism for ensuring 
they are completed and submitted to OMB for review. In particular, it 
does not have a reporting mechanism that provides for public 
transparency into the results of these assessments, which the IT 
Dashboard could provide. Having such a mechanism for the performance of 
steady state systems is consistent with Clinger-Cohen Act requirements 
that call for OMB to analyze and report on the performance of IT 
capital investments. Moreover, such public disclosure promotes 
increased transparency which is one of OMB's goals in establishing the 
IT Dashboard. 

Treasury officials from the department's office of the CIO told us they 
decided not to perform OAs in 2011 and instead decided to use the time 
to develop a policy. However, the officials stated that they did not 
anticipate the policy to be completed until the end of this calendar 
year. 

Until these agencies establish policies and begin performing OA 
assessments for their steady state investments, there is increased risk 
that these agencies will not know whether the multibillion dollar 
investments fully meet their intended objectives, therefore increasing 
the potential for waste and duplication. 

Conclusions: 

The federal government has made a multibillion dollar commitment to 
operating and maintaining its IT investments. OMB has established 
guidance for federal agencies to use to evaluate the performance of 
such investments, including whether a sound basis exists for agencies 
to continue funding them. DHS and HHS had developed policies in 
accordance with OMB guidance and performed analyses, but did not do so 
for all of their investments and their analyses did not address all key 
factors. During the course of this review, DHS reiterated the 
importance of performing OAs and issued a memorandum with initiatives 
to address the department's shortcomings. Further, DOD, Treasury, and 
VA had not developed a policy nor had they performed OAs. Taken 
together, these five agencies continue to invest billions of dollars 
each year on IT steady state investments without ensuring that they are 
continuing to meet agency needs and are delivering value. These 
shortcomings are due in part to a number of factors, including agencies 
relying on budget submission processes through their annual exhibit 300 
submissions, which are not intended as a substitute for OAs, and not 
viewing performance of these assessments as a priority. Although OMB's 
current guidance does not require agencies to report on OAs to OMB, 
using existing oversight and transparency tools like the IT Dashboard 
could help ensure that these important performance assessments are 
completed and available for public viewing. Nonetheless, until the 
agencies address these shortcomings and ensure all their steady state 
investments are fully assessed, there is increased potential for these 
multibillion dollar investments to result in unnecessary waste and 
duplication. 

Recommendations for Executive Action: 

To ensure that major steady state IT investments are being adequately 
analyzed, we recommend that the Secretaries of Defense, Veterans 
Affairs, and the Treasury direct appropriate officials to develop an OA 
policy, annually perform OAs on all investments, and ensure the 
assessments include all key factors. 

In addition, we recommend that the Secretaries of Homeland Security and 
Health and Human Services direct their Chief Information Officers to 
ensure OAs are performed annually on all major steady state investments 
and the assessments include all key factors. 

Further, to ensure that OA policies are developed and that annual 
analyses are conducted and to promote transparency into the results of 
these analyses, we recommend that the Director of OMB revise existing 
guidance to include directing agencies to report on the IT Dashboard 
the results from the OAs of their steady state investments. 

Agency Comments and Our Evaluation: 

In commenting on a draft of this report, OMB and the five agencies 
agreed with our findings and recommendations. Their comments are 
discussed in more detail below. 

* In oral comments, staff from OMB's Office of E-Government and 
Information Technology concurred with our recommendations and stated 
that OMB had recently initiated an effort to address the specific 
recommendation directed to it. Specifically, the staff stated that 
OMB's fiscal year 2014 budget guidance (dated August 3, 2012) directs 
agencies to include OAs as part of their exhibit 300 submissions to 
OMB. 

* In written comments--signed by DOD's Deputy Chief Information Officer 
for Information Enterprise and reprinted in appendix III--DOD concurred 
with our recommendation and said it plans to establish an OA policy in 
coordination with OMB. 

* In written comments--signed by the Director of the Departmental GAO-
OIG Liaison Office and reprinted in appendix IV--DHS concurred with our 
recommendation. The department, after receiving our draft report, 
identified and provided to us OAs that it had performed for 3 
additional investments in fiscal year 2011. DHS also provided technical 
comments which we incorporated in the report as appropriate. 

* In comments provided via e-mail from its GAO Intake Coordinator 
within the Office of the Assistant Secretary for Legislation, HHS 
stated that it did not have any general comments on the report. The 
department did provide technical comments which we incorporated in the 
report as appropriate. 

* In written comments--signed by the Deputy Assistant Secretary for 
Information Systems and Chief Information Officer and reprinted in 
appendix V--Treasury agreed with the report's recommendations. In 
addition, after receiving our draft report, the department identified 
and provided to us OAs that it had performed on 9 of its 17 investments 
in fiscal year 2011. 

* In written comments--signed by its Chief of Staff and reprinted in 
appendix VI--VA generally agreed with our conclusions and concurred 
with the recommendation to it. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to interested 
congressional committees; the Secretaries of Defense, Homeland 
Security, Health and Human Services, the Treasury, Veterans Affairs; 
and other interested parties. In addition, the report will be available 
at no charge on the GAO website at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions on the matters discussed in 
this report, please contact me at (202) 512-9286 or pownerd@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in appendix VII. 

Signed by: 

David A. Powner: 
Director, Information Technology Management Issues: 

[End of section] 

Appendix I: Objective, Scope, and Methodology: 

Our objective was to determine the extent to which selected federal 
agencies assess the performance of steady state information technology 
(IT) investments in accordance with Office of Management and Budget 
(OMB) guidance. To accomplish our objective, we selected the five 
agencies (Departments of Defense (DOD), Health and Human Services 
(HHS), Homeland Security (DHS), the Treasury (Treasury), and Veterans 
Affairs (VA)) that have the largest budgets for major steady state IT 
investments; collectively, these investments accounted for 
approximately $37 billion annually or about 70 percent of all reported 
IT operations and maintenance (O&M) spending. In particular, we focused 
on these agencies' 75 major IT investments valued at $4.6 billion 
annually that were strictly in the steady state phase as opposed to the 
agencies' other O&M investments--called mixed life-cycle investments by 
the agencies and OMB--which are not solely in O&M; these mixed life-
cycle investments have projects under development as well as projects 
being placed into O&M. 

We analyzed OMB's guidance and identified a key practice called 
operational analysis (OA) that agencies are to use to assess the 
performance of existing O&M investments. We also interviewed OMB 
officials to corroborate our understanding of the key practice. 

We then determined whether the five agencies developed OA policies as 
called for by the OMB guidance. Specifically, we compared each agency's 
policy, if they had one, to the OMB criteria to determine the extent of 
compliance and where there were variances. We further determined 
whether the agencies were conducting analyses. Specifically, for the 75 
major investments, we determined whether the agencies had performed an 
OA on each of them. In those cases where one had been performed, we 
analyzed the agencies' efforts to address the OMB criteria in the 
analysis and categorized the extent to which the OMB key factors had 
been addressed using the following criteria: 

* Yes: if all aspects of the key factor specified in the OMB criteria 
were fully addressed. 

* No: if none of the aspects of the key factor were addressed. 

* Partial: if some, but not the entire key factor was addressed. 

In cases where agencies did not fully address factors (i.e., partially 
or not all), we analyzed documentation and interviewed agency 
officials, including staff from the offices of the chief information 
officers responsible for overseeing these investments, to assist in 
identifying causes for shortfalls and any actions planned or underway 
to address the causes. 

We conducted this performance audit from October 2011 to September 2012 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objective. 

[End of section] 

Appendix II: Extent to Which DHS's and HHS's OAs Addressed Key Factors: 

Tables 5-20 show our analysis for DHS's investments with OAs in fiscal 
year 2011. 

Table 5: U.S. Customs and Border Protection: Analysis of Extent to 
Which Infrastructure's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Customs and Border Protection 
included an analysis of performance measures against a performance 
baseline for, among other things, software and hardware maintenance 
and network availability; however, a measure of cost against its 
baseline was not included. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Customs and Border Protection 
found that the investment was currently meeting established 
performance goals. The assessment did not include an analysis on 
whether opportunities to improve the system’s efficiency had been 
identified. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 6: Office of the Chief Information Officer: Analysis of Extent 
to Which Homeland Secure Data Network's Fiscal Year 2011 OA Addressed 
OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Partial; 
Summary of partial rating: The Office of the Chief Information Officer 
identified which strategic goal the investment supports, namely to “
Strengthen and Unify DHS Operations and Management,” but it did not 
include measures of how well the investment contributes to achieving 
the goal. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Partial; 
Summary of partial rating: The Office of the Chief Information Officer 
assessed current performance with pre-established performance 
measures, but it did not analyze current cost performance against a 
cost baseline. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Partial; 
Summary of partial rating: The Office of the Chief Information Officer 
identified the investment’s risk management plan, as containing 
current risks, their status, and associated mitigation efforts. It 
also identified that current risks are reported at monthly meetings; 
however, it did not address specific risks in its assessment and the 
impact of these risks. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 7: Federal Emergency Management Agency: Analysis of Extent to 
Which Disaster Management E-Government Initiative's Fiscal Year 2011 
OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: No. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: No. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Partial; 
Summary of partial rating: Federal Emergency Management Agency 
identified the strategic mission, goals, and objectives that this 
investment supports, but it did not measure the effect the investment 
is having on the department and its mission. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Partial; 
Summary of partial rating: Federal Emergency Management Agency 
identified the strategic mission, goals, and objectives that this 
investment supports, but it did not provide any analysis on metrics 
showing the extent to which the investment is contributing to 
achieving the department’s business needs and strategic goals. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Partial; 
Summary of partial rating: Federal Emergency Management Agency 
identified how the investment’s project management office monitored 
risks through a risk management plan and risk register, which is to be 
updated biweekly, but it did not assess risks against investment cost, 
schedule, and performance. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: No. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 8: Federal Emergency Management Agency: Analysis of Extent to 
Which Integrated Financial Management Information System's Fiscal Year 
2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Partial; 
Summary of partial rating: Federal Emergency Management Agency used 
the investment’s risk management plan to identify risks and potential 
impact on schedule milestones, but this assessment did not identify 
the risk and potential impact on cost. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 9: U.S. Immigration and Customs Enforcement: Analysis of Extent 
to Which Intelligence Fusion System's Fiscal Year 2011 OA Addressed 
OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Immigration and Customs 
Enforcement identified opportunities for innovation in the areas of 
strategic and business results and customer satisfaction; it did not 
address whether there were any innovation opportunities in the area of 
financial performance. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Yes. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: No. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 10: National Protection and Programs Directorate: Analysis of 
Extent to Which National Security and Emergency Preparedness Priority 
Telecommunication Services's Fiscal Year 2011 OA Addressed OMB Key 
Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: Yes. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 11: Transportation Security Administration: Analysis of Extent 
to Which Federal Air Marshal Service Mission Scheduling and 
Notification System's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Partial; 
Summary of partial rating: Transportation Security Administration 
developed a schedule performance table and a framework to analyze 
results, but the schedule assessment omitted analysis of key schedule 
aspects, including actual start dates and percent completed. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Yes. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 12: Transportation Security Administration Analysis of Extent to 
Which Federal Air Marshal Service Network's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: No. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 13: Transportation Security Administration Fiscal Analysis of 
Extent to Which Hazardous Materials Endorsement Threat Assessment 
Program's Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Yes. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: Yes. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 14: Transportation Security Administration: Analysis of Extent 
to Which Information Technology Infrastructure Program's Fiscal Year 
2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Partial; 
Summary of partial rating: Transportation Security Administration 
stated that customer satisfaction is to be measured using surveys, 
service legal agreements, and key performance measures. However, the 
assessment did not include results of these efforts, and thus did not 
identify if the investment was meeting customer needs. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: No. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 15: Transportation Security Administration: Analysis of Extent 
to Which Secure Flight's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Yes. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 16: U.S. Coast Guard: Analysis of Extent to Which Coast Guard 
Business Intelligence's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Partial; 
Summary of partial rating: In its assessment, U.S. Coast Guard 
described how it has established two major goals for the investment, 
namely, to provide products and services which leverage standardized 
Enterprise Measures while ensuring repeatable answers, and to provide 
Enterprise Solutions for lower level reporting requirements; but the 
department did not analyze the performance to date against these 
established investment goals. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Partial; 
Summary of partial rating: U.S. Coast Guard described the investment 
as organizational knowledge that is directly relevant to decision 
making towards organizational goals, but it did not measure the effect 
the investment had on the component in performing its mission. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: Partial; 
Summary of partial rating: In its assessment, U.S. Coast Guard 
identified an area of focus for fiscal year 2011 that was to determine 
how the investment aligns strategically/tactically with other business 
intelligence investments. U.S. Coast Guard did not identify whether 
the Business Intelligence investment overlapped or duplicated 
functions performed by other business intelligence investments. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 17: U.S. Coast Guard: Analysis of Extent to Which Core 
Accounting System Suite's Fiscal Year 2011 OA Addressed OMB Key 
Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: Yes. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: Yes. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 18: U.S. Coast Guard: Analysis of Extent to Which Direct 
Access's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: Yes. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 19: U.S. Coast Guard: Analysis of Extent to Which Marine 
Information for Safety and Law Enforcement's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Coast Guard provided a 
summary of the investments’ performance goals for the past year, 
including significant deliverables, but it did not include how or when 
these goals were to be achieved nor did it include if any other goals 
remained outstanding. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 20: United States Citizenship and Immigration Services: Fiscal 
Analysis of Extent to Which Immigration Computer Linked Application 
Information Management System 3's Year 2011 OA Addressed OMB Key 
Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Citizenship and Immigration 
Services identified cost centers and said that they managed costs of 
the program well, but did not assess current costs against life-cycle 
costs. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Citizenship and Immigration 
Services identified schedule milestones, the management of scheduling, 
and identified assurance of the project moving forward on schedule, 
but it did not measure current progress on milestones against the 
established schedule. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Partial; 
Summary of partial rating: The assessment identified the financial 
management background of the system, its scope, and major costs. It 
also identified the financial performance results, and identified 
planned actions to improve financial performance, but it did not 
provide a comparison between the pre-established cost baseline and 
current performance. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Partial; 
Summary of partial rating: United States Citizenship and Immigration 
Services identified risks associated with the system, including risks 
that could affect schedule, among other things; however, it did not 
detail potential costs associated with the risks. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: Yes. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: Yes. 

Source: GAO analysis of DHS data. 

[End of table] 

Tables 21-27 show our analysis for HHS's investments with OAs in 
fiscal year 2011. 

Table 21: Centers for Disease Control and Prevention: Analysis of 
Extent to Which National Select Agency Registry's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: No. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 22: Centers for Medicare and Medicaid Services: Analysis of 
Extent to Which Beneficiary e-Services's Fiscal Year 2011 OA Addressed 
OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Partial; 
Summary of partial rating: Centers for Medicare and Medicaid Services 
determined that it improved population health by providing Medicare 
information via public websites and providing customer service 
channels for beneficiaries to manage their health; it did not analyze 
or identify the effect the investment had on the department and its 
mission. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: No. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: No. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 23: Centers for Medicare and Medicaid Services: Analysis of 
Extent to Which Health Care Quality Improvement and Evaluation 
System's Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: No. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: No. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: No. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: No. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: No. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: No. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 24: Health Resources and Services Administration: Analysis of 
Extent to Which Electronic Handbooks Program Management Office's 
Fiscal Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Partial; 
Summary of partial rating: Health Resources and Services 
Administration identified standard and unscheduled maintenance efforts 
performed as scheduled, but it did not comparatively analyze these 
efforts against completion dates and goals. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Yes. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: Yes. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: Yes. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: Yes. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 25: Health Resources and Services Administration: Analysis of 
Extent to Which National Practitioner Data Bank's Fiscal Year 2011 OA 
Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Partial; 
Summary of partial rating: Health Resources and Services 
Administration identified schedule assessment results that reflect a “
green” score for the last four quarters. It also identifies that 
earned value management is reporting no discrepancies in cost and 
schedule. However, the department did not perform a comparative 
analysis of the planned and actual schedule milestones. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Partial; 
Summary of partial rating: Health Resources and Services 
Administration identified the investment’s cost, schedule, and 
performance outcomes; but did not address the status of risks, and 
their potential impacts were not identified. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: Yes. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: Yes. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 26: Indian Health Service: Analysis of Extent to Which 
Infrastructure, Office Automation, and Telecommunications's Fiscal 
Year 2011 OA Addressed OMB Key Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: Yes. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: Yes. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Yes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: Yes. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: Yes. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: No. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: Yes. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: Partial; 
Summary of partial rating: Indian Health Service analyzed cost and 
schedule progress, but it did not include an assessment of the risks 
and potential impacts. 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: Yes. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: Yes. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: Yes. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

Table 27: National Institutes of Health: Analysis of Extent to Which 
Business Intelligence System's Fiscal Year 2011 OA Addressed OMB Key 
Factors: 

Key factor: 1. Assesses current costs against life-cycle cost; 
Yes, no, or partial: No. 

Key factor: 2. Includes a structured schedule assessment; 
Yes, no, or partial: No. 

Key factor: 3. Includes a structured assessment of performance goals; 
Yes, no, or partial: Yes. 

Key factor: 4. Identifies whether the investment supports customer 
processes as designed and is delivering the goods and services it was 
designed to deliver; 
Yes, no, or partial: Yes. 

Key factor: 5. Measures the effect the investment has on the 
performing organization itself; 
Yes, no, or partial: Partial; 
Summary of partial rating: National Institutes of Health identified 
that the investment was in line with the appropriate component’s 
enterprise architecture, but did not identify the effect the 
investment had on other aspects of the component, such as its mission 
and business processes. 

Key factor: 6. Includes a measure of how well the investment 
contributes to achieving the organization's business needs and 
strategic goals; 
Yes, no, or partial: No. 

Key factor: 7. Compares current performance with a pre-established 
cost baseline and estimates; 
Yes, no, or partial: No. 

Key factor: 8. Identifies any areas for innovation in the areas of 
customer satisfaction, strategic and business results, and financial 
performance; 
Yes, no, or partial: No. 

Key factor: 9. Identifies if the agency revisited alternative methods 
for achieving the same mission needs and strategic goals; 
Yes, no, or partial: Yes. 

Key factor: 10. Addresses issues such as greater utilization of 
technology or consolidation of investments to better meet 
organizational goals; 
Yes, no, or partial: No. 

Key factor: 11. Includes an ongoing review of the status of the risks 
identified in the investment's planning and acquisition phases; 
Yes, no, or partial: 

Key factor: 12. Identifies a need to redesign, modify, or terminate 
the investment; 
Yes, no, or partial: No. 

Key factor: 13. Includes an analysis on the need for improved 
methodology (i.e., better ways for the investment to meet cost and 
performance goals); 
Yes, no, or partial: Yes. 

Key factor: 14. Identifies any lessons learned; 
Yes, no, or partial: No. 

Key factor: 15. Identifies if the investment had a cost or schedule 
variance; 
Yes, no, or partial: Yes. 

Key factor: 16. Identifies recommendations to redesign or modify an 
asset before it becomes a problem; 
Yes, no, or partial: No. 

Key factor: 17. Includes information on the overlap of the investment 
with other systems; 
Yes, no, or partial: No. 

Source: GAO analysis of DHS data. 

[End of table] 

[End of section] 

Appendix III: Comments from the Department of Defense: 

Department of Defense: 
Chief Information Officer: 
6000 Defense Pentagon: 
Washington, D.C. 20301-6000: 

September 21, 2012: 

Mr. David A. Powner: 
Director, information Technology Management: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Powner, 

this is the Department of Defense (DoD) response to the GAO draft 
report, GA0-12-963, 'Information Technology: Agencies Need to 
Strengthen Oversight of Billions of Dollars in Operations and 
Maintenance,' dated August 28, 2012 (GAO Code 311261). 

Our comments on the draft report are attached. My point of contact is 
Mr. Kevin Garrison, 571-372-4473, Kevin.warrison@osd.mil. 

Sincerely, 

Signed by: 

David L. DeVries: 
Deputy Chief Information Officer for Information Enterprise: 

[End of letter] 

GAO Draft Report Dated August 2012: 
GAO-12-963 (GAO Code 311261): 

Information Technology: Agencies Need to Strengthen Oversight of
Billions of Dollars in Operations and Maintenance: 

Department of Defense Comments To The GAO Recommendations: 

Recommendation 1: "To ensure that major steady state IT investments 
arc being adequately analyzed, we recommend that the Secretaries of 
Defense. Veterans Affairs, and Treasury direct appropriate officials 
to develop an OA policy, annually perform ()As on all investments, and 
ensure the assessments include all key factors." 

DoD Response: DoT] concurs with the recommendation and will promulgate
Operational Analysis (OA) guidance in coordination with OMB. 

[End of section] 

Appendix IV: Comments from the Department of Homeland Security: 

U.S. Department of Homeland Security: 
Washington, DC 20528: 

September 21, 2012: 

David A. Powner: 
Director, Information Technology Management Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Re: Draft Report GAO-12-963, "Information Technology: Agencies Need To
Strengthen Oversight of Billions of Dollars In Operations and 
Maintenance Investments" 

Dear Mr. Powner: 

Thank you for the opportunity to review and comment on this draft 
report. The U.S. Department of Homeland Security (DHS) appreciates the 
U.S. Government Accountability Office's (GAO's) work in planning and 
conducting its review and issuing this report. 

The Department is pleased to note GAO's positive acknowledgment that 
the Department recently implemented steps to make Operational Analyses 
(OAs) a priority and to ensure consistent application of departmental 
and Office of Management and Budget (OMB) policy. 

The draft report contained one recommendation directly involving DHS 
with which the Department concurs. Specifically, GAO recommended that 
the Secretary of Homeland Security direct the Chief Information 
Officer (CIO) to: 

Recommendation: Ensure OAs are performed annually on all major steady 
state investments and the assessments include all key factors. 

Response: Concur. On May 24, 2012, the Department's CIO signed a 
memorandum requiring OAs to be completed annually. Operational 
Analysis Guidance (DHS Instruction Guide 102-02-002) has been drafted 
and is pending approval. The draft guidance includes all the key 
factors as prescribed by OMB in Circular A-130, Management of Federal 
Information Resources. In addition, a scoring template was created. 
The guidance has been distributed to the Capital Planning and 
Investment Control community to be shared with their Program Managers 
for the FY 2014 Exhibit 300 submission. Each OA submitted with the FY 
2014 Exhibit 300s has been reviewed on the basis of guidance, and 
feedback has been provided to the Components. The signed OAs will be 
sent to OMB in February 2013 with the final Exhibit 300s for the 
President's budget. 

DHS will continue to review the OAs annually for quality and adherence 
to our template. In the future, the annual OA review process will be 
incorporated in the portfolio review process, annual information 
technology budget review process, and/or CIO health assessment review 
process. 

Again, thank you for the opportunity to review and comment on this 
draft report. Technical comments were provided previously under 
separate cover. Please feel free to contact me if you have any 
questions. We took forward to working with you in the future. 

Sincerely, 

Signed by: 

Jim H. Crumpacker: 
Director: 
Departmental GAO-OIG Liaison Office: 

[End of section] 

Appendix V: Comments from the Department of the Treasury: 

Department of The Treasury: 
Washington, D.C. 20220: 

September 14, 2012: 

David A. Powner: 
Director, Information Technology Management Issues: 
U.S. Government Accountability Office: 
441 G Street NW: 
Washington, DC 20548: 

Dear Mr. Powner: 

Thank you for the opportunity to provide comments on GAO's Draft 
Report, Information Technology Agencies Need to Strengthen Oversight 
of Billions of Dollars in Operations and Maintenance Investments (GAO-
12-963). 

Treasury appreciates GAO's efforts in developing this report. We agree 
with GAO's recommendations for Treasury and anticipate we will publish 
our revised Operational Analyses (OA) policy by the end of calendar 
year 2012. A draft of this policy was provided to GAO on August 23, 
2012. Additionally, on September 13, 2012, we provided a report to GAO 
summarizing OAs for 9 of the 16 investments identified in GAO's report 
as not having an OA, and we are developing OAs for the remaining 7. 

Please contact me at 202-622-1200 if you need anything further. 

Sincerely, 

Signed by: 

[Illegible] for: 
Robyn East: 
Deputy Assistant Secretary for Information Systems and Chief 
Information Officer: 

[End of section] 

Appendix VI: Comments from the Department of Veterans Affairs: 

Department of Veterans Affairs: 
Washington DC 20420: 

September 13, 2012: 

Mr. David A. Powner: 
Director, Information Technology Management Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Powner: 

The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, "Information Technology:
Agencies Need to Strengthen Oversight of Billions of Dollars in 
Operations and Maintenance Investments" (GAO-12-963). VA generally 
agrees with GAO's conclusions and concurs with GAO's recommendation to 
the Department. 

The enclosure specifically addresses GAO's recommendation to the draft 
report. 

VA appreciates the opportunity to comment on your draft report. 

Sincerely, 

Signed by: 

John R. Gingrich: 
Chief of Staff: 

Enclosure: 

Department of Veterans Affairs (VA) Comments to Government 
Accountability Office (GAO) Draft Report: "Information Technology: 
Agencies Need to Strengthen Oversight of Billions of Dollars in 
Operations and Maintenance Investments" (GA0-12-963): 

GAO Recommendation: To ensure that major steady state IT investments are
being adequately analyzed, we recommend that the Secretary of Veterans 
Affairs direct appropriate officials to develop an OA policy, annually 
perform OA's on all investments, and ensure the assessments include 
all key factors. 

VA Response: Concur. VA understands and acknowledges the need for 
appropriate officials to develop an operational analyses (OA) policy, 
annually perform OA's on all investments, and ensure the assessments 
include all key factors required by Office of Management and Budget. 
VA's Chief Information Officer anticipates that the policy will be 
published by the second quarter of fiscal year (FY) 2013. 

VA will develop a process to comply with all 17 required elements of 
"operational assessment" performed annually which may include standing 
up an office using Resource and Performance Management, Customer 
Satisfaction, and Information Technology Service Management Program 
Analysis to synthesize the information. VA's Chief Information Officer 
anticipates that the process will be implemented by the third quarter 
of FY 2013. 

[End of section] 

Appendix VII: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

David A. Powner, (202) 512-9286 or pownerd@gao.gov: 

Staff Acknowledgments: 

In addition to the contact name above, individuals making contributions 
to this report included Gary Mountjoy (Assistant Director), Gerard 
Aflague, Rebecca Eyler, Lori Martinez, and Teresa Smith. 

[End of section] 

Footnotes: 

[1] The 26 federal departments and agencies that report to OMB on their 
IT investments are the Departments of Agriculture, Commerce, Defense, 
Education, Energy, Health and Human Services, Homeland Security, 
Housing and Urban Development, Interior, Justice, Labor, State, 
Transportation, the Treasury, and Veterans Affairs; Environmental 
Protection Agency, General Services Administration, National 
Aeronautics and Space Administration, National Archives and Records 
Administration, National Science Foundation, Nuclear Regulatory 
Commission, Office of Personnel Management, Small Business 
Administration, Smithsonian Institution, Social Security 
Administration, and U.S. Agency for International Development. 

[2] 40 U.S.C. § 11302(c). 

[3] OMB Circular No. A-11, Preparation, Submission, and Execution of 
the Budget (August 2012). 

[4] According to OMB guidance, a major investment is a system or 
acquisition requiring special management attention because of its 
importance to the mission or function of the agency, a component of the 
agency, or another organization; is for financial management and 
obligates more than $500,000 annually; has significant program or 
policy implications; has high executive visibility; has high 
development, operating, or maintenance costs; is funded through other 
than direct appropriations; or is defined as major by the agency's 
capital planning and investment control process. 

[5] GAO, IT Dashboard: Accuracy Has Improved and Additional Efforts Are 
Under Way to Better Inform Decision Making, [hyperlink, 
http://www.gao.gov/products/GAO-12-210] (Washington, D.C.: Nov. 7, 
2011); Information Technology: OMB Has Made Improvements to Its 
Dashboard, but Further Work Is Needed by Agencies and OMB to 
Ensure Data Accuracy, [hyperlink, 
http://www.gao.gov/products/GAO-11-262] (Washington, D.C.: Mar. 15, 
2011); and 
Information Technology: OMB's Dashboard Has Increased Transparency and 
Oversight, but Improvements Needed, [hyperlink, 
http://www.gao.gov/products/GAO-10-701] (Washington, D.C.: July 
16, 2010). 

[6] Capital Programming Guide, Supplement to OMB Circular A-11, Part 7 
(July 2012); OMB Memorandum M-10-27 (June 2010), requires agencies to 
establish a policy for performing OAs on steady state investments as a 
part of managing and monitoring investment baselines. 

[7] DHS, Operational Analysis Guidance, Version 1.1, May 2008. 

[8] HHS, Enterprise Performance Life Cycle Framework, Practices Guide, 
Annual Operational Analysis, September 2008. 

[9] HHS, Centers for Disease Control and Prevention: Operational 
Analysis Guide, Version 9.0, January 2011 and HHS, National Institutes 
of Health: NIG Operational Analysis Management Framework: A How-to 
Guide, Version 1.2, August 2010. 

[10] GAO, Information Technology: Agencies Need to Improve the Accuracy 
and Reliability of Investment Information, [hyperlink, 
http://www.gao.gov/products/GAO-06-250] (Washington, D.C: Jan.12, 
2006). 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548. 

[End of document]