This is the accessible text file for GAO report number GAO-11-873 
entitled 'Quadrennial Homeland Security Review: Enhanced Stakeholder 
Consultation and Use of Risk Information Could Strengthen Future 
Reviews' which was released on October 5, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

September 2011: 

Quadrennial Homeland Security Review: 

Enhanced Stakeholder Consultation and Use of Risk Information Could 
Strengthen Future Reviews: 

GAO-11-873: 

GAO Highlights: 

Highlights of GAO-11-873, a report to congressional requesters. 

Why GAO Did This Study: 

The United States continues to face a range of evolving threats, such 
as the 2010 attempted attack on the nation’s air cargo system, that 
underscore why homeland security planning efforts are crucial to the 
security of the nation. The Implementing Recommendations of the 9/11 
Commission Act of 2007 required the Department of Homeland Security 
(DHS) to provide a comprehensive examination of the U.S. homeland 
security strategy every 4 years. In response, DHS issued its first 
Quadrennial Homeland Security Review (QHSR) report in February 2010 
and a Bottom-Up Review (BUR) report in July 2010, to identify 
initiatives to implement the QHSR. As requested, this report addresses 
the extent to which DHS (1) consulted with stakeholders in developing 
the QHSR, (2) conducted a national risk assessment, and (3) developed 
priorities, plans, monitoring mechanisms, and performance measures for 
implementing the QHSR and BUR initiatives. GAO analyzed relevant 
statutes and DHS documents on the QHSR and BUR processes and, in 
response to a request for comments on the processes, received comments 
from 63 of the 85 federal and nonfederal stakeholders it contacted. 
Their responses are not generalizable, but provided perspectives on 
the processes. 

What GAO Found: 

DHS solicited input from various stakeholder groups in conducting the 
first QHSR, but DHS officials, stakeholders GAO contacted, and other 
reviewers of the QHSR noted concerns with time frames provided for 
stakeholder consultations and outreach to nonfederal stakeholders. DHS 
consulted with stakeholders-—federal agencies; department and 
component officials; state, local, and tribal governments; the private 
sector; academics; and policy experts—-through various mechanisms, 
such as the solicitation of papers to help frame the QHSR and a web-
based discussion forum. DHS and these stakeholders identified benefits 
from these consultations, such as DHS receiving varied perspectives. 
However, stakeholders also identified challenges in the consultation 
process. Sixteen of 63 stakeholders who provided comments to GAO noted 
concerns about the time frames for providing input into the QHSR or 
BUR. Nine DHS stakeholders, for example, responded that the limited 
time available for development of the QHSR did not allow DHS to have 
as deep an engagement with stakeholders. Further, 9 other stakeholders 
commented that DHS consultations with nonfederal stakeholders, such as 
state, local, and private sector entities, could be enhanced by 
including more of these stakeholders in QHSR consultations. In 
addition, reports on the QHSR by the National Academy of Public 
Administration, which administered DHS’s web-based discussion forum, 
and a DHS advisory committee comprised of nonfederal representatives 
noted that DHS could provide more time and strengthen nonfederal 
outreach during stakeholder consultations. By providing more time for 
obtaining feedback and examining mechanisms to obtain nonfederal 
stakeholders’ input, DHS could strengthen its management of 
stakeholder consultations and be better positioned to review and 
incorporate, as appropriate, stakeholders’ input during future reviews. 

DHS identified threats confronting homeland security in the 2010 QHSR 
report, such as high-consequence weapons of mass destruction and 
illicit trafficking, but did not conduct a national risk assessment 
for the QHSR. DHS officials stated that at the time DHS conducted the 
QHSR, DHS did not have a well-developed methodology or the analytical 
resources to complete a national risk assessment that would include 
likelihood and consequence assessments-—key elements of a national 
risk assessment. To develop an approach to national risk assessments, 
DHS created a study group as part of the QHSR process that developed a 
national risk assessment methodology. DHS officials plan to implement 
a national risk assessment in advance of the next QHSR, which DHS 
anticipates conducting in fiscal year 2013. 

DHS developed priorities, plans, monitoring mechanisms, and 
performance measures, but did not consider risk information in making 
its prioritization efforts. DHS considered various factors in 
identifying high-priority BUR initiatives for implementation in fiscal 
year 2012 but did not include risk information as one of these 
factors, as called for in GAO’s prior work and DHS’s risk management 
guidance, because of differences among the initiatives that made it 
difficult to compare risks across them, among other things. 
Consideration of risk information during future implementation efforts 
could help strengthen DHS’s prioritization of mechanisms for 
implementing the QHSR, including assisting in determinations of which 
initiatives should be implemented in the short or longer term. 

What GAO Recommends: 

GAO recommends that for future reviews, DHS provide the time needed 
for stakeholder consultations, explore options for consulting with 
nonfederal stakeholders, and examine how risk information could be 
considered in prioritizing QHSR initiatives. DHS concurred with our 
recommendations. 

To view the full product, including the scope and methodology, click 
on [hyperlink, http://www.gao.gov/products/GAO-11-873]. For more 
information, contact David C. Maurer at (202) 512-9627 or 
maurerd@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DHS Could Strengthen Stakeholder Consultations by Expanding Time 
Frames, Nonfederal Participation, and Role Definitions: 

DHS Plans to Implement a National Risk Assessment as Part of the Next 
QHSR: 

DHS Developed BUR Monitoring Mechanisms and Measures but Could 
Strengthen Its Prioritization Efforts by Using Risk Information: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendix I: DHS Strategic Documents and the National Security Strategy 
Align with the QHSR: 

Appendix II: Scope and Methodology: 

Appendix III: Comments from the Department of Homeland Security: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Mechanisms Used by DHS for Obtaining Input on the QHSR from 
Various Stakeholder Groups: 

Table 2: Characteristics of the HSNRA: 

Table 3: DHS Ranking of 14 High-Priority BUR Initiatives to Be 
Implemented in Fiscal Year 2012: 

Table 4: DHS Risk Assessment Tools That DHS Reports Could Be Leveraged 
to Prioritize QHSR Implementation Mechanisms: 

Table 5: Alignment between QHSR Mission Areas and DHS Strategic 
Documents and the NSS: 

Figures: 

Figure 1: 2010 QHSR Missions, Goals, and Objectives: 

Figure 2: Example of BUR Implementation Linkage to QHSR Missions and 
DHS Programs and Activities: 

Figure 3: Example of HSNRA Output for Use by DHS Decision Makers: 

Figure 4: DHS Planning, Programming, Budgeting, and Execution Phases 
of the Budget Process: 

Abbreviations: 

BUR: Bottom-Up Review: 

CBP: U.S. Customs and Border Protection: 

CG: U.S. Coast Guard: 

DHS: Department of Homeland Security: 

DOD: Department of Defense: 

FACA: Federal Advisory Committee Act: 

FEMA: Federal Emergency Management Agency: 

FYHSP: Future Years Homeland Security Program: 

HSNRA: Homeland Security National Risk Assessment: 

ICE: U.S. Immigration and Customs Enforcement: 

IRMF: Integrated Risk Management Framework: 

NAPA: National Academy of Public Administration: 

NSS: National Security Strategy: 

PPBE: Planning, Programming, Budgeting and Execution: 

QHSR: Quadrennial Homeland Security Review: 

QRAC: Quadrennial Review Advisory Committee: 

RAP: Resource Allocation Plan: 

RAPID: Risk Analysis Process for Informed Decision-making: 

RMA: Risk Management and Analysis: 

Sub-IPC: Sub-Interagency Policy Committee: 

TSA: Transportation Security Administration: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

September 15, 2011: 

The Honorable Joseph I. Lieberman: 
Chairman: 
The Honorable Susan M. Collins: 
Ranking Member: 
Committee on Homeland Security and Governmental Affairs 
United States Senate: 

The Honorable Daniel K. Akaka: 
Chairman: 
Subcommittee on Oversight of Government Management, the Federal 
Workforce, and the District of Columbia: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate: 

The United States continues to face a myriad of broad and evolving 
threats, such as the October 2010 attempted attack on the nation's air 
cargo system, that underscore why the federal government places a high 
priority on homeland security and efforts to coordinate security 
roles, responsibilities, and activities across a wide variety of 
stakeholders, including state, local, and tribal government; private 
sector; nongovernmental; and international partners. The Implementing 
Recommendations of the 9/11 Commission Act of 2007 (9/11 Commission 
Act) requires that beginning in fiscal year 2009 and every 4 years 
thereafter the Department of Homeland Security (DHS) conduct a review 
that provides a comprehensive examination of the homeland security 
strategy of the United States.[Footnote 1] According to the act, the 
review is to delineate the national homeland security strategy, 
outline and prioritize critical homeland security missions, and assess 
the organizational alignment of DHS with the homeland security 
strategy and missions, among other things. The act requires that DHS 
conduct the quadrennial review in consultation with stakeholders, such 
as heads of federal agencies; state, local, and tribal governments; 
private sector representatives; and academics and other policy experts. 

In February 2010, DHS issued its first Quadrennial Homeland Security 
Review (QHSR) report, outlining a strategic framework for homeland 
security to guide the activities of homeland security partners, 
including federal, state, local, and tribal government agencies; the 
private sector; and nongovernmental organizations.[Footnote 2] The 
report identified five homeland security missions--(1) Preventing 
Terrorism and Enhancing Security, (2) Securing and Managing Our 
Borders, (3) Enforcing and Administering Our Immigration Laws, (4) 
Safeguarding and Securing Cyberspace, and (5) Ensuring Resilience to 
Disasters--and goals and objectives to be achieved within each 
mission. The QHSR report also identified threats and challenges 
confronting U.S. homeland security, strategic objectives for 
strengthening the homeland security enterprise, and federal agencies' 
roles and responsibilities for homeland security.[Footnote 3] In 
addition to the QHSR report, in July 2010 DHS issued a report on the 
results of its Bottom-Up Review (BUR), a departmentwide assessment to 
implement the QHSR strategy by aligning DHS's programmatic activities, 
such as investigating drug smuggling and inspecting cargo at ports of 
entry, and its organizational structure with the missions and goals 
identified in the QHSR.[Footnote 4] The BUR report described DHS's 
current activities contributing to (1) QHSR mission performance, (2) 
departmental management, and (3) accountability. The BUR report also 
identified priority initiatives, such as strengthening aviation 
security and enhancing the department's risk management capability, to 
strengthen DHS's mission performance, improve departmental management, 
and increase accountability. In December 2010, we issued a report on 
the extent to which the QHSR addressed the 9/11 Commission Act's 
required reporting elements.[Footnote 5] We reported that of the nine 
9/11 Commission Act reporting elements for the QHSR, DHS addressed 
three and partially addressed six.[Footnote 6] Elements DHS addressed 
included a description of homeland security threats and an explanation 
of underlying assumptions for the QHSR report. Elements addressed in 
part included a prioritized list of homeland security missions, an 
assessment of the alignment of DHS with the QHSR missions, and 
discussions of cooperation between the federal government and state, 
local, and tribal governments. 

You asked us to review DHS's QHSR, including DHS's process for 
conducting the review and for implementing the QHSR strategy. This 
report addresses the following question: To what extent did DHS (1) 
consult with stakeholders in developing the QHSR strategy; (2) conduct 
a national risk assessment to develop the QHSR; and (3) develop 
priorities, plans, monitoring mechanisms, and performance measures for 
implementing the QHSR and BUR initiatives? This report also provides 
information on the extent to which DHS's strategic documents and the 
National Security Strategy align with the QHSR (see appendix I). 

To address these objectives, we analyzed DHS documents related to the 
QHSR, BUR, and budget development processes, including the QHSR 
report, BUR report, fiscal year 2012 budget request, and Fiscal Years 
2012-2016 Future Years Homeland Security Program (FYHSP).[Footnote 7] 
We identified criteria for evaluating these processes by analyzing our 
prior reports on key characteristics of effective national strategies, 
key practices for effective interagency collaboration, strategic 
planning, performance measurement, and standards for internal control, 
among others.[Footnote 8] Based on these reports, we identified those 
key practices and characteristics applicable to quadrennial reviews, 
like the QHSR. The key practices we identified were involving 
stakeholders in defining QHSR missions and outcomes; defining homeland 
security problems and assessing risks; including homeland security 
strategy goals, subordinate objectives, activities and performance 
measures; including resources, investments, and risk management; 
including organizational roles, responsibilities, and coordination 
across the homeland security enterprise; and establishing DHS 
processes for managing implementation of BUR initiatives. We vetted 
the key practices with our subject matter experts--staff with legal 
and methodological expertise and experience analyzing the Quadrennial 
Defense Review--and provided them to DHS officials for review, and 
incorporated their comments as appropriate.[Footnote 9] As we 
developed our report, we grouped these key practices into three areas--
stakeholder involvement, risk assessment, and implementation processes 
for the QHSR and BUR initiatives. 

To determine the extent to which DHS consulted with stakeholders in 
developing the QHSR, we requested comments on the QHSR process from 79 
QHSR stakeholder organizations identified by DHS.[Footnote 10] The 
stakeholders solicited by us for comments included 22 federal 
departments and agencies; 10 state, local, and tribal organizations; 
28 DHS components, directorates and offices; and 19 Quadrennial Review 
Advisory Committee (QRAC) members.[Footnote 11] We also solicited 
comments from 6 subject matter experts hired by DHS to facilitate QHSR 
study groups.[Footnote 12] We received comments from 63 of the 85 
stakeholders and study group facilitators we contacted (74 percent), 
including 21 of 22 federal departments; 6 of 10 state, local, and 
tribal organizations; 26 of the 28 DHS components, directorates, and 
offices; 7 of the 19 QRAC members; and 3 of the 6 study group 
facilitators. We asked open-ended questions regarding the QHSR 
stakeholder consultation process, such as suggestions for improving 
future QHSRs, examples of positive ways DHS involved stakeholders, and 
involvement in determining agency roles and responsibilities listed in 
the QHSR report. We relied on respondents to raise and comment on 
their views of the QHSR process; therefore we could not determine 
whether respondents shared similar views or identified similar 
benefits or challenges to the QHSR process unless respondents 
identified them in their responses to our requests for comments. 
[Footnote 13] We analyzed the comments provided by the 63 respondents 
to determine common benefits and challenges they identified regarding 
DHS consultations during the QHSR. We also conducted follow-up 
interviews with 14 QHSR stakeholders that we selected based on their 
responses, to obtain clarification of their responses to our requests 
for comments. The comments received from these respondents are not 
generalizable to the entire group of stakeholders, but the feedback 
provided insights into stakeholder perspectives on how QHSR 
stakeholder consultations were conducted and how they could be 
improved. Further, we reviewed reports on the QHSR by the National 
Academy of Public Administration (NAPA) and the QRAC, both of which 
were based upon each organization's collaboration experiences with DHS 
in developing the QHSR report.[Footnote 14] We compared DHS's 
stakeholder consultation efforts to our prior work on effective 
practices for collaboration and consultation. 

To determine the extent to which DHS conducted a national risk 
assessment to develop the QHSR, we analyzed risk analysis-related 
documents produced as part of the QHSR process, such as DHS risk 
assessment tools, and interviewed DHS officials responsible for 
developing risk analyses for use at DHS. We compared DHS's risk 
assessment process in the QHSR to our prior work on key 
characteristics for risk assessment as well as DHS risk analysis 
guidance documents. 

To determine the extent to which DHS developed priorities, 
implementation plans, monitoring mechanisms, and performance measures, 
we analyzed DHS's BUR implementation priorities and plans, such as 
DHS's fiscal year 2012 budget request; DHS monitoring mechanisms, such 
as BUR initiative scorecards; our Standards for Internal Control in 
the Federal Government; and DHS's strategic and management performance 
measures. We also interviewed DHS officials responsible for managing 
and monitoring implementation of the BUR initiatives. We compared 
DHS's processes for prioritizing, monitoring, and measuring 
implementation efforts to our prior work on key practices for risk 
management and implementation and monitoring of strategic initiatives. 
We also compared DHS's performance measures for fiscal year 2011 to 
our criteria on key attributes of successful performance measures. 
[Footnote 15] A more detailed discussion of our scope and methodology 
is contained in appendix II. 

We conducted this performance audit from January 2011 through 
September 2011 in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

Background: 

DHS approached the 9/11 Commission Act requirement for a quadrennial 
homeland security review in three phases. In the first phase, DHS 
defined the nation's homeland security interests, identified the 
critical homeland security missions, and developed a strategic 
approach to those missions by laying out the principal goals, 
objectives, and strategic outcomes for the mission areas. DHS reported 
on the results of this effort in the February 2010 QHSR report in 
which the department identified five homeland security missions, 14 
associated goals and 43 objectives, as shown in figure 1. The QHSR 
report also identified a strategy for maturing and strengthening the 
homeland security enterprise, with 18 associated objectives. In the 
second phase--the BUR--DHS identified its component agencies' 
activities, aligned those activities with the QHSR missions and goals, 
and made recommendations for improving the department's organizational 
alignment and business processes. DHS reported on the results of this 
second phase in the July 2010 BUR report. In the third phase DHS 
developed its budget plan necessary to execute the QHSR missions. DHS 
presented this budget plan in the President's fiscal year 2012 budget 
request, issued February 14, 2011, and the accompanying Fiscal Year 
2012-2016 FYHSP, issued in May 2011. DHS officials stated that 
together, these three phases and their resulting reports and documents 
address the 9/11 Commission Act requirement for the quadrennial 
homeland security review. 

Figure 1: 2010 QHSR Missions, Goals, and Objectives: 

[Refer to PDF for image: illustration] 

Interactive Graphic: Roll mouse over section header for additional 
information. 

Preventing Terrorism and Enhancing Security Mission Goals and 
Objectives: 

Goal 1.1: Prevent Terrorist Attacks: Malicious actors are unable to
conduct terrorist attacks within the United States; 
Objectives: 
* Understand the threat: Acquire, analyze, and appropriately share
intelligence and other information on current and emerging threats. 
* Deter and disrupt operations: Deter, detect, and disrupt surveillance,
rehearsals, and execution of operations by terrorists and other 
malicious actors. 
* Protect against terrorist capabilities: Protect potential targets 
against the capabilities of terrorists, malicious actors, and their 
support networks to plan and conduct operations. 
* Stop the spread of violent extremism: Prevent and deter violent
extremism and radicalization that contributes to it. 
* Engage communities: Increase community participation in efforts to
deter terrorists and other malicious actors and mitigate radicalization
toward violence. 

Goal 1.2: Prevent the unauthorized acquisition or use of Chemical,
Biological, Radiological, and Nuclear (CBRN) Materials and Capabilities:
Malicious actors, including terrorists, are unable to acquire or move 
dangerous chemical, biological, radiological, and nuclear materials or 
capabilities within the United States.
Objectives: 
* Anticipate emerging threats: Identify and understand potentially 
dangerous actors, technologies, and materials.
* Control access to CBRN: Prevent terrorists and other malicious 
actors from gaining access to dangerous materials and technologies.
* Control movement of CBRN: Prevent the illicit movement of dangerous 
materials and technologies.
* Protect against hostile use of CBRN: Identify the presence of and
effectively locate, disable, or prevent the hostile use of CBRN.

Goal 1.3: Manage risks to critical infrastructure, key leadership, and 
events: Key sectors actively work to reduce vulnerability to attack
or disruption.
Objectives: 
* Understand and prioritize risks to critical infrastructure: 
Identify, attribute, and evaluate the most dangerous threats to 
critical infrastructure and the categories of critical infrastructure 
most at risk.
* Protect critical infrastructure: Prevent high-consequence events by 
securing critical infrastructure assets, systems, networks, or 
functions—-including linkages through cyberspace—-from attacks or 
disruption.
* Make critical infrastructure resilient: Enhance the ability of 
critical infrastructure systems, networks, and functions to withstand 
and rapidly recover from damage and disruption and adapt to changing 
conditions.
* Protect government leaders, facilities, and special events: Preserve 
continuity of government and ensure security at events of national 
significance.

Securing and Managing Borders Mission Goals and Objectives: 

Goal 2.1: Effectively control U.S air, land, and sea borders: Prevent
the illegal flow of people and goods across U.S. air, land, and sea 
borders while expediting the safe flow of lawful travel and commerce.
Objectives: 
* Prevent illegal entry: Prevent the illegal entry of people, weapons, 
dangerous goods, and contraband, and protect against cross-border 
threats to health, food, environment, and agriculture, while 
facilitating the safe flow of lawful travel and commerce.
* Prevent illegal export and exit: Prevent the illegal export of 
weapons, proceeds of crime, and other dangerous goods, and the exit of 
malicious actors. 

Goal 2.2: Safeguard lawful trade and travel: Ensure security and 
resilience of global movement systems.
Objectives: 
* Secure key nodes: Promote the security and resilience of key nodes 
of transaction and exchange within the global supply chain.
* Secure conveyances: Promote the safety, security, and resilience of 
conveyances in the key global trading and transportation networks.
* Manage the risk posed by people and goods in transit.

Goal 2.3: Disrupt and Dismantle Transnational Criminal Organizations: 
Disrupt and dismantle transnational organizations that engage in 
smuggling and trafficking across the U.S. border. 
Objectives: 
* Identify, disrupt, and dismantle transnational criminal and 
terrorist organizations: Disrupt transnational criminal or terrorist 
organizations involved in cross-border smuggling, trafficking, or 
other cross-border crimes; dismantle their infrastructure; and 
apprehend their leaders.
* Disrupt illicit pathways: Identify, disrupt, and dismantle illicit 
pathways used by transnational criminal and terrorist organizations.
Enforcing and Administering Our Immigration Laws Mission Goals & 
Objectives

Enforcing and Administering Our Immigration Laws Mission Goals and 
Objectives: 

Goal 3.1: Strengthen and effectively administer the immigration system:
Promote lawful immigration, expedite administration of immigration 
services, and promote the integration of lawful immigrants into 
American society.
Objectives: 
* Promote lawful immigration: Clearly communicate to the public 
information on immigration services and procedures.
* Effectively administer the immigration services system: Create a user-
friendly system that ensures fair, consistent, and prompt decisions.
* Promote the integration of lawful immigrants into American society:
Provide leadership, support, and opportunities to immigrants to 
facilitate their integration into American society and foster 
community cohesion. 

Goal 3.2: Prevent unlawful immigration: Reduce conditions that 
encourage foreign nationals to illegally enter and remain in the United
States, while identifying and removing those who violate our laws.
Objectives: 
* Reduce demand: Eliminate the conditions that encourage illegal 
employment.
* Eliminate systemic vulnerabilities: Prevent fraud, abuse, and 
exploitation, and eliminate other systemic vulnerabilities that 
threaten the integrity of the immigration system.
* Prevent entry or admission: Prevent entry or admission of criminals, 
fugitives, other dangerous foreign nationals, and other unauthorized 
entrants.
* Arrest, detain, prosecute, and remove: Arrest, detain, prosecute, 
and remove criminal, fugitive, dangerous, and other unauthorized 
foreign nationals consistent with due process and civil rights 
protections.

Safeguarding and Securing Cyberspace Mission Goals and Objectives: 

Goal 4.1: Create a safe, secure, and resilient cyber environment:
Ensure that malicious actors are unable to effectively exploit 
cyberspace, impair its safe and secure use, or attack the nation's 
information infrastructure.
Objectives: 
* Understand and prioritize cyber threats: Identify and evaluate the 
most dangerous threats to federal civilian and private sector networks 
and the nation.
* Manage risks to cyberspace: Protect and make resilient information 
systems, networks, and personal and sensitive data.
* Prevent cyber crime and other malicious uses of cyberspace: Disrupt 
the criminal organizations and other malicious actors engaged in high-
consequence or wide-scale cyber crime.
* Develop a robust public-private cyber incident response capability:
Manage cyber incidents from identification to resolution in a rapid 
and replicable manner with prompt and appropriate action.

Goal 4.2: Promote cybersecurity knowledge and innovation. Ensure that 
the nation is prepared for the cyber threats and challenges of tomorrow.
Objectives: 
* Enhance public awareness: Ensure that the public recognizes 
cybersecurity challenges and is empowered to address them.
* Foster a dynamic workforce: Develop the national knowledge base and 
human capital capabilities to enable success against current and 
future threats.
* Invest in innovative technologies, techniques, and procedures: 
Create and enhance science, technology, governance mechanisms, and 
other elements necessary to sustain a safe, secure, and resilient 
cyber environment.

Ensuring Resilience to Disasters Mission Goals and Objectives: 
 
Goal 5.1: Mitigate hazards: Strengthen capacity at all levels of 
society to withstand threats and hazards.
Objectives: 
* Reduce the vulnerability of individuals and families: Improve 
individual
and family capacity to reduce vulnerabilities and withstand disasters.
* Mitigate risks to communities: Improve community capacity to 
withstand disasters by mitigating known and anticipated hazards.
 
Goal 5.2: Enhance preparedness: Engage all levels and segments of 
society in improving preparedness.
Objectives: 
* Improve individual, family, and community preparedness: Ensure 
individual, family, and community planning, readiness, and capacity 
building for disasters.
* Strengthen capabilities: Enhance and sustain nationwide disaster 
preparedness capabilities, to include life safety, law enforcement, 
information sharing, mass evacuation and shelter-in-place, public 
health, mass care, and public works.
 
Goal 5.3: Ensure effective emergency response: Strengthen response 
capacity nationwide.
Objectives: 
* Provide timely and accurate information to the public: Establish and 
strengthen pathways for clear, reliable, and current emergency 
information, including effective use of new media.
* Conduct effective disaster response operations: Respond to disasters 
in an effective and unified manner.
* Provide timely and appropriate disaster assistance: Improve 
governmental, nongovernmental, and private sector delivery of disaster 
assistance.

Goal 5.4: Rapidly recover: Improve the nation's ability to adapt and 
rapidly recover.
Objectives: 
* Enhance recovery capabilities: Establish and maintain nationwide 
capabilities for recovery from major disasters.
* Ensure continuity of essential services and functions: Improve 
capabilities of families, communities, private sector organizations, 
and all levels of government to sustain essential services and 
functions.

Source: GAO analysis of DHS data. 

[End of figure] 

DHS initiated the QHSR in August 2007. Led by the DHS Office of 
Policy, in July 2009 the department issued its QHSR terms of 
reference, outlining the framework for conducting the quadrennial 
review and identifying threats and assumptions to be used in 
conducting the review.[Footnote 16] Through the terms of reference, 
DHS identified the initial four homeland security missions to be 
studied, which were refined during the QHSR process--Counterterrorism 
and Domestic Security Management; Securing Our Borders; Smart and 
Tough Enforcement of Immigration Laws; and Preparing For, Responding 
To, and Recovering from Disasters--as well as three other separate, 
nonmission study areas to be part of the review--DHS Strategic 
Management, Homeland Security National Risk Assessments, and Homeland 
Security Planning and Capabilities. The fifth QHSR mission on 
Safeguarding and Securing Cyberspace was added after DHS issued the 
terms of reference. A sixth category of DHS activities--Providing 
Essential Support to National and Economic Security--was added in the 
fiscal year 2012 budget request but was not included in the 2010 QHSR 
report. 

DHS established seven study groups for the QHSR, which were composed 
of officials from across DHS offices and components. The study groups 
were each led by a DHS official and facilitated by an independent 
subject matter expert from the Homeland Security Studies and Analysis 
Institute.[Footnote 17] These study groups conducted their analysis 
over a 5-month period and shared their work products, such as outlines 
of missions and assumptions, with other stakeholder groups in order to 
develop goals and objectives for each mission. At the end of the study 
group period, DHS senior leadership, including the Deputy Secretary of 
Homeland Security, the General Counsel, and office and component 
heads, met multiple times to review and discuss the study group 
recommendations. The DHS Office of Policy consolidated the study 
groups' recommendations into a draft QHSR report and obtained and 
incorporated feedback on the draft report from other federal agencies 
and stakeholder groups, including the stakeholders listed in the 9/11 
Commission Act, with which DHS was to consult in conducting the 
QHSR.[Footnote 18] Agreement on the QHSR report's final content was 
reached between the Secretary for Homeland Security and senior White 
House officials. DHS issued the final QHSR report in February 2010. 

DHS initiated the BUR in November 2009. Each DHS directorate, 
component, and office created an inventory of its activities and 
categorized them according to the QHSR missions. For example, U.S. 
Immigration and Customs Enforcement (ICE) identified one of its 
activities as investigating human smuggling and trafficking, which it 
categorized under the Securing and Managing our Borders QHSR mission. 
The BUR resulted in a catalog of about 1,300 DHS activities organized 
under each of the five QHSR missions or categorized as mission or 
business support activities.[Footnote 19] DHS identified over 300 
potential initiatives for increasing mission performance and 
accountability and improving department management, derived 43 
priority initiatives from this list, and highlighted them in the July 
2010 BUR report.[Footnote 20] For example, under the Enforcing and 
Administering our Immigration Laws mission, DHS identified as priority 
initiatives improving DHS's immigration services processes and 
dismantling human-smuggling organizations (see figure 2). 

Figure 2: DHS Planning, Programming, Budgeting, and Execution Phases 
of the Budget Process: 

[Refer to PDF for image: table] 

Time period: 2007--February 2010; 
Strategic exercise: OHSR mission 2: Securing and managing our borders; 
Output of exercise: Mission 2 goals: 
* Effectively control U.S. air, land, and sea borders; 
* Safeguard lawful trade and travel; 
* Disrupt and dismantle transnational criminal organizations. 

Time period: February 2010--April 2010; 
Strategic exercise: BUR mission 2: Inventory BUR inventory of 653 
existing Mission 2-related programs and activities; 
Output of exercise: Program/activity examples connected to Mission 2 
goals: 
* Cargo Inspection and Security: Conduct agricultural inspections 
(CBP); 
* Immediate border security between the ports of entry (CBP); 
* Passenger processing: conduct agricultural inspections (CBP); 
* Prevent illegal export (CBP); 
* Conduct bulk cash smuggling/illegal money services investigations 
(ICE); 
* Conduct trade fraud investigations (ICE); 
* Assess, safeguard, and defend ports (CG); 
* Combat maritime terrorism (CG); 
* International air cargo inspectors (TSA). 

Time period: April 2010--July 2010; 
Strategic exercise: BUR initiatives: Five initiatives developed to 
enhance DHS programs/activities to implement QHSR Mission 2; 
Output of exercise: Mission 2 BUR initiatives: 
* Expand joint operations and intelligence capabilities, including 
enhanced domain awareness; 
* Prioritize immigration and customs investigations; 
* Enhance the security and resilience of global trade and travel 
systems; 
* Strengthen and expand DHS-related security assistance 
internationally; 
* Enhance North American Security. 

Sources: GAO analysis of DHS documents. 

Note: The DHS budget process consists of overlapping planning, 
programming, budgeting, and execution phases; therefore, some of DHS's 
key activities are carried out concurrently. 

[End of figure] 

In addition, DHS categorized its 43 BUR initiatives according to 
whether they require organizational, programmatic, policy, or 
legislative activities in order to be implemented. DHS defines these 
categories as (1) organizational, where implementation requires some 
type of departmental reorganization (e.g., create a cybersecurity and 
infrastructure resilience operational component within DHS); (2) 
programmatic, where implementation requires budgetary activity, such 
as a funding increase (e.g., increase efforts to detect and counter 
nuclear and biological weapons and dangerous materials); (3) policy, 
where implementation requires a policy decision but no additional 
funding (e.g., enhance the department's risk management capability); 
and (4) legislative, where implementation requires a change in 
legislation or congressional approval because DHS does not have the 
legislative authority to implement the initiative (e.g., restore the 
Secretary's reorganization authority for DHS headquarters). According 
to DHS officials, some BUR initiatives require one or more of these 
types of changes to be implemented, such as the initiative to 
strengthen internal DHS counterintelligence capabilities, which 
requires policy and programmatic changes. 

DHS's fiscal year 2012 budget request highlighted funding requests to 
support projects and programs within each QHSR mission. For example, 
for QHSR mission 1, Preventing Terrorism and Enhancing Security, DHS's 
fiscal year 2012 budget request includes requests for 18 projects and 
programs to support that mission. These requests include items such as 
$273 million for explosive detection systems at airports and $12.4 
million for enhanced watchlist vetting of airline passengers. 
According to DHS officials, DHS intends to include longer-term project 
and program funding plans for QHSR missions through annual iterations 
of its FYHSP. For example, the Fiscal Year 2012-2016 FYHSP contains 
initiatives and planned performance information aligned with the 
missions of the department. 

According to the 9/11 Commission Act, DHS is to report on the results 
of its QHSR every 4 years with the next report due by December 31, 
2013. DHS plans to issue its next QHSR report in accordance with the 
act. 

DHS Could Strengthen Stakeholder Consultations by Expanding Time 
Frames, Nonfederal Participation, and Role Definitions: 

DHS Used Various Mechanisms to Solicit Stakeholder Views during the 
QHSR Process: 

DHS solicited input from various stakeholder groups in conducting the 
first QHSR. The 9/11 Commission Act required DHS to consult with seven 
federal agencies in conducting the QHSR--the Departments of 
Agriculture, the Treasury, Justice, State, Defense, and Health and 
Human Services and the Office of the Director of National 
Intelligence.[Footnote 21] DHS consulted with these agencies and also 
sought input from a range of other stakeholders, including its 
directorates, offices, and components; other federal agencies; and 
nonfederal governmental and nongovernmental entities and 
representatives, such as state and local governmental associations and 
individuals working in academia. 

In obtaining input from these stakeholders, DHS used a variety of 
mechanisms, such as multiagency working groups, solicitation of 
homeland security research papers, and a web-based forum, referred to 
as the National Dialogue, as shown in table 1. 

Table 1: Mechanisms Used by DHS for Obtaining Input on the QHSR from 
Various Stakeholder Groups: 

Stakeholder coordination mechanism: Study Groups; 
Lead agency/office: DHS Office of Policy; 
each study group was chaired by a DHS official and facilitated by a 
subject matter expert from the Homeland Security Studies and Analysis 
Institute; 
Stakeholder participants: DHS directorates, components, offices, 
subject matter experts, and research analysts; 
Nature of collaboration and activities: Provided analysis over a 5-
month period with work products that defined the nature and purpose of 
the homeland security missions to collaboratively share with other 
stakeholder groups. 

Stakeholder coordination mechanism: Steering Committee; 
Lead agency/office: DHS - Deputy Assistant Secretary for Policy 
(Strategic Plans); 
Stakeholder participants: DHS study group chairs and independent 
facilitators, Director of DHS's Office of Program Analysis and 
Evaluation, and representatives from DHS's Office of Intergovernmental 
Affairs, Science and Technology Directorate, Office of International 
Affairs, Office of General Counsel, and Office of Intelligence and 
Analysis; 
Nature of collaboration and activities: Provided day-to-day management 
and oversight of the QHSR report process. According to the QHSR 
report, they met weekly to review and integrate study group materials 
into the QHSR report. The committee also held monthly meetings during 
which each study group presented its progress toward developing 
recommendations and issues that required leadership consideration and 
decision. 

Stakeholder coordination mechanism: Senior Leadership Meetings; 
Lead agency/office: DHS; 
Stakeholder participants: DHS senior leadership, such as the Deputy 
Secretary of Homeland Security, and the heads of directorates and 
components; 
Nature of collaboration and activities: Reviewed and provided 
concurrence on study group recommendations for the QHSR mission goals 
and objectives. 

Stakeholder coordination mechanism: National Security Staff Sub- 
Interagency Policy Committees (Sub-IPC); 
Lead agency/office: National Security Staff and DHS officials led each 
of six Sub-IPCs; 
Stakeholder participants: Twenty-six federal departments and agencies 
and 6 entities within the Executive Office of the President.a 
departments and agencies participated in Sub-IPCs based on whether 
they had roles or activities related to the Sub-IPCs' mission areas; 
Nature of collaboration and activities: Provided a forum for study 
groups to gather interagency input as the study groups developed 
proposals for QHSR mission goals, objectives, and other report content. 

Stakeholder coordination mechanism: Strategy Coordination Group; 
Lead agency/office: DHS - Deputy Assistant Secretary for Policy 
(Strategic Plans); 
Stakeholder participants: Representatives of DHS and other federal 
agencies and White House staff; 
Nature of collaboration and activities: In addition to the Sub-IPCs, 
interagency input was provided by the Strategy Coordination Group to 
allow strategy and policy planners from across federal agencies an 
opportunity to share their feedback and perspectives on the review. 
According to the QHSR report, monthly meetings allowed federal 
officials responsible for similar strategic reviews to share lessons 
learned and best practices regarding their respective reviews and 
planning processes. 

Stakeholder coordination mechanism: Solicitation of Stakeholder 
Position Papers; 
Lead agency/office: DHS; 
Stakeholder participants: Various homeland security stakeholder 
organizations representing state, local, tribal, territorial, 
nongovernmental, private sector, and professional interests; 
Nature of collaboration and activities: Solicited position papers from 
118 stakeholder groups, such as the All Hazards Consortium and the 
Airports Council International North America. DHS study groups used 
the 43 documents submitted by the stakeholders groups to help frame 
and inform study group discussions. 

Stakeholder coordination mechanism: Web-based Discussion Forum 
(referred to as the National Dialogue); 
Lead agency/office: DHS with NAPA; 
Stakeholder participants: Open to anyone, including the members of the 
general public, who wanted to provide input on the QHSR content; 
DHS engaged in deliberate outreach to organizations with interests in 
homeland security such as business and academia; 
Nature of collaboration and activities: Provided a series of three web-
based discussions to obtain direct input and perspectives from 
participants to comment on study group materials. According to DHS, 
this forum resulted in over 3,000 comments on study group materials. 
The study groups used this information to inform the QHSR analyses and 
posted updated materials on each successive dialogue to show 
participants how their comments informed study group work. 

Stakeholder coordination mechanism: Executive Committee; 
Lead agency/office: DHS; 
Stakeholder participants: Ten stakeholder associations, such as the 
National Governors Association and the U.S. Conference of Mayors; 
Nature of collaboration and activities: Provided monthly 
teleconferences with associations throughout the review to keep the 
associations apprised of the review progress. 

Source: GAO analysis of DHS information. 

[A] The 26 federal departments and agencies and 6 entities within the 
Executive Office of the President were the Department of Homeland 
Security, Department of State, Department of Justice, Department of 
Defense, Department of Transportation, Department of Energy, 
Department of Health and Human Services, Office of the Director of 
National Intelligence, Department of the Treasury, National 
Counterterrorism Center, United States Postal Service, General 
Services Administration, Office of Management and Budget, National 
Security Staff, Office of Global Maritime Situational Awareness, 
Department of Labor, Domestic Policy Council, United States Trade 
Representative, Council of Economic Advisors, National Economic 
Council, Department of Education, Department of Agriculture, Nuclear 
Regulatory Commission, Office of Science and Technology Policy, Office 
of Personnel Management, Department of Veterans Affairs, Environmental 
Protection Agency, United States Geological Survey, United States Army 
Corps of Engineers, National Guard Bureau, National Institute of 
Standards and Technology, and Department of Housing and Urban 
Development. 

[End of table] 

We obtained comments from 63 stakeholders who DHS consulted with 
through these mechanisms. The 63 stakeholders who responded to our 
request for comments on the QHSR process noted that DHS conducted 
outreach to them to solicit their views and provided opportunities for 
them to give input on the QHSR. For example, DHS stakeholders, 
including its directorates, offices, and components, reported 
participating in the QHSR process by, for example, helping develop 
strategic outcomes and measurable end states for the QHSR missions, 
assigning representatives to the various QHSR study groups, and 
helping to draft QHSR report language.[Footnote 22] Stakeholders from 
21 federal agencies other than DHS and its components that responded 
to our request for comments noted that they provided input during the 
QHSR process by, among other things, having representatives attend 
QHSR meetings, participating in sub-interagency policy committee 
meetings, and commenting on draft versions of the QHSR report. 
[Footnote 23] Additionally, 6 nonfederal stakeholders reported to us 
that DHS consulted with them by, for example, sending a representative 
to association meetings, participating in conference calls to discuss 
the QHSR, and holding stakeholder briefings to discuss QHSR strategic 
goals, outcomes, and responsibilities. 

DHS, QHSR stakeholders, and other entities, specifically the QRAC and 
NAPA, that reviewed aspects of the QHSR identified various benefits 
from DHS's consultation efforts throughout the QHSR. For example, the 
Deputy Assistant Secretary for Policy (Strategic Plans) stated that 
stakeholder position paper submissions obtained at the beginning of 
the QHSR process were beneficial in that study groups had stakeholder 
input at the outset of the work. The Deputy Assistant Secretary also 
stated that the National Dialogue was beneficial in that it gave DHS 
the ability to gauge reactions to proposals for including information 
in the QHSR in real time, as the National Dialogue represented a 
virtual discussion among stakeholders. Further, 33 respondents to our 
request for comments on the QHSR reported that one positive aspect of 
DHS's consultations during the QHSR was the range of stakeholders DHS 
contacted. Two DHS stakeholders reported, for example, that DHS made 
extensive efforts to involve a wide range of stakeholders and that 
involvement of federal non-DHS agencies was beneficial in helping DHS 
obtain views on the QHSR outside of the department. One DHS 
stakeholder noted that the benefit of involving state, local, and 
private industry in the QHSR study group discussions via the National 
Dialogue was that the study groups were able to systematically 
consider viewpoints of the public during the course of developing the 
QHSR mission goals and objectives. The public perspectives offered 
different views than those provided by DHS and other federal 
stakeholders. Similarly, 2 federal stakeholders responded that the 
interagency meetings and the National Dialogue were positive ways in 
which DHS involved stakeholders during the QHSR, and that DHS's 
consultations provided a mechanism for interagency collaboration to 
discuss QHSR goal and objective areas. Additionally, one QRAC member 
noted that DHS involved and coordinated well with federal agencies; 
reached out reasonably well to state, local and tribal organizations; 
included a large number of academics and other policy experts; and 
gave the American public an opportunity to comment through the 
National Dialogue. 

Moreover, in its report on the QHSR, the QRAC noted that while not 
privy to the details of all inputs received, the QHSR report 
represented a synthesis of stakeholder consultations that was designed 
to set forth a shared vision of homeland security in order to achieve 
unity of purpose across the homeland security enterprise. In addition, 
with regard to the National Dialogue, NAPA reported that by engaging 
stakeholders at all levels, DHS was able to incorporate ground-level 
expertise and specialized knowledge into the review. According to 
NAPA, by conducting a process accessible to all interested parties, 
the National Dialogue provided the opportunity to strengthen trust 
among stakeholders and create potential buy-in for later 
implementation of policies and priorities they helped to shape. 

Time Constraints, Nonfederal Stakeholder Participation, and Definition 
of Stakeholders' Roles Hindered QHSR Consultations: 

DHS consulted with a range of stakeholders through various mechanisms, 
but DHS officials and stakeholders identified challenges that hindered 
DHS's consultation efforts in conducting the QHSR. These challenges 
were (1) consultation time frames, (2) inclusion of nonfederal 
stakeholders, and (3) definition of stakeholders' roles and 
responsibilities. 

Consultation Time Frames: 

According to DHS officials, the department consulted with stakeholders 
primarily over a 5-month period--from May through September 2009-- 
during the QHSR process. In response to our request for comments on 
the QHSR process, 16 stakeholders noted concerns regarding the time 
frames they had for providing input into the QHSR or BUR. Nine DHS 
stakeholders, for example, responded that in their view, the limited 
time available for development of the QHSR did not allow DHS to have 
as broad and deep an engagement with stakeholders as DHS could have 
experienced if more time had been allotted to stakeholder 
consultations. DHS stakeholders also reported to us that DHS's time 
frames for conducting the BUR were short and that the BUR process was 
hampered by an overly aggressive timeline for deliberation and 
decision making. Two of the study group facilitators who responded to 
our request for comments reported that in their view, stakeholders 
needed more time to review draft work products and hold more 
discussions. Three federal stakeholders suggested that the process be 
initiated earlier than it was for the first QHSR to provide more time 
for DHS to consider and resolve stakeholder comments, draft the 
report, and provide stakeholders with an opportunity to review the 
draft report. One of these federal stakeholders stated that more 
detail and other viewpoints would have been added to the QHSR if DHS 
had conducted outreach earlier in the QHSR process while another noted 
that it was difficult to keep up with the changes in the QHSR draft 
report and therefore to fully participate in providing comments. There 
were multiple drafts and no dialogue on how the comments were 
incorporated, according to this stakeholder. This federal stakeholder 
stated that more lead time in the provision of QHSR materials would 
have allowed for stakeholders to better consider the information and 
provide DHS with feedback. Two state and local associations responded 
that more lead time for the arrangement of meetings and a review of 
the complete QHSR report prior to its release would have been helpful. 

In addition, NAPA identified challenges associated with time frames 
for conducting aspects of the QHSR. Specifically, in its report on the 
National Dialogue, NAPA stated that the abbreviated turnaround time 
between phases of the National Dialogue--approximately 3 weeks on 
average--resulted in very constrained time periods for the study 
groups to fully review stakeholder feedback, incorporate it into the 
internal review process, and use it to develop content for subsequent 
phases. NAPA reported that for DHS to improve online stakeholder 
engagement it should build sufficient time for internal review and 
deliberations into its timetable for public engagement on the QHSR, 
and provide the public an opportunity to see that it is being heard in 
each QHSR phase. Thus, related to the National Dialogue, NAPA 
recommended that DHS build a timetable that allows ample time for 
internal deliberations that feed directly into external transparency. 
According to the Deputy Assistant Secretary for Policy (Strategic 
Plans) at DHS, addressing NAPA's recommendations, in general, is part 
of the QHSR project planning to begin during summer 2011 for the next 
QHSR. The official stated that DHS is considering NAPA's 
recommendations and is looking for opportunities for additional 
stakeholder involvement during the next QHSR. 

DHS identified those stakeholders to be consulted and various 
consultation mechanisms to be used prior to initiation of stakeholder 
consultations, but planned the consultation time periods based on the 
limited time available between when the QHSR process began and when 
the report was due, contributing to the time frame concerns raised by 
the 16 QHSR stakeholders and NAPA. Our prior work on strategic studies 
has shown that when federal agencies are defining missions and 
outcomes, such as DHS did in developing the QHSR report, involving 
stakeholders is a key practice.[Footnote 24] According to program 
management standards, stakeholder and program time management are 
recognized practices, among others, for operating programs 
successfully.[Footnote 25] Stakeholder management defines stakeholders 
as those whose interests may be affected by the program outcomes and 
that play a critical role in the success of any program; it should 
ensure an active exchange of accurate, consistent, and timely 
information that reaches all relevant stakeholders. Time management is 
necessary for program components and entities to keep the overall 
program on track, within defined constraints, and produce a final 
product. 

According to the Deputy Assistant Secretary for Policy (Strategic 
Plans) at DHS, constrained time periods for stakeholder consultations 
are part of the challenge of executing a time-limited process with a 
broad stakeholder base, such as the QHSR. According to the Deputy 
Assistant Secretary, longer time periods for stakeholder consultations 
could be beneficial, but a tradeoff to consider is that the review as 
a whole would be more time consuming. DHS officials determined time 
periods for consultation by planning from the QHSR issuance date and 
then building in stakeholder consultation periods for white paper 
solicitation and receipt, the National Dialogue, and executive 
committee meetings. Stakeholder consultation time frames were built 
into the QHSR project plan, with planned time periods such as 23 days 
between white paper solicitation notifications and the deadline for 
submissions from stakeholders, which was dictated by the necessities 
of the December 31, 2009 issuance deadline. The National Security 
Staff set timelines for report review by other federal agencies, 
according to the Deputy Assistant Secretary. Moreover, this official 
said that setting target time frames for stakeholder consultations 
during the next QHSR is something that DHS plans to address during 
project planning. By considering ways to build more time for 
stakeholder consultations into the timeline or target time frames for 
the next QHSR, DHS could be better positioned to manage stakeholder 
consultations and feedback received throughout the process, including 
determining and communicating how much time stakeholders will be given 
for providing feedback and commenting on draft products. In addition, 
DHS could be better positioned to ensure that stakeholders have the 
time needed for reviewing QHSR documents and providing input. 

Inclusion of Nonfederal Stakeholders: 

DHS consulted with a range of stakeholders, including federal and 
nonfederal entities, during the QHSR, and these consultations provided 
DHS with a variety of perspectives for consideration as part of the 
QHSR process. However, the department faced challenges in obtaining 
feedback from nonfederal stakeholders. Our prior work on key practices 
for performance management has shown that stakeholder involvement is 
important to help agencies ensure that their efforts and resources 
target the highest priorities.[Footnote 26] Involving stakeholders in 
strategic planning efforts can also help create a basic understanding 
among the stakeholders as to the agency's programs and results they 
are intended to achieve. Without this understanding successful 
implementation can be difficult because nonfederal stakeholders help 
clarify DHS's missions, reach agreement on DHS's goals, and balance 
the needs of other nonfederal stakeholders who at times may have 
differing or even competing goals. As we have previously reported, 
nonfederal entities have significant roles in homeland security 
efforts.[Footnote 27] For example, state, local and private sector 
entities own large portions of critical infrastructure in the United 
States and have responsibilities for responding to and recovering from 
homeland security incidents. Thus we have previously reported that it 
is vital that the public and private sectors work together to protect 
these assets. Further, we have reported on the need for federal and 
nonfederal entities to more effectively communicate their emergency 
preparedness and response roles, responsibilities, and activities. 
[Footnote 28] For example, we have reported that effective public 
warning depends on the expertise, efforts, and cooperation of diverse 
stakeholders, such as state and local emergency managers and the 
telecommunications industry. 

In responding to our request for comments, 9 stakeholders commented 
that DHS consultations with nonfederal stakeholders, such as state, 
local and private sector representatives, could be enhanced. For 
example, 1 stakeholder noted that state, local and private sector 
representatives, such as those with responsibility for securing 
critical infrastructure and key resources, the maritime sector, and 
overseas interests, should be further consulted during the next QHSR 
process. One federal stakeholder noted that state and local 
involvement is critical for homeland security and that a review of 
state and local readiness would be beneficial to determine the gaps 
that would need to be filled at the federal level. DHS could map out 
what state and local officials need in case of an emergency and 
include the various federal agencies in these discussions. Further, 
another stakeholder noted that DHS faced challenges in consulting 
specifically with the private sector during the 2010 QHSR. DHS 
consulted with private sector entities primarily through (1) the QRAC, 
whose membership comprises of individuals from academia, nonprofit 
research organizations, private consultants, and nonprofit service 
providers and advocacy organizations; and (2) the National Dialogue. 
With regard to the QRAC, it met nine times during which it received 
information from DHS leadership regarding the QHSR design, analysis, 
and interim conclusions, and its members provided feedback and advice 
to DHS. However, one QRAC respondent noted that the council's members 
were predominately consultants and not representatives of industries 
affected by homeland security threats, such as critical infrastructure 
sectors, which resulted in views that were not representative of one 
of the most affected members of the nonfederal homeland security 
community. This respondent stated that enhancing participation of 
private sector representatives is important for the next QHSR, as it 
would help DHS obtain views from entities that provide homeland 
security and emergency responses services, such as one corporation 
providing water to victims after Hurricane Katrina. According to this 
stakeholder, private sector entities could help offer DHS views on, 
for example, best practices for how to prepare for and respond to 
homeland security events or technology enhancements for homeland 
security. 

With regard to the National Dialogue--one of the primary mechanisms 
used for soliciting input from nonfederal stakeholders--17 
stakeholders who responded to our request for comments on the QHSR, as 
well as NAPA, identified challenges. As an example of comments we 
received from these 17 stakeholders, 1 federal stakeholder reported 
that the National Dialogue did not appear to have significant impact 
on the QHSR because in interagency meetings involving this 
stakeholder, information from the National Dialogue was not discussed. 
In an additional example, one QRAC member responded that the National 
Dialogue included a small number of comments from the private sector 
and did not reflect the significant number of stakeholders around the 
country with homeland security responsibilities. This respondent 
stated that the National Dialogue was an important exercise but was 
not an effective means for obtaining representative views specifically 
of the private sector. Further, as another example, one state and 
local association responded that in its view, DHS's conclusions on 
QHSR strategy had been reached prior to initiation of the National 
Dialogue, making it appear to the association that although DHS was 
soliciting its input, the department did not view the association as 
playing a consultative role in the QHSR development. In addition, NAPA 
reported that engaging nonfederal associations, such as the National 
Association of Counties, did not necessarily equate to reaching out to 
individual nonfederal entities, such as cities and counties. 
Therefore, according to NAPA's report, through the National Dialogue, 
DHS notified approximately 1,000 contact members of nonfederal 
associations in an effort to include a range of nonfederal homeland 
security practitioners. Based on this outreach effort, NAPA's report 
recommended continuing efforts to gain significant buy-in from 
nonfederal associations to ensure that DHS obtains access to the 
nonfederal stakeholders it wishes to consult regarding the QHSR. 

DHS faced challenges in obtaining nonfederal input during the QHSR 
process for two reasons. First, convening state and local government 
officials for consultation, especially from individual nonfederal 
stakeholders, on the QHSR was a significant logistical challenge, 
according to DHS officials. Because of this challenge, DHS opted to 
consult with national associations that could represent the 
perspectives of state, local, and tribal homeland security 
stakeholders. Second, according to the Deputy Assistant Secretary for 
Policy (Strategic Plans) at DHS, the Federal Advisory Committee Act 
(FACA), which establishes standards and uniform procedures for the 
establishment, operation, administration, and duration of advisory 
committees, affected how DHS was able to consult with private sector 
stakeholders when developing the QHSR report.[Footnote 29] 
Specifically, the Deputy Assistant Secretary noted that the department 
was limited in its ability to consult with private sector groups on an 
ongoing basis without forming additional FACA committees specifically 
for conducting consultations on the QHSR. DHS was also limited in its 
ability to seek feedback from established FACA committees that had 
been convened for other purposes. The meeting schedules of those 
committees did not align well with the QHSR study period, and there 
were significant logistical challenges to scheduling additional 
meetings of those groups to address QHSR. In addition, the Deputy 
Assistant Secretary for Policy (Strategic Plans) stated that under 
FACA DHS could not invite members of established FACA committees 
convened for other purposes to join meetings of the QRAC for the 
purpose of providing advice and feedback. One study group facilitator 
commented that the FACA consideration significantly reduced the role 
that nonfederal stakeholders played in the QHSR. According to this 
respondent, addressing the FACA requirements and including appropriate 
FACA-compliant groups with a broader range of academics and others 
could have affected the outcome of the study group's deliberations. 
However, according to the Deputy Assistant Secretary, establishing new 
FACA committees in addition to the QRAC, which DHS established as a 
FACA-compliant committee specifically for QHSR consultations, was 
prohibitively time consuming within the time frames DHS had for 
conducting the 2010 QHSR. 

Four respondents to our request for comments made suggestions for 
alternative approaches for obtaining viewpoints of nonfederal 
stakeholders in future QHSRs. For example, one study group facilitator 
stated that state and local associations could put together a group of 
their members to engage in the QHSR process and be part of the study 
groups. In addition, the National Dialogue could have provided more 
focused questions and provided to a broad group of state and local 
experts questions on specific issues, such as housing disaster 
resiliency. This approach could have allowed more state, local, 
private sector, academic and nongovernmental organizations into the 
QHSR process, according to the facilitator. DHS officials noted, 
though, that the National Dialogue was not intended to address 
individual initiatives, as the QHSR was intended to focus on broader 
homeland security issues. Further, one local government association 
suggested that this association could put together a crosscutting 
group of local officials who could discuss specific issues, such as 
national preparedness. In addition to alternative approaches for 
obtaining viewpoints of nonfederal stakeholders provided by 
respondents outside of DHS, one DHS stakeholder responded they held in-
person or teleconferencing meetings with numerous interest groups and 
associations, while another DHS stakeholder responded that the 
component sent emails to its stakeholder groups soliciting the groups' 
views on the QHSR. Additionally, in our prior work on the Federal 
Emergency Management Agency's (FEMA) process for updating the National 
Response Framework, we identified examples of ways in which FEMA 
involved nonfederal stakeholders in the process. For example, FEMA 
posted a spreadsheet that included the comments made by nonfederal 
stakeholders and the final disposition DHS assigned to each of those 
comments to allow stakeholders to see how DHS did or did not 
incorporate their comments. Further, FEMA had agency leaders appoint 
advisory council members who represented a geographic and substantive 
cross section of officials from the nonfederal community.[Footnote 30] 

Given the significant role that state and local governments and the 
private sector play in homeland security efforts, which is 
acknowledged by DHS in the QHSR report, examining mechanisms, such as 
those proposed by QHSR stakeholders or used by components, could help 
DHS include a broader segment of these representatives during the QHSR 
process and better position DHS to consider and incorporate, as 
appropriate, nonfederal concerns and capabilities related to homeland 
security in the next QHSR. 

Definition of Stakeholders' Roles and Responsibilities: 

DHS identified stakeholders' roles and responsibilities in the QHSR 
report primarily by referencing other homeland security-related 
documents, such as the National Response Framework and National 
Infrastructure Protection Plan, that describe homeland security roles 
and responsibilities.[Footnote 31] With regard to federal agencies, 
the QHSR report described homeland security roles and responsibilities 
with brief summaries of federal agencies' leadership roles for 
coordinating homeland security-related efforts. For example, the QHSR 
report listed the Attorney General's responsibilities as conducting 
criminal investigations of terrorist acts or threats by individuals or 
groups, collecting intelligence on terrorist activity within the 
United States, and leading the Federal Bureau of Investigation, the 
Drug Enforcement Administration, and the Bureau of Alcohol, Tobacco, 
Firearms and Explosives in their respective areas of homeland security 
responsibilities. With regard to nonfederal stakeholders' roles and 
responsibilities, the QHSR report provided summaries of roles and 
responsibilities, based on these and other homeland security-related 
documents, such as identifying that critical infrastructure owners and 
operators are responsible for developing protective programs and 
measures to ensure that systems and assets are secure from and 
resilient to threats. 

Our prior work has shown that agencies that work together to define 
and agree on their respective roles and responsibilities when 
implementing federal strategies that cross agency boundaries can 
enhance the effectiveness of interagency collaboration.[Footnote 32] 
In doing so, agencies clarify who will do what, organize their joint 
and individual efforts, and facilitate decision making. Further, our 
work on key characteristics for effective national strategies 
identified, among others, one desirable characteristic as defining the 
roles and responsibilities of the specific federal departments, 
agencies, or offices involved and, where appropriate, the different 
sectors, such as state, local, private, or international sectors. 
[Footnote 33] Inclusion of stakeholders' roles and responsibilities in 
a strategy that crosses agency boundaries is useful to agencies and 
other stakeholders in clarifying specific roles, particularly where 
there is overlap, and thus enhancing both implementation and 
accountability. In addition, we have reported that DHS needs to form 
effective and sustained partnerships with a range of other entities, 
including other federal agencies, state and local governments, and the 
private and nonprofit sectors.[Footnote 34] Successful partnering 
involves collaborating and consulting with stakeholders to develop and 
agree on goals, strategies, and roles to achieve a common purpose. 

In responding to our request for comments on the QHSR, 10 federal 
stakeholders noted that the roles and responsibilities listed in the 
QHSR report, as derived from other documents, such as the National 
Response Framework and National Infrastructure Protection Plan, 
reflected their homeland security missions and activities. For 
example, 1 federal stakeholder responded that the roles and 
responsibilities listed in the QHSR report were established in 
previous documents and were accurate, and another federal stakeholder 
noted that the roles and responsibilities listed in the QHSR report 
were derived from previously published material for which the 
stakeholder had provided input. However, DHS and 10 other respondents 
to our request for comments noted that the department could strengthen 
its definition of homeland security roles and responsibilities in the 
next QHSR by better reflecting the range of the stakeholders' roles 
and responsibilities. Specifically, in the QHSR report DHS identified 
the need to better assess stakeholders' homeland security roles and 
responsibilities, noting that although the report was not intended to 
describe stakeholders' roles and responsibilities, the division of 
operational roles and responsibilities among federal departments and 
agencies for various homeland security goals and objectives emerged as 
a major area requiring further study following the QHSR report. DHS 
reported that an analysis of roles and responsibilities across the 
homeland security missions would help resolve gaps or unnecessary 
redundancies between departments and agencies going forward. Further, 
10 stakeholders commented to us that the definitions of roles and 
responsibilities in the QHSR report could be enhanced to better 
reflect the range of homeland security stakeholders' responsibilities. 
For example, 3 federal stakeholders reported that roles and 
responsibilities definitions in the QHSR could be enhanced by, for 
example, recognizing the variety of agency or administration-level 
responsibilities of the cabinet departments. In particular, one of 
these federal stakeholders suggested that the next QHSR may want to 
include a more detailed delineation of the roles and responsibilities 
of departments to support the homeland security enterprise by (1) 
reflecting the broad nature of responsibilities across a broad 
spectrum of threats and (2) identifying readiness and resource 
requirements to address the stated roles and responsibilities. The 
brief narrative on roles and responsibilities in the 2010 QHSR report 
presented a shortened version of the roles and responsibilities that 
the federal stakeholder has in supporting the homeland security 
enterprise. Another federal stakeholder noted that formalizing the 
process to elicit values and judgments from individual agencies would 
help ensure adequate representation of each agency's role in the next 
version of the QHSR report. The formalized process, according to the 
federal stakeholder, would convene agency officials and facilitate a 
discussion, resulting in a common understanding of how agency roles 
and responsibilities are defined for executing the QHSR strategy. 

In our December 2010 report on the extent to which the QHSR report 
addressed reporting elements that the 9/11 Commission Act specified 
for the report, we noted that DHS partially addressed two reporting 
elements for the QHSR report related to roles and responsibilities for 
homeland security stakeholders. These elements were for the QHSR 
report to include a discussion of the status of (1) cooperation among 
federal agencies in the effort to promote national security and (2) 
cooperation between the federal government and state, local, and 
tribal governments in preventing terrorist attacks and preparing for 
emergency response to threats to national homeland security.[Footnote 
35] With regard to the first element, we reported that although the 
QHSR and BUR reports discussed homeland security roles and 
responsibilities for federal agencies, they did not discuss 
cooperation on homeland security efforts among federal agencies other 
than DHS. We reported that while the QHSR discussion of roles and 
responsibilities as found in other documents was helpful for 
understanding which federal agencies lead particular homeland security 
efforts, the QHSR report did not provide a description of how federal 
agencies cooperate with one another in addressing homeland security 
efforts. With regard to the second element, we reported that although 
the QHSR and BUR reports provided descriptions of cooperation between 
DHS and state, local, and tribal governments, they did not discuss the 
status of cooperation between other federal agencies that have 
homeland security responsibilities and state, local, and tribal 
governments. DHS officials stated that DHS solicited comments from 
other federal departments and state, local, and tribal governments on 
the role and responsibility descriptions for each of these entities 
listed in the QHSR report. 

According to the Deputy Assistant Secretary for Policy (Strategic 
Plans) at DHS, during the QHSR process the department did not attempt 
to discuss the status of cooperation among other federal departments 
and state, local, and tribal governments. DHS officials stated that 
the department viewed such a discussion as outside its authority to 
conduct and that those discussions were conducted in other venues, 
such as the National Infrastructure Protection Plan and the National 
Response Framework. Because the National Response Framework and the 
National Infrastructure Protection Plan were completed during DHS's 
launch of the QHSR, in 2008, use of those definitions in the QHSR was 
appropriate, according to the official. DHS did not obtain comments 
from all stakeholders on the definitions listed in the QHSR report, 
but looked at stakeholder comments on roles and responsibilities 
received during the National Infrastructure Protection Plan and 
National Response Framework drafting processes. The definitions listed 
in the QHSR report were also shared with DHS's Office of 
Intergovernmental Affairs, which solicited comments from stakeholders, 
as necessary, based on any roles that may have changed since the 
National Infrastructure Protection Plan and National Response 
Framework were published, according to the official. 

In its May 2010 report on the QHSR, the QRAC noted that the QHSR 
report included a summary of roles and responsibilities of key 
stakeholders that was derived from existing statutes, among other 
documents. However, according to the QRAC report, the QHSR report did 
not provide a mapping of these roles and responsibilities to the QHSR 
missions and further work was required to deconflict and potentially 
supplement existing homeland security stakeholder role and 
responsibility policies and directives. According to the QRAC report, 
the QHSR report was designed to create a shared vision of homeland 
security in order to achieve unity of purpose; a vital next step was 
the delineation of key roles and responsibilities for individual QHSR 
goals and objectives to generate unity of effort. A comprehensive 
mapping of stakeholder roles and responsibilities to QHSR missions, 
goals, and objectives was needed to (1) enable assessment of the 
current state of cooperation and coordination between all public and 
private sector stakeholder communities; (2) identify potential gaps, 
conflicts, or both in current policies and directives from an 
enterprise perspective; and (3) underpin follow-on planning efforts. 
The QRAC recommended that DHS map goals to objectives for each core 
QHSR mission and key stakeholder communities to delineate the 
stakeholders' respective roles and responsibilities. In response to 
this recommendation, DHS plans to map the existing QHSR mission goals 
and objectives to stakeholder roles and responsibilities during the 
pre-execution year for the next QHSR, if possible. This setup work on 
role and responsibility mapping would allow for work at the end of the 
QHSR process to map roles and responsibilities to final QHSR goals and 
objectives developed during the next QHSR. Consistent with the QRAC's 
recommendation and DHS's planned actions, by seeking to further define 
homeland security stakeholders' roles and responsibilities in the next 
QHSR, DHS could be better positioned to identify, understand, and 
address any potential gaps in roles and responsibilities or areas for 
additional or enhanced cooperation and coordination. 

DHS Plans to Implement a National Risk Assessment as Part of the Next 
QHSR: 

Through the QHSR, DHS identified various threats confronting homeland 
security but did not conduct a risk assessment for the QHSR. In the 
2010 QHSR report, DHS identified six threats confronting homeland 
security, such as high-consequence weapons of mass destruction and 
illicit trafficking and related transnational crime, as well as five 
global challenges, including economic and financial instability and 
sophisticated and broadly available technology. According to the QHSR 
report, these threats and challenges were the backdrop against which 
DHS planned to pursue its homeland security efforts. The threats and 
global challenges listed in the QHSR report were developed through 
discussions with federal national security officials and through 
reviews of intelligence community materials, according to DHS 
officials. 

Multiple DHS guidance documents emphasize the importance of 
considering risk assessment information when engaging in strategic 
decisions. For example, DHS's Integrated Risk Management Framework 
(IRMF), published in January 2009, calls for DHS to use risk 
assessments to inform DHS-wide decision-making processes.[Footnote 36] 
Risk assessments, which include assessing and analyzing risk in terms 
of threats, vulnerabilities, and consequences of a potential homeland 
security incident, are the foundation for developing alternative 
strategies for managing risk, according to the IRMF. Similarly, the 
QHSR report includes an objective for DHS to establish an approach for 
national-level homeland security risk assessments, specifically 
calling for development and implementation of a methodology to conduct 
national-level homeland security risk assessments. Our prior work on 
federal strategic studies has also found that establishing an analytic 
framework to assess risks is a key aspect of developing a strategy to 
address national problems, such as homeland security.[Footnote 37] 
Consistent with the IRMF, we define risk assessment as a qualitative 
determination, a quantitative determination, or both of the likelihood 
of an adverse event occurring and the severity, or impact, of its 
consequences.[Footnote 38] 

DHS has called for the use of national risk assessments for homeland 
security but did not conduct such an assessment as part of the 2010 
QHSR. DHS officials stated that at the time DHS conducted the QHSR, 
DHS did not have a well-developed methodology or the analytical 
resources to complete a national risk assessment that would include 
likelihood and consequence assessments. The QHSR terms of reference, 
which established the QHSR process, also stated that at the time the 
QHSR was launched, the homeland security enterprise lacked a process 
and a methodology for consistently and defensibly assessing risk at a 
national level and using the results of such an assessment to drive 
strategic prioritization and resource decisions. 

In recognition of a need to develop a national risk assessment 
methodology, the QHSR National Risk Assessment Study Group was created 
as part of the QHSR process. In establishing the study group, the QHSR 
Terms of Reference stated that assessing national risk was a 
fundamental and critical element of an overall risk management 
process, with the ultimate goal of improving the ability of decision 
makers to make rational judgments about tradeoffs between courses of 
action to manage homeland security risks. The QHSR National Risk 
Assessment Study Group consulted with subject matter experts from the 
federal government, academia, and the private sector and, in October 
2009, produced the Homeland Security National Risk Assessment (HSNRA) 
methodology, which established a process for conducting a national 
risk assessment in the future.[Footnote 39] According to DHS 
officials, because the HSNRA methodology was developed as part of the 
QHSR process and finalized as the QHSR report was being completed in 
late 2009, it was not intended to be implemented during the 2010 QHSR. 

The HSNRA is to use a methodology for assessing risk across a range of 
hazards for use by DHS in its decisions on strategy and policy 
development, planning priorities, resource allocation, and capability 
requirements development. The HSNRA includes definitions or 
descriptions of the scope of incidents it applies to, risk formula, 
and likelihood and consequence (see table 2). 

Table 2: Characteristics of the HSNRA: 

HSNRA characteristics: Scope; 
Description of characteristics: The scope of the assessment includes 
natural disasters, terrorism, transnational incidents (such as mass 
migration), cyberattacks, public health emergencies or incidents, and 
industrial accidents. The HSNRA is future looking, evaluating risks up 
to 7 years from the assessment date. 

HSNRA characteristics: Risk formula; 
Description of characteristics: Risk is defined as a function of the 
likelihood and the consequences of a homeland security scenario, such 
as a major metropolitan area being hit by a hurricane or a terrorist 
group detonating a 10-kiloton nuclear weapon in a major population 
center. Approximately 100 different scenarios would be used in an 
HSNRA. For risk comparison purposes, the risk formula provides 
absolute risk, which allows for comparisons across different 
categories of scenarios because it provides annualized loss estimates 
in a common numerical framework. For example, a hypothetical example 
provided in the HSNRA estimates 10 lives lost per year for a set of 
human and agricultural disease scenarios compared to approximately 100 
lives lost per year for a set of cyberattack scenarios. 

HSNRA characteristics: Likelihood and consequence; 
Description of characteristics: Likelihood is expressed as the 
expected frequency of a scenario occurring per unit of time. 
Consequences are expressed in categories of losses depending on 
available data, such as deaths, economic damages, and environmental 
impacts. Values for the likelihood and consequence data used for each 
scenario would be derived from existing DHS risk evaluations and from 
elicitation of opinions from subject matter experts. 

Source: GAO analysis of HSNRA documents. 

[End of table] 

Outputs from the HSNRA calculations could be expressed in a number of 
ways, such as plotting scenarios on a two-dimensional graph with 
scenario frequency estimates on the x axis and scenario consequence 
estimates on the y axis, as shown in figure 3. 

Figure 3: Example of HSNRA Output for Use by DHS Decision Makers: 

[Refer to PDF for image: illustration] 

All data notional - not actual risk information. 

The illustration depicts frequency (number of expected events per year), for the following: 
Terrorism Scenarios (T); 
Natural Disaster Scenarios (ND); 
Disease (Human and Agricultural) Scenarios (D); 
Industrial Accident Scenarios (IA); 
Cyber Attack Scenarios (CA); 
Transnational Crime Scenarios (TC). 

The following are also depicted: 
Terrorism consequence uncertainty; 
Other hazard categories’ uncertainty. 

Source: GAO analysis of DHS documents. 

[End of figure] 

In accordance with the QHSR goal of implementing a national risk 
assessment and with issuance of Presidential Policy Directive 8, which 
calls for risk analysis across a range of homeland security threats, 
DHS is planning to conduct a national risk assessment as part of its 
next QHSR.[Footnote 40] In determining how to conduct a national risk 
assessment, DHS is considering various factors, such as how to 
incorporate and use an assessment's results, the time frames and costs 
for conducting an assessment, and what alternatives exist to 
conducting a national assessment. 

* Use of national risk assessment results. DHS officials stated that 
one consideration in determining how to conduct a national risk 
assessment is the manner in which the department would use the results 
of such an assessment to inform the QHSR. Specifically, DHS officials 
told us that a national risk assessment, such as the HSNRA, should be 
one of multiple inputs considered in conducting the QHSR, with other 
inputs including such factors as privacy and civil liberties concerns, 
economic interests, and administration priorities. DHS's risk 
assessment guidance makes a similar point, stating that risk 
information is usually one of multiple factors decision makers 
consider and is not necessarily the sole factor influencing a 
decision. There may be times when the strategy selected and 
implemented does not optimally reduce risk and decision makers should 
consider all factors when selecting and implementing strategies. The 
National Research Council of the National Academies also recently 
reported that risk analysis is one input to decision making, designed 
to inform decisions but not dictate outcomes.[Footnote 41] DHS 
officials noted that it would be important to communicate this to its 
stakeholders, including Congress, the public, and others, to manage 
any expectations that QHSR decisions would be solely based on risk 
assessment results. 

* National risk assessment time frames and costs. According to DHS 
officials, the HSNRA would require 12 months to complete and would 
need to be completed before launching the next QHSR, since the 
assessment would help frame how the QHSR missions are defined. If the 
next QHSR is conducted during fiscal year 2013 and reported by 
December 2013, as anticipated by DHS, the HSNRA would need to be 
completed during fiscal year 2012 to help inform the QHSR, according 
to DHS officials. With regard to financial costs, DHS officials 
estimated that conducting the HSNRA over a 12-month period would cost 
from $3 million to $6 million. DHS's Deputy Assistant Secretary for 
Policy (Strategic Plans) stated that the HSNRA is a sound methodology 
that should be used as part of the next QHSR, and officials within 
DHS's unit responsible for developing the HSNRA, the Office of Risk 
Management and Analysis, stated that the benefits of having risk 
information available for input into developing the QHSR are worth the 
costs. 

* National risk assessment alternatives. In order to identify risks 
and inform mission areas for the next QHSR, DHS could consider 
alternatives to conducting a national risk assessment, according to 
DHS officials. These officials stated that one alternative approach 
would involve using segments of the HSNRA process to help provide risk 
information to department decision makers, such as eliciting expert 
judgments and surveying nonfederal experts about perspectives on the 
risks DHS should address. The officials stated that this approach 
would not be as useful as a complete HSNRA because a full HSNRA 
provides likelihood and consequence estimates for various homeland 
security incident scenarios, which offers a more complete picture of 
the risks DHS must address. Another approach, according to the 
officials, would be to identify risks through existing DHS analyses, 
such as the Homeland Security Threat Assessment or the National 
Planning Scenarios.[Footnote 42] The officials stated that 
identification of risks through these tools would also be limited and 
would not be as effective as completing the HSNRA. For example, the 
HSNRA includes likelihood estimates for scenarios, which these other 
tools do not include, and therefore provides a more complete picture 
of risk by addressing threats, likelihoods, and consequences. 

Consistent with DHS's plans, a national risk assessment conducted in 
advance of the next QHSR could assist DHS in developing QHSR missions 
that target homeland security risks and could allow DHS to demonstrate 
how it is reducing risk across multiple hazards. 

DHS Developed BUR Monitoring Mechanisms and Measures but Could 
Strengthen Its Prioritization Efforts by Using Risk Information: 

DHS Prioritized Its BUR Initiatives but Could Benefit from Considering 
Risk Information in Future Efforts: 

DHS considered various factors in identifying high-priority BUR 
initiatives for implementation in fiscal year 2012 but did not include 
risk information as one of these factors. Through the BUR, DHS 
identified 43 initiatives aligned with the QHSR mission areas to help 
strengthen DHS's activities and serve as mechanisms for implementing 
those mission areas. According to DHS officials, the department could 
not implement all of these initiatives in fiscal year 2012 because of, 
among other things, resource constraints and organizational or 
legislative changes would need to be made to implement some of the 
initiatives. In identifying which BUR initiatives to prioritize for 
implementation in fiscal year 2012, DHS leadership considered (1) 
"importance," that is, how soon the initiative needed to be 
implemented; (2) "maturity," that is, how soon the initiative could be 
implemented; and (3) "priority," that is, whether the initiative 
enhanced secretarial or presidential priorities.[Footnote 43] 
Component leadership officials, as subject matter experts, completed a 
survey instrument indicating their assessment of each BUR initiative 
based on these criteria. The results were then aggregated and 
presented to DHS's Program Review Board--which is the body that 
oversees DHS program reviews and the budgeting process. With the 
Deputy Secretary's leadership, the Program Review Board evaluated the 
results of the survey and refined the prioritization. The BUR 
initiative prioritization process resulted in the Secretary and Deputy 
Secretary of Homeland Security ranking and selecting 14 high-priority 
BUR initiatives to be implemented in fiscal year 2012, as shown in 
table 3. 

Table 3: DHS Ranking of 14 High-Priority BUR Initiatives to Be 
Implemented in Fiscal Year 2012: 

BUR initiative priority ranking: 1; 
QHSR mission: Mission 1; 
BUR initiative: Strengthen aviation security[A]. 

BUR initiative priority ranking: 2; 
QHSR mission: Mission 1; 
BUR initiative: Create an integrated departmental information-sharing 
architecture. 

BUR initiative priority ranking: 3; 
QHSR mission: Mission 4; 
BUR initiative: Increase DHS predictive and forensic capabilities for 
cyber intrusions and cyberattacks[A]. 

BUR initiative priority ranking: 4; 
QHSR mission: Mission 3; 
BUR initiative: Improve the detention and removal process. 

BUR initiative priority ranking: 5; 
QHSR mission: Mission 1; 
BUR initiative: Promote safeguards for access to secure areas in 
critical facilities. 

BUR initiative priority ranking: 6; 
QHSR mission: Mission 4; 
BUR initiative: Strengthen DHS ability to protect cyber networks[A]. 

BUR initiative priority ranking: 7; 
QHSR mission: Mission 2; 
BUR initiative: Prioritize immigration and customs investigations. 

BUR initiative priority ranking: 8; 
QHSR mission: Mission 4; 
BUR initiative: Promote cybersecurity public awareness. 

BUR initiative priority ranking: 9; 
QHSR mission: Mission 5; 
BUR initiative: Improve DHS's ability to lead in emergency management. 

BUR initiative priority ranking: 10; 
QHSR mission: Mission 5; 
BUR initiative: Make individual and family preparedness and critical 
facility resilience inherent in community preparedness. 

BUR initiative priority ranking: 11; 
QHSR mission: Mission 2; 
BUR initiative: Expand joint operations and intelligence capabilities, 
including enhanced domain awareness. 

BUR initiative priority ranking: 12; 
QHSR mission: Mission 2; 
BUR initiative: Enhance the security and resilience of global trade 
and travel systems. 

BUR initiative priority ranking: 13; 
QHSR mission: Mission 3; 
BUR initiative: Improve DHS immigration services processes. 

BUR initiative priority ranking: 14; 
QHSR mission: Mission 3; 
BUR initiative: Focus on fraud detection and national security 
vetting[A]. 

Source: DHS. 

[A] BUR initiative with request for additional funding in the 
President's fiscal year 2012 budget proposal. 

[End of table] 

We and DHS have called for the use of risk information in making 
prioritization, resource, and investment decisions. For example, DHS's 
IRMF states that DHS is to use risk information to inform strategies, 
processes, and decisions to enhance security and to work in a unified 
manner to manage risks to the nation's homeland security. The IRMF 
states that one of its objectives is to use an integrated risk 
management process to inform resource allocations on a departmentwide 
basis, which is critical to balance resources across the set of DHS 
strategic objectives. Likewise, our prior work has shown the 
importance of using risk information to inform resource prioritization 
decisions.[Footnote 44] For example, our risk management approach 
advises using risk information to inform resource allocation decisions 
so that management can consider which risks should be managed 
immediately and which risks can be deferred and addressed at a later 
time. 

According to DHS officials, using risk information as an input into 
DHS's prioritization of the initiatives was difficult for several 
reasons. For example, the BUR initiatives were highly differentiated, 
making comparisons based on risks the initiatives address impossible, 
according to DHS officials. Some of the BUR initiatives focus on 
organizational changes at DHS; others are extremely broad, addressing 
multiple and overlapping risks; and others focus on specific risks. 
For example, comprehensive immigration reform is a broad BUR 
initiative, addressing broad illegal immigration risks, while 
promoting safeguards for access to secure areas in critical facilities 
targets more specific risks. According to the officials, the variance 
in how the initiatives were defined allowed DHS to align initiatives 
with the QHSR strategy and consideration of such variance, in addition 
to risks addressed by QHSR implementation mechanisms, such as BUR 
initiatives, would be important in defining implementation mechanisms 
and initiatives for future QHSRs. However, DHS could not apply its 
existing risk assessment tools to evaluating and prioritizing BUR 
initiatives for the 2010 QHSR. 

For future QHSRs, DHS officials described several characteristics of 
mechanisms for implementing QHSR missions that would enable risk 
information to be used among prioritization criteria. First, the 
implementation mechanisms or initiatives to be prioritized based on 
risk information should be comparable in terms of the nature of the 
risks addressed. For example, comparing mechanisms to address DHS 
organizational changes that do not directly reduce homeland security 
risks with mechanisms that are designed to directly prevent terrorism 
risks would be an inappropriate comparison. Second, expected outcomes 
of the mechanisms or initiatives should be defined so that the risks 
reduced by the mechanisms can be estimated. For example, the BUR 
initiatives do not indicate the degree to which investments will 
change DHS's security capabilities. Knowing the increase (or decrease) 
in security capabilities associated with an implementation mechanism 
would allow estimates of risks reduced, which could be compared in 
prioritization efforts. Third, an implementation mechanism or 
initiatives should have a "line of sight" directly between the DHS 
activities associated with the mechanism and the risk reduced by those 
activities. In other words, according to the officials, DHS operations 
need to be closely aligned with identified risk reductions in order 
for risk reduction calculations to be accurately achieved. For 
example, U.S. Border Patrol efforts to stop illegal border crossings 
are closely aligned with reducing risks of illegal immigration. 

DHS officials stated that although existing DHS risk assessment tools 
could not be used to systematically prioritize the BUR initiatives for 
the 2010 QHSR, there is utility in thinking qualitatively about risks 
addressed by the initiatives when making future prioritization 
decisions. Risk information should not be the sole input but should be 
considered along with other criteria, according to Office of Risk 
Management and Analysis (RMA) officials and the Deputy Assistant 
Secretary for Policy (Strategic Plans). DHS has various tools that 
could, with some limitations, provide risk information for 
consideration when prioritizing implementation of QHSR mission 
objectives, as shown in table 4. Two tools, the Risk Analysis Process 
for Informed Decision-making (RAPID) and the methodology for 
conducting the HSNRA, were created to provide risk information for 
decision making across DHS mission areas. Risk analyses conducted 
within DHS components could also provide risk information useful for 
prioritizing QHSR implementation mechanisms, according to DHS 
officials. The officials stated that at least five current risk 
assessments used by DHS components could be useful for prioritizing 
QHSR implementation efforts within the mission areas relevant to the 
risk assessment tools. 

Table 4: DHS Risk Assessment Tools That DHS Reports Could Be Leveraged 
to Prioritize QHSR Implementation Mechanisms: 

DHS-wide risk assessment tools: 

DHS risk assessment tool: RAPID; 
Description of tool: RAPID is the process for conducting risk analysis 
to support DHS risk management tradeoffs. RAPID has three key 
deliverables: (1) a quantitative multihazard homeland security risk 
baseline (i.e. annualized expected loss across a range of terrorism, 
transnational crime, and natural hazard events), (2) a map of major 
DHS programs to homeland security hazards that shows how programs 
interact to manage the risk of a specific hazard, and (3) a program-
based risk reduction analysis that shows the risk reduction of DHS 
individual programs. One of RAPID's objectives is to compare homeland 
security risks within and across incident types to support 
prioritization of DHS's efforts. Risk information for 80 percent of 
DHS's fiscal year 2012 budget and approximately 50 percent of DHS 
programs is available via RAPID, according to DHS documents; 
Tool limitations: DHS officials stated that RAPID was not developed to 
assess risks by QHSR mission area, but that it can be applied to some 
QHSR mission areas more effectively than others. In addition, RAPID is 
not a tool for evaluating the entire QHSR, by comparing risks reduced 
for each mission objective. For example, within QHSR mission 1, 
preventing terrorism, RAPID has extensive data on terrorism risks 
reduced by DHS programs. Within mission 1, these data could be used to 
help make prioritization decisions based on the relative amount of 
risk reduced by various DHS antiterrorism programs. However, RAPID is 
challenged, according to the officials, in making such comparisons 
across mission areas where risks may overlap and RAPID data may be 
limited. For example, mission 1, preventing terrorism, and mission 2, 
securing our borders, address many of the same risks, which makes 
prioritization using risk reduction information inappropriate, 
according to the officials. 

DHS risk assessment tool: HSNRA; 
Description of tool: The HSNRA is designed to provide risk information 
to enable prioritization of required DHS capabilities across all 
hazards, according to the HSNRA proposal; 
Tool limitations: DHS officials stated that for prioritization 
purposes, in future QHSRs DHS would have to agree on what unit of 
analysis to use when comparing risks addressed by QHSR mission areas. 
The current QHSR missions are too broad and risks addressed too 
interdependent for effective risk-based prioritization across missions 
using the HSNRA, according to the officials. However, at the goal and 
objective level, risks addressed become less interdependent and more 
easily compared based on risks addressed. 

DHS component risk assessment tools: 

DHS risk assessment tool: Regional Resiliency Assessment Projects; 
Description of tool: These projects are risk-based assessments of the 
resiliency of clusters of critical infrastructure; 
Tool limitations: According to RMA officials, these risk assessment 
could not be aggregated to inform prioritization across mission areas 
because the tools were not designed for that purpose. However, the 
tools could determine within mission areas which objectives may be 
more important than others, in terms of the risks addressed. The 
officials provided the example of a risk assessment that identifies 
which communities are more vulnerable to natural hazards than others, 
which could help prioritize QHSR mission areas that have objectives 
related to natural hazard resiliency; 
Tool limitations: same as above. 

DHS risk assessment tool: Biological Threat Risk Assessment; 
Description of tool: This is a computationally intensive, 
probabilistic event-tree model for assessing bioterrorism risks; 
Tool limitations: same as above. 

DHS risk assessment tool: Chemical Threat Risk Assessment; 
Description of tool: This is a computationally intensive, 
probabilistic event-tree model for assessing chemical terrorism risks; 
Tool limitations: same as above. 

DHS risk assessment tool: Integrated Chemical-Biological-Radiological- 
Nuclear Risk Assessment; 
Description of tool: This assessment is a computationally intensive, 
probabilistic event-tree model for developing an integrated assessment 
of the risk of terrorist attacks using biological, chemical, 
radiological, or nuclear weapons; 
Tool limitations: same as above. 

DHS risk assessment tool: National Maritime Strategic Risk Assessment; 
Description of tool: This process is used by the U.S. Coast Guard to 
identify risks to achieving its performance goals and identifying 
mitigation options; 
Tool limitations: same as above. 

Source: GAO analysis of DHS information and the National Research 
Council of the National Academies, Review of the Department of 
Homeland Security's Approach to Risk Analysis (Washington, DC: 2010). 

[End of table] 

DHS officials stated that there are benefits to considering risk 
information in resource allocation decisions; however, DHS has not yet 
examined the extent to which risk information could be used when 
implementing subsequent QHSRs. Consideration of risk information could 
help strengthen DHS's prioritization of mechanisms for implementing 
the QHSR, including determining which initiatives or programs should 
be implemented in the short or longer term and the resources required 
for implementation. Such information could also help the department to 
more effectively make decisions about implementing initiatives and 
allocating resources across initiatives that address different levels 
and types of risks. 

DHS Has Developed Plans and Scorecards for Managing and Monitoring 
Implementation of BUR Initiatives: 

DHS is managing and monitoring its implementation of BUR initiatives 
primarily through its budget development and execution process, called 
the Planning, Programming, Budgeting, and Execution (PPBE) process. 
[Footnote 45] The objective of the PPBE process is to articulate DHS's 
goals, objectives, and priorities; align DHS programs with those 
goals; guide the development of the department's budget request; and 
set guidelines for implementing the current budget. To manage 
implementation of the BUR initiatives, beginning with the fiscal year 
2012 budget request, DHS officials told us that DHS developed 
implementation plans for each of the 43 BUR initiatives during the 
planning phase of the PPBE process (see figure 4). DHS assigned a 
directorate, component, or office to lead departmentwide 
implementation efforts for each initiative, including developing the 
implementation plans. According to DHS, each implementation plan 
included what needs to be done to accomplish the BUR initiative, what 
is currently being done to address identified implementation problems, 
a description of stakeholders involved in the implementation effort, 
and a discussion of next steps. DHS initiative leads submitted BUR 
implementation plans to the department for review and discussion 
during the fiscal year 2012 budget development process and the fiscal 
years 2012-2016 budget review process, and these plans served as a 
basis for components to develop their Resource Allocation Plans (RAP)--
components' descriptions of funding needs for fiscal year 2012. 
[Footnote 46] 

Figure 4: Example of BUR Implementation Linkage to QHSR Missions and 
DHS Programs and Activities: 

[Refer to PDF for image: illustration, including 6 photographs and 
accompanying data] 

BUR implementation plans developed by DHS components (input into 
Resource Allocation Plans): 

Fiscal year 2012 PPBE process: 

Planning: 
Integrated Planning Guidance (IPG)(BUR replaced FY 2012 IPG): 
Key activities: DHS is to establish 10+ year planning horizon to set 
overarching direction and program priorities, including strategic 
goals and objectives, projected operating environment and threat 
assessment, mission needs and program priorities, risk analysis, and 
target capabilities. 
Outputs: The Quadrennial Homeland Security Review, Homeland Security 
Threat Assessment, and other documents used to develop the annual DHS 
Integrated Planning Guidance, which is the final output of the 
planning phase and provides guidance to subsequent PPBE phases. 

Programming: 
Resource Allocation Plans and FY 2012-2016 Future Years Homeland 
Security Program; 
Key activities: DHS is to translate planning priorities into 5-year 
resource and performance plan (the Future Years Homeland Security 
Program), and allocate limited resources to best meet the prioritized 
needs. 
Outputs: Secretary’s Resource Allocation Decisions, DHS Future Years 
Homeland Security Program. 

Budgeting: 
Issuance of Resource Allocation Decisions and Budget Guidance; 
Key activities: DHS is to develop detailed budget estimates of 
approved resource plans for budget year justification and 
presentation, and work with the Office of Management and Budget and 
Congress to get a budget enacted. 
Outputs: Budget request sent to the Office of Management and Budget 
and Congress’ briefing and information sent to Congress. 

Execution: 
All DHS components execute missions, spend resources, and report on 
performance; 
Key activities: DHS Chief Financial Officer monitors accountability 
and execution of budget authority and reports results and makes 
recommendations on realigning resources. Funds are apportioned to the 
directorates and components in accordance with apportionment 
guidelines. 
Outputs: Monthly Budget Execution Reports, Midyear Review, 
congressionally directed reports, Annual Financial Report, and Annual 
Performance Report. 

Fiscal year 2012 PPBE process: 

Planning: 
Integrated Planning Guidance; 
DHS developed BUR initiative monitoring scorecards as a part of the FY 
2013 Integrated Planning Guidance. 

Source: GAO analysis of DHS documents. 

Notes: DHS officials stated that programs and activities can support 
multiple mission areas. DHS components listed in figure 2 are U.S. 
Customs and Border Protection (CBP), ICE, the U.S. Coast Guard (CG), 
and the Transportation Security Administration (TSA). 

[End of figure] 

DHS plans to implement the BUR initiatives primarily through 
components' existing programs and activities. For example, DHS plans 
to implement the Strengthen Aviation Security BUR initiative through 
its existing aviation security programs, such as checking airline 
passengers against watchlists and screening passengers at airports. 
Through the PPBE process for fiscal year 2012, DHS requested 
additional funding for select BUR initiatives, above base funding for 
programs and activities that support those initiatives.[Footnote 47] 
For example, under the Improve Detention and Removal Process BUR 
initiative, DHS requested about $222 million in increased funding to 
support its existing Secure Communities program and effort to 
rightsize detention bed space.[Footnote 48] In addition to increased 
funding requested for select BUR initiatives, according to DHS, the 
department planned to fund existing programs and activities that 
support the other BUR initiatives through its base funding. For 
example, the Domestic Nuclear Detection Office stated that it plans to 
fund its programs that support the BUR initiative to increase efforts 
to detect and counter nuclear and biological weapons and dangerous 
materials through its base funding. 

To monitor implementation of the BUR initiatives, DHS established 
scorecards as part of its Integrated Planning Guidance--which DHS 
developed during the planning phase of the fiscal year 2013 PPBE 
process to provide guidance to DHS components for the programming and 
budgeting phases.[Footnote 49] The scorecards depict the status of 
implementing BUR initiatives, including, among other things, whether 
DHS requested funding for BUR initiatives in fiscal year 2012 or plans 
to request funding in future years. The scorecards also allow DHS to 
periodically assess progress made on implementing individual BUR 
initiatives and the status of BUR implementation as a whole. For those 
BUR initiatives for which the department did not identify specific 
funding needs in future years, DHS officials told us that they have 
discussions with DHS components and directorates during midyear budget 
review meetings to discuss progress made toward implementing BUR 
initiatives. In addition, DHS officials told us that because the BUR 
initiatives reflect existing DHS priorities, the initiatives are 
monitored through the Secretary of Homeland Security's discussions 
with component and directorate leadership, such as discussions on 
progress being made on a particular BUR initiative like strengthening 
aviation security. 

DHS Has Undertaken Efforts to Link Its Performance Measures to QHSR 
Mission Areas: 

DHS has taken action to develop and strengthen its performance 
measures, including linking them to QHSR missions and goals and 
ensuring limited overlap among measures. While DHS has not developed 
performance measures for all QHSR missions, goals, and objectives, DHS 
has efforts under way to develop measures to address those missions, 
goals, and objectives. Our prior work on key practices for performance 
measurement has shown that measuring performance allows organizations 
to track the progress they are making toward their goals and gives 
managers critical information on which to base decisions for improving 
their performance.[Footnote 50] We also have previously reported on 
attributes of successful performance measures that include ensuring 
that measures are linked to agencies' missions and goals.[Footnote 51] 

Since issuance of the QHSR report, DHS has undertaken efforts to 
develop new performance measures and link its existing measures to the 
QHSR missions and goals. These efforts included DHS providing guidance 
to components that outlines how to assess QHSR missions, goals, and 
objectives and achievement of QHSR outcomes. DHS also provided 
components with performance measure development training and formed 
working groups to discuss performance measurement best practices. To 
support these efforts, in 2010, we provided technical assistance to 
DHS and its components as they developed and revised their performance 
measures to align with the strategic missions and goals of the QHSR. 
[Footnote 52] Our feedback ranged from pointing out components' 
limited use of outcome-oriented performance measures to assess the 
results or effectiveness of programs to raising questions about the 
steps taken by DHS or its components to ensure the reliability and 
verification of performance data.[Footnote 53] While we offered advice 
on best practices for performance measurement and developing outcome- 
oriented measures, we did not suggest specific performance measures or 
targets or recommend methodologies for collecting, analyzing, and 
reporting performance measure data. Therefore, there was no 
expectation that we and DHS reached agreement on the performance 
measures, and thus decisions related to performance measures were 
fundamentally an executive branch management responsibility. In 
response to this feedback and its internal review efforts, DHS took 
action to develop and revise its performance goals and measures to 
strengthen its ability to assess its outcomes and progress in key 
mission areas. In DHS's fiscal years 2010-2012 Annual Performance 
Report, DHS identified 57 new performance measures for fiscal year 
2011, retained 28 measures from the fiscal year 2010 measure set, and 
is in the process of refining the methodologies for additional 
measures that the department plans to implement in fiscal year 2012. 
[Footnote 54] 

DHS's actions to strengthen its performance measures have helped the 
department link its measures to QHSR missions, goals, and objectives. 
DHS has not yet developed performance measures for all of the QHSR 
goals and objectives but has plans to do so. Specifically, DHS has 
established new performance measures, or linked existing measures, to 
13 of 14 QHSR goals, and to 3 of the 4 goals for the sixth category of 
DHS activities--Providing Essential Support to National and Economic 
Security. DHS reported these measures in its fiscal years 2010-2012 
Annual Performance Report. At the time of issuance of that report, DHS 
had not yet developed performance measures for QHSR Goal 2.3, Disrupt 
and Dismantle Transnational Criminal Organizations, or one of the 
goals for its sixth category of activities--Provide Specialized 
National Defense Capabilities. However, since then, DHS officials told 
us that the department has developed performance measures for these 
goals and plans to publish them in their budget justification to 
Congress upon approval of the measures by DHS leadership and the 
Office of Management and Budget. Further, within QHSR Goal 4.2, 
Promote Cybersecurity Knowledge and Innovation, DHS has not yet 
developed measures for two of the three objectives--foster a dynamic 
workforce and invest in innovative technologies, techniques, and 
procedures. DHS officials told us that the department is collaborating 
with the Office of Personnel Management on a multiyear effort to 
identify competencies and more accurately gauge workforce needs for 
cybersecurity professionals and is working to develop a measure 
related to innovative technologies that have been developed and 
deployed. 

Conclusions: 

Homeland security includes a vast range of mission areas--from 
preventing terrorism to securing U.S. borders, safeguarding 
cyberspace, and ensuring resilience to disasters. It also involves a 
wide variety of stakeholders and partners, including federal 
departments and agencies; state, local, and tribal governments; and 
nongovernmental entities, including the private sector. Given the 
scope and magnitude of the homeland security enterprise, it is 
important for the federal government to set clear goals, objectives, 
and priorities for securing the United States and making resource 
allocation decisions. DHS's 2010 QHSR--the department's first 
quadrennial review--was a massive undertaking to review the nation's 
homeland security strategy and identify homeland security missions and 
organizational objectives. It involved the input of numerous 
stakeholders with homeland security roles and responsibilities, 
including other federal agencies, state and local government entities, 
and academics. DHS plans to initiate its next QHSR in fiscal year 2013 
and to report on that review's results in fiscal year 2014. In 
conducting this next review, DHS could leverage lessons learned from 
the 2010 QHSR to strengthen its planning and risk management efforts. 
Specifically, given the array of federal and nonfederal stakeholders 
involved in implementing homeland security missions, building more 
time for obtaining stakeholders' feedback and input and examining 
additional mechanisms to obtain nonfederal stakeholders' input could 
strengthen DHS's planning and management of stakeholder consultations 
and better position it to obtain, review, and incorporate, as 
appropriate, stakeholders' feedback. 

Risk assessment in the homeland security realm is an evolving field, 
although DHS has developed methodologies, human capital, and 
departmental policies for integrating risk information into DHS 
decision-making processes. Such information can help decision makers 
identify and assess homeland security threats and vulnerabilities 
facing the nation and evaluate strategies for mitigating or addressing 
those threats and vulnerabilities. Using existing risk assessment 
tools could assist DHS in prioritizing QHSR implementation mechanisms. 
Specifically, examining the extent to which risk information could be 
used to help prioritize implementation mechanisms for the next QHSR 
could help DHS determine how to incorporate and use such information 
to strengthen prioritization and resource allocation decisions. 

Recommendations for Executive Action: 

To strengthen DHS's planning, management, and execution of the next 
QHSR, we recommend that the DHS Assistant Secretary for Policy take 
the following three actions: 

* Provide more time for consulting with stakeholders during the QHSR 
process to help ensure that stakeholders are provided the time needed 
to review QHSR documents and provide input into the review, and build 
this time into the department's project planning for the next QHSR. 

* Examine additional mechanisms for obtaining input from nonfederal 
stakeholders during the QHSR process, such as whether panels of state, 
local, and tribal government officials or components' existing 
advisory or other groups could be useful, and use them for obtaining 
nonfederal stakeholders' input, as appropriate, during the next QHSR. 

* Examine the extent to which risk information could be used as one 
input to prioritize QHSR implementing mechanisms, including reviewing 
the extent to which the mechanisms could include characteristics, such 
as defined outcomes, to allow for comparisons of the risks addressed 
by each mechanism. To the extent that DHS determines that risk 
information could be used, consider such information as one input into 
the decision-making process for prioritizing the QHSR implementation 
mechanisms. 

Agency Comments: 

We requested comments on a draft of this report from DHS. On September 
12, 2011, DHS provided written comments, which are reprinted in 
appendix III. DHS concurred with our three recommendations and 
described actions planned to address them. With regard to our first 
recommendation that DHS provide more time for consulting with 
stakeholders during the QHSR process and to build this time into the 
department's project planning for the next QHSR, DHS stated that it 
would endeavor to incorporate increased opportunities and time for 
stakeholder engagement during the next QHSR. Regarding our second 
recommendation that DHS examine additional mechanisms for obtaining 
input from nonfederal stakeholders during the QHSR process and use 
them for obtaining nonfederal stakeholders' input, DHS stated that it 
will examine using panels of state, local, and tribal government 
officials and existing advisory groups to obtain input. With regard to 
our third recommendation that DHS examine the extent to which risk 
information could be used as one input into prioritizing QHSR 
implementing mechanisms and to consider such information, if 
appropriate, when prioritizing QHSR implementation, DHS stated that it 
intends to conduct risk analysis specific to the QHSR in advance of 
the next review. DHS stated that it plans to consider the results of 
such analysis, along with other factors, as an input into decision-
making related to QHSR implementation. DHS also provided technical 
comments which we incorporated as appropriate. 

We also requested comments on a draft of this report from the 
Departments of Agriculture, Defense, Health and Human Services, State, 
the Treasury, and Justice and the Office of the Director of National 
Intelligence. The Department of Defense provided technical comments 
which we incorporated as appropriate. In e-mails received from 
departmental liaisons September 7, 2011, the Departments of 
Agriculture, State, and Justice indicated that they had no comments on 
the report. In e-mails received September 7, 2011 from Treasury's 
Director for Emergency Programs and Health and Human Service's Office 
of the Assistant Secretary for Legislation, the Treasury and Health 
and Human Services indicated that they had no comments on the report. 
In an email received September 9, 2011 from a departmental liaison, 
the Office of the Director of National Intelligence indicated that it 
had no comments on the report. 

We are sending copies of this report to the Secretaries of 
Agriculture, Defense, Health and Human Services, Homeland Security, 
State, and the Treasury; the Attorney General; the Director of 
National Intelligence; and selected congressional committees. This 
report is also available at no charge on the GAO website at 
[hyperlink, http://www.gao.gov]. 

If you or your staff have any questions, please contact me at (202) 
512-9627 or maurerd@gao.gov. Key contributors to this report are 
listed in appendix IV. 

Signed by: 

David C. Maurer: 
Director, Homeland Security and Justice: 

[End of section] 

Appendix I: DHS Strategic Documents and the National Security Strategy 
Align with the QHSR: 

The Department of Homeland Security's (DHS) strategic documents, such 
as component strategic plans and budget requests, align with the 
Quadrennial Homeland Security Review (QHSR) missions. The May 2010 
National Security Strategy (NSS) also identifies strategic elements 
related to homeland security that are identified in the QHSR report, 
such as similar listings of homeland security threats. 

Each of the DHS strategic documents we reviewed includes language 
explicitly aligning at least some aspects of the strategy with the 
QHSR report, as shown in table 5.[Footnote 55] According to DHS 
officials, DHS does not have an explicit policy that strategic 
documents, such as component strategic plans, be consistent with the 
missions, goals, and objectives listed in the QHSR report. However, 
such consistency is expected by DHS senior management, according to 
the officials. We also identified 17 references to homeland security 
within the NSS that relate to DHS responsibilities.[Footnote 56] While 
not explicitly linked to the QHSR report in the NSS document, each of 
the 17 statements link to aspects of the QHSR report, such as homeland 
security threats identified or specific QHSR goals or objectives. 

Table 5: Alignment between QHSR Mission Areas and DHS Strategic 
Documents and the NSS: 

Strategic document: DHS fiscal years 2010-2012 Annual Performance 
Report[A]; 
Description of alignment with the QHSR: Performance measures were 
aligned with the QHSR missions and goals, with suites of measures 
identified for the QHSR goals; 
Examples of alignment: QHSR mission 2, goal 2.2--Safeguard Lawful 
Trade and Travel--has performance measures listed, such as the 
following: 
* Percentage of maritime facilities in compliance with security 
regulations as they have not received a notice of violation, civil 
penalty, or both; 
* Percentage of air carriers operating flights from foreign airports 
that serve as last point of departure to the United States in 
compliance with leading security indicators; 
* Percentage of imports compliant with applicable U.S. trade laws. 

Strategic document: DHS fiscal year 2012 Budget-in-Brief[B]; 
Description of alignment with the QHSR: Funding requests, 
accomplishments, and reforms were listed for the six QHSR missions as 
well as for the maturing and strengthening of the homeland security 
enterprise strategy described in the QHSR report. In the strategic 
context documents submitted to Congress for each DHS component as part 
of the fiscal year 2012 budget request, components listed resource 
requests as they relate to QHSR mission areas; 
Examples of alignment: QHSR mission 1--Preventing Terrorism and 
Enhancing Security--has the following budget elements listed: 
* Funding request examples: explosive detection systems, enhanced 
watchlist vetting, canine teams, and state and local law enforcement 
training; 
* Accomplishment and reform examples: screening 100 percent of 
passengers on flights from, within, or bound for the United States 
against government terrorist watchlists through the Secure Flight 
program and new enhanced security measures for all air carriers with 
international flights to the United States to strengthen the safety 
and security of all passengers; 
* Strategic context for component-level budget requests: U.S. Customs 
and Border Protection, for example, requested funding for programs 
that support mission 1, such as Border Security Inspections and Trade 
Facilitation and Air and Marine Interdiction. National Protection and 
Programs Directorate requested funding for mission 1 programs, such as 
Infrastructure Protection, Federal Protective Service, and 
Cybersecurity and Communications. 

Strategic document: DHS Future Years Homeland Security Program 
(FYHSP), fiscal years 2012-2016[C]; 
Description of alignment with the QHSR: Fiscal year 2010 
accomplishments and reforms are listed by QHSR mission. Funding 
requests are specified for selected DHS Bottom-Up Review (BUR) 
initiatives that implement QHSR mission areas; 
Examples of alignment: For QHSR mission 4, Safeguarding and Securing 
Cyberspace, below are examples of accomplishments/reforms and BUR 
initiative funding requests listed in the FYHSP: 
* Accomplishments/reforms. Memorandum of agreement to align and 
enhance the United States' capabilities to protect against threats to 
critical civilian and military computer systems and networks. The 
agreement embeds Department of Defense (DOD) cyberanalysts within DHS 
and sends DHS privacy, civil liberties, and legal personnel to DOD's 
National Security Agency; 
* Funding to support BUR initiative--Strengthen DHS's ability to 
protect cyber networks: 
- The fiscal year 2012 budget request includes $40.9 million to 
support the department's efforts to strengthen federal network 
security of large and small agencies by conducting an estimated 66 
network assessments to improve security across the federal executive 
branch; 
- Additionally, $24.5 million is requested to provide high-quality, 
cost-effective virtual cybersecurity education and training to develop 
and grow a robust cybersecurity workforce that is able to protect 
against and respond to national cybersecurity threats and hazards; 
- The fiscal year 2012 request also includes $1.3 million to enable 
DHS to coordinate national cybersecurity operations and interface with 
DOD's National Security Agency at Fort Meade, Maryland. 

Strategic document: U.S. Immigration and Customs Enforcement (ICE) 
Strategic Plan, fiscal year 2010-2014[D]; 
Description of alignment with the QHSR: ICE's strategic goals were 
explicitly aligned with QHSR missions1 through 3; 
Examples of alignment: For QHSR mission 3--Enforcing and Administering 
our Immigration Laws--six ICE strategic goals are listed: 
* Detain and removing aliens seeking illegal entry; 
* Create a culture of employer compliance; 
* Prosecute and remove criminals and gang members; 
* Protect the integrity of the immigration system; 
* Achieve efficiency in the removal process; 
* Reform the detention system to meet the needs of ICE. 

Strategic document: Federal Emergency Management Agency (FEMA) 
Strategic Plan fiscal years 2011-2014[E]; 
Description of alignment with the QHSR: Linkage between FEMA's 
strategic plan elements and the QHSR was through statements indicating 
that FEMA's strategy was informed by the QHSR. Our comparison of 
FEMA's strategic plan with the QHSR report determined that FEMA's 
strategic plan elements are consistent with QHSR objectives; 
Examples of alignment: According to FEMA's strategic plan, it "links 
to, and supports, the President's National Security Strategy (NSS), 
the QHSR, DHS missions, and the priorities expressed in the FEMA 
Administrator's Intent." Examples of linkage include the following: 
* FEMA's initiative to "Build the Nation's capacity to stabilize and 
recover from a catastrophic event" reflects the QHSR goal 5.4--Improve 
the nation's ability to adapt and rapidly recover--which includes 
objectives for establishing and maintaining nationwide capabilities 
for recovery from major disasters; 
* FEMA's initiative to "Enhance FEMA's ability to learn and innovate 
as an organization" reflects the QHSR objective for enhancing DHS 
systems for training and evaluating capabilities using, among other 
things, simulated event exercises to improve capabilities. 

Strategic document: National Security Strategy (NSS); 
Description of alignment with the QHSR: Narrative in the NSS regarding 
homeland security strategic elements is reflected in the QHSR reports; 
Examples of alignment: Examples of NSS statements and linkages to the 
QHSR report include the following: 
* The NSS states that "To improve our preparedness, we are integrating 
domestic all hazards planning at all levels of government and building 
key capabilities to respond to emergencies." Similarly, QHSR goal 5.1--
Strengthen capacity at all levels of society to withstand threats and 
hazards--describes improving community capacities for withstanding and 
reducing the consequences of disasters; 
* The NSS states that "We will emphasize individual and community 
preparedness and resilience through frequent engagement that provides 
clear and reliable risk and emergency information to the public." 
Similarly, QHSR goal 5.3--Ensure Effective Emergency Response--
includes the goal to provide timely and accurate information to the 
public. 

Source: GAO analysis of DHS documents and the NSS. 

[A] DHS, U.S. Department of Homeland Security Annual Performance 
Report, Fiscal Years 2010-2012 (Washington, D.C.: Feb. 14, 2011). 

[B] DHS, Budget-in-Brief Fiscal Year 2012 (Washington, D.C.: Feb. 14, 
2011): 

[C] DHS, Future Years Homeland Security Program, Fiscal Years 2012-16 
(Washington, D.C.: May 2011). 

[D] U.S. Immigration and Customs Enforcement. U.S. Immigration and 
Customs Enforcement Strategic Plan, FY 2010-2014 (Washington, D.C.). 

[E] Federal Emergency Management Agency, FEMA Strategic Plan Fiscal 
Years 2011-2014 (Washington, D.C.: February 2011). 

[End of table] 

[End of section] 

Appendix II: Scope and Methodology: 

The objectives for this report were to evaluate the extent to which 
the Department of Homeland Security (DHS) (1) consulted with 
stakeholders in developing the Quadrennial Homeland Security Review 
(QHSR) strategy; (2) conducted a national risk assessment to develop 
the QHSR; and (3) developed priorities, plans, monitoring mechanisms, 
and performance measures for implementing the QHSR and Bottom-Up 
Review (BUR) initiatives. 

To address our objectives, we analyzed DHS documents related to the 
QHSR, BUR, and budget development processes, including the QHSR 
report, BUR report, fiscal year 2012 budget request, and Fiscal Years 
2012-2016 Future Years Homeland Security Program. We identified 
criteria for evaluating these processes by analyzing our prior reports 
on key characteristics of effective national strategies, key practices 
for effective interagency collaboration, strategic planning, 
performance measurement, and standards for internal control, among 
others. For a listing of these prior reports, see the related products 
listed at the end of this report. Based on these reports, we 
identified those key practices and characteristics applicable to 
quadrennial reviews, like the QHSR. The key practices we identified 
were involving stakeholders in defining QHSR missions and outcomes; 
defining homeland security problems and assessing risks; including 
homeland security strategy goals, subordinate objectives, activities, 
and performance measures; including resources, investments, and risk 
management; including organizational roles, responsibilities, and 
coordination across the homeland security enterprise; and establishing 
a DHS process for managing implementation of BUR initiatives. We 
vetted the key practices with our subject matter experts--staff with 
legal and methodological expertise and experience analyzing the 
Quadrennial Defense Review--and provided them to DHS officials for 
review and incorporated their comments as appropriate.[Footnote 57] As 
we developed our report, we grouped these key practices into three 
areas--stakeholder involvement, risk assessment, and implementation 
processes for the QHSR and BUR initiatives. 

To determine the extent to which DHS consulted with stakeholders in 
developing the QHSR, we requested comments on the QHSR process from 79 
QHSR stakeholders identified by DHS.[Footnote 58] The stakeholders 
solicited by us for comments included 22 federal departments and 
agencies; 10 state, local, and tribal organizations; 28 DHS 
components, directorates, and offices; and 19 Quadrennial Review 
Advisory Committee (QRAC) members.[Footnote 59] We also solicited 
comments from 6 subject matter experts hired by DHS to facilitate QHSR 
study groups.[Footnote 60] We received comments from 63 of the 85 
stakeholders and study group facilitators we contacted (74 percent), 
including 21 of 22 federal departments; 6 of 10 state, local, and 
tribal organizations; 26 of the 28 DHS components, directorates, and 
offices; 7 of the 19 QRAC members; and 3 of the 6 study group 
facilitators. We asked open-ended questions regarding the QHSR 
stakeholder consultation process, such as suggestions for improving 
future QHSRs, examples of positive ways DHS involved stakeholders, and 
involvement in determining agency roles and responsibilities listed in 
the QHSR report. We relied on respondents to raise and comment on 
their views of the QHSR process; therefore we could not determine 
whether respondents shared similar views or identified similar 
benefits or challenges to the QHSR process unless respondents 
identified them in their responses to our requests for comments. 
[Footnote 61] We analyzed the comments provided by the 63 respondents 
to determine common benefits and challenges they identified regarding 
DHS consultations during the QHSR. We also conducted follow-up 
interviews with 14 QHSR stakeholders that we selected based on their 
responses, to obtain clarification of their responses to our requests 
for comments. The comments received from these respondents are not 
generalizable to the entire group of stakeholders, but the feedback 
provided insights into stakeholder perspectives on how QHSR 
stakeholder consultations were conducted and how they could be 
improved. Further, we reviewed reports on the QHSR by the National 
Academy of Public Administration (NAPA) and the QRAC, both of which 
were based upon each organization's collaboration experiences with DHS 
in developing the QHSR report.[Footnote 62] During the QHSR, NAPA 
partnered with DHS to conduct three National Dialogues, which allowed 
any member of the public to review draft QHSR material and provide 
online suggestions for the QHSR. According to the QRAC's report, the 
QRAC served as a forum in which committee members, who were nonfederal 
representatives, shared independent advice with DHS on the QHSR 
process. We compared DHS's stakeholder consultation efforts to our 
prior work on effective practices for collaboration and consultation. 
For example, based on a key practice in federal agency collaboration, 
we analyzed the extent to which DHS worked with stakeholders to 
establish agency roles and responsibilities when developing the QHSR. 
[Footnote 63] 

To determine the extent to which DHS conducted a national risk 
assessment to develop the QHSR, we analyzed risk analysis-related 
documents produced as part of the QHSR process, such as DHS risk 
assessment tools, and interviewed DHS officials responsible for 
developing risk analyses for use at DHS. We compared DHS's risk 
assessment process in the QHSR to our prior work on key 
characteristics for risk assessment as well as DHS risk analysis 
guidance documents. For example, we reviewed our previous reports on 
key practices in risk management, including risk assessment 
approaches, and compared them to DHS's effort to develop a national 
risk assessment methodology.[Footnote 64] In addition, we reviewed DHS 
guidance for use of risk assessment information and compared the 
guidance with DHS's QHSR risk assessment process. 

To determine the extent to which DHS developed priorities, 
implementation plans, monitoring mechanisms, and performance measures, 
we analyzed DHS's BUR implementation priorities and plans, such as 
DHS's fiscal year 2012 budget request; monitoring mechanisms, such as 
BUR initiative scorecards; and DHS's performance measures. We also 
interviewed DHS officials responsible for managing and monitoring the 
implementation of the BUR initiatives. We compared DHS's processes for 
prioritizing, monitoring, and measuring implementation efforts to our 
prior work on key practices for risk management and implementation and 
monitoring of strategic initiatives. For example, we identified 
practices in our past reports and DHS guidance for using risk 
information in resource prioritization decisions and compared DHS's 
efforts to prioritize and implement the QHSR strategy with those 
practices. We also compared DHS's strategic-level performance measures 
for fiscal year 2011 to our criteria on key attributes of successful 
performance measures. Because DHS focused on aligning its performance 
measures with QHSR missions, we selected three key attributes of 
successful performance measures that were most relevant--linkage, core 
program activity, and limited overlap.[Footnote 65] In applying the 
attributes, we analyzed documentation, such as the QHSR report and 
DHS's fiscal years 2010-2012 Annual Performance Report. We also 
interviewed DHS officials who are involved in overseeing the 
development and reporting of DHS performance measures. 

We conducted this performance audit from January 2011 through 
September 2011 in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

[End of section] 

Appendix III: Comments from the Department of Homeland Security: 

U.S. Deportment of Homeland Security: 
Washington, DC 20528: 

September 12, 2011: 

Mr. David Maurer: 
Director, Homeland Security and Justice: 
U.S. Government Accountability Office: 
441 G Street N.W. 
Washington, D.C. 20548: 

Re: Draft Report GA0-11-873, "Quadrennial Homeland Security Review
Enhanced Stakeholder Consultation and Use of Risk Information Could 
Strengthen Future Reviews" 

Dear Mr. Maurer: 

Thank you for the opportunity to review and comment on the draft 
report. The U.S. Department of Homeland Security (DHS) appreciates the 
U.S. Government Accountability Office's (GAO's) work in planning and 
conducting its review and issuing this report. 

The Department is pleased to note GAO's positive acknowledgment of 
DHS's efforts in issuing its first Quadrennial Homeland Security 
Review (QHSR)[Footnote 1] and recognition that this was a "massive" 
undertaking to review the nation's homeland security strategy and 
identify homeland security missions and organizational objectives. The 
draft report contains three recommendations, with which DHS concurs. 
Specifically, GAO recommended that the DHS Assistant Secretary for 
Policy: 

Recommendation 1: Provide more time for consulting with stakeholders 
during the QHSR process to help ensure that stakeholders are provided 
the time needed to review QHSR documents and provide input into the 
review, and build this time and into the department's project planning 
for the next QHSR. 

Response: Concur. While there was an unprecedented level of 
stakeholder engagement in the execution of the first QHSR. to include 
innovation in our engagement tools, additional time for consultation 
is an area that DHS internally identified for improvement following the
QHSR. We will endeavor to incorporate increased opportunities and time 
for meaningful stakeholder engagement and input into our next QHSR 
scheduled for FY 2013, the planning of which began last month and will 
continue through FY 2012. 

Recommendation 2: Examine additional mechanisms for obtaining input 
from nonfederal stakeholders during the QHSR process, such as whether 
panels of state, local and tribal government officials or components' 
existing advisory or other groups could be useful, and use them for 
obtaining nonfederal stakeholders' input, as appropriate, during the 
next QHSR. 

Response: Concur. The Department will examine the use of panels of 
state, local, and tribal government officials and existing advisory 
groups to obtain input. DHS will aim for mechanisms that are simple 
and accessible while facilitating meaningful and substantive input 
into QHSR analyses. 

Recommendation 3: Examine the extent to which risk information could 
be used as one input to prioritize QHSR implementing mechanisms or BUR 
initiatives, including reviewing the extent to which the mechanisms or 
initiatives could include characteristics, such as defined outcomes, 
to allow for comparisons of the risks addressed by each initiative. To 
the extent that DHS determines that risk information could be used, 
consider such information as one input into the decision-making 
process for prioritizing the QHSR implementation mechanisms. 

Response: Concur. DHS continues to believe that risk information and 
analysis is integral to homeland security decision-making, including 
the QHSR process. The Department intends to conduct risk analysis 
specific to the QHSR in advance of the next review, including a review 
of the strategic risks facing the Nation. Such analysis will then be 
considered, along with other factors, as an input into decision-making 
related to the implementation of the QHSR. 

Again, thank you for the opportunity to review and comment on this 
draft report. Technical and sensitivity comments were provided under 
separate cover. We look forward to working with you on future Homeland 
Security related issues. 

Sincerely, 

Signed by: 

Jim H. Crumpacker: 
Director: 
Departmental GAO-OIG Liaison Office: 

Footnote: 

[1] DHS, Quadrennial Homeland Security Review Report: A Strategic 
Framework for a Secure Homeland (Washington, D.C.: February 2010), 
[hyperlink, http://www.dhs.gov/xlibrarv/assets/qhsr_report.pdf]. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

David C. Maurer, (202) 512-9627 or maurerd@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Rebecca Gambler, Assistant 
Director, and Ben Atwater, Analyst-in-Charge, managed this assignment. 
Jean Orland and Janay Sam made significant contributions to this work. 
Michele Fejfar assisted with design and methodology, and Tracey King 
provided legal support. Labony Chakraborty, Jessica Orr, and Robert 
Robinson assisted with report preparation. 

[End of section] 

Related GAO Products: 

Defense Transportation: Additional Information Is Needed for DOD's 
Mobility Capabilities and Requirements Study 2016 to Fully Address All 
of Its Study Objectives. [hyperlink, 
http://www.gao.gov/products/GAO-11-82R]. Washington, D.C.: December 8, 
2010. 

Department of Homeland Security: Actions Taken Toward Management 
Integration, but a Comprehensive Strategy Is Still Needed. [hyperlink, 
http://www.gao.gov/products/GAO-10-131]. Washington, D.C.: November 
20, 2009. 

Transportation Security: Comprehensive Risk Assessments and Stronger 
Internal Controls Needed to Help Inform TSA Resource Allocation. 
[hyperlink, http://www.gao.gov/products/GAO-09-492]. Washington, D.C.: 
March 27, 2009. 

Quadrennial Defense Review: Future Reviews Could Benefit from Improved 
Department of Defense Analyses and Changes to Legislative 
Requirements. [hyperlink, http://www.gao.gov/products/GAO-07-709]. 
Washington, D.C.: September 14, 2007. 

Risk Management: Further Refinements Needed to Assess Risks and 
Prioritize Protective Measures at Ports and Other Critical 
Infrastructure. [hyperlink, http://www.gao.gov/products/GAO-06-91]. 
Washington, D.C.: December 15, 2005. 

Results-Oriented Government: Practices That Can Help Enhance and 
Sustain Collaboration among Federal Agencies. [hyperlink, 
http://www.gao.gov/products/GAO-06-15]. Washington, D.C.: October 21, 
2005. 

Results-Oriented Government: Improvements to DHS's Planning Process 
Would Enhance Usefulness and Accountability. [hyperlink, 
http://www.gao.gov/products/GAO-05-300]. Washington, D.C.: March 31, 
2005. 

Combating Terrorism: Evaluation of Selected Characteristics in 
National Strategies Related to Terrorism. [hyperlink, 
http://www.gao.gov/products/GAO-04-408T]. Washington, D.C.: February 
3, 2004. 

Tax Administration: IRS Needs to Further Refine Its Tax Filing Season 
Performance Measures. [hyperlink, 
http://www.gao.gov/products/GAO-03-143]. Washington, D.C.: November 
22, 2002. 

Homeland Security: Proposal for Cabinet Agency Has Merit, But 
Implementation Will be Pivotal to Success. [hyperlink, 
http://www.gao.gov/products/GAO-02-886T]. Washington, D.C.: June 25, 
2002. 

Standards for Internal Control in the Federal Government. [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. Washington, D.C.: 
November 1999. 

Executive Guide: Effectively Implementing the Government Performance 
and Results Act. [hyperlink, 
http://www.gao.gov/products/GAO/GGD-96-118]. Washington, D.C.: June 
1996. 

[End of section] 

Footnotes: 

[1] Pub. L. No. 110-53, § 2401(a), 121 Stat. 266, 543-45 (2007) 
(codified at 6 U.S.C. § 347). 

[2] DHS, Quadrennial Homeland Security Review Report: A Strategic 
Framework for a Secure Homeland (Washington, D.C.: February 2010). 
Although the act requires the first QHSR to be conducted in 2009--see 
6 U.S.C. § 347(c)--the QHSR report was issued in February 2010 and we 
refer to it in this report as the 2010 QHSR. 

[3] In the QHSR report, the term enterprise refers to the collective 
efforts and shared responsibilities of federal, state, local, tribal, 
territorial, nongovernmental, and private sector partners--as well as 
individuals, families, and communities--to maintain critical homeland 
security capabilities. 

[4] DHS, Bottom-Up Review Report (Washington, D.C.: July 2010). 

[5] GAO, Quadrennial Homeland Security Review: 2010 Reports Addressed 
Many Required Elements, but Budget Planning Not Yet Completed, 
[hyperlink, http://www.gao.gov/products/GAO-11-153R] (Washington, 
D.C.: Dec. 16, 2010). 

[6] We considered an element addressed if all portions of it were 
explicitly included in either the QHSR or BUR reports, addressed in 
part if one or more but not all portions of the element were included, 
and not addressed if neither the QHSR nor the BUR reports explicitly 
addressed any part of the element. 

[7] The FYHSP provides a summary and breakdown of DHS program 
resources over a 5-year period, including resource alignment by goals, 
component appropriations, and component programs, as well as program 
descriptions, milestones, performance measures, and targets. 

[8] See the Related GAO products list at the end of this report. 

[9] The Quadrennial Defense Review is a legislatively mandated review 
that articulates the Department of Defense's strategic plan for 
meeting future threats. 

[10] DHS identified a total of 102 QHSR stakeholders, including 11 
individual staff members within White House offices or the National 
Security Staff. We did not request comments on the QHSR from the White 
House and National Security offices or individual staff members 
because we focused our review on DHS's interactions with executive 
branch departments and agencies and state, local, and private sector 
entities. In addition, there were 6 other individuals DHS identified 
for whom we could not obtain contact information; we did not request 
comments on the QHSR from these individuals. 

[11] The QRAC was a subcommittee of the Homeland Security Advisory 
Council established to provide DHS with recommendations during the 
QHSR. 

[12] Convened by DHS as part of the QHSR process, the study groups 
provided analysis that defined the QHSR mission goals and objectives 
and shared results of the analyses with QHSR stakeholders. 

[13] Because respondents volunteered information about their views on 
the QHSR, we do not know the extent to which other officials within 
the same organization shared these views. 

[14] NAPA and the QRAC reported their observations and recommendations 
based on their involvement in the QHSR process. We determined that 
their reports were sufficiently reliable for the purposes of reporting 
their observations on the QHSR process. During the QHSR, NAPA 
partnered with DHS to conduct three National Dialogues, which allowed 
any member of the public to review draft QHSR material and provide 
online suggestions for the QHSR. According to the QRAC's report, the 
QRAC served as a forum in which committee members, all of whom are 
nonfederal representatives, shared independent advice with DHS on the 
QHSR process. See NAPA. The National Dialogue on the Quadrennial 
Homeland Security Review: Panel Report (Washington, D.C.: April 2010) 
and Homeland Security Advisory Council, Quadrennial Review Advisory 
Committee Final Report (May 27, 2010). 

[15] GAO, Tax Administration: IRS Needs to Further Refine Its Tax 
Filing Season Performance Measures, [hyperlink, 
http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22, 
2002). Pub. L. No. 103-62, 107 Stat. 285 (1993). The Government 
Performance and Results Act of 1993 was intended to address several 
broad purposes, including strengthening the confidence of the American 
people in their government; improving federal program effectiveness, 
accountability, and service delivery; and enhancing congressional 
decision making by providing more objective information on program 
performance. 

[16] DHS distributed the draft QHSR terms of reference for internal 
DHS review in May 2009. The final draft QHSR terms of reference was 
distributed to study group lead officials in early June 2009. The 
Secretary of Homeland Security signed the QHSR terms of reference in 
July 2009. 

[17] The Homeland Security Studies and Analysis Institute is a 
federally funded research and development center that advises DHS in 
areas of policy development. 

[18] The stakeholders listed in the 9/11 Commission Act were the 
Attorney General; Secretaries of State, Defense, Health and Human 
Services, the Treasury, and Agriculture; the Director of National 
Intelligence; key officials of the department, and other relevant 
governmental and nongovernmental entities, including state, local, and 
tribal government officials, members of Congress, private sector 
representatives, academics, and other policy experts. 6 U.S.C. § 
347(a)(3). 

[19] The 1,300 DHS activities represent all activities conducted by 
each DHS unit, some of which were the same type of activity performed 
in different areas of DHS, such as business support processes. 
According to DHS officials, there were about 860 distinct activities 
counted in the activities inventory. Business support activities are 
enabling activities providing enterprise business services, such as 
information technology, human resources, and legal counsel. Mission 
support activities provide a product or service for, and tailored to, 
mission or operational activities, such as logistical support, and 
research and development. 

[20] While 43 initiatives are listed in the BUR report, DHS tracks 40 
BUR initiatives because 6 initiatives were consolidated into 3 for 
implementation purposes, according to DHS officials. The initiatives 
were combined as follows: Deliver infrastructure protection and 
resilience capabilities to the field was combined with Explore 
opportunities with the private sector to "design in" greater 
resilience for critical infrastructure; Create a cyber security and 
infrastructure resilience operational component within DHS was 
combined with Align DHS operational activities in order to achieve 
maximum effectiveness; and Increase analytic capability and capacity 
was combined with Improve performance measurement and accountability. 

[21] 6 U.S.C. § 347(a)(3)(A). 

[22] Key strategic outcomes and measurable end states refer to the 
portions of the QHSR report that identify five selected strategic 
outcomes for each mission with key actions for each of the mission 
objectives. 

[23] Throughout this report, when we refer to federal stakeholders, we 
mean non-DHS federal departments and agencies that participated in the 
QHSR process. 

[24] GAO, Executive Guide: Effectively Implementing the Government 
Performance and Results Act, [hyperlink, 
http://www.gao.gov/products/GAO/GGD-96-118] (Washington, D.C.: June 
1996). 

[25] The Project Management Institute, The Standard for Program 
Management (2008). 

[26] [hyperlink, http://www.gao.gov/products/GAO/GGD-96-118]. 

[27] GAO, Critical Infrastructure: Challenges Remain in Protecting Key 
Sectors, [hyperlink, http://www.gao.gov/products/GAO-07-626T] 
(Washington, D.C.: Mar. 20, 2007); Emergency Preparedness: Improved 
Planning and Coordination Necessary for Development of Integrated 
Public Alert and Warning System, [hyperlink, 
http://www.gao.gov/products/GAO-09-1044T] (Washington, D.C.: Sept. 30, 
2009); and Results-Oriented Government: Practices That Can Help 
Enhance and Sustain Collaboration among Federal Agencies, [hyperlink, 
http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21, 
2005). 

[28] [hyperlink, http://www.gao.gov/products/GAO-09-1044T]. 

[29] See 5 U.S.C. App. 2. For example, according to DHS's charter for 
the Homeland Security Advisory Council, of which the QRAC is a 
subcommittee, the Council operates in accordance with the provisions 
of FACA. 

[30] GAO, National Response Framework: FEMA Need Policies and 
Procedures to Better Integrate Non-Federal Stakeholders in the 
Revision Process, [hyperlink, http://www.gao.gov/products/GAO-08-768] 
(Washington, D.C.: June 11, 2008). 

[31] According to the QHSR report, the National Infrastructure 
Protection Plan, the National Response Framework and other homeland 
security documents define roles and responsibilities for many actors 
across the homeland security enterprise and provided a basis for the 
definitions included in the QHSR report. The National Response 
Framework presents the guiding principles that enable all response 
partners to prepare for and provide a unified national response to 
disasters and emergencies. It establishes a comprehensive, national, 
all-hazards approach to domestic incident response. The National 
Infrastructure Protection Plan provides the unifying structure for the 
integration of a wide range of efforts for the enhanced protection and 
resiliency of the nation's critical infrastructure and key resources 
into a single national program. 

[32] [hyperlink, http://www.gao.gov/products/GAO-06-15]. 

[33] GAO, Combating Terrorism: Evaluation of Selected Characteristic 
in National Strategies Related to Terrorism, [hyperlink, 
http://www.gao.gov/products/GAO-04-408T] (Washington, D.C.: Feb. 3, 
2004). 

[34] GAO, Department of Homeland Security: Progress Report on 
Implementation of Mission and Management Functions, [hyperlink, 
http://www.gao.gov/products/GAO-07-454] (Washington, D.C.: Aug. 17, 
2007). 

[35] [hyperlink, http://www.gao.gov/products/GAO-11-153R]. We 
considered a reporting element addressed in part if one or more but 
not all portions of the element were included in the QHSR or BUR 
reports. 

[36] DHS's Risk Management Fundamentals: Homeland Security Risk 
Management Doctrine, published in April 2011, provides more detail on 
the steps used in a risk management framework and supports the concept 
of risk information being used to inform DHS-wide decision-making 
processes. The report also states that homeland security risks can be 
assessed through evaluations of the likelihood and consequences of 
certain homeland security incidents. 

[37] See GAO, Transportation Security: Comprehensive Risk Assessments 
and Stronger Internal Controls Needed to Help Inform TSA Resource 
Allocation, [hyperlink, http://www.gao.gov/products/GAO-09-492] 
(Washington, D.C.: Mar. 27, 2009), and [hyperlink, 
http://www.gao.gov/products/GAO-04-408T]. 

[38] [hyperlink, http://www.gao.gov/products/GAO-09-492]. 

[39] The sufficiency of the HSNRA as a risk assessment methodology was 
not evaluated as part of this review. 

[40] Presidential Policy Directive 8: National Preparedness (Mar. 30, 
2011). This directive calls for development of a national preparedness 
goal informed by the risk of specific threats and vulnerabilities in 
an effort to prepare the nation for threats that pose the greatest 
risk to the security of the nation, including acts of terrorism, cyber 
attacks, pandemics, and catastrophic national disasters. 

[41] National Research Council of the National Academies Review of the 
Department of Homeland Security's Approach to Risk Analysis 
(Washington, D.C.: 2010). 

[42] The Homeland Security Threat Assessment identifies threats that 
could result in incidents of national significance based on analytical 
judgments of DHS's intelligence divisions. The National Planning 
Scenarios, developed by national and homeland security officials, are 
intended to form the basis for identifying the capabilities needed to 
respond to a wide range of homeland security emergencies. The 
scenarios focus on the consequences that federal, state, and local 
first responders will have to address and are intended to illustrate 
the scope and magnitude of large-scale, catastrophic events for which 
the nation needs to be prepared. 

[43] According to DHS officials, risk was not included as an element 
of any of these three criteria. 

[44] See GAO, Risk Management: Further Refinements Needed to Assess 
Risks and Prioritize Protective Measures at Ports and Other Critical 
Infrastructure, [hyperlink, http://www.gao.gov/products/GAO-06-91] 
(Dec. 15, 2005) and [hyperlink, 
http://www.gao.gov/products/GAO-09-492]. 

[45] PPBE guidance is contained in DHS Management Directive 1330. 

[46] According to DHS, the RAP submission process was restructured to 
show resources in a common set of expenditure categories according to 
the activities identified in the BUR. By collecting the data in this 
way, DHS has been able to systematically look at costs associated with 
salaries (people), expenses, investments (planning, acquisition, and 
maintenance), research and development, and assistance payments 
(grants) across components, which DHS has not been able to do in the 
past. 

[47] DHS defines base funding as year-end actual funding from the 
prior year. 

[48] Through the Secure Communities program, ICE aims to leverage 
existing information-sharing capability between DHS and the Department 
of Justice to quickly and accurately identify aliens who are arrested 
for a crime and booked into local law enforcement custody. The 
objective of the rightsize detention bed space initiative is to 
increase ICE's use of larger, strategically located facilities, 
thereby increasing program consistency, improving conditions of 
confinement, and lowering detention costs. 

[49] The Integrated Planning Guidance is to provide guidance to DHS 
components, directorates and offices for the programming and budgetary 
phases of the PPBE process. It is intended to translate national 
homeland security strategy and policy into actionable guidance for 
programming, budgeting, and execution, including investment and 
acquisition. 

[50] [hyperlink, http://www.gao.gov/products/GAO/GGD-96-118]. 

[51] [hyperlink, http://www.gao.gov/products/GAO-03-143]. Linkage 
refers to the extent to which a measure is aligned with division and 
agencywide goals and missions and clearly communicated throughout the 
organization. 

[52] Over the past 2 years, we provided technical assistance to DHS 
and its components on the department's performance measures at the 
request of the Senate Homeland Security and Governmental Affairs 
Committee. Technical assistance we provided was based on published 
work and our subject matter knowledge of DHS, and our knowledge of 
performance measurement guidance and best practices. 

[53] In August and September 2010, we provided feedback on the 
department's proposals for performance measures aligned with the 
QHSR's goals and objectives. 

[54] The Annual Performance Report presents the department's proposed 
performance measures and applicable results, associated performance 
targets, and information on the department's priority goals. 

[55] DHS strategic documents we reviewed were limited to those 
published after the February 2010 release of the QHSR report. 

[56] White House, National Security Strategy (Washington, D.C.: May 
2010). 

[57] The Quadrennial Defense Review is a legislatively mandated review 
that articulates the Department of Defense's strategic plan for 
meeting future threats. 

[58] DHS identified a total of 102 QHSR stakeholders, including 11 
individual staff members within White House offices or the National 
Security Staff. We did not request comments on the QHSR from the White 
House and National Security Staff offices or individual staff members 
because we focused our review on DHS's interactions with executive 
branch department agencies, and state, local, and private sector 
entities. In addition, there were 6 other individuals DHS identified 
for whom we could not obtain contact information; we did not request 
comments on the QHSR from these individuals. 

[59] The QRAC was a subcommittee of the Homeland Security Advisory 
Council established to provide DHS with recommendations during the 
QHSR. 

[60] Convened by DHS as part of the QHSR process, the study groups 
provided analysis that defined the QHSR mission goals and objectives 
and shared results of the analyses with QHSR stakeholders. 

[61] Because respondents volunteered information about their views on 
the QHSR, we do not know the extent to which other officials within 
the same organizations shared these views. 

[62] NAPA, The National Dialogue on the Quadrennial Homeland Security 
Review: Panel Report, and Homeland Security Advisory Council, 
Quadrennial Review Advisory Committee Final Report. We determined that 
the QRAC and NAPA reports were sufficiently reliable for the purpose 
of providing their views on the QHSR process. 

[63] [hyperlink, http://www.gao.gov/products/GAO-06-15]. 

[64] [hyperlink, http://www.gao.gov/products/GAO-09-492] and 
[hyperlink, http://www.gao.gov/products/GAO-06-91]. 

[65] [hyperlink, http://www.gao.gov/products/GAO-03-143]. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: