This is the accessible text file for GAO report number GAO-11-260T 
entitled 'Measuring Disaster Preparedness: FEMA Has Made Limited 
Progress in Assessing National Capabilities' which was released on 
March 17, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Testimony: 

Before the Committee on Homeland Security and Governmental Affairs,
United States Senate: 

For Release on Delivery: 
Expected at 3:15 p.m. EST:
Wednesday, March 17, 2011: 

Measuring Disaster Preparedness: 

FEMA Has Made Limited Progress in Assessing National Capabilities: 

Statement of William O. Jenkins, Jr. Director:
Homeland Security and Justice: 

GAO-11-260T: 

Mr. Chairman and Members of the Committee: 

I appreciate the opportunity to participate in today's hearing and to 
discuss the efforts of the Federal Emergency Management Agency (FEMA)--
a component of the Department of Homeland Security (DHS)--to measure 
and assess national capabilities to respond to a major disaster. 
According to the Congressional Research Service, from fiscal years 
2002 through 2010, Congress appropriated over $34 billion for homeland 
security preparedness grant programs to enhance the capabilities of 
state, territory, local, and tribal governments to prevent, protect 
against, respond to, and recover from terrorist attacks and other 
disasters. Congress enacted the Post-Katrina Emergency Management 
Reform Act of 2006 (Post-Katrina Act) to address shortcomings in the 
preparation for and response to Hurricane Katrina that, among other 
things, gave FEMA responsibility for leading the nation in developing 
a national preparedness system.[Footnote 1] The Post-Katrina Act 
requires that FEMA develop a national preparedness system and assess 
preparedness capabilities--capabilities needed to respond effectively 
to disasters--to determine the nation's preparedness capability levels 
and the resources needed to achieve desired levels of capability. 
Figure 1 provides an illustration of how federal, state, and local 
resources provide capabilities for different levels of "incident 
effect" (i.e., the extent of damage caused by a natural or manmade 
disaster). FEMA's National Preparedness Directorate within its 
Protection and National Preparedness organization is responsible for 
developing and implementing a system for measuring and assessing 
national preparedness capabilities. 

Figure 1: Conceptual Illustration for Assessing Capability 
Requirements and Identifying Capability Gaps for National Preparedness: 

[Refer to PDF for image: illustrated graph] 

The graph plots Level of Incident Effect against Time. 
A line in the graph depicts "Capability Requirements". 
A stacking of Federal, State, and Local resources is depicted, with 
the difference between the available resources and capability 
requirements being indicated as "Capability Gaps." 

Source: GAO. 

[End of figure] 

The need to define measurable national preparedness capabilities is a 
well-established and recognized issue. For example, in December 2003, 
the Advisory Panel to Assess Domestic Response Capabilities noted that 
preparedness (for combating terrorism) requires measurable 
demonstrated capacity by communities, states, and private sector 
entities throughout the United States to respond to threats with well- 
planned, well-coordinated, and effective efforts. This is consistent 
with our April 2002 testimony on national preparedness, in which we 
identified the need for goals and performance indicators to guide the 
nation's preparedness efforts and help to objectively assess the 
results of federal investments.[Footnote 2] We reported that FEMA had 
not yet defined the outcomes of where the nation should be in terms of 
domestic preparedness. Thus, identifying measurable performance 
indicators could help FEMA: 

* track progress toward established goals, 

* provide policy makers with the information they need to make 
rational resource allocations, and: 

* provide program managers with the data needed to effect continual 
improvements, measure progress, and to enforce accountability. 

In September 2007, DHS issued the National Preparedness Guidelines 
that describe a national framework for capabilities-based preparedness 
as a systematic effort that includes sequential steps to first 
determine capability requirements and then assess current capability 
levels. According to the Guidelines, the results of this analysis 
provide a basis to identify, analyze, and choose options to address 
capability gaps and deficiencies, allocate funds, and assess and 
report the results. This proposed framework reflects critical 
practices we have identified for government performance and results. 
[Footnote 3] (See appendix 1 for an illustration of the Guidelines and 
critical practices.) 

My remarks today are based on our prior work issued from July 2005 
through October 2010 on DHS's and FEMA's efforts to develop and 
implement a national framework for assessing preparedness capabilities 
at the federal, state, and local levels, as well as DHS's and FEMA's 
efforts to develop and use metrics to define capability levels, 
identify capability gaps, and prioritize national preparedness 
investments to fill the most critical capability gaps.[Footnote 4] As 
requested, my testimony today focuses on the extent to which DHS and 
FEMA have made progress in measuring national preparedness by 
assessing capabilities and addressing related challenges. To conduct 
our work, we analyzed documentation, such as FEMA's National 
Preparedness Guidelines and Target Capabilities List--a list of 37 
capabilities that federal, state, and local stakeholders need to 
possess to respond to natural or manmade disasters--and interviewed 
relevant DHS, FEMA, state, and local officials. We conducted this work 
in accordance with generally accepted government auditing standards. 
More detailed information on our scope and methodology appears in our 
published work. 

In summary, DHS and FEMA have implemented a number of efforts with the 
goal of measuring preparedness by assessing capabilities and 
addressing related challenges, but success has been limited. DHS first 
developed plans to measure preparedness by assessing capabilities, but 
did not fully implement those plans. FEMA then issued the target 
capabilities list in September 2007 but has made limited progress in 
developing preparedness measures and addressing long-standing 
challenges in assessing capabilities, such as determining how to 
aggregate data from federal, state, local, and tribal governments. At 
the time of our review of FEMA's efforts in 2008 and in 2009, FEMA was 
in the process of refining the target capabilities to make them more 
measurable and to provide state and local jurisdictions with 
additional guidance on the levels of capability they need. We 
recommended in our April 2009 report that FEMA enhance its project 
management plan with, among other things, milestones to help it 
implement its capability assessment efforts; FEMA agreed with our 
recommendation. We reported in October 2010 that FEMA had enhanced its 
plan with milestones in response to our prior recommendation and that 
officials said they had an ongoing effort to develop measures for 
target capabilities--as planning guidance to assist in state and local 
assessments--rather than as requirements for measuring preparedness by 
assessing capabilities; FEMA officials had not yet determined how they 
plan to revise the list. 

FEMA Has Made Limited Progress in Measuring Preparedness by Assessing 
Capabilities and Addressing Long-Standing Challenges: 

DHS Developed Plans for Assessing Capabilities, but Did Not Fully 
Implement Them: 

In July 2005, we reported that DHS had established a draft Target 
Capabilities List that provides guidance on the specific capabilities 
and levels of capability that FEMA would expect federal, state, local, 
and tribal first responders to develop and maintain. We reported that 
DHS defined these capabilities generically and expressed them in terms 
of desired operational outcomes and essential characteristics, rather 
than dictating specific, quantifiable responsibilities to the various 
jurisdictions. DHS planned to organize classes of jurisdictions that 
share similar characteristics--such as total population, population 
density, and critical infrastructure--into tiers to account for 
reasonable differences in capability levels among groups of 
jurisdictions and to appropriately apportion responsibility for 
development and maintenance of capabilities among levels of government 
and across these jurisdictional tiers. According to DHS's Assessment 
and Reporting Implementation Plan, DHS intended to implement a 
capability assessment and reporting system based on target 
capabilities that would allow first responders to assess their 
preparedness to identify gaps, excesses, or deficiencies in their 
existing capabilities or capabilities they will be expected to access 
through mutual aid. In addition, this information could be used: 

* to measure the readiness of federal civil response assets and the 
use of federal assistance at the state and local level and: 

* to provide a means of assessing how federal assistance programs are 
supporting national preparedness. 

In implementing this plan, DHS intended to collect preparedness data 
on the capabilities of the federal government, states, local 
jurisdictions, and the private sector to provide information about the 
baseline status of national preparedness.[Footnote 5] 

DHS's efforts to implement these plans were interrupted by the 2005 
hurricane season. In August 2005, Hurricane Katrina--the worst natural 
disaster in our nation's history--made final landfall in coastal 
Louisiana and Mississippi, and its destructive force extended to the 
western Alabama coast. Hurricane Katrina and the following Hurricanes 
Rita and Wilma--also among the most powerful hurricanes in the 
nation's history--graphically illustrated the limitations at that time 
of the nation's readiness and ability to respond effectively to a 
catastrophic disaster, that is, a disaster whose effects almost 
immediately overwhelm the response capacities of affected state and 
local first responders and require outside action and support from the 
federal government and other entities. In June 2006, DHS concluded 
that target capabilities and associated performance measures should 
serve as the common reference system for preparedness planning. 

In September 2006, we reported that numerous reports and our work 
suggest that the substantial resources and capabilities marshaled by 
federal, state, and local governments and nongovernmental 
organizations were insufficient to meet the immediate challenges posed 
by the unprecedented degree of damage and the resulting number of 
hurricane victims caused by Hurricanes Katrina and Rita. We also 
reported that developing the capabilities needed for catastrophic 
disasters should be part of an overall national preparedness effort 
that is designed to integrate and define what needs to be done, where, 
based on what standards, how it should be done, and how well it should 
be done. In October 2006, Congress passed the Post-Katrina Act that 
required FEMA, in developing guidelines to define target capabilities, 
ensure that such guidelines are specific, flexible, and measurable. In 
addition, the Post-Katrina Act calls for FEMA to ensure that each 
component of the national preparedness system, which includes the 
target capabilities, is developed, revised, and updated with clear and 
quantifiable performance metrics, measures, and outcomes.[Footnote 6] 
We recommended, among other things, that DHS apply an all-hazards, 
risk management approach in deciding whether and how to invest in 
specific capabilities for a catastrophic disaster; DHS concurred, and 
FEMA said it planned to use the Target Capabilities List to assess 
capabilities to address all hazards. 

FEMA Issued the Target Capabilities List in September 2007 but Has 
Made Limited Progress in Developing Preparedness Measures and 
Addressing Long-standing Challenges in Assessing Capabilities: 

In September 2007, FEMA issued the Target Capabilities List to provide 
a common perspective to conduct assessments to determine levels of 
readiness to perform critical tasks and to identify and address any 
gaps or deficiencies. According to FEMA, policymakers need regular 
reports on the status of capabilities for which they have 
responsibility to help them make better resource and investment 
decisions and to establish priorities. Further, FEMA officials said 
that emergency managers and planners require assessment information: 

* to help them address deficiencies; 

* to identify alternative sources of capabilities (e.g., from mutual 
aid or contracts with the private sector); and: 

* to identify which capabilities should be tested through exercises. 

Also, FEMA said that agencies or organizations that are expected to 
supplement or provide capabilities during an incident need assessment 
information to set priorities, make investment decisions, and position 
capabilities or resources, if needed. 

In April 2009, we reported that establishing quantifiable metrics for 
target capabilities was a prerequisite to developing assessment data 
that can be compared across all levels of government. At the time of 
our review, FEMA was in the process of refining the target 
capabilities to make them more measurable and to provide state and 
local jurisdictions with additional guidance on the levels of 
capability they need. Specifically, FEMA planned to develop 
quantifiable metrics--or performance objectives--for each of the 37 
target capabilities that are to outline specific capability targets 
that jurisdictions (such as cities) of varying size should strive to 
meet, being cognizant of the fact that there is not a "one size fits 
all" approach to preparedness. However, FEMA has not yet completed 
these quantifiable metrics for its 37 target capabilities, and it is 
unclear when it plans to do so. 

In October 2009, in responding to congressional questions regarding 
FEMA's plan and timeline for reviewing and revising the 37 target 
capabilities, FEMA officials said they planned to conduct extensive 
coordination through stakeholder workshops in all 10 FEMA regions and 
with all federal agencies with lead and supporting responsibility for 
emergency support-function activities associated with each of the 37 
target capabilities. The workshops were intended to define the risk 
factors, critical target outcomes, and resource elements for each 
capability. The response stated that FEMA planned to create a Task 
Force comprised of federal, state, local, and tribal stakeholders to 
examine all aspects of preparedness grants, including benchmarking 
efforts such as the Target Capabilities List. FEMA officials have 
described their goals for updating the list to include establishing 
measurable target outcomes, providing an objective means to justify 
investments and priorities, and promoting mutual aid and resource 
sharing. In November 2009, FEMA issued a Target Capabilities List 
Implementation Guide that described the function of the list as a 
planning tool and not a set of standards or requirements. 

We reported in July 2005 that DHS had identified potential challenges 
in gathering the information needed to assess capabilities, including 
determining how to aggregate data from federal, state, local, and 
tribal governments and others and integrating self-assessment and 
external assessment approaches. In reviewing FEMA's efforts to assess 
capabilities, we further reported in April 2009 that FEMA faced 
methodological challenges with regard to (1) differences in data 
available, (2) variations in reporting structures across states, and 
(3) variations in the level of detail within data sources requiring 
subjective interpretation. We recommended that FEMA enhance its 
project management plan to include milestone dates, among other 
things, a recommendation to which DHS concurred. In October 2010, we 
reported that FEMA had enhanced its project management plan. 

Nonetheless, the challenges we reported in July 2005 and April 2009 
faced by DHS and FEMA, respectively, in their efforts to measure 
preparedness and establish a system of metrics to assess national 
capabilities have proved to be difficult for them to overcome. We 
reported that in October 2010, in general, FEMA officials said that 
evaluation efforts they used to collect data on national preparedness 
capabilities were useful for their respective purposes, but that the 
data collected were limited by data reliability and measurement issues 
related to the lack of standardization in the collection of data. 

For example, FEMA's Deputy Director for Preparedness testified in 
October 2009 that the "Cost-to-Capabilities" (C2C) initiative 
developed by FEMA's Grant Programs Directorate (at that time already 
underway for 18 months) had a goal as a multiyear effort to manage 
homeland security grant programs and prioritize capability-based 
investments.[Footnote 7] We reported in October 2010, that as a result 
of FEMA's difficulties in establishing metrics to measure enhancements 
in preparedness capabilities, officials discontinued the C2C program. 
Similarly, FEMA's nationwide, multiyear Gap Analysis Program 
implementation, proposed in March 2009, was "to provide emergency 
management agencies at all levels of government with greater 
situational awareness of response resources and capabilities." 
However, as we reported in October 2010, FEMA noted that states did 
not always have the resources or ability to provide accurate 
capability information into its Gap Analysis Program response models 
and simulation; thus, FEMA had discontinued the program. 

FEMA officials reported that one of its evaluation efforts, the State 
Preparedness Report, has enabled FEMA to gather data on the progress, 
capabilities, and accomplishments of a state's, the District of 
Columbia's, or a territory's preparedness program, but that these 
reports included self-reported data that may be subject to 
interpretation by the reporting organizations in each state and not be 
readily comparable to other states' data. The officials also stated 
that they have taken steps to address these limitations by, for 
example, creating a Web-based survey tool to provide a more 
standardized way of collecting state preparedness information that 
will help FEMA officials validate the information by comparing it 
across states. 

We reported in October 2010 that FEMA officials said they had an 
ongoing effort to develop measures for target capabilities--as 
planning guidance to assist in state and local assessments --rather 
than as requirements for measuring preparedness by assessing 
capabilities; FEMA officials had not yet determined how they plan to 
revise the list and said they are awaiting the completed revision of 
Homeland Security Presidential Directive 8, which is to address 
national preparedness. As a result, FEMA has not yet developed 
national preparedness capability requirements based on established 
metrics to provide a framework for national preparedness assessments. 
Until such a framework is in place, FEMA will not have a basis to 
operationalize and implement its conceptual approach for assessing 
federal, state, and local preparedness capabilities against capability 
requirements to identify capability gaps for prioritizing investments 
in national preparedness. 

Mr. Chairman, this completes my prepared statement. I would be pleased 
to respond to any questions that your or other Members of the 
Committee may have at this time. 

[End of section] 

Contacts and Staff Acknowledgments: 

For further information about this statement, please contact William 
O. Jenkins Jr., Director, Homeland Security and Justice Issues, at 
(202) 512-8777 or jenkinswo@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this statement. In addition to the contact named above, the 
following individuals from GAO's Homeland Security and Justice Team 
also made major contributions to this testimony: Chris Keisling, 
Assistant Director; C. Patrick Washington, Analyst, and Lara Miklozek, 
Communications Analyst. 

[End of section] 

Appendix I: National Preparedness Guidelines and Critical Practices 
for Performance Measurement: 

This appendix presents additional information on the Federal Emergency 
Management Agency's National Preparedness Guidelines as well as key 
steps and critical practices for measuring performance and results. 

Figure 2: National Preparedness Guidelines Describe Steps for 
Assessing Capabilities: 

[Refer to PDF for image: illustration] 

1. Convene Working Group. 

2. Determine Capability Requirements. 

3. Assess Current Capability Levels. 

4. Identify, Analyze, and Choose Options. 

5. Update Plans and Strategies. 

6. Allocate Funds. 

7. Update and Execute Program Plans. 

8. Assess and Report. 

Number 3 through 8 form a continuous circle of steps. 

Source: DHS; National Preparedness Guidelines, September 2007. 

[End of figure] 

Figure 3: Key Steps and Critical Practices for Performance and Results: 

[Refer to PDF for image: illustration] 

Step 1: Define Mission and Desired Outcomes Practices:
1. Involve stakeholders; 
2. Assess environment; 
3. Align activities, core processes, and resources. 

Step 2: Measure Performance Practices: 
4. Produce measures at each organizational level that: 
* demonstrate results, are limited to the vital few, 
* respond to multiple priorities, and, 
* link to responsible programs; 
5. Collect data. 

Step 3: Use Performance Information Practices: 
6. Identify performance gaps; 
7. Report information; 
8. Use information. 

Reinforce GPRA implementation: 
Practices: 
9. Devolve decision making with accountability; 
10. Create incentives; 
11. Build expertise; 
12. Integrate management reforms. 

Source: GAO. 

[End of figure] 

[End of section] 

Footnotes: 

[1] The Post-Katrina Act was enacted as Title VI of the Department of 
Homeland Security Appropriations Act, 2007, Pub. L. No. 109-295, 120 
Stat. 1355 (2006). The provisions of the Post-Katrina Act are codified 
in numerous sections of the U.S. Code. The applicable U.S. Code 
citations are included in this statement. The provisions of the Post- 
Katrina Act became effective upon enactment, October 4, 2006, with the 
exception of certain organizational changes related to FEMA, most of 
which took effect on March 31, 2007. 

[2] GAO, National Preparedness: Integration of Federal, State, Local, 
and Private Sector Efforts Is Critical to an Effective National 
Strategy for Homeland Security [hyperlink, 
http://www.gao.gov/products/GAO-02-621T] Apr. 11, 2002. 

[3] GAO, Executive Guide: Effectively Implementing the Government 
Performance and Results Act, [hyperlink, 
http://www.gao.gov/products/GGD-96-118], (Washington, D.C.: June 1, 
1996). 

[4] GAO, Homeland Security: DHS' Efforts to Enhance First Responders' 
All-Hazards Capabilities Continue to Evolve, [hyperlink, 
http://www.gao.gov/products/GAO-05-652] (Washington D.C.: July 11, 
2005); Catastrophic Disasters: Enhanced Leadership, Capabilities, and 
Accountability Controls Will Improve the Effectiveness of the Nation's 
Preparedness, Response, and Recovery System, [hyperlink, 
http://www.gao.gov/products/GAO-06-618] (Washington D.C.: Sept. 6, 
2006); GAO, National Preparedness: FEMA Has Made Progress, but Needs 
to Complete and Integrate Planning, Exercise, and Assessment Efforts, 
[hyperlink, http://www.gao.gov/products/GAO-09-369] (Washington, D.C.: 
Apr. 30, 2009); and GAO, FEMA Has Made Limited Progress in Efforts to 
Develop and Implement a System to Assess National Preparedness 
Capabilities, [hyperlink, http://www.gao.gov/products/GAO-11-51R] 
(Washington, D.C.: Oct. 29, 2010). 

[5] DHS intended to use federal regulatory agencies and other 
appropriate sources to collect private-sector data. 

[6] 6 U.S.C. §§ 744(b)(1), 746(c), 749(b). 

[7] Statement Of The Honorable Timothy W. Manning Deputy 
Administrator, National Preparedness Federal Emergency Management 
Agency U.S. Department Of Homeland Security Before the House Committee 
On Homeland Security Subcommittee On Emergency Communications, 
Preparedness And Response "Preparedness: What Has $29 Billion In 
Homeland Security Grants Bought And How Do We Know?" U.S. House Of 
Representatives Washington, D.C., October 27, 2009. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: