This is the accessible text file for GAO report number GAO-11-51R 
entitled 'FEMA Has Made Limited Progress in Efforts to Develop and 
Implement a System to Assess National Preparedness Capabilities' which 
was released on October 29, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

GAO-11-51R: 

United States Government Accountability Office: 
Washington, DC 20548: 

October 29, 2010: 

The Honorable Frank Lautenberg:
Interim Chairman:
The Honorable George Voinovich:
Ranking Member:
Subcommittee on Homeland Security:
Committee on Appropriations:
United States Senate: 

The Honorable David E. Price:
Chairman:
The Honorable Harold Rogers:
Ranking Member:
Subcommittee on Homeland Security:
Committee on Appropriations:
House of Representatives: 

Subject: FEMA Has Made Limited Progress in Efforts to Develop and 
Implement a System to Assess National Preparedness Capabilities: 

This letter formally transmits a briefing we provided to your staff in 
draft form on September 29, 2010, and subsequent agency comments. We 
provided this briefing in response to a mandate in the conference 
report to the Department of Homeland Security (DHS) Appropriations Act, 
2010.[Footnote 1] In accordance with the direction in that report and 
in consultation with your staff, we provided interim oral briefings in 
March and July 2010 and are reporting the results of our final briefing 
on the Federal Emergency Management Agency's (FEMA) efforts to assess 
national preparedness. Specifically, we are reporting on (1) the 
usefulness and limitations of the national preparedness capabilities 
data that have been collected to date through selected evaluation 
efforts as described by FEMA, and (2) the extent to which FEMA has made 
progress in its national preparedness capability assessment efforts 
since we last reported on this issue in April 2009.[Footnote 2] To 
conduct this work, we analyzed information, such as system user guides 
and project plans for six of FEMA's evaluation efforts that FEMA 
officials identified as being key in assessing preparedness; reviewed 
the Post-Katrina Emergency Management Reform Act[Footnote 3] to 
identify legislative requirements associated with assessing national 
preparedness capabilities; and interviewed FEMA officials. 

In summary, FEMA officials said that evaluation efforts used to collect 
data on national preparedness capabilities were useful for their 
respective purposes, but that the data collected were limited by data 
reliability and measurement issues related to the lack of 
standardization in the collection of data. For example, FEMA officials 
reported that one of its evaluation efforts, the State Preparedness 
Report, has enabled FEMA to gather data on the progress, capabilities, 
and accomplishments of a state's, the District of Columbia's, or a 
territory's preparedness program, but that these reports include self- 
reported data that may be subject to interpretation by the reporting 
organizations in each state and not be readily comparable to other 
states' data. They also stated that they have taken steps to address 
these limitations, for example by creating a Web-based survey tool to 
provide a more standardized way of collecting state preparedness 
information that will help them validate the information by comparing 
it across states. However, since April 2009, FEMA has made limited 
progress in assessing preparedness capabilities. Since that time, its 
primary efforts to assess national preparedness have focused on the 
ongoing implementation of the Comprehensive Assessment System (a five- 
step process for analyzing available preparedness data) and efforts to 
streamline preparedness data-reporting requirements for state, tribal, 
and local stakeholders. However, FEMA has not yet developed national 
preparedness capability requirements based on established metrics to 
provide a framework for these assessments. Further, FEMA has not yet 
fully implemented the five-step Comprehensive Assessment System because 
of delays in completing the fourth step--reporting national 
preparedness capabilities--and issuing the first National Preparedness 
Report. Until such a framework is in place, FEMA will not have a basis 
to operationalize and implement its conceptual approach for assessing 
local, state, and federal preparedness capabilities against capability 
requirements to identify capability gaps for prioritizing investments 
in national preparedness. For additional information on a summary of 
our work, see enclosure I, slides 13 through 15. Based on the results 
of our review, we are not making any recommendations for congressional 
consideration or agency action. 

We provided a draft of this briefing to DHS for review and comment. DHS 
provided written comments, which are reprinted in enclosure II. In 
commenting on a draft of this briefing, DHS stated that FEMA is working 
toward refining its programs, and GAO's analysis of its program 
performance greatly benefits its ability to continually improve its 
activities. In addition, DHS commented that FEMA believes it has made 
much progress since 2009 in meeting the Post-Katrina Emergency 
Management Reform Act and other legislative requirements and 
highlighted some of its specific achievements, such as the 
establishment of a working group to help consolidate and streamline 
reporting requirements for state, tribal, and local stakeholders. The 
actions FEMA has taken are discussed in more detail in the enclosed 
briefing. DHS also provided technical comments, which we incorporated 
into the report as appropriate. 

We are sending copies of this report to other appropriate congressional 
committees. We are also sending copies to the Secretary of Homeland 
Security, the FEMA Administrator, and the Director of the Office of 
National Preparedness Directorate. This report will also be available 
at no charge on our Web site at [hyperlink, http://www.gao.gov]. Should 
you or your offices have any questions concerning this report, please 
contact me at (202) 512-8757 or JenkinsWO@gao.gov. Contact points for 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. Key contributors to this report were 
Chris Keisling, Assistant Director; Frederick Lyles, Jr., Analyst-in-
Charge; Jared Hermalin; Tracey King; Cynthia Saunders; and Adam Vogt. 

Signed by: 

William O. Jenkins: 
Director, Homeland Security and Justice Issues: 

Enclosures--2: 

[End of section] 

Enclosure I: Briefing to Subcommittees on Homeland Security, Senate and 
House Committees on Appropriations: 

FEMA Has Made Limited Progress in Efforts to Develop and Implement a 
System to Assess National Preparedness Capabilities: 

3rd Quarterly Briefing for Congressional Requesters: 

October 29, 2010: 

Contents: 

* Introduction: 
* Objectives: 
* Scope and Methodology: 
* Summary: 
* Findings: 
* Appendixes: 

Introduction: 

The attacks of 9/11 and Hurricane Katrina were, respectively, the most 
destructive terrorist and natural disasters in our nation’s history and 
highlighted gaps in the nation’s readiness to respond effectively to 
large scale catastrophes. To strengthen the nation’s preparedness for 
such incidents, in December 2003 the President issued Homeland Security 
Presidential Directive 8 (HSPD-8) that called on the Secretary of 
Homeland Security to enhance preparedness capabilities of federal, 
state, and local entities.[Footnote 4] In October 2006, the Post-
Katrina Emergency Management Reform Act (Post-Katrina Act) charged the 
Federal Emergency Management Agency (FEMA)—a component of the 
Department of Homeland Security (DHS)—with responsibility for leading 
the nation in developing a national preparedness system.[Footnote 5] 

The Post-Katrina Act requires that FEMA develop a national preparedness 
system and assess preparedness capabilities—capabilities needed to 
respond effectively to disasters—to determine the nation’s preparedness 
capability levels and the resources needed to achieve desired levels of 
capability.[Footnote 6] Figure 1 provides an illustration of how local, 
state, and federal resources provide capabilities for different levels 
of incident effect. 

FEMA’s National Preparedness Directorate within its Protection and 
National Preparedness organization is responsible for developing and 
implementing a system for measuring and assessing national preparedness 
capabilities (see app. I for FEMA’s organizational chart). 
Organizational responsibilities related to national preparedness 
assessments are summarized in appendix II. 

Figure 1: Conceptual Illustration for Assessing Capability Requirements 
and Identifying Capability Gaps for National Preparedness: 

[Refer to PDF for image: graph] 

This graph shows Level of incident effect on the Y axis, and Time on 
the X axis. The three levels are: 

Local resources; 
State resources; 
Federal resources. 

Federal resources has the highest peak of level of incident effect. 
There is a circular line going over the three sections, representing 
capability requirements. Capability gaps are indicated as the difference 
between capability requirements and cumulative resources. 

Source: GAO. 

[End of figure] 

In April 2009, we reported that FEMA was developing the Comprehensive 
Assessment System in response to the Post-Katrina Act requirement to 
assess the nation’s capabilities and overall preparedness for 
preventing, responding to, and recovering from natural and man-made 
disasters.[Footnote 7] We reported that FEMA (1) lacked a comprehensive 
approach to managing the development of policies and plans to define 
emergency response roles and responsibilities; (2) faced challenges in 
meeting statutory and program requirements in conducting the National 
Exercise Program; (3) had not established a clearly defined course of 
action to assess capabilities or defined outcomes of where the nation 
should be in terms of domestic preparedness goals and measurable 
performance indicators for the nation’s preparedness programs; and (4) 
had not established a strategic plan for integrating elements of the 
national preparedness system. We concluded that without defining 
capability requirements, FEMA and its local, state, tribal, and federal 
preparedness stakeholders cannot implement a standardized approach to 
identifying capability gaps. We recommended that FEMA take action to 
address these concerns, and FEMA concurred with our recommendations. 
Appendix III provides a summary of the status of the agency’s 
implementation of our recommendations.[Footnote 8] 

The conference report accompanying the Department of Homeland Security 
Appropriations Act, 2010, directed us to provide quarterly reports to 
Congress in fiscal year 2010 on the status of FEMA’s efforts to assess 
national preparedness.[Footnote 9] This is the final in our series of 
briefings. 

As part of our first quarterly briefing (March 2010) to staff of the 
Senate Appropriations Committee, Homeland Security Subcommittee, and 
the House Appropriations Committee, Homeland Security Subcommittee, we 
reported that FEMA had initiated efforts to develop a framework to 
assess preparedness capabilities by, for example, creating a working 
group to assist in the development of an integrated approach for 
assessing national preparedness.[Footnote 10] We also reported that for 
fiscal years 2008 through 2010, FEMA had budgeted about $58 million to 
develop and implement the evaluation efforts they had identified as 
being key in assessing preparedness. FEMA officials said these efforts 
were useful for their respective purposes, but had certain limitations 
such as data reliability given that the data collected were in many 
cases self-reported by states and grant recipients and FEMA did not 
have a verification mechanism for them. 

As part of our second quarterly briefing (July 2010) to staff of the 
Senate Appropriations Committee, Homeland Security Subcommittee, and 
the House Appropriations Committee, Homeland Security Subcommittee, we 
reported that the working group FEMA created to assist in the 
development of an integrated approach for assessing national 
preparedness had concluded its efforts and made several recommendations 
to FEMA management regarding the requirements that any future data-
reporting system related to national preparedness should address. We 
also reported that FEMA had developed a Comprehensive Assessment System 
to assess the nation's preparedness for disasters but faced challenges 
in collecting data needed to implement the system because the data are 
collected by many organizations in different forms. 

Objectives: 

In accordance with this mandate, our objectives were to determine the 
following: 

1. How has FEMA described the usefulness and limitations of the 
national preparedness capabilities data that have been collected to 
date through selected evaluation efforts? 

2. To what extent has FEMA made progress in its national preparedness 
capability assessment efforts since April 2009? 

Scope and Methodology: 

To identify how FEMA has described the usefulness and limitations of 
the national preparedness capabilities data they have collected as of 
September 2010 through the selected evaluation efforts, we: 

* obtained and reviewed information such as system user guides and 
project plans for six of FEMA’s evaluation efforts that FEMA officials 
identified as being key in assessing preparedness—the State 
Preparedness Reporting system, the National Incident Management System 
Compliance Assistance Support Tool, the Grants Reporting Tool, the 
Logistics Capability Assessment Tool, the Gap Analysis Program, and the 
Cost-to-Capability pilot program. Figure 2 provides a description of 
each of the six evaluation efforts. We previously reported on the State 
Preparedness Reporting system, the National Incident Management System 
Compliance Assistance Support Tool, the Gap Analysis Program, and the 
Cost-to-Capability pilot program in our April 2009 report, and FEMA 
officials identified the Grants Reporting Tool and the Logistics 
Capability Assessment Tool as additional key evaluation efforts, and: 

* interviewed FEMA officials to obtain information on the status, 
usefulness, and limitations of these evaluation efforts. 

Figure 2: Six Evaluation Efforts Selected for This Review 

[Refer to PDF for image: chart] 

State Preparedness Reports: 

The Post-Katrina Act requires all states receiving preparedness funding 
to report annually on the state's level of preparedness, such as 
current and target capabilities and resource needs to meet identified 
preparedness priorities. Continuing annual requirements for 
jurisdictions that receive DHS preparedness assistance.[A] 

National Incident Management System Compliance Assistant Support Tool: 

Web-based self--assessment tool for jurisdictions to evaluate and 
report their achievement of National Incident Management System 
implementation.[B] Continuing annual reporting requirement for DHS 
preparedness recipients such as states, territories, tribes, and 
localities. 

Grants Reporting Tool: 

Recipients of FEMA grants are required to use this tool to report twice 
a year on planned and actual grant obligations and progress made in 
developing preparedness capabilities through grant projects for 
financial and program management oversight. Continuing biannual 
reporting requirement for FEMA grantees such as states, territories, 
tribes, and localities. 

Logistics Capability Assessment Tool: 

Logistics pilot program created to assist states in conducting self--
assessments to determine their operations readiness to respond to 
disasters in terms of planning, operations, distribution and property 
management of emergency . Program piloted in 12 states. 

Gap Analysis Program: 

Program goal was to identify "gaps" between estimated incident response 
needs for operational planning and equipment and the actual capability 
of levels of government and the private sector to address that need. 
Collection and submission of data was voluntary. FEMA now considers 
this effort to be a planning tool rather than a stand-alone program. 

Cost-to-Capability: 

The pilot program was developed as an approach to directly measure the 
effect of grants on grantee preparedness. FEMA suspended this effort in 
response to concerns of stakeholders. 

Source: FEMA. 

[A] Includes the states, territories, and the District of Columbia. 

[B] The National Incident Management System provides a standardized 
program of planning, organization, and resources for management of 
emergency incidents. 

[End of figure] 

To determine the extent to which FEMA has made progress in its national 
preparedness capability assessment efforts since we reported on its 
status in April 2009, we reviewed the status of FEMA’s efforts to 
implement the Comprehensive Assessment System, streamline preparedness 
reporting requirements, and work with the congressionally-mandated 
Local, State, Tribal and Federal Preparedness Task Force (Task Force). 
[Footnote 11] Specifically, we: 

* reviewed the Post-Katrina Act sections that required FEMA to develop 
and implement the Comprehensive Assessment System to identify 
legislative requirements associated with assessing national 
preparedness capabilities and analyzed FEMA’s contracts for the system 
and associated documents including project and implementation plans, 
and monthly progress reports; 

* observed several meetings and reviewed the meeting minutes of the 
Reporting Requirements Working Group, which was responsible for 
reviewing the agency’s preparedness data collection efforts and 
reducing the reporting burden on state, local, and tribal governments, 
to identify FEMA’s planned approach and progress achieved to date; 

* observed the Local, State, Tribal, and Federal Preparedness Task 
Force meetings in April, June, July, and September 2010 and reviewed 
its meeting minutes to obtain information on its efforts to develop 
recommendations related to policy and guidance, grant programs, and 
capabilities and assessments; and: 

* interviewed officials from national stakeholder organizations 
identified by FEMA, including the International Association of 
Emergency Managers, the National Advisory Council, the National Council 
on Disability, and the National Emergency Management Association to 
obtain their views on FEMA’s preparedness capability assessment 
efforts.[Footnote 12] Although the views of the officials from these 
organizations can not be generalized to all of FEMA’s national 
stakeholder organizations, they provided useful perspectives on FEMA’s 
preparedness capability assessment efforts. 

To ensure the technical accuracy of the briefing, we provided a draft 
of this briefing to DHS and FEMA and met with officials to obtain 
technical comments, which we incorporated as appropriate. 

We conducted this performance audit from December 2009 through October 
2010, in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Summary: 

FEMA officials said that their evaluation efforts were useful for their 
respective purposes, but the national preparedness capabilities data 
that have been collected to date through selected evaluation efforts 
were limited by data reliability and measurement issues related to the 
lack of standardization in the collection of data. However, FEMA 
officials stated that the cost of using federal employees or 
contractors to collect and validate preparedness data at the state and 
local levels would be cost-prohibitive. In terms of usefulness of the 
preparedness capabilities data provided by the evaluation efforts we 
selected for this review, each of the evaluation efforts has provided 
data of specific use to its respective users. For example, FEMA 
officials reported that the Logistics Capability Assessment Tool helped 
states and localities conduct their self-assessments in identifying 
disaster response capabilities for operational readiness (i.e., short 
term actions for an immediate event), and target areas for improvement. 
In terms of limitations, FEMA has identified data reliability as a 
primary limitation of the evaluation efforts that have been 
implemented. For example, FEMA officials said State Preparedness 
Reports include self-reported data that may be subject to 
interpretation by the reporting organizations in each state and not be 
readily comparable to other states’ data. They also stated that they 
have taken steps to address these limitations, for example by creating 
a Web-based survey tool to provide a more standardized way of 
collecting state preparedness information that will help them validate 
the information by comparing it across states. FEMA officials stated 
that they believe overall the data they have are reliable. 

Since April 2009, FEMA has made limited progress in assessing 
preparedness capabilities. Its primary efforts to assess national 
preparedness since our April 2009 report have focused on the ongoing 
implementation of the Comprehensive Assessment System and efforts to 
streamline preparedness data reporting requirements for state, tribal, 
and local stakeholders. Specifically: 

* FEMA has developed a Comprehensive Assessment System—a five-step 
process for analyzing available preparedness data—but has not developed 
national preparedness capability requirements based on established 
metrics to provide a framework for these assessments. Further, FEMA has 
not yet fully implemented the five-step process because of delays in 
completing the fourth step—reporting national preparedness capabilities—
and issuing the first National Preparedness Report. 

* FEMA officials said they planned to consider the Task Force’s October 
2010 report and its recommendations, and their progress in assessing 
national preparedness capabilities depended on the outcome of the 
administration's effort to revise HSPD-8, the national preparedness 
presidential directive; the revision was still in process as of October 
2010. 

* Until such a framework is in place, FEMA will not have a basis to 
operationalize and implement its conceptual approach for assessing 
local, state, and federal preparedness capabilities against capability 
requirements to identify capability gaps for prioritizing investments 
in national preparedness. 

* FEMA established a Reporting Requirements Working Group, which met 
eight times from August 2009 through April 2010 to discuss analysis 
efforts underway by FEMA’s offices and directorates. Although the goal 
of the group was to develop an agencywide information system and 
provide recommendations for streamlining data requests, FEMA 
discontinued the working group prior to it developing a system or 
specific recommendations for streamlining preparedness assessment 
reporting requirements for state, local, and tribal stakeholders. 

Objective 1: 

FEMA Reports That Its Evaluation Efforts Have Been Useful for Their 
Intended Purposes, but Limitations Exist: 

* FEMA officials said that, in general, the data developed through the 
evaluation efforts we selected for this review have provided 
information of specific use to FEMA programs responsible for supporting 
the development of national preparedness capabilities. Program 
officials identified data reliability as a primary limitation of the 
evaluation efforts since they all rely on self-reporting of 
preparedness data and assessments. For example, FEMA officials 
identified issues regarding whether data are complete and accurate and 
regarding establishing and applying objective ways to measure 
preparedness capabilities; they said the ongoing development of 
capability measures is intended to address this issue. They also stated 
that they have taken steps to address these limitations, for example by 
creating a Web-based survey tool to provide a more standardized way of 
collecting state preparedness information that will help them validate 
the information by comparing it across states. FEMA officials stated 
that they believe overall the data they have are reliable. 

FEMA Reports That State Preparedness Reports Have Been Useful for 
Assessing Capabilities, but Limitations Exist: 

Usefulness: 

* State Preparedness Reports, which are required by the Post-Katrina 
Act, have enabled FEMA to gather data on the progress, capabilities, 
and accomplishments of a state’s, the District of Columbia’s, or a 
territory’s preparedness program. States, territories, and the District 
of Columbia submitted their first state preparedness reports to FEMA in 
2008. 

* The reports track information regarding planning and incident 
management efforts, current preparedness capability levels, targeted 
levels of capability, preparedness expenditures, and estimates of 
needed monetary resources. 

* For fiscal year 2010 reporting, FEMA developed a Web-based survey to 
make the report more user-friendly. According to FEMA officials, the 
new format uses quantitative scores and common metrics to help assess 
capabilities in the Target Capabilities List. 

Limitations: 

* Prior to fiscal year 2010, FEMA had not yet established common 
metrics, which made comparisons among states’ prior years’ reports 
difficult. 

* FEMA required that states assess 10 to 15 "priority" capabilities in 
prior years’ reports, rather than all 37 capabilities in the Target 
Capabilities List. 

* The reliability of the data in reports could be limited because FEMA 
relies on states to self-report data, which makes it difficult to 
ensure data are consistent and accurate. 

Status: This is a continuing annual requirement for jurisdictions 
receiving preparedness assistance from DHS. 

FEMA Reports That the National Incident Management System Compliance 
Assistance Support Tool Has Been Useful for Assessing Compliance, but 
Limitations Exist: 

Usefulness: 

* This Web-based tool enabled FEMA to collect data from stakeholders at 
all levels of government and report on compliance with National 
Incident Management System (N I MS) at the state level. 

* Users identify successes, shortfalls, and corrective action plans (if 
necessary) for each NIMS compliance objective. 

* The tool is widely used; as of January 2010, fiscal year 2009 
compliance data were submitted by 50 states, the District of Columbia, 
and territories; 406 state agencies; 10,877 local agencies; and 36 
tribal nations.[Footnote 13] 

Limitations: 

* While state reporting is standardized and required, no 
standardization exists across states regarding the type of 
jurisdictions within each state reporting on their implementation of 
NIMS. Tribal accounts may be created under FEMA regions, and are not 
necessarily linked to state accounts. 

* Because jurisdictions input data and information through the tool 
based on self-assessments, data reliability issues exist. 

Status: DHS preparedness recipients such as states, territories, 
tribes, and localities are to report annually on NIMS compliance. 
Homeland Security Presidential Directive (HSPD-5) requires the DHS 
Secretary to develop standards and guidelines for state and local 
adoption of NIMS.[Footnote 14] 

FEMA Reports That the Grants Reporting Tool Has Been Useful, but 
Limitations Exist: 

Usefulness: 

* The Grants Reporting Tool database provided: (a) standardized and ad 
hoc reports to Congress, DHS, auditors, program administrators, and 
grantees; (b) grantee audit trails; and (c) links between project data 
and Homeland Security strategies and target capabilities. 

* Using the tool, FEMA tracked 21 grant programs from fiscal years 2004 
through 2009. 

* The tool allowed FEMA staff to track the entire grant life cycle, and 
assisted in financial and program management oversight by collecting 
information on, among other things, state, local, and tribal grant 
recipients’ planned and actual obligations and expenditures. 

Limitations: 

* Because the system is not designed to capture information on 
preparedness capability improvements, it can be difficult to measure 
the relationship between expenditures and preparedness capabilities. 

* While validation mechanisms are built into the Web-based system, data 
reliability is an issue because the tool relies on state and local 
officials to self-report data for preparedness assessments that are 
subject to individual interpretation; thus, the consistency of 
information across grantees is uncertain. For example, grantees may be 
asked to report whether they have made progress toward implementation 
of national priorities. 

Status: Grant recipients are required to use the Grants Reporting Tool 
on a continuing biannual basis to provide information for FEMA 
monitoring and oversight. 

FEMA Reports That the Logistics Capability Assessment Tool Has Been 
Useful, but Limitations Exist: 

Usefulness: 

* This tool was designed for the exclusive use of states to help them 
conduct self-assessments to identify their disaster logistics planning 
and response capabilities for operational readiness (i.e., short term 
actions for an immediate event), and target areas for improvement. 

* The tool focuses on four core logistics competencies (related to 
planning, operations, distribution, and property management) and their 
subcategories; respondents choose from a range of competency levels. 

* The results of the tool are intended to measure local logistics 
capability levels compared to logistics needs. 

Limitations: 

* The tool was neither designed nor intended to be used to evaluate 
logistical capabilities at the federal or state level. 

* Differences in states’ logistics capabilities make it difficult to 
summarize capabilities to identify, inventory, dispatch, mobilize, 
transport, and track resources for incident management at the regional 
level because the tool was not designed for this purpose. 

Status: This pilot evaluation program is to be completed in fiscal year 
2010; FEMA is exploring opportunities to continue the effort. 

FEMA Reports That Its Gap Analysis Program Provided Useful Information, 
but Limitations Exist Usefulness: 

* The program helped state and local governments to identify potential 
preparedness shortfalls (gaps) and enhance their operational disaster 
capabilities. 

* The program helped provide flexibility to states to use scenarios 
that are tailored to their risks, such as hurricanes or earthquakes. 

Limitations: 

* To estimate response requirements in the absence of actual disaster 
events, states may not have the resources or ability to provide 
accurate capability information into Gap Analysis Program response 
models and simulations. 

* The depth and transparency of analysis is uneven across states, which 
casts a degree of uncertainty on reported results, and the lack of a 
national planning system and collaborative federal, state, and local 
planning is an impediment to identification of capability requirements. 

Status: FEMA revised its approach to operational planning assistance. 
As a result, FEMA officials no longer consider this evaluation program 
as a stand alone program; instead program officials said they will 
continue to support these types of evaluations by offering the analysis 
tools to assist in state and local planning activities. 

FEMA Reports That Its Cost-to-Capability Program Was Useful, but 
Limitations Resulted in its Discontinuation: 

Usefulness: 

* FEMA pilot testing of the system in fiscal year 2009 was its first 
attempt to study how grants help build preparedness capabilities, and 
the pilot program provided useful information about the limitations of 
the cost-to-capability model. 

Limitations: 

* FEMA officials identified difficulties in establishing metrics to 
measure enhancements in preparedness capabilities; FEMA has an ongoing 
effort to develop measures for target capabilities as planning guidance 
to assist in state and local assessments; officials plan to complete 
their efforts after the revision of HSPD-8. 

* FEMA said that state officials expressed concerns that there is no 
standardized formula for determining the projected dollar amounts that 
are necessary for "gaining" or "sustaining" levels of capability; users 
apply their judgment in assessing the status of preparedness 
capabilities. 

Status: FEMA officials discontinued the program in 2010; however, FEMA 
officials said they plan to continue to explore ways to assess the 
effect of grant funds on state and local preparedness capabilities, 
including quantitative and qualitative measures. 

Objective 2: 

FEMA Has Made Limited Progress in Implementing the Comprehensive 
Assessment System and Streamlining Reporting Requirements: 

FEMA has made limited progress in implementing its national 
preparedness capabilities assessment efforts since we reported on its 
efforts in April 2009. Specifically, FEMA has not: 

* fully implemented its Comprehensive Assessment System—its primary 
effort intended to bring together multiple sources of preparedness 
information and data (including data from the evaluation efforts 
included in this review)—nor determined a framework for assessing and 
reporting national preparedness capabilities based on established 
metrics and: 

* streamlined reporting requirements as a result of the Reporting 
Requirements Working Group’s efforts. 

Until such a framework is in place, FEMA will not have a basis to 
operationalize and implement its conceptual approach for assessing 
local, state, and federal preparedness capabilities against capability 
requirements to identify capability gaps for prioritizing investments 
in national preparedness. 

FEMA officials said that the Comprehensive Assessment System was fully 
functional and would continue to be developed and improved, although 
they did not identify a time frame for publication of the National 
Preparedness Report. They said that implementation of the working group’
s recommendations would be part of their ongoing efforts. 

FEMA Has Made Limited Progress Implementing Its Comprehensive 
Assessment System: 

FEMA has developed a system to assess preparedness that it defines as a 
five-step process (defining, collecting, analyzing, reporting, and 
improving preparedness data). FEMA reported facing challenges in 
collecting data on preparedness activities needed to implement the 
system because such data are collected by many organizations in many 
different forms. FEMA officials said they had produced a draft report 
assessing national preparedness utilizing the Comprehensive Assessment 
System process. According to FEMA officials, FEMA began developing the 
first National Preparedness Report in October 2008.[Footnote 15] Since 
then, the report has undergone a series of revisions and the latest 
version of the draft report was submitted for review in May 2010. FEMA 
officials said they plan to submit the draft report for review and 
approval by the end of calendar year 2010. 

Based on discussions with FEMA officials and our review of its 
evaluation efforts, system documentation and products, we developed an 
illustration that reflects the relationship between the six evaluation 
efforts and the way FEMA collects and plans to report on preparedness 
capabilities data—see figure 3. FEMA has not yet determined how 
information technology will be used to collect and store these data and 
make them accessible to FEMA and state and local governments. FEMA 
officials said that decisions regarding the technological approach for 
gathering and disseminating information on national preparedness 
capabilities will be finalized once they take actions to address the 
working group and task force recommendations. FEMA officials agreed 
that the figure reflects a general concept of how FEMA might gather and 
disseminate preparedness data to the stakeholders. 

Figure 3: Conceptual Illustration of How FEMA Might Gather and 
Disseminate Preparedness Data: 

[Refer to PDF for image: Chart] 

The following graph has four tiers separated by arrows. 

First Tier: Input from stakeholders including state, local, tribal, and 
territorial governments. 

Second Tier: 

State Preparedness (Data transferred to the Comprehensive Assessment 
System); 
National Incident Management Systems Compliance Assistance Support 
Tool; 
Grants Reporting Tool; 
Single access point for queries; 
Logistics Capability Assessment Tool; 
Gap Analysis Program; 
Cost-to-Capability Program[A] (Data transferred to the Comprehensive 
Assessment System). 

Third Tier: 

Comprehensive Assessment System: Intended to be a data repository and 
system for national preparedness assessment, designed to give a clear, 
scalable picture of state, local, and tribal preparedness and to 
facilitate capability-based planning, reporting, and resource 
allocation. 

Fourth Tier: 

Pointing to Third Tier: Additional resources include: training, 
exercise, and real event data; federal and state plans; studies and 
research conducted by consultants/scholars; and data from other 
agencies--including the Centers for Disease Preparedness Prevention and 
U.S. Department of Health and Human Services. 

FEMA--generated reports: 

National Preparedness Reports. 

Source GAO analysis of FEMA data. 

[A] FEMA now considers the Gap Analysis Program to be a planning tool 
rather than a stand-alone program, and the Cost-to-Capability 
Initiative evaluation effort was discontinued; related evaluation 
efforts continue as part of a new Grants Effectiveness Analysis effort, 
according to FEMA officials. 

[End of figure] 

According to FEMA officials, the requests for revisions to the draft 
National Preparedness Report reflect management concerns regarding 
methodological issues with the preparedness assessment. For example, 
initial drafts utilized four scenarios of catastrophic events—two man-
made and two natural—to identify critical capabilities and shortfalls 
in those capabilities at federal, state, and local levels.[Footnote 16] 
However, management reviewers directed program officials to use a 
mission-oriented framework (prepare, protect, respond to, recover from, 
and mitigate) to identify critical capabilities and shortfalls for the 
report. As a result, FEMA has not yet completed implementation of its 
five-step Comprehensive Assessment System process for defining, 
collecting, analyzing, reporting, and improving preparedness data. 
Program officials plan to submit the draft report for review and 
approval by the end of calendar year 2010. At such time, the fourth 
step should be completed, and FEMA plans to address the fifth step to 
improve preparedness data to reflect ongoing and evolving national 
preparedness reporting requirements. 

In our 2009 report on the National Preparedness System,[Footnote 17] we 
recommended that FEMA improve its approach for developing the 
Comprehensive Assessment System by enhancing its project management 
plan—to include milestone dates, an assessment of risk, and addressing 
concerns of stakeholders in comprehensively collecting and reporting on 
disparate information sources—to reflect best practices for project 
management established by the Project Management Institute.[Footnote 
18] The practices include, among other things, 

* developing milestone dates to identify points throughout the project 
to reassess efforts underway to determine whether project changes are 
necessary, 

* managing project risk to increase the probability and effect of 
positive events and decrease the probability and effect of adverse 
events on the project, and: 

* adapting the specifications, plans, and approach to the different 
concerns and expectations of the various stakeholders involved in the 
project. 

In response, FEMA: 

* developed a project and implementation plan in August 2009 that 
provides milestone dates and identifies key assessment points 
throughout the project to determine whether project changes are 
necessary, 

* developed an October 2009 proposal for a risk management plan, and: 

* drafted a November 2009 plan for improving customer service for state 
and local preparedness stakeholders. 

FEMA officials stated that they continue to experience the challenges 
we identified in our April 2009 report regarding the quality and 
availability of data needed as inputs for the Comprehensive Assessment 
System’s five-step process to measure national preparedness.[Footnote 
19] Specifically, because responsibilities for protecting against, 
responding to, and mitigating emergencies are shared by different 
federal, state, and local organizations, data on emergency preparedness 
are collected by many organizations for different purposes, in many 
different forms, and to differing degrees of thoroughness and 
completeness. 

FEMA officials said their effort to implement sections in the Post-
Katrina Act prescribing that the system assess, among other things, the 
nation’s preparedness capability levels against target capability 
levels and the resources needed to achieve such capability levels is 
being revised to incorporate pending anticipated changes to national 
homeland security preparedness policies and the national-level review 
of preparedness by the task force. They did not know when the revised 
policies would be issued. The task force issued its report in October 
2010. According to FEMA officials, the agency plans to complete 
revisions to the Target Capabilities List by the end of 2010 and 
provide them to the states, with changes based on the recommendations 
of the task force and FEMA leadership guidance. 

As a result, FEMA has not yet established measurable capability 
requirements for federal, state, tribal, and local preparedness efforts 
to provide a framework for analyzing the results of capability 
assessments provided by the Comprehensive Assessment System process to 
identify capability gaps and prioritize national preparedness resource 
investments. 

To improve the availability and quality of preparedness data, officials 
from FEMA’s National Preparedness Directorate said they are: 

* working with state and local officials to improve the quality of 
information obtained through evaluation of federal, regional, state, 
and local emergency response exercises; 

* promoting a new Emergency Management Institute training course to 
enhance standardization in the way evaluators observe and report on the 
results of exercises; and[Footnote 20] 

* developing a program with the Emergency Management Accreditation 
Program to better assess exercises.[Footnote 21] 

However, FEMA officials said they had not yet taken steps to implement 
our April 2009 recommendation to the National Exercise Division to 
improve its implementation of statutory and program requirements and 
its efforts to measure program performance. As a result, FEMA’s data 
for measuring the effectiveness of the National Exercise Program are 
incomplete. On August 17, 2010, Secretary of Homeland Security Janet 
Napolitano announced plans to revise the program by the end of November 
2010. In addition, according to FEMA officials, they are reviewing 
after-action reporting requirements for exercises conducted by homeland 
security grant program recipients as part of their efforts to reduce 
the reporting burden on state and local stakeholders. 

FEMA Has Made Limited Progress in Streamlining Reporting Requirements 

In August 2009, FEMA established a Reporting Requirements Working Group 
consisting of 41 officials from FEMA; state, local, and tribal 
governments; and the National Emergency Management Association. 
[Footnote 22] The goal of the group was to develop an agencywide 
information-gathering system that is fielded to state, local, tribal, 
and territorial governments and provide recommendations for 
streamlining data requests. The working group met eight times from 
August 2009 through April 2010 to discuss analysis efforts underway by 
FEMA’s offices and directorates. 

The Reporting Requirements Working Group established by FEMA reviewed 
analyses developed by National Preparedness Directorate staff and 
contractors supporting the working group who took a number of steps to 
assess reporting requirements. For example, they: 

* worked with FEMA’s Office of Records Management to identify all FEMA 
reporting requirements; 

* performed an analysis of FEMA data collection tools and systems to 
determine opportunities for consolidation, which identified over 900 
data-reporting requirements and suggested phasing out two systems to 
eliminate 376 of 988 questions; 

* met with representatives of the owners and operators of FEMA data-
collection systems (Logistics, Response, and National Preparedness and 
Protection) to share information and consider possible consolidations; 
and: 

* drafted a report summarizing data-collection and reporting 
methodologies that concluded using a survey tool is the most effective 
approach for collecting data from state, local, and tribal 
stakeholders. 

According to the Deputy Administrator for Protection and National 
Preparedness, FEMA discontinued the working group effort after 
achieving its purpose to consult with state, local, and tribal 
stakeholders related to streamlining efforts. He also said that FEMA 
plans to consider the working group’s recommendations in the near 
future in its efforts to implement a system that meets the principles 
and goals approved by FEMA’s senior leadership. FEMA informed the 
working group’s state, local, and tribal members of the cessation of 
the effort in June 2010. 

The Deputy Administrator said the group’s efforts resulted in seven 
recommendations. The recommendations focused on goals for an agencywide 
information-gathering system but did not provide specific 
recommendations for streamlining data requests. 

The working group made general recommendations in 2009 and reiterated 
them in April 2010: 

1. FEMA should better communicate the reasons, needs, and desired 
analysis prior to requesting information from state, local, and tribal 
governments; 

2. data submitted for analysis should meet the information needs of all 
stakeholders; 

3. a data collection and warehouse should be easily queried by all 
stakeholders; 

4. a data warehouse must have a high level of security; 

5. FEMA region offices should serve as a conduit for preparedness-
related data requests to state, local, and tribal governments; 

6. FEMA should develop a 12-month calendar that outlines the frequency 
of existing FEMA reporting requirements to identify current effect and 
eliminate duplicative efforts; and: 

7. FEMA should work with other partners, especially within DHS to 
ensure awareness of the initiative and improve interdepartmental 
consistency. 

In notifying working group members that the working group was being 
discontinued, FEMA’s Deputy Administrator for Protection and National 
Preparedness identified the following action items to implement the 
working group recommendations and continue efforts to develop an 
agencywide Information-gathering system and streamline data-collection 
efforts to be carried out by FEMA offices and divisions: 

* review the approximately 100 identified data-collection efforts 
throughout FEMA, analyze them and make recommendations for 
consolidation of all data-collection systems within FEMA; 

* identify an optimal data-collection methodology and related processes 
across FEMA; 

* develop an overarching, agencywide FEMA data-collection policy that 
outlines standards to govern existing and future data collection; 

* develop requirements for a unified preparedness, capability, and 
readiness information-collection system that has utility for meeting 
all stakeholders’ needs; and: 

* solidify an annual timeline for data collection based on the current 
state and the stakeholders desired end state. 

Next Steps by FEMA to Further National Preparedness Capabilities 
Assessment Efforts: 

According to program officials, FEMA’s efforts to define a framework 
within which its capability assessments can be effectively applied rely 
on the results of two key ongoing efforts: the first report of the 
congressionally-mandated Local, State, Tribal and Federal Preparedness 
Task Force, and planned revisions to Homeland Security Presidential 
Directive-8 (HSPD-8). 

In response to the conference report accompanying the 2010 DHS 
appropriations act, FEMA established the Local, State, Tribal, and 
Federal Preparedness Task Force.[Footnote 23] The task force consists 
of 35 members of federal, state, local, and tribal governments and is 
to make recommendations for all levels of government. The task force 
evaluated (1) which policies and guidance need updating and the most 
appropriate process to do so; (2) which grant programs work the most 
efficiently and where programs can be improved; and (3) the most 
appropriate way to collectively assess national preparedness 
capabilities and identify capability gaps. Accordingly, the task force 
established three working groups—policy and guidance, grants programs, 
and capabilities and assessment—and held meetings in April, June, July, 
and September 2010. The working groups used teleconferencing and a task 
force information-sharing site to facilitate their work between 
meetings of the entire task force. 

According to FEMA officials, the administration is planning to issue a 
revision of HSPD-8; the revision will significantly affect FEMA’s 
national preparedness policies and plans. 

FEMA has identified a number of steps that the agency plans to take to 
further implement its ongoing initiatives once the administration has 
issued its planned revisions to HSPD-8, including the following: 

* Continuing refinement of the Comprehensive Assessment System and 
issuing the first National Preparedness Report. 

* Implementing recommendations from the Reporting Requirements Working 
Group. 

In addition, FEMA identified additional preparedness assessment efforts 
related to our recommendations for the National Preparedness System 
from April 2009 that it plans to take, including the following: 

* Integrating HSPD-8 revisions into national preparedness system 
elements including the Integrated Planning System, Comprehensive 
Assessment System, and strategic planning efforts. 

* Determining how to revise the Target Capabilities List and 
incorporate recommendations from the Local, State, Tribal, and Federal 
Preparedness Task Force. 

* Revising the National Exercise Program.[Footnote 24] 

Appendix I: 

Figure 4: FEMA’s Protection and National Preparedness Organization: 

[Refer to PDF for image: Organizational Chart] 

First Tier: 

Protection and National Preparedness Deputy Administrator: 

Second Tier: 

Senior Counselor: 

Office of Counterterorism and Security Preparedness Director: 

Office of Preparedness Integration and Coordination Director: 

Senior Policy Advisor: 

Strategic Resource Management Office Director: 

Preparedness Taskforce Executive Director: 

Third Tier: 

National Preparedness Directorate, Assistant Deputy Administrator: 

National Community Program Directorate, Assistant Administrator: 

Grants Program Directorate, Assistant Deputy Administrator: 

National Capital Region Coordination, Director: 

Fourth Tier: 

National Preparedness Directorate, Assistant Deputy Administrator: 

Individual and Community Preparedness Division, Director; 
National Exercise Division, Director; 
National Integration Center, Assistant Director; 
Technological Hazards Division, Director; 
National Preparedness Assessment Division Director. 

National Preparedness Directorate, Assistant Deputy Administrator: 
National Training and Education, Assistant Administrator: 

Center for Domestic Preparedness, Superintendent; 
Emergency Management Institute, Superintendent; 
National Training and Education Division, Deputy Assistant 
Administrator. 

National Community Program Directorate, Assistant Administrator: 

Continuity of Operations Division, Director; 
Operations Division, Director; 
Readiness Division, Director; 
Integrated Public Alert Warning System Division, Director; 
Mount Weather Emergency Operations Center, Deputy Executive 
Administrator. 

Grants Program Directorate, Assistant Deputy Administrator: 

Grants Development and Administration Division, Director; 
Grants Management Division, Director. 

National Capital Region Coordination, Director: 

Operations Coordination Division, Associate Director; 
Preparedness, Planning, and Readiness Division, Associate Director; 
Risk Reduction Division, Team Leader; 
Technology Integration Division, Associate Director. 

Source: FEMA. 

[End of figure] 

Appendix II: 

Table 1: National Preparedness Assessment Responsibilities: 

Organization: Local, State, Tribal, and Federal Preparedness Task 
Force; 
Define Preparedness metrics/requirements: Serve as a collaborative body 
for considering input from preparedness community about current policy, 
and FEMA's strategic goals; recommend preparedness assessment metrics; 
Data collection: No Responsibility; 
Data analysis: No Responsibility; 
Report products and outputs: Provide feedback on the analysis and 
conclusions provided by the FEMA; 
FEMA--issued improvements and recommendations: Review current policy, 
communicate with internal stakeholders, make recommendations on how to 
better define preparedness. 

Organization: Reporting Requirements Working Group[A]; 
Define Preparedness metrics/requirements: Identify where duplicative 
efforts may exist among FEMA's data--collection initiative; 
Data collection: Consult on data collection approach; 
Data analysis: No Responsibility; 
Report products and outputs: Provide feedback on the analysis and 
conclusions provided by FEMA; 
FEMA--issued improvements and recommendations: Share feedback from 
stakeholders and provide recommendation for improved data collection. 

Organization: FEMA's National Preparedness Directorate; 
Define Preparedness metrics/requirements: Define the overall 
preparedness assessment process, coordinate assessment efforts among 
internal stakeholders, identify data required to develop defined 
preparedness metrics; 
Data collection: Collect, manage and standardize data sets for 
analysis, coordinate data collection for all preparedness measurement; 
Data analysis: Integrate the most applicable and effective methods of 
assessment into its analysis process; 
Report products and outputs: Generate ad hoc and standard product and 
service outputs bases on analysis of the preparedness data. Communicate 
findings based on analytic results; 
FEMA--issued improvements and recommendations: Refine approach and 
methodologies of the assessment process based on feedback from internal 
and external stakeholders, including the recommendations of the 
Reporting Requirements Working Group. 

Source: GAO analysis of FEMA data. 

[A] Note: According to the Deputy Administrator for Protection and 
National Preparedness, in June 2010 FEMA discontinued the working group 
effort after achieving its purpose to consult with state, local, and 
tribal stakeholders related to streamlining efforts. 

[End of table] 

Appendix III: 

Table 2: FEMA Has Fully Implemented 3 of the 11 Actions Recommended in 
Our April 2009 Report: 

Action 1: Develop a program management plan to ensure the completion of 
the key national preparedness policies and plans; 
Current Status of recommendation: Action not fully implemented. 

Action 2: Develop policies for issuing after-action reports for 
National Level Exercises in 6 months or less; 
Current Status of recommendation: Action not fully implemented. 

Action 3: Establish policies and procedures for documenting corrective 
actions from Principal-Level Exercises; 
Current Status of recommendation: Action partially implemented. 

Action 4: Obtain information from past principal-level exercises and 
conduct remedial action tracking and long-term trend analysis; 
Current Status of recommendation: Action fully implemented. 

Action 5: Revise FEMA's grant monitoring guidance to ensure states' 
compliance with Homeland Security Exercise and Evaluation requirements; 
Current Status of recommendation: Action not fully implemented. 

Action 6: Involve the National Council on Disability on committees 
involved in the design and execution of national-level exercises, 
especially on issues related to population with special needs; 
Current Status of recommendation: Action fully implemented. 

Action 7: Develop internal control policies and procedures that 
validate the completeness and accuracy of data used to measure National 
Exercise Program performance; 
Current Status of recommendation: Action not fully implemented. 

Action 8: Revise the National Council on Disability on committees 
involved in the design and execution of national--level exercises, 
especially on the issues related to population with special needs; 
Current Status of recommendation: Action not fully implemented. 

Action 9: Develop procedures for including "lessons learned" from real-
world incidents in the Corrective Action Program system; 
Current Status of recommendation: Action not fully implemented. 

Action 10: Enhance FEMA's project management plan for developing a 
comprehensive assessment system; 
Current Status of recommendation: Action fully implemented. 

Action 11: Develop a strategic plan for implementing the national 
preparedness system that includes the key characteristics of a 
strategic plan; 
Current Status of recommendation: Action not fully implemented. 

Source: GAO analysis of FEMA data. 

Note: Our April 2009 report was: GAO, National Preparedness: FEMA Has 
Made Progress, but Needs to Complete and Integrate Planning, Exercise, 
and Assessment Efforts, GAO-09-369 (Washington, D.C.: Apr. 30, 2009). 

[End of section] 

Enclosure II: Comments from the Department of Homeland Security: 

U.S. Department of Homeland Security: 
Washington, DC 20528: 

Homeland Security: 

Mr. Bill Jenkins: 
Director, Homeland Security and Justice: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Jenkins: 

Subject: Draft Report GAO-11-51R, FEMA Has Made Limited Progress in 
Efforts to Develop and Implement a System to Assess National 
Preparedness. capabilities (Job Code 440843): 

The Department of Homeland Security (DHS) appreciates the opportunity 
to review and comment on the draft report referenced above. As the 
Federal Emergency Management Agency (FEMA) works towards refining its 
programs, the GAO's analysis of our program performance greatly 
benefits our ability to continually improve our activities. 

FEMA believes that it has made much progress since 2009 in meeting the 
Post-Katrina Emergency Management Reform Act of 2006 (PKEMRA) and other 
legislative requirements and would like to take this opportunity to 
highlight some of its specific achievements. FEMA has taken the 
following actions since GAO's original assessment to address many of 
GAO's findings: 

* Established a congressionally-mandated Local, State, Tribal and 
Federal Task Force that is charged with "making recommendations for all 
levels of government regarding: disaster and emergency guidance and 
policy, federal grants; and federal requirements, including measuring 
efforts. The task force will also evaluate... the most appropriate way 
to collectively assess our capabilities and our capability gaps." 

* Put an immediate moratorium on any new reporting/assessment systems. 

* Established a Reporting Requirement Working Group (RRWG) with 
representatives from each FEMA Directorate/Office, the Regions, and 
national stakeholder groups to work toward consolidation of the various 
reporting requirements/systems and to develop an integrated approach 
for assessing national preparedness. 

* The RRWG has since concluded its work, and FEMA is working on 
implementing the recommendations, including the following examples: 

- Combining all FEMA data into a central data warehouse that may be 
easily queried by multiple jurisdictions. This will reduce redundancy. 

- Putting a policy in place restricting new data collections as well as 
establishing a new process internal to FEMA. 

- Analyzing the approximately 102 data requirements for redundancy and 
opportunities for consolidation. 

* Halted all activity on the Cost to Capability (C2C) project, and 
consolidated it with the Comprehensive Assessment System, as outlined 
in the Post-Katrina Emergency Management Reform Act of 2006 (13KEMRA). 

* While the Target Capabilities List (TCL) 2.0 is still in use (through 
grant guidance, exercises, state preparedness reports, etc.), halted 
further development of TCL 3.0 pending recommendations from the 
Preparedness Task Force. 

* Transitioned the FEMA/Response Gap Analysis Program (GAP) from an 
assessment/reporting system to a tool that assists in the development 
of catastrophic, evacuation, transportation, emergency communications, 
and other planning initiatives. 

* Transformed the State Preparedness Reports from narrative based 
reports to more quantitative assessments. 

* Consolidated five congressionally mandated reporting requirements 
into one National Preparedness Report, which is currently in the 
clearance stage. 

* Working with the National Security Staff to ensure revisions to 
preparedness policy reconcile past conflicts and more clearly 
articulate assessment requirements. 

• Recently reorganized the National Preparedness Directorate (NPD), 
including: 

- Transferring the preparedness policy, planning and technical 
assistance staff, resources, and functions out of NPD's Preparedness 
Policy, Planning and Analysis, and renaming it to the National 
Preparedness Assessment Division which: 1) focuses the division 
strictly on assessments and reporting; and 2) more closely aligns 
programs such as the TCL with similar activities (e.g., resource 
typing, credentialing, etc.) in NPD's National Integration Center. 

- Transferring the grant assessments staff and activities from Grant 
Programs Directorate (GPD) to NPD to more closely align activities such 
as grant effectiveness with preparedness assessments. 

* Consolidating the TCL with the National Incident Management System 
(NIMS) Resource Typing and Credentialing to develop better performance 
measures that are more output-based, and are based on hazard 
identification and risk assessments. Thank you again for the 
opportunity to comment on this Draft Report. We look forward to working 
with you on other issues as we both strive to improve FEMA. 

Sincerely, 

Signed by: 

Jerald E. Levine: 
Director: 
Departmental GAO-OIG Liaison Office: 

[End of section] 

Footnotes: 

[1] H.R. Rep. No. 111-298, at 109-110 (Conf. Rep.) The conference 
report accompanied the Department of Homeland Security Appropriations 
Act, 2010, Pub. L. No. 111-83, 123 Stat. 2142 (2009). 

[2] GAO, National Preparedness: FEMA Has Made Progress, but Needs to 
Complete and Integrate Planning, Exercise, and Assessment Efforts, 
[hyperlink, http://www.gao.gov/products/GAO-09-369] (Washington, D.C.: 
Apr. 30, 2009). 

[3] The Post-Katrina Emergency Management Reform Act was enacted as 
Title VI of the Department of Homeland Security Appropriations Act, 
2007, Pub. L. No. 109-295, 120 Stat. 1355 (2006). The act defines 
capability as "the ability to provide the means to accomplish one or 
more tasks under specific conditions and to specific performance 
standards." Id. at § 641, 120 Stat. at 1424 (codified at 6 U.S.C. § 
728). 

[4] Homeland Security Presidential Directive 8 (HSPD-8)—National 
Preparedness (Dec. 17, 2003). 

[5] The Post-Katrina Act was enacted as Title VI of the Department of 
Homeland Security Appropriations Act, 2007, Pub. L. No. 109-295, 120 
Stat. 1355 (2006). The act defines capability as “the ability to 
provide the means to accomplish one or more tasks under specific 
conditions and to specific performance standards.” 

[6] Pub. L. No. 109-295, § 649, 120 Stat. 1355, 1428 (2006) (codified 
at 6 U.S.C. § 749). 

[7] GAO, National Preparedness: FEMA Has Made Progress, but Needs to 
Complete and Integrate Planning, Exercise, and Assessment Efforts, 
[hyperlink, http://www.gao.gov/products/GAO-09-369] (Washington, 1 
D.C.: Apr. 30, 2009). 

[8] FEMA established a Target Capabilities List—a list of 37 specific 
preparedness capabilities related to the four homeland security mission 
areas: Prevent, Protect, Respond, and Recover--intended as planning 
guidance for state, tribal, and local stakeholders. It defines and 
provides the basis for assessing preparedness. 

[9] H.R. Rep. No. 111-298, at 109-110 (Conf. Rep.). The conference 
report accompanied the Department of Homeland Security Appropriations 
Act, 2010, Pub. L. No. 111-83, 1 123 Stat. 2142 (2009). 

[10] FEMA established a Reporting Requirements Working Group consisting 
of 41 officials from FEMA, state, local, and tribal governments, and 
the National Emergency Management Association to develop an agencywide 
information-gathering system that is fielded to state, local, tribal, 
and territorial governments and provide recommendations for 
streamlining data requests. 

[11] The Conference Report accompanying the 2010 Department of Homeland 
Security appropriations act called for a Task Force responsible for 
"making recommendations for all levels of government regarding: 
disaster and emergency guidance and policy; federal grants; and federal 
requirements, including measuring efforts." H.R. Rep. No. 111-298, at 
102 (2009) (Conf. Rep.). This Task Force is comprised of 35 members of 
federal, state, local, and tribal governments. 

[12] The International Association of Emergency Managers is a nonprofit 
educational organization dedicated to promoting the goals of saving 
lives and protecting property during emergencies and 1
disasters. The National Advisory Council was established by the Post-
Katrina Act to advise the FEMA Administrator on all aspects of 
emergency management, incorporating state, local, and tribal government 
and private-sector input in the development and revision of national 
preparedness policies and plans. Pub. L. No. 109-295, § 508, 120 Stat. 
1355, 1403 (2006) (codified at 6. U.S.C. § 318). The National Council 
on Disability is an independent federal agency that provides advice to 
the President, Congress, and executive branch agencies to promote 
policies for individuals with disabilities. The National Emergency 
Management Association is a nonprofit, nonpartisan association of 
emergency management and homeland security professionals. 

[13] One territory did not respond, according to FEMA officials. 

[14] According to HSPD-5, the purpose is to enhance the ability of the 
United States to manage domestic incidents by establishing a single, 
comprehensive national incident management system. 

[15] Homeland Security Presidential Directive 8 (HSPD-8) directs the 
Secretary of Homeland Security to report annually on the nation’s level 
of the preparedness, including state capabilities. The report is 
referred to as the National Preparedness Report (Dec. 17, 2003). 

[16] FEMA utilized the following four scenarios of catastrophic events 
in its preparedness assessment: (1) influenza pandemic, (2) category 3 
and 5 hurricanes, (3) terrorist use of explosives, and (4) improvised 
nuclear devices. 

[17] [hyperlink, http://www.gao.gov/products/GAO-09-369]. 

[18] Project Management Institute, A Guide to the Project Management 
Body of Knowledge (PMBOK Guide), 4th ed. (Newton Square, Pa. 2008). 

[19] [hyperlink, http://www.gao.gov/products/GAO-09-369]. 

[20] The Emergency Management Institute is FEMA’s national emergency 
management training, exercising, and education institution. The 
institute promotes integrated emergency management principles and 
practices through application of the National Response Framework, 
National Incident Management System, and an all-hazards approach. 

[21] The Emergency Management Accreditation Program is a voluntary 
review process for state and local emergency management programs. 
Created by a group of national organizations to foster continuous 
improvement in emergency management capabilities, the program provides 
emergency management programs the opportunity to be recognized for 
compliance with national standards, to demonstrate accountability, and 
to focus attention on areas and issues where resources are needed. 

[22] The National Emergency Management Association is a nonprofit, 
nonpartisan association of emergency management and homeland security 
professionals. 

[23] H.R. Rep. No. 111-298, at 102 (2009) (Conf. Rep.). 

[24] The National Exercise Program provides an organized approach to 
set priorities for exercises, reflect those priorities in a multiyear 
schedule of exercises that serves the strategic and policy goals of the 
U.S. government, and address findings from those exercises through a 
disciplined interagency process. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: