National Preparedness:

FEMA Has Made Progress, but Needs to Complete and Integrate Planning, Exercise, and Assessment Efforts

GAO-09-369: Published: Apr 30, 2009. Publicly Released: May 29, 2009.

Additional Materials:

Contact:

William O. Jenkins, Jr
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Hurricane Katrina was the most destructive disaster in our nation's history and it highlighted gaps in preparedness for a catastrophic disaster. The Federal Emergency Management Agency (FEMA), a component within the Department of Homeland Security (DHS), is the lead federal agency responsible for developing a national preparedness system. The system includes policies and plans as well as exercises and assessments of capabilities across many public and private entities. GAO was asked to assess the extent to which FEMA has (1) developed policies and plans that define roles and responsibilities; (2) implemented the National Exercise Program, a key tool for examining preparedness; (3) developed a national capabilities assessment; and (4) developed a strategic plan that integrates these elements of the preparedness system. GAO analyzed program documents, such as after-action reports, and visited six states located in disaster regions. While the results of these visits are not generalizable, they show how select states carry out their efforts.

While most policies (41 of 50) that define roles and responsibilities have been completed, such as the National Response Framework, 68 percent (49 of 72) of the plans to implement these policies, including several for catastrophic incidents, are not yet complete. As a result, the roles and responsibilities of key officials involved in responding to a catastrophe have not been fully defined and, thus, cannot be tested in exercises. The lack of clarity in response roles and responsibilities among the diverse set of responders contributed to the disjointed response to Hurricane Katrina and highlighted the need for clear, integrated disaster preparedness and response policies and plans. Although best practices for program management call for a plan that includes key tasks and their target completion dates, FEMA does not have such a plan. With such a plan, FEMA would be better positioned to ensure that the policies and plans are completed and integrated with each other as intended as well as with other elements of the preparedness system. Since 2007, FEMA has taken actions to implement the National Exercise Program at the federal and state levels by developing, among other things, program guidance and systems to track corrective actions; however, FEMA faces challenges in ensuring that the exercises are carried out consistent with program guidance. For example, the Homeland Security Council (an interagency entity responsible for coordinating homeland security policy) and state participants did not systematically track whether corrective actions had been taken to address deficiencies identified by exercises as called for by program guidance. As a result, FEMA lacks reasonable assurance that entities have taken actions aimed at improving preparedness. FEMA has made progress in developing a system for assessing national preparedness capabilities by, among other things, establishing reporting guidance for state preparedness, but it faces challenges in completing the system and required reports to assess preparedness. While FEMA has developed a project management plan for the new system, the plan does not fully identify milestones and program risks for developing quantifiable metrics necessary for measuring preparedness capabilities. A more complete project plan that identifies milestones and program risks would provide FEMA with greater assurance that it can produce a system to assess capabilities and inform decisions related to improving national preparedness. FEMA's strategic plan for fiscal years 2008-2013 recognizes that each of its components need to develop its own strategic plans that integrate the elements of national preparedness. FEMA's National Preparedness Directorate has yet to develop its strategic plan, but instead plans to use a draft annual operating plan to guide its efforts. This plan does not include all elements of a strategic plan, such as how the directorate will integrate the various elements of the system over time to improve national preparedness. Having a strategic plan would provide FEMA with a roadmap for addressing the complex task of guiding and building a national preparedness system.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the Disaster Operations Directorate and the National Preparedness Directorate to improve their approach to developing policies and plans that define roles and responsibilities and planning processes by developing a program management plan, in coordination with DHS and other federal entities, to ensure the completion of the key national preparedness policies and plans called for in legislation, presidential directives, and existing policy and doctrine, to define roles and responsibilities and planning processes, as well as to fully integrate such policies and plans into other elements of the national preparedness system. The program management plan, among other things, should (1) identify the specific schedule of activities that needs to be performed to complete, and identify dependencies among, all policy and planning development and integration activities; (2) identify the type and quantities of resources required to perform, and the schedule for completing, all policy and planning development and integration activities; (3) analyze activity sequences, durations (including the time required to partner and coordinate on an interagency basis with other federal entities), resource requirements, and schedule constraints to create and update the individual policy and plan development project schedules; and (4) control for changes to the project schedules precipitated by outside forces. When outstanding policies and plans are completed, FEMA should also integrate them into training and exercise efforts to ensure that roles and responsibilities are fully communicated and fully understood by emergency response stakeholders.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: FEMA's National Preparedness Directorate (NPD) issued a 2011 report on its Strategic Intent for Fiscal Year 2012-2014/Fiscal Year 2012 Operating Plan. The Operating Plan focuses NPD's efforts primarily on implementation of Presidential Policy Directive-8 and addresses development of a National Preparedness Goal, National Preparedness System, 5 planning frameworks, and a National Preparedness Report. The Plan includes many of the elements outlined in GAO's recommendation, including goals, timelines, milestones and measurements of progress, as well as resources associated with implementing the plan. As a result, this recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by coordinating with the Department of Homeland Security to develop policies and procedures for issuing after-action reports for National Level Exercises (i.e., TOPOFF (top Officials 4 Exercise)) in 6 months or less, as required by the implementation plan for the National Exercise Program.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: FEMA developed policies and procedures to meet the timelines delineated in the National Exercise Program (NEP) Implementation Plan (I-Plan) concerning the release of After Action Reports (AARs) for National Level Exercises (NLEs) and, according to agency officials, FEMA has been working with Department of Homeland Security leadership to ensure more rapid approval and release of AARs through the NEP. As an example, they noted that the NLE 2010 AAR was completed and sent to the interagency for review and comment within 60 days of the exercise. According to officials, following NLE 11, FEMA's National Exercise Division initiated a 14-day turnaround for a Missions Critical Finding Reports and presented findings to the Domestic Resilience Group (DRG) within that timeframe. This recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by collaborating with the Homeland Security Council to establish policies and procedures for documenting corrective actions from Principal Level Exercises that are consistent with Homeland Security Exercise and Evaluation Program (HSEEP) guidance and the implementation plan for the National Exercise Program.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: In an updated response, FEMA reported in January 2012 that officials met with the National Security Staff (NSS) in June 2011 to explore future processes for Principal Level Exercises (PLEs) corrective actions and to clarify requirements. The NSS informed FEMA that it is committed to ensuring effective identification and tracking of corrective actions from PLEs, and this process is being incorporated into the revised NEP I-Plan. In the meantime FEMA continues to work with NSS to identify and monitor corrective actions from these exercises. For example, recently the NSS and FEMA co-chaired a Corrective Action/Lessons Learned Conference based on issues identified from previous NLEs and (PLEs). During the conference, Federal Departments and Agencies were asked to provide the status on each of their respective corrective actions from these exercises, and follow-up sessions are scheduled to resolve any outstanding items.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by collaborating with the Homeland Security Council to provide FEMA with the information it needs from past principal level exercises to enable it to conduct remedial action tracking and long-term trend analysis, as required by the Post-Katrina Act.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: The Federal Emergency Management Agency (FEMA) reported in September 2010 that the agency has obtained access to after action reports from past Principal Level Exercises, noting that remedial action tracking is done at the National Security Staff level for most Cabinet level exercises and that FEMA is working with the National Security Staff of the Homeland Security Council to explore ways to gather data that may be used by key players that are responsible for assessments to conduct analysis of Principal Level Exercises, while protecting sensitive information. FEMA's response noted that long term trend analysis to identify comprehensive and supportable findings is important; there are individual findings that provide significant near-term value and lead to rapid corrective actions that significantly improve preparedness, noting that the findings are captured and reviewed for action and implementation by the National Security Staff and that FEMA is working with the staff to enhance the corrective action identification and tracking process. As a result, this recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by ensuring compliance by states that receive grant funds with HSEEP requirements by revising FEMA's grant monitoring guidance, for example by including a checklist of specific HSEEP requirements for state validation and certification.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Not Implemented

    Comments: According to the agency response in its 31 U.S. Code section 720 letter, dated September 1, 2010, Federal Emergency Management Agency (FEMA) officials did not concur with this recommendation. They said the Homeland Security Exercise Evaluation Program (HSEEP) is a tool to assist State and local jurisdictions plan, conduct, and evaluate exercises. However, because of the emphasis on "HSEEP compliance" over the past several years, this resource has been perceived as unwieldy and often burdensome to stakeholders; detracting from the ultimate value of HSEEP. FEMA is working to identify all the various reporting requirements that are levied on state/local partners and to ultimately reduce and/or consolidate them so that FEMA is getting the information it requires in a manner that is least burdensome to stakeholders. This recommendation is closed as not implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by involving the National Council on Disability on committees involved in the design and execution of national level exercises, especially on issues related to populations with special needs.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: The Federal Emergency Management Agency (FEMA) reported in September 2010 that the agency has added the National Council on Disability to the invitee list for the annual National Exercise Program planning conferences, including National Level Exercises. According to FEMA, the National Exercise Division has begun collaborative discussions directly with the National Council on Disability and through the FEMA Office of Disability Integration and Coordination on how best to incorporate the unique life experiences of people with disabilities in the design and execution of exercises, including national level exercises.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by developing internal control policies and procedures that validate the completeness and accuracy of data used to measure program performance. Such procedures could involve checking whether states and federal agencies are providing data and information needed to measure the performance of the program.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: Since issuing an initial National Exercise Program (NEP) charter memorandum and related implementation plan in 2007, FEMA issued a new NEP base plan in March 2011 to replace the original charter. In June 2012, FEMA issued an updated NEP implementation plan, which further clarifies the roles and responsibilities of stakeholders, and details specific processes and procedures for carrying out the program. The implementation plan includes guidance on exercise evaluation planning. The implementation plan requires FEMA?s National Exercise Division, in coordination with the federal interagency and FEMA?s National Preparedness Assessment Division, to develop an evaluation plan for each NEP two-year exercise cycle. The evaluation plan is to be approved by the National Security Staff at the White House. Each evaluation plan is to cover three key elements: evaluation methodology, performance measures, and evaluation tools and templates. The implementation plan further notes that for each NEP exercise, the exercise?s sponsoring organization will be responsible for conducting the exercise evaluation, using the methodology, performance measures, and evaluation tools described in the evaluation plan for a given NEP cycle. Exercise Directors are to ensure that all exercise evaluators have completed general and specific exercise evaluation training directed by the Homeland Security Exercise Evaluation Program (HSEEP) guidance, which was revised and reissued in April 2013. Additionally, the implementation plan states that Exercise Directors are to mandate and enforce the use of the approved exercise evaluation guide for the exercise to help provide consistent standards and guidelines for observation, data collection, analysis, and development of exercise-related reports. Finally, the NEP implementation plan notes that the adherence to these exercise evaluation policies and procedures is intended to enable the generation of several analytical products for the benefit of senior leaders, exercise planners, and participants. These analytical products are to include an NEP rolling summary report, exercise lessons learned reports and related corrective action improvement plans, and a concluding report at the end of each 2-year NEP cycle for National Security Staff review. Based on the internal control policies and procedures established in the June 2012 NEP implementation plan, this recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by revising the National Exercise Program Implementation Plan to require the use of FEMA's Corrective Action Program for all federal exercises that involve interagency testing of roles and responsibilities or require that federal agencies submit a report to FEMA on the status of their corrective actions resulting from such exercises.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: Since issuing an initial National Exercise Program (NEP) charter memorandum and related implementation plan in 2007, FEMA issued a new NEP base plan in March 2011 to replace the original charter. In June 2012, FEMA issued an updated NEP implementation plan, which further clarifies the roles and responsibilities of stakeholders, and details specific processes and procedures for carrying out the program. The base plan states that FEMA's National Exercise Division (NED) will manage all logistics, coordination, and administrative tasks for the NEP. The implementation plan includes guidance on information management systems to be used to facilitate the evaluation of exercises, including the Corrective Action Program (CAP) System. According to the implementation plan, FEMA's CAP System is a web-based application that enables users (e.g., NED) to prioritize, track, and analyze improvement plans for corrective actions identified through exercises and real-world events. The implementation plan states that all federal corrective actions developed from NEP exercises will be tracked by the federal interagency through the CAP System, from identification through implementation, unless security/classification restrictions prevent the posting of such information to CAP. Based on the June 2012 NEP implementation plan's requirement that all NEP corrective actions be tracked through the CAP system, this recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Exercise Division to improve its implementation of statutory and program requirements by developing procedures for including "lessons learned" from real-world incidents in the Corrective Action Program system.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: On February 8, 2013, FEMA issued management directive FD107-1 Lessons Learned/Continuous Improvement Program. The directive superseded FEMA's prior Remedial Action Management Program. The directive is intended to establish a single FEMA system for lessons learned information collection, analysis, collaboration, identification of findings, continuous improvement development and tracking, and trend analysis. It establishes policy, assigns responsibilities, and applies to all FEMA components, including FEMA headquarter and regional offices, as well as temporary emergency response field organizations and entities. According to the directive, lessons learned are knowledge and experience derived from actual incidents (i.e., real-world events) as well as from observations and historical study of operations, training, and exercises. Continuous improvement refers to a broad range of improvement actions, processes, and activities that support learning in FEMA and improve operational and programmatic performance. Continuous improvement includes: identification of issues or capability gaps from real-world events, among other things, and resolution of these issues through applied corrective actions; identification and incorporation of best practices from real-world events, among other things; and sharing of lessons learned, best practices, and corrective action information as part of a collaborative learning environment. The Corrective Action Program (CAP) System designed to tracks the status of continuous improvement actions identified through post-event analyses is a component of the Homeland Security Exercise and Evaluation Program developed by DHS. According to the directive, all corrective actions identified for assignment and tracking based on lessons learned (including from real-world events) that are presented to the FEMA Office of Policy and Program Analysis chaired FEMA Continuous Improvement Working Group, shall be submitted for inclusion in the CAP System. Based on the February 2013 management directive's requirement that all corrective actions identified for assignment and tracking as a result of lessons learned, including from real-world events, shall be submitted for inclusion in the CAP System, this recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the Office of Preparedness Policy, Planning, and Analysis to improve its approach for developing a comprehensive assessment system by enhancing its project management plan to include milestone dates, an assessment of risk, and related mitigation strategies for (1) comprehensively collecting and reporting on disparate information sources, (2) developing quantifiable metrics for target capabilities that are to be used to collect and report preparedness information, and (3) reporting on the results of preparedness assessments to help inform homeland security resource allocation decisions.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: In response to our recommendation, the Federal Emergency Management Agency reported in September 2010 that the agency's efforts in implementing the comprehensive assessment system reflected its improved approach. Program officials provided an updated project plan that included milestone dates that included risk assessment and mitigation. As a result, this recommendation is closed as implemented.

    Recommendation: To ensure that key elements of the national preparedness system are developed in a timely and integrated fashion, the Administrator of the Federal Emergency Management Agency should direct the National Preparedness Directorate to take a more strategic approach to developing the national preparedness system by developing a strategic plan for implementing the national preparedness system that includes the key characteristics of a strategic plan, including coordination, integration, and implementation approaches.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

    Status: Closed - Implemented

    Comments: In April 2009, we reported that developing goals and metrics to measure progress towards achieving an integrated system would help FEMA prioritize actions, requirements, and national investments in preparedness and that a strategic plan for the National Preparedness Directorate that describes how it will approach these challenges and mitigate these weaknesses would help FEMA partner with the many organizations whose cooperation and resources are necessary for success. We recommended that FEMA develop a strategic plan for implementing the national preparedness system that includes the key characteristics of a strategic plan, including coordination, integration, and implementation approaches. In response, NPD issued a 2011 report on its Strategic Intent for Fiscal Year 2012-2014. The report defines the mission and responsibilities of the Directorate, its strategic priorities, and the major outcomes anticipated from each of the priorities for the years covered by the plan, many of the program management elements called for in this recommendation. As a result of these efforts to describe and document a strategic approach, FEMA's oversight and management of national preparedness efforts can better ensure effective participation and implementation by federal, state and local stakeholders.

    Jul 16, 2014

    Jun 27, 2014

    Jun 24, 2014

    Jun 23, 2014

    Jun 18, 2014

    Jun 16, 2014

    Jun 11, 2014

    Jun 10, 2014

    Looking for more? Browse all our products here