Catastrophic Planning:

States Participating in FEMA's Pilot Program Made Progress, but Better Guidance Could Enhance Future Pilot Programs

GAO-11-383: Published: Apr 8, 2011. Publicly Released: May 9, 2011.

Additional Materials:

Contact:

Brian J. Lepore
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Hurricane Katrina in 2005 highlighted gaps in the nation's preparedness to respond effectively to catastrophic incidents. The Department of Homeland Security's (DHS) Federal Emergency Management Agency (FEMA) designed the Task Force for Emergency Readiness (TFER) pilot program to advance and integrate state and federal catastrophic planning efforts. TFER, first envisioned by the Department of Defense (DOD), ran from September 2008 to September 2010, and FEMA provided the five participating states--Hawaii, Massachusetts, South Carolina, Washington, and West Virginia--with $350,000 each to develop plans, build relationships with stakeholders, and document lessons learned (i.e., TFER's stated objectives). As requested, GAO evaluated the extent to which (1) FEMA followed sound management practices in designing, administering, and evaluating TFER and (2) the five participating states satisfied TFER's stated objectives. GAO analyzed FEMA guidance, such as the TFER Pilot Information Package, conducted site visits to all five participant states, and met with relevant FEMA and DOD officials, to evaluate FEMA's management of TFER and the states' implementation of it.

FEMA developed program objectives and procedures for administering the TFER pilot, but did not develop other elements of sound management practices in designing, administering, and evaluating pilot programs that GAO identified from its prior work and social science literature. FEMA developed objectives for the pilot, but did not document standards for determining the pilot's success. FEMA also provided resources such as funding, training, and support, but FEMA did not always follow the procedures it established for TFER. For example, FEMA did not consistently conduct biweekly conference calls with the states, and four states reported that it would have been helpful if FEMA provided more guidance. FEMA did not develop a data analysis plan, which could have better ensured FEMA collected data on the extent to which the pilot states coordinated with key stakeholders and provided reasonable assurance that FEMA conducted a systematic assessment of TFER using comparable data across the five pilot states. In Spring 2010, FEMA announced TFER would not continue past the pilot stage before evaluating TFER on its merits in strengthening and advancing state catastrophic planning, but FEMA allowed the states to draw down the remaining TFER funds and continue to complete initiatives started under TFER through August 2011. GAO previously reported in April 2009 that FEMA faced challenges in assessing pilot program data, and FEMA officials reported the agency does not have pilot program policy guidance. In the absence of this, FEMA lacks a systematic approach to developing, administering, and evaluating pilot programs. FEMA could better ensure other pilot programs meet their intended goals by developing policies and guidance that include sound management practices. All five states have taken steps to follow FEMA's guidance to address TFER's objectives, but no state has fully addressed them all. First, two of the five states have completed draft catastrophic plans, and all five states reported following FEMA's planning process. Second, all five states built relationships with stakeholders such as state agencies and FEMA, but state officials said coordination with DOD--a key federal stakeholder who may be called upon to assist in disaster response--was limited. State officials reported not coordinating with DOD because they did not have draft plans for DOD officials to review. Third, all states have documented lessons learned to date, but four states have not exercised TFER plans to determine their effectiveness in the event of an emergency. GAO recommends that FEMA develop policies and guidance that follow sound management practices for future pilot programs, and share TFER results with stakeholders. FEMA agreed with GAO's recommendations.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: In August 2013, Federal Emergency Management Agency (FEMA) National Preparedness Directorate (NPD) officials reported that they do not plan to develop FEMA-wide policies regarding the piloting of programs because it is not within NPD's authority to develop such guidance. Instead, these officials reported that NPD will adhere to the policies and procedures developed by the Grant Programs Directorate for any future state or local grant pilot programs. Further, these officials reported that future pilot efforts that result in the development of doctrine are to use FEMA Directive 112-12, Policy, Directive, and Doctrine Process Guidance. However, FEMA 112-12 does not include specific policies and guidance for pilot programs. As a result, we consider this recommendation to be closed as not implemented.

    Recommendation: To help ensure future pilot programs achieve their intended results and provide the performance information needed to make effective management decisions for broader implementation, the Administrator, Federal Emergency Management Agency, should develop and implement policies and guidance for pilot programs that follow sound management practices. This guidance should include, at a minimum, requirements for (1) a clearly articulated methodology with objectives reflective of overall program goals and standards for determining program performance; (2) procedures for monitoring program performance; (3) a data collection plan; (4) a data analysis plan; and (5) a process to disseminate the results and lessons learned that emerge from the pilot.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

  2. Status: Closed - Implemented

    Comments: We found that the Federal Emergency Management Agency (FEMA) developed program objectives and procedures for administering the Task Force for Emergency Readiness (TFER) pilot, but did not develop other elements of sound management practices in designing, administering, and evaluating pilot programs that GAO identified from its prior work and social science literature. For example, according to sound management practices, results and recommendations that emerge from evaluations must be disseminated in ways that meet the needs of stakeholders, but FEMA did not commit to sharing the TFER evaluation results with stakeholders. We recommended that FEMA disseminate the evaluation results and recommendations that emerge from the TFER pilot program in ways that meet the needs of current and future stakeholders. In response, FEMA developed a TFER pilot final report in March 2011 that documents lessons learned and recommendations for improvement. According to FEMA officials, FEMA shared the final report with points of contact at each of the five pilot states, the Defense Science Board, and the Congress. FEMA officials reported that, since the TFER pilot program has ended, there are no future stakeholders with whom to share the final report with. As a result, we believe FEMA has met the intent of our recommendation and consider this recommendation to be closed as implemented.

    Recommendation: To help ensure stakeholders receive valuable information regarding catastrophic preparedness from lessons learned during the TFER pilot program, the Administrator, Federal Emergency Management Agency, should disseminate the evaluation results and recommendations that emerge from the TFER pilot program in ways that meet the needs of current and future stakeholders.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency

 

Explore the full database of GAO's Open Recommendations »

Dec 17, 2014

Dec 12, 2014

Dec 11, 2014

Oct 23, 2014

Jul 29, 2014

Jun 6, 2014

May 14, 2014

May 9, 2014

Feb 6, 2014

Looking for more? Browse all our products here