Disaster Cost Estimates:

FEMA Can Improve Its Learning from Past Experience and Management of Disaster-Related Resources

GAO-08-301: Published: Feb 22, 2008. Publicly Released: Feb 22, 2008.

Additional Materials:

Contact:

Susan J. Irving
(202) 512-9142
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Public Law No. 110-28 directed GAO to review how the Federal Emergency Management Agency (FEMA) develops its disaster cost estimates. Accordingly, GAO addressed the following questions: (1) What is FEMA's process for developing and refining its cost estimates for any given disaster? (2) From 2000 through 2006, how close have cost estimates been to the actual costs for noncatastrophic (i.e., federal costs under $500 million) natural disasters? (3) What steps has FEMA taken to learn from past experience and improve its management of disaster-related resources and what other opportunities exist? To accomplish this, GAO reviewed relevant FEMA documents and interviewed key officials. GAO also obtained and analyzed disaster cost data and determined that they were sufficiently reliable for the purposes of this review.

After a disaster is declared, FEMA staff deployed to a joint field office work with state and local government officials and other relevant parties to develop and refine cost estimates. The overall estimate comprises individual estimates for FEMA's assistance programs plus any related tasks assigned to other federal agencies (mission assignments) and FEMA administrative costs. The methods used to develop these estimates differ depending on program requirements including, in some cases, historical knowledge. FEMA officials told GAO that cost estimates are updated on a continuing basis. Decision makers need accurate information to make informed choices and learn from past experience. FEMA officials stated that by 3 months after a declaration estimates are usually within 10 percent of actual costs--which they defined as reasonable. GAO's analysis showed that decision makers did not have cost information within this 10 percent band until 6 months after the disaster declaration. These results cannot be generalized since this comparison could only be made for the 83 (24 percent) noncatastrophic natural disaster declarations for which final financial decisions had been made. Disaster coding issues also hamper FEMA's ability to learn from past experience. For example, in several instances the code for the incident type and the description of the disaster declaration did not match. Officials described several ways in which FEMA has learned from past disasters and improved its management of disaster-related resources. For example, FEMA uses a national average to predict costs for expected applicants for Individual Assistance. FEMA has also taken several actions to professionalize and expand the responsibilities of its disaster comptrollers. Nonetheless, FEMA could further learn from past experience by conducting sensitivity analyses to identify the marginal effect various factors have on causing fluctuations in its estimates. FEMA could improve its management of disaster-related resources by developing standard procedures for staff involved in entering and updating cost estimate data in its database.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: FEMA has not yet completed this analysis, as the Office of the Chief Financial Officer (OCFO) had been attempting to bolster its analytical skills by hiring a skilled operations research (OR) analyst. The OR analyst would complete this analysis as well as other analytical studies required to aid the agency's cost estimation process. OCFO is went through three announcement announcement cycles and did not find a suitable candidate. Additionally, Congressional Committee report language directs FEMA to hire a consultant to evaluate its cost estimating process. They would not begin any sensitivity analysis before they have begun working with a contractor to evaluate cost estimating processes.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to conduct sensitivity analyses to determine the marginal effects of key cost drivers to provide a range for the uncertainty created by factors beyond FEMA's control.

    Agency Affected: Department of Homeland Security

  2. Status: Closed - Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA review the effect that their own processes have on fluctuations in disaster cost estimates and take steps to limit the impact they have on estimates. FEMA contracted with International Business Machines (IBM) to conduct an objective assessment of the agency's process for formulating disaster cost estimates, and the results are captured in "As-Is" and "To-Be" reports, which were issued on February 27, 2008 and June 16, 2008, respectively. The "To-Be" report presents the recommendations of IBM, with the goal being to improve future disaster-level and aggregate fund level cost estimates. Consistent with GAO's recommendations, IBM's key recommendations that would improve FEMA's processes for disaster cost estimates include aligning cost assessment skills with hiring, training, and the organizational structure; integrating variance analysis and response capability with current cost assessment processes and related tools; developing process output metrics for the Disaster Projection Report (DPR), Disaster Financial Status Report (DFSR), Disaster Relief Fund (DRF) Spend Plan, and DRF Burn Rate Report; providing historical costs summary reports to stakeholders and incorporating historical qualitative disaster information for the DPR; and adding tool functionality to the DFSR to allow calculation of actual total projection amount change from month to month. Per a 6/28/10 discussion with FEMA Director Financial Systems & Management Reporting, a contractor is now redeveloping their database in order to address these improvements.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to review the effect FEMA's own processes have on fluctuations in disaster cost estimates and take steps to limit the impact they have on estimates.

    Agency Affected: Department of Homeland Security

  3. Status: Closed - Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA review the reasons why it takes 6 months or more for estimates to reasonably predict actual costs and focus on improving them to shorten the time frame. FEMA contracted with International Business Machines (IBM) to conduct an objective assessment of the agency's process for formulating disaster cost estimates, and the results are captured in "As-Is" and "To-Be" reports, which were issued on February 27, 2008 and June 16, 2008, respectively. The "To-Be" report presents the recommendations of IBM, with the goal being to improve future disaster-level and aggregate fund level cost estimates. Consistent with GAO's recommendations, IBM's key recommendations that would improve cost estimates include aligning cost assessment skills with hiring, training, and the organizational structure; integrating variance analysis and response capability with current cost assessment processes and related tools; developing process output metrics for the Disaster Projection Report (DPR) and Disaster Financial Status Report (DFSR); providing historical costs summary reports to stakeholders and incorporating historical qualitative disaster information for the DPR; and adding tool functionality to the DFSR to allow calculation of actual total projection amount change from month to month. Per a 6/28/10 discussion with FEMA Director Financial Systems & Management Reporting, a contractor is now redeveloping their database in order to address these improvements.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to review the reasons why it takes 6 months or more for estimates to reasonably predict actual costs and focus on improving them to shorten the time frame.

    Agency Affected: Department of Homeland Security

  4. Status: Closed - Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA undertake efforts - similar to those FEMA used to develop its model to predict hurricane costs - to better predict costs for other types of disasters, informed by historical costs and other data. FEMA contracted with International Business Machines (IBM) to conduct an objective assessment of the agency's process for formulating disaster cost estimates, and the results are captured in "As-Is" and "To-Be" reports, which were issued on February 27, 2008 and June 16, 2008, respectively. The "To-Be" report presents the recommendations of IBM, with the goal being to improve future disaster-level and aggregate fund level cost estimates. Consistent with GAO's recommendations, IBM's key recommendation that would incorporate historical costs and other data in disaster cost estimates is to provide historical costs summary reports to stakeholders and incorporate historical qualitative disaster information for the Disaster Projection Report (DPR). This will help comptrollers assess estimates from program staff, and it will enable FEMA to understand, analyze, and benchmark the estimates being developed and assist in the refinement of cost estimates over time. Per a 6/28/10 discussion with FEMA Director Financial Systems & Management Reporting, a contractor is now redeveloping their database in order to address these improvements.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to undertake efforts--similar to those FEMA used to develop its model to predict hurricane costs--to better predict costs for other types of disasters, informed by historical costs and other data.

    Agency Affected: Department of Homeland Security

  5. Status: Closed - Not Implemented

    Comments: According to the Director of FEMA's Financial Systems and Management Reporting (OCFO), as of May 2012, this evaluation is still under review. Additionally, FEMA has no plans to begin such an analysis until a contractor to complete a Congressionally mandated study of cost estimating process has been hired. This analysis may be part of that study.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to evaluate the benefits of using geographically specific averages in addition to national averages to better project Individual Assistance costs.

    Agency Affected: Department of Homeland Security

  6. Status: Closed - Not Implemented

    Comments: According to the Director of FEMA's Director of Financial Systems and Management reporting, after some deliberation FEMA has decided not to resume distributing surge costs to individual disasters.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to resume the distribution of surge account costs to individual disasters, as appropriate, to make cost data from past, current, and future disasters comparable.

    Agency Affected: Department of Homeland Security

  7. Status: Closed - Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA review and revise incident coding types to ensure that they are accurate and useful for learning from past experience, so that the codes match the descriptions and are consistently entered to reflect what occurred. As described in emails from Acting Special Assistant, FEMA Recovery Directorate, on July 7 and 15, 2010, the agency has taken actions that address this recommendation. FEMA Declarations Unit staff are solely responsible for entering the incident in the National Emergency Management Information System (NEMIS) exactly as it appears in the declaration letters signed by the President. Each staff member has been trained/advised of the responsibility, and has been provided an internal desktop reference as a check and balance to ensure that this action is taken for each gubernatorial request. The internal desktop reference is an 'eyes only' reference that is used for the entire declaration process, including the steps to be followed in NEMIS. The agency has not changed incident codes, but instead chose to implement the previously mentioned improvements as a more time and cost effective method to address the issue.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to review and revise incident coding types to ensure that they are accurate and useful for learning from past experience. At a minimum, incident codes should match the descriptions and be consistently entered and reflect what occurred, which may require permitting multiple incident types for each declaration.

    Agency Affected: Department of Homeland Security

  8. Status: Closed - Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA develop training and standard operating procedures for all staff entering incident type and cost information into the Disaster Financial Status Report (DFSR) database. In response to GAO?s report, FEMA staff took action in early 2010 by reclassifying the incident types in the DFSR database so that the incident types agree with those in the declaration model in National Emergency Management Information System (NEMIS). The process has been standardized so that first, the Declaration Unit determines the disaster type and enters this data into the declaration model; and second, the Disaster Relief Fund (DRF) Oversight Team ensures that the data in the DFSR database agree with the declaration model.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to develop training and standard operating procedures for all staff entering incident type and cost information into the DFSR database.

    Agency Affected: Department of Homeland Security

  9. Status: Closed - Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA review reasons why older disasters remain open and take action to close/reconcile them if possible. On November 7, 2009, the FEMA Administrator directed the FEMA Chief Financial Officer (CFO) to lead an agency-wide effort aimed at closing open disasters and emergencies and fire management assistance grants. The Executive Officer from FEMA's Office of the CFO (OCFO) led the Administrator's closeout initiative and formed a Disaster Closeout Workgroup (DCW), which consisted of representatives from the Headquarter (HQ) program offices, mission support offices, and the FEMA regional offices. OCFO tasked the Regional Administrators with providing an Open Disaster Assessment by April 1, 2010. The assessment was conducted to identify the status of all open projects, programs, and FEMA-State Agreements (FSAs); establish target closeout dates; and identify impediments to timely closeout. The DCW will develop consistent closeout procedures and training materials across the agency. HQ program offices are reviewing program timelines and tools to enforce compliance by the states with those timelines. Further, the OCFO will assess the enhancement of its current reporting system for open disasters to incorporate a more robust aging/tracking component. Of the 357 open disasters as of 10/1/2009 (under the purview of the DCW initiative), 118 were closed as of June 28, 2010.

    Recommendation: To better mitigate the effect of factors both beyond and within FEMA's control to improve the information provided to decision makers; to better inform future estimates, including the ability to incorporate past experience in those estimates; and to improve the management of FEMA's disaster-related resources, the Secretary of Homeland Security should instruct FEMA's Administrator to review reasons why "older" disasters remain open and take action to close/reconcile them if possible.

    Agency Affected: Department of Homeland Security

  10. Status: Closed - Not Implemented

    Comments: In February 2008, GAO reported that opportunities exist for the Federal Emergency Management Agency (FEMA) to learn from past disasters and improve its management of disaster-related resources (GAO-08-301). GAO recommended that FEMA work with Congress and OMB to provide more complete information on known costs from prior disasters and costs associated with catastrophic disasters as part of the annual budget request. However, per a 6/28/10 discussion with FEMA Director Financial Systems & Management Reporting, FEMA has been unable to address this item because they would need direction from Congress, but Congress has been unwilling to take on the issue. As a result, GAO will close out this recommendation as not implemented.

    Recommendation: To promote a more informed debate about budget priorities and trade-offs, the Secretary of Homeland Security also should instruct FEMA's Administrator to work with OMB and Congress to provide more complete information on known costs from prior disasters and costs associated with catastrophic disasters as part of the annual budget request.

    Agency Affected: Department of Homeland Security

 

Explore the full database of GAO's Open Recommendations »

Oct 23, 2014

Jul 29, 2014

Jun 6, 2014

May 14, 2014

May 9, 2014

Feb 6, 2014

Jan 22, 2014

Jan 15, 2014

Sep 18, 2013

Looking for more? Browse all our products here