This is the accessible text file for GAO report number GAO-09-68 
entitled 'Wildland Fire Management: Interagency Budget Tool Needs 
Further Development to Fully Meet Key Objectives' which was released on 
November 25, 2008. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Committee on Energy and Natural Resources, 

U.S. Senate: 

United States Government Accountability Office: 

GAO: 

November 2008: 

Wildland Fire ManAgement: 

Interagency Budget Tool Needs Further Development to Fully Meet Key 
Objectives: 

Fire Program Analysis: 

GAO-09-68: 

GAO Highlights: 

Highlights of GAO-09-68, a report to the Chairman, Committee on Energy 
and Natural Resources, U.S. Senate. 

Why GAO Did This Study: 

Wildland fires have become increasingly damaging and costly. To deal 
with fire’s threats, the five federal wildland fire agencies—the Forest 
Service in the Department of Agriculture and four agencies in the 
Department of the Interior (Interior)—rely on thousands of 
firefighters, fire engines, and other assets. To ensure acquisition of 
the best mix of these assets, the agencies in 2002 began developing a 
new interagency budget tool known as fire program analysis (FPA). FPA 
underwent major changes in 2006, raising questions about its ability to 
meet its original objectives. GAO was asked to examine (1) FPA’s 
development to date, including the 2006 changes, and (2) the extent to 
which FPA will meet its objectives. To do so, GAO reviewed agency 
policies and FPA documentation and interviewed agency officials. 

What GAO Found: 

FPA is both a computer model and a broader management system for 
developing the five agencies’ wildland fire budget requests and 
allocating funds. FPA is intended to allow the agencies to analyze 
potential combinations of firefighting assets and potential strategies 
for reducing vegetation and fighting fires to determine the most cost-
effective mix of assets and strategies. The agencies began developing 
FPA in 2002 and completed the first part of the model in October 2004. 
As the agencies began using FPA, however, agency officials raised 
concerns about its underlying science and the extent to which it met 
agency management and policy objectives. As a result, in 2006 the 
agencies conducted a review of FPA, which questioned FPA’s basic 
modeling approach. The agencies made substantial changes to FPA after 
the review, some of which followed from the review’s recommendations. 
For example, as recommended, the agencies established a new oversight 
body comprising senior agency leaders. The agencies also made 
fundamental changes to FPA’s modeling approach for analyzing the 
firefighting assets needed to respond to fires, but these changes went 
beyond the review’s recommendations and, despite FPA’s importance and 
cost, the reasons for these changes were not fully documented. The 
agencies expected to complete the FPA model in November 2008—about a 
year later than initially estimated—and to begin using FPA’s results in 
spring 2009 to develop their fiscal year 2011 budget requests, a delay 
of about 3 years from their initial goal of using FPA’s preliminary 
results in 2006. FPA is expected to cost about $54 million to develop. 

Although it is not yet complete and GAO conducted only a limited review 
of its available components, FPA shows promise in achieving some of the 
key objectives originally established for it; nevertheless, the 
approach the agencies have taken hampers FPA from meeting other key 
objectives. Among the most important objectives, FPA will (1) provide a 
common framework for the five federal agencies to analyze firefighting 
assets and develop budget requests across agency jurisdictions, (2) 
analyze the most important fire management activities, and (3) 
recognize the presence of certain nonfederal firefighting assets that 
may be available to respond to fires on federal land. FPA falls short, 
however, with respect to other key objectives. First, FPA has limited 
ability to project the effects of different levels of vegetation 
reduction treatments and firefighting strategies over time, meaning 
that agency officials lack information that could help them analyze the 
long-term impact of changes in their approach to wildland fire 
management. Second, the modeling approach the agencies are taking 
cannot identify the most cost-effective mix and location of federal 
firefighting assets for a given budget but, rather, analyzes a limited 
number of combinations of assets and strategies to identify the most 
cost-effective among them. More broadly, the current FPA approach 
involves considerable discretion on the part of agency officials, 
increasing the importance of making decisions in a transparent manner 
so that Congress, the public, and officials throughout the agencies 
understand FPA’s role in budget development and allocation. 

What GAO Recommends: 

GAO is recommending, among other things, that the agencies develop a 
strategic plan for the continued development of FPA and provide 
Congress with annual updates on (1) their progress in completing the 
steps outlined in that plan and (2) how they used FPA in developing 
their budgets. Interior disagreed with the need to develop a strategic 
plan. In response to Forest Service and Interior comments on GAO 
findings on FPA’s cost-effectiveness approach, GAO’s recommendation to 
develop a strategic plan was revised to provide more flexibility. The 
agencies generally concurred with the other recommendations. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/products/GAO-09-68]. For more 
information, contact Robin Nazzaro at (202) 512-3841 or 
nazzaror@gao.gov. Highlights 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Concerns about FPA's Early Performance Led to Significant Changes, Not 
All of Which Were Transparent: 

FPA Shows Promise in Achieving Some Objectives but Falls Short of 
Others, Although the Agencies Are Considering Changes That May Improve 
Its Performance: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Agriculture, Forest 
Service: 

GAO Comments: 

Appendix III: Comments from the Department of the Interior: 

GAO Comments: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Table: 

Table 1: Agencies' Cost Estimates for Developing FPA: 

Abbreviations: 

Agriculture: Department of Agriculture: 

FPA: fire program analysis: 

Interior: Department of the Interior: 

OMB: Office of Management and Budget: 

[End of section] 

United States Government Accountability Office: 

Washington, DC 20548: 

November 24, 2008: 

The Honorable Jeff Bingaman: 
Chairman: 
Committee on Energy and Natural Resources: 
United States Senate: 

Dear Mr. Chairman: 

Wildland fires increasingly threaten communities and natural resources, 
and the cost of responding to those fires has risen dramatically. To 
deal with fire's threats, the five federal agencies responsible for 
managing wildland fires--the Forest Service in the Department of 
Agriculture (Agriculture) and the Bureau of Indian Affairs, Bureau of 
Land Management, Fish and Wildlife Service, and National Park Service 
in the Department of the Interior (Interior)--call upon thousands of 
firefighters and station fire engines, aircraft, and other equipment on 
or near federal land across the country. The agencies also conduct 
treatments to reduce vegetation, in an effort to lessen the potential 
for severe wildland fires, decrease the damage caused by fires, and 
restore and maintain healthy ecosystems. Despite these efforts, the 
average number of acres burned annually in recent years has grown by 
about 70 percent, and federal appropriations to prepare for and respond 
to wildland fires have nearly tripled since the mid-1990s, to more than 
$3 billion annually. Several factors have contributed to the increased 
risk and cost, including (1) uncharacteristic accumulations of 
vegetation, in part due to past land management activities and fire 
suppression policies; (2) increasing human development in or near 
wildlands, an area commonly known as the wildland-urban interface; and 
(3) severe drought and other stresses, in part due to climate change. 
Combined, these factors have contributed to wildland fires' burning 
more intensely and spreading more quickly at the same time that 
development has continued in fire-prone areas. Long-standing concerns 
about the mounting risk from and cost of wildland fires, along with 
growing recognition of the long-term fiscal challenges facing the 
nation, have led Congress, the agencies, and others to focus on 
ensuring that federal wildland fire activities are appropriate and 
carried out in a cost-effective and efficient manner. 

A key initial step in this effort was the development of the 1995 
federal wildland fire management policy.[Footnote 1] The policy 
recognized that new approaches to managing wildland fire were needed if 
the agencies were to respond effectively to changing conditions. The 
policy also found that differences in budgeting processes among the 
five agencies hindered their response to wildland fires. Subsequently, 
congressional committees directed the agencies to develop a common 
budget process. In 2001, the agencies commissioned a report that 
established a vision for an interagency budget process, a report the 
agencies adopted as the basis for a new budget-planning system known as 
fire program analysis, or FPA.[Footnote 2] 

As envisioned in the 2001 agency report, as well as in congressional 
committee and Office of Management and Budget (OMB) reports, FPA was 
intended to help the agencies develop their wildland fire budget 
requests and allocate funds. FPA's objectives include: 

* providing a common budget framework to analyze firefighting assets 
without regard for agency jurisdictions; 

* examining the full scope of fire management activities, including 
preparing for fires by acquiring and positioning firefighting assets 
for the fire season, mobilizing assets to suppress fires, and reducing 
potentially hazardous fuels; 

* considering the availability of nonfederal firefighting assets, such 
as state or county firefighters, that typically help respond to fires 
on federal lands; 

* considering the communities and resources to be protected and agency 
land management objectives; 

* modeling the effects over time of differing strategies for responding 
to wildland fires and treating lands to reduce hazardous fuels; and: 

* using this information to identify the most cost-effective mix and 
location of federal wildland fire management assets. 

* In addition, FPA was expected to be externally peer reviewed, which 
could improve Congress's and the agencies' understanding of its 
strengths and weaknesses. 

To realize this vision, the agencies in 2002 began to develop FPA, 
designing it as a computer model that analyzed numerous potential 
combinations and locations of firefighting assets and, for any given 
budget level, identified the optimal mix of these assets--that is, the 
mix and locations of firefighting assets that would best protect 
resources at risk. Data on potential combinations and locations of 
assets were to be entered by fire officials at agency field units, and 
the model's analysis of these combinations would then be evaluated by 
agency budget officials at the national level. The agencies estimated 
that FPA would cost more than $40 million to develop and would take 
about 5 years to complete. In 2006, after 4 years of work, the agencies 
conducted an internal review of FPA, in part because of concerns about 
how well the computer model reflected the realities of the agencies' 
fire management activities.[Footnote 3] Subsequently, the agencies made 
substantial changes in how FPA analyzes needed firefighting assets and 
determines where best to locate them. These changes raised questions 
about the extent to which FPA would meet its original objectives. In 
this context, you asked us to review FPA. This report examines (1) how 
the agencies have developed FPA to date, including the process followed 
as part of the internal review, and FPA's current status; and (2) the 
extent to which FPA will meet its original objectives. 

To address our objectives, we reviewed agency documents on FPA 
development, including the interagency report and project charter that 
provide FPA's foundation, the reports resulting from the internal 
review, and numerous technical papers and other documentation 
describing particular aspects of FPA. To further our understanding of 
FPA's development, including changes made to FPA after the internal 
review, we interviewed Forest Service and Interior officials in 
Washington, D.C; FPA project staff in Boise, Idaho; and agency 
officials in the field who were familiar with FPA. We also interviewed 
agency and other scientists who have helped develop FPA. At the time of 
our review, however, substantial portions of the model remained 
incomplete, and the agencies had not documented the model sufficiently 
to allow a comprehensive evaluation. We therefore limited our review to 
a broad examination of FPA's various components and how they interact, 
as well as a comparison of FPA's current approach and capabilities with 
its original objectives. We did not compare the capabilities of the 
current approach to those of the approach taken before the internal 
review. Appendix I describes our scope and methodology in more detail. 
We conducted this performance audit from September 2007 through 
November 2008 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Results in Brief: 

The Forest Service's and Interior agencies' initial development and 
implementation of FPA gave rise to concerns about its performance, 
leading to an internal review and subsequent changes to the model. 
These changes, however, went beyond the review's recommendations and 
were not always clearly explained or fully documented. The agencies 
began developing FPA in 2002 and completed the first part of the model 
in October 2004. But as field units began to use the first part, senior 
agency officials and some field staff raised fundamental concerns-- 
including concerns about the underlying science and the extent to which 
FPA met agency management and policy objectives. As a result, in 2006 
the agencies conducted an internal review of FPA, which questioned its 
modeling approach and concluded, among other things, that agency 
leadership needed to become more involved if FPA were to succeed. The 
agencies made substantial changes to FPA after the review, some of 
which followed from the review's recommendations. For example, the 
agencies established a new oversight body comprising senior agency 
leaders and an interagency science team. The agencies, with the 
approval of the oversight body, also made fundamental changes to FPA's 
modeling approach for analyzing the firefighting assets needed to 
respond to fires, but these changes went beyond the review's 
recommendations. The review, for example, did not conclude that a 
different modeling approach was needed, instead recommending that the 
agencies continue testing the initial model and refine it as necessary. 
Rather than follow this recommendation, however, the agencies adopted 
an entirely new modeling approach. Yet despite FPA's importance and 
cost, the reasons for these changes were not fully documented, and a 
formal, documented comparison of the original and revised approaches 
was never conducted. The agencies expected to complete the FPA model in 
November 2008--about a year later than initially estimated--and to 
begin using FPA's results in spring 2009 to develop their fiscal year 
2011 budget requests, a delay of about 3 years from their initial goal 
of using FPA's preliminary results in 2006. Ultimately, FPA is expected 
to cost about $54 million to develop. 

Although it is not yet complete and we conducted only a limited review 
of its available components, FPA shows promise in achieving some of the 
key objectives that congressional committees, OMB, and the agencies 
themselves established for it. Nevertheless, the approach the agencies 
have taken hampers FPA from meeting other key objectives. Once FPA is 
more fully developed and documented, a detailed, external peer review 
may reveal more about the extent to which it will help the agencies 
develop their wildland fire budget requests and allocate funds. Among 
the most important objectives it is likely to achieve, FPA is to 
provide a common framework for the five federal agencies to develop 
their wildland fire budget requests and analyze needed firefighting 
assets across agency jurisdictions--a significant step forward--and is 
to analyze the three most important fire management activities 
(preparedness, fire suppression, and fuel reduction). The agencies also 
have developed FPA to be capable of recognizing the presence of 
nonfederal firefighting assets that may be available to respond to 
fires on federal land--another key objective--although the extent to 
which these assets is to be included in the analysis is not yet clear. 
And finally, FPA is also to consider specific land management 
objectives and resources at risk, as suggested by the 1995 federal 
wildland fire management policy, rather than simply assume that all 
fires should be suppressed as quickly as possible (although if 
implemented as currently developed, FPA will likely not allow the 
agencies to consistently identify the locations that are most important 
to protect from a national perspective). FPA falls short, however, with 
respect to other key objectives in two critical areas. First, FPA's 
ability to project the effects of different levels of fuel reduction 
treatments and firefighting strategies over time appears limited. 
Agency officials are therefore likely to lack information that would 
help them analyze the extent to which increasing or decreasing funding 
for fuel reduction treatments and responding more or less aggressively 
to fires in the short term could affect the expected cost of responding 
to wildland fires over the long term. Second, regardless of the extent 
to which other key objectives are met, the modeling approach the 
agencies have taken is unlikely to identify the most cost-effective mix 
and location of federal firefighting assets for a given budget but only 
whether a particular mix of assets is more or less cost-effective than 
another. Since the different mixes of assets analyzed are limited to 
the number of alternatives developed by agency units in the field, 
these alternatives, even taken together, are unlikely to include the 
single most cost-effective mix of assets nationwide. In addition, other 
aspects of FPA may complicate its further development and 
implementation, including the lack of an external peer review of the 
model to date. Agency officials recognize many of these shortcomings 
and have said that they are considering taking actions--such as further 
adjusting the model (to better identify the most highly valued 
resources to protect, for example) and submitting the model for peer 
review--that have the potential to move FPA closer to meeting its key 
objectives. Regardless of the specific objectives FPA achieves, the 
modeling approach the agencies selected for FPA involves considerable 
discretion on the part of agency decision makers, increasing the 
importance of making decisions in a manner transparent enough that 
Congress, the public, and officials throughout the agencies understand 
how the decisions were made and FPA's role in them. 

To improve the agencies' ability to use FPA in developing their 
wildland fire management budget requests and allocating funds in a cost-
effective manner and to promote transparency in decision making-- and 
recognizing that FPA is still under development and that completing it 
will be an iterative process requiring the agencies' continued effort 
to improve--we are recommending that the Secretaries of Agriculture and 
the Interior (1) direct the agencies to develop a strategic plan for 
the continued development of FPA, (2) report annually to Congress on 
their progress in completing the steps outlined in this plan and on 
FPA's ability to meet each of its key original objectives, (3) report 
to Congress each year on how the agencies used FPA to develop their 
budget requests and allocate funds, and (4) submit the model for 
external peer review. 

In written comments on a draft of this report, the Forest Service and 
Interior disagreed with our finding that FPA is unlikely to allow the 
agencies to identify the most cost-effective mix of firefighting 
assets, stating they believed that FPA will allow them to meet the goal 
of cost-effectiveness. They also commented that the revised approach 
they are taking in developing FPA is more realistic and appropriate 
than their original approach. We continue to believe, however, that, 
regardless of the comparative strengths and weaknesses of the original 
and revised approaches, FPA as it is being developed is unlikely to 
allow the agencies to identify the most cost-effective location and mix 
of assets and strategies--one of the agencies' original objectives for 
FPA. To account for the agencies' views that the revised approach is 
more realistic, we are modifying our recommendation that the agencies 
develop a strategic plan for the continued development of FPA, adding 
that the agencies should clearly state whether they believe any of 
FPA's key original objectives are no longer appropriate. The Forest 
Service commented that it fundamentally agreed with our recommendations 
but believes there are better alternative approaches to carrying some 
of them out. The agency described the steps it intended to take in 
addressing two of them, but we do not believe that the steps outlined 
in the letter are specific and transparent enough to meet the intent of 
our recommendations. Interior concurred with three of our 
recommendations but disagreed with our recommendation that the agencies 
develop a strategic plan for the continued development of FPA, stating 
that developing such a plan would delay deployment and increase the 
cost of FPA. We do not agree that creating a strategic plan would 
necessarily delay the agencies' implementation of FPA; further, because 
our review raised questions about FPA's ability to meet certain key 
objectives, we continue to believe it is important for the agencies to 
create a strategic plan that directly and transparently evaluates FPA's 
ability to meet its original objectives and identifies ways to improve 
FPA to better meet those objectives. Comments from the Forest Service 
and Interior, along with our responses to those comments, are reprinted 
in appendixes II and III, respectively. 

Background: 

The agencies' wildland fire management program has three major 
components: preparedness, suppression, and fuel reduction.[Footnote 4] 
To prepare for a wildland fire season, the agencies acquire 
firefighting assets--including firefighters, engines, aircraft, and 
other equipment--and station them either at individual land management 
units (such as national forests or national parks) or at centralized 
dispatch locations. The primary purpose of these assets is to respond 
to fires before they become large--a response referred to as initial 
attack--thus forestalling threats to communities and natural and 
cultural resources. The speed with which the agencies are able to 
respond to a fire can be critical to their ability to suppress it while 
it is small; increasing the number of firefighting assets available to 
respond, and the number of locations they can respond from, can 
therefore improve the agencies' initial attack success, although the 
marginal utility of adding more firefighting assets decreases as the 
number of assets goes up. The assets the agencies use for initial 
attack are funded primarily from the agencies' preparedness budget 
accounts. 

In the relatively rare instances in which fires escape initial attack 
and grow large, the agencies respond using an interagency system, in 
which additional firefighting assets from federal, state, and local 
agencies, as well as private contractors, are mobilized, regardless of 
which agency or agencies have jurisdiction over the burning 
lands.[Footnote 5] Federal agencies typically fund the costs of these 
activities from their suppression budget accounts. To reduce the 
potential for severe wildland fires, lessen the damage caused by fires, 
limit the spread of flammable invasive species, and restore and 
maintain healthy ecosystems, the agencies also reduce potentially 
hazardous vegetation that can fuel fires. They remove or modify fuels 
using prescribed fire, mechanical thinning, herbicides, certain grazing 
methods, or combinations of these or other approaches. 

The federal government's cost of preparing for and responding to 
wildland fires has increased substantially over the past decade--an 
increase that has led federal agencies to fundamentally reexamine their 
approach to wildland fire management. For decades, federal agencies 
aggressively suppressed wildland fires and generally succeeded in 
reducing the number of acres burned. In some parts of the country, 
however, rather than eliminating severe wildland fires, decades of 
suppression contributed to the disruption of ecological cycles and 
began to change the structure and composition of forests and 
rangelands, thereby making lands more susceptible to fire. 
Increasingly, the agencies have recognized the role that fire plays in 
many ecosystems and the role that it could play in the agencies' 
management of forests and watersheds. As a result, the agencies have 
increased their efforts to reduce fuels and their emphasis on using 
less aggressive firefighting strategies, which typically cost less and 
can reduce fuels across a broader area than if fires are aggressively 
suppressed. Such strategies are to be used only in appropriate 
situations, such as in responding to fires that are not expected to 
threaten communities or damage important natural or cultural resources. 

This approach to managing wildland fires requires close integration of 
planning and budgeting systems so that the agencies are able to 
holistically analyze the full wildland fire management program. The 
agencies historically have used different planning and budgeting 
systems to help develop their budget requests and allocate the funds 
Congress appropriates. The agencies have identified shortcomings with 
this approach and have recognized that the existing systems were not 
capable of analyzing the trade-offs among initial attack, the full 
range of suppression strategies, and fuel reduction. Aggressively 
suppressing a fire, for example, may cost less in the short term but 
contribute to continued accumulation of vegetation, which can increase 
both the risk from and cost of responding to fires in the future; 
conversely, increasing investment in reducing fuels may cost more in 
the short term but can provide future benefits. The agencies, following 
congressional committee direction, committed to developing a new 
system, which came to be known as FPA. FPA is a strategic tool that 
agency budget officials expect to use to develop their wildland fire 
budget requests and allocate their fire management funds to the field, 
and that agency fire officials expect to use to model the effect that 
differing mixes and locations of firefighting assets, and differing 
levels of investment in reducing fuels, will have on their ability to 
protect communities and resources. Because it is a strategic tool 
rather than a tactical one, agency fire managers would not use FPA to 
help the agencies respond to actual fires. 

In developing and using FPA, the agencies must consider the process and 
time frames of the annual federal budget cycle, which begins about 2 
years before the fiscal year for which funds are being requested. 
Agencies develop their budget requests in late spring and summer and 
submit them to OMB in September. OMB prepares budget materials to 
submit to the President in January. The President approves a budget 
proposal and sends it to Congress by the first Monday in February. To 
develop their fiscal year 2011 budgets, for example, the agencies, in 
conjunction with their respective departments, expect to begin 
developing their budget requests in spring 2009 and to submit them to 
OMB in September 2009; subsequently, the President would submit his 
budget request to Congress in February 2010 for Congress's 
consideration. 

Concerns about FPA's Early Performance Led to Significant Changes, Not 
All of Which Were Transparent: 

Concerns about FPA's early performance and about the policy and 
scientific approaches the agencies used in FPA's early development led 
the agencies to conduct an internal review of FPA in 2006. 
Subsequently, the agencies made several significant changes to FPA, but 
these changes went beyond those recommended by the review, and the 
reasons for several of the changes were not fully documented. The 
agencies do not expect to use preliminary FPA results to develop their 
budget requests until 2009--3 years later than they had initially 
planned--but the cost of completing FPA appears in line with previous 
estimates. 

Concerns During FPA's Initial Implementation Led to an Internal Review: 

The staff who began to develop FPA in 2002, following congressional 
committee direction, initially focused on developing the portion of the 
model that analyzed the agencies' ability to successfully contain 
wildland fires during initial attack. The staff selected an approach 
that relied primarily on a modeling technique known as optimization. 
Using this approach, FPA was to analyze, for any given budget level, 
all possible combinations and locations of the firefighting assets 
typically available to agency field units and identify the combination 
of these assets that resulted in optimal protection of communities and 
resources. To provide data on different potential firefighting assets 
and locations, the agencies divided the country into 139 interagency 
"fire planning units," each of which encompassed land managed by one or 
more of the federal agencies responsible for wildland fire.[Footnote 6] 
Fire management officials in each of these planning units then 
identified the relative importance of protecting each acre within that 
planning unit by assigning a weighting factor indicating each acre's 
importance relative to other acres. The most important acres to 
protect, such as those in the wildland-urban interface, were assigned a 
weight of 1.0, while less important acres were assigned proportionately 
lower weights. After analyzing historic fire occurrence and weather 
patterns associated with each planning unit to determine where and when 
fires were likely to start, and considering the relative importance of 
acres to be protected, FPA was to analyze, for any given budget level, 
all possible mixes and locations of firefighting assets typically 
available to those units in order to determine which mix and locations 
would afford the best level of protection. 

Development of this "preparedness module" was completed in October 
2004, and over the next 16 months, officials in the field began using 
it to analyze their preparedness assets and budgets. By February 2006, 
nearly all the fire planning units had submitted FPA results for their 
units to the agencies' Washington offices, which in turn analyzed the 
FPA results in an effort to identify the optimal mix and location of 
firefighting assets across the country. During this time, however, 
senior agency officials, as well as some field officials, began to 
raise fundamental concerns about FPA's modeling approach. Weighting the 
importance of individual acres within a fire planning unit, for 
example, was a central component of FPA's early approach, and some 
officials believed that even where resources to be protected were 
similar across planning units, officials in those units assigned 
substantially different weights to the resources, thereby undermining 
the reliability of the results. Other officials were concerned that the 
early FPA approach placed insufficient emphasis on containing fires 
during initial attack, although the officials who developed this 
approach noted that it reflected the interagency policy of responding 
to fires on the basis of specific land and fire management objectives, 
rather than simply assuming that all fires should be suppressed as 
quickly as possible. Still other officials were concerned that the 
initial approach could result in unrealistic shifts in the mix and 
location of assets; a small change in budget, for example, could have 
led FPA to suggest moving a large quantity of assets from one planning 
unit to another or to dramatically change the relative proportion of 
firefighters, engines, and aircraft within a planning unit. 

Despite these concerns, senior agency officials told us, the early 
development of FPA represented an important first effort, given the 
difficulty of the project; in hindsight, they also recognized that 
greater involvement by policy and budget officials and agency 
scientists might well have averted some of the concerns and helped FPA 
develop more quickly. In any case, despite having told congressional 
and OMB staff that they intended to use the results of this initial 
analysis to help develop their 2008 budget requests, agency officials 
decided that the concerns about FPA were too great to justify doing so, 
and they instead initiated a two-part review of FPA to evaluate the 
issues that had surfaced. 

The agencies conducted this two-part review of FPA in late 2005 and 
early 2006. The reviews--performed by agency land managers, fire and 
budget officials, and scientists, as well as a representative from a 
state forestry agency--consisted of (1) an evaluation of the extent to 
which FPA helped the agencies achieve their management and policy 
objectives and (2) an evaluation of particular aspects of the 
underlying science and modeling approach. The reviews reaffirmed FPA's 
original objectives as articulated in the 2001 report, but they 
identified several challenges to meeting these objectives and made 
several recommendations intended to strengthen FPA's ability to do so. 
The management review, for example, recommended that the agencies more 
fully involve senior officials and scientists, submit FPA for external 
peer review,[Footnote 7] and complete their analysis of the initial FPA 
results. The science review likewise recommended that FPA be peer- 
reviewed and, in addition, that the agencies further test and improve 
the model and the data it uses. The agencies conducted the reviews 
quickly, however, and did not intend them to be a comprehensive 
evaluation of FPA; the science review, in particular, examined only 
certain aspects of FPA. 

The Agencies Made Significant Changes to FPA, Not All of Which Were 
Fully Documented: 

After the reviews, the agencies made several changes to the process 
used for developing FPA, changes that generally followed from the 
reviews' recommendations. In April 2006, the agencies established a new 
oversight body comprising senior officials from the Forest Service and 
Interior. This group was formed to make strategic decisions about FPA's 
scope, determine how FPA would be used to help the agencies make 
funding allocation decisions, and address any policy issues that FPA's 
development raised. The group also was to keep the Wildland Fire 
Leadership Council[Footnote 8] informed of FPA's status, including 
issues that the council needed to resolve. The agencies also 
established an interagency science team, made up of scientists from 
both the Forest Service and Interior, as well as university scientists 
outside the agencies. This science team was to assist FPA's developers 
by reviewing and evaluating FPA's modeling approach and identifying 
data sources and analytical techniques that could further FPA's 
development. 

The agencies also changed FPA's modeling approach considerably. Rather 
than continue to use the initial optimization-based approach 
(evaluating, for a given budget level, all possible combinations and 
locations of firefighting assets typically available to local units and 
identifying the asset combination that provided optimal protection of 
communities and resources), the agencies switched to a simulation 
modeling approach that evaluates a much smaller number of potential 
asset combinations along with different options for fuel reduction 
treatments and ranks them according to certain performance criteria-- 
which also differ from those used previously. The new approach is no 
longer to simply assess the extent to which each asset combination 
protects the areas field officials have identified as most important. 
Instead, it is to evaluate each combination's predicted performance 
against five separate performance measures the agencies have 
established: 

* total projected cost of suppressing fires; 

* total number of acres burned in the wildland-urban interface; 

* total number of acres meeting fire and fuels management objectives, 
such as reducing the likelihood of intense fires; 

* total number of acres burned containing resources the agencies define 
as being highly valued, such as endangered species habitat or municipal 
watersheds; and: 

* percentage of fires contained while small (i.e., the initial attack 
success rate).[Footnote 9] 

The revised FPA approach encompasses both a computer model and a 
management system to help the agencies develop their budget requests 
and allocate funds. Agency officials in each of the 139 planning units 
are to develop, for each of a given number of budget levels, an option 
specifying the mix and location of firefighting assets they would 
choose to acquire and an option specifying the number of acres they 
would treat to reduce fuels. For example, officials might develop one 
mix and location of firefighting assets and the acreage that would be 
treated if their fire planning unit's budget remained the same as the 
previous year, another option corresponding to a budget decrease from 
the previous year, and a third option corresponding to a budget 
increase from the previous year. The number of options the planning 
units are to develop and the budget levels to which these options 
correspond will depend on annual field guidance prepared by the 
agencies' headquarters offices. Senior agency officials told us that 
during FPA's initial implementation they were considering directing the 
units to develop three preparedness options and three fuel treatment 
options. These options were to correspond to each unit's 2007 budget 
level and plus and minus 10 percent of these 2007 levels. As of 
November 2008, however, the agencies had not finalized this step. 

Once the planning units have developed their options and entered 
information about firefighting assets and fuel treatments into the FPA 
system, the computer model is to then analyze historical data on local 
fire occurrence; local vegetation, geography, and weather; and the 
predicted effect on fire behavior of reducing fuels. From this 
analysis, FPA is to model the likelihood that wildland fire will damage 
communities and resources within the fire planning unit, considering 
the different mixes of assets and fuel treatments reflected in the 
proposed options. To provide comparable information across planning 
units, FPA is to evaluate each unit's options against the five 
performance measures. 

The FPA model is to then calculate a performance score for each of the 
"alternatives" developed by each planning unit. (An alternative 
consists of one preparedness option paired with one fuel treatment 
option. If planning units were directed to prepare three preparedness 
and three fuel treatment options, for example, nine alternatives would 
be possible.) FPA is to then "roll up" the performance scores for each 
alternative in all 139 planning units, so that senior agency officials 
can evaluate the effects on the agencies' performance measures 
nationwide of different combinations of alternatives. The senior agency 
officials would then use FPA results in conjunction with other budget 
information and processes to develop their budget requests. 

The extent to which any particular alternative, or set of alternatives, 
is considered cost-effective relies on the weights assigned to each of 
the five measures. The agencies could weight these measures in several 
ways to reflect their relative importance. If one measure were 
overwhelmingly more important than the others--if the agencies wanted 
to minimize suppression costs regardless of any other outcome, for 
example--the agencies could select a mix of firefighting assets and 
fuel reduction options predicted to maximize their ability to achieve 
that measure and consider the other measures only to help them choose 
between different mixes with similar outcomes for the most important 
measure. The agencies could also group two or more measures as more 
important than the others, or they could identify desired target levels 
for each measure and select the mix of firefighting assets projected to 
come closest to these targets. Senior officials will be able to use FPA 
to explore the modeled effects of weighting the measures differently--
in effect, to evaluate the trade-offs associated with weighting any 
particular measure more heavily than the others--as well as to identify 
alternatives with high performance scores regardless of the weights 
ultimately selected. 

The agencies will also need to determine whether the relative 
importance of the five measures is the same across different geographic 
regions of the country. Some officials and scientists involved with 
developing FPA have questioned whether applying a single weighting 
system across the country would accurately reflect national priorities 
or whether it is appropriate to emphasize different measures in 
different locations. For example, protecting the wildland-urban 
interface might be the most important consideration in some parts of 
the country, but reducing the likelihood of intense fires or protecting 
endangered species habitat might be more important elsewhere. FPA 
officials said that the model could perform this type of analysis, and 
agency budget officials said they would consider different approaches 
to weighting the measures once FPA was completed and the field units 
had submitted their different combinations of firefighting assets for 
analysis. 

The leaders of FPA's science team told us that this new approach 
addressed specific concerns they had with the old approach. First, they 
said that using multiple measures to evaluate different mixes and 
locations of firefighting assets--rather than a single measure as in 
the old approach--better reflected the complexity of wildland fire 
management. Second, they said that the new approach is to analyze many 
more potential fire scenarios, thus evaluating the asset alternatives' 
predicted performance across a broader range of conditions than in the 
old approach.[Footnote 10] Third, they said that because the old 
approach relied on weighting the relative importance of acres, they 
were concerned that different units would weight similar resources 
differently, thus preventing meaningful comparisons among units, or 
that some units might intentionally inflate the weights in an effort to 
gain advantage. Fourth, they said that because the new approach is to 
rely on alternatives developed by officials in the field, it can 
identify possible mixes and locations of assets that are likely to be 
more easily implemented than those identified through the old approach, 
which considered all possible combinations of assets typically 
available to local units and could suggest changes that might be 
unrealistic. Finally, they said that because the new approach allows 
field officials to identify the firefighting assets they would 
typically dispatch to fires burning in specific areas under certain 
conditions, it more closely follows how the agencies actually respond 
to fires. 

The changes to FPA's modeling approach, however, were not among the 
recommendations stemming from the science review, which recommended 
that the agencies further test the initial model and improve it. But 
such testing and improvement of the initial model did not take place. 
The leaders of the science team told us that refining the initial model 
would not be useful, because the team had determined that the model was 
fundamentally flawed and a new approach was needed. Instead, the 
science team, in summer and fall 2006, developed five options for 
continuing to develop FPA and presented these options to the Wildland 
Fire Leadership Council, which selected one option in December 2006. 
This process was generally consistent with the management review, which 
recommended that an interagency science team examine the modeling 
approach FPA initially used. 

Still, the agencies' rationale for making the changes to FPA's modeling 
approach was not fully documented, even though FPA is a major project 
whose outcome is expected to influence the allocation of billions of 
dollars. Although the science team's leaders told us they believed that 
the changes improved FPA, they provided no documents describing either 
the reasons for the changes or the process used to identify FPA's new 
approach. For example, a formal, documented comparison of the old and 
new approaches was never done; the science team's leaders told us they 
considered the relative strengths and weaknesses of the old approach 
and other possible approaches but did not document this consideration. 
In any event, each of the five development options the science team 
presented to the Wildland Fire Leadership Council included the same two 
fundamental changes in modeling approach. Without a formal, documented 
comparison of the old and new approaches, and without the opportunity 
to consider options that used other modeling approaches, the council 
lacked information that might have informed its choice. 

FPA's Completion Has Been Delayed, but Costs Appear in Line with 
Previous Estimates: 

In addition, the changes apparently prevented the agencies from meeting 
their commitment to use preliminary FPA results beginning in 2006. 
Although FPA was not expected to be complete until late 2007, agency 
officials believed they would be able to make some use of its 
preliminary results in 2006. Accordingly, officials told congressional 
committee staff and OMB in early 2006 that the agencies would begin 
using FPA results that year to allocate their fiscal year 2007 funds 
and to develop their fiscal year 2008 budget requests.[Footnote 11] 
Agency officials told us, however, that they subsequently decided not 
to use FPA's preliminary results because they did not believe it was 
prudent in light of the concerns that arose during the internal review. 
While it seems appropriate to delay using the model for budget 
decisions until concerns about its utility have been resolved, the 
agencies' position has been less than transparent; in August 2006--well 
after they realized that FPA would be undergoing substantial changes-- 
they repeated their commitment to begin using FPA results in September 
of that year. 

The agencies now expect that the FPA model will be completed in 
November 2008--about a year later than initially estimated--and that 
they will begin using FPA's results in 2009 to develop their 2011 
budget requests, a delay of about 3 years from their initial goal of 
using preliminary results in 2006. When they began developing FPA in 
2002, the agencies reported that FPA would be completed by the end of 
2007. After the internal review, the agencies reported that a fully 
functional FPA system would be developed by June 30, 2008, and used in 
spring 2009 to inform the agencies' fiscal year 2011 budget requests. 
In spring 2008, the agencies repeated their commitment to this time 
frame. Agency officials attribute the delay in completing FPA to the 
project's complexity. When our review ended, agency officials said they 
expected fire planning units to begin using FPA in late 2008; about 
half the field units are expected to complete their alternatives by 
February 2009, with the remaining units completing their alternatives 
by June 2009. Meeting this time frame, however, will require the 
agencies to complete both the model and the guidance directing the 
field on how to develop the options the FPA model will analyze--both of 
which have experienced recent delays. Nonetheless, the agencies' 
Washington offices remained committed to using FPA results beginning in 
2009. Agency field officials, however, have worried that the delay in 
completing FPA places an undue burden on the field by shortening the 
time available for planning units to develop their alternatives. Field 
officials also observed that senior agency officials have not clearly 
articulated how the results from FPA's first year would be used, 
although senior agency officials have stated that 2009 is to be a 
"learning year" and that they do not expect FPA to influence 
substantial changes to funding allocations in the first year. 

The expected cost for completing FPA has been little affected by the 
substantial changes it has undergone since 2006. FPA's project 
development costs are expected to total about $43.9 million, according 
to an April 2008 estimate by the senior project manager responsible for 
FPA's budget.[Footnote 12] This cost is generally in line with the 
agencies' previous estimates, particularly those developed after the 
agencies began to determine FPA's full scope (see table 1). Agency 
salaries and benefits, which were not included in yearly estimates of 
project development costs, represent an estimated $9.7 million in 
additional costs--for a total of about $53.6 million. According to the 
senior project manager, the agencies did not begin to develop FPA's 
second phase until 2005 and were still determining the scope of that 
phase when they submitted projected cost estimates in 2003 and 2004. 
The increase from 2003 to 2005 in the estimated cost for the second 
phase therefore reflects the agencies' better understanding FPA's scope 
and not a cost overrun, the project manager said.[Footnote 13] 

Table 1: Agencies' Cost Estimates for Developing FPA: 

Dollars in millions. 

Year of cost estimate: 2003; 
Phase 1 (fiscal years 2002-2006): $11.9; 
Phase 2 (fiscal years 2005-2010): $22.0; 
Total (fiscal years 2002-2010): $33.9. 

Year of cost estimate: 2004; 
Phase 1 (fiscal years 2002-2006): 12.2; 
Phase 2 (fiscal years 2005-2010): 30.0; 
Total (fiscal years 2002-2010): 42.2. 

Year of cost estimate: 2005; 
Phase 1 (fiscal years 2002-2006): 12.1; 
Phase 2 (fiscal years 2005-2010): 36.2; 
Total (fiscal years 2002-2010): 48.3. 

Year of cost estimate: 2006; 
Phase 1 (fiscal years 2002-2006): 11.6[A]; 
Phase 2 (fiscal years 2005-2010): 31.2; 
Total (fiscal years 2002-2010): 42.8. 

Year of cost estimate: 2007; 
Phase 1 (fiscal years 2002-2006): 11.6[A]; 
Phase 2 (fiscal years 2005-2010): 32.3; 
Total (fiscal years 2002-2010): 43.9. 

Year of cost estimate: 2008; 
Phase 1 (fiscal years 2002-2006): 11.6[A]; 
Phase 2 (fiscal years 2005-2010): 32.3; 
Total (fiscal years 2002-2010): 43.9. 

Source: GAO analysis of Forest Service data. 

Note: Costs do not include salaries and benefits for all agency 
employees who worked on the FPA project. The senior project manager 
responsible for FPA's budget estimated these costs at $9.7 million. 

[A] Actual, not estimated; the agencies completed phase 1 of FPA in 
2005 at a cost of $11.6 million. 

[End of table] 

FPA Shows Promise in Achieving Some Objectives but Falls Short of 
Others, Although the Agencies Are Considering Changes That May Improve 
Its Performance: 

Although FPA is not yet complete and our review was limited, FPA shows 
promise in achieving some key objectives, including establishing a 
common, interagency budget framework that includes important wildland 
fire program activities. Nevertheless, FPA is unlikely to achieve all 
its key objectives, including the critical objectives of analyzing the 
effect over time of different funding allocation strategies and 
identifying the most cost-effective mix of firefighting assets. The 
agencies recognize that FPA will not fully meet all its key objectives 
in 2008 and are considering several changes that may improve its 
ability to meet certain objectives in the future. But because the 
modeling approach the agencies selected for FPA involves considerable 
discretion on the part of agency decision makers, transparency is 
particularly vital. 

FPA Is to Provide the Foundation for an Interagency Framework for 
Analyzing Needed Firefighting Assets and Is to Examine Key Fire 
Management Program Activities and Objectives: 

If implemented as currently developed, FPA will provide the foundation 
for a single framework for the five federal agencies to develop their 
budget requests and allocate funds, a key objective. It is also likely 
to help the agencies achieve another key objective by analyzing the 
most important wildland fire management activities. The agencies have 
developed FPA so that it can recognize the presence of nonfederal 
firefighting assets that may be available to respond to fires on 
federal land--a third key objective--although the extent to which these 
assets will be included in the analysis is not yet clear. And finally, 
FPA should help the agencies move toward a fourth key objective-- 
responding to wildland fires in ways that meet specific land and fire 
management objectives, rather than simply assuming that all fires 
should be suppressed as quickly as possible--although its ability to 
fully achieve this objective is likewise uncertain. 

FPA Is to Provide the Foundation for an Interagency Budgeting 
Framework: 

As the agencies are developing it, FPA is to provide the foundation for 
a single framework for the five federal agencies to help develop their 
wildland fire budget requests and allocate their fire management funds, 
as envisioned in congressional guidance and the 2001 agency report--a 
significant step forward. In implementing FPA, officials are to work 
across agencies, both in the field and at headquarters. In the field, 
officials from each agency will need to work together to identify 
different mixes and locations of firefighting assets--information that 
will enable the FPA model to analyze the effect of different mixes of 
firefighting assets without regard to agency jurisdictional boundaries. 
At headquarters, agency officials are to work together to determine how 
to weight the five performance measures FPA incorporates to identify 
the best mix of firefighting assets. 

FPA Is to Analyze the Three Most Important Fire Management Program 
Activities: 

FPA also substantially moves the agencies toward achieving another key 
objective by analyzing the three most important fire management 
activities: preparedness, fuel reduction, and suppression. FPA is to 
directly analyze preparedness and fuel reduction and then model the 
effects that varying investments in these activities might have on 
suppression costs.[Footnote 14] Together, these activities constitute 
most of the agencies' overall fire management budgets. 

To analyze the agencies' preparedness for wildland fires, FPA is to 
model the potential effect of wildland fire on communities and 
resources, depending on the mix and location of firefighting assets 
that would be stationed in an area. FPA is to consider historical fire 
occurrence and weather patterns to model the likelihood that a fire 
might occur in specific areas. Using an interagency database known as 
LANDFIRE to identify the fuel types and topography in the location 
where a fire is predicted to ignite,[Footnote 15] FPA is to then model 
a fire's likely intensity and rate of spread. Finally, FPA is to 
identify the location of specific firefighting assets available for 
initial attack and, considering the fire's intensity and rate of 
spread, determine whether firefighters are likely to contain the fire 
before it grows too large and whether the fire is likely to damage 
communities or resources. 

To analyze the effect of fuel reduction treatments within FPA, 
officials in the field are to begin by identifying the attributes of 
the fuel reduction treatments they most often undertake in their area, 
including vegetation type (such as trees, shrubs, or grasses) and the 
treatments' effect on vegetation density, height, and other 
characteristics. The FPA model is to then predict the effect of those 
treatments on fire behavior and compare the effectiveness of fuel 
treatments at reducing fire damage in different areas. 

Finally, to analyze suppression costs, FPA is to consider different 
levels of investment in preparedness and fuel reduction and, for each 
investment level (including the mix and location of firefighting 
assets), estimate the number of fires likely to escape initial 
suppression efforts. For each such "large" fire, FPA is to use another 
model the agencies have developed to predict the cost of suppressing 
the fire on the basis of the costs from previous fires with similar 
characteristics, including fire size, fuel types, fire intensity, 
physical terrain, proximity to the nearest community, and total value 
of structures close to the fire. The costs of past fires with similar 
characteristics vary widely, however, which limits the model's ability 
to accurately predict suppression costs. Moreover, the model is based 
on historical costs, and since the agencies have recently begun 
emphasizing less aggressive strategies, it may not accurately predict 
suppression costs for fires.[Footnote 16] The agencies are continuing 
to improve this model, however, which could improve the accuracy of the 
cost estimates. 

FPA Is to Be Able to Analyze Nonfederal Firefighting Assets, but the 
Extent to Which the Agencies Will Include These Assets in Their 
Analysis Is Unclear: 

Although the agencies are developing FPA to recognize the presence of 
nonfederal firefighting assets that may be available to respond to 
fires on federal land[Footnote 17]--a key objective of FPA--the extent 
to which these assets will be included in the analysis is not yet 
clear. When officials in the field enter into FPA the different 
combinations of federal firefighting assets they would acquire for a 
given budget level, they can also include nonfederal firefighting 
assets that are stationed nearby, such as firefighters or fire engines 
belonging to state agencies or area communities. FPA is then to 
consider the availability of these nonfederal assets when it analyzes 
the effect of different combinations of federal assets on the five 
performance measures. 

FPA officials recognize, however, that some nonfederal entities may 
object to federal agencies' including nonfederal assets in their 
analysis, for fear that doing so would lead to fewer federal 
firefighting assets stationed in certain locations, which in turn could 
lead to an additional workload for nonfederal entities in those 
locations. The inclusion of nonfederal assets raised significant 
concerns among nonfederal entities when the first FPA analysis was 
conducted in 2006. And while FPA guidance to planning units in the 
field generally directs them to include nonfederal assets, FPA 
officials acknowledged the likelihood that field units would receive 
"strong objections" to this direction from some nonfederal entities. 
Such objections might cause field units to omit nonfederal assets from 
the FPA analysis to satisfy the concerns of their nonfederal partners, 
with whom they must maintain relationships. FPA officials said they 
expect concerns from nonfederal officials to lessen over time, as those 
officials become more knowledgeable about how FPA operates. Ultimately, 
however, if agency planning units do not include nonfederal assets that 
may be available to respond to fires, FPA will model fewer firefighting 
assets than are actually present--and may therefore underestimate the 
effectiveness of a given set of federal assets. In addition, if some 
planning units include nonfederal assets and others do not, FPA's 
ability to identify the best combination of federal firefighting assets 
nationwide is likely to be compromised. 

FPA Is to Consider Land and Fire Management Objectives, but Some 
Shortcomings Remain to Be Addressed: 

FPA should also help the agencies move toward achieving a fourth key 
objective--responding to wildland fires so as to meet specific land and 
fire management objectives, as suggested by the 1995 federal wildland 
fire management policy, rather than simply assuming that all fires 
should be suppressed as quickly as possible--although some agency 
officials have concerns about how well FPA will consider land 
management objectives. FPA should help the agencies move closer to this 
objective in two ways. First, officials in the field are to be 
responsible for identifying the number and type of firefighting assets 
they would typically dispatch to a fire that ignited in a particular 
location under particular conditions. The intent is to recognize that 
agency responses vary from fire to fire, and fire managers are more 
likely to dispatch more assets to a fire that threatens communities or 
highly valued resources or ignites under conditions conducive to rapid 
spread than to a fire ignited where it threatened few important 
resources or was unlikely to spread. The FPA model is to use this 
information to identify locations where stationing proportionately more 
firefighting assets might be helpful. Second, officials in the field 
are also to estimate the fire intensity beyond which resources in a 
particular area are likely to be damaged. In some areas, for example, 
officials might establish a relatively high intensity threshold to 
recognize that moderate, or even severe, fires might be acceptable, 
while in other areas--such as the wildland-urban interface--officials 
would likely determine that any fire is undesirable. In evaluating 
different mixes and locations of firefighting assets, FPA is to take 
into account this variation in acceptable fire intensity. In 
determining both the firefighting assets they would dispatch and the 
intensity threshold, field officials are expected to use information 
contained in local land and fire management plans, which the agencies 
are required to develop.[Footnote 18] 

Several issues, however, must be addressed for FPA to move the agencies 
more fully toward achieving their objective of responding to fires 
according to specific land and fire management objectives. First, one 
of the measures FPA is to use in evaluating alternative mixes and 
locations of firefighting assets is the predicted success of containing 
fires before they become large. Although containing fires when they are 
small is desirable in many circumstances, the agencies themselves have 
also recognized that their legacy of successful suppression has 
contributed substantially to the current increase in burned acres and 
fire intensity. As noted, it is not clear how the agencies will weight 
the relative importance of containing fires early (or indeed how they 
will weight any of the five measures) in FPA, but early guidance to the 
field indicates that early containment may be weighted heavily, which 
would keep FPA from fully recognizing the potential benefits of fire in 
some areas. Second, officials from the Fish and Wildlife Service and 
National Park Service have expressed concern that FPA is to evaluate 
the effects of reducing fuels solely by how the reduction affects the 
likelihood of a severe fire, without considering whether the fuel 
reduction treatment helps the agencies achieve broader land management 
objectives, such as improving the ecological condition of the land over 
time, as the 2001 report envisioned. A senior Fish and Wildlife Service 
official also noted that many wildlife refuges consist of small parcels 
of federal land interspersed among larger parcels of nonfederal land 
and that FPA is not designed to consider the effects of fragmented 
ownership. 

Third, although FPA is to consider specific local land and fire 
management objectives that recognize that some areas are more important 
to protect than others, it will likely not allow the agencies to 
consistently identify the locations that are most important to protect 
from a national perspective. Within the five performance measures 
evaluating the effects of different mixes and locations of firefighting 
assets and fuel treatment options, FPA is to consider all acres as 
equally important, despite significant variation in the resources on 
those acres. For example, the agencies have established protection of 
the wildland-urban interface as one of their most important policy 
objectives, and FPA is to treat all interface acres identically, 
regardless of whether an acre contains one or several houses. 
Similarly, the agencies intend to increase the number of acres that are 
meeting fire and fuel management objectives, such as reducing the 
likelihood of uncharacteristically intense fires, and FPA is to 
consider all acres within this measure identically. For example, FPA is 
to consider an acre of a relatively common forest type, such as 
ponderosa pine, the same as a relatively rare type, such as giant 
sequoia--even though agency managers may place a much greater priority 
on the condition of a sequoia forest. As a result, FPA will not likely 
allow the agencies to give high priority to meeting objectives in 
particularly important or rare areas. FPA is also to predict the 
percentage of fires likely to be contained in initial attack. In 
evaluating the effect that different mixes and locations of 
firefighting assets have on this measure, however, FPA is to weight all 
fires equally, regardless of the fires' potential to damage communities 
or valuable natural or cultural resources. Agency officials analyzing 
FPA results may therefore consider it more important to try to contain 
multiple fires that do not pose a great threat than to try to contain a 
single fire that does. The presence of the other measures helps to 
mitigate this shortcoming, because if an uncontained fire damages 
communities or valuable resources, the agencies' ability to meet the 
other objectives will be compromised. The relative weights of the five 
measures, however, have not yet been determined, and it is not clear 
how the measures' interactions will play out. 

One of the five measures the agencies will ultimately use to evaluate 
different mixes and locations of firefighting assets specifically 
considers resources the agencies regard as highly valued, which could 
improve the agencies' ability to identify some of the most important 
resources to protect. Nevertheless, FPA would still consider all acres 
within a particular performance measure identically and therefore not 
recognize that it is more important to protect some acres than others. 
In August 2008, the agencies decided to include only two types of 
resources in this measure in their 2009 analysis: municipal watersheds 
and habitat for some endangered species. Senior officials from the four 
Interior agencies, however, have criticized the approach the agencies 
are developing for FPA to consider highly valued resources because it 
does not sufficiently consider their agencies' land management 
objectives. 

As Designed, FPA Will Not Achieve All Its Key Objectives, Including 
Examining the Effects over Time of Differing Funding Allocation 
Strategies and Identifying the Most Cost-Effective Mix of Firefighting 
Assets: 

Even though FPA is likely to achieve several of its key objectives, it 
is unlikely to help the agencies achieve others. In particular, the 
modeling approach the agencies are taking has limited ability to 
examine the effects over time of different funding allocation 
strategies and is unlikely to allow them to identify the most cost- 
effective mix of firefighting assets. Other aspects of FPA, including 
the lack of an external peer review of the model, may complicate its 
further development and implementation. 

FPA's Ability to Examine the Temporal Effects of Differing Funding 
Allocation Strategies Appears Limited: 

FPA was envisioned as a way to help the agencies determine the extent 
to which, in the short term, increasing or decreasing funding for fuel 
reduction treatments and responding more or less aggressively to fires 
would affect the expected cost of responding to wildland fires over the 
long term. Although FPA is to analyze funding for both preparedness and 
fuel reduction, its ability to evaluate the trade-offs associated with 
increasing or decreasing one of these activities appears to be limited 
to short-term effects. Spending funds to reduce fuels, however, is 
generally considered a long-term investment, one whose value increases 
over time as more of the landscape is treated. If FPA considers only 
short-term effects, it may underestimate the benefit of reducing fuels 
and may lead the agencies to place greater emphasis on suppressing 
fires than warranted--with potentially far-reaching consequences. 

FPA officials told us in September 2008 that they were working with the 
interagency science team to develop an approach that would allow FPA to 
better analyze the long-term effect of reducing fuels; the officials 
expected to incorporate this approach into FPA by November 
2008.[Footnote 19] Because the agencies had begun to develop this 
approach only toward the end of our review, we were unable to evaluate 
it. On the basis of our discussions with FPA officials and members of 
the interagency science team, and from our review of the limited 
documentation describing the approach, it appears that FPA's ability to 
help the agencies achieve this objective will be limited. 

Moreover, FPA is unlikely to examine the effects over time of different 
firefighting strategies. Since adopting the 1995 fire management 
policy, the agencies have increasingly emphasized appropriate 
management response. FPA is to recognize that fire managers choose to 
respond less aggressively in some cases; for example, it is to allow 
field officials to model dispatching fewer firefighting assets to fires 
that are unlikely to threaten important resources. Fires responded to 
less aggressively are likely to burn many more acres than fires 
suppressed quickly. Less aggressive strategies may therefore reduce 
fuels on more acres--which in some cases could lower the risk from 
future large fires. FPA does not recognize this benefit, however, and 
will therefore be unable to help agency officials understand how 
responding less aggressively now may reduce the size and intensity of 
fires later, which could in turn help the agencies protect communities 
and resources and lower the cost of suppressing fires. 

More broadly, FPA's limited ability to examine the effect over time of 
reducing fuels and implementing appropriate management response could 
also limit its ability to help the agencies develop a long-term, 
cohesive strategy for responding to wildland fires. We have long 
recommended that the agencies develop a cohesive strategy identifying 
available long-term options and associated funding for reducing 
hazardous fuels and responding to wildland fires.[Footnote 20] Such a 
strategy is fundamental if the agencies and Congress are to fully 
understand the potential choices, and associated costs, for addressing 
wildland fire problems. The agencies have consistently concurred with 
our recommendation,[Footnote 21] and agency officials cited FPA as a 
key step in enabling them to develop a cohesive strategy. In its 
current state of development, however, FPA lacks important capabilities 
to help inform strategic decisions about how to invest the agencies' 
limited funds. 

FPA Is Unlikely to Allow the Agencies to Identify the Most Cost- 
Effective Mix of Firefighting Assets: 

A primary objective of FPA, established by the 2001 agency report, is 
to identify the most cost-effective fire management program for a given 
budget. Accomplishing this objective requires the agencies to define 
the fire management objectives they are trying to achieve and then to 
identify the combination of fuel reduction treatments and suppression 
strategies, including the best mix and location of firefighting assets, 
that would result in the most effective use of program funds. The 
modeling approach the agencies are using in FPA, however, does not 
allow the agencies to meet this objective. Rather than analyzing all 
possible combinations of assets typically available to local units, as 
well as fuel reduction and fire suppression strategies, to identify the 
most cost-effective combination, the approach the agencies are taking 
allows them to compare only a limited number of asset mixes and 
firefighting strategies (including fuel reduction options) to determine 
whether one mix of assets and strategies is more or less cost-effective 
than another. Because FPA is to compare only a limited number of 
alternatives, the evaluated alternatives are unlikely to include the 
most cost-effective mix of assets nationwide. Further, because the 
evaluated alternatives are likely to reflect minor variations in budget 
levels (e.g., plus and minus 5 percent or 10 percent of the prior 
year's budget for each planning unit), the present FPA approach is 
likely to generate results that differ only incrementally from the 
asset mixes and strategies already in place, rather than evaluate 
whether significantly different alternatives could yield significantly 
better results. 

Agency officials, including key scientists involved in FPA development, 
told us they believed that although the modeling approach has not been 
designed to identify the single most cost-effective mix of firefighting 
assets, FPA would nevertheless provide useful information to help the 
agencies develop their budget requests. In fact, several officials told 
us they preferred the flexibility currently built into FPA, which 
allows them to consider multiple potential budget scenarios--what one 
official termed a "family of solutions"--over the rigidity built into 
the old approach, which resulted in a single solution. Officials told 
us that it would be unrealistic to expect that the complexities of 
wildland fire management could be modeled accurately enough to yield a 
single solution that is truly optimal and that by examining multiple 
possible budget scenarios developed by officials in the field, FPA's 
new approach would yield results that would be "among the most cost- 
effective solutions," according to one official. Nevertheless, it is 
not clear that examining only a small number of alternatives for each 
planning unit will generate results that are among the most cost- 
effective, particularly given current guidance to the field to consider 
only slight variations from current funding levels when developing 
alternatives. 

Moreover, in analyzing trade-offs among different mixes and locations 
of firefighting assets, FPA is to consider only those assets that are 
stationed at individual management units, not those that are centrally 
located and under regional or national control. These central assets, 
which include large air tankers and helicopters and many of the most 
qualified firefighters, are some of the agencies' most costly, 
representing about $200 million of their budgets, according to agency 
estimates. The agencies use these assets in two ways: to assist local 
units with initial attack on small fires and to help suppress large 
fires. FPA is to consider the presence of these central assets when 
analyzing the likelihood that firefighters will be able to contain a 
fire during initial attack--important because otherwise the model would 
suggest that more firefighting assets would be needed at local planning 
units. FPA is to consider the number of centrally located assets as a 
given, however--that is, as a fixed input to the model, not a variable-
-rather than analyze the effects of changing the number of centrally 
located assets or the proportion of assets under local or national 
control. As a result, the model is not likely to determine the effect 
of changing the number or type of these assets on the agencies' 
firefighting abilities and costs, thus further limiting its ability to 
identify the most cost-effective mix of assets nationwide. 

Other Aspects of FPA Complicate Further Development and Implementation: 

Although FPA is a new system, it will rely on many data sources, 
models, and systems the agencies developed earlier, some of which have 
known shortcomings. For example, FPA is to use data from the LANDFIRE 
system to identify the fuel types across the country; yet the accuracy 
of LANDFIRE data has been questioned, as has the frequency with which 
the system will be updated to recognize changes in fuel conditions over 
time due to insect outbreaks, large wildland fires, or other 
disturbances. Over the past several years, FPA and LANDFIRE project 
officials have worked together to develop a process to update LANDFIRE 
data, which should benefit FPA. It is too early to tell how effective 
the planned LANDFIRE improvements will be. 

To predict how quickly a fire may spread in different fuel, weather, 
and geographical conditions, FPA is to use the results of FSPro, a fire 
growth model developed by Forest Service scientists. Fire officials 
have recognized that the spread rate predicted by the model is not 
always consistent with the rate of spread they observe during real 
fires. To help compensate for this difference, the FPA model is to 
allow field officials to calibrate the data used by FSPro to model 
spread rates so that they more closely reflect conditions typically 
observed in a particular area. It is not clear, however, how the 
agencies will ensure that the calibrations are made consistently or 
what the effects may be on the mix of firefighting assets FPA 
identifies as most appropriate. 

Even with their known shortcomings, some of FPA's component elements 
are well-established applications that have been used by wildland fire 
managers for many years and, in some cases, are based on peer-reviewed 
science. The FPA model as a whole, however, including its component 
parts, has not been externally peer-reviewed. This lack comes in part 
because documentation on FPA's development and capabilities has not 
been sufficiently developed to allow for peer review; instead, 
according to agency officials, project staff have been devoting time to 
model development. Until the model is peer-reviewed--including 
validation that the overall logic is sound, the methods used are state 
of the art, the results are consistent with empirical evidence, and the 
system is adequate for its intended purpose--neither the agencies nor 
outside parties will have a full understanding of FPA's strengths and 
limitations or know how much confidence they should place in the 
model's analysis. A peer review, moreover, may identify limitations not 
revealed by our review. 

Finally, the utility of FPA in identifying the best mix of firefighting 
assets will depend heavily on the alternatives and data developed by 
officials in the field, but some field officials expressed concerns 
about this component of FPA. For example, some field officials are 
concerned that they will receive little training on how to use FPA, 
which may prevent them from developing the most realistic options, and 
that the time needed to enter data into FPA and develop alternatives 
for national consideration will substantially increase their workload. 
Other officials told us that they are concerned that field staff may 
try to "game" FPA in an attempt to get the model to identify their area 
as needing more assets. Staff could, for example, develop a less- 
effective alternative for their low-budget scenario to make their mid- 
or high-budget scenarios appear more effective; similarly, staff may 
find it expedient to develop alternatives that ensure that each agency 
in their planning unit gains or loses comparable quantities of 
firefighting assets in order to promote equity among the agencies, 
rather than develop alternatives likely to best protect important 
resources but which might affect one agency more than another. Senior 
agency officials told us that gaming is a concern with any budgeting 
system and that they are planning to establish a two-stage process to 
review field submissions. In the first stage, officials from other 
field units would review the alternatives to ensure that interagency 
guidance was followed and information entered correctly; in the second, 
regional officials would review the results to ensure they met regional 
priorities. The exact steps this review process would follow, however, 
have not been determined, so it is not yet clear whether this process 
will ensure that only appropriately developed alternatives are 
submitted. 

The Agencies Are Considering Changes That May Increase FPA's Ability to 
Meet Certain Key Objectives, Although These Changes Do Not Address All 
Shortcomings: 

Senior agency officials told us they recognize that in 2008 FPA will 
not fully meet all its key objectives but said they are considering 
making several changes that may improve its ability to meet certain key 
objectives, including the following: 

* submitting FPA for an external peer review; 

* continuing to develop FPA's process for identifying the resources the 
agencies consider to be highly valued and assessing the agencies' 
ability to protect these resources; and: 

* working with the interagency science team to improve how FPA is to 
consider land management objectives, such as improving the ecological 
condition of the land over time, when evaluating the benefits of 
reducing fuels. 

* Although these steps have the potential to move FPA closer to meeting 
some of its key objectives, it is too early to determine how successful 
they will be. Moreover, these steps do not address all shortcomings we 
or others have identified, and taking these or other steps to improve 
FPA will carry an additional cost, which is not included in current 
agency estimates. The approximately $54 million estimated cost for FPA 
includes basic operation and maintenance through fiscal year 2010 but, 
according to agency officials, does not include funds to make the above 
improvements. 

The Approach Selected for FPA Increases the Importance of Transparency 
in Decision Making: 

The approach the agencies have taken in developing FPA allows for 
considerable discretion on the part of agency decision makers in three 
key areas: determining the relative importance (that is, the weights) 
of the five performance measures used to evaluate locally developed 
alternatives; using FPA results in combination with other information 
to develop agency budget requests; and using FPA results, likewise in 
combination with other information, to allocate funds to the field. 
Although it is important that decision makers have the flexibility to 
consider various options, that same flexibility makes it essential for 
the agencies to ensure that these processes are fully transparent. 
Otherwise, Congress, the public, and agency officials cannot be assured 
of fully understanding the rationale behind decisions or FPA's role in 
them. Although any changes to the existing allocation of funds among 
agencies or across different geographical areas are likely to be 
incremental at first, the agencies could consider larger funding 
reallocations as their understanding of FPA increases--which would make 
transparent decision making even more important. 

First, as noted, the extent to which any particular alternative or set 
of alternatives is considered cost-effective will depend on the 
relative importance assigned to the five performance measures, 
including any variation in their relative importance in different 
regions of the country. FPA officials and the leaders of the 
interagency science team said that FPA is being designed to allow for 
the agencies to evaluate different weighting schemes, which senior 
agency officials referred to as "exploring the decision space." Others, 
however, have raised concerns that the flexibility inherent in setting 
weights for the different performance measures will allow the agencies 
to manipulate these weights until they reach a predetermined outcome. 
Without understanding the weights assigned to each measure and the 
rationale for assigning those weights--that is, without transparency in 
this process--Congress and others will find it impossible to understand 
and evaluate the reasonableness of FPA's results, and skepticism about 
FPA's usefulness will be difficult to quell. 

Second, senior agency officials emphasized that, despite its 
importance, FPA will not be the sole determining factor in developing 
their budget requests and allocating appropriated funds; rather, senior 
agency officials would consider FPA results along with other 
information and exercise managerial discretion in making these 
decisions. Agency officials said, for example, that they would continue 
to involve national and regional officials from the various agencies to 
help ensure that their budget requests reflected differences in 
priorities among the agencies or regions, although they recognized that 
this process might lead FPA results to be used differently by different 
agencies or in different regions. Although considering these factors is 
important, as with the setting of the weights, it will also be 
important for the agencies to clarify the additional factors beyond FPA 
that they consider in developing their budget requests, so that 
Congress and others can understand FPA's role in the process. 

And third, once Congress has appropriated funds to the agencies, it is 
not clear how the agencies will use FPA to help allocate these funds to 
the field. If one agency allocated funds differently than suggested by 
FPA--or if one agency's field unit acquired a different mix of assets 
than it modeled--it could affect the other agencies' ability to protect 
important resources, as well as the overall effectiveness of the 
agencies' fire management program. Agency officials said they intended 
for each agency to consider FPA results in allocating its funds and for 
field units to consider FPA results in acquiring firefighting assets. 
They also said they would not decide how much to deviate from the 
allocation suggested by FPA until they had begun to analyze the first 
year's results. Officials also said that it is important to recognize 
that more than 2 years could elapse between field units' developing 
their alternatives and Congress's appropriating funds on the basis of 
that information--and that priorities could change substantially in the 
interim, leading the agencies to allocate funds differently than 
suggested by FPA. Agency officials also said that, in addition to FPA 
results, they would consider specific congressional earmarks and 
appropriations guidance when allocating funds. Moreover, the agencies 
have existing systems outside of FPA for allocating fuel reduction 
funds, which they have been working in recent years to 
improve.[Footnote 22] As of November 2008, agency officials did not 
know how they would consider the information from FPA in relation to 
the agencies' other systems in allocating fuel reduction funds. 

Conclusions: 

As fires become more severe and development in fire-prone lands 
continues, the Forest Service and Interior agencies face difficult 
decisions about how to best protect the nation's communities and 
natural and cultural resources. In particular, the agencies must 
determine the best mix and location of firefighting assets to respond 
to wildland fires, and they must balance the need to spend money 
preparing for and fighting fires against the need to invest in reducing 
potentially hazardous fuels so as to lower both the cost of suppressing 
future fires and the risk to communities and important resources. 
Complicating these decisions, our nation's long-term fiscal challenges 
have constrained agency budgets, simultaneously limiting available 
choices and making it even more important to spend funds efficiently 
and effectively. The agencies believe that FPA can be a useful tool in 
making these difficult choices, which will drive billions of dollars in 
federal expenditures each year and directly affect millions of citizens 
living in fire-prone areas. By establishing an interagency budget 
framework that analyzes trade-offs among the most important fire 
management program activities, FPA represents an important first step 
in improving the agencies' cost-effectiveness. 

Achieving the full potential of FPA, however, will depend on the extent 
to which the agencies improve FPA's ability to live up to the promises 
that were made on its behalf--namely, that it would allow the agencies 
to develop rational budgets and allocate funds in a way that maximizes 
the agencies' ability to manage wildland fire. Living up to these 
promises presents a daunting challenge, given the inherent difficulty 
of modeling the complexities and uncertainties of wildland fire and 
given that FPA remains a work in progress. Nevertheless, an early 
assessment of the model's capabilities raises several issues. The 
overall modeling approach the agencies have chosen does not allow them 
to identify the most cost-effective mix and location of firefighting 
assets, one of FPA's key objectives. Moreover, without improvements, 
FPA will be unable to identify, from a national perspective, the most 
important resources to protect or the relative priority of different 
values at risk; to evaluate the effect of different investments in fuel 
reduction treatments and firefighting strategies over time; or to 
analyze the effect of changes in the number of aircraft and experienced 
firefighters that are under regional or national control. Without such 
improvements, the agencies will continue to lack important information 
on which to base decisions about how best to allocate scarce funds. 
Further, the agencies have not yet determined how they will weigh the 
relative importance of FPA's five performance measures or exactly how 
they will use FPA to develop their budget requests and allocate funds. 
Given the importance of and the uncertainty surrounding these 
decisions, Congress--as well as the agencies and other interested 
parties--would benefit if these fundamental budget decisions were made 
in a transparent manner. And finally, an external peer review by an 
independent entity, such as the National Academy of Sciences, would 
achieve one of FPA's objectives and help the agencies identify the 
strengths and limitations of the model, which could increase confidence 
in their decisions and help them make needed changes more quickly. 

Recommendations for Executive Action: 

We recommend that the Secretaries of Agriculture and the Interior take 
four actions to improve their agencies' abilities to develop their 
budget requests and allocate funds using FPA. 

First, to improve the FPA model's ability to identify needed 
firefighting assets and the best locations for these assets--and 
recognizing that developing FPA will be an iterative process that will 
require the agencies' continued effort to improve--we recommend that 
the Secretaries of Agriculture and the Interior direct the agencies to 
develop a strategic plan for the continued development of FPA, which 
would (1) include an evaluation of FPA's ability to meet its key 
original objectives; (2) identify ways to improve the model to better 
meet these objectives; (3) clearly state whether the agencies believe 
any of the original objectives are no longer appropriate, and why; and 
(4) identify the steps the agencies plan to take to improve FPA and the 
expected time frames and associated budget needs for completing these 
steps. To allow the agencies sufficient time to identify issues that 
may arise as they implement FPA, the Secretaries of Agriculture and the 
Interior should submit this plan to Congress no later than September 
30, 2010. In particular, we believe that the strategic plan should, at 
a minimum, address ways to improve FPA's ability to: 

* evaluate different mixes and locations of firefighting assets, so 
that FPA recognizes the relative priority of different values at risk 
when assessing how best to protect the wildland-urban interface and 
increase the number of acres meeting fire management objectives; 

* identify the most highly valued resources, such as endangered species 
habitat or important cultural sites, that the agencies seek to protect; 

* model the effects over time of different investments in fuel 
reduction treatments and firefighting strategies on the cost of 
suppressing future wildland fires; and: 

* analyze trade-offs between increases and decreases in firefighting 
assets that are under national or regional control. 

Second, we recommend that the Secretaries of Agriculture and the 
Interior report annually to Congress on (1) their progress in 
completing the steps outlined in the strategic plan for the continued 
development of FPA and (2) FPA's ability to meet each of its key 
objectives. 

Third, to increase agency transparency in using FPA to develop their 
budget requests and allocate funds, we recommend that the Secretaries 
of Agriculture and the Interior report annually to Congress on FPA's 
role in the budget development and allocation process. This report 
should include, at a minimum, information on (1) how the agencies 
weighted the measures FPA uses to evaluate different mixes and 
locations of firefighting assets and the rationale for those weights, 
(2) how FPA results were used in conjunction with other information in 
developing the agencies' budget requests, and (3) the extent to which 
the agencies' funding allocations to their field units reflected the 
FPA results for a given year. 

Fourth, to increase Congress's and the agencies' understanding of the 
strengths and limitations of FPA--including the extent to which it 
achieves the key objectives envisioned by the 2001 report--and to 
fulfill one of the original objectives established for FPA, we 
recommend that the Secretaries of Agriculture and the Interior direct 
the agencies to submit the FPA model to external peer review. This 
review should be initiated as soon as FPA is complete enough to allow 
for a thorough review, but no later than November 2009, so that its 
results can inform decisions about how FPA may be improved and the 
extent to which additional funding should be provided to the project. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, the Forest Service and 
Interior disagreed with our finding that FPA is unlikely to allow the 
agencies to identify the most cost-effective mix of firefighting 
assets. The Forest Service commented that it fundamentally agreed with 
our recommendations and described the steps the agency intended to take 
in addressing them, but we do not believe that the steps outlined in 
the letter are specific and transparent enough to meet the intent of 
our recommendations. Interior disagreed with our recommendation that 
the agencies develop a strategic plan for the continued development of 
FPA but concurred with our other recommendations. 

The Forest Service and Interior commented that they believe FPA will 
allow them to meet the goal of cost-effectiveness. As their letters 
state, we previously discussed our conclusions on this issue with the 
agencies but did not resolve the differing points of view. As stated in 
our report, FPA compares only a limited number of mixes of firefighting 
assets and firefighting strategies, and the alternatives it evaluates 
are likely to reflect only minor variations in budget levels. Given 
this structure, we continue to believe that FPA is unlikely to allow 
the agencies to identify the most cost-effective location and mix of 
assets and strategies nationwide--an objective the agencies themselves 
established in their 2001 report. In their responses, both agencies 
raised questions about this objective. The Forest Service's comments 
seek to invalidate the objective altogether, stating that identifying 
the single most cost-effective mix of assets and strategies is not 
realistic. Interior did not question the validity of the objective but 
stated that the approach FPA is taking is more realistic than the 
approach the agencies had taken when they first began developing FPA. 
While we are not altering our conclusion that FPA's current approach 
will likely keep the agencies from identifying the most cost-effective 
solution, we are modifying our first recommendation to state that in 
the strategic plan, the agencies not only identify ways to improve the 
model to better meet FPA's original objectives, but also clearly state 
whether they believe any of the original objectives are no longer 
appropriate--and, if not, why not--in order to ensure that Congress and 
other interested parties are fully informed about what they can 
reasonably expect from FPA. 

Regarding our recommendation that the agencies develop a strategic plan 
for the continued development of FPA, the Forest Service concurred with 
our recommendation and stated that it has a strategy for completing 
FPA, although it is not clear from the letter whether this strategy is 
or will be articulated in a written document directly addressing the 
elements of our recommendation. In contrast, Interior disagreed with 
this recommendation, stating that developing a strategic plan would 
delay the deployment and increase the cost of FPA. Regarding Interior's 
position, we are not suggesting that the agencies delay implementing 
FPA until they have developed the strategic plan we recommend; rather, 
we believe that such a plan can be developed concurrently with 
implementation and in fact may benefit from incorporating lessons 
learned during early use of FPA. More broadly, because of FPA's 
importance and the concerns about its development--including the 
questions raised in our review about its ability to meet its key 
objectives--we believe it is important for the agencies to create a 
strategic plan for FPA's continued development that directly and 
transparently evaluates FPA's ability to meet its original objectives, 
identifies ways to improve FPA to better meet those objectives, and 
identifies the steps the agencies plan to take to improve FPA. Given 
the agencies' comments about FPA's cost-effectiveness objective, 
however, we modified the language of our recommendation on developing a 
strategic plan, as discussed above. 

The Forest Service and Interior generally agreed with our 
recommendations to report annually to Congress on the continued 
development of FPA and on FPA's role in the budget development process, 
and to submit the FPA model to external peer review. The Forest 
Service, however, also provided clarifications on two of our 
recommendations that did not appear to be fully responsive in terms of 
the amount of information and transparency we believe is warranted. 
Specifically, in response to our recommendations that the agencies 
report annually to Congress on (1) their progress in completing the 
strategic plan for FPA's continued development, and on FPA's current 
ability to meet each of its key objectives, and (2) FPA's role in the 
agencies' budget development and allocation process, the Forest Service 
stated that it has always been--and will continue to be--responsive to 
congressional requests for information and that it would include 
information on FPA's role in budget development and allocation in its 
annual budget requests. We are not convinced, however, that this 
approach will furnish Congress with the consistent, transparent, and 
complete information we believe it needs--particularly given FPA's 
importance in helping the agencies manage their $3 billion wildland 
fire program and the concerns about its development. We continue to 
recommend, therefore, that the Secretaries of Agriculture and the 
Interior prepare an annual report to Congress about the status of FPA's 
development and how the agencies have used FPA to help develop their 
budget requests and allocate funds. The Forest Service's and Interior's 
letters are reprinted in appendixes II and III, respectively, along 
with our evaluation of specific comments. 

We are sending copies of this report to interested congressional 
committees, the Secretaries of Agriculture and the Interior; the Chief 
of the Forest Service; the Directors of the Bureau of Indian Affairs, 
Bureau of Land Management, Fish and Wildlife Service, and National Park 
Service; and other interested parties. The report is also available at 
no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff have questions about this report, please contact 
me at (202) 512-3841 or nazzaror@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. Key contributors to this report are 
listed in appendix IV. 

Sincerely yours, 

Signed by: 

Robin M. Nazzaro: 

Director, Natural Resources and Environment: 

[End of section] 

Appendix I: Scope and Methodology: 

To determine how the agencies have developed the fire program analysis 
(FPA) budget-planning system to date, we reviewed agency documents from 
each stage of FPA's development. To identify the key objectives 
originally established for FPA, we reviewed congressional committee and 
Office of Management and Budget guidance to the agencies; a 2001 
report, commissioned and later adopted by the agencies, that 
established the vision, key objectives, time frames, and rationale for 
what FPA was intended to accomplish; the interagency memorandum of 
agreement and project charter that established FPA as an interagency 
project; and other agency documents. To further our understanding of 
the broader context for the shortcomings FPA was intended to address, 
we reviewed key agency documents, including the 1995 and 2001 federal 
wildland fire management policies, the national fire plan, and related 
documents. To identify changes the agencies made to FPA in 2006, and 
the reasons for those changes, we reviewed the report the agencies 
issued after their review of FPA's policy and scientific approaches; a 
response to that report prepared by those who had helped to develop 
FPA; and internal agency briefing materials about the changes. To 
identify the likely capabilities of FPA as the agencies have been 
developing it since 2006, we reviewed the draft interagency science 
team report that formed the basis for FPA's new modeling approach and 
numerous technical papers and other documentation describing particular 
aspects of FPA. To further our understanding of FPA's development at 
each of these stages, we interviewed Forest Service and Department of 
the Interior officials in Washington, D.C; FPA project staff in Boise, 
Idaho; and agency officials in the field familiar with FPA. We also 
interviewed agency and other scientists who have helped develop FPA. 

To determine the extent to which FPA meets its original objectives, we 
compared--to the extent possible--the capabilities of FPA as the 
agencies developed it with those envisioned in congressional committee 
guidance and the 2001 report. At the time of our review, however, 
substantial portions of the model remained incomplete, and the agencies 
had not sufficiently documented the model to allow a comprehensive 
evaluation. We therefore limited our review to a broad examination of 
FPA's various components and how they interact. We also interviewed 
senior agency officials, FPA project staff, agency field officials, and 
agency and other scientists to obtain their views on the extent to 
which FPA appears capable of meeting its key objectives, as well as 
possible changes that could improve the model's ability to meet those 
objectives. 

We conducted this performance audit between September 2007 and November 
2008 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix I: Comments from the Department of Agriculture, Forest 
Service: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

United States Department of Agriculture: 
Forest Service: 
Washington Office: 
1400 Independence Avenue, SW: 
Washington, DC 20250: 

File Code: 1420/5100: 

Ms. Robin M. Nazzaro: 
Director, Natural Resources and Environment: 
United States Government Accountability Office: 
4441 G Street, N.W.: 
Washington, DC 20548: 

Dear Ms. Nazzaro:

Thank you for the opportunity to comment on the draft Government 
Accountability Office (GAO) report, GAO-09-68, "Wildland Fire 
Management: Interagency Budget Tool Needs Further Development to Fully 
Meet Key Objectives." The Fire Program Analysis (FPA) project is a very 
significant and challenging undertaking by the Federal wildland fire 
agencies. We were pleased the audit team discussed the complexities 
inherent in developing an interagency planning and budget system. While 
we generally view the audit as supportive, we respectfully disagree 
with GAO's conclusion that our approach hampers FPA from meeting the 
key objective of cost effectiveness. We discussed this with the audit 
team; they were receptive to the discussion and recommended that we 
document our concern. 

As indicated, we worked closely with GAO, commenting and clarifying 
statements; however, not all of these have been reflected in key 
aspects of the draft report. We believe GAO has not accurately 
portrayed the system's ability to meet the cost effectiveness objective 
and actions we have taken to assure that FPA is a useful planning and 
budgeting tool. GAO takes exception to FPA system design modifications 
in 2006 that it says compromises the agencies' ability to fully achieve 
key goals. We strongly disagree that these modifications compromise 
cost effectiveness. 

See comment 1. 

GAO contends that the analytical approach developed by the agencies 
cannot identify the most cost effective mix and location of federal 
firefighting assets for a given budget, but rather can only compare the 
relative cost effectiveness of a small set of alternatives. This 
assertion fails to recognize or acknowledge several key points. First, 
the notion of a single most cost-effective solution is conceptual; it 
is not based in reality. Any reasonable alternative can be shown to be 
optimal under a hypothetical set of assumptions about fire occurrence, 
weather, and values placed on the consequences of individual fires. 
That same alternative would be judged inferior to other alternatives 
under differing sets of assumptions, even if the differences are minor. 
Furthermore, building an optimization model requires gross 
simplification of the system being modeled in order to calculate a 
solution. This simplification can lead to distorted or inaccurate 
representations of the relationships between actions and outcomes, 
which reduces the confidence placed in a given solution. These and 
other shortcomings of the classical optimization approach were manifest 
in the first phase of FPA and led the agencies to change their 
approach. 

The analytical approach recommended by the Interagency Science Team and 
adopted by the agencies remains faithful to the goal of improving 
firefighting effectiveness. It also improves the ability to: A) address 
the uncertainty inherent in wildland fire due to random variations in 
weather, fuels, and topography; B) more realistically model fire 
behavior and the strategic and tactical choices made by wildland fire 
managers and; C) build upon the corporate intelligence gained from 
decades of firefighting by the agencies and their partners. 

See comment 2. 

This approach will allow us to systematically evaluate alternative 
investment strategies and identify options that best reduce fire 
losses, improve ecological conditions and increase cost efficiency. The 
system is designed to explicitly address uncertainty and risk in 
predicting future wildland fires. A combination of simulation models 
and goal programming will array alternatives using quantitative 
performance measures that display inherent risks and trade- offs at 
both local and national levels. This approach is a more robust basis 
for modeling real- world complexities than the linear optimization 
approach originally used in FPA, while maintaining the ability to 
compare the performance and effectiveness of alternative funding 
decisions. 

We would also like to offer comments relative to the system's ability 
to evaluate fuel reduction investments over time. GAO states that we 
are working "to develop an approach that would allow FPA to better 
analyze the long-term effect of reducing fuels" but that due to the 
development time frame they were unable to evaluate it, reaching the 
opinion "it appears that FPA's ability to help the agencies achieve 
this objective will be limited." GAO concludes that "without 
improvements FPA will be unable to evaluate the effect of different 
investments in fuel reduction and firefighting strategies over time." 
We want to reaffirm our commitment to ensure that the system is useful 
and that it supports, both near-term and long-term, fire planning and 
budgeting. To that end, our development and science teams are 
aggressively analyzing temporal modeling approaches for fuel treatments 
which could be released later this year. In addition, the performance 
metric "proportion of land meeting or trending toward the attainment of 
fire and fuels management objectives" will recognize and consider acres 
managed under the appropriate management response. The system now being 
deployed is within the scope approved by the Agency. However, the 
Forest Service has recognized the potential for a more comprehensive 
analysis of vegetation and fuel treatments that will address the 
concerns expressed by GAO. 

See comment 3. 

GAO Recommendations - The report identifies four recommendations to 
improve FPA and its use in the budget process. The Forest Service 
fundamentally agrees with GAO's recommendations, but believes there are 
better alternative approaches for implementing three of the 
recommendations than those proposed by GAO. 

Recommendation 1: The Secretaries of Agriculture and the Interior 
direct the agencies to develop a strategic plan for the continued 
development of FPA, which would include: 1) an evaluation of the 
strengths and weaknesses of FPA; 2) identify ways to improve the model 
to better meet its intended objectives, and; 3) identify the steps the 
agencies plan to take to improve FPA and expected time frames and 
associated budget. 

Response - The Forest Service has a strategy for completing development 
and implementation of the FPA system consistent with the project's 
charter, and its associated plans. The Forest Service has recognized 
the need to implement FPA in an adaptive manner in FY 2009 through a 
staged approach that allows for system adjustments as experience 
dictates. This approach is both helping to avoid workload issues for 
the system and development of personnel and facilitating our ability to 
address system issues as they arise. The enclosed graphic displays the 
schedule and methodology for implementation across all Fire Planning 
Units. As we use the system and review and analyze its outputs, its 
strengths and weaknesses will be identified and documented through 
existing business processes. In addition, the planned external peer 
review will provide insight into FPA's strengths and weaknesses. 

See comment 4. 

Enclosure not reprinted. 

Major development of FPA will be complete in FY 2009 when the system 
will transition from development to operation and maintenance. The 
system's Operation and Maintenance Plan will provide for some 
enhancements to the system's models. In addition, the Forest Service 
has recognized other modeling components, such as a focused analysis of 
national resources or an expanded analysis of fuels and vegetative 
treatments, which could be useful and potentially provide a more 
comprehensive range of alternatives. These will be considered as the 
system is deployed and insights into outputs, and their utility, become 
known. The FPA Executive Oversight Group will consider these and other 
enhancements and provide guidance relative to their future inclusion 
and development. 

Key aspects of these actions and activities will be conveyed in 
accordance with the Forest Service's response to Recommendations 2 and 
3. 

Recommendation 2: The Secretaries of Agriculture and the Interior 
report annually to Congress on: 1) their progress in completing the 
steps outlined in the strategic plan for the continued development of 
FPA; and, 2) FPA's ability to meet the key objectives initially 
established for it. 

Response - The Forest Service agrees with informing Congress about the 
progress in implementing FPA and how the results of FPA are being used 
in agency decision processes. The agency has always been responsive to 
Congressional requests for information through informal briefings, 
responses to written questions, and formal testimony. The Forest 
Service will continue to respond to any requests for information by 
members and committees. In addition, the results and use of FPA 
information will be clearly highlighted in the Agency's formal annual 
budget requests to Congress. 

See comment 5. 

Recommendation 3: To increase agency transparency in using FPA to 
develop their budget requests and allocate funds, the Secretaries of 
Agriculture and the Interior report annually to Congress on FPA's role 
in budget development and allocation process. 

Response – Please see response to recommendation 2. 

Recommendation 4: The Secretaries of Agriculture and the Interior 
direct the agencies to submit the FPA model to external peer review. 

Response - The Forest Service agrees with the recommendation. An 
external peer review is planned as part of the FY 2009 development and 
implementation strategy. 

Please contact Sandy T. Coleman, Forest Service Assistant Director for 
GAO/OIG Audit Liaison staff, at 703-605-4699, with any questions. 

We look forward to working with GAO in the future. 

Sincerely,

Signed by: 

Abigail R. Kimbell: 
Chief: 

cc: Sandy T Coleman, Clarice Wesley, Tom Harbour, Bill Breedlove, Rick 
Prausa: 

The following are GAO's comments on the Department of Agriculture, 
Forest Service's letter dated November 5, 2008. 

GAO Comments: 

1. As the Forest Service's comment letter indicates, we have had 
extensive discussions with the agency on FPA's ability to identify the 
most cost-effective mix of firefighting assets, without resolving our 
differing points of view. As we describe in our report, FPA compares 
only a limited number of mixes of firefighting assets and firefighting 
strategies, and further, the alternatives it evaluates are likely to 
reflect only minor variations in budget levels. Given this structure, 
we continue to believe that FPA is unlikely to allow the agencies to 
identify the most cost-effective location and mix of assets and 
strategies--an objective the agencies themselves established in their 
2001 report. Rather than directly contradicting our conclusion, 
however, the Forest Service's letter seeks instead to invalidate this 
objective altogether. The Forest Service commented that identifying the 
most cost-effective mix of assets and strategies is not a realistic 
objective and that FPA's current combination of simulation models and 
goal programming is a preferable approach. We did not compare the 
agencies' current approach with their initial approach, nor have we 
concluded whether one approach is more suitable or realistic than the 
other. Rather, in accordance with the objectives of our review, we 
simply evaluated the extent to which FPA as it is currently being 
developed is likely to meet the objectives originally established for 
it. While we are not altering our conclusion that FPA's current 
approach will likely not result in identifying the most cost-effective 
solution, we are modifying our first recommendation to suggest that the 
agencies clarify which of FPA's original objectives they believe are no 
longer appropriate and why. See comment 4 below. 

2. The Forest Service stated that FPA remains faithful to the goal of 
improving firefighting effectiveness and that FPA's approach will 
provide a more robust basis for systematically evaluating alternative 
investment strategies. As discussed above, however, and as we noted in 
our draft report, the objective originally established for FPA was to 
identify the most cost-effective mix and location of firefighting 
assets and strategies, not simply to improve firefighting 
effectiveness. 

3. The Forest Service's letter reaffirmed the agency's commitment to 
ensuring that FPA is able to support both near-term and long-term 
planning considerations in evaluating fuel reduction investments. In 
reaffirming this commitment, the Forest Service stated that FPA, as it 
is being developed, fulfills the scope that has been approved by the 
agency. This approved scope, however, has evolved during FPA's 
development and is not fully consistent with the objectives initially 
established for FPA. Our conclusions about FPA are based on our 
comparison of its current capabilities with the objectives originally 
established for it. We did not determine whether the agencies were 
developing FPA in a manner that fulfilled the scope approved by the 
agencies in subsequent documents. 

4. The Forest Service stated that it has a strategy for completing FPA 
"consistent with the project's charter and its associated plans." It is 
not clear from the letter, however, whether this strategy is, or will 
be, articulated in a written document directly addressing the elements 
of our recommendation. Because of FPA's importance, and the concerns 
that have arisen during its development, we believe it is important for 
the agencies to develop a single document that addresses these issues 
transparently. Given the agencies' comments, however, we modified our 
recommendation to suggest that the agencies use this plan not only to 
identify ways to improve the model to better meet FPA's original 
objectives, but also to clearly state whether they believe any of the 
original objectives are no longer appropriate, and why, in order to 
ensure that Congress and other interested parties are fully informed 
about what they can reasonably expect from FPA. 

5. The Forest Service stated it would respond to any requests for 
information by Congress and would highlight how FPA's results were used 
in the agency's annual budget request. We are not convinced, however, 
that this approach will provide Congress with the consistent, 
transparent, and complete information we believe it needs--particularly 
given FPA's importance in helping the agencies manage their $3 billion 
wildland fire program and the concerns about its development. We 
continue to recommend, therefore, that the Secretaries of Agriculture 
and the Interior prepare an annual report to Congress about the status 
of FPA's development and how the agencies have used FPA to help develop 
their budget requests and allocate funds. 

[End of section] 

Appendix II: Comments from the Department of the Interior: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

The Associate Deputy Secretary Of The Interior: 
Washington: 

November 4, 2008: 

Ms. Robin M. Nazzaro: 
Director. Natural Resources and Environment: 
Government Accountability Office: 
441 G Street, NW: 
Washington, D.C. 20548-0001: 

Dear Ms. Nazzaro: 

We appreciate the opportunity to review and comment on the draft 
Government Accountability Office report entitled, "Wildland Management 
Interagency Budget Tool Needs Further Development to Fully Meet Key 
Objectives." (GAO-09-68). The Fire Program Analysis project is a very 
significant and challenging undertaking by the Federal wildland fire 
agencies We were pleased the audit team engaged in numerous and 
constructive discussions to understand the complexities inherent in 
developing an interagency planning and budget system. While we view the 
audit as supportive of our effort we respectfully disagree with GAO's 
conclusion that our approach hampers FPA from meeting key objectives. 
We discussed this concern with the audit team: they were receptive to 
the discussion and recommended that we document our concern. 

We arc concerned that our comments and clarifying information have not 
been reflected in the report. In particular, we believe that FPA will 
allow us to meet the cost effectiveness objective and that actions we 
have taken will ensure that FPA will he a useful planning and budgeting 
tool. The FPA will he useful in meeting the cost effective objective 
and he a useful budgeting and planning took because FPA is an 
interagency analysis that is based on collaboration at the local level. 
The results of the collaborative analysis process will display 
efficiencies and effectiveness identified at the local planning level. 
We also believe that the 2006 system modifications support the goal of 
cost effectiveness. The cost effectiveness is based on quantitative 
performance measures that display trade-offs locally and nationally 
between various budget levels. The FPA system allows the local planning 
unit to determine the most effective mix of resources and fuel 
treatments at various budget levels. The FPA does not provide one 
finite answer but instead evaluates the changes in modeled 
effectiveness for each investment level analyzed. 

See comment 1. 

We believe that one of FPA's greatest strengths is the capability to 
evaluate alternative investment strategies and identify options that 
best reduce lire losses, improve ecology conditions. and increase cost 
efficiency. This ability to array alternatives Mal display risks and 
trade-offs at the local and national levels provide a robust basis for 
comparing potential performance and cost effectiveness will optimize 
the use of resources. We believe this is a much more realistic approach 
then the single-dimensional optimization approach for the most cost 
effective mix and location of firefighting asset for a given budget. 
Our Interagency Science Team has recommended the more analytical 
approach that we are now developing. 

With respect to the system's ability to evaluate fuel reduction 
investments over time, it is important to note that our development and 
science teams are analyzing temporal modeling approaches for fuel 
treatments, which may be released later this year. We agree with GAO 
that there is potential for a more comprehensive analysis of vegetation 
and fuel treatments. 

See comment 2. 

Our comments on the recommendations are as follows: 

Recommendation 1— The Secretaries of Agriculture and the Interior 
develop a strategic plan for the continued development of FPA, which 
would (1) include an evaluation of the strengths and weaknesses of FPA, 
(2) identify ways to improve the model to better men its intended 
objectives, and (3) identify the steps the agencies plan to take to 
improve FPA and expected time frames and associated budget needs for 
completing these steps. 

Response— The results of the reviews of FPA that were conducted by the 
science team and the management team support the current course of 
action. The system is expected to provide results that can be used in 
2009 and to be available in time to develop the fiscal year 2011 budget 
request. The fire community is anxiously awaiting access to this system 
that will give them new tools to use in decision-making and improve 
their ability to allocate resources in a cost effective manner. 
Development of a strategic plan will further delay system deployment 
and result in increased costs. We would prefer to continue forward with 
the planned, staged approach that allows for adaptive use and 
modification. We believe that this approach will allow us to ensure 
that the system optimizes capabilities for evaluation of different 
mixes and locations of firefighting assets, protection of valued 
resources, modeling investments, and analyzing trade-offs in the 
allocation of firefighting assets. In addition, our planned external 
peer review is expected to further assist in identifying improvements 
to the system. 

See comment 3. 

Recommendation 2— The Secretaries of Agriculture and the Interior 
report annually to Congress on (I) their progress in completing the 
steps outlined in the strategic plan for the continued development of 
FPA, and (2) FPA's current ability to meet each of the key objectives 
initially established for it. Response— The Department concurs with 
GAO's recommendation to report annually to Congress on progress and 
achievement of objectives. 

Recommendation 3— Increase agency transparency in using FPA to develop 
their budget requests and allocate funds by reporting to Congress on 
FPA's role in budget development and allocation process. 

Response— The Department concurs with GAO's recommendation and will 
ensure that the results and use of FPA information is clearly depicted 
in the budget request to Congress. 

Recommendation 4— Submit the FPA model to external peer review. 
Response—The Department concurs with GAO's recommendation and is 
planning an external peer review. 

We have closely coordinated our response with the U.S. Forest Service 
and hold concurrent views. If you have any questions or concerns, 
please contact Barbara Loving at the Office of Wildland Fire 
Coordination at 202-606-3108. 

We look forward to working with GAO in the future. 

Sincerely,

Signed by: 

James E. Cason: 
Enclosure: 

Enclosure not printed. 

The following are GAO's comments on the Department of the Interior's 
letter dated November 5, 2008. 

GAO Comments: 

1. As Interior's comment letter indicates, we have had extensive 
discussions with agency officials on FPA's ability to identify the most 
cost-effective mix of firefighting assets without resolving our 
differing points of view. Interior commented that FPA will allow the 
department to meet FPA's cost-effectiveness objective by evaluating 
alternative investment strategies and identifying options that best 
reduce fire losses, improve ecological conditions, and increase cost 
efficiencies and that the agencies' current approach is much more 
realistic than the approach taken initially. As we describe in our 
report, FPA compares only a limited number of mixes of firefighting 
assets and firefighting strategies, and further, the alternatives it 
evaluates are likely to reflect only minor variations in budget levels. 
Given this structure, we continue to believe that FPA is unlikely to 
allow the agencies to identify the most cost-effective location and mix 
of assets and strategies--an objective the agencies themselves 
established in their 2001 report. And as noted in our response to the 
Forest Service's comments, we did not compare the agencies' current 
approach with their initial approach, nor do we conclude whether one 
approach is more suitable or realistic than the other. Rather, in 
accordance with the objectives of our review, we simply evaluated the 
extent to which FPA as it is currently being developed is likely to 
meet the objectives originally established for it. While we are not 
altering our conclusion that FPA's current approach will likely keep 
the agencies from identifying the most cost-effective solution, we are 
modifying our first recommendation to suggest that the agencies clarify 
which of FPA's original objectives they believe are no longer 
appropriate and why. See comment 3 below. 

2. Interior commented that the agencies are continuing to analyze how 
FPA evaluates fuel reduction investments over time and may begin using 
a new modeling approach later in 2008. It is not clear from Interior's 
letter whether it believes the new approach will allow the agencies to 
meet FPA's original objective of modeling the effects over time of 
differing strategies for responding to wildland fires and treating 
lands to reduce hazardous fuels. Our review of the limited 
documentation describing this approach suggests that this approach is 
unlikely to allow FPA to fully meet this key objective. 

3. Interior stated that developing a strategic plan for the continued 
development of FPA, as we are recommending, would further delay 
deployment and increase the cost of FPA. We recognize it is important 
for the agencies to continue to develop FPA, and we are not suggesting 
that the agencies delay implementing FPA until they have developed the 
strategic plan we recommend. On the contrary, we believe that such a 
plan can be developed concurrently with implementation and that the 
agencies may benefit from incorporating lessons learned during FPA's 
early use into the plan. In any event, our review raised questions 
about FPA's ability to meet certain of its key objectives, even with 
the changes the agencies are considering making to FPA--and because of 
FPA's importance, and the concerns about its development, we believe it 
is important for the agencies to develop a single document that 
directly and transparently evaluates FPA's ability to meet its original 
objectives and identifies ways to improve FPA to better meet those 
objectives. Given the agencies' comments, however, we modified our 
recommendation to suggest that the agencies use this plan not only to 
identify ways to improve the model to better meet FPA's original 
objectives, but also to clearly state whether they believe any of the 
original objectives are no longer appropriate, and why, in order to 
ensure that Congress and other interested parties are fully informed 
about what they can reasonably expect from FPA. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Robin M. Nazzaro, (202) 512-3841 or nazzaror@gao.gov: 

Staff Acknowledgments: 

In addition to the contact person named above, Steve Gaty, Assistant 
Director; David P. Bixler; Ellen W. Chu; Jonathan Dent; Richard 
Johnson; Chester Joy; Mehrzad Nadji; Jacqueline Nowicki; and Dae Park 
made key contributions to this report. 

[End of section] 

Footnotes: 

[1] Departments of Agriculture and the Interior, Federal Wildland Fire 
Management Policy and Program Review (Washington, D.C., December 1995). 
This policy was subsequently reaffirmed and updated in 2001: 
Departments of the Interior, Agriculture, Energy, Defense, and 
Commerce; Environmental Protection Agency; Federal Emergency Management 
Agency; and National Association of State Foresters, Review and Update 
of the 1995 Federal Wildland Fire Management Policy (Washington, D.C., 
January 2001). 

[2] Forest Service and Department of the Interior, Developing an 
Interagency, Landscape-scale Fire Planning Analysis and Budget Tool 
(Washington, D.C., November 2001). 

[3] Departments of Agriculture and the Interior, Fire Program Analysis: 
Scientific Review Team Report (Washington, D.C., January 2006); and 
Management Review Team Report of the Fire Program Analysis (FPA) 
Preparedness Module (Washington, D.C., March 2006). 

[4] Together, preparedness, suppression, and fuel reduction make up 
approximately 80 percent of the agencies' wildland fire management 
budgets. Other federal wildland fire program components include 
financial assistance to state foresters for fire management activities, 
research and development, and rehabilitating burned federal lands. 

[5] Federal and nonfederal agencies have established a framework to 
share the costs of responding to fires that threaten both federal and 
nonfederal resources. See GAO, Wildland Fire Suppression: Lack of Clear 
Guidance Raises Concerns about Cost Sharing between Federal and 
Nonfederal Entities, GAO-06-570 (Washington, D.C.: May 30, 2006). 

[6] The number of planning units established by the agencies has 
fluctuated over the course of FPA development. In this report, we refer 
to the 139 planning units in existence at the time our review ended but 
recognize that the actual number at any particular time may differ. 

[7] Peer review is a process by which scientific research or technical 
projects are subject to an independent assessment by scientists not 
involved with the project who have knowledge and expertise comparable 
to that of the scientists whose work they review. 

[8] The Wildland Fire Leadership Council consists of senior Agriculture 
and Interior officials, including the Agriculture Undersecretary for 
Natural Resources and Environment; the Interior Assistant Secretary for 
Policy, Management, and Budget; and the heads of the five federal 
firefighting agencies. Other members include representatives of the 
Intertribal Timber Council, the National Association of State 
Foresters, and the Western Governors' Association, and a local fire 
department chief. 

[9] The size threshold for fires to be considered contained while small 
is to vary according to criteria established by officials in each 
planning unit, considering the circumstances under which they typically 
consider a fire in their area "escaped" and then request additional 
firefighting assets to help suppress it. This measure also includes the 
number of fires the model predicts would be averted because of the 
agencies' efforts to prevent human-caused fires. 

[10] The program staff who helped develop the old approach told us that 
they had recognized the small number of potential fire scenarios in 
that approach limited its capabilities and that they were considering 
how to improve it, but the agencies determined that a new modeling 
approach was needed before they could make improvements. 

[11] The agencies have provided brief updates on the status of FPA to 
Congress in their annual budget justifications, and have provided 
periodic briefings to congressional committee and OMB staff. 

[12] These figures include the cost of developing FPA and operating and 
maintaining it through fiscal year 2010. 

[13] Similarly, the project manager said that the decrease in estimated 
cost for the second phase from 2005 to 2006 was also due to a better 
understanding of the project's scope. 

[14] FPA is also to help the agencies model their investment in 
preventing fires. The agencies carry out activities, such as increased 
law enforcement patrols and public education programs, intended to 
reduce the number of human-caused wildland fires. FPA is to predict the 
number of fires that would have started if not for the agencies' 
prevention activities. 

[15] LANDFIRE is a geospatial data and modeling system designed to 
assist the agencies in identifying the extent, severity, and location 
of wildland fire threats to the nation's communities and ecosystems. At 
the time of our review, LANDFIRE data were not available for the 
eastern United States or for Alaska and Hawaii. FPA officials said that 
until LANDFIRE data are available nationwide, they are using other 
available data to provide similar information. FPA officials expect 
that LANDFIRE data will be available nationwide by 2009. 

[16] We have previously reported limitations of the model the agencies 
use to predict suppression costs. See GAO, Wildland Fire Management: 
Lack of Clear Goals or a Strategy Hinders Federal Agencies' Efforts to 
Contain the Costs of Fighting Fires, GAO-07-655 (Washington, D.C.: June 
1, 2007). 

[17] In some cases, federal firefighting assets are also available to 
respond to fires on nonfederal land. 

[18] We have previously reported on the status of the agencies' 
development of these plans. See GAO, Wildland Fire Management: Update 
on Federal Agency Efforts to Develop a Cohesive Strategy to Address 
Wildland Fire Threats, GAO-06-671R (Washington, D.C.: May 1, 2006). 

[19] The interagency science team in 2006 proposed an option for 
developing FPA that might have helped the agencies to better achieve 
this objective, but the Wildland Fire Leadership Council did not 
approve this option out of concern that the agencies would be unable to 
complete it within the time and budget available. Interagency science 
team members told us they could, if directed, continue to develop that 
option and incorporate it into FPA later. 

[20] GAO, Western National Forests: A Cohesive Strategy Is Needed to 
Address Catastrophic Wildfire Threats, GAO/RCED-99-65 (Washington, 
D.C.: Apr. 2, 1999); Wildland Fire Management: Important Progress Has 
Been Made, but Challenges Remain to Completing a Cohesive Strategy, GAO-
05-147 (Washington, D.C.: Jan. 14, 2005); and GAO-06-671R. 

[21] In 2008, however, we reported that the agencies had begun 
retreating from their commitment to develop a cohesive strategy. See 
GAO, Wildland Fire Management: Federal Agencies Lack Key Long-and Short-
Term Management Strategies for Using Program Funds Effectively, GAO-08-
433T (Washington, D.C.: Feb. 12, 2008). 

[22] For information on the agencies' approaches to allocating fuel 
reduction funds, see GAO, Wildland Fire Management: Better Information 
and a Systematic Process Could Improve Agencies' Approach to Allocating 
Fuel Reduction Funds and Selecting Projects, GAO-07-1168 (Washington, 
D.C.: Sept. 28, 2007). 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: