This is the accessible text file for GAO report number GAO-05-927 
entitled 'Managing for Results: Enhancing Agency Use of Performance 
Information for Management Decision Making' which was released on 
September 9, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Subcommittee on Government Management, Finance 
and Accountability, Committee on Government Reform, House of 
Representatives: 

September 2005: 

Managing for Results: 

Enhancing Agency Use of Performance Information for Management Decision 
Making: 

GAO-05-927: 

GAO Highlights: 

Highlights of GAO-05-927, a report to the Chairman, Subcommittee on 
Government Management, Finance and Accountability, House Committee on 
Government Reform: 

Why GAO Did This Study: 

The Government Performance and Results Act (GPRA) of 1993 has laid a 
foundation of results-oriented agency planning, measurement, and 
reporting in the federal government. Performance planning and 
measurement have slowly, yet increasingly, become a part of agencies’ 
cultures. For planning and performance measurement to be effective, 
federal managers need to use performance information to identify 
performance problems and look for solutions, develop approaches that 
improve results, and make other important management decisions. 
According to GAO’s periodic surveys, federal managers reported having 
more performance measures in 2003 than in 1997. However, the data also 
showed that managers’ reported use of performance information for 
program management activities has remained essentially unchanged from 
1997 levels. GAO was asked to identify (1) how federal agencies can use 
performance information to make management decisions and (2) practices 
that can enhance or facilitate the use of performance information to 
make management decisions. 

Technical comments from the case agencies were incorporated where 
appropriate. 

What GAO Found: 

Agencies can use performance information to make various types of 
management decisions to improve programs and results. Agencies can also 
implement a number of practices that can enhance or facilitate the use 
of performance information. GAO identified four broad types of 
management decisions for which federal managers can use performance 
information and five different types of practices that can contribute 
to greater use of performance information. (See figure.)

Practices That Can Enhance or Facilitate the Use of Performance 
Information for Management Decision Making: 

[See PDF for image] 

[End of figure]

The five federal agencies that GAO examined for this report provide 
examples of how agencies can use performance information for key 
management decisions and practices that can enhance or facilitate such 
use. While agencies face different management conditions and challenges 
and operate under different authorities, the general uses and practices 
highlighted in this report could be adapted by other agencies. Helpful 
next steps would be for agency experiences in using performance 
information to be more widely shared; and, for agencies to be 
encouraged to adapt the practices to their unique situations. 

www.gao.gov/cgi-bin/getrpt?GAO-05-927. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Bernice Steinhardt at 202-
512-6543 or steinhardtb@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Agencies Can Use Performance Information to Manage for Results: 

Management Practices Can Enhance the Use of Performance Information: 

Conclusions: 

Agency Comments: 

Appendixes: 

Appendix I: Scope and Methodology: 

Appendix II: GAO Contact and Staff Acknowledgments: 

Bibliography: 

Related GAO Products: 

Table: 

Table 1: Background Information on Agencies Selected to Illustrate 
Performance Information Uses and Practices: 

Figures: 

Figure 1: Practices That Can Enhance or Facilitate the Use of 
Performance Information for Management Decision Making: 

Figure 2: Example of Using Performance Information to Manage for 
Results: 

Figure 3: The Percentage of Federal Managers Who Reported Using 
Information Obtained from Performance Measures to a Great or Very Great 
Extent for Various Management Activities Has Not Increased 
Significantly Since 1997: 

Figure 4: Uses of Performance Information: 

Figure 5: Practices That Can Enhance or Facilitate the Use of 
Performance Information for Management Decision Making: 

Figure 6: SBA Data Validation Table: 

Figure 7: NOAA Quad Chart Template: 

Figure 8: NOAA Display Board Template: 

Figure 9: FAA Scorecard for Runway Incursions: 

Abbreviations: 

BIC: Business Information Center: 

COO: Chief Operating Officer: 

DOC: Department of Commerce:  

DOL: Department of Labor: 

DOT: Department of Transportation: 

EBSA: Employee Benefits Security Administration: 

EMS: Enforcement Management System: 

ETA: Employment and Training Administration: 

FAA: Federal Aviation Administration: 

FARS: Fatality Analysis Reporting System: 

FFMP: Flash Flood Monitoring and Prediction Program: 

GPRA: Government Performance and Results Act: 

GS: General Schedule: 

NBLP: National Business Learning Partnership: 

NHTSA: National Highway Traffic Safety Administration: 

NOAA: National Oceanic and Atmospheric Administration: 

NWS: National Weather Service:  

OMB: Office of Management and Budget: 

OPA: Office of Participant Assistance: 

OSI: Organizational Success Increase: 

PEP: Performance Enhancement Project: 

SBA: Small Business Administration: 

SES: Senior Executive Service:  

VA: Department of Veterans Affairs: 

VHA: Veterans Health Administration: 

WFO: Weather Forecast Office: 

WIA: Workforce Investment Act: 

The Honorable Todd Platts: 
Chairman, Subcommittee on Government Management, Finance and 
Accountability: 
Committee on Government Reform: 
House of Representatives: 

Dear Mr. Chairman: 

The federal government is in a period of profound transition and faces 
an array of challenges and opportunities to enhance performance, ensure 
accountability, and position the nation for the future. These 21ST 
century challenges include the nation's large, long-term fiscal 
imbalance, evolving national and homeland security threats, increasing 
global interdependence, the global shift to market-oriented knowledge- 
based economies, an aging and more diverse population, and rapid 
advances in science and technology. In the face of these pressures, it 
is vital to maximize the performance of federal agencies in achieving 
their long-term goals. The Government Performance and Results Act of 
1993 (GPRA)[Footnote 1] planning and reporting requirements can provide 
the essential information needed to assess federal agencies' 
performance, hold agencies accountable for achieving results, and help 
the baseline review of federal programs, operating functions, and 
activities needed to respond to the nation's long-term structural 
fiscal imbalance. GPRA requires executive agencies to develop strategic 
plans, prepare annual performance plans, measure progress toward the 
achievement of the goals in the annual performance plans, and report 
annually on their progress in program performance reports. 

For planning and performance measurement to be effective, federal 
managers need to use performance information to identify performance 
problems and look for solutions, develop approaches that improve 
results, and make other important management decisions. A 2004 GAO 
report that reviewed the effectiveness of GPRA found that while the 
percentage of federal managers who report having performance measures 
for their programs has increased over time, their use of performance 
information in making key management decisions, such as adopting new 
program approaches or changing work processes, has not.[Footnote 2] 
Federal agencies also appear to differ considerably in the extent to 
which they use performance information to manage. 

Identifying practices associated with the increased use of performance 
information could be helpful to federal agencies seeking to better use 
performance information to inform their decisions and ultimately 
achieve results. Although performance information could also be helpful 
for external purposes, such as congressional decision making, 
transparency, and public accountability, this report focuses on use of 
performance information by federal agency managers. 

This report responds to your request that we identify (1) how federal 
agencies can use performance information to make management decisions 
and (2) practices that can facilitate the use of performance 
information. 

To address the objectives, we reviewed relevant literature, including 
previous GAO reports, spoke to experts in using performance 
information, and held group discussions with federal program managers. 
We also interviewed individuals within five federal agencies and 
reviewed documentation to illustrate in greater detail how program 
managers have used performance information to make decisions and 
specific agency practices that facilitated those uses. We selected the 
five agencies--the Departments of Commerce, Labor, Transportation, and 
Veterans Affairs, and the Small Business Administration--on the basis 
of a number of factors, including a relatively high agency score on our 
2000 GPRA survey regarding the extent to which agency managers reported 
using performance information, and relatively good agency marks in the 
September 2004 Executive Branch Management Scorecard (President's 
Management Agenda). We also considered information obtained from 
interviews with experts, and diversity in terms of program type and 
size. It should be noted that we did not attempt to identify all 
agencies and programs that could have illustrated these uses and 
practices. In addition, we reviewed existing relevant information 
regarding the quality of performance information where available but 
did not systematically assess the quality of the performance 
information used in the examples we cite. Finally, for most of the 
examples, we describe how performance information was used to make 
decisions, but we did not attempt to verify that the use ultimately 
resulted in improved outcomes. (See appendix I for a more detailed 
discussion of our scope and methodology and for further information on 
the five agencies.) We conducted our work in accordance with generally 
accepted government auditing standards from September 2004 through 
August 2005. 

Results in Brief: 

Agencies can use performance information to make various types of 
management decisions to improve programs and results. Agencies can also 
implement a number of practices that can enhance or facilitate the use 
of performance information. We identified four broad types of 
management decisions for which federal managers can use performance 
information and five different types of practices that can contribute 
to greater use of performance information. (See fig. 1.)

Figure 1: Practices That Can Enhance or Facilitate the Use of 
Performance Information for Management Decision Making: 

[See PDF for image] 

[End of figure] 

The five federal agencies that GAO examined for this report provide 
examples of how agencies can use performance information for key 
management decisions and practices that can enhance or facilitate such 
use. While agencies face different management conditions and challenges 
and operate under different authorities, the general uses and practices 
highlighted in this report could be adapted by other agencies. Helpful 
next steps would be for agency experiences in using performance 
information to be more widely shared; and, for agencies to be 
encouraged to adapt the practices to their unique situations. In figure 
2 we describe how an agency employed several practices to use 
performance information to make management decisions. 

Figure 2: Example of Using Performance Information to Manage for 
Results: 

[See PDF for image] 

[A] The National Safety Council is a nonprofit, nongovernmental, 
international public service organization dedicated to protecting life 
and promoting health. 

[End of figure] 

We provided a draft of this report to the Secretaries of Commerce, 
Labor, Transportation, and Veterans Affairs, and the Administrator of 
the Small Business Administration for their review and comment. 
Technical comments provided were incorporated, where appropriate, in 
the report. 

Background: 

GPRA helped create a governmentwide focus on results by establishing a 
statutory framework for performance management and accountability, with 
the necessary infrastructure to generate meaningful performance 
information. GPRA introduced planning and reporting requirements that 
seek to shift the focus of federal management and decision making from 
a preoccupation with the number of program tasks or activities 
completed or services provided to a more direct consideration of the 
results of programs. The Act was intended to improve federal program 
effectiveness, accountability, and service delivery. GPRA requires 
federal agencies to develop strategic plans with long-term, outcome- 
oriented goals and objectives, annual goals linked to the long-term 
goals, and annual reports on the results achieved. Agencies are 
required to measure progress toward the achievement of the goals in 
annual performance plans and report annually on their progress in 
program performance reports. 

As we reported in our 10-year retrospective report on the effectiveness 
of GPRA, the Act's requirements have laid a solid foundation of results-
oriented agency planning, measurement, and reporting that has begun to 
address the purposes of GPRA.[Footnote 3] Performance planning and 
measurement have slowly yet increasingly become a part of agencies' 
cultures. According to three government-wide, random sample surveys of 
federal managers conducted by us in 1997, 2000, and 2003, managers 
reported having significantly more of the types of performance measures 
called for by GPRA, particularly outcome-oriented performance measures, 
in 2003 than in 1997, when GPRA went into effect governmentwide. 

The benefit of collecting performance information is only fully 
realized when this information is actually used by managers to make 
decisions oriented toward improving results. While our surveys found 
that managers reported having more performance measures in 2003 than in 
1997, the data showed that the use of performance information for 
program management activities did not increase significantly from 1997 
levels. In our survey, we asked managers about their use of performance 
information for specific purposes, such as "setting program 
priorities", "allocating resources", or "adopting new program 
approaches." As shown in figure 3, the majority of managers who 
expressed an opinion reported using performance information for these 
uses to a great or very great extent in 2003. While the reported use of 
performance information for these uses to a great or very great extent 
fluctuated somewhat between 1997 and 2003, the extent of use in 2003 
was not significantly different from 1997 levels for any category of 
use except "adopting new program approaches or changing work 
processes", where use actually decreased. 

Figure 3: The Percentage of Federal Managers Who Reported Using 
Information Obtained from Performance Measures to a Great or Very Great 
Extent for Various Management Activities Has Not Increased 
Significantly Since 1997: 

[See PDF for image] 

[A] There was a statistically significant difference between the 1997 
and 2003 surveys. 

[End of figure] 

From our 2000 survey,[Footnote 4] we also know that use of performance 
information has varied widely among agencies. For example, while 56 
percent of managers overall reported using performance information when 
setting program priorities in 2000, the percentage of managers within 
specific agencies reporting this use to a great or very great extent 
ranged from 26 percent to 64 percent. Similarly, wide variation among 
agencies was seen in managers' reported use of performance information 
for other types of management decisions. For example, the reported use 
of performance information to a great or very great extent in agencies 
ranged from 24 to 66 percent when allocating resources, from 25 to 64 
percent when adopting new program approaches or changing work 
processes, and from 16 to 66 percent when setting individual job 
expectations. 

Agencies Can Use Performance Information to Manage for Results: 

Federal agencies can use performance information to make various types 
of management decisions to improve programs and results. We have 
previously reported that leading organizations that have progressed 
toward results-oriented management use performance information as a 
basis for decision making.[Footnote 5] The full benefit of collecting 
performance information is realized only when managers actually use it 
to manage. From our review of the literature, agency documents and 
interviews with experts and agency officials, we identified four 
categories of management decisions for which federal agency managers 
can use performance information. (See fig. 4).[Footnote 6] Managers can 
use performance information to identify problems in existing programs, 
to try to identify the causes of problems, and/or to develop corrective 
actions. They can also use program performance information to develop 
strategies, plan and budget, identify priorities, and make resource 
allocation decisions to affect programs in the future. Agency managers 
can also use performance information to reward individuals or 
organizations who meet or exceed expectations. Finally, managers can 
use performance information to identify more effective approaches to 
program implementation and share those approaches more widely across 
the agency. 

Figure 4: Uses of Performance Information: 

[See PDF for image] 

[End of figure] 

Performance Information Can Be Used to Identify Problems in Programs 
and Take Corrective Action: 

Agencies can use performance information to identify problems or 
weaknesses in programs, to try to identify factors causing the 
problems, and to modify a service or process to try to address 
problems. We have reported that leading organizations that have 
progressed toward results-oriented management have used performance 
information to identify gaps in performance, improve organizational 
processes, and improve their performance.[Footnote 7] We found several 
examples where federal program managers used performance data to 
identify problems and corrective actions at the agencies we examined. 

Officials in the Office of Participant Assistance (OPA) at the Employee 
Benefits Security Administration (EBSA) in the Department of Labor 
(DOL) told us that when data for a performance measure of customer 
satisfaction indicated a problem, they changed a process to try to 
improve performance. One of EBSA's functions is to provide information 
and assistance for individuals covered by private retirement and health 
and welfare plans, plan sponsors, and members of the employee benefits 
community. EBSA has contracted with an outside organization since 2003 
to collect data through a specialized survey on customer satisfaction 
with EBSA services. Data indicated that customers whose cases were 
referred to the enforcement side of EBSA were reporting lower 
satisfaction rates. An analysis of data for specific survey questions 
identified the need for improved communication. Investigating an 
inquiry or complaint could sometimes take up to 2 years, and a customer 
might not receive any information during that time. As a result of the 
performance data, EBSA changed a standard operating procedure in 
January 2005 so that customers would be kept informed about the status 
of their open inquiries or complaints. Under the new procedure, when an 
inquiry is referred to the enforcement staff for investigation, the OPA 
benefits advisor informs the inquirer about the referral and sets 
realistic expectations for future contact by an investigator. 
Investigative staff are to make initial contact with the customer no 
later than 30 days following receipt of the referral and maintain 
contact with the inquirer on a quarterly basis until the matter is 
resolved. Managers plan to continue to monitor customer satisfaction to 
see whether the changes lead to improved performance. 

Through the Performance Enhancement Project (PEP), the Office of 
Workforce Investment in DOL has used performance information to 
identify the technical assistance needs of state and local employment 
and training programs and has then provided targeted assistance to try 
to improve performance. The PEP project, which began in October 2002, 
has provided technical assistance and training for targeted state and 
local workforce organizations that receive funds under the Workforce 
Investment Act (WIA).[Footnote 8] Poorly performing states, as shown by 
the performance measures (supplemented by analysis from regional DOL 
staff), have received technical assistance designed to improve their 
performance through PEP. DOL's Office of Workforce Investment sent a 
questionnaire to all the regional offices and held a conference with 
them at the close of the first PEP project[Footnote 9] to assess the 
effectiveness of the technical assistance that had been given. This 
information was used to improve the second year's project. Online 
tutorials have also been developed under PEP in response to requests 
from states.[Footnote 10]

Officials in the Veterans Health Administration (VHA) network that 
includes facilities located in Maryland, Virginia, West Virginia, and 
Pennsylvania,[Footnote 11] stated that in response to one of the 
network facilities not meeting the heart failure performance measure 
target, all of the facilities adopted a new process to improve the 
network's performance. According to an official, in the first quarter 
of 2004, the network was at risk of not meeting the entire set of 10 
cardiovascular performance measures if one of its facilities did not 
improve its low score on the heart failure performance measure by the 
end of the year.[Footnote 12] The heart failure performance measure is 
a cumulative performance measure that assesses whether discharged heart 
failure patients receive six types of instructions, including 
instructions for monitoring weight after discharge and taking 
medications. The instructions are critical because patients' 
noncompliance with physicians' instructions is often a cause for 
rehospitalization. An agency official stated that the entire network 
implemented a new set of mandatory data fields in patients' electronic 
medical records. The data fields include the six types of instructions 
and have to be filled out when a facility discharges a heart failure 
patient. According to this official, the result of the implementation 
of the new mandatory fields in the patients' electronic records was 
that the facility that was not meeting the heart failure performance 
measure target in the first quarter improved its score to the "fully 
satisfactory" category in the fourth quarter. 

Performance Information Can Be Used to Develop Agency Strategy, Plan 
and Budget, Identify Priorities and Allocate Resources: 

Agencies can use performance information to make decisions that affect 
future strategies, planning and budgeting, identifying priorities, and 
allocating resources. We have previously reported that outcome-based 
performance information should be used for the allocation of resources 
and in deciding among competing priorities in a results-oriented 
management system.[Footnote 13] In addition, linking cost with 
performance information infuses performance concerns into planning and 
budgetary deliberations, prompting agencies to reassess their 
performance goals and strategies and to more clearly understand the 
cost of performance. Performance information also allows program 
managers to compare their programs' results with goals and thus 
determine where to target program resources to improve performance. 
When managers are faced with reduced resources, the same analysis can 
help them target reductions to minimize negative impacts on program 
results. We found a number of examples among our case agencies of 
managers using performance information to identify priorities and 
allocate resources, and to shape future program strategy. 

The National Highway Traffic Safety Administration (NHTSA) has used 
performance information to set priorities in budgeting and to target 
resources. The Senior Associate Administrator for Traffic Injury 
Control annually sends out a memo that identifies agency priorities 
that support the Department of Transportation's (DOT) strategic goals 
and are based on performance information for traffic-injury-related 
measures, such as safety belt use and impaired driver rates.[Footnote 
14] The memo directs managers to focus their programs and proposals to 
support these priorities. In addition, NHTSA used performance 
information on alcohol-related injuries and fatalities to target grant 
funding and specific program strategies to states with the highest 
impaired driver rates. Using information reported through the Fatality 
Analysis Reporting System (FARS),[Footnote 15] NHTSA identified 13 
states that represented 46 percent of the total number of alcohol- 
related fatalities in the United States. Officials said NHTSA worked 
closely with these 13 states to implement the impaired-driving program 
strategies it had designed. The strategies involved state-sponsored, 
high-visibility law enforcement activities in conjunction with a NHTSA- 
designed media campaign. A NHTSA official said that the agency brought 
senior policy and law enforcement officials from these states together 
and presented the performance information and evaluations showing 
impaired driving rates and the effectiveness of the high visibility law 
enforcement and media campaign. In order to receive funding, the 
targeted states had to agree to implement the program that NHTSA had 
designed, including the media campaign. After the publicized crackdown 
periods, NHTSA provided the targeted states with additional evaluation 
resources. 

The Small Business Administration (SBA) used performance and cost 
information to reallocate resources to better meet priorities. In the 
1990s, SBA created Business Information Centers (BIC), a community 
partnership service to provide entrepreneurs with access to computers 
and the Internet to help them access business information and develop 
business plans. Officials said that, as a partnership using hardware 
and software and library resources donated from private technology 
companies, the program was designed to require minimal SBA employee 
support or capital costs. However, after analyzing financial and human 
capital resources, including the data collected through their annual 
survey of how much support SBA employees provide for various programs, 
they found that substantial employee resources were being devoted to 
the BICs, on the order of $11 million. This cost information led SBA 
officials to reconsider the service. Since computers had become more 
accessible to entrepreneurs than when the BIC program was designed, and 
because assistance to entrepreneurs could be provided more efficiently 
though other programs, SBA decided to end its participation in the 
program and has phased out its role in the community partnerships. 

Performance Information Can Be Used to Recognize and Reward 
Performance: 

Agencies can use performance information to affect pay decisions and to 
reward individuals, and in some cases, grantees.[Footnote 16] Using 
performance information this way reinforces accountability and creates 
incentives for achieving results. We have previously reported that high-
performing public-sector organizations create a clear "line of sight" 
between individual performance and organizational success[Footnote 17] 
showing how team, unit, and individual performance can contribute to 
overall organizational results. There is growing recognition that the 
government needs to transform how it compensates its employees to 
achieve greater results. To this end, some agencies have implemented 
performance agreements as a vehicle for bringing results-oriented 
performance information into evaluations of performance. A revised 
performance-based pay system for members of the Senior Executive 
Service (SES) across the federal government has been in effect since 
January 2004, as authorized by Congress and overseen by the Office of 
Personnel Management (OPM), where pay adjustments for SES members are 
to be based on the alignment of individual performance and 
contributions to the agency's mission and strategic goals, among other 
things. The human capital reforms under way at the Departments of 
Defense and Homeland Security are intended to help them manage their 
human capital to achieve results. We found several examples where case 
agencies used performance information to recognize and reward 
performance, as described next. 

The Federal Aviation Administration (FAA) uses performance information 
to affect pay decisions. FAA has linked employee annual pay increases 
to the achievement of agencywide goals through the performance-based 
pay system.[Footnote 18] At the end of fiscal year 2004, 78 percent of 
employees were included in the agency's performance-based pay system, 
which allocates increases on the basis of individual and organizational 
performance.[Footnote 19] At the individual level, employees can 
receive two types of increases. Employees earn the first increase for 
meeting individual goals and the second increase for achieving 
outstanding individual performance. At the organizational level, 
employees are eligible for the organizational success increase (OSI). 
This is an agencywide pay increase based on the performance of the 
agency as a whole on the 31 key performance measures in the agency's 
strategic plan, called the Flight Plan.[Footnote 20] The OSI funding 
pool consists of the amount of the general increase for General 
Schedule (GS) employees in other federal agencies plus an additional 1 
percent (which reflects a portion of the funds previously spent on 
within-grade increases). Receiving the full OSI depends on whether 
actual performance meets the goals. In calendar year 2004, FAA 
employees received an OSI increase of 2.13 percent. FAA also offers 
Short Term Incentives to FAA senior staff for achieving key performance 
targets and specified accomplishments in implementing Flight Plan 
initiatives. 

VHA uses performance information to create incentives for network 
directors. We previously reported that the use of performance 
information in performance agreements helped VHA define directors' 
accountability for specific goals and monitor progress during the year, 
and contributed to their evaluations. VHA accomplishes this through the 
performance-based review process, which links a director's performance 
rating and compensation to the performance of the network that he or 
she oversees. Specifically, a director's annual ratings and bonuses are 
based on how well he or she meets the goals in the performance 
contract. As detailed in the contract, the network director's 
compensation is affected by that network's scores on performance 
measures, which include clinical waiting times, the percentage of 
patients receiving cancer screenings, and patient satisfaction. Half of 
the network director's performance evaluation is based upon the 
cumulative score on the network's performance measures. 

The Employment and Training Administration (ETA) uses performance 
information to provide incentives for and apply sanctions to state 
programs that receive grants under Title I of the Workforce Investment 
Act. The authority to hold grantees accountable for performance in this 
manner is provided for in the WIA legislation.[Footnote 21] Each state 
negotiates performance goals with DOL prior to the beginning of the 
program year for the 17 WIA indicators measuring employment, retention, 
earnings, attainment of credentials, attainment of skills, and customer 
satisfaction.[Footnote 22] States are eligible for incentive awards if 
they have exceeded performance goals set for programs funded under 3 
separate programs, one of which is WIA.[Footnote 23] In general, 
financial sanctions may be levied if performance is less than 80 
percent of the negotiated performance levels for 2 consecutive years or 
if the state fails to submit an accurate and complete annual 
performance report.[Footnote 24] Specific mitigating factors may be 
considered, such as performance relative to other states, improvement 
efforts, and economic conditions, among others. 

Performance Information Can Be Used to Identify and Share More 
Effective Processes and Approaches to Program Implementation: 

We have reported that high-performing organizations continuously assess 
and benchmark performance and efforts to improve performance.[Footnote 
25] They evaluate their efforts using fact-based understandings of how 
their activities contribute to accomplishing the mission and broader 
results, and optimize their efforts through continuous improvement. 
Managers can use performance information to identify and increase the 
use of program approaches that are working well, and consider 
alternative processes in areas where goals are not being met. Pilots 
and demonstration projects can be used to identify innovative ways to 
improve performance--by allowing for experiences to be rigorously 
evaluated, and shared systematically with others, and by allowing for 
new procedures to be adjusted as appropriate--before receiving wider 
application. Organizations can also use the process improvement 
technique of benchmarking--comparing their processes with those of 
private and public organizations that are thought to be the best in 
their fields. By benchmarking its own processes against those of 
leading business and government entities, an organization can learn how 
much change it needs to make and what changes might be the right ones. 
We found examples of these types of uses of performance information in 
the case agencies, as described next. 

On the basis of performance information from a pilot test, the National 
Weather Service (NWS) adopted a more effective flash flood monitoring 
and prediction tool on a national basis. NWS provides flash flood 
warning services and verifies the timeliness and accuracy of its 
warnings. It aims to improve its performance in this area through the 
flash flood performance goals entitled, flash flood warning lead time 
and flash flood warning accuracy, as outlined in its strategic 
plan.[Footnote 26] A prototype flash flood monitoring application was 
developed and first tested at the Pittsburgh Weather Forecast Office 
(WFO) beginning in 1995. On the basis of improved performance data 
achieved with the prototype and case studies of several flash flood 
events in the Pittsburgh area, NWS decided to develop and test an 
enhanced version of the pilot system. The enhanced version, called the 
Flash Flood Monitoring and Prediction (FFMP) Program, continuously 
monitors rainfall rates and hydrologic conditions to provide automated 
alerts when a dangerous flood situation may be developing on a given 
stream or catchment. In 2002 NWS began implementing FFMP nationwide, 
with the aim of improving flash flood services and helping NWS to meet 
its flash flood performance goals. Initial training on how to use the 
FFMP system was provided for all WFOs and additional training is 
provided at routinely offered courses. According to a recent survey, 
nearly all of the WFOs were using the system. It is expected that 
implementing the system will facilitate improvements in the accuracy 
and timeliness of official NWS flash flood warnings, and enable 
forecasters to provide more specific location information within a 
flash flood warning. 

ETA has used performance information on state and local workforce 
investment areas to identify and nominate participants for the National 
Business Learning Partnership's (NBLP) peer-to-peer, mentoring 
initiative. The NBLP, which started in October of 2003, links local 
workforce areas[Footnote 27] that want to improve their services 
(protégés) with workforce areas that have exceeded performance 
standards (mentors). The program focuses on the delivery of workforce 
services tailored to the needs of business and industry in that 
specific workforce area. Overall, 19 mentor sites and 25 protégé sites 
have participated. Examples of the types of performance and related 
information considered by ETA in selecting workforce areas to nominate 
as mentors include the following: successful engagement with businesses 
to meet the workforce needs of local industry; high performance on the 
17 legislated WIA performance indicators measuring employment, 
retention, earnings, attainment of credentials, attainment of skills, 
and customer satisfaction; and the transferability of mentor practices 
to other areas. Protégés were selected on the basis of their desire to 
improve their performance and services to businesses, and also on the 
basis of whether they had problems meeting the WIA performance 
measures. Each protégé site was matched with a mentor site to help it 
develop a work plan. The protégés and mentors conducted at least two 
site visits to each other's respective areas to study the local 
workforce investment operations and share information. In addition to 
peer-to-peer consultation, NBLP case studies have been developed to 
provide a wider audience with access to participants' experiences and 
learning. Each case study includes practices and principles to improve 
performance outcomes by addressing the workforce needs of businesses 
and industries, as well as a guide to facilitate shared learning and 
promote action.[Footnote 28] To assess the success of these 
partnerships, each protégé's performance on selected WIA measures will 
be compared from performance year 2003 through performance year 2005. 
Additional measures of success will include such considerations as 
contacts with businesses, resources leveraged, and workers trained and 
placed in high-growth careers. 

Management Practices Can Enhance the Use of Performance Information: 

Agencies can adopt or apply a number of practices that can enhance the 
use of performance information for policy and program decisions aimed 
at improving results. The concept of practices as used in this report 
includes administrative or management tools, systems, or processes that 
agencies can implement. From our interviews with experts and agency 
officials and our review of related literature and agency documents, we 
identified five types of practices that can facilitate greater use of 
performance information. As illustrated in figure 5, the five types of 
practices are demonstrating management commitment; aligning agency 
goals, objectives and measures; improving the usefulness of performance 
information to better meet management's needs; developing agency 
capacity to effectively use performance information; and frequently and 
effectively communicating performance information within the agency. We 
discuss the 5 practices in detail following the figure. 

Figure 5: Practices That Can Enhance or Facilitate the Use of 
Performance Information for Management Decision Making: 

[See PDF for image] 

[End of figure] 

Demonstrating Management Commitment: 

The commitment of agency managers to results-oriented management is 
critical to increased use of performance information for policy and 
program decisions. As we previously reported,[Footnote 29] 
demonstrating the willingness and ability to make decisions and manage 
programs on the basis of results, and inspiring others to embrace such 
a model, are important indicators of management's commitment. Agency 
officials from our case illustration programs also identified 
management commitment as central to encouraging the use of performance 
information in decisions. Officials from these agencies described how 
management showed this type of commitment by leading frequent, regular 
performance review meetings to discuss progress made toward the 
achievement of results, and by involving staff from different 
organizational levels in performance review meetings. These methods 
assisted agencies in identifying performance problems and in developing 
performance improvement plans based on collected performance 
information. 

SBA conducts monthly meetings with the associate administrator of each 
mission and functional office to review that office's performance. The 
administrator, deputy administrator, and chief operating officer (COO) 
participate. Each SBA office has performance measures organized along 
three components of performance--office strategic goals, production 
goals, and project goals--on a scorecard. Prior to meeting, the COO's 
staff review the monthly and cumulative performance information for 
each office and ask each office to respond in writing to questions or 
concerns based on the data. This analysis is the basis for the regular 
performance review meetings. Officials said management's commitment to 
regularly reviewing performance increases performance ownership among 
staff and competition among the offices to meet performance targets. 

Similarly, FAA management demonstrates commitment through monthly, day- 
long, agencywide performance review meetings that are led by the 
Administrator and key associate administrators who serve as goal leads 
for each FAA Flight Plan goal area: Increased Safety, Greater Capacity, 
International Leadership, and Organizational Excellence. At these 
meetings, officials said that they discuss performance for the agency's 
31 key performance metrics and the strategic initiatives supporting 
each. When a business line is not meeting the performance targets for 
specific metrics, officials report on efforts planned or under way to 
improve performance. For metrics where performance targets have been 
met, officials discuss the actions that were taken to achieve the 
targets. Officials said that during this performance review, the 
Administrator identifies IOUs, outlining agreed-upon actions to be 
implemented. Officials said that they provide updates on the status of 
these IOUs at the following performance review meeting. 

At the Department of Veterans Affairs (VA), managers are involved in 
reviewing performance at different levels of the organization. Senior 
management hold monthly departmentwide meetings to review performance. 
According to officials, the meetings are chaired by the Deputy 
Secretary and are attended by the heads of the three administrations: 
VHA, the Veterans Benefit Administration, and the National Cemetery 
Administration, as well as all of the staff offices within VA. Each 
administration reports financial status, workload, and key performance 
measures. The Deputy Secretary has recently implemented reporting by 
exception at the monthly performance review. Reporting by exception 
requires that administration and staff offices offer an explanation for 
less-than-desired performance, describe plans or ongoing efforts to 
improve performance and discuss when results were expected. At the 
following month's performance review, status information is presented 
to demonstrate progress. In addition, success stories and the 
activities used to achieve them are also reported. According to 
officials, within VHA, mid-and lower-level managers are also involved 
in regular performance review meetings, including meetings involving 
the Deputy Under Secretary and the directors of the regional networks, 
meetings of network directors and their facility directors, and 
meetings between facility directors and their staff. In addition, 
according to officials, each of the facilities in one of the networks 
we visited, which includes VA facilities in California, Nevada, Hawaii 
and the Philippines, assigns a "champion" who is responsible for 
monitoring and presenting results for a specific performance measure at 
weekly facility management meetings. 

Aligning Agencywide Goals, Objectives and Measures: 

Agencies can encourage greater use of performance information by 
aligning agencywide goals and objectives, and by aligning program 
performance measures at each operating level with those goals and 
objectives. GPRA requires that agencies use performance measurement to 
reinforce the connection between their long-term strategic goals and 
the day-to-day activities of their managers and staff. As we reported: 

previously,[Footnote 30] in order to meet the GPRA requirements, an 
agency should cascade its goals and objectives throughout the 
organization and should align performance measures to the objectives 
from the executive level down to the operational levels. This forms 
hierarchies of goals and objectives and performance information that 
are appropriate to the managerial responsibilities and controls at each 
level of the organization. This alignment increases the usefulness of 
the performance information collected to decision makers at each level, 
and reinforces the connection between strategic goals and the day-to- 
day activities of managers and staff. We have also reported that a 
greater focus on results can be created by cascading organizational 
goals and objectives down to the individual performance level. Earlier 
in this report, we described how agencies can use performance 
information to recognize and reward performance and encourage the 
achievement of results. Alignment facilitates the linking of individual 
performance to organizational performance. 

FAA promotes alignment by requiring that all organizations use the same 
alignment framework and approach in preparing their annual business 
plans. Goals cascade throughout each of FAA's 16 organizations to each 
individual employee. The agency goals detailed in FAA's strategic plan 
(or Flight Plan) drive new strategic initiatives and the ongoing, day- 
to-day operations of FAA, referred to as their core business functions. 
Key elements required in the business plans for each of FAA's four 
lines of business and 12 offices are developed using a systematic 
alignment approach that guides the organizations through the step-by- 
step process of defining and developing performance elements that 
support the Flight Plan goals. Organizations align themselves with the 
Flight Plan through this process, and targets cascade down through the 
organization into employee performance plans. These plans identify 
specific work expectations and outcomes, and are required for every FAA 
employee. This alignment facilitates FAA recognizing and rewarding 
performance by using performance information to affect pay decisions, 
as described earlier in this report. 

VHA's performance measures are aligned through different levels of the 
organization. The same performance measures are used to assess VHA's 
performance from the VHA-wide level down through the network levels to 
the individual facilities. For example, the performance measure of the 
percentage of patients receiving breast cancer screening is used at all 
three levels. As mentioned earlier in this report, the performance 
score of a facility can have an effect upon the entire network to which 
it belongs. As an illustration of how performance is "rolled up," 
officials stated that in order to get a fully satisfactory rating at 
the network level, 80 percent of the hospitals in that network must 
have met their goals. Officials stated that if any facility is at the 
failing level for a performance measure, the entire network will also 
be rated as failing the measure. In addition, alignment contributes to 
VHA's ability to use performance information to provide incentives for 
network directors, through the performance-based review process 
described earlier. 

Improving the Usefulness of Performance Information to Better Meet 
Management's Decision-Making Needs: 

To ensure that performance information will be both useful and used in 
decision making throughout the organization, agencies need to consider 
users' differing policy and management information needs. Practices 
that improve the usefulness of performance information can help to meet 
those needs. We reported previously[Footnote 31] that to be useful, 
performance information must meet users' needs for completeness, 
accuracy, consistency, timeliness, validity, and ease of use. Other 
attributes that affect the usefulness of information include, but are 
not limited to, relevance, credibility, and accessibility. We have 
reported previously on a number of practices that improve the 
usefulness of performance information to different users.[Footnote 32] 
Measures should be selected specifically on the basis of their ability 
to inform the decisions made at each organizational level, and should 
be appropriate to the responsibilities and control at each level. In 
that regard, involving managers in the development of performance goals 
and measures is critical to increasing the relevance and therefore the 
usefulness of performance information to their day-to-day activities. 
Agency officials in our case illustrations identified a number of 
practices that increased the usefulness of performance information, 
including its relevance, ease of use, timeliness, and accessibility. 

As discussed earlier, EBSA contracted with an outside organization to 
collect more useful customer satisfaction information. By increasing 
the scope and depth of customer satisfaction data, the information has 
been made more relevant, and therefore, more useful to management's day-
to-day activities. Prior to this contract, EBSA used another contractor 
that provided a general high-level index of customer satisfaction that 
provided comparability to other organizations. However, in order to 
provide this comparability, the survey questions had to be standardized 
in a manner that was not as relevant for EBSA as for other 
organizations. In addition, the first customer satisfaction survey did 
not provide a means for EBSA to disaggregate data to identify specific 
aspects of its performance. Recognizing that the information did not 
fully meet their needs, EBSA staff identified a new contractor to 
create and conduct a specialized customer satisfaction survey of 
clients that will provide comparability from year to year even if it 
does not provide comparability among other organizations. Survey 
questions are now tailored to EBSA's needs. Specific questions can be 
added and data disaggregated and analyzed as needed to focus in on 
potential problem areas. For example, data can be broken down by 
categories such as regional office, benefit advisor, and type of 
inquiry. 

The Enforcement Office of the Department of Labor's EBSA Office 
implemented a new Enforcement Management System (EMS) that officials 
say improved the usefulness of performance information, making it more 
timely, accessible, and easy to use. The system, implemented in 1999, 
manages information related to investigations into the potential 
violation of benefits laws. These investigations are conducted by 385 
investigators located at EBSA field offices across the country. Prior 
to the development of the EMS, officials said that the Enforcement 
Office had a simple database system located at headquarters and that 
field offices used a separate case management system.[Footnote 33] The 
EMS was custommade for the Enforcement Office. It tracks information 
about every investigation nationwide throughout its lifecycle, 
providing both field and headquarters staff access to real-time 
information. According to officials, not only does the system make 
information more timely and accessible, it has increased the ease of 
use and relevance of performance information by incorporating user- 
friendly windows technology and allowing users to define their own 
information needs with customized queries and reports. In addition, 
over 30 standardized statistical and case reports are available. 
Standardized statistical reports allow managers to view, by selected 
geographic area, summaries of such information as the different 
investigation types (e.g., fiduciary and criminal), the cases referred 
for litigation, and closed cases along with the associated results. 
Headquarters provides additional analysis of the information regarding 
overall progress toward policy goals, allowing adjustments to be made 
in the field offices. 

SBA has involved its program managers in the development of the 
agency's performance measurement system, allowing offices to determine 
their own goals and measures. This practice increases the usefulness of 
performance information to the activities of offices and programs, and 
SBA officials credit it with creating buy-in and ownership of the 
performance measurement system. Measures and targets must meet certain 
requirements as overseen by the chief operating officer. For example, 
measures should support the strategic goals; measures should be valid, 
measurable, and verifiable; and the set of measures as a whole should 
contain both qualitative and quantitative measures. At the beginning of 
each year when the measures and targets are being selected, assistance 
is provided for any program upon request to help with selecting 
appropriate measures. 

SBA has also initiated a practice aimed at documenting and improving 
the overall usefulness of performance information and increasing its 
use. Programs are required to identify the intended use of each measure 
in decisions related to policy and to provide an assessment of how fit 
the information is for this use. The importance of this practice is 
that it promotes the selection of performance measures that better meet 
managements' needs, providing information that is relevant, and 
sufficiently complete, accurate, consistent, and timely for 
management's decision-making needs. To facilitate and document the 
practice, SBA's Office of Analysis, Planning and Accountability 
developed an assessment tool for its indicators in the form of the Data 
Validation Table,"[Footnote 34] shown in figure 6. These tables are 
updated annually for every indicator published in the SBA Performance 
and Accountability Report. In addition to documenting the intended use, 
the office must provide an assessment of the information's 
completeness, accuracy, consistency, and timeliness. Validation Tables 
also document the associated office and program; what outcomes are 
being assessed; how each outcome is aligned with agency goals; the 
system in which the data are kept; and any limitations to the use of 
the information in the intended manner and associated remedies, among 
other items. 

Figure 6: SBA Data Validation Table: 

[See PDF for image] 

[End of figure] 

Developing Agency Capacity: 

The practice of building analytical capacity to use performance 
information--both in terms of staff trained to do analysis and 
availability of research and evaluation resources--is critical to using 
performance information in a meaningful fashion. Our 2003 GPRA survey 
found that there was a positive relationship between agencies providing 
training and development on setting program performance goals and the 
use of performance information when setting or revising performance 
goals. We have previously reported[Footnote 35] on the need for 
agencies to expend resources on effective training and professional 
development to equip federal employees to work effectively. 
Specifically, managers and staff need competencies and skills to plan 
strategically, develop robust measures of performance, and analyze what 
the performance data mean. Performance management literature also 
states that training is a key factor in improving employees' 
capabilities and enabling employee involvement in achieving performance 
improvements. We found examples where our case illustration agencies 
built capacity through providing staff with training in the performance 
management system and by providing access to technical assistance, such 
as having designated evaluation support staff or performance 
measurement experts available. 

The VHA network that includes health care facilities located in 
California, Nevada, Hawaii, and the Philippines builds agency capacity 
by providing annual training on performance measures for facility 
managers. The training is coordinated by a performance measurement 
committee composed of clinical leadership and quality management staff 
from each of the facilities. The committee's mission includes providing 
education on the performance measures beyond the annual training, as 
well as sharing and implementing strategies to facilitate meeting the 
performance measurement targets. At the annual training conference, all 
of the performance measures are discussed, including existing measures 
as well as any new measures. The training lasts for a day or two, and 
facility managers can participate in person, on the telephone, or by 
videoconference. In addition, the performance measurement committee is 
also a resource throughout the year for questions on the performance 
measures. 

NHTSA has established capacity to use performance information through 
its research and evaluation office. According to one official, NHTSA 
has had internal analytic capacity through a research office for more 
than 25 years. The evaluation office provides the capacity to conduct 
research and evaluations on the effectiveness of NHTSA's Traffic Injury 
Control Office programs and initiatives, which address issues such as 
safety related to alcohol-and drug-impaired driving, pedestrians, 
bicycles, motorcycles, school buses, emergency medical services, older 
drivers, and driver fatigue and inattention. We previously reported 
that NHTSA's evaluation office provides the capacity for a three-phase 
evaluation process.[Footnote 36] First, studies identify the nature of 
the problem and possible solutions. Second, cost-benefit analyses 
identify the expected consequences or alternative approaches. Third, 
follow-up studies assess the consequences of policy or regulatory 
changes, since effects of some changes may not be apparent until 5 or 
more years after the introduction of changes. According to a NHTSA 
official, it is important to have this capacity because the Traffic 
Injury Control Office does not have any regulatory authority, and needs 
the analysis and evaluations to demonstrate that the proposed programs 
are effective in order to persuade states to implement them. In 
addition, we have previously reported that evaluations can improve 
agency use of performance information by providing performance 
information that may otherwise be unavailable; by validating the 
accuracy of performance data; by explaining the reasons for observed 
performance; or, finally, by identifying ways to improve performance. 

Communicating Performance Information Frequently and Effectively: 

Improving the communication of performance information among staff and 
stakeholders can facilitate the use of performance information by 
agency managers. Improvements can be achieved through frequent and 
routine communication, and the use of effective communication tools, 
such as visual aids. We previously reported that frequent, regular 
communication is key for managers to inform staff and other 
stakeholders of their commitment to achieve the agency's goals and to 
keep these goals in mind as they pursue their day-to-day 
activities.[Footnote 37] Frequently reporting performance information 
also allows managers to review the information in time to take action 
to make improvements. Program managers can also communicate performance 
information upward through the management hierarchy, and across 
operating units. We found a number of vehicles through which agencies 
communicated performance information effectively. For example, agencies 
maintained frequent, routine communication among managers and staff 
through visual tools such as poster displays, performance scorecards, 
and intranet sites. Some agencies provided performance updates through 
regular e-mail or the distribution of monthly performance review 
meeting minutes. In addition, officials said publicizing performance 
information can inspire ownership of a unit's performance, as well as 
competition between units. For example, scorecards or other visual 
presentations of performance information enable staff to compare and 
analyze performance, resulting in a competitive interest in improving 
performance. We found examples in selected agencies of the practice of 
frequent and routine communication and using visual tools to 
communicate performance information. 

NWS uses a number of display tools to convey performance information to 
staff and management at different levels of the organization. Quad 
charts and display boards are two examples. Quad charts arose out of 
the National Oceanic and Atmospheric Administration's (NOAA) efforts to 
improve budgeting and planning systems, and have also been used at the 
NWS. They are one-page summaries that provide pertinent information for 
decision-making purposes. One particular type of quad chart is prepared 
to inform agency executives about program performance. These quad 
charts are prepared by each major program and presented during NOAA's 
quarterly executive panels. These panels are composed of the Deputy 
Under Secretary of NOAA and the deputies of the line offices. A one- 
page chart is divided into four quadrants containing the following four 
areas: Performance Parameters (metrics and threshold values for 
selected performance indicators); Schedule of Activities (including 
schedule status); Key Issues and Risks; and Funding (budgetary issues). 
For each of these topics, status is also color-coded in green, yellow, 
or red. An example of this type of quad chart is shown in figure 7. 
Similar types of quad charts are prepared by managers requesting 
program adjustments (such as changes in funding) and for selected NWS 
and NOAA programs. Some quad charts also include GPRA performance 
measures when applicable to a NOAA or NWS program. 

Figure 7: NOAA Quad Chart Template: 

[See PDF for image] 

[End of figure] 

NWS also uses display boards at all of its field and regional offices 
to communicate information to all staff. As shown in figure 8, display 
boards convey national, regional, and site performance information on 
GPRA measures relevant to specific site activities. These charts allow 
site personnel to compare the site's performance with performance 
measured at the regional and national levels. Sites can update their 
own performance measures as often as monthly, while national data are 
updated annually. Comparable site-level information is available for 
all sites on the NOAA Intranet. 

Figure 8: NOAA Display Board Template: 

[See PDF for image] 

[End of figure] 

FAA communicates performance information to all staff through a monthly 
performance scorecard. The scorecard is organized into a table, and 
includes the actual and targeted performance levels for the agency's 31 
key metrics, as identified in FAA's strategic plan, or Flight Plan. 
Each metric is color coded, using the stoplight system of red, yellow, 
or green, according to how closely actual performance met the targeted 
performance for each month. The scorecard is posted on FAA's Intranet 
and is updated monthly. The same information is reported in more detail 
on a quarterly basis on FAA's external Web site.[Footnote 38] An 
example of one of the metrics in the quarterly report, Runway 
Incursions,[Footnote 39] is shown in figure 9. 

Figure 9: FAA Scorecard for Runway Incursions: 

[See PDF for image] 

[End of figure] 

Conclusions: 

Our 2003 survey of federal managers found that, despite the fact that 
managers reported having more performance measures, the use of 
performance information for program management activities did not 
increase significantly from 1997 levels when GPRA was first implemented 
governmentwide. Creating results-oriented cultures in which performance 
information is routinely used to make key management decisions will 
require the sustained attention and commitment of top agency leadership 
and more widespread adoption of the practices identified in this 
report. The five federal agencies that we examined for this report 
provide examples of how agencies can use performance information for 
key management decisions and practices that can facilitate such use. 
The specific ways in which the case agencies used performance 
information or implemented the practices may not be appropriate for 
wholesale adoption throughout the federal government because agencies 
face different management conditions and challenges, and operate under 
different authorities. Nevertheless, the general uses and practices 
highlighted in this report are universal and could be adapted by each 
agency. Helpful next steps would be for the relevant experiences of 
agencies in using performance information and adopting practices that 
facilitate use of performance information to be more widely shared, and 
for agencies to be encouraged to adapt practices to their unique 
situations. 

Agency Comments: 

The Departments of Commerce, Labor, Transportation, and Veterans 
Affairs, and the Small Business Administration provided technical 
comments that were incorporated where appropriate throughout the 
report. 

As agreed with your office, we will send copies of this report to the 
Director of the Office of Management and Budget and other interested 
parties. We will also make copies available to others upon request. In 
addition, the report will be available at no charge on GAO's Web site 
at [Hyperlink, http://www.gao.gov]. 

Please contact me on (202) 512-6543 if you or your staff have any 
questions about this report. Contact points for our Office of 
Congressional Relations and Public Affairs may be found on the last 
page of this report. Other contacts and staff acknowledgments are 
listed in appendix II. 

Sincerely yours,

Signed by: 

Bernice Steinhardt: 
Director, Strategic Issues: 

[End of section]

Appendixes: 

Appendix I: Scope and Methodology: 

To identify uses of performance information and practices that 
encourage the use of performance information, we reviewed our prior 
reports and other relevant literature and interviewed agency officials. 
In addition, we interviewed experts in using performance information 
from the following organizations: the School of Public Policy at the 
University of Maryland; the School of Policy, Planning and Development 
at the University of Southern California; the Institute for the Study 
of Public Policy Implementation at American University; the School of 
Public Policy and Public Administration at George Washington 
University; the Urban Institute; the IBM Center for the Business of 
Government; and the Mercatus Center. 

To identify examples of ways in which managers use performance 
information and practices that enhance their use of that information, 
we selected five federal agencies that were identified as having a 
greater likelihood of using performance information: the Departments of 
Commerce (DOC), Labor (DOL), Transportation (DOT), and Veterans Affairs 
(VA); and the Small Business Administration (SBA). Among the factors 
that we considered in guiding our selection processs were that the 
agencies received relatively high scores on the 2000 GAO Government 
Performance and Results Act survey results regarding the extent to 
which managers reported using performance information,[Footnote 40] 
relatively good marks on the September 2004 Executive Branch Management 
Scorecard (President's Management Agenda), and/or recommendations from 
interviews with experts. We also considered diversity in terms of 
program type and size. 

Table 1 contains brief background information on the agencies within 
the selected departments that are highlighted in our case examples. 

Table 1: Background Information on Agencies Selected to Illustrate 
Performance Information Uses and Practices: 

Department or agency: DOL; 
Selected agencies or offices: Employment and Training Administration 
(ETA); ETA provides job training, employment and labor market 
information, and income maintenance services. Within ETA, the Office of 
Workforce Investment (OWI) is responsible for providing national 
leadership, oversight, policy guidance, and technical assistance for 
the one-stop public employment service system and the youth and adult 
employment and training programs funded under the Workforce Investment 
Act (WIA); 
Selected program type: Block/formula grants. 

Department or agency: DOL; 
Selected agencies or offices: Employee Benefits Security Administration 
(EBSA); EBSA is tasked with protecting the integrity of pensions, 
health plans, and other employee benefits. EBSA's Office of Participant 
Assistance (OPA) provides information and assistance for individuals 
covered by private retirement and health and welfare plans, plan 
sponsors, and members of the employee benefits community. EBSA's goal 
is to enable participants to better understand and exercise their 
rights under the law, and when possible, recover any benefits to which 
they may be entitled. They also seek to give practitioners the 
opportunity to better understand and comply with the law. The Office of 
Enforcement is tasked with protecting pension and welfare benefits 
under ERISA. The Office of Enforcement's investigative field staff 
conduct investigations to detect and correct violations of Title I of 
ERISA and related criminal laws; 
Selected program type: Regulatory- based. 

Department or agency: VA; 
Selected agencies or offices: Veterans Health Administration (VHA); The 
mission of VHA is to assist veterans by providing comprehensive 
healthcare services including primary care, specialized care, and 
related medical and social support. Comprehensive health care services 
are provided for veterans through VHA's field facilities and Veterans 
Integrated Service Networks. VHA's medical system consists of 21 of 
these integrated networks of care whose focus is pooling and aligning 
resources; 
Selected program type: Direct federal. 

Department or agency: SBA; 
Selected agencies or offices: Small Business Administration (SBA); The 
principal mission of the Small Business Administration is to maintain 
and strengthen the nation's economy by aiding, counseling, assisting 
and protecting the interests of small businesses and by helping 
families and businesses recover from national disasters. Within SBA, 
the Office of Capital Access assists small businesses in obtaining 
loans necessary for growth by being a gap lender, assisting small 
businesses that otherwise would not qualify for un-guaranteed financing 
in obtaining equity, or taking advantage of procurement opportunities. 
The Office of Government Contracting and Business Development promotes 
increased small business participation in the federal procurement 
market for goods and services and fulfills SBA's statutory mission to 
ensure that a fair share of federal procurement goes to small 
businesses. The Office of Entrepreneurial Development, through a 
network of partners and its own on-line Small Business Training 
Network, provides counseling, training and business information to 
small entrepreneurs. These resources help to start and grow small 
businesses throughout the country. The Office of Disaster Assistance 
provides affordable, timely, and accessible financial assistance for 
homeowners, renters, and businesses of all sizes affected by disaster. 
Financial assistance is available in the form of low-interest, long-
term loans; 
Selected program type: Direct federal; credit; block/formula grants; 
competitive grant. 

Department or agency: DOT; 
Selected agencies or offices: National Highway Traffic Safety 
Administration (NHTSA); NHTSA's mission is to save lives, prevent 
injuries, and reduce economic costs due to road traffic crashes. NHTSA 
establishes and enforces automobile safety regulations, including crash 
avoidance and crashworthiness standards, and consumer protection 
standards, including fuel efficiency standards, bumper standards, and 
regulations relating to odometer tampering and domestic content 
labeling. NHTSA also investigates safety defects and carries out the 
duties and the powers of DOT to provide for aspects of highway safety 
other than highway safety design features, such as driver performance; 
Selected program type: Block/formula grant; regulatory-based. 

Department or agency: DOT; 
Selected agencies or offices: Federal Aviation Administration (FAA); 
The mission of the FAA is to improve the safety and efficiency of 
aviation. FAA promotes safe flight of civil aircraft in air commerce by 
prescribing standards for the design; material; construction; quality 
of work; and performance of aircraft, aircraft engines, and propellers. 
In addition, FAA issues certificates such as those for airworthiness 
and airport operations. FAA also helps develop and maintain a safe and 
efficient nationwide system of public-use airports that meets the 
present and future needs of civil aeronautics. Finally, FAA licenses 
commercial space launches and the operation of nonfederal launch sites 
within the United States and by U.S. citizens abroad; 
Selected program type: Direct federal; competitive grant; regulatory-
based. 

Department or agency: DOC; 
Selected agencies or offices: National Oceanic and Atmospheric 
Administration (NOAA); The mission of NOAA is to understand and predict 
changes in the earth's environment and to conserve and manage coastal 
and marine resources to meet our nation's economic, social, and 
environmental needs. A part of NOAA, the National Weather Service (NWS) 
provides weather, hydrologic, and climate forecasts and warnings for 
the United States, its territories, adjacent waters, and ocean areas; 
for the protection of life and property; and for the enhancement of the 
national economy. NWS produces and maintains data in a national 
information database, the products and infrastructure of which are used 
by other governmental agencies, the private sector, the public, and the 
international community; 
Selected program type: Direct federal. 

Source: GAO. 

[End of table]

Within each of the five selected agencies, we conducted interviews with 
program/policy management and staff to obtain their suggestions 
regarding specific programs in which performance information has been 
used for management. We then interviewed staff involved with those 
programs in more detail and reviewed related documents to obtain 
further information about how those programs have used performance 
information and any associated practices that have contributed to those 
uses. We selected specific examples to highlight each use and practice 
and to illustrate particular aspects of how that use or practice was 
implemented within specific agency program contexts and across various 
program types. We did not attempt to identify all agencies and programs 
that could have illustrated these uses and practices. In addition, 
while we did review existing relevant information regarding performance 
data quality where available, we did not systematically assess the 
quality of the performance information used in the examples we cite. 
Finally, for most of the examples, we describe how performance 
information was used to make decisions, but we did not attempt to 
verify that the use ultimately resulted in improved outcomes. 

From our review of the literature and interviews with experts and staff 
from the five agencies, we developed a conceptual framework identifying 
four categories of uses of performance information and five categories 
of practices that contribute to using performance information. The 
framework and categories of uses and practices are consistent with the 
themes identified in our March 2003 report on results-oriented cultures 
and effective performance management, and are consistent with 
additional GAO reports on GPRA and results-oriented 
government.[Footnote 41] The four use categories in our framework that 
we identified through our independent analysis are similar to the four 
types of uses identified in How Federal Programs Use Outcome 
Information: Opportunities for Federal Managers, by Harry Hatry, Elaine 
Morley, Shelli Rossman, and Joseph Wholey, published by the IBM 
Endowment for The Business of Government in May 2003. We recognize that 
alternative categories of uses and practices could have been developed 
and that there may be additional uses and practices that we did not 
identify. 

To provide additional support and further illustrate the uses and 
practices, we conducted two discussion groups. The first discussion 
group included about 15 agency officials from the 5 selected case 
agencies and was intended to gain further perspective on those 
agencies' uses and practices. To broaden the range of experiences we 
could draw upon, we also conducted a second discussion group consisting 
of about 15 representatives from 5 different agencies that were not in 
the pool of selected case agencies: the Departments of Agriculture, 
Health and Human Services, Housing and Urban Development, the Interior, 
and Justice. For each discussion group, we asked agencies to nominate 
potential participants who had hands-on experience with using 
performance information, at least 3 years of management experience, and 
were at a GS-13 level or higher. We selected participants on the basis 
of their self-described experience using performance information and 
selected them to reflect a range of programs within agencies. 

We conducted our review from September 2004 through August 2005 in 
accordance with generally accepted government auditing standards. 

[End of section]

Appendix II: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Bernice Steinhardt (202) 512-6543 or [Hyperlink, steinhardtb@gao.gov]. 

Acknowledgments: 

In addition to the contact name above, Elizabeth Curda, Assistant 
Director; Maya Chakko; Chelsa Gurkin; Anne Inserra; Anne Marie 
Morillon; and Susan Wilschke made significant contributions to this 
report. In addition, Tom Beall provided key assistance. 

[End of section]

Bibliography: 

Ammons, David N. Municipal Benchmarks: Assessing Local: 

Performance and Establishing Community Standards. Thousand Oaks, 
California: Sage Publications, 1996. 

Fountain, James, Wilson Campbell, Terry Patton, Paul Epstein, Mandi: 

Cohn, Mark Abrahams and Jonathan Walters. Reporting Performance 
Information: Suggested Criteria for Effective Communication. 
Governmental Accounting Standards Board: Norwalk, Connecticut, 2003. 

Hatry, Harry P., Performance Measurement: Getting Results. Washington,

D.C.: The Urban Institute, 1999. 

------"Public and Private Agencies Need to Manage for Results, Not 
Just: 

Measure Them." Washington, D.C.: The Urban Institute, August 31, 2004. 
http://www.urban.org/url.cfm?ID=900731: 

Hatry, Harry P., Elaine Morley, Shelli B. Rossman and Joseph S. Wholey. 

How Federal Programs Use Outcome Information: Opportunities for Federal 
Managers. Washington, D.C.: IBM Center for the Business of Government, 
2003. 

Morley, Elaine, Harry P. Hatry and Jake Cowan. Making Use of Outcome: 

Information for Improving Services: Recommendations for Nonprofit 
Organizations. Washington, D.C.: The Urban Institute, 2002. 

Newcomer, Kathryn, Edward T. Jennings, Jr., Cheryle Broom and Allen: 

Lomax. Meeting the Challenges of Performance-Oriented Government. 
Washington, D.C.: American Society for Public Administration, 2002. 

Osborne, David and Peter Hutchinson. The Price of Government: Getting: 

the Results We Need in an Age of Permanent Fiscal Crisis. New York: 
Basic Books, 2004. 

Wholey, Joseph S. Overcoming the Challenges in Managing for Results. 

Washington, D.C.: IBM Center for the Business of Government, 2004. 

[End of section]

Related GAO Products: 

Workforce Investment Act: States and Local Areas Have Developed 
Strategies to Assess Performance, but Labor Could Do More to Help. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-651]. Washington, 
D.C.: June 1, 2004. 

Results-Oriented Government: GPRA Has Established a Solid Foundation 
for Achieving Greater Results. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-38]. Washington, D.C.: March 10, 2004. 

Highlights of a GAO Forum: High Performing Organizations: Metrics, 
Means, and Mechanisms for Achieving High Performance in the 21ST 
Century Public Management Environment. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-04-343SP]. Washington, D.C.: 
February 13, 2004. 

Human Capital: Key Principles for Effective Strategic Workforce 
Planning. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-39]. 
Washington, D.C.: December 11, 2003. 

Results-Oriented Government: Using GPRA to Address 21ST Century 
Challenges. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-
1166T]. Washington, D.C.: September 18, 2003. 

Program Evaluation: An Evaluation Culture and Collaborative 
Partnerships Help Build Agency Capacity. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-454]. Washington, D.C.: May 2, 
2003. 

Results-Oriented Cultures: Creating a Clear Linkage between Individual 
Performance and Organizational Success. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-488]. Washington, D.C.: March 
14, 2003. 

Human Capital Management: FAA's Reform Effort Requires a More Strategic 
Approach. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-156]. 
Washington, D.C.: February 3, 2003. 

Pension and Welfare Benefits Administration: Opportunities Exist for 
Improving Management of the Enforcement Program. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-02-232]. Washington, D.C.: March 
15, 2002. 

Results-Oriented Budget Practices in Federal Agencies. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-01-1084SP]. Washington, D.C.: 
August 2001. 

Managing for Results: Federal Managers' Views on Key Management Issues 
Vary Widely Across Agencies. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-01-592]. Washington, D.C.: May 25, 2001. 

Managing for Results: Federal Managers' Views Show Need for Ensuring 
Top Leadership Skills. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-01-127]. Washington, D.C.: October 20, 2000. 

Performance Plans: Selected Approaches for Verification and Validation 
of Agency Performance Information. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/GGD-99-139]. Washington, D.C.: July 30, 1999. 

Executive Guide: Effectively Implementing the Government Performance 
and Results Act. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/GGD-
96-118]. Washington, D.C.: June 1996. 

Results-Oriented Cultures: Insights for U.S. Agencies from Other 
Countries' Performance Management Initiatives. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-02-862]. Washington, D.C.: August 
2, 2002. 

Managing for Results: Efforts to Strengthen the Link Between Resources 
and Results at the Veterans Health Administration. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-03-10]. Washington, D.C.: 
December 10, 2002. 

Results-Oriented Cultures: Implementation Steps to Assist Mergers and 
Organizational Transformations. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-03-669]. Washington, D.C.: July 2, 2003. 

Human Capital: Senior Executive Performance Management Can Be 
Significantly Strengthened to Achieve Results. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-04-614]. Washington, D.C.: May 
26, 2004. 

Workforce Investment Act: Labor Should Consider Alternative Approaches 
to Implement New Performance and Reporting Requirements. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-05-539]. Washington, D.C.: May 
27, 2005. 

The Government Performance and Results Act: 1997 Governmentwide 
Implementation Will Be Uneven. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/GGD-97-109]. Washington, D.C.: June 2, 1997. 

Tax Administration: IRS Needs to Further Refine Its Tax Filing Season 
Performance Measures. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-
03-143]. Washington, D.C.: November 22, 2002. 

Architect of The Capitol: Midyear Status Report on Implementation of 
Management Review Recommendations. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-966]. Washington, D.C.: August 31, 2004. 

Managing For Results: Emerging Benefits From Selected Agencies' Use of 
Performance Agreements. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-01-115]. Washington, D.C.: October 30, 2000. 

Human Capital: Observations on Final DHS Human Capital Regulations. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-391T]. Washington, 
D.C.: March 2, 2005. 

Managing For Results: Strengthening Regulatory Agencies' Performance 
Management Practices. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/GGD-00-10]. Washington, D.C.: October 28, 1999. 

Human Capital: A Self-Assessment Checklist for Agency Leaders. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/OCG-00-14G]. 
Washington, D.C.: September 2000. 

Managing For Results: Challenges Agencies Face in Producing Credible 
Performance Information. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/GGD-00-52]. Washington, D.C.: February 4, 2000. 

Human Capital: Preliminary Observations on Proposed DOD National 
Security Personnel System Regulations. [Hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-05-432T]. Washington, D.C.: March 
15, 2005. 

(450365): 

FOOTNOTES

[1] GPRA, Pub. L. No. 103-62 (Aug. 3, 1993), was enacted to help 
resolve the long-standing management problems that undermined the 
federal government's efficiency and effectiveness and provide greater 
accountability for results. 

[2] GAO, Results-Oriented Government: GPRA Has Established a Solid 
Foundation for Achieving Greater Results, GAO-04-38 (Washington, D.C.: 
Mar. 10, 2004). 

[3] GAO-04-38. 

[4] GAO, Managing for Results: Federal Managers' Views on Key 
Management Issues Vary Widely Across Agencies, GAO-01-592 (Washington, 
D.C.: May 25, 2001). The 2000 survey provided the most current data on 
specific agencies, while the 2003 survey provided governmentwide data. 

[5] GAO, Executive Guide: Effectively Implementing the Government 
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 
1996); and GAO, Results-Oriented Cultures: Creating a Clear Linkage 
between Individual Performance and Organizational Success, GAO-03-488 
(Washington, D.C.: Mar. 14, 2003). 

[6] The four use categories we identified through our independent 
analysis are similar to the four types of uses identified in How 
Federal Programs Use Outcome Information: Opportunities for Federal 
Managers, by Harry Hatry, Elaine Morley, Shelli Rossman and Joseph 
Wholey, published by the IBM Endowment for the Business of Government 
in May 2003. Furthermore, these categories are consistent with, but not 
identical to, the types of uses captured in questions for our surveys 
of federal agency managers described earlier. 

[7] GAO-04-38, GAO-01-592, and GAO/GGD-96-118. 

[8] Pub. L. No. 105-220 (Aug. 7, 1998). The Workforce Investment Act of 
1998 required 17 employment and training programs, funded by four 
different agencies, to centralize service delivery through a one-stop- 
center system. WIA also established performance measures that look at a 
broad array of participant outcomes, such as job placement and 
retention, earnings, skill gains, and customer satisfaction. 

[9] The first PEP project ended in June 2004. The second PEP project 
began in July 2004 and is currently scheduled to end in August 2005. 

[10] A tutorial became available online at www.workforcetools.org in 
December 2004. 

[11] VHA networks are regional VHA units composed of health care 
facilities. 

[12] As will be discussed in more detail later in this report, if one 
of the facilities is failing a performance measure, then the entire 
network fails the performance measure. 

[13] GAO, Results-Oriented Government: Using GPRA to Address 21ST 
Century Challenges, GAO-03-1166T (Washington, D.C.: Sept. 18, 2003); 
GAO, Results-Oriented Budget Practices in Federal Agencies, GAO-01- 
1084SP (Washington, D.C.: August 2001); and GAO, Human Capital: Key 
Principles for Effective Strategic Workforce Planning, GAO-04-39 
(Washington, D.C.: December 2003). 

[14] Impaired driving is driving under the influence of alcohol or 
other drugs. 

[15] FARS is a database containing information on fatal crashes in the 
United States, including specific details about crash circumstances and 
vehicle and victim demographic information. It is maintained by NHTSA's 
National Center for Statistics and Analysis. 

[16] Agencies can also use performance information to recognize and 
reward contractors for achieving results. However, we did not analyze 
any examples of performance-based contracting for this report. 

[17] GAO-03-488. 

[18] We previously reviewed FAA's human capital reforms and found that 
FAA had little data with which to assess the effects of its reform 
effort. See GAO, Human Capital Management: FAA's Reform Effort Requires 
a More Strategic Approach, GAO-03-156, (Washington, D.C.: Feb. 3, 
2003). 

[19] Section 347 of the Fiscal Year 1996 Department of Transportation 
Appropriations Act, Pub. L. No. 104-50 (Nov. 15, 1995), provided FAA 
with special flexibilities with regard to its human capital policies. 
49 U.S.C. § 40122. 

[20] FAA's 2005-2009 Flight Plan can be found online at http://
www.faa.gov. 

[21] 29 U.S.C. § 2871(g). The authority to provide financial incentives 
and levy sanctions based on performance is specific to WIA legislation 
and associated WIA programs. Similar authority may or may not be 
provided for in legislation for other grant programs. 

[22] We have reported on quality issues regarding the performance data 
generated by state WIA programs. In our report, Workforce Investment 
Act: States and Local Areas Have Developed Strategies to Assess 
Performance, but Labor Could Do More to Help, GAO-04-657 (Washington, 
D.C.: June 1, 2004) we discuss OMB's proposed common measures, and data 
quality issues such as the lack of consistent definitions which leads 
to variations in reporting. 

[23] The two other programs are adult education and vocational 
education, authorized by the Adult Education and Family Literacy Act 
and the Carl D. Perkins Vocational and Technical Education Act, 
respectively. 

[24] More specific criteria for sanctions and incentives implementation 
can be found in DOL's Training and Employment Guidance Letter No. 19- 
02, located online at http://wdr.doleta.gov/directives/attach/TEGL19- 
02.cfm. 

[25] GAO, Highlights of a GAO Forum: High Performing Organizations: 
Metrics, Means, and Mechanisms for Achieving High Performance in the 
21ST Century Public Management Environment, GAO-04-343SP (Washington, 
D.C.: Feb. 13, 2004). 

[26] The NWS Strategic Plan for 2005-2010 can be found online at http:/
/www.nws.noaa.gov/sp/. 

[27] Workforce investment areas are designated within states by 
governors. 

[28] Three case studies have been published so far, and another five 
are being developed. 

[29] GAO, Managing for Results: Federal Managers' Views Show Need for 
Ensuring Top Leadership Skills, GAO-01-127 (Washington, D.C.: Oct. 20, 
2000). 

[30] GAO/GGD-96-118. 

[31] GAO-04-38. 

[32] GAO, Performance Plans: Selected Approaches for Verification and 
Validation of Agency Performance Information, GAO/GGD-99-139 
(Washington, D.C.: July 30, 1999). 

[33] We reviewed EBSA's previous database system in GAO, Pension and 
Welfare Benefits Administration: Opportunities Exist for Improving 
Management of the Enforcement Program, GAO-02-232 (Washington, D.C., 
Mar. 15, 2002). 

[34] Tables for specific SBA metrics can be found online at http://
www.sba.gov/PAR.pdf/strategicgoal1.html. 

[35] GAO-04-38. 

[36] GAO, Program Evaluation: An Evaluation Culture and Collaborative 
Partnerships Help Build Agency Capacity, GAO-03-454 (Washington, D.C.: 
May 2, 2003). 

[37] GAO/GGD-96-118. 

[38] The FAA Quarterly Performance Report can be found online at http:/
/www.faa.gov/about/plans_reports/performance/performancetargets/book0/
section0/section.html. 

[39] A runway incursion is any occurrence in the airport runway 
environment involving an aircraft, vehicle, person, or object on the 
ground that creates a collision hazard or results in a loss of required 
separation with an aircraft taking off, intending to take off, landing, 
or intending to land. 

[40] GAO-01-592. 

[41] GAO-01-592, GAO-03-488, and GAO-04-38. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: