This is the accessible text file for GAO report number GAO-07-58 
entitled 'NASA Procurement: Use of Award Fees for Achieving Program 
Outcomes Should Be Improved' which was released on February 16, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Committee on Science, House of Representatives: 

United States Government Accountability Office: 

GAO: 

January 2007: 

NASA Procurement: 

Use of Award Fees for Achieving Program Outcomes Should Be Improved: 

GAO-07-58: 

GAO Highlights: 

Highlights of GAO-07-58, a report to the Chairman, Committee on 
Science, House of Representatives 

Why GAO Did This Study: 

Cost-plus-award-fee contracts accounted for almost half of the National 
Aeronautic and Space Administration’s (NASA) obligated contract dollars 
for fiscal years 2002-2004. Since 1990, we have identified NASA’s 
contract management as a high-risk area—in part because of a lack of 
emphasis on end results. You asked us to examine (1) the extent NASA’s 
guidance on award fees addresses problems previously identified with 
the use of award-fee contracts and (2) whether NASA follows its 
guidance in using award fees to achieve desired outcomes. We reviewed 
the top 10 dollar value award-fee contracts active from fiscal years 
2002 through 2004. 

What GAO Found: 

NASA guidance on the use of cost-plus-award-fee (CPAF) contracts 
provides criteria to improve the effectiveness of award fees. For 
example, the guidance emphasizes outcome factors that are good 
indicators of success in achieving desired results, cautions against 
using numerous evaluation factors, prohibits rollover of unearned fee, 
and encourages evaluating the costs and benefits of such contracts 
before using this contract type. 

However, NASA does not always follow the preferred approach laid out in 
its guidance. For example, some evaluation criteria contained input or 
process factors, such as program planning and organizational 
management. Moreover, some contracts included numerous supporting 
subfactors that may dilute emphasis on any specific criteria. Although 
the Federal Acquisition Regulation and NASA guidance require 
considering the costs and benefits of choosing a CPAF contract, NASA 
did not perform such analyses. 

In some cases there appears to be a significant disconnect between 
program results and fees paid. For example, NASA paid the contractor 
for the Earth Observing System Data and Information System Core System 
97 percent of the available award fee despite a delay in the completion 
of the contract by over 2 years and an increase in the cost of the 
contract of more than 50 percent. 

NASA officials expressed satisfaction with the results of the contracts 
we reviewed, and this was further evidenced by the extent of fee paid. 
NASA’s satisfaction was based on its evaluations of contractor 
performance against criteria established in the award-fee plan. While 
NASA’s evaluations would indicate generally good contractor 
performance, that performance did not always translate into desired 
program outcomes. That disconnect raises questions as to the extent 
NASA is achieving the effectiveness it sought through the establishment 
of guidance on the use of award fees. NASA has not evaluated the 
overall effectiveness of award fees and does not have metrics in place 
for conducting such evaluations. 

What GAO Recommends: 

We recommend NASA improve its current use of award fees by 
reemphasizing tying award-fee payments to desired outcomes, limiting 
the number of factors used in contractor evaluations as its guidance 
recommends, and by using this contract type only when justified by a 
consideration of costs and benefits. We also recommend that NASA 
develop metrics for measuring the effectiveness of award fees, 
establish a system for collecting data on the use of award-fee 
contracts, and regularly examine the effectiveness of award fees in 
achieving desired outcomes. In commenting on a draft of this report, 
NASA concurred with all three recommendations. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-58]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Ann M. Calvaresi-Barr, 
(202) 512-4841, calvaresibarra@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

NASA's Award-Fee Policy Addresses Many Cost-Plus-Award-Fee Contracting 
Issues Identified as Problematic: 

NASA Has Not Always Followed the Preferred Approach Laid Out in Its 
Award-Fee Guidance: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Summary Description of the 10 NASA Contracts GAO Reviewed: 

Appendix III: Comments from the National Aeronautics and Space 
Administration: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Award Fees as a Negotiated Percentage of Contract Value: 

Table 2: Percentage of Available Award Fee Pool Paid on Contracts 
Examined: 

Abbreviations: 

CPAF: cost-plus-award-fee: 

CSOC: Consolidated Space Operations Contract: 

DOD: Department of Defense: 

ECS: EOSDIS Core System: 

EOSDIS: Earth Observing System Data and Information System: 

ETM+: Enhanced Thematic Mapper Plus: 

FAR: Federal Acquisition Regulation: 

FDO: fee determination official: 

IEMP: Integrated Enterprise Management Program: 

ISS: International Space Station: 

JPL: Jet Propulsion Laboratory: 

MSES: Mechanical System Engineering Services: 

NASA: National Aeronautics and Space Administration: 

PEB: performance evaluation board: 

SEAT: Science, Engineering, Analysis, and Test: 

United States Government Accountability Office: 
Washington, DC 20548: 

January 17, 2007: 

The Honorable Bart Gordon: 
Chairman: 
Committee on Science: 
House of Representatives: 

The Federal Acquisition Regulation (FAR) states that cost-plus-award- 
fee (CPAF) contracts are intended to motivate excellent contractor 
performance in areas such as quality, timeliness, technical ingenuity, 
and cost-effective management.[Footnote 1] During the early 1990s, the 
National Aeronautics and Space Administration (NASA) Inspector General 
and NASA internal studies raised concerns about NASA's use of CPAF 
contracts.[Footnote 2] As a result, NASA developed specific guidance to 
improve the effectiveness of award fees. The CPAF contract type 
continues to be used extensively by NASA for obtaining both goods and 
services, accounting for almost half of NASA contract dollars for 
fiscal years 2002 through 2004.[Footnote 3] Given this, you requested 
that we examine NASA's use of award-fee contracts and determine (1) the 
extent NASA's guidance addresses the problems previously identified 
with the use of award-fee contracts and (2) whether NASA follows its 
guidance in using award fees to achieve desired outcomes. 

To address these objectives, we reviewed a sample of NASA CPAF 
contracts that were active from fiscal years 2002 through 
2004.[Footnote 4] We reviewed contract files, obtained information from 
program and contracting officials through the use of a structured 
questionnaire, and discussed the application of award-fee criteria with 
NASA officials involved in the award-fee process. Our work was 
conducted between August 2005 and October 2006 in accordance with 
generally accepted government auditing standards. For a complete 
description of our scope and methodology, see appendix I. 

Results in Brief: 

NASA regulations and guidance on the use of cost-plus-award-fee 
contracts address many of the issues and problems identified by NASA 
and the NASA Inspector General and provide criteria for appropriately 
using award-fee contracts. For example, NASA encourages tying fees to 
outcomes, prohibits the rollover of unearned award fee, authorizes the 
use of interim fees on end item contracts until final product delivery, 
and encourages the use of performance-based contracts for the 
procurement of services and supplies. Further, because of the cost and 
administrative burden associated with managing this type of contract, 
the FAR and NASA's Award Fee Contracting Guide recommend evaluating the 
costs and benefits of using a cost-plus-award-fee contract before 
committing to this contract type. 

NASA did not consistently implement key aspects of the agency's 
guidance on major award-fee contracts that we reviewed. NASA's award- 
fee guide states that while it is sometimes valuable to consider input 
factors when evaluating contractor performance, it is NASA's preference 
to use outcome-based criteria, and each of the contracts we reviewed 
had some outcome based criteria. However, some criteria used to 
evaluate performance were process or input factors, such as awarding 
portions of the fee for the quality and effectiveness of the 
contractor's scheduling system, and program planning and organizational 
management. Other contracts used numerous subfactors for evaluating 
contractor performance, which, according to NASA's award-fee guide, can 
dilute emphasis on any specific criteria. For example, one contract 
specified 3 primary performance evaluation factors, but included 96 
subfactors for fiscal year 2004 and 108 subfactors for fiscal year 
2005. Also, although the FAR and NASA's award-fee guide specify 
consideration of the costs and benefits before using a CPAF contract 
because of the cost and administrative burden associated with these 
contracts, no analysis of costs and benefits was conducted for the 
contracts we reviewed. Finally, NASA officials expressed satisfaction 
with the results of the contracts we reviewed. NASA's satisfaction was 
based on its evaluations of contractor performance against criteria 
established in the award-fee plan. While NASA's evaluations would 
indicate generally good contractor performance, such performance did 
not always translate into desired program outcomes. That disconnect 
raises questions as to the extent NASA is achieving the effectiveness 
it sought through the establishment of guidance on award fees. 
Specifically, our analysis showed that NASA paid significant amounts of 
available fee on all of the contracts we reviewed, including those end 
item contracts that did not deliver a capability within initial cost, 
schedule, and performance parameters. For example, NASA paid the 
contractor for the Earth Observing System Data and Information System 
Core System 97 percent of the available award fee despite a delay in 
the completion of the contract of more than 2 years and an increase in 
the cost of the contract of more than 50 percent. Since revising its 
guidance, NASA has not evaluated the effectiveness of award fees in 
achieving program results and does not have metrics in place for 
measuring the effectiveness of award fees. 

We are recommending that NASA improve its current implementation of 
cost-plus-award-fee contracts by reemphasizing its current guidance on 
tying award-fee payments, particularly on major end item contracts, to 
outcome factors and limiting the number of subfactors used in 
evaluations and to use this contract type only when justified by a 
consideration of costs and benefits. Further, we are recommending that 
NASA develop metrics for measuring and a mechanism for evaluating the 
effectiveness of award fees in achieving desired outcomes. In 
commenting on a draft of this report, NASA concurred with all three 
recommendations. 

Background: 

In January 2004, the President announced a new "Vision for Space 
Exploration" calling for human and robotic missions to the Moon, Mars, 
and beyond. Over the next two decades, NASA plans to spend over $100 
billion to develop a number of new capabilities, supporting 
technologies, and facilities that are critical to enabling space 
exploration missions. Development of the critical capabilities and 
technologies will be largely dependent on NASA contractors, who 
constitute more than two-thirds of NASA's workforce.[Footnote 5] 
According to NASA officials, 87 percent of NASA's $16.6 billion budget 
for fiscal year 2006 was spent on work performed by its contractors. 

Since 1990, we have designated NASA's contract management as a high- 
risk area. This is based primarily on NASA's lack of a modern 
integrated financial management system that can provide reliable 
information on contract spending and performance as well as NASA's lack 
of emphasis on end results, product performance, and cost control. For 
example, our most recent high-risk report stated that while NASA has 
taken actions to improve its contract management function, it continues 
to face considerable challenges in implementing its contracts 
effectively.[Footnote 6] 

NASA is organized under four mission directorates--Aeronautics 
Research, Exploration, Science, and Space Operations--each of which 
covers a major area of the agency's research and development efforts. 
The agency is composed of NASA headquarters, 10 field centers, and the 
contractor-operated Jet Propulsion Laboratory.[Footnote 7] 

Cost-Plus-Award-Fee Contracts: 

NASA and other federal agencies can choose among numerous contract 
types for acquiring goods and services that can differ in part 
according to the nature of the fee that agencies offer to the 
contractor for achieving or exceeding specified objectives or goals. 
According to the FAR, a CPAF contract is appropriate to use when it is 
difficult to measure key elements of performance. It is widely used to 
procure nonroutine services such as the development of new systems. 
Typically, award-fee contracts emphasize several aspects of contractor 
performance, such as schedule performance, technical performance, and 
cost control. Because development and administration of award-fee 
contracts involve substantially more effort over the life of a contract 
than other types of contracts, the FAR and NASA's Award Fee Contracting 
Guide specify that the expected benefits of using an award-fee contract 
must exceed the additional administrative effort and cost 
involved.[Footnote 8] 

The theory behind CPAF contracts is that although the government 
assumes most of the cost risk, it retains control over most or all of 
the contractor's potential profit as leverage. On CPAF contracts, the 
award fee is often the only source of potential fee for the contractor. 
According to the NASA FAR Supplement and NASA's Award Fee Contracting 
Guide[Footnote 9], these contracts can include a base fee of anywhere 
from 0 to 3 percent of the estimated value of a nonservice contract. 
However, NASA's regulations and guide do not allow the use of a base 
fee on service contracts. Table 1 shows the percentage of award fee 
available on the contracts we examined. (See app. II for a description 
of these contracts.) 

Table 1: Award Fees as a Negotiated Percentage of Contract Value: 

Contract: Jet Propulsion Laboratory[A]; 
Award-fee percentage: 1.5. 

Contract: International Space Station[B]; 
Award-fee percentage: 11.0. 

Contract: Consolidated Space Operations; 
Award-fee percentage: 10.0. 

Contract: Joint Base Operation and Support; 
Award-fee percentage: 8.0. 

Contract: Science, Engineering, Analysis, and Test; 
Award-fee percentage: 6.0. 

Contract: Engineering and Technical Support for Life Sciences; 
Award- fee percentage: 5.0. 

Contract: Program Information Systems Mission Services; 
Award-fee percentage: 6.0. 

Contract: Earth Observing System Data and Information System Core 
System; 
Award-fee percentage: 9.9. 

Contract: Mechanical Systems Engineering Services; 
Award-fee percentage: 9.2. 

Contract: Landsat-7; 
Award-fee percentage: 7.0. 

Sources: NASA submissions to GAO and contract and award-fee 
documentation; GAO (analysis). 

Note: Half of the 10 contracts we reviewed also included other types of 
fee or incentives including performance incentives and fixed fees. 

[A] Although this contract for the operation and management of the Jet 
Propulsion Laboratory is a CPAF contract, it differs from the other 
contracts we reviewed in that the contractor, the California Institute 
of Technology (Caltech), is a nonprofit educational institution. 
According to the NASA FAR Supplement 1815.404-471-6(a), it is NASA's 
policy not to pay profit or fee on contracts with educational 
institutions. This contract is an exception. According to NASA 
officials, the fee paid becomes part of Caltech's investment in the 
institution's educational programs and the infrastructure supporting 
the research efforts made by Caltech. 

[B] Eleven percent is the maximum award fee available as negotiated in 
the December 2003 International Space Station contract extension. 

[End of table] 

NASA's Use of Cost-Plus-Award-Fee Contracts: 

NASA relies heavily on CPAF contracts. This contract type accounted for 
48 percent of obligated contract dollars and 7.7 percent of contract 
actions from fiscal years 2002 through 2004. By comparison, between 
fiscal years 1999 and 2003, award-fee contracts accounted for 13 
percent of the contract dollars and 3.4 percent of contract actions at 
the Department of Defense (DOD). A CPAF contract includes an estimate 
of the total cost of what is being contracted for, may include a fee 
with a possible base amount fixed at the inception of the contract, and 
includes an award amount that is intended to motivate excellence in 
contract performance. The award fee is paid based upon the government's 
periodic judgmental evaluations of contractor performance. 

Award-Fee Criteria: 

When developing evaluation plans, NASA's award-fee guide indicates that 
evaluation plans may include outcomes, outputs, inputs, or a 
combination of these elements. NASA's guide expresses a preference for 
outcome factors. It notes that while it is sometimes valuable to 
consider input and output factors when evaluating contractor 
performance, outcome factors are better indicators of success relative 
to the desired result. 

* An outcome factor is an assessment of the results of an activity 
compared to its intended purpose. Outcome-based factors are the least 
administratively burdensome type of performance evaluation factor, and 
should provide the best indicator of overall success. Outcome-based 
factors should therefore be the first type of evaluation factor 
considered for use, and are often ideal for nonroutine efforts. 

* An output factor is the tabulation, calculation, or recording of 
activity or effort and can be expressed in a quantitative or 
qualitative manner. Output factors may be more desirable for routine 
efforts, but are administratively more burdensome than outcome factors 
due to the tabulation, calculation, or recording requirements. When 
output factors are used, care should be taken to ensure that there is a 
logical connection between the reported measures and the program's 
mission, goals, and objectives. 

* Input factors refer to intermediate processes, procedures, actions, 
or techniques that are key elements influencing successful contract 
performance. These may include testing and other engineering processes 
and techniques; quality assurance and maintenance procedures; 
subcontracting plans; purchasing department management; and inventory, 
work assignment, and budgetary controls. 

Evaluation of Award-Fee Contracts: 

For CPAF contracts, NASA personnel conduct periodic, typically 
semiannual evaluations of contractor's performance against the criteria 
specified in a performance evaluation plan. During the course of the 
evaluation period, performance monitors track contractor performance, 
and once the period is over they assess the performance and report to 
the performance evaluation board (PEB). The PEB considers the reports 
as well as any other pertinent information and prepares a report for 
the fee determination official (FDO) with findings and recommendations. 
The contractor is given an opportunity to provide a self-assessment of 
its performance during the evaluation period, which is often a written 
report. The FDO may meet with the PEB to discuss the report, after 
which a final determination is made in writing as to the amount of fee 
to be paid. The FDO provides the determination to the contracting 
officer and a copy of the related document to the contractor. 

Acquisition Environment Can Affect Acquisition Outcomes: 

When discussing award-fee contracts, it is important to acknowledge the 
acquisition environment in which they are used. Award fees are intended 
to motivate excellent contractor performance, which should result in 
excellent program outcomes. However, award fees should not be used to 
make up for factors internal or external to the acquisition environment 
that hinder the success of acquisition outcomes. These factors may 
include inadequate resources and financial management systems, lack of 
knowledge prior to starting the acquisition, or unsound acquisition 
practices. We have reported that in some cases, NASA's failure to 
define requirements adequately and develop realistic cost estimates 
resulted in projects costing more, taking longer, and achieving less 
than originally planned.[Footnote 10] The persistence of these problems 
in NASA contract management is not only indicative of undisciplined 
processes or practices such as these, but may also reflect the fact 
that the design, development, and production of major space systems are 
extremely complex technical processes that must operate within complex 
budget and political processes. Even properly run programs can 
experience problems that may arise from unknowns, such as technical 
obstacles and changes in circumstances. Only a few things need to go 
wrong to cause major problems, and many things must go right for a 
program to be successful. 

NASA's Award-Fee Policy Addresses Many Cost-Plus-Award-Fee Contracting 
Issues Identified as Problematic: 

The NASA FAR Supplement and NASA's Award Fee Contracting Guide address 
many of the issues and problems identified by NASA on the use of award- 
fee contracts and provide criteria for appropriately using such 
contracts. Much of the guidance on award-fee contracting was issued in 
response to weaknesses in CPAF contracting practices identified by NASA 
internal reviews and NASA's Office of Inspector General in the early 
1990s.[Footnote 11] Those weaknesses included the awarding of excessive 
fees with limited emphasis on acquisition outcomes (end results, 
product performance, and cost control); rollover of unearned fee; use 
of base fee; and the failure to use both positive and negative 
incentives. NASA updated its award-fee guide in 1994, 1997, and 2001 to 
explain and elaborate on its award-fee policy. The 2001 revision also 
reflects the FAR's additional emphasis on using performance-based 
contracts. 

NASA's award-fee guide emphasizes tying fees to outcome factors. The 
guide states that outcome-based factors are the least administratively 
burdensome type of evaluation factor and should provide the best 
indicator of overall success. The award-fee guide warns against 
micromanaging performance and diluting the emphasis of criteria by 
spreading the potential award fee over a large number of performance 
evaluation factors. Instead, the guide recommends selecting broad 
performance evaluation factors, such as technical factors, project 
management, and cost control supplemented by a limited number of 
subfactors under these factors. 

Cost control is required to be a key performance evaluation factor in 
award-fee performance evaluation plans, largely because of past 
performance issues in which contractors were paid millions of dollars 
in fees on contracts that were experiencing hundreds of millions of 
dollars in cost overruns.[Footnote 12] The NASA FAR Supplement states 
that cost control shall be no less than 25 percent of the total 
weighted evaluation factors when explicit evaluation factor weightings 
are used. The NASA FAR Supplement states that emphasis on cost control 
should be balanced against other performance requirement objectives, 
and the contractor should not be incentivized to pursue cost control to 
the point that overall performance is significantly degraded. 

NASA's regulations prohibit rolling over unearned fee to subsequent 
evaluation periods for service contracts. For such contracts, each 
interim evaluation and the last evaluation are final. Another key 
element of the current award-fee regulations is an increased emphasis 
on overall contractor performance and the end product, rather than on 
incremental progress. NASA requires conducting interim evaluations on 
end item contracts until final product delivery to monitor performance 
prior to contract completion and establish the basis for making interim 
payments. At the end of the contract, a final evaluation is conducted 
and the contractor's total performance is evaluated against the award- 
fee plan to determine total earned award fee. For example, the 
contractor may be evaluated and paid an interim fee once every 6 months 
until the product is delivered. During the final evaluation, the 
contractor's performance is evaluated to determine total earned award 
fee. The final evaluation may result in the contractor retaining the 
fee previously awarded or receiving additional or less fee than 
previously awarded and thus refunding a portion of the fee to the 
government. The final evaluation provides NASA the opportunity to make 
an award-fee decision based on actual quality, total cost, and ability 
to meet the contract schedule at the point the final product is 
delivered. 

Further, under the award-fee policy in effect prior to the 1994 and 
subsequent revisions to the guidance, base fee was allowed on all CPAF 
contracts. NASA's current regulations prohibit the use of base fee on 
service contracts and restrict the use of base fee on end item 
contracts, such as for hardware. When base fee is used, it is not to 
exceed 3 percent of estimated contract costs and it should only be paid 
if the final award-fee evaluation is satisfactory or better. We note 
that base fee, which was paid on two of the three end item contracts we 
reviewed, did not exceed 3 percent, and none of the seven service 
contracts included base fee. 

Another issue addressed by NASA's regulations is the use of both 
positive and negative performance incentives in its CPAF contracts. The 
NASA FAR Supplement provides that award-fee contracts with primary 
deliverables of hardware and with a total estimated cost and fee of 
greater than $25 million require both kinds of incentives based on 
measurements of hardware performance against objective 
criteria.[Footnote 13] Performance incentives are separate and distinct 
from award fee and measure contractor performance up to delivery and 
acceptance. Performance incentives are designed to reward contractors 
when performance of delivered hardware is above minimum contract 
requirements. For example, if the government establishes a specified 
level of objective performance for a product that the contractor 
exceeds, the contractor can be paid a performance incentive in addition 
to the award fee already paid. If the contractor just meets this 
measure, it cannot receive an additional performance incentive and 
keeps the award fee already paid. If the contractor fails to meet the 
measure, however, it must pay a negative performance incentive fee that 
reduces or eliminates the entire award fee. 

To address inconsistencies among NASA centers in how they evaluate 
contractor performance, the current award-fee regulations also provide 
a uniform rating system to be used for all NASA award-fee contracts. It 
includes adjectival ratings as well as a numerical scoring system of 0- 
100. Scores of 61-70 percent are considered satisfactory, and the 
regulations specify that contractors receiving a rating of less than 61 
percent will not receive any fee. A contractor is not to be paid any 
base fee or award fee for less than satisfactory overall performance. 

NASA's award-fee guide encourages the use of performance-based 
contracts for the procurement of services and supplies. The guide 
states that constructing performance-based contracts that clearly 
define performance requirements, include easily understood performance 
standards, and have an objective incentive mechanism will result in 
contractors having a clearer understanding of the government's 
expectations and will ultimately facilitate enhanced contractor 
performance. 

Finally, because of the cost and administrative burden associated with 
administering award-fee contracts, the FAR and NASA's award-fee guide 
specify consideration of the costs and benefits of using a CPAF 
contract before committing to this contract type. Through an evaluation 
of the administrative costs versus the expected benefits, the 
contracting officer should be able to assess whether the benefits the 
government gains through a CPAF contract will outweigh the additional 
costs of overseeing and administering the contract. The award-fee guide 
provides an example of how to calculate the administrative cost and 
states that benefits could be measured in dollars saved through cost 
control or enhanced technical capability. 

NASA Has Not Always Followed the Preferred Approach Laid Out in Its 
Award-Fee Guidance: 

Although the revisions in NASA's regulations and guidance on award-fee 
contracts address many weaknesses previously identified, the contracts 
that we reviewed did not always demonstrate use of award fees by the 
centers in the way that NASA prefers as outlined in its guidance. Some 
performance evaluation plans or reports included input evaluation 
factors, which are not the best indicators of success relative to the 
desired result, although they are allowed by the guidance. Other 
contracts included numerous subcategories for evaluating the contractor 
that can lessen the importance of any particular subcategory and reduce 
the leverage of the award fee on any particular criterion. Also, 
although the FAR and NASA's award-fee guide calls for a consideration 
of the costs and benefits of using cost-plus-award-fee contracts 
because of the cost and administrative burden involved, we found no 
examples of a documented analysis of costs and benefits. Finally, NASA 
officials expressed satisfaction with the results of the contracts 
based on their evaluations of contractor performance against criteria 
established in the award-fee plan. Those evaluations would indicate 
generally good performance. However, that performance did not always 
translate into desired program outcomes. NASA paid a majority of the 
available award fee on all of the contracts we reviewed, including 
those end item contracts that did not deliver a capability within 
initial cost, schedule, and performance parameters.[Footnote 14] That 
disconnect raises questions as to the extent NASA is achieving the 
effectiveness it sought through the establishment of guidance on the 
use of award fees. Further, NASA has not evaluated the overall 
effectiveness of award fees in promoting program outcomes and does not 
have metrics in place for measuring their effectiveness in achieving 
program outcomes. 

Some Evaluation Factors Were Not Outcome Factors as Preferred by NASA 
Guidance: 

Some performance evaluation subfactors included in performance 
evaluation plans or reports were not outcome oriented. NASA's award-fee 
guide states that while it is sometimes valuable to consider input and 
output factors when evaluating contractor performance, it is NASA's 
preference when feasible to tie fees to evaluation factors that are 
based on outcomes because outcome-based factors provide the best 
indicator of overall success. The award-fee guide recommends selecting 
broad performance evaluation factors, such as technical factors, 
project management, and cost control, and cautions that factors related 
to intermediate processes, procedures, and actions may cause the 
contractor to divert its attention from the overall desired outcome. 
The guide states that those types of factors, while allowed, are not 
always true indicators of the contractor's performance and should be 
relied on with caution. Further, with service contracts, input factors 
may be of little or no value as a basis for evaluation. While the 
contracts we reviewed generally used outcome factors as part of the 
evaluation of performance, some supporting subfactors that formed the 
basis of the ratings for performance measured compliance with process 
or input factors that may not provide the best indicators of success 
relative to the desired results. 

For example, a part of the award fee on the Mechanical System 
Engineering Services (MSES) contract was to be awarded for program and 
business management performance. There were five subfactors under this 
primary performance factor. Two of these subfactors, program planning 
and organizational management and business management were input 
subfactors. These two input subfactors measure contractor processes or 
inputs, but do not focus on final results. Subfactors in the Landsat-7 
contract included input subfactors such as responsiveness of the 
contractor's corporate management, quality and effectiveness of the 
contractor's scheduling system, and prudent utilization of manpower and 
timely removal of manpower upon completion of tasks. 

Some Contracts Did Not Limit the Number of Subfactors for Evaluating 
Contractor Performance as NASA Guidance Recommends: 

The NASA award-fee guide cautions that spreading the potential award 
fee over a large number of performance evaluation factors dilutes 
emphasis on any particular performance evaluation criterion, increases 
the prospect of any one item being too small and thus overlooked, and 
increases the administrative burden. It encourages broad performance 
evaluation factors such as technical factors, project management, and 
cost control, which should be supplemented by only a limited number of 
subfactors describing significant evaluation elements over which the 
contractor has effective management control. Our analysis showed that a 
large number of subfactors were used to evaluate contractor performance 
for some contracts. 

For example, the Jet Propulsion Laboratory (JPL) contract, which 
includes both service and product deliverables defined in task orders 
under the contract, uses three primary performance evaluation factors 
for measuring contractor performance--programmatic, scientific, and 
engineering; institutional management; and support to outreach 
initiative programs. Although the JPL performance evaluation plan 
characterizes award-fee subfactors as representing major areas of 
emphasis during the performance period, the award-fee subfactors used 
to support the broad performance evaluation factors were numerous--96 
subfactors were used to evaluate the contractor's performance in fiscal 
year 2004, and 108 subfactors were used in fiscal year 2005.[Footnote 
15] The Engineering and Technical Support for Life Sciences contract 
used three broad performance evaluation factors also--technical 
performance, schedule performance and contract management, and cost 
control--but evaluated the contractor on numerous supporting subfactors 
identified as tasks or subtasks in the contractor performance 
evaluation reports. For example, on one task order under this contract, 
performance evaluation reports for various evaluation periods showed as 
many as 50 different subtasks used to evaluate the contractor's 
performance for the primary evaluation criteria: (1) technical 
performance and (2) schedule performance and contract management. 

The Landsat-7 contract also included a number of subfactors. Contractor 
performance under this contract was evaluated in several different 
areas each time the performance evaluation board met. Technical 
performance and program management were grouped together in one primary 
performance evaluation factor, and business management and cost 
performance were grouped together in the other primary performance 
evaluation factor. There were 9 subfactors under technical performance 
and 12 subfactors under program management, including quality and 
effectiveness of the contractor's scheduling system. Under business 
management and cost performance, 17 evaluation subfactors and elements 
were to be considered, including compliance with general contract 
provisions and clauses and weekly scheduling of teleconferences to 
determine schedule status. In addition to the number of subfactors that 
fell under the two primary performance evaluation factors, there were 
nine additional evaluation criteria, including resourcefulness, 
communication, and responsiveness. 

NASA Did Not Perform Cost-Benefit Analyses: 

Although the FAR and NASA's award-fee guide require consideration of 
the costs and benefits of using a CPAF contract before committing to 
this contract type to determine whether the benefits outweigh the 
additional cost and administrative burden of managing the contract, we 
found no instances where a documented cost-benefit analysis had been 
done for any of the contracts under our review. According to the 
guidance, since award-fee contracts require additional administrative 
effort, they should be used only when the contract values, performance 
period, and expected benefits are sufficient to warrant that additional 
management effort. Careful selection of the most appropriate contract 
type and careful tailoring should prevent a situation in which the 
burden of administering the award fee is out of proportion to the 
improvements expected in the quality of the contractor's performance 
and in overall management. In addition, CPAF contracts can be 
particularly costly and burdensome for NASA to administer because of 
contract reporting and review requirements. Major cost drivers include 
the number of award-fee periods, performance monitors, and performance 
evaluation board members necessary for implementing the award-fee 
process. For example, according to NASA's Award Fee Contracting Guide's 
conservative estimate, it would cost about $387,000 to administer the 
award-fee process over the life of a 5-year contract. The guide notes 
that the estimate does not represent all associated administrative cost 
that might arise. Although NASA procurement officials acknowledged that 
formal cost-benefit analyses were not prepared, some officials referred 
to determination and findings statements or acquisition strategy 
meeting documents associated with specific contracts as providing some 
evidence of consideration given to whether or not CPAF contracts should 
be used. 

Award-Fee Payments at Times Did Not Reflect Program Outcomes: 

While NASA officials expressed satisfaction with the results of the 
contracts, in some cases there appeared to be a disconnect between the 
fee paid and program results. NASA paid most of the available fee on 
all of the contracts we reviewed--including on projects that showed 
cost increases, schedule delays, and technical problems. The total 
estimated value of the 10 contracts we reviewed was more than $31 
billion. NASA paid between 80 and 99 percent of the maximum award fee 
possible on these contracts. The average was 90 percent, which equated 
to almost a billion dollars in total award fees paid under the 10 
contracts. Table 2 shows the percentage of award fee paid for each of 
the 10 contracts we reviewed. 

Table 2: Percentage of Available Award Fee Pool Paid on Contracts 
Examined: 

Contract: Jet Propulsion Laboratory[A]; 
Award-fee percentage: 90. 

Contract: International Space Station[A]; 
Award-fee percentage: 92. 

Contract: Consolidated Space Operations; 
Award-fee percentage: 80. 

Contract: Joint Base Operation and Support[A]; 
Award-fee percentage: 90. 

Contract: Science, Engineering, Analysis, and Test; 
Award-fee percentage: 80. 

Contract: Engineering and Technical Support for Life Sciences; 
Award-fee percentage: 91. 

Contract: Program Information Systems Mission Services; 
Award-fee percentage: 89. 

Contract: Earth Observing System Data and Information System Core 
System; 
Award-fee percentage: 97. 

Contract: Mechanical Systems Engineering Services; 
Award-fee percentage: 90. 

Contract: Landsat-7; 
Award-fee percentage: 99. 

Sources: NASA contract and award-fee documentation; GAO (analysis). 

[A] These contracts were still active at the conclusion of our audit 
work. The remaining 7 contracts were either closed or were in the 
process of being closed. 

Note: Numbers are rounded to the nearest whole percentage. 

[End of table] 

NASA officials expressed satisfaction with contract results, which was 
further evidenced by its evaluations of contractor performance against 
criteria established in the award-fee plan. While NASA's evaluations 
would indicate generally good performance, such performance did not 
always translate into desired program outcomes. That disconnect raises 
questions as to the extent NASA is achieving the effectiveness it 
sought through the establishment of guidance on the use of award-fees. 
On the end item contracts we reviewed, although there were some periods 
in which NASA paid a lesser percentage of the available fee, NASA 
ultimately paid more than 90 percent of the available fee based on its 
evaluation of contractor performance against criteria in the award-fee 
plan even when those contracts did not deliver capability within 
initial cost, schedule, and performance parameters. For example: 

* The prime contractor for the International Space Station (ISS) has 
received 92 percent of the total award fee available--$425.3 million-- 
although the cost increased by 131 percent, from $5.6 billion to $13 
billion, in part due to increased contract scope and delays caused by 
the Columbia accident, but also contractor cost overruns. In addition, 
the contractor estimates that it will incur an additional $76 million 
in overruns by the time the contract is completed. Further, the 
completion date for space station assembly under the prime contract was 
delayed by 8 years. In some cases these delays were caused by actions 
not within the control of the contractor, such as problems with the 
shuttle program and actions by the international partners. 

* The contractor for the Earth Observing System Data and Information 
System (EOSDIS) Core System (ECS) was paid 97 percent of the available 
award fee--$103.2 million--despite a delay in the completion of the 
contract by more than 2 years and an increase in the cost of the 
contract from $766 million to $1.2 billion. Technical problems, 
schedule delays, and cost control problems led to a major restructuring 
of the contract. 

* The Landsat-7 contractor was paid 99 percent of the available award 
fee or more than $17 million.[Footnote 16] The original contract was 
managed by the Air Force but was subsequently transferred to NASA and 
rebaselined. The cost of the contract when transferred to NASA and 
rebaselined was $342.7 million. The Landsat-7 launch was delayed by 9 
months and although the original scope of the work under the contract 
was significantly reduced[Footnote 17], the cost of the contract 
increased. By the time the contract was complete, costs had risen 20 
percent to $409.6 million. 

While some NASA officials pointed out that problems encountered on 
these contracts were at times outside the control of the contractor, 
difficulties such as these with achieving program results have resulted 
in NASA contract management being considered a high-risk area by GAO. 
We did not review these contracts to determine responsibility for 
undesirable results and therefore make no conclusion as to whether the 
fee paid was appropriate on each particular contract. However, the high 
fees paid on contracts where programs experienced disappointing results 
raise questions as to the effectiveness of award fees as a tool for 
obtaining desired program outcomes. 

For the service contracts we reviewed, NASA officials reported that 
they were satisfied with the results and quality of services provided. 
While we could not assess these contracts against cost, schedule, and 
performance outcomes as we could with the end item contracts, we did 
assess the award-fee criteria used in these contracts against NASA 
guidance. Here we found instances of process and input-oriented 
subfactors and the inclusion of numerous subfactors in evaluating 
performance. Further, we found no evidence that a cost-benefit analysis 
had been performed prior to choosing the contract type. Taken together, 
this is not the preferred approach according to NASA guidance, which 
raises questions as to the degree to which performance outcomes-- 
getting the quality of service desired--was actually the basis for 
judging contractor performance and awarding fee. 

NASA Has Not Assessed the Effectiveness of Award Fees in Achieving 
Program Outcomes or Developed Metrics for Conducting Such Evaluations: 

NASA views CPAF contracts as a viable and often preferred mechanism for 
acquiring the types of goods and services that the agency procures. 
NASA's satisfaction with the results of these contracts is evidenced by 
the level of fee paid on all of the contracts we reviewed and is based 
on NASA's evaluation of compliance with criteria contained in its award-
fee plans. However, the agency has not evaluated the overall 
effectiveness of award fees in promoting desired outcomes. As noted, 
NASA developed its new policies on award-fee contracts because the 
agency and its Office of Inspector General found that it was paying 
excessive fees with limited emphasis on acquisition outcomes. However, 
according to NASA officials, the agency has not completed any 
assessments of the effectiveness of award fees since the award-fee 
policy was restructured in the 1990s, nor has it developed metrics or 
performance measures to conduct such evaluations. Further, NASA lacks 
an agencywide system with the capability of compiling and aggregating 
award-fee information and for identifying trends and outcomes. 
According to NASA officials, even NASA's modern Integrated Enterprise 
Management Program (IEMP) will not provide this capability. Thus, NASA 
cannot meaningfully judge how well award fees are improving or can 
improve contractor performance and program outcomes. 

Conclusions: 

NASA could better link its award fees to desired results by making 
greater use of outcome factors, its preferred criteria for evaluating 
award fee contracts. While NASA has established policies and guidance 
that provide an appropriate framework for their use, the agency has not 
always used award fees as preferred by its guidance. To the extent that 
NASA uses input evaluation factors and numerous subfactors for 
evaluating performance, NASA may be diluting the leverage of award fees 
in achieving desired results. Our review raises questions as to the 
extent NASA is achieving the effectiveness it sought through the 
establishment of guidance on the use of award fees. However, NASA has 
not evaluated the overall effectiveness of its implementation of award 
fees. 

Recommendations for Executive Action: 

We are making the following three recommendations to increase the 
likelihood that the award fees NASA pays incentivize high performance 
from its suppliers. 

* We recommend that the NASA Administrator reemphasize to the NASA 
centers the importance of tying award-fee criteria to desired outcomes 
and limiting the number of subfactors used in evaluations. 

* To ensure that cost-plus-award-fee contracts are used only when their 
benefits outweigh the costs, we recommend that the NASA Administrator 
direct the centers to consider costs and benefits in choosing this 
contract type by requiring documentation explaining how the perceived 
benefits will offset the additional cost associated with its 
administration as required by the FAR. 

* Finally, we recommend that the NASA Administrator require the 
development of metrics for measuring the effectiveness of award fees, 
establish a system for collecting data on the use of award-fee 
contracts, and regularly examine the effectiveness of award fees in 
achieving desired acquisition outcomes. 

Agency Comments and Our Evaluation: 

In commenting on a draft of this report, NASA concurred with our 
recommendations and indicated that it would reemphasize its current 
guidance as recommended, address the issues raised by the report in 
training, and cover those issues in its internal reviews of procurement 
operations at the individual Space Centers. In terms of our 
recommendation to develop metrics for measuring the effectiveness of 
award fees and establish a system for collecting data on the use of 
award-fee contracts, NASA concurred and indicated it would explore the 
best way to develop and use metrics for evaluating the effectiveness of 
award fees and set up a system for collecting data on award-fee 
contracts. NASA said it planned to contact the Department of Defense to 
obtain information on its process, since DOD is also developing such a 
data collection system and metrics for measuring the effectiveness of 
award fees. NASA also provided technical comments on the draft, which 
have been incorporated as appropriate. 

As agreed with your office, unless you announce its contents earlier, 
we will not distribute this report further until 30 days from its date. 
At that time, we will send copies to interested congressional 
committees and the NASA Administrator. We will also make copies 
available to others upon request. In addition, the report will be 
available at no charge on the GAO Web site at [Hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-4841 or calvaresibarra@gao.gov. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. Key contributors to this report 
are acknowledged in appendix IV. 

Signed by: 

Ann M. Calvaresi-Barr: 
Director: 
Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Scope and Methodology: 

Scope and Methodology: 

Our objectives were to determine (1) the extent the National 
Aeronautics and Space Administration's (NASA) guidance addresses the 
problems previously identified with the use of award-fee contracts and 
(2) whether NASA follows its guidance in using award fees to achieve 
desired outcomes. 

We selected 10 NASA cost-plus-award-fee (CPAF) contracts to review. Our 
selection was based on contract data from the Federal Procurement Data 
System. We extracted information on all NASA contracts active between 
fiscal years 2002 and 2004 that were coded as CPAF. To ensure the 
validity of the database from which we drew our contracts, we confirmed 
the contract type of each of the 10 contracts we selected through NASA 
contracting officers and contract documentation. The contracts we 
selected were the top 10 dollar value contracts active from fiscal 
years 2002 through 2004. These contracts account for about $7.6 
billion, or 44 percent, of obligated cost-plus-award-fee-dollars for 
the 3-year period. 

To determine the extent NASA's guidance addresses the problems 
previously identified with the use of award-fee contracts and whether 
NASA follows its guidance in using award fees to achieve desired 
outcomes, we interviewed responsible program and procurement officials 
at NASA headquarters and six NASA centers. We also reviewed the Federal 
Acquisition Regulation (FAR), the NASA FAR Supplement, and NASA's Award 
Fee Contracting Guide. We conducted a literature review and examined 
previous reports, studies, and analyses done by GAO, NASA, the NASA 
Inspector General, or others that included information related to 
NASA's use of award fees and other relevant issues. 

Additionally, we reviewed contract files, obtained information from 
program and contracting officials through the use of a structured 
questionnaire, and discussed the application of award-fee criteria with 
NASA officials involved in the award-fee process. The contract 
documents we reviewed contained information related to the development 
and implementation of the award fee. This information included the 
basic contract and statement of work, acquisition planning documents, 
award-fee modifications, performance evaluation plan documentation 
describing fee criteria for specific evaluation periods, contractor 
self-assessments, performance evaluation board reports, and fee 
determination documents. We used this information to corroborate and 
supplement the information provided by NASA officials in response to 
structured questionnaires we prepared and interviews we conducted. We e-
mailed the questionnaires and received written responses for all 10 of 
the contracts. We conducted structured interviews with contracting and 
program officials concerning the development, implementation, and 
effectiveness of the award-fee structure for some of the contracts. 

To accomplish our work, we visited NASA headquarters in Washington, 
D.C. We also visited and held teleconferences with Goddard Space Flight 
Center in Greenbelt, Maryland, responsible for managing 3 of the 
contracts we reviewed; Johnson Space Center in Houston, Texas, 
responsible for managing 3 of the contracts; and Marshall Space Flight 
Center in Huntsville, Alabama, responsible for managing 1 of the 
contracts. We held teleconferences with officials at the Jet Propulsion 
Laboratory in Pasadena, California; Kennedy Space Center in Cape 
Canaveral, Florida; and Ames Research Center in Moffett Field, 
California, responsible for managing 1 contract each under our review. 

Our work was conducted from August 2005 through October 2006 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: Summary Description of the 10 NASA Contracts GAO Reviewed: 

NAS5-60000-Earth Observing System Data and Information System Core 
System-Goddard Space Flight Center: 

NAS5-60000 was an end item hardware cost-plus-award-fee contract 
between NASA and Hughes Applied Information Systems Incorporation. 
Raytheon Information Systems Company acquired Hughes in December 1999 
and became the prime contractor. The contract, currently closed, was 
managed by Goddard Space Flight Center. The 10-year research and 
development contract was awarded in March 1993 for the development and 
operation of the Earth Observing System Data and Information System 
Core System. The period of performance on the contract actually ended 
in April 2005, and the contract has since been closed. According to 
Goddard Space Flight Center procurement officials, the desired program 
outcome or objective of the contract was to develop a technically 
capable system to process data from NASA's satellites at a reasonable 
cost. Procurement officials stated that the Earth Observing System Data 
and Information System Core System, a state-of-the-art data-processing 
system, is currently dedicated to the processing and dissemination of 
NASA Earth Science satellite data. 

NAS15-10000-International Space Station-Johnson Space Center: 

NAS15-10000 is an end item hardware cost-plus-award-fee contract 
between NASA and the Boeing Company. The contract, currently active, is 
managed by the Johnson Space Center. A letter contract was awarded in 
November 1993 and was definitized in January 1995 as a cost-plus- 
incentive-fee award-fee contract. In October 1999, during a 
restructuring of the contract, the cost-plus-incentive-fee award-fee 
contract was converted to a cost-plus-award-fee contract. The contract 
was extended in December 2003, partially because of the Columbia 
accident. This planned 10-year contract is for the design, development, 
manufacture, and on-orbit assembly of the U.S. on-orbit segment of the 
International Space Station. The contract also included provisions for 
a level of effort that included (1) sustaining engineering, (2) multi- 
element integrated testing, (3) logistics and maintenance-post 
production support, (4) technical definition of contract changes, and 
(5) other engineering support. According to Johnson Space Center 
procurement officials, the desired program outcomes or objectives of 
this contract are (1) completion of the U. S. on-orbit segment, 
delivery, and on-orbit acceptance of the space station; (2) sustaining 
engineering of the U.S. on-orbit segment hardware and software and 
common hardware and software provided to international partners/ 
participants and payloads; (3) post-production support of the U.S. on- 
orbit segment hardware and common hardware provided to the 
international partners/participants; and (4) space station end-to-end 
subsystems management for the majority of the subsystems and specialty 
engineering disciplines. 

NAS5-32633-Landsat-7 Spacecraft-Goddard Space Flight Center: 

NAS5-32633 was an end item hardware cost-plus-award-fee contract 
between NASA and Lockheed Martin Missiles and Space. The contract, 
currently closed, was managed by Goddard Space Flight Center. The 
research and development contract was initially awarded by the Air 
Force in October 1992 and transferred to NASA in May 1994. The contract 
was for the design, development, fabrication, integration, test, and 
pre-and post-launch support of the Landsat-7 spacecraft. Landsat-7 was 
launched in April 1999; the contract was completed in 2005. The purpose 
of the Landsat-7 satellite is to obtain continuous remotely sensed, 
high-resolution imagery of the earth's surface for environmental 
monitoring, disaster assessment, land use and regional planning, 
cartography, range management, and oil and mineral exploration. 
According to Goddard Space Flight Center procurement officials, the 
desired program outcome or objective of the contract was to develop an 
operational satellite that met the science requirements of users and 
the laws requiring the data be obtained at a reasonable cost. 

NAS8-60000-Program Information Systems Mission Services-Marshall Space 
Flight Center: 

NAS8-60000 was a cost-plus-award-fee service contract between NASA and 
Computer Sciences Corporation. The contract, managed by the Marshall 
Space Flight Center, was in the process of being closed as of June 
2006. It was awarded in May 1994, and covered a 2-year period of 
performance, but included options to extend the period of performance 
for an additional 6 years--through April 30, 2002. The contract was 
extended three times, with the period of performance ending on March 
30, 2004. The primary purpose of the contract was to provide services 
in the area of program information system mission services. The 
contractor's responsibilities were to manage, be responsible for, and 
provide information services to meet requirements of the Information 
Systems Services Office and its customers. According to Marshall Space 
Flight Center procurement officials, the desired program outcome or 
objective of the contract was to provide services including operating 
and maintaining existing equipment and software; gathering, analyzing, 
defining, and documenting systems requirements; and planning, 
designing, developing, acquiring, integrating, testing, and 
implementing new systems or enhancements to existing systems. 

NAS2-14263-Engineering and Technical Support for Life Sciences-Ames 
Research Center: 

NAS2-14263 was a cost-plus-award-fee service contract between NASA and 
Lockheed Martin Engineering and Science Company, defined under task 
orders. The contract, managed by Ames Research Center, was in the 
process of being closed as of June 2006. Its period of performance 
ended in September 2003. The 5-year research and development contract 
was awarded in May 1995 for the provision of engineering and technical 
support services for Ames Research Center life sciences. The work to be 
performed included engineering and technical support for life sciences 
projects, including space shuttle life sciences payloads, other life 
science payloads, the Space Station Biological Research Project, ground-
based life sciences research, and advanced life support technology 
development. According to Ames Research Center procurement officials, 
the desired program outcome or objective of the contract was to achieve 
support for space life science projects, life sciences research, and 
related technology. 

NAS9-19100-Science, Engineering, Analysis, and Test-Johnson Space 
Center: 

NAS9-19100 was a cost-plus-award-fee service contract between NASA and 
Lockheed Martin with indefinite delivery, indefinite quantity task 
orders; performance-based; and level-of-effort provisions. Following 
the merger of Lockheed and Martin in 1995, NASA consolidated two 
existing contracts to form NAS9-19100 with an effective date of October 
1, 1996. The contract, managed by Johnson Space Center, was in the 
process of being closed as of June 2006. The period of performance 
ended in January 2005. The contract included requirements related to 
hardware, government-furnished crew equipment, facilities, laboratory 
maintenance, life sciences, flight hardware, and support for the 
science and engineering requirements of the Space Shuttle Program and 
the International Space Station Program. According to Johnson Space 
Center procurement officials, the desired program outcomes or 
objectives of the contract were to provide engineering and science 
support to all engineering directorates at Johnson Space Center as well 
as support both the science and engineering requirements of the shuttle 
and space station programs. 

NAS9-98100-Consolidated Space Operations Contract-Johnson Space Center: 

NAS9-98100 was a cost-plus-award-fee service contract between NASA and 
Lockheed Martin Space Operations Company, with task orders and level- 
of-effort provisions. The contract, which was in the process of being 
closed as of June 2006, was managed by the Johnson Space Center. It was 
awarded on September 25, 1998, with a basic 5-year period of 
performance and an option for an additional 5-year period. NASA chose 
not to exercise the option for the second 5-year period of performance. 
The contract required (1) developing an integrated operations approach 
to spacecraft design, operations, and data processing that minimized 
cost and the support infrastructure required to conduct space 
operations; (2) obtaining a highly capable and accountable contractor 
that would be responsible for providing space operations mission and 
data services; and (3) providing a contract and management structure 
that would enable outsourcing, commercialization, or privatization of 
some or all service under the contract. According to Johnson Space 
Center procurement officials, the desired program outcomes or 
objectives of the contract were to (1) provide excellent quality and 
reliable mission and data services at a significantly reduced cost; (2) 
move end-to-end mission and service responsibility and accountability 
to industry; (3) implement an integrated architecture that reduces 
overlap, eliminates unnecessary duplication, and reduces life cycle 
costs; (4) define streamlined processes that minimize intermediaries 
required to define requirements and deliver services; and (5) adopt 
private sector commercial practices and services. 

NAS10-99001-Joint Base Operation and Support-Kennedy Space Center: 

NAS10-99001 is a cost-plus-award-fee service contract between NASA and 
Space Gateway Support. The contract, currently active, is managed by 
Kennedy Space Center. The contract was awarded on October 1, 1998, for 
a basic 5-year period of performance and included an option for an 
additional 5 years. NASA exercised that option on October 1, 2003. The 
purpose was to provide for base operations support at NASA's Kennedy 
Space Center and the Air Force's Cape Canaveral Air Force Station, as 
well as specific requirements at Patrick Air Force Base and Florida 
Annexes into one consolidated contract. In addition to NASA and the Air 
Force, other primary customers include the Navy, Department of 
Interior, Spaceport Florida, and commercial customers such as Boeing, 
Lockheed Martin, Orbital Science, and Astrotech. According to Kennedy 
Space Center procurement officials, the desired program outcomes or 
objectives of the contract are to (1) enhance safety for the public and 
on-site workforce; (2) provide protection of human, national, and 
environmental resources; (3) provide high-quality and responsive 
service to customers; (4) reduce the cost of doing business for NASA 
and the Air Force; (5) provide flexibility to respond to new 
requirements and unplanned events; (6) improve supportability and 
reliability through innovative methodologies and concepts; (7) provide 
common support practices and systems; and (8) increase small business 
subcontracting goals. 

NAS5-01090-Mechanical System Engineering Services-Goddard Space Flight 
Center: 

NAS5-01090 is a cost-plus-award-fee service contract between NASA and 
Swales and Associates, with a line item for indefinite delivery, 
indefinite quantity task orders. The contract, currently active, is 
managed by Goddard Space Flight Center. NAS5-01090 was awarded in 
January 2001 with a period of performance of 5 years and 30 days. 
According to Goddard Space Flight Center procurement officials, the 
period of performance was extended and was currently scheduled to end 
on August 15, 2006. The purpose of the contract is to provide 
engineering services for the study, design, development, fabrication, 
integration, testing, verification, and operations of space flight and 
ground system hardware and software, including development and 
validation of new technologies to enable future science missions. 
According to Goddard Space Flight Center procurement officials, the 
desired program outcomes or objectives of the contract were to obtain 
high-quality performance, desired results, and output. 

NAS7-03001-Jet Propulsion Laboratory: 

NAS7-03001 is a cost-plus-award-fee contract between NASA and the 
California Institute of Technology, a private nonprofit educational 
institution, which establishes the relationship for the operation of 
the Jet Propulsion Laboratory (JPL) federally funded research and 
development center. The contract, currently active, is a 5-year 
research and development contract awarded in November 2002 for the 
operation and management of JPL. The contract allows for extension or 
decrease to the initial period of performance in 3-or 9-month 
increments. JPL is a NASA-owned facility as well as an operating 
division of Caltech. Caltech has operated JPL as a federally funded 
research and development center since 1959 to meet certain government 
research and development needs, which, according to the contract, could 
not be met as effectively by existing government resources or normal 
contractor relationships. The contract includes both service and 
product deliverables, which are defined in task orders issued under the 
contract. The contract encompasses a large number of discrete programs 
and projects--approximately 500 active task orders. According to NASA 
procurement officials, the desired program outcomes or objectives of 
the contract are specific performance requirements defined in task 
orders issued under the contract. The contract encompasses support of 
exploration of the solar system, including earth-based investigations, 
investigations and studies to support NASA missions in the fields of 
earth science and astrophysics and astrobiology, as well as development 
of supporting fundamental technologies. 

[End of section] 

Appendix III: Comments from the National Aeronautics and Space 
Administration: 

National Aeronautics and Space Administration: 
Office of the Administrator: 
Washington, DC 20546-0001: 

January 12, 2007: 

Ms. Ann Calvaresi-Bare: 
Director: 
Acquisition and Sourcing Management: 
United States Government Accountability Office: 
Washington, DC 20548: 

Dear Ms. Calvaresi-Barr: 

The National Aeronautics and Space Administration (NASA) appreciates 
the opportunity to comment on your draft report entitled "NASA 
Procurement: Use of Award Fees for Achieving Program Outcomes Can Be 
Improved" (GAO-07-58). The information provided in your report should 
help us make better use of award-fee contracts. 

In the draft report, GAO recommends that the NASA Administrator take 
the following three actions: 

Recommendation 1: Reemphasize to the NASA Centers the importance of 
tying award-fee criteria to desired outcomes and limiting the number of 
sub factors used in evaluations. 

Concur - NASA concurs with this recommendation. NASA's award-fee 
guidance already expresses a preference for tying award fees to 
evaluation factors that are based on outcomes as opposed to inputs or 
outputs. NASA's guidance also cautions against spreading the potential 
award fee over a large number of performance evaluation factors that 
dilute the emphasis on any particular performance evaluation criteria. 
NASA will reemphasize its guidance, address it in training, and cover 
this area in its ongoing internal reviews of Center procurement 
operations. 

Recommendation 2: To ensure that cost-plus-award-fee contracts are only 
used when their benefits outweigh their costs, direct the Centers to 
consider costs and benefits in choosing this contract type by requiring 
documentation explaining how the perceived benefits will offset the 
additional costs associated with its administration, as required by the 
Federal Acquisition Regulation (FAR). 

Concur - NASA concurs with this recommendation. NASA guidance already 
requires that the award-fee contract type only be selected when it is 
demonstrated that the costs of administration of award-fee contracts 
are offset by the perceived benefits. Although the FAR does not require 
written documentation of the cost benefit analysis, NASA will amend its 
FAR supplement to do so. NASA will reemphasize this existing 
requirement, address it in training, and cover this area in its ongoing 
internal reviews of Center procurement operations. 

Recommendation 3: Require the development of metrics for measuring the 
effectiveness of award fees, establish a system for collecting data on 
the use of award-fee contracts, and regularly examine the effectiveness 
of award fees in achieving desired acquisition outcomes. 

Concur - NASA concurs with this recommendation. Although we understood 
from the auditors that no other agency currently performs the 
recommended activities, NASA will explore the best way to develop and 
use metrics for evaluating the effectiveness of award fees, as well as 
set up a system for collecting data on the use of award-fee contracts. 
This system will allow NASA to measure whether the fees awarded are 
commensurate with the contract goals achieved. The auditors stated that 
the Department of Defense (DoD) was in the process of developing 
metrics and a system for collecting data on its award-fee contracts. 
NASA will contact DoD to obtain information on its processes and assess 
their compatibility with NASA's processes. 

Although NASA concurs with GAO's draft report recommendations, 
technical comments will be provided under separate cover. 

Thank you for the opportunity to respond to this draft report. 

Sincerely, 

Signed by: 

Shana Dale: 
Deputy Administrator: 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Ann M. Calvaresi-Barr, (202) 512-4841: 

Staff Acknowledgments: 

In addition to the individual named above, Thomas Denomme, Assistant 
Director; James Beard; Shirley Johnson; Julia Kennon; Heather Barker 
Miller; Kenneth Patton; Sylvia Schatz; and Robert Swierczek made key 
contributions to this report. 

FOOTNOTES 

[1] FAR Part 16.405-2(a)(2). 

[2] In December 2005, we issued a report on the use of award and 
incentive fees at the Department of Defense that identified many of the 
same issues as had been identified by NASA. GAO, Defense Acquisitions: 
DOD Has Paid Billions in Award and Incentive Fees Regardless of 
Acquisition Outcomes, GAO-06-66 (Washington, D.C., Dec. 19, 2005). 

[3] Fiscal year 2004 was the last year for which award-fee contract 
data were available in the Federal Procurement Data System when we 
began our work. 

[4] The sample was 10 contracts that were the top 10 dollar value award-
fee contracts in terms of obligations active in that time period. The 
estimated value of the 10 contracts totaled over $31 billion as of June 
2006, and the contracts accounted for 44 percent of obligated CPAF 
dollars for the 3-year period. Three of the contracts were for end 
items, while 7 were service contracts. 

[5] This workforce composition also includes NASA grantees. 

[6] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: 
January 2005). 

[7] NASA's field centers include Ames Research Center, Dryden Flight 
Research Center, Glenn Research Center, Goddard Space Flight Center, 
Johnson Space Center, Kennedy Space Center, Langley Research Center, 
Marshall Space Flight Center, NASA Shared Services Center, Stennis 
Space Center, and the Jet Propulsion Laboratory, which is a federally 
funded research and development center and operated by the California 
Institute of Technology. 

[8] FAR Part 16.404(b)(1) and 16.405-2(b)(1)(iii). 

[9] NASA's regulatory award-fee policy is found in the NASA FAR 
Supplement. We refer to the NASA FAR Supplement as NASA's "regulations" 
throughout this report. NASA's Award Fee Contracting Guide, dated June 
2001, states that the purpose of the guide is to explain and elaborate 
on NASA's regulatory policy, providing examples and dealing with 
practical concerns that could not be addressed in the NASA FAR 
Supplement. 

[10] GAO, NASA: Long-Term Commitment to and Investment in Space 
Exploration Program Requires More Knowledge, GAO-06-817R (Washington, 
D.C.: July 17, 2006). 

[11] These weaknesses in NASA CPAF contracting practices were cited in 
our 1994 report, GAO, NASA Procurement: Challenges Remain in 
Implementing Improvement Reforms, GAO/NSIAD-94-179 (Washington, D.C., 
Aug. 18, 1994). 

[12] GAO/NSIAD-94-179. 

[13] An exception is made for those awarded under the commercial item 
procedures of FAR Part 12. 

[14] By initial cost, schedule, and performance parameters, we mean 
those parameters agreed to at initial contract award. 

[15] In comments on our draft report, NASA officials told us that the 
number of subfactors used to evaluate the contractor's performance in 
fiscal year 2006 was 57, and the number of subfactors planned for 
fiscal year 2007 will be further reduced to 45. 

[16] The 99 percent of award fee paid on the Landsat-7 contract 
includes 6 years of on-orbit incentive fee, which is based on the 
satellite meeting the government's requirements. 

[17] Under the original contract, managed by the Air Force, the 
contractor was to assemble the satellite and its payload---the Enhanced 
Thematic Mapper Plus (ETM+) instrument. The original estimated cost of 
the contract, before it was transitioned to NASA and rebaselined, was 
$372.4 million. When the contract was transferred from the Air Force to 
NASA, the subcontract for the ETM+ instrument was split into a separate 
prime contract. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: