This is the accessible text file for GAO report number GAO-11-8 
entitled 'Defense Contracting: Enhanced Training Could Strengthen DOD’s 
Best Value Tradeoff Decisions' which was released on October 28, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 

GAO: 

October 2010: 

Defense Contracting: 

Enhanced Training Could Strengthen DOD's Best Value Tradeoff Decisions: 

Best Value Contracting: 

GAO-11-8: 

GAO Highlights: 

Highlights of GAO-11-8, a report to congressional committees. 

Why GAO Did This Study: 

The Department of Defense (DOD) obligated about $380 billion in fiscal 
year 2009 to acquire products and services. One approach DOD can take 
to evaluate offerors’ proposals is the best value tradeoff process in 
which the relative importance of price varies compared to non-cost 
factors. The National Defense Authorization Act for Fiscal Year 2010 
required GAO to review DOD’s use of the best value tradeoff process, 
specifically when non-cost factors were more important than price. In 
response, GAO determined (1) how often and for what types of contracts 
DOD used the best value tradeoff process; (2) why and how DOD used such 
an approach; and (3) challenges, if any, DOD faces in using the best 
value tradeoff process. GAO identified a probability sample of new, 
competitively awarded fiscal year 2009 contracts in which DOD obligated 
$25 million or more. GAO reviewed guidance, solicitations, source 
selection decisions, and other documents for 129 contracts and 
interviewed DOD contracting and program staff about the use of the best 
value tradeoff process. 

What GAO Found: 

In fiscal year 2009, DOD used best value processes for approximately 95 
percent of its new, competitively awarded contracts in which $25 
million or more was obligated. Almost half of DOD’s contracts—47 
percent—were awarded using a tradeoff process in which non-cost 
evaluation factors, when combined, were more important than price. DOD 
used best value tradeoffs principally to acquire services, such as 
construction of troop housing, as well as for professional management 
services. 

DOD used the best value tradeoff process in 88 of the 129 contracts GAO 
reviewed. For 60 of the 88 contracts, DOD weighted non-cost factors as 
more important than price. In these cases, DOD was willing to pay more 
for a contractor that demonstrated it understood complex technical 
issues more thoroughly, could provide a needed good or service to meet 
deadlines, or had a proven track record in successfully delivering 
products or services of a similar nature. In making tradeoff decisions, 
GAO found that DOD selected a lower priced proposal nearly as often as 
it selected a higher technically rated, but more costly proposal. 
Overall, GAO found that DOD paid a combined total of more than $230 
million in price differentials—the difference in price between the 
awardee and the offeror next in line for award—on 21 contracts, but 
chose not to pay more than $800 million in proposed costs by selecting 
a lower priced offer over a higher technically rated offer in 18 
contracts. DOD does not track whether the use of best value tradeoff 
processes correlates with the contractor successfully meeting the terms 
of the contract and noted that many factors ultimately contribute to an 
acquisition’s success or failure. 

DOD officials identified several challenges in using the best value 
tradeoff process, including the difficulty in determining meaningful 
evaluation factors and the business judgment of acquisition staff 
required. DOD officials also noted that the complexity of the tradeoff 
process increases the risk of bid protests. For example, GAO found that 
15 of the 88 contracts awarded using a best value tradeoff process 
reviewed were protested to GAO, resulting in 4 cases in which DOD 
terminated the contract or made a new source selection decision when 
DOD determined that it failed to adhere to the solicitations’ 
requirements. Such concerns are heightened given the expected influx of 
more than 6,400 new contracting personnel over the next few years. 
According to DOD officials, making sound tradeoff decisions, and in 
particular, deciding whether or not a price differential is warranted, 
is one of the most difficult aspects of using a best value tradeoff 
process. DOD is developing a new departmentwide source selection guide 
and intends to subsequently revise its training curriculum, but neither 
the guide nor DOD’s current training curriculum provides agency 
personnel with information on assessing price differentials when 
performing tradeoff analyses. 

What GAO Recommends: 

GAO recommends that to help DOD effectively employ best value tradeoff 
processes, DOD develop training elements, such as case studies, that 
focus on reaching tradeoff decisions, as it updates its training 
curriculum. DOD concurred with this recommendation. 

View [hyperlink, http://www.gao.gov/products/GAO-11-8] or key 
components. For more information, contact John Hutton at (202) 512-4841 
or huttonj@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DOD Relies Heavily on Best Value Processes to Evaluate Contractor 
Offers: 

DOD Used a Best Value Tradeoff Process to Address Complex or Time 
Sensitive Needs, but Paid Relatively Few Price Differentials: 

DOD Faces Several Challenges in Using the Best Value Tradeoff Process: 

Conclusions: 

Recommendation for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Table: 

Table 1: Estimated Percentages of Source Selection Approaches Reviewed: 

Figures: 

Figure 1: DOD Contract Obligations in Fiscal Year 2009 (Dollars in 
billions): 

Figure 2: Estimated Frequencies of Source Selection Approaches Used in 
Fiscal Year 2009 for New, Competitively Awarded DOD Contracts 
Obligating over $25 million: 

Figure 3: Estimated Frequencies of Selected Characteristics of 
Contracts or Agreements Awarded using a Best Value Tradeoff Process: 

Figure 4: Most Frequently Selected Non-cost Evaluation Factors and 
Their Relative Importance in 88 Contracts Using a Tradeoff Process: 

Figure 5: Price Differentials in the 68 DOD Contracts Reviewed in Which 
a Tradeoff Analysis Was Conducted: 

Abbreviations: 

ACE: Acquisition Center of Excellence: 

DAU: Defense Acquisition University: 

DFARS: Defense Federal Acquisition Regulation Supplement: 

DLA: Defense Logistics Agency: 

DOD: Department of Defense: 

FAR: Federal Acquisition Regulation: 

FPDS-NG: Federal Procurement Data System-Next Generation: 

IDIQ: Indefinite Delivery Indefinite Quantity: 

LPTA: Lowest Price Technically Acceptable: 

[End of section] 

United States Government Accountability Office: 

Washington, DC 20548: 

October 28, 2010: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Ike Skelton: 
Chairman: 
The Honorable Howard P. McKeon: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Department of Defense (DOD) obligated about $380 billion in fiscal 
year 2009[Footnote 1] to acquire products and services needed to 
support its missions. DOD has several approaches it can use to evaluate 
offerors' proposals. For example, DOD can use a best value tradeoff 
process, in which it can vary the relative importance of cost or price 
to other factors, such as technical approach or past performance in its 
solicitations to offerors. In these cases, DOD may award a contract to 
other than the lowest-priced offeror if DOD determines that a higher- 
priced proposal provides a greater benefit to DOD, and this greater 
benefit is worth paying an additional cost, or price differential. 
Proper execution of the tradeoff process is essential, however, to 
ensure that DOD lays the foundation for successful acquisition 
outcomes. We have identified DOD contract management as a long-standing 
high-risk area, due in part to DOD's use of ill-suited business 
arrangements that have contributed to unmet expectations and placed the 
department at risk of potentially paying more than necessary.[Footnote 
2] 

Section 845 of the National Defense Authorization Act for Fiscal Year 
2010 directed GAO to report on DOD's use of the best value tradeoff 
process, and specifically for cases in which DOD determined that it 
would evaluate contractors' proposals on factors other than cost or 
price, if these non-cost factors, when combined, were to be considered 
more important than cost or price.[Footnote 3] To respond to the 
mandate, we determined (1) how often and for what types of contracts 
DOD used the best value tradeoff process; (2) why and how DOD used the 
best value tradeoff process; and (3) what challenges, if any, DOD faces 
in using the best value tradeoff process. 

To conduct our work we used the Federal Procurement Data System-Next 
Generation (FPDS-NG)[Footnote 4] to identify new, competitively awarded 
contracts in which DOD obligated $25 million or more in fiscal year 
2009. We selected the $25 million threshold based on a Defense Federal 
Acquisition Regulation Supplement (DFARS) requirement that contracts 
for production or services with $25 million or more in estimated total 
costs for any fiscal year have written acquisition plans, which contain 
information on the anticipated source selection approach.[Footnote 5] 
This analysis identified 363 contracts on which DOD had obligated a 
total of $39.2 billion, or about 10 percent of the total amount DOD 
obligated on contracts in fiscal year 2009. 

From this population, we selected a random sample of 160 contracts, 
including 60 indefinite delivery contracts.[Footnote 6] We identified 
errors in the information provided by FPDS-NG on 31 contracts, 
including contracts that were incorrectly coded as competitively 
awarded or had incorrect award amounts.[Footnote 7] We excluded these 
contracts from our sample and determined that FPDS-NG was sufficiently 
reliable for the purposes of our review after adjusting for these 
errors. Based on our analysis of the remaining 129 sample contracts, we 
produced estimates of source selection approaches used, contract type, 
and whether a product or service was acquired. To determine the type of 
and rationale for the source selection process used for the contracts, 
we reviewed the associated acquisition plans, solicitations, source 
selection decision memorandums, and other relevant documents for each 
of the contracts in our sample. In particular, we reviewed source 
selection decision documents for contracts that used a tradeoff process 
to determine whether DOD paid a price differential. For the purposes of 
this report, we defined a price differential as the difference in price 
between the awardee's price and the price of the offeror next in line 
for award. 

From our sample, we also judgmentally selected buying activities from 
each military department and one defense agency based on such factors 
as the number of best value contracts, the type of contract, and the 
type of product or service acquired. Results based on these selected 
buying activities are not generalizable to a larger population. At each 
activity, we reviewed contract files and interviewed program and 
contracting officials to discuss their rationale for selecting a 
tradeoff process including the selection of non-cost evaluation factors 
for 34 contracts and 23 task orders as illustrative case studies. We 
also reviewed contract documents, DOD and military department source 
selection guidance, and interviewed program and contracting officials 
from DOD, the military departments, and one defense agency to identify 
what challenges, if any, DOD faces in using the best value tradeoff 
process. A more detailed description of our scope and methodology is 
included in appendix I. 

We conducted this performance audit from March 2010 through October 
2010 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Background: 

The Federal Acquisition Regulation (FAR) Part 15 allows the use of 
several best value competitive source selection techniques to meet 
agency needs. Within the best value continuum, DOD may choose an 
approach that it considers the most advantageous to the government, 
including the lowest price technically acceptable (LPTA) process and 
the tradeoff process. 

DOD may elect to use the LPTA process in acquisitions where the 
requirement is clearly definable and the risk of unsuccessful contract 
performance is minimal. In such cases, DOD may determine that cost or 
price should play a dominant role in source selection. When using the 
LPTA process, DOD specifies its minimal technical requirements in the 
solicitation. Once DOD determines that the contractors meet or exceed 
the technical requirements, no tradeoffs between cost or price and non- 
cost factors are permitted and the award is made based on the lowest 
price offered to the government. 

By contrast, DOD may elect to use a tradeoff process in acquisitions 
where the requirement is less definitive, more development work is 
required, or the acquisition has greater performance risk. In these 
instances, non-cost evaluation factors, such as technical capabilities 
or past performance, may play a dominant role in the source selection 
and tradeoffs among price and non-cost factors allow DOD to accept 
other than the lowest priced proposal. This report focuses on DOD's use 
of the tradeoff process, and specifically, in which non-cost factors, 
when combined, were considered more important than cost or price. 

When using a tradeoff process, the FAR requires that evaluation factors 
and significant subfactors that affect contract award and their 
relative importance be clearly stated in the solicitation; and the 
solicitation must provide whether all evaluation factors other than 
cost or price, when combined, are significantly more important than, 
approximately equal to, or significantly less important than cost or 
price[Footnote 8]. Additionally, the FAR requires that each factor 
represent key areas of importance and emphasis to be considered in the 
source selection decision and that support meaningful comparison and 
discrimination between and among competing proposals[Footnote 9]. The 
FAR also requires the source selection authority document the perceived 
benefits of the higher priced proposal and the rationale for tradeoffs 
in the contract file[Footnote 10]. The resulting source selection 
decision should be based on a comparative assessment of proposals 
against all source selection criteria in the solicitation. The decision 
must also include the rationale for any business judgments and 
tradeoffs made or relied on by the source selection official, including 
benefits associated with additional costs. Although the rationale for 
the source selection decision must be documented, the documentation 
need not quantify the tradeoffs that led to the decision. 

In fiscal year 2009, DOD obligated about $380 billion on contracts for 
goods and services. Our analysis of data reported by DOD to FPDS-NG 
indicates that $69.9 billion or 18 percent of DOD's obligations were 
made on new contracts competitively awarded in fiscal year 2009 (see 
figure 1). By contrast, about $176 billion were modifications to or 
orders issued under contracts that were awarded prior to fiscal year 
2009 and $133 billion were awarded non-competitively,[Footnote 11] 
which in combination totaled to nearly 82 percent of DOD's reported 
contract obligations in fiscal year 2009. 

Figure 1: DOD Contract Obligations in Fiscal Year 2009[A] (Dollars in 
billions): 

[Refer to PDF for image: pie graph and bar graph] 

Pie graph: 

Non-competed contracts[B]: $133.2; 
Completed Contracts: $264.4. 

Bar graph: 

New contracts: $39.2; 
More than $25 million obligated. 

New contracts: $30.7; 
Less than $25 million obligated. 

Orders and modifications on contracts awarded prior to FY 2009: $125.1; 
More than $25 million obligated. 

Orders and modifications on contracts awarded prior to FY 2009: $51.3; 
Less than $25 million obligated. 

Source: GAO analysis of FPDS-NG obligations data. 

[A] Dollar amounts may not sum due to rounding. 

[B] The non-competed contracts category consists of those contracts 
awarded under other than full and open competition as defined in FAR 
subpart 6.3. 

[End of figure] 

Properly managing the acquisition of goods and services requires an 
acquisition workforce with the right skills and capabilities. In March 
2009, however, we reported that DOD lacked complete information on the 
skill sets of the current acquisition workforce and whether these skill 
sets were sufficient to accomplish its missions.[Footnote 12] In April 
2009, the Secretary of Defense announced his intent to grow the 
acquisition workforce by 15 percent by fiscal year 2015. As part of 
this strategy, DOD indicated that it intends to grow its contracting 
career field by more than 6,400 personnel, an increase of more than 28 
percent from fiscal year 2008 staffing levels. 

DOD Relies Heavily on Best Value Processes to Evaluate Contractor 
Offers: 

DOD relies heavily on the use of the best value process to evaluate 
offers from potential contractors. DOD chose a best value process for 
approximately 95 percent of its new, competitively awarded contracts on 
which it had obligated $25 million or more in fiscal year 
2009.[Footnote 13] Almost half of DOD's contracts--47 percent--were 
awarded using a tradeoff process in which non-cost evaluation factors, 
when combined, were more important than price. Figure 2 shows how often 
DOD used the different best value processes and other source selection 
approaches. 

Figure 2: Estimated Frequencies of Source Selection Approaches Used in 
Fiscal Year 2009 for New, Competitively Awarded DOD Contracts 
Obligating over $25 million[A]: 

[Refer to PDF for image: pie graph] 

Best value processes (95%): Lowest price technically acceptable: 26%; 
Best value tradeoff prices (69%): Non-cost factors less important than 
price: 2%; 
Best value tradeoff prices (69%): Non-cost factors equal to price: 20%; 
Best value tradeoff prices (69%): Non-cost factors more important than 
price: 47%; 
Sealed bid[B]: 5%. 

Source: GAO analysis of DOD contract documents. 

[A] The 95 percent confidence intervals for estimates in this graph are 
within +/-8 percentage points of the estimates themselves. 

[B] In sealed bidding, award is made to the responsible bidder whose 
bid conforms to the invitation for bid and is the most advantageous for 
the government considering only price and price-related factors 
included in the invitation. 

[End of figure] 

In 69 percent of the contract awards, DOD used the best value tradeoff 
process. When doing so, it acquired services approximately four times 
as often as it acquired products. Over half of these procurements were 
for building or civil engineering construction services, including 
projects for troop housing, administrative facilities, and hurricane 
protection systems. Other services procured using the tradeoff process 
were equipment maintenance and professional management services. For 
example, in fiscal year 2009, the Army Corps of Engineers awarded a 
contract worth more than $963 million to construct one of the largest 
pumping stations in the world, along with floodgates and floodwalls for 
hurricane protection. Similarly, the Air Force awarded the Contract 
Field Team program multiple-award contract, with an estimated base 
value of $2.6 billion for modification, maintenance and repair of 
systems including aircraft and missile defense for the departments of 
the Army, Navy, Air Force and several federal agencies. Small arms and 
electronic countermeasure equipment were among the products most 
frequently procured using a tradeoff process, including the contracts 
for the Squad Automatic Weapons--lightweight, automatic rifles issued 
to each Army and Marine rifle squad--and an Army contract to procure 
devices that counteract radio-controlled improvised explosives. Our 
analysis of selected characteristics of contracts awarded using a best 
value tradeoff process in fiscal year 2009 is shown in figure 3. 

Figure 3: Estimated Frequencies of Selected Characteristics of 
Contracts or Agreements Awarded using a Best Value Tradeoff Process[A]: 

[Refer to PDF for image: circular chart] 

Contract pricing[B]; 
Firm fixed price: 78%; 
Cost reimbursement: 2%; 
Other fixed price: 2%; 
Combination: 17%[C]. 

Contract structure; 
Indefinite delivery: 31%; 
Stand-alone[D]: 66%; 
Other[E]: 3%. 

Contract purpose; 
Product: 17%; 
Product/Service: 5%; 
Service: 78. 

Source: GAO analysis of DOD contract documents and data obtained from 
FPDS-NG. 

[A] Note: The 95 percent confidence intervals for estimates in this 
graph are within +/-8 percentage points of the estimates themselves. 

[B] Percentages may not sum to 100 percent due to rounding. 

[C] Combination refers to contracts that allow for orders to be placed 
using more than one pricing arrangement. 

[D] Stand-alone refers to contracts that do not allow for individual 
orders to be placed against the contract. 

[E] Other includes basic ordering agreements and blanket purchase 
agreements. 

[End of figure] 

As part of our work, we reviewed 10 IDIQ contracts that had been 
awarded using a best value tradeoff process and 23 task and delivery 
orders under these contracts which had obligations ranging from $11 
million to over $319 million. In most cases, DOD did not issue the task 
or delivery orders we reviewed using a tradeoff process. For example, 
an Air Force official explained that the initial task orders for the 
Contract Field Team program, including 13 orders in our sample, were 
issued under an LPTA process because data needed to assess contractor 
performance and timeliness were not yet available to use given the 
short time between award of the base contract and issuance of the first 
orders. An Air Force official indicated that once they had obtained 
sufficient performance data, they intended to issue task orders using a 
tradeoff process when possible. DOD officials issued six other task 
orders on the basis of negotiating with a contractor who had been 
awarded a single award IDIQ contract. The four remaining orders were 
awarded using a tradeoff process. For example, the Army wanted infrared 
vision enhancement equipment for nighttime and battlefield use in Iraq 
and Afghanistan to be delivered as quickly as possible. Consequently, 
the Army used a contractor's ability to meet delivery requirements as 
the principal evaluation factor in selecting the contractor for 
delivery order award. 

Some DOD officials noted that the use of various source selection 
evaluation methods can change over time. For example: 

* Defense Logistics Agency (DLA) officials noted that they have 
recently transitioned from principally using the tradeoff process to 
using the LPTA process for most fuel purchases because the majority of 
their procurements were for a commercial product in relatively stable 
domestic and international markets. They noted, however, that they 
still use the tradeoff process in less stable areas, such as Iraq and 
Afghanistan, where they require more information about vendors' past 
performance and technical capability when operating in war zones. 

* Conversely, Army Corps officials in New Orleans reported that they 
have been using the tradeoff process more frequently since the increase 
in civil works construction projects following Hurricane Katrina. While 
they typically used sealed bids in the past, they told us that use of 
the tradeoff process enabled them to better assess contractors' ability 
to meet safety and schedule requirements. 

DOD Used a Best Value Tradeoff Process to Address Complex or Time 
Sensitive Needs, but Paid Relatively Few Price Differentials: 

DOD officials tended to use a best value tradeoff process with non-cost 
factors weighted more important than price when they were willing to 
accept a higher price if a contractor could demonstrate certain 
advantages, such as meet a deadline, demonstrate that it understood 
complex technical issues, or propose an innovative approach. DOD often 
indicated in tradeoff solicitations that non-cost factors would be 
significantly more important than price in making award decisions, but 
our analysis indicated that DOD selected a lower priced proposal among 
those offerors remaining in the final competition almost as often as it 
selected a higher technically rated, but more costly, proposal. 
Overall, DOD paid a price differential--the difference in the price of 
the offeror awarded the contract and the price of the offeror next in 
line for award--in 21 of the 68 contracts in which a price differential 
was considered. Most differentials were less than 5 percent. While DOD 
officials told us that the tradeoff process provides an essential tool 
to obtain desired capabilities, they rely on the case-by-case judgment 
of contracting and program officials to determine the best acquisition 
approach suited to program requirements and do not specifically track 
whether use of the tradeoff process is in DOD's interest. 

Contract Requirements Drove DOD's Selection of Technical and Past 
Performance Evaluation Factors: 

The FAR and DOD guidance generally provide acquisition staff 
flexibility to develop evaluation factors that meet their procurement 
needs and does not indicate which evaluation factors should be most 
important. The FAR requires that DOD officials consider, among other 
things, past performance on all negotiated competitive acquisitions 
exceeding $100,000, but DOD officials have broad discretion in 
selecting other non-cost factors and their relative importance. The 
factors are intended to provide meaningful discriminators to evaluate 
proposals. Army, Navy, and Air Force officials told us that they formed 
interdisciplinary teams that developed evaluation factors and the 
factors' relative importance by consensus. 

We found that 88 of the 129 contracts we reviewed used a best value 
tradeoff process. Our analysis shows that DOD considered past 
performance and technical evaluation factors as the most important 
among the non-cost factors. Figure 4 shows the five most frequently 
used non-cost evaluation factors for the 88 contracts in our review in 
which a tradeoff was conducted and how often the technical and past 
performance factors were most important among the non-cost factors. 

Figure 4: Most Frequently Selected Non-cost Evaluation Factors and 
Their Relative Importance in 88 Contracts Using a Tradeoff Process: 

[Refer to PDF for image: chart] 

Army; 
Past performance: Occurrence of non-cost factor: 19; 
Past performance: Occurrence of past performance or technical factor as 
most important: 25; 
Technical: Occurrence of non-cost factor: 5; 
Technical: Occurrence of past performance or technical factor as most 
important: 36; 
Small business: Occurrence of non-cost factor: 33; 
Small business: Occurrence of past performance or technical factor as 
most important: 0; 
Experience: Occurrence of non-cost factor: 26; 
Experience: Occurrence of past performance or technical factor as most 
important: 0; 
Management: Occurrence of non-cost factor: 27; 
: Occurrence of past performance or technical factor as most important: 
0. 


Navy; 
Past performance: Occurrence of non-cost factor: 8; 
Past performance: Occurrence of past performance or technical factor as 
most important: 13; 
Technical: Occurrence of non-cost factor: 0; 
Technical: Occurrence of past performance or technical factor as most 
important: 14; 
Small business: Occurrence of non-cost factor: 13; 
Small business: Occurrence of past performance or technical factor as 
most important: 0; 
Experience: Occurrence of non-cost factor: 15; 
Experience: Occurrence of past performance or technical factor as most 
important: 0; 
Management: Occurrence of non-cost factor: 9; 
Management: Occurrence of past performance or technical factor as most 
important: 0.

Air Force; 
Past performance: Occurrence of non-cost factor: 0; 
Past performance: Occurrence of past performance or technical factor as 
most important: 5; 
Technical: Occurrence of non-cost factor: 0; 
Technical: Occurrence of past performance or technical factor as most 
important: 0; 
Small business: Occurrence of non-cost factor: 3; 
Small business: Occurrence of past performance or technical factor as 
most important: 0; 
Experience: Occurrence of non-cost factor: 0; 
Experience: Occurrence of past performance or technical factor as most 
important: 0; 
Management: Occurrence of non-cost factor: 3; 
Management: Occurrence of past performance or technical factor as most 
important: 0. 

DLA; 
Past performance: Occurrence of non-cost factor: 0; 
Past performance: Occurrence of past performance or technical factor as 
most important: 1; 
Technical: Occurrence of non-cost factor: 0; 
Technical: Occurrence of past performance or technical factor as most 
important: 1; 
Small business: Occurrence of non-cost factor: 0; 
Small business: Occurrence of past performance or technical factor as 
most important: 0; 
Experience: Occurrence of non-cost factor: 0; 
Experience: Occurrence of past performance or technical factor as most 
important: 0; 
Management: Occurrence of non-cost factor: 0; 
Management: Occurrence of past performance or technical factor as most 
important: 0. 

Other; 
Past performance: Occurrence of non-cost factor: 1; 
Past performance: Occurrence of past performance or technical factor as 
most important: 0; 
Technical: Occurrence of non-cost factor: 0; 
Technical: Occurrence of past performance or technical factor as most 
important: 1; 
Small business: Occurrence of non-cost factor: 0; 
Small business: Occurrence of past performance or technical factor as 
most important: 0; 
Experience: Occurrence of non-cost factor: 0; 
Experience: Occurrence of past performance or technical factor as most 
important: 0; 
Management: Occurrence of non-cost factor: 0; 
Management: Occurrence of past performance or technical factor as most 
important: 0. 

Percentage of contracts using this factor: Past performance: 82%; 
Percentage of contracts using this factor: Technical: 65%; 
Percentage of contracts using this factor: Small business: 56%; 
Percentage of contracts using this factor: Experience: 47%; 
Percentage of contracts using this factor: Management: 44%. 

Source: GAO analysis of DOD contract documents. 

Note: For the purposes of this report, in cases where multiple non-cost 
factors were considered to be of equal importance, GAO considered all 
of these factors to be most important. The small business factors 
typically consisted of evaluating the offerors' proposed use of small 
businesses. 

[End of figure] 

DOD officials told us that the selection of these evaluation factors 
and their relative importance was based on specific acquisition 
requirements, such as the ability to meet production deadlines, ensure 
compatibility with existing ship and aircraft systems, or provide 
needed security for delivery of goods in war zones. Our analysis found 
that DOD considered non-cost factors more important than price in 60 of 
the 88 contracts awarded using a tradeoff process. The following 
illustrate instances where DOD's acquisition needs led them to make non-
cost factors the principle criteria for source selection. 

* Army officials had to quickly meet surge requirements based on a 
Joint Urgent Operational Needs Statement[Footnote 14] for roadside bomb 
detectors as well as services to provide training and support the 
system once fielded, and accordingly, made technical capability the 
most important evaluation factor. The acquisition plan specified that 
deliveries of the critical technology and support services needed to be 
made within 6 months of contract award. 

* Army officials sought contractors with innovative approaches and a 
superior understanding on how to counter the threat of roadside bombs 
in awarding a professional services contract for a range of training 
programs to be used within the Military Service Combat Training 
Centers. According to the Army, the selected contractor provided a 
proposal that was superior in nearly all the technical categories 
sought by the Army. 

* The Navy considered contractors' proposed technical approach the most 
important evaluation factor for a helicopter upgrade kit procurement 
because the design had to be compatible with existing helicopters. In 
this case, the timing of fleet deployment was also critical and the 
Navy sought a contractor that could meet their schedule. 

* DLA used a tradeoff process primarily for commercial fuel contracts 
in dangerous areas, such as Iraq and Afghanistan, due to the heightened 
need for contractor reliability in these war zones. In these 
situations, DLA officials explained the tradeoff process allowed them 
to emphasize security and past performance in their evaluations to 
mitigate acquisition risks, especially since they do not know the 
vendors well. 

* Army Corps of Engineers officials that needed to procure construction 
services for barracks for wounded soldiers made the technical and 
performance capability factors most important because they needed to be 
responsive to new schedule and price targets. These officials used the 
tradeoff process to incentivize timeliness and price reductions, and 
they were also able to obtain better features, such as more durable 
materials. 

DOD officials also told us they used these non-cost factors to 
encourage contractors to provide innovative solutions to meet DOD's 
needs. For example, Army officials expressed a need for technological 
innovation in a solicitation for equipment, field support services, and 
associated maintenance needed to intercept enemy communications. The 
Army encouraged the contractors to develop a system that would enable 
them to upgrade the equipment frequently over the life of the contract. 
The statement of work clarified that these upgrades would be essential 
to maintain relevancy in the battlefield and keep pace with technology 
advancements. Similarly, Marine Corps officials we spoke to about an 
urban warfare training system told us that they used the tradeoff 
process to seek innovative designs when awarding a 5-year, $1 billion 
dollar contract. Marine Corps officials indicated that they had a 
system that worked, but wanted to push industry to come up with a 
solution that allowed the Marines to reconfigure building structures 
more quickly and to provide more realistic and current combat scenarios 
prior to deployment. In the winning design, the offeror proposed using 
modular building sets that Marines could assemble more quickly to 
maximize the training opportunities available in the field. 

In contrast, our analysis found that the 28 cases in which DOD 
officials considered non-cost factors as equal to or less important 
than price were nearly all related to construction projects. For 
example, in 15 cases we reviewed, the Army Corps of Engineers 
considered non-cost factors such as management and technical, 
experience, and past performance as equal to price to address less 
complex project requirements such as building a new runway for aircraft 
and constructing a maintenance facility. In these instances, the 
contracting and program officials were able to request and review 
information from potential contractors and conduct a tradeoff process 
that would not be available through an LPTA approach, but still 
considered price of equal importance to non-cost factors in the award 
decision. 

DOD Paid Relatively Few Price Differentials Despite Best Value Tradeoff 
Solicitations That Emphasized Non-Cost Factors: 

For the 88 contracts awarded using a best value tradeoff process, DOD 
considered whether to pay a price differential in 68 
contracts.[Footnote 15] Our analysis indicated that DOD selected the 
lower priced option nearly as often as it selected the highest rated, 
but more costly, proposal. In the 18 cases in which DOD officials 
decided not to pay a price differential, they determined that the lower 
price outweighed the advantages of the offeror with the higher 
technical rating. In doing so, DOD officials decided not to pay over 
$800 million in price differentials. In 29 other cases, DOD awarded 
contracts to the offerors that had both the lowest price and the 
highest non-cost factor rating. 

DOD accepted a higher price in 21 of the 68 contracts in which a price 
differential was considered, for a combined difference of more than 
$230 million. Most differentials paid were less than 5 percent above 
the price submitted by the offeror next in line for award. The largest 
price differential from the contracts in our sample was 48 percent 
higher, or roughly $13.6 million more, than the next in line offeror's 
price. In this case, Marine Corps officials determined that the 
product--burn resistant clothing for use by soldiers in Iraq--was worth 
the price difference because it provided substantially greater 2nd and 
3rd degree burn protection than the product proposed by the other 
offeror. Figure 5 shows the frequency with which DOD elected to pay or 
not pay a price differential for the 68 contracts in which a price 
differential was considered, as well as the value of the price 
differentials either paid or not paid. 

Figure 5: Price Differentials in the 68 DOD Contracts Reviewed in Which 
a Tradeoff Analysis Was Conducted: 

[Refer to PDF for image: bar graph] 

Contracts with a price differential paid: 21; 
Contracts with no price differential paid: Proposals selected had 
lowest price and highest non-cost ratings: 29; 
Contracts with no price differential paid: Lower priced proposals 
selected outweighed advantages of higher non-cost ratings: 18; 
Total: 68. 

Contracts with a price differential paid: 21; 
Value of price differentials as a percentage of the contract value: 
Less than or equal to 5%: 12; 
Value of price differentials as a percentage of the contract value: 5% 
to 20%: 7; 
Value of price differentials as a percentage of the contract value: 
Greater than 20%: 2. 

Contracts with no price differential paid: 18; 
Value of price differentials as a percentage of the contract value: 
Less than or equal to 5%: 1; 
Value of price differentials as a percentage of the contract value: 5% 
to 20%: 6; 
Value of price differentials as a percentage of the contract value: 
Greater than 20%: 11. 

Source: GAO analysis of DOD contract documents. 

[End of figure] 

DOD Relies on the Judgment of Contracting and Program Officials to 
Determine Whether Use of Best Value Tradeoffs Meet DOD's Needs: 

DOD contracting and program officials believed that the use of best 
value tradeoffs provide DOD an essential tool, which allows them to 
obtain better insights into the contractors' capabilities and their 
understanding of the government's needs, and the reasonableness of the 
contractor's approach. DOD and military department officials stated 
that they do not specifically track use of the tradeoff process to 
determine if DOD's interests are met. Instead, they rely on the 
judgment of contracting and program officials to select the best 
acquisition approach suited to program requirements on a case by case 
basis. Further, DOD officials stated that they do not track whether the 
solicitation approach used correlated with whether the contractor 
successfully met the terms of the contract and noted that many factors 
ultimately contribute to the success or failure of an individual 
acquisition that may not have been foreseeable when awarding the 
contract. For example, DOD officials noted that DOD would often use a 
best value tradeoff process to award a contract to develop a major 
weapons system. As our work has found, DOD often encounters cost 
increases, schedule delays, and performance shortfalls on its major 
systems.[Footnote 16] 

DOD Faces Several Challenges in Using the Best Value Tradeoff Process: 

DOD officials acknowledged several challenges in using the best value 
tradeoff process such as the difficulties in developing meaningful 
evaluation factors, the additional time investment needed to conduct 
best value procurements, and the business judgment required of 
acquisition staff when compared to other acquisition approaches. DOD 
officials also noted that the complexity of the tradeoff process 
increases the risk of bid protests. To help address source selection 
challenges, DOD is drafting a source selection guide to improve 
consistency and standardize source selection procedures for 
competitively awarded negotiated procurements. 

DOD officials told us that developing non-cost factors that 
meaningfully discriminate between offers is a challenging part of the 
tradeoff process. They noted that as the complexity of the acquisition 
increases, so does the need for individuals with the expertise to help 
develop the evaluation factors. For example, Army Corps of Engineers 
officials told us that the contract for one of the world's largest 
flood pump stations required experts with experience in issues ranging 
from water flow management to real estate to develop evaluation 
factors. Further, Navy officials explained that while they often use 
past performance as a non-cost discriminator, it can be difficult to 
identify differences between contractor proposals because contractors 
often provide their best performance examples and the government often 
lacks data to evaluate additional contractor projects. Our past work 
has also identified governmentwide challenges in obtaining needed past 
performance information to support contract award decisions.[Footnote 
17] Further, the absence of meaningful non-cost discriminators can 
result in offerors receiving equal scores on the factors that were 
identified as being significantly more important than price. As such, 
the decision may default simply to a consideration of price alone. For 
example, Air Force officials noted that they are considering updating 
factors used to award task orders under the Contract Field Team 
contract because contractors tend to receive the highest ratings for 
each non-cost factor reviewed, so price is typically the only 
discriminator. 

DOD officials also noted that using the best value tradeoff process is 
often far more time-consuming than other approaches. Navy officials 
told us that the tradeoff process is administratively burdensome and 
requires a large time investment from program staff, which can make it 
challenging to keep the same acquisition team together for an entire 
procurement. During our site visits, many contract and program staff 
told us that the tradeoff process often takes between 18 and 24 months. 
In addition, in Afghanistan and Iraq, the challenges of conducting a 
tradeoff process have contributed to decisions by the CENTCOM Joint 
Theater Support Contracting Command and Army Corps of Engineers to 
discourage its use. For example, recent Army Corps of Engineer projects 
in Afghanistan have emphasized using simpler, less complex designs or 
requirements that are more suitable for the use of a lowest price 
technically acceptable approach. 

The complex nature of the best value tradeoff process, including 
decisions on whether to pay a price differential, requires much greater 
business judgment when compared to other acquisition approaches. DOD 
officials stated that making tradeoff decisions, particularly when to 
pay a price differential, is among the most difficult aspects of the 
tradeoff process, which will become more challenging with less 
experienced staff coming into the acquisition workforce. DOD officials 
indicated that DOD intends to increase the size of its contracting 
career field by more than 6,400 personnel through fiscal year 2015. 
With the influx of new staff, many of the contracting officers we met 
with noted challenges in preparing staff to conduct the tradeoff 
process. For example, a Navy contracting officer told us that guidance 
and training only go so far to prepare acquisition staff to conduct 
best value tradeoff procurements. Instead, acquisition staff need to be 
involved in a number of best value tradeoff procurements to develop the 
business judgment necessary to conduct a successful acquisition. 

DOD officials stated that the complexity of the tradeoff process also 
increases the risk of bid protests. Of the 88 contracts we reviewed 
that used a tradeoff process, 15 were the subject of a bid protest to 
GAO. While most of the protests were denied, DOD took corrective 
actions in 5 cases, including 4 cases in which DOD terminated the 
contract or made a new source selection decision when it determined 
that it failed to adhere to the solicitations' requirements. 

Some of the services have developed initiatives to address these 
challenges. For example, the Air Force set up an Acquisition Center of 
Excellence (ACE) at Tinker Air Force Base, which provides pre-award 
source selection assistance to contract and program staff. Air Force 
officials stated that ACE reviews the evaluation factors within 
individual source selection plans, serves in an advisory capacity on 
source selection teams and holds workshops for contracting officers. 
Similarly, Army officials at Ft. Monmouth's Communications-- 
Electronics Command have developed an online business tool--the ASSIST 
tool--that shepherds contracting officers through the solicitation 
process. For example, the tool provided a list of steps that must be 
completed for best value tradeoff procurements and automatically routes 
documents through source selection evaluation boards and other 
participating officials for review, as required. 

DOD is also drafting a departmentwide source selection guide to improve 
consistency and standardize source selection procedures for 
competitively awarded negotiated procurements. Given the influx of new 
acquisition staff, DOD officials stated they wanted to develop a more 
prescriptive guide for best value procurements. While the DOD draft 
source selection guidance contains information on various aspects of 
the best value process, such as the source selection decision document, 
DOD officials told us it does not address price differentials. Numerous 
DOD officials underscored the importance of training in the use of the 
best value process, particularly training that addresses the tradeoff 
decision that acquisition staff must make. For example, one Army Corps 
of Engineers official told us that source selection officials would 
benefit by training that contains real life lessons on how other 
officials have made price differential decisions during the tradeoff 
process. Similarly, Marine Corps and Army officials told us that while 
decisions are made on a case-by-case basis, informal rules of thumb 
regarding price differentials can come into play and indicated that 
additional guidance or training, especially case studies or scenarios, 
would be helpful. 

The Defense Acquisition University (DAU) is responsible for providing 
training to the DOD acquisition workforce. According to DAU officials, 
they offer more than 10 courses that contain elements of the best value 
tradeoff process, but none of the current courses provide case studies 
or scenarios that focus on reaching price differential decisions during 
source selection. They noted that once the new source selection 
guidance is implemented, which is anticipated for January 2011, they 
plan to augment existing contracting courses to reflect the new 
guidance. 

Conclusions: 

The best value tradeoff process underlies the vast majority of DOD 
competitively awarded contracts, and effective use of this process 
hinges on making sound tradeoffs between price and non-cost factors. By 
focusing on non-cost factors, DOD anticipates that it will obtain 
technical solutions that are innovative and address complex and time- 
sensitive program requirements. Applying a tradeoff process, however, 
does not guarantee successful acquisitions, nor is it without other 
challenges. In particular, using a tradeoff process can be more complex 
and take more time than other source selection methods, and requires 
that acquisition staff have proper guidance, needed skills, and sound 
business judgment. With the anticipated influx of more than 6,400 DOD 
contracting personnel over the next few years, providing a firm 
foundation for use of the tradeoff process is essential. While DOD and 
the military departments have taken steps to improve source selection 
procedures, acquisition personnel noted a lack of training to assist 
them in deciding whether or not a price differential is warranted when 
making tradeoff decisions. For example, while DOD's new source 
selection guide provides insights on the source selection process, it 
is silent on how to reach decisions on when to pay a price 
differential, as is DOD's current training curriculum. DOD has an 
opportunity as it updates its training curriculum to provide 
acquisition staff with better insights using real life examples on 
reaching tradeoff decisions. Taking this step can help DOD minimize the 
risk of paying a price differential when not warranted or losing the 
benefit of a technically superior solution. 

Recommendation for Executive Action: 

To help DOD effectively employ the best value tradeoff process, we 
recommend that the Secretary of Defense direct the Director of Defense 
Procurement and Acquisition Policy to work with the Defense Acquisition 
University to develop training elements, such as case studies or 
scenarios that focus on reaching tradeoff decisions, including 
consideration of price differentials, as it updates the source 
selection curriculum. 

Agency Comments and Our Evaluation: 

DOD provided written comments on a draft of this report. DOD concurred 
with our recommendation and intends to request the Panel on Contracting 
Integrity--comprised of senior DOD leaders tasked, in part, to help 
improve DOD's performance--to assist the Defense Acquisition University 
in developing training case studies and scenarios that focus on 
reaching tradeoff decisions. DOD's letter is reprinted in appendix II. 

We are sending copies of this report to interested congressional 
committees and the Secretary of Defense. In addition, this report will 
be available at no charge on GAO's Web site at [hyperlink, 
http://www.gao.gov]. 

Should you or your staff have any questions on the matters covered in 
this report, please contact me at (202) 512-4841 or huttonj@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in appendix III. 

Signed by: 

John P. Hutton, Director: 
Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Scope and Methodology: 

Section 845 of the National Defense Authorization Act for Fiscal Year 
2010 directed GAO to report on the Department of Defense's (DOD) use of 
the best value tradeoff process, and specifically for cases in which 
DOD evaluated contractors' proposals on factors other than cost or 
price, if these non-cost factors, when combined, were considered more 
important than cost or price.[Footnote 18] To respond to the mandate, 
we determined (1) how often and for what types of contracts DOD used 
the best value tradeoff process; (2) why and how DOD used the best 
value tradeoff process; and (3) what challenges, if any, DOD faces in 
using the best value tradeoff process. 

To determine how often and for what types of contracts DOD used the 
best value tradeoff process, we used data from the Federal Procurement 
Data System-Next Generation (FPDS-NG) as of January 2010 to identify a 
population of contracts based on the following criteria: (1) newly 
awarded by DOD in fiscal year 2009; (2) competitively awarded, and (3) 
had obligations of $25 million or more in fiscal year 2009. 

We established the $25 million threshold because the Defense Federal 
Acquisition Regulation Supplement (DFARS) requires contracts with total 
estimated costs of $25 million or more in any fiscal year to prepare 
written acquisition plans, which contain information about the source 
selection approach. This analysis identified 363 contracts. From this 
population, we selected a probability sample of 160 contracts, 
including 60 indefinite delivery contracts, and reviewed associated 
solicitations, source selection decision documents, and other contract 
documents to determine the solicitation approach DOD used. 

We verified the obligations and contract award fields in FPDS-NG with 
contract data to ensure that the contracts within our sample were 
within scope. Thirty-one contracts from our initial sample of 160 
contracts were outside the scope of our review because they were 
incorrectly coded in key parameters, such as being coded as 
competitively awarded when they were not or had misreported the amount 
of obligations made on the contract or task order. We excluded these 
contracts from our sample and determined that FPDS-NG was sufficiently 
reliable for the purposes of our review after adjusting for these 
errors. 

Because we followed a probability procedure based on random selections, 
our sample is only one of a large number of samples that we might have 
drawn. Since each sample could have provided different estimates, we 
express our confidence in the precision of our particular sample's 
results as a 95 percent confidence interval (e.g., plus or minus 8 
percentage points). This is the interval that would contain the actual 
population value for 95 percent of the samples we could have drawn. 
Unless otherwise noted, percentage estimates based on our sample have 
95 percent confidence intervals that are within plus or minus 8 
percentage points of the estimate itself. Confidence intervals for 
other numeric estimates are reported along with the estimate itself. 
Table 1 summarizes the estimated percentage of contracts of various 
source selection approaches reviewed. 

Table 1: Estimated Percentages of Source Selection Approaches Reviewed: 

Approach: Best value: Tradeoff process; 
Estimated percent representation: 55%. 

Approach: Best value: Lowest price technically acceptable; 
Estimated percent representation: 21%. 

Approach: Best value: Sealed bid; 
Estimated percent representation: 4%. 

Approach: Best value: Erroneously reported as being within our scope of 
review; 
Estimated percent representation: 19%. 

Source: GAO analysis of DOD contract information. 

Note: Percentages do not sum to 100 percent due to rounding. 

[End of table] 

Based on our analysis of the remaining 129 sample contracts, we 
estimate that the total number of best value tradeoff, lowest price 
technically acceptable, or sealed bid award decisions (in-scope 
contracts) in the full population of interest was about 293.[Footnote 
19] For contracts that utilized a best value tradeoff process, we 
categorized them based on the relative importance placed on price. In 
addition, we determined contract type, the type of procurement (product 
versus services), and the type of product or service for our sample 
contracts using FPDS-NG data and verified this information with the 
contract documents. 

To determine why and how DOD used the best value tradeoff process and, 
in particular, when non-cost factors were considered more important 
than price, we obtained and reviewed DOD and service level acquisition 
guidance related to source selection policies and procedures that 
describe how and when the tradeoff process may be used, including those 
used for issuing task orders. In addition, for each of the contracts 
within our sample, we obtained contract documentation including the 
acquisition plan, solicitation, and source selection decision 
memorandum and reviewed them in preparation for interviews with DOD 
officials. In several cases, the solicitation was unclear as to which 
type of tradeoff process was used. In these cases, we relied on the 
source selection decision document to categorize the tradeoff process 
used. 

We judgmentally selected buying activities to visit based on factors 
including the number of contracts awarded on a best value tradeoff 
basis, contract type, and goods or services procured. Buying activities 
included at least one command from each military department as well as 
a defense agency. We reviewed 27 contracts and 23 task orders through 
our site visits. Specifically, we judgmentally selected 23 of 48 task 
orders for review by compiling all task orders issued on indefinite 
delivery/indefinite quantity contracts obligating over $10 million that 
were administered by the officials at the sites we visited. We chose 
this dollar threshold to exceed the FAR requirement to provide fair 
opportunity notices for task orders valued at $5 million or more. 
Results from these selected contracts or task orders cannot be 
generalized beyond the specific contract contracts or task orders 
selected. 

During the course of our review, we interviewed officials from the 
following commands: 

* Department of the Army, U.S. Army Corps of Engineers, New Orleans 
District Office, Louisiana, and Afghanistan Engineering District, Kabul 
and Kandahar, Afghanistan; 

* Department of the Army, Armament Research, Development and 
Engineering Center, Picatinny Arsenal, New Jersey; 

* Department of the Army, Communications-Electronics Command, Fort 
Monmouth, New Jersey; 

* Department of the Navy, Marine Corps Combat Development Command, 
Quantico, Virginia; 

* Department of the Navy, Naval Air Systems Command, Patuxent River, 
Maryland and Lakehurst, New Jersey; 

* Department of the Air Force, Air Force Materiel Command, Tinker Air 
Force Base, Oklahoma; 

* Defense Logistics Agency Energy, Ft. Belvoir, Virginia; and: 

* Joint Theater Support Contracting Command, U.S. Central Command, 
Kabul and Kandahar, Afghanistan, and Baghdad, Iraq. 

We interviewed DOD acquisition and contracting officials to identify 
their rationale for the selected source selection approach (e.g., the 
thought process behind why a best value approach was chosen over other 
approaches). For award decisions that used a best value tradeoff 
process, we discussed why the evaluation factors were chosen and how 
their relative weights were assigned. We also interviewed officials 
about the process used and the underlying rationale when issuing 
selected task orders. We also interviewed officials to determine what 
the expected outcomes were from using the best value tradeoff process. 

We reviewed applicable DOD source selection decision documents and 
related memoranda to determine how often DOD paid a price differential, 
the amount of the price differential, and the reasons that were given 
underlying the decision to pay a higher price. We defined a price 
differential as a positive difference in price between the offeror who 
received the award and the offeror next in line for award. 

To determine what challenges if any, DOD faces in using the best value 
tradeoff process, we reviewed DOD guidance and interviewed officials 
from Defense Procurement and Acquisition Policy, the military 
departments and defense agencies. 

We conducted this performance audit from March 2010 through October 
2010 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Office of the Under Secretary of Defense: 
3000 Defense Pentagon: 
Washington, DC: 
20301-3000: 

Acquisition, Technology, and Logistics: 

October 20, 2010: 

Mr. John Hutton: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G. Street, N.W.: 
Washington, DC 20548: 

October 20, 2010: 

Dear Mr, Hutton: 

This is the Department of Defense (DoD) response to the GAO draft 
report 11-8 "Defense Contracting" Enhanced Training Could Strengthen 
DoD's Best Value Tradeoff Decisions," dated October 2010, GAO Code 
120890. 

The Department has reviewed the draft report and concurs with its 
recommendation. I intend to request the Panel on Contracting Integrity, 
Subcommittee 5, Appropriate Contracting Approaches and Techniques, to 
assist the Defense Acquisition University in developing the training 
case studies and scenarios that you recommended. I thank you for the 
opportunity to comment on the draft report. 

Sincerely, 

Signed by: 

Shay D. Assad: 
Director, Defense and Procurement Policy: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

John P. Hutton, (202) 512-4841 or huttonj@gao.gov: 

Acknowledgments: 

In addition to the contact named above, Timothy DiNapoli, Assistant 
Director; William Russell, Katheryn Hubbell, Paige Muegenburg, Jodi 
Munson, Anna Russell, Sylvia Schatz, Roxanna Sun, and Bob Swierczek 
made key contributions to this report. 

[End of section] 

Footnotes: 

[1] According to the Federal Procurement Data System-Next Generation, 
DOD's total obligations in fiscal year 2009 were about $366 billion. 
However, this figure reflects an approximately $13.9 billion downward 
adjustment made by DOD to correct an administrative error made in 
fiscal year 2008. As this adjustment significantly affected DOD's 
reported obligations in fiscal year 2009, the $380 billion figure we 
report reflects what DOD's total obligations would have been had the 
error not occurred. 

[2] GAO, High-Risk Series: An Update, [http://www.gao.gov/products/GAO-
09-271] (Washington, D.C.: January 2009). 

[3] Pub. L. No. 111-84 (2009). 

[4] The Federal Procurement Data System-Next Generation is the federal 
government's system for tracking information on contracting actions. 

[5] DFARS 207.103(d)(i)(B). 

[6] Indefinite delivery contracts can be used when the government needs 
flexibility in the timing of orders within a specified period of time. 
One type of indefinite delivery contract is an indefinite delivery/ 
indefinite quantity (IDIQ), which allows the government to order 
unspecified quantities, within stated limits, of products or services 
during a fixed period when they cannot predetermine their needs. After 
award of the base IDIQ contract, products and services are procured 
through individual delivery or task orders during the contract period 
based on governmental needs. IDIQ contracts may be issued as a single 
award to one contractor, or preferably on a multiple award basis to 
several contractors, in which case the FAR requires that each awardee 
be given a fair opportunity to compete for subsequent orders. 

[7] We have previously reported on data reliability issues with FPDS- 
NG. See, e.g., GAO, Federal Contracting: Observations on the 
Government's Contracting Data Systems, [http://www.gao.gov/products/GAO-
09-1032T] (Washington, D.C.: Sept. 29, 2009); and Contract Management: 
Minimal Compliance with New Safeguards for Time-and-Materials Contracts 
for Commercial Services and Safeguards Have Not Been Applied to GSA 
Schedules Program, [http://www.gao.gov/products/GAO-09-579] 
(Washington, D.C.: June 24, 2009). 

[8] Hereafter for the purposes of this report, we use the term "non- 
cost factors" for both non-cost and non-price evaluation factors and 
the term "price" for both cost and price, despite the different ways 
that they are evaluated under the FAR. 

[9] FAR 15.304(b). 

[10] FAR 15.308. 

[11] For additional information see GAO, Federal Contracting: 
Opportunities Exist to Increase Competition and Assess Reasons When 
Only One Offer is Received, [http://www.gao.gov/products/GAO-10-833] 
(Washington D.C.: July 26, 2010). 

[12] GAO, Department of Defense: Additional Actions and Data Are Needed 
to Effectively Manage and Oversee DOD's Acquisition Workforce, 
[http://www.gao.gov/products/GAO-09-342] (Washington D.C.: March 25, 
2009). 

[13] All percentage and fraction estimates reported within this section 
have 95 percent confidence intervals within plus or minus 8 percentage 
points of the estimates themselves. 

[14] For additional information on joint urgent operational needs, see 
GAO, Warfighter Support: Improvements to DOD's Urgent Needs Processes 
Would Enhance Oversight and Expedite Efforts to Meet Critical 
Warfighter Needs, [http://www.gao.gov/products/GAO-10-460] (Washington, 
D.C.: Apr. 30, 2010). 

[15] In the other 20 contracts, DOD did not consider a price 
differential because only one vendor was considered for award or these 
involved a multiple award contract. 

[16] GAO, Defense Acquisitions: Assessment of Selected Weapon Systems, 
[http://www.gao.gov/products/GAO-10-388SP] (Washington D.C.: March 30, 
2010). 

[17] See GAO, Federal Contractors: Better Performance Information 
Needed to Support Agency Contract Award Decisions, 
[http://www.gao.gov/products/GAO-09-374] (Washington D.C.: April 23, 
2009). 

[18] For the purposes of this report, we use the term "non-cost 
factors" for both non-cost and non-price evaluation factors and the 
term "price" for both cost and price, despite the different ways that 
they are evaluated under the FAR. 

[19] The 95 percent confidence interval for this estimate is from 276 
to 309 contracts. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site ([hyperlink, http://www.gao.gov]). Each 
weekday afternoon, GAO posts on its Web site newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select "E-
mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's Web 
site, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, DC 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. 
Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548: