This is the accessible text file for GAO report number GAO-14-584 
entitled 'Defense Contracting: Factors DOD Considers When Choosing 
Best Value Processes Are Consistent with Guidance for Selected 
Acquisitions' which was released on July 30, 2014. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Committees: 

July 2014: 

Defense Contracting: 

Factors DOD Considers When Choosing Best Value Processes Are Consistent 
with Guidance for Selected Acquisitions: 

GAO-14-584: 

GAO Highlights: 

Highlights of GAO-14-584, a report to congressional committees. 

Why GAO Did This Study: 

DOD obligated about $310 billion in fiscal year 2013 for products and 
services needed to support its mission. To competitively acquire what 
is needed, DOD may use best value processes-—including tradeoff and 
LPTA—-to evaluate vendors' proposals. When using the tradeoff process, 
DOD weighs the relative importance of price against non-cost factors. 
By contrast, DOD may use the LPTA process and award the contract based 
on lowest price once technical requirements are met. 

Congress mandated GAO to review DOD's use of best value processes. GAO 
identified, among other things, (1) the extent to which DOD used best 
value processes in fiscal year 2013, (2) the factors DOD considers 
when choosing a source selection process, and (3) training DOD 
provides to its acquisition personnel on source selection processes. 

GAO identified and reviewed solicitations for a projectable sample of 
183 contracts out of 2,851 new, competitively awarded contracts that 
DOD awarded in fiscal year 2013 with obligations over $1 million. GAO 
also reviewed DOD and military departments' guidance regarding their 
use of the best value process. GAO selected 16 contracts for case 
studies based on military department, best value process used, and 
other factors. GAO reviewed contract documents and interviewed program 
and contracting officials for these case studies. GAO also reviewed 
DAU and military departments' training on source selection procedures. 

DOD provided technical comments that GAO incorporated as appropriate. 

What GAO Found: 

The Department of Defense (DOD) used two best value processes—tradeoff 
and lowest price technically acceptable (LPTA)—for approximately 93 
percent of the 2,851 new, competitively awarded contracts awarded in 
fiscal year 2013 with obligations greater than $1 million. DOD used 
the tradeoff process most often in GAO's sample of contracts to 
acquire services, regardless of obligation value. For contracts with 
higher obligations, DOD used the LPTA process primarily to acquire 
commercial products, such as fuel. In contrast, for contracts in GAO's 
sample with lower obligations, DOD used the LPTA process to acquire 
both products and services. Several contracting and program officials 
said that their commands gave more attention to whether LPTA is an 
alternative option in light of declining budgets and efficiency 
initiatives. For contracts with obligations of $25 million or more, 
GAO found that DOD increased its use of LPTA since GAO last reported 
on this issue in October 2010 using fiscal year 2009 data. GAO's prior 
report did not include contracts with lower obligations. 

Table: Estimated Source Selection Processes Used in Fiscal Years 2009 
and 2013 for New, Competitively Awarded Contracts with Obligations of 
$25 million or More[A]: 

Source selection process: Tradeoff; 
Fiscal year 2009 percent: 69; 
Fiscal year 2013 percent: 58; 
Statistically significant change[A]: Yes. 

Source selection process: Lowest price technically acceptable; 
Fiscal year 2009 percent: 26; 
Fiscal year 2013 percent: 36; 
Statistically significant change[A]: Yes. 

Source selection process: Other; 
Fiscal year 2009 percent: 5; 
Fiscal year 2013 percent: 6; 
Statistically significant change[A]: No. 

Source: GAO analysis of DOD contract and solicitation documents. GAO-
14-584. 

[A] The 95 percent confidence intervals for estimates in this table 
are within +/- 8 percentage points of the estimates themselves. 

[End of table] 

DOD's ability to clearly define its requirements and its knowledge of 
potential vendors were key factors that underpinned decisions about 
whether to use tradeoff or LPTA in GAO's 16 case studies. In the eight 
case studies in which DOD used LPTA, contracting and program officials 
generally stated that they had sufficient knowledge of the 
requirements or vendors to feel confident that the lowest priced 
vendor, meeting DOD's technical requirements, could deliver the 
product or service. In contrast, in the eight tradeoff case studies, 
contracting and program officials were less certain about 
requirements, were looking for innovative solutions, or wanted to use 
non-cost factors to differentiate vendors. For example, the United 
States Army Corps of Engineers used technical non-cost factors to 
evaluate vendors' abilities to use robotics for explosives disposal. 
These factors are generally consistent with guidance in the Federal 
Acquisition Regulation and DOD's March 2011 source selection 
procedures. 

DOD, through courses offered by the Defense Acquisition University 
(DAU) and the military departments, provides both classroom and online 
training related to source selection processes to its acquisition 
personnel. Both DAU and military department officials stressed, 
however, the importance of on-the-job training in preparing personnel 
to make informed source selection determinations. For example, Naval 
Facilities Engineering Command officials told GAO that determining 
when requirements are better suited for tradeoff or LPTA is learned 
through gaining experience from on-the-job training. 

View [hyperlink, http://www.gao.gov/products/GAO-14-584]. For more 
information, contact Timothy J. DiNapoli at (202) 512-4841 or 
dinapolit@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DOD Predominately Used Best Value Processes in Fiscal Year 2013, but 
Increased Its Use of LPTA for Higher Dollar Contracts since Fiscal Year 
2009: 

Knowledge of Requirements and Potential Vendors Underpin Decisions 
about Source Selection Process: 

DOD Provides Online and Classroom Training on Source Selection 
Processes, but On-the-Job Training Considered Essential for Making 
Sound Source Selection Decisions: 

Concluding Observations: 

Agency Comments: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Body Armor Vest Acquisitions in Fiscal Year 2013: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: DOD's Use of Non-Cost Factors for the 96 Sample Contracts 
Awarded Using Tradeoff in Fiscal Year 2013 GAO Reviewed, by Dollar 
Obligation: 

Table 2: Estimated Frequencies of Source Selection Processes Used in 
Fiscal Years 2009 and 2013 for New, Competitively Awarded Contracts 
with Obligations of $25 million or More: 

Table 3: Factors Affecting the Source Selection Decision Making Process 
for Selected Case Studies: 

Table 4: Defense Acquisition University (DAU) Courses: 

Table 5: Examples of Source Selection Courses Offered by Military 
Departments: 

Table 6: Total Population and Sample Size for the New, Competitively 
Awarded Contracts in Fiscal Year 2013: 

Figures: 

Figure 1: Source Selection Processes on the Best Value Continuum: 

Figure 2: DOD Contract Obligations for Products and Services in Fiscal 
Year 2013 (Dollars in billions): 

Figure 3: DOD's Use of Best Value Processes for the 171 Sample 
Contracts Awarded in Fiscal Year 2013 GAO Reviewed, by Dollar 
Obligation: 

Abbreviations: 

DAU: Defense Acquisition University: 

DAWIA: Defense Acquisition Workforce Improvement Act: 

DFARS: Defense Federal Acquisition Regulation Supplement: 

DLA: Defense Logistics Agency: 

DOD: Department of Defense: 

FAR: Federal Acquisition Regulation: 

FPDS-NG: Federal Procurement Data System-Next Generation: 

IDIQ: Indefinite Delivery Indefinite Quantity: 

LPTA: Lowest Price Technically Acceptable: 

USACE: United States Army Corps of Engineers: 

USD(AT&L): Under Secretary of Defense for Acquisition, Technology, and 
Logistics: 

[End of section] 

United States Government Accountability Office: 
GAO:
441 G St. N.W. 
Washington, DC 20548: 

July 30, 2014: 

Congressional Committees: 

The Department of Defense (DOD) obligated about $310 billion in fiscal 
year 2013 to acquire products and services needed to support its 
missions. DOD has several source selection processes it can use to 
evaluate vendors' proposals. For example, DOD can use a best value 
tradeoff process, in which it can vary the relative importance of cost 
or price to other factors such as technical capability or past 
performance in its solicitations to vendors. In these cases, DOD may 
award a contract to other than the lowest-priced vendor if DOD 
determines that a higher-priced vendor provides a greater benefit to 
DOD, and this greater benefit is worth paying an additional cost, or 
price differential. Alternatively, DOD may also use a best value, 
lowest price technically acceptable (LPTA) process. In the LPTA 
process, DOD awards to the lowest priced vendor that is technically 
acceptable, and no tradeoffs are permitted. Proper implementation of 
these processes is essential to ensure that DOD lays the foundation for 
successful acquisition outcomes, especially at a time of declining 
spending. 

Committee reports from the Senate and House Armed Services committees 
and the Joint Explanatory Statement accompanying the National Defense 
Authorization Act for Fiscal Year 2014 mandated GAO to report on DOD's 
use of best value processes.[Footnote 1] GAO determined (1) the extent 
to which DOD used best value processes in fiscal year 2013, (2) the 
factors DOD considers when choosing a source selection process, and (3) 
training DOD provides to its acquisition personnel on source selection 
processes. In addition, in response to a matter identified in a 2013 
report from the House Armed Services Committee, appendix II includes 
information on the military departments' acquisitions of body armor 
vests in fiscal year 2013. 

To determine the extent DOD used best value processes in fiscal year 
2013, we used the Federal Procurement Data System-Next Generation 
(FPDS-NG) to identify new, competitively awarded DOD contracts with 
obligations of over $1 million.[Footnote 2] From a population of 2,851 
contracts with over $1 million in obligations, we selected a random 
projectable sample of 227 contracts. We then used a $25 million 
threshold to divide our data set based on a Defense Federal Acquisition 
Regulation Supplement (DFARS) requirement that contracts for products 
or services with $25 million or more in estimated total costs for any 
fiscal year have written acquisition plans, which contain information 
on the anticipated source selection process.[Footnote 3] For contracts 
with obligations of $25 million or more, we compared the percentage of 
contracts solicited using the best value processes to fiscal year 2009 
data that we reported in October 2010.[Footnote 4] Our prior report did 
not include contracts with lower obligations. Ninety-three contracts 
from this sample had higher obligations of $25 million or more and 134 
contracts had lower obligations over $1 million and less than $25 
million. We determined that 44 contracts were either incorrectly coded, 
such as non-competed[Footnote 5] contracts that were identified as 
competed or contracts that used source selection processes other than 
the best value processes. We excluded these contracts, which resulted 
in a total of 183 contracts in our sample that we reviewed, including 
86 contracts with obligations of $25 million or more and 97 contracts 
with obligations over $1 million and less than $25 million. After 
accounting for these errors and assessing the reliability of FPDS-NG 
data by electronically testing the data to identify problems with 
consistency, completeness, or accuracy and reviewing relevant 
documentation, we determined that FPDS-NG data were sufficiently 
reliable for the purposes of our review. We obtained and analyzed 
solicitation documents for all contracts in our sample to identify what 
source selection process DOD used. Further, we reviewed our sample of 
contracts that DOD awarded using tradeoff to identify how often DOD 
considered non-cost factors more important than price, and identified 
the most frequently used non-cost factors. We also used FPDS-NG and 
contract and solicitation documentation to identify characteristics of 
the contracts in our sample. 

To identify what factors DOD considers when choosing a source selection 
process, we analyzed the Federal Acquisition Regulation (FAR), DFARS, 
and DOD and military departments' policy and guidance on source 
selection. We supplemented the review of these documents with 
interviews of senior DOD policy officials at Defense Procurement and 
Acquisition Policy, and at the Army, Navy, and Air Force headquarters. 
We also interviewed officials from at least two commands--based upon 
such factors as the number of contract actions and obligation amounts-
-at each military department (Army, Navy, and Air Force), as well as 
the Defense Logistics Agency (DLA) to discuss factors affecting their 
decision on which source selection process to use. In addition, we 
selected 16 new, competitively awarded contracts from our sample with 
obligations ranging from $1.1 million to $150.7 million in fiscal year 
2013 to further our understanding of why acquisition--including 
contracting and program--officials chose a particular source selection 
process. Our 16 case studies included a selection from each military 
department and DLA, different product and service types, and a mix of 
tradeoff and LPTA processes. We also interviewed industry associations 
to identify their perspectives about DOD's source selection processes. 

To determine what training DOD provides to its acquisition personnel on 
source selection processes, we met with Defense Acquisition University 
(DAU) officials and instructors and reviewed training materials. We 
also obtained workforce data from the DOD Office of the Under Secretary 
of Defense (Acquisition, Technology and Logistics), Human Capital 
Initiatives. Further, we collected and reviewed DOD and military 
departments' training documents to identify additional source selection 
training that is given by them. A more detailed description of our 
scope and methodology is included in appendix I. 

We conducted this performance audit from September 2013 through July 
2014 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Background: 

FAR Part 15 allows the use of several competitive source selection 
processes to meet agency needs. Within the best value continuum, DOD 
may choose a process that it considers the most advantageous to the 
government, either the LPTA or the tradeoff process (see figure 1). 

Figure 1: Source Selection Processes on the Best Value Continuum: 

[Refer to PDF for image: process illustration] 

When proposals are deemed technically acceptable, price is the 
determining factor: 

Tradeoff process continuum: Lowest price technically acceptable 
process: 
Non-cost factors less important than price; 
Non-cost factors equal to price; 
Non-cost factors more important than price. 

Requirements clearly defined Development work low Contract performance 
risk low; 
Importance of price decreasing: 
Requirements less defined Development work high Contract performance 
risk high. 

Source: GAO analysis of Federal Acquisition Regulation Part 15.101. 
GAO-14-584. 

[End of figure] 

DOD may elect to use the LPTA process where the requirement is clearly 
defined and the risk of unsuccessful contract performance is minimal. 
In such cases, DOD may determine that cost or price should play a 
dominant role in the source selection. When using the LPTA process, DOD 
specifies its requirements in the solicitation. Contractors submit 
their proposals and DOD determines which of the contractors meet or 
exceed those requirements, no tradeoffs between cost or price and non-
cost factors are permitted, and the award is made based on the lowest 
price technically acceptable proposal submitted to the government. 

By contrast, DOD may elect to use a tradeoff process in acquisitions 
where the requirement is less definitive, more development work is 
required, or the acquisition has a greater performance risk. In these 
instances, non-cost evaluation factors, such as technical capabilities 
or past performance, may play a dominant role in the source selection 
process. Tradeoffs among price and non-cost factors allow DOD to accept 
other than the lowest priced proposal. The FAR requires DOD to state in 
the solicitation whether all evaluation factors other than cost or 
price, when combined, are significantly more important than, 
approximately equal to, or significantly less important than cost or 
price. 

In October 2010, we reported that DOD used best value processes for 
approximately 95 percent of its new, competitively awarded contracts in 
which $25 million or more was obligated in fiscal year 2009.[Footnote 
6] DOD awarded approximately 26 percent using the LPTA process and 69 
percent using the tradeoff process. DOD awarded the remaining 5 percent 
using sealed bidding, which is a competitive process where award is 
made to the responsible bidder whose bid conforms to the invitations 
for bid and is most advantageous for the government considering only 
price and price-related factors included in the solicitation.[Footnote 
7] At that time, we found that the majority of the contracts were 
awarded using a tradeoff process in which all evaluation factors other 
than cost or price, when combined, were significantly more important 
than cost or price. Our analysis showed that DOD considered past 
performance and technical capability evaluation factors as the most 
important among the non-cost factors. Further, we found using a 
tradeoff process can be more complex and take more time than other 
source selection methods, and requires that acquisition staff have 
proper guidance, needed skills, and sound business judgment. While DOD 
and the military departments had taken steps to improve source 
selection procedures, acquisition personnel noted a lack of training to 
assist them in deciding whether or not a price differential is 
warranted when making tradeoff decisions. We recommended that to help 
DOD effectively employ best value tradeoff processes, DOD develop 
training elements, such as case studies, that focus on reaching 
tradeoff decisions, as it updates its training curriculum. DOD 
concurred and implemented the recommendation in August 2012. 

DOD issued new guidance that emphasizes affordability and 
standardization of best value processes since our analysis of fiscal 
year 2009 contracts. In September 2010, the Under Secretary of Defense 
for Acquisition, Technology, and Logistics (USD(AT&L)) issued a 
memorandum that established its Better Buying Power Initiative to 
obtain greater efficiency and productivity in defense spending. In its 
memorandum, USD(AT&L) emphasized that DOD must prepare to continue 
supporting the warfighter through the acquisition of products and 
services in potentially fiscally constrained times. USD(AT&L) noted 
that DOD must "do more without more." In April 2013, USD(AT&L) issued 
another memorandum to update the Better Buying Power Initiative. This 
memorandum identifies seven areas USD(AT&L) is pursuing to increase 
efficiency and productivity in defense spending. One area is 
incentivizing productivity and innovation in industry and government. 
As part of this guidance, USD(AT&L) states that "best value" in a 
competitive source selection should generally indicate that the 
government is open to paying more (up to some amount) than the minimum 
price bid in return for a product that provides more than the minimum 
needed performance. In addition, USD(AT&L) states that LPTA should be 
used in situations where DOD would not realize any value from a 
proposal exceeding its minimum technical or performance requirements 
and that another process should be used when standards of performance 
and quality are subjective. A second area of this guidance includes 
improving the professionalism of the total acquisition workforce. DOD 
has previously reported that training is a critical element of 
improving and sustaining a high quality workforce with the right skills 
and capabilities. 

USD(AT&L) also issued source selection procedures in March 2011 to 
standardize the methodology and process that DOD uses to conduct 
competitively negotiated source selections. For example, USD(AT&L) 
outlined a common set of principles and procedures for conducting 
acquisitions using the best value processes including the use of 
standardized rating criteria and descriptions for technical capability 
and past performance factors. Further, similar to information presented 
in the Better Buying Power Initiative, USD(AT&L) stated in the 
procedures that the LPTA process may be used in situations where the 
government would not realize any value from a proposal exceeding 
minimum technical or performance requirements, often for acquisitions 
of commercial or non-complex services or supplies which are clearly 
defined and expected to be low risk. In its April 2013 memorandum 
updating the Better Buying Power Initiative, USD(AT&L) directed the 
director of Defense Procurement and Acquisition Policy to update the 
guidance to describe the characteristics of a technically acceptable 
solution by July 1, 2013. As of July 2014, DOD officials are 
coordinating comments on a draft revision of the guidance. The Defense 
Procurement and Acquisition Policy official in charge of the revision 
told us the original due date of July 1, 2013 was established before 
they decided to do a more comprehensive update of the guidance, which 
has contributed to the date slipping for its completion. 

During the time that USD(AT&L) issued these initiatives and guidance--
specifically, between fiscal years 2009 and 2013--DOD experienced a 
number of changes in its contracting activity, including: 

* Total obligations for products and services decreased from $380 
billion in fiscal year 2009 to $310 billion in fiscal year 2013, 

* Obligations on new, competed contracts decreased from $70 billion in 
fiscal year 2009 to $43 billion in fiscal year 2013, and: 

* Obligations on new, competed contracts of $25 million or more 
decreased from $39 billion in fiscal year 2009 to $24 billion in fiscal 
year 2013.[Footnote 8] 

See figure 2 for our analysis of DOD's contract obligations from FPDS-
NG for fiscal year 2013. 

Figure 2: DOD Contract Obligations for Products and Services in Fiscal 
Year 2013 (Dollars in billions)[A]: 

[Refer to PDF for image: pie-chart and sub-chart] 

Non-competed contracts;B]: $132.4 billion; 
Competed contracts: $169.8 billion. 

Of the Competed contracts: 
Orders and modifications on contracts awarded prior to fiscal year 
2013: 
- $1 million or less: $0.7 billion; 
- Over $1 million and less than $25 million: $37.2 billion; 
- $25 million or more: $88.6 billion; 
New contracts: 
- $1 million or less: $7.3 billion; 
- Over $1 million and less than $25 million: $12.2 billion; 
- $25 million or more: $23.8 billion. 

Source: GAO analysis of Federal Procurement Data System-Next 
Generation obligations data. GAO-14-584. 

[A] Excludes obligations of $7.8 billion on non-DOD contracts. Values 
are based on Federal Procurement Data System-Next Generation as of 
January 2014. 

[B] The non-competed contracts category consists of those contracts 
awarded under other than full and open competition as defined in 
Federal Acquisition Regulation subpart 6.3. 

[End of figure] 

Even though DOD's obligations decreased between fiscal year 2009 and 
2013, it did acquire a similar mix of products and services in both 
years. In addition, the percentage of commercial items purchased in 
those 2 fiscal years was approximately the same. 

DOD Predominately Used Best Value Processes in Fiscal Year 2013, but 
Increased Its Use of LPTA for Higher Dollar Contracts since Fiscal Year 
2009: 

DOD predominately used best value processes--tradeoff and LPTA--to 
evaluate offers from potential vendors in fiscal year 2013. DOD used 
best value processes for approximately 93 percent of the 2,851 new, 
competed contracts for which it had obligated over $1 million in fiscal 
year 2013 and used sealed bid for approximately 7 percent.[Footnote 9] 
For contracts with obligations of $25 million or more, DOD used the 
tradeoff process for approximately 58 percent of the contracts and the 
LPTA process for approximately 36 percent of the contracts. For 
contracts with obligations over $1 million and less than $25 million, 
DOD used tradeoff and LPTA at about the same overall rate--47 percent 
and 45 percent, respectively.[Footnote 10] 

In our sample of 171 contracts that used best value processes, DOD used 
tradeoff for 96 contracts and LPTA for 75 contracts. We found some 
variation in terms of what process was used to acquire products and 
services at the different thresholds we reviewed (see figure 3). 

Figure 3: DOD's Use of Best Value Processes for the 171 Sample 
Contracts Awarded in Fiscal Year 2013 GAO Reviewed, by Dollar 
Obligation: 

[Refer to PDF for image: illustrated table] 

Obligations of $25 million or more: 

Tradeoff: 
Product: 9 contracts; 
Services: 41 contracts; 
Total: 50 contracts. 

Lowest price technically acceptable: 
Product: 27 contracts; 
Services: 4 contracts; 
Total: 31 contracts. 

Obligations over $1 million and less than $25 million: 

Tradeoff: 
Product: 7 contracts; 
Services: 39 contracts; 
Total: 46 contracts. 

Lowest price technically acceptable: 
Product: 24 contracts; 
Services: 20 contracts; 
Total: 44 contracts. 

Source: GAO analysis of DOD solicitation and contract documents. GAO-
14-584. 

[End of figure] 

As seen in the above figure, DOD used the tradeoff process most often 
in our sample to acquire services, including those related to 
construction projects, aircraft maintenance, and other support 
services, regardless of obligation amount. For contracts with 
obligations of $25 million or more, DOD used the LPTA process primarily 
to acquire commercial products such as fuel. In contrast, for contracts 
with obligations over $1 million and less than $25 million, DOD used 
the LPTA process to acquire a mix of products and services, including 
fuel, aircraft parts, computer equipment, construction-related 
services, engineering support services, and ship maintenance and 
repairs. 

The desire to weigh non-cost factors such as technical approach and 
past performance was a key factor cited in the majority of the 
solicitations issued for the 96 contracts in our sample that DOD 
awarded using the tradeoff process, regardless of obligation value (see 
table 1). 

Table 1: DOD's Use of Non-Cost Factors for the 96 Sample Contracts 
Awarded Using Tradeoff in Fiscal Year 2013 GAO Reviewed, by Dollar 
Obligation: 

Contract obligation amount: Over $1 million and less than $25 million; 

Non-cost factors less important than price: 0; 
Non-cost factors equal to price: 5; 
Non-cost factors more important than price: 41. 

Contract obligation amount: $25 million or more; 
Non-cost factors less important than price: 0; 
Non-cost factors equal to price: 15; 
Non-cost factors more important than price: 35. 

[End of table] 

Source: GAO analysis of DOD solicitation and contract documents. 
GAO-14-584: 

For the 76 contracts for which non-cost factors were more important 
than price, DOD acquired both products and services, such as computer 
equipment, aircraft maintenance services, and communication network 
support services.[Footnote 11] In addition, our analysis found that 
technical approach and past performance were the factors most often 
identified as more important than price among the non-cost factors. For 
example, 48 out of the 76 contracts in our sample identified technical 
approach as the most important factor. Additionally, 23 out of the 76 
contracts in our sample identified past performance as the most 
important factor. Other non-cost factors considered in some of the 
solicitations with much less frequency than technical approach and past 
performance include small business participation and delivery schedule. 

While data on DOD's use of source selection processes were not readily 
available, our analysis found that DOD increased its use of LPTA from 
fiscal year 2009 to fiscal year 2013 for contracts with obligations of 
$25 million or more (see table 2). We cannot make a comparison between 
fiscal year 2009 and fiscal year 2013 for the lower dollar range, 
because our prior report only focused on contracts with obligations of 
$25 million or more in fiscal year 2009.[Footnote 12] 

Table 2: Estimated Frequencies of Source Selection Processes Used in 
Fiscal Years 2009 and 2013 for New, Competitively Awarded Contracts 
with Obligations of $25 million or More[A]: 

Source selection process: Tradeoff; 
Fiscal year 2009 percent: 69; 
Fiscal year 2013 percent: 58; 
Statistically significant change[A]: yes. 

Source selection process: Lowest price technically acceptable; 
Fiscal year 2009 percent: 26; 
Fiscal year 2013 percent: 36; 
Statistically significant change[A]: yes. 

Source selection process: Sealed bid; 
Fiscal year 2009 percent: 5; 
Fiscal year 2013 percent: 6; 
Statistically significant change[A]: No. 

Source: 2009: GAO-11-8; 2013: GAO analysis of DOD solicitation 
documents. GAO-14-584. 

[A] The 95 percent confidence intervals for estimates in this table are 
within +/-8 percentage points of the estimates themselves. 

[End of table] 

Several contracting and program officials said that their commands gave 
more attention to whether LPTA is an alternative option in light of 
declining budgets and Better Buying Power Initiatives. Further, 
declining budgets encouraged contracting and program officials to 
streamline requirements. For example: 

* The Executive Director of Army Contracting Command--Aberdeen Proving 
Ground, one of five Army Contracting Command centers--said that overall 
there is an increased cost consciousness regarding acquisitions, 
resulting from the Better Buying Initiatives and declining budgets. As 
a part of that increased cost consciousness, there is an increased 
willingness and necessity to re-examine tools that could present better 
prices. For example, the Executive Director referred to LPTA as "a tool 
that has been at the bottom of the source selection tool box collecting 
dust for some time." As it became necessary to take a look at what is 
really needed, they have "dusted off" the LPTA tool and had more 
discussions about how to set the technical acceptability at an 
appropriate level where there is no additional benefit from paying for 
more than that level. 

* Contracting officials from Naval Facilities and Engineering Command 
stated that in the current fiscal environment of "doing more with 
less," they are educating their contracting personnel to use LPTA when 
appropriate. For example, on March 28, 2013, the Command sent an email 
communication to its contracting staff that provided guidance on the 
use of LPTA for task orders on multiple award contracts that are less 
than $10 million. The guidance stated that the contracting officer may 
choose to consider only price or cost for award purposes when the 
requirement is valued at less than $10 million, considered to be non-
complex, and where non-cost factors are deemed unnecessary. These 
officials stated LPTA is less complex and less time consuming than 
tradeoff, and as a result, they can save personnel resources. In 
addition to internal guidance, Navy officials told us that the Better 
Buying Power Initiative also directs acquisition personnel to look for 
efficiencies and streamlining in acquisitions. 

* Contracting officials from Naval Supply Systems Command stated they 
increased their scrutiny on tradeoff acquisitions, which has 
contributed to a cultural shift to increase the consideration of LPTA 
as an alternative source selection process. The command issued an 
October 9, 2012 memorandum to contracting activities that states if 
non-cost factors are more important than price, the acquisition must be 
reviewed by a senior level acquisition executive. 

* Similarly, Air Force Materiel Command contracting and program 
officials stated that given the budget environment, it is increasingly 
difficult to justify higher dollar solutions from a technical 
standpoint when solutions may exist that meet the minimum requirement. 

* DLA contracting officials stated that in light of resource 
constraints, it is increasingly common to purchase products that meet 
the program's needs without overstating the requirement. These 
officials told us LPTA is a good choice for mature, commercial 
requirements where there is no added value in conducting a tradeoff 
given the need to stretch budgets. 

Knowledge of Requirements and Potential Vendors Underpin Decisions 
about Source Selection Process: 

Our review of contract documents and interviews with program and 
contracting officials from our 16 case studies found that for these 
specific acquisitions, DOD's ability to clearly define its requirements 
and its knowledge of potential vendors were the key factors that 
underpinned the decisions about whether to use tradeoff or LPTA. For 
example, in the eight case studies in which DOD used LPTA, DOD 
contracting and program officials generally stated they had sufficient 
knowledge of the requirements or vendors to feel confident that the 
lowest priced vendor, after meeting technical acceptability 
requirements, could deliver the product or service. In contrast, in our 
eight tradeoff case studies, contracting and program officials were 
less certain about requirements, were looking for innovative solutions, 
or wanted to use non-cost factors, such as past performance, as a 
differentiator when selecting the vendor. We found that for these 16 
case studies DOD's reasons for choosing LPTA or tradeoff were generally 
consistent with guidance in the FAR and DOD's source selection 
procedures. Table 3 provides several highlights from the case studies 
that illustrate where DOD's ability to clearly define its requirements 
and its knowledge of potential vendors affected the source selection 
decision making process. 

Table 3: Factors Affecting the Source Selection Decision Making Process 
for Selected Case Studies: 

Source selection process: Lowest price technically acceptable; 
Military department/fiscal year 2013 obligation value/contract type: 
Navy; $5.2 million; 
Contract type: firm-fixed-price; 
Description of the solicitation and requirements: This solicitation 
was for a follow-on contract to convert a '.mil' website to a '.edu' 
website with email services; 
Product or service: service; 
Number of bids received: 3; 
Awarded to incumbent: no; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: Knowledge of requirements: 
Navy contracting and program officials stated that the requirements 
for the system were well-defined as a result of prior work and had 
clear minimum technical standards that a vendor needed to meet. For 
example, one of the technical standards required vendors to have 
proven expertise with using and installing a particular commercially-
available software program; 
Knowledge of vendors: Navy contracting and program officials had a 
high level of confidence in vendor capability, because they had 
experience with and knowledge of industry vendors from a prior 
contract with similar requirements. Through market research, 
contracting officials determined that vendors had experience in 
providing similar services in previous acquisitions. 

Source selection process: Lowest price technically acceptable; 
Military department/fiscal year 2013 obligation value/contract type: 
Air Force; $44.0 million; 
Contract type: firm-fixed-price; 
Description of the solicitation and requirements: This solicitation 
was to build a system that mimics anti-aircraft missile systems. The 
Air Force previously awarded a sole source contract for this 
requirement. Following a management review, the solicitation was 
determined to be appropriate for competition; 
Product or service: product; 
Number of bids received: 2; 
Awarded to incumbent: yes; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: Knowledge of requirements: 
Air Force contracting and program officials told us that this follow-
on requirement was to replicate the functionality of an existing 
system for a planned procurement of 15 new units. Based on the 
historical experience of the contracting and program team, they were 
able to clearly define and convey the desired technical capability of 
the system; 
Knowledge of vendors: Air Force contracting and program officials had 
previous experience and knowledge about the technical engineering 
capability of the vendors that can produce these missile systems. 
Through market research, contracting officials determined that several 
vendors had adequate technical capability to support competition. 

Source selection process: Lowest price technically acceptable; 
Military department/fiscal year 2013 obligation value/contract type: 
Navy; $66.4 million; 
Contract type: firm-fixed-price; 
Description of the solicitation and requirements: This solicitation 
was to construct a replacement fuel pier; 
Product or service: service; 
Number of bids received: 7; 
Awarded to incumbent: no; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: Knowledge of requirements: 
Navy contracting officials told us that this requirement was clearly 
defined as the solicitation was for the construction of the fully 
completed design that the Navy provided to the vendors at the time of 
solicitation. In addition, the requirement had previously been 
accomplished in a similar location so officials deemed it very low 
risk; 
Knowledge of vendors: Navy contracting officials told us that they had 
knowledge of the size and capability of the local vendors, because 
some of the potential vendors had experience working on similar 
projects in the past. 

Source selection process: Lowest price technically acceptable; 
Military department/fiscal year 2013 obligation value/contract type: 
Air Force; $1.3 million; 
Contract type: firm-fixed-price; 
Description of the solicitation and requirements: This solicitation 
was for a weatherproof, name brand communication and control switch 
equipment; 
Product or service: product; 
Number of bids received: 3; 
Awarded to incumbent: no; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: Knowledge of requirements: 
Air Force contracting officials stated that the requirement was for a 
modified commercial off-the-shelf item that had known vendors; 
Knowledge of vendors: Air Force contracting officials were confident 
that multiple vendors could provide the name brand specific routers 
after conducting market research. 

Source selection process: Tradeoff; 
Military department/fiscal year 2013 obligation value/contract type: 
Defense Logistics Agency; $3.4 million; 
Contract type: fixed-price with economic price adjustment; 
Description of the solicitation and requirements: This solicitation was 
for fabric dyeing for production of dress uniform fabric; 
Product or service: product; 
Number of bids received: 1; 
Awarded to incumbent: yes; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: 
Knowledge of requirements/vendors: Defense Logistics Agency 
contracting officials determined that tradeoff on past performance was 
required to ensure that the vendor would be able to provide dyed 
fabric within an acceptable color range. Officials told us that the 
stringent color specifications required vendors to have a level of 
competency and experience. Further, officials said that trading off on 
past performance informed them on vendor qualifications and mitigated 
the risk that an inexperienced vendor would bid at the lowest price to 
try and win the contract. 

Source selection process: Tradeoff; 
Military department/fiscal year 2013 obligation value/contract type: 
United States Army Corps of Engineers (USACE); $2.2 million; 
Contract type: firm-fixed-price; 
Description of the solicitation and requirements: This solicitation was 
for use of robotics to dispose of explosives; 
Product or service: service; 
Number of bids received: 3; 
Awarded to incumbent: no; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: Knowledge of requirements: 
USACE contracting officials told us they were seeking to differentiate 
vendors on their innovative technical capability in which robotics 
could be incorporated that improved the disposal process and reduced 
safety risks; 
Knowledge of vendors: USACE contracting officials told us that they 
had performed market research to better understand the maturity of the 
robotic technology industry. Through market research, contracting 
officials determined that several vendors could potentially provide 
robotic technology applicable to explosives disposal. 

Source selection process: Tradeoff; 
Military department/fiscal year 2013 obligation value/contract type: 
Navy; $41.8 million; 
Contract type: firm-fixed-price; 
Description of the solicitation and requirements: This solicitation 
was for Vertical Launching System Canisters, which are used to launch 
missiles from ships. The requirement included a technical data package 
containing the exact specifications that DOD required; 
Product or service: product; 
Number of bids received: 1; 
Awarded to incumbent: yes; 
Description of DOD knowledge of requirements and knowledge of vendors 
affecting the source selection decision: Knowledge of requirements: 
Navy contracting and program officials told us that from their 
extensive knowledge and experience with these requirements, they knew 
that technical capability was a critical factor given the missile-
related nature of the requirement. As a result, the program office 
wanted the discretion to discriminate on technical and safety factors 
to ensure demonstrated capability; 
Knowledge of vendors: Navy's market research included releasing a 
draft solicitation, holding an Industry Day to gauge industry 
interest, and sending the technical data package to multiple vendors. 
While the market research indicated that a number of vendors could 
potentially provide the desired product, only the incumbent bid for 
the requirement. 

Source: GAO analysis of DOD solicitation and contract documents, 
Federal Procurement Data System-Next Generation data, and interviews 
with DOD officials. GAO-14-584. 

[End of table] 

Policy officials from some military departments noted that setting 
technical acceptability levels is important for contracts awarded 
through LPTA to be successful. Defense Procurement and Acquisition 
Policy officials told us the ongoing efforts to revise DOD's 2011 
source selection procedures is intended, in part, to further define how 
to conduct best value processes. According to these officials, the 
revised guidance will emphasize that for LPTA, the solicitation must 
clearly describe the minimum evaluation standards. In addition, they 
expect the guide will provide additional information on how to 
determine when to pay a price premium. 

DOD Provides Online and Classroom Training on Source Selection 
Processes, but On-the-Job Training Considered Essential for Making 
Sound Source Selection Decisions: 

DOD, through courses offered by DAU and the military departments, 
provides both classroom and online training related to source selection 
processes to its acquisition personnel. Both DAU and military 
department officials stressed, however, the importance of on-the-job 
training in preparing personnel to make informed source selection 
determinations. 

Congress passed the Defense Acquisition Workforce Improvement Act 
(DAWIA) in 1990 to both ensure effective and uniform education, 
training, and career development of members of the acquisition 
workforce, including contracting and other career fields, and 
established DAU to provide training.[Footnote 13] The act also required 
DOD to establish career paths, referred to by DOD as certification 
requirements, for the acquisition workforce. DOD military departments 
must track acquisition workforce personnel to ensure that they meet 
mandatory standards established for level I (basic or entry), level II 
(intermediate or journeyman), or level III (advanced or senior) in a 
career field, such as contracting, life cycle logistics, and program 
management. Similar requirements and levels are established for each of 
the acquisition career fields identified by DOD. 

DOD identified a need to increase the capacity and size of the 
acquisition workforce over the past several years. For example, in a 
DOD assessment of the contracting workforce completed in September 
2008, senior DOD contracting leaders identified the importance of not 
only mastering the "what," but in using critical thinking and sound 
judgment to apply the knowledge--thus mastering the "how" of 
contracting among its entry-level and mid-career personnel. To help 
address concerns that DOD had become too reliant on contractors to 
support core functions and to rebuild the capacity and skill sets that 
eroded in the years that followed the downsizing of the workforce in 
the 1990s, DOD increased its number of acquisition workforce positions 
from 133,103 in fiscal year 2009 to 151,355 in fiscal year 2013--
including a 9.5 percent increase or an additional 2,616 positions--in 
the contracting career field. 

DAU officials identified five training courses that are taken either 
online or in the classroom to provide acquisition personnel, including 
contracting and program officials, the knowledge and skills necessary 
to make source selection decisions. Contracting personnel are required 
or recommended to complete all five of the identified training courses 
at some point in their career to obtain specific DAWIA certifications. 
Additionally, DAU makes these classes available to personnel outside 
the DAWIA acquisition workforce. Based on our analysis of student self-
reported exit data in fiscal year 2013 and our discussion with DAU 
officials, we found that many graduates for these courses did not 
indicate their career field when completing the course registration or 
exit survey, particularly for online courses, which makes it difficult 
to know how many personnel outside of the DAWIA workforce with 
acquisition-related responsibilities took these courses. In September 
2011, we reported on personnel working on service acquisitions who are 
outside the DAWIA acquisition workforce with acquisition-related 
responsibilities and found the number of these individuals to be 
substantial. As such, we recommended that the Secretary of Defense 
establish criteria and a time frame for identifying personnel outside 
the DAWIA acquisition workforce with acquisition-related 
responsibilities.[Footnote 14] DOD concurred with the recommendation 
and, as of June 2014, is developing a way to identify all of the non-
DAWIA personnel with acquisition-related responsibilities and the 
appropriate training curriculum they should receive. Table 4 outlines 
each of these five courses. 

Table 4: Defense Acquisition University (DAU) Courses: 

Course title: CLC 007: Contract; Source Selection; 
Course description: 
An online, continuous learning module that contains three lessons 
focused solely on source selection. Learning objectives include: 
* roles and responsibilities of each source selection participant; 
* procedures for safeguarding source selection information; 
* key elements of a source selection plan; 
* evaluation criteria to best support an award recommendation; 
* appropriate elements of debriefing; and; 
* importance of effective dialogue with vendors; 
Targeted attendees: Personnel new to the contracting career field or 
non-contracting personnel that play a role in the acquisition process; 
Other career field required or recommended attendees: 
* Cost estimating; 
* Financial management; 
* Life cycle logistics; 
Production, quality and manufacturing. 

Course title: CON 121: Contract Planning; 
Course description: 
An 8-part online contract planning training course. Lesson 8 covers 
acquisition strategy development, which includes: 
* characteristics of best value; 
* elements of and criteria used in developing a written acquisition 
plan; and; 
* elements of source selection planning; 
Targeted attendees: Personnel new to the contracting career field or 
non-contracting personnel that play a role in the acquisition process; 
Other career field required or recommended attendees: 
* Financial management; 
* Industrial and/or contract property management; 
* Life cycle logistics; 
* Program management; 
* Purchasing. 

Course title: CON 200: Business Decisions for Contracting; 
Course description: 
An 11-part online training course that builds on Level I (basic or 
entry) pre-award business and contracting knowledge to process complex 
procurements. Lesson 10 provides an overview on source selection 
strategy and discusses the tradeoff versus LPTA decision; 
Targeted attendees: Contracting personnel with at least 2 years of 
contracting experience in the contracting career field seeking 
intermediate certification; 
Other career field required or recommended attendees: 
* Industrial and/or contract property management. 

Course title: CON 280: Source Selection and Administration of Service 
Contracts; 
Course description: A 10-day in-classroom training course. The primary 
focus of this course is on the acquisition of services under FAR Part 
15 procedures with an emphasis on performance-based acquisitions for 
services; contract types, incentives, and administration; as well as 
source selection. On days 4-7, students participate in group exercises 
designed to replicate the source selection development, evaluation, 
and decision processes; 
Targeted attendees: Contracting personnel with at least 2 years of 
contracting experience in the contracting career field seeking 
intermediate certification; 
Other career field required or recommended attendees: 
* Life cycle logistics. 

Course title: CON 334: Advanced Contingency Contracting Officer's 
Course; 
Course description: A 4-day in-classroom training course with 
exercises focused on the unique aspects of contingency operations, 
critical thinking skills, and the execution of appropriate contractual 
instruments. In lesson 6, participants are introduced to the elements 
of the formal source selection process, which they utilize later to 
prepare a mock solicitation; 
Targeted attendees: Contracting career field personnel who are in 
deployable positions and will be the designated Chief of the 
Contracting Office or Senior Contracting Official; 
Other career field required or recommended attendees: 
* Not applicable. 

Source: GAO analysis of DAU documents. GAO-14-584. 

[End of table] 

We also found that military departments provided source selection 
training--offering both overview and refresher courses--to contracting 
staff and others involved in the source selection process. Table 5 
identifies examples of the training courses offered by various military 
departments. 

Table 5: Examples of Source Selection Courses Offered by Military 
Departments: 

Military department/course title: Air Force Source Selection Training 
Phase I and Phase II; 
Course description: This course provides basic refresher training on 
best value acquisitions to the source selection team. During Phase I 
of the training, the source selection team develops and jointly plans 
the acquisition strategy based upon the customer identified 
requirements for a current acquisition. Phase II, also known as "just-
in-time" training assists the source selection team in evaluating and 
executing the contract for a current acquisition; 
Targeted attendees: Every member involved in the acquisition process 
from source selection to contract award as well as those in the 
contracting career field. Additionally, this course is offered in 
abbreviated format for Senior Executive Service level participants. 

Military department/course title: Army (Medical Research Acquisition 
Activity) Lowest Price Technically Acceptable (LPTA) Source Selection 
Process: Banging the Basics; 
Course description: This course provides an overview of best value 
acquisitions. It includes discussion of the best value as outlined in 
Federal Acquisition Regulation (FAR) Part 15, identifies for class 
participants when LPTA is the appropriate source selection process for 
a particular acquisition, and stresses the importance of consistency 
within the request for proposal; 
Targeted attendees: Contracting and other personnel involved in 
choosing the source selection process. 

Military department/course title: Defense Logistics Agency Back to 
Basics; 
Course description: This course is a refresher training to address 
areas where there may be skill gaps in contracting. It identifies 
situations when tradeoff versus LPTA might be used and roles and 
responsibilities of those involved in source selection; 
Targeted attendees: Contracting personnel. 

Military department/course title: Marine Corps Systems Command Kapstone 
Source Selection Training; 
Course description: This course outlines the source selection process 
and the roles of those involved in source selection. It identifies 
situations when tradeoff versus LPTA might be used; 
Targeted attendees: Contracting and program personnel. 

Military department/course title: Navy CTC 415: Source Selection 
Technical Evaluation; 
Course description: This course provides students with an overview of 
the source selection process and responsibilities of those involved in 
source selection; 
Targeted attendees: Contracting personnel and program managers 
involved in the source selection process. 

Source: GAO analysis of military department documents and interviews 
with military department officials. GAO-14-584. 

[End of table] 

DAU and military department officials we spoke with pointed to their 
training as providing educational resources from which the acquisition 
workforce can understand the basics of appropriate source selection 
processes. These officials also stressed the role on-the-job training 
plays when making such determinations. For example, policy officials 
within the office of the Assistant Secretary of the Army for 
Acquisition, Technology, and Logistics told us that on-the-job training 
provides important exposure for less experienced acquisition staff to 
the source selection decision making processes. As a result, 
contracting officials have a better understanding of situations where a 
particular source selection process may be more appropriate than 
others. Many officials told us that contracting officials can best 
understand the acquisition process and apply their in-classroom 
training through making real world source selection decisions. As such, 
several military department officials, including contracting officials 
from our case studies, provided examples of why they consider on-the-
job training to be important, including the following: 

* Air Force Installation Contracting Agency contracting officials from 
one of our case studies and a command official told us that on-the-job 
training and experience are important factors that affect the source 
selection process determination. They stated that on-the-job training 
provides experience and opportunities for contracting officers to make 
critical decisions that can only occur in a source selection 
environment. To that end, these officials told us that informal 
mentoring relationships are established wherein newer, less experienced 
staff is assigned to work with more senior staff. 

* Naval Facilities Engineering Command officials and contracting 
officials from one of our case studies stated that the task of 
identifying when requirements would better suit a particular source 
selection process is learned through gaining experience from on-the-job 
training. 

* Naval Sea Systems Command officials from one of our case studies 
stated that the best training they received is on-the-job training. 
These officials explained that more senior contracting officers help 
newer contracting staff with their acquisitions. They consider mentor 
type training invaluable in learning how to conduct an acquisition. 

Concluding Observations: 

Best value processes continued to underlie the vast majority of DOD's 
new, competitively awarded contracts. DOD has increased its use of the 
LPTA process in recent years for higher value contracts, and its 
decision making regarding which source selection process to use did not 
appear to be ill-advised. Its decision making was generally rooted in 
knowledge about the requirements and vendors. In our sample of 16 
cases, we identified instances in which DOD used LPTA for what appeared 
to be complex acquisitions, such as the system to mimic an anti-
aircraft missile, but the acquisition team had considerable knowledge 
about the requirements or vendors. In other cases, DOD used the 
tradeoff process for what appeared to be relatively simple 
acquisitions, such as fabric dyeing, yet the acquisition team 
identified complexities about the proposed acquisition. Amid the 
climate of rapidly building fiscal pressures and cost consciousness, 
selecting the right source selection approach continues to be essential 
to ensure the department acquires what it needs without paying more 
than necessary. 

Agency Comments: 

We are not making recommendations in this report. We provided a draft 
of this report to DOD for comment. DOD did not provide written comments 
on this report but did provide technical comments, which we 
incorporated as appropriate. 

We are sending copies of this report to appropriate congressional 
committees and the Secretary of Defense. The report will be available 
at no charge on GAO's website at [hyperlink, http://www.gao.gov]. 

If you or your staff have questions about this report, please contact 
me at (202) 512-4841 or dinapolit@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions to 
this report are listed in appendix III. 

Signed by: 

Timothy J. DiNapoli: 
Director: 
Acquisition and Sourcing Management: 

List of Committees: 

The Honorable Carl Levin: 
Chairman: 
The Honorable James Inhofe: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

The Honorable Richard J. Durbin: 
Chairman: 
The Honorable Thad Cochran: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
United States Senate: 

The Honorable Howard P. "Buck" McKeon: 
Chairman: 
The Honorable Adam Smith: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Honorable Rodney Frelinghuysen: 
Chairman: 
The Honorable Pete Visclosky: 
Ranking Member: 
Subcommittee on Defense: 
Committee on Appropriations: 
House of Representatives: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Committee reports from the Senate and House Armed Services committees 
and the Joint Explanatory Statement accompanying the National Defense 
Authorization Act for Fiscal Year 2014 mandated GAO to report on the 
Department of Defense's (DOD) use of best value processes.[Footnote 15] 
We determined 1) the extent to which DOD used best value processes in 
fiscal year 2013; (2) the factors DOD considers when choosing a source 
selection process; and (3) training DOD provides to its acquisition 
personnel on source selection processes. In addition, in response to a 
matter identified in a 2013 report from the House Armed Services 
Committee, appendix II includes information on the military 
departments' acquisitions of body armor vests in fiscal year 2013. 

To determine the extent DOD used the best value processes in fiscal 
year 2013, we used data from the Federal Procurement Data System-Next 
Generation (FPDS-NG) as of October 2013 to identify a population of 
contracts based on the following criteria: (1) newly awarded by DOD in 
fiscal year 2013, (2) competitively awarded, and (3) had obligations of 
over $1 million in fiscal year 2013. This analysis identified a 
population of 2,851 contracts, and from this population we selected a 
stratified random sample of 227 contracts, with the strata defined by 
whether the contract had obligations of $25 million or more, or whether 
its obligations totaled over $1 million and less than $25 million. We 
divided the data into two groups including contracts with higher 
obligations of $25 million or more and contracts with lower obligations 
over $1 million and less than $25 million. We used the $25 million 
threshold to divide our data set based on a Defense Federal Acquisition 
Regulation Supplement (DFARS) requirement that contracts for products 
or services with $25 million or more in estimated total costs for any 
fiscal year have written acquisition plans, which contain information 
on the anticipated source selection process.[Footnote 16] For contracts 
with obligations of $25 million or more, we compared the percentage of 
contracts solicited using best value processes to fiscal year 2009 data 
we reported in October 2010.[Footnote 17] Our prior report did not 
include contracts with lower obligations of less than $25 million. 

We obtained and analyzed the solicitation documents for all of the 
contracts in our sample to identify the source selection process DOD 
used. We verified the contract award fields in FPDS-NG with contract 
and solicitation data to ensure that the contracts within our sample 
were in-scope. Based on that analysis, we determined that a total of 44 
contracts were out of scope for our review. These 44 contracts were 
excluded from our analysis, because they were either incorrectly coded 
in our key parameters, or were awarded using processes outside of the 
Federal Acquisition Regulation (FAR) Part 14 on sealed bidding or Part 
15 on contracting by negotiation (which includes best value processes) 
and consequently should not have been in our sample, resulting in a 
total of 183 contracts in our review (see table 6). After accounting 
for these errors, we assessed the reliability of FPDS-NG data by 
electronically testing the data to identify problems with consistency, 
completeness, or accuracy and reviewed relevant documentation. We 
determined that the FPDS-NG data were sufficiently reliable for the 
purposes of our review. 

Table 6: Total Population and Sample Size for the New, Competitively 
Awarded Contracts in Fiscal Year 2013: 

Obligation strata: $25 million or more; 
Population: 193; 
Sample size: 93; 
In-scope: 86; 
Out-of-scope: 7. 

Obligation strata: Over $1 million and less than $25 million; 

Population: 2,658; 
Sample size: 134; 
In-scope: 97; 
Out-of-scope: 37. 

Obligation strata: Total; 
Population: 2,851; 
Sample size: 227; 
In-scope: 183; 
Out-of-scope: 44. 

Source: GAO analysis of DOD solicitation and contract documents and 
Federal Procurement Data System-Next Generation data. GAO-14-584. 

[End of table] 

Because we followed a probability procedure based on random selection, 
our sample is only one of a large number of samples that we might have 
drawn. Since each sample could have provided different estimates, we 
express our confidence in the precision of our particular sample's 
results as a 95 percent confidence interval (e.g., plus or minus 8 
percentage points). This is the interval that would contain the actual 
population value for 95 percent of the samples we could have drawn. 
Unless otherwise noted, percentage estimates of contracts with 
obligations of $25 million or more have 95 percent confidence intervals 
within +/-8 percentage points of the estimate itself. Similarly, for 
contracts with obligations over $1 million and less than $25 million, 
percentage estimates have confidence intervals within +/-10 percentage 
points of the estimate itself. 

In addition, to compare characteristics of contracts in our sample that 
used best value processes for both strata, we determined contract type, 
the type of procurement (product versus service), and if commercial 
item procedures were used for our sample using FPDS-NG data and 
conducted data reliability analysis on these fields, by verifying this 
information with the contract and solicitation documents. For the 
contracts identified as tradeoff, we analyzed the contract and 
solicitation documentation to identify the most frequently used non-
cost evaluation factors and their relative importance to price. 

To identify what factors DOD considers when choosing a source selection 
process, we analyzed the FAR, DFARS, and DOD and military departments' 
regulation, policy, and guidance on source selection. We interviewed 
senior DOD policy officials at Defense Procurement and Acquisition 
Policy, and at the Army, Navy, and Air Force headquarters. We also 
interviewed officials from at least two buying commands--based upon 
such factors as the number of contract actions and obligation amounts--
at each military department (Army, Navy, and Air Force), as well as 
the Defense Logistics Agency (DLA) to discuss factors affecting their 
decision process on which source selection process to use. In addition, 
we analyzed our sample of 183 contracts and selected 16 new, 
competitively awarded contracts with obligations ranging from $1.1 
million to $150.7 million to further our understanding of why 
acquisition officials chose the source selection process. Our 16 case 
studies--8 tradeoff and 8 LPTA--included at least 1 from each military 
department and DLA, different product and service types, and amount of 
dollars obligated in fiscal year 2013. For the case studies, we 
interviewed DOD contracting and program officials and reviewed contract 
documentation, including the acquisition plan, solicitation, and source 
selection decision memorandum to further understand the source 
selection decision making process. The results from our review of these 
selected contracts cannot be generalized beyond the specific contracts 
selected. 

During the course of our review, we also interviewed officials from the 
following commands: 

* Department of the Army, Army Contracting Command, Aberdeen Proving 
Ground, Maryland; Medical Command, Fort Detrick, Maryland; and 
Intelligence and Security Command, Fort Belvoir, Virginia: 

* Department of the Army, United States Army Corps of Engineers, 
Washington, D.C., and Huntsville Center, Alabama: 

* Department of the Navy, Naval Air Systems Command, Patuxent River, 
Maryland; Naval Facilities Command, Navy Yard, Washington, D.C.; and 
Naval Supply Systems Command, Mechanicsburg, Pennsylvania: 

* Department of the Navy, United States Marine Corps Installations and 
Logistics Command, Navy Annex, Virginia; and Marine Corps Systems 
Command, Quantico, Virginia: 

* Department of the Air Force, Installation Contracting Agency and Air 
Force Materiel Command, both located at Wright-Patterson Air Force 
Base, Ohio: 

* Defense Logistics Agency-Energy, Ft. Belvoir, Virginia; and Defense 
Logistics Agency-Troop Support, Philadelphia, Pennsylvania: 

* Joint Theater Support Contracting Command, Kabul, Afghanistan. 

To determine what training DOD provides to its acquisition personnel on 
source selection processes, we met with Defense Acquisition University 
(DAU) officials and instructors and reviewed training materials. We 
also obtained attendance and workforce data from the DOD Office of the 
Under Secretary of Defense (Acquisition, Technology, and Logistics), 
Human Capital Initiatives. Further, we collected and reviewed military 
department and command specific training documents to identify if 
additional source selection training is given in addition to DAU 
provided training. We also interviewed DOD policy officials at Defense 
Procurement and Acquisition Policy, several commands at the military 
departments, as well as contracting and program personnel at the 
contracting offices of the selected military departments from the 16 
case studies on training provided related to source selection 
processes. 

We supplemented these case studies with interviews with industry 
associations to identify their perspectives about DOD's source 
selection processes. 

We conducted this performance audit from September 2013 through July 
2014 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Body Armor Vest Acquisitions in Fiscal Year 2013: 

The Marine Corps, Defense Logistics Agency (DLA), and Army bought 
similar soft body armor vests made of ballistic material in fiscal year 
2013 using different source selection processes. Knowledge of 
requirements or vendors were key considerations in each acquisition, 
but distinct needs led to different decisions about which source 
selection process to use even when acquiring the similar product. 

* The Marine Corps issued one delivery order to purchase soft body 
armor vests for $2.3 million in fiscal year 2013 using the lowest price 
technically acceptable (LPTA) process. It issued this order from a 
multiple award, indefinite delivery indefinite quantity (IDIQ) 
contract[Footnote 18] awarded to two vendors in fiscal year 2009 using 
the LPTA process. The contracting officer told us they chose to use 
LPTA for the base contract because they consider soft body armor vests 
to be a commodity product with clearly defined technical performance 
specifications. Further, the contracting officer, in consultation with 
the program office, saw no opportunity for tradeoff above industry 
standard because the industry standard met their current needs. Ongoing 
research and development showed that any tradeoff for enhanced 
performance would lead to the armor being heavier, an unacceptable 
outcome. For the base contracts, the Marine Corps awarded to the second 
and third lowest priced vendors, because the lowest priced vendor was 
deemed non-responsible.[Footnote 19] 

* DLA issued 23 delivery orders to purchase soft body armor vests for 
$288.1 million in fiscal year 2013. It issued these orders from three 
separate IDIQ contracts awarded to three vendors in fiscal years 2011 
and 2012 using the tradeoff process. DLA contracting officials told us 
that they chose to use the tradeoff process for these contracts because 
they wanted to use past performance as a key discriminator, which is 
generally not allowed using the LPTA process. Further, because DLA buys 
for sustainment purposes and its quantity needs fluctuate, officials 
told us that past performance was a critical determination factor 
requiring the use of the tradeoff process, in addition to the vendor's 
historic production capacity, delivery schedule, and other performance 
capabilities. 

* The Army issued one delivery order to purchase soft body armor vests 
for $10,201 in fiscal year 2013 using the LPTA process. It issued this 
order from one of the multiple award, IDIQ contracts awarded to eight 
vendors in fiscal years 2009 and 2010 using the tradeoff process. Army 
contracting officials told us they chose to use the tradeoff process 
for the base contract, because it provided the Army more discretion in 
evaluating past performance as well as leaving open the possibility 
that industry vendors might offer a more innovative solution. Once the 
Army had a group of qualified vendors on contract, they could then use 
the LPTA process for subsequent buys. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Timothy J. DiNapoli, 202-512-4841 or dinapolit@gao.gov: 

Staff Acknowledgments: 

In addition to the contact name above, the following staff members made 
key contributions to this report: Molly Traci, Assistant Director; 
James Kim; Anh Nguyen; Erin Stockdale; Jina Yu; Claire Li; Jessica 
Drucker; Danielle Greene; Roxanna Sun; John Krump; Mark Ramage; Julia 
Kennon; Virginia Chanley; and Carol Petersen. 

[End of section] 

Footnotes: 

[1] See S. Rep. No. 113-44, at 142 (2013), H.R. Rep. No. 113-102, at 
185 (2013), and 159 Cong. Rec. H7894, H7922 (December 12, 2013). 

[2] FPDS-NG is the federal government's system for tracking information 
on contracting actions. 

[3] DFARS § 207.103(d)(i)(B). 

[4] GAO, Defense Contracting: Enhanced Training Could Strengthen DOD's 
Best Value Tradeoff Decisions, [hyperlink, 
http://www.gao.gov/products/GAO-11-8] (Washington, D.C.: Oct. 28, 
2010). 

[5] For the purposes of this report, non-competed contracts consists of 
those contracts awarded under other than full and open competition as 
defined in Federal Acquisition Regulation subpart 6.3. 

[6] [hyperlink, http://www.gao.gov/products/GAO-11-8]. Estimates of 
contracts based in this report were subject to sampling error. For 
example, the percentage estimates for contracts with obligations of 
$25 million or more have 95 percent confidence intervals within +/-8 
percentage points of the estimate itself. 

[7] FAR § 14.101(e). 

[8] All dollar values are in nominal terms. Values for fiscal year 2013 
are based on FPDS-NG data as of January 2014. 

[9] Sealed bids represented 12 of the 183 contracts in our sample. 

[10] The percentage estimates for contracts with obligations of $25 
million or more have a 95 percent confidence interval with a +/-8 
percent margin of error. The percentage estimates for contracts with 
obligations over $1 million and less than $25 million have a 95 percent 
confidence interval with a +/-10 percent margin of error. 

[11] The majority of these contracts were firm-fixed-price. Other 
contract types for the remaining contracts included fixed-price with 
economic price adjustment and cost-plus-fixed-fee. 

[12] [hyperlink, http://www.gao.gov/products/GAO-11-8]. 

[13] Pub. L. No. 101-510, § 1202(a) (1990). 

[14] GAO, Defense Acquisition Workforce: Better Identification, 
Development, and Oversight Needed for Personnel Involved in Acquiring 
Services, [hyperlink, http://www.gao.gov/products/GAO-11-892] 
(Washington, D.C.: Sept. 28, 2011). 

[15] See S. Rep. No. 113-44, at 142 (2013), H.R. Rep. No. 113-102, at 
185 (2013), and 159 Cong. Rec. H7894, H7922 (December 12, 2013). 

[16] DFARS § 207.103(d)(i)(B). 

[17] GAO, Defense Contracting: Enhanced Training Could Strengthen DOD's 
Best Value Tradeoff Decisions, [hyperlink, 
http://www.gao.gov/products/GAO-11-8] (Washington, D.C.: Oct. 28, 
2010). 

[18] Agencies may award more than one contract to more than one vendor 
(known as a multiple award contract) from a solicitation. An IDIQ 
contract is a type of indefinite delivery contract that provides for an 
indefinite quantity of supplies or services within stated limits, 
during a fixed period. 

[19] According to the Federal Acquisition Regulation (FAR), Prospective 
Contractors, to be determined responsible, must have among other 
things, adequate financial resources, be able to comply with the 
required or proposed delivery or performance schedule, have a 
satisfactory performance record, and have a satisfactory record of 
integrity and business ethics. FAR § 9.104-1. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO's commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
"E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]