This is the accessible text file for GAO report number GAO-11-740 
entitled 'Aviation Security: TSA Has Enhanced Its Explosives Detection 
Requirements for Checked Baggage, but Additional Screening Actions Are 
Needed' which was released on July 12, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

July 2011: 

Aviation Security: 

TSA Has Enhanced Its Explosives Detection Requirements for Checked 
Baggage, but Additional Screening Actions Are Needed: 

GAO-11-740: 

GAO Highlights: 

Highlights of GAO-11-740, a report to congressional requesters. 

Why GAO Did This Study: 

Explosives represent a continuing threat to aviation security. The 
Transportation Security Administration (TSA), within the Department of 
Homeland Security (DHS), seeks to ensure through the Electronic 
Baggage Screening Program (EBSP) that checked-baggage-screening 
technology is capable of detecting explosives. Generally, the 
explosives detection system (EDS) is used in conjunction with 
explosives trace detection (ETD) machines to identify and resolve 
threats in checked baggage. As requested, GAO assessed the extent to 
which: (1) TSA revised explosives detection requirements and deployed 
technology to meet those requirements, and (2) TSA’s approach to the 
current EDS acquisition meets best practices for schedules and cost 
estimates and includes plans for potential upgrades of deployed EDSs. 
GAO analyzed EDS requirements, compared the EDS acquisition schedule 
against GAO best practices, and interviewed DHS officials. This is a 
public version of a sensitive report that GAO issued in May 2011. 

What GAO Found: 

TSA revised EDS explosives detection requirements in January 2010 to 
better address current threats and plans to implement these 
requirements in a phased approach. The first phase, which includes 
implementation of the previous 2005 requirements, is to take years to 
fully implement. However, deploying EDSs that meet 2010 requirements 
could prove difficult given that TSA did not begin deployment of EDSs 
meeting 2005 requirements until 4 years later in 2009. As of January 
2011, some number of the EDSs in TSA’s fleet are detecting explosives 
at the level established in 2005. The remaining EDSs in the fleet are 
configured to meet the 1998 requirements because TSA either has not 
activated the included software or has not installed the needed 
hardware and software to allow these EDS to meet the 2005 
requirements. Developing a plan to deploy and operate EDSs to meet the 
most recent requirements could help ensure EDSs are operating most 
effectively and should improve checked-baggage screening. However, TSA 
has faced challenges in procuring the first 260 EDSs to meet 2010 
requirements. For example, due to the danger associated with some 
explosives, TSA and DHS encountered challenges in developing simulants 
and collecting data on the explosives’ physical and chemical 
properties needed by vendors and agencies to develop detection 
software and test EDSs prior to the current acquisition. Also, TSA’s 
decision to pursue EDS procurement during data collection complicated 
both efforts and resulted in a delay of over 7 months for the current 
acquisition. Completing data collection for each phase of the 2010 
requirements prior to pursuing EDS procurements that meet those 
requirements could help TSA avoid additional schedule delays. 

TSA has established a schedule for the current EDS acquisition, but it 
does not fully comply with best practices, and TSA has not developed a 
plan to upgrade its EDS fleet. For example, the schedule is not 
reliable because it does not reflect all planned program activities 
and does not include a timeline to deploy EDSs or plans to procure 
EDSs to meet subsequent phases of the 2010 requirements. Developing a 
reliable schedule would help TSA better monitor and oversee the 
progress of the EDS acquisition. TSA officials stated that to meet the 
2010 requirements, TSA will likely upgrade many of the current fleet 
of EDSs as well as the first 260 EDS machines to be purchased under 
the current acquisition. However, TSA has no plan in place outlining 
how it will approach these upgrades. Because TSA is implementing the 
2010 requirements in a phased approach, the same EDS machines may need 
to be upgraded multiple times. TSA officials stated that they were 
confident the upgrades could be completed on deployed machines. 
However, without a plan, it will be difficult for TSA to provide 
reasonable assurance that the upgrades will be feasible or cost-
effective. 

What GAO Recommends: 

GAO recommends that TSA, among other things, develop a plan to ensure 
that new machines, as well as those machines currently deployed in 
airports, will be operated at the levels in established requirements, 
collect explosives data before initiating new procurements, and 
develop a reliable schedule for the EBSP. DHS concurred with all of 
GAO’s recommendations and has initiated actions to implement them. 

View [hyperlink, http://www.gao.gov/products/GAO-11-740] or key 
components. For more information, contact Steve Lord at (202) 512-8777 
or lords@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

TSA Has Revised Explosives Detection Requirements for Checked-Baggage- 
Screening Systems, but Faces Challenges in Deploying Equipment to Meet 
the Requirements: 

Improved Acquisition Planning Could Help TSA Avoid Further Delays and 
Potential Cost Overruns for the EDS Procurement: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Department of Homeland Security Efforts to Conduct 
Computer Modeling to Establish a Tier of the Explosives Detection 
Requirements for the Explosives Detection System: 

Appendix III: GAO's Assessment of the Extent to Which the Electronic 
Baggage-Screening-Program Schedule Meets Established Best Practices: 

Appendix IV: Comments from the Department of Homeland Security: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Implementation of Best Practices in Electronic-Baggage- 
Screening Program Schedule as of December 2010: 

Table 2: Assessment of the Extent to Which the Electronic Baggage- 
Screening-Program Schedule Meets Best Practices: 

Figures: 

Figure 1: Stand-alone Explosives Detection System (EDS): 

Figure 2: Explosives Trace Detection (ETD) Machine: 

Figure 3: TSA's Plan to Deploy EDSs That Meet One Tier of the 2010 
Requirements in Phases: 

Abbreviations: 

AFB: Air Force Base: 

AFRL: Air Force Research Laboratory: 

ATSA: Aviation and Transportation Security Act: 

CAD: Cost Analysis Division: 

CRT: certification readiness testing: 

CT: computed tomography: 

DHS: Department of Homeland Security: 

DOD: Department of Defense: 

EBSP: Electronic Baggage Screening Program: 

EDS: explosives detection system: 

EXD: Explosives Division: 

ETD: explosives trace detection: 

FAR: Federal Acquisition Regulation: 

IMS: integrated master schedule: 

IPT: integrated product team: 

LCCE: lifecycle cost estimate: 

OT&E: operational test and evaluation: 

SRA: schedule risk analysis: 

S&T: Science and Technology Directorate: 

TSA: Transportation Security Administration: 

TSIF: TSA Systems Integration Facility: 

TSL: Transportation Security Laboratory: 

WBS: work breakdown structure: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

July 11, 2011: 

The Honorable John L. Mica:
Chairman:
Committee on Transportation and Infrastructure:
House of Representatives: 

The Honorable Charles W. Dent:
House of Representatives: 

Explosives represent a continuing threat to aviation security, 
according to the Department of Homeland Security (DHS). The 
Transportation Security Administration (TSA), the agency within DHS 
responsible for overseeing and ensuring civil aviation security, is 
seeking to ensure that the technology it uses to screen checked 
baggage is capable of detecting these threats. 

TSA's Electronic Baggage Screening Program (EBSP), one of the largest 
acquisition programs within DHS, certifies and acquires systems used 
to screen checked baggage at 462 commercial airports throughout the 
United States.[Footnote 1] TSA certifies explosives detection-
screening technologies to ensure they meet explosives detection 
requirements developed in conjunction with the DHS Science and 
Technology Directorate (S&T) along with input from other agencies, 
such as the Federal Bureau of Investigation and Department of Defense 
(DOD). S&T conducts research and development of new technologies while 
its Transportation Security Laboratory (TSL) is responsible for, among 
other things, independent test and evaluation of new technologies, 
including conducting certification testing of checked-baggage- 
screening technologies. 

Pursuant to the Aviation and Transportation Security Act (ATSA), 
enacted in November 2001, TSA deploys explosives detection systems 
(EDS) and explosives trace detection (ETD) machines to screen all 
checked baggage transported by U.S. and foreign air carriers departing 
from U.S. commercial airports. An EDS machine uses computed tomography 
(CT) technology to automatically measure the physical characteristics 
of objects in baggage. The system automatically triggers an alarm when 
objects that exhibit the physical characteristics of explosives are 
detected. An ETD machine is used to chemically analyze trace materials 
after a human operator swabs checked baggage to identify any traces of 
explosive material. 

In January 2010, TSA revised explosives detection requirements for the 
EDS (hereinafter referred to as "2010 EDS requirements") to better 
address current threats. The specific details included in the 2010 EDS 
requirements such as the physical characteristics and minimum masses 
of each of the explosives types that EDS machines must detect are 
classified. As highlighted in TSA's 2010 EBSP acquisition strategy, to 
improve its existing checked-baggage-screening capability, TSA plans 
to procure 260 EDSs (that is, the current acquisition) capable of 
meeting the 2010 EDS requirements. In addition to acquiring new EDSs 
to better address current threats, TSA is planning to procure new EDSs 
because many of the currently deployed machines are nearing the end of 
their expected service lives. 

You requested that we review TSA's efforts to enhance explosives 
detection requirements for checked-baggage-screening technologies and 
to ensure that newly acquired and currently deployed explosives 
detection technologies meet the enhanced requirements. Specifically, 
this report assesses (1) the extent to which TSA has revised 
explosives detection requirements and deployed EDSs and ETDs to meet 
these revised requirements; (2) any challenges that TSA and S&T have 
experienced in implementing the EDS acquisition; and (3) the extent to 
which TSA's approach to its EDS acquisition meets best practices for 
schedule and cost estimates and includes plans for potential upgrades 
to deployed EDSs. 

This report is a public version of the prior sensitive report that we 
provided to you in May 2011. DHS deemed some of the information in the 
prior report as sensitive, which must be protected from public 
disclosure. Therefore, this report omits certain sensitive information 
about EDS and ETD explosives detection requirements, including 
descriptions of those requirements as well as timeframes for their 
implementation, the number of EDSs meeting the requirements, and 
challenges associated with data collection for explosives. This report 
addresses the same questions as the sensitive report. Also, the 
overall methodology used for both reports is the same. 

To determine the status of TSA's efforts to revise explosives 
detection requirements for checked baggage screening, we reviewed 
TSA's explosives detection requirements for EDSs established in 
January 2010 and compared them to explosives detection requirements 
previously established in 2005 and 1998 to determine how the 
requirements differed. We also identified, analyzed, and discussed 
with TSA officials how the 2010 EDS requirements compare with current 
explosives detection requirements for the ETD established in 2006. We 
also visited three of the Department of Energy's national 
laboratories--Lawrence Livermore National Laboratory, Los Alamos 
National Laboratory, and Sandia National Laboratories--to determine 
the status of additional efforts to further revise the requirements. 

To identify any challenges that TSA is experiencing in implementing 
the EDS acquisition, we reviewed documentation from TSA's EBSP--the 
program responsible for operational testing, procurement, deployment, 
and maintenance of checked-baggage-screening technologies. Among other 
things, we reviewed the 2010 EBSP acquisition strategy, the program's 
risk management plan, the technical specifications for its procurement 
of new EDSs, and DHS acquisition guidance and directives. We conducted 
multiple interviews with EBSP program officials regarding the 
program's approach to the current EDS acquisition and received updates 
on revisions to TSA's planned approach to the acquisition and 
timelines. We also compared TSA's acquisition efforts with internal 
control standards.[Footnote 2] In addition, we conducted site visits 
and/or telephone interviews with all six EDS vendors competing in the 
current EDS procurement and also obtained their views regarding TSA's 
approach to the competitive procurement. While information we obtained 
from these interviews may not be generalized across the industry as a 
whole, we were able to obtain the perspectives of all companies 
planning to compete for the current EDS procurement, and they were 
able to provide an understanding of their companies' abilities to 
develop EDSs that meet the 2010 EDS requirements. 

To assess the extent to which TSA's approach for its EDS acquisition 
meets best practices for schedule and cost estimates, and includes 
plans for potential upgrades to deployed EDSs, we assessed the 
original and revised schedule for the current EDS acquisition against 
relevant best practices in our Cost Estimating and Assessment Guide to 
determine the extent to which the schedule reflects key estimating 
practices that are fundamental to having a reliable schedule.[Footnote 
3] We compared TSA's efforts with recommended practices that we 
previously identified for sound acquisition planning.[Footnote 4] We 
also conducted interviews with DHS officials and conducted site visits 
to S&T's TSL to obtain additional perspective on TSA's efforts to 
deploy EDSs that meet the 2010 requirements. Further, we visited 
Tyndall Air Force Base (AFB), Florida, where the Air Force Research 
Laboratory, TSL, and the Department of Energy's Lawrence Livermore 
National Laboratory are assisting S&T's Explosives Division's data 
collection efforts. Regarding TSA's planning for potential upgrades to 
already deployed EDSs, we interviewed TSA and S&T officials to 
identify the explosives detection technologies that are currently used 
for checked-baggage-screening. We also identified the number of 
currently deployed EDSs that meet the 1998 and 2005 EDS explosives 
detection requirements, the number of currently deployed ETD that meet 
the 2002 and 2006 ETD requirements, any challenges involved and 
expected in upgrading EDS detection capabilities, and TSA's plans to 
upgrade EDSs to meet its 2010 requirements. Additionally, in our site 
visits and telephone interviews with the six vendors as we previously 
discussed, we asked vendors to also provide their perspectives on 
TSA's approach to upgrading currently deployed EDSs as well as those 
to be deployed in the future. 

We conducted this performance audit from September 2009 through May 
2011 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
During the course of our review, we revised the engagement objectives 
and scope to facilitate a broader examination of TSA's efforts to 
revise its explosives detection requirements and related schedule for 
the EDS acquisition, a revision that increased the time for completing 
this audit. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. Appendix I contains additional information on the 
objectives, scope, and methodology of our review. 

Background: 

The Roles of TSA and S&T: 

After the September 11, 2001, terrorist attacks, the Aviation and 
Transportation Security Act (ATSA) was enacted.[Footnote 5] Among 
other things, ATSA required that TSA provide for the screening of all 
checked baggage for explosives transported on flights departing U.S. 
commercial airports.[Footnote 6] Pursuant to ATSA, TSA deployed EDS 
and ETD equipment to screen checked baggage and identify potential 
threats from explosives.[Footnote 7] While TSA is responsible for 
operating or overseeing the operation of checked-baggage-screening 
equipment, TSA and S&T share responsibilities for the research and 
development of checked-baggage-screening technologies. During fiscal 
year 2006, most research and development functions within DHS, 
including TSA, were consolidated within S&T.[Footnote 8] After this 
consolidation, S&T assumed primary responsibility for the research, 
development, and related test and evaluation of airport checked-
baggage-screening technologies. S&T also assumed responsibility from 
TSA for the TSL, which tests and evaluates technologies under 
development against TSA-established detection requirements. TSA 
continues to be responsible for: identifying the requirements for new 
checked-baggage-screening technologies; operationally testing and 
evaluating technologies in airports; and procuring, deploying, and 
maintaining technologies. TSA relies on S&T as the central 
coordination point to manage all work related to explosives that 
involve the TSL, Lawrence Livermore National Laboratory, Los Alamos 
National Laboratory, Sandia National Laboratories, and the Air Force 
Research Laboratory at Tyndall AFB. 

Deploying EDSs and ETDs: 

According to TSA, since fiscal year 2001, TSA has made over $8 billion 
available to EBSP for activities related to checked-baggage screening. 
TSA uses two types of technology for checked baggage screening--the 
EDS and the ETD--at 462 U. S. commercial airports. EDS is used to 
identify suspicious bulk items or anomalies in checked baggage that 
could be explosives or detonation devices. In airports that have EDS, 
it is used for primary screening of checked baggage while ETD machines 
are used for secondary screening of checked baggage to help resolve 
EDS alarms. Additionally, at airports without EDS, the ETD machines 
are used for primary screening of checked baggage. See figure 1 for a 
photograph of an EDS machine and figure 2 for a photograph of an ETD 
machine. 

Figure 1: Stand-alone Explosives Detection System (EDS): 

[Refer to PDF for image: photograph] 

Source: GAO. 

[End of figure] 

Figure 2: Explosives Trace Detection (ETD) Machine: 

[Refer to PDF for image: photograph] 

Source: TSA. 

[End of figure] 

TSA deploys EDSs in multiple configurations, such as an in-line 
configuration and a stand-alone configuration. The in-line 
configuration integrates EDS with an airport's baggage handling system-
-the conveyor system that sorts and transports baggage for loading 
onto an aircraft. EDS in stand-alone configurations are separate 
baggage screening units that are not integrated with a baggage-
handling system and are typically located in an airport lobby although 
they may also be located in other airport locations. Checked baggage 
is manually loaded and unloaded on stand-alone EDS machines. 

As of October 2010, TSA had 2,297 EDS machines in its fleet, 1,938 of 
which were deployed at airports in the United States.[Footnote 9] At 
airports and terminals that do not use EDSs, ETD machines are used for 
primary checked-baggage screening. Typically, ETDs are used for 
primary screening of checked baggage at smaller airports. These 
airports typically do not have EDSs for primary screening of checked 
baggage. As of February 2011, TSA estimated that there were about 
5,200 ETD machines used for the primary or secondary screening of 
checked baggage at U.S. commercial airports. 

S&T and TSA's Certification and Testing Process: 

TSA certifies the EDS it deploys to commercial airports for screening 
checked baggage, based on tests performed by the TSL.[Footnote 10] 
Specifically, TSA certifies that EDSs, alone or as part of an 
integrated system, can detect, under realistic operating conditions, 
the amounts, configurations, and types of explosive material which 
would be likely to be used to cause catastrophic damage to an 
aircraft, using requirements developed in consultation with experts 
from outside TSA.[Footnote 11] Furthermore, TSA periodically reviews 
threats to civil aviation security, including: 

* explosive material that presents the most significant threat to 
civil aircraft; 

* the minimum amounts, configurations, and types of explosive material 
that can cause, or would be expected to cause, catastrophic damage to 
aircraft in air transportation; and: 

* the amounts, configurations, and types of explosive material that 
can be detected reliably by existing or reasonably anticipated, near-
term explosive detection technologies.[Footnote 12] 

Currently, TSA requires that EDSs undergo three types of testing-- 
certification testing, integration testing, and operational testing-- 
before it will purchase such equipment. First, TSA verifies that 
vendors' explosives detection systems meet--that is, are capable of 
detecting in accordance with--the TSA established explosives detection 
requirements through the certification testing process. TSA's decision 
to certify an EDS relies on the results of independent test and 
evaluation performed at the TSL. Prior to certification testing, TSL 
conducts preliminary evaluations of vendors' EDS, known as 
certification readiness testing (CRT) and pre-certification, to 
determine the extent to which vendors are ready to enter certification 
testing. During CRT, TSL provides feedback to vendors on their EDS's 
strengths and weaknesses in detecting explosives in order to help 
vendors make necessary adjustments to their detection software. 
Second, in addition to being certified that the EDS can meet 
explosives detection requirements, EDSs being deployed in an in-line 
configuration must also undergo integration testing. As part of this 
testing, machines deployed in an in-line configuration must 
demonstrate in a controlled environment that they can be successfully 
integrated within the baggage-handling systems used for checked 
baggage. Finally, following certification and integration testing, 
EDSs undergo operational testing in an airport setting to demonstrate 
that they can reliably and effectively function in a live airport 
environment. 

TSA Has Revised Explosives Detection Requirements for Checked-Baggage- 
Screening Systems, but Faces Challenges in Deploying Equipment to Meet 
the Requirements: 

In 2005, TSA revised explosives detection requirements for the EDS; 
however, some number of the EDSs are currently operating at the levels 
to detect explosives as set forth only in the 2005 
requirements.[Footnote 13] When TSA established the 2005 requirements, 
it did not have a plan that identified the appropriate time frames 
needed to deploy EDSs to meet the requirements. In January 2010, TSA 
again revised the EDS explosives detection requirements and plans to 
deploy EDSs meeting these requirements in a tiered and phased approach 
over a number of years. One tier of requirements consists of three 
levels and expanded the number and types of explosives that EDSs must 
detect.[Footnote 14] TSA is in the process of developing another tier 
of requirements, which will refine the amount (for example, minimum 
mass) of an explosive that can cause catastrophic damage to an 
aircraft. If TSA deploys EDSs that fully meet the one tier of the 
requirements, TSA must ensure that ETD machines are capable of 
detecting all of the explosives that EDSs will be able to detect to 
minimize any potential screening difference between the EDS and ETD. 

TSA Did Not Establish a Plan to Ensure That Currently Deployed EDSs 
Meet the 2005 Requirements: 

In November 2005, TSA revised its explosives detection requirements 
for EDS that had previously been established in 1998 by the Federal 
Aviation Administration. However, as of January 2011, some number of 
the EDSs in TSA's fleet are configured to detect explosives at the 
levels established only in the 2005 requirements. The remaining EDSs 
are configured to detect explosives at 1998 levels. When TSA 
established the 2005 requirements, it did not have a plan with the 
appropriate time frames needed to deploy EDSs to meet the 
requirements. Standard practices for program and project management 
state that specific desired outcomes or results should be 
conceptualized, defined, and documented in the planning process as 
part of a road map, along with the appropriate steps, time frames, and 
milestones needed to achieve those results. 

Despite the absence of a plan, TSA officials stated that they must 
conduct testing to compare the false alarm rates for machines 
operating at one level of requirements to those operating at another 
level of requirements.[Footnote 15] According to TSA officials, the 
results of this testing would allow them to determine if additional 
staff are needed at airports to help resolve false alarms once the 
EDSs are configured to operate at a certain level of requirements. 
[Footnote 16] TSA officials reported that they had anticipated this 
operational testing was to be completed in March 2011. 

According to agency officials, TSA did not begin this operational 
testing immediately after the previous explosives detection 
requirements were established in November 2005 because the agency 
officials were aware at the time of a potential further revision of 
the requirements based upon a planned computer modeling effort to 
revise the detection standards that became known as Project Newton. 
TSA and S&T officials told us they had planned to use the results from 
Project Newton to further revise the explosives detection requirements 
to reflect the mass of an explosive that would cause catastrophic 
damage to an aircraft. Although Project Newton did not begin until 
2007, TSA officials told us that they were aware of plans to further 
revise the requirements prior to the initiation of Project Newton and 
delayed operational testing in anticipation of the results of the 
computer modeling effort. As of April 2011, the EDS explosives 
detection requirements have not been changed based on results of the 
computer modeling because Project Newton is still under way, though 
TSA officials told us that they plan to use the results of Project 
Newton to later define a tier of the 2010 EDS requirements. (We 
discuss the status of Project Newton in more detail in appendix II.) 

However, once it became apparent that the results of Project Newton 
would not become available to further revise the requirements, TSA did 
not establish a plan with time frames for completing the additional 
testing related to staffing.[Footnote 17] Standard practices for 
planning and project management suggest TSA should have defined the 
operational testing plan and milestones as part of a road map for 
assessing potential staffing changes when these EDSs were first 
deployed after the 2005 requirements were established. Establishing 
reasonable time frames to complete the operational testing could help 
TSA ensure it achieves its desired goal of activating EDSs capable of 
detecting the explosives established in the 2005 requirements in a 
timely manner. 

TSA Plans to Deploy EDSs That Meet the 2010 Explosives Detection 
Requirements Using a Phased Approach: 

In January 2010, TSA revised the EDS checked-baggage explosives 
detection requirements partly in response to credible and immediate 
threats to civil aviation. TSA plans to meet the 2010 EDS requirements 
using a tiered and phased approach: one tier is to be implemented over 
a number of years and expands the types of explosives that EDSs must 
detect, and another tier adds the use of the results of the computer- 
modeling effort known as Project Newton. TSA is not certain when this 
tier of requirements will be implemented. TSA plans to deploy EDSs 
that meet one tier of the 2010 requirements in a phased approach 
beginning in late fiscal year 2011 as part of its planned EDS 
acquisition. Regarding the other tier, TSA officials stated that 
Project Newton is to enhance TSA's understanding of explosives effects 
by simulating hundreds of explosives tests using computer modeling to 
determine the effects explosives will have when placed in different 
locations within an aircraft. TSA's and S&T's understanding of how 
explosives affect aircraft has largely been based on data obtained 
from live-fire explosive tests on retired aircraft hulls and other 
data. Project Newton is jointly managed and funded by DHS's S&T and 
TSA. Through fiscal year 2009, S&T and TSA had invested about $12.5 
million in national laboratories for computer modeling activities as 
part of Project Newton, according to a senior TSA official. We discuss 
Project Newton and its budget in more detail in appendix II. 

TSA plans to implement one tier of requirements in a phased approach 
that consists of three levels (see fig. 3).[Footnote 18] As our past 
work has shown, an incremental or phased approach to implementing 
requirements can reduce risk and make a program more achievable by 
providing more time to develop and test key technologies.[Footnote 19] 
TSA officials told us that the ability to develop EDS is likely to 
become increasingly complex as the implementation of requirements 
progresses. According to TSA, it expects to begin procuring EDSs to 
meet the 2010 requirements in July 2011. Consequently, if TSA is 
successful in deploying EDSs that meet all three levels of the 2010 
requirements, TSA's EDS fleet would be certified to detect more 
explosives than a fleet meeting the 1998 or 2005 requirements. 

Figure 3: TSA's Plan to Deploy EDSs That Meet One Tier of the 2010 
Requirements in Phases: 

[Refer to PDF for image: illustration] 

Level C: 
EDS procurement estimated to begin July 2011. 

Level B: 
Requirements to be implemented over a number of years. 

Level A: 
Requirements to be implemented over a number of years. 

Source: GAO analysis of TSA data. 

[End of figure] 

Deploying EDSs That Meet Subsequent Levels of the 2010 Requirements 
Could Affect How EDS Alarms Are Resolved: 

ETD machines are currently certified to detect some different 
explosives than those EDSs that meet the 2005 EDS detection 
requirements. However, if TSA purchases and deploys EDSs that fully 
meet Level C of the 2010 EDS requirements, ETD machines are not 
required to detect all of the explosives that can be detected by EDSs. 
TSA's existing ETD explosives detection requirements identify the 
types and quantities of explosives materials, that is, traces of 
explosives, that must be detected and the minimum detection rate for 
each category of explosive. 

According to TSA officials, the ETD explosives detection requirements 
have not been revised because TSA wanted to first focus on revising 
the EDS explosives detection requirements in time for its planned EDS 
acquisition, which is aimed at replacing and upgrading its fleet of 
EDSs used to screen checked baggage. TSA officials stated that they 
are developing a combined set of explosives detection requirements 
that could eventually result in the ETD and EDS machines detecting the 
same explosives. TSA officials stated that the combined set of EDS and 
ETD requirements would not be expected to be approved until sometime 
in calendar year 2011. Although combined detection requirements for 
ETD and EDS are to help ensure that both machines can detect the same 
explosives, the machines are not expected to be required to detect the 
same amounts of explosives because the purpose of the ETD is to detect 
traces of explosives in nanograms while the EDS is designed to detect 
larger amounts of explosives. 

At all airports that use EDSs to screen checked baggage, ETD machines 
are used in conjunction with EDSs to screen checked baggage for 
explosives. At these airports, if an EDS alarms--indicating that 
checked baggage may contain an explosive or explosive device that 
cannot be cleared--ETD machines are used as a secondary screening 
device in order to attempt to resolve the alarm. However, the 
differences between the EDS and ETD requirements may impact the 
resolution of EDS alarms by the ETD in the future. According to TSA's 
2010 EBSP Acquisition Strategy, additional equipment--other than the 
ETDs currently deployed--is to be employed to support alarm resolution 
when EDSs that meet the new checked-baggage explosives detection 
requirements are deployed. However, the acquisition strategy does not 
specify what additional equipment or screening protocols will be 
employed to resolve alarms nor does the strategy discuss whether TSA 
will continue to use ETD equipment to resolve EDS alarms.[Footnote 20] 
According to TSA, the agency is currently evaluating what additional 
technologies and/or changes to screening protocols may be needed to 
address any potential gap in capability between the newly certified 
EDSs and ETDs used for alarm resolution. If TSA begins operating EDSs 
that detect explosives in subsequent phases of the 2010 requirements, 
this potential screening difference between EDS and ETD will exist 
until TSA deploys additional equipment and/or implements new screening 
protocols that could be used for secondary screening. 

In commenting on this issue, TSA officials stated that checked-baggage-
screening technologies are only one layer of security and that other 
layers of security exist to help address potential threats to the 
aviation security system. However, officials agreed that they have not 
yet developed new screening protocols or deployed additional equipment 
that will address the potential gap in screening capability between 
EDS and ETD if the new EDSs are deployed. Standards for program 
management require that specific desired outcomes or results be 
conceptualized, defined, and documented in the planning process as 
part of a road map, along with the appropriate steps and time frames 
needed to achieve those results.[Footnote 21] Because TSA decided to 
revise explosives detection requirements for EDSs prior to revising 
the ETD requirements, the differences in the requirements may affect 
TSA's capability to detect the 2010-required levels until TSA 
identifies technologies or protocols needed to address the potential 
gap. Without a plan to ensure that secondary-screening devices or 
protocols are in place to resolve EDS alarms if EDSs are deployed with 
additional capability, it will be difficult for TSA to provide 
assurances that the potential capability gap has been resolved. 

TSA Has Faced Challenges in Implementing Plans for the Current EDS 
Acquisition: 

TSA has developed an EBSP acquisition strategy to guide its efforts to 
improve its fleet of checked-baggage-screening machines, but has faced 
several challenges in implementing plans for the current EDS 
acquisition under this strategy. First, TSA has experienced challenges 
in collecting explosives data on the physical and chemical properties 
of certain explosives needed by vendors to develop EDS detection 
software and needed by TSA before procuring and deploying EDSs to meet 
the 2010 requirements. TSA and S&T have experienced these challenges 
because of problems associated with safely handling and consistently 
formulating some explosives. Second, the challenges related to data 
collection for certain explosives have resulted in problems carrying 
out the EDS procurement as planned. Specifically, attempting to 
collect data for certain explosives while simultaneously pursuing the 
EDS procurement has delayed the EDS acquisition schedule by at least 7 
months. Finally, EDS vendors have expressed concerns about the extent 
to which TSA is communicating with the business community about the 
current EDS procurement. 

TSA's EBSP Acquisition Strategy Is Being Used to Guide Efforts to 
Improve the Explosive Detection Capabilities of the EDS: 

In July 2010, DHS approved TSA's current acquisition strategy for the 
EDS, and under this strategy, TSA plans to increase the threat 
detection capabilities of the EDS using a competitive procurement to 
purchase and deploy EDS beginning in 2011. According to TSA officials, 
most of the previous EDS acquisitions were sole source procurements. 
[Footnote 22] However, TSA is implementing a competitive procurement 
for the current EDS acquisition to, in part, meet the EBSP acquisition 
strategy's goals and objectives.[Footnote 23],[Footnote 24] 
Furthermore, the EBSP acquisition strategy calls for acquiring new 
EDSs as part of the recapitalization plan to replace aging EDS. Under 
the current EDS procurement, TSA plans to award contracts to purchase 
260 EDSs, including those in its recapitalization plan, at an 
estimated cost of $256 million during the fourth quarter of fiscal 
year 2011.[Footnote 25] Until TSA begins purchasing machines under the 
current EDS acquisition to meet 2010 requirements, the agency has 
continued to purchase EDSs under existing contracts with current 
vendors. 

TSA Faces Challenges in Collecting Explosives Data Needed to Procure 
EDSs That Meet the 2010 Requirements: 

TSA and S&T have experienced a number of challenges related to 
collecting data on some explosives data needed to procure and deploy 
EDSs that meet the 2010 requirements. These data are needed both by 
vendors to develop EDS detection software and by the TSL for the 
certification testing process and includes such information as the 
physical and chemical properties of explosives. Participants in the 
data collection effort include S&T; the TSL, which is taking the lead; 
the Air Force Research Laboratory (AFRL) at Tyndall AFB, Florida; and 
Lawrence Livermore National Laboratory. The AFRL is assisting the TSL 
because the AFRL facility at Tyndall AFB, Florida, is better equipped 
to safely handle certain explosives as part of the testing and data 
collection efforts than the TSL facility, according to S&T and Air 
Force officials[Footnote 26].: 

In the course of collecting data, TSL and AFRL officials determined 
that some of the explosives were very unstable or volatile and special 
care and procedures were required to reliably and safely handle them. 
This caused delays as TSA and S&T were unable to provide vendors with 
all of the data on the explosives or simulants, so that vendors could 
create and test the software used to detect them.[Footnote 27] TSL and 
AFRL also collected scans of explosives as another means to provide 
vendors with needed data. Because micro-computed technology (micro-CT) 
data are provided by scanning smaller, less dangerous, amounts of each 
explosive, TSL and AFRL officials did not face the same challenges in 
safely synthesizing and determining the physical and chemical 
properties of the micro-CT data that they faced in working with larger 
amounts, known as full threat weight, of explosives. TSL officials 
told us that they provided the micro-CT data to help vendors in 
developing their explosives detection software. Specifically, TSA was 
able to distribute the micro-CT data to vendors in early fiscal year 
2010, and five of the six vendors we interviewed stated that these 
data were of limited use in developing their explosives detection 
software. For example, one vendor stated that the micro-CT data 
provided some guidance, but that there were too many unknowns to fully 
use the data to develop their explosives detection software. Further, 
TSL officials stated that providing the micro-CT data to vendors only 
served as an intermediary step to providing full threat-weight data 
that vendors needed to develop their explosives detection software. 
TSL officials stated that the micro-CT data could not provide vendors 
with all of the data they needed to fully develop their explosives 
detection software to meet the 2010 EDS requirements because vendors 
need scans of the full threat weight of explosives on their respective 
EDSs to finish developing their detection software. 

Because of the limitations of simulants and micro-CT data, TSA and S&T 
decided to collect and distribute scans of the explosives to vendors 
using the full threat weight of the explosives specified in the 2010 
EDS requirements. These scans were collected using each vendor's 
respective EDS equipment.[Footnote 28] TSA has distributed some, but 
not all, of the full threat-weight data needed by vendors to develop 
EDS detection software. TSL officials stated that they needed full 
threat-weight data to conduct certification testing. Additionally, 
five of the six vendors we interviewed agreed that the full threat-
weight data will be necessary in order for vendors to develop their 
explosives detection software. However, all six vendors noted that 
because of concerns about the safety of handling certain explosives, 
they are relying on TSA for the full threat-weight data. Further, four 
of the vendors said that in the past they had access to some 
explosives and could collect their own data to develop and test their 
detection software in order to prepare for certification testing. 
However, because the vendors cannot safely handle certain explosives, 
they are reliant upon the data provided by TSA. Consequently, until 
S&T completes the data collection on all identified explosives being 
performed at TSL and the AFRL facility at Tyndall AFB, TSA cannot 
provide all of the data that vendors need to develop their explosives 
detection software and prepare for certification, nor can the TSL 
start certification testing of new equipment as part of the current 
EDS acquisition. 

Challenges Related to Collecting Explosives Data Have Delayed the EDS 
Acquisition Schedule: 

TSA's plans to award contracts for the current EDS acquisition have 
been delayed by at least 7 months, in part, due to the challenges 
experienced by S&T related to collecting explosives data. TSA 
officials stated that, initially, they planned to conduct the current 
EDS acquisition separately from efforts to collect the data needed to 
deploy EDSs that meet the 2010 requirements. Specifically, TSA 
officials stated that they planned to complete the data collection 
before initiating the procurement to buy EDSs that meet the 2010 
requirements. However, officials stated that they subsequently decided 
to collect explosives data at the same time as implementing the 
current EDS acquisition because TSA and other stakeholders believed 
that the data collection effort would be straightforward and that the 
new requirements could be easily applied to machines procured in the 
current EDS acquisition. Additionally, program officials stated that 
procuring and deploying EDSs that meet the 2010 requirements in a 
phased approach (that is, implementing Level C first, then Level B, 
then Level A) would help to mitigate any additional challenges and 
some of the risks associated with collecting data needed for the 2010 
requirements. 

However, TSA and S&T officials acknowledged that pursuing the 
competitive procurement and explosives data collection at the same 
time had been more challenging than originally anticipated and had 
presented problems for the current EDS acquisition. TSA officials 
stated that all of the 260 EDSs they plan to purchase in 2011 will be 
upgraded to meet all of the 2010 EDS requirements at a later date. 

In our prior reports regarding acquisitions, we reported on the 
elevated risk of poor program outcomes from the substantial overlap of 
development, test, and production activities. Specifically, we have 
identified development cost increases, additional delays in 
manufacturing and testing schedules, and increased financial risk due 
to pursuing procurement before testing is complete.[Footnote 29] By 
separating the effort to collect data on explosives needed to meet the 
new requirements from the related competitive procurement, TSA and S&T 
would have more time to collect data identifying the physical and 
chemical properties of explosives, provide vendors with the time 
needed to develop detection software, and attempt to pass CRT and 
certification testing without the added pressure of an acquisition 
deadline. For example, by completing data collection for each of the 
phases of the 2010 EDS requirements prior to pursuing procurements for 
EDSs that meet those requirements, TSA could avoid additional delays 
to the acquisition schedule due to any data collection challenges. To 
help avoid these challenges in the future, TSA officials stated that 
they do not plan for subsequent procurements of EDS capable of meeting 
the more stringent explosives detection requirements until after the 
data collection for these explosives has been fully completed. 

We recognize that it is difficult in such situations to identify firm 
milestones. However, TSA has not documented its revised approach for 
conducting the needed data collection and related procurements 
sequentially rather than simultaneously. TSA does not yet have a 
documented strategy in place for deploying EDSs beyond July 2011; such 
a strategy would be valuable because TSA plans to complete the 
implementation of all of the requirements at an undetermined time 
after July 2011. Standard practices for program management state that 
the successful execution of any plan includes identifying in the 
planning process the schedule that establishes the timeline for 
delivering the plan.[Footnote 30] Documenting a plan to separate data 
collection efforts and certification from future procurements could 
help TSA ensure it avoids the challenges it has encountered during the 
current procurement. 

EDS Vendors Report Concerns about the Extent to Which TSA is 
Communicating Effectively about the Current Procurement: 

Officials from five of six EDS vendors we interviewed expressed 
concerns about the extent to which TSA has communicated effectively 
with vendors interested in the current procurement. Specifically, 
these five vendors expressed concerns about the timeliness in which 
TSA responded to their questions regarding the current procurement or 
the manner in which TSA communicated important schedule changes, or 
both. Standards for Internal Control in the Federal Government state 
that management should ensure there are adequate means of 
communicating with and obtaining information from external 
stakeholders that may have a significant impact on the agency 
achieving its goals. Additionally, the Federal Acquisition Regulation 
(FAR) encourages exchanges of information among all interested 
parties, from the earliest identification of a requirement through the 
receipt of the proposal. The FAR further states that the purpose of 
exchanging information is to improve the understanding of government 
requirements and industry capabilities, thereby allowing vendors to 
judge whether or how they can satisfy the government's requirements. 
The improved understanding resulting from such information exchange 
also enhances the government's ability to obtain quality supplies and 
services at reasonable prices and, among other things, potentially 
increases efficiency in vendors' proposal preparations. However, five 
out of six vendors we interviewed said TSA often did not provide 
information or respond to their questions in a timely manner, if at 
all.[Footnote 31] For example, four out of these five vendors said TSA 
did not answer their questions in a timely manner, in one case taking 
several months to provide answers to questions posted via a question 
tracker accessible online to all interested vendors. Meanwhile four of 
the five vendors' officials stated TSA did not respond at all to some 
of their questions, while officials from the fifth vendor stated they 
were frustrated with how long it took TSA to answer their questions. 

Officials from two vendors stated that the lack of timely 
communication regarding schedule changes for the EDS acquisition 
caused them to incur additional costs allocating extra resources and 
time to meet the original deadline. Specifically, officials from one 
vendor noted that they spent additional costs on personnel to 
aggressively pursue software development for the planned start of 
certification readiness testing (CRT), despite not having all of the 
full threat-weight explosives data TSA had intended to provide. 
Subsequently, these officials told us, TSA did not announce to vendors 
that CRT would be delayed until one week prior to the original 
deadline. EBSP officials stated that, because vendors had not yet 
received all of the full threat weight explosives data, they should 
have been aware that CRT was not going to happen according to the 
established schedule. However, EBSP officials agreed that providing 
vendors with a revised schedule prior to the previously established 
deadline would have helped promote greater vendor understanding about 
the proposed changes to TSA's acquisition strategy. 

TSA stated that it has taken a number of important steps to alleviate 
confusion and provide as much information to the vendors as possible. 
Among other things, at the start of the current procurement, TSA 
conducted three conferences with industry, called "industry days," to 
provide a forum for sharing information with the vendor community 
regarding the current EDS acquisition. TSA also reported sharing 
multiple draft versions of the requirements documents and soliciting 
vendor comments. Additionally, TSA officials stated that they shared 
draft copies of the detection requirements and held individual 
classified meetings during the industry days with each interested 
vendor to obtain input regarding the acquisition. Finally, TSA stated 
that it also allowed vendors to use government owned equipment and 
paid for engineering services associated with the testing to help 
offset vendor costs. Although EBSP officials stated that they have 
made a concerted effort to be responsive to vendors' questions and to 
call vendors directly when issues such as schedule changes arose, EBSP 
officials agreed that the agency did not always effectively 
communicate with vendors in a timely manner. Establishing a process 
for more timely communication with vendors competing for the current 
EDS procurement could help TSA to ensure that vendors have all of the 
information they need to meet TSA's needs for new checked baggage 
screening equipment. 

Improved Acquisition Planning Could Help TSA Avoid Further Delays and 
Potential Cost Overruns for the EDS Procurement: 

TSA does not have an integrated master schedule (IMS) for the EBSP, 
and TSA's schedule for the current EDS acquisition, which is only a 
part of the program, does not fully meet best practices for preparing 
an acquisition schedule.[Footnote 32] Additionally, while TSA 
completed an initial cost estimate for the EBSP, TSA officials 
reported that the current cost estimate does not reflect the 
anticipated costs of purchasing EDSs to meet the 2010 EDS 
requirements. To meet the explosives detection requirements 
established in January 2010, TSA plans to upgrade the detection 
software of a currently unknown number of the deployed EDSs and 260 of 
the EDSs to be purchased under the current acquisition after they are 
deployed to airports. However, TSA has not yet developed a plan or 
cost estimate for the planned upgrades. 

TSA Does Not Have a Schedule for the EBSP and Has Not Established a 
Reliable Schedule for the Current EDS Acquisition: 

EBSP: 

As part of EBSP's responsibility to provide equipment to screen all 
checked baggage originating at U.S. commercial airports, it is 
acquiring and deploying explosives detection technology to replace 
aging systems and meet emerging threats. While TSA established the 
EBSP as a long-term program to procure, test, deploy, and maintain 
checked-baggage-screening equipment, TSA officials confirmed in 
December 2010 that there is currently no IMS for the EBSP. Among other 
things, best practices and related federal guidance call for a program 
schedule to be programwide in scope, meaning that it should include 
the integrated breakdown of the work to be performed by both the 
government and its contractors over the expected life of the program. 
[Footnote 33] Without an IMS identifying long-term plans for the EBSP, 
it is difficult for TSA to have a comprehensive program view of the 
work that must be completed to deliver explosive detection technology 
to replace aging systems and meet emerging threats. Without such a 
view, a sound basis does not exist for knowing with any degree of 
confidence when and how the program will be completed. 

EDS Acquisition: 

While there is no IMS for the EBSP, TSA has established a schedule for 
the current EDS acquisition. However, while the schedule identifies 
activities through the first contract award--scheduled for July 2011-- 
of the current EDS procurement, our analysis shows that it does not 
identify activities planned for subsequent award windows.[Footnote 34] 
Additionally, TSA has encountered a number of challenges in 
implementing the schedule. For example, according to TSA officials, 
TSA originally planned to award the first EDS contract in December 
2010 in order to procure machines required to meet Level C of the 2010 
EDS requirements. However, TSA has since revised the schedule due to 
the challenges of collecting explosives data needed before development 
of EDS can be completed and certification testing of the machines can 
begin.[Footnote 35] 

Based on the revised schedule, certification testing began in late 
2010, according to TSA, so that in July 2011 the first EDS contract 
can be awarded to procure machines that meet one part of the Level C 
explosives detection requirements. Furthermore, while TSA has stated 
that it plans to procure and deploy 640 additional EDSs at an 
estimated cost of approximately $964 million during fiscal years 2012 
through 2015, it is unclear when TSA plans for those machines to meet 
the remaining 2010 EDS requirements. 

As of March 2011, TSA officials estimate that it will take a number of 
years to certify EDSs that meet all three levels--C, B, and A--of the 
2010 requirements. However, the officials stated that they cannot 
fully develop these plans until they can evaluate the capability of 
the equipment to meet these requirements. This is expected to happen 
during the testing process associated with the current EDS 
procurement. TSA officials stated that they plan to deploy EDSs that 
meet the full set of Level C, B, and A requirements, but more precise 
planning, including establishing timelines, cannot occur until TSA 
better understands the potential for the EDS equipment to meet those 
requirements. However, best practices state that a comprehensive 
schedule should at least reflect all activities planned for a project 
even though some activities may be tentative and there may be 
uncertainties in schedule estimates due to, among other things, 
limited data.[Footnote 36] 

In addition to the challenges TSA has encountered in carrying out the 
schedule as originally planned, based on our analysis, the current 
schedule leading up to the first contract award is not reliable. 
[Footnote 37] Best practices state that the success of a large-scale 
system acquisition, such as the current EDS acquisition, depends in 
part on having a reliable schedule that identifies: 

* when the program's set of work activities and milestone events will 
occur, 

* how long they will take, and: 

* how they are related to one another. 

Best practices also call for the schedule to expressly identify and 
define the relationships and dependencies among work elements and the 
constraints affecting the start and completion of work elements. 
Additionally, best practices indicate that a well-defined schedule 
also helps to identify the amount of human capital and fiscal 
resources that are needed to execute an acquisition. However, based on 
our assessment of both the original as well as an updated version of 
the schedule, TSA's schedule for the current EDS acquisition does not 
fully comply with nine best practices for preparing a schedule as 
shown in table 1. Appendix III has additional information about GAO's 
assessment of the extent to which TSA's schedule meets each best 
practice. 

Table 1: Implementation of Best Practices in Electronic-Baggage- 
Screening Program Schedule as of December 2010: 

Best practice: 1. Capturing all activities; 
Explanation of best practice: Defining in detail the work to be 
completed, including activities to be performed; 
Degree of implementation: Minimally met[B]. 

Best practice: 2. Sequencing all activities; 
Explanation of best practice: Listing activities in the order in which 
they are to be carried out; 
Degree of implementation: Partially met[A]. 

Best practice: 3. Assigning resources to all activities; 
Explanation of best practice: Identifying the resources needed to 
complete the activities; 
Degree of implementation: Minimally met[B]. 

Best practice: 4. Establishing the duration of all activities; 
Explanation of best practice: Determining how long each activity will 
take to execute; 
Degree of implementation: Partially met[A]. 

Best practice: 5. Integrating all activities horizontally and 
vertically; 
Explanation of best practice: Achieving aggregated products or 
outcomes by ensuring that products and outcomes associated with other 
sequenced activities are arranged in the right order, and dates for 
supporting tasks and subtasks are aligned; 
Degree of implementation: Partially met[A]. 

Best practice: 6. Establishing the critical path for all activities; 
Explanation of best practice: Identifying the path in the schedule 
with the longest duration through the sequenced list of key activities; 
Degree of implementation: Minimally met[B]. 

Best practice: 7. Identifying reasonable float between activities; 
Explanation of best practice: Determining the amount of time that a 
predecessor activity can slip before the delay affects successor 
activities; 
Degree of implementation: Minimally met[B]. 

Best practice: 8. Conducting a schedule risk analysis; 
Explanation of best practice: Using statistical techniques to predict 
the level of confidence in meeting a project's completion date; 
Degree of implementation: Not met[C]. 

Best practice: 9. Updating the schedule using logic and durations to 
determine the dates for all activities; 
Explanation of best practice: Continuously updating the schedule to 
determine realistic start and completion dates for program activities 
based on current information; 
Degree of implementation: Minimally met[B]. 

Source: GAO analysis of TSA information. 

Note: We intended to assess the EBSP schedule based on the nine best 
practices, but when we identified that TSA did not have an IMS for the 
EBSP, we assessed the EDS acquisition schedule. 

[A] The program provided evidence that satisfies about half of the 
criterion (partially met). 

[B] The program provided evidence that satisfied less than half of the 
criterion (minimally met). 

[C] The program did not provide evidence that satisfies any of the 
criterion (not met). 

[End of table] 

Although TSA's schedule does not fully comply with any of the nine 
best practices, TSA has taken action to partially or minimally meet 
eight of the best practices. For example, consistent with best 
practice 4, the schedule establishes the duration of all activities 
and properly reflects how long each activity should take. However, 
while the schedule establishes the duration of all activities, 61 
percent of activities represented in the schedule are based on a 7-day 
calendar that does not account for holidays. Similarly, our analysis 
found that, consistent with best practice 5, the schedule is 
vertically integrated; however, issues with sequencing logic in the 
schedule prevent it from being fully horizontally integrated. Vertical 
and horizontal integration ensures that products and outcomes 
associated with other sequenced activities are arranged in the right 
order and that dates for supporting tasks and subtasks are aligned. 
Other areas of the schedule that remain unaddressed also reflect 
weaknesses that limit its usefulness as a program management tool. For 
example, the schedule does not fully identify the resources needed to 
do the work or the availability of these resources. Specifically, the 
schedule does not reflect what labor, material, and overhead are 
needed to complete key activities for the program. Resource 
information would assist the program office in forecasting the 
likelihood of activities being completed based on their projected end 
dates. If the current schedule does not allow for insight into current 
or projected over-allocation of resources, then the risk of the 
program slipping is significantly increased. 

Additionally, TSA officials did not complete a schedule risk analysis 
when developing the schedule. A schedule risk analysis may be used to 
determine the level of uncertainty and to help identify and mitigate 
the associated risks. In the absence of a schedule risk analysis, the 
acquisition faces the risk of delays to the scheduled completion date 
if any delays were to occur on critical path activities. Furthermore, 
without this information, TSA is limited in its ability to answer 
questions such as how likely it is to complete the project on time and 
which risks are most likely to delay the project. Similarly, without a 
valid critical path, EBSP management lacks a clear picture of the 
tasks that must be performed to achieve the acquisition's target 
completion date. While TSA officials noted that they had no staff or 
expertise to complete a schedule risk analysis, TSA provided no 
explanation as to why a schedule consistent with the other eight best 
practices had not been developed. TSA officials stated that the EDS 
acquisition is one of the largest acquisition programs in DHS. 
However, the absence of a reliable schedule makes it difficult for 
management to predict with any degree of confidence whether the 
estimated completion date for the acquisition is realistic. 
Furthermore, without the development of a schedule that meets 
scheduling best practices, TSA is limited in its ability to monitor 
and oversee the progress of the billions of dollars being invested in 
the procurement of new EDSs. 

The EBSP's Current Life-Cycle Cost Estimate Does Not Reflect 
Anticipated Costs for Purchasing EDSs That Meet the 2010 EDS 
Requirements: 

The EBSP does not yet have an up-to-date approved life-cycle cost 
estimate in place, and as a result, DHS has no reliable basis for 
understanding how much the program will cost. While TSA reported that 
it had completed a life-cycle cost estimate (LCCE) for the EBSP in May 
2010, program officials reported in February 2011 that the estimate is 
currently being revised to reflect assumptions related to the current 
EDS acquisition.[Footnote 38] Specifically, officials indicated that 
the May 2010 LCCE did not include the anticipated costs for purchasing 
any EDSs that meet the revised 2010 requirements. TSA officials stated 
that they are working to revise the LCCE to reflect the anticipated 
costs of the current EDS acquisition and expected to complete the LCCE 
by the end of April 2011. Additionally, after conducting a review of 
the LCCE that was completed in May 2010, DHS's Cost Analysis Division 
(CAD) found that the LCCE needed more comprehensive data and that its 
accuracy could not be determined.[Footnote 39] As a result, the DHS 
Acquisition Review Board directed the CAD to develop an appropriate 
cost estimate, including a reconciliation with the EBSP's 
LCCE.[Footnote 40] In January 2011, officials in DHS's CAD stated that 
they had initiated work on the independent cost estimate for the EBSP 
but were only able to complete the portion of the estimate related to 
current detection capabilities in the Level C requirements for one 
tier. Officials stated that the lack of detail in program requirements 
for some of Level C and all of Levels B and A limited their ability to 
develop an estimate that would be usable for budgetary purposes. CAD 
officials further noted that significant portions of the total EBSP 
program have yet to be defined and estimated. During the course of our 
review, the anticipated completion date of the LCCE has been delayed 
multiple times and was expected to be completed at the end of April 
2011. As a result, we were unable to evaluate TSA's approach to 
developing the cost estimates for the program. 

We reported in June 2010 that inaccurate or incomplete cost estimates 
were often a factor in cost growth for DHS programs we previously 
reviewed. We also reported that initial cost estimates for most DHS 
programs were often developed after the start of acquisition 
activities, so they do not capture earlier cost changes.[Footnote 41] 
Further, our best practices for cost estimation state that estimates 
are integral to determining and communicating a realistic view of 
likely cost and schedule outcomes that can be used to support a 
program including planning the work necessary to develop, produce, and 
install equipment. However, because TSA had not established a cost 
estimate that accurately reflects the anticipated costs of the 
acquisition prior to initiating the current EDS procurement, it is 
unclear how DHS could determine if the budget for the EBSP is 
reasonable. Furthermore, in the absence of an approved cost estimate 
and baseline financial information for the current EDS acquisition, 
including the costs of purchasing machines that meet the 2010 EDS 
requirements, TSA has limited information to make essential cost-
informed program decisions. 

Although we were unable to evaluate TSA's cost estimates for the 
program, the fact that TSA's schedule for the EDS acquisition does not 
meet best practices for schedule estimating also raises questions 
about the credibility of the program's LCCE. For example, the absence 
of a schedule risk analysis would have made it difficult for officials 
to account for the cost effects of schedule slippage when developing 
the LCCE. Best practices for cost estimation state that because some 
program costs such as labor, supervision, rented equipment, and 
facilities cost more if the program takes longer, a reliable schedule 
can contribute to an understanding of the cost impact if the program 
does not finish on time.[Footnote 42] The program's success depends on 
the quality of its schedule and an integrated schedule is key to 
managing program performance and is necessary for determining what 
work remains and the expected cost to complete the work. In a memo 
from the DHS Under Secretary for Management dated July 10, 2008, DHS 
endorsed the use of best practices that we identified and stated that 
DHS would be utilizing them as a "best practices" approach in the 
future.[Footnote 43] However, in the absence of a reliable schedule to 
guide cost estimates, having a current cost estimate that reflects 
anticipated costs for the EDS acquisition, or submitting the revised 
LCCE to DHS for departmental approval, it is unclear how TSA utilized 
a best practices approach in developing cost estimates for the program. 

TSA Has No Plan in Place Outlining How It Will Upgrade Deployed EDSs 
to Fully Meet the 2010 Requirements: 

TSA officials stated that they expect to upgrade an unknown number of 
the current fleet of 2,297 EDSs and 260 of the EDSs to be purchased 
under the current acquisition after they are deployed to airports to 
fully meet all phases of the 2010 requirements.[Footnote 44] However, 
similar to when TSA revised the EDS explosives detection requirements 
in 2005, it has no plan in place outlining how it will approach these 
upgrades. Specifically, TSA has not established an upgrade plan or 
conducted an analysis to determine what type of approach to upgrading 
deployed EDSs is likely to be most feasible, efficient, or effective. 
TSA officials stated that there are too many unknowns at this time 
regarding potential approaches to upgrading the fleet of EDSs. 
Standards for program management require that specific desired 
outcomes or results be conceptualized, defined, and documented in the 
planning process as part of a road map, along with the appropriate 
steps and time frames needed to achieve those results.[Footnote 45] 
Until TSA develops a plan identifying how it will approach the 
upgrades for currently deployed EDSs--and the plan includes such items 
as estimated costs, the number of machines that can be upgraded, and 
the number of times a given machine must be upgraded to meet the 2010 
EDS requirements--it will be difficult for TSA to provide reasonable 
assurance that its upgrade approach is feasible or cost-effective. 

TSA's 2010 acquisition strategy identifies the planned purchase of 900 
EDSs over the next 5 years, but it does not indicate what level 
requirements those EDSs will be required to meet. For the currently 
deployed equipment that will not be replaced, TSA would need to 
upgrade the equipment to meet the 2010 requirements. TSA officials 
stated that they may not upgrade all of the current fleet of EDSs to 
the 2010 EDS requirements because in some cases, certain models of the 
EDS may not be upgradeable and in other cases, it may ultimately be 
more cost effective to replace older EDSs with new machines. According 
to TSA, upgrading EDSs will require an assessment of currently 
deployed EDS' detection capabilities and that the results of the 
assessment will affect the EDS program's schedule, budget, and 
detection goals. TSA was working with a consulting firm to modify a 
computer model that will be used to project the costs of the upgrades 
if TSA were to use a time-phased installation for the upgrades. While 
TSA officials were working with a consulting firm, they have not yet 
established a plan for how they will approach the upgrades. TSA 
officials further stated that the number of upgrades TSA performs on 
currently deployed equipment will depend on the cost of the upgrades, 
the level of complexity of the upgrades, and whether the upgrades can 
be conducted in the airports or must be performed in the factory. 

TSA's approach to deploy EDSs that meet the 2010 requirements could 
result in the same EDSs being upgraded multiple times in order to 
first meet all of the Level C requirements and to then meet the Levels 
B and A explosives detection requirements.[Footnote 46] For example, 
TSA's decision to revise its acquisition strategy and deploy EDSs that 
meet the Level C requirements in a phased approach could result in 
upgrading the same currently deployed machines twice before they may 
have to be upgraded a third time to meet Level B requirements and then 
upgraded a fourth time to meet Level A requirements. Moreover, based 
on TSA's schedule for the current EDS acquisition, by the time some or 
all of the 260 new EDSs under the current EDS acquisition have been 
deployed in airports, TSA may have approved a subsequent tier of the 
EDS explosives detection requirements, which could involve upgrading 
the machines again or replacing these newly purchased and deployed 
machines because they cannot meet the subsequent tier of explosives 
detection requirements. Therefore, TSA may procure and deploy 260 EDSs 
that will only be used in airports for a short period of time before 
they will need to be upgraded, possibly multiple times, or replaced 
with new machines. TSA officials told us that they will evaluate the 
need to upgrade EDSs to a subsequent tier at the time those 
requirements are finalized. 

TSA officials stated that they initially delayed the analysis of the 
upgrade approach until the 2010 EDS explosives detection requirements 
were approved, an approval that occurred in January 2010. TSA 
officials subsequently stated that their plan to upgrade deployed EDSs 
is included in the recapitalization strategy due to be completed at 
the end of May 2011. According to TSA, vendors that have previously 
sold EDSs to TSA are to be asked to also include proposals to upgrade 
their currently deployed machines when submitting proposals for the 
current EDS procurement. Specifically, vendors are to be asked to 
include a plan for upgrading their currently deployed EDS equipment as 
well as cost estimates for the upgrades. TSA plans to then analyze the 
feasibility and costs of the vendors' proposals. However, TSA 
officials stated that the equipment upgrades may or may not be 
implemented as part of the contract award and that TSA has discretion 
regarding which aspects of the contracts to implement. According to 
TSA, the total number of EDSs to be upgraded and the associated costs 
will not be known until the agency receives proposed upgrade plans and 
cost estimates from EDS vendors in summer 2011. According to TSA 
officials, any upgrades are not to occur until calendar year 2012 at 
the earliest and will depend on available funding and complexity of 
the upgrades. 

TSA officials as well as officials from three of six current EDS 
vendors told us that they are confident that currently-deployed EDSs 
can be upgraded to meet Level C requirements.[Footnote 47] 
Specifically, TSA officials stated that the EDS vendors can rewrite 
the detection software to provide the capability to detect the 2010 
EDS requirements. Minor hardware changes, such as new computer chips, 
are also expected to be made as part of these upgrades according to 
the TSA officials. The officials stated that, after they approve the 
software and hardware upgrades, EDS vendors will install the upgrades 
on the machines in the airports. Officials from three EDS vendors 
noted that they believe upgrades to the EDSs can be made in the 
airports when regularly-scheduled routine maintenance work is 
conducted. 

Once deployed EDSs have been upgraded to fully meet the Level C 
requirements, TSA will have to make decisions about how to ensure 
these machines can meet subsequent phases of the 2010 EDS requirements 
(Levels B and A). Officials from all six EDS vendors stated that given 
the absence of additional data on the explosives that will be included 
in subsequent phases of the 2010 EDS requirements, it is difficult to 
know precisely what must be done to upgrade newly-purchased equipment. 
Therefore, none of the officials from the six vendors could provide 
estimates for the cost to upgrade EDSs to meet all of the requirements 
for one tier. However, officials from two vendors estimated the cost 
to upgrade new EDSs to meet Level C requirements at $50,000 to 
$150,000 per machine.[Footnote 48] An official from one vendor stated 
that the CT technology currently used in EDSs might not be sufficient 
to detect Level A requirements and that an as yet undeveloped 
technology may be needed. The official noted that this could result in 
substantially higher costs to upgrade the current fleet of machines to 
Level A requirements than it would cost to upgrade machines to Level B 
requirements. Similarly, officials from two other vendors stated that 
meeting Level A requirements may require either new technology or a 
combination of current technologies instead of only using an EDS. 

Although TSA and vendor officials expressed confidence that deployed 
EDSs can be upgraded, TSA officials also confirmed that the agency has 
never previously upgraded the detection software of deployed EDS or 
ETD machines to meet revised explosives detection requirements. 
Additionally, even though TSA has estimated that it will take a number 
of years to certify new EDSs to fully meet Levels B and A of the 2010 
requirements, TSA has not yet developed similar time frames to upgrade 
deployed equipment. Given the number of unknowns associated with 
upgrading EDSs, it is unclear how long it will take the agency to 
upgrade deployed EDSs to meet Levels C, B, and A of the 2010 
requirements. Furthermore, TSA has identified the EDS upgrade effort 
as a high program risk. Consequently, TSA and vendor officials' 
confidence that it will be feasible and cost effective to upgrade 
deployed machines at airports may be unwarranted as it has not been 
based on experience, supported by analysis, or a documented plan. 

Conclusions: 

TSA faces a complex task in its efforts to address explosives threats 
in its current and future procurements and existing fleet of checked- 
baggage-screening systems. The complexity of this task is amplified 
when taking into account the large volume of checked baggage that TSA 
must screen for explosives without disrupting commerce. TSA's plan to 
procure and deploy EDSs that meet the 2010 requirements in a phased 
approach that spans a number of years is aimed at allowing more time 
to collect necessary explosives data, test key technologies, and 
provide a means for TSA to continue to purchase EDSs to meet its needs 
for new checked-baggage-screening equipment at the nation's commercial 
airports. However, TSA officials recognized that if TSA deploys EDS 
capable of detecting all explosives included in the 2010 EDS 
requirements, TSA must ensure that ETD machines are capable of 
detecting all of the explosives that EDSs will be able to detect to 
minimize any potential screening difference between the EDS and ETD. 
Without a plan to help ensure that additional screening devices or 
protocols are in place to resolve EDS alarms if EDSs are deployed that 
detect a broader set of explosives than existing ETD machines used to 
resolve EDS screening alarms, it will be difficult for TSA to provide 
reasonable assurance that a potential capability gap has been resolved. 

By separating the effort to collect data needed to meet the 2010 EDS 
requirements from the related competitive procurement, TSA would have 
more time to identify the physical and chemical properties of the 
explosives, collect full threat weight data, provide vendors with the 
time needed to develop detection software, and attempt to pass CRT and 
certification testing without the added pressure of an acquisition 
deadline. TSA also faces additional challenges related to the agency's 
plans for implementing the current EDS procurement. For example, the 
lack of timely communication with vendors may impact vendors' 
abilities to ensure they can meet TSA's needs for the current EDS 
acquisition. By establishing a process to communicate with vendors in 
a timely manner, TSA could help ensure that vendors have the 
information necessary to meet TSA's needs for new checked-baggage-
screening equipment. Moreover, by addressing challenges related to 
planning for the acquisition, TSA may be able to better avoid further 
delays and potential cost overruns for the current procurement. 
Specifically, completing a reliable IMS that fully meets the nine best 
practices could help DHS and TSA management to predict whether the 
estimated acquisition completion date is realistic and manage program 
performance. Once a reliable schedule is in place, TSA can in turn 
revise current cost estimates for the program to better reflect actual 
acquisition costs including, for example, the potential cost impacts 
resulting from schedule slippage to give program decision-makers a 
more accurate and comprehensive view of current and projected program 
costs. 

As TSA plans to deploy EDSs that meet the 2010 requirements, it is 
critical that TSA plans its approach to ensure that all airports with 
EDS equipment are capable of detecting the required explosives. 
Because TSA has not yet upgraded most of the deployed EDSs to meet 
certain requirements, many EDSs are only capable of detecting certain 
explosives. Moreover, of the EDSs currently deployed, TSA is currently 
operating some number of them at the capability needed to detect the 
explosives identified in the 2005 requirements, although activating 
the software and operationally testing the machines to detect the 2005 
requirements would help address this issue. 

As part of TSA's phased approach to meet the 2010 EDS requirements, 
TSA may have to upgrade many of its currently deployed EDSs and 
hundreds of newly purchased EDSs over a period of years, upgrades that 
may require significant investments in new technologies to help meet 
more stringent explosives detection requirements. However, until TSA 
develops a plan identifying how it will approach the upgrades for 
currently deployed EDSs--and the plan includes such items as estimated 
costs, the number of machines that can be upgraded, time frames for 
upgrading them, and the number of times a given machine must be 
upgraded to meet the 2010 EDS requirements--it will be difficult for 
TSA to provide reasonable assurance that its upgrade approach is 
feasible or cost-effective. 

Recommendations for Executive Action: 

To help ensure that TSA takes a comprehensive and cost-effective 
approach to the procurement and deployment of EDSs that meet the 2010 
EDS requirements and any subsequent revisions, we recommend that the 
Assistant Secretary for TSA take the following six actions: 

* Develop a plan to ensure that screening devices or protocols are in 
place to resolve EDS alarms if EDSs are deployed that detect a broader 
set of explosives than existing ETD machines used to resolve EDS 
screening alarms. 

* Develop a plan to ensure that TSA has the explosives data needed for 
each of the planned phases of the 2010 EDS requirements before 
starting the procurement process for new EDSs or upgrades included in 
each applicable phase. 

* Establish a process to communicate information to EDS vendors in a 
timely manner regarding TSA's EDS acquisition, including information 
such as changes to the schedule. 

* Develop and maintain an integrated master schedule for the entire 
Electronic Baggage Screening Program in accordance with the nine best 
practices identified by GAO for preparing a schedule. 

* Ensure that key elements of the program's final cost estimate 
reflect critical issues, such as the potential cost impacts resulting 
from schedule slippage identified once an integrated master schedule 
for the Electronic Baggage Screening Program has been developed in 
accordance with the nine best practices identified by GAO for 
preparing a schedule. 

* Develop a plan to deploy EDSs that meet the most recent EDS 
explosives-detection requirements and ensure that new machines, as 
well as machines deployed in airports, will be operated at the levels 
established in those requirements. This plan should include the 
estimated costs for new machines and upgrading deployed machines, and 
the time frames for procuring and deploying new machines and upgrading 
deployed machines. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to DHS on June 23, 2011, for review 
and comment. On July 6, 2011, DHS provided written comments, which are 
presented in appendix IV. We also provided relevant excerpts of our 
draft report to DOD and the Department of Energy for review and 
comment. In commenting on our report, DHS stated that it agreed with 
our six recommendations and identified actions planned or under way to 
implement them. DOD provided written technical comments and the 
Department of Energy provided technical comments in an e-mail. Both 
stated that the draft report excerpts related to their respective 
agencies contained accurate information. 

Overall, DHS stated that, because of the urgent need to meet ongoing 
requirements, TSA began addressing many of the issues identified by 
this audit while the audit was being conducted. However, as DHS noted 
in its letter, TSA still needs to complete many actions to resolve the 
issues identified in this report. Additionally, TSA stated that it 
suspended the implementation of the 2005 requirements because of the 
computer modeling effort known as "Project Newton" and then issued the 
2010 detection standards when Project Newton did not yield timely 
results. Thus, in its comments, TSA confirmed that it is using some 
number of EDSs that meet requirements established in 1998 by the FAA, 
as we reported, an approach that raises questions about how well some 
of its deployed equipment detects current explosives threats. 

In addition, DHS stated that TSA is currently taking steps to collect 
the operational data necessary to support the upgrade of deployed 
equipment based on 2010 detection standards and that the operational 
data-collection effort is to be completed in 2011. However, as 
discussed in the report, this could be a difficult endeavor as TSA is 
still in the process of conducting operational testing to determine 
the staffing implications of operating EDSs that meet 2005 explosives 
detection requirements. Therefore, it will have taken TSA 6 years from 
the time that the 2005 EDS explosives detection requirements were 
issued until this operational testing is to be completed. Furthermore, 
if the results of the operational testing show that operating EDS 
machines, to meet the 2005 requirements, will require additional TSA 
staff and/or slow down the rate of checked-baggage screening, TSA may 
have to make difficult decisions and trade-offs that could affect 
aviation security and commerce, and also affect the schedule for 
meeting the 2010 requirements. 

DHS concurred with our first recommendation to develop a plan to 
ensure that screening devices or protocols are in place to resolve EDS 
alarms if EDSs are deployed that detect a broader set of explosives 
than existing ETD machines used to resolve EDS screening alarms. DHS 
stated that TSA convened a working group to assess capability gaps for 
secondary screening technology, evaluate current technology 
capabilities against the capabilities of future EDSs, and prepare a 
plan to procure any additional technology required to ensure alarms 
can be resolved. DHS expects this plan to be finalized by the fourth 
quarter of fiscal year 2012. While these actions and planned actions 
represent positive steps to fully implement the recommendation, TSA 
should develop a plan to ensure that screening devices or protocols 
are in place to resolve EDS alarms if EDSs are deployed that detect a 
broader set of explosives than existing ETD machines used to resolve 
EDS screening alarms. 

DHS concurred with our second recommendation to develop a plan to 
ensure that TSA has the data needed for each of the planned phases of 
the 2010 EDS requirements before starting the procurement process for 
new EDSs or upgrades included in each applicable phase. DHS commented 
that TSA modified its strategy for the EDS's competitive procurement 
in July 2010 in response to the challenges in working with the 
explosives for data collection and alerted the vendor community on 
September 3, 2010. DHS stated that the new baseline schedule removed 
data collection from the acquisition process. Additionally, DHS stated 
that TSA is working with DHS S&T to establish a laboratory by summer 
2011 to support further data collection and independent test and 
evaluation. Although these actions respond in part to the intent of 
our recommendation, separating data collection from the acquisition 
process does not necessarily ensure that the needed data will be 
available before starting the procurement process for the new EDSs or 
upgrading currently deployed EDSs. Consequently, we continue to 
believe that, to fully address our recommendation, a plan is needed to 
establish a process for ensuring that data are available before 
starting the procurement process for new EDSs or upgrades for each 
applicable phase. Developing and following such a plan would assist 
TSA in implementing the acquisition and making upgrades in an 
efficient and effective manner and would benefit DHS in its oversight 
role of TSA by allowing DHS to determine progress against the plan. 

DHS concurred with our third recommendation to establish a process to 
communicate information to EDS vendors in a timely manner regarding 
TSA's EDS acquisition, including information such as changes to the 
schedule. In the letter, DHS stated that TSA has a process for 
communicating information to the vendor community and will continue to 
follow this process in adherence with guidelines outlined in the 
Federal Acquisition Regulation. DHS also stated that TSA significantly 
changed the business model to procure checked-baggage-screening 
equipment from what has historically been a sole-source environment to 
a competitive environment, resulting in significant improvements in 
communication with industry. In addition, according to DHS, TSA has 
already made a number of efforts to improve the quality and frequency 
of communication with industry, but TSA recognizes the complexity 
associated with many of the acquisitions currently ongoing. As such, 
TSA acknowledged that there are opportunities to continue to improve 
communication with the vendor community and will take steps to ensure 
that vendors are provided with the most current information possible 
in an efficient manner. Since the agency did not provide us with 
evidence of how it plans to ensure more timely and effective 
communications with vendors in the future, we continue to believe that 
such a process is needed to ensure that TSA officials are aware of the 
specific guidelines to follow to communicate with vendors about 
current and future acquisitions. Our meetings with vendors indicated 
that TSA's communications with them continue to leave room for 
improvement. 

DHS concurred with our fourth recommendation to develop and maintain 
an IMS for the entire EBSP in accordance with the nine best practices 
identified by GAO for preparing a schedule. DHS commented that TSA has 
already begun working with key stakeholders to develop and define 
requirements for an IMS and to ensure that the schedule aligns with 
the best practices outlined by GAO. DHS stated that this effort is 
expected to be completed by the second quarter of fiscal year 2012. In 
addition, DHS stated that, as the program matures and increases its 
focus on flexible and upgradeable technology, an IMS will ensure close 
coordination among the program's procurement, deployment, 
recapitalization, and upgrade capabilities, and that the EBSP IMS will 
be updated as a result of these efforts to be in accordance with the 
nine best practices. While these actions and planned actions are steps 
toward implementing our recommendation, to fully implement the 
recommendation, TSA needs to develop and maintain an IMS for the 
entire EBSP in accordance with the nine best practices identified by 
GAO for preparing a schedule. 

DHS concurred with our fifth recommendation to ensure that key 
elements of the program's final cost estimate reflect critical issues, 
such as the potential cost impacts resulting from schedule slippage. 
Such a slippage might be identified once an IMS for the EBSP has been 
developed in accordance with the nine best practices identified by GAO 
for preparing a schedule. DHS stated that TSA is working to update the 
EBSP LCCE to incorporate cost estimates associated with enhanced 
detection, work that should be completed in the fourth quarter of 
fiscal year 2011. DHS also stated that, per the recommendations of GAO 
and DHS, TSA is developing a master schedule to document timelines 
associated with various projects. DHS further stated that risks to the 
costs and schedules will be analyzed and that the risk analysis will 
produce confidence intervals for the life-cycle costs to the program. 
Although TSA discussed activities to address the EBSP LCCE, to fully 
implement this recommendation, it will be important that key elements 
of the program's final cost estimate reflect critical issues, such as 
the potential cost impacts resulting from schedule slippage identified 
once an IMS for the EBSP has been developed in accordance with the 
nine best practices. 

DHS concurred with our sixth recommendation to develop a plan to 
deploy EDSs that meet the most recent EDS explosives detection 
requirements and ensure that new machines, as well as machines 
deployed in airports, will be operated at the levels established in 
those requirements. This plan should include the estimated costs for 
new machines and upgrading deployed machines, and the time frames for 
procuring and deploying new machines and upgrading deployed machines. 
DHS commented that TSA has a plan in place to evaluate and implement 
the most recent certified algorithms on the existing fleet of deployed 
EDSs, assuming the evaluation results in minimal to no operational 
impact. In contrast, our recommendation calls for a plan to deploy new 
EDSs as well as to upgrade existing EDSs in airports to meet the 2010 
EDS explosives detection requirements and, importantly, ensure that 
new machines will be operated at the levels established in those 
requirements. As we discussed in the report, some number of the EDSs 
in airports are operating at a level that meets the 2005 explosives 
detection requirements. Our recommendation is intended to ensure that 
TSA operates all EDSs in airports to meet the most recent 
requirements, which are currently the 2010 requirements. Consequently, 
we continue to believe that a plan is needed describing the approach 
that TSA will use to deploy EDSs that meet the most recent EDS 
explosives detection requirements and ensure that new machines, as 
well as machines deployed in airports, will be operated at the levels 
established in those requirements. 

TSA also provided written technical comments, which we incorporated in 
the report, as appropriate. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 10 days 
from the report date. At that time, we will send copies of this report 
to the Secretary of Homeland Security, the Assistant Secretary of the 
Transportation Security Administration, and appropriate congressional 
committees. This report also will be available at no charge on the GAO 
Web site at [hyperlink, http://www.gao.gov]. 

If you or your staffs have any questions about this report, please 
contact me at (202) 512-8777 or LordS@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions 
to this report are listed in appendix V. 

Signed by: 

Stephen M. Lord: 
Director: 
Homeland Security and Justice: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

This report discusses: (1) the extent to which TSA has revised 
explosives detection requirements and deployed EDSs and ETDs to meet 
these revised requirements; (2) any challenges that TSA and the 
Department of Homeland Security's (DHS) Science and Technology 
Directorate (S&T) have experienced in implementing the EDS 
acquisition; and (3) the extent to which TSA's approach to its EDS 
acquisition meets best practices for schedule and cost estimates, and 
includes plans for potential upgrades to deployed EDSs. 

To determine the extent to which TSA has revised explosives detection 
requirements for checked baggage screening, we reviewed TSA's EDS 
explosives detection requirements for checked baggage screening and 
assessed the extent to which the 2010 detection requirements differed 
from the 2005 detection requirements for EDSs. We compared specific 
explosives 1998, 2005, and 2010 detection requirements to identify 
commercial and homemade variants of the explosives. We also 
identified, analyzed, and discussed with TSA and S&T officials the 
differences between the tiers and multiple levels of explosives 
detection requirements in the 2010 EDS explosives detection 
requirements. We discussed with TSA officials the 2002 explosives 
detection requirements for the ETD and reviewed the 2006 explosives 
detection requirements for ETD. We also compared the 2010 EDS 
requirements with the 2006 ETD requirements and discussed with TSA and 
S&T officials the differences between the explosives detection 
requirements for the EDS and ETD. We discussed TSA's Standard 
Operating Procedures for resolving EDS and ETD alarms with TSA 
officials. Finally, we visited three of the Department of Energy's 
national laboratories, Lawrence Livermore National Laboratory, Los 
Alamos National Laboratory, and Sandia National Laboratories, to 
determine the status of Project Newton. (For more on Project Newton, 
see appendix II.) 

To identify any challenges that TSA has experienced implementing the 
EDS acquisition, we reviewed documentation from TSA's Electronic 
Baggage Screening Program (EBSP), the program responsible operational 
testing, procurement, deployment, and maintenance of checked-baggage- 
screening technologies. Among other things, we reviewed available 
program documentation on the status of its EDS acquisition including 
EBSP strategic plans from previous years as well as the most recent 
EBSP strategy approved in July 2010. We also reviewed documentation 
from the program's first Acquisition Review Board review, the EBSP 
risk management plan, the most recent procurement specifications for 
the EDS, information posted by EBSP for interested vendors on 
FedBizOpps.gov, and DHS acquisition guidance and directives. We also 
interviewed EBSP program officials, including the EBSP program 
manager, regarding the program's approach to the current EDS 
acquisition and received updates on revisions to the program's EDS 
acquisition strategy and timelines for the current procurement. 

To further understand the challenges TSA and S&T face in preparing for 
new EDSs to meet revised detection requirements, we reviewed 
documentation provided by TSA outlining the agency's plan for 
deploying EDSs that meet the 2010 requirements as well as 
documentation regarding S&T's approach to testing and certification 
carried out at the Transportation Security Laboratory (TSL). We also 
conducted interviews with TSA and S&T officials and conducted site 
visits to the TSL in Atlantic City, New Jersey, and to the Air Force 
Research Lab (AFRL) facility at Tyndall Air Force Base (AFB), Florida, 
to obtain information on efforts to certify EDSs that meet the 2010 
requirements. We visited the TSL because that is where S&T tests and 
evaluates transportation technologies including checked baggage 
screening technologies. We visited AFRL because they are assisting TSL 
in their efforts to collect data regarding the physical and chemical 
properties of explosives included in the 2010 EDS requirements in 
preparation to certify EDSs for the current procurement. Additionally, 
we conducted site visits and/or telephone interviews with all six EDS 
vendors competing in the first phase of the current EDS procurement. 
These vendors were able to provide us with an understanding of their 
companies' views regarding TSA's approach to the current procurement 
as well as potential challenges they believe vendors face in preparing 
to compete for the current EDS procurement. While information we 
obtained from these interviews may not be generalized across the 
industry as a whole, we were able to obtain the perspectives of all 
companies planning to compete for the current EDS procurement, and 
they were able to provide an understanding of their companies' 
abilities to develop EDSs that meet the 2010 requirements. We also 
reviewed TSA documentation to identify the explosives detection 
technologies that are used for checked baggage screening. 
Additionally, we interviewed TSA and S&T officials to identify the 
number of currently-deployed explosives detection machines that meet 
the previous and most recent detection requirements, and found the 
data for the number of machines to be sufficiently reliable. 

To determine the extent to which TSA's approach to its EDS acquisition 
meets best practices for schedule and cost estimates and includes 
plans for potential upgrades to deployed EDSs, we determined the 
extent to which TSA had established an integrated master schedule 
(IMS) for the EBSP, and due to the lack of an IMS, assessed the EDS 
acquisition schedule against nine best practices in our Cost 
Estimating and Assessment Guide. We conducted this assessment to 
determine the extent to which the schedule reflects key estimating 
practices that are fundamental to having and maintaining a reliable 
schedule. In doing so, we independently assessed the schedule for the 
current EDS acquisition and its underlying activities against our nine 
best practices, as provided to us in July 2010. We subsequently 
interviewed cognizant program officials to discuss their use of best 
practices in creating the schedule and to discuss the findings 
resulting from our review of the schedule. After TSA revised the 
schedule to reflect changes in some of the timelines and provided it 
to us in October 2010, we reviewed the updated schedule and compared 
it to information in the original schedule in order to understand how 
the new schedule was constructed and to determine to what extent TSA 
had resolved weaknesses that we identified in its original schedule. 
We also assessed the schedule against relevant best practices in our 
Cost Estimating and Assessment Guide to determine the extent to which 
it reflects key estimating practices that are fundamental to having a 
reliable schedule. We compared TSA's efforts with internal control 
standards and recommended practices we previously identified for sound 
acquisition planning. 

To further evaluate TSA's planning for the current EDS acquisition, we 
also interviewed TSA, S&T, and EDS vendors' officials to identify any 
challenges involved and expected in upgrading EDS detection 
capabilities, and TSA's plans to upgrade equipment to meet future 
implementations of the 2010 EDS requirements. Also, during our site 
visits and telephone interviews with the six vendors, as discussed 
previously, vendors provided their perspectives on TSA's approach to 
upgrade currently deployed EDSs as well as those to be deployed in the 
future. We also obtained from the six vendors their perspectives on 
how upgrades to deployed EDSs might be accomplished and potential 
costs involved in performing the upgrades. 

We conducted this performance audit from September 2009 through May 
2011 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
During the course of our review, we revised the engagement objectives 
and scope to facilitate a broader examination of TSA's efforts to 
revise its explosives detection requirements and related schedule for 
the EDS acquisition, which increased the time for completing this 
audit. We believe that the evidence obtained provides a reasonable 
basis for our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Department of Homeland Security Efforts to Conduct 
Computer Modeling to Establish a Tier of the Explosives Detection 
Requirements for the Explosives Detection System: 

The Department of Homeland Security (DHS) Science and Technology 
Directorate (S&T) and Transportation Security Administration (TSA) 
began a project in 2007, known as Project Newton, to identify the 
minimum mass of explosives that could cause catastrophic damage to an 
aircraft. Through 2009, S&T and TSA had invested approximately $12.5 
million in Project Newton modeling activities, according to a senior 
TSA official. A different senior TSA official stated that TSA 
allocated an additional $2.5 million to $3.1 million for Project 
Newton as of August 2010: $1.0 million to $1.6 million for incremental 
development of computer models and $1.5 million to develop a plan to 
validate the models.[Footnote 49] As part of the effort to understand 
the effects of explosives detonations on aircraft, S&T and TSA have 
been working to simulate the complex dynamics of explosive blast 
effects on an in-flight aircraft by using computer models at three 
Department of Energy national laboratories--Lawrence Livermore 
National Laboratory, Los Alamos National Laboratory, and Sandia 
National Laboratories. According to TSA officials, the current 
understanding of the effects of explosives on aircraft has been 
largely based on data from live-fire explosives tests conducted with 
retired aircraft hulls at ground level. These officials stated that, 
compared to running a computer simulation, live-fire tests can be more 
expensive, which limits the number of live-fire tests conducted and, 
therefore, the amount of data available for analysis. S&T, TSA, and 
national laboratory officials stated that computer modeling can cost-
effectively simulate the effects of explosives detonations in various 
locations of different types of aircraft at ground level, which can 
provide significant data for analysis. 

In January 2010, TSA revised the explosives detection requirements for 
the explosives detection system (EDS) and established tiers of 
explosives that are required to be detected. One tier of requirements 
is to be implemented over a number of years. TSA plans to incorporate 
the computer-modeling results into the requirements for a subsequent 
tier. Although TSA expected that the computer-modeling results would 
be used to revise EDS explosives detection requirements as early as 
2012, as of December 2010, TSA officials were uncertain when the 
computer-modeling results will be used for this purpose because the 
computer models had not been validated. In 2009, TSA established a 
Blue Ribbon Panel to, among other things, assess the three national 
laboratories' computer models and their results and comment on whether 
they were valid to be used to revise explosives detection 
requirements. The panel members included DHS and TSA officials as well 
as officials from academia and the private sector. In March 2010, the 
Blue Ribbon Panel recommended that, among other things, before the 
computer modeling results are used to revise EDS explosives detection 
requirements, the computer models and their results should be 
validated, according to a senior TSA official. This official stated 
that the panel also recommended specific locations to add to the 
computer models, so that the models can simulate the effects of 
explosives detonations in those additional locations on the aircraft. 
Validating the computer models and their results is essential before 
relying on them to revise explosives detection requirements. A senior 
TSA official stated that it will take a number of months to validate 
the computer models, validation that is expected to be completed later 
in 2011. 

[End of section] 

Appendix III: GAO's Assessment of the Extent to Which the Electronic 
Baggage-Screening-Program Schedule Meets Established Best Practices: 

In determining the extent to which the Transportation Security 
Administration's (TSA's) Electronic Baggage Screening Program (EBSP) 
schedule meets established best practices, we identified that TSA did 
not have an integrated master schedule (IMS) for the program. As a 
result, we assessed the explosives detection system (EDS) acquisition 
schedule against each of nine best practices. Specifically, we 
assessed TSA's initial schedule, which was provided to GAO in July 
2010, and met with TSA officials to discuss our assessment and 
provided officials with suggestions for corrective action. TSA later 
revised the schedule and provided GAO with an updated version in 
October 2010. We completed a separate assessment of TSA's revised 
schedule. The following table presents our two assessments. 

Table 2: Assessment of the Extent to Which the Electronic Baggage- 
Screening-Program Schedule Meets Best Practices: 

Best practice: 1. Capturing all activities; 
Explanation: The schedule should reflect all activities as defined in 
the project's work breakdown structure, which defines in detail the 
work necessary to accomplish a project's objectives, including 
activities to be performed by both the owner and contractors; 
Criterion met (July 2010): Minimally met; 
Criterion met (October 2010): Minimally met; 
GAO analysis: Initial analysis: TSA officials reported that there is 
currently no IMS for the EBSP and that the initial EDS acquisition 
schedule provided to GAO in July 2010 only reflects government effort 
associated with the pending contract award for procurement of EDS. The 
EDS acquisition schedule also contains key activities performed by 
other DHS components involved in the acquisition, such as TSA, the 
Department of Homeland Security (DHS) Science and Technology 
Directorate's (S&T) Explosives Division (EXD), the Transportation 
Security Laboratory (TSL), the TSA Systems Integration Facility 
(TSIF), Operational Testing and Evaluation (OT&E), and Air Force 
Research Lab at Tyndall Air Force Base. The schedule includes pre-
solicitation and solicitation effort for three planned EDS contract 
awards.[A] Our analysis found that none of the 901 tasks within the 
schedule are mapped to program work breakdown structure (WBS). EBSP 
officials were not aware of how a program WBS would align to either a 
schedule WBS or a WBS being used by DHS Cost Analysis Division for an 
ongoing Independent Cost Estimate. A WBS is a valuable communication 
tool because it provides a clear picture of what needs to be 
accomplished and how the work will be done. Accordingly, it is an 
essential element for identifying activities in a program's integrated 
master schedule. EBSP officials stated that activities beyond the 
third contract award are not scheduled. While separate airport 
schedules exist that govern the deployment of machines purchased under 
the current procurement contract, those schedules are not integrated 
with the EDS acquisition schedule because the effort is handled under 
a separate office within TSA. However, without an integrated master 
schedule that accounts for all planned government and contractor 
effort, management is unable to reliably estimate planned dates beyond 
the current schedule's end date of December 19, 2011. Management also 
has no information of the effects on other phases of the program--for 
example, production, deployment, and maintenance activities of 
equipment--if the third contract award is delayed. 
Updated analysis: The revised schedule provided to GAO in October 2010 
represents the new approach to the EDS acquisition, with Level C now 
divided into sub-phases. However, the schedule only includes 
activities through the first contract award to purchase EDSs to meet 
part of Level C of the 2010 EDS requirements under the first contact 
award. The schedule does not include planned contract awards for 
Windows 2 or 3 nor does it reflect when EDSs will be required to meet 
all of Levels C, B, or A. In addition, the schedule includes only 
government effort, does not link to any external schedules, and does 
not map activities to a program WBS. 

Best practice: 2. Sequencing all activities; 
Explanation: The schedule should be planned so that critical project 
dates can be met. Activities need to be logically sequenced--that is, 
listed in the order in which they are to be carried out. In 
particular, activities that must be completed before other activities 
can begin (predecessor activities), as well as activities that cannot 
begin until other activities are completed (successor activities), 
should be identified. This helps ensure that interdependencies among 
activities that collectively lead to the accomplishment of events or 
milestones can be established and used as a basis for guiding work and 
measuring progress; 
Criterion met (July 2010): Minimally met; 
Criterion met (October 2010): Partially met; 
GAO analysis: Initial analysis: Our analysis of the initial EDS 
acquisition schedule provided to GAO in July 2010 shows that 146 of 
the 502 remaining activities, or 29 percent, have missing logic--that 
is, these activities are missing necessary predecessors or successors 
which in turn reduces the credibility of the calculated dates. If an 
activity that has no logical successor slips, the schedule will not 
reflect the effect on the critical path, float, or scheduled start 
dates of downstream activities. In addition, we found 21 remaining 
activities, or 4 percent, have "dangling" logic--that is, these 
activities are missing successors off their finish date. In other 
words, the activities could continue indefinitely and not affect the 
start or finish dates of downstream activities. We found 176 remaining 
activities (35 percent) with Start No Earlier Than constraints. These 
are considered "soft" constraints in that they allow the activity to 
slip into the future based on what happens to their predecessor 
activities. While activities may be soft constrained, for example, to 
represent receipt of delivery of equipment, in general, constraining 
an activity's start date prevents managers from accomplishing work as 
soon as possible and consumes flexibility in the project. In addition, 
we found 39 Finish No Earlier Than constraints (8 percent). These are 
also considered "soft" date constraints because they prevent 
activities from finishing earlier than their constraint date. Program 
officials stated they are working to reduce the number of constraints 
within the schedule. They stated that many constraints are in place to 
dictate the start and finish dates of testing activities that rely on 
the availability of testing equipment. However, assigning calendar-
based test equipment resources to testing activities eliminates the 
need for date constraints and allows the activities' start and finish 
dates to respond dynamically to changes in the schedule. 
Of the remaining activities, 170 activities (34 percent) are linked to 
their successor activities with lags, including 11 negative lags. Lags 
are often used to put activities on a specific date or to insert a 
buffer for risk; however, these lags persist even when predecessor 
activities are delayed (that is, when the buffer should be consumed). 
Lags should be justified because they cannot have risk or uncertainty. 
Updated analysis: The updated schedule provided to GAO in October 2010 
has corrected all but two missing dependencies on remaining activities 
(less than 1 percent), a marked improvement from the last schedule 
version. The updated schedule also includes a lower number of 
activities with dangling logic: 12 remaining activities (3 percent) 
are missing successors off their finish date. In addition, the new 
schedule includes fewer date constraints than the previous version, 
which improves the dynamic nature of the schedule. There are now 13 
remaining activities with Start No Earlier Than constraints (3 
percent), and all Finish No Earlier Than constraints have been 
removed. Program officials justified the use of these Start No Earlier 
Than constraints as necessary because test units cannot be deployed to 
each test location before January 10, 2011, due to a "black-out" 
period between November 15 and January 10. This black-out period, 
enforced by TSA Security Operations, prohibits any changes to airport 
operations other than for emergency purposes. 
However, a large number of lags remain in the approved baseline 
schedule. There are 102 remaining activities with lags (27 percent), 
including 11 remaining activities (3 percent) with leads (negative 
lags). Negative lags are typically discouraged since negative time is 
not demonstrable. 

Best practice: 3. Assigning resources to all activities; 
Explanation: The schedule should reflect what resources (for example, 
labor, materials, and overhead) are needed to do the work, whether all 
required resources will be available when needed, and whether any 
funding or time constraints exist; 
Criterion met (July 2010): Minimally met; 
Criterion met (October 2010): Minimally met; 
GAO analysis: Initial analysis: Program officials stated that beyond 
assigning scheduled activities to integrated product teams (IPT), the 
schedule does not account for resources. In addition, our analysis 
shows 42 (11 percent) of the 369 remaining detail activities have IPT 
assignments. Resource information would assist the program office in 
forecasting the likelihood of activities being completed based on 
their projected end dates. If the current schedule does not allow for 
insight into current or projected over-allocation of resources, then 
the risk of the program slipping is significantly increased. 
Updated analysis: While the revised schedule provided to GAO in 
October 2010 is not used to monitor resources, the revised schedule 
now includes 10 resource group names and every remaining activity 
within the schedule is assigned a resource group name. The majority of 
remaining activities are assigned to TSIF (229 activities, or 60 
percent), and another third of the activities are assigned to OT&E 
(121 activities, or 32 percent). The other 8 percent of remaining 
activities are assigned between DHS, TSL, the EBSP, and other 
government agencies. Three remaining activities (1 percent) are 
assigned to the EBSP, and represent a total duration of 8 days of 
effort. Program officials stated that while there are only three 
activities assigned to the EBSP, the program office has oversight on 
all scheduled activities. However, if there are three activities, or 
only 8 days of effort being performed by the EBSP through July 2011, 
assuming there are 22 working days per month, there are 168 days where 
the EBSP has no scheduled work. As a result, all EBSP effort may not 
be captured in the schedule. 

Best practice: 4. Establishing the duration of all activities; 
Explanation: The schedule should realistically reflect how long each 
activity will take to execute. In determining the duration of each 
activity, the same rationale, historical data, and assumptions used 
for cost estimating should be used. Durations should be as short as 
possible and have specific start and end dates. The schedule should be 
continually monitored to determine when forecasted completion dates 
differ from planned dates; this information can be used to determine 
whether schedule variances will affect subsequent work; 
Criterion met (July 2010): Partially met; 
Criterion met (October 2010): Partially met; 
GAO analysis: Initial analysis: The majority of remaining activities 
meet best practices for durations. There are 21 (6 percent) remaining 
activities with baseline (planned) durations longer than 44 days, 
which exceeds the best practice for activity duration. Additionally, 
no activity duration exceeds 84 days, and the two longest duration 
activities represent data collection efforts being performed by 
stakeholders outside of the EBSP program office. Representing effort 
in the schedule that is performed by outside organizations is 
considered a best practice because it keeps management informed of 
ongoing work that might easily be forgotten until the deliverable is 
due, and the impact on future activities if the deliverable is behind 
schedule. 
Schedule activities are based on one of three work calendars: a 7-day 
workweek, a 5-day workweek, and the default 5-day workweek provided 
with the scheduling software. Our analysis found the majority of 
remaining activities (59 percent) are assigned to the default 
calendar, which does not account for holidays. In addition, there are 
168 activities (33 percent) assigned to a 7-day workweek calendar, 
which also does not account for holidays. EBSP officials stated that 
some activities, such as testing, occur continuously regardless of 
weekends or holidays and are, therefore, assigned to the 7-day 
workweek calendar. However, we found activities such as site design, 
equipment delivery, integration and installation, and documentation 
tasks scheduled to occur on weekends. 
Updated analysis: The updated schedule meets best practices for 
durations, with 94 percent of remaining activities being less than 44 
days, and 99 percent less than 45 days. The schedule includes five 
calendars, four of which represent 8-hour, 5-day workweeks and account 
for holidays. However, the majority of remaining activities (61 
percent) is based on a 7-day calendar that does not account for 
holidays. Activities based on a 7-day workweek with no holidays 
include tasks such as documentation, data analysis, training, and 
report deliveries. While it is possible that activities assigned to 
the 7-day calendar are related to activities that can run through 
holidays, some of the activities assigned to this calendar do not 
typically belong on a 7-day calendar. 

Best practice: 5. Integrating schedule activities horizontally and 
vertically; 
Explanation: The schedule should be horizontally integrated, meaning 
that it should link products and outcomes associated with other 
sequenced activities. These links are commonly referred to as 
"handoffs" and serve to verify that activities are arranged in the 
right order to achieve aggregated products or outcomes. The schedule 
should also be vertically integrated, meaning that the dates for 
starting and completing activities in the integrated master schedule 
should be aligned with the dates for supporting tasks and subtasks. 
Such mapping or alignment among levels enables different groups to 
work to the same master schedule; 
Criterion met (July 2010): Partially met; 
Criterion met (October 2010): Partially met; 
GAO analysis: Initial Analysis: The schedule is vertically integrated, 
with low-level tasks and milestones being traceable to higher-level 
summary tasks. However, issues with sequencing logic prevent the 
schedule from being fully horizontal-integrated. Extending durations 
of some key activities had no effect on future milestone activities. 
For example, increasing the duration of the "Evaluate Pricing" 
activity associated with the first contract award from 40 days to 500 
days has no effect on any of three contract award dates. Similarly, 
extending the duration of the first contracting activity, "Request 
decision to move forward with purchase" from 1 day to 90 days has no 
effect on any of the three contract award dates. 
Updated analysis: Issues remain with horizontal integration because 
extending the duration of some key activities to extremely long values 
does not affect future activities. For example, increasing the 
currently in-progress, non-critical task "[OT&E] Site Surveys" from 45 
days to 500 days does not cause the task to become critical and has no 
effect on successor activities, including the first contract award 
date. Similarly, extending the durations of four activities related to 
Windows 1 and 2 Qualification Data Packages to 500 days each has no 
effect on future activities, including the first contract award. 

Best practice: 6. Establishing the critical path for all activities; 
Explanation: Scheduling software should be used to identify the 
critical path, which represents the chain of dependent activities with 
the longest total duration. Establishing a project's critical path is 
necessary to examine the effects of any activity slipping along this 
path. Potential problems along or near the critical path should also 
be identified and reflected in scheduling the duration of high-risk 
activities; 
Criterion met (July 2010): Minimally met; 
Criterion met (October 2010): Minimally met; 
GAO analysis: Initial analysis: Our analysis could not determine a 
valid critical path within the EBSP schedule due to missing logic 
links and incorrect statusing of activities. The EBSP schedule 
critical path begins via a data collection activity that is not 
logically linked to any predecessor activities, has a constrained 
start date, and is marked as 100 percent complete on June 18, 2010--2 
months into the future according to the schedule status date of April 
16, 2010. The critical path continues through two successor activities 
that are marked as having started several months into the future as 
well; then it continues through 24 successor activities until the 
third contract award milestone of December 12, 2011. However, 
according to the schedule, 11 of these critical activities are 334 
days behind schedule; 3 activities are 341 days behind schedule, and 
10 are 477 days behind schedule. In other words, these critical 
activities are shown as between 1.3 and 1.8 working years behind 
schedule, assuming a 5-day workweek calendar. As there are 612 
calendar days left until the project completion date of December 19, 
2011 (approximately 2.3 working years), negative float fewer than -300 
days does not appear to be a realistic representation of the true 
status of these activities. 
Schedules should include complete logic that addresses the 
relationships between predecessor and successor activities because any 
activity can become critical under some circumstances. Without clear 
insight into a critical path at the project level, management will not 
be able to monitor critical or near-critical detail activities that 
may have a detrimental impact on downstream activities if delayed. 
Updated analysis: Our analysis could not determine a valid critical 
path for the updated schedule. The critical path begins with an 
equipment delivery activity scheduled for November 14, 2010--7 months 
into the future relative to the status date, and one month into the 
actual future relative to the actual date of this analysis. A given 
activity cannot start in the future -that is, activities cannot 
actually start one month from now. Additionally, the status date is 
the time of the last update to the schedule, yet the critical path 
starts 7 months into the future relative to this status date. 
Therefore, the critical path starts at the wrong relative time, and 
the wrong actual time. The critical path continues through 25 
activities, which range in float values from -230 days to -336 days. 
In other words, according to the new revised plan, the 26 activities 
driving the revised date for the first contract award are 10 to 15 
months behind schedule, assuming a 5-day workweek calendar. 

Best practice: 7. Identifying reasonable float between activities; 
Explanation: The schedule should identify the float--the amount of 
time by which a predecessor activity can slip before the delay affects 
successor activities--so that a schedule's flexibility can be 
determined. As a general rule, activities along the critical path have 
the least float. Total float is the total amount of time by which an 
activity can be delayed without delaying the project's completion, if 
everything else goes according to plan; 
Criterion met (July 2010): Minimally met; 
Criterion met (October 2010): Minimally met; 
GAO analysis: Initial analysis: Our analysis found that float 
calculations within the schedule are not reliable because of missing 
logic links and the high number of date constraints. If the schedule 
is missing dependencies or if activities are linked incorrectly, float 
estimates will be miscalculated because float is directly related to 
the logical sequencing of events. 
For example, 112 remaining activities (22 percent) have more than 300 
days of total float meaning that, according to the schedule, these 
activities can slip more than one work-year and not affect the finish 
milestone. This includes 25 remaining activities (5 percent of all 
remaining activities) with 500 or more days of float. Furthermore, 
there are 125 remaining activities (25 percent) with negative float. 
Negative float implies that there is not enough time scheduled for the 
task. Of the activities with negative float, 98 remaining activities 
(20 percent) appear as more than 50 days behind schedule, including 40 
activities (8 percent) that are 300 or more days behind schedule. As 
there are 612 calendar days left until the project completion date of 
December 19, 2011, float greater than 300 days or fewer than -300 days 
does not appear to be a realistic representation of the true status of 
these activities. 
Without reliable float estimates management may be unable to allocate 
resources from non-critical activities to activities that cannot slip 
without affecting the project finish date. 
Updated analysis: The schedule continues to reflect unreasonable 
amounts of negative float. Program officials stated that the negative 
float is a result of changes in the acquisition strategy resulting in 
a change to activity start dates. However, 98 percent of all remaining 
activities have negative float, ranging from -336 to -90 days. Only 
five remaining activities have positive float, which ranges from 10 to 
486 days. 

Best practice: 8. Conducting a schedule risk analysis; 
Explanation: A schedule risk analysis should be performed using 
statistical techniques to predict the level of confidence in meeting a 
project's completion date. This analysis focuses not only on critical 
path activities, but also on activities near the critical path since 
they can affect the project's status; 
Criterion met (July 2010): Not met; 
Criterion met (October 2010): Not met; 
GAO analysis: Initial analysis: The program office has not performed a 
schedule risk analysis (SRA). Officials stated that because DHS does 
not require an SRA, TSA has no contracts in place to support 
conducting the analysis and no knowledgeable support is available 
within TSA to perform an SRA. Program officials stated that in their 
opinion, an SRA appears to be too time-consuming and that they have no 
plans to seek support to perform the analysis. 
A comprehensive schedule risk analysis is an essential tool for 
decision makers. An SRA can be used to determine a level of confidence 
in meeting the completion date or whether proper reserves have been 
incorporated into the schedule. An SRA will calculate schedule 
reserve, which can be set aside for those activities identified as 
high-risk. Without this reserve, the program faces the risk of delays 
to the scheduled completion date if any delays were to occur on 
critical path activities. However, if the schedule risk analysis is to 
be credible, the program must have a quality schedule that reflects 
reliable logic and clearly identifies the critical path--conditions 
that the schedule does not meet. 
Updated analysis: No change to initial assessment. 

Best practice: 9. Updating the schedule using logic and durations to 
determine dates; 
Explanation: The schedule should be continuously updated using logic 
and durations to determine realistic start and completion dates for 
program activities. The schedule should be analyzed continuously for 
variances to determine when forecasted completion dates differ from 
planned dates. This analysis is especially important for those 
variations that affect activities identified as being on a project's 
critical path and can affect a scheduled completion date; 
Criterion met (July 2010): Minimally met; 
Criterion met (October 2010): Minimally met; 
GAO analysis: Initial analysis: EBSP officials stated that EBSP 
conducts weekly meetings regarding the schedule and updates the status 
accordingly. From these weekly meetings, officials generate weekly 
reports that identify key areas of concern with regard to schedule 
shifts and potential impacts on milestones. However, our analysis of 
the schedule shows 30 activities (6 percent) that should have started 
and finished according to the schedule status date, but do not have 
actual start or finish dates. In addition, as previously mentioned, 
the critical path in part consists of activities that are marked as 
having started several months into the future. A status date denotes 
the date of the latest update to the schedule and thus defines the 
point in time at which completed work and remaining work are 
calculated. The schedule also contains 43 activities (9 percent) with 
actual start and finish dates in the future relative to the schedule 
status date. 
EBSP officials stated that relevant scheduling guidance, such as 
program or agency directives that govern the creation, maintenance, 
structure, and statusing of the schedule, does not exist. However, the 
schedule should be continually monitored to determine when forecasted 
completion dates differ from the planned dates, which can be used to 
determine whether schedule variances will affect downstream work. 
Maintaining the integrity of the schedule logic is not only necessary 
to reflect true status, but is also required before conducting a 
schedule risk analysis. 
Updated analysis: The updated schedule has the same April 16, 2010, 
status date as the original schedule, which EBSP officials stated is 
an error that will be corrected in future versions. Officials stated 
that while the status date was not correctly updated, updates to 
scheduled activities have been made to reflect actual dates on a 
weekly basis. However, the incorrect status creates the impression 
that 34 activities have occurred in the future. The updated schedule 
records actual start and actual finish dates for all completed 
activities but one. This activity should have finished on April 2, 
2009, but has no actual finish date. 

Source: GAO analysis of TSA information. 

Notes: 

Fully met: The program provided complete evidence that satisfies all 
of the entire criteria for the identified best practice. 

Partially met: The program provided evidence that satisfies about half 
of the criteria for the identified best practice. 

Minimally met: The program provided evidence that satisfies less than 
half of the criteria for the identified best practice. 

Not met: The program did not provide evidence that satisfies any of 
the criteria for the identified best practice. 

[A] In its initial schedule provided to GAO in July 2010, TSA 
established planned dates for the first three EDS contract awards. 
However, TSA's revised schedule provided to GAO in October 2010 plans 
for activities only through the first EDS contract award scheduled for 
July 2011. 

[End of table] 

[End of section] 

Appendix IV: Comments from the Department of Homeland Security: 

U.S. Department of Homeland Security: 
Washington, DC 20528: 

July 6, 2011: 

Stephen M. Lord: 
Director, Homeland Security and Justice: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Re: Draft Report GA0-11-740, "Aviation Security: TSA Has Enhanced its
Explosives Detection Requirements for Checked Baggage, but Additional 
Screening Actions Are Needed" 

Dear Mr. Lord: 

Thank you for the opportunity to review and comment this draft report. 
The U.S. Department of Homeland Security (DHS) appreciates the U.S. 
Government Accountability Office's (GAO) work in planning and 
conducting its review and issuing this report. 

Because of the urgent need of the program to meet ongoing 
requirements, the Transportation Security Administration (TSA) began 
actively resolving many of the issues identified in the audit in 
parallel to the audit being conducted. Contracts used by TSA to meet 
ongoing needs for Explosives Detection Systems (EDS) will expire in 
2011 and must be replaced to enable TSA to continue meeting baggage 
screening requirements. TSA also awarded 41 facility modifications to 
36 airports under the American Recovery and Reinvestment Act and 
fiscal year (FY) 2009 funding. As a result, a critical need exists to 
meet airport schedules for installation of EDS into baggage handling 
systems that are currently being built. To meet these demands for 
equipment, TSA began modifying its strategy for the EDS competitive 
procurement in 2010. This modification removed data collection from 
the EDS procurement critical path and will enable TSA to achieve 
higher levels of detection for EDS more quickly than would have been 
achieved through the original procurement strategy. 

Additionally, TSA must procure and install large quantities of EDS 
equipment to replace the aging fleet of EDS. TSA began developing a 
recapitalization plan for EDS in 2010 with the goal of prioritizing 
projects and leveling out the recapitalization of EDS equipment while 
continuing to meet emerging operational requirements of ongoing 
airport projects. As the recapitalization effort moves forward, TSA 
will replace this equipment with systems possessing higher levels of 
detection. 

TSA's Strategy for Increasing Detection Capability: 

In December 2007. TSA decided to suspend the dill implementation of 
the 2005 standard, including an operational impact evaluation, because 
of a computer modeling effort known as "Project Newton," which had the 
potential to modify the 2005 detection standard. When Project
Newton did not yield timely results. TSA published the 2010 detection 
standard that was a requirement for the EDS Competitive Procurement 
Process planned for contract awards in 2011, TSA is currently taking 
steps to collect the operational data necessary to support the upgrade 
of deployed equipment based on 2010 detection standards. This data 
collection effort to support the upgrade is planned to be completed in 
2011. In addition, vendor proposals for upgrading the existing fleet 
to the 2010 detection standard have been requested and contract awards 
for upgrades are planned to be complete by the end of 2011. While 
achieving deployment of the 2010 detection standard, TSA is also 
working with the OHS Science and Technology (S&T) Directorate toward 
establishing a laboratory by the summer of 201 1 to support further 
vendor data collection and independent test and evaluation. 

It is important to note that the detection requirements are dynamic in 
nature due to continuous intelligence driven threat assessments. As 
noted in the GAO report, TSA's current EDS competitive procurement 
strategy separates the procurement into multiple windows; only the 
first window's schedule has been outlined at this time. Each phase 
will require an incremental increase in detection capability, which 
reduces program risk, increases opportunities for vendors to qualify 
their technology, and enables deployment of the most current detection 
capability possible to the field. Achieving all detection levels will 
require collaborative research and development on the part of DEIS 
S&T, TSA, and the vendor community. 

Additionally, TSA is developing a plan to address Explosives Trace 
Detection (ETD) units to resolve secondary alarms during the checked 
baggage screening process. Deployed technology meets current detection 
needs but TSA may need to pursue technology solutions to address the 
next level of detection within the necessary timeframe. The initial 
planning stages of this effort have already been started, and TSA is 
actively coordinating with DEIS S&T to identify available technologies. 

Vendor Communication: 

TSA significantly changed the business model for procurement of 
checked baggage screening equipment from what has historically been a 
sole source environment to a competition driven environment resulting 
in significant improvements in communication with industry. The 
changes TSA implemented drove business model changes on behalf of the 
equipment manufacturers that are now forced to compete for contract 
awards. Throughout the competitive procurement, TSA Electronic Baggage 
Screening Program (EBSP) leadership worked closely with the vendors 
and provided numerous opportunities for industry to offer feedback to 
the program. These opportunities included an initial kick-off with all 
vendors (2009); multiple industry days (2009 and 2010); individual 
classified sessions to review detection requirements (2010); and the 
publication of advance copies of the EDS specifications (2009) to 
offer vendors the opportunity to comment on planned detection and 
operational requirements prior to the time the competitive procurement 
process began and finalization of the requirements. In addition,
TSA leadership visited each vendor site in 2009 to discuss the 
upcoming procurement and concerns of the vendors, and to communicate 
ISA's procurement plans. 

As a result of this extensive engagement and feedback from industry 
subject-matter experts on the capabilities of EDS technology today, 
TSA revised the detection and operational requirements, as well as the 
procurement strategy and schedule, to promote competition and to drive 
enhanced EDS capabilities. This included revising the acquisition 
schedule to move the release of the Request for Proposal after the 
testing process had been significantly completed to promote continued 
open communication between TSA and the vendor community. TSA's ability 
to adhere to defined plans and schedules relies partially with 
industry's ability to meet their own commitments. As the acquisition 
process continues to advance, vendor performance has not always been 
consistent with the commitments that were made. 

TSA learned of industry concerns through these engagements and made 
the following adjustments to ensure these concerns were addressed: 

* TSA implemented bailment/Other Transaction Agreements with each 
vendor to pay for engineering support services throughout the testing 
and data collection processes. — The Operational Requirements Document 
was modified to reflect minimum and non-minimum requirements. 

* Additional time was added to the schedule to support Operational 
Test and Evaluation after completion of certification testing and 
integration testing. 

* Data was collected on each system submitted by each vendor for the 
procurement to support algorithm development. 

* Multiple windows of opportunity to compete in the procurement were 
created to allow industry the opportunity to go through the 
certification and testing process multiple times and to promote 
competition. 

In conclusion, TSA believes that its modified strategy for the EDS 
competitive procurement will increase detection capability in the most 
achievable, cost-effective approach possible. The Agency has taken a 
number of steps to address the issues discussed in this report and is 
actively developing plans to ensure that airports across the country 
are operating with the latest detection capability. Additionally, TSA 
has been responsive to industry, and extensive communication has taken 
place to promote competition; communicate TSA goals, objectives, and 
requirements; and to ensure success of both TSA and industry in the 
competitive procurement process. 

Recommendations: 

The draft report contained six recommendations. As discussed below, 
DHS concurs with all six recommendations. Specifically, to help ensure 
that TSA takes a comprehensive and cost effective approach to the 
procurement and deployment of EDSs that meet the 2010 EDS requirements 
and any subsequent revisions. GAO recommended that the Assistant 
Secretary for TSA take the following actions: 

Recommendation 1: Develop a plan to ensure that screening devices or 
protocols are in place to resolve EDS alarms if EDS are deployed that 
detect a broader set of explosives than existing ETD machines used to 
resolve EDS screening alarms. 

Response: Concur. TSA recognizes that evolving detection standards for 
primary EDS screening may require an enhanced portfolio of 
technologies or procedures to support secondary threat resolution. TSA 
will be able to meet current detection needs but will pursue 
technology solutions to address the next level of detection 
requirements within the necessary timeframe. TSA convened a working 
group to assess capability gaps for secondary screening technology, 
evaluate current technology capabilities against the capabilities of 
future EDS, and prepare a plan to procure any additional technology 
required to ensure alarms can be resolved. This plan will be finalized 
by the fourth quarter of FY 2012. TSA is also collaborating with DHS 
S&T to drive research and development efforts toward the next level of 
trace detection technology. 

Recommendation 2: Develop a plan to ensure that TSA has the data 
needed for each of the planned phases of the 2010 EDS requirements 
before starting the procurement process for new EDS or upgrades 
included in each applicable phase. 

Response: Concur. TSA modified its strategy for the EDS competitive 
procurement in July 2010 in response to challenges regarding safety 
issues and characteristics of the explosives used for data collection, 
and posted this change to the vendor community through Amendment 13 on
September 3, 2010. The new baseline schedule outlines an incremental 
development process for detection that removes data collection from 
the acquisition process. Additionally, TSA is working with DHS S&T to 
establish a laboratory by summer 2011 to support further vendor data 
collection and independent test and evaluation. 

Recommendation 3: Establish a process to communicate information to 
EDS vendors in a timely manner regarding TSA's EDS acquisition, 
including information such as changes to the schedule. 

Response: Concur. TSA has a process for communicating information to 
the vendor community and will continue to follow this process in 
adherence with guidelines outlined in the Federal Acquisition 
Regulation. The Agency has already made a number of efforts to improve 
the quality and frequency of communication with industry as detailed 
earlier. However, TSA recognizes the complexity associated with many 
of the acquisitions currently ongoing. As such, TSA acknowledges there 
are opportunities to continue to improve communications with the 
vendor community and looks forward to ensuring that vendors are 
provided with the most current information possible in an efficient 
manner. 

Recommendation 4: Develop and maintain an Integrated Master Schedule 
for the entire Electronic Baggage Screening Program in accordance with 
the nine best practices identified by GAO for preparing a schedule. 

Response: Concur. TSA has already begun working with key stakeholders 
to develop and define requirements for an Integrated Master Schedule 
(IMS) and ensure that our schedule is in alignment with the best 
practices outlined by GAO. This effort is expected to be completed by 
the second quarter of FY 2012. As the program matures and increases 
its focus on flexible and upgradable technology, an IMS will ensure 
close coordination between the program's procurement, deployment, 
recapitalization, and upgrade capabilities. The EBSP IMS will he 
updated as a result of these efforts to be in accordance with the nine 
best practices. 

Recommendation 5: Ensure that key elements of the program's final cost 
estimate reflect critical issues, such as the potential cost impacts 
resulting from schedule slippage identified once an Integrated Master 
Schedule for the Electronic Baggage Screening Program has been 
developed in accordance with the nine best practices identified by GAO 
for preparing a schedule. 

Response: Concur. TSA agrees that the program's cost estimate should 
reflect potential cost increases and schedule risks associated with 
enhanced detection capabilities. TSA is working to update the EBSP 
life cycle cost estimate (LCCE) to incorporate cost estimates 
associated with enhanced detection, which should be complete in the 
fourth quarter of FY 2011. In addition, per recommendations of both 
GAO and DHS. TSA is also developing a master schedule to document 
timelines associated with various projects. Risks to the costs and 
schedules will be analyzed within the LCCE qualitatively in text and 
quantitatively via a scenario-based risk analysis approach recommended 
by DHS. The risk analysis will produce confidence intervals for the 
life cycle costs to the program. 

Recommendation 6: Develop a plan to deploy EDS that meets the most 
recent EDS explosives detection requirements and ensure that new 
machines. as well as machines deployed in airports, will be operated 
at the levels established in those requirements. This plan should 
include the estimated costs for new machines and upgrading deployed 
machines, and the timeframes for procuring and deploying new machines 
and upgrading deployed machines. 

Response: Concur. TSA has a plan in place to evaluate and implement 
the most recent certified algorithms on the existing fleet of deployed 
EDS, assuming the evaluation results has minimal to no operational 
impact. 

Again, thank you for the opportunity to review and comment on this 
draft report. Sensitivity and technical comments on the report have 
been provided under separate cover. We look forward to working with 
you on future Homeland Security engagements. 

Sincerely, 

Signed by: 

Jim H. Crumpacker: 
Director: 
Departmental GAO/OIG Liaison Office: 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Stephen M. Lord, (202) 512-8777 or LordS@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Glenn Davis, Assistant 
Director, and Joseph E. Dewechter, Analyst-in-Charge, managed this 
assignment. Scott Behen, Samantha Carter, and Orlando Copeland made 
major contributions to the planning and all other aspects of the work. 
David Alexander and Richard Hung assisted with design, methodology, 
and data analysis. Jason Lee and Karen Richey assisted with 
acquisition cost and schedule analysis. John Hutton and Nathan 
Tranquilli assisted with acquisition and contracting issues. Katherine 
Davis provided assistance in report preparation. Thomas Lombardi and 
Tracey King provided legal support. 

[End of section] 

Footnotes: 

[1] As of January 2011, TSA provides for, or oversees the provision 
of, screening and other security activities at 462 airports in the 
United States that operate under TSA-approved security programs 
pursuant to 49 C.F.R. part 1542, referred to in this report as 
"commercial airports." At some of these airports, a private-screening 
company under contract to TSA and subject to TSA oversight carries out 
the screening function using private screeners through TSA's Screening 
Partnership Program. See 49 U.S.C. § 44920. 

[2] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: November 1999). 

[3] GAO, GAO Cost Estimating and Assessment Guide, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[4] See, for example, GAO, Department of Homeland Security: 
Assessments of Selected Complex Acquisitions, [hyperlink, 
http://www.gao.gov/products/GAO-10-588SP] (Washington, D.C.: June 
2010). 

[5] Aviation and Transportation Security Act (ATSA), Pub. L. No. 107- 
71, 115 Stat. 597 (2001). 

[6] Pub. L. No. 107-71, § 110(b), 115 Stat. at 614-16 (codified as 
amended at 49 U.S.C. § 44901). 

[7] 49 U.S.C. § 44901(d)-(e) (as amended, requiring that TSA deploy 
sufficient equipment to ensure that all checked baggage be screened 
using explosives detection equipment (EDS and ETD) as soon as 
practicable but in no event later than December 31, 2003). 

[8] Consistent with the Homeland Security Act of 2002, DHS undertook 
to coordinate and integrate most of its research, development, 
demonstration, testing, and evaluation activities within S&T. See Pub. 
L. No. 107-296, § 302(12) 116 Stat. 2135, 2164 (2002). Whereas TSA 
received a separate appropriation for research and development 
activities related to aviation security through fiscal year 2005, 
beginning in fiscal year 2006, TSA research and development activities 
were rolled into the S&T appropriation. 

[9] According to TSA officials, 359 EDS units are currently stored in 
warehouses. EDS vendors deliver new EDS machines to TSA's warehouses 
where they are prepared for deployment to airports. 

[10] TSA also certifies the ETD that it deploys to commercial airports 
for screening checked baggage; however, this report primarily focuses 
on the EDS. 

[11] See 49 U.S.C. § 44913(a)(1). 

[12] See 49 U.S.C. § 44912(b). 

[13] The specific details regarding this issue are considered 
sensitive security information. 

[14] For purposes of this report, unlike TSA officials who defined the 
implementation of EDS explosives detection requirements as having 
occurred at the time that EDSs are certified, we define implementation 
as having occurred at the time that EDSs are certified, purchased, and 
deployed to airports nationwide. 

[15] A false alarm is when the system alarms even though a threat is 
not present. A false alarm rate is defined as the percentage of times 
that a false alarm occurs in a given number of trials. In addition to 
a false alarm rate, there are other metrics for system performance, 
such as the detection rate. 

[16] TSA officials told us that although this testing was not 
completed prior to pursuing procurements for previous acquisitions, 
they plan to complete this testing as a part of the current EDS 
acquisition. 

[17] TSA officials have reported that all EDSs purchased under the 
2010 requirements will have 2010 detection software installed and 
activated once the machines are deployed. Thus, new machines certified 
to meet the 2010 requirements will operate at these detection levels 
in airports. 

[18] The 2010 EDS explosives detection requirements contain classified 
information; therefore, this report does not discuss the names of the 
explosives or other specific information about the explosives. 

[19] See, for example, GAO, Defense Acquisitions: Restructured JTRS 
Program Reduces Risk, but Significant Challenges Remain, [hyperlink, 
http://www.gao.gov/products/GAO-06-955] (Washington, D.C.: September 
2006). 

[20] We refer to the screening protocols to identify that they are 
part of the checked-baggage screening process; although we reviewed 
the protocols, we did not evaluate them because we focused on the 
checked-baggage screening technologies for this report. 

[21] Project Management Institute, The Standard for Program Management 
(2006). 

[22] The Federal Acquisition Regulation defines a "sole source 
acquisition" as "a contract for the purchase of supplies or services 
that is entered into or proposed to be entered into by an agency after 
soliciting and negotiating with only one source." See 48 C.F.R. § 
2.101(b). For purposes of this report, the terms "acquisition" and 
"procurement" are interchangeable. 

[23] See H.R. Conf. Rep. No. 111-298 (accompanying H.R. 2892, 
Department of Homeland Security Appropriations Act, 2010) (Oct. 13, 
2009) at 78 (providing that TSA shall move to a fully competitive EDS 
procurement process no later than September 30, 2010). 

[24] The acquisition strategy's objectives also include meeting the 
EBSP mission needs by minimizing life-cycle costs, minimizing impact 
to airport operations, and providing a flexible security 
infrastructure capable of meeting future airline traffic and industry 
needs. 

[25] TSA's Passenger Screening Program acquires ETD machines on behalf 
of the EBSP for use in checked-baggage screening. 

[26] The AFRL at Tyndall Air Force Base is supporting S&T, TSL, and 
TSA in developing explosives data through fiscal year 2012. S&T is in 
the process of identifying other facilities that can safely test 
explosives. 

[27] A simulant is a "safe to handle" material designed to appear to 
EDS technology as a real explosive. 

[28] The EDS uses a computed tomography X-ray source that rotates 
around a bag, obtaining a large number of cross-sectional images that 
are integrated by a computer which automatically triggers an alarm 
when objects with the characteristic of explosives are detected. 

[29] See, for example, GAO, Joint Strike Fighter: Significant 
Challenges Remain as DOD Restructures Program, [hyperlink, 
http://www.gao.gov/products/GAO-10-520T] (Washington, D.C.: March 11, 
2010). 

[30] Project Management Institute, The Standard for Program Management 
© (Second Edition, 2008). 

[31] Officials from the sixth vendor did not comment on the extent to 
which TSA responded to questions in a timely manner. 

[32] TSA established the EBSP as a long-term program to procure, test, 
deploy, and maintain checked-baggage-screening equipment; it is 
expected to reach full capacity in 2019. 

[33] See, for example, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP]; and OMB, Capital Programming 
Guide V 2.0, Supplement to Office of Management and Budget Circular A-
11, Part 7: Planning, Budgeting, and Acquisition of Capital Assets 
(Washington, D.C.: June 2006). 

[34] TSA refers to the first planned EDS contract award as Window 1 
and subsequent contract awards as Windows 2 and 3, respectively. 
However, TSA has not yet established contract award dates for Windows 
2 and 3 in the schedule. 

[35] We assessed TSA's initial schedule, which was provided to GAO in 
July 2010; met with TSA officials to discuss our assessment; and 
provided officials with suggestions for corrective action. 
Subsequently, TSA revised the schedule and provided it to GAO in 
October 2010. We completed a separate assessment of TSA's revised 
schedule. 

[36] GAO Cost Estimating and Assessment Guide, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[37] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[38] Life-cycle costs include all resources and associated cost 
elements required to develop, produce, deploy, and sustain a 
particular program from initial concept through operations, support, 
and disposal. Acquisition costs include costs for all supplies and 
services for a designated investment. 

[39] According to DHS, the Director, Cost Analysis Division, located 
in the Office of the Chief Procurement Officer, serves as the focal 
point within DHS for cost analysis and estimating policy, process, and 
procedure. 

[40] The DHS Acquisition Review Board is responsible for reviewing 
major DHS acquisitions for executable business strategy, resources, 
management, accountability, and alignment to strategic initiatives. 
According to DHS, the EBSP is either at or very near the top of the 
Acquisition Review Board's priority list. 

[41] GAO, Department of Homeland Security: Assessments of Selected 
Complex Acquisitions, [hyperlink, 
http://www.gao.gov/products/GAO-10-588SP] (Washington, D.C.: June 
2010). 

[42] See [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[43] DHS endorsed the use of best practices outlined in GAO's 2007 
Cost Estimating Guide [hyperlink, 
http://www.gao.gov/products/GAO-07-1134SP]. 

[44] According to TSA officials, while the current fleet of EDSs 
totals 2,297 machines, 1,938 of the machines are currently deployed in 
airports. 

[45] Project Management Institute, The Standard for Program Management 
(2006). 

[46] TSA officials stated that, if they purchase EDSs that meet a part 
of the Level C requirements, they plan to upgrade those machines to 
meet other parts of the requirements. Once EDSs are purchased to meet 
all parts of the Level C requirements, the machines will need to be 
upgraded to meet Levels B and A requirements. 

[47] Officials from another vendor stated that upgrades could be made 
to the EDSs in airports, but did not specify when the upgrades could 
be made. Officials from the other two EDS vendors do not have EDSs 
currently deployed in airports and, therefore, did not comment on the 
feasibility of upgrades. 

[48] The four other EDS vendors did not have cost estimates for 
upgrades to existing machines. 

[49] Computer model validation includes steps to ensure that a model 
sufficiently simulates the actual system. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: