Aviation Security:

DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges

GAO-10-128: Published: Oct 7, 2009. Publicly Released: Oct 29, 2009.

Additional Materials:

Contact:

Stephen M. Lord
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Since fiscal year 2002, the Transportation Security Administration (TSA) and the Department of Homeland Security (DHS) have invested over $795 million in technologies to screen passengers at airport checkpoints. The DHS Science and Technology Directorate (S&T) is responsible, with TSA, for researching and developing technologies, and TSA deploys them. GAO was asked to evaluate the extent to which (1) TSA used a risk-based strategy to prioritize technology investments; (2) DHS researched, developed, and deployed new technologies, and why deployment of the explosives trace portal (ETP) was halted; and (3) DHS coordinated research and development efforts with key stakeholders. To address these objectives, GAO analyzed DHS and TSA plans and documents, conducted site visits to research laboratories and nine airports, and interviewed agency officials, airport operators, and technology vendors.

TSA completed a strategic plan to guide research, development, and deployment of passenger checkpoint screening technologies; however, the plan is not risk-based. According to TSA officials, the strategic plan and its underlying strategy for the Passenger Screening Program were developed using risk information, such as threat information. However, the strategic plan and its underlying strategy do not reflect some of the key risk management principles set forth in DHS's National Infrastructure Protection Plan (NIPP), such as conducting a risk assessment based on the three elements of risk--threat, vulnerability, and consequence--and developing a cost-benefit analysis and performance measures. TSA officials stated that, as of September 2009, a draft risk assessment for all of commercial aviation, the Aviation Domain Risk Assessment, was being reviewed internally. However, completion of this risk assessment has been repeatedly delayed, and TSA could not identify the extent to which it will address all three elements of risk. TSA officials also stated that they expect to develop a cost-benefit analysis and establish performance measures, but officials could not provide timeframes for their completion. Without adhering to all key risk management principles as required in the NIPP, TSA lacks assurance that its investments in screening technologies address the highest priority security needs at airport passenger checkpoints. Since TSA's creation, 10 passenger screening technologies have been in various phases of research, development, test and evaluation, procurement, and deployment, but TSA has not deployed any of these technologies to airports nationwide. The ETP, the first new technology deployment initiated by TSA, was halted in June 2006 because of performance problems and high installation costs. Deployment has been initiated for four technologies--the ETP in January 2006, and the advanced technology systems, a cast and prosthesis scanner, and a bottled liquids scanner in 2008. TSA's acquisition guidance and leading commercial firms recommend testing the operational effectiveness and suitability of technologies or products prior to deploying them. However, in the case of the ETP, although TSA tested earlier models, the models ultimately chosen were not operationally tested before they were deployed to ensure they demonstrated effective performance in an operational environment. Without operationally testing technologies prior to deployment, TSA does not have reasonable assurance that technologies will perform as intended. DHS coordinated with stakeholders to research, develop, and deploy checkpoint screening technologies, but coordination challenges remain. Through several mechanisms, DHS is taking steps to strengthen coordination within the department and with airport operators and technology vendors.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In October 2009, we reported that the Transportation Security Administration (TSA) had completed a strategic plan to guide research, development, and deployment of passenger checkpoint screening technologies; however, the plan was not risk-based. Specifically, the strategic plan and its underlying strategy for the Passenger Screening Program (PSP) were developed using risk information, such as threat information, but were not based on the three elements of risk--threat, vulnerability, and consequence. We recommended that TSA conduct a complete risk assessment, including threat, vulnerability, and consequence assessments, which would apply to the PSP. TSA concurred and, in response, initiated approaches to identify risks at both the strategic and detailed level. At the strategic level, in March 2010, TSA updated its Aviation Modal Risk Assessment (AMRA), previously called the Aviation Domain Risk Assessment, which includes a comprehensive threat, vulnerability, and consequence assessment of the risk of a terrorist attack on the international and domestic aviation system. TSA has indicated that it will use AMRA to inform its PSP. At a more detailed level, TSA developed a simulation model that can be used to model specific risk scenarios and risk-reduction potential of specific technologies or other countermeasures. According to TSA, the tool will allow the agency to perform analysis at a level not previously available while not disrupting real world checkpoint operations. TSA indicated that the outputs of this tool are being used to inform the strategic planning and research and development processes. This recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the research, development, test and evaluation (RDT&E), procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should conduct a complete risk assessment, including threat, vulnerability, and consequence assessments, which would apply to the Passenger Screening Program (PSP).

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  2. Status: Closed - Implemented

    Comments: In October 2009, we reported that the Transportation Security Administration (TSA) had not completed a cost-benefit analysis to prioritize and fund the Passenger Screening Program's (PSP) priorities for investing in checkpoint technologies, as required by the National Infrastructure Protection Plan's (NIPP) risk management framework. At the time of our report, TSA had not developed life-cycle cost estimates of each screening technology the PSP is developing, procuring, or deploying, and could not provide us with information on their priorities for the research and development of checkpoint screening technologies or the processes they followed to develop these priorities. We recommended that TSA develop cost-benefit analyses to assist in prioritizing investments in new checkpoint screening technologies. TSA concurred and completed a life-cycle cost estimate for the PSP in November 2011. While completing a life-cycle cost estimate was an important first step in addressing our recommendation, it did not include cost-benefit analyses to assist in prioritizing investments in new checkpoint screening technologies, as required by the NIPP. In January 2014, TSA completed a cost-benefit analysis of a new checkpoint screening technology that compared the benefits of adding that technology to the passenger screening checkpoint environment with the overall cost to develop, deploy, and implement the technology. In doing so, TSA analyzed the technology's impact on overall checkpoint system security effectiveness, impact on the traveling public, staff utilization, and passenger throughput. While TSA does not require that cost-benefit analyses be conducted for all new technologies in the PSP, TSA's audit liaison told us in April 2014 that program officials said that they intend to conduct a cost-benefit analysis for all future technologies and for the process used for the January 2014 analysis to be repeatable. Weighing the costs and benefits of each checkpoint technology that TSA invests in relative to the costs and benefits of other solutions will help decision makers determine whether investments in technologies or in other security programs will provide the greatest mitigation of risk for the resources that are available. This recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should develop cost-benefit analyses to assist in prioritizing investments in new checkpoint screening technologies.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  3. Status: Closed - Implemented

    Comments: We reported in October 2009 that the Transportation Security Administration's (TSA) strategy for the Passenger Screening Program (PSP) did not have a mechanism - such as performance measures or other evaluation methods - to monitor, assess, or test the extent to which investments in new checkpoint technologies reduce or mitigate the risk of terrorist attacks. We recommended that TSA develop quantifiable performance measures to assess the extent to which investments in research, development, and deployment of checkpoint screening technologies achieve performance goals for enhancing security at airport passenger checkpoints. TSA concurred and, in March 2011, told us that all major technologies that are intended to be deployed to the checkpoint are required to undergo extensive testing in an operational environment during the Operational Testing and Evaluation (OT&E) phase. In addition, TSA said that it has gained an enhanced capability by opening the TSA Systems Integration Facility (TSIF), which allows TSA to mirror the constraints of an operational environment while reducing the impact on the flying public. Finally, according to TSA, baseline comparisons are performed, when relevant, for the previously installed technologies compared to the proposed upgrades to determine if the increase in abilities is acceptable. In addition to the testing performed prior to deployment, TSA uses several methods to evaluate technologies on an ongoing basis to ensure operational effectiveness. For example, TSA has implemented the Aviation Screening Assessment Program (ASAP) during which detection is measured and data are collected to identify trends. Also, TSA performs covert testing of the vulnerabilities of both the technologies and the processes used at the checkpoint. In addition, in January 2014, TSA completed a cost-benefit analysis of a new checkpoint technology that analyzed the technology's impact on overall checkpoint system security effectiveness, impact on the traveling public, staff utilization, and passenger throughput. The quantifiable measures were used to provide TSA decision makers with information to assess and prioritize their investment resources, and ensure they are investing in screening technologies to address the security needs of the passenger screening checkpoint with the highest priority. While TSA does not require that cost-benefit analyses be conducted for all new technologies in the PSP, TSA's audit liaison told us in April 2014 that program officials said that they intend to conduct a cost-benefit analysis for all future technologies and for the process used for the January 2014 analysis to be repeatable. This recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should develop quantifiable performance measures to assess the extent to which investments in research, development, and deployment of checkpoint screening technologies achieve performance goals for enhancing security at airport passenger checkpoints.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  4. Status: Closed - Implemented

    Comments: We reported in October 2009 that the Transportation Security Administration (TSA) completed a strategic plan to guide research, development, and deployment of passenger checkpoint screening technologies; however, the plan was not risk-based. According to TSA officials, the strategic plan and its underlying strategy for the Passenger Screening Program (PSP) were developed using risk information, such as threat information. However, the strategic plan and its underlying strategy did not reflect some of the key risk management principles set forth in DHS's National Infrastructure Protection Plan (NIPP), such as conducting a risk assessment based on the three elements of risk--threat, vulnerability, and consequence--and developing a cost-benefit analysis and performance measures. Without adhering to all key risk management principles as required in the NIPP, TSA lacked assurance that its investments in screening technologies address the highest priority security needs at airport passenger checkpoints. We recommended that TSA, after conducting a complete risk assessment and completing cost-benefit analyses and quantifiable performance measures for the PSP, incorporate the results of these efforts into the PSP strategy as determined appropriate. TSA concurred and told us in January 2014 that the agency was updating its PSP strategic plan and expected to complete it by summer 2014. In June 2014 TSA informed GAO that TSA has moved away from developing strategic plans and has rolled their planning into the Acquisition Program Baseline (APB) which is updated annually. TSA officials stated the APB serves as an operational guide for program management, including laying out the steps to implementing a program and showing the requirements and timeframes needed in rolling out programs. TSA provided an approved APB in September 2014 which included the three elements of risk, and provided quantifiable performance measures for the PSP. As a result of TSA's actions this recommendation is now closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should, after conducting a complete risk assessment and completing cost-benefit analyses and quantifiable performance measures for the PSP, incorporate the results of these efforts into the PSP strategy as determined appropriate.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  5. Status: Closed - Implemented

    Comments: In October 2009, we reported that although the Transportation Security Administration (TSA) had tested earlier models of the explosives trace portal (ETP), the models ultimately chosen were not operationally tested before they were deployed to ensure they demonstrated effective performance in an operational environment. We recommended that TSA, to the extent feasible, ensure that technologies have completed operational tests and evaluations before they are deployed. TSA concurred with the recommendation and has taken actions to address it. For example, TSA implemented a test and evaluation process for all of its technology procurements in accordance with the Department of Homeland Security's policy. This includes developmental testing in the lab and operational testing in the field. Specifically, according to TSA, Operational Test and Evaluation (OT&E) focuses on validating operational effectiveness and suitability. A Test and Evaluation Master Plan (TEMP) is developed to describe the test and evaluation strategy for determining a system's operational effectiveness and suitability. An operational test plan is developed and approved, and an Operational Test Readiness Review is conducted prior to the start of testing. According to TSA, these elements ensure that operational tests and evaluations have been successfully completed before deploying checkpoint screening technologies to airport checkpoints. Subsequently, TSA provided a configuration management plan that, if implemented as required, should ensure that the specific configuration of a checkpoint screening technology that passes OT&E is the same one that is deployed to airports for day-to-day operational use. This recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should, to the extent feasible, ensure that operational tests and evaluations have been successfully completed before deploying checkpoint screening technologies to airport checkpoints.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  6. Status: Closed - Implemented

    Comments: In October 2009, we reported that the Transportation Security Administration (TSA) had relied on technologies in day-to-day airport operations that had not been demonstrated to meet their functional requirements in an operational environment. For example, TSA substituted existing screening procedures with screening by the Whole Body Imager (later known as Advanced Imaging Technology) even though its performance had not yet been validated by testing in an operational environment. We recommended that TSA evaluate whether current passenger screening procedures should be revised to require the use of appropriate screening procedures until TSA determined that existing emerging technologies meet its functional requirements in an operational environment. TSA agreed with this recommendation. Subsequently, TSA provided us with documentation verifying that the agency had compared the effectiveness of its existing screening procedures and technologies that had not yet completed operational testing and were emerging at the time of our October 2009 report, and considering modifications to those procedures until operational testing had been completed. Therefore, this recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should evaluate whether TSA's current passenger screening procedures should be revised to require the use of appropriate screening procedures until it is determined that existing emerging technologies meet their functional requirements in an operational environment.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  7. Status: Closed - Implemented

    Comments: In October 2009, we reported that the Transportation Security Administration (TSA) had relied on technologies in day-to-day airport operations that had not been demonstrated to meet their functional requirements in an operational environment. For example, TSA substituted existing screening procedures with screening by the Whole Body Imager (now known as Advanced Imaging Technology) even though its performance had not yet been validated by testing in an operational environment. We recommended that TSA should, in the future, prior to testing or using all checkpoint screening technologies at airports, determine whether TSA's passenger screening procedures should be revised to require the use of appropriate screening procedures until the performance of the technologies has been validated through successful testing and evaluation. TSA agreed with this recommendation. In response, TSA revised the Test and Evaluation Master Plan for its Passenger Screening Program to include consideration of how new technologies will integrate with existing screening procedures at the checkpoint and evaluation of the procedures during operational testing. TSA also provided documentation that such consideration had occurred as part of operational testing and evaluation of several new passenger screening technologies. Therefore, this recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should, in the future, prior to testing or using all checkpoint screening technologies at airports, determine whether TSA's passenger screening procedures should be revised to require the use of appropriate screening procedures until the performance of the technologies has been validated through successful testing and evaluation.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  8. Status: Closed - Implemented

    Comments: In October 2009, we reported that the Transportation Security Administration (TSA) procured and deployed explosives trace portal equipment even though TSA officials were aware that earlier tests did not demonstrate reliable performance in an airport environment. We recommended that TSA conduct an evaluation and determine whether it was cost effective to continue to use these machines. TSA concurred with this recommendation and later halted further deployment of these machines due to performance, maintenance, and installation problems. As of April 2011, TSA reported and provided documented evidence that it had removed from airports all 101 machines that it had deployed. This recommendation is closed as implemented.

    Recommendation: To help ensure that DHS's Science and Technology Directorate (S&T) and Transportation Security Administration (TSA) take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should valuate the benefits of the Explosives Trace Portals that are being used in airports, and compare the benefits to the costs to operate and maintain this technology to determine whether it is cost-effective to continue to use the machines in airports.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

 

Explore the full database of GAO's Open Recommendations »

Dec 17, 2014

Nov 6, 2014

Oct 14, 2014

Sep 30, 2014

Sep 24, 2014

Sep 18, 2014

Sep 17, 2014

Sep 10, 2014

Looking for more? Browse all our products here