Advanced Imaging Technology: TSA Needs Additional Information before Procuring Next-Generation Systems
Highlights
What GAO Found
The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) does not collect or analyze available information that could be used to enhance the effectiveness of the advanced imaging technology (AIT) with automated target recognition (ATR) system. Specifically, TSA does not collect or analyze available data on drills using improvised explosive devices (IED) at the checkpoint that could provide insight into how well screening officers (SO) resolve anomalies, including objects that could pose a threat to an aircraft, identified by AIT systems, because it does not enforce compliance with its operational directive. TSA's operational directive requires personnel at airports to conduct drills to assess SO compliance with TSA's screening standard operating procedures and to train SOs to better resolve anomalies identified by AIT-ATR systems. GAO found that TSA personnel at about half of airports with AIT systems did not report any IED checkpoint drill results on those systems from March 2011 through February 2013. According to TSA, it does not ensure compliance with the directive at every airport because it is unclear which office should oversee enforcing the directive. Without data on IED checkpoint drills, TSA lacks insight into how well SOs resolve anomalies detected by AIT systems, information that could be used to help strengthen existing screening processes. Potential weaknesses in the screening process could be caused by TSA not clarifying which office is responsible for overseeing TSA's operational directive, directing that office to ensure enforcement of the directive in conducting these drills, and analyzing the data. Further, when determining AIT-ATR system effectiveness, TSA uses laboratory test results that do not reflect the combined performance of the technology, the personnel who operate it, and the process that governs AIT-related security operations. TSA officials agreed that it is important to analyze performance by including an evaluation of the technology, operators, and processes and stated that TSA is planning to assess the performance of all layers of security. By not measuring system effectiveness based on the performance of the technology and SOs who operate the technology or taking into account current processes and deployment strategies, DHS and TSA are not ensuring that future procurements meet mission needs.
TSA completed the installation of ATR software upgrades intended to address privacy concerns for all deployed AIT systems; however, it has not met proposed milestones for enhancing capabilities as documented in its AIT roadmap—a document that contains milestones for achieving enhanced capabilities to meet the agency's mission needs. For example, TSA began operational test and evaluation for Tier II upgrades 17 months after the expected start date. Moreover, TSA did not use available scientific research or information from experts from the national laboratories or vendors on the technological challenges that it faces in developing requirements and milestones, because, according to TSA, it relied on time frames proposed by vendors. Thus, TSA cannot ensure that its roadmap reflects the true capabilities of the next generation of AIT systems by using scientific evidence and information from DHS's Science and Technology Directorate, the national laboratories, and vendors to develop a realistic schedule with achievable milestones that outlines the technological advancements, estimated time, and resources needed to achieve enhanced capabilities as outlined in TSA's roadmap.
Why GAO Did This Study
TSA accelerated the deployment of AIT systems, or full-body scanners, in response to the December 25, 2009, attempted terrorist attack on Northwest Airlines Flight 253. Pursuant to the Federal Aviation Administration Modernization and Reform Act of 2012, TSA was mandated to ensure that AIT systems were equipped with ATR software, which displays generic outlines of passengers rather than actual images, by June 1, 2013. All deployed AIT systems were equipped with ATR software by the deadline. GAO was asked to evaluate TSA's AIT-ATR systems' effectiveness. This report addresses the extent to which (1) TSA collects and analyzes available information that could be used to enhance the effectiveness of the AIT-ATR system and (2) TSA has made progress toward enhancing AIT capabilities to detect concealed explosives and other threat items, and any challenges that remain. GAO analyzed testing results conducted by the Transportation Security Laboratory and TSA personnel at airports and interviewed DHS and TSA officials. This is a public version of a classified report that GAO issued in December 2013. Information DHS and TSA deemed classified or sensitive has been omitted, including information and recommendations related to improving AIT capabilities.
Recommendations
GAO recommends that TSA, among other things, clarify which office should oversee its operational directive, better measure system effectiveness, and develop a realistic schedule before procuring future generations. TSA concurred with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Transportation Security Administration | To help ensure that TSA improves SO performance on AIT-ATR systems and uses resources effectively, the Administrator of the Transportation Security Administration should clarify which office is responsible for overseeing TSA's IED screening checkpoint drills operational directive, direct the office to ensure enforcement of the directive in conducting these drills, and analyze the data to identify any potential weaknesses in the screening process. |
In response to this recommendation, the Transportation Security Administration (TSA) conducted a review of programs that contribute to assessing screening performance and determined that the Improvised Explosive Device Checkpoint Drills training program did not result in meaningful data that could be analyzed to identify potential weaknesses. Further, TSA determined that the program is resource intensive and no longer fulfills an existing need within TSA. In May 2014, TSA provided GAO with a memo cancelling the checkpoint drills operational directive, which was approved by the TSA Administrator on May 20, 2014. Based on these actions, we are closing this recommendation as not implemented.
|
| Transportation Security Administration | To help ensure that TSA improves SO performance on AIT-ATR systems and uses resources effectively, the Administrator of the Transportation Security Administration should establish protocols that facilitate the capturing of operational data on secondary screening of passengers at the checkpoint to determine the extent to which AIT-ATR system false alarm rates affect operational costs once AIT-ATR systems are networked together. |
In March 2014, we reported that TSA was in the process of networking all AIT systems equipped with automated target recognition (AIT-ATR) that were deployed to airports nationwide so that information could be collected at the headquarters level, and when this process is complete, TSA would be able to centrally collect operational data that could provide information on secondary screening outcomes, which provide insight into the operational false alarm rate. We recommended that the TSA Administrator establish protocols that facilitate the capturing of operational data on secondary screening of passengers at the checkpoint to determine the extent to which AIT-ATR system false alarm rates affect operational costs once AIT-ATR systems are networked together. In July 2015, TSA officials stated that the agency had not completed the networking of AIT-ATR systems, and in response to this recommendation, TSA evaluated false alarm rates against existing staffing levels as part of the test and evaluation for the second generation of AIT systems, referred to as AIT-2 systems, to determine the impact of the new AIT systems on staffing. However, evaluating false alarm rates under test conditions, which occur for a limited period of time, did not address the intent of our recommendation to collect and analyze operational data on an ongoing basis under normal operating conditions. After considering other options to address this recommendation, such as leveraging existing processes to gather AIT false alarm information using a statistically representative sample of airports, in December 2016 TSA reported that it is not feasible to implement this recommendation in a live airport environment, given expected costs and methodological challenges. Based on these actions, we are closing this recommendation as not implemented.
|
| Transportation Security Administration | To help ensure that TSA invests in screening technology that meets mission needs, the Administrator of the Transportation Security Administration should measure system effectiveness based on the performance of the AIT-2 technology and screening officers who operate the technology, while taking into account current processes and deployment strategies before procuring AIT-2 systems. |
TSA provided documentation showing that, while conducting operational testing of the AIT-2 system, the agency considered screening officer performance and measured AIT-2 system effectiveness based on both the performance of the AIT-2 technology and the screening officers who operate it. Based on these actions, we are closing this recommendation as implemented.
|
| Transportation Security Administration | To help ensure that TSA invests in screening technology that meets mission needs, the Administrator of the Transportation Security Administration should use scientific evidence and information from DHS's Science and Technology Directorate, and the national laboratories, as well as information and data provided by vendors to develop a realistic schedule with achievable milestones that outlines the technological advancements, estimated time, and resources needed to achieve TSA's Tier IV end state before procuring AIT-2 systems. |
In response to this recommendation, TSA sought input from the DHS Science & Technology Directorate, U.S. government national laboratories, industry, and academia to develop a technology maturation roadmap for Advanced Imaging Technology (AIT). This technology roadmap, which was prepared by The Johns Hopkins University Applied Physics Laboratory on behalf of TSA, is designated for official use only. TSA subsequently produced a significantly scaled down public version of the technology roadmap, which it published on the Federal Business Opportunities website in February 2015. Because TSA's public roadmap does not contain all elements outlined in the original recommendation (technological advancements, estimated time, and resources needed to achieve TSA's Tier IV end state), we are closing this recommendation as implemented based on the technology roadmap produced by Johns Hopkins University.
|