This is the accessible text file for GAO report number GAO-11-461T entitled 'Aviation Security: TSA Is Taking Steps to Validate the Science Underlying Its Passenger Behavior Detection Program, but Efforts May Not Be Comprehensive' which was released on April 6, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Testimony: Before the Subcommittee on Investigations and Oversight, Committee on Science, Space, and Technology, House of Representatives: For Release on Delivery: Expected at 10:00 a.m. EDT: Wednesday, April 6, 2011: Aviation Security: TSA Is Taking Steps to Validate the Science Underlying Its Passenger Behavior Detection Program, but Efforts May Not Be Comprehensive: Statement of Stephen M. Lord, Director: Homeland Security and Justice Issues: GAO-11-461T: GAO Highlights: Highlights of GAO-11-461T, a testimony before the Subcommittee on Investigations and Oversight, Committee on Science, Space, and Technology, House of Representatives. Why GAO Did This Study: The attempted passenger aircraft bombing of Northwest flight 253 on December 25, 2009, provided a vivid reminder that the civil aviation system remains an attractive terrorist target. To enhance aviation security, in October 2003 the Department of Homeland Security’s (DHS) Transportation Security Administration (TSA) began testing of its Screening of Passengers by Observation Techniques (SPOT) program to identify persons who may pose a risk to aviation security. The SPOT program utilizes behavior observation and analysis techniques to identify potentially high-risk passengers. This testimony provides information on (1) the extent to which TSA has validated the scientific basis for SPOT and (2) other operational challenges. This statement is based on a prior report GAO issued in May 2010 on SPOT, including selected updates made in March 2011. For the updates, GAO reviewed documentation on TSA’s progress in implementing the report’s recommendations. What GAO Found: As GAO reported in May 2010, TSA deployed its behavior detection program nationwide before first determining whether there was a scientifically valid basis for the program. According to TSA, the program was deployed before a scientific validation of the program was completed in response to the need to address potential security threats. However, a scientific consensus does not exist on whether behavior detection principles can be reliably used for counterterrorism purposes, according to a 2008 report of the National Research Council of the National Academy of Sciences. DHS is conducting a study on the scientific basis of SPOT. Thus, in May 2010, GAO recommended that DHS convene an independent panel of experts to review the methodology of its study. DHS concurred and stated that it is convening an independent panel to review its current efforts to help validate the scientific basis for the program, which is expected to complete its work by early April 2011. Nonetheless, DHS’s study to assess SPOT is not designed to fully validate whether behavior detection can be used to reliably identify individuals in an airport environment who pose a security risk. For example, factors such as the length of time behavior detection officers (BDO) can observe passengers without becoming fatigued are not part of the plan and could provide additional information on the extent to which SPOT can be effectively implemented. The results of a panel to review DHS’s methodology could help ensure a rigorous, scientific validation of SPOT. As GAO previously reported, TSA experienced SPOT operational challenges, including not systematically collecting and analyzing information obtained by BDOs on passengers who may pose a threat to the aviation system. Better utilizing existing resources would enhance TSA’s ability to quickly verify passenger identity and could help TSA to more reliably “connect the dots” with regard to persons who pose a threat. Thus, GAO recommended that TSA clarify BDO guidance for inputting information into the database used to track suspicious activities, and develop a schedule to expand access to this database across all SPOT airports. TSA agreed and in March 2011 stated that it has revised the SPOT standard operating procedures on how BDOs are to input data into the database used to report suspicious activities. TSA plans to implement these revised procedures in April 2011. TSA also reported that all SPOT airports have access to this database as of March 2011. In addition, GAO reported that individuals allegedly involved in six terrorist plots transited SPOT airports. GAO recommended in May 2010 that TSA study the feasibility of using airport video recordings of the behaviors exhibited by persons transiting airport checkpoints who were later charged with or pleaded guilty to terrorism-related offenses. GAO reported that such recordings could provide insights about behaviors that may be common among terrorists or could demonstrate that terrorists do not generally display any identifying behaviors. TSA agreed that studying airport videos could be a useful tool in understanding terrorist behaviors in the airport environment and in March 2011 reported that it is exploring ways to better utilize such recordings. What GAO Recommends: GAO has made recommendations in prior work to strengthen TSA’s SPOT program. TSA generally concurred with the recommendations and has actions under way to address them. GAO provided the updated information to TSA. TSA had no comment. View [hyperlink, http://www.gao.gov/products/GAO-11-461T] or key components. For more information, contact Stephen M. Lord at (202) 512- 4379 or lords@gao.gov' [End of section] Chairman Broun, Ranking Member Edwards, and Members of the Subcommittee: I appreciate the opportunity to participate in today's hearing to discuss the Transportation Security Administration's (TSA) behavior- based passenger screening program known as the Screening of Passengers by Observation Techniques (SPOT) program. The attempted U.S. passenger aircraft bombing of Northwest flight 253 on December 25, 2009, provided a vivid reminder that civil aviation remains an attractive terrorist target and underscores the need for effective passenger screening. To help enhance aviation security, in October 2003, the Department of Homeland Security's (DHS) TSA began testing its SPOT program to identify persons who may pose a risk to aviation security. The SPOT program utilizes behavior observation and analysis techniques to identify potentially high-risk passengers. TSA designed SPOT to provide behavior detection officers (BDO) with a means of identifying persons who may pose a potential security risk at TSA-regulated airports by focusing on behaviors and appearances that deviate from an established baseline and that may be indicative of stress, fear, or deception. In instances when a passenger's SPOT indicators place him or her above a numerical threshold, he or she will be directed to the second step of SPOT, referral screening. This involves additional questioning and physical search of his or her person and property by BDOs and transportation security officers. This referral screening occurs in the checkpoint area. A referral to a law enforcement officer (LEO) is a potential third step in the SPOT process. After a passenger has been referred by the BDOs to a LEO, the LEO is then expected to independently determine, through additional investigation, such as questioning the passenger and, if appropriate, conducting an identity verification and background check through the Federal Bureau of Investigation's (FBI) National Crime Information Center (NCIC), whether sufficient grounds exist to take further action, such as detaining or arresting the passenger. BDOs have been selectively deployed to 161 of the 462 TSA-regulated airports in the United States. The conference report accompanying the fiscal year 2010 DHS appropriations act provided $211.9 million for the SPOT program.[Footnote 1] The administration has requested $232 million for SPOT for fiscal year 2011, a $20.2 million (9.5 percent) increase over the fiscal year 2010 funding level, to support 3,350 BDOs. If this increase is appropriated, TSA will have invested over $800 million in the program since fiscal year 2007. In addition, DHS has requested about $254 million, a $21.9 million increase, in fiscal year 2012 to support an additional 350 BDOs. My statement today discusses TSA's and DHS's efforts to validate the scientific basis of the SPOT program, as well as steps that TSA is taking to address operational challenges in deploying SPOT to airports. My comments are based primarily on our May 2010 report. [Footnote 2] It also includes selective updates we obtained in March 2011. For our May 2010 report, we reviewed relevant literature on behavior analysis by subject matter experts. This included a 2008 study by the National Research Council of the National Academy of Sciences that included a discussion section on the issue of deception and behavioral surveillance, as well as other issues related to behavioral analysis.[Footnote 3] We interviewed recognized experts in the field, as well as cognizant officials from other U.S. government agencies that utilize behavior analysis in their work, including U.S. Customs and Border Protection (CBP), the U.S. Secret Service, the Federal Air Marshal Service (FAMS), and the FBI. To better understand how SPOT incorporated expertise on behavior analysis for aviation security, we also interviewed current and retired officials of Israel's El Al Airlines,[Footnote 4] whose security processes TSA cites as providing part of the basis of the SPOT program. To identify any challenges that emerged during implementation of the SPOT program, we conducted field site visits to 15 TSA-regulated airports with SPOT, which represent almost 10 percent of the 161 TSA-regulated airports with SPOT to observe operations and meet with key program personnel. To obtain comparative data on how SPOT had been implemented at different airports across the nation, we conducted a survey of all federal security directors responsible for security operations at TSA- regulated airports with SPOT. We obtained a 100 percent response rate. In addition, to determine if individuals who were later charged with or pleaded guilty to terrorism-related offenses had transited SPOT airports and whether TSA could obtain information from these transits to enhance its understanding of terrorist behaviors, we reviewed CBP and Department of Justice information to (1) identify individuals who were charged with or pleaded guilty to terrorism-related offenses and (2) determine if these individuals had, prior to being charged, transited airports where SPOT had been deployed. For the updates, we reviewed documentation from TSA on the steps it has taken to implement the recommendations from our May 2010 report. More detailed information about our scope and methodology is included in our May 2010 report. We conducted this work in accordance with generally accepted government auditing standards. TSA Did Not Validate the Science Underlying the SPOT Program before Deploying SPOT: As discussed in our May 2010 report, TSA deployed SPOT nationwide before first determining whether there was a scientifically valid basis for using behavior and appearance indicators as a means for reliably identifying passengers who may pose a risk to the U.S. aviation system. A validation study by DHS's Science and Technology Directorate is under way now, but questions exist regarding whether the study's methodology is sufficiently comprehensive to validate the SPOT program. Specifically, DHS's plan to assess SPOT is not designed to fully validate whether behavior detection can be used to reliably identify individuals in an airport environment who pose a security risk. The results of an independent assessment are needed to determine whether current validation efforts are sufficiently comprehensive to validate the program, and to support future requests for increased funding. According to TSA, SPOT was deployed before a scientific validation of the program was completed, but TSA stated that this deployment was made in response to the need to address potential threats to the aviation system, such as suicide bombers. TSA also stated that the program was based upon scientific research available at the time regarding human behaviors. Moreover, TSA stated that no other large- scale U.S. or international screening program incorporating behavior- and appearance-based indicators has ever been rigorously scientifically validated. However, a 2008 report issued by the National Research Council of the National Academy of Sciences stated that the scientific evidence for behavioral monitoring is preliminary in nature.[Footnote 5] The report also noted that an information-based program, such as a behavior detection program, should first determine if a scientific foundation exists and use scientifically valid criteria to evaluate its effectiveness before deployment. The report added that such programs should have a sound experimental basis and that the documentation on the program's effectiveness should be reviewed by an independent entity capable of evaluating the supporting scientific evidence. [Footnote 6] As we reported in May 2010, an independent panel of experts could help DHS develop a comprehensive methodology to determine if the SPOT program is based on valid scientific principles that can be effectively applied in an airport environment for counterterrorism purposes. Thus, we recommended that the Secretary of Homeland Security convene an independent panel of experts to review the methodology of the validation study on the SPOT program being conducted to determine whether the study's methodology is sufficiently comprehensive to validate the SPOT program. We also recommended that this assessment include appropriate input from other federal agencies with expertise in behavior detection and relevant subject matter experts.[Footnote 7] DHS concurred and stated that its current validation study includes an independent review of the study that will include input from a broad range of federal and operational agencies and relevant experts, including those from academia. According to DHS's Science and Technology Directorate, this independent review is expected to be completed in early April 2011. As discussed in our May 2010 report, DHS has contracted with the American Institutes for Research to conduct its validation study. DHS stated that the ongoing independent review will include, among other things, recommendations on additional studies that should be undertaken to more fully validate the science underlying the SPOT screening process. As we noted in our report, research on other issues, such as determining the number of individuals needed to observe a given number of passengers moving at a given rate per day in an airport environment or the duration that such observation can be conducted by BDOs before observation fatigue affects effectiveness, could provide additional information on the extent to which SPOT can be effectively implemented in airports. Additional research could also help determine the need for periodic refresher training for the BDOs since research has not yet determined whether behavior detection is easily forgotten or can be potentially degraded with time or lack of use. Because such questions exist, the results of an independent panel of experts to assess the methodology of the study could provide DHS with additional assurance regarding whether the study's methodology is sufficiently comprehensive to validate the SPOT program. Moreover, DHS stated that its current effort to validate the science underlying SPOT includes 3 years of operational SPOT referral data and preliminary results indicate that it is supportive of SPOT. However, in May 2010, we reported weaknesses in TSA's process for maintaining operational data from the SPOT program database. Because of these data- related issues, we reported that meaningful analyses could not be conducted to determine if there is an association between certain behaviors and the likelihood that a person displaying certain behaviors would be referred to a law enforcement officer or whether any behavior or combination of behaviors could be used to distinguish deceptive from nondeceptive individuals.[Footnote 8] As we reported in March 2011, Congress may wish to consider limiting program funding pending receipt of an independent assessment of TSA's SPOT program.[Footnote 9] We identified potential budget savings of about $20 million per year if funding were frozen at current levels until validation efforts are complete. Specifically, in the near term, we reported that Congress could consider freezing appropriation levels for the SPOT program at the 2010 level until the validation effort is completed. Assuming that TSA is planning to expand the program at a similar rate each year, this action could result in possible savings of about $20 million per year, or $100 million over 5 years, since TSA is seeking about a $20 million increase for SPOT in fiscal year 2011. We also reported that upon completion of the validation effort, Congress may also wish to consider the study's results--including those on the program's effectiveness in using behavior-based screening techniques to detect terrorists in the aviation environment--in making future funding decisions regarding the program. TSA Is Taking Steps to Address Operational Challenges in Implementing the SPOT Program: In May 2010, we reported that TSA is not fully utilizing the resources it has available to systematically collect the information obtained by BDOs on passengers whose behaviors and appearances resulted in either a referral to a BDO or to a LEO, and who thus may pose a risk to the aviation system. As we previously reported, TSA does not provide official guidance on how or when BDOs or other TSA personnel should enter data into the Transportation Information Sharing System or which data should be entered.[Footnote 10] Official guidance on what data should be entered into the system on passengers could better position TSA personnel to be able to consistently collect information to facilitate synthesis and analysis in "connecting the dots" with regard to persons who may pose a threat to the aviation system. Moreover, as of May 2010, TSA had not developed a schedule or milestones by which database access would be deployed to SPOT airports, or a date by which access at all SPOT airports would be completed. Setting milestones for expanding Transportation Information Sharing System access to all SPOT airports, and setting a date by which the expansion will be completed, could better position TSA to identify threats to the aviation system that may otherwise go undetected and help TSA track its progress in expanding Transportation Information Sharing System access as management intended. Thus, we previously recommended that TSA provide guidance in the SPOT standard operating procedures or other directives to BDOs, and to other TSA personnel as appropriate, on how and when to input data into the Transportation Information Sharing System database.[Footnote 11] In March 2011, TSA stated that it has taken steps to implement our recommendation by revising SPOT standard operating procedures to provide guidance directing the input of BDO data into the Transportation Information Sharing System. TSA plans to implement these revised procedures in April 2011. In addition, all SPOT airports have access to the Transportation Information Sharing System as of March 2011 according to TSA. In addition, as we previously reported, studying airport video recordings of the behaviors exhibited by persons transiting airport checkpoints who were later charged with or pleaded guilty to terrorism- related offenses could provide important insights about behaviors that may be common among terrorists or could demonstrate that terrorists do not generally display any identifying behaviors. In addition, such images could help determine if BDOs are looking for the right behaviors or seeing the behaviors they have been trained to observe. Using CBP and Department of Justice information, we examined the travel of key individuals allegedly involved in six terrorist plots that have been uncovered by law enforcement agencies.[Footnote 12] We determined that at least 16 of the individuals allegedly involved in these plots moved through 8 different airports where the SPOT program had been implemented.[Footnote 13] Six of the 8 airports were among the 10 highest-risk airports, as rated by TSA in its Current Airport Threat Assessment. In total, these individuals moved through SPOT airports on at least 23 different occasions. For example, according to Department of Justice documents, in December 2007 an individual who later pleaded guilty to providing material support to Somali terrorists boarded a plane at the Minneapolis-Saint Paul International Airport en route to Somalia. Similarly, in August 2008, an individual who later pleaded guilty to providing material support to al Qaeda boarded a plane at Newark Liberty International Airport en route to Pakistan to receive terrorist training to support his efforts to attack the New York subway system. Our survey of federal security directors at 161 SPOT airports indicated that most checkpoints at SPOT airports have surveillance cameras installed. Thus, we reported that TSA may be able to utilize the information collected from the video infrastructure at the nation's airports to study the behavior of persons who were later charged with or pleaded guilty to terrorism-related offenses to help improve and refine the existing SPOT program. As a result, in our May 2010 report, we recommended that if the current validation effort determines that the SPOT program has a scientifically validated basis for using behavior detection for counterterrorism purposes in the airport environment, then TSA should study the feasibility of using airport checkpoint surveillance video recordings to enhance its understanding of terrorist behaviors.[Footnote 14] DHS agreed with our recommendation and noted that TSA agrees this could be a useful tool and is working with DHS's Science and Technology Directorate to utilize video case studies of terrorists, if possible. TSA officials agreed that examining video recordings of individuals who were later charged with or pleaded guilty to terrorism-related offenses, as they used the aviation system to travel to overseas locations allegedly to receive terrorist training or to execute attacks, could help inform the SPOT program's identification of behavioral indicators. In March 2011, TSA stated that it is exploring ways to better utilize video recordings to identify these behavioral indicators. Chairman Broun, Ranking Member Edwards, and Members of the Subcommittee, this concludes my statement. I look forward to answering any questions that you may have at this time. Contacts and Acknowledgments: For questions about this statement, please contact Stephen M. Lord at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this testimony are David M. Bruno, Assistant Director; Ryan Consaul; Katherine Davis; Emily Gunn; and Tracey King. [End of section] Footnotes: [1] H.R. Rep. 111-298, at 77 (2009) (Conf. Rep.). [2] See GAO, Aviation Security: Efforts to Validate TSA's Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges, [hyperlink, http://www.gao.gov/products/GAO-10-763] (Washington, D.C.: May 20, 2010). [3] National Research Council, Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Assessment (Washington, D.C.: National Academies Press, 2008). The report's preparation was overseen by the National Academy of Sciences Committee on Technical and Privacy Dimensions of Information for Terrorism Prevention and Other National Goals. Although the report addresses broader issues related to privacy and data mining, a senior National Research Council official stated that the committee included behavior detection as a focus because any behavior detection program could have privacy implications. [4] Although SPOT is based in some respects on El Al's aviation security program, El Al's processes differ in substantive ways from those used by the SPOT program. In particular, El Al does not use a list of specific behaviors with numerical values for each, or a numerical threshold to determine whether to question a passenger; rather, El Al security officers utilize behavioral indicators as a basis for interviewing all passengers boarding El Al passenger aircraft, and access relevant intelligence databases, when deemed appropriate. According to these officials, El Al also permits what is termed "profiling," in which passengers may be singled out for further questioning based on their nationality, ethnicity, religion, appearance, or other descriptive characteristics, but these are not the only bases on which a passenger may be questioned. The scale of El Al operations is considerably smaller than that of major airlines operating within the United States. In Israel, El Al operates out of one hub airport; in contrast, there are 462 TSA-regulated airports in the United States. [5] Specifically, the report states that the scientific support for linkages between behavioral and physiological markers and mental state is strongest for elementary states, such as simple emotions; weak for more complex states, such as deception; and nonexistent for highly complex states, such as when individuals hold terrorist intent and beliefs. [6] A study performed by the JASON Program Office raised similar concerns. The JASON Program Office is an independent scientific advisory group that provides consulting services to the U.S. government on matters of defense science and technology. [7] See [hyperlink, http://www.gao.gov/products/GAO-10-763]. [8] See [hyperlink, http://www.gao.gov/products/GAO-10-763]. [9] See GAO, Opportunities to Reduce Potential Duplication in Government Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink, http://www.gao.gov/products/GAO-11-318SP] (Washington, D.C.: Mar. 1, 2011). [10] The Transportation Information Sharing System is a database owned by TSA's FAMS component; the data entered into it may be shared with other federal, state, or local law enforcement and law enforcement support entities. Federal air marshals file reports related to the observation of suspicious activities and input this information, as well as incident reports submitted by airline employees and other individuals within the aviation domain, into the Transportation Information Sharing System. [11] See [hyperlink, http://www.gao.gov/products/GAO-10-763]. [12] The analysis included only flights leaving the United States. Department of Justice data show that more than 400 individuals have been convicted in the United States for terrorism-related offenses since September 11, 2001. We did not examine the travel itineraries of all these individuals. [13] The events included the Mumbai, India, attack of 2008; a plot to attack the Quantico, Virginia, Marine base in 2008; an effort by five Americans to receive training and fight in Pakistan in December 2009; a plot to attack infrastructure in New York City in 2009; an effort to provide men and support for terrorists in Somalia in 2008; and an attack on a U.S. base in Afghanistan by an American who received training in Pakistan. We were unable to confirm whether BDOs were stationed at the checkpoints used by these individuals at the time they traveled. [14] See [hyperlink, http://www.gao.gov/products/GAO-10-763]. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: