Federal Air Marshal Service: Additional Actions Needed to Ensure Air Marshals' Mission Readiness
Highlights
What GAO Found
The Transportation Security Administration's (TSA) Office of Training and Development (OTD) assesses air marshals' training needs using several information sources, but opportunities exist to obtain more feedback from air marshals on whether the training courses they must take met their needs. OTD primarily assesses air marshals' training needs by holding curriculum development and review conferences composed of OTD officials, training instructors, and other subject matter experts. In assessing courses, conference participants use, among other things, the results of surveys that some air marshals complete on the effectiveness of their training. However, while OTD administers these surveys for air marshal candidates and newly graduated air marshals, it does not use them to obtain feedback from incumbent air marshals on the effectiveness of their annual recurrent training courses. Systematically gathering feedback from incumbent air marshals would better position OTD to fully assess whether the training program is meeting air marshals' needs. Additionally, among the training surveys that OTD does currently administer to air marshals, the response rates have been low. For example, among newly hired air marshals and their supervisors from 2009 through 2011—the last three full years in which the Federal Air Marshal Service (FAMS) hired air marshals—the survey response rates ranged from 16 to 38 percent. Until OTD takes steps to achieve sufficient response rates, OTD cannot be reasonably assured that the feedback it receives represents the full spectrum of views held by air marshals.
FAMS relies on its annual recurrent training program to ensure incumbent air marshals' mission readiness, but additional actions could strengthen FAMS's ability to do so. First, FAMS does not have complete and timely data on the extent to which air marshals have completed their recurrent training. For example, nearly one-quarter of all training records for calendar year 2014 had not been entered into FAMS's training database within the required time period. Policies that specify who is responsible at the headquarters level for overseeing these activities could help FAMS ensure its data on air marshals' recurrent training are accurate and up to date. Second, FAMS requires air marshals to demonstrate proficiency in marksmanship by achieving a minimum score of 255 out of 300 on the practical pistol course every quarter. However, for the remaining recurrent training courses FAMS does not assess air marshals' knowledge or performance in these courses against a similarly identified level of proficiency, such as by requiring examinations or by using checklists or other objective tools. More objective and standardized methods of determining incumbent air marshals' mission readiness, as called for by the Department of Homeland Security's (DHS) Learning Evaluation Guide, could help FAMS better and more consistently assess air marshals' skills and target areas for improvement. Additionally, in 2015 FAMS developed a health, fitness, and wellness program to improve air marshals' overall health and wellness, but it is too early to gauge the program's effectiveness. This is a public version of a sensitive report that GAO issued in June 2016. Information that TSA deems “Sensitive Security Information” has been removed.
Why GAO Did This Study
FAMS, within TSA, is the federal entity responsible for promoting confidence in the nation's aviation system through deploying air marshals to protect U.S. air carriers, airports, passengers, and crews.
GAO was asked to assess FAMS's training program for federal air marshals. This report examines (1) how TSA assesses the training needs of air marshal candidates and incumbent air marshals, and any opportunities that exist to improve this assessment, and (2) the extent to which FAMS ensures that incumbent air marshals are mission ready.
GAO analyzed FAMS training data for calendar year 2014, the last year of available data, reviewed TSA, OTD and DHS guidance and policies on FAMS's air marshal training program, interviewed TSA and FAMS headquarters officials, and visited the TSA Training Center and 7 of FAMS 22 field offices selected based on size and geographic dispersion.
Recommendations
GAO recommends that OTD implement a mechanism for regularly collecting incumbent air marshals' feedback on their recurrent training, and take steps to improve the response rates of training surveys it conducts. GAO also recommends that FAMS specify in policy who at the headquarters level has oversight responsibility for ensuring that recurrent training records are entered in a timely manner, and develop and implement standardized methods to determine whether incumbent air marshals continue to be mission ready in key skills. DHS concurred with all of the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Transportation Security Administration | To ensure effective evaluation of air marshal training, the TSA Administrator should direct OTD to implement a mechanism for regularly collecting and incorporating incumbent air marshals' feedback on the training they receive from field office programs. |
In September 2016, we reported that the Transportation Security Administration (TSA) conducted surveys to obtain feedback from air marshal candidates and newly graduated air marshals on the effectiveness of their training, but did not systematically collect training feedback from incumbent air marshals on field-based training. We recommended that TSA's Office of Training and Development implement a mechanism for regularly collecting and incorporating incumbent air marshals' feedback on the training they receive from field office programs. In July 2017, TSA reported that its Office of Training and Development developed a survey (Field Office Recurrent Training Survey) that is intended to measure the effectiveness of FAMS training curriculum, field office training personnel, and training facilities. Additionally, in November 2016, TSA resumed the field office training assessments to evaluate field office training programs and their instructors. As part of these assessments, TSA sends surveys to supervisors and air marshals in the field with questions on the effectiveness of the field office's training program. TSA documentation indicates that completed surveys are compiled and analyzed by the TSA Training Center. TSA officials report using this analysis to identify whether corrective action is required. As a result of these actions, this recommendation is closed as implemented.
|
| Transportation Security Administration | To ensure effective evaluation of air marshal training, the TSA Administrator should direct OTD to take additional steps to improve the response rates of the training surveys it conducts. |
In September 2016, we reported that the response rates for federal air marshals training surveys were consistently low. Specifically, we found that the training survey response rate for graduates of new-hire training were about 19 to 38 percent in the period 2009 to 2011 and, according to OTD officials, the combined response rate for surveys that training assessment teams conducted from June 2012 through March 2013 was about 16 percent. The Transportation Security Administration (TSA) uses the survey results to identify training gaps and determine how to appropriately address them. We recommended that TSA take additional steps to improve the response rates of the surveys it conducts. In response to our recommendation, TSA's Office of Training and Development added the training surveys to the On-line Learning Center, which provides a tracking mechanism for program managers to ensure that personnel complete the surveys. TSA documentation shows that response rates have since improved. Specifically, TSA documentation shows that the overall response rate for training surveys administered to graduates of new-hire training in 2017 and 2018 was about 67 percent and the combined response rate for surveys that training assessment teams conducted in 2017 was 79 percent. As a result, this recommendation is closed as implemented.
|
| Transportation Security Administration | To provide reasonable assurance that air marshals are complying with recurrent training requirements and have the capability to carry out FAMS's mission, the TSA Administrator should direct FAMS to specify in policy who at the headquarters level has oversight responsibility for ensuring that field office Supervisory Air Marshals-in-Charge or their designees meet their responsibilities for ensuring that training completion records are entered in a timely manner. |
In September 2016, we reported that the U.S. Federal Air Marshal Service (FAMS) did not have complete and timely data on the extent to which air marshals have completed their recurrent training. Specifically, nearly one-quarter of all training records for calendar year 2014 had not been entered into FAMS training database--Federal Air Marshal Information System (FAMIS)--within 5 days after an air marshal had completed the required training, although FAMS policy requires this information to be recorded within 5 days. Additionally, FAMS officials responsible for reconciling completion of recurrent training agency-wide reported that there were a significant number of air marshals for whom Field Office Supervisory Air Marshals-in-Charge or their designees had not entered training records. FAMS officials acknowledged that the agency had not specified in policy who had oversight responsibility at the headquarters level for ensuring that each field office had entered recurrent training data in a timely manner. As a result, we recommended that FAMS specify in policy who at the headquarters level has this responsibility. According to TSA, FAMS revised the recurrent training policy in November 2017 in response to GAO's recommendation. The revised policy specifies that Regional Directors (who are based in headquarters) or their designees are responsible for ensuring that air marshals under their supervision comply with all mandatory training requirements and field office Training Supervisory Air Marshals-in-Charge or their designees are responsible for recording completion of training in FAMIS. In April 2018, TSA reported that the revised policy has helped FAMS ensure compliance with the required reporting and tracking of all FAMS training--resulting in complete and up to date data on air marshals' recurrent training. As a result, this recommendation is closed as implemented.
|
| Transportation Security Administration | To provide reasonable assurance that air marshals are complying with recurrent training requirements and have the capability to carry out FAMS's mission, the TSA Administrator should direct FAMS to specify in policy who at the headquarters level is responsible for ensuring that headquarters personnel enter approved air marshals' training exemptions into the Federal Air Marshal Information System, and define the timeframe for doing so. |
In September 2016, we reported that the U.S. Federal Air Marshal Service (FAMS) did not have complete and timely data on the extent to which air marshals had received exemptions from completing required training. Specifically, nearly one-third of all training exemptions granted to air marshals in calendar year 2014 had not been entered into FAMS's training database--Federal Air Marshal Information System (FAMIS)--until July 2015, which was seven months after the calendar year ended. At the time of our review, FAMS had not specified in policy who had oversight responsibility at the headquarters level for ensuring that headquarters personnel have entered air marshals exemptions into FAMIS within a defined timeframe. As a result, we recommended that FAMS specify in policy who at the headquarters level has this responsibility and define the timeframe for doing so. In response to our recommendation, in November 2017, FAMS revised its recurrent training requirements policy to require that FAMS's Field Operations Division, Tactical Support Section verify that FAMIS entries are made for all training exemptions within 5 business days of the approval of the exemptions. In April 2018, TSA reported that the revised policy has helped FAMS ensure compliance with the required reporting and tracking of FAMS training, including training exemptions received by air marshals. As a result, this recommendation is closed as implemented.
|
| Transportation Security Administration | To provide reasonable assurance that air marshals are complying with recurrent training requirements and have the capability to carry out FAMS's mission, the TSA Administrator should direct FAMS to develop and implement standardized methods, such as examinations and checklists, for determining whether incumbent air marshals continue to be mission ready in key skills. |
In September 2016, we reported that the Federal Air Marshal Service (FAMS) requires incumbent air marshals to demonstrate their proficiency in marksmanship by achieving a minimum score on the practice pistol course every quarter; however, for the remaining recurrent training courses, FAMS does not assess air marshals' knowledge or performance against a similarly identified level of proficiency. As a result, we recommended that the Transportation Security Administration (TSA) develop and implement standardized methods, such as examinations and checklists, for determining whether air marshals continue to be mission ready in key skills. TSA has since taken some steps to further consider this recommendation but has not implemented it. TSA officials report that in 2016 they convened an Integrated Project Team to develop a mission ready assessment measure for incumbent air marshals. In June of 2017, a group of executive and management level leaders considered the Integrated Project Team's conclusions and recommended that FAMS not adopt any formalized testing for its workforce; determining that processes in place ensure that the workforce remains mission ready. In December 2019, FAMS reaffirmed this position, stating that the training measures and processes already in place ensure that air marshals are mission ready. They noted that air marshals are required to complete and pass numerous trainings. FAMS reaffirmed this position again in July 2020, stating that FAMS does not intend to take further steps to implement this recommendation. We maintain, however, that requiring the completion of trainings is distinct from using a standardized method, such as an examination or checklist, to determine whether incumbent air marshals continue to be mission ready in key skills. As a result, and given the passage of time since GAO made this recommendation, GAO is closing this recommendation as not implemented.
|