Transportation Security: Action Needed to Strengthen TSA's Security Threat Assessment Process
Highlights
What GAO Found
The Transportation Security Administration's (TSA) Adjudication Center performance data show mixed results, and the center's performance measurement practices have limitations. The Adjudication Center relies on contractors to adjudicate security threat assessments and uses three primary measures to evaluate their performance--timeliness for completing adjudication, adjudication accuracy, and caseload status. GAO found that the Adjudication Center contractor met its timeliness and accuracy measures, but faced challenges in meeting its caseload measure. The Adjudication Center's timeliness and accuracy measures did not capture key data. According to TSA officials, the Adjudication Center's accuracy rate is based on a review of all cases where adjudicators had disqualified an applicant. However, this calculation generally does not include the accuracy rate for those applicants adjudicators had approved--which account for roughly 90 percent of the Adjudication Center's caseload. In this way, the accuracy rate provides a limited assessment of adjudicator performance. By developing an accuracy rate that includes data on both incorrectly disqualified and incorrectly approved applicants, TSA can better identify and addresses performance issues among its workforce.
Two TSA offices that share responsibility for implementing security threat assessments--the Program Management Division in the Office of Intelligence and Analysis and the Adjudication Center in the Office of Law Enforcement/Federal Air Marshal Service--can improve coordination on workforce planning. While the offices share information on workload completion, they do not have a process in place to ensure that information in the Adjudication Center's staffing plan--which the Adjudication Center periodically updates to reflect caseload projections and associated staffing needs--reflects the mutual understanding of both offices. For example, program managers in the Office of Intelligence and Analysis reported to GAO that they were unfamiliar with the staffing plan and they disagreed with workload projections in the plan. Establishing a mechanism for the offices to share and reconcile information in the plan can help better support the Adjudication Center's workforce planning.
TSA has been delayed in addressing risks posed by using contractors to adjudicate security threat assessments. In October 2011 TSA's Balanced Workforce Strategy Working Group completed its assessment for the Adjudication Center and determined that an excessive risk exists by allowing contractors to make security threat assessment approvals without sufficient federal oversight. The Working Group recommended that TSA convert to an all government workforce. According to a May 2012 implementation plan, TSA planned to convert this workforce by the end of calendar year 2013. However, delays have rendered the timelines and cost information in its plan outdated and TSA has not updated the plan or determined a revised implementation schedule. Completing this review and updating the plan would help TSA and Department of Homeland Security (DHS) decision makers by providing a roadmap for moving forward. Finally, providing this plan to DHS for review will be important to help ensure TSA can begin its conversion and mitigate identified risks of using contract adjudicators to conduct security threat assessments.
Why GAO Did This Study
TSA implements programs that, for example, ensure individuals with unescorted access to secure areas of the nation's critical transportation infrastructure do not pose a security threat. Key to these programs are security threat assessments that screen individuals for links to terrorism, criminal history, and immigration status. TSA's Adjudication Center serves as the primary operational component in this process. GAO was asked to examine the performance and staffing strategy of the center. This report addresses the extent to which 1) TSA has measured performance for the center and what the data show; 2) TSA offices have coordinated to meet security threat assessment workload; and 3) TSA addressed potential risks posed by using a mix of government employees and contractors to adjudicate security threat assessments. GAO analyzed TSA data describing the center's performance since October 2010; reviewed documentation, including staffing plans; and interviewed TSA officials about data measurement and staffing practices.
Recommendations
GAO recommends that TSA, among other things: direct the Adjudication Center to calculate an accuracy rate that includes adjudicator performance for cases where applicants were both approved and disqualified; share adjudicator staffing plans among key program offices; and update its Adjudication Center workforce conversion plan and provide it to DHS for review and approval. DHS concurred with our recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Homeland Security | To ensure that the Adjudication Center accuracy rate effectively captures the center's accuracy in completing security threat assessments, the Secretary of Homeland Security should direct the TSA Administrator to take the following action. The Adjudication Center should develop an accuracy rate measure that includes accuracy data for cases where adjudicators both approved and disqualified applicants, document this methodology, and implement the process. |
In July 2013, we reported that the Transportation Security Administration (TSA) faced challenges in ensuring the effectiveness of its Adjudication Center's security threat assessment process. We found that while the Adjudication Center uses three primary measures to evaluate the performance of the Adjudication Center's contract workforce, it had not documented its methods and its measures for doing so were limited because they did not capture key data. In particular, we found that the Adjudication Center's accuracy rate generally did not include cases in which contract adjudicators incorrectly approved an applicant. For example, according to TSA officials, the Adjudications Center's accuracy rate is based on a review of all cases where adjudicators had disqualified an applicant. However, this calculation generally did not include the accuracy rate for those applicants adjudicators had approved---which account for roughly 90 percent of the Adjudication Center's caseload. In this way, the accuracy rate provided a limited assessment of adjudicator performance. We recommended that DHS direct TSA's Adjudication Center to develop an accuracy rate measure that includes accuracy data for cases where adjudicators both approved and disqualified applicants, document this methodology, and implement this process. DHS concurred with our recommendation. TSA reported that it had modified the Adjudication Center's quality control process for measuring accuracy to include both approved and disqualified cases. In June 2014, TSA officials reported documenting this methodology in Adjudication Center Standard Operating Procedures and Excel spreadsheets. For example, according to its January 2014 Adjudication Center Standard Operating Procedures, on average, mid-level adjudicators will conduct a review of 50 approved cases per day for quality assurance. TSA officials provided documentation of the calculations it used to determine its accuracy rate which showed that the accuracy rate included both approved and disqualified cases. TSA's actions are consistent with the intent of our recommendation.
|
| Department of Homeland Security | To ensure continuity of case reporting, the Secretary of Homeland Security should direct the TSA Administrator to document its case reporting performance management processes. |
In July 2013, we reported that the Transportation Security Administration (TSA) faced challenges in ensuring the effectiveness of its Adjudication Center's security threat assessment process. Among other things, we reported that since beginning operations in 2005 Adjudication Center management officials have used a complex, manual process to track the performance data of its contract adjudicator workforce. In particular, because of functional limitations of TSA's Screening Gateway systems, officials reported that the Adjudication Center lacked an automated process for tracking adjudicator performance of the estimated 7,500 to 10,000 security threat assessment cases that adjudicators process each week. As a result, Adjudication Center management had used a cumbersome, manual process to track case production and performance of its contract adjudicator workforce. However, officials reported that only two officials were familiar with the process and it was not documented. To ensure continuity of case reporting, we recommended that DHS direct TSA to document the Adjudication Center's case reporting system. DHS concurred with our recommendation. In response, TSA documented its case reporting system in a manual which lays out elements in the Adjudication Center's case management process and the responsibilities of Adjudication Center officials in this process. TSA's actions are consistent with the intent of our recommendation.
|
| Department of Homeland Security | The Secretary of Homeland Security should direct the TSA Administrator to ensure workforce planning is based on accurate workload projections, establish a mechanism for TSA's Office of Intelligence and Analysis (OIA) Program Management Division and Office of Law Enforcement/Federal Air Marshal Service (OLE/FAMS) Adjudication Center to share and reconcile information included in the Adjudication Center's staffing plan updates, such as timelines for anticipated workload growth. |
In July 2013, we reported that the Transportation Security Administration (TSA) faced challenges in ensuring the effectiveness of its Adjudication Center's security threat assessment process. Among other things, we found that two TSA offices that share responsibility for implementing security threat assessments---the Program Management Division in the Office of Intelligence and Analysis (OIA) and the Adjudication Center in the Office of Law Enforcement/Federal Air Marshall Service (OLE/FAMS)could improve coordination on workforce planning. In particular, we noted that while the offices shared information on workload completion, they did not have a process in place to ensure that information in the Adjudication Center's staffing plan---which the Adjudication Center periodically updated to reflect caseload projections and associated staffing needs---reflected the mutual understanding of both offices. To ensure workforce planning is based on accurate workload projections, we recommended that DHS direct the TSA to establish a mechanism for TSA's OIA and OLE/FAMS to share and reconcile information included in the Adjudication Center's staffing plan updates, such as timelines for anticipated workload growth. In response, TSA reported formalizing a quarterly process for meetings between officials from the two offices to review staffing plans, program workload projections, and other areas. According to TSA documentation, the first quarterly meeting was held in February 2014. TSA's actions are consistent with the intent of our recommendation.
|
| Department of Homeland Security | To advance efforts to address risks identified in the Adjudication Center Balanced Workforce Strategy (BWS) assessment, the Secretary of Homeland Security should direct the TSA Administrator to update and document its Adjudication Center insourcing conversion plan to reflect revised schedule timeframes, cost and hiring level information. |
In July 2013, we reported that the Transportation Security Administration (TSA) faced challenges in ensuring the effectiveness of its Adjudication Center's security threat assessment process. Among other things, we reported that TSA had been delayed in addressing risks posed by using contractors to adjudicate security threat assessments. Specifically, we found that in October 2011 TSA's Balanced Workforce Strategy Working Group completed its assessment for the Adjudication Center and determined that an excessive risk exists by allowing contractors to make security threat assessment approvals without sufficient federal oversight. According to a May 2012 implementation plan, TSA planned to convert this workforce by the end of calendar year 2013. However, delays had rendered the timelines and cost information in its plan outdated and TSA had not updated the plan or determined a revised implementation schedule. To advance efforts to address risks identified in the Adjudication Center Balanced Workforce Strategy assessment, we recommended that DHS direct TSA to update and document its Adjudication Center insourcing conversion plan to reflect revised schedule timeframes, cost and hiring level information. DHS concurred with our recommendation. TSA officials reported updating the insourcing conversion plan. This plan, dated January 29, 2014, recommended approval to convert 46 contractor positions to 46 Federal positions in fiscal year 2014 and provided updated cost information for doing so. TSA's actions are consistent with the intent of our recommendation.
|
| Department of Homeland Security | To advance efforts to address risks identified in the Adjudication Center BWS assessment, the Secretary of Homeland Security should direct the TSA Administrator to review the updated Adjudication Center insourcing conversion plan, and provide it to TSA and DHS leadership for review and implementation approval. |
In July 2013, we reported that the Transportation Security Administration (TSA) faced challenges in ensuring the effectiveness of its Adjudication Center's security threat assessment process. Among other things, we reported that TSA had been delayed in addressing risks posed by using contractors to adjudicate security threat assessments. Specifically, we found that in October 2011 TSA's Balanced Workforce Strategy Working Group completed its assessment for the Adjudication Center and determined that an excessive risk exists by allowing contractors to make security threat assessment approvals without sufficient federal oversight. According to a May 2012 implementation plan, TSA planned to convert this workforce by the end of calendar year 2013. However, delays had rendered the timelines and cost information in its plan outdated and TSA had not updated the plan or determined a revised implementation schedule. To advance efforts to address risks identified in the Adjudication Center Balanced Workforce Strategy assessment, we recommended that DHS direct TSA to review and provide an updated Adjudication Center insourcing conversion plan to TSA and DHS leadership for review and implementation approval. DHS concurred with our recommendation. On January 29, 2014, the TSA Deputy Administrator approved and signed the updated insourcing conversion plan recommending approval to convert 46 contractor positions to 46 Federal positions in fiscal year 2014. TSA officials reported that 21 adjudicators had been hired as of November 2014 and they expected to have their planned hiring completed by the end of February 2015. TSA's actions are consistent with the intent of our recommendation.
|