Transportation Worker Identification Credential: Progress Made in Enrolling Workers and Activating Credentials but Evaluation Plan Needed to Help Inform the Implementation of Card Readers

We found that weaknesses in the Transportation Security Administration?s (TSA) scheduling practices, such as not including all pilot activities or establishing the duration of activities, could hinder the usefulness of the pilot?s schedule as a management tool and the ability to obtain a clear insight into each phase of the pilot assessment. As a result, we recommended that TSA fully incorporate best practices, such as ensuring that it captures all pilot activities and assigns sufficient resources, into the pilot schedule. In response, the Department of Homeland Security (DHS) said in March 2010 that the current Transportation Worker Identification Credential (TWIC) reader pilot schedule has implemented best practices and is tailored to specifically meet the management requirements relative to the complex and unique constraints of the pilot program. While TSA made improvements to the TWIC pilot schedule, TSA did not fully implemented the practices that we recommended and the schedule remained unreliable for managing stakeholder participation and projecting the pilot's completion date. According to TSA, the pilot was completed on May 31, 2011. A report on the findings of the pilot was to be issued by April 2010. However, as of November 2011, the report remains to be issued.

We reported that the Transportation Security Administration (TSA) and Coast Guard had not developed an evaluation plan that fully identified the scope of the pilot and specified how information from the pilot would be analyzed. This shortfall partly hindered their efforts to ensure that the pilot is broadly representative of deployment conditions and will yield the information needed. We recommended that TSA and Coast Guard develop an evaluation plan to guide the remainder of the pilot that includes performance standards, a clearly articulated evaluation methodology, and a data analysis plan. However, as we reported in May 2013, while TSA developed a data analysis plan, TSA and USCG reported that they did not develop an evaluation plan with an evaluation methodology or performance standards, as we recommended. The data analysis plan was a positive step because it identified specific data elements to be captured from the pilot for comparison across pilot sites. If accurate data had been collected, adherence to the data analysis plan could have helped yield valid results. However, TSA and the independent test agent did not utilize the data analysis plan. According to officials from the independent test agent, they started to use the data analysis plan but stopped using the plan because they were experiencing difficulty in collecting the required data and TSA directed them to change the reporting approach. TSA officials stated that they directed the independent test agent to change its collection and reporting approach because of TSA's inability to require or control data collection to the extent required to execute the data analysis plan. We are therefore closing this recommendation as not implemented.

We reported that the Transportation Worker Identification Credential (TWIC) reader pilot was to, among other things, test the technology, business processes, and operational impacts required to deploy card readers at secure areas of the marine transportation system. However, we found that the Transportation Security Administration (TSA) had not developed a detailed evaluation plan to describe how the collected data was to be used to track the program's performance and evaluate the effectiveness of using TWIC with biometric card readers. We recommended that that TSA and Coast Guard identify how they will compensate for areas where the TWIC reader pilot will not provide the necessary information needed to report to Congress and implement the card reader rule. In April 2013, Coast Guard further reported that it augmented TWIC pilot data by conducting additional studies regarding physical access control systems and congestion delays, as well as other analyses, which it used to complete the TWIC reader Notice of Proposed Rulemaking published in March of 2013. As we reported in May 2013, the studies did not compensate for all of the challenges we identified in our November 2009 report. We are therefore closing this recommendation as not implemented.