Transportation Security: Comprehensive Risk Assessments and Stronger Internal Controls Needed to Help Inform TSA Resource Allocation
GAO-09-492
Published: Mar 27, 2009. Publicly Released: Apr 22, 2009.
Skip to Highlights
Highlights
The Department of Homeland Security (DHS) has called for using risk-informed approaches to help prioritize its investments, develop plans, and allocate resources in a way that balances security and commerce. Within DHS, the Transportation Security Administration (TSA) is responsible for making risk-informed investments to secure the transportation system. GAO evaluated to what extent TSA (1) implemented a risk management approach to inform the allocation of resources across the transportation sector and (2) followed internal control standards in its efforts to implement and use a risk management approach to inform resource allocation.
In conducting this work, GAO analyzed, among other things, DHS and TSA documents, such as TSA’s risk management methodology, and compared them to DHS’s risk management framework for infrastructure protection, compared TSA’s management activities to criteria in federal internal control standards, and interviewed DHS and TSA officials.
To promote effective use of risk management, GAO is recommending, among other things, that the Assistant Secretary, TSA, work with DHS to validate its risk management approach, conduct comprehensive risk assessments, and establish related internal controls. DHS concurred with all of our recommendations.
TSA has taken some actions but has not fully implemented a risk management approach to inform the allocation of resources across the transportation modes (aviation, mass transit, highway, freight rail, and pipeline). DHS’s risk management framework for infrastructure protection consists of six sequential steps that are used to systematically and comprehensively identify risk and establish risk-informed security priorities. TSA has taken some actions that the six steps require but has not conducted comprehensive risk assessments. For example, TSA collected information related to threat, vulnerability, and consequence within the transportation modes but has not conducted risk assessments that integrate these three components for each mode or the transportation sector as a whole. Identifying and prioritizing risk in this way is essential to efforts to allocate resources to address the highest priority risks. TSA developed an approach to prioritization based primarily on intelligence instead of comprehensive risk assessments. However, DHS has not reviewed or validated this methodology; thus, TSA lacks assurance that its approach provides the agency and DHS information needed to guide investment decisions to ensure resources are allocated to the highest risks. TSA also did not have a plan specifying the degree to which risk assessments are needed for the sector, the appropriate level of resources required to complete them, and time frames for completing its risk assessment efforts. Without a plan to identify the scope, resource requirements, and timeline for risk assessments, it will be difficult for TSA to ensure that it conducts timely and cost-effective risk assessments to inform resource allocation.
TSA has not followed federal internal control standards to assist it in implementing DHS’s risk management framework and informing resource allocation. Specifically, TSA lacked the following:
- An organizational structure that allows the agency to direct and control operations to achieve agency objectives. Although TSA officials acknowledged that a focal point for TSA’s risk management activities is needed, the agency has not yet established such a focal point.
- Policies, procedures, and guidance to assist its offices in ensuring that DHS’s National Infrastructure Protection Plan (NIPP) risk management framework and related activities, such as risk assessments, are implemented as DHS and TSA intended for the transportation sector and its individual modes.
- A mechanism to monitor the quality of performance. While TSA reports to DHS on the implementation of its risk management activities, it did not discuss all of the steps necessary to implement DHS’s risk management framework, such as the status of efforts taken to complete risk assessment activities including threat, vulnerability, and consequence assessments.
Without effectively implementing such controls, TSA cannot provide reasonable assurance that its resources are being used effectively and efficiently to achieve security priorities and that accountability and oversight regarding the quality of risk management activities implemented exists.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Transportation Security Administration | To promote the effective use of risk management at TSA, and to help provide assurances that resources are allocated to the highest priority risks across the transportation sector, the Assistant Secretary of TSA should better ensure that its risk management approach includes (1) adopting security goals that define specific outcomes, conditions, end points, and performance targets; and (2) conducting comprehensive risk assessments for the transportation sector that meet the NIPP criteria and combine individual assessments of threat, vulnerability, and consequence and analyzing these assessments to produce a comparative analysis of risk across the entire transportation sector to guide current and future investment decisions. |
Closed – Implemented
In March 2009, we reported that the Transportation Security Administration (TSA) had taken some actions to implement a risk management approach, including conducting assessments of threat, vulnerability, and consequence within the major transportation modes, but it had not conducted comprehensive risk assessments that integrate these three components for each mode or the transportation sector as a whole, as called for by the National Infrastructure Protection Plan (NIPP) and the Transportation Systems Sector-Specific Plan (TSSP). We recommended that TSA conduct risk assessments that combine threat, vulnerability, and consequence to help the agency produce a comparative analysis of risk across the entire transportation sector, which the agency could use to guide current and future investment decisions. TSA concurred, and in June 2010, TSA produced a Transportation Sector Security Risk Assessment (TSSRA), which assessed risk within and across the aviation, mass transit, highway, freight rail, and pipeline modes, and incorporated threat, vulnerability, and consequence. A September 2009 letter from the Director of the Department of Homeland Security's (DHS) Office of Risk Management and Analysis noted that in developing the TSSRA, TSA was making progress toward developing a strategic and comprehensive risk management approach that would better align with DHS's risk management framework and address our recommendations. However, TSA noted limitations in the June 2010 TSSRA report that could limit its usefulness in guiding investment decisions across the transportation sector as a whole. For example, the TSSRA excluded the maritime sector and certain types of threats, such as "lone wolf" operators. In June 2011, agency officials stated that TSA is working to address these limitations in the next version, which is scheduled for completion in 2012. TSA also said that it is strengthening and enhancing the TSSRA methodology based on a 2011 independent verification and validation. It will be important for TSA to continue to improve future versions of its risk assessment to better inform investment decisions. Regarding the second half of this recommendation, we reported in March 2009 that TSA did not define specific outcomes, conditions, end points, or performance targets for its security goals as called for by the NIPP, and we recommended that they do so. TSA officials stated in April 2010 that the agency had completed revised versions of its TSSP and related modal annexes, which they said would incorporate security goals that define specific outcomes, conditions, end points, and performance targets. TSA provided us with the revised TSSP and modal annexes in August 2011. While this document included strategic goals for transportation security, it did not define specific outcomes, conditions, end points, or performance targets for those goals. DHS revised its NIPP in 2009; the new NIPP calls for agencies to identify goals and objectives that describe the desired risk management posture and that consider distinct assets, systems, networks, functions, operational processes, business environments, and risk management approaches; define the risk management posture that partners seek to attain; and express this posture in terms of the outcomes and objectives sought. In August 2012, TSA submitted technical corrections to its revised TSSP and modal annexes to DHS for review and publication. These technical corrections address the requirements of the revised NIPP and our recommendation. TSA also created a performance dashboard to monitor progress across all modes of transportation in mitigating risk identified through the TSSRA, which should assist the agency in promoting the effective use of risk management and help provide assurances that resources are allocated to the highest priority risks across the transportation sector.
|
| Transportation Security Administration | To promote the effective use of risk management at TSA, the Assistant Secretary of TSA should establish an approach for gathering data on state and private sector security partners' investments in transportation security. |
Closed – Not Implemented
In March 2009, we reported that the Transportation Security Administration's (TSA) annual report included information on federal spending but not on investments made by states or private sector security partners, making it difficult for TSA to avoid potentially redundant efforts in transportation security and to identify security gaps within the transportation sector that have not been addressed by federal, private sector, or state security investments. While TSA communicated with its partners through its government coordinating council (GCC) and modal sector coordinating councils (SCC), TSA did not use the GCC and SCCs to help identify these private and state transportation security investments. We recommended that they establish an approach for gathering such data. In June 2010, TSA stated that for each mode, TSA developed a rough order of magnitude estimate of the total annual transportation security spending in the United States on the part of both federal and non-federal entities. Additionally, through the Transportation Systems GCC, TSA solicited input from its federal partners on state and private sector security spending data they have collected. In June 2011, however, TSA officials stated that the estimates were "very rough" and therefore of limited utility. The results were not particularly meaningful and did not justify the amount of effort and time involved in collecting the information. TSA therefore decided not to update it. As a result, this recommendation is closed as not implemented.
|
| Transportation Security Administration | To promote the effective use of risk management at TSA, the Assistant Secretary of TSA should establish a plan and milestones for conducting risk assessments for the transportation sector that identify the scope of the assessments and resource requirements for completing them. |
Closed – Implemented
In March 2009, we reported that Transportation Security Administration (TSA) officials told us that the agency uses an intelligence-driven approach to risk management to guide strategic investment decisions across the transportation sector because of concerns about the high cost of implementing the National Infrastructure Protection Plan's (NIPP) risk management framework and the methodological limitations of this approach. TSA officials cited that it is costly and time consuming to follow the NIPP's risk management framework - particularly in conducting comprehensive vulnerability and consequence assessments. However, TSA officials were not able to provide estimates of the time and resources needed to do so. For example, TSA did not have a plan specifying the degree to which risk assessments of the sector are needed, the scope of the risk assessments, the appropriate level of resources required to complete these assessments, and time frames for completing its risk assessment efforts. We recommended that TSA establish a plan and milestones for conducting transportation sector risk assessments that identify the scope of the assessments and resource requirements for completing them. TSA issued its first cross-sector risk assessment - the Transportation Sector Security Risk Assessment (TSSRA) - in June 2010 and is working on an annual update, which TSA expects to complete by December 2011. In July 2010, TSA officials stated that the agency worked with the National Protection and Programs Directorate's Office of Risk Management and Analysis (RMA) to frame the TSSRA during the report review process. A September 2009 letter from the RMA Director supports this statement, noting that TSA was making progress toward developing a strategic and comprehensive risk management approach that would better align with DHS's risk management framework and address our recommendation. In developing the TSSRA, TSA moved away from an intelligence-driven approach to risk management and instead followed the NIPP's risk management framework. As a result, this recommendation is closed implemented.
|
| Transportation Security Administration | To promote the effective use of risk management at TSA, the Assistant Secretary of TSA should work with DHS to validate its risk management approach by establishing a plan and time frame for assessing the appropriateness of TSA's intelligence-driven risk management approach for managing risk at TSA and document the results of this review once completed. |
Closed – Implemented
In March 2009, we reported that the Transportation Security Administration (TSA) had not worked with the Department of Homeland Security (DHS) to validate its risk management approach and therefore TSA lacked assurance that its approach would provide the agency and DHS information needed to guide investment decisions to ensure resources are allocated to the highest risks. We recommended that they establish a plan and time frame for assessing the appropriateness of TSA's intelligence-driven risk management approach. In July 2010, TSA officials stated that the agency worked with the National Protection and Programs Directorate's (NPPD) Office of Risk Management and Analysis during the framing of TSA's June 2010 transportation security risk assessment and during the report review process. They stated that TSA's risk management framework fully supports the threat-vulnerability-consequence risk methodology prescribed in the National Infrastructure Protection Plan (NIPP) and said that the cross-modal analysis is intelligence-based. A September 2009 letter from the RMA Director supports this statement, noting that TSA was making progress toward developing a strategic and comprehensive risk management approach that would better align with DHS's risk management framework and address our recommendation. As a result, this recommendation is closed as implemented.
|
| Transportation Security Administration | To promote the effective use of risk management at TSA, the Assistant Secretary of TSA should work with the Director of National Intelligence to determine the best approach for assigning uncertainty or confidence levels to analytic intelligence products and apply this approach to intelligence products. |
Closed – Implemented
In March 2009, we reported that Transportation Security Administration (TSA) officials did not assign uncertainty or confidence levels to the intelligence information TSA used to identify threats and guide long-range planning and strategic investment, and recommended that TSA work with the Office of the Director of National Intelligence (ODNI) to determine the best approach for assigning uncertainty or confidence levels to analytic intelligence products and apply this approach to intelligence products. As part of its Transportation Sector Security Risk Assessment (TSSRA) effort, TSA officials stated in August 2009 that they were independently reviewing criteria that other intelligence agencies, such as the U.S. Coast Guard and the Federal Bureau of Investigation, use to assign confidence levels and uncertainty to their intelligence products. In July 2010, TSA officials stated that TSA engaged ODNI and DHS's Office of Intelligence and Analysis to determine the best approach for assigning uncertainty or confidence levels to TSA's analytic intelligence products. As a result, the overall risk results in TSA's June 2010 TSSRA include uncertainty bands ranging from a "high" to a "low" estimate around a "best" estimate for each attack scenario that was used to inform the TSSRA. These uncertainty bands stemmed from "high," "best," and "low" estimates for the vulnerability and consequence of each scenario, but not for threat. TSA officials stated in June 2011 that the next version of the TSSRA would likely include uncertainty bands for threat as well as vulnerability and consequence. In addition, in June 2011 TSA officials stated that TSA's approach for assigning uncertainty and confidence levels to TSA's analytic intelligence products now follows ODNI's guidance. As a result, this recommendation is closed as implemented.
|
| Transportation Security Administration | To promote the effective use of risk management at TSA, the Assistant Secretary of TSA should establish internal controls, including (1) a focal point and clearly defined roles and responsibilities for ensuring the risk management framework is implemented; (2) policies, procedures, and guidance that require the implementation of its framework and completion of related work activities; and (3) a system to monitor and improve how effectively the framework is being implemented. |
Closed – Implemented
In March 2009, we reported that the Transportation Security Administration (TSA) could strengthen its internal controls to help implement the National Infrastructure Protection Plan's (NIPP) risk management framework. We recommended that they establish internal controls, including an organizational structure with a focal point and clearly defined roles and responsibilities to organize TSA's efforts to implement the framework; policies, procedures and guidance that require the use of the NIPP's risk management framework; and a mechanism to monitor the implementation of the NIPP's risk management framework to help ensure that results are achieved and performance improved. TSA established an Executive Risk Steering Committee (ERSC) in February 2009; according to TSA officials, the ERSC met 12 times between March 2009 and April 2011. In June 2011, TSA officials clarified that the ERSC serves as a focal point for strategic risk management at TSA and also provides an organizational structure for implementing the NIPP's risk management framework and for overseeing and monitoring the implementation of that framework. The ERSC's charter states the purpose of the committee and describes its membership, as well as clear lines of reporting. According to TSA, its ERSC primarily serves to monitor progress of TSA's Transportation Sector Security Risk Assessment process. As a result, this recommendation is closed as implemented.
|
Full Report
Office of Public Affairs
Topics
Agency missionsClassified defense informationComputer securityCost analysisCost effectiveness analysisCritical infrastructureCritical infrastructure protectionEmergency preparednessHomeland securityInternal controlsQuality assuranceReporting requirementsRisk assessmentRisk managementSecurity assessmentsStrategic planningSystems analysisTerrorismTransportation industryTransportation planningTransportation safetyTransportation security