Supply Chain Security: CBP Needs to Conduct Regular Assessments of Its Cargo Targeting System
Highlights
What GAO Found
U.S. Customs and Border Protection (CBP), within the Department of Homeland Security (DHS), employs a risk-based approach that uses the Automated Targeting System (ATS) and other tools to identify (target) maritime cargo shipments for further examination. ATS is a web-based enforcement and decision support system that includes a set of rules to assess the risk level for each arriving cargo shipment. This set of rules is referred to as the maritime national security weight set (weight set) because each rule in the set has a specific weighted value assigned to it. CBP classifies the risk scores from the weight set as low, medium, or high risk. CBP policy states that a shipment's risk score is to determine, in part, actions taken by CBP officers (targeters) at the ports. Specifically, targeters are generally required to review shipment data for all medium-risk and high-risk shipments and hold high-risk shipments for examination. The risk score, however, is not the sole factor that determines whether a targeter reviews the data for a shipment or whether CBP examines a shipment. In particular, targeters at each of the six ports GAO visited explained that they use the ATS risk score as a starting point for the targeting process but that their decisions regarding which shipments to examine are ultimately based on additional research. Targeters at the six ports GAO visited said they also use tools outside of ATS, such as web searches, to research shipments.
CBP efforts to assess the weight set's effectiveness in identifying the risk of shipments have been limited. CBP has performance measures--represented by the percentage of shipments targeted as high risk that contain a threat and the percentage of shipments targeted as high risk that do not contain a threat--that enable CBP to determine the accuracy of the weight set, given a particular workload or examination rate. However, CBP did not assess the weight set to verify its effectiveness when implementing an updated version in early 2011. Prior to implementing the updated version of the weight set, CBP assessed the potential impact of the update on CBP's workload but did not conduct an assessment to determine whether the updated version of the weight set would be more effective in identifying high-risk shipments than the previous version or other alternatives. Assessing the potential effectiveness of alternative versions of the weight set prior to selecting one for implementation could help CBP make more informed decisions about future updates. Doing so could also provide CBP reasonable assurance that the version it selects is the most effective of the alternatives and is more effective than the previous version it replaces. Furthermore, since implementing the updated version of the weight set in early 2011, CBP has not regularly assessed the weight set to monitor its performance and to help determine when changes are needed. For example, CBP conducted the first assessment of the current version of the weight set, using the performance measures, in the summer of 2012--18 months after the weight set's implementation in early 2011. Regular assessments of the weight set's effectiveness could help CBP determine when updates are needed in a timelier manner and ensure that targeters have the best information available to make targeting decisions. Moreover, CBP has not established targets for the performance measures so that it is not clear whether a particular change in the weight set's performance is significant enough to suggest that changes are needed to improve the effectiveness of the weight set.
Why GAO Did This Study
The U.S. economy is dependent on the expeditious flow of millions of tons of cargo each day. Cargo containers are an important instrument of global trade but also can present security concerns. CBP is responsible for administering container security programs, and its strategy for securing maritime cargo containers includes analyzing information to identify shipments that may contain terrorist weapons or other contraband. Because CBP has insufficient resources to examine every container, targeters use ATS to target which container shipments should be examined. GAO was asked to assess CBP's targeting efforts. This report addresses (1) how ATS supports CBP's targeting of maritime cargo container shipments for national security purposes and (2) the extent to which CBP assesses the effectiveness of ATS's national security targeting rules. GAO analyzed fiscal year 2011 CBP data on shipments and containers arriving at U.S. ports and containers scanned at these ports. GAO also visited six CBP units selected on the basis of the percentage of maritime shipments that were scored as high risk or medium risk for national security purposes at these locations in fiscal year 2011, among other factors. GAO also analyzed documents, such as CBP's ATS performance measures.
Recommendations
GAO recommends that CBP (1) ensure that future updates to the weight set are based on assessments of its performance and (2) establish targets for performance measures and use those measures to regularly assess effectiveness of the weight set. DHS concurred with these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| United States Customs and Border Protection | To enhance its targeting of maritime cargo containers and better position CBP to provide reasonable assurance of the effectiveness of ATS, the Commissioner of CBP should ensure that future updates to the weight set are based on results of assessments that demonstrate that the chosen version of the weight set is more effective than other alternatives, including the existing version. |
In May 2015, CBP revised its National Security Weight Set, Maritime Standard Operating Procedures (SOP) to address the new requirements for the maintenance, review, and update of the national security weight set (maritime) in ATS. The SOP requires program managers to compare proposed versions of the national security weight set against the existing version as part of the process for determining whether to implement a proposed new version of the weight set. Doing so will help provide reasonable assurance that changes to the weight set will improve the effectiveness of CBP's targeting of high-risk maritime cargo container shipments.
|
| United States Customs and Border Protection | To enhance its targeting of maritime cargo containers and better position CBP to provide reasonable assurance of the effectiveness of ATS, the Commissioner of CBP should establish targets for CBP's performance measures and use those measures to assess the effectiveness of the weight set on a regular basis to better determine when updates to the weight set are needed. |
In May 2015, CBP revised its National Security Weight Set, Maritime Standard Operating Procedures (SOP) to address the new requirements for the maintenance, review, and update of the national security weight set (maritime) in ATS. The SOP established a performance measure and associated target that will assist CBP in determining whether the weight set is effectively targeting high-risk maritime cargo container shipments. The SOP requires CBP to review the national security weight set for revisions if the weight set does not meet the performance target in two consecutive quarters. By assessing the weight set regularly against a performance target, CBP will be better positioned to determine when updates to the weight set are needed to ensure continued effectiveness in targeting of high-risk maritime cargo container shipments.
|