Supply Chain Security:

CBP Needs to Conduct Regular Assessments of Its Cargo Targeting System

GAO-13-9: Published: Oct 25, 2012. Publicly Released: Nov 26, 2012.

Additional Materials:

Contact:

Stephen L. Caldwell
(202) 512-9610
caldwells@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

U.S. Customs and Border Protection (CBP), within the Department of Homeland Security (DHS), employs a risk-based approach that uses the Automated Targeting System (ATS) and other tools to identify (target) maritime cargo shipments for further examination. ATS is a web-based enforcement and decision support system that includes a set of rules to assess the risk level for each arriving cargo shipment. This set of rules is referred to as the maritime national security weight set (weight set) because each rule in the set has a specific weighted value assigned to it. CBP classifies the risk scores from the weight set as low, medium, or high risk. CBP policy states that a shipment's risk score is to determine, in part, actions taken by CBP officers (targeters) at the ports. Specifically, targeters are generally required to review shipment data for all medium-risk and high-risk shipments and hold high-risk shipments for examination. The risk score, however, is not the sole factor that determines whether a targeter reviews the data for a shipment or whether CBP examines a shipment. In particular, targeters at each of the six ports GAO visited explained that they use the ATS risk score as a starting point for the targeting process but that their decisions regarding which shipments to examine are ultimately based on additional research. Targeters at the six ports GAO visited said they also use tools outside of ATS, such as web searches, to research shipments.

CBP efforts to assess the weight set's effectiveness in identifying the risk of shipments have been limited. CBP has performance measures--represented by the percentage of shipments targeted as high risk that contain a threat and the percentage of shipments targeted as high risk that do not contain a threat--that enable CBP to determine the accuracy of the weight set, given a particular workload or examination rate. However, CBP did not assess the weight set to verify its effectiveness when implementing an updated version in early 2011. Prior to implementing the updated version of the weight set, CBP assessed the potential impact of the update on CBP's workload but did not conduct an assessment to determine whether the updated version of the weight set would be more effective in identifying high-risk shipments than the previous version or other alternatives. Assessing the potential effectiveness of alternative versions of the weight set prior to selecting one for implementation could help CBP make more informed decisions about future updates. Doing so could also provide CBP reasonable assurance that the version it selects is the most effective of the alternatives and is more effective than the previous version it replaces. Furthermore, since implementing the updated version of the weight set in early 2011, CBP has not regularly assessed the weight set to monitor its performance and to help determine when changes are needed. For example, CBP conducted the first assessment of the current version of the weight set, using the performance measures, in the summer of 2012--18 months after the weight set's implementation in early 2011. Regular assessments of the weight set's effectiveness could help CBP determine when updates are needed in a timelier manner and ensure that targeters have the best information available to make targeting decisions. Moreover, CBP has not established targets for the performance measures so that it is not clear whether a particular change in the weight set's performance is significant enough to suggest that changes are needed to improve the effectiveness of the weight set.

Why GAO Did This Study

The U.S. economy is dependent on the expeditious flow of millions of tons of cargo each day. Cargo containers are an important instrument of global trade but also can present security concerns. CBP is responsible for administering container security programs, and its strategy for securing maritime cargo containers includes analyzing information to identify shipments that may contain terrorist weapons or other contraband. Because CBP has insufficient resources to examine every container, targeters use ATS to target which container shipments should be examined. GAO was asked to assess CBP's targeting efforts. This report addresses (1) how ATS supports CBP's targeting of maritime cargo container shipments for national security purposes and (2) the extent to which CBP assesses the effectiveness of ATS's national security targeting rules. GAO analyzed fiscal year 2011 CBP data on shipments and containers arriving at U.S. ports and containers scanned at these ports. GAO also visited six CBP units selected on the basis of the percentage of maritime shipments that were scored as high risk or medium risk for national security purposes at these locations in fiscal year 2011, among other factors. GAO also analyzed documents, such as CBP's ATS performance measures.

What GAO Recommends

GAO recommends that CBP (1) ensure that future updates to the weight set are based on assessments of its performance and (2) establish targets for performance measures and use those measures to regularly assess effectiveness of the weight set. DHS concurred with these recommendations.

For more information, contact Stephen L. Caldwell at (202) 512-9610 or caldwells@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: In May 2014, CBP officials stated that CBP's Office of Intelligence and Investigative Liaison plans to develop a standard operating procedure that is to include descriptions of how CBP uses performance measures to (1) assess the effectiveness of an Automated Targeting System ( ATS) weight set and (2) make subsequent decisions regarding which updates to implement for that weight set. CBP plans to have this standard operating procedure issued by June 2015.

    Recommendation: To enhance its targeting of maritime cargo containers and better position CBP to provide reasonable assurance of the effectiveness of ATS, the Commissioner of CBP should ensure that future updates to the weight set are based on results of assessments that demonstrate that the chosen version of the weight set is more effective than other alternatives, including the existing version.

    Agency Affected: Department of Homeland Security: United States Customs and Border Protection

  2. Status: Open

    Comments: In May 2014, CBP officials stated that CBP's Office of Intelligence and Investigative Liaison plans to develop a standard operating procedure that is to include descriptions of how CBP uses performance measures to (1) assess the effectiveness of an Automated Targeting System (ATS) weight set and (2) make subsequent decisions regarding which updates to implement for that weight set. CBP plans to have this standard operating procedure issued by June 2015.

    Recommendation: To enhance its targeting of maritime cargo containers and better position CBP to provide reasonable assurance of the effectiveness of ATS, the Commissioner of CBP should establish targets for CBP's performance measures and use those measures to assess the effectiveness of the weight set on a regular basis to better determine when updates to the weight set are needed.

    Agency Affected: Department of Homeland Security: United States Customs and Border Protection

 

Explore the full database of GAO's Open Recommendations »

Sep 26, 2014

Sep 25, 2014

Sep 23, 2014

Sep 12, 2014

Jul 31, 2014

Jul 23, 2014

Jun 25, 2014

Jun 24, 2014

Looking for more? Browse all our products here