Combating Nuclear Smuggling:

DHS's Cost-Benefit Analysis to Support the Purchase of New Radiation Detection Portal Monitors Was Not Based on Available Performance Data and Did Not Fully Evaluate All the Monitors' Costs and Benefits

GAO-07-133R: Published: Oct 17, 2006. Publicly Released: Oct 17, 2006.

Additional Materials:

Contact:

Eugene E. Aloise
(202) 512-6870
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Since the attacks of September 11, 2001, combating terrorism has been one of the nation's highest priorities. As part of that effort, preventing nuclear and radioactive material from being smuggled into the United States--perhaps to be used by terrorists in a nuclear weapon or in a radiological dispersal device (a "dirty bomb")--has become a key national security objective. The Department of Homeland Security (DHS) is responsible for providing radiation detection capabilities at U.S. ports-of-entry. Until April 2005, U.S. Customs and Border Protection (CBP) under DHS managed this program. However, on April 15, 2005, the president directed the establishment, within DHS, of the Domestic Nuclear Detection Office (DNDO), whose duties include acquiring and supporting the deployment of radiation detection equipment. CBP continues its traditional screening function at ports-of-entry to interdict dangerous nuclear and radiological materials through the use of radiation detection equipment, including portal monitors. The Pacific Northwest National Laboratory (PNNL), one of the Department of Energy's (DOE) national laboratories, manages the deployment of radiation portal monitors for DHS. Current portal monitors, which cost about $55,000 per monitor, detect the presence of radiation, but cannot distinguish between harmless radiological materials, such as naturally occurring radiological material in some ceramic tile, and dangerous nuclear materials, such as highly enriched uranium (HEU). CBP officers also use radioactive isotope identification devices (RIIDs), which are handheld devices designed to identity different types of radioactive material, such as radioactive material used in medicine or industry, a naturally occurring source of radiation, or weapons-grade material. These devices have limitations in their ability to detect and identify nuclear material. DHS would like to improve the capabilities of its radiation detection equipment in order to better distinguish between different types of nuclear and radiological materials. As a result, DHS sponsored research, development, and testing activities in 2005 that were designed to produce portal monitors that, in addition to detecting, would also identify the type of nuclear or radiological material. Portal monitors with this new identification technology currently cost about $377,000 or more per monitor. In these same tests, DHS also tested the performance of currently deployed portal monitors. In July 2006, DHS announced that it had awarded contracts to three vendors to further develop and purchase $1.2 billion worth of new portal monitors over 5 years. DHS plans to deploy these monitors at U.S. ports of entry. For fiscal year 2007, DNDO plans to acquire the first installment of 104 new portal monitors that use new identification technology at a cost of $80.2 million. Congress, however, has curtailed DNDO's ability to do so by restricting the availability of funding for full scale procurement of new radiation detection portal monitors until DHS certifies that a significant increase in operational effectiveness will be achieved. In response to our recommendations in an earlier March 2006 report, in May 2006, DNDO issued a cost-benefit analysis for the acquisition and deployment of new portal monitors. In this document, DNDO stated that the purpose of its analysis was to help provide a "robust" defense against nuclear smuggling, to limit the negative impacts to legitimate trade and travel between nations, and to provide a sound financial investment for the United States government. In this context, by agreement with Congressional staff, we reviewed DNDO's cost-benefit analysis to determine the extent to which it provides a sound analytical basis for acquiring and deploying new portal monitors.

DNDO's cost-benefit analysis does not provide a sound analytical basis for DNDO's decision to purchase and deploy new portal monitor technology. DNDO did not use the results of its own performance tests in its cost-benefit analysis and instead relied on assumptions of the new technology's anticipated performance level. Performance tests also showed that the ability of new radiation detection portal monitors to correctly identify masked HEU (placed next to or within another, usually more benign, radiological substance) was even more limited. According to the cost-benefit analysis and radiation detection experts to whom we spoke, masked HEU is a significant concern because it is difficult to detect. DNDO also focused the analysis exclusively on identifying HEU and did not consider in the analysis how well (either as a goal or in testing) new portal monitor technology can correctly detect or identify other dangerous radiological or nuclear materials. Furthermore, the analysis did not include the results from side-by-side tests that DNDO conducted of the advanced portal monitors and current portal monitors. The cost-benefit analysis for acquiring and deploying portal monitors is also incomplete because it does not include all of the major costs and benefits required by DHS guidelines. In particular, DNDO did not assess the likelihood that radiation detection equipment would either misidentify or fail to detect nuclear or radiological material. Rather, it focused its analysis on reducing the time necessary to screen traffic at border check points and reduce the impact of any delays on commerce. DNDO also used questionable assumptions about the procurement costs of portal monitor technology. DNDO assumed a purchase price for current portal monitor technology that is more than twice what CBP typically pays.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: DNDO announced that it is no longer pursuing the acquisition of ASPs as originally intended. While DNDO did not perform a formal cost benefits analysis, the agency did consider the performance of the ASP over the last 3 years of testing in deciding to shutdown the program. In short, the ASP had difficulty meeting its expected performance standards, and did not deliver the expected benefits that would make its costs worthwhile.

    Recommendation: DHS and DNDO should conduct a new cost-benefit analysis to justify the purchase of new radiation detection portal monitors by using sound analytical methods, including actual performance data and a complete accounting of all major costs and benefits as required by DHS guidelines.

    Agency Affected: Department of Homeland Security: Domestic Nuclear Detection Office

  2. Status: Closed - Implemented

    Comments: The purpose of this recommendation was to prevent DHS/DNDO from going forward with a large procurement and deployment of ASPs without first doing a valid cost-benefit analysis. As a direct result of a series of GAO reports and testimonies about the technical and operational shortcomings of the ASP, DNDO's director announced it was cancelling the ASP program in testimony before the House Homeland Security Subcommittee on Cybersecurity, Infrastructure Protection, and Security Technologies. In addition, because of all the difficulties encountered by the ASP in testing, DNDO delayed undertaking the expense of a cost benefit analysis until these difficulties could be overcome and procurement and deployment became a viable option. Thus, the overall purpose of the recommendation was accomplished because DNDO did not proceed with the program without doing a valid cost benefit analysis.

    Recommendation: DHS and DNDO should conduct a new cost-benefit analysis to justify the purchase of new radiation detection portal monitors by using sound analytical methods, including actual performance data and a complete accounting of all major costs and benefits as required by DHS guidelines.

    Agency Affected: Department of Homeland Security: Domestic Nuclear Detection Office

  3. Status: Closed - Implemented

    Comments: As discussed in GAO reports on advanced spectroscopic portal (ASP) testing (GAO-09-655) issued in May, 2009, DHS increased the rigor of testing in comparison with previous tests, and the improvements added credibility to the test results. For example, DNDO addressed limitations in the previous round of performance testing comparing ASPs to handheld identification devices. In particular, Customs and Border Protection officers operated the devices and adhered as closely to the standard operating procedure as test conditions allowed. In general, GAO concluded that the design for the most recent round of testing was sufficient to identify statistically significant differences between the new technology and current-generation systems when there were relatively large differences in performance.

    Recommendation: DNDO should conduct realistic testing for both advanced spectroscopic portal monitors (ASPs) and plastic scintillators in order to meet the Congressional requirement outlined in the Homeland Security Appropriations Act for Fiscal Year 2007 that DHS certify that a "significant increase in operational effectiveness" will be achieved by ASPs before spending additional funds to purchase ASPs for deployment.

    Agency Affected: Department of Homeland Security

 

Explore the full database of GAO's Open Recommendations »

Sep 24, 2014

Sep 18, 2014

Sep 17, 2014

Sep 10, 2014

Sep 9, 2014

Sep 8, 2014

Jul 31, 2014

Jul 29, 2014

Looking for more? Browse all our products here