Anthrax:

DHS Faces Challenges in Validating Methods for Sample Collection and Analysis

GAO-12-488: Published: Jul 31, 2012. Publicly Released: Sep 11, 2012.

Additional Materials:

Contact:

Timothy M. Persons, Ph.D.
(202) 512-3000
personst@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

A workgroup—led by the U.S. Department of Homeland Security (DHS) and made up of DHS and the Centers for Disease Control and Prevention (CDC), the Environmental Protection Agency (EPA), the Federal Bureau of Investigation (FBI), and the National Institute of Standards and Technology (NIST)—has attempted to address GAO’s recommendations to (1) validate environmental sampling methods for detecting Bacillus anthracis and (2) conduct studies to develop probability-based sampling approaches for indoor environments. This workgroup has taken some actions to validate environmental sampling methods (collection, transportation, preparation, analysis) and develop statistically based sampling approaches that will provide confidence statements when test results are negative. These activities were projected to be completed by fiscal year 2013, but delays are now expected.

While progress has been made in validating sampling methods for detecting Bacillus anthracis spores in indoor environments, their validation is not yet complete. Some studies have not begun. Although more is known about the methods’ performance characteristics—such as their limits of detection—other aspects of the methods are unknown, such as false negative rates. CDC has validated the preparation and analysis but not the collection methods for the swab and wipe. CDC states that field validation would be too difficult and laboratory validation of collection methods is not required. However, experts GAO talked to stated that collection methods could be validated in a laboratory.

Agencies that perform environmental sampling take the lead in validating the sampling methods. The FBI does not typically use CDC’s environmental sampling methods and validating its methods is outside the scope of the DHS-led workgroup. The FBI’s environmental sampling methods are not validated but the agency relies on DHS’s National Bioforensic Analysis Center (NBFAC) to validate its microbial forensic analytical methods. Thus, the FBI, through NBFAC, and CDC are attempting to validate analytical methods for Bacillus anthracis but neither is validating the collection methods. Nevertheless, improvements in sample collection procedures for the swab and wipe could be useful to the FBI in developing its sampling plans or in evaluating its sampling methods.

The workgroup must address several remaining challenges before the validation project can be completed: (1) clarifying the strategic plan’s scope—some agencies believe it is overly ambitious and differ on whether it includes linking sampling results to a risk-based decision process—and determining whether the workgroup is to continue; (2) reaching consensus on the range of sampling approaches that should be available to decision makers in different phases of a response; (3) establishing realistic estimates of the time for completing prioritized validation activities; (4) addressing scientific gaps, such as assessing risk in the absence of dose-response data; and (5) ensuring the availability of funds for critical tasks. While validating the methods provides information on performance characteristics, human health risks from any particular level of exposure remain uncertain. Since the workgroup has invested about $12 million and considerable resources over about 7 years, it would be prudent for it to complete prioritized tasks. Thus, the workgroup may wish to consider carefully what work is needed and think strategically in terms of its investments and their potential benefits.

Why GAO Did This Study

In 2005, assessing federal agencies’ activities for detecting Bacillus anthracis in postal facilities, GAO reported that the test results of their sampling were largely negative. GAO found that the agencies had not used validated sampling methods and approaches that would have given a defined level of confidence for negative results. Consequently, GAO recommended several actions. In this study, GAO was asked to identify the extent to which (1) DHS’s actions have addressed GAO’s recommendations regarding sampling, (2) the environmental sampling methods for B. anthracis spore detection in initial public health sampling and microbial forensic investigations have been validated, and (3) any challenges remain to completing validation. GAO analyzed agency documents and interviewed agency officials.

What GAO Recommends

To ensure validated sampling methods and approaches are available for decision makers to respond to an indoor Bacillus anthracis release, DHS should (1) update the strategic plan and its roadmap with an agreed scope and timelines, and (2) complete the validation project. The Secretary of HHS and the Administrator of EPA should support DHS’s goal of achieving validated sampling methods and a statistically based sampling approach. DHS agreed with our recommendations; EPA and HHS disagreed with our recommendation to them, stating that such an approach was not feasible or necessary. We continue to believe a validated statistical sampling approach will provide a broader range of options for decision makers responding to future incidents.

For more information, contact Timothy M. Persons, Chief Scientist, at (202) 512-6412 or personst@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To ensure that federal agencies have validated sampling methods for detecting B. anthracis in indoor environments and--in the case of negative results--the option of using appropriate sampling approaches to make statistical confidence statements about the likelihood that a building is free of contamination when potentially there has been a low-level release, the Secretary of Homeland Security should take steps to complete the validation project. Statistically-based sampling designs for such purposes would encompass any sampling with a statistical basis, including a probabilistic only approach as well as one that combines judgmental and probabilistic sampling. Achieving a sufficiently rigorous validation of the sampling methods and ensuring that statistically rigorous and mutually acceptable sampling approaches are available will provide options that will better prepare decision makers to respond to a future bioterrorism incident.

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of Homeland Security should update the strategic plan and its roadmap with an agreed-on scope and revised timelines.

    Agency Affected: Department of Homeland Security

  3. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of Homeland Security should complete the validation project, including validating the collection methods in a laboratory setting in a manner that determines the potential sources of variation in collection method performance, including variation that could be introduced by individual samplers, and related ongoing studies.

    Agency Affected: Department of Homeland Security

  4. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of the Department of Health and Human Services and the Administrator of the Environmental Protection Agency should support DHS in its goal of achieving (1) validated sampling methods to understand the limitations of the data that would be provided to decision makers, and (2) a mutually acceptable statistically-based sampling approach that can be employed when decision makers--such as Incident Commanders and others--conclude that statistical confidence statements need to be made about the level of contamination in a particular indoor environment.

    Agency Affected: Department of Health and Human Services

  5. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of the Department of Health and Human Services and the Administrator of the Environmental Protection Agency should support DHS in its goal of achieving (1) validated sampling methods to understand the limitations of the data that would be provided to decision makers, and (2) a mutually acceptable statistically-based sampling approach that can be employed when decision makers--such as Incident Commanders and others--conclude that statistical confidence statements need to be made about the level of contamination in a particular indoor environment.

    Agency Affected: Environmental Protection Agency

 

Explore the full database of GAO's Open Recommendations »

Dec 19, 2014

Dec 17, 2014

Nov 6, 2014

Oct 14, 2014

Sep 30, 2014

Sep 24, 2014

Sep 18, 2014

Looking for more? Browse all our products here