This is the accessible text file for GAO report number GAO-05-493T entitled 'Anthrax Detection: Agencies Need to Validate Sampling Activities in Order to Increase Confidence in Negative Results' which was released on April 5, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony before the Chairman, Subcommittee on National Security, Emerging Threats, and International Relations, House Committee on Government Reform, House of Representatives: United States Government Accountability Office: GAO: For Release on Delivery Expected at 2:00 p.m. Tuesday, April 5, 2005: Anthrax Detection: Agencies Need to Validate Sampling Activities in Order to Increase Confidence in Negative Results: Statement of Keith A. Rhodes, Chief Technologist, Center for Technology and Engineering, Applied Research and Methods: GAO-05-493T: GAO Highlights: Highlights of GAO-05-493T, a testimony before Chairman, Subcommittee on National Security, Emerging Threats, and International Relations, House Committee on Government Reform, House of Representatives: Why GAO Did This Study: In September and October 2001, letters laced with Bacillus anthracis (anthrax) spores were sent through the mail to two U.S. senators and to members of the media. These letters led to the first U.S. cases of anthrax disease related to bioterrorism. In all, 22 individuals, in four states and Washington, D.C., contracted anthrax disease; 5 died. These cases prompted the Subcommittee to ask GAO to describe and assess federal agencies' activities to detect anthrax in postal facilities, assess the results of agencies' testing, and assess whether agencies' detection activities were validated. What GAO Found: GAO recommends that the Department of Homeland Security (DHS) develop a coordinated approach to working with federal agencies, so that appropriate validation studies of various activities involved in detecting anthrax are conducted. The DHS Secretary should also ensure that an agreed-on definition of validation is developed; appropriate investments are made to explore improved sampling strategies; and agencies' policies, procedures, and guidelines reflect the results of all these efforts. DHS stated that while it has the overall responsibility for coordination, EPA and HHS have the lead roles in responding to biological attacks. DHS said that it would coordinate with EPA to ensure that appropriate investments are made to explore improved sampling. What GAO Recommends: The U.S. Postal Service, Centers for Disease Control and Prevention (CDC), and Environmental Protection Agency (EPA) conducted several interdependent activities, including sample collection and analytic methods, to detect anthrax in postal facilities in 2001. They developed a sampling strategy and collected, transported, extracted, and analyzed samples. They primarily collected samples from specific areas, such as mail processing areas, using their judgment about where anthrax would most likely be found--that is, targeted sampling. The agencies did not use probability sampling in their initial sampling strategy. Probability sampling would have allowed agencies to determine, with some defined level of confidence, when all results are negative, whether a building is contaminated. This is important, considering that low levels of anthrax could cause disease and death in susceptible individuals. [See PDF for image] [End of figure] The results of the agencies' testing in 286 postal facilities were largely negative--no anthrax was detected. But negative results do not necessarily mean that a facility is free from anthrax. In addition, agencies' detection activities (for example, sample collection and analytical methods) were not validated. Validation is a formal, empirical process in which an authority determines and certifies the performance characteristics of a given method. Consequently, the lack of validation of agencies' activities, coupled with limitations associated with their targeted sampling strategy, means that negative results may not be reliable. In preparing for future incidents, the agencies have (1) made some changes based on what has been learned about some of the limitations of their sampling strategies, (2) made some revisions to their guidelines, and (3) funded some new research. In addition, the Department of Homeland Security (DHS) has taken on the role of coordinating agencies' activities and has undertaken several new initiatives related to anthrax and other biothreat agents. However, while the actions DHS and other agencies have taken are important, they do not address the issue of validating all activities related to sampling. Finally, the agencies have not made appropriate and prioritized investments to develop and validate all activities related to anthrax and other biothreat agents. www.gao.gov/cgi-bin/getrpt?GAO-05-493T. To view the full product, including the scope and methodology, click on the link above. For more information, contact Keith Rhodes, 202-512- 6412, email@example.com. [End of section] April 5, 2005: Mr. Chairman and Members of the Subcommittee: We are pleased to participate in this hearing by presenting our assessment of the federal agencies'--U.S. Postal Service (USPS), Centers for Disease Control and Prevention, and Environmental Protection Agency (EPA)--activities conducted to detect anthrax in postal facilities in 2001. My statement is based on our report, entitled Anthrax Detection: Agencies Need to Validate Sampling Activities in Order to Increase Confidence in Negative Results, which was issued on March 31, 2005.[Footnote 1] As you know, in September and October 2001, contaminated letters laced with Bacillus anthracis, or anthrax spores,[Footnote 2] were sent through the mail to two senators, Thomas Daschle and Patrick Leahy, and members of the media. The postal facilities in New Jersey and Washington, D.C., that processed the senators' letters became heavily contaminated.[Footnote 3] Other mail routed through these facilities, as well as additional ones in the postal network, also became contaminated. In addition, numerous federal facilities in the Washington, D.C., area were later found to be contaminated. The letters led to the first cases of anthrax disease related to bioterrorism in the United States. In all, 22 individuals contracted anthrax disease in four states (Connecticut, Florida, New Jersey, and New York) as well as in Washington, D.C. Five of these 22 individuals died. The threat of bioterrorism has been recognized for a considerable time. Long before the anthrax incidents, several hoax letters indicating the presence of anthrax had been mailed to federal and state agencies, as well as to private sector organizations. In calendar year 2000, the Federal Bureau of Investigation (FBI) responded to about 250 cases potentially involving weapons of mass destruction. Of these, 200 were related to anthrax, although all turned out to be hoaxes. Nevertheless, these events raised the possibility that facilities could become contaminated and would therefore have to be evaluated for environmental contamination. However, federal agencies have not been fully prepared to deal with environmental contamination, that is, anthrax released through the mail, including the potential for multiple dispersals in indoor environments.[Footnote 4] Before I discuss our assessment, let me first present some background. (See appendix I for a discussion of our scope and methodology.) Background: Although anthrax can infect humans, it is most commonly found in plant- eating animals. Human anthrax infections are rare in the United States, and when infection does occur, it usually results from occupational exposure to infected animals or contaminated animal products, such as wool, hides, or hair. Anthrax infection can occur (1) cutaneously, usually from a cut or abrasion on the skin; (2) gastrointestinally, by ingesting undercooked, contaminated meat; and (3) through inhalation, by breathing aerosolized, or airborne, spores into the lungs. The response to the incident in the American Media Incorporated building in Florida in September 2001 led to the identification of mail as the potential source of contamination; eventually, it led to the sampling of the postal facilities. The agencies began sampling on October 12, 2001, in Florida and stopped on April 21, 2002, when the Wallingford, Connecticut, facility was sampled for the last time. Four contractors conducted USPS sampling. The mission of USPS is to provide affordable, universal mail service. As of May 28, 2004, more than 800,000 workers processed more than 200 billion pieces of mail a year. The USPS headquarters office is in Washington, D.C. USPS has nine area offices; approximately 350 P&DCs; and about 38,000 post offices, stations, and branches; the P&DCs vary widely in size and capacity. The USPS mail system is involved in collecting, distributing, and delivering letters, flats (that is, catalogs and magazines), and parcels, as well as other items that vary in size and capacity. The federal agencies involved in the response in the postal facilities had differing responsibilities. The Centers for Disease Control and Prevention (CDC) and state and local health departments primarily provided public health advice and assistance to USPS. CDC has had primary responsibility for national surveillance of specific diseases, including anthrax; it has also conducted epidemiologic investigations to determine, among other things, the source of the disease. The FBI has been responsible for criminal investigations involving interstate commerce and the mail and crimes committed on federal property. The Environmental Protection Agency (EPA) has been the nation's lead agency for responding to a release of hazardous substances into the environment. On October 8, 2001, the President created the Office of Homeland Security to develop and coordinate a comprehensive national strategy for dealing with domestic terrorist threats or attacks. The office, which had limited involvement in the 2001 response, was superseded by the Homeland Security Act of 2002, which transferred many of its functions to the Department of Homeland Security (DHS); it became operational in 2003. DHS was created by combining many previously separate agencies and is assigned a lead role in coordinating the efforts of federal agencies that respond to acts of terrorism in the United States. In addition, the Laboratory Response Network (LRN) was developed in 1999 to coordinate clinical diagnostic testing for bioterrorism. The primary purpose on the biological side was to detect the presence of biothreat agents in a number of specimen and sample types. These laboratories function as first responders that can perform standard initial tests to rule out, but not definitively confirm, anthrax. Now I will discuss our assessment of the following federal agencies' activities: (1) federal agencies' activities to detect anthrax contamination in the postal facilities; (2) the results of the federal agencies' testing in the postal facilities; and (3) whether agencies' activities were validated and, if not, discuss any issues that arose from the lack of validation and any actions they took to address these issues. Federal Agencies' Activities to Detect Anthrax Contamination in the Postal Facilities: CDC, EPA, and USPS, the federal agencies involved in sampling the postal facilities in 2001 to detect anthrax, undertook several activities: (1) sampling strategy development, followed by (2) sample collection, (3) transportation, (4) extraction, and (5) analysis of the samples (see fig. 1). Figure 1: Agency Sampling Activities: [See PDF for image] [End of figure] Neither these activities nor the overall process has been validated for anthrax testing. Consequently, the agencies had only limited information available for reliably choosing one method over another and no information on the limits of detection to use when evaluating negative results. In addition, the sampling strategy used by the agencies could not provide any statistical confidence with regard to the basic question: Is this building contaminated? Therefore, in the future, in the absence of a positive result, a different strategy is needed that will provide statistical confidence, at a defined level, to answer this question. Activity 1: Sampling Strategy Development: The first activity involved agencies' developing a sampling strategy, which included deciding how many samples to collect, where to collect them from, and what collection methods to use. The agencies primarily used a targeted strategy: They collected samples from specific areas considered more likely to be contaminated, based on judgments. Such judgments can be effective in some situations, for example, in determining (1) the source of contamination in a disease outbreak investigation or (2) whether a facility is contaminated when information on the source of potential contamination is definitive. However, in the case of a negative finding, when the source of potential contamination is not definitive, the basic question--Is this building contaminated?--will remain unanswered. Agencies Primarily Used a Targeted Strategy: The targeted strategy the agencies used was reflected in their site- specific sampling activities. Sample sizes varied by facility and circumstances, increased over time, and excluded probability sampling. In the beginning, in each USPS facility, 23 samples were to be collected from specific areas relating to mail processing and up to 20 additional "discretionary" samples were to be collected, depending on the type and size of the facility. Later, USPS increased the number of samples required to a minimum of 55, with up to 10 additional discretionary samples for larger facilities. Consequently, the number of samples collected varied by facility, from a low of 4 to a high of 148. CDC's and EPA's site-specific strategies were primarily discretionary. The number of samples CDC collected varied by facility, ranging from a low of 4 to a high of 202. The number of samples EPA collected ranged from a low of 4 to a high of 71. According to CDC, a targeted sampling strategy may be effective in detecting contamination in a facility when sufficient site-specific information exists to narrow down the locations in which the release and contamination are most likely to have occurred. CDC's assumptions for this strategy are that at the outset, (1) a scenario where all locations have an equal chance of being contaminated is generally the exception rather than the rule; (2) information collected about the event, combined with technical judgment about exposure pathways, can be used to identify locations where contamination is most likely to be found; (3) contamination levels of the highest public health concern can usually be detected using a variety of available methods, despite their limitations; and (4) there is important public health value in quickly identifying contaminated locations. However, these assumptions may not always apply. For example, there may be limitations in the available information that restrict the ability to reliably identify target locations. The method of contamination spread could conceivably be via a mechanism where there is an equal chance of any area being contaminated. Lastly, all results may be negative, which will lead to a requirement for additional testing, as was the case in Wallingford. This, in turn, will result in the loss of the critical time needed for public health intervention. CDC and USPS officials said that they used a targeted strategy for several reasons, including limitations on how many samples could be collected and analyzed. They also said that in 2001 they lacked the data necessary to develop an initial sampling strategy that incorporated probability sampling. We disagree with this interpretation. Probability sampling is statistically based and does not depend solely on empirical criteria regarding the details of possible contamination. Incorporating Probability Sampling Would Allow Greater Confidence in Negative Results: We consider probability sampling to be a viable approach that would address not only the immediate public health needs but also the wider public health protection, infrastructure cleanup, and general environmental contamination issues. We recognize that in a major incident, the number of samples that may need to be collected and analyzed may challenge available laboratory resources. Accordingly, there is a need to develop innovative approaches to use sampling methods that can achieve wide-area coverage with a minimal number of individual samples to be analyzed. For example, high-efficiency particulate air (HEPA) vacuum techniques, in combination with other methods, appear to be one such approach that could achieve this. In addition, because of limited laboratory capacity, samples may need to be stored after collection for subsequent analysis, on a prioritized basis. The situation in 2001 was unique, and the agencies were not fully prepared to deal with environmental contamination. In the future, if the agencies decide to use a targeted rather than a probability sampling strategy, they must recognize that they could lose a number of days if their targeted sampling produces negative test results. In this case, additional samples would need to be collected and analyzed, resulting in critical time, for public health interventions, being lost. This was so at the Wallingford postal facility in the fall of 2001, when about 3 weeks elapsed between the time the first sampling took place and the results of the fourth testing, which revealed positive results. Furthermore, about 5 months elapsed between the time of the first sampling event and the time anthrax was found in the Wallingford facility's high-bay area. Therefore, in the future, strategies that include probability sampling need to be developed in order to provide statistical confidence in negative results. Further, even if information on all the performance characteristics of methods is not yet available, a probability sampling strategy could be developed from assumptions about the efficiency of some of the methods. And even if precise data are not available, a conservative, approximate number could be used for developing a sampling strategy. This would enable agencies and the public to have greater confidence in negative test results than was associated with the sampling strategy used in 2001. Activity 2: Collecting Samples: The agencies used a variety of sample collection methods. USPS primarily used the dry swab method. CDC and EPA used premoistened and dry sterile, synthetic (noncotton) swabs, wet synthetic wipes, and HEPA vacuums for sampling. To determine whether anthrax was airborne, CDC performed air sampling in the Brentwood facility 12 days after the contaminated letters were processed. Airborne anthrax spores pose a health risk because they can cause inhalational anthrax, the most serious form of the disease. Agency officials stated that laboratory requirements had influenced the choice of sample collection methods. For example, in the New York area, CDC used only dry swabs, following a requirement by New York public health laboratories. The majority of the samples were collected by the dry swab method, which experts and others we interviewed considered the least effective. Single methods were involved in 304 sampling events--that is, CDC and USPS collecting dry swab samples (185) and CDC and others collecting premoistened swabs (119).[Footnote 5] However, for some sampling events, CDC used wet wipes, HEPA vacuum, and air samples at Brentwood and swabs, wet wipes, and HEPA vacuum samples at Wallingford. USPS officials said that the choice of dry swabs was based on advice from CDC and an APHL working group, which had coordinated with the head of LRN. CDC stated that the reason for the use of swabs was an accommodation USPS had reached with APHL. According to APHL officials, the working group consulted with CDC's NCID in November 2001. APHL said that an NCID official, who was a member of the group, agreed that the dry synthetic swab method could be used but that premoistened swabs would pick up more spores. During our fieldwork, we tried to determine what specific advice CDC gave the Association of Public Health Laboratories (APHL) on using dry swabs. In responding to our inquiry, CDC did not specifically deny APHL's statement that an official from CDC's National Center for Infectious Diseases (NCID) told APHL that dry swabs could be used. However, an official from CDC's National Institute for Occupational Safety and Health (NIOSH), which was not a member of the working group, said that CDC has always recommended using premoistened swabs. Nevertheless, according to APHL, "the NIOSH recommendation was not known by the NCID working group members, nor did they advocate on its behalf." The decision to use dry rather than premoistened swabs stemmed partly from the concern of some public health officials, including APHL officials we interviewed, that moistened swabs would allow anthrax spores to germinate, growing into vegetative cells instead of remaining as spores. Other public health officials we interviewed said it was highly unlikely that anthrax spores would germinate into vegetative cells in a premoistened swab. APHL officials said that it was feared that such vegetative cells would be destroyed during certain analytic procedures. However, none of the agencies' collection methods were evaluated for anthrax detection in environmental samples. In the absence of empirical research, agencies had no information available for reliably choosing one method over another and no information on the limits of detection to use when evaluating negative results.[Footnote 6] Activity 3: Transporting Samples: Agencies transported samples by land or air to laboratories for extraction and analysis (activities 4 and 5). The USPS sample collection plan included shipping instructions that were based on regulations for shipping infectious substances and designed to prevent their inadvertent release. EPA's sample collection plan did not refer to transportation requirements. According to CDC's guidelines, anthrax samples were to be considered infectious substances and packaged according to applicable federal regulations enforced by the Department of Transportation. These regulations were aimed at "ensuring that the public and the workers in the transportation chain are protected from exposure to any agent that might be in the package."[Footnote 7] Among other potential requirements, infectious material must be contained in a securely sealed, pressure resistant, watertight, primary receptacle surrounded by an absorbent and cushioning material. This material must, in turn, be enclosed in a securely sealed, watertight, and durable secondary packaging, which has to be enclosed in an outer packaging constructed of fiberboard or equivalent material, as well as shock absorbent material if more than 50 milliliters are shipped in one package. However, these regulations did not address one of the most important issues--maintaining the biological integrity of samples while being transported. Failure to do so could result in false negative test results. For example, analysis by culture requires that spores can germinate, divide and multiply, so that tests can determine whether a sample contains anthrax. Temperature and exposure to certain kinds of light, such as ultraviolet light, can be deleterious to some microorganisms. Therefore, it is important that every sample collected retain its original physical form before and during transportation. We did not attempt to ascertain (1) the specific transit times for delivering all the samples to laboratories, (2) whether sample transportation was delayed, and (3) if it was, how long it was delayed. We also did not attempt to ascertain the environmental conditions the samples were shipped under or when they were received at the laboratories. Finally, we did not attempt to ascertain the degree to which spores could have been exposed to varying environmental conditions from the time of release to the time of sample collection, which could have affected sample integrity. Anthrax spores are robust, compared with other pathogenic microorganisms, but whether transportation affected their viability cannot be known because the conditions of their transportation were not validated. Transport conditions, once validated, would have to be standardized to ensure reproducibility. Activity 4: Extracting Samples: LRN protocols required that sample material be extracted with specific extraction procedures and fluids (such as sterile saline or water) and that the extracted fluid be subjected to specific analytic methods. For the samples USPS collected under the APHL agreement, the extraction methods included adding a sample processing solution to the conical tubes containing the dry swabs before "plating." This process was adapted from LRN protocols for extracting swabs. However, the private laboratory (not part of LRN) that originally analyzed the samples for USPS did not use an extraction fluid; it inoculated the noncotton, rayon-tipped dry swab directly onto a culture plate. Several factors could have affected extraction efficiency. For example, according to public health officials and other experts, the degree to which swabs or wipes can retain spores depends on the material they are made of. Cotton is more retentive than some artificial fibers like rayon and may be more difficult for extraction of spores for analysis. Other factors affecting spore extraction are the physical nature of the collection device and surface properties. For example, swabs are easier to manipulate and immerse in extract fluid than more bulky wipes are. CDC has acknowledged that "the recovery efficiency of the analytical methods has not been adequately evaluated." The reproducibility of the results when an extraction fluid is used can also be an issue. For example, a U. S. Army Medical Research Institute for Infectious Diseases (USAMRIID) official we interviewed told us of an unpublished USAMRIID study conducted to determine the efficiency of extracting anthrax from swabs; the study showed that even if the same procedure was followed, the results were not always the same.[Footnote 8] Although the importance of reproducibility has been recognized, definitive scientific information regarding extraction efficiency is lacking. In its absence, it is not clear whether sampling results were affected, particularly with respect to samples that may have contained few spores. Without knowing the extraction efficiency, a false negative result may potentially be seen as a true negative. Activity 5: Analyzing Samples: Analyzing the samples involved a variety of methods and required two steps--preliminary and confirmatory--to generate a final result. The laboratory analytic methods that were used for detecting anthrax in clinical samples already existed, but they had not been used for environmental samples. As a result, different analytic approaches were taken at the preliminary step, involving adaptations of such protocols. Samples deemed positive at the preliminary step were not always confirmed as positive, as was to be expected. However, this could cause problems for the agencies. In addition, some agencies considered preliminary analyses by field-based instruments unreliable, while others maintained that they were reliable but had been used inappropriately. However, once sample extracts were subjected to the required confirmatory tests, a positive result was indeed a positive. In analyzing the postal samples, laboratories used a variety of methods for preliminary and confirmatory testing. Preliminary tests included colony morphology, Gram's stain, hemolysis, and motility tests.[Footnote 9] Any culture isolates that could not be ruled out in the preliminary step of testing were considered presumptively positive and referred for confirmatory testing. Confirmatory tests included culture analyses (traditional microbiological and biochemical analyses), gamma phage lysis (a test that identifies the susceptibility of the organism to anthrax-specific viruses that create a kill zone in anthrax cultures), and direct fluorescent antibody assay, or antibody analyses employing a two-component test that detects the cell wall and capsule, or outer covering, produced by vegetative cells of anthrax. Other specialized tests, such as molecular subtyping, were also conducted to determine what strain of anthrax was involved. The test results were reported as positive--anthrax was found--or negative-- anthrax was not found. Traditional microbiological analyses require 18 to 24 hours before a result can be generated, depending on the laboratory protocols and procedures. In a few instances, results were also reported as number of colony forming units (CFU) per gram of sample material. According to CDC guidelines, LRN laboratories were to analyze samples by appropriate LRN protocols. According to CDC, all LRN laboratories were qualified to perform the preliminary tests, and most could perform confirmatory and other specialized tests. While a lower level of LRN laboratory could analyze swab samples for preliminary testing, all other samples--such as bulk, wipes, air samples, or vacuum samples-- were to be analyzed at a higher level of LRN laboratory. Samples could also be analyzed at CDC laboratories. Presumptive positives found at a lower level LRN laboratory had to be referred to an appropriately qualified laboratory for confirmatory testing. The problems agencies encountered in preliminary testing included issues related to training and quality control, as well as problems with using field-based analytic methods with limitations that were not well understood. In preliminary testing, a suspect organism must first be selected; at this point, human error or quality control issues can affect the results. For example, we identified a problem involving culture in the preliminary tests--that is, a reliance on the naked human eye to identify and select the growth of anthrax on the petri dish. Many different types of organisms could be growing that looked like, but were not, anthrax. This is significant because when negative results were obtained during preliminary testing, no further testing was to be done. The agencies were also faced with problems when deciding how to respond to preliminary positive results that might eventually turn out to be confirmed otherwise. For example, agencies did not have clear criteria for when to close facilities. In addition, although hand-held assays (HHA) were considered preliminary tests, concerns were raised that the negative results might lead to a false sense of security.[Footnote 10] During the 2001 incidents, USPS kept the Brentwood facility open, following CDC's advice that closing it was not warranted. According to USPS officials, the correctness of this advice appeared to be confirmed by the HHA results obtained on October 18, 2001. When CDC confirmed a case of inhalation anthrax in a Brentwood employee on October 21, 2001, the facility was closed that day. According to USPS, it was not until October 22, 2001, that the laboratory's culture tests of the other samples, collected on October 18, revealed positive results. In a more recent instance, on November 6, 2003, USPS shut down 11 postal facilities in and around Washington, D.C., after a preliminary test-- not a confirmed result--from a routine air sample taken on November 5 indicated that a naval mail processing facility might be contaminated with anthrax. USPS tracked the flow of mail through its own facilities and closed 11 postal facilities that delivered mail to the naval facility. The subsequent confirmatory tests were negative, and the facilities were reopened about 3 days later. All the activities discussed above are interdependent, and many variables for each one can affect the results. Further, problems associated with any one of these activities could affect the validity of the results generated by the overall process. Given that there are so many variables, the use of different sample collection strategies, reflected in site-specific plans, could yield different results. For example, three potential sample collection plans could be used in one facility--plan A, using one collection method (for example, a swab); plan B, using two methods (for example, a swab and wipe); and plan C, using three methods (for example, swab, wipe, and HEPA vacuum). How these collection methods are to be applied--that is, how they are physically used and how much area each sample covers--is a variable. Within each plan, sample transportation protocols could differ, involving variables such as temperature--plans A and B might require transporting at ambient temperature, while plan C might require freezing temperature--the sample collection method's moistness during transport, and the size and construction of the packaging. In addition, within each plan, laboratory extraction and analysis protocols could differ, involving variables such as (1) different manufacturers' different formulations of extraction fluids, (2) different ways to physically release spores from a particular collection method (such as a swab) into the liquid extract (such as by shaking or vortexing), and (3) a combination of analytic methods, such as culture or polymerase chain reaction (PCR) for deoxyribonucleic acid (DNA) amplification to identify anthrax. Any problems experienced with any of these variables across any of these plans could affect the final result. The Sampling Results Were Largely Negative: The results of the CDC, EPA, and USPS testing in 286 postal facilities were largely negative. Of 286 facilities, 23 tested positive. For 2 of these 23 facilities, test results were negative at first but positive on a subsequent testing. However, in 1 of these facilities--the Wallingford, Connecticut, facility--it was not until the fourth testing that positive results were obtained. Testing results differed between the primary facilities and Wallingford. First, in the three primary facilities, results were positive each time a facility was tested, with the important exception of the two quick tests in Brentwood. In Wallingford, considered less likely to be contaminated, results were positive only on the fourth sampling. Second, in the primary facilities, sampling with a single method produced some positive results, regardless of the sample collection method. In Wallingford, neither dry nor premoistened swabs produced any positive results. Third, in the primary facilities, both single and multiple methods produced positive results; in Wallingford, only multiple methods produced positive results. When comparing the positive results, obtained with dry swabs, across the primary facilities, the proportions differed. For example, in one sampling event in Brentwood, out of 29 samples collected using dry swabs, 14 were positive (48 percent), whereas in Morgan, out of 56, only 7 were positive (13 percent). In addition, for the West Palm Beach, Florida, facility, sampled several times during one sampling event, out of 38 dry swab samples collected, only 1 was positive (about 3 percent). While we did not define this facility as primary, it was suspected of processing a contaminated letter, although none was found. However, the use of both wet and dry swabs produced positive results in this facility. USPS and CDC sampled facilities that processed mail from the primary facilities to determine whether any other facilities had become contaminated. The majority of test results from these facilities were negative: Of 286 facilities sampled, 23 tested positive, including the 3 primary facilities, and 263 tested negative. For some of the positive facilities, excluding the primary ones: * Generally, only 1 or 2 of the total samples collected for each facility were positive, such as several post offices that received mail from Brentwood, including Dulles (11 samples collected, 1 positive), Friendship Station (32, 1 positive), Pentagon Station (17, 2 positive), and Raleigh, North Carolina (42, 1 positive). These facilities were considered cross-contaminated. * West Palm Beach and Wallingford tested positive only on retesting, whereas initially they had tested negative. The West Palm Beach facility tested positive on the second testing. According to CDC, the sampling strategy used in this facility was found to have limitations and was not used again. However, Wallingford did not test positive until the fourth testing. These results underscore the importance of retesting and cast doubt on the efficiency of the testing. Of the 263 facilities that tested negative, only 9 were sampled more than once. A facility in West Trenton tested negative, even though an employee had contracted cutaneous anthrax. The facility in West Trenton was tested twice by the FBI and once by CDC, during which a total of 57 samples were collected, with negative results. Final, or confirmed, results will be negative if contamination is not present in a facility. However, a result can be negative for several other reasons, such as (1) the sampling method was not efficient enough, (2) samples were not collected from places where contamination was present, (3) not enough samples were collected, (4) not enough spores were recovered from the sample material, or (5) analysis of the sample extract was not sensitive enough to detect anthrax spores that were present (that is, the result was a false negative). Agencies' Activities Were Not Validated: None of the agencies' activities to detect anthrax contamination in the postal facilities were validated. Without validation, the sampling activities could have been based on false assumptions. Using an ineffective method or procedure could result in a finding of no contamination when in fact there is contamination--a false negative. Because the sampling methods are not validated, it is not known to what extent they will underestimate contamination. Thus, in the case of a negative result, agencies would have no sound basis for taking public health measures for the occupants of the contaminated facility. Validation, as it is generally understood, is a formal, empirical process in which the overall performance characteristics of a given method are determined and certified by a validating authority as (1) meeting the requirements for the intended application and (2) conforming with applicable standards. Because the agencies did not use an empirical process to validate their testing methods, the agencies had limited information available for reliably choosing one method over another and no information on the detection limit to use when evaluating negative results. Validating the overall process is important because operational and health-related decisions are made on the basis of testing results generated by that process. In addition, validation would offer assurance that the results of using a particular method, which is part of that process, are robust enough to be reproduced, regardless of which agency, contractor, or laboratory is involved. Thus, agencies and the public could be reasonably confident that any test results generated by a process that includes that method would be reliable and, in particular, that any negative results would mean that a sample was free from contamination (within the method's limits of detection). In preparing for future incidents, the agencies have (1) made some changes based on what has been learned about some of the limitations of their sampling strategies, (2) made some revisions to their guidelines to reflect some of this knowledge and experience or developed new ones, (3) funded some new research, and (4) planned or conducted conferences addressing some of the issues we have identified. In addition, DHS has taken on the role of coordinating agencies' activities and has undertaken several new initiatives related to dealing with anthrax and other biothreat agents. However, while the actions DHS and other agencies have taken are important, they do not address the issue of validating all activities related to sampling. Since the fall of 2001, studies have been performed, or are under way, that may contribute to the validation of the individual activities. Nonetheless, these studies address only some aspects of an individual activity rather than the overall process. Finally, the agencies have not made appropriate and prioritized investments to develop and validate all activities related to anthrax and other biothreat agents. Conclusions: The lack of validated methods for assessing contamination in postal facilities impeded the agencies in responding to the incidents. The need that all methods, from sampling to final analysis, be validated, so that their performance characteristics can be clearly understood, is not in doubt. But any combination of methods that makes up the overall process should also be validated because the effect of different permutations of methods may not be predictable. It must be recognized, however, that an inability to validate the entire process reduces, to some degree, the level of confidence in the results. To assess the impact of relying on the validation of individual activities, experiments could be performed with a limited number of processes, combining different methods. The issues we have raised in this report apply to any anthrax incident, including the March 2005 incident involving DOD facilities in the Washington, D.C. area. In addition, while the 2001 events involved anthrax, many other biothreat agents exist. Differences in their characteristics mean different solutions. Accordingly, efforts to develop sampling strategies and to validate methods should address requirements specific to those biothreat agents as well. However, since addressing other agents would consume resources and time, these efforts should be prioritized in a long-term strategy. The several agencies that dealt with the anthrax attacks generally worked well together, but we have identified areas that would have benefited from one agency's taking the lead in coordinating the response. Given the mission of DHS and its responsibilities, it appears that DHS is now well positioned to take a lead role in promoting and coordinating the activities of the various agencies that have technical expertise related to environmental testing. In addition, it is important that all participating agencies recognize and support DHS in that role and that they have an effective structure for participating in identifying and addressing the appropriate issues. Recommendations for Executive Action: Accordingly, in our report, we recommended that to improve the overall process for detecting anthrax and to increase confidence in negative test results generated by that process, the Secretary of Homeland Security develop a coordinated approach. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should (1) take a lead role in promoting and coordinating the activities of the various agencies with technical expertise related to environmental testing; (2) ensure that a definition of validation is developed and agreed on; (3) guarantee that the overall process of sampling activities, including methods, is validated so that performance characteristics, including limitations, are clearly understood and results can be correctly interpreted; (4) see that appropriate investments are made in empirical studies to develop probability-based sampling strategies that take into account the complexities of indoor environments; (5) ensure that appropriate, prioritized investments are made for all biothreat agents; and (6) ensure that agency policies, procedures, and guidelines reflect the results of such efforts. We obtained written comments on a draft of this report from CDC, DHS, and USPS. We also obtained written comments from APHL on excerpts from the draft that pertained to its role in anthrax testing. Although we requested comments from DOD and EPA, DOD said it had no comments and EPA provided only technical comments. CDC, DHS, and USPS, as well as APHL, agreed with our conclusion-- methods for detecting anthrax contamination in facilities were not validated--and with the thrust of our recommendations--calling for a coordinated, systematic effort to validate the methods to be used for such testing. In response, DHS stated that while it has the overall responsibility for coordination for future biological attacks, EPA has "the primary responsibility of establishing the strategies, guidelines, and plans for the recovery from a biological attack while HHS has the lead role for any related public health response and guidelines." DHS further stated that EPA "is developing specific standards, protocols, and capabilities to address the risks of contamination following a biological weapons attack and developing strategies, guidelines, and plans for decontamination of persons, equipment, and facilities." DHS pointed out that in the Conference Report on H.R. 4818, the conferees expressed their expectation that EPA will enter into a comprehensive MOU [memorandum of understanding] with DHS no later than August 1, 2005 that will define the relationship and responsibilities of these entities with regard to the protection and security of our Nation. The Conferees expect the MOU to specifically identify areas of responsibilities and the potential costs (including which entity pays, in whole or part) for fully meeting such responsibilities. EPA shall [is to] submit to the House and Senate Committees on Appropriations a plan no later than September 15, 2005 that details how the agency will meet its responsibilities under the MOU, including a staffing plan and budget. Finally, DHS stated, "Even though DHS is in charge during a biological attack, EPA is primarily responsible for the coordination of the recovery process. So, DHS will coordinate with EPA to ensure appropriate investments are made to explore improved sampling." With respect to our recommendation that DHS develop probability-based sampling strategies, DHS said that it must first define the necessary requirements for the sampling process and then evaluate targeted and probability-based sampling strategies against those requirements. DHS said that targeted sampling may be beneficial for some applications. We agree with DHS on the need to define the requirements for the sampling process and to evaluate sampling approaches against those requirements. On the basis of the work we have done on this review, we believe that (1) DHS will find that targeted sampling will not always meet all the requirements to answer the question of whether a facility is contaminated and (2) probability-based sampling will be necessary when information on the source and path of potential contamination is not definitive. In our view, probability sampling will be necessary in order for DHS to achieve its goal of having a "scientifically defensible sampling strategy and plan." Mr. Chariman, this concludes my prepared statement. I will be happy to answer any questions you or Members of the Subcommittee may have. Contacts and Acknowledgments: If you or your staff have any questions about this report or would like additional information, please contact me at (202) 512-6412, or Sushil Sharma, PhD., DrPH, at (202) 512-3460. We can also be reached by e-mail at firstname.lastname@example.org and email@example.com. Other staff that contributed to this report include Hazel Bailey, Heather Balent, Venkareddy Chennareddy, Crystal Jones, Jack Melling, Penny Pickett, Laurel Rabin, Mark Ramage, and Bernard Ungar. [End of section] Appendix I: Scope and Methodology: To respond to your request, we interviewed officials from federal agencies involved in sampling the postal facilities. The federal agencies included the Centers for Disease Control and Prevention (CDC), the Environmental Protection Agency (EPA). We also interviewed U.S. Postal Service (USPS), Association of Public Health Laboratories (APHL), public health and private sector laboratories, and experts on microbial detection in indoor environments. We reviewed documentation provided or developed by CDC, EPA, and USPS, including sample collection strategies, guidance, environmental collection and analytical methods and protocols. In addition, we reviewed and analyzed test results data, that is, sample collection and analytical data collected by federal agencies, their contractors, and public health laboratories. We did not independently verify these data. We conducted site visits to some postal facilities affected by anthrax and some public health and private sector laboratories that were involved in analyzing samples. We also reviewed studies on sampling methods for detecting biological substances, including anthrax, on surfaces and in the air. We conducted our review from May 2003 through November 2004 in accordance with generally accepted government auditing standards. Although our study focused on anthrax testing relating to 2001 anthrax incident, we believe that the issue we identified concerning the need for validated methods and sound sampling strategies would apply to similar incidents in future. This is particularly evident given the consequences arising from the March 2005 incident involving facility closures following preliminary anthrax testing in the Washington, D.C. area. FOOTNOTES  GAO, Anthrax Detection: Agencies Need to Validate Sampling Activities in Order to Increase Confidence in Negative Results, GAO-05- 251 (Washington, D.C.: March 31, 2005). www.gao.gov.  "Anthrax" in this testimony reflects commonly used terminology. Technically, the term refers only to the disease caused by the microorganism Bacillus anthracis, not the bacterium itself or its spores.  Anthrax contamination had been found earlier in several Florida postal facilities that processed mail for the American Media Incorporated building there. However, no letter containing anthrax was ever found.  According to the head of the Postal Inspection Service, more than 7,000 hoaxes, threats, and suspicious letters and packages--an average of almost 600 a day--were reported to his agency in the weeks following the first anthrax incident. As a result, nearly 300 postal facilities had to be evacuated.  We use "sampling event" to refer to initial sample collection by a specific agency on a specific day and at a specific time in a specific facility. Multiple agencies collected samples on the same day in some of the same facilities; therefore, each agency's sample collection is considered a separate sampling event. As a result, there were more sampling events than the total number of facilities sampled.  The published literature provided some information on the efficiency of a few sample collection methods. In all the methods studied, swabs were always premoistened before samples were collected. However, according to one study, the most efficient method caused problems when used with certain analytic methods.  Department of Transportation, 49 C.F.R. subchapter C--Hazardous Materials Regulation. The USPS regulations mirror the Department of Transportation regulations. However, to be transported as mail, material must be classified as mailable. By statute, infectious materials, such as anthrax spores, that are "disease germs or scabs, [or] other natural or artificial articles, compositions, or material which may kill or injure another" cannot be mailed. Such materials are termed "nonmailable matter." Knowingly mailing such material is a criminal offense, and doing so with the intent to kill or injure is a felony. When an etiologic material is not "outwardly or of [its] own force dangerous or injurious to life, health, or property," USPS may allow it to be mailed, subject to appropriate rules and regulations governing its preparation and packing. As a result, USPS allows the mailing of small quantities of appropriately packaged infectious material, but only if it is intended for medical or veterinary use, research, or laboratory certification related to public health.  Using synthetic swabs and a particular type of buffer could lead to 70 to 75 percent extraction. However, repeating the test with the same type of buffer made by different companies yielded different results. The official said that this test showed that there were too many variables. Even when analysts followed the same procedure, the results were not always reproducible, casting doubt on the reliability of the test results.  When bacteria stained with Gram's stain retained the color of the primary stain (crystal violet), they were considered gram-positive, a characteristic of anthrax. Hemolysis, a procedure involving culturing, identified whether the colonies gave no evidence of red blood cell lysis, a characteristic of anthrax. Motility refers to whether the colonies showed no movement in microscopic observation, another characteristic of anthrax.  See GAO, U.S. Postal Service: Better Guidance Is Needed to Ensure an Appropriate Response to Anthrax Contamination, GAO-04-239 (Washington D.C.: Sept. 9, 2004).