This is the accessible text file for GAO report number GAO-10-798 entitled 'New Drug Approval: FDA's Consideration of Evidence from Certain Clinical Trials' which was released on August 30, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: July 2010: New Drug Approval: FDA's Consideration of Evidence from Certain Clinical Trials: GAO-10-798: GAO Highlights: Highlights of GAO-10-798, a report to congressional requesters. Why GAO Did This Study: Before approving a new drug, the Food and Drug Administration (FDA)—an agency of the Department of Health and Human Services (HHS)—assesses a drug’s effectiveness. To do so, it examines information contained in a new drug application (NDA), including data from clinical trials in humans. Several types of trials may be used to gather this evidence. For example, superiority trials may show that a new drug is more effective than an active control—a drug known to be effective. Non- inferiority trials aim to demonstrate that the difference between the effectiveness of a new drug and an active control is small—small enough to show that the new drug is also effective. Drugs approved on this basis may provide important benefits, such as improved safety. Because non-inferiority trials are difficult to design and interpret, they have received attention within the research community and FDA. FDA has issued guidance on these trials. GAO was asked to examine FDA’ s use of non-inferiority trial evidence. This report (1) identifies NDAs for new molecular entities—potentially innovative new drugs not FDA-approved in any form—that included evidence from non-inferiority trials, (2) examines the characteristics of these trials, and (3) describes FDA’s guidance on these trials. GAO reviewed NDAs submitted to FDA between fiscal year 2002 (the first full year that FDA documentation was available electronically) and fiscal year 2009 (the last full year of submissions), examined FDA’s guidance, and interviewed agency officials. What GAO Found: Evidence from non-inferiority trials was included in about one- quarter, or 43, of the 175 NDAs for new molecular entities that were submitted to FDA for review from fiscal years 2002 through 2009. Many of these applications were for antimicrobial drugs, such as those treating bacterial, viral, and fungal infections. As of December 31, 2009, FDA approved 18 of the 43 NDAs on the basis of evidence from non- inferiority trials. Of the remaining 25 NDAs, FDA approved 11 based on other evidence, such as proof that the new drug was more effective than a placebo (no treatment), and decided not to approve 14. The non-inferiority trials included in these NDAs varied with respect to their characteristics. FDA generally requires sponsors to provide evidence of a drug’s effectiveness as shown in more than one trial. For the 18 NDAs that were approved based on evidence from non- inferiority trials, the number of non-inferiority trials used to provide primary support for approval ranged from one to four, with an average of 2 such trials per NDA. Half of these applications included non-inferiority trials that tested the effectiveness of the new drug against more than one active control. The non-inferiority margins—the maximum clinically acceptable extent to which the new drug can be less effective than the active control and still show evidence of an effect— ranged from 5 to 20 percent among trials that supported approval. Among the other 25, FDA identified nine NDAs that included poorly designed non-inferiority trials which did not provide primary evidence for approval. Some of these problems included an inappropriate selection of an active control and an improper calculation of a non- inferiority margin. FDA notified sponsors of its concerns with the poorly designed trials prior to the sponsors’ submissions of all NDAs that included such trials. In March 2010 FDA issued draft guidance which focused solely on the use of non-inferiority trials. This guidance presents detailed and comprehensive recommendations on how non-inferiority trials may be used to provide evidence of a drug’s effectiveness. For example, it provides advice on how to select an active control and how to set the non-inferiority margin, as well as how to interpret the trials. This guidance offers broad, generally applicable recommendations to supplement indication-specific guidance documents that FDA had previously issued. These indication-specific guidance documents include FDA’s advice on many issues related to the development of drugs for particular indications, some of which are related to the use of non-inferiority trials. GAO’s review of FDA’s guidance showed that the agency has become more conservative in allowing evidence from non- inferiority trials to demonstrate a drug’s effectiveness. First, FDA has limited the indications for which these trials may be used. Second, the agency has also become more rigorous in its review of evidence from non-inferiority trials. We sent a draft of this report to HHS for review. HHS provided us with technical comments, which we incorporated as appropriate. View [hyperlink, http://www.gao.gov/products/GAO-10-798] or key components. For more information, contact Marcia Crosse at (202) 512- 7114 or crossem@gao.gov. [End of section] Contents: Letter: Background: One-Quarter of NDAs Included Evidence from Non-Inferiority Trials and FDA Approved a Majority of These Applications: Non-Inferiority Trial Characteristics Varied and Not All Such Trials Provided Primary Evidence for the Approval of NDAs: FDA Has Issued Detailed and Comprehensive Guidance on the Use of Non- Inferiority Trials to Establish the Effectiveness of New Drugs: Agency Comments: Appendix I: New Drug Applications Approved on the Basis of Evidence from Non-Inferiority Trials: Appendix II: FDA Guidance Documents Issued with Information on the Use of Non-Inferiority Trials: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: NDAs Submitted from Fiscal Years 2002 through 2009 with Evidence from at Least One Non-Inferiority Trial, by Drug Type: Table 2: Characteristics of Non-Inferiority Trials for 18 NDAs Submitted for FDA Review from Fiscal Years 2002 through 2009 and Approved on the Basis of Primary Evidence from Non-Inferiority Trials: Table 3: Summary of FDA Guidance Regarding the Use of Non-Inferiority Trials for Particular Indications: Table 4: NDAs Submitted for FDA Review from Fiscal Years 2002 through 2009 Where the Basis of Approval Included Primary Evidence from Non- Inferiority Trials: Table 5: FDA-Issued Guidance Including Recommendations on the Use of Non-Inferiority Trials, January 2002 through June 2010: Figures: Figure 1: Examples of Clinical Trials Demonstrating Non-Inferiority Compared to Those Showing Inferiority and Superiority: Figure 2: NDAs Submitted from Fiscal Years 2002 through 2009 with Evidence from at Least One Non-Inferiority Trial: Figure 3: Approval Status of 43 NDAs Submitted from Fiscal Years 2002 through 2009 with Evidence from Non-Inferiority Trials: Abbreviations: FDA: Food and Drug Administration: HHS: Department of Health and Human Services: HIV: human immunodeficiency virus: NDA: new drug application: [End of section] United States Government Accountability Office: Washington, DC 20548: July 30, 2010: Congressional Requesters: Testing new drugs on human volunteers is an essential step in the drug development process. These tests, known as clinical trials, are instrumental in determining whether a drug is safe and effective. Their purpose is to measure the effect of a new drug separately from other influences, such as a spontaneous change in the course of a disease. Before a new drug can be marketed in the United States, the drug's sponsor--typically a manufacturer--must submit a new drug application (NDA) to the Food and Drug Administration (FDA) for approval. The agency will only approve the NDA if it determines that the drug is safe and effective for its intended use. To make this determination, FDA reviews the results of clinical trials that sponsors submit as part of their NDAs. Sponsors must provide evidence of a drug's effectiveness based on adequate and well-controlled trials.[Footnote 1] Several different types of trials may be used to gather this evidence. Some clinical trials are designed to test whether a new drug is more effective than a placebo or no treatment at all. However, in some instances it would be unethical to offer a placebo or withhold treatment because it would deprive human volunteers of an available treatment known to prevent death, irreversible injury, or other serious harm. In such instances, a clinical trial may be conducted to measure the effect of a new drug compared to an active control--a drug already known to be effective. For example, a superiority trial could be conducted to show that a new drug is more effective than an active control. Another type of clinical trial, a non-inferiority trial, is intended to establish that the difference in the effectiveness of a new drug and an active control is small. Non-inferiority trials that demonstrate a small difference in the effectiveness of the two drugs may be able to support a conclusion that the new drug is effective because the effectiveness of the active control is known. FDA considers the use of non-inferiority trials to be acceptable in certain circumstances and has approved drugs on the basis of evidence from such trials. Although non-inferiority trials may prove that new drugs are no more effective than the active controls they are compared to, new drugs approved on the basis of non-inferiority may provide patients with other important benefits, such as improved safety, fewer drug-to-drug interactions, convenience of administration, or a lower cost. However, non-inferiority trials are more complicated to design and their results are more difficult to interpret than other types of clinical trials. For example, in interpreting the results of non- inferiority trials, sponsors not only must examine the difference in the effectiveness of the two drugs as measured in the non-inferiority trial, they must also assess whether the active control proved to be as effective as expected. If the active control did not demonstrate its expected effect in the trial, results showing the similarity of the two drugs are meaningless. As a result of these and other issues, non-inferiority trials have received attention within the research community and FDA. To assist sponsors in appropriately using non- inferiority trials to establish a drug's effectiveness, FDA has issued written guidance conveying the agency's understanding of non- inferiority trials and how such trials may be used to support a drug's approval. You asked us to review FDA's use of evidence from non-inferiority trials to establish a drug's effectiveness and support approval. In this report, we (1) identify the type and status of drug applications submitted for FDA review that included evidence from non-inferiority trials; (2) examine the characteristics of non-inferiority trials FDA considered in making approval decisions; and (3) describe FDA's guidance for establishing a drug's effectiveness on the basis of non- inferiority trials. To identify the type and status of drug applications submitted to FDA for review that included evidence from non-inferiority trials, we limited our scope to a subset of drugs. FDA officials told us that the majority of non-inferiority trials for drugs listed in NDAs were for new molecular entities--potentially innovative drugs containing active ingredients that have never been approved for marketing in the United States in any form. We therefore examined FDA data on the 223 NDAs for new molecular entities that were submitted to FDA during the last 8 years--fiscal years 2002 through 2009 (October 1, 2001, through September 30, 2009).[Footnote 2] Fiscal year 2002 was the first full year that records documenting FDA's review of NDAs were available in electronic format, and so we did not include prior years in our scope. Fiscal year 2009 was the last full year for which data was available at the time we requested data from FDA and so we excluded NDAs submitted to FDA after that period. We limited our scope to prescription drugs intended to prevent or treat diseases or other medical conditions. We excluded 18 NDAs for nonprescription drugs, drugs used to aid in diagnosing diseases, or drugs aiding in the absorption of other drugs. We further limited our scope to those NDAs for which FDA had completed its review by December 31, 2009--excluding 30 NDAs that were either withdrawn or pending review as of that date. These two limitations excluded 48 of the 223 NDAs from our review and we conducted our analysis on the remaining 175 NDAs for new molecular entities.[Footnote 3] As our analysis was limited to a subset of all NDAs received by FDA, the results of our review may not be generalizable to other types of applications. To determine whether NDAs within our scope included evidence from non- inferiority trials, we examined documents summarizing the results of clinical trials that we obtained from FDA's Web site[Footnote 4] (for approved NDAs) and from agency officials (for NDAs not approved as of December 31, 2009). Specifically, we reviewed FDA's statistical and medical reviews to determine whether the application included a non- inferiority trial. For those NDAs with at least one non-inferiority trial, we determined whether FDA considered data from these trials as pivotal--that is, providing the primary evidence of effectiveness to support the NDA.[Footnote 5] Because FDA could not readily identify the NDAs that it had received that included evidence from non- inferiority trials, we submitted the results of our analysis to FDA. We asked agency officials to confirm that we had correctly identified the NDAs that included evidence from non-inferiority trials--including those that were approved primarily based on this evidence, approved based on other primary evidence, or that had not been approved as of December 31, 2009. To examine the characteristics of non-inferiority trials FDA considered in making approval decisions, we analyzed FDA's statistical and medical reviews for those drugs that included evidence from at least one non-inferiority trial in their NDAs. For those drugs FDA ultimately approved based on evidence from non-inferiority trials, we gathered descriptive information about the non-inferiority trials that FDA considered. In particular, we determined the number of non- inferiority trials conducted to support each application, identified the active controls used in these trials, and reviewed whether and how each trial supported the conclusion that the drug was effective. We also reviewed whether the characteristics of the non-inferiority trials providing primary evidence for approval revealed evidence of biocreep. Biocreep refers to a concern that successive generations of drugs approved based on non-inferiority trials, with the active control changing in each new generation, could lead to the adoption of decreasingly effective drugs, culminating in the approval of drugs that are no more effective than a placebo. We examined how FDA assessed the effectiveness of the active controls used in non- inferiority trials to ensure that comparisons to these drugs were appropriate--particularly where successive generations of non- inferiority trials occurred. We also reviewed FDA's correspondence with sponsors about their non-inferiority trials to analyze the information FDA communicated with sponsors before, during, and after their clinical trials. We gathered information on the concerns FDA identified and communicated with sponsors regarding their non- inferiority trials and determined when FDA notified sponsors about these issues. To describe FDA's guidance for establishing drug effectiveness on the basis of non-inferiority trials, we reviewed guidance FDA issued from January 2002 through June 2010 that was related to the design of such trials. From these published guidance documents we gathered FDA's recommendations on how to use non-inferiority trials to support the effectiveness of new drugs. We interviewed FDA officials to obtain contextual information regarding their issuance of this guidance. We also interviewed experts, such as biostatisticians and physician specialists, including those affiliated with the Infectious Diseases Society of America and the American Society of Clinical Oncology, to obtain their perspectives regarding the content and clarity of FDA's guidance. We did not assess the extent to which sponsors' non- inferiority trials or FDA's approval decisions on applications that included evidence from these types of trials comported with agency guidance. We conducted this performance audit from July 2009 to July 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: FDA, an agency within the Department of Health and Human Services (HHS), is responsible for overseeing the safety and effectiveness of drugs marketed within the United States. These responsibilities begin before a product is brought to the market, and include reviewing drug sponsors' proposals for conducting clinical trials, providing advice and publishing guidance regarding these trials, as well as reviewing applications for new drugs. The Use of Non-Inferiority Trials in Obtaining Evidence of a Drug's Effectiveness: Once a drug sponsor identifies a promising chemical compound it believes to be capable of curing or treating diseases, the sponsor may decide to conduct clinical trials on humans to gather the evidence necessary to demonstrate to FDA that the drug is safe and effective for its intended use. Before beginning clinical trials in the United States, a sponsor generally must submit an investigational new drug application to FDA for review.[Footnote 6] This application provides FDA with extensive information about the drug, including safety and manufacturing information, and outlines the sponsor's plans for clinical trials, which gradually introduce new drugs to increasingly larger numbers of patients.[Footnote 7] FDA assesses the information in the application--which is later included as part of the NDA--to ensure that the drug is reasonably safe to begin studying in humans. [Footnote 8] Sponsors may use these clinical trials to gather evidence of a drug's safety and effectiveness. In general, FDA requires sponsors to submit the results of more than one clinical trial demonstrating effectiveness in order to provide substantial evidence that a drug is effective for the intended indication and population.[Footnote 9] FDA has issued regulations and guidance that provide industry with information to properly design, conduct, and interpret these trials. For example, in 1985 FDA substantially revised its regulations including the provision addressing the characteristics of adequate and well-controlled trials and the types of controlled trials that can be used to gather evidence of a new drug's effectiveness.[Footnote 10] Sponsors may use trials of varying designs to obtain evidence of a drug's effectiveness. One type of clinical trial is a non-inferiority trial. The objective of a non-inferiority trial is to show that any difference in the effectiveness of two drugs is small enough to allow a conclusion that the new drug is also effective, but not substantially less effective than the active control. To conduct a non- inferiority trial, sponsors must make many decisions regarding how the trial will measure the new drug's effectiveness. For example, they must select the trial's primary endpoint, the principal measure used to determine a drug's effectiveness. The primary endpoint may be a clinical endpoint--a direct measure of how a patient feels, functions, or survives--or, in some cases, a surrogate endpoint--a laboratory measure or physical sign used as a substitute for a clinical endpoint that reasonably predicts a clinical benefit.[Footnote 11] Sponsors must also determine when to measure the trial's endpoint--for example, are patients cured within 7, 14, or 30 days after starting treatment-- in addition to determining the number and type of patients to be enrolled in each trial. Sponsors conducting non-inferiority trials must also make decisions to account for the new drug's comparison to the active control. Sponsors must identify an available treatment for use as an active control in the non-inferiority trial.[Footnote 12] They must then use evidence of the active control's effectiveness as shown in prior clinical trials to estimate the effect that the active control will have in the planned non-inferiority trial, adjusting for any differences between the prior and planned trials. Using this estimate, sponsors determine the trial's non-inferiority margin--the maximum clinically acceptable extent to which the new drug can be less effective than the active control and still show evidence of an effect. FDA considers the selection of a margin to be the single greatest challenge in designing, conducting, and interpreting non-inferiority trials. Its calculation is not only dependent on a string of other decisions related to the trial--for example, the data collected on the active control's effectiveness in other trials--but also includes the application of clinical judgment to determine the maximum amount of effectiveness that could be lost without having a substantial impact on the drug's effectiveness. If a non-inferiority margin is incorrectly calculated and is set too large, a drug that is not effective may appear to be effective; if the margin is too small, an effective drug may appear to be ineffective. In a non-inferiority trial, patients are randomly assigned to receive either the new drug or an active control. After the trial, the sponsor identifies the observed effect of each drug in the trial, and calculates the observed difference in the drugs' effectiveness. The actual difference in the drugs' effectiveness in the entire population could be greater or less than what is observed in the trial. For that reason, sponsors calculate a confidence interval around the observed difference in effectiveness between the new drug and active control drug. The confidence interval provides a range of values for the difference in effectiveness within which the true difference is likely to be found. The confidence interval around the observed difference in effectiveness is used to determine if the new drug is non-inferior to the active control. It is compared to the non-inferiority margin--the maximum clinically acceptable extent to which the new drug can be less effective than the active control. If the confidence interval is within the non-inferiority margin, and the sponsor provides adequate evidence that the active control demonstrated its expected effect in the trial, the new drug may be deemed non-inferior to the active control.[Footnote 13] A new drug can be non-inferior to an active control even if the estimated difference in effectiveness and its confidence interval lies entirely below zero, meaning that the active control drug is more effective than the new drug, but by an irrelevant amount. However, if the confidence interval shows that the effect of the drug could be below the margin--even if the observed effect of the drug was within the margin--the drug would not have shown an effect, and is therefore considered inferior. In addition, if the confidence interval lies entirely above zero--demonstrating that the new drug is more effective than the active control--the drug can be considered superior. (See figure 1.) Figure 1: Examples of Clinical Trials Demonstrating Non-Inferiority Compared to Those Showing Inferiority and Superiority: [Refer to PDF for image: illustration] The illustration depicts clinical trials with the following information: Difference in effectiveness (new drug effectiveness minus active control drug effectiveness); Confidence interval for the difference. The trials are shown plotted against the following plot line: Active control drug more effective; -X%; 0% difference in effectiveness; New drug more effective. Clinical trial A: Difference in effectiveness: Inferior. Clinical trial B: Difference in effectiveness: Inferior. Clinical trial C: Difference in effectiveness: Non-inferior. Clinical trial D: Difference in effectiveness: Non-inferior. Clinical trial E: Difference in effectiveness: Non-inferior. Clinical trial F: Difference in effectiveness: Superior. Clinical trial G: Difference in effectiveness: Superior. Source: GAO. [End of figure] Since issuing regulations that address the elements of adequate and well-controlled trials, FDA has also periodically issued guidance documents to provide updates on the agency's current thinking on a range of topics. These guidance documents encompass broad issues such as statistical principles for use in clinical trials and how to select an appropriate control, whereas others are more focused and serve to consolidate relevant recommendations on the development of drugs treating a particular indication. In addition to disseminating guidance on non-inferiority trials, FDA provides specific advice regarding the design of clinical trials at the request of sponsors. For example, sponsors may ask FDA to review and provide advice on a trial's proposed active control, non- inferiority margin, or endpoint before the given trial has begun. After the conclusion of their clinical trials, sponsors may consult with FDA regarding the interpretation of trial results or to discuss the information the agency would expect to see submitted in an NDA. FDA's advice and recommendations to sponsors are considered advisory; sponsors are not required to implement any of the agency's suggestions.[Footnote 14] If sponsors believe they have successfully demonstrated a new drug's safety and effectiveness, they may submit an NDA to FDA for review. The NDA contains information about the safety and effectiveness of the drug as demonstrated in clinical trials and other research, such as studies in animals. Once the agency receives an NDA, the application is reviewed by one of FDA's medical review divisions, depending on the indication the drug has been proposed to treat. If FDA determines that the drug is safe and effective for its intended use--that its clinical benefits outweigh its potential health risks--and that other requirements are met, it will approve the application. After approving a new drug, FDA's responsibilities continue as it is charged with monitoring the safety, effectiveness, and promotion of approved drugs. FDA executes these responsibilities in the same manner regardless of whether drugs were approved on the basis of evidence from non- inferiority trials. Issues Unique to Non-Inferiority Trials: Non-inferiority trials present unique issues in measuring the effectiveness of new drugs. For example, the use of these trials can raise uncertainties about the true effectiveness of new drugs because non-inferiority trials cannot measure this directly. Instead, these trials measure the effectiveness of the new drug relative to the active control, and sponsors must assess whether the active control can be considered to be as effective in the non-inferiority trial as was expected based on past experience. Using data from the non- inferiority trial and from prior trials measuring the effectiveness of the active control, the effectiveness of the new drug is estimated-- but not ever fully known. In addition, non-inferiority trials are more prone to certain biases than superiority trials. For example, if patients in a superiority trial do not take the new drug as directed, this poor compliance will dilute the measured effectiveness of the new drug, making it less likely that the trial will successfully demonstrate superiority. In a non-inferiority trial, however, poor compliance by patients taking the active control drug can have a different effect. It can reduce the difference in the measured effectiveness between the new drug and the active control, making the treatments appear more similar than they might otherwise be. As such, poor compliance in a non-inferiority trial can increase the likelihood that an ineffective drug is concluded to be effective. The use of non-inferiority trials over time also raises concerns about the potential for "biocreep" to occur. This term is used to describe the concern that successive generations of drugs approved based on non- inferiority trials, with the active control changing in each new generation, could lead to the adoption of decreasingly effective drugs and ultimately to the approval of drugs that are no more effective than a placebo. Non-inferiority trials that are poorly designed are especially prone to biocreep. The selection of inappropriate active controls--that is, drugs that are not known to be consistently effective, or drugs that were themselves approved on the basis of non- inferiority trials--could lead to biocreep. Even if successive generations of non-inferiority trials are conducted and each trial is itself well-designed, biocreep may still occur because placebo controls are not included in these trials. Non- inferiority trials are only able to measure the effectiveness of the new drug relative to the active control, not a placebo. As a result, the true effectiveness of any of the new drugs, compared to a placebo, is not measured. Without this metric, it is impossible to determine the extent to which the effectiveness of the new drug is similar to that of a placebo and whether biocreep has occurred. FDA has acknowledged some concerns over the uncertainties inherent in non-inferiority trials and the potential these trials create for biocreep. For example, FDA stated in a 1992 guidance document that, in order to avoid biocreep, sponsors should consult with the agency regarding the active controls they were considering for their trials.[Footnote 15] In other guidance documents, FDA has also encouraged sponsors to consult with the agency regarding their planned non-inferiority trials. One-Quarter of NDAs Included Evidence from Non-Inferiority Trials and FDA Approved a Majority of These Applications: One-quarter of NDAs submitted to FDA for review from fiscal years 2002 through 2009 included evidence from non-inferiority trials, and many of these applications were for antimicrobial drugs. FDA approved a majority of the applications that included evidence from these trials. One-Quarter of NDAs, Many for Antimicrobial Drugs, Included Evidence from Non-Inferiority Trials: Forty-three, or one-quarter, of the 175 NDAs we reviewed that were submitted to FDA from fiscal years 2002 through 2009 included evidence from at least one non-inferiority trial. The number of NDAs with evidence from non-inferiority trials varied from year to year and generally declined from fiscal years 2002 through 2009. On average, FDA received five NDAs each year that included evidence from non- inferiority trials. (See figure 2.) Figure 2: NDAs Submitted from Fiscal Years 2002 through 2009 with Evidence from at Least One Non-Inferiority Trial: [Refer to PDF for image: vertical bar graph and pie-chart] 175 NDAs: Applications including evidence from non-inferiority trials: 43; All other applications: 132. 43 NDAs Including Evidence from Non-Inferiority Trials: FDA received an average of 5 NDAs each year. Fiscal year of submission: 2002; Number of NDAs: 9. Fiscal year of submission: 2003; Number of NDAs: 9. Fiscal year of submission: 2004; Number of NDAs: 2. Fiscal year of submission: 2005; Number of NDAs: 6. Fiscal year of submission: 2006; Number of NDAs: 5. Fiscal year of submission: 2007; Number of NDAs: 4. Fiscal year of submission: 2008; Number of NDAs: 7. Fiscal year of submission: 2009; Number of NDAs: 1. Source: GAO analysis of FDA documents. Note: Our review was limited to NDAs for new molecular entities. [End of figure] About half of the 43 NDAs submitted with evidence from at least one non-inferiority trial--or 22--were for antimicrobial drugs,[Footnote 16] such as those that treat bacterial, viral, or fungal infections. The remaining portion of NDAs submitted with evidence from these trials represented a variety of drug types. (See table 1.) Table 1: NDAs Submitted from Fiscal Years 2002 through 2009 with Evidence from at Least One Non-Inferiority Trial, by Drug Type: Antimicrobial drugs: Drug Type: Antibacterial drugs; Number: 10; Percentage: 23%. Drug Type: Antiviral drugs; Number: 5; Percentage: 12%. Drug Type: Antifungal drugs; Number: 3; Percentage: 7%. Drug Type: Other antimicrobial drugs; Number: 4; Percentage: 9%. Antimicrobial drugs subtotal: Number: 22; Percentage: 51%. Other drug types: Drug Type: Metabolism and endocrinology drugs; Number: 5; Percentage: 12%. Drug Type: Ophthalmology drugs; Number: 3; Percentage: 7%. Drug Type: Analgesia drugs; Number: 2; Percentage: 5%. Drug Type: Hematology drugs; Number: 2; Percentage: 5%. Drug Type: Pulmonary drugs; Number: 2; Percentage: 5%. Drug Type: Cardiovascular and renal drugs; Number: 2; Percentage: 5%. Drug Type: Gastroenterology drugs; Number: 1; Percentage: 2%. Drug Type: Psychiatry drugs; Number: 1; Percentage: 2%. Drug Type: Reproductive drugs; Number: 1; Percentage: 2%. Drug Type: Rheumatology drugs; Number: 1; Percentage: 2%. Drug Type: Transplant drugs; Number: 1; Percentage: 2%. Other drug types subtotal: Number: 21; Percentage: 49%. Drug Type: Total; Number: 43; Percentage: 100%. Source: GAO analysis of FDA documents. Note: Our review was limited to NDAs for new molecular entities. [End of table] FDA Approved a Majority of NDAs That Included Evidence from Non- Inferiority Trials: FDA approved 29 of the 43 NDAs submitted for review from fiscal years 2002 through 2009 that included evidence from at least one non- inferiority trial. Most NDAs--18 of the 29--were approved based on evidence from pivotal non-inferiority trials.[Footnote 17] FDA approved the remaining 11 applications based on other evidence, such as the superiority of the new drug compared to a placebo or an active control. As of December 31, 2009, FDA had decided not to approve 14 applications that included evidence from non-inferiority trials. (See figure 3.) Figure 3: Approval Status of 43 NDAs Submitted from Fiscal Years 2002 through 2009 with Evidence from Non-Inferiority Trials: [Refer to PDF for image: pie-chart] Approved based on primary evidence from non-inferiority trials: 18; Approved based on other primary evidence: 11; Reviewed but not approved: 14. Source: GAO analysis of FDA documents. Notes: Our review was limited to NDAs for new molecular entities. Approval status is as of December 31, 2009. [End of figure] Many NDAs including evidence from non-inferiority trials were for antimicrobial drugs, and the majority of approvals based on this evidence were also for these types of drugs. Two-thirds, or 12 of the 18, NDAs approved on the basis of non-inferiority trials were for antimicrobial drugs.[Footnote 18] The remaining one-third of NDAs approved on the basis of non-inferiority trials were for various other types of drugs, including those treating diabetes and chemotherapy- induced nausea and vomiting. See appendix I for a list of all 18 NDAs approved based on evidence from non-inferiority trials, including fiscal year of approval, drug type, and approved indication. Non-Inferiority Trial Characteristics Varied and Not All Such Trials Provided Primary Evidence for the Approval of NDAs: Characteristics varied among the non-inferiority trials providing primary evidence to support FDA's approval of 18 NDAs. Some other applications also included non-inferiority trials that FDA identified as being poorly designed; these trials did not provide primary evidence for approval. Characteristics Varied Among the Non-Inferiority Trials That Provided Primary Evidence for Approval of 18 NDAs: Characteristics varied among the non-inferiority trials that provided primary evidence for the approval of the 18 NDAs. FDA relied on primary evidence from multiple pivotal non-inferiority trials to support the approval of most of these applications. The number of pivotal non-inferiority trials used as primary evidence for these 18 NDAs ranged from one to four, with an average of two pivotal non- inferiority trials supporting the approval of each application. In addition to including evidence from pivotal non-inferiority trials, five applications included evidence from other types of pivotal trials; for example, trials demonstrating superiority to a placebo or active control drug. Thirteen of the 18 applications included only pivotal non-inferiority trials in their applications. Of these applications, FDA approved four based on evidence from a single pivotal non-inferiority trial. Two-thirds, or 12, of the 18 NDAs included trials that measured drug effectiveness using a surrogate, rather than a clinical, primary endpoint in at least one of their pivotal trials.[Footnote 19] Although FDA generally prefers that drug sponsors demonstrate the effectiveness of a new drug by showing its impact on a clinical endpoint, in certain cases, it will consider a surrogate endpoint if it determines it is a reasonable substitute. However, all experts we interviewed who commented on this topic noted that the approval of drugs on the basis of both non-inferiority trials and surrogate endpoints increases uncertainty in the drugs' true effectiveness. Half of the 18 NDAs FDA approved on the basis of non-inferiority trials tested the effectiveness of the new drug against more than one active control. A majority of the active controls used in non- inferiority trials were FDA-approved for the indication. However, three applications included evidence from non-inferiority trials that used one active control that was not FDA-approved for the indication. For example, in fiscal year 2003, FDA approved Cubicin for the treatment of complicated skin and skin structure infections on the basis of evidence from two pivotal non-inferiority trials that used a total of five different active control drugs. While three of these active control drugs were FDA-approved to treat this indication, two were not.[Footnote 20] In addition, some of the active controls used in non-inferiority trials were themselves approved on the basis of evidence from other trials that compared the drug to another active control.[Footnote 21] However, FDA reviewed the selection of nearly all of the active controls used in the pivotal non-inferiority trials that supported the approval of the 18 NDAs, and found the active controls appropriate for use in these trials.[Footnote 22] FDA officials also told us that if a new drug was approved on the basis of evidence from non-inferiority trials, the active control used in these trials would most likely also be used in subsequent trials, except in cases where the newer drug proved to be superior to the active control.[Footnote 23] The margins used for most of the 18 NDAs approved on the basis of evidence from non-inferiority trials ranged from 5 to 20 percent, with the most commonly used margin being 10 percent.[Footnote 24] That is, for trials using a 10 percent non-inferiority margin, the new drug could be estimated to be up to 10 percent less effective than the active control. However, the observed difference in the effectiveness of the new drug and active control, as measured in the clinical trials, would be less than 10 percent. At the time of its review of the NDAs, FDA agreed with the non- inferiority margins set for all of the pivotal trials submitted for the majority of drugs approved on the basis of evidence from non- inferiority trials.[Footnote 25] All of the pivotal trials submitted for these drugs--that is, those where FDA agreed with the margin-- demonstrated that the new drug was non-inferior to the active control drug as measured on the primary endpoint, with one exception.[Footnote 26] These trials showed that the confidence interval for the difference in the drugs' effectiveness was within the non-inferiority margin. FDA did not agree with the non-inferiority margins set for pivotal trials submitted with three applications, though the agency approved these drugs based on evidence from these trials. For two drugs, Exjade and Reyataz, FDA stated that the proposed margins could not be used to measure the drugs' effectiveness. FDA conducted additional analyses of data from pivotal trials submitted in these drugs' applications which showed that the drugs were superior to a placebo. For the third drug, Noxafil, FDA did not agree with the sponsor's proposed justification of the margin for one trial, although this trial showed the difference in the drugs' effectiveness to be less than the disputed margin. * FDA approved Exjade in fiscal year 2006 to treat chronic iron overload in certain patients receiving blood transfusions. Exjade's NDA included evidence from one pivotal non-inferiority trial that had an objective of showing that Exjade lowered iron levels to a similar extent as the active control. Upon reviewing the application, FDA disagreed with the non-inferiority margin proposed for this trial. FDA analyzed data from the trial which showed that Exjade was effective in lowering patients' iron levels despite ongoing blood transfusions (which typically result in increased iron levels), particularly among those patients who began the trial with very high iron levels. FDA approved Exjade on the basis of this evidence, which showed that the drug would have been more effective than a placebo. In addition, FDA officials noted that Exjade presented a valuable alternative in the treatment of this indication.[Footnote 27] * FDA approved Reyataz in fiscal year 2003 for the treatment of human immunodeficiency virus (HIV) infection. Reyataz's NDA included evidence from two pivotal non-inferiority trials, including one in patients that were naïve to HIV treatment and one in patients that had experience receiving HIV treatment. FDA agreed with the margin proposed for the trial conducted in the treatment-naïve population, which was successful in demonstrating that Reyataz was non-inferior to its active control. However, FDA disagreed with the margin proposed for the trial conducted in the treatment-experienced population. Agency officials analyzed data from this trial which showed that Reyataz was effective in treatment-experienced patients, and this effect was greater than what would have been expected with a placebo. FDA approved Reyataz to treat HIV infection on the basis of this evidence, as well as other pivotal evidence of effectiveness in the treatment-naïve population. In addition, FDA officials noted that Reyataz presented an alternative to HIV-infected patients that were not responding to available HIV treatments. * FDA approved Noxafil in fiscal year 2006 for the prevention of invasive Aspergillus and Candida infections in certain patients on the basis of evidence from two pivotal trials. In its review of this NDA, FDA noted that the sponsor had not adequately explained the relevance of the proposed 15 percent non-inferiority margin. One of these trials demonstrated that Noxafil was superior to its active control, and the other trial demonstrated that the drug was at most three percent less effective than the active control. FDA approved this drug on the basis of this evidence of effectiveness. Table 2 provides a summary of the characteristics of non-inferiority trials for the 18 NDAs we identified as approved on the basis of evidence from non-inferiority trials. Table 2: Characteristics of Non-Inferiority Trials for 18 NDAs Submitted for FDA Review from Fiscal Years 2002 through 2009 and Approved on the Basis of Primary Evidence from Non-Inferiority Trials: Drug name: Alinia; Number of pivotal non-inferiority trials: 1; Type of primary endpoint(s): clinical; Number of active controls: 1[A]; Non-inferiority margin(s): 20%. Drug name: Aloxi; Number of pivotal non-inferiority trials: 3; Type of primary endpoint(s): clinical; Number of active controls: 2; Non-inferiority margin(s): 15%. Drug name: Apidra; Number of pivotal non-inferiority trials: 3; Type of primary endpoint(s): surrogate; Number of active controls: 2; Non-inferiority margin(s): 0.4%[B]. Drug name: Cubicin; Number of pivotal non-inferiority trials: 2; Type of primary endpoint(s): clinical; Number of active controls: 5[A]; Non-inferiority margin(s): 10%. Drug name: Doribax; Number of pivotal non-inferiority trials: 3[C]; Type of primary endpoint(s): clinical and surrogate; Number of active controls: 2; Non-inferiority margin(s): 10% and 15%. Drug name: Eraxis; Number of pivotal non-inferiority trials: 2; Type of primary endpoint(s): clinical and surrogate; Number of active controls: 1; Non-inferiority margin(s): 10% and 20%. Drug name: Exjade; Number of pivotal non-inferiority trials: 1; Type of primary endpoint(s): surrogate; Number of active controls: 1; Non-inferiority margin(s): 15%. Drug name: Levemir; Number of pivotal non-inferiority trials: 3; Type of primary endpoint(s): surrogate; Number of active controls: 2; Non-inferiority margin(s): 0.4%[B]. Drug name: Livalo; Number of pivotal non-inferiority trials: 4[C]; Type of primary endpoint(s): surrogate; Number of active controls: 3; Non-inferiority margin(s): 6%[D]. Drug name: Mycamine; Number of pivotal non-inferiority trials: 2; Type of primary endpoint(s): clinical and surrogate; Number of active controls: 1; Non-inferiority margin(s): 10%. Drug name: Noxafil; Number of pivotal non-inferiority trials: 1[C]; Type of primary endpoint(s): clinical; Number of active controls: 1; Non-inferiority margin(s): 15%. Drug name: Pylera; Number of pivotal non-inferiority trials: 1; Type of primary endpoint(s): surrogate; Number of active controls: 1; Non-inferiority margin(s): 15%. Drug name: Reyataz; Number of pivotal non-inferiority trials: 2; Type of primary endpoint(s): surrogate; Number of active controls: 2; Non-inferiority margin(s): 10%[E]. Drug name: Tindamax; Number of pivotal non-inferiority trials: 4[C]; Type of primary endpoint(s): surrogate; Number of active controls: 2[A]; Non-inferiority margin(s): 5%. Drug name: Tygacil; Number of pivotal non-inferiority trials: 4; Type of primary endpoint(s): clinical; Number of active controls: 3; Non-inferiority margin(s): 10%. Drug name: Tyzeka; Number of pivotal non-inferiority trials: 1; Type of primary endpoint(s): surrogate; Number of active controls: 1; Non-inferiority margin(s): 15%. Drug name: Uloric; Number of pivotal non-inferiority trials: 1[C]; Type of primary endpoint(s): surrogate; Number of active controls: 1; Non-inferiority margin(s): 10%. Drug name: Vibativ; Number of pivotal non-inferiority trials: 2; Type of primary endpoint(s): clinical; Number of active controls: 1; Non-inferiority margin(s): 10%. Source: GAO analysis of FDA documents. Notes: This list reflects NDAs for new molecular entities that FDA approved as of December 31, 2009. NDAs that included more than one non- inferiority trial may have each used different endpoints and different margins. Unless otherwise noted, all active controls were FDA-approved for the indication, and all margins represent the maximum acceptable difference in the portion of patients achieving a successful outcome. [A] At least one active control was not FDA approved for the indication. [B] The non-inferiority margin for this NDA represents the maximum acceptable difference in blood sugar levels. [C] The NDA also included data from at least one other pivotal clinical trial that was not a non-inferiority trial; for example, a superiority trial. [D] The non-inferiority margin for this NDA represents the maximum acceptable difference in cholesterol levels. [E] In addition to the margin listed, this NDA included primary evidence from another pivotal trial that set a margin of .5 log copies/mL, representing the maximum acceptable difference in HIV viral load levels. [End of table] We found that FDA reviewed the characteristics of the non-inferiority trials supporting the approval of the 18 NDAs to ensure that the drugs it approved were more effective than a placebo. FDA's review therefore minimized the potential for biocreep. Similarly, our examination of the trials' characteristics also revealed no evidence of biocreep. Some Non-Inferiority Trials Were Poorly Designed and Did Not Provide Primary Evidence for Approval: While non-inferiority trials provided primary evidence of effectiveness to support the approval of 18 NDAs, other non- inferiority trials were poorly designed and did not provide such evidence. Of the other 25 NDAs that included evidence from non- inferiority trials, FDA identified 9 applications that included poorly designed non-inferiority trials.[Footnote 28] These trials were unable to accurately measure the new drugs' effectiveness and did not provide primary evidence for the approval of these drugs.[Footnote 29] Some of the concerns FDA identified with sponsors' non-inferiority trials were: * inappropriate use of non-inferiority trials for the indication being treated, * inappropriate selection of an active control, including cases where the drug was not FDA-approved or the sponsor did not provide an adequate justification, and: * improper calculation or justification of the non-inferiority margin. FDA informed sponsors of its concerns with all of these applications' non-inferiority trials prior to the sponsors' submission of the NDAs. Specifically, FDA notified the sponsors between 1 month and 94 months before submission, with an average of about 30 months prior to submission. With the exception of one application, FDA notified all sponsors at least 6 months prior to submission. * For example, FDA advised one sponsor before the sponsor began its non-inferiority trials--24 months prior to submitting its NDA--that the agency did not consider it appropriate to use non-inferiority trials to support the approval of the drug for the indication being sought--treatment of schizophrenia. FDA reiterated this position on another occasion prior to the NDA submission. FDA did not consider the results of this trial to provide primary evidence to support its approval decision. The agency ultimately approved the drug based on evidence that the drug was superior to placebo as demonstrated in several other trials. * In another case, a sponsor conducted the non-inferiority trial outside of the U.S. and had not requested FDA's input while planning or conducting the trial. The sponsor requested a meeting with FDA to discuss its planned NDA. During this meeting, which occurred 1 month before FDA received the NDA, the agency learned of the sponsor's non- inferiority trial and communicated its concerns regarding the design of the trial. FDA did not consider the results of this non-inferiority trial in its approval decision, but ultimately approved the drug based on evidence of superiority to placebo as demonstrated in another trial. FDA Has Issued Detailed and Comprehensive Guidance on the Use of Non- Inferiority Trials to Establish the Effectiveness of New Drugs: In March 2010, FDA issued draft guidance on non-inferiority trials that provides detailed recommendations on using these trials to provide evidence of a new drug's effectiveness. This March 2010 draft guidance offers broader and more comprehensive information to supplement other indication-specific guidance documents the agency previously issued. FDA's March 2010 Draft Guidance Provides Detailed Recommendations on the Use of Non-Inferiority Trials in Establishing the Effectiveness of New Drugs: In March 2010, FDA issued new draft guidance on non-inferiority trials that provides detailed recommendations on how these trials may be used to establish the effectiveness of new drugs.[Footnote 30] Although FDA had previously issued guidance documents that included information regarding the use of non-inferiority trials for certain indications, this March 2010 guidance is the first focused solely on the use of non- inferiority trials. It explains the key principles involved in using a non-inferiority trial to demonstrate the effectiveness of a drug and provides detailed recommendations for such trials, including how to select an active control and how to set the non-inferiority margin (that is, determining the maximum clinically acceptable extent to which the new drug can be less effective than the active control), among other things. The March 2010 guidance also explains why the agency considers its recommendations appropriate, offers answers to frequently asked questions, and lists detailed examples to illustrate some common challenges in designing and interpreting non-inferiority trials. FDA officials told us that they developed the March 2010 guidance on non-inferiority trials because it was clear to them that these trials were not well understood. The concepts elaborated on in the March 2010 guidance are not new, however. They have been part of FDA's considerations since at least 1985 when the agency substantially revised NDA regulations to include a provision describing the characteristics of adequate and well-controlled trials. These concepts have also been addressed, in part, in other agency guidance documents. However, FDA officials saw the need for more detailed guidance as they noticed many errors, especially related to the selection of a non- inferiority margin, in sponsors' execution of these trials. FDA officials also expect that the use of non-inferiority trials will rise as more drugs become available to prevent death or serious illness and the use of placebos may become unethical. FDA's March 2010 guidance explains when non-inferiority trials may be used to establish a drug's effectiveness. The guidance states that these trials are generally used when an available treatment is known to provide an important benefit--for example, the prevention of death or irreversible harm. In these cases, it would be considered unethical to use a placebo in a clinical trial. The guidance also states that non-inferiority trials may only be used when they are capable of measuring the effect of the new drug in the study--that is, when the active control is able to consistently demonstrate its expected effect in the non-inferiority trial.[Footnote 31] FDA's March 2010 guidance explains that non-inferiority trials may not be able to demonstrate the effectiveness of drugs treating certain indications because not all drugs have a consistent effect in treating these indications. [Footnote 32] The guidance also offers suggestions for other types of trials that may be useful in demonstrating a drug's effectiveness in cases where a non-inferiority trial is unable to provide evidence of effectiveness. The March 2010 guidance provides detailed recommendations on how to select an active control. For example, when more than one potential active control exists, the guidance recommends that the most effective drug be chosen as the active control. In addition, the frequently asked questions section also clarifies that the active control does not need to be FDA-approved for the indication. However, FDA officials we interviewed stated that active controls used in non-inferiority trials are usually FDA-approved. If the active control is not FDA- approved, FDA asks sponsors to provide evidence of the active control's effectiveness. FDA's March 2010 guidance also offers detailed advice on a range of other topics related to the use and interpretation of non-inferiority trials. For example, it suggests two methodologies that can be used to set the margin, offers step-by-step instructions on how to use each of these approaches, and addresses the role of clinical judgment in determining the margin. It also explains how to adjust the margin to account for some of the uncertainties related to non-inferiority trials, such as differences between the planned non-inferiority trial and prior trials that measured the effectiveness of the active control. The guidance offers advice on how to determine the proper number and type of patients to enroll in the trial, and how to select an endpoint. For example, the guidance states that the endpoint should be "one for which there is a good basis for knowing the effect of the active control." Most of the experts we interviewed who reviewed FDA's March 2010 guidance told us that they thought the recommendations it included were clear and detailed, and addressed the key principles involved in conducting non-inferiority trials. Some experts noted that the guidance's frequently asked questions and examples were useful in illustrating the key principles described in the document, and said that FDA's recommendations would help sponsors appropriately use these trials to prove a drug's effectiveness. While experts we interviewed who reviewed FDA's March 2010 guidance noted that it addressed key principles, most identified additional technical issues that they would have liked this guidance to have addressed. For example, the March 2010 guidance does not address how the use of a surrogate endpoint impacts the design and interpretation of a non-inferiority trial. FDA officials told us that the guidance applies to non-inferiority trials that use surrogate endpoints. However, some experts we interviewed noted that such trials are difficult to design and interpret; therefore, additional guidance on this topic may be helpful. Since the non-inferiority margin represents the maximum clinically acceptable extent to which the new drug can be less effective than the active control, experts told us that sponsors would need to translate the drug's effect on a surrogate endpoint into its expected effect on a clinical endpoint in order to calculate the non-inferiority margin and interpret the trials' results. Some experts also noted that the guidance does not include enough detailed instructions on how to estimate the effect of the active control in the non-inferiority trial. Finally, some experts who reviewed FDA's March 2010 guidance told us that they wished the guidance more emphatically stated that non-inferiority trials should only be used as a last resort when seeking drug approval. FDA's March 2010 Draft Guidance Provides Broader and More Comprehensive Recommendations to Supplement Previously Issued Indication-Specific Guidance Documents: FDA's March 2010 draft guidance provides broader and more comprehensive information about the use of non-inferiority trials, supplementing other indication-specific guidance documents the agency had already issued. The objective and content of these two types of guidance documents differ. The March 2010 guidance offers comprehensive information on one topic, non-inferiority trials, that may be generally applied for all drugs using these trials. In contrast, FDA's indication-specific guidance documents present recommendations on many topics--including trial design--for consideration in developing drugs to treat a particular indication or set of indications. Some of these indication-specific documents provide recommendations on how to use non-inferiority trials for that particular indication; for example, by suggesting a specific margin or a specific endpoint. However, unlike FDA's March 2010 guidance, not all indication-specific guidance documents include information on all of the key principles involved in using a non-inferiority trial to establish a drug's effectiveness. In addition, these indication- specific guidance documents do not include the same level of detail on the key principles that is in the March 2010 guidance. For example, several of FDA's indication-specific guidance documents state that sponsors should justify their selection of non-inferiority margins in their NDAs. However, in these documents FDA does not elaborate on the methods sponsors could use to select or justify the margins. In contrast, the March 2010 non-inferiority guidance provides detailed instructions on how to calculate the margin. FDA's indication-specific guidance documents provide sponsors with additional clarity on when non-inferiority trials may be used to establish the effectiveness of drugs treating a particular indication. From January 2002 through June 2010, FDA issued 17 guidance documents that state the agency's position regarding the use of non-inferiority trials in demonstrating the effectiveness of drugs treating certain indications. In these indication-specific guidance documents, FDA stated that non-inferiority trials may be able to demonstrate the effectiveness of drugs treating eight indications, including those for HIV, cancer, diabetes mellitus, and certain severe infections. During the same period, FDA also issued nine indication-specific guidance documents which state that non-inferiority trials may not be able to demonstrate the effectiveness of drugs treating other indications-- including some less severe infections such as sinusitis and acute bacterial otitis media (ear infections)--because the agency has been unable to identify available drugs that have a consistent effect and could serve as active controls in non-inferiority trials. (See table 3.) Appendix II identifies the guidance documents FDA has issued with information on the use of non-inferiority trials from January 2002 through June 2010, including indication-specific documents as well as the March 2010 draft guidance on non-inferiority trials. Table 3: Summary of FDA Guidance Regarding the Use of Non-Inferiority Trials for Particular Indications: Indications for which non-inferiority trials may be able to demonstrate a new drug's effectiveness (guidance issuance date): 1. HIV (Oct. 2002); 2. Gingivitis (June 2005); 3. Cancer (May 2007); 4. Malaria (June 2007); 5. Certain infections[A] (Oct. 2007); 6. Diabetes mellitus (Feb. 2008); 7. Community-acquired bacterial pneumonia (Mar. 2009); 8. Helicobacter pylori (Oct. 2009). Indications for which non-inferiority trials may not be able to demonstrate a new drug's effectiveness (guidance issuance date): 1. Complications arising from the smallpox vaccine with vaccina virus (Mar. 2004); 2. Sinusitis treated with non-antibiotic drugs (Nov. 2006); 3. Acute bacterial sinusitis (Oct. 2007); 4. Chronic obstructive pulmonary disease (Nov. 2007); 5. Acute bacterial otitis media (Jan. 2008); 6. Acute bacterial exacerbations of chronic bronchitis in patients with chronic obstructive pulmonary disease (Aug. 2008); 7. Influenza (Feb. 2009); 8. Systemic lupus erythematosus (June 2010); 9. Lupus nephritis caused by systemic lupus erythematosus (June 2010). Source: GAO analysis of FDA documents. [A] FDA's guidance noted that non-inferiority trials may be able to provide evidence of effectiveness for drugs treating some, but not all infections. [End of table] Our review of FDA's indication-specific guidance showed that the agency has become more conservative in allowing evidence from non- inferiority trials to demonstrate the effectiveness of new drugs. First, FDA has revised its view regarding when non-inferiority trials may be used. Prior to 2007, for example, FDA had approved drugs treating several less severe infections--including acute bacterial sinusitis, acute bacterial otitis media, and acute bacterial exacerbations of chronic bronchitis--on the basis of evidence from non- inferiority trials.[Footnote 33] Experts we interviewed noted that these infections can often be resolved without treatment--and thus it is difficult to estimate the effect that an active control drug would have in a non-inferiority trial. In 2007 and 2008, FDA issued several guidance documents stating that non-inferiority trials may not be able to demonstrate the effectiveness of drugs treating these indications. Second, FDA has become more rigorous in its review of evidence from non-inferiority trials. For example, prior to 2001, FDA's guidance on the development of anti-infective drugs had not advised sponsors to scientifically calculate or justify their selected non-inferiority margins--a step that FDA's March 2010 guidance recommends. Agency Comments: We provided a draft of this report to HHS for review. We received technical comments from HHS, which we incorporated as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Commissioner of FDA and appropriate congressional committees. The report also will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or crossem@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Signed by: Marcia Crosse Director, Health Care: List of Requesters: The Honorable Charles E. Grassley: Ranking Member: Committee on Finance: United States Senate: The Honorable Henry A. Waxman: Chairman: The Honorable John D. Dingell: Chairman Emeritus: Committee on Energy and Commerce: House of Representatives: The Honorable Bart Stupak: Chairman: Subcommittee on Oversight and Investigations: Committee on Energy and Commerce: House of Representatives: The Honorable Edward J. Markey: House of Representatives: [End of section] Appendix I: New Drug Applications Approved on the Basis of Evidence from Non-Inferiority Trials: The Food and Drug Administration (FDA) approved 18 new drug applications (NDA) that were submitted from fiscal year 2002 through fiscal year 2009 on the basis of evidence from non-inferiority trials. The majority of these were antimicrobial drugs, such as those that treat bacterial, viral, and fungal infections. (See table 4.) Table 4: NDAs Submitted for FDA Review from Fiscal Years 2002 through 2009 Where the Basis of Approval Included Primary Evidence from Non- Inferiority Trials: Drug name (active ingredient): Alinia (nitazoxanide); NDA number: 21498; Fiscal year of approval: 2003; Drug type: Antimicrobial; Approved indication: Treatment of diarrhea caused by the pathogen Giardia lamblia[A]. Drug name (active ingredient): Aloxi (palonosetron hydrochloride); NDA number: 21372; Fiscal year of approval: 2003; Drug type: Gastroenterology; Approved indication: Prevention of acute nausea and vomiting associated with initial and repeated courses of moderately and highly emetogenic cancer chemotherapy[B]. Drug name (active ingredient): Cubicin (daptomycin); NDA number: 21572; Fiscal year of approval: 2003; Drug type: Antimicrobial; Approved indication: Treatment of complicated skin and skin structure infections caused by susceptible isolates of gram-positive microorganisms. Drug name (active ingredient): Reyataz (atazanavir sulfate); NDA number: 21567; Fiscal year of approval: 2003; Drug type: Antimicrobial; Approved indication: Treatment of human immunodeficiency virus-1 infection. Drug name (active ingredient): Apidra (insulin glulisine); NDA number: 21629; Fiscal year of approval: 2004; Drug type: Metabolism and endocrinology; Approved indication: Treatment of diabetes mellitus in adult patients. Drug name (active ingredient): Tindamax (tinidazole); NDA number: 21618; Fiscal year of approval: 2004; Drug type: Antimicrobial; Approved indication: Treatment of trichomoniasis. Drug name (active ingredient): Levemir (insulin detemir); NDA number: 21536; Fiscal year of approval: 2005; Drug type: Metabolism and endocrinology; Approved indication: Treatment of diabetes mellitus in adult patients. Drug name (active ingredient): Mycamine (micafungin sodium); NDA number: 21506; Fiscal year of approval: 2005; Drug type: Antimicrobial; Approved indication: Prevention of Candida infections in patients undergoing hematopoietic stem cell transplantation. Drug name (active ingredient): Tygacil (tigecycline); NDA number: 21821; Fiscal year of approval: 2005; Drug type: Antimicrobial; Approved indication: Treatment of complicated skin and skin structure infections and complicated intra-abdominal infections. Drug name (active ingredient): Eraxis (anidulafungin); NDA number: 21632; Fiscal year of approval: 2006; Drug type: Antimicrobial; Approved indication: Treatment of esophageal candidiasis. Drug name (active ingredient): Exjade (deferasirox); NDA number: 21882; Fiscal year of approval: 2006; Drug type: Hematology; Approved indication: Treatment of chronic iron overload due to blood transfusions in patients 2 years of age and older. Drug name (active ingredient): Noxafil (posaconazole); NDA number: 22003; Fiscal year of approval: 2006; Drug type: Antimicrobial; Approved indication: Prevention of invasive Aspergillus and Candida infections in patients, 13 years of age and older, who are at high risk of developing these infections. Drug name (active ingredient): Pylera (biskalcitrate, metronidazole, and tetracycline hydrochloride); NDA number: 50786; Fiscal year of approval: 2006; Drug type: Antimicrobial; Approved indication: Treatment of patients with Helicobacter pylori infection and duodenal ulcer disease. Drug name (active ingredient): Tyzeka (telbivudine); NDA number: 22011; Fiscal year of approval: 2007; Drug type: Antimicrobial; Approved indication: Treatment of chronic hepatitis B in certain populations of adults. Drug name (active ingredient): Doribax (doripenem); NDA number: 22106; Fiscal year of approval: 2008; Drug type: Antimicrobial; Approved indication: Treatment of complicated intra-abdominal infections and complicated urinary tract infections caused by certain microorganisms. Drug name (active ingredient): Livalo (pitavastatin); NDA number: 22363; Fiscal year of approval: 2009; Drug type: Metabolism and endocrinology; Approved indication: Treatment of primary hyperlipidemia and mixed dyslipidemia. Drug name (active ingredient): Uloric (febuxostat); NDA number: 21856; Fiscal year of approval: 2009; Drug type: Rheumatology; Approved indication: Treatment of hyperuricemia in patients with gout. Drug name (active ingredient): Vibativ (telavancin); NDA number: 22110; Fiscal year of approval: 2009; Drug type: Antimicrobial; Approved indication: Treatment of complicated skin and skin structure infections caused by susceptible gram-positive bacteria in adults. Source: GAO analysis of FDA data. Note: Our review was limited to NDAs for new molecular entities. [A] As part of the same NDA, Alinia was approved for the treatment of diarrhea caused by the pathogen Cryptosporidium parvum based on evidence from two pivotal placebo-controlled trials. [B] As part of the same NDA, Aloxi was approved for the treatment of delayed nausea and vomiting associated with initial and repeated courses of moderately emetogenic cancer chemotherapy based on evidence of superiority to an active control in the same trial. [End of table] [End of section] Appendix II: FDA Guidance Documents Issued with Information on the Use of Non-Inferiority Trials: From January 2002 through June 2010, FDA issued 17 indication-specific guidance documents that included information about non-inferiority trials, and one guidance document that included broad recommendations regarding the use of non-inferiority trials. Table 5: FDA-Issued Guidance Including Recommendations on the Use of Non-Inferiority Trials, January 2002 through June 2010: Guidance title: Guidance for Industry, Antiretroviral Drugs Using Plasma HIV RNA Measurements--Clinical Considerations for Accelerated and Traditional Approval; Issuance month and year: October 2002. Guidance title: Guidance for Industry, Vaccinia Virus--Developing Drugs to Mitigate Complications from Smallpox Vaccination; Issuance month and year: March 2004. Guidance title: Guidance for Industry, Gingivitis: Development and Evaluation of Drugs for Treatment or Prevention; Issuance month and year: June 2005. Guidance title: Guidance for Industry, Sinusitis: Designing Clinical Development Programs of Nonantimicrobial Drugs for Treatment; Issuance month and year: November 2006. Guidance title: Guidance for Industry, Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics; Issuance month and year: May 2007. Guidance title: Guidance for Industry, Malaria: Developing Drug and Nonvaccine Biological Products for Treatment and Prophylaxis; Issuance month and year: June 2007. Guidance title: Guidance for Industry, Acute Bacterial Sinusitis: Developing Drugs for Treatment; Issuance month and year: October 2007. Guidance title: Guidance for Industry, Antibacterial Drug Products: Use of Noninferiority Studies to Support Approval; Issuance month and year: October 2007. Guidance title: Guidance for Industry, Chronic Obstructive Pulmonary Disease: Developing Drugs for Treatment; Issuance month and year: November 2007. Guidance title: Guidance for Industry, Acute Bacterial Otitis Media: Developing Drugs for Treatment; Issuance month and year: January 2008. Guidance title: Guidance for Industry, Diabetes Mellitus: Developing Drugs and Therapeutic Biologics for Treatment and Prevention; Issuance month and year: February 2008. Guidance title: Guidance for Industry, Acute Bacterial Exacerbations of Chronic Bronchitis in Patients with Chronic Obstructive Pulmonary Disease: Developing Antimicrobial Drugs for Treatment; Issuance month and year: August 2008. Guidance title: Guidance for Industry, Influenza: Developing Drugs for Treatment and/or Prophylaxis; Issuance month and year: February 2009. Guidance title: Guidance for Industry, Community-Acquired Bacterial Pneumonia: Developing Drugs for Treatment; Issuance month and year: March 2009. Guidance title: Guidance for Industry, Helicobacter pylori-Associated Duodenal Ulcer Disease in Adults: Developing Drugs for Treatment; Issuance month and year: October 2009. Guidance title: Guidance for Industry, Non-Inferiority Clinical Trials; Issuance month and year: March 2010. Guidance title: Guidance for Industry, Lupus Nephritis Caused By Systemic Lupus Erythematosus--Developing Medical Products for Treatment; Issuance month and year: June 2010. Guidance title: Guidance for Industry, Systemic Lupus Erythematosus-- Developing Medical Products for Treatment (previously issued in March 2005 as Guidance for Industry, Systemic Lupus Erythematosus-- Developing Drugs for Treatment); Issuance month and year: June 2010. Source: FDA documents. [End of table] [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Marcia Crosse, (202) 512-7114 or crossem@gao.gov: Acknowledgments: In addition to the contact named above, Geri Redican-Bigott, Assistant Director; Kathleen Diamond; Carolyn Garvey; Cathy Hamann; Julian Klazkin; Kaitlin McConnell; and Patricia Roy made key contributions to this report. [End of section] Footnotes: [1] 21 U.S.C. § 355(d), 21 C.F.R. § 314.126 (2009). [2] NDAs for new molecular entities represented about 25 percent of all NDAs submitted during this period. [3] Unless otherwise noted, we use the term NDA throughout this report to refer to NDAs for new molecular entities. [4] We obtained these materials between July 2009 and January 2010 from FDA's Web site, [hyperlink, http://www.accessdata.fda.gov/scripts/cder/drugsatfda/]. [5] Alternatively, FDA may determine that other clinical trials provide support for the overall application, but do not provide the primary evidence of effectiveness. [6] 21 C.F.R. § 312.20 (2009). Sponsors conducting clinical trials outside the United States are not required to submit data to, or consult with, FDA prior to or during the trials, although they may choose to do so at any time. [7] 21 C.F.R. § 312.23 (2009). [8] FDA will allow clinical trials to proceed as long as the participants are not exposed to an unreasonable and significant risk of illness or injury, and that other requirements are met. 21 C.F.R. § 312.42(b) (2009). [9] FDA does not require sponsors to show that a drug is more effective than other available treatments, although such evidence may be used to support a drug's approval. [10] 21 C.F.R. § 314.126 (2009). [11] For example, demonstrating that a drug can lower blood pressure may be used as a surrogate endpoint to predict whether the drug is effective in preventing strokes. A drug sponsor can demonstrate the effect of a new drug on a surrogate endpoint based on smaller and shorter trials than would be required to prove the drug's effectiveness on a clinical endpoint. For additional information on the use of surrogate endpoints in the drug approval process, see GAO, New Drug Approval: FDA Needs to Enhance Its Oversight of Drugs Approved on the Basis of Surrogate Endpoints, GAO-09-866 (Washington, D.C.: Sept. 23, 2009). [12] According to FDA officials, the active controls used in non- inferiority trials are generally FDA-approved for that particular indication and population. However, sponsors may use active control drugs that are not FDA-approved; for example, when clinical trials take place outside the United States. In such cases, FDA asks sponsors to provide, as part of their NDA, evidence of the active control's effect in treating the indication and population as studied in adequate and well-controlled trials. [13] Non-inferiority trials that show a small difference between a new drug and an active control do not necessarily demonstrate that the new drug is effective--it could also mean that neither was effective in the trial; evidence that the active control drug was effective in the trial is therefore critical. [14] 21 C.F.R. § 312.41(c) (2009). [15] Food and Drug Administration, Guidance For Industry: Clinical Development and Labeling of Anti-infective Drug Products (Rockville, Md.: Oct. 26, 1992). [16] Seventy-six percent of all NDAs for antimicrobial drugs included evidence from at least one non-inferiority trial. [17] The 18 NDAs that FDA approved on the basis of evidence from pivotal non-inferiority trials represent a small share, about 14 percent, of the total number of NDAs FDA approved during this period-- 125. These 18 approvals also reflect a 42 percent approval rate for all NDA applications submitted with evidence from non-inferiority trials from fiscal years 2002 through 2009, lower than the respective approval rate--71 percent--for all NDA applications submitted during this period. [18] These 12 approvals reflect 50 percent of all NDAs for antimicrobial drugs that were approved during this time frame. [19] For example, in 2004 FDA approved the drug Apidra for the treatment of diabetes mellitus in adult patients based on evidence of effectiveness on a surrogate endpoint as measured in three pivotal non- inferiority trials. Through the use of a blood test, the drug's sponsor demonstrated that Apidra reduced patients' hemoglobin A1c--a measure of blood sugar levels. This surrogate endpoint was used as an alternative to measuring clinical endpoints of diabetes-related morbidity, such as eye or kidney disease. [20] One of the two unapproved active control drugs was similar to an FDA-approved drug; both had the same active ingredient but used different formulations and routes of administration. [21] For some of these drugs, we were unable to distinguish between trials demonstrating superiority and non-inferiority to active controls because FDA has not always included evidence of its statistical testing--for example, confidence intervals--in its review documentation. [22] We could not determine whether FDA reviewed two active controls that were used in pivotal non-inferiority trials supporting two NDAs. Both NDAs were approved based on evidence from multiple pivotal non- inferiority trials, and each included one other active control which FDA reviewed. [23] In its review documentation, FDA identified and acknowledged that one of the drugs it approved on the basis of evidence from a non- inferiority trial would be inappropriate to use as an active control drug in future trials. This type of identification and acknowledgment reduces the potential for biocreep to occur in the future. [24] Fourteen of the 18 NDAs FDA approved included non-inferiority trials that measured the effectiveness of the new drug and active control drugs with a success rate representing the portion of patients achieving a successful outcome for the relevant indication. Trials for 3 of the 18 NDAs did not measure effectiveness this way; these trials measured effectiveness as a percentage change in outcome, for example, the percentage change in cholesterol or blood sugar, and therefore the non-inferiority margins used for these drugs are not directly comparable. In addition, the one remaining NDA included evidence from two non-inferiority trials that measured the effect of the new drug in both ways, measuring with a success rate in one trial and a percentage change in outcome in another trial. [25] We did not assess whether FDA's agreement with these margins was appropriate. [26] One pivotal trial for Livalo--a drug FDA approved in fiscal year 2009 for the treatment of patients with high cholesterol--was unsuccessful in demonstrating non-inferiority to its active control in one patient group. However, this drug's application also included the results from four other pivotal trials which tested the drug in different patient groups using three different active controls. One of these four trials showed that Livalo was superior to its active control, and three trials demonstrated that the drug was non-inferior to the trials' respective active controls. FDA approved the drug based on the collective evidence of effectiveness as demonstrated in all five trials. [27] Exjade presented an opportunity for increased convenience in administration and a potential for increased compliance with treatment. Exjade is available in a once-daily tablet. Previously, the only other available treatment was an infusion administered between 8 and 24 hours per day. [28] The remaining 16 NDAs also included evidence from non-inferiority trials but were either approved on the basis of other primary evidence, or were not approved as of December 31, 2009. Some of these trials were unsuccessful in demonstrating non-inferiority to an active control or had demonstrated superiority to an active control or placebo. Other applications provided evidence of effectiveness through non-inferiority trials but FDA had not approved the drug due to other concerns, such as safety. [29] As of December 31, 2009, FDA had approved seven of these applications based on other evidence, such as superiority to placebo or active control. It decided not to approve the remaining two drugs. [30] See Food and Drug Administration, Guidance for Industry: Non- Inferiority Clinical Trials (Silver Spring, Md.: Mar. 1, 2010). FDA guidance documents reflect the agency's current thinking on a topic. Such documents include non-binding recommendations and do not establish legally enforceable requirements. FDA solicited comments from the public on this draft guidance through June 1, 2010, and received comments from 13 interested parties as of that date. Agency officials told us that they plan to review these comments and issue a final version of the guidance, although they were unsure of when this guidance would be issued in final form. [31] FDA recommends that the active control consistently demonstrate its expected effect because non-inferiority trials are only able to measure the new drug's effectiveness relative to the active control. If the active control did not demonstrate its expected effect in the non-inferiority trial, the results of the trial may not be able to support the conclusion that the new drug is effective. [32] For example, drugs treating depression and anxiety often fail to consistently demonstrate effectiveness compared to placebo. Therefore, the agency considers non-inferiority trials incapable of providing evidence of effectiveness to support the approval of new drugs treating these indications. [33] For example, FDA approved Factive in fiscal year 2003 for the treatment of acute bacterial exacerbations of chronic bronchitis on the basis of evidence from non-inferiority trials. FDA also approved Ketek in fiscal year 2004 for the treatment of acute bacterial sinusitis, acute bacterial exacerbations of chronic bronchitis, and community acquired pneumonia (of mild to moderate severity) on the basis of evidence from non-inferiority trials. However, in fiscal year 2007 FDA removed two indications--acute bacterial sinusitis and acute bacterial exacerbations of chronic bronchitis--from Ketek's labeling as a result of concerns related to the product's safety. The NDAs for both of these drugs were submitted for review prior to fiscal year 2002 and were therefore not included within our scope. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: