This is the accessible text file for GAO report number GAO-03-494
entitled 'Dietary Supplements: Review of Health-Related Call Records
for Users of Metabolife 356' which was released on April 30, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to the Chairman, Subcommittee on Wellness and Human Rights,
Committee on Government Reform, House of Representatives:
United States General Accounting Office:
GAO:
March 2003:
DIETARY SUPPLEMENTS:
Review of Health-Related Call Records for Users of Metabolife 356:
Metabolife 356 Adverse Event Reports:
GAO-03-494:
Contents:
Letter:
Results in Brief:
Background:
Consumer Information in Call Records Was Limited, Sometimes Difficult
to Interpret, and Not Consistently Recorded:
Metabolife International Had Thousands of Call Records Reporting
Adverse Events Associated with Metabolife 356:
Findings of Different Reviews of Metabolife International Call Records
Vary:
Summary:
Agency and Metabolife International Comments and Our Evaluation:
Appendix I: Scope and Methodology for Categorizing the Call Records:
Call Records And Supplementary Information:
Exclusion of Duplicate and Nonhealth-Related Call Records:
Classification of Records and Data Entry Procedures:
Appendix II: Metabolife 356 Label:
Appendix III: Requirements for Reporting Adverse Events to FDA:
Appendix IV: Comments from the Food and Drug Administration:
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Acknowledgments:
Tables:
Table 1: Percentage of Call Records in Which Consumer and Response
Details Were Recorded:
Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke,
Seizure, or Death:
Table 3: Metabolife 356 Call Records Reporting Adverse Events Described
as Serious or Potentially Serious in FDA's 1997 Proposed Rule:
Table 4: Number of Call Records Containing Reports of Heart Attack,
Stroke, Seizure, Death, or Cardiac Arrest Reported in Reviews of
Metabolife International Call Records:
Table 5: Requirements for Reporting Adverse Events to FDA:
Abbreviations:
ANADA: Abbreviated New Animal Drug Application:
ANDA: Abbreviated New Drug Application:
FDA: Food and Drug Administration:
HHS: Department of Health and Human Services:
NADA: New Animal Drug Application:
NDA: New Drug Application:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO's product.
United States General Accounting Office:
Washington, DC 20548:
March 31, 2003:
The Honorable Dan Burton
Chairman
Subcommittee on Wellness and Human Rights
Committee on Government Reform
House of Representatives:
Dear Mr. Chairman:
Medical experts have expressed concerns about the safety of dietary
supplements containing ephedra or ephedrine alkaloids, which are used
by millions of Americans annually.[Footnote 1] On February 28, 2003,
the Food and Drug Administration (FDA) announced several proposed
changes to its regulation of dietary supplements containing ephedra,
including requiring a product label warning that "Heart attack, stroke,
seizure, and death have been reported after consumption of ephedrine
alkaloids."[Footnote 2] As of September 27, 2002, FDA had received
approximately 1,800 adverse event reports regarding consumers of
dietary supplements containing ephedra. Of these, 322 concerned
Metabolife 356, a weight loss product first marketed in 1995 by
Metabolife International, a large manufacturer of dietary supplements
containing ephedra. Adverse event reports can signal potential health
problems that deserve additional investigation, but, on their own,
generally are not sufficient to establish that the reported problems
were caused by use of the product.
Metabolife International has also received complaints about adverse
health events among users of Metabolife 356.[Footnote 3] Between August
and December 2002, Metabolife International made available to the
public redacted[Footnote 4] copies of nearly 16,000 pages of
documentation that it identified as containing reports of adverse
events among consumers of Metabolife 356.[Footnote 5] These complaints,
which were received between May 1997 and July 2002, had not been
previously released to FDA. Most of them were records of calls received
through a consumer health information phone line established by
Metabolife International in 1998 and staffed by its nurses.[Footnote 6]
Metabolife International officials told us that the phone line was
established to provide information to consumers regarding appropriate
use of Metabolife 356. In letters to the Texas Department of Health and
FDA,[Footnote 7] company officials described the phone line as a
"safety monitoring procedure" for the reporting of medical complaints.
The call records ranged from handwritten notes to printed versions of
records that had been entered into a database developed by Metabolife
International. These call records have been the subject of six previous
reviews: one by Metabolife International,[Footnote 8] three by
consultants commissioned by Metabolife International,[Footnote 9] one
by the minority staff of the Committee on Government Reform, House of
Representatives,[Footnote 10] and one by the RAND Corporation.[Footnote
11]
You asked us to review the content of all health-related call records
made public by Metabolife International. Specifically, you asked us to
answer the following questions. (1) To what extent was consumer
information in the call records comprehensive, interpretable, and
consistently recorded? (2) How many call records reported health-
related problems, and how many of those were serious? (3) How do our
counts of reported serious adverse events compare with those of other
reviews for those events counted by Metabolife International?
In responding to your request, we reviewed all the pages of
documentation voluntarily provided to us by Metabolife International.
We did not independently verify that we received all of the call
records held by Metabolife International. We excluded from our review
call records that duplicated other records.[Footnote 12] To determine
the extent to which consumer information was comprehensive,
interpretable, and consistently recorded in the call records, we
recorded information about the adverse event, demographic information
about the individual consumer, and other details in the call record.
Specifically, we recorded background information similar to that used
by FDA in the reporting of adverse events, including age, sex, weight,
height, the amount of Metabolife 356 used, the duration of use, and
whether any medical history was noted in the call record.
To assess how many call records reported health-related problems and
how many of those were serious, we first counted the number of call
records that reported at least one adverse event. Within this set of
call records, we then counted the number of reports of heart attack,
stroke, seizure, and death--the types of serious adverse events
identified in FDA's proposed label warning. We also counted the number
of reports of the
23 other types of adverse events that were described as serious or
potentially serious in FDA's 1997 proposed rule on dietary supplements
containing ephedrine alkaloids.[Footnote 13] For call records that did
not report any of the above adverse events, we counted the number of
records, but did not count the number of other specific types of
adverse events reported.
We classified events in the call records based solely on the words and
phrases therein; we did not attempt to diagnose a consumer's condition
or to otherwise interpret the information presented.[Footnote 14] We
did not apply medical judgment in the process of classifying events and
we did not independently verify the accuracy of the information in the
records or determine the validity of the claims made in the call
records. We also did not attempt to determine whether Metabolife 356
caused the reported adverse events. Our results may either overestimate
or underestimate the number and severity of adverse events because the
call records generally do not include medical diagnoses made by
qualified professionals.[Footnote 15]
To determine how our counts of reported serious adverse events compare
with those of other reviews, we examined the six previous reviews of
Metabolife International's call records. In addition, we interviewed
Metabolife International and FDA officials. Appendix I describes our
methodology in more detail. We conducted our work from September 2002
through March 2003 in accordance with generally accepted government
auditing standards.
Results in Brief:
The information in the Metabolife International call records was
limited, sometimes difficult to understand and interpret, and consumer
information was not consistently recorded. In some cases, the evidence
for a report of an adverse event was limited to a single word on a call
record. In addition, most call records did not record complete
information about the consumer's age, sex, weight, and height.
Information about both the amount of product used and duration of use
was recorded for 60 percent of the call records. Further, handwritten
call records were difficult to read and understand. Different versions
of the call records sometimes contained different information about the
consumer and the symptoms they reported. Nearly all of the reports of
adverse events that contained information about the amount of
Metabolife 356 used and duration of use were for consumers who reported
following the usage guidelines on the product label, not for consumers
who reported that they took too much Metabolife 356 or used it for too
long a period.
We categorized 14,684 call records from Metabolife International as
containing reports of at least one adverse event associated with
Metabolife 356. We found that there were 92 reports of the serious
adverse events identified in FDA's proposed label warning for dietary
supplements containing ephedrine alkaloids: 18 reported heart attacks,
26 reported strokes, 43 reported seizures, and 5 reported deaths. Among
the other adverse events reported that were identified as serious or
potentially serious in FDA's 1997 proposed rule, we found, for example,
93 reports of significant elevation of blood pressure, 31 reports of
abnormal heart rhythm, 47 reports of loss of consciousness, and
181 reports of systemic rash. Because of the inherent limitations of
adverse event reports and the incomplete nature of these call records,
we cannot establish that the reported adverse events were caused by the
use of Metabolife 356.
All of the reviews of the Metabolife International call records,
including ours, counted reports of serious adverse events, although
none of the reviews reported identical results. For those adverse
events that Metabolife International counted--heart attacks, strokes,
seizures, deaths, and cardiac arrests--our counts of reported events
are similar to the counts from the other reviews. We counted 96 such
reported events. Metabolife International counted 78, and the counts of
the other reviews ranged from 65 to 107.
In commenting on a draft of this report, FDA discussed the value of
reports of adverse events in helping to understand the causes of such
events.
Background:
Metabolife 356, which claims to raise the body's metabolism and help
dieters lose weight while maintaining high energy levels, contains
32 ingredients, including ephedra, guarana (an herbal source of
caffeine), bee pollen, and caffeine.[Footnote 16] The product label
recommends that adults take one to two caplets two to three times per
day or every 4 hours, not to exceed eight caplets per day. Warnings on
the product label suggest that a health care professional be consulted
by individuals who are using any other dietary supplement, prescription
drug, or over-the-counter drug containing ephedrine alkaloids or who
have, or have a family history of, any of 11 health conditions,
including heart disease, high blood pressure, diabetes, recurrent
headaches, and depression. The label also recommends that persons
should not use the product for more than
12 weeks and that exceeding the recommended amount may cause serious
adverse health effects including heart attack or stroke. Other possible
side effects mentioned on the label include rapid heartbeat, dizziness,
severe headache, and shortness of breath. The complete product label is
in appendix II.
The Dietary Supplement Health and Education Act of 1994 created a
framework for FDA's regulation of dietary supplements as part of its
oversight of food safety. Dietary supplements are generally marketed
without prior FDA review of their safety and effectiveness.[Footnote
17] Manufacturers of dietary supplements are responsible for ensuring
the safety of the dietary supplements they sell. Therefore, FDA relies
on voluntary reports of adverse events from consumers, health
professionals, and others in its effort to oversee the safety of
marketed dietary supplements.
Although there are no adverse event reporting requirements for
manufacturers of dietary supplements, there are such requirements for
many other products regulated by FDA. Various types of adverse events
associated with the use of human drugs and biologics,[Footnote 18]
animal drugs, animal feeds containing animal drugs, medical devices,
infant formulas, and radiation-emitting devices must be reported to
FDA. In addition to dietary supplements, other products regulated by
FDA that do not require adverse event reporting are foods, cosmetics,
and color additives. (See app. III for details about adverse event
reporting requirements.):
Voluntary adverse event reporting systems can be valuable tools for
identifying potentially serious health issues that may be associated
with the use of a product and for maintaining ongoing surveillance. FDA
has used adverse event reports to identify issues for further
investigation and, as we previously reported, it has used adverse event
reports to help identify dietary supplements for which evidence of harm
existed, and has issued warnings and alerts for dietary
supplements.[Footnote 19] However, by themselves, adverse event
reporting systems generally are not sufficient to establish that a
product caused the reported health problem. As we noted in 1999, all
voluntary surveillance systems, including FDA's adverse event reporting
system, have certain weaknesses.[Footnote 20] These include
underreporting, reporting biases, difficulties estimating population
exposure, and poor report quality. For example, the Department of
Health and Human Services (HHS) Inspector General reported that a study
commissioned by FDA estimated that FDA receives reports for less than
1 percent of adverse events associated with dietary supplements.
[Footnote 21] In addition, it is often difficult to rule out other
possible explanations for the event; for example, the event may have
been caused by preexisting medical conditions, or by the concurrent use
of prescription drugs, over-the-counter drugs, or other supplements.
For these reasons, data from adverse event reports alone cannot be used
to determine if the occurrence of a symptom among product users is
unusually high.
Between August and December 2002, Metabolife International released
copies of 15,948 pages of documents that it said contained call records
that reported adverse events associated with Metabolife 356 that the
company had received from May 1997 through July 2002. Some pages of
call records contained information about more than one call while
others did not contain reports of adverse events. Some pages were
photocopies or duplicates of other pages.
Consumer Information in Call Records Was Limited, Sometimes Difficult
to Interpret, and Not Consistently Recorded:
The information about reported adverse events in the 14,684 health-
related call records we examined was limited. Most of the call records
we reviewed did not completely record demographic or medical history
information about the consumer. Information about age, sex, weight,
height, the amount of product used, and the duration of use was
frequently not recorded. Handwritten call records were difficult to
read and interpret. Information was often inconsistent across different
versions of the same call record.
The call records contained limited information about reported adverse
events and consumers. In some cases the evidence for a report of an
adverse event was a single health-related word on the call record, such
as "seizure" or "stroke." In addition, demographic and medical history
information was not consistently recorded in the call records. Most of
the call records we reviewed did not record information about the
consumers' sex, age, weight, or height. Eighty-eight percent of the
call records did not record at least one of these variables. In
addition, information about the amount of Metabolife 356 used and the
duration of use was not recorded in 27 and 33 percent of the call
records, respectively. (See table 1.) The absence of this information
makes it difficult to assess whether the call records represent a
signal of health concerns related to the consumption of Metabolife
356.[Footnote 22]
Table 1: Percentage of Call Records in Which Consumer and Response
Details Were Recorded:
Type of detail recorded: Age; Percentage of call records: with
information: (n=14,684): 42%.
Type of detail recorded: Sex[A]; Percentage of call records: with
information: (n=14,684): 41.
Type of detail recorded: Weight; Percentage of call records: with
information: (n=14,684): 62.
Type of detail recorded: Height; Percentage of call records: with
information: (n=14,684): 34.
Type of detail recorded: Amount of Metabolife used; Percentage of call
records: with information: (n=14,684): 73.
Type of detail recorded: Duration of use; Percentage of call records:
with information: (n=14,684): 67.
Type of detail recorded: Medical history; Percentage of call records:
with information: (n=14,684): 45.
Source: GAO.
Note: Analysis of 14,684 health-related call records provided by
Metabolife International. Where information was not recorded, we do not
know if Metabolife International did not record information in the call
records or if the caller did not provide the information.
[A] Metabolife International likely has more information about
consumers' sex than we did because in many cases the company had access
to the names of consumers to help make that determination. Consumers'
names had been removed from the records Metabolife International
provided us to protect consumer privacy.
[End of table]
Both the amount of product used and duration of use were recorded for
60 percent of the calls reporting adverse events. Relatively few of
these records involved consumers who reported taking too much
Metabolife 356 or using it for too long a period. Specifically, among
call records containing information on the amount of product used or
duration of use, 99 and 91 percent of consumers, respectively, reported
using the product within the guidelines recommended on the label.
The format of the call records varied from brief handwritten notes to
typed notations to printed versions of a form used by Metabolife
International. In general, less information was recorded for the one-
third of call records that were handwritten than all other types of
records. For example, calls recorded on a typed form more frequently
recorded additional information such as recommendations by Metabolife
International to discontinue Metabolife 356 (62 percent) or contact a
doctor (54 percent) than did those on handwritten forms (13 percent and
8 percent, respectively).
Further, it was often difficult to read handwritten call records. We
could not always determine how many calls were reported on a single
page since there was rarely a clear delineation of events. Because
handwritten call records did not follow a template, we were unable to
determine if some information was medical history or symptom
information, or if a number was a weight, heart rate, or blood
pressure.
Information in call records was sometimes inconsistent. Where duplicate
call records were available, information about consumers and their
usage of Metabolife 356 was sometimes presented differently in the
different records of the same consumer call. In addition, Metabolife
International officials told us that its nurses sometimes used several
different terms to document the same type of adverse event.
Metabolife International Had Thousands of Call Records Reporting
Adverse Events Associated with Metabolife 356:
We found that 14,684 of the Metabolife International call records
reported at least one adverse event. Ninety-two of these were for the
serious adverse events identified in the proposed label warning for
dietary supplements containing ephedra that FDA announced on February
28, 2003. Other adverse events reported included significant elevation
of blood pressure, abnormal heart rhythm, loss of consciousness, and
systemic rash. We cannot establish that any of the reported adverse
events were caused by the use of Metabolife 356.
Reports of Adverse Events Identified as Serious in FDA's Proposed Label
Warning:
We counted 92 reports of heart attack, seizure, stroke, or death--the
serious adverse events identified in FDA's proposed label warning for
dietary supplements containing ephedra (see table 2).[Footnote 23]
Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke,
Seizure, or Death:
Type of adverse event: Heart attack; Number[A]: 18.
Type of adverse event: Stroke; Number[A]: 26.
Type of adverse event: Seizure; Number[A]: 43.
Type of adverse event: Death; Number[A]: 5.
Source: GAO.
Note: Analysis of 14,684 health-related call records provided by
Metabolife International.
[A] The counts do not represent unique consumers because a single call
record may have more than one complaint and because some consumers
called the Metabolife health information phone line more than once.
[End of table]
Other Adverse Events:
In its 1997 proposed rule on dietary supplements, FDA also identified
other types of adverse events as serious or potentially serious. Table
3 shows our counts for almost all such events.[Footnote 24] The serious
and potentially serious types of adverse events described in FDA's June
4, 1997, proposed rule were reported to the agency prior to June 7,
1996. FDA officials report that some other types of adverse events not
included in the table may be considered serious or potentially serious
but had not been reported to FDA during the time period considered by
its proposed rule.
Table 3: Metabolife 356 Call Records Reporting Adverse Events Described
as Serious or Potentially Serious in FDA's 1997 Proposed Rule:
Category of event: Cardiovascular.
Event reported: Cardiovascular: Category of event: Chest pain;
Number[A]: Cardiovascular: Category of event: 433.
Event reported: Cardiovascular: Category of event: Significant
elevation in blood pressure[B]; Number[A]: Cardiovascular: Category of
event: 93.
Event reported: Cardiovascular: Category of event: Abnormal heart
rhythm (alternative names for this event include dysrhythmia,
ventricular tachycardia, ventricular fibrillation, atrial
fibrillation, atrial flutter)[C]; Number[A]: Cardiovascular: Category
of event: 31.
Event reported: Cardiovascular: Category of event: Cardiomyopathy;
Number[A]: Cardiovascular: Category of event: 7.
Event reported: Cardiovascular: Category of event: Cardiac arrest;
Number[A]: Cardiovascular: Category of event: 4.
Event reported: Cardiovascular: Category of eventNervous system:
Angina; Number[A]: Cardiovascular: Category of eventNervous system: 3.
Category of event: Nervous system.
Event reported: Cardiovascular: Category of event: Loss of
consciousness; Number[A]: Cardiovascular: Category of event: 47.
Event reported: Cardiovascular: Category of event: Psychosis;
Number[A]: Cardiovascular: Category of event: 7.
Event reported: Cardiovascular: Category of event: Altered
consciousness (including disorientation or confusion); Number[A]:
Cardiovascular: Category of event: 4.
Event reported: Cardiovascular: Category of event: Suicidal; Number[A]:
Cardiovascular: Category of event: 3.
Event reported: Cardiovascular: Category of event: Vestibular (inner
ear) disturbance; Number[A]: Cardiovascular: Category of event: 2.
Event reported: Cardiovascular: Category of event: Severe depression;
Number[A]: Cardiovascular: Category of event: 2.
Event reported: Cardiovascular: Category of eventOther: Mania;
Number[A]: Cardiovascular: Category of eventOther: 1.
Category of event: Other.
Event reported: Cardiovascular: Category of event: Systemic rash;
Number[A]: Cardiovascular: Category of event: 181.
Event reported: Cardiovascular: Category of event: Urinary infection;
Number[A]: Cardiovascular: Category of event: 110.
Event reported: Cardiovascular: Category of event: Urinary retention;
Number[A]: Cardiovascular: Category of event: 72.
Event reported: Cardiovascular: Category of event: Elevations of liver
function tests; Number[A]: Cardiovascular: Category of event: 54.
Event reported: Cardiovascular: Category of event: Prostatitis;
Number[A]: Cardiovascular: Category of event: 24.
Event reported: Cardiovascular: Category of eventCardiovascular:
Hepatitis; Number[A]: Cardiovascular: Category of eventCardiovascular:
1.
Source: GAO.
Note: Analysis of 14,684 health-related call records provided by
Metabolife International.
[A] The counts do not represent unique consumers because a single call
record may have more than one complaint and because some consumers
called the Metabolife health information phone line more than once.
[B] We used the MEDLINE Plus Medical Encyclopedia to define significant
elevations in blood pressure as a measurement of greater than 160
millimeters of mercury systolic or 100 millimeters of mercury
diastolic. This count does not include call records that mentioned
"high blood pressure" or "elevated blood pressure" without specifying
these levels.
[C] Alternative names for abnormal heart rhythm were determined using
the MEDLINE Plus Medical Encyclopedia (www.nlm.nih.gov/medlineplus/
encyclopedia.html) (downloaded December 2002 through February 2003).
[End of table]
In addition, the 14,684 call records with health-related reports
presented a broad range of types of adverse events. Many of the call
records contained reports of jitters, insomnia, hair loss, bruising,
menstrual irregularities, and sexual dysfunction, as well as vague
references to events such as "side effect" or "felt sick." Some
reported blood in stool, blood in urine, or blood clots. There were
also some reports of visits to emergency departments and hospital
admissions. Some call records contained reports of diseases such as
pulmonary embolus (a blockage of an artery in the lungs), multiple
myeloma, and inflammation of heart tissue.
Causal Role of Metabolife 356 Cannot Be Established:
We cannot establish that any of the adverse events reported in the
Metabolife International call records were caused by the use of
Metabolife 356. As we noted earlier, adverse event reports by
themselves are generally not sufficient to establish that a health
problem was caused by the use of a particular product. For example, for
many adverse event reports it is difficult to rule out other possible
explanations for the event--the event may have been caused by
preexisting medical conditions, or by the concurrent use of
prescription drugs, over-the-counter drugs, or other dietary
supplements. In addition, the limited information available in the
Metabolife International call records means that we cannot confirm that
a particular adverse event occurred, much less identify a specific
cause for it.
Findings of Different Reviews of Metabolife International Call Records
Vary:
All the reviews of the Metabolife International call records, including
ours, counted reports of serious adverse events. None of the reviews
reported identical tabulations of these events. For the set of adverse
events that Metabolife International counted--heart attack, stroke,
seizure, death, and cardiac arrest--our counts are similar to those of
the other reviews (see table 4). In total, we counted 96 such events,
Metabolife International counted 78, and the counts of the other
reviews ranged from 65 to 107.
Table 4: Number of Call Records Containing Reports of Heart Attack,
Stroke, Seizure, Death, or Cardiac Arrest Reported in Reviews of
Metabolife International Call Records:
Events: Heart attack; GAO: 18; Metabolife: 16[F]; Karch[A]: 17;
Mozayani[B]: 13; Molgaard[C]: 13; Minority Staff, Committee on
Government Reform, House of Representatives[D]: 20; RAND[E]: 22.
Events: Stroke; GAO: 26; Metabolife: 20; Karch[A]: 24; Mozayani[B]: 19;
Molgaard[C]: 13; Minority Staff, Committee on Government Reform, House
of Representatives[D]: 24; RAND[E]: 31.
Events: Seizure; GAO: 43; Metabolife: 35; Karch[A]: 40; Mozayani[B]:
52; Molgaard[C]: 36; Minority Staff, Committee on Government Reform,
House of Representatives[D]: 40; RAND[E]: 46.
Events: Death; GAO: 5; Metabolife: 3; Karch[A]: 2; Mozayani[B]: 4;
Molgaard[C]: 3; Minority Staff, Committee on Government Reform, House
of Representatives[D]: 3; RAND[E]: 5.
Events: Cardiac arrest; GAO: 4; Metabolife: 4[F]; Karch[A]: 4;
Mozayani[B]: 5; Molgaard[C]: NC; Minority Staff, Committee on
Government Reform, House of Representatives[D]: NC; RAND[E]: 3.
Events: Total; GAO: 96; Metabolife: 78; Karch[A]: 87; Mozayani[B]: 93;
Molgaard[C]: 65; Minority Staff, Committee on Government Reform, House
of Representatives[D]: 87; RAND[E]: 107.
Source: GAO and others.
Notes: "NC" indicates that these types of events were not counted by
these reviews. The counts do not represent unique consumers because a
single call record may have more than one complaint and because some
consumers called the Metabolife health information phone line more than
once.
[A] Steven B. Karch, An Analysis of Metabolife 356 HealthLine Contacts
(August 2002), www.ephedrafacts.com/metabolife.html (downloaded Dec.
12, 2002), and An Analysis of an Additional 3268 HealthLine Records
(Jan. 17, 2003).
[B] Ashraf Mozayani, Analysis of Metabolife 356 Health Line Reports
(August 2002), and Supplemental Report of Analysis of Metabolife 356
Health Line Reports (January 2003).
[C] Craig A. Molgaard, Epidemiologic Assessment of Health Line Reports
about a Dietary Supplement (August 2002), and An Analysis of Additional
HealthLine Records (Jan. 17, 2003).
[D] Minority Staff Report, Special Investigations Division, Committee
on Government Reform, House of Representatives, Adverse Event Reports
from Metabolife (October 2002), www.house.gov/reform/min (downloaded
Dec. 11, 2002). This review did not include at least 1,480 pages of
call records Metabolife International later made available to us and
other reviews.
[E] Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues,
Ephedra and Ephedrine for Weight Loss and Athletic Performance
Enhancement: Clinical Efficacy and Side Effects. Evidence Report/
Technology Assessment No. 76 (Prepared by Southern California Evidence-
based Practice Center, RAND, under Contract No. 290-97-0001, Task Order
No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for
Healthcare Research and Quality, February 2003.
[F] Metabolife International provided to us a list of call records it
considered to report cardiac events. Because the other reviews counted
heart attacks and cardiac arrests separately, we examined the events
that Metabolife International classified as cardiac events to
categorize them as cardiac arrest or heart attack.
[End of table]
There are several possible reasons for the slightly different counts of
serious adverse events in the different reviews. First, the call
records themselves are often difficult to understand and interpret.
Second, not all of the reviews included the same set of call records,
both because some were completed before all of the Metabolife
International call records were released and because the reviews
adopted different procedures for identifying and discarding duplicate
records. Third, the reviews used different definitions of particular
events or established different thresholds for categorizing a
particular event. For example, we included reports of "convulsions" in
our count of seizures, while some other reviews may not have.
Specifically, the counts we report in table 4 for our review and the
reviews by Metabolife International and Karch include reports of
convulsions, while it is not clear if the other reviewers' counts did.
Similarly, we did not count as a report of a heart attack a call record
that reported "heart attack?", while at least one other review did.
Summary:
The information in the Metabolife International call records was
limited, sometimes difficult to understand and interpret, and consumer
information was not consistently recorded. Most call records contained
only limited information about a consumer and the event being reported,
and handwritten records were difficult to read and understand. We
categorized 14,684 call records from Metabolife International as
containing reports of at least one adverse event associated with
Metabolife 356. We found that there were 92 reports of the types of
serious adverse events identified in FDA's proposed label warning for
dietary supplements containing ephedrine alkaloids. All of the reviews
of the Metabolife International call records, including ours, counted
reports of serious adverse events, although none of the reviews
reported identical results. We counted 96 reports of the types of
events counted by Metabolife International--heart attack, stroke,
seizure, death, and cardiac arrest--and the counts of the other reviews
ranged from 65 to 107.
Agency and Metabolife International Comments and Our Evaluation:
We provided a draft of this report to FDA and Metabolife International
for their review. FDA asked us to clarify that it has not conducted its
own review of the Metabolife International call records, that we only
reviewed reports of adverse events contained in the Metabolife
International call records, and that we did not review other reports of
adverse events among users of Metabolife 356 that have been received by
FDA. In addition, FDA pointed out that, when combined with other
information, adverse event reports can help establish that an adverse
event was caused by a particular health product. FDA's comments are
included as appendix IV. FDA also provided technical comments, which we
incorporated as appropriate.
In its comments, Metabolife International was primarily concerned about
our use of the term "adverse events" to describe the health-related
complaints that were reported in the call records we reviewed. We
believe that our use of the term is accurate and consistent with its
use by FDA and others in the field. Metabolife International also
wanted us to clarify that, while it did identify some call records as
containing references to types of specific adverse events that have
been categorized as serious by others, it has not identified any call
records as reporting "serious adverse events." We have made revisions
so as not to imply that Metabolife International labeled these events
as serious adverse events. Metabolife International also made other
comments, which we incorporated as appropriate.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
from the date of this report. At that time, we will send copies to the
Secretary of HHS, the Commissioner of FDA, and others who are
interested. We will also provide copies to others upon request. In
addition, the report will be available at no charge on GAO's Web site
at http://www.gao.gov.
If you or your staff have any questions, please contact me at (202)
512-7119. Another contact and major contributors to this report are
listed in appendix V.
Sincerely yours,
Marcia Crosse
Acting Director, Health Care--Public Health and Science Issues:
Signed by Marcia Crosse:
[End of section]
Appendix I: Scope and Methodology for Categorizing the Call Records:
We reviewed call records and supplementary information voluntarily
provided to us by Metabolife International to (1) determine the extent
to which information was comprehensive, interpretable, and consistently
recorded in the call records, and (2) count the number of call records
reporting health-related problems associated with Metabolife 356, and
how many of them were serious. During our review we removed duplicate
call records and records that did not report health-related events. For
each record we recorded demographic information about the individual
consumer, other details about the call record and the consumer, and
categorized the reported events.
Call Records And Supplementary Information:
From August 2002 through December 2002, Metabolife International
voluntarily provided to us 15,948 pages[Footnote 25] of documentation
relating to reports of adverse events among consumers of Metabolife
356. Most of these records were from calls made to the company's
consumer health information phone line from May 1997 through July 2,
2002.[Footnote 26] Other records included e-mail messages and letters
that had been sent to the company. Nurses on the staff of Metabolife
International documented the calls to the consumer HealthLine in a
variety of formats. The records included handwritten notes on a page,
typed and handwritten letters, forms with handwritten entries, e-mails,
and printed versions of records that had been entered into a database
developed by Metabolife International. Many kinds of forms were used to
record calls, ranging from simple forms with few spaces or check boxes
to full-page forms with multiple boxes for consumer and event-related
information. Metabolife International officials told us that health
complaints that were noted on product return forms that it received
were not in the call records provided to us.
Metabolife International also provided to us copies of 46 redacted
medical records and a list of corresponding call records. After
reviewing these records we found 8 that were not associated with other
call records. Five of these records contained enough information to
determine the nature of the adverse event and were coded in the same
way as other call records. The other medical records were used as
additional sources of information for documenting the events and
consumer information reported in their corresponding records.
While most pages of call records contained information about a single
call, some included information about multiple calls on the same page,
other calls spanned multiple pages, and some did not include any report
of adverse events. Records that spanned multiple pages were often
letters to the company, some of which were sent with additional
information (such as medical bills). Records that did not report an
adverse event were either incomplete printouts of other records from
the database, product questions, complaints about not losing weight, or
reports of consumer satisfaction. As a result, the number of pages of
call records that we received from Metabolife International does not
correspond to the number of reports of adverse events.
The call records and medical records we received were redacted by
Metabolife International to remove personal identifying information
such as name, phone number, address, fax number, and e-mail address to
protect consumer privacy. Metabolife International officials told us
that in the process of redacting the records, some relevant adverse
event information was also inadvertently removed.
Exclusion of Duplicate and Nonhealth-Related Call Records:
Metabolife International officials told us that there were duplicate
call records in the set of call records they provided to us. Some
duplicate reports were photocopies of the same call record. In other
cases, there were multiple versions of the same call record in
different formats. Metabolife International officials reported these
multiple versions were the result of nurses taking handwritten notes
and later entering the same information directly into a database
established in September 1999.
Metabolife International gave us lists of those call records it
believed to be duplicates. Over the course of our review, it identified
more than
2,200 records for which there were at least one duplicate. Metabolife
International officials reported that they identified the duplicates on
the basis of the name of the consumer. Duplicates may have included
subsequent calls about different events from the same individual. We
examined the duplicate call records identified in the lists provided
throughout our review by Metabolife International. Because identifying
information was removed, we examined the date of the call record,
demographic information about the consumer (such as age, height,
weight, the amount of the product used, and duration of use), and event
details to determine if they were duplicate records. Where this
information was the same or similar, we considered the records to be
duplicates and excluded the extra records from our review. We did,
however, include in our analysis any additional information that
appeared on the duplicate records. For example, if one version included
height and another weight, we recorded both of these.
We agreed with Metabolife International that most of the more than
2,200 records it identified as duplicates were, in fact, duplicates.
However, we did not exclude records that represented multiple calls
from the same consumer for different events if the dates on the call
records differed by more than a few days or the symptoms were clearly
different. During the course of our review, we also identified
duplicates not previously identified by Metabolife International,
including photocopied records and records that used identical language
in event descriptions. We do not know if all duplicate call records
were identified.
We also excluded from our analysis records in which there was no health
complaint or the health complaint could not be clearly determined. We
also excluded call records that reported third-hand knowledge of
adverse events (such as a friend of a friend who experienced an adverse
event). In addition, we did not count call records that clearly
referred to nutrition bars or other ephedra-free products manufactured
by Metabolife International. In total, we determined that the 15,948
pages of documentation provided by Metabolife International contained
14,684 separate health-related call records.
Classification of Records and Data Entry Procedures:
We classified the adverse events reported in each call record and
entered the appropriate codes into a database. We classified the
reported adverse events as either one of the events FDA identified as
serious in its
February 28, 2003, announcement regarding a proposed label warning for
dietary supplements containing ephedra (heart attack, stroke, seizure,
or death) or as an other adverse event. All serious events reported
within a particular call record were counted. Therefore an individual
could have reported multiple serious adverse events, though this
happened in few records. For other adverse events, we documented
whether the call record reported one or more adverse events. We did not
count the number of reports for every type of event reported in the
record. We did, however, count the number of all but 1 of the 24 other
types of adverse events that were described as serious or potentially
serious in FDA's June 4, 1997, proposed rule on dietary supplements
containing ephedrine alkaloids.[Footnote 27] The set of events
identified by FDA in the proposed rule is not an exhaustive list of the
adverse events that may be associated with the use of dietary
supplements containing ephedrine alkaloids. FDA officials told us that
some other types of adverse events may be considered serious or
potentially serious but had not yet been reported to FDA during the
time period considered by its proposed rule.
We did not apply medical judgment in the process of identifying and
classifying events. Our classification of events in the call records
was based solely on the words and phrases therein; we did not diagnose
a consumer's condition or otherwise interpret the information
presented. For example, if a report said "poss. heart attack," "heart
attack symptoms," or "heart attack?", we did not classify it as a heart
attack since it was not clear that a heart attack was reported. Also,
while we counted "blood pressure 210/120" as an instance of
significantly elevated blood pressure because it reported measurements
greater than 160 systolic or 100 diastolic, we did not place in the
same category call records that reported only "high blood pressure"
because they did not contain the specific measurements needed for that
determination. We used MEDLINE Plus Medical Encyclopedia definitions to
further clarify individual symptoms related to these
categories.[Footnote 28] We also did not attempt to determine whether
Metabolife 356 caused the reported adverse events.
[End of section]
Appendix II: Metabolife 356 Label:
[See PDF for image]
Source: Metabolife International, February 12, 2003.
[End of section]
Appendix III: Requirements for Reporting Adverse Events to FDA:
Adverse events about many types of products regulated by FDA are
required to be reported to the agency. Such products include human
drugs, biologics, animal drugs, animal feeds containing animal drugs,
medical devices, infant formulas, and radiation-emitting devices. There
are, however, no reporting requirements for adverse events associated
with other products regulated by FDA, including food and food
additives, dietary supplements, cosmetics, or color additives. (See
table 5.):
Table 5: Requirements for Reporting Adverse Events to FDA:
Product: Human drugs (including over-the-counter drugs) with approved
New Drug Application (NDA) or Abbreviated New Drug Application
(ANDA)[A]; Adverse events that must be reported to FDA: Serious and
unexpected adverse drug experiences from all sources (domestic and
foreign).[B]; Who reports: NDA and ANDA applicants, and any person
whose name is on the label of an approved drug as manufacturer, packer,
or distributor (" nonapplicants").; When reported: As soon as possible
but within 15 calendar days. Nonapplicants may, instead, submit reports
to applicants within 5 calendar days.
Adverse events that must be reported to FDA: New information obtained
as result of follow-up investigation of earlier reports.; Who reports:
Same as above.; When reported: Within 15 calendar days of receipt of
new information or as requested by FDA. Nonapplicants may, instead,
submit reports to applicants within 5 calendar days.
Adverse events that must be reported to FDA: Adverse experiences that
occur domestically and that are serious and expected or not serious
(expected or unexpected).; Who reports: NDA and ANDA applicants.; When
reported: At quarterly intervals for the first 3 years after approval
and then annually or at different times upon written notice by FDA.
Adverse events that must be reported to FDA: Serious and unexpected
adverse drug experiences described in scientific literature as case
reports or as the result of a formal clinical trial, or from or during
postmarketing studies where the applicant concludes that there is a
reasonable possibility that drug caused reaction.[B]; Who reports: NDA
and ANDA applicants and nonapplicants.; When reported: Within 15
calendar days.
Product: Human drugs without approved NDAs/ANDAs[C]; Adverse events
that must be reported to FDA: Serious and unexpected adverse drug
experiences from all sources (domestic and foreign).[B]; Who reports:
Any person whose name is on the label as a manufacturer, packer, or
distributor; and the manufacturer even if its name does not appear on
the label, when it receives adverse drug experience reports directly
from a packer or distributor.; When reported: As soon as possible but
within 15 calendar days; packers and distributors may, instead, submit
reports to manufacturers within
5 calendar days.
Adverse events that must be reported to FDA: Serious and unexpected
adverse drug experiences from a postmarketing study where there is
reasonable possibility that drug caused reaction.[B]; Who reports: Same
as above.; When reported: Same as above.
Adverse events that must be reported to FDA: New information obtained
as result of follow-up investigation of 15-day alert reports.; Who
reports: Same as above.; When reported: Within 15 calendar days of
obtaining the information or as requested by FDA.
Product: Biologics[D]; Adverse events that must be reported to FDA:
Serious and unexpected adverse experiences from all sources described
in scientific literature, or described in postmarketing clinical
studies where there is a reasonable possibility product caused
reaction.[B]; Who reports: Licensed manufacturerse and manufacturers,
packers, distributors, or other manufacturing participants whose name
appears on the label.; When reported: As soon as possible but no later
than 15 calendar days. Packers, distributors, and other nonlicensees
required to report may submit reports to licensed manufacturers within
5 calendar days.
Adverse events that must be reported to FDA: New information obtained
as a result of follow-up of 15-day alert reports.; Who reports: Same as
above.; When reported: Within 15 days of receipt of new information or
as requested by FDA. Packers, distributors, and other unlicensed firms
required to report may submit reports to licensees within 5 calendar
days.
Adverse events that must be reported to FDA: Adverse experiences that
are expected or nonserious.; Who reports: Licensed manufacturers.; When
reported: At quarterly intervals for the first 3 years after license
approval and then annually or at different times upon written notice by
FDA.
Adverse events that must be reported to FDA: Certain reactions
associated with administration of vaccines listed in 42 U.S.C. §300aa-
14.; Who reports: Vaccine manufacturers and health care providers.;
When reported: Within 7 days of the administration of listed vaccines
or as specified.[F].
Adverse events that must be reported to FDA: Fatality resulting from
blood collection or transfusion.[G]; Who reports: Collecting facilities
in the event of donor reaction; facilities performing compatibility
tests in the event of transfusion reaction.; When reported: As soon as
possible by telephone, facsimile, express mail, or electronic
transmission with a written report to follow within 7 days.
Product: Animal drugs[H]; Adverse events that must be reported to FDA:
Unexpected side effects, injury, toxicity, sensitivity, reaction;
unexpected incidence or severity, or unusual failure to exhibit
expected pharmacological activities.; Who reports: Applicants for New
Animal Drug Application (NADA) or Abbreviated New Animal Drug
Application (ANADA), including those whose name appears on the labeling
as a manufacturer, packer, distributor, or who are engaged in
manufacturing, processing, packing, or labeling of drug.; When
reported: As soon as possible but within 15 working days of receipt by
the applicant.
Adverse events that must be reported to FDA: Mix-up in new animal drug
or its labeling with another article, bacteriological or significant
physical or other change or deterioration in the drug, or failure to
meet specifications.; Who reports: Same as above.; When reported:
Immediately (generally within
3 days).
Product: Animal feeds bearing or containing animal drugs[I]; Adverse
events that must be reported to FDA: Mix-up with another drug or its
labeling with another article; bacteriological or significant chemical,
physical, or other change or deterioration in the drug; or failure to
meet specifications.; Who reports: NADA and ANADA applicants.; When
reported: Immediately (generally within
3 days).
Adverse events that must be reported to FDA: Information concerning
unexpected side effect, injury, toxicity, sensitivity reaction, any
unexpected incidences or severity, or unusual failure to exhibit
expected pharmacological activities.; Who reports: Same as above.; When
reported: As soon as possible but within 15 working days of receipt by
the applicant.
Product: Medical devices[J]; Adverse events that must be reported to
FDA: Device-related deaths or serious injuries.; Who reports: Device
user facilities.[K]; When reported: Within 10 work days of receiving
relevant information; annual reports must summarize all reported
events.
Adverse events that must be reported to FDA: Device-related deaths or
serious injuries.; Who reports: Importers.; When reported: Within 30
days of becoming aware of event.
Adverse events that must be reported to FDA: Device-related deaths or
serious injuries.[L]; Who reports: Device manufacturers.; When
reported: Within 30 days of becoming aware of event, or within 5 days
if the event requires remedial action to prevent an unreasonable risk
of substantial harm to the public health or if FDA has made a written
request.
Adverse events that must be reported to FDA: Information that would
have had to have been reported earlier but was unknown or unavailable.;
Who reports: Same as above.; When reported: Within 1 month of receiving
information.
Product: Infant formula[M]; Adverse events that must be reported to
FDA: Possible causal connection between consumption of an infant
formula and infant death.; Who reports: Manufacturers.; When reported:
Within 15 days, conduct an investigation and notify FDA.
Product: Radiation-emitting devices[N]; Adverse events that must be
reported to FDA: Injurious or potentially injurious exposure to
radiation from nonmedical electronic products.[O]; Who reports:
Manufacturers.; When reported: Immediately.
Product: Food and food additives; Adverse events that must be reported
to FDA: No requirements to report adverse events.; Who reports: Not
applicable.; When reported: Not applicable.
Product: Dietary supplements; Adverse events that must be reported to
FDA: No requirements to report adverse events.; Who reports: Not
applicable.; When reported: Not applicable.
Product: Cosmetics; Adverse events that must be reported to FDA: No
requirements to report adverse events.; Who reports: Not applicable.;
When reported: Not applicable.
Product: Color additives; Adverse events that must be reported to FDA:
No requirements to report adverse events.; Who reports: Not
applicable.; When reported: Not applicable.
[A] 21 C.F.R. §§ 314.80, 314.98 (2002). Over-the-counter drugs are subject
to FDA's adverse event reporting requirements only to the extent they
are covered by approved NDAs or ANDAs. On March 14, 2003, FDA published
a proposed rule which includes requirements for reporting suspected
adverse events associated with drugs and biological products ("Safety
Reporting Requirements for Human Drug and Biological Products," 68 Fed.
Reg. 12406).
[B] FDA refers to these as 15-day alert reports.
[C] 21 C.F.R. § 310.305. Adverse events associated with investigational
new drugs are required to be reported under sections 312.32 and 312.33
of Title 21 of the Code of Federal Regulations. Also see FDA's proposed
rule at 68 Fed. Reg. 12406 (Mar. 14, 2003).
[D] 21 C.F.R. § 600.80. There are no reporting requirements for
manufacturers of whole blood or components of whole blood. 21 C.F.R. §
600.80(k)(1). Also see FDA's proposed rule at 68
Fed. Reg. 12406 (Mar. 14, 2003).
[E] In vitro diagnostic products are subject to the reporting
requirements for devices.
21 C.F.R. § 600.80(k)(2).
[F] 42 U.S.C. § 300aa-25(b).
[G] 21 C.F.R. § 606.170(b).
[H] 21 C.F.R. § 510.300. FDA is in the process of redrafting the
adverse event reporting rules for approved animal drugs.
[I] 21 C.F.R. § 510.301. Certain medicated items incorporated into
animal feeds are also subject to the animal drug reporting
requirements. See 21 C.F.R. § 514.80(a)(4).
[J] 21 C.F.R. pt. 803. Not all medical device adverse events must be
reported to FDA; user facilities are required to report serious
injuries to FDA only if the manufacturers are not known. 21 C.F.R. §
803.30(a)(2). Adverse events associated with devices under
Investigational Device Exemptions must be reported and summaries must
be included in applications submitted to FDA for premarket approval. 21
C.F.R. §§ 812.150, 814.20.
[K] Device user facilities do not include physician offices, school
nurse offices, and employee health units. 21 C.F.R. § 803.3(f).
[L] Manufacturers must also report to FDA if a device has malfunctioned
and such malfunction, were it to recur, would be likely to cause or
contribute to a death or serious injury. 21 C.F.R. § 803.50(a)(2).
[M] 21 C.F.R. § 106.100(k)(3). Manufacturers must promptly report to
FDA knowledge about an infant formula it has processed and that has
left its establishment if the infant formula may be adulterated or
misbranded and that may present a risk to human health. 21 C.F.R. §
106.120(b).
[N] 21 C.F.R. §1002.20.
[O] 21 C.F.R. §§ 1000.3, 1002.20. Nonmedical electronic products
include, for example, microwave ovens and infrared alarm systems. If a
product is classified as a medical device, the normal medical device
reporting requirements apply.
[End of table]
[End of section]
Appendix IV: Comments from the Food and Drug Administration:
DEPARTMENT OF HEALTH & HUMAN SERVICES:
Food and Drug Administration Rockville MD 20857:
March 20, 2003:
Marcia Crosse, Ph.D.
Acting Director, Health Care-Public Health and Science Issues United
States General Accounting Office 441 G Street, NW:
Washington, DC 20548:
Dear Dr. Crosse:
Please find the enclosed comments from the Food and Drug Administration
on the GAO draft report entitled, DIETARY SUPPLEMENTS: Review of
Reports of Adverse Events Among Users of Metabolite 356 (GAO-03-494).
The Agency provided extensive technical comments directly to your
staff.
We appreciate the opportunity to review and comment on this draft
report before its publication as well as the opportunity to work with
your staff in developing this report.
Sincerely,
Mark B. McClellan, M.D., Ph.D Commissioner of Food and Drugs:
Signed for Mark B. McClellan
Enclosure:
General Comments by the Department of Health and Human Service's Food
and Drug Administration (FDA) on General Accounting Office's (GAO)
Draft Report DIETARY SUPPLEMENTS Review of Reports of Adverse Events
among Users of Metabolife 356 (GAO-03-494):
FDA appreciates the opportunity to comment on GAO's draft report which
focuses additional attention on the area of adverse event reporting
associated with dietary supplements.
We have a few general comments regarding the overall report, as
follows:
The draft report implies that FDA conducted its own review and analyses
of the adverse event reports submitted by Metabolife. This is not
accurate.
There are multiple sets of adverse event reporting systems and
databases related to dietary supplements containing ephedrine alkaloids
and Metabolife and multiple databases. The GAO report references two
different reporting systems (FDA's and Metabolife's) and discusses
various interpretations of data subsets from these reporting systems
(RAND, Minority House staff, etc.). The draft report is not
sufficiently clear about which subset of data was used for this review.
FDA encourages GAO to make additional clarifications regarding these
systems and databases in the final report.
We conclude with our concern about authoritative statements made
against the use of adverse events to prove, determine, or establish
causality. While it may be true that causality can only rarely be
definitively established from a reported adverse event, this does not
mean that causality can never be established in an individual adverse
event. Aggregated adverse events can not be used to establish risks in
a population because this requires more complete and accurate
information about the size of population exposed to a particular agent,
and the number of individuals experiencing a particular type of adverse
event (in exposed and non-exposed persons); for these reasons,
aggregated adverse events are used to signal a problem that requires
further study.
Statements to the effect that "adverse event reports are not sufficient
on their own to definitively establish causality" while technically
true, are not an adequate reflection of current scientific standards
for adverse event assessment. With enough supporting evidence, such as
supporting medical documents, dechallenge, rechallenge, temporality,
biological plausibility, dose response, etc., a causal association may
be determined.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Martin T. Gahart, (202) 512-3596:
Acknowledgments:
Carolyn Feis Korman, Chad Davenport, Julian Klazkin, and Roseanne Price
also made major contributions to this report.
FOOTNOTES
[1] It has been estimated that 12 million Americans consumed dietary
supplements with ephedra in 1999 (C. A. Haller and N. L. Benowitz,
"Adverse Cardiovascular and Central Nervous System Events Associated
with Dietary Supplements Containing Ephedra Alkaloids," The New England
Journal of Medicine, vol. 343, no. 25 (2000)).
[2] See 68 Fed. Reg. 10417 (Mar. 5, 2003). FDA also announced that it
is reopening the comment period for its June 4, 1997, proposed rule,
"Dietary Supplements Containing Ephedrine Alkaloids" (62 Fed. Reg.
30678).
[3] There is no information available about the extent to which reports
of particular adverse events may have been reported to both FDA and
Metabolife International.
[4] The redaction consisted primarily of the removal of personal
identifying information (such as names, phone numbers, addresses, and
e-mail addresses) to protect consumer privacy. Although data relevant
to the adverse event being reported were not supposed to be removed,
Metabolife International officials noted that such information was
occasionally accidentally removed.
[5] The number of adverse event reports does not equal the pages of
documentation because some pages contained reports of more than one
call reporting an adverse health event, some reports of adverse health
events spanned several pages, and some pages included reports not
related to negative health consequences.
[6] In addition to phone calls, some call records were letters and e-
mails sent to Metabolife International.
[7] The letter to FDA is available at http://www.fda.gov/ohrms/dockets/
dockets/98n0148/2.htm (letter from Metabolife International received
February 10, 1999) (downloaded March 24, 2003).
[8] Metabolife International has not issued a report on its review of
the call records, but provided to us a list of the calls it believed to
report heart attack, stroke, seizure, death, and cardiac arrest.
[9] Each of the consultants reviewed the first set of approximately
12,700 pages of Metabolife International records released in August
2002. Steven B. Karch, An Analysis of Metabolife 356 HealthLine
Contacts (August 2002) www.ephedrafacts.com/metabolife.html
(downloaded Dec. 12, 2002); Craig A. Molgaard, Epidemiologic Assessment
of Health Line Reports about a Dietary Supplement (August 2002); Ashraf
Mozayani, Analysis of Metabolife 356 Health Line Reports (August 2002).
After more pages of call records were made available, each of the
consultants completed updated reviews with these additional reports.
Steven B. Karch, An Analysis of an Additional 3268 HealthLine Records
(Jan. 17, 2003); Craig A. Molgaard, An Analysis of Additional
HealthLine Records (Jan. 17, 2003); Ashraf Mozayani, Supplemental
Report of Analysis of Metabolife 356 Health Line Reports (January
2003).
[10] Minority Staff Report, Special Investigations Division, Committee
on Government Reform, House of Representatives, Adverse Event Reports
from Metabolife (October 2002). www.house.gov/reform/min (downloaded
Dec. 11, 2002).
[11] Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues,
Ephedra and Ephedrine for Weight Loss and Athletic Performance
Enhancement: Clinical Efficacy and Side Effects. Evidence Report/
Technology Assessment No. 76 (Prepared by Southern California Evidence-
based Practice Center, RAND, under Contract No. 290-97-0001, Task Order
No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for
Healthcare Research and Quality, February 2003.
[12] Metabolife International officials identified call records they
believed were duplicates of each other. We reviewed the relevant call
records to determine which were duplicates. Call records identified by
Metabolife International officials as duplicates were either
photocopies of specific call records, multiple entries of the same call
(such as handwritten notes that were later also entered into the
database, creating two pages of call records for the same call), or
multiple calls about the same consumer describing different events. We
considered the first two instances, but not the third, to be
duplicates. We did not include in our review reports that we considered
duplicates. We also identified additional call records that were
duplicates and removed them from our review.
[13] FDA's June 4, 1997, proposed rule identified serious or
potentially serious adverse events associated with the use of ephedra
based on a review of the literature and an analysis of 600 adverse
event reports that FDA had received by June 7, 1996. See "Dietary
Supplements Containing Ephedrine Alkaloids," 62 Fed. Reg. 30678. We did
not count reports of one of the events FDA identified, "altered serum
enzymes," because the proposed rule did not specify threshold values.
[14] We required that certain words be in the call record for it to be
counted as a specific type of event. For example, for a call record to
meet the criteria for a stroke, it needed to specifically include the
word "stroke," not related terms like "stroke-like symptoms."
[15] Our findings may either overestimate or underestimate the number
and severity of adverse events. Our findings may overestimate the
number of adverse events because we accepted the events as they were
reported on the page. For example, if a call record reported a stroke,
we counted it as a stroke even though the consumer may not have
actually had a stroke. Conversely, our findings may underestimate the
number and severity of adverse events because individual adverse events
we categorized as other adverse events may collectively suggest a more
serious event. For example, we categorized a call record reporting
left-side numbness and tingling and left-side face drooping as an other
adverse event where a physician or other health professional might have
determined that these symptoms actually represented a stroke.
[16] According to Metabolife International officials, the only
ingredient change since Metabolife 356 was placed on the market was
made in early 2001, when bovine complex was removed from the product.
Some other inactive ingredients may vary by manufacturing facility.
Metabolife International officials told us that the same labels are
used for products sold in all states.
[17] FDA officials reported that the agency conducts a premarket review
of safety information for certain supplements that contain new dietary
ingredients.
[18] Biologics are any virus, therapeutic serum, toxin, antitoxin,
vaccine, blood, blood component or derivative, allergenic product, or
analogous product applicable to the prevention, treatment, or cure of a
human disease or condition. Biological products include, but are not
limited to, bacterial and viral vaccines, human blood and plasma and
their derivatives, and certain products produced by biotechnology, such
as interferons and erythropoietins.
[19] U.S. General Accounting Office, Health Products for Seniors:
"Anti-Aging" Products Pose Potential for Physical and Economic Harm,
GAO-01-1129 (Washington, D.C.: Sep. 7, 2001).
[20] U.S. General Accounting Office, Dietary Supplements: Uncertainties
in Analyses Underlying FDA's Proposed Rule on Ephedrine Alkaloids, GAO/
HEHS/GGD-99-90 (Washington, D.C.: July 2, 1999).
[21] HHS Office of Inspector General, Adverse Event Reporting for
Dietary Supplements: An Inadequate Safety Valve, OEI-01-00-00180
(Washington, D.C.: April 2001).
[22] We previously reported that adverse event reports should optimally
include demographic data (GAO/HEHS/GGD-99-90). Such information is
useful for determining whether or not the adverse events reported would
be unexpected in a specific population of users, for example, heart
attacks in young adults.
[23] See 68 Fed. Reg. 10417 (Mar. 5, 2003).
[24] We counted all reports of 23 of the 24 other types of adverse
events FDA identified as serious or potentially serious in its 1997
proposed rule. We did not count reports of "altered serum enzymes"
since the proposed rule did not specify threshold values. The other
serious or potentially serious adverse events--coma, myopathies,
exfoliative dermatitis, and epididymitis--are not reported in the table
because we did not find any reports of them in the call records
provided by Metabolife International.
[25] These 15,948 pages contained 14,684 call records that we
categorized as reporting adverse events. The number of adverse event
reports does not equal the pages of documentation because some pages
contained reports of more than one call reporting an adverse health
event, some reports of adverse health events spanned several pages, and
some pages included reports not related to negative health
consequences.
[26] Metabolife International received the call records we reviewed
primarily from mid
1998 through July 2002, although 12 call records were from 1997 and
some were from early 1998.
[27] FDA's June 4, 1997, proposed rule identified serious or
potentially serious adverse events associated with the use of ephedra
based on a review of the literature and an analysis of 600 adverse
event reports that FDA had received by June 7, 1996. See "Dietary
Supplements Containing Ephedrine Alkaloids," 62 Fed. Reg. 30678.
[28] www.nlm.nih.gov/medlineplus/encyclopedia.html (downloaded
December 2002 through February 2003).
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to daily E-mail alert for newly
released products" under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548: