National Institutes of Health:

Better Oversight Needed to Help Ensure Continued Progress Including Women in Health Research

GAO-16-13: Published: Oct 22, 2015. Publicly Released: Oct 23, 2015.

Additional Materials:

Contact:

Linda Kohn
(202) 512-7114
KohnL@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Data from the National Institutes of Health (NIH) show that more women than men were enrolled in NIH-funded clinical research for fiscal years 2005-2014, but NIH does not make certain enrollment data readily available to interested parties or examine other detailed data to identify potential challenges to enrolling women in specific research and disease or condition areas. In fiscal year 2014, for example, NIH reported that across all of the clinical research studies it funded—including phase III clinical trials, the largest studies involving human subjects— 57 percent of enrollees (16.4 million) were women. NIH collects enrollment data from individual awardees through its Institutes and Centers (IC)—which generally fund studies in different research areas—and publicly reports data on aggregate enrollment as part of its implementation of the Inclusion Policy developed to implement provisions of the NIH Revitalization Act of 1993. However, NIH does not make the IC-level enrollment data from each of the 25 ICs that report data readily available to interested parties, so that interested parties must make an effort to seek out this data. In addition, NIH does not routinely examine more detailed enrollment data, such as enrollment data organized by the disease and condition being studied. As a result, NIH is limited in its ability to identify whether women are sufficiently represented in studies in specific areas—such as cardiovascular disease—or if the agency-wide data inadvertently mask enrollment challenges. By not examining more detailed data on enrollment below the aggregate level, NIH cannot know whether it is adequately including women in all of the research it supports, in a manner consistent with its Inclusion Policy. Further, NIH's reporting and monitoring in this area is inconsistent with federal internal control standards, which call for agencies to have controls to help ensure effective information flow and effective monitoring of agency activities.

NIH requires that phase III clinical trial awardees consider whether analysis of potential differences in outcomes between women and men is needed in their studies—one of the key requirements of its Inclusion Policy; however, the agency does not maintain, analyze, or report summary data to oversee whether analysis of outcomes by sex are planned or conducted. NIH officials told GAO that they rely on peer review and program officer monitoring to ensure awardee compliance with the analysis requirement. However, NIH program officers do not have a required field in a reporting system or other means to record the information they collect to monitor awardees' analysis plans and compliance with the Inclusion Policy requirement. In addition, there is no data element in NIH's data system to indicate whether an awardee's study should or does include plans for an analysis of potential differences in research outcomes by sex. As a result, NIH lacks summary data, such as the percentage of awardees in a given year with trials designed to identify potential differences in clinical outcomes by sex. Without this summary information, NIH cannot report this information in the agency's biennial reports to Congress and other stakeholders. The lack of summary data and reporting compromises NIH's monitoring of its implementation of the Inclusion Policy and conflicts with federal internal control standards, which call for agencies to ensure the flow of information about agency activities, provide for internal and external communication, and conduct periodic monitoring. Further, it limits NIH's assurance that it is supporting research that can be used to shape improved medical practice for both women and men.

Why GAO Did This Study

Women make up over half the U.S. population, but historically have been underrepresented in clinical research supported by NIH and others. As a result, differences in the manifestation of certain diseases and reactions to treatment in women compared with men were not identified. For example, there have been instances of women having adverse effects that differed from those of men related to medications and other treatments. NIH's Inclusion Policy established requirements governing women's inclusion in its clinical research.

GAO was asked to provide information on women's participation in NIH research. Among other reporting objectives, GAO examined (1) women's enrollment and NIH's efforts to monitor this enrollment in NIH-funded clinical research; and (2) NIH's efforts to ensure that NIH-funded clinical trials are designed and conducted to analyze potential sex differences, when applicable. To do this, GAO reviewed relevant laws and policies, including the Inclusion Policy, and federal standards for internal control; reviewed and analyzed NIH enrollment data from fiscal years 2005-2014; and interviewed NIH and IC officials and other experts.

What GAO Recommends

GAO recommends that NIH examine and report more detailed data on women's enrollment in NIH-funded studies, and collect, examine, and report data on the extent to which these studies include analyses of potential differences between women and men. NIH agreed with GAO's recommendations and plans to take action to implement them.

For more information, contact Linda Kohn at (202) 512-7114 or KohnL@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: In October 2015, as part of our review of the National Institute or Health's (NIH) oversight of women's participation in clinical trials, we recommended that, to ensure effective implementation of the Inclusion policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should make IC-level enrollment data readily available through public means. NIH agreed with our recommendation and began discussions in October 2015 regarding the format of a potential report with this information and how this information would be shared. NIH anticipates that this will be completed and made public in 2017.

    Recommendation: To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should make IC-level enrollment data readily available through public means, such as NIH's regular biennial report to Congress on the inclusion of women in research, or through NIH's website.

    Agency Affected: Department of Health and Human Services: Public Health Service: National Institutes of Health

  2. Status: Open

    Comments: In October 2015, as part of our review of the National Institute or Health's (NIH) oversight of women's participation in clinical trials, we recommended that, to ensure effective implementation of the Inclusion policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, we recommended the NIH Director examine approaches for aggregating more detailed enrollment data at the disease and condition level, and report on the status of this examination to key stakeholders and through its biennial report to Congress. NIH agreed with our recommendation and began discussions in November 2015 regarding conducting these types of analyses. NIH plans to provide this information in its next biennial report, which is expected to be published in 2017.

    Recommendation: To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should examine approaches for aggregating more detailed enrollment data at the disease and condition level, and report on the status of this examination to key stakeholders and through its regular biennial report to Congress on the inclusion of women in research.

    Agency Affected: Department of Health and Human Services: Public Health Service: National Institutes of Health

  3. Status: Closed - Implemented

    Comments: In October 2015, as part of our review of the National Institute of Health's (NIH) oversight of women's participation in clinical trials, we recommended that to ensure effective implementation of the agency's Inclusion Policy, the NIH Director should ensure that program officers have a means for recording information obtained from monitoring awardees' plans for and progress in conducting analyses of potential differences in outcome by sex for phase III trials. NIH agreed with our recommendation and, in response, beginning in fiscal year 2016, the agency incorporated questions specific to documenting this information for phase III trials into the checklist used by program officers in reviewing new awards and assessing progress on annual awards.

    Recommendation: To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should ensure that program officers have a means for recording information obtained from monitoring awardees' plans for and progress in conducting analyses of potential differences in outcomes by sex.

    Agency Affected: Department of Health and Human Services: Public Health Service: National Institutes of Health

  4. Status: Open

    Comments: In October 2015, as part of our review of the National Institute or Health's (NIH) oversight of women's participation in clinical trials, we recommended that, to ensure effective implementation of the Inclusion policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, we recommended the NIH Director systematically collect and analyze summary data regarding awardees' plans to conduct analyses of sex differences. NIH agreed with our recommendation and is considering several approaches to addressing this recommendation.

    Recommendation: To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should, on a regular basis, systematically collect and analyze summary data regarding awardees' plans to conduct analyses of potential sex differences, such as the proportion of trials being conducted that intend to analyze differences in outcomes for men and women.

    Agency Affected: Department of Health and Human Services: Public Health Service: National Institutes of Health

  5. Status: Open

    Comments: In October 2015, as part of our review of the National Institute of Health's (NIH) oversight of women's participation in clinical trials, we recommended that, to ensure effective implementation of the Inclusion policy, in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should make IC-level enrollment data readily available through public means. NIH agreed with our recommendation and began discussions in October 2015 regarding the format of a potential report with this information and how this information would be shared. NIH anticipates that this will be completed and made public in 2017.

    Recommendation: To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should report on this summary data and the results of this analysis in NIH's regular biennial report to Congress on the inclusion of women in research.

    Agency Affected: Department of Health and Human Services: Public Health Service: National Institutes of Health

 

Explore the full database of GAO's Open Recommendations »

Jul 7, 2016

Jun 23, 2016

Jun 20, 2016

Jun 15, 2016

Jun 9, 2016

Jun 7, 2016

Jun 3, 2016

Jun 1, 2016

May 24, 2016

Looking for more? Browse all our products here