Skip to main content

National Institutes of Health: Better Oversight Needed to Help Ensure Continued Progress Including Women in Health Research

GAO-16-13 Published: Oct 22, 2015. Publicly Released: Oct 23, 2015.
Jump To:
Skip to Highlights

Highlights

What GAO Found

Data from the National Institutes of Health (NIH) show that more women than men were enrolled in NIH-funded clinical research for fiscal years 2005-2014, but NIH does not make certain enrollment data readily available to interested parties or examine other detailed data to identify potential challenges to enrolling women in specific research and disease or condition areas. In fiscal year 2014, for example, NIH reported that across all of the clinical research studies it funded—including phase III clinical trials, the largest studies involving human subjects— 57 percent of enrollees (16.4 million) were women. NIH collects enrollment data from individual awardees through its Institutes and Centers (IC)—which generally fund studies in different research areas—and publicly reports data on aggregate enrollment as part of its implementation of the Inclusion Policy developed to implement provisions of the NIH Revitalization Act of 1993. However, NIH does not make the IC-level enrollment data from each of the 25 ICs that report data readily available to interested parties, so that interested parties must make an effort to seek out this data. In addition, NIH does not routinely examine more detailed enrollment data, such as enrollment data organized by the disease and condition being studied. As a result, NIH is limited in its ability to identify whether women are sufficiently represented in studies in specific areas—such as cardiovascular disease—or if the agency-wide data inadvertently mask enrollment challenges. By not examining more detailed data on enrollment below the aggregate level, NIH cannot know whether it is adequately including women in all of the research it supports, in a manner consistent with its Inclusion Policy. Further, NIH's reporting and monitoring in this area is inconsistent with federal internal control standards, which call for agencies to have controls to help ensure effective information flow and effective monitoring of agency activities.

NIH requires that phase III clinical trial awardees consider whether analysis of potential differences in outcomes between women and men is needed in their studies—one of the key requirements of its Inclusion Policy; however, the agency does not maintain, analyze, or report summary data to oversee whether analysis of outcomes by sex are planned or conducted. NIH officials told GAO that they rely on peer review and program officer monitoring to ensure awardee compliance with the analysis requirement. However, NIH program officers do not have a required field in a reporting system or other means to record the information they collect to monitor awardees' analysis plans and compliance with the Inclusion Policy requirement. In addition, there is no data element in NIH's data system to indicate whether an awardee's study should or does include plans for an analysis of potential differences in research outcomes by sex. As a result, NIH lacks summary data, such as the percentage of awardees in a given year with trials designed to identify potential differences in clinical outcomes by sex. Without this summary information, NIH cannot report this information in the agency's biennial reports to Congress and other stakeholders. The lack of summary data and reporting compromises NIH's monitoring of its implementation of the Inclusion Policy and conflicts with federal internal control standards, which call for agencies to ensure the flow of information about agency activities, provide for internal and external communication, and conduct periodic monitoring. Further, it limits NIH's assurance that it is supporting research that can be used to shape improved medical practice for both women and men.

Why GAO Did This Study

Women make up over half the U.S. population, but historically have been underrepresented in clinical research supported by NIH and others. As a result, differences in the manifestation of certain diseases and reactions to treatment in women compared with men were not identified. For example, there have been instances of women having adverse effects that differed from those of men related to medications and other treatments. NIH's Inclusion Policy established requirements governing women's inclusion in its clinical research.

GAO was asked to provide information on women's participation in NIH research. Among other reporting objectives, GAO examined (1) women's enrollment and NIH's efforts to monitor this enrollment in NIH-funded clinical research; and (2) NIH's efforts to ensure that NIH-funded clinical trials are designed and conducted to analyze potential sex differences, when applicable. To do this, GAO reviewed relevant laws and policies, including the Inclusion Policy, and federal standards for internal control; reviewed and analyzed NIH enrollment data from fiscal years 2005-2014; and interviewed NIH and IC officials and other experts.

Recommendations

GAO recommends that NIH examine and report more detailed data on women's enrollment in NIH-funded studies, and collect, examine, and report data on the extent to which these studies include analyses of potential differences between women and men. NIH agreed with GAO's recommendations and plans to take action to implement them.

Recommendations for Executive Action

Agency Affected Recommendation Status
National Institutes of Health To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should make IC-level enrollment data readily available through public means, such as NIH's regular biennial report to Congress on the inclusion of women in research, or through NIH's website.
Closed – Implemented
NIH agreed with our recommendation and in response, NIH has made the most currently reported IC-level enrollment data (from fiscal years 2013-2014) on the inclusion of women in clinical research available online through NIH's RePORT website, as of August 2017. NIH also plans to continue to make the most currently reported IC-level enrollment data available on this website going forward, making this information more readily available to the public and other interested parties.
National Institutes of Health To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should examine approaches for aggregating more detailed enrollment data at the disease and condition level, and report on the status of this examination to key stakeholders and through its regular biennial report to Congress on the inclusion of women in research.
Closed – Implemented
NIH agreed with our recommendation and began discussions in November 2015 regarding conducting these types of analyses. In April and May 2016, NIH officials identified a preliminary strategy for summarizing inclusion data at the disease and condition level. In December 2019, NIH reported that it had published aggregated enrollment data at the research area, condition, and disease category, accounting for single-sex, single-race, single-ethnicity, completed, and non-completed studies earlier in the year. These data are available on the NIH RePORT website and referenced in NIH's most recent inclusion reports, which are available to Congress and on the NIH public website.
National Institutes of Health To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should ensure that program officers have a means for recording information obtained from monitoring awardees' plans for and progress in conducting analyses of potential differences in outcomes by sex.
Closed – Implemented
NIH agreed with our recommendation and, in response, beginning in fiscal year 2016, the agency incorporated questions specific to documenting this information for phase III trials into the checklist used by program officers in reviewing new awards and assessing progress on annual awards.
National Institutes of Health To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should, on a regular basis, systematically collect and analyze summary data regarding awardees' plans to conduct analyses of potential sex differences, such as the proportion of trials being conducted that intend to analyze differences in outcomes for men and women.
Closed – Implemented
In June 2020, NIH provided evidence that its Institutes and Centers (IC) have begun to systematically collect and analyze, on a regular basis, data on awardees' plans to conduct analyses of potential sex differences. They also reported that its Human Subjects System, launched in 2018, has provided an avenue to more easily collect and analyze this information on an ongoing basis.
National Institutes of Health To ensure effective implementation of the Inclusion Policy in a manner consistent with the Revitalization Act's provisions regarding the design of certain clinical trials, the NIH Director should report on this summary data and the results of this analysis in NIH's regular biennial report to Congress on the inclusion of women in research.
Closed – Implemented
In December 2021, NIH published its biennial report to Congress on the inclusion of women in research for Fiscal Years 2019-2020, which includes data on awardees' plans to conduct valid analysis by sex/gender.

Full Report

GAO Contacts

Office of Public Affairs

Topics

WomenClinical trialsHealth careMedical researchDiseasesHealth researchCompliance oversightHealth care standardsInternal controlsAdvisory committees