Women's Health: NIH Has Increased Its Efforts to Include Women in
Research (Letter Report, 05/02/2000, GAO/HEHS-00-96).
Pursuant to a congressional request, GAO provided information on the
National Institutes of Health's (NIH) efforts to include women in
clinical research, focusing on: (1) the extent to which women are being
included in clinical research that NIH funds; (2) the activities and
accomplishments of the NIH Office of Research on Women's Health (ORWH)
in promoting women's health research at NIH; and (3) how much funding
NIH has allocated to research on health issues that affect women.
GAO noted that: (1) NIH has made significant progress in implementing a
strengthened policy on including women in clinical research; (2) NIH
issued guidelines to implement the 1993 NIH Revitalization Act and
conducted extensive training for scientists and reviewers; (3) the
review process for extramural research now treats the inclusion of women
and minorities as a matter of scientific merit, which affects a
proposal's eligibility for funding, and it appears that NIH staff and
researchers are working to ensure that, when appropriate, study findings
will apply to both women and men; (4) NIH implemented a centralized
inclusion tracking data system that is an important tool for monitoring
the implementation of the inclusion policy; (5) NIH has made less
progress in implementing the requirement that certain clinical trials be
designed and carried out to permit valid analysis by sex, which could
reveal whether interventions affect women and men differently; (6) more
than 50 percent of the participants in clinical research studies that
NIH funded in fiscal year (FY) 1997 were women, according to NIH; (7)
minority women were well represented, especially black and Asian and
Pacific Islander women, however, the proportion of Hispanic women
enrolled was below their proportion in the general population; (8) ORWH
has lead responsibility for ensuring that women and minorities are
included in clinical research that NIH funds; (9) its budget grew from
$9.4 million in FY 1993 to about $20 million in FY 2000; (10) ORWH uses
its budget to leverage increased funding for research on women's health
by the institutes and centers; (11) it has carried out extensive
training and education on the inclusion policy for staff members,
investigators, and institutional review boards; (12) however, ORWH has
not conducted updated training on the data tracking system to ensure
that its data are accurate and consistent; (13) NIH annually reports how
much it spends on women's health, men's health, and conditions that
affect both women and men; (14) however, the nature of scientific
inquiry makes it impossible to predict how research will affect specific
populations, especially with regard to the basic research that receives
a substantial portion of NIH resources, and GAO found inconsistencies in
the methods NIH staff use to produce its expenditure estimates; (15)
according to NIH, spending on women's health conditions grew by 39
percent between fiscal years 1993 and 1999; and (16) NIH's total
spending on diseases and conditions unique to or more prevalent in women
grew more rapidly than NIH's overall spending from FY 1993 to FY 1999.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-00-96
TITLE: Women's Health: NIH Has Increased Its Efforts to Include
Women in Research
DATE: 05/02/2000
SUBJECT: Research grants
Health research programs
Medical research
Health statistics
Women
Minorities
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO/HEHS-00-96
Appendix I: Objectives, Scope, and Methodology
30
Appendix II: Women's Health Initiative
32
Appendix III: Percentages of NIH Studies Tracked and Not Tracked,
Fiscal Year 1997
33
Appendix IV: Comments From the National Institutes of Health
34
Appendix V: GAO Contacts and Staff Acknowledgments
37
Table 1: Enrollment of Minority Women in NIH Extramural and
Intramural Research as a Percentage of Total Female
Enrollment, Fiscal Year 1997 17
Table 2: The Distribution of Fiscal Year 1997 Clinical Studies by the
Percentage of Women Participants 18
Table 3: Percentages of NIH Expenditures for Research on Women,
Men, and Both, Fiscal Years 1993-99 24
Figure 1: Aggregate Enrollment for NIH Extramural and Intramural Research,
Fiscal Year 1997 16
Figure 2: Rate of Adjusted Expenditure Growth for Selected
Conditions and Total NIH Expenditures, Fiscal Years
1993-99 26
AIDS acquired immunodeficiency syndrome
HHS Department of Health and Human Services
NIH National Institutes of Health
ORWH Office of Research on Women's Health
PHS Public Health Service
WHI Women's Health Initiative
Health, Education, and
Human Services Division
B-284039
May 2, 2000
The Honorable Tom Harkin
The Honorable Barbara A. Mikulski
The Honorable Olympia J. Snowe
United States Senate
The Honorable Henry A. Waxman
Ranking Minority Member
Committee on Government Reform
House of Representatives
In the 1980s, public health leaders and advocates drew attention to
inequities in the health research agenda and the fact that in particular
women and minorities were being excluded from research studies. Many of the
major research studies the National Institutes of Health (NIH) funded
included only men, making it uncertain whether the studies' results applied
also to women. NIH developed a policy to include women in research study
populations, but in 1990, we found that NIH had been slow and ineffective in
implementing it.1 NIH began to take more comprehensive measures to increase
research on health problems affecting women, and in 1993, the Congress
passed the NIH Revitalization Act, which provided statutory authority and
specific guidance for NIH to follow regarding research on women's health.2
In light of these developments, you asked us to assess NIH's progress in
conducting research on women's health in the past decade. In response to
your request, we have assessed NIH's progress in implementing its new
guidelines on including women in clinical research, including the
requirement that certain studies be designed to permit analysis of
differences between women and men. We also provide information about the
extent to which women are being included in clinical research that NIH
funds, review the activities and accomplishments of the NIH Office of
Research on Women's Health (ORWH) in promoting women's health
research at NIH, and provide information about how much funding NIH has
allocated to research on health issues that affect women.
To assess NIH's progress, we conducted work in several NIH institutes,
centers, and offices, including the National Cancer Institute; National
Heart, Lung, and Blood Institute; National Institute of Allergy and
Infectious Diseases; National Institute of Arthritis and Musculoskeletal and
Skin Diseases; National Institute on Aging; National Institute of Child
Health and Human Development; ORWH; and Warren Grant Magnuson Clinical
Center for intramural research. We also reviewed documents related to the
review and approval of a sample of grants and cooperative agreements from
the National Heart, Lung, and Blood Institute; National Institute of Allergy
and Infectious Diseases; and National Institute of Arthritis and
Musculoskeletal and Skin Diseases. We analyzed NIH's tracking data on the
inclusion of women and minorities in clinical research, for NIH and for
individual institutes and centers, and we analyzed NIH's data on
expenditures on women's health. (For additional information on our
methodology, see app. I.) We conducted our work between November 1999 and
April 2000 in accordance with generally accepted government auditing
standards.
In the past decade, NIH has made significant progress in implementing a
strengthened policy on including women in clinical research. It issued
guidelines to implement the 1993 NIH Revitalization Act and conducted
extensive training for scientists and reviewers. The review process for
extramural research now treats the inclusion of women and minorities as a
matter of scientific merit, which affects a proposal's eligibility for
funding, and it appears that NIH staff and researchers are working to ensure
that, when appropriate, study findings will apply to both women and men. The
intramural research program now also implements the inclusion policy. NIH
implemented a centralized inclusion tracking data system that is an
important tool for monitoring the implementation of the inclusion policy.
However, the data need to be more accurate and consistent. NIH is taking
steps to improve the system, but ongoing training is needed to ensure the
quality of the data. NIH has made less progress in implementing the
requirement that certain clinical trials be designed and carried out to
permit valid analysis by sex, which could reveal whether interventions
affect women and men differently. We are therefore recommending that the
Director of NIH ensure that NIH staff and reviewers implement this aspect of
the inclusion policy, a recommendation with which NIH concurs.
More than 50 percent of the participants in clinical research studies that
NIH funded in fiscal year 1997 were women, according to NIH. Even when
studies with female-only or male-only protocols are removed from the data,
the proportion of women enrolled exceeded 50 percent. Minority women were
well represented, especially black and Asian and Pacific Islander women;
however, the proportion of Hispanic women enrolled was below their
proportion in the general population. There was some variation in enrollment
levels by institute.
ORWH, established in 1990, has lead responsibility for ensuring that women
and minorities are included in clinical research that NIH funds. Its budget
grew from $9.4 million in fiscal year 1993 to about $20 million in fiscal
year 2000. With input from the scientific and advocacy community, ORWH
developed an agenda for research on women's health. ORWH uses its budget to
leverage increased funding for research on women's health by the institutes
and centers. It has carried out extensive training and education on the
inclusion policy for staff members, investigators, and institutional review
boards. However, ORWH has not conducted updated training on the data
tracking system to ensure that its data are accurate and consistent.
Determining which NIH expenditures fund research that may affect women's
health is complex and imprecise. NIH annually reports how much it spends on
women's health, men's health, and conditions that affect both women and men.
However, the nature of scientific inquiry makes it impossible to predict how
research will affect specific populations, especially with regard to the
basic research that receives a substantial portion of NIH resources, and we
found inconsistencies in the methods NIH staff use to produce its
expenditure estimates. Therefore, NIH's data on spending on conditions that
affect women, men, or both should be interpreted with caution. According to
NIH, spending on women's health conditions grew by 39 percent between fiscal
years 1993 and 1999 when adjusted for inflation, in comparison with 23
percent and 27 percent increases, respectively, in spending on men's health
and on research that affects both women and men. Using a different
analytical approach, we found that NIH's total spending on diseases and
conditions unique to or more prevalent in women grew more rapidly than NIH's
overall spending from fiscal year 1993 to fiscal year 1999, but spending
growth for specific diseases varied.
NIH, the lead federal agency for medical research, is made up of 25
institutes and centers that had a total fiscal year 1999 budget of
approximately $15.6 billion. NIH institutes and centers, each focusing on a
particular research area such as cancer, infectious diseases, or heart,
lung, and blood conditions, accomplish their missions chiefly through
extramural and intramural research. Extramural research, constituting about
80 percent of NIH's budget, involves grants, contracts, and other funding
awards to scientists who are employed outside the federal government and
conduct research at their institutions. Intramural research, which
represents about 10 percent of the NIH budget, entails government scientists
conducting research in NIH's own laboratories, hospitals, and clinics or
off-site at selected clinics and hospitals.
In 1986, NIH announced a policy of encouraging grant applicants to include
women in study populations. However, in 1990 we found that NIH had been slow
to implement this policy. Moreover, it had not been well communicated or
understood within NIH or the research community, and it was applied
inconsistently in research proposal reviews. In 1990, the inclusion policy
applied only to extramural research, not to intramural research. Although
the original policy announcement advised researchers to analyze findings by
gender, NIH had done little to implement that part of the policy.
In response to criticism about the lack of progress on research on women's
health, NIH took a number of actions. In 1990, it established ORWH in the
Office of the Director to take the lead on women's health issues at NIH, and
the following year, it launched the Women's Health Initiative (WHI), a large
prevention study to examine the major causes of death, disability, and
frailty in postmenopausal women. (See app. II.)
The 1993 NIH Revitalization Act required NIH to strengthen its previous
policy on the inclusion of women and members of minority groups in clinical
research supported by NIH.3 The act contained provisions that required NIH
to
� ensure that women and minority groups are included as subjects in clinical
research except in cases in which it is inappropriate with respect to the
health of the subjects or the purpose of the research or under circumstances
that NIH's Director designates,
� conduct or support outreach programs for recruiting women and members of
minority groups as subjects in clinical research,
� ensure that clinical trials that include women and minorities are designed
and carried out in a manner sufficient to provide for the valid analysis of
differences in effect for them,
� establish guidelines about including women and minorities as subjects in
clinical research projects, and
� establish a data system to collect, store, analyze, retrieve, and
disseminate information about women's health research that NIH supports or
conducts.
In addition to strengthening the policy on inclusion, the Revitalization Act
statutorily established ORWH, requiring it to carry out certain activities,
and called for establishing the Advisory Committee for Research on Women's
Health, a panel of nonfederal health practitioners, scientists, and other
women's health experts. The act defined women's health conditions as
conditions (1) unique to women or more serious or more prevalent in women,
(2) for which the factors of medical risk or types of medical intervention
are different for women or for which it is unknown whether such factors or
types are different for women, or (3) for which clinical research using
women as subjects or clinical data on women have been insufficient.
Included in Studies but Less Progress in Encouraging Analysis by Sex
NIH's guidelines implementing the 1993 Revitalization Act strengthened its
policies on the inclusion of women and minorities in clinical research. NIH
conducted extensive internal training on the new requirements, educated the
scientific community, and revised its application form for new grants. The
review process for recommending extramural research proposals to fund now
treats the inclusion of women and minorities as a factor that affects the
scientific merit of a proposal, which in turn affects the likelihood that
NIH will fund it. Moreover, NIH staff work with researchers to try to ensure
that studies continue to meet the inclusion requirements as they proceed.
The intramural research program now actively implements the inclusion policy
as well. A centralized tracking data system provides information on the
extent to which women are included in study populations, but there are some
inconsistencies in the data, and updated training sessions have not been
conducted. NIH has made less progress in implementing the requirement that
phase III clinical trials be designed and carried out to permit the analysis
of whether interventions affect women and men differently.
Reviewers
In March 1994, NIH published in the Federal Register new guidelines on
including women and minorities as subjects in NIH-supported clinical
research. They became effective in March 1994 for proposals submitted on or
after June 1, 1994, and addressed all relevant provisions of the 1993 NIH
Revitalization Act. In addition to continuing the NIH requirement to
generally include women and minority groups in all NIH-supported biomedical
and behavioral research involving human subjects, the new guidelines
broadened the definition of clinical research to include all research
involving human subjects, instructed reviewers to consider the planned study
population when assessing the scientific merit of a grant proposal, and
required that certain phase III clinical trials be designed to allow for
valid analyses of differences by sex or minority status.4 According to the
new guidelines, cost is not an acceptable reason for excluding these groups
from clinical studies. Furthermore, NIH must conduct or support outreach
programs to recruit and retain women and minorities as subjects in clinical
research.
To facilitate researchers' compliance with the inclusion policy, NIH revised
its application form for new grants. The application for a Public Health
Service (PHS) grant now contains instructions on compliance with the
inclusion policy and displays the format for investigators to use to report
data on the study population by sex and by racial and ethnic group.
Applicants are instructed to report the composition of the proposed study
population at the start of a project and report current enrollment data
annually when they apply to continue a grant.
After the 1994 guidelines were promulgated, NIH engaged in a series of
activities to communicate the new policy to its own reviewers, program
officers, and grants management staff, as well as to the broader scientific
community. NIH trained more than 1,000 staff members on the inclusion
requirements. The newly trained staff, in turn, worked closely with grant
applicants in the research community to guide application development,
submission, and review. NIH staff also presented information at professional
meetings and in workshops with the chairs of institutional review boards.5
NIH Funding Decisions
NIH reviewers now routinely examine the planned study populations in
extramural research proposals when evaluating their scientific merit.
Initial review groups and study sections in NIH's Center for Scientific
Review or the institutes and centers are responsible for determining whether
grant applications for clinical research comply with NIH's inclusion policy.
Under the guidance of a scientific review administrator, peer reviewers
assess the scientific merit of proposals and assign an overall priority
score to each application, which affects NIH decisions on studies to fund.6
This score is supposed to reflect, in part, the level of adherence to the
inclusion policy. Following the initial review, the scientific review
administrator prepares a summary statement for each application, which is
supposed to include all concerns about the research proposal, including any
concern about the inclusion of women.
Our reviews of grant files and discussions with NIH program officers
indicated that during the initial review process, NIH staff and many peer
reviewers do pay attention to proposed study populations' inclusion of
women. Most summary statements we reviewed at least mentioned the
anticipated participation of women and minorities, and we found numerous
instances of reviewers either noting that the level of planned participation
was good or raising concerns about the adequacy of inclusion or strategies
for recruitment and retention. We were not, however, able to assess the
extent to which observations about study populations affected the priority
scores given specific applications. In some instances, comments from
individual reviewers submitted before the initial review group met
identified concerns about the study population, but the final assessment of
the initial review group was that the population was acceptable. This could
have resulted from a difference of opinion among the reviewers or the
investigator's having provided additional information that responded to
those initial concerns.
NIH data indicate that in fiscal year 1997, first-level peer reviewers found
that about 94 percent of the extramural applications with human subjects met
NIH's standard for including women. If an application is found not to meet
NIH's standard for inclusion, it receives an administrative
"bar-to-funding." The bar-to-funding may be removed by NIH staff if
additional information is provided or if the study design is modified. About
4 percent of the extramural applications with human subjects were found to
be unacceptable in regard to the inclusion of women and received a
bar-to-funding.7
When the review group identifies deficiencies in applications with respect
to inclusion, NIH program staff explain to investigators the kinds of
changes they need to make to their proposals so that the studies will comply
with the inclusion requirements. We observed that some applicants have been
requested to provide additional information about or improve strategies for
including women before their applications were able to move forward.
Summary statements for applications recommended for further consideration
are sent to the institutes and centers for review by their national advisory
councils, which review research proposals not only for scientific and
technical merit but also for their relationship to the institute's or
center's mission, priorities, and research portfolio. Although the advisory
council reviews are considered another element of NIH's process for ensuring
the appropriate inclusion of women, most of the work of identifying and
resolving problems related to inclusion occurs before the applications reach
the councils. Council members are expected to consider a study population's
inclusion of women when making a recommendation, but advisory councils
typically focus on inclusion issues only when program staff flag problems as
a result of the initial review process.
Ensuring that women are included in research does not end with the initial
approval of a grant. We found evidence that both investigators who have
received funding and NIH staff continue to focus on inclusion issues after a
study is under way. Investigators submitting annual progress reports
sometimes call attention to problems with the recruitment of women and
outline strategies for improving lagging enrollment. For example, they may
say that they plan to concentrate all recruitment efforts in the coming year
on enrolling women, outline planned outreach activities, or report that they
will add new study sites that should provide greater access to the
population whose enrollment is below target.
NIH program staff told us that when there is a large gap between planned and
actual enrollment of women, they take steps to help investigators recruit
more women. A National Heart, Lung, and Blood Institute official told us
that he and other staff are consistently concerned about recruiting women to
studies and clearly inform researchers when there is a problem. In general,
NIH's approach is to work with researchers to expand recruitment strategies
or identify reasons for difficulties in enrolling women. Often the research
team pursues the necessary actions to improve enrollment, but a grant can be
terminated if the enrollment pattern is not corrected.
Differences by Sex Is Incomplete
Progress in carrying out the Revitalization Act's requirement that clinical
trials be designed to permit analysis of whether interventions affect women
and men differently has been less substantial than progress in implementing
other aspects of the inclusion policy. NIH requires that phase III clinical
trials include women in sufficient numbers to allow for the valid analysis
of differences in intervention effects between women and men.8 The agency
limited the requirement to phase III trials because officials believed that
the Congress had in mind studies that are well advanced on the biomedical
research spectrum and that can contribute more immediately to the
development of policy and standards of care than more
basic scientific research.9 As specifically authorized by the Revitalization
Act, the guidelines also exempt certain clinical trials from this
requirement. When substantial scientific data demonstrate no significant
difference between women and men, NIH does not require that sex be
considered in selecting study subjects but strongly encourages including
women.
In practice, these guidelines leave room for interpretation as to when phase
III clinical trials must be designed to allow for analysis by sex. A number
of clinical trials whose files we reviewed were designed to include women
but not in numbers large enough to allow analysis that would definitively
measure different outcomes for women and men. For example, the target
populations for certain trials related to acquired immunodeficiency syndrome
(AIDS) or sudden cardiac death were based on prevalence data for the general
population or the study's catchment area, which meant the goal for women's
enrollment was in the 20 to 25 percent range. NIH program officers told us
that for these and certain other studies, previous research indicated no
reason to expect different outcomes for women and men, so the trials did not
need to be designed for that type of analysis. They said, however, that the
researchers would review data for each population subgroup, so that even if
the studies were not designed to produce valid results separately for women
and men, the researchers would recognize whether the data revealed
differences that suggested the need for further study by sex.
Although the grant documents we reviewed provided strong evidence that NIH
staff and reviewers routinely focus on studies' general inclusion of women,
evidence as to whether they were taking care to implement the requirement
related to analysis by sex was scant. For example, in discussing some of the
phase III clinical trials we examined, program officers told us that under
the NIH guidelines, they were exempt from the requirement. However, because
the grant applications did not indicate plans to analyze the data by sex and
the summary statements did not explicitly mention this issue, we were not
able to assess whether reviewers had considered whether the studies should
be designed to permit valid analysis by sex. Furthermore, when we reviewed
requests for proposals and requests for applications for clinical trials
that NIH issued, we found that while they instructed potential applicants
that studies must include
women, they did not mention the need to design and carry out studies to
permit valid analysis of outcomes by sex.
Because of the difficulty in determining from the available documents
whether analysis by sex is occurring, we looked for evidence in the
published literature. A study by the Society for Women's Health Research
reviewed articles published in four major medical journals and found that
only one-quarter to one-third of the non-sex-specific studies that included
women analyzed data by sex of the subjects, with no significant change from
1993 to 1998.10 However, even when scientists are performing such analyses,
the results are not always published. NIH officials told us that when an
analysis reveals no difference in outcome, journals publishing the analysis
may omit this information because editors often discourage researchers from
including "no news" information in their results. NIH officials also
observed that because researchers who received funding after the new
guidelines took effect are just now beginning to publish their study
results, the amount of analysis by sex may increase.
Policy
As required by the Revitalization Act, NIH's intramural studies also must
comply with the inclusion policy. All the patients admitted to the
intramural program are treated under a research protocol that one of NIH's
14 institutional review boards must review and approve. To approve a
research protocol, a board must ensure that the selection of patients is
appropriate. In the review process for the continuation of protocols, a
board receives accrual data on the distribution of women and men for each of
the intramural program's almost 1,000 research protocols. According to NIH,
the boards play an active role in evaluating the inclusiveness of the
intramural research designs and monitor the accrual data closely.
About 6 years ago, the intramural program developed and began implementing a
new curriculum to train investigators that covers all aspects of clinical
research practice, including NIH's inclusion policy. In addition, all newly
appointed review board members are required to complete NIH's computer-based
training on the protection of human subjects and a one-on-one orientation
with the Office of Human Subjects Research that addresses the inclusion of
women and minorities.
Inclusion Policy but Needs Improvement
In 1995, NIH established an automated, centralized data tracking system to
monitor demographic information on study populations and help assess whether
the inclusion policy is being carried out. The Information Systems Branch of
the Division of Research Grants developed this system in collaboration with
ORWH, the Office of Research on Minority Health, the Office of Extramural
Research, and the Office of Intramural Research, and it was designed to be
compatible with NIH's existing data management system. Investigators
manually submit data on their study populations to program staff at each
institute, who either manually or electronically transmit them to the
centralized tracking data system.11 An NIH Tracking and Inclusion Committee,
consisting of representatives from each of the institutes and centers and
cochaired by the ORWH director and an NIH senior program official from a
major institute, provides guidance on the tracking system and communicates
tracking policies to the institutes and centers. Each year the committee
reports on aggregate enrollment information in the tracking system. These
data are generally about 2 years old because of the time it takes
investigators to report and NIH staff to compile and process the data.
Not all research studies that NIH funds are entered into the tracking
system. NIH officials have decided that ten different categories of studies
do not need to report on the inclusion of women and minorities. These
include basic research studies, studies involving only tissue or body fluid
specimens, and very small studies. Also, if a study is part of a multicenter
study or is funded by a supplemental grant, its data are presumed to be
reported by its coordinating center or the parent study.12 In fiscal year
1997, about 41 percent of NIH studies were tracked. Among the six institutes
we reviewed, this percentage ranged from 13 percent of studies at the
National Institute of Allergy and Infectious Diseases to 60 percent of
studies at the National Institute of Child Health and Human Development.
(See app. III.) The relatively small percentage of studies tracked at NIH
overall and at some institutes may in part reflect the fact that a majority
of NIH research is nonclinical.
The inclusion data are subject to inconsistencies because of differing
interpretations of which studies should be tracked. When the tracking system
was first established, NIH staff involved in entering the data received
training, but updated training has not been conducted. NIH officials told us
that the system and the data derived from it have improved and have been
refined and that the system is now capturing more meaningful information,
such as enrollment targets. Some of the earlier data are being reconciled by
the Office of Extramural Research, which is also transferring the system to
a new NIH-wide data management system. NIH is also beginning to implement an
electronic reporting system for applications that includes reporting the
composition of study populations.
More than half of the participants in clinical research studies NIH funded
in fiscal year 1997 were women, according to NIH's tracking data. Even when
studies with female-only or male-only protocols were removed from the
analysis, the proportion of women enrolled was greater than 50 percent.
Minority women were well represented, especially black and Asian and Pacific
Islander women; however, the proportion in Hispanic women enrolled was below
their proportion in the general population. There was some variation in
enrollment levels by institute.
Well Represented
NIH's tracking system indicates a large percentage of women enrolled in
clinical research that NIH funds. Aggregate enrollment data for all
extramural research protocols funded in fiscal year 1997 show that 61.9
percent of the study subjects were women and 37.1 percent were men. Phase
III clinical studies funded that year included 74.8 percent women and 24.3
percent men. For intramural research projects, women represented about half
of the study participants for both on-site and off-site protocols.13 (See
fig. 1.) When studies with female-only or male-only protocols were
removed from the aggregate extramural research enrollment data, the
proportion of women enrolled was 52.1 percent.14
Figure 1: Aggregate Enrollment for NIH Extramural and Intramural Research,
Fiscal Year 1997
NIH's data on the enrollment of minority women in extramural research
projects indicate that blacks represented the largest proportion of minority
women at 17.2 percent. Asians and Pacific Islanders represented 15.4 percent
of women enrolled, and Hispanics represented 7.7 percent. (See table 1.)
These percentages compare favorably with the proportions of black women at
12.4 percent and Asian and Pacific Islander women at 3.6 percent in the
general population; however, there is a larger proportion of Hispanic women
in the general population, at 10.6 percent, than is represented in
extramural research protocols at NIH.
Table 1: Enrollment of Minority Women in NIH Extramural and Intramural
Research as a Percentage of Total Female Enrollment, Fiscal Year 1997
American
Black, White,
Type of Indians Asian and Other and
research and Pacific Not Hispanic Not unknown
Alaska Islanders
Natives Hispanic Hispanic
All
extramural 1.1 15.4 17.2 7.7 52.7 5.9
Extramural
phase III 1.3 2.0 11.6 4.4 79.5 1.3
Intramural
On-site 0.1 4.3 12.4 3.6 78.2 1.4
Off-site 0.2 55.3 3.8 0.5 38.4 1.8
NIH officials are cautious, however, about directly comparing enrollment
data with national census figures. They believe that the goal of the NIH
policy is to conduct biomedical and behavioral research in such a manner
that scientific knowledge can be generalized to the entire U.S. population,
not to satisfy proportional representation. NIH considers that the
appropriate numbers of women or minority subgroups included in a particular
study depend on the scientific question addressed in the study and the
prevalence among women and minority subpopulations of the disease, disorder,
or condition under investigation. Some studies, for example, would require a
rate of minority participation higher than a group's presence in the overall
population, and others would be sound with lower minority participation.
Across Institutes
When we examined the fiscal year 1997 distribution of studies by the
percentage of women participating, most (52.5 percent) had enrolled between
30 and 60 percent women participants. Seventy percent of the studies had
populations that were at least 40 percent women. (See table 2.) Among the
six institutes we reviewed in detail, the range in aggregate enrollment of
women in clinical research in fiscal year 1997 ran from the National Cancer
Institute's highest at 70.9 percent to the National Institute of Allergy and
Infectious Diseases' lowest at 48.1 percent.
Table 2: The Distribution of Fiscal Year 1997 Clinical Studies by the
Percentage of Women Participants
Percent of women Number of studies Percent of studies
100 740 14.5
90 up to 100 45 0.9
80 up to 90 105 2.1
70 up to 80 196 3.9
60 up to 70 388 7.6
50 up to 60 1,115 21.9
40 up to 50 974 19.1
30 up to 40 586 11.5
20 up to 30 377 7.4
10 up to 20 224 4.4
1 up to 10 95 1.9
0 244 4.8
Total 5,089 100.0
Note: Excludes 74 studies in which it was not known whether the participants
were women or men.
Develop NIH's Women's Health Research Agenda
ORWH has a mandate to ensure that women and minorities are included in
clinical research that NIH funds, and it is specifically responsible for
carrying out the guidelines on inclusion and coordinating the data tracking
system. It conducted training and education on the inclusion policy when the
policy first took effect but has not provided NIH staff with ongoing
training on the data tracking system. ORWH took the lead at NIH in
developing an agenda for research on women's health, obtaining input from
individuals and groups outside NIH through numerous conferences, workshops,
and its Advisory Committee for Research on Women's Health. ORWH has provided
funding support to institutes and centers for women's health research that
it considers important.
ORWH serves as a focal point for women's health research at NIH.15 ORWH's
estimated budget of about $20 million for fiscal year 2000 is a component of
the budget of the Office of the Director. The ORWH budget more than doubled
from fiscal year 1993 ($9.4 million) to fiscal year 1999 ($19.6 million),
which was a 67 percent increase when adjusted for inflation. The size of
ORWH's full-time-equivalent staff also increased, from 3
full-time-equivalent employees in fiscal year 1991 to 16 in fiscal year
1999. ORWH is required to prepare a report on its activities for the NIH
director every 2 years. The most recent report, for fiscal years 1997-98,
was still in draft in April 2000 and had not yet been published.
ORWH played a central role implementing NIH's 1994 guidelines on the
inclusion of women and was responsible for some of the training and
educational activities we described earlier in this report. For example,
ORWH formed a task force in 1993 on the recruitment and retention of women
in clinical studies; it also sponsored a public hearing following the
publication of the guidelines. Recommendations from this hearing were
incorporated into the planning of a scientific meeting, "Recruitment and
Retention of Women in Clinical Studies," later that year, and in the fall of
1994, ORWH published a summary report.16
ORWH convened a meeting at NIH with institutional review board chairs from
around the country after the 1994 inclusion guidelines were announced. The
meeting focused on the role of board chairs in implementing the guidelines,
discussed lessons learned during early implementation of the NIH policy, and
suggested strategies for increasing the recruitment of women and minorities.
In 1996, ORWH reconvened the board chairs to discuss their experience in
implementing the guidelines.
To provide ongoing education and guidance to investigators as they consider
scientific questions to research, appropriate study designs, and methods to
facilitate the enrollment of study participants, ORWH published an outreach
notebook in 1997, which is available on the Internet. This notebook provides
advice on inclusion criteria in the form of a decision tree, provides
information on the key elements of the outreach process--such as
understanding the study population and evaluating the efficacy of
recruitment and retention strategies--and discusses a number of practical
considerations. It describes the review criteria for including women and
minorities and presents the format for the summary table of planned and
actual enrollment.
In collaboration with the Office of Extramural Research, the Office of
Intramural Research, and other components of NIH, ORWH is responsible for
coordinating the collection and analysis of the inclusion tracking data and
publishing reports with these data. ORWH organized and convened the NIH-wide
Tracking and Inclusion Committee to ensure that uniform standards and
definitions are used in reporting data on inclusion. However, as we
indicated earlier, there have been inconsistencies in the reported data, and
ORWH has not conducted updated training on the tracking system. The annual
reports on the data have generally been limited to NIH-wide summary counts,
with limited analysis.
The Revitalization Act also required NIH to establish a system of data on
research on women's health, including a clinical trials registry, for health
care providers, researchers, and the general public. ORWH worked with the
National Library of Medicine to develop an NIH-wide clinical trials
registry. In early 2000, NIH unveiled an Internet website listing all
clinical trials sponsored primarily by NIH, and it plans to add studies from
other federal agencies and the pharmaceutical industry later in 2000.
Community to Develop a Research Agenda
To develop a research agenda for women's health, ORWH has sought input from
scientific experts and interest groups outside NIH. ORWH has sponsored or
cosponsored with the institutes and centers numerous scientific conferences
on women's health. Among the cosponsored conferences, topics covered since
fiscal year 1993 include
� Directions for menopause research,
� Older women, health, and retirement: A demographic perspective,
� The recruitment and retention of minorities in clinical cancer research,
� African-American women's health care,
� The health of women with physical disabilities,
� Psychological and behavioral factors in women's health,
� HIV/AIDS in women: The changing face of AIDS,
� The U.S.-Mexico border and women's health,
� Neuroscience and endocrinology of fibromyalgia, and
� Research priorities in eating disorders.
One of ORWH's early actions was to convene a scientific workshop in
September 1991 at which experts in the basic and clinical sciences,
practitioners in women's health, and representatives of women's
organizations developed recommendations for research activities in women's
health, focusing on the major divisions of a woman's life span and the
diseases and impairments that might affect her health and well-being over
the course of her life. The deliberations and findings of that meeting were
published as Report of the National Institutes of Health: Opportunities for
Research on Women's Health.17 According to the Director of ORWH, this report
served as the basis for NIH's research priorities in women's health for 7
years.
In 1996 and 1997, ORWH and a task force of federal and nonfederal scientists
and advocates convened three regional meetings around the country and a
final national meeting to examine progress in research on women's health and
to set an agenda for the future. Each meeting included a public hearing and
a scientific workshop. Participants identified continuing or emerging
issues, gaps in knowledge, and new models for designing and conducting
research studies. The deliberations of these meetings were published in the
1999 report, Agenda for Research on Women's Health for the 21st Century.18
ORWH does not have a formal mechanism for monitoring how NIH's institutes
and centers implement the women's health research agenda. Its primary means
for collaborating with the institutes and centers is the Coordinating
Committee for Research on Women's Health. The committee members, who are
institute or center directors or, more commonly, their delegates, advise
ORWH on NIH research issues regarding women's health and provide a liaison
and a forum for communication among the institutes and centers and ORWH.
ORWH now generally convenes committee meetings a few times a year to provide
an opportunity for institutes and centers to share information on women's
health research priorities and activities in their respective organizations.
Representatives bring ORWH priorities back to their respective institutes
and centers for consideration.
Although ORWH does not fund research projects directly, it uses its budget
to leverage funding for research on women's health by the NIH institutes and
centers. From fiscal year 1991 through fiscal year 1999, ORWH provided
almost $90 million to institutes and centers through various funding
mechanisms such as making supplemental grants and cofunding studies. For
example, one institute representative reported that funding assistance from
ORWH led his institute to fund large-scale projects for women's health
research. Some of the research projects that ORWH has cofunded have focused
on reproductive health and sexually transmitted diseases, urological health,
autoimmune diseases, occupationally related diseases and disabilities,
physical disability in aging women, cancer, cardiovascular disease,
depression, and violence.
The Advisory Committee for Research on Women's Health advises the ORWH
director on appropriate women's health research activities for NIH to
undertake and is charged with helping monitor compliance with the guidelines
on including women in clinical research. To meet the Revitalization Act's
requirement that the Advisory Committee issue a biennial report on its
activities, it issues a report jointly with ORWH and the Coordinating
Committee for Research on Women's Health.19 Some Advisory Committee members
have also taken an active role in ORWH conferences to develop the research
agenda.
The ORWH director serves as co-study director of WHI with the National
Heart, Lung, and Blood Institute director, receives periodic updates on the
progress of the study's recruitment, and participates in WHI Advisory
Committee meetings. In addition, the ORWH director serves as a member of
WHI's Institute Directors Committee, which meets periodically to ensure
communication among institutes and centers with an interest in WHI. A member
of the ORWH Advisory Committee also serves as an ad hoc member of the WHI's
Advisory Committee.
The Department of Health and Human Services (HHS) has an Office of Women's
Health that serves as a focal point for women's health issues in the
department by coordinating and promoting women's health activities across
all HHS agencies. For example, the Office of Women's Health, along with
ORWH, sponsored the Secretary's Conference to Establish a National Action
Plan on Breast Cancer. The conference involved collaboration with the
National Breast Cancer Coalition, advocacy communities, scientific
communities, and numerous components of HHS and other federal agencies.
It is difficult to estimate NIH's expenditures on research on women's
health, in part because it is often not possible to predict who will benefit
from research, especially basic research. NIH's estimates of expenditures
for health conditions that affect women, men, or both women and men must
therefore be interpreted with caution. According to NIH's estimates, NIH's
spending on women's health conditions grew by 39 percent between fiscal year
1993 and fiscal year 1999, when adjusted for inflation, in comparison with
23 percent and 27 percent increases, respectively, in spending on men's
health and on research that affects both women and men. Using a different
approach, we found that NIH's spending on diseases and conditions unique to
or more prevalent in women grew more rapidly than its overall spending,
although this varied among specific diseases.
NIH annually compiles a table that estimates expenditures for 124 specific
disease categories. NIH classifies expenditures into three
categories--spending on women's health conditions, spending on men's health
conditions, and spending that affects both women's and men's health.
Determining which NIH expenditures should be considered spending on women's
health is challenging. The nature of biomedical research makes it difficult
to predict the eventual uses of scientific findings and which segments of
the population will benefit from particular studies. This is especially true
for basic research, which is a substantial portion of the research that NIH
funds. It is relatively easy to classify some expenditures, such as for
certain research on ovarian cancer or prostate cancer, as related to women's
health or men's health. Other expenditures, however, such as for certain
cardiovascular research, could be divided between women's and men's health
or placed in the category of spending that affects both. We found
inconsistencies in the methods NIH staff use to produce the annual NIH
expenditure table. For example, for clinical research that affects both
women and men, some institutes use their data on the numbers of women and
men enrolled in studies to place proportional
expenditures in the "women" and "men" categories. Others, however, enter the
entire amount into the "both" category. One official noted that sufficient
judgment is involved in making these decisions to suggest that even the same
person might not make the same judgment twice.20
In light of the difficulties in assigning expenditures to the categories of
women's health, men's health, and both, the following data must be
interpreted with caution. NIH reported that it spent about $2.3 billion on
women's health in fiscal year 1999, about 15.5 percent of total
expenditures.21 About 78.1 percent of expenditures that year fell into the
"both" category, and about 6.4 percent went to men's health conditions. This
distribution fluctuated to only a small extent from fiscal year 1993 through
fiscal year 1999, but the share of total expenditures in the women's health
category grew during that period while the proportions in the two other
categories decreased (see table 3). When adjusted for inflation, spending in
the women's health category grew by 39 percent from fiscal year 1993 to
fiscal year 1999; during the same period, spending on men's health grew by
23 percent and spending on both grew by 27 percent.
Table 3: Percentages of NIH Expenditures for Research on Women, Men, and
Both, Fiscal Years 1993-99
1993 1994 1995 1996 1997 1998 1999
Women 14.3 14.5 16.1 16.0 16.2 16.0 15.5
Men 6.7 6.0 6.2 5.7 6.1 6.1 6.4
Both 79.0 79.4 77.6 78.3 77.7 77.9 78.1
We also analyzed NIH spending data using a different approach--comparing the
growth in total NIH spending from fiscal year 1993 to fiscal
year 1999 with growth in spending on diseases and conditions that are unique
to or more prevalent in women.22 In the past several years, NIH's spending
on these diseases and conditions has grown more rapidly than its overall
spending. While overall expenditures, when adjusted for inflation, increased
by about 29 percent from fiscal year 1993 to fiscal year 1999, adjusted
expenditures for the conditions unique to or more prevalent in women grew by
37 percent. The rates of growth for specific women's conditions varied
greatly. Adjusted spending on osteoarthritis and depression and mood
disorders, conditions more prevalent in women, increased by more than 70
percent. Adjusted spending on pregnancy, at 21 percent, and osteoporosis, at
16 percent, however, grew more slowly than overall NIH spending. Adjusted
expenditures on breast cancer and ovarian cancer grew by 59 percent and 53
percent, respectively. (See fig. 2.)
Figure 2: Rate of Adjusted Expenditure Growth for Selected Conditions and
Total NIH Expenditures, Fiscal Years 1993-99
Note: 1999 expenditures have been adjusted to 1993 dollars.
An underlying goal of the 1993 NIH Revitalization Act and NIH's policy on
including women and minorities in research is for every study funded by NIH
to answer its research questions for as wide a segment of the population as
is scientifically appropriate. NIH has made significant progress toward this
goal. In contrast to our review of this issue 10 years ago, we found
widespread awareness and understanding of most aspects of the policy, and
many people in NIH and the research community are making concerted efforts
to translate the policy into improved science.
For the policy to have its intended effect, however, NIH needs to expand its
focus beyond simple inclusion and to ensure that, when it is scientifically
appropriate, researchers conducting clinical trials enroll populations and
analyze study data in ways that enable them to learn whether interventions
affect women and men differently. NIH staff, peer reviewers, and Advisory
Council members should carry out this dimension of the inclusion policy as
conscientiously as they attend to its other components. For example, when
NIH initiates a phase III clinical trial through a request for application
or proposal, unless NIH officials have determined that this particular trial
is exempt from the requirement, the request should inform the applicants
that they should design the trial to allow for a valid analysis of
differences between women and men. As reviewers examine each application to
receive funding for a phase III clinical trial, they need to explicitly
consider whether the study should be structured to allow for analysis by
sex. If the review group concludes that a particular trial should be exempt
from this requirement--for example, because of previous research
findings--noting this decision in the summary statement would increase
confidence that NIH is implementing this aspect of the inclusion policy.
Advisory Council members can carefully review these decisions to ensure that
all research NIH funds is producing the scientifically appropriate
information.
NIH's tracking system is an important tool for monitoring the implementation
of the inclusion policy, and the system is beginning to capture more
information, such as enrollment targets, that will help assess the policy's
success. NIH's recent steps to improve the system and move toward increased
electronic reporting should help improve the accuracy and timeliness of the
data and improve the system's ability to measure progress. However,
follow-up training on the requirements and purpose of the tracking system is
needed.
Finally, NIH's data on spending on women's health must be interpreted with
care. Determining which expenditures affect women's health is so complex and
imprecise that it is easy to overstate or understate the extent of NIH's
efforts. The nature of scientific inquiry makes it impossible to predict all
the effects of research, and this is especially true for the basic research
that makes up a large part of NIH's portfolio.
To strengthen the capacity of biomedical research to produce information on
health applicable to all segments of the population, we recommend that the
Director of NIH ensure that the agency implements the requirement that phase
III clinical trials be designed and carried out to allow for the valid
analysis of differences between women and men as fully as it implements
other elements of the inclusion policy. Specifically, we recommend that NIH
appropriately communicate this requirement to applicants, that peer review
groups explicitly determine whether each proposed phase III clinical trial
is required to have such a study design, and that summary statements
document the initial reviewers' decisions.
To improve the accuracy of NIH's tracking data on the inclusion of women and
minorities, we recommend that the Director of NIH ensure that NIH staff who
transmit data to the tracking system receive ongoing training on the
requirements and purpose of the system.
We provided a draft of this report to NIH for comment. NIH concurred with
our recommendations. Regarding our recommendation to ensure that phase III
clinical trials be designed and carried out to permit analysis by sex
whenever appropriate, NIH said it will again emphasize to NIH staff and
applicants that sex analysis should be included in research plans when it is
scientifically justified and can be done in a way that protects the
interests of study participants. The agency also indicated that it plans to
take additional steps to implement the requirements related to phase III
clinical trials. In response to our recommendation that ongoing training
related to NIH's tracking system be provided, NIH said that its staff who
transmit data to the tracking system will receive ongoing training and said
that ORWH will work with the Office of Extramural Research to ensure that
NIH staff are well informed about the data collection requirements of the
current system. NIH also intends to conduct training on the tracking module
in NIH's new grants administration system when it is implemented. Finally,
NIH provided technical comments, which we incorporated where appropriate.
We are sending copies of this report to the Honorable Donna E. Shalala,
Secretary of Health and Human Services, and to Dr. Ruth L. Kirschstein, the
Acting Director of NIH; appropriate congressional committees; and others who
are interested. We will also make copies available to others on request.
If you or your staff have any questions, please contact me at (202) 512-7119
or Helene Toiv at (202) 512-7162. Other major contributors are listed in
appendix V.
Janet Heinrich
Associate Director, Health Financing
and Public Health Issues
Objectives, Scope, and Methodology
This report assesses the progress the National Institutes of Health (NIH)
has made in implementing its 1994 guidelines on including women in clinical
research, including the requirement that certain studies be designed to
permit analysis by sex. The report also provides information about the
extent to which women are being included in clinical research that NIH
funded, reviews the activities and accomplishments of NIH's Office of
Research on Women's Health (ORWH) in promoting women's health research at
NIH, and provides information about how much funding NIH has allocated to
research on issues that affect women.
To assess NIH's progress in implementing its inclusion policy, we
interviewed officials in central NIH offices, six institutes--the National
Cancer Institute; National Heart, Lung, and Blood Institute; National
Institute of Allergy and Infectious Diseases; National Institute of
Arthritis and Musculoskeletal and Skin Diseases; National Institute on
Aging; and National Institute of Child Health and Human Development--and the
Warren Grant Magnuson Clinical Center for intramural research. We also
reviewed pertinent agency documents. We reviewed small judgmental and random
samples of extramural clinical research grant files from the National Heart,
Lung, and Blood Institute, the National Institute of Allergy and Infectious
Diseases, and the National Institute of Arthritis and Musculoskeletal and
Skin Diseases to assess adherence to the required procedures for ensuring
that funded research complies with the policy to include women in study
populations. We did not review any files involving intramural clinical
research.
To determine the extent to which women have been included in clinical
research supported by NIH, we analyzed NIH's tracking data for all
extramural and intramural studies involving human subjects funded during
fiscal year 1997, for NIH as a whole and for individual institutes. Fiscal
year 1997 is the latest year for which data were available.
To assess the activities and resources of ORWH and its coordination and
collaboration among NIH institutes and centers, we interviewed NIH's
director and representatives to the Coordinating Committee and the Tracking
and Inclusion Committee from the six institutes in our review. We examined
ORWH's budget, mission, research agenda, and reports to obtain additional
information on its resources and activities. To further examine coordination
and collaboration issues, we interviewed individuals at other public and
private agencies.
Finally, using the Department of Health and Human Services' (HHS) definition
of health issues that affect women, we analyzed NIH's data on expenditures
on women's health for fiscal years 1993 to 1999.23 We compared NIH-wide and
institute expenditures on women's health research with expenditures for
research on men and on both women and men. We also analyzed NIH expenditures
for conditions that are unique to or more prevalent in women. The Director
of ORWH helped us identify the conditions to include in those categories.
Finally, we interviewed NIH and institute and center budget officers about
the methods they used to produce the data on expenditures on research on
women's health.
We conducted our work between November 1999 and April 2000 in accordance
with generally accepted government auditing standards.
Women's Health Initiative
The Women's Health Initiative (WHI) includes three research arms: (1) a
large-scale randomized clinical trial to examine the effect of hormone
replacement therapy, diet, and calcium and vitamin D supplementation in the
primary prevention of cardiovascular disease, breast cancer, and
osteoporosis in women; (2) an observational study to identify predictors of
disease; and (3) a study of community approaches to enhance the development
of healthy behavior.
WHI has reported significant progress since recruitment began in September
1993. The clinical trial is being conducted in 40 clinical centers
nationwide and has enrolled more than 68,000 postmenopausal women between
the ages of 50 and 79. This clinical trial has three components: a hormone
replacement therapy trial, a dietary modification trial, and a calcium and
vitamin D supplementation trial. The hormone component tests whether
long-term therapy reduces coronary heart disease and fractures without
increasing breast cancer risk. The final participant in this trial was
enrolled in December 1998. The calcium and vitamin D component enrolls women
who have already been in the hormone or dietary trials for up to 2 years.
The observational study, which will track the medical history of
approximately 100,000 women, is designed to examine the relationship between
lifestyle, health, and risk factors and specific disease outcomes. Women who
joined this study fill out periodic health forms and also visit a clinic 3
years after enrollment. Participants are not required to take any medication
or change their health habits. The observational study completed enrollment
on December 31, 1998, and participants will be followed for 8 to 12 years.
The community prevention study, which began in October 1995, is a
collaboration between NIH and the Centers for Disease Control and
Prevention's National Center for Chronic Disease Prevention and Health
Promotion. Eight community prevention centers have been conducting and
evaluating efforts to encourage women, especially minority women, to adopt
healthy behaviors such as improved diet, nutritional supplementation,
smoking cessation, exercise, and early detection of treatable health
problems. The goal is to develop model programs that can be implemented in
communities throughout the United States. This study is expected to conclude
in fiscal year 2000.
Percentages of NIH Studies Tracked and Not Tracked, Fiscal Year 1997
Tracking NIH
code Description NCI NHLBI NIAID NIAMS NIA NICHD total
0 Must be tracked 31.88 31.58 13.24 20.40 40.64 60.11 40.66
Basic research, far
1 removed from 11.09 38.49 20.29 21.89 26.97 0.65 17.08
immediate clinical
application
Early stages of
2 technology or 7.52 5.64 0.87 4.23 4.19 1.85 3.87
methodology
development
3 Fewer than 10 0.59 1.01 1.64 2.99 0.74 0.98 1.02
participants
Multicenter
4 project--not 11.51 6.72 0.87 0.25 1.60 4.24 5.5
coordinating center
Supplement--data
5 captured in parent 2.98 5.77 3.38 3.73 3.69 4.13 4.05
award
6 Unidentifiable 0.80 0.51 0 0.25 0.49 0.33 0.67
gender and minority
Awarded before
7 implementing gender 0.29 0 0.58 0.50 1.85 0 0.84
or minority policy
8 Substudy of existing 2.98 0 1.93 2.24 2.83 0.11 1.73
reported population
9 Analysis of existing 2.90 3.55 6.96 1.00 8.87 7.28 3.48
study data
10 Only tissue or body 21.42 0 46.28 40.30 4.19 13.04 16.36
fluid specimens
Note: NCI = National Cancer Institute. NHLBI = National Heart, Lung, and
Blood Institute. NIAID = National Institute of Allergy and Infectious
Diseases. NIAMS = National Institute of Arthritis and Musculoskeletal and
Skin Diseases. NIA = National Institute on Aging. NICHD = National Institute
of Child Health and Human Development.
Comments From the National Institutes of Health
GAO Contacts and Staff Acknowledgments
Helene Toiv, (202) 512-7162
Anne Dievler, (202) 512-7006
In addition to the persons named above, Nila Garces-Osorio, Brenda James,
Darryl Joyce, and Katherine Tomber made important contributions to this
report.
(101883)
Table 1: Enrollment of Minority Women in NIH Extramural and
Intramural Research as a Percentage of Total Female
Enrollment, Fiscal Year 1997 17
Table 2: The Distribution of Fiscal Year 1997 Clinical Studies by the
Percentage of Women Participants 18
Table 3: Percentages of NIH Expenditures for Research on Women,
Men, and Both, Fiscal Years 1993-99 24
Figure 1: Aggregate Enrollment for NIH Extramural and Intramural Research,
Fiscal Year 1997 16
Figure 2: Rate of Adjusted Expenditure Growth for Selected
Conditions and Total NIH Expenditures, Fiscal Years
1993-99 26
1. National Institutes of Health: Problems in Implementing Policy on Women
in Study Populations (GAO/T-HRD-90-38, June 18, 1990 ).
2. Public Law 103-43.
3. Although the Revitalization Act addressed the inclusion of minorities as
well as women, we focus in this report primarily on women.
4. The 1994 guidelines use "gender" rather than "sex." However, since
scientists have begun to use the term "sex" to denote biologically based
differences and "gender" to indicate culturally shaped variations between
women and men, in this report we use "sex" when referring to analysis of
different study results for women and men. For further discussion of this
issue, see Jennifer R. Fishman, Janis G. Wick, and Barbara A. Koenig, "The
Use of `Sex' and `Gender' to Define and Characterize Meaningful Differences
Between Men and Women," in NIH, Office of the Director, Office of Research
on Women's Health, Agenda for Research on Women's Health for the 21st
Century: A Report of the Task Force on the NIH Women's Health Research
Agenda for the 21st Century, Vol.1, Executive Summary, NIH publication
99-4385 (Bethesda, Md.: HHS, NIH, 1999).
5. Institutional review boards are associated with a research institution or
university and are responsible for implementing federal human subject
protection requirements for research conducted at or supported by the
institution or university.
6. A scientific review administrator is an NIH staff member who serves as
the primary contact with applicants during the initial phase of the peer
review process until the conclusion of the first-level peer review process.
7. The remainder of the applications, about 2 percent, were found to be
unacceptable in regard to the inclusion of minorities and received a
bar-to-funding.
8. To implement this policy, NIH developed a special definition of clinical
trial to distinguish these trials from other types of clinical research that
NIH supports. NIH defined clinical trial as "a broadly based prospective
Phase III clinical investigation, usually involving several hundred or more
human subjects, for the purpose of evaluating an experimental intervention
in comparison with a standard or control intervention or comparing two or
more existing treatments." Phase I and phase II clinical trials are earlier
phases of research in which interventions or treatments are tested in a
smaller number of people.
9. J. LaRosa, and others, "Including Women and Minorities in Clinical
Research," Applied Clinical Trials, Vol. 4, No. 5 (May 1995), pp. 31-38.
10. Regina M. Vidaver and others, "Women in NIH-funded Clinical Research:
Lack of Progress in Both Representation and Analysis by Sex in the Medical
Literature," forthcoming in Journal of Women's Health and Gender-Based
Medicine.
11. There is no verification of the data that investigators submit on their
study enrollment; however, falsifying these data would be a federal felony.
12. The categories are (1) basic research; (2) studies in early stages of
technology or methodology development; (3) studies with fewer than 10
participants; (4) multicenter projects, not at a coordinating center; (5)
supplemental grants, when data are reported in the parent study; (6)
unidentifiable gender and minority status; (7) awards made before
implementing the revised inclusion policy; (8) substudy of existing reported
population; (9) analysis of existing study data; and (10) only tissue or
body fluid specimens are involved.
13. Where these percentages do not add up to 100, the remainder of study
subjects were unknown.
14. Some studies restrict enrollment to women or to men because they focus
on reproductive health, on diseases that affect women or men
disproportionately or exclusively, or on the action of particular disease
processes in only one or the other.
15. ORWH's mission also includes promoting the recruitment, retention,
reentry, and advancement of women in biomedical careers. This report does
not examine this aspect of ORWH's mission.
16. NIH, Recruitment and Retention of Women in Clinical Studies, NIH
publication 95-3756 (Bethesda, Md.: 1994).
17. NIH publication 92-3457 (Bethesda, Md.: Sept. 1992).
18. Agenda for Research on Women's Health for the 21st Century is a
six-volume report. Volume 1 is the executive summary, NIH publication
99-4385 (Bethesda, Md.: 1999).
19. See Report of the Office of Research on Women's Health and of NIH
Support for Research on Women's Health Issues, Fiscal Years 1995-1996, NIH
publication 99-4304 (Washington, D.C.: n.d.).
20. One factor contributing to the inconsistencies is that NIH staff receive
two sets of guidelines, one written by the HHS-wide Coordinating Committee
on Women's Health and one by the NIH Coordinating Committee for Research on
Women's Health, which differ in part.
21. The total expenditure figures we are using in this report are the totals
reported in the expenditure tables organized by the categories "women,"
"men," and "both." These figures may differ from other total expenditure
figures NIH reports in other contexts, such as the annual budget
justification, because the table figures do not include all NIH
expenditures, such as buildings and facilities.
22. These spending categories include the following diseases, conditions,
and programs: breast cancer; cervical cancer; ovarian cancer; vaginal,
uterine, and other reproductive cancers; contraception; infertility; female
reproductive physiology; hysterectomy; endometriosis and leiomyomas (of
uterus); pregnancy, pregnancy prevention, and maternal health; diseases
related to DES exposure; menopause; hormone replacement therapy;
incontinence; osteoarthritis; osteoporosis; thyroid diseases and conditions;
violence; women as caregivers; depression and mood disorders; eating
disorders; toxic shock syndrome; chronic fatigue syndrome; Sjogren's
syndrome; temporomandibular disorders; fibromyalgia; urinary tract
infections; arthritis; lupus erythematosus; multiple sclerosis; scleroderma;
migraine; female genital mutilation; homeless women; WHI; and ORWH.
23. According to HHS, a health issue that affects women is a disease or
condition that is unique to women, more prevalent in women, or more serious
among women or that has different risk factors or different interventions
for women.
*** End of document. ***