GAO-11-32, VA Health Care: VA Spends Millions on Post-Traumatic Stress Disorder Research and Incorporates Research Outcomes into Guidelines and Policy for Post-Traumatic Stress Disorder Services


This is the accessible text file for GAO report number GAO-11-32 
entitled 'VA Health Care: VA Spends Millions on Post-Traumatic Stress 
Disorder Research and Incorporates Research Outcomes into Guidelines 
and Policy for Post-Traumatic Stress Disorder Services' which was 
released on January 24, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to the Ranking Member, Subcommittee on Health, Committee on 
Veterans' Affairs, House of Representatives: 

January 2011: 

VA Health Care: 

VA Spends Millions on Post-Traumatic Stress Disorder Research and 
Incorporates Research Outcomes into Guidelines and Policy for Post- 
Traumatic Stress Disorder Services: 

GAO-11-32: 

GAO Highlights: 

Highlights of GAO-11-32, a report to the Ranking Member, Subcommittee 
on Health, Committee on Veterans’ Affairs, House of Representatives. 

Why GAO Did This Study: 

In addition to providing health care to veterans, the Department of 
Veterans Affairs (VA) funds research that focuses on health conditions 
veterans may experience. According to VA, experts estimate that up to 
20 percent of Operation Enduring Freedom and Operation Iraqi Freedom 
veterans have experienced post-traumatic stress disorder (PTSD) and 
demand for PTSD treatment is increasing. Because of the importance of 
research in improving the services that veterans receive, GAO was 
asked to report on VA’s funding of PTSD research, and its processes 
for funding PTSD research proposals, reviewing and incorporating 
research outcomes into clinical practice guidelines (CPG)—-tools that 
offer clinicians recommendations for clinical services but do not 
require clinicians to provide one service over another-—and 
determining which PTSD services are required to be made available at 
VA facilities. To do this work, GAO obtained and summarized VA data on 
the funding of PTSD research from its medical and prosthetic research 
appropriation through its intramural research program. GAO also 
reviewed relevant VA documents, such as those for developing CPGs and 
those related to VA’s 2008 Uniform Mental Health Services in VA 
Medical Centers and Clinics handbook (Handbook), which defines certain 
mental health services that must be made available at VA facilities. 
GAO also interviewed VA officials. 

What GAO Found: 

Based on VA data GAO obtained and summarized, GAO found that the 
amount of funding VA provided for intramural PTSD research increased 
from $9.9 million in fiscal year 2005 to $24.5 million in fiscal year 
2009. From fiscal year 2005 through fiscal year 2009, intramural PTSD 
research funding ranged from 2.5 percent to 4.8 percent of VA’s 
medical and prosthetic research appropriation. In addition, the number 
of PTSD research studies VA funded through the Merit Review Program 
and the Cooperative Studies Program (CSP)-—VA’s two primary funding 
mechanisms in its intramural research program-—increased from 47 in 
fiscal year 2005 to 96 in fiscal year 2009. 

According to VA officials, intramural research proposals, including 
those on PTSD, are funded primarily according to scientific merit in 
both the Merit Review Program and CSP. Proposals are evaluated by a 
panel of reviewers and scored based on their scientific merit. 
Directors of VA’s research and development services—-offices that 
focus on different research areas and administer VA’s intramural 
research program-—fund proposals based on their scores, typically up 
to a specified percentile. The number of proposals funded may vary 
based on budgetary considerations and, for a small number of 
proposals, responsiveness to VA research priority areas. 

VA has a process to review and incorporate relevant research outcomes 
to develop CPGs for a number of topics, including PTSD. VA relies on 
the policies of a joint VA and Department of Defense (DOD) work group—
comprised of VA and DOD officials—to ensure that systematic reviews of 
relevant research outcomes are conducted when issuing CPGs. In brief, 
a systematic review is conducted to identify the most methodologically 
rigorous research studies that are applicable to each clinical 
question contained in the CPG. A group of subject matter experts then 
assesses the individual research studies in order to determine the 
overall quality of evidence available for each particular clinical 
question, considers the potential benefits and harms of a clinical 
intervention to determine its net effect, and, based on an assessment 
of the overall quality of the evidence and the net effect of an 
intervention, develops recommendations for the CPG. 

According to VA officials, the decision to require that two PTSD 
services—cognitive processing therapy and prolonged exposure therapy—
be made available at VA facilities by including them in the Handbook 
was based on a review of research outcomes and the availability of 
existing resources. Specifically, VA officials told GAO that these two 
services were strongly recommended in the 2004 PTSD CPG and had 
greater evidence supporting their effectiveness than other PTSD 
services. VA also told GAO that prior to the Handbook’s 2008 issuance, 
VA had already begun investing resources in training programs for 
cognitive processing therapy in 2006 and prolonged exposure therapy in 
2007. While VA provided some documentation regarding the decision-
making process for PTSD services, VA officials explained that clinical 
decision-making processes are not typically expected to be documented 
in a formal manner. VA officials told GAO that they are currently 
clarifying language in the Handbook but do not plan to revise any 
requirements relating to PTSD services at this time. 

VA provided technical comments that GAO incorporated as appropriate. 

View [hyperlink, http://www.gao.gov/products/GAO-11-32] or key 
components. For more information, contact Randall B. Williamson at 
(202) 512-7114 or williamsonr@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

PTSD Research Funding Increased from Fiscal Year 2005 through Fiscal 
Year 2009: 

VA Intramural PTSD Research Studies Are Funded Primarily According to 
Scientific Merit: 

VA Has Established a Standardized Process to Review All Relevant PTSD 
Research Outcomes and Incorporate Them into a PTSD CPG: 

VA Reviewed Research Outcomes and Used Existing Resources in 
Determining Which PTSD Services to Require Its Facilities to Provide 
or Make Available to Veterans: 

Agency Comments: 

Appendix I: VA Research Centers and Programs That Conduct or Support 
PTSD Research: 

Appendix II: The Review Process for VA's Merit Review Program: 

Appendix III: The Review Process for VA's Cooperative Studies Program: 

Appendix IV: The Process for Developing VA/DOD Evidence-Based Clinical 
Practice Guidelines: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: The Office of Research and Development's Research and 
Development Services: 

Table 2: Department of Veterans Affairs (VA) Intramural Research Study 
Funding Limits and Maximum Length of Award by Research and Development 
Service: 

Table 3: Department of Veterans Affairs (VA) Intramural Post-Traumatic 
Stress Disorder (PTSD) Research Funding and VA's Medical and 
Prosthetic Research Appropriation, Fiscal Years 2005 through 2009: 

Table 4: Scientific Merit Scoring Criteria in the Department of 
Veterans Affairs' (VA) Merit Review Program: 

Table 5: Overview of Funding Recommendations Provided by the 
Department of Veterans Affairs' (VA) Cooperative Studies Scientific 
Merit Review Board: 

Table 6: Department of Veterans Affairs (VA) Research Centers and 
Programs That Conduct or Support Post-Traumatic Stress Disorder (PTSD) 
Research: 

Table 7: Scoring Guidelines for the Department of Veterans Affairs' 
(VA) Merit Review Program: 

Table 8: Categorization of Ratings Used by the Guideline Working Group 
Established by the Department of Veterans Affairs (VA)/Department of 
Defense (DOD) Evidence-Based Practice Work Group to Identify the Level 
of Evidence of Research Outcomes: 

Table 9: Categorization of Ratings Used by the Guideline Working Group 
Established by the Department of Veterans Affairs (VA)/Department of 
Defense (DOD) Evidence-Based Practice Work Group to Determine the 
Overall Quality of Research: 

Table 10: Categorization of Ratings Used by the Guideline Working 
Group Established by the Department of Veterans Affairs 
(VA)/Department of Defense (DOD) Evidence-Based Practice Work Group to 
Determine the Net Effect of the Intervention as Described in the 
Research: 

Table 11: Grade of Evidence-Based Recommendation Assigned by the 
Guideline Working Group Established by the Department of Veterans 
Affairs (VA)/Department of Defense (DOD) Evidence-Based Practice Work 
Group Based on the Overall Quality of Evidence and the Net Effect of 
the Intervention as Described in the Research: 

Figures: 

Figure 1: Department of Veterans Affairs (VA) Intramural Funding for 
Post-Traumatic Stress Disorder (PTSD) Research Studies and Other PTSD 
Research-Related Funding, Fiscal Years 2005 through 2009: 

Figure 2: Department of Veterans Affairs (VA) Intramural Post-
Traumatic Stress Disorder (PTSD) Funding for Merit Review Program 
Studies and Cooperative Studies Program (CSP) Studies, Fiscal Years 
2005 through 2009: 

Figure 3: Number of Department of Veterans Affairs (VA) Intramural 
Post-Traumatic Stress Disorder (PTSD) Research Studies Funded per 
Fiscal Year, Fiscal Years 2005 through 2009: 

Figure 4: Overview of Submission, Review, and Funding Process in 
Department of Veterans Affairs' (VA) Merit Review Program: 

Figure 5: Overview of Submission, Review, and Funding for a Department 
of Veterans Affairs' (VA) Cooperative Studies Program (CSP) Proposal: 

Figure 6: Summary of the Department of Veterans Affairs' (VA) Merit 
Review Process for the Merit Review Program: 

Figure 7: Summary of the Department of Veterans Affairs' (VA) 
Cooperative Studies Program (CSP) Review Process: 

Figure 8: Overview of the Process Used by the Guideline Working Group 
Established by the Department of Veterans Affairs (VA)/Department of 
Defense (DOD) Evidence-Based Practice Work Group to Grade Evidence- 
Based Recommendations for Clinical Practice Guidelines: 

Abbreviations: 

CPG: clinical practice guideline: 

CSP: Cooperative Studies Program: 

DOD: Department of Defense: 

Handbook: Uniform Mental Health Services in VA Medical Centers and 
Clinics handbook: 

ORD: Office of Research and Development: 

PTSD: post-traumatic stress disorder: 

VA: Department of Veterans Affairs: 

January 24, 2011: 

The Honorable Michael H. Michaud: 
Ranking Member: 
Subcommittee on Health: 
Committee on Veterans' Affairs: 
House of Representatives: 

Dear Mr. Michaud: 

In addition to providing health care to about 5 million veterans each 
year, the Department of Veterans Affairs (VA) also funds research that 
focuses on the specific health conditions that veterans may 
experience. One condition that is examined in VA-funded research is 
post-traumatic stress disorder (PTSD), an anxiety disorder that can 
occur after a person is exposed to a life-threatening event.[Footnote 
1] Veterans diagnosed with PTSD may experience problems sleeping, 
maintaining relationships, and returning to their previous civilian 
lives.[Footnote 2] According to VA, experts estimate that up to 20 
percent of Operation Enduring Freedom and Operation Iraqi Freedom 
veterans,[Footnote 3] up to 10 percent of Gulf War veterans, and up to 
30 percent of Vietnam War veterans have experienced PTSD.[Footnote 4] 
Consequently, demand for PTSD treatment continues to grow. VA data 
show that from fiscal year 2004 through fiscal year 2008, the number 
of unique veterans receiving treatment for PTSD increased by 60 
percent from over 274,000 to over 442,000. In particular, the number 
of Operation Iraqi Freedom and Operation Enduring Freedom veterans who 
received VA treatment for PTSD increased from nearly 4,400 in fiscal 
year 2004 to over 69,000 in fiscal year 2008. 

Veterans diagnosed with PTSD can receive a range of treatments to 
manage their symptoms, including individual and group therapy and 
medication. Veterans can receive PTSD treatment on an outpatient basis 
at VA facilities such as medical centers, community-based outpatient 
clinics, and Vet Centers.[Footnote 5] Veterans can also receive 
intensive treatment through VA medical center inpatient settings for 
acute care needs and through residential rehabilitation treatment 
programs for more prolonged rehabilitative care. 

VA has guidance and policies in place related to PTSD services. For 
example, in 2004, the joint VA/Department of Defense (DOD) Evidence- 
Based Practice Work Group[Footnote 6] issued a CPG for PTSD.[Footnote 
7] The PTSD CPG is an educational tool for clinicians that provides 
evidence-based recommendations[Footnote 8] for PTSD services based on 
a review of PTSD research outcomes. It does not require clinicians to 
provide one service over another. However, in a handbook issued in 
2008--Uniform Mental Health Services in VA Medical Centers and Clinics 
(Handbook)--VA for the first time required certain mental health 
services to be made available to veterans throughout the system. 
[Footnote 9] For PTSD, the Handbook requires that cognitive processing 
therapy or prolonged exposure therapy, two evidence-based 
psychotherapies, be provided or made available at VA facilities. 
[Footnote 10] 

Through its intramural research program, VA funds studies on different 
topics, such as PTSD. VA intramural studies are conducted by VA 
investigators--that is, researchers who conduct VA research studies. 
VA intramural research refers to research that is funded by and 
conducted within VA. In fiscal year 2009, VA received an appropriation 
of $510 million for medical and prosthetic research.[Footnote 11] This 
appropriation funds VA's intramural research program, including 
providing funding for necessary equipment and supplies.[Footnote 12] 
VA also receives three medical care appropriations that support VA's 
intramural research by paying some of the costs associated with this 
research, for example, the salaries of VA investigators who are also 
VA clinicians.[Footnote 13] According to VA, in fiscal year 2009, VA's 
medical care appropriations totaled about $41 billion and VA provided 
$433 million of that amount to support all research conducted at VA 
facilities.[Footnote 14] 

Because of the growing demand for PTSD services and the importance of 
research in improving the health care services that veterans receive, 
you asked us to report on VA's funding of PTSD research, VA's 
processes for reviewing PTSD research proposals, VA's incorporation of 
research outcomes into clinical practice, and VA's process for 
determining which PTSD services it requires VA facilities to provide 
or make available. This report will describe: 

* how much funding VA provided for intramural PTSD research from its 
medical and prosthetic research appropriations from fiscal year 2005 
through fiscal year 2009, 

* how VA determines which intramural PTSD research studies will be 
funded, 

* how VA reviews PTSD research outcomes and incorporates them into its 
PTSD CPG, and: 

* how VA determines which PTSD services it requires VA facilities to 
provide or make available. 

To describe how much funding VA provided for PTSD research from its 
medical and prosthetic research appropriations through its intramural 
research program from fiscal year 2005 through fiscal year 2009, we 
obtained and summarized data provided by VA's Office of Research and 
Development (ORD), the office that manages the Merit Review Program 
and the Cooperative Studies Program (CSP)--the two primary funding 
mechanisms within VA's intramural research program.[Footnote 15] VA 
officials define PTSD research as "the effort to acquire generalizable 
knowledge about causes, epidemiology, susceptibility and genetics, 
resilience, pathophysiology, prevention and treatment of post- 
traumatic stress disorder." Based on these data, for fiscal years 2005 
through 2009, we calculated totals and trends for (1) overall 
intramural PTSD research funding--including PTSD research studies and 
other PTSD research-related funding, such as career development awards 
provided to junior VA investigators to conduct PTSD studies, salaries 
for VA investigators who are not VA clinicians,[Footnote 16] funding 
for PTSD research conducted within ORD research centers, and PTSD 
research meetings, and (2) the number and type of intramural PTSD 
research studies funded. According to VA, the funding data we obtained 
from VA do not include funds provided to conduct or support intramural 
PTSD research from VA's medical care appropriations.[Footnote 17] 
Because of this, they do not necessarily represent all the funding VA 
provided for PTSD research during this time. In addition, to obtain 
information about how VA funds research conducted at VA, we 
interviewed officials from VA's Veterans Health Administration's 
Office of Finance, ORD, and Veterans Integrated Service Networks. 
[Footnote 18] We also reviewed VA documents regarding its total 
medical and prosthetic research appropriations. We tested the internal 
consistency and reliability of the PTSD research data by reviewing the 
data for obvious outliers and performing consistency checks. We found 
the data to be sufficiently reliable for the purposes of this report. 
[Footnote 19] 

To describe how VA determines which intramural PTSD research studies 
will be funded, we interviewed VA officials responsible for managing 
VA's intramural research program. We also obtained and analyzed VA 
documents that describe VA's policies for submitting, reviewing, and 
funding intramural research proposals. In addition, to understand the 
process VA uses to evaluate intramural research proposals, we observed 
a meeting in which research proposals were evaluated. 

To describe how VA reviews PTSD research outcomes and incorporates 
them into its PTSD CPG to inform clinical practice at VA, we 
interviewed VA officials who are responsible for developing CPGs. We 
also obtained and reviewed the PTSD CPG and documents related to CPG 
development. 

To describe how VA determines which PTSD services it requires VA 
facilities to provide or make available to veterans, we interviewed VA 
officials responsible for determining these requirements. We also 
obtained and reviewed documents that describe the PTSD service 
requirements. 

We conducted this performance audit from May 2010 through November 
2010 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Background: 

VA manages its intramural research program through ORD. According to 
ORD's 2009 to 2014 strategic plan, ORD has 10 research priority areas, 
which are topics of research that are considered important to VA. 
[Footnote 20] The research priority areas are the health care needs of 
veterans who have served in Operation Enduring Freedom and Operation 
Iraqi Freedom, aging-related conditions, mental health care and well- 
being, chronic diseases, long-term care and caregiving, deployment- 
related exposure to hazardous environmental agents, equity in care, 
access in rural areas, women's health, and personalized medicine. 
According to VA officials, all of these research priority areas could 
include PTSD research. 

VA funds intramural research through the following: 

* VA's Merit Review Program: This program supports research studies 
typically conducted by one VA investigator at one VA facility and is 
administered by ORD's four research and development services, each of 
which has a different research focus. (See table 1.) Each research and 
development service is responsible for soliciting, reviewing, 
selecting, and funding research proposals submitted to the service. 

Table 1: The Office of Research and Development's Research and 
Development Services: 

Research and development service: Biomedical Laboratory; 
Focus of research: Biological or physiological principles in humans or 
animals; 
Examples of research: Investigations of tissues, blood, or other 
biologic specimens. 

Research and development service: Clinical Science; 
Focus of research: Human beings; 
Examples of research: Clinical studies examining interventions and 
effectiveness; epidemiological studies. 

Research and development service: Health Services; 
Focus of research: The interface of health care systems, patients, and 
health care outcomes; 
Examples of research: Studies examining quality, access, patient 
outcomes, and costs. 

Research and development service: Rehabilitation; 
Focus of research: Improving the quality of life of impaired and 
disabled veterans; 
Examples of research: Veteran rehabilitation studies; studies focused 
on identifying technical solutions for impaired and disabled veterans. 

Source: GAO analysis of Department of Veterans Affairs information. 

[End of table] 

* VA's CSP: This program, which is administered by Clinical Science, 
funds larger-scale, multi-site clinical trials and epidemiological 
research studies on key diseases that impact veterans. 

The Merit Review Program has research award funding limits, which are 
set by VA. In some cases, intramural research awards may only be 
funded for a certain number of years. See table 2 for more information. 

Table 2: Department of Veterans Affairs (VA) Intramural Research Study 
Funding Limits and Maximum Length of Award by Research and Development 
Service: 

Service: Biomedical Laboratory; 
Funding limit per award per fiscal year: Pilot study[A]: N/A; 
Funding limit per award per fiscal year: Full study: $150,000[B]; 
Maximum length of award (in fiscal years): Pilot study[A]: N/A; 
Maximum length of award (in fiscal years): Full study: 4. 

Service: Clinical Science; 
Funding limit per award per fiscal year: Pilot study[A]: N/A; 
Funding limit per award per fiscal year: Full study: $150,000[B]; 
Maximum length of award (in fiscal years): Pilot study[A]: N/A; 
Maximum length of award (in fiscal years): Full study: 4 (5 for 
clinical trials). 

Service: Cooperative Studies Program; 
Funding limit per award per fiscal year: Pilot study[A]: N/A; 
Funding limit per award per fiscal year: Full study: No limit; 
Maximum length of award (in fiscal years): Pilot study[A]: N/A; 
Maximum length of award (in fiscal years): Full study: No limit. 

Service: Health Services; 
Funding limit per award per fiscal year: Pilot study[A]: $100,000; 
Funding limit per award per fiscal year: Full study: $300,000[C]; 
Maximum length of award (in fiscal years): Pilot study[A]: 1; 
Maximum length of award (in fiscal years): Full study: 4. 

Service: Rehabilitation; 
Funding limit per award per fiscal year: Pilot study[A]: $75,000; 
Funding limit per award per fiscal year: Full study: $300,000; 
Maximum length of award (in fiscal years): Pilot study[A]: 2; 
Maximum length of award (in fiscal years): Full study: 4. 

Source: GAO analysis of VA information. 

[A] A pilot study is a study to establish feasibility or to develop 
data, a technique, a concept, or a procedure, which is preliminary to 
undertaking a full study. Two of VA's Office of Research and 
Development's research and development services--Health Services and 
Rehabilitation--accept pilot study proposals in addition to full study 
proposals. 

[B] According to VA officials, this limit may be exceeded for funding 
the salaries of investigators who are not VA clinicians or for 
equipment. 

[C] According to VA officials, the total over all years may not exceed 
$925,000. 

[End of table] 

In addition to individual studies conducted at VA facilities, VA has 
several research centers and programs that conduct or support PTSD 
research. For example, the National Center for PTSD focuses on PTSD 
research. VA also has Research Enhancement Award Programs, which help 
support PTSD research by providing staff and other resources to 
investigators. (For more information on VA research centers and 
research programs that conduct or support PTSD research, see appendix 
I.) According to a VA official from the National Center for PTSD, VA 
does not fund most of the PTSD research that is being conducted today. 

Origin and Criteria for Intramural Research Proposals: 

Intramural research proposals may be service-directed--solicited by 
ORD on specific topics--or investigator-initiated--submitted by 
investigators to ORD on their own initiative. Investigators submit 
proposals either in response to a request for proposals on a specific 
topic (for service-directed proposals) or to an open request for 
proposals (for investigator-initiated proposals). For both the Merit 
Review Program and CSP, proposals are typically evaluated in two 
review cycles per year.[Footnote 21] 

To be considered for intramural research funding:[Footnote 22] 

* The proposal must be veteran-centric. 

* The proposal must have received approval from the director of the 
medical center and the research and development office of the medical 
center where the lead investigator, known as a principal investigator, 
is based.[Footnote 23] 

* The principal investigator and any co-principal investigators must 
demonstrate a primary professional commitment to VA, as demonstrated 
by at least a 5/8 time VA appointment at the time the funding is 
awarded and previous VA experience, including experience in research 
and patient care.[Footnote 24] 

* Research must be conducted primarily on VA premises. The principal 
investigator and any co-principal investigators must have designated 
research space within a VA medical center.[Footnote 25] 

PTSD Research Funding Increased from Fiscal Year 2005 through Fiscal 
Year 2009: 

Overall intramural PTSD research funding from VA's medical and 
prosthetic research appropriation increased from $9.9 million in 
fiscal year 2005 to $24.5 million in fiscal year 2009. The number of 
intramural PTSD research studies funded through the Merit Review 
Program and CSP increased from 47 in fiscal year 2005 to 96 in fiscal 
year 2009. 

VA Intramural PTSD Research Funding Increased from $9.9 Million in 
Fiscal Year 2005 to $24.5 Million in Fiscal Year 2009: 

Based on the VA data we obtained and summarized, we found that overall 
intramural PTSD research funding from VA's medical and prosthetic 
research appropriation increased from about $9.9 million in fiscal 
year 2005 to about $24.5 million in fiscal year 2009, or by about 150 
percent (see figure 1). Overall intramural PTSD research funding 
included funding for specific PTSD studies as well as for other PTSD 
research-related funding, such as career development awards provided 
to junior VA investigators to conduct PTSD studies, salaries for VA 
investigators who are not VA clinicians,[Footnote 26] funding for PTSD 
research conducted within ORD research centers, and PTSD research 
meetings. 

Figure 1: Department of Veterans Affairs (VA) Intramural Funding for 
Post-Traumatic Stress Disorder (PTSD) Research Studies and Other PTSD 
Research-Related Funding, Fiscal Years 2005 through 2009: 

[Refer to PDF for image: stacked vertical bar graph] 

Fiscal year: 2005; 
Funding for PTSD research studies: $9.4 million; 
Other PTSD-related research funding: $0.5 million; 
Total: $9.9 million. 

Fiscal year: 2006; 
Funding for PTSD research studies: $12.7 million; 
Other PTSD-related research funding: $0.6 million; 
Total: $13.3 million. 

Fiscal year: 2007; 
Funding for PTSD research studies: $14.8 million; 
Other PTSD-related research funding: $0.9 million; 
Total: $15.7 million. 

Fiscal year: 2008; 
Funding for PTSD research studies: $19.8 million; 
Other PTSD-related research funding: $1.4 million; 
Total: $21.2 million. 

Fiscal year: 2009; 
Funding for PTSD research studies: $23.5 million; 
Other PTSD-related research funding: $1.0 million; 
Total: $24.5 million. 

Source: GAO analysis of VA data. 

[End of figure] 

Of the $80.2 million provided for PTSD studies from fiscal year 2005 
through fiscal year 2009, $51.3 million, or about 64 percent, was for 
studies funded through the Merit Review Program. The remaining 
approximately $28.9 million, or about 36 percent, was for CSP studies. 
(See figure 2.) 

Figure 2: Department of Veterans Affairs (VA) Intramural Post-
Traumatic Stress Disorder (PTSD) Funding for Merit Review Program 
Studies and Cooperative Studies Program (CSP) Studies, Fiscal Years 
2005 through 2009: 

[Refer to PDF for image] 

[Refer to PDF for image: stacked vertical bar graph] 

Fiscal year: 2005; 
Funding for Merit Review Program studies: $6.4 million; 
Funding for CSP studies: $3.0 million; 
Total: $9.4 million. 

Fiscal year: 2006; 
Funding for Merit Review Program studies: $8.3 million; 
Funding for CSP studies: $4.4 million; 
Total: $12.7 million. 

Fiscal year: 2007; 
Funding for Merit Review Program studies: $10.7 million; 
Funding for CSP studies: $4.1 million; 
Total: $14.8 million. 

Fiscal year: 2008; 
Funding for Merit Review Program studies: $12.2 million; 
Funding for CSP studies: $7.5 million; 
Total: $19.7 million. 

Fiscal year: 2009; 
Funding for Merit Review Program studies: $13.6 million; 
Funding for CSP studies: $9.9 million; 
Total: $23.5 million. 

Source: GAO analysis of VA data. 

[End of figure] 

From fiscal year 2005 through fiscal year 2009, intramural PTSD 
research funding ranged from 2.5 percent to 4.8 percent of VA's 
medical and prosthetic research appropriation. (See table 3 for VA 
intramural PTSD research funding and VA's medical and prosthetic 
research appropriations from fiscal year 2005 through fiscal year 
2009.) For comparison,[Footnote 27] according to a 2009 report 
prepared by ORD staff for VA's National Research Advisory Council, 
[Footnote 28] for fiscal year 2009, funding for intramural traumatic 
brain injury research was about $14.6 million, 2.9 percent of the 
medical and prosthetic research appropriation. Funding for spinal cord 
injury research was $27.2 million, 5.3 percent of the medical and 
prosthetic research appropriation. Funding for intramural 
cardiovascular disease and stroke research was $53.1 million, 10.4 
percent of the medical and prosthetic research appropriation. 

Table 3: Department of Veterans Affairs (VA) Intramural Post-Traumatic 
Stress Disorder (PTSD) Research Funding and VA's Medical and 
Prosthetic Research Appropriation, Fiscal Years 2005 through 2009: 

Fiscal year: 2005; 
VA intramural PTSD research funding: $9.9 million; 
VA's medical and prosthetic research appropriation: $390.2 million; 
Percentage: 2.5%. 

Fiscal year: 2006; 
VA intramural PTSD research funding: $13.3 million; 
VA's medical and prosthetic research appropriation: $412.0 million; 
Percentage: 3.2%. 

Fiscal year: 2007; 
VA intramural PTSD research funding: $15.7 million; 
VA's medical and prosthetic research appropriation: $446.5 million; 
Percentage: 3.5%. 

Fiscal year: 2008; 
VA intramural PTSD research funding: $21.3 million; 
VA's medical and prosthetic research appropriation: $480.0 million; 
Percentage: 4.4%. 

Fiscal year: 2009; 
VA intramural PTSD research funding: $24.5 million; 
VA's medical and prosthetic research appropriation: $510.0 million; 
Percentage: 4.8%. 

Total: 
VA intramural PTSD research funding: $84.7 million; 
VA's medical and prosthetic research appropriation: $2.239 billion; 
Percentage: 3.8%. 

Source: GAO analysis of VA data. 

Note: VA intramural research is also supported by VA's medical care 
appropriations, which are not represented in this table. For example, 
the medical care appropriations pay for the salaries of VA 
investigators who are VA clinicians (according to VA, about 70 percent 
of VA investigators are VA clinicians), administrative costs such as 
payroll and human resources, and logistical and infrastructure costs. 

[End of table] 

The Number of PTSD Studies Funded through VA's Intramural Research 
Program Increased from 47 in Fiscal Year 2005 to 96 in Fiscal Year 
2009: 

Similarly, we found that the number of PTSD studies funded from VA's 
medical and prosthetic research appropriations through VA's intramural 
research program increased from fiscal year 2005 through fiscal year 
2009. (See figure 3.) Specifically, in fiscal year 2005, 47 intramural 
PTSD research studies were funded while in fiscal year 2009, 96 
intramural PTSD research studies were funded.[Footnote 29] This 
represented an increase of more than 100 percent. Of all the studies 
funded each fiscal year, only a small number were CSP studies. 

Figure 3: Number of Department of Veterans Affairs (VA) Intramural 
Post-Traumatic Stress Disorder (PTSD) Research Studies Funded per 
Fiscal Year, Fiscal Years 2005 through 2009: 

[Refer to PDF for image: stacked vertical bar graph] 

Fiscal year: 2005; 
Cooperative Studies Program studies: 44; 
Merit Review Program studies: 3; 
Total: 47. 

Fiscal year: 2006; 
Cooperative Studies Program studies: 60; 
Merit Review Program studies: 3; 
Total: 63. 

Fiscal year: 2007; 
Cooperative Studies Program studies: 70; 
Merit Review Program studies: 3; 
Total: 73. 

Fiscal year: 2008; 
Cooperative Studies Program studies: 79; 
Merit Review Program studies: 5; 
Total: 84. 

Fiscal year: 2009; 
Cooperative Studies Program studies: 80; 
Merit Review Program studies: 6; 
Total: 86. 

Source: GAO analysis of VA data. 

[End of figure] 

VA Intramural PTSD Research Studies Are Funded Primarily According to 
Scientific Merit: 

According to VA officials, intramural research proposals, including 
those on PTSD, are reviewed and funded in VA's Merit Review Program 
and VA's CSP primarily according to scientific merit. 

VA's Merit Review Program: 

Intramural research proposals submitted to VA's Merit Review Program 
are reviewed through a series of steps prior to funding. See figure 4 
for an overview of the submission, review, and funding process for 
proposals submitted to the Merit Review Program. (For more detailed 
information on this process, see appendix II.) 

Figure 4: Overview of Submission, Review, and Funding Process in 
Department of Veterans Affairs' (VA) Merit Review Program: 

[Refer to PDF for image: illustration] 

1) Principal investigator submits proposal electronically[A]; 

2) Proposal is assigned to a merit review panel based on topic; 

3) Merit review panel scores proposal for scientific merit; proposals 
are ranked based on scores; 

4) Director funds highest-ranked proposals and may fund proposals that 
respond to research priority areas. 

Source: GAO analysis of VA documents and GAO interviews with VA 
officials. 

[A] Proposals are submitted electronically to grants.gov, the 
government's central grant identification and proposal portal. 
Proposals are then transferred to eRA Commons, an electronic system 
for grant administration functions, for VA processing and review. 

[End of figure] 

First, investigators submit proposals electronically. Investigators 
typically submit Merit Review Program proposals to grants.gov, the 
government's central grant identification and proposal portal, 
[Footnote 30] in response to a request for proposals. Submitted 
proposals are then transferred to eRA Commons, an electronic system 
for grant administration functions, for VA processing and review. 
[Footnote 31] 

Second, each proposal is assigned to a merit review panel for 
evaluation. Each merit review panel reviews proposals in a specific 
research topic area, and is composed of panelists, typically associate-
level professors, who are selected based on their expertise in this 
area.[Footnote 32] According to VA documents, as of 2010, there were a 
total of 35 merit review panels across VA's research and development 
services.[Footnote 33] The merit review panels evaluate each proposal 
based on its scientific merit.[Footnote 34] Panelists consider several 
criteria in evaluating the overall scientific merit of a proposal. 
(See table 4 for criteria used to determine scientific merit.) 

Table 4: Scientific Merit Scoring Criteria in the Department of 
Veterans Affairs' (VA) Merit Review Program: 

Criteria: Significance; 
Description: Supports/advances the health and health care of veterans 
and research field in general; addresses important scientific 
question/area; makes a potential contribution to scientific literature. 

Criteria: Approach; 
Description: Incorporates current scientific/theoretical bases; 
is hypothesis-driven; uses appropriate research design/methods for 
addressing hypothesis; ensures that feasibility of methods is clear. 

Criteria: Innovation; 
Description: Addresses new concepts, gaps, or both, in the research 
area; addresses potential for impact of findings on existing field of 
research, treatment paradigms, or both. 

Criteria: Environment; 
Description: Has appropriate knowledge/background and resources (e.g., 
equipment and staff) to ensure completion of study. 

Criteria: Feasibility; 
Description: Provides sufficient evidence to determine that the 
proposed study can be successfully conducted and completed. 

Criteria: Investigator; 
Description: Ensures that investigators involved in the proposed study 
are appropriately trained and have expertise in the proposed area of 
research. 

Source: GAO analysis of VA information. 

[End of table] 

Third, the merit review panelists score the proposals to determine 
their rank. Each panelist provides a score to each of the proposals 
reviewed by the panel. The scores are averaged to create a "priority 
score" for the proposal.[Footnote 35] (See appendix II for specific 
scoring guidelines given to panelists in all research and development 
services.) All proposals scored by the merit review panel are then 
ranked by priority score among all of the proposal scores recently 
assigned by the merit review panel. The rank of the proposal is used 
to determine the "percentile" of the proposal.[Footnote 36] 

Finally, research and development service directors determine how many 
proposals to fund. All of the proposals scored by all merit review 
panels in a research and development service in the review cycle are 
ranked together by their percentiles to be considered for funding. 
According to VA officials, research and development service directors 
typically fund up to the 25th percentile of proposals in a review 
cycle, beginning with those with the most scientific merit, although 
the number of proposals funded may vary depending on the budget. 
According to VA, research and development service directors may also 
choose to fund a small number of additional proposals at the margin 
that respond to research priority areas. For example, if the fundable 
range determined by a research and development service director was up 
to the 25th percentile, proposals at the 26th percentile related to 
research priority areas could also be considered for funding.[Footnote 
37] 

VA's Cooperative Studies Program: 

VA intramural research proposals submitted to CSP are reviewed and 
scored in a process similar to that of the Merit Review Program prior 
to consideration for funding. To help develop the CSP proposal, 
investigators are assisted by members of a CSP center,[Footnote 38] a 
VA entity that provides guidance and support for research across 
multiple sites. (See figure 5 for an overview of the process for 
submitting, reviewing, and funding a CSP proposal. For more 
information on ORD's CSP review process, see appendix III.) 

Figure 5: Overview of Submission, Review, and Funding for a Department 
of Veterans Affairs' (VA) Cooperative Studies Program (CSP) Proposal: 

[Refer to PDF for image: illustration] 

1) CSP center submits proposal to CSP central office[A]; 

2) CSP Merit Review Board scores proposals and recommends funding for 
those with the most scientific merit; 

3) Proposals recommended for funding are provided to the Clinical 
Science Director; 

4) Clinical Science Director funds proposals with the most scientific 
merit. 

Source: GAO analysis of VA documents and GAO interviews with VA 
officials. 

[A] Prior to submitting the proposal to CSP central office, the 
principal investigator must have a letter of intent approved by the 
Director of Clinical Science. 

[End of figure] 

Before submitting a research proposal, investigators submit a letter 
of intent, or a preliminary outline of a proposal, to the Director of 
Clinical Science to be approved for planning a CSP proposal.[Footnote 
39] Based on the merit of the letter of intent, as determined by three 
or more reviewers,[Footnote 40] the Clinical Science Director decides 
whether to fund planning efforts to develop a CSP proposal.[Footnote 
41] 

When the principal investigator receives approval to begin planning 
efforts, the Clinical Science Director assigns a CSP center to provide 
statistical and methodological guidance to the investigator. The 
director of the CSP center designates a project manager and 
methodologist, such as a person with expertise in biostatistics, to 
provide guidance to the principal investigator. The Clinical Science 
Director, with recommendations from the principal investigator, then 
forms a planning committee of additional experts to assist in 
developing a CSP proposal. The planning committee develops a CSP 
proposal over the course of two planning meetings. 

Once a proposal is developed, the CSP center, on behalf of the 
principal investigator, submits a hard copy proposal to the CSP 
central office for evaluation by the Cooperative Studies Scientific 
Merit Review Board.[Footnote 42] This board consists of reviewers who 
have extensive experience in clinical research and the conduct of 
clinical trials or epidemiology studies.[Footnote 43] Reviewers 
evaluate CSP proposals based on scientific merit. According to VA, the 
scientific merit of a CSP proposal is defined by the importance of the 
proposal, its feasibility, the clarity and achievability of its 
objectives, the adequacy of the plan of investigation, the correctness 
of the technical details, and the adequacy of the safeguards for the 
welfare of the patients.[Footnote 44] Based on these criteria, 
reviewers discuss the general scientific merit of the proposal. 
Reviewers vote on whether to unconditionally approve, conditionally 
approve, reject or defer with recommendation for resubmittal, or 
reject each proposal. (See table 5 for an overview of funding 
recommendations provided by the board.) 

Table 5: Overview of Funding Recommendations Provided by the 
Department of Veterans Affairs' (VA) Cooperative Studies Scientific 
Merit Review Board: 

Recommendation for funding: Unconditional approval; 
Definition: The proposal is approved without changes or additions to 
the proposal. 

Recommendation for funding: Conditional approval; 
Definition: The proposal is approved pending certain changes or 
additions to the proposal. 

Recommendation for funding: Rejection or defer with recommendation for 
resubmittal; 
Definition: The proposal is considered to be worthwhile, but is in 
need of major revisions. A revised proposal can be resubmitted to the 
Clinical Science Director. 

Recommendation for funding: Rejection; 
Definition: The proposal is rejected. The principal investigator must 
submit a new request for planning to submit a revised proposal. 

Source: GAO analysis of a VA document. 

[End of table] 

After the reviewers vote, they each provide scores for a proposal 
recommended for funding based on scientific merit.[Footnote 45] The 
scores are then averaged to provide a priority score for a proposal. 
Finally, the Clinical Science Director considers the priority scores 
of all the proposals in that review cycle and selects the proposals 
with the strongest priority scores for funding. According to VA 
officials, the number of proposals funded may vary depending on the 
budget.[Footnote 46] 

VA Has Established a Standardized Process to Review All Relevant PTSD 
Research Outcomes and Incorporate Them into a PTSD CPG: 

The VA/DOD Evidence-Based Practice Work Group, which is responsible 
for developing and updating all of VA's CPGs, has a standardized and 
reproducible process to review all relevant research outcomes when 
developing or updating all CPGs, including the PTSD CPG. To develop or 
update a CPG, the VA/DOD Evidence-Based Practice Work Group identifies 
and assigns a group of VA and DOD clinical leaders and experts who are 
knowledgeable in the subject area to work on the CPG.[Footnote 47] 
Generally, the process to develop or update a CPG consists of the 
following steps. 

* First, the assigned group of VA and DOD clinical leaders and experts 
identifies "clinical questions" that will be answered in the CPG. 
According to VA officials, clinical questions can be either broad or 
specific. For example, the 2004 PTSD CPG contained clinical questions 
regarding whether early intervention is more effective than later 
intervention, and whether certain interventions, such as different 
psychotherapies, are more effective than others. 

* Second, in order to minimize bias, an external contractor conducts a 
systematic review of relevant research and selects and summarizes the 
most methodologically rigorous research studies that are applicable to 
each of the clinical questions. 

* Third, after receiving summaries of the studies with the highest 
level of evidence, the VA and DOD group of clinical leaders and 
experts rates the research using an established grading scheme that 
considers: 

- the level of evidence of each research study--the scope and 
methodological rigor of an individual study; 

- the overall quality of evidence--the overall quality of all of the 
research that addresses a particular clinical question, considering 
the level of evidence of all the studies considered; and: 

- the net effect of an intervention--according to the collective 
results of the studies considered, the intervention's benefits minus 
the intervention's harms. 

* Finally, the assigned group of VA and DOD clinical leaders and 
experts assigns a grade to each evidence-based recommendation based on 
an assessment of the overall quality of evidence and the net effect of 
the intervention. (See appendix IV for a detailed description of the 
process used to develop evidence-based VA/DOD CPGs.) 

The process for conducting a systematic review of research outcomes to 
develop or update a CPG is repeated as often as is deemed necessary by 
the VA/DOD Evidence-Based Practice Work Group according to its written 
procedures and designated time frames.[Footnote 48] According to 
VA/DOD Evidence-Based Practice Work Group documents, routine updates 
to the CPGs should ideally occur approximately every 2 years. However, 
updates to CPGs often do not occur every 2 years, and VA officials 
told us that some CPGs are updated more frequently than others based 
on availability of resources and priority areas. Additionally, VA 
officials reported that a CPG will be immediately updated if any 
evidence-based recommendation contained in it is identified as harmful 
to patients. 

According to VA, the VA/DOD Evidence-Based Practice Work Group 
approved an update to the 2004 PTSD CPG on October 25, 2010, and 
published the update on VA's Web site on November 17, 2010.[Footnote 
49] According to VA officials, the systematic process outlined above 
was used to review all relevant research outcomes and make evidence-
based recommendations for PTSD services to both develop and update the 
PTSD CPG. 

VA Reviewed Research Outcomes and Used Existing Resources in 
Determining Which PTSD Services to Require Its Facilities to Provide 
or Make Available to Veterans: 

According to VA officials, the decision to require that cognitive 
processing therapy and prolonged exposure therapy be made available to 
veterans diagnosed with PTSD at VA facilities--as indicated in the 
Handbook, which established certain requirements for mental health 
services within VA--was based on a review of research outcomes and the 
availability of existing resources. 

* Review of research outcomes. According to VA, agency officials and 
qualified subject matter experts reviewed relevant research outcomes 
and the quality of the research to determine the most efficacious PTSD 
treatments available when determining which PTSD services to include 
in the Handbook and make available to veterans. Specifically, VA 
officials told us that their decision to include cognitive processing 
therapy and prolonged exposure therapy in the Handbook was influenced 
by the fact that both of these had been graded as level "A" treatments 
in the 2004 PTSD CPG (indicating that the intervention is always 
indicated and acceptable).[Footnote 50] Furthermore, VA officials said 
that these two therapies had greater evidence supporting their 
effectiveness than other PTSD services also graded as level "A" in the 
2004 PTSD CPG.[Footnote 51] In addition, VA officials added that their 
decision was validated by the results of a VA-commissioned Institute 
of Medicine study published in 2008 that reviewed the evidence for 
existing PTSD treatments. According to VA, the study found that 
cognitive processing therapy and prolonged exposure therapy were 
considered efficacious treatments for PTSD.[Footnote 52] While the 
Institute of Medicine report was released after VA had already decided 
to include cognitive processing therapy and prolonged exposure therapy 
in the Handbook, VA officials explained that the Institute of Medicine 
report was the basis for the decision not to include other PTSD 
services in the Handbook. 

* Availability of existing resources. VA officials told us that prior 
to issuing the Handbook in 2008, VA had already begun investing 
considerable resources to implement national training programs for 
cognitive processing therapy and prolonged exposure therapy in 2006 
and 2007, respectively. VA officials said that they decided to 
implement the national training programs because VA realized the need 
to create sufficient capacity so that evidence-based PTSD treatments 
could be available to veterans throughout the VA system. VA explained 
that the national training programs were rolled out in advance of the 
Handbook's issuance as part of the implementation of VA's 
Comprehensive Veterans Health Administration Strategic Plan for Mental 
Health Services, which called for rapid implementation of evidence-
based treatments. VA did this to ensure that it had the capacity to 
provide cognitive processing therapy and prolonged exposure therapy to 
all veterans with PTSD for whom these treatments were clinically 
appropriate.[Footnote 53] VA officials said that they were able to 
begin implementing national training programs for cognitive processing 
therapy in 2006 and prolonged exposure therapy in 2007 because VA had 
qualified instructors to administer the programs and money available 
to fund them.[Footnote 54] 

Unlike the written and standardized process that the VA/DOD Evidence- 
Based Practice Work Group established to develop CPGs, VA does not 
have a formal written process or framework to explain its decision for 
including cognitive processing therapy and prolonged exposure therapy 
in the Handbook.[Footnote 55] VA officials explained that they 
followed a process when choosing cognitive processing therapy and 
prolonged exposure therapy, but added that clinical decision-making 
processes are not typically expected to be documented in a formal 
manner. 

VA officials told us that they plan to assess the implementation of 
the Handbook and will update PTSD requirements in it as needed or as 
new information or unexpected obstacles arise in the future.[Footnote 
56] VA officials stated that they are currently clarifying the 
language regarding some of the requirements, but do not plan to revise 
any of the requirements relating to PTSD services at this time. 

Agency Comments: 

We provided a draft of this report to VA and received technical 
comments, which we incorporated into our report as appropriate. 

We are sending a copy of this report to the Secretary of Veterans 
Affairs. The report also is available at no charge on the GAO Web site 
at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. GAO staff members who made key 
contributions to this report are listed in appendix V. 

Sincerely yours, 

Signed by: 

Randall B. Williamson: 
Director, Health Care: 

Enclosures - V: 

[End of section] 

Appendix I: VA Research Centers and Programs That Conduct or Support 
PTSD Research: 

In addition to post-traumatic stress disorder (PTSD) research studies 
that are conducted by individual Department of Veterans Affairs (VA) 
investigators, or researchers, VA also funds a number of research 
centers or programs that conduct or support PTSD research. See table 6 
for a description of these VA research centers and programs. 

Table 6: Department of Veterans Affairs (VA) Research Centers and 
Programs That Conduct or Support Post-Traumatic Stress Disorder (PTSD) 
Research: 

Office of Mental Health Services: 

VA research centers/programs: The National Center for PTSD; 
Description: Advances the clinical care and social welfare of veterans 
through research, education, and training in the science, diagnosis, 
and treatment of PTSD and stress-related disorders; 
Established by Congress? Yes; 
Number of centers or programs with a primary focus on PTSD: 1[A]. 

VA research centers/programs: Centers of Excellence; 
Description: Research the causes and treatments of mental disorders 
and use the dissemination of education to implement new knowledge into 
routine VA clinical practices; 
Established by Congress? Yes; 
Number of centers or programs with a primary focus on PTSD: 2. 

VA research centers/programs: Mental Illness Research, Education, and 
Clinical Centers; 
Description: Research the causes and treatments of mental disorders 
and use education to implement knowledge into routine clinical 
practice in VA; 
Established by Congress? Yes; 
Number of centers or programs with a primary focus on PTSD: 4. 

Office of Research and Development: 

VA research centers/programs: Centers of Excellence; 
Description: Create or maintain a core program of investigators to 
support and facilitate the development of research studies and train 
and mentor new investigators; 
Established by Congress? No; 
Number of centers or programs with a primary focus on PTSD: 8. 

VA research centers/programs: Quality Enhancement Research Initiative 
Program; 
Description: Enhances the quality and outcomes of VA health care by 
implementing research findings and evidence-based recommendations into 
routine clinical practice; 
Established by Congress? No; 
Number of centers or programs with a primary focus on PTSD: 1[B]. 

VA research centers/programs: Research Enhancement Award Programs; 
Description: Support groups of VA investigators that address specific 
medical problems of veterans to promote innovative research, train new 
medical research investigators, and foster new research collaborations 
among investigators; 
Established by Congress? No; 
Number of centers or programs with a primary focus on PTSD: 1. 

Source: GAO analysis of VA information. 

[A] The National Center for PTSD, headquartered in White River 
Junction, Vermont, has seven divisions in five locations across the 
country. 

[B] Although the Mental Health Quality Enhancement Research Initiative 
(one of nine Quality Enhancement Research Initiative Centers) focuses 
on depression and schizophrenia, Office of Research and Development 
officials stated that within the past few years it has become more 
involved with PTSD research. 

[End of table] 

[End of section] 

Appendix II: The Review Process for VA's Merit Review Program: 

Research proposals submitted to the Department of Veterans Affairs' 
(VA) Merit Review Program are evaluated in merit review panels that 
each review proposals in a specific research topic area. Each merit 
review panel is comprised of panelists, typically associate-level 
professors, who are selected based on their expertise in the area. 
Panelists are responsible for scoring proposals based on scientific 
merit to provide funding recommendations. See figure 6 for a detailed 
description of the Merit Review Program's process for reviewing 
research proposals and table 7 for the Merit Review Program's scoring 
guidelines. 

Figure 6: Summary of the Department of Veterans Affairs' (VA) Merit 
Review Process for the Merit Review Program: 

[Refer to PDF for image: illustration] 

1. The portfolio manager, an official responsible for managing a 
specific area of research, assigns three panelists as the primary, 
secondary, and tertiary panelists for each research proposal. These 
panelists are responsible for reviewing the proposal prior to the 
panel meeting, and individually submitting a preliminary score and 
summary of the proposal into eRA Commons, an electronic system for 
grant administration functions, for VA processing and review. The 
preliminary scores range from 1.0 and 5.0 (with scores closer to 1.0 
being more meritorious). 

2. The primary, secondary, and tertiary panelists present their 
preliminary scores to the entire panel[A] and respond to feedback 
provided by the other panelists about the proposal. 

3. Based on the discussion at the panel meeting, the primary, 
secondary, and tertiary panelists can change their preliminary scores. 
The primary, secondary, and tertiary panelists must announce to the 
panel the final score each plans to give the proposal. The final 
scores also range from 1.0 and 5.0 (with scores closer to 1.0 being 
more meritorious). 

4. The portfolio manager asks if any panelist plans to score the 
research proposal 0.3 points above or below the range of the final 
scores provided by the primary, secondary, and tertiary panelists. If 
any panelist intends to score outside of this range, the panelist must 
discuss this decision with the rest of the panel and explain reasons 
for doing so. 

5. All panelists on the review panel then submit a final score for the 
research proposal. The average of the final scores multiplied by 100 
is known as the priority score. In addition, the primary panelist 
writes a summary of the discussion of the panel meeting and submits 
the summary into eRA Commons. 

Source: GAO analysis of VA documents and GAO interviews with VA 
officials. 

[A] According to VA's policy, panelists who have a conflict of 
interest with a given proposal are required to leave the meeting 
before the proposal is discussed and do not score the proposal. 
According to VA, a conflict of interest exists when a reviewer has an 
interest in a research proposal that is likely to bias his or her 
evaluation of it. Panelists are expected to inform the portfolio 
manager if they have a conflict of interest. 

[B] In some instances, research proposals are streamlined, meaning 
that they are evaluated only by the primary, secondary, and tertiary 
panelists, and are not discussed by the entire merit review panel. 
Proposals may be streamlined if the average of the primary, secondary, 
and tertiary panelists' preliminary scores falls in the upper half of 
the range of scores, indicating less merit. According to VA officials, 
proposals with a preliminary score of lower than 2.2, indicating very 
good or excellent, are also streamlined for two research and 
development services. (See table 7 for merit review process scoring 
guidelines.) In these instances, the administrator of the merit review 
panel meeting asks the panelists if anyone has an objection to 
streamlining the proposal. If a panelist objects, the proposal is 
discussed in the merit review panel meeting and it is not streamlined. 
Streamlined proposals with preliminary scores that fall in the upper 
half of the range of scores, indicating less merit, are not considered 
for funding. 

[End of figure] 

Table 7: Scoring Guidelines for the Department of Veterans Affairs' 
(VA) Merit Review Program: 

Scoring range and grade: 1.0 - 1.5 Excellent; 
Description: Proposed research addresses important scientific area 
that lacks needed knowledge. Hypothesis(es) is clearly stated and 
research design/methodology is appropriate. Research is innovative, 
representing state-of-the-art science. Potential findings may have a 
vital role in advancing the health and health care of veterans and 
research in general. Resources listed suggest a very high probability 
of the study's completion. 

Scoring range and grade: 1.6 - 2.2 Very good; 
Description: Proposed research addresses important scientific area. 
Hypothesis(es) is clearly stated and the research design/methodology 
is appropriate with a few minor exceptions. Potential findings may 
have an important role to the health and health care of veterans and 
research in general. Resources listed suggest a high probability of 
the study's completion. 

Scoring range and grade: 2.3 - 2.8 Good; 
Description: Proposed research addresses a valid area of 
investigation. Hypothesis(es) is clearly stated, but research 
design/methodology contains key flaws that should be corrected. 
Potential findings may contribute to the health and health care of 
veterans and research in general. Resources listed suggest that the 
study could be completed. 

Scoring range and grade: 2.9 - 3.4 Fair; 
Description: Proposed research requires further preliminary data to 
warrant investigation as a viable area of research. Hypothesis(es) is 
not clear, research design/methods contain significant flaws, or both. 
It is not clear how potential findings would contribute to the health 
and health care of veterans and research in general. It is unclear 
whether the resources listed are sufficient to ensure study completion. 

Scoring range and grade: 3.5 - 5.0 Poor; 
Description: Proposed research does not appear to address an important 
scientific question/area. Hypothesis(es) is not clearly stated, 
research design/methodology is inappropriate or contains uncorrectable 
flaws, or both. Design/methodological limitations hinder any 
significant conclusions that would contribute to the health and health 
care of veterans, research in general, or both. Resources listed do 
not suggest that the study will be completed. 

Source: GAO analysis of VA documents. 

[End of table] 

[End of section] 

Appendix III: The Review Process for VA's Cooperative Studies Program: 

Research proposals submitted to the Department of Veterans Affairs' 
(VA) Cooperative Studies Program (CSP) are reviewed and scored by the 
Cooperative Studies Scientific Merit Review Board. Reviewers on the 
board are chosen based on their expertise in clinical or 
epidemiological research. They typically serve 4-year terms, and ad 
hoc members can be added depending on specific expertise that may be 
needed to review a proposal. According to VA, as of October 2010, 
there were six reviewers on the board. 

During the research proposal review process, the study team--which 
includes the lead researcher (referred to as the principal 
investigator) and a methodologist, such as a person with expertise in 
biostatistics--has an interactive discussion with the board regarding 
the proposal.[Footnote 57] Reviewers evaluate CSP proposals based on 
scientific merit and provide scores to reflect their funding 
recommendations. See figure 7 for a detailed description of the review 
process for CSP research proposals. 

Figure 7: Summary of the Department of Veterans Affairs' (VA) 
Cooperative Studies Program (CSP) Review Process: 

[Refer to PDF for image: illustration] 

1. The Deputy Director of CSP assigns three individuals on the 
Cooperative Studies Scientific Merit Review Board to be the initial 
reviewers for a CSP proposal. One of the initial reviewers is 
designated as the primary reviewer. All three initial reviewers 
prepare written reviews of the proposal based on scientific merit 
prior to the board meeting.[A] 

2. A first board meeting is held to summarize and discuss the key 
critiques of the written reviews provided by the initial reviewers. 
The primary reviewer is responsible for introducing the proposal to 
the board. The study team, which includes the principal investigator 
and a methodologist, such as a person with expertise in biostatistics, 
is not present at this meeting. 

3. A second board meeting is held with the study team.[B] At the 
beginning of this meeting, the principal investigator is provided a 
summary of the critiques discussed in the first board meeting. The 
study team is then given 15 minutes to provide a summary of the 
proposal, including its importance to VA, and to address the 
critiques.[B] 

4. After the study team responds to the board, the board and the study 
team engage in an interactive discussion regarding the proposal. 

5. After the interactive discussion, the study team is dismissed and 
the board votes on whether to unconditionally approve, conditionally 
approve, reject or defer with recommendation for resubmittal, or 
reject funding of the proposal.[D] For proposals that are 
conditionally and unconditionally approved, each member of the board 
assigns a numeric rating from 10 to 50, with lower scores indicating 
more meritorious proposals, based on scientific merit. The average of 
the scores for each proposal is considered the priority score.[E] 

Source: GAO analysis of VA documents and GAO interviews with VA 
officials. 

[A] According to a VA document, the primary reviewers are typically a 
biostatistician and a clinician. Reviewers are asked to comment on the 
importance of the proposal, its feasibility, the clarity and 
achievability of its objectives, the adequacy of the plan of 
investigation, the correctness of the technical details, the adequacy 
of the safeguards for the welfare of the patients, and any other 
pertinent features of the proposal. According to VA, these criteria 
are components of scientific merit. 

[B] For CSP proposals that involve economic analysis, a health 
economist is also present at the board meeting to discuss the 
proposal. In addition, the principal investigator may ask the Clinical 
Science Director to allow consultants to be present at the board 
meeting if review of the proposal requires expertise in a specific 
area. 

[C] According to VA, if there are any economic issues raised relating 
to the proposal, the study team will be given an additional 5 minutes 
to address these issues with a health economist. 

[D] According to a VA document, unconditional approval means a 
proposal is approved by the board without changes and is recommended 
for funding. Conditional approval means that the proposal is approved 
by the board pending certain changes or additions to the proposal. 
Reject or defer with recommendation for resubmittal means the board 
considers the proposal to be worthwhile, but it is need of major 
revisions. In this case, if the principal investigator chooses to 
submit a revised proposal, the Clinical Science Director may waive the 
requirement to submit a letter of intent. If the board rejects the 
proposal, the principal investigator must submit a new letter of 
intent to resubmit a revised proposal. 

[E] After the board scores the proposal, a human rights committee 
typically provides a general assessment of the protection of patients' 
rights and welfare as described in the proposal. For studies with 
unique ethical considerations, this review may occur prior to the 
review by the Cooperative Studies Scientific Merit Review Board. 

[End of figure] 

[End of section] 

Appendix IV: The Process for Developing VA/DOD Evidence-Based Clinical 
Practice Guidelines: 

In 1999, the Department of Veterans Affairs (VA) and the Department of 
Defense (DOD) formed the VA/DOD Evidence-Based Practice Work Group to 
issue joint VA/DOD clinical practice guidelines (CPG)--tools that 
provide guidance and evidence-based recommendations to clinicians 
regarding the most effective interventions and services for a variety 
of health care topics.[Footnote 58] To develop or update a CPG, the 
VA/DOD Evidence-Based Practice Work Group has a standardized process 
to ensure that systematic reviews of relevant research outcomes are 
conducted in order to formulate evidence-based recommendations for 
prevention, assessment, and treatment services. 

To develop or update a CPG, the VA/DOD Evidence-Based Practice Work 
Group identifies two clinical leaders--one from VA and one from DOD-- 
who then help identify not more than 15 to 20 other experts in the 
subject area to form a "guideline working group."[Footnote 59] A 
member of the VA/DOD Evidence-Based Practice Work Group is also 
selected to be an evidence chaperone for each CPG to ensure that 
conformity to prevailing standards for conducting high-quality 
systematic reviews is upheld.[Footnote 60] 

To determine the scope of the CPG, the guideline working group, the 
evidence chaperone, and a facilitator are responsible for identifying 
clinical questions that are to be answered by a systematic review of 
relevant research outcomes.[Footnote 61] According to VA officials, 
clinical questions can be both broad and specific. For example, the 
2004 post-traumatic stress disorder CPG contained clinical questions 
regarding whether early intervention is more effective than later 
intervention and whether certain interventions, such as different 
psychotherapies, are more effective than others.[Footnote 62] 

According to VA, in order to answer these clinical questions, an 
external evidence center--an entity that conducts systematic reviews 
of research on a variety of topics--is contracted to collect and 
review all relevant research (including, but not limited to, VA-and 
DOD-sponsored research) to assess its applicability to each clinical 
question under consideration using explicit and reproducible methods. 
[Footnote 63] The evidence center then focuses its review on the best 
available research, that is, high-quality, methodologically rigorous 
studies that address health issues that impact VA and DOD populations 
and consider the effectiveness as well as the harms and benefits of 
the intervention at issue. According to VA officials, the evidence 
center provides summaries of only the best available research to the 
guideline working group for review.[Footnote 64] 

After receiving the summaries, the guideline working group reviews the 
research in sequential steps using an established rating scheme 
developed by the U.S. Preventive Services Task Force to formulate 
evidence-based recommendations.[Footnote 65] See figure 8 for an 
overview of the steps that the guideline working group uses to 
formulate evidence-based recommendations. 

Figure 8: Overview of the Process Used by the Guideline Working Group 
Established by the Department of Veterans Affairs (VA)/Department of 
Defense (DOD) Evidence-Based Practice Work Group to Grade Evidence- 
Based Recommendations for Clinical Practice Guidelines: 

[Refer to PDF for image: illustration] 

1) Relevant research is assessed by its level of evidence. 

2) Overall quality of evidence that addresses a particular clinical 
question is assessed. 

3) The net effect of the intervention is assessed (benefits minus 
harms). 

4) Evidence-based recommendations are graded (based on overall quality 
of evidence + net effect of the intervention). 

Source: GAO analysis of VA/DOD documents and GAO interviews with VA 
officials. 

[End of figure] 

Level of evidence. First, the guideline working group reviews the 
summaries to identify the level of evidence, or the level of 
methodological rigor. For example, research studies that have the 
highest quality are categorized as "I" (indicating at least one 
properly done randomized controlled trial), while research studies of 
the lowest quality are categorized as "III" (indicating that the 
research reflects the opinion of respected authorities, descriptive 
studies, case reports, and expert committees). (See table 8.) 

Table 8: Categorization of Ratings Used by the Guideline Working Group 
Established by the Department of Veterans Affairs (VA)/Department of 
Defense (DOD) Evidence-Based Practice Work Group to Identify the Level 
of Evidence of Research Outcomes: 

Rating: I; 
Description: At least one properly done randomized controlled trial. 

Rating: II-1; 
Description: Well-designed controlled trial without randomization. 

Rating: II-2; 
Description: Well-designed cohort or case-control analytic study, 
preferably from more than one source. 

Rating: II-3; 
Description: Multiple time series evidence with/without intervention, 
dramatic results of uncontrolled experiment. 

Rating: III; 
Description: Opinion of respected authorities, descriptive studies, 
case reports, and expert committees. 

Source: VA/DOD document. 

[End of table] 

Overall quality of research. After determining the level of evidence 
of individual research studies, the guideline working group makes a 
determination regarding the overall quality of all of the research 
that addresses a particular clinical question. The overall quality 
takes into account the number, quality, and size of all of the 
individual research studies together as well as the consistency of the 
results between research outcomes to determine the collective overall 
strength of the research. Based on this review, the guideline working 
group determines the overall quality of the evidence to be good, fair, 
or poor.[Footnote 66] (See table 9.) 

Table 9: Categorization of Ratings Used by the Guideline Working Group 
Established by the Department of Veterans Affairs (VA)/Department of 
Defense (DOD) Evidence-Based Practice Work Group to Determine the 
Overall Quality of Research: 

Rating: Good; 
Description: High-grade evidence (I or II-1) linked to health outcome. 

Rating: Fair; 
Description: High-grade evidence (I-or II-1) linked to intermediate 
outcome; Or: Moderate-grade evidence (II-2 or II-3) directly linked to 
health outcome. 

Rating: Poor; 
Description: Level III evidence or no linkage of evidence to health 
outcome. 

Source: VA/DOD document. 

[End of table] 

Net effect of the intervention. For interventions that were supported 
by studies of "fair" or "good" overall quality, the guideline working 
group evaluates the benefits and the potential harms to determine the 
net effect of the intervention.[Footnote 67] The net effect of an 
intervention takes into account the benefits of the intervention minus 
the harms to determine the overall potential clinical benefit that the 
intervention may provide to patients. The net effect of the 
intervention ranges from "substantial" (meaning the benefit 
substantially outweighs the harm) to "zero or negative" (meaning it 
has no impact or a negative impact on patients). (See table 10.) 

Table 10: Categorization of Ratings Used by the Guideline Working 
Group Established by the Department of Veterans Affairs 
(VA)/Department of Defense (DOD) Evidence-Based Practice Work Group to 
Determine the Net Effect of the Intervention as Described in the 
Research: 

Rating: Substantial; 
Description: More than a small relative impact on a frequent condition 
with a substantial burden of suffering; Or: A large impact on an 
infrequent condition with a significant impact on the individual 
patient level. 

Rating: Moderate; 
Description: A small relative impact on a frequent condition with a 
substantial burden of suffering; Or: A moderate impact on an 
infrequent condition with a significant impact on the individual 
patient level. 

Rating: Small; 
Description: A negligible relative impact on a frequent condition with 
a substantial burden of suffering; Or: A small impact on an infrequent 
condition with a significant impact on the individual patient level. 

Rating: Zero or negative; 
Description: Negative impact on patients; Or: No relative impact on 
either a frequent condition with a substantial burden of suffering or 
an infrequent condition with a significant impact on the individual 
patient level. 

Source: VA/DOD document. 

[End of table] 

Grade of evidence-based recommendation. In the final step, the 
guideline working group uses its assessment of the overall quality of 
the evidence and the net effect of the intervention to grade evidence- 
based recommendations. (See table 11.) 

Table 11: Grade of Evidence-Based Recommendation Assigned by the 
Guideline Working Group Established by the Department of Veterans 
Affairs (VA)/Department of Defense (DOD) Evidence-Based Practice Work 
Group Based on the Overall Quality of Evidence and the Net Effect of 
the Intervention as Described in the Research: 

Overall quality of evidence: Good; 
The net benefit of the intervention: 
Substantial: A; 
Moderate: B; 
Small: C; 
Zero or negative: D. 

Overall quality of evidence: Fair; 
The net benefit of the intervention: 
Substantial: B; 
Moderate: B; 
Small: C; 
Zero or negative: D. 

Overall quality of evidence: Poor; 
The net benefit of the intervention: 
Substantial: I; 
Moderate: I; 
Small: I; 
Zero or negative: I. 

Grade of recommendation: A; 
Description of recommendation: A strong recommendation that the 
intervention is always indicated and acceptable. 

Grade of recommendation: B; 
Description of recommendation: A recommendation that the intervention 
may be useful/effective. 

Grade of recommendation: C; 
Description of recommendation: A recommendation that the intervention 
may be considered. 

Grade of recommendation: D; 
Description of recommendation: A recommendation that a procedure may 
be considered not useful/effective or may be harmful. 

Grade of recommendation: I; 
Description of recommendation: Insufficient evidence to recommend for 
or against--the clinician will use clinical judgment. 

Source: VA/DOD document. 

[End of table] 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Randall B. Williamson, (202) 512-7114 or williamsonr@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Mary Ann Curran, Assistant 
Director; Susannah Bloch; Stella Chiang; Martha R. W. Kelly; Melanie 
Krause; Lisa Motley; Michelle Paluga; Rebecca Rust; and Suzanne Worth 
made key contributions to this report. 

[End of section] 

Related GAO Products: 

VA Health Care: Progress and Challenges in Conducting the National 
Vietnam Veterans Longitudinal Study. [hyperlink, 
http://www.gao.gov/products/GAO-10-658T]. Washington, D.C.: May 5, 
2010. 

VA Health Care: Status of VA's Approach in Conducting the National 
Vietnam Veterans Longitudinal Study. [hyperlink, 
http://www.gao.gov/products/GAO-10-578R]. Washington, D.C.: May 5, 
2010. 

VA Health Care: Preliminary Findings on VA's Provision of Health Care 
Services to Women Veterans. [hyperlink, 
http://www.gao.gov/products/GAO-09-899T]. Washington, D.C.: July 16, 
2009. 

DOD and VA Health Care: Challenges Encountered by Injured 
Servicemembers during Their Recovery Process. [hyperlink, 
http://www.gao.gov/products/GAO-07-589T]. Washington, D.C.: March 5, 
2007. 

VA Health Care: Spending for Mental Health Strategic Plan Initiatives 
Was Substantially Less Than Planned. [hyperlink, 
http://www.gao.gov/products/GAO-07-66]. Washington, D.C.: November 21, 
2006. 

VA Health Care: Preliminary Information on Resources Allocated for 
Mental Health Strategic Plan Initiatives. [hyperlink, 
http://www.gao.gov/products/GAO-06-1119T]. Washington, D.C.: September 
28, 2006. 

VA Health Care: VA Should Expedite the Implementation of 
Recommendations Needed to Improve Post-Traumatic Stress Disorder 
Services. [hyperlink, http://www.gao.gov/products/GAO-05-287]. 
Washington, D.C.: February 14, 2005. 

VA and Defense Health Care: More Information Needed to Determine If VA 
Can Meet an Increase in Demand for Post-Traumatic Stress Disorder 
Services. [hyperlink, http://www.gao.gov/products/GAO-04-1069]. 
Washington, D.C.: September 20, 2004. 

[End of section] 

Footnotes: 

[1] American Psychiatric Association, Diagnostic and Statistical 
Manual of Mental Disorders, Fourth Edition, Text Revision (Washington, 
D.C., 2000). 

[2] Those diagnosed with PTSD may also suffer from other ailments, 
such as depression and substance abuse. 

[3] Operation Enduring Freedom, which began in October 2001, supports 
combat operations in Afghanistan and other locations, and Operation 
Iraqi Freedom, which began in March 2003, supports combat operations 
in Iraq and other locations. In September 2010, Operation Iraqi 
Freedom became known as Operation New Dawn. 

[4] Estimates for veterans who have experienced PTSD vary. For 
example, according to VA, regarding Operation Enduring Freedom and 
Operation Iraqi Freedom veterans, one 2008 RAND study found that 
approximately 14 percent of these veterans have experienced PTSD, 
while another 2004 study by Hoge et al. estimated that from 6 to 20 
percent of those veterans have experienced PTSD. 

[5] Vet Centers offer readjustment counseling--a wide range of 
psychosocial services that includes individual and group counseling 
and screening and referrals for medical issues--to eligible veterans 
and their families. 

[6] Formed in 1999 and composed of VA and DOD officials, the VA/DOD 
Evidence-Based Practice Work Group makes decisions about which 
clinical practice guidelines (CPG) for specific conditions will be 
developed and oversees their development. As of August 2010, the 
VA/DOD Evidence-Based Practice Work Group had issued 24 CPGs on 
conditions such as chronic heart failure and major depressive 
disorder. Since VA and DOD issue the CPGs in a joint effort, they are 
intended to be used by both VA and DOD clinicians. The VA/DOD Evidence-
Based Practice Work Group is supported by VA's Office of Quality and 
Performance and the U.S. Army Medical Department's Office of Quality 
Management. 

[7] The PTSD CPG is formally known as the VA/DOD Clinical Practice 
Guideline for the Management of Post-Traumatic Stress (January 2004). 

[8] Evidence-based care refers to approaches that have consistently 
been shown in controlled research to be effective for a particular 
condition or conditions. 

[9] Veterans Health Administration Handbook 1160.01, Uniform Mental 
Health Services in VA Medical Centers and Clinics (Sept. 11, 2008). 
According to the Handbook, all veterans with PTSD must have access to 
cognitive processing therapy or prolonged exposure therapy. VA medical 
centers and very large community-based outpatient clinics (serving 
more than 10,000 unique veterans each year) must be able to provide 
staff to administer such services at the facilities. Large (serving 
5,000 to 10,000 unique veterans each year) and midsized (serving 1,500 
to 5,000 unique veterans each year) community-based outpatient clinics 
may provide these services through telemental health or contract care 
when necessary. In March 2010, VA issued the Veterans Health 
Administration Handbook 1160.03, Programs for Veterans with Post-
Traumatic Stress Disorder (PTSD), to provide additional information 
regarding the implementation of the PTSD requirements contained in the 
2008 Handbook. 

[10] Psychotherapies focus on changing individuals' behaviors, 
thoughts, perceptions, and emotions. Cognitive processing therapy 
utilizes trauma-specific cognitive techniques to help patients move 
past trauma-related thoughts and progress toward recovery. Prolonged 
exposure therapy works by helping individuals approach trauma-related 
thoughts, feelings, and situations that they have been avoiding 
because of the distress they cause. Repeated exposure to these 
thoughts, feelings, and situations can assist with reducing the 
likelihood they will cause distress. 

[11] VA's medical and prosthetic research appropriation pays for 
necessary expenses in carrying out programs of medical and prosthetic 
research and development as authorized by chapter 73 of title 38, 
United States Code, the authorizing laws for the Veterans Health 
Administration. See Pub. L. No. 110-329, 122 Stat. 3574, 3706 (2008). 

[12] The medical and prosthetic research appropriation funds research 
on different topics. It is not limited to funding research on PTSD. 

[13] VA's three medical care appropriations are (1) medical services, 
which provides funds for the provision of veterans' health care 
services; (2) medical support and compliance, which provides funds for 
expenses related to the administration of veterans' health care 
services; and (3) medical facilities, which provides funds for the 
operation and maintenance of VA's health care facilities. Each of 
these appropriations provides support to all research conducted at VA 
facilities. For example, the medical care appropriations pay for the 
salaries of VA investigators who are VA clinicians (according to VA, 
about 70 percent of VA investigators are VA clinicians), 
administrative costs such as those for payroll and human resources, 
and logistical and infrastructure costs. 

[14] VA could not provide an estimate for the amount of funding VA 
spent from the medical care appropriations for intramural PTSD 
research. Of the $433 million, more than half went to personnel costs, 
including the salaries of clinicians and other VA facility staff. 

[15] The data provided by ORD reflect the funding amounts VA 
investigators were authorized to use, not necessarily the amounts that 
were spent. VA officials said that the authorized amounts are very 
similar to the amounts that are spent. 

[16] Salaries for VA investigators who are VA clinicians are paid from 
the medical services appropriation, one of the medical care 
appropriations. 

[17] For example, VA's Veterans Integrated Service Networks, which 
receive funding from the medical care appropriations and include VA's 
medical facilities, may also fund research by VA investigators, but 
this research funding is not considered part of VA's intramural 
research program. According to ORD officials, about 95 percent of the 
PTSD research funded by VA is funded by VA's intramural research 
program. 

[18] The management of VA's medical facilities is decentralized to 21 
regional networks referred to as Veterans Integrated Service Networks. 

[19] In 2010, the VA Office of Inspector General found issues with 
ORD's validation of some research expenditure data reported by 
investigators at the VA Maryland Health Care System facility. See 
Department of Veterans Affairs, Office of Inspector General, Health 
Care Inspection: Inappropriate Research and Development Data Entries 
Affecting Veterans Equitable Research Allocation (VERA) Funding VA 
Maryland Health Care System Baltimore, MD (Washington, D.C., Sept. 23, 
2010). According to VA officials, the PTSD research funding data we 
obtained for this report were generated from a different data system 
than the data systems specifically examined by the VA Office of 
Inspector General. In addition, the PTSD research funding data are 
entered by VA officials at the central office and checked for accuracy 
by VA officials at both the central office and the facilities. 

[20] In 2009, VA's Office of Inspector General found that appropriated 
funds for VA research addressed the broad spectrum of medical issues 
with which veterans contend. See Department of Veterans Affairs, 
Office of Inspector General, Healthcare Inspection: Review of the 
Veterans Health Administration's Use of Appropriated Funds for 
Research (Washington, D.C., 2009). 

[21] According to VA, proposals may be reviewed outside of a review 
cycle depending on when ORD solicits requests for proposals. In rare 
cases, service-directed proposals are reviewed by ad hoc reviewers or 
solely by research and development service directors. 

[22] Each research and development service may have additional 
requirements. 

[23] The approval indicates that the medical center has agreed to 
commit the resources, such as space and staff, necessary to conduct 
the research study. 

[24] Investigators who do not demonstrate a primary professional 
commitment to VA can request a waiver of this requirement. According 
to a VA document, a waiver may be granted by VA officials depending on 
the circumstances of the request, such as for a retired investigator 
who previously received funding through the Merit Review Program. 
According to VA, waivers are granted for about 35 investigators per 
year. 

[25] The principal investigator and any co-principal investigators may 
also have a designated research space in an approved non-VA facility 
or other VA facility, such as a community-based outpatient clinic. 

[26] Salaries for VA investigators who are VA clinicians are paid from 
the medical services appropriation, one of the medical care 
appropriations. 

[27] Because of potential overlap in research areas, percentage 
estimates should not be added. 

[28] The National Research Advisory Council is responsible for 
advising the Secretary of Veterans Affairs and the Under Secretary for 
Health on matters related to VA's research. The Office of Management 
and Budget requires that the National Research Advisory Council 
conduct an annual assessment of VA's intramural research program. 

[29] VA intramural research studies may be funded for multiple years 
(see table 2). Therefore, the same intramural PTSD research study 
funded in one fiscal year may also be funded the following fiscal 
year. In our analysis, this study would be counted as a funded study 
each fiscal year. In addition, some studies were not funded each 
fiscal year of their duration. For example, a study that was conducted 
from fiscal year 2006 through fiscal year 2009 may not have received 
funding in fiscal year 2007. 

[30] Depending on the specific requirements of the request for 
proposal, a letter of intent (a preliminary outline of a research 
proposal) may be required prior to the submission of a research 
proposal. 

[31] Electronic research administration, or eRA Commons, is managed 
and supported by the National Institutes of Health to provide support 
for the funding of research proposals. 

[32] Merit review panelists may or may not be employed by VA. 
According to VA officials, most merit review panelists have previous 
experience reviewing proposals. According to a VA document, the number 
of panelists on a merit review panel may vary. For example, the number 
of panelists on Rehabilitation's merit review panels ranges from 16 to 
59. 

[33] Biomedical Laboratory and Clinical Science share the same merit 
review panels. 

[34] In addition, for Health Services, along with scientific merit, 
reviewers take into account research priority areas of Health Services 
when scoring proposals. 

[35] For Health Services, prior to assigning a priority score for a 
proposal, merit review panelists vote on whether to approve, 
conditionally approve, or reject the proposal. Conditional approval 
means the proposal is approved pending additions or changes. Only 
approved and conditionally approved proposals are assigned a priority 
score. 

[36] According to VA, the percentile is calculated by the proposal's 
rank, determined by its priority score in comparison to the scores of 
the current and previous three review cycles, minus 0.5, divided by 
the total number of applications being considered. For example, if a 
proposal received the fifth best priority score and there were a total 
of 60 proposals reviewed over the four cycles, the percentile for that 
application would be 5 minus 0.5 divided by 60, equaling .075. 

[37] After a proposal is approved for funding by the research and 
development service director, the proposal must undergo review by an 
institutional review board, an independent committee that reviews 
research based on ethical considerations. In addition, depending on 
the topic of the proposal, it may be reviewed by additional committees 
at this time, such as a data monitoring committee. 

[38] There are nine CSP centers: VA operates five CSP Coordinating 
Centers to support multi-site clinical trials and four Epidemiological 
Research and Information Centers to support epidemiology studies. In 
addition to receiving support from a designated CSP center, CSP 
proposals that involve drugs or medical devices also receive 
assistance from the CSP Clinical Research Pharmacy Coordinating 
Center, and CSP proposals that involve economic analysis receive 
support from VA's Health Economics Resource Center. 

[39] A letter of intent typically contains an outline of the proposed 
research study, including justification for multiple sites where the 
research will be conducted and a list of experts who can assist in the 
initial planning efforts of the research study. 

[40] Reviewers of letters of intent are selected based on their 
expertise in areas related to the topic of the letter of intent. 

[41] Planning efforts include travel funds for personnel involved in 
developing the proposal. According to VA, there are two planning 
meetings that involve approximately 15 people. 

[42] CSP proposals are currently not submitted electronically; 
however, ORD officials estimate that electronic submission of CSP 
proposals through eRA Commons will begin in fall 2011. 

[43] According to a VA document, members of the Cooperative Studies 
Scientific Merit Review Board serve 4-year terms, and ad hoc members 
can be added depending on specific expertise that may be needed to 
review a proposal. As of October 2010, there were six members of the 
Cooperative Studies Scientific Merit Review Board. 

[44] CSP proposals are also reviewed by a human rights committee at 
the designated CSP center to determine if the protection of patients' 
rights and welfare is adequate. CSP proposals must be approved by a 
human rights committee prior to being funded. 

[45] Scores are given only to conditionally and unconditionally 
approved proposals. 

[46] After a CSP proposal has received approval for funding, the 
proposal must undergo additional steps before study subjects are 
enrolled. For example, the proposal must be reviewed by an 
institutional review board, and an executive committee meets to review 
the operational and monitoring aspects of the study. 

[47] In addition, representatives from VA's Office of Patient Care 
Services are responsible for identifying VA and DOD clinical leaders. 

[48] According to VA, the development or update of a CPG typically 
takes 18 months. 

[49] According to VA, the update of the PTSD CPG mainly strengthens 
the evidence that was already contained in the 2004 PTSD CPG. 
According to a VA official, the updated PTSD CPG will include a module 
on complementary alternative medicine, which was not included in the 
2004 PTSD CPG. 

[50] In the CPG, cognitive processing therapy is graded a level "A" 
treatment specifically for female sexual assault-related PTSD. 
However, VA officials told us that they also consider cognitive 
processing therapy to be effective for PTSD in veterans. See appendix 
IV for a detailed description of the process used to determine grades 
for evidence-based recommendations. 

[51] Two other psychotherapies, stress inoculation training and eye 
movement desensitization and reprocessing, were also graded as level 
"A" treatments for PTSD in the 2004 PTSD CPG, but were not included in 
the Handbook. 

[52] Institute of Medicine, Treatment of Posttraumatic Stress 
Disorder: An Assessment of the Evidence (Washington, D.C., 2008). The 
Institute of Medicine is an independent, nonprofit organization that 
works outside of government to provide authoritative advice to 
decision makers and the public. The Institute of Medicine found that 
exposure therapies, including prolonged exposure therapy and elements 
of cognitive processing therapy, are efficacious treatments for PTSD. 

[53] VA officials reported that incorporating cognitive processing 
therapy and prolonged exposure therapy into the Handbook helped ensure 
that clinicians who had been trained in these therapies prior to 
development of the Handbook would be able to offer and provide them in 
VA facilities as needed. In May 2010, VA's Office of Inspector General 
reported that as of November 17, 2009, the total number of mental 
health practitioners who had completed either the cognitive processing 
therapy or the prolonged exposure therapy training was 3,086. See 
Department of Veterans Affairs, Office of Inspector General, 
Healthcare Inspection: Progress in Implementing the Veterans Health 
Administration's Uniform Mental Health Services Handbook (Washington, 
D.C., 2010). 

[54] To help implement VA's Comprehensive Veterans Health 
Administration Strategic Plan for Mental Health Services (November 
2004), VA allocated additional resources to fund mental health 
strategic plan initiatives in fiscal years 2005 and 2006. According to 
VA officials, a part of these additional resources was used to pay for 
the national cognitive processing therapy and prolonged exposure 
therapy training programs until the costs of the programs could be 
integrated into VA's congressional budget submission. 

[55] However, VA officials provided documentation regarding 
consultation with experts and a list of VA entities that participated 
in reviewing the Handbook. 

[56] According to VA officials, VA policies must be recertified every 
5 years to remain in effect. The Handbook must be recertified on or 
before September 2013. 

[57] The study team may also include a health economist, in cases 
where a CSP proposal involves economic analysis. In addition, the 
principal investigator may ask the Clinical Science Director to allow 
other consultants to be a part of the study team, as necessary. 

[58] Evidence-based care refers to approaches that have consistently 
been shown in controlled research to be effective for a particular 
condition or conditions. The VA/DOD Evidence-Based Practice Work Group 
makes decisions about which clinical practice guidelines for specific 
conditions will be developed and oversees their development. As of 
August 2010, the VA/DOD Evidence-Based Practice Work Group had issued 
24 CPGs on conditions such as chronic heart failure and major 
depressive disorder. Since VA and DOD issue the CPGs in a joint 
effort, they are intended to be used by both VA and DOD clinicians. 
The VA/DOD Evidence-Based Practice Work Group is supported by VA's 
Office of Quality and Performance and the U.S. Army Medical 
Department's Office of Quality Management. 

[59] In addition, representatives from VA's Office of Patient Care 
Services are responsible for identifying VA and DOD clinical leaders. 

[60] VA and DOD designees from the VA/DOD Evidence-Based Practice Work 
Group, with assistance from VA's Office of Patient Care Services, are 
responsible for selecting the VA and DOD clinical leaders and the 
evidence chaperone. In addition, they also select another member of 
the VA/DOD Evidence-Based Practice Work Group to monitor the 
development of the CPG. 

[61] The facilitator is also responsible for keeping the guideline 
working group focused during meetings and to ensure that it focuses on 
the quality of research outcomes when formulating evidence-based 
recommendations in the CPG. 

[62] The post-traumatic stress disorder CPG is formally known as The 
VA/DOD Clinical Practice Guideline for the Management of Post-
Traumatic Stress (January 2004). 

[63] According to a VA official, an external evidence center is used 
to minimize bias. 

[64] The evidence center provides summaries of research of lower 
quality only if no higher-quality research is available. 

[65] The U.S. Preventive Services Task Force was first convened by the 
Department of Health and Human Services' Public Health Service in 
1984, and since 1998 has been sponsored by the Department of Health 
and Human Services' Agency for Healthcare Research and Quality. The 
U.S. Preventive Services Task Force uses explicit criteria to grade 
the scientific evidence for a broad range of clinical services and 
develop evidence-based recommendations for clinicians and health 
systems. While the U.S. Preventive Services Task Force updated its 
criteria in May 2007, the VA/DOD Evidence-Based Practice Work Group 
currently uses U.S. Preventive Services Task Force criteria developed 
prior to May 2007. 

[66] The overall quality of the research is considered "good" when the 
research outcomes are consistent from a number of higher-level 
research studies and have been conducted across a broad range of 
populations, thereby providing a high degree of confidence that the 
research outcomes are true. A "fair" overall quality indicates that 
the research outcomes could have been a result of true effects or 
biases present across some or all of the research studies. A "poor" 
rating for overall quality indicates that any conclusion about the 
research outcomes is uncertain because of serious methodological 
shortcomings, sparse data, or inconsistent results. 

[67] The net effect of the intervention of research outcomes that are 
found to be of poor overall quality is not assessed because the 
evidence is insufficient to make a recommendation for or against 
routinely providing the intervention. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: