This is the accessible text file for GAO report number GAO-12-27R
entitled 'Department of Defense: Use of Neurocognitive Assessment Tools
in Post-Deployment Identification of Mild Traumatic Brain Injury which
was released on October 24, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
October 24, 2011:
Congressional Requesters:
Subject: Department of Defense: Use of Neurocognitive Assessment Tools
in Post-Deployment Identification of Mild Traumatic Brain Injury:
Traumatic brain injury (TBI) has emerged as a serious concern among
U.S. forces serving in military operations in Afghanistan and Iraq. The
widespread use of improvised explosive devices in these conflicts
increases the likelihood that servicemembers will sustain a TBI, which
the Department of Defense (DOD) defines as a traumatically induced
structural injury and/or physiological disruption of brain function as
a result of an external force.[Footnote 1] TBI cases within DOD are
generally classified as mild, moderate, severe, or penetrating. From
2000 to March 2011 there were a total of 212,742 TBI cases reported by
the Defense and Veterans Brain Injury Center within DOD. A majority of
these cases, 163,181, were classified as mild traumatic brain injuries
(mTBI)--commonly referred to as concussions.[Footnote 2]
Early detection of injury is critical in TBI patient management.
Diagnosis of moderate and severe TBI usually occurs in a timely manner
due to the obvious and visible nature of the head injury.
Identification of mTBI presents a challenge due to its less obvious
nature. With mTBI, there may be no observable head injury. In addition,
in the combat theater, an mTBI may not be identified if it occurs at
the same time as other combat injuries that are more visible or life-
threatening, such as orthopedic injuries or open wounds. Furthermore,
some of the symptoms of mTBI--such as irritability and insomnia--are
similar to those associated with other conditions, such as post-
traumatic stress disorder.
Although the majority of patients with mTBI recover quickly with
minimal intervention, a subset of patients develops lingering symptoms
that interfere with social and occupational functioning. Accurate and
timely identification of mTBI is important as treatment can mitigate
the physical, emotional, and cognitive effects of the injury.
Neurocognitive deficits associated with mTBI can be identified by
neurocognitive assessment tools. These tools generally consist of a
series of tests that measure cognitive performance areas that may be
impaired by an mTBI such as attention, judgment, and memory.
Identification of mTBI in servicemembers who served in Afghanistan and
Iraq has been the subject of recent media attention, with particular
attention focused on the proper use of neurocognitive assessment tools
to screen all servicemembers post-deployment for deficits or symptoms
related to mTBI. In this context and in response to your request, this
report describes (1) DOD's post-deployment policy on the use of
neurocognitive assessment tools as a stand-alone initial screen to
identify servicemembers who may have sustained an mTBI during
deployment; (2) what informed DOD's decisions to establish this post-
deployment policy; and (3) mTBI experts' views on the science related
to DOD's policy decision.[Footnote 3] Additionally, you have expressed
the importance of recording mTBI in a servicemember's medical history
to ensure proper treatment. In response to this concern, we are
initiating a review of DOD's in-theater documentation of servicemembers
involved in potentially concussive events.
To describe DOD's post-deployment policy on the use of neurocognitive
assessment tools as a stand-alone initial screen to identify
servicemembers who may have sustained an mTBI during deployment, and
what informed DOD's decisions to establish this post-deployment policy,
we reviewed relevant DOD policy and guidance and interviewed DOD
officials involved in DOD TBI policy. We also reviewed documents that
DOD cited as support for its decision, such as task force and expert
panel reports, and scientific studies. To describe mTBI experts' views
on the science related to the policy decision, we interviewed 15 mTBI
experts within and outside of DOD in the fields of military concussion
and sports concussion. We selected experts through a snowball sampling
method and through structured searches of the military and sports
concussion literature.[Footnote 4] We interviewed sports concussion
experts because the sports population has experience in the use of
neurocognitive assessment tools to help identify mTBI. The views of the
experts we interviewed cannot be assumed to be the views of all mTBI
experts.
We conducted this performance audit from April 2011 through October
2011 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Results in Brief:
DOD does not require that all servicemembers be screened post-
deployment using a neurocognitive assessment tool but does require that
all servicemembers be screened using a set of TBI screening questions.
According to DOD officials, this policy was informed by findings and
recommendations from several task forces and expert panel reports, and
scientific studies. Additionally, mTBI experts told us that the
scientific evidence supports DOD's policy. For example, these experts
told us that neurocognitive assessment tools cannot determine whether
low cognitive function is caused by an mTBI. These experts told us,
however, that neurocognitive assessment tools can be useful as part of
a full clinical evaluation for a person who has already screened
positive for a possible mTBI.
DOD Policy Does Not Require Screening All Servicemembers with a
Neurocognitive Assessment Tool Post-Deployment:
DOD policy does not require that all servicemembers receive an
assessment with a neurocognitive assessment tool as a stand-alone
initial screen for mTBI post-deployment. Instead, DOD requires initial
screening of all servicemembers using TBI screening questions as part
of the post-deployment health assessment (PDHA) process. The PDHA
process is designed to self-identify and refer for further evaluation
servicemembers with health concerns as a result of deployment.[Footnote
5] During the PDHA process, a servicemember completes a form that
includes TBI screening questions.[Footnote 6] A health care provider
reviews the completed form and may refer the servicemember to a
clinician for further evaluation for a possible mTBI.[Footnote 7] As
part of that evaluation, a DOD official told us that clinicians may
choose, but are not required, to use a neurocognitive assessment tool
to help identify symptoms consistent with an mTBI. DOD anticipates
issuing in the first quarter of calendar year 2012 additional policy on
post-deployment neurocognitive assessment tools, according to a DOD
official. The DOD policy will specify the use of a particular tool that
clinicians should use if they choose to use a neurocognitive assessment
tool during the clinical evaluation of a servicemember referred post-
deployment through the PDHA process.
A May 28, 2008, DOD interim guidance requires that neurocognitive
assessments using the Automated Neuropsychological Assessment Metrics
(ANAM) neurocognitive assessment tool be administered to all
servicemembers prior to deployment. According to a DOD official, the
upcoming policy on post-deployment neurocognitive assessment will also
specify use of the ANAM, if clinicians choose to use a neurocognitive
assessment tool. DOD is currently conducting a head-to-head study of
five neurocognitive assessment tools (including the ANAM) in order to
identify the tool best suited to military use. After significant delay,
DOD anticipates results of that study in 2015. DOD officials told us
that DOD will then specify that the selected tool be used consistent
with DOD policy on neurocognitive assessment.
Reports and Scientific Studies Informed DOD's Policy:
According to a DOD official with responsibility for DOD's policy on
mTBI, the use of neurocognitive assessment tools post-deployment was
informed by findings and recommendations from several task forces and
expert panel reports, and scientific studies. The DOD official told us
that DOD determined that these sources did not provide evidence to
support the use of neurocognitive assessment tools as a stand-alone
initial screen for mTBI for all servicemembers post-deployment. The DOD
official cited several reports as supporting their policy of not
requiring that all servicemembers be assessed with a neurocognitive
assessment tool post-deployment because these reports are silent on the
use of such tools post-deployment.[Footnote 8] Additionally, the DOD
official told us that one report and two scientific studies provided
further support for not using a neurocognitive assessment tool as a
stand-alone initial screen for mTBI for all servicemembers when they
return from deployment.
The following describes the recommendation and suggestions of the
report and studies and the DOD official's explanation of how they
provided support for DOD's policy.
The 2007 Army TBI Task Force report to the Surgeon General[Footnote 9]
recommended that DOD implement a post-deployment neuropsychological
evaluation using the ANAM. However, the report reiterated several
limitations of using a neurocognitive assessment tool as a stand-alone
initial screen for mTBI, including limitations that could affect the
accuracy of the tool. The DOD official told us that the report's
recommendation in conjunction with the limitations regarding the use of
neurocognitive assessment tools stated in the report supports their
policy of using the PDHA, rather than the ANAM.
* A study conducted at Fort Bragg and published in 2009[Footnote 10]
found no association between poor ANAM performance after deployment and
self-reported history of mTBI. The DOD official told us that this study
provides evidence that screening all servicemembers post-deployment
with a neurocognitive assessment tool is not likely to be useful.
* DOD has stated that findings from a study conducted at Ft.
Campbell[Footnote 11] support a selective use of the ANAM post-
deployment but not as a stand-alone initial screen for mTBI for all
servicemembers. The study found that at post-deployment, servicemembers
reporting a history of mTBI, but not current mTBI symptoms, did not
show a decline in cognitive functioning as measured by the ANAM. The
DOD official told us that using the ANAM as a stand-alone initial
screen for mTBI can result in false negative results for these
servicemembers because the assessment tool was not able to identify
that an mTBI had occurred for servicemembers without symptoms at post-
deployment.[Footnote 12]
MTBI Experts Told Us the Science Supports DOD's Policy:
Experts in mTBI told us that the science related to neurocognitive
assessment tools supports DOD's post-deployment policy with respect to
screening for mTBI. These experts told us they do not believe that
screening all servicemembers post-deployment with a neurocognitive
assessment tool is beneficial to determine who may have sustained an
mTBI during deployment. However, most of the experts told us they do
believe that such tools can be useful to help a clinician determine
whether a servicemember has sustained an mTBI in situations where a
servicemember has already screened positive on the PDHA and was
subsequently referred to a clinician for a full evaluation.
Experts gave several reasons why they support DOD's policy with respect
to mTBI screening. For one, experts told us that neurocognitive
assessment tools can indicate low cognitive function but cannot
determine whether or not it is caused by an mTBI. They told us it is
likely that confounding variables,[Footnote 13] such as lack of sleep,
stress, and influence of medication--not mTBI--can be the cause of
change in cognition in servicemembers at post-deployment. They said
that these confounding variables could result in a large number of
false positives[Footnote 14] when all servicemembers are assessed.
Another reason is that such tools are designed to measure cognitive
function and generally do not measure the physical aspects of mTBI,
such as headaches, impaired balance, and sensitivity to light or noise.
This oversight may potentially result in false negatives. Most experts
told us that in order to reduce the number of false positives or
negatives from assessments at post-deployment with a neurocognitive
assessment tool, such tools should only be used during a full clinical
evaluation for someone who already screened positive for the
possibility of an mTBI on the PDHA. During that evaluation, the
clinician can evaluate a servicemember's event history, symptoms, and
cognitive functioning by means of a neurocognitive assessment tool to
determine whether someone has sustained an mTBI.
A third reason experts gave for why they support DOD's policy involves
the potential negative effect on the servicemember from a false
positive outcome. Most experts told us that individuals who initially
test positive for mTBI with a neurocognitive assessment tool--but after
further evaluation are determined not to have sustained an mTBI--could
still believe they sustained an mTBI and suffer lasting psychological
effects from the initial false positive outcome. A few experts
disagreed, saying an appropriate explanation by a clinician would
likely mitigate any negative psychological effect of a false positive
outcome.
Agency Comments:
We provided a draft of this report to DOD for comment. DOD concurred
with our findings and provided written technical comments, which we
incorporated as appropriate. DOD comments appear in enclosure I.
We are sending copies of this correspondence to the Secretary of
Defense and appropriate congressional committees. In addition, this
correspondence will also be available at no charge on the GAO Web site
at [hyperlink, http://www.gao.gov].
If you or your staffs have any questions regarding this report, please
contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major contributions
to this report are listed in enclosure II.
Signed by:
Randall B. Williamson:
Director, Health Care:
Enclosures -2:
List of Requesters:
The Honorable Joseph I. Lieberman:
Chairman:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Mark Begich:
United States Senate:
The Honorable Scott P. Brown:
United States Senate:
The Honorable Richard Burr:
United States Senate:
The Honorable Saxby Chambliss:
United States Senate:
The Honorable Tom Coburn:
United States Senate:
The Honorable Susan M. Collins:
United States Senate:
The Honorable James M. Inhofe:
United States Senate:
The Honorable Mark Udall:
United States Senate:
The Honorable Tammy Baldwin:
House of Representatives:
The Honorable Sanford D. Bishop Jr.:
House of Representatives:
The Honorable Tom Cole:
House of Representatives:
The Honorable Joe Courtney:
House of Representatives:
The Honorable Mark Critz:
House of Representatives:
The Honorable Rush Holt:
House of Representatives:
The Honorable Dave Loebsack:
House of Representatives:
The Honorable Mike Michaud:
House of Representatives:
The Honorable Grace F. Napolitano:
House of Representatives:
The Honorable Bill Pascrell, Jr.:
House of Representatives:
The Honorable Todd Russell Platts:
House of Representatives:
[End of section]
Enclosure I: Comments from the Department of Defense:
Office Of The Assistant Secretary Of Defense:
Washington, DC 20301-1200:
Health Affairs:
October 5, 2011:
Mr. Randall B. Williamson:
Director, Health Care:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Williamson:
Thank you for the opportunity to review the Government Accountability
Office (GAO) draft report, "Defense Health Care: Use of Neurocognitive
Assessment Tools in Post-Deployment Identification of Mild Traumatic
Brain Injury," dated September 15, 2011 (Job Code: 290921). After
careful review, we concur with the report but would like to offer the
following technical comments.
(1) The enclosed DoD definition of traumatic brain injury (TBI) should
be used and referenced within GAO's report rather than the definition
released by the Centers for Disease Control and Prevention discussed in
paragraph 1. The DoD definition incorporates pertinent elements of the
CDC definition but is targeted specifically to the population that we
serve. Within DoD, TBI cases are generally classified as mild,
moderate, severe and penetrating. The penetrating category has been
omitted from the draft report and is clearly relevant for DoD.
(2) As written, paragraph 3, lines 4 and 5, imply neurocognitive
assessment results can help determine whether a mild traumatic brain
injury (mTBI) has occurred. Evaluation can be assisted and some mTBI-
related impairment can be quantified, but identification of TBI is
based on an injury event and alteration or loss of consciousness.
Neurocognitive assessments can reveal a change in a cognitive baseline
or show cognitive deficits that might be a product of a brain injury,
but it is important not to contribute to the common misunderstanding
that neurocognitive assessments can detect mTBI. Therefore, we
recommend the sentence be changed to one of the following:
* Neurocognitive deficits associated with mTBI can be identified by
neurocognitive assessment tools, or:
* Symptoms associated with mTBI can be identified by neurocognitive
assessment tools.
(3) Finally, paragraph 4, sentence 1, again implies neurocognitive
assessment tools can detect cases of mTBI in Service members returning
from deployment. In an attempt to maintain consistent language
throughout the report regarding the function of neurocognitive
assessment tools, we offer the following change:
* Identification of mTBI in Service members who served in Afghanistan
and Iraq has been the subject of recent media attention, with
particular attention focused on the proper use of neurocognitive
assessment tools to screen all Service members post-deployment for
deficits or symptoms related to mTBI.
We sincerely thank the GAO for its thorough review and analysis of
issues regarding the use of neurocognitive assessment tools in post-
deployment identification of mTBI, and look forward to the release of
the final report Questions or concerns regarding this response may be
addressed to Ms. Elizabeth Fudge at (703) 578-8596, or
elizabeth.fudge@trna.osd.mil.
Sincerely,
Signed by:
George Peach Taylor, Jr., MD:
Deputy Assistant Secretary of Defense:
Force health Protection and Readiness:
Enclosure:
As stated:
[End of section]
Enclosure II: GAO Contact and Staff Acknowledgments:
Contact:
Randall B. Williamson at (202) 512-7114 or williamsonr@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, key contributors to this report
were Marcia A. Mann, Assistant Director; Lori Fritz; Teresa Tam; and
Rasanjali Wickrema. Monica Perez-Nelson provided legal support and
Laurie Pachter assisted in the message and correspondence development.
{2909210)
[End of section]
Footnotes:
[1] Department of Veterans Affairs/DOD, Clinical Practice Guideline for
Management of Concussion/mild Traumatic Brain Injury (April 2009).
[2] DOD specifies that a person may be designated as having an mTBI
only if the severity of the injury does not include: (1) loss of
consciousness that lasted longer than 30 minutes; (2) alteration of
consciousness for more than 24 hours; (3) post-traumatic amnesia
lasting longer than 24 hours; or (4) an initial score of less than 13
on the Glasgow Coma Score, a widely-used 15-point scoring system for
assessing coma and impaired consciousness. (Higher scores indicate a
less severe injury while lower scores indicate a more severe injury.)
[3] For the purposes of this report, we define post-deployment as
within 30 days after servicemembers' return from deployment.
[4] A snowball sample includes an initial list of cases, each of whom
is asked for referrals to additional people, who are then interviewed
and asked for referrals, and so on. We repeated this process until we
were consistently receiving substantively similar information from each
additional interview.
[5] DOD policy requires that the PDHA be completed by servicemembers
within 30 days before or 30 days after return from deployment.
[6] The TBI screening questions on the PDHA are designed to be
completed by the servicemember in four series. The sequence of
questions specifically assesses (1) events that may have increased the
risk of a TBI, (2) immediate symptoms following the event, (3) new or
worsening symptoms following the event, and (4) current symptoms. If
there is a positive response to any question in the first series, the
servicemember completes the second and third series; if there is a
positive response to any question in the third series, the
servicemember completes the fourth series about current symptoms. The
form directs the health care provider to refer the servicemember based
on the servicemember's current symptoms.
[7] According to the Department of Veterans Affairs/DOD Clinical
Practice Guideline for Management of Concussion/mild Traumatic Brain
Injury, a clinical evaluation for mTBI should include (1) obtaining a
detailed medical history (including details of the injury event and
identification of symptoms); (2) a psychosocial assessment (including
assessment of conditions that may exacerbate mTBI symptoms, such as
post-traumatic stress disorder); and (3) a physical exam.
[8] August 2006 report of the Armed Forces Epidemiology Board; April
2007 Independent Review Group on Rehabilitative Care and Administrative
Processes at Walter Reed Army Medical Center and National Naval Medical
Center; June 2007 report of the Defense Health Board Task Force on
Mental Health; October 2007 report of the Scientific Advisory Panel on
the Use of the ANAM; and Gulf War and Health: Volume 7: Long-Term
Consequences of Traumatic Brain Injury, Committee on Gulf War and
Health: Brain Injury in Veterans and Long-Term Health Outcomes
(Institute of Medicine, 2008).
[9] Traumatic Brain Injury Task Force, Report to the Surgeon General
(May 2007).
[10] Brian J. Ivins, Robert Kane, Karen A. Schwab, "Performance on the
Automated Neuropsychological Assessment Metrics in a Nonclinical Sample
of Soldiers Screened for Mild TBI After Returning From Iraq and
Afghanistan: A Descriptive Analysis," Journal of Head Trauma
Rehabilitation, vol. 24, no. 1 (2009), 24-31.
[11] Tresa Roebuck-Spenser, Andrea S. Vincent, David A. Twille, Bret W.
Logan, Mary Lopez, Stephen Grate, Robert E. Schlegel, Kirby Gilliland,
Cognitive Change Associated with Deployment-Related Mild Traumatic
Brain Injury Sustained During the OEF/OIF Conflicts.This is an
unpublished manuscript provided to us by the Center for the Study of
Human Operator Performance (C-SHOP) at the University of Oklahoma. The
University of Oklahoma holds the license for ANAM technology.
[12] A false negative is an incorrect result of a diagnostic test or
procedure that falsely indicates the absence of a finding, condition,
or disease.
[13] A confounding variable is another variable that distorts the
association being studied between the two main variables.
[14] A false positive is a test result that wrongly indicates the
presence of a disease or other condition the test is designed to reveal.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: