This is the accessible text file for GAO report number GAO-12-676 
entitled 'Military Disability System: Improved Monitoring Needed to 
Better Track and Manage Performance' which was released on September 
19, 2012.  

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO:  

Report to Congressional Committees:  

August 2012:  

Military Disability System: 

Improved Monitoring Needed to Better Track and Manage Performance: 

GAO-12-676: 

GAO Highlights: 

Highlights of GAO-12-676, a report to congressional committees.  

Why GAO Did This Study: 

Since 2007, DOD and VA have jointly operated IDES—-which is intended 
to expedite benefits for injured servicemembers. IDES replaced the 
departments’ separate processes for evaluating servicemembers for 
fitness and disability. Initially a pilot at 3 military treatment 
facilities, IDES is now in place at military treatment facilities 
worldwide. In previous reports, GAO identified a number of challenges 
as IDES expanded to more facilities, including staffing shortages and 
difficultly meeting timeliness goals.  

In light of IDES’ expansion, GAO was asked to examine: (1) the extent 
to which DOD and VA are meeting IDES timeliness and servicemember 
satisfaction performance goals, and (2) steps the agencies are taking 
to improve IDES performance. GAO analyzed IDES timeliness and customer 
satisfaction data, visited six IDES sites with varying performance, 
and interviewed DOD and VA officials.  

What GAO Found: 

Case processing times under the Integrated Disability Evaluation 
System (IDES) have increased over time, and measures of servicemember 
satisfaction have shortcomings. Since 2008, annual average processing 
times for IDES cases have steadily climbed, while the percentage of 
cases meeting established timeliness goals declined. Average case 
processing times reached 394 and 420 days for active and reserve 
component members in fiscal year 2011—-compared to goals of 295 and 
305 days, respectively, and just 19 percent of active duty and 18 
percent of guard or reserve servicemembers completed the process and 
received benefits within established goals. Of the four phases 
comprising IDES, the medical evaluation board phase increasingly fell 
short of timeliness goals, while the physical evaluation board phase, 
although meeting goals, was taking increasingly more time to complete. 
With respect to servicemember satisfaction with the IDES process, GAO 
found shortcomings in how these data are collected and reported, such 
as unduly limiting who is eligible to receive a survey and computing 
average satisfaction scores in a manner that may overstate them. 
Department of Defense (DOD) officials told GAO they are considering 
alternatives for gauging satisfaction with the process.  

Figure: Timeliness for IDES Cases Resulting in VA Benefits (by year 
case completed):  

[Refer to PDF for image: vertical bar graph]  

Total days in IDES:  

Fiscal year: 2008; 
Active forces: 283; 
Reserves/National Guard: 297.  

Fiscal year: 2009; 
Active forces: 313; 
Reserves/National Guard: 316.  

Fiscal year: 2010; 
Active forces: 357; 
Reserves/National Guard: 370.  

Fiscal year: 2011; 
Active forces: 394; 
Reserves/National Guard: 420.  

Goal: 
Active forces: 295; 
Reserves/National Guard: 295.  

Source: GAO analysis of DOD and VA data.  

[End of figure]  

DOD and Veterans Affairs (VA) are taking steps to improve IDES 
performance, but progress to date is uneven and it is too early to 
assess their overall impact. For example, VA increased resources for 
completing exams and disability ratings while the Army is hiring 
additional staff for its medical evaluation boards. VA has met exam 
timeliness goals in the past several months, but other resources have 
yet to translate into lower processing times. DOD and VA are pursuing 
system upgrades so that staff and managers at IDES facilities can 
better track and manage the progress of servicemembers’ cases. IDES 
officials have been working with the military services to correct case 
data that were inaccurately entered into VA’s IDES tracking system, 
but have not yet identified a permanent solution to improve the 
accuracy of data input. Finally, DOD, with VA’s assistance, is in the 
early stages of an in-depth review of the entire IDES process and its 
supporting IT systems. This effort is intended to improve 
understanding of how each step contributes to overall processing times 
and identify opportunities to streamline the process and supporting 
systems. However, timeframes for completing the review or issuing 
recommendations have yet to be established.  

What GAO Recommends: 

To improve monitoring of IDES timeliness and satisfaction, GAO 
recommends that DOD and VA work together to (1) develop plans for 
completing the ongoing business process review and implementing any 
resulting recommendations and (2) improve the accuracy of case 
information at the point of data entry; and that (3) DOD consider 
alternative approaches to measuring satisfaction. DOD and VA concurred 
with GAO’s recommendations.  

View [hyperlink, http://www.gao.gov/products/GAO-12-676]. For more 
information, contact Daniel Bertoni at (202) 512-7215 or 
bertonid@gao.gov.  

[End of section]  

Contents: 

Letter: 

Background: 

IDES Processing Times Increased Over Time, While Measures of 
Servicemember Satisfaction Have Shortcomings: 

Recent Actions and Ongoing Initiatives May Improve IDES Performance, 
but It Is Too Early to Assess Their Overall Impact: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Additional Timeliness and Satisfaction Analyses: 

Appendix III: Monthly DOD Timeliness Data for Active Duty Cases in 
Fiscal Year 2012: 

Appendix IV: Comments from the Department of Defense: 

Appendix V: Comments from the Department of Veterans Affairs: 

Appendix VI: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Average Processing Times for MEB Phase for Completed Cases by 
Fiscal Year of Completion (in days): 

Table 2: Average Processing Time in PEB Phase of IDES for Completed 
Cases (in days): 

Table 3: Average Processing Time for PEB Stages of IDES for Completed 
Cases (in days): 

Table 4: Average Processing Time in Transition Phase of IDES for 
Completed Cases (in days): 

Table 5: Average Processing Time in Benefits Phase of IDES for 
Completed Cases (in days): 

Table 6: Survey Response and Coverage Rates: 

Table 7: Selected Characteristics of IDES Pilot Sites Visited as of 
May 2011: 

Table 8: Survey phase response and coverage rates: 

Table 9: Selected Servicemember Characteristics of Respondents and 
Nonrespondents to the MEB, PEB and Transition Surveys: 

Table 10: Questions Used in DOD Indices of Overall Servicemember 
Satisfaction: 

Table 11: Overall IDES Satisfaction by Subgroups: 

Table 12: Perceived Timeliness of IDES by Actual Processing Times: 

Table 13: DOD Reported Monthly Average Processing Times for Active 
Component Servicemembers in Fiscal Year 2012 (in days): 

Figures: 

Figure 1: Steps of the IDES Process and Timeliness Goals: 

Figure 2: IDES Cases Enrolled and Completed by Fiscal Year: 

Figure 3: Distribution of New IDES Cases Each Year by Military Service 
and Status: 

Figure 4: Average Processing Times and Number of Completed Cases 
Resulting in Benefits by Fiscal Year of IDES Completion: 

Figure 5: Percentage of Cases Resulting in Benefits Meeting and Not 
Meeting Overall Processing Time Goals by Fiscal Year of IDES 
Completion: 

Figure 6: Percent of Cases Meeting Timeliness Goals for each Phase of 
IDES by Fiscal Year of Completion: 

Figure 7: Percent of Cases Completing MEB Within or in Excess of Goals 
All Fiscal Years Combined: 

Figure 8: Timeliness of Cases Enrolled in MEB Stage as of December 
2011: 

Figure 9: Comparison of Percent of Servicemembers Satisfied Using GAO 
and DOD Calculations for Overall Satisfaction: 

Figure 10: Average IDES Processing Times by Year of Completion, for 
Completed Cases Resulting in Benefits: 

Figure 11: Average IDES Processing Times by Fiscal Year of Enrollment 
for Completed Cases Resulting in Benefits: 

Figure 12: Average IDES Processing Times by Year of Completion, for 
All Completed Cases Regardless of Outcome: 

Figure 13: Average IDES Processing Times by Fiscal Year of Enrollment 
for All Completed Cases Regardless of Outcome: 

Figure 14: Percentage of Completed Cases Meeting and Missing the 
Overall Processing Time Goals, by Fiscal Year of IDES Completion, for 
Servicemembers Receiving Benefits: 

Figure 15: Percentage of Completed Cases Meeting and Missing the 
Overall Processing Time Goals, by Fiscal Year of IDES Enrollment, for 
Servicemembers Receiving Benefits: 

Figure 16: Average Processing Time for Each IDES Phase, by Fiscal Year 
in Which the Phase Was Completed: 

Figure 17: Average Processing Time for Each IDES Phase, by Fiscal Year 
of Enrollment in Each Phase: 

Figure 18: Percent of Cases Meeting Timeliness Goals for each Phase of 
IDES, by Fiscal Year in Which the Phase Was Completed: 

Figure 19: Percent of Cases Meeting Timeliness Goals for each Phase of 
IDES, by Fiscal Year of Enrollment in Each Phase: 

Abbreviations: 

DOD: Department of Defense:  

IDES: integrated disability evaluation system:  

MEB: medical evaluation board:  

PEB: physical evaluation board:  

VA: Department of Veterans Affairs:  

VTA: Veterans Tracking Application:  

WCP: Office of Warrior Care Policy:  

WWCTP: Office of Wounded Warrior Care & Transition Policy:  

[End of section]  

United States Government Accountability Office: 
Washington, DC 20548:  

August 28, 2012:  

The Honorable Patty Murray: 
Chairman: 
The Honorable Richard Burr: 
Ranking Member: 
Committee on Veterans’ Affairs: 
United States Senate:  

The Honorable Joseph I. Lieberman: 
Chairman: 
The Honorable Susan M. Collins: 
Ranking Member: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate:  

Servicemembers who are injured in war or as the result of accidents 
and illnesses may face a difficult transition as they leave the 
military and become veterans. In response to concerns that wounded, 
ill, or injured servicemembers had to undergo two complex disability 
evaluations-—first by the Department of Defense (DOD) then by the 
Department of Veterans Affairs (VA)—-DOD and VA jointly designed a new 
integrated disability evaluation process to expedite the delivery of 
benefits to servicemembers. In November 2007, DOD and VA began pilot 
testing the integrated disability evaluation system (IDES) at three 
military treatment facilities in the Washington, D.C. area, and 
expanded the number of sites over time. As of October 1, 2011, IDES 
had replaced the military services’ existing-—or “legacy”-—disability 
evaluation systems for almost all new disability cases.  

Past GAO work highlighted challenges DOD and VA experienced while 
piloting the IDES and recommended a number of improvements. For 
instance, we reported in December 2010[Footnote 1] that insufficient 
staff and logistical challenges contributed to delays in completing 
IDES cases and recommended the agencies take steps to ensure adequate 
staffing levels and develop a systematic process for monitoring 
caseloads. In response to ongoing concerns with IDES performance, this 
report provides information on (1) the extent to which DOD and VA are 
meeting IDES performance goals, and (2) steps DOD and VA are taking to 
improve IDES performance.  

In conducting our work, we obtained DOD timeliness and customer 
satisfaction data from the inception of IDES in 2007 to December 2011. 
We assessed the reliability of these data and analyzed them to look 
for changes in performance over time; factors that may help or hinder 
performance; and relationships between servicemember satisfaction and 
case outcomes and timeliness. We supplemented these analyses with site 
visits to six military treatment facilities, where we spoke with DOD 
and VA staff as well as some servicemembers involved in the IDES 
process.[Footnote 2] We selected these facilities to obtain 
perspectives from sites in different military services and 
geographical regions and with varying caseloads and performance 
outcomes. For both research objectives, we interviewed key officials 
involved with IDES at DOD, VA, and each of the military services, and 
reviewed pertinent reports, guidance, plans, relevant federal laws, 
regulations, directives, and other documents. We conducted this 
performance audit from May 2011 to August 2012, in accordance with 
generally accepted government auditing standards. These standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings 
and conclusions based on our audit objectives. We believe the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives.  

Background: 

The Disability Evaluation Process: 

The IDES process begins at a military treatment facility when a 
physician identifies one or more conditions that may interfere with a 
servicemember's ability to perform his or her duties.[Footnote 3] The 
process involves four main phases: the Medical Evaluation Board (MEB), 
the Physical Evaluation Board (PEB), transition out of military 
service (transition), and VA benefits. 

MEB phase: In this phase, medical examinations are conducted and 
decisions are made by the MEB regarding a servicemember's ability to 
continue to serve in the military. This phase involves four stages: 
(1) the servicemember is counseled by a DOD board liaison on what to 
expect during the IDES process; (2) the servicemember is counseled by 
a VA case manager on what to expect during the IDES process and 
medical exams are scheduled;[Footnote 4] (3) medical exams are 
conducted according to VA standards for exams for disability 
compensation by VA, DOD, or contractor physicians, and (4) exam 
results are used by the MEB to identify conditions that limit the 
servicemember's ability to serve in the military.[Footnote 5] Also 
during this stage, a servicemember can seek a rebuttal, or an 
impartial medical review by a physician not on the MEB, or both. 

PEB phase: In this subsequent phase, decisions are made about the 
servicemember's fitness for duty, disability rating and DOD and VA 
disability benefits, and the servicemember has opportunities to appeal 
those decisions. This includes: (1) the informal PEB stage, an 
administrative review of the case file by the relevant military 
branch's PEB without the presence of the servicemember; (2) VA rating 
stage, where a VA rating specialist prepares a rating that covers the 
conditions that DOD determined made a servicemember unfit for duty and 
any other conditions claimed by the servicemember to VA.[Footnote 6] 
This rating is prepared for use by both agencies in determining 
disability benefits. In addition, servicemembers have several 
opportunities to appeal different aspects of their disability 
evaluations: a servicemember dissatisfied with the decision on whether 
he or she is fit for duty may request a hearing with a "formal" PEB; a 
servicemember who disagrees with the formal PEB fitness decision can, 
under certain conditions, appeal to the reviewing authority of the 
PEB;[Footnote 7] and a servicemember can ask for VA to reconsider its 
rating, but only for conditions found unfitting by the PEB. 

Transition phase: If the servicemember is found unfit to serve, he or 
she enters the transition phase and begins the process of separating 
from the military. During this time, the servicemember may take 
accrued leave. Also, DOD board liaisons and VA case managers provide 
counseling on available benefits and services, such as job assistance. 

VA benefits phase: A servicemember found unfit and separated from 
service becomes a veteran and enters the VA benefits phase. VA 
finalizes its disability rating after receiving evidence of the 
servicemember's separation from military service. VA then starts to 
award monthly disability compensation to the veteran. 

DOD and VA established timeliness goals for the IDES process to 
provide VA benefits to active duty servicemembers within 295 days of 
being referred into the process, and to reserve component members 
within 305 days (see figure 1). DOD and VA also established interim 
timeliness goals for each phase and stage of the IDES process. The 
overall timeframes are intended to represent an improvement over the 
legacy disability evaluation system, which was estimated to take 540 
days to complete. 

Figure 1: Steps of the IDES Process and Timeliness Goals: 

[Refer to PDF for image: process outline]  

Servicemember referred to the IDES: 
    
MEB phase: 
100 days Active; 140 days Reserves:  

DOD board liaison meets with servicemember, compiles medical and 
personnel records (30 days Reserves); 
IDES goals (in days): 10.  

VA case manager meets member, files VA claim (30 days Reserves); 
IDES goals (in days): 10.  

VA, DOD, or contracted providers perform medical exam; 
IDES goals (in days): 45.  

MEB identifies potentially unfitting conditions; 
IDES goals (in days): 30.  

PEB phase: 
120 days Active and Reserves:  

Informal PEB determines fitness for duty; 
IDES goals (in days): 15.  

VA completes ratings; 
IDES goals (in days): 15.  

Member may appeal fitness decision to Formal PEB;  
IDES goals (in days): 30. 
  
Member may appeal rating decision to VA; 
IDES goals (in days): 15.  

Member may appeal Formal PEB decision to military department; 
IDES goals (in days): 30.  

Administrative processing throughout PEB phase; 
IDES goals (in days): 15.  

Transition phase: 
45 days Active and Reserves:   

Servicemember separates from military; 
IDES goals (in days): 45.  

VA Benefit phase[A]: 
30 days (Active only):  

VA issues benefits letter (Benefit phase not applicable to Reserves).  

Total days: 
295 (305 for Reserves[B]).  

Source: GAO analysis of DOD and VA policies. 

[A] Not all reservists complete the VA benefit phase and thus DOD does 
not apply the 30-day goal for this phase to reservists. For those 
reservists who do go through the VA benefits phase, this time is 
reflected in the overall time in IDES. 

[B] DOD guidance allows 40 more days for reserve component members 
than for active duty members in completing the first two steps of the 
process, in order to provide sufficient time for employer 
notification, establish orders for active duty, and compile medical 
records. However, DOD and VA's goal for total IDES processing time is 
only 10 days longer for reserve component members than for active duty 
members because the VA benefit phase goal of 30 days is not applied to 
reserve component members.  

[End of figure]  

In addition to timeliness, the agencies also established a performance 
goal of having 80 percent of servicemembers satisfied with the IDES 
process. DOD measures satisfaction through surveys conducted after the 
completion of the MEB, PEB, and transition phases. Each survey 
consists of approximately 30 questions, including 4 questions that ask 
about the servicemember's satisfaction with the overall IDES process 
up to that point. Reported satisfaction rates for each phase are based 
on an average of responses to these four questions, and reported 
overall satisfaction with IDES (which is used to track the percent 
satisfied under the performance goal) is an average of satisfaction 
rates for the three phases. 

Rollout of IDES and Enrollment: 

From the original 3 pilot military treatment facilities in the 
Washington, D.C., area,[Footnote 8] the IDES has expanded to 139 
military treatment facilities in the U.S. and several other countries. 
DOD and VA first added 24 military treatment facilities to the pilot 
in fiscal years 2009 and 2010, bringing the pilot total to 27. In 
2010, DOD and VA leadership decided to implement the IDES world-wide, 
and did so in 4 stages between October 2010 and September 2011, adding 
112 military treatment facilities. As IDES expanded, the number of new 
cases enrolled in IDES has also increased, totaling 18,651 in fiscal 
year 2011 (see figure 2). 

Figure 2: IDES Cases Enrolled and Completed by Fiscal Year: 

[Refer to PDF for image: vertical bar graph]  

Fiscal year: 2008; 
Cases enrolled: 702; 
Cases completed: 210.  

Fiscal year: 2009; 
Cases enrolled: 4,155; 
Cases completed: 1,183.  

Fiscal year: 2010; 
Cases enrolled: 9,713; 
Cases completed: 4,484.  

Fiscal year: 2011; 
Cases enrolled: 18,651; 
Cases completed: 7,106.  

Source: GAO analysis of DOD and VA data.  

Note: Cases completed include those who exited the IDES process, 
including those who received benefits or returned to duty.  

[End of figure]  

IDES caseloads vary by service, but the Army manages the bulk of IDES 
cases. Of new cases referred to IDES in fiscal year 2011, about 64 
percent were in the Army, and much of the growth in caseload has been 
in the Army. Additionally, active duty servicemembers make up the 
majority of IDES cases, with about 88 percent of new cases in fiscal 
year 2011 involving this group (see figure 3). 

Figure 3: Distribution of New IDES Cases Each Year by Military Service 
and Status: 

[Refer to PDF for image: 4 stacked vertical bar graphs]  

Air Force:  

Fiscal year: 2008; 
Active forces: 122; 
Reserves/National Guard: 12; 
Total number of cases: 134.  

Fiscal year: 2009; 
Active forces: 207; 
Reserves/National Guard: 41; 
Total number of cases: 248.  

Fiscal year: 2010; 
Active forces: 436; 
Reserves/National Guard: 54; 
Total number of cases: 490.  

Fiscal year: 2011; 
Active forces: 1,698; 
Reserves/National Guard: 181; 
Total number of cases: 1,879.  

Army:  

Fiscal year: 2008; 
Active forces: 212; 
Reserves/National Guard: 99; 
Total number of cases: 311.  

Fiscal year: 2009; 
Active forces: 1,892; 
Reserves/National Guard: 414; 
Total number of cases: 2,306.  

Fiscal year: 2010; 
Active forces: 5,330; 
Reserves/National Guard: 1,067; 
Total number of cases: 6,397.  

Fiscal year: 2011; 
Active forces: 9,970; 
Reserves/National Guard: 1,938; 
Total number of cases: 11,908.  

Navy:  

Fiscal year: 2008; 
Active forces: 109; 
Reserves/National Guard: 7; 
Total number of cases: 116.  

Fiscal year: 2009; 
Active forces: 572; 
Reserves/National Guard: 31; 
Total number of cases: 603.  

Fiscal year: 2010; 
Active forces: 1,014; 
Reserves/National Guard: 53; 
Total number of cases: 1,067.  

Fiscal year: 2011; 
Active forces: 1,975; 
Reserves/National Guard: 100; 
Total number of cases: 2,075.  

Marine Corps:  

Fiscal year: 2008; 
Active forces: 117; 
Reserves/National Guard: 23; 
Total number of cases: 140.  

Fiscal year: 2009; 
Active forces: 948; 
Reserves/National Guard: 50; 
Total number of cases: 998.  

Fiscal year: 2010; 
Active forces: 1,665; 
Reserves/National Guard: 89; 
Total number of cases: 1,754.  

Fiscal year: 2011; 
Active forces: 2,712; 
Reserves/National Guard: 76; 
Total number of cases: 2,788.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

IDES Processing Times Increased Over Time, While Measures of 
Servicemember Satisfaction Have Shortcomings: 

Overall IDES Case Processing Times Steadily Increased Since the Start 
of IDES: 

IDES timeliness has worsened since the inception of the program. Since 
fiscal year 2008, the average number of days for servicemember cases 
to be processed and receive benefits increased from 283 to 394 for 
active duty cases (compared to the goal of 295 days) and from 297 to 
420, for reserve component cases (compared to the goal of 305 days) 
(see figure 4).[Footnote 9] 

Figure 4: Average Processing Times and Number of Completed Cases 
Resulting in Benefits by Fiscal Year of IDES Completion: 

[Refer to PDF for image: horizontal bar graph]  

Total time in IDES:  

Active forces goal: 295 days; 
Reserves/National Guard goal: 305 days.  

Fiscal year: 2008; 
Completed cases: 154; 
Active forces: 283 days; 
Reserves/National Guard: 297 days.  

Fiscal year: 2009; 
Completed cases: 956; 
Active forces: 313 days; 
Reserves/National Guard: 316 days.  

Fiscal year: 2010; 
Completed cases: 3,878; 
Active forces: 357 days; 
Reserves/National Guard: 370 days.  

Fiscal year: 2011; 
Completed cases: 6,312; 
Active forces: 394 days; 
Reserves/National Guard: 420 days. 

Source: GAO analysis of DOD and VA data.  

[End of figure] 

Along with increasing average processing times, the percent of IDES 
cases awarded benefits within timeliness goals has steadily declined. 
DOD's and VA's current goal is to complete 60 percent of IDES cases on 
time. In fiscal year 2008, an average of 63 percent of cases for 
active duty servicemembers and 65 percent for reservists completed the 
process and received benefits within the timeliness goals; by fiscal 
year 2011 this was down to 19 and 18 percent respectively (see figure 
5). These trends also hold when considering all cases that completed 
the IDES process regardless of outcome, although overall processing 
times were shorter. (See appendix III for more information on case 
processing times regardless of outcome.)  

Figure 5: Percentage of Cases Resulting in Benefits Meeting and Not 
Meeting Overall Processing Time Goals by Fiscal Year of IDES 
Completion: 

[Refer to PDF for image: horizontal bar graph]  

Percentage of IDES cases:  

Active forces:  

Fiscal year 2008: 
Not meeting service goals: 37%; 
Meeting service goals: 63%.  

Fiscal year 2009: 
Not meeting service goals: 50%; 
Meeting service goals: 50%.  

Fiscal year 2010: 
Not meeting service goals: 68%; 
Meeting service goals: 32%.  

Fiscal year 2011: 
Not meeting service goals: 81%; 
Meeting service goals: 19%.  

Reserves/National Guard:  

Fiscal year 2008: 
Not meeting service goals: 35%; 
Meeting service goals: 65%.  

Fiscal year 2009: 
Not meeting service goals: 48%; 
Meeting service goals: 52%.  

Fiscal year 2010: 
Not meeting service goals: 63%; 
Meeting service goals: 37%.  

Fiscal year 2011: 
Not meeting service goals: 82%; 
Meeting service goals: 18%.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

Key Contributors to Timeliness Problems Include Lengthy Medical 
Evaluations and Servicemember Separation Activities: 

When examining timeliness across the four phases that make up IDES, 
data show that average processing time regularly fell short of goals 
for three--MEB, Transition, and VA Benefits. For example, for cases 
that completed the MEB phase in fiscal year 2011, active duty and 
reserve component members' cases took an average of 181 and 188 days 
respectively to be processed, compared to goals of 100 and 140 days. 
For the PEB phase, processing times increased over time, but were 
still within the established goal of 120 days. Along with increasing 
average processing times, the percentage of cases meeting goals for 
most phases has generally declined (see figure 6). In particular, the 
MEB and Transition phases have lower percentages of cases meeting 
goals than the other phases in most years, especially for active duty 
cases. 

Figure 6: Percent of Cases Meeting Timeliness Goals for each Phase of 
IDES by Fiscal Year of Completion: 

[Refer to PDF for image: 4 vertical bar graphs]  

MEB phase:  

Fiscal year: 2008; 
Reserves/National Guard: 69%; 
Active forces: 40%.  

Fiscal year: 2009; 
Reserves/National Guard: 46%; 
Active forces: 41%.  

Fiscal year: 2010; 
Reserves/National Guard: 50%; 
Active forces: 34%.  

Fiscal year: 2011; 
Reserves/National Guard: 33%; 
Active forces: 18%.  

PEB phase:  

Fiscal year: 2008; 
Reserves/National Guard: 92%; 
Active forces: 97%.  

Fiscal year: 2009; 
Reserves/National Guard: 92%; 
Active forces: 90%.  

Fiscal year: 2010; 
Reserves/National Guard: 79%; 
Active forces: 81%.  

Fiscal year: 2011; 
Reserves/National Guard: 62%; 
Active forces: 78%.  

Transition phase:  

Fiscal year: 2008; 
Reserves/National Guard: 13%; 
Active forces: 10%.  

Fiscal year: 2009; 
Reserves/National Guard: 29%; 
Active forces: 17%.  

Fiscal year: 2010; 
Reserves/National Guard: 34%; 
Active forces: 19%.  

Fiscal year: 2011; 
Reserves/National Guard: 25%; 
Active forces: 17%.  

Benefits phase:  

Fiscal year: 2008; 
Reserves/National Guard: N/A; 
Active forces: 64%.  

Fiscal year: 2009; 
Reserves/National Guard: N/A; 
Active forces: 52%.  

Fiscal year: 2010; 
Reserves/National Guard: N/A; 
Active forces: 46%.  

Fiscal year: 2011; 
Reserves/National Guard: N/A; 
Active forces: 54%.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

MEB Phase: 

As noted above, the MEB phase was a key contributor to increases in 
overall processing times between 2008 and 2011 for both active duty 
servicemembers and reservists for cases that have completed the IDES 
process regardless of outcome (table 1). 

Table 1: Average Processing Times for MEB Phase for Completed Cases by 
Fiscal Year of Completion (in days): 

Component: Active; 
Goal: 100; 
FY 2008: 114; 
FY2009: 126; 
FY2010: 141; 
FY2011: 181. 

Component: Reserve; 
Goal: 140; 
FY 2008: 128; 
FY2009: 153; 
FY2010: 158; 
FY2011: 188. 

Source: GAO analysis of DOD and VA data.  

[End of table]  

To obtain a better understanding of more recent timeliness trends 
within the MEB phase, GAO analyzed MEB timeliness of all cases--all 
fiscal years combined--that completed the MEB process by sorting them 
into two groups: (1) those that completed the entire IDES process, and 
(2) those that had not yet completed IDES but completed the MEB phase. 
As shown in figure 7, for the group that completed IDES, 30 percent of 
active duty servicemembers and 18 percent of reservists missed the 
goal by more than 90 days. For those still in IDES, representing more 
recent data, the picture is slightly better for active duty 
servicemembers with 37 percent of cases meeting the MEB goal and 25 
percent missing the goal by more than 90 days. However, the percentage 
of reserve component members who missed the goal by more than 90 days 
increased from 18 to 28 percent. 

Figure 7: Percent of Cases Completing MEB Within or in Excess of Goals 
All Fiscal Years Combined: 

[Refer to PDF for image: 4 pie-charts]  

Completed MEB and IDES:  

Active forces: 12,272 cases: 
Met goal: 25%; 
Less than 45 days over goal: 25%; 
45 to 90 days over goal: 20%; 
More than 90 days over goal: 30%.  

Reserves/National Guard: 1,251 cases: 
Met goal: 44%; 
Less than 45 days over goal: 24%; 
45 to 90 days over goal: 14%; 
More than 90 days over goal: 18%.  

Completed MEB but are still enrolled in IDES:  

Active forces: 10,467 cases: 
Met goal: 37%; 
Less than 45 days over goal: 23%; 
45 to 90 days over goal: 16%; 
More than 90 days over goal: 25%.  

Reserves/National Guard: 1,222 cases: 
Met goal: 44%; 
Less than 45 days over goal: 15%; 
45 to 90 days over goal: 13%; 
More than 90 days over goal: 28%.  

Source: GAO analysis of DOD and VA data.  

Note: Numbers may not total 100 percent due to rounding.  

[End of figure]  

For those servicemembers who were still enrolled in the MEB phase as 
of December 2011, the data show that 41 percent of active duty and 33 
percent of reserve component servicemember cases had already missed 
the goal processing times (see figure 8). Of these, 15 percent of 
active duty and 10 percent of reservist component servicemember cases 
missed the goal by more than 90 days.[Footnote 10] 

Figure 8: Timeliness of Cases Enrolled in MEB Stage as of December 
2011: 

[Refer to PDF for image: 2 pie-charts]  

Active forces: 8,493 cases: 
Met goal: 59%; 
Less than 45 days over goal: 16%; 
45 to 90 days over goal: 10%; 
More than 90 days over goal: 15%; 
Total missed goal: 41%.  

Reserves/National Guard: 1,745 cases: 
Met goal: 67%; 
Less than 45 days over goal: 9%; 
45 to 90 days over goal: 15%; 
More than 90 days over goal: 10%; 
Total missed goal: 33%.  

Source: GAO analysis of DOD and VA data.  

Note: Numbers may not match numbers in the body of this report or 
total 100 percent due to rounding.  

[End of figure]  

Within the MEB phase, significant delays have occurred in completing 
medical examinations (medical exam stage) and delivering an MEB 
decision (the MEB stage). For cases completing the MEB phase in fiscal 
year 2011, 31 percent of active duty and 29 percent of reservist cases 
met the 45-day goal for the medical exam stage and 20 percent of 
active duty and 17 percent of reservist cases met the 35-day goal for 
the MEB stage. Officials at some sites we visited told us that MEB 
phase goals were difficult to meet and not realistic given current 
resources. For example: 

* Some military officials noted that they did not have sufficient 
numbers of doctors to write the narrative summaries of exam results 
needed to complete the MEB stage in a timely manner.[Footnote 11] One 
facility noted that while they have 7 doctors, they would need 11 
additional doctors and 10 technician assistants to process cases 
through the initial medical exam and other additional disability 
specific examinations in a timely manner. Further, officials at 
another Army base we visited noted that there was a shortage of 
doctors and DOD board liaisons and that they had difficulty recruiting 
such staff due to the remote location of the base. 

* At all the facilities we visited, officials told us DOD board 
liaisons and VA case managers had large case loads. While DOD has 
established a goal of 1 board liaison for every 20 
servicemembers,[Footnote 12] the ratios varied widely by military 
treatment facility with a range from 1:1 as the lowest to the highest 
of 1:75 according to recent data. Because of high case loads and a 
reported increase in the complexity of cases, staff at one facility 
reported a liaison to servicemember ratio of 1:80 and noted that 
liaisons must often prioritize cases to deal with the most pressing 
issues first. As a result, cases that might otherwise be quick to 
process take longer simply because they are waiting to be processed. 
Liaisons are often working overtime and weekends to keep up with cases. 

Monthly data produced by DOD subsequent to the data we analyzed show 
significantly improved timeliness for the medical exam stage (66 
percent of active duty cases met the goal in June 2012) and some 
improvement for the MEB stage (40 percent of active duty cases met the 
goal in the month of June 2012). However, it is too early to tell 
whether these improvements will continue going forward. (See appendix 
III for DOD reported monthly data, October 2011 - June 2012[Footnote 
13].)  

PEB Phase: 

Since fiscal year 2008, the majority of cases have completed the PEB 
phase under the goal of 120 days, however, PEB timeliness has still 
worsened over time. In 2011, 78 percent of active duty and 62 percent 
of reservist cases that completed the entire IDES process met the PEB 
goal. The average processing time was 93 days for active duty 
servicemembers and 116 for reservists (see table 2). 

Table 2: Average Processing Time in PEB Phase of IDES for Completed 
Cases (in days): 

Component: Active; 
Goal: 120; 
FY 2008: 42; 
FY 2009: 58; 
FY 2010: 80; 
FY 2011: 93. 

Component: Reserve; 
Goal: 120; 
FY 2008: 41; 
FY 2009: 60; 
FY 2010: 83; 
FY 2011: 116. 

Source: GAO analysis of DOD and VA data.  

[End of table]  

Despite meeting the overall PEB goal in fiscal year 2011, established 
goals were not met for any of the interim PEB stages, including the 
informal PEB and VA rating stages which are the two stages all 
servicemembers must complete. For all cases that completed the PEB 
phase in fiscal year 2011, only 38 percent of active duty and 38 
percent of reservists' cases received an informal PEB decision within 
the 15 days allotted. Further, only 32 percent of active duty and 27 
percent of reservist cases received a preliminary VA rating within the 
15-day goal. (see table 3). 

Table 3: Average Processing Time for PEB Stages of IDES for Completed 
Cases (in days): 

Stage of PEB Phase: Informal Physical Evaluation Board (IPEB):  

Component: Active; 
Goal: 15; 
FY 2008: 17; 
FY2009: 12; 
FY2010: 21; 
FY2011: 26. 

Component: Reserve; 
Goal: 15; 
FY 2008: 15; 
FY2009: 12; 
FY2010: 19; 
FY2011: 29. 

Stage of PEB Phase: VA preliminary rating:  

Component: Active; 
Goal: 15; 
FY 2008: 6; 
FY2009: 20; 
FY2010: 27; 
FY2011: 33. 

Component: Reserve; 
Goal: 15; 
FY 2008: 7; 
FY2009: 26; 
FY2010: 32; 
FY2011: 43. 

Stage of PEB Phase: VA rating reconsideration[A]:  

Component: Active; 
Goal: 15; 
FY 2008: 3; 
FY2009: 11; 
FY2010: 14; 
FY2011: 32. 

Component: Reserve; 
Goal: 15; 
FY 2008: 8; 
FY2009: 29; 
FY2010: 18; 
FY2011: 30. 

Stage of PEB Phase: Formal Physical Evaluation Board (FPEB)[A]:  

Component: Active; 
Goal: 30; 
FY 2008: 13; 
FY2009: 39; 
FY2010: 63; 
FY2011: 81. 

Component: Reserve; 
Goal: 30; 
FY 2008: 33; 
FY2009: 38; 
FY2010: 55; 
FY2011: 69. 

Stage of PEB Phase: FPEB appeal[A]:  

Component: Active; 
Goal: 30; 
FY 2008: 196; 
FY2009: 5; 
FY2010: 100; 
FY2011: 103. 

Component: Reserve; 
Goal: 30; 
FY 2008: 20; 
FY2009: 23; 
FY2010: 13; 
FY2011: 169. 

Source: GAO analysis of DOD and VA data: 

[A] These stages are appeals. Fewer servicemembers completed each of 
the appeal stages compared to the non-appeal stages, but the average 
days spent within the appeal stages generally and significantly 
surpassed the goal times.  

[End of table]  

Regarding delays with the VA rating, VA officials told us that 
staffing has been a challenge at their IDES rating sites and that this 
has slowed case processing. Monthly data produced by DOD subsequent to 
the data we analyzed show similar trends for the informal PEB and VA 
preliminary rating stages. As of June 2012 (most recent data 
available), active duty cases showed slight improvements in timeliness 
for the informal PEB stage (41 percent of cases meeting the goal and 
processing times averaging 24 days). The VA rating stage, on the other 
hand, showed slight declines in timeliness (31 percent of cases 
meeting the established goal and processing times of 35 days) relative 
to FY 2011 averages for active duty servicemembers. However, as noted 
before, it is too early to tell the extent to which such trends will 
continue. (See appendix III for DOD reported monthly data, October - 
June 2012.)  

Also during this phase, IDES planners allocated the majority of 
overall PEB processing time (75 out of the 120 days) for appeals--
including a formal PEB hearing and a reconsideration of the VA 
ratings. According to officials, while the three appeal stages do not 
happen for every case, appeals can significantly increase processing 
times for any one case. However, only 20 percent of cases completed in 
fiscal year 2011 actually had any appeals; calling into question DOD 
and VA's assumption on the prevalence and average effect of appeals, 
and potentially masking processing delays in other mandatory parts of 
the PEB phase. 

Transition Phase: 

The transition phase has consistently taken longer than its 45-day 
goal--almost twice as long on average. While processing times improved 
slightly for cases that completed this phase in fiscal year 2011 (from 
79 days in fiscal year 2010 to 76 days in fiscal year 2011 for active 
duty cases), timeliness has remained consistently problematic since 
fiscal year 2008 (see table 4). 

Table 4: Average Processing Time in Transition Phase of IDES for 
Completed Cases (in days): 

Component: Active; 
Goal: 45; 
FY 2008: 88; 
FY 2009: 80; 
FY 2010: 79; 
FY 2011: 76. 

Component: Reserve; 
Goal: 45; 
FY 2008: 81; 
FY 2009: 73; 
FY 2010: 82; 
FY 2011: 75. 

Source: GAO analysis of DOD and VA data.  

[End of table]  

DOD lacks comprehensive data on how servicemembers spend their time in 
the transition phase, which includes many different activities related 
to separation from the military. These activities vary widely 
depending on the case.[Footnote 14] For example, during this phase 
servicemembers receive mandatory training such as job training through 
the Transition Assistance Program and may also receive counseling such 
as pre-discharge Vocational Rehabilitation and Employment counseling. 
In addition, servicemembers may be placed on temporary duty while 
house hunting, or to allow for a servicemember's children to complete 
the school year before moving. Servicemembers may also take earned 
leave time--to which they are entitled--before separating from the 
service. For example, an Army official said that Army policy allows 
servicemembers to take up to 90 days of earned leave prior to 
separating, and that average leave time was about 80 days. Because 
many of these activities can occur simultaneously or in small 
intermittent segments of time, DOD officials said it is difficult to 
track which activities servicemembers participate in or determine how 
much time each activity takes. DOD is exploring options for better 
tracking how time is spent in this phase. Because a potentially 
substantial amount of the time in this phase may be for the personal 
benefit of servicemembers, DOD recently began reporting time in IDES 
with and without the transition phase included. 

Benefits Phase: 

Processing time improved somewhat for the benefits phase (48 days in 
fiscal year 2010 to 38 days in fiscal year 2011), but continued to 
exceed the 30-day goal for active duty servicemembers (see table 
5).[Footnote 15] 

Table 5: Average Processing Time in Benefits Phase of IDES for 
Completed Cases (in days): 

Component: Active; 
Goal: 30; 
FY 2008: 29; 
FY 2009: 43; 
FY 2010: 48; 
FY 2011: 38. 

Source: GAO analysis of DOD and VA data.  

[End of table]  

Several factors may contribute to delays in this final phase. VA 
officials told us that cases cannot be closed without the proper 
discharge forms and that sometimes they do not receive this 
information in a timely manner from the military services. 
Additionally, if data are missing from the IDES tracking system (e.g., 
the servicemember already separated, but this was not recorded in the 
database), processing time will continue to accrue for cases that 
remain open in the system. Officials could not provide data on the 
extent to which these factors had an impact on processing times for 
pending cases, but said that once errors are detected and addressed, 
reported processing times are also corrected. 

Shortcomings in the Design and Administration of Servicemember Surveys 
Hamper their Usefulness: 

In addition to timeliness, DOD and VA evaluate IDES performance using 
the results of servicemember satisfaction surveys. In principle, all 
members have an opportunity to complete satisfaction surveys at the 
end of the MEB, PEB, and transition phases; however, under current 
survey procedures servicemembers become ineligible to complete a 
survey for either the PEB or transition phases if they did not 
complete a survey in an earlier phase. Additionally, servicemembers 
who start but do not complete a phase are not surveyed. As such, DOD 
may be missing opportunities to obtain input from servicemembers who 
did not complete a prior survey or exited IDES in the middle of a 
phase.[Footnote 16] Further, response rates may be affected because 
DOD does not survey servicemembers once they separate from the service 
and become veterans. While it is not necessary for DOD to survey all 
servicemembers at the end of every phase,[Footnote 17] the percentage 
and characteristics of servicemembers covered by the survey (i.e., who 
completed a phase and were ultimately interviewed) may be insufficient 
to establish that the survey results are representative of 
servicemember satisfaction, especially for later phases. (See table 6 
for response and coverage rates.) DOD officials recently told us that 
they will consider alternative survey eligibility requirements, 
including working with the Office of Management and Budget for 
permission to interview veterans. (For additional information 
regarding the timing of the survey, see appendix II). 

Table 6: Survey Response and Coverage Rates: 

Phase: MEB; 
Servicemembers who completed phase (N): 25,212; 
Servicemembers eligible to be surveyed (i.e., completed both phase and 
survey for prior phase, if applicable): all; 
Servicemembers who completed survey: 9,604; 
Response rate (Servicemembers who completed survey/Servicemembers 
eligible): 38.1%; 
Coverage rate (Servicemembers who completed survey/Servicemembers who 
completed phase): 38.1%. 

Phase: PEB; 
Servicemembers who completed phase (N): 18,296; 
Servicemembers eligible to be surveyed (i.e., completed both phase and 
survey for prior phase, if applicable): 8,968; 
Servicemembers who completed survey: 4,795; 
Response rate (Servicemembers who completed survey/Servicemembers 
eligible): 53.5%; 
Coverage rate (Servicemembers who completed survey/Servicemembers who 
completed phase): 26.2%. 

Phase: Transition; 
Servicemembers who completed phase (N): 12,352; 
Servicemembers eligible to be surveyed (i.e., completed both phase and 
survey for prior phase, if applicable): 3,996; 
Servicemembers who completed survey: 2,893; 
Response rate (Servicemembers who completed survey/Servicemembers 
eligible): 72.4%; 
Coverage rate (Servicemembers who completed survey/Servicemembers who 
completed phase): 23.4%. 

Source: GAO analysis of DOD and VA data.  

[End of table]  

In addition, alternate survey measures show lower satisfaction rates 
than those reported by DOD. Using DOD's measure, we found an overall 
satisfaction rate of about 67 percent since the inception of IDES. DOD 
defines a servicemember as satisfied if the average of his or her 
responses across several surveys is above 3 on a 5-point scale, with 3 
denoting neither satisfied nor dissatisfied. However, using our 
alternate measure that defines servicemembers as satisfied only when 
all of their responses are 4 or above,[Footnote 18] we calculated the 
satisfaction rate to be about 24 percent (see figure 9). 

Figure 9: Comparison of Percent of Servicemembers Satisfied Using GAO 
and DOD Calculations for Overall Satisfaction: 

[Refer to PDF for image: horizontal bar graph]  

Active forces, 8,692 cases: 
DoD's calculation: 67.4%; 
GAO's calculation: 23.7%.  

Reserve/National Guard, 1,173 cases: 
DoD's calculation: 63.5%; 
GAO's calculation: 24.6%.  

Source: GAO analysis of DOD and VA data.  

Note: GAO combined reserve and national guard, while DOD reported 
satisfaction scores for guard and reserve separately.  

[End of figure]  

Our calculation is a more conservative measure of satisfaction, 
because it rules out the possibility that a servicemember is deemed 
"satisfied" even when he or she is dissatisfied on one or more 
questions in the scale. While not incorrect, DOD's scale can mask 
pockets of servicemember dissatisfaction. For example, an individual 
may indicate that he or she is very dissatisfied with one phase of the 
program, but satisfied with other phases, and the overall satisfaction 
score can be the same as one for a servicemember who is generally 
satisfied across all phases of the process. Measuring satisfaction, or 
even dissatisfaction, in different ways may provide a more complete 
picture of satisfaction and how it varies in different circumstances, 
and thus may reveal areas where DOD could focus on improving 
management and performance. 

Finally, using either DOD's or our calculated measure, we found that 
overall satisfaction did not vary much according to differences in the 
experiences of servicemembers. For example, our model estimated that 
satisfaction varied by no more than approximately five percentage 
points across branch, component, disenrollment outcome, sex, MEB exam 
provider, enlisted and officer personnel classes, and the number of 
claimed and referred conditions. While lack of variation could be a 
positive outcome signaling consistent treatment, it could equally mean 
that the survey does not measure opinions in enough detail to 
discriminate among servicemembers' experiences. Either way, such 
results provide little insight into identifying areas for improvement 
or effective practices. Further, while we found some association 
between servicemembers satisfaction and the timeliness of their case 
processing, we also found many servicemembers were highly dissatisfied 
even when their cases were completed on time, and many were highly 
satisfied even when their cases were not. For example, 68 percent of 
those who said that PEB timeliness was "very poor" completed the phase 
on time, and 55 percent of those who said that MEB timeliness was 
"very good" did not complete on time. The lack of variation and/or 
correlation between satisfaction and experiences of servicemembers--
coupled with low coverage rates--raise questions about the value of 
the survey results as a performance measure and program evaluation 
tool. (See appendix II for more information on servicemember 
satisfaction results.)  

DOD is reconsidering its options for measuring customer satisfaction, 
but has yet to select a particular approach. As noted above, possible 
changes might include widening the criteria for who is eligible for 
the survey, modifying survey questions, changing when and how the 
survey is delivered, and changing how satisfaction is calculated. 
Officials already concluded that the survey, in its current form, is 
not a useful management tool for determining what changes are needed 
in IDES and said that it is expensive to administer--costing 
approximately $4.3 million in total since the start of the IDES pilot. 
Navy officials told us they believed that the satisfaction surveys 
could be made more useful if they knew whether servicemember's 
satisfaction was actually influenced by the servicemember's desired or 
actual outcome of the IDES process. Further, Army officials already 
determined that the DOD survey is of limited value, and are proceeding 
with plans to field their own survey in the hopes of obtaining more 
detailed information at the facility level. Because of fiscal 
constraints, DOD suspended the survey in December 2011, but officials 
told us that they hope to resume collecting data in fiscal year 2013. 

We identified two potential alternatives to assessing servicemember 
experiences. 

* Surveying a sample of servicemembers: While a census gives each 
servicemember a chance to describe his or her experiences with IDES, 
DOD could collect the same data at a lower cost by surveying a 
probability sample of servicemembers. If appropriately designed and 
executed, a sample would accurately represent all groups of 
servicemembers and produce the necessary data for important subgroups, 
such as facilities or branches. Since the cost of administering a 
survey is strongly related to the number of people surveyed, 
probability sampling could also allow DOD to assess servicemember 
experiences while substantially reducing data collection costs. 

* Exit interviews: In-depth interviews with servicemembers, completed 
at disenrollment from IDES, could also yield more detailed and 
actionable information about the program. Although the current survey 
includes open-ended questions, it is primarily designed to collect 
standardized, quantitative measures of satisfaction with broad aspects 
of IDES, such as fairness and the performance of DOD board liaisons 
and VA case managers. As a result, the survey provides a limited 
amount of detailed feedback on particular facilities, staff members, 
and stages of the process that managers might use to improve the 
servicemember experience, decrease processing times, or reduce cost. 
In contrast, semi-structured exit interviews would allow 
servicemembers to provide this type of qualitative, detailed feedback. 
Interviewing servicemembers at the end of the process would also allow 
servicemembers to assess their overall experiences with IDES rather 
than at an earlier stage, without having completed the entire process. 
Exit interviews could also reach servicemembers who exit IDES without 
completing the process such as those who are returned to duty. Exit 
interviews, however, have the potential to be labor intensive and 
expensive. 

Recent Actions and Ongoing Initiatives May Improve IDES Performance, 
but It Is Too Early to Assess Their Overall Impact: 

DOD and VA Took Steps to Address Previously Identified IDES Challenges: 

DOD and VA have undertaken a number of actions to address IDES 
challenges--many of which we identified in our past work. Some 
actions--such as increased oversight and staffing--represent important 
steps in the right direction, but progress is uneven in some areas. 

Increased monitoring and oversight: We identified the need for agency 
leadership to provide continuous oversight of IDES in 2008[Footnote 
19] and the need for system-wide monitoring mechanisms in 2010. Since 
then, agency leadership has established mechanisms to improve 
communication, monitoring, and accountability. 

* The secretaries of DOD and VA have met several times since February 
2011 to discuss progress in improving IDES timeliness and have tasked 
their agencies to find ways to streamline the process so that the 
timeliness goals can be shortened. The secretaries also tasked their 
agencies to expand the use of expedited disability evaluations for 
severely combat-wounded servicemembers;[Footnote 20] and develop a 
system to electronically transfer case files between DOD and VA 
locations. 

* Senior Army and Navy officials regularly hold conferences to assess 
performance and address performance issues, including at specific 
facilities. With respect to the Army, meetings are led by the Army's 
vice-chief of staff and VA's chief of staff, and include reviews of 
performance where regional and local facility commanders provide 
feedback on best practices and challenges. For example, recent Army-VA 
conferences focused on delays in completion of preliminary ratings for 
Army PEBs by VA's Seattle rating site, efforts by the Army to increase 
MEB staffing, development of Army-wide IDES standardization guidance, 
and Army-VA electronic records interchange. Periodic meetings are also 
held between senior Navy medical and VA officials to discuss 
performance issues at Navy military treatment facilities. 

VA holds its own biweekly conferences with local staff responsible for 
VA's portion of the process. These conferences are supplemented by a 
bi-weekly IDES "dashboard" that tracks performance data for portions 
of the IDES for which VA is responsible. According to VA officials, in 
addition to identifying best practices, these conferences focus on 
sites with performance problems and identify potential corrective 
actions. For example, officials said a recent conference addressed 
delays at Fort Benning, Georgia, and discussed how they could be 
reduced. VA officials noted that examiner staff were reassigned to 
this site and worked on weekends to address the problems at this site. 
In addition, senior VA health care officials hold periodic conferences 
with officials responsible for exams at IDES sites, to monitor 
performance. 

Ensuring sufficient medical exam resources: In our December 2010 
report, we noted that VA struggled to provide enough medical examiners 
(both VA employees and contractors) to meet demand and deliver exam 
summaries within its 45-day goal. For example, significant 
deficiencies in examiner staffing (particularly for mental health 
exams) at Fort Carson contributed to exams for active duty members 
taking an average of 140 days. To improve exam timeliness, VA hired 
more examiners and is devoting more resources at those sites where VA 
clinicians perform IDES exams. In addition, in July 2011, VA awarded a 
revised compensation and pension (including IDES) contract that 
provides more flexibility for VA to have contractors perform IDES 
exams at sites needing additional resources. As a result, VA can use 
contractors to conduct exams for regional offices beyond the 10 
offices for which the contractor normally provides services. Also, VA 
contracted with 5 companies to provide short-term exam assistance at 
IDES sites needing it. Further, VA procedures allow reserve component 
servicemembers in remote locations to receive exams close to their 
homes. VA exam timeliness has improved and the agency met its 45-day 
goal for active component members in every month from August 2011 
through June 2012. VA officials attributed improved exam timeliness, 
in part, to additional exam resources provided to IDES sites. (See 
appendix III for additional information on fiscal year 2012 
timeliness.)  

Ensuring sufficient exam summaries: In our December 2010 report, we 
noted that some cases were delayed because VA medical exam summaries 
were not complete and clear enough for use in making rating and 
fitness decisions and needed to be sent back to examiners for 
additional work. VA officials told us that they have been reinforcing 
the importance of training and communication between rating staff and 
medical examiners as ways to improve exam summary sufficiency. For 
example, VA identified types of information which, if missing from an 
exam summary, would cause it to be insufficient, and has been training 
examiners to include such information. Additionally, VA noted that VTA 
now has the ability to track cases with insufficient exams by allowing 
staff to annotate information on exam summaries. However, staff are 
not required to provide this information and rules and procedures for 
its use have not been established. 

Ensuring sufficient MEB staffing: In our December 2010 report, we 
noted that some sites had insufficient MEB physicians, leading to 
delays in completing the MEB phase. At that time, most of the 27 pilot 
sites were not meeting the 35-day goal, with average times for active 
component cases as high as 109 days. Meanwhile, DOD did not have 
sufficient board liaison staff to handle IDES caseloads. The Army is 
in the midst of a major hiring initiative intended to more than double 
staffing for its MEBs over its October 2011 level, which will include 
additional board liaison and MEB physician positions.[Footnote 21] The 
Army reported having 610 full-time equivalent MEB staff positions in 
October 2011, and planned to hire up to 1,410; this would include 172 
MEB physician and 513 board liaison positions. The Army also planned 
to hire an average of one contact representative per board liaison; 
these staff members assist the board liaisons with clerical functions, 
freeing more of the liaisons' time for counseling servicemembers. As 
of June 2012, the Army had filled 1,219 (86 percent) of the planned 
1,410 positions. 

Ensuring sufficient VA rating staff: In our December 2010 report, we 
noted that VA had insufficient staff at one of its rating sites to 
handle the demand for preliminary ratings, rating reconsiderations, 
and final VA benefit decisions. VA officials said that the agency has 
more than tripled the staffing at its IDES rating sites-from 78 to 262 
positions. Further, VA has moved staff resources to IDES rating sites 
from other VA regional offices to provide short-term help in working 
down rating backlogs. Recent monthly data show an increase in the 
number of preliminary VA ratings completed, and a slight improvement 
in processing times. However, as noted before, it is too early to tell 
the extent to which such trends will continue. (See appendix III for 
additional information on fiscal year 2012 timeliness.)  

Improving completeness of reserve component members' records: Service 
officials noted that incomplete medical records and administrative 
documentation, especially for reserve component members, often 
contribute to delays in the early IDES stages, including the VA exam 
stage. For example, a reserve unit may not have complete medical 
records for a member who received care from a private provider. When 
the servicemember enters the IDES, a board liaison is responsible for 
obtaining the private provider records before handing off the case to 
VA for exams. To address issues with reserve component servicemembers' 
records, the Army established an interim office in Pinellas Park, 
Florida in January 2011. For reserve component servicemembers who may 
require IDES referral, this office is tasked with obtaining records 
from the member's reserve unit; reviewing them to identify missing 
information; and, if necessary, requiring the reserve unit to obtain 
additional records to complete the case file. Staff at this office 
also determine whether the member needs IDES referral.[Footnote 22] 
Army officials indicated that this office is expected to help reduce 
the backlog of Army reserve component cases in the IDES. However, Army 
officials noted that they are providing training to reserve units to 
improve their ability to maintain complete records on their 
servicemembers and eventually, the Army may discontinue this office if 
no longer needed. 

Improving MEB documentation and decisions: In response to delays in 
completing the MEB stage, the Navy and Army have initiatives underway 
to help ensure the timely completion of narrative summaries and MEB 
decisions. For example, the Navy piloted electronic narrative summary 
preparation at Naval Hospital Camp Lejeune, North Carolina. In May 
2012, after determining that the piloted process led to improved MEB 
completion timeliness, the Navy deployed electronic narrative summary 
preparation Navy-wide. In March 2011, the Army also deployed an 
abbreviated MEB narrative summary format, intended to provide better 
information for MEB and PEB decision making while helping reduce 
delays in the completion of these summaries by MEB clinicians. 
Incorporating feedback from its MEBs and PEBs, the Army expects the 
revised IDES template to reduce redundant information, make summaries 
simpler and easier to use, and standardize summary preparation across 
their sites. 

Resolving diagnostic differences: In our December 2010 report, we 
identified differences in diagnoses between DOD physicians and VA 
examiners, especially regarding mental health conditions, as a 
potential source of delay in IDES. We also noted inconsistencies among 
services in providing guidance and a lack of a tracking mechanism for 
determining the extent of diagnostic differences. In response to our 
recommendation, DOD commissioned a study on the subject. The resulting 
report confirmed the lack of data on the extent and nature of such 
differences, and noted that the Army has established guidance more 
comprehensive than the guidance DOD was developing. It also 
recommended that DOD or the other services develop similar guidance. A 
DOD official told us that consistent guidance across the services, 
similar to the Army's, was included in DOD's December 2011 IDES 
manual. Also, in response to our recommendation, VA took steps to 
modify the VTA database used to track IDES to collect information on 
diagnostic differences. The VTA upgrade was completed in June 2012 
after several delays. The report also recommended that DOD and VA 
establish a committee to improve the accuracy of posttraumatic stress 
disorder ratings. DOD noted that training on diagnostic differences 
has been incorporated into its continuing medical education curriculum 
for military clinicians, but DOD considers the issue of posttraumatic 
stress disorder ratings largely resolved. Meanwhile, the Army's new 
IDES narrative summary template includes a section where the MEB 
clinician identifies any inconsistencies in the case record, including 
any diagnostic differences with VA examiners. 

DOD and VA Are Addressing Shortcomings in Information Systems, but 
Efforts to Date Are Limited: 

DOD and VA are working to remedy shortcomings in information systems 
that support the IDES process. These shortcomings include VTA's lack 
of capability for local sites to track cases, and the potential for 
erroneous and missing data in VTA, affecting timeliness measurement. 
However, some efforts related to information systems are causing work 
inefficiencies, are still in progress, or otherwise are limited. 

Improving local IDES reporting capability: DOD and VA are implementing 
solutions to improve the ability of local military treatment 
facilities to track their IDES cases, but multiple initiatives may 
result in redundant work efforts. Officials told us that the VTA--
which is the primary means of tracking the completion of IDES cases--
has limited reporting capabilities and staff at local facilities are 
unable to use it for monitoring the cases for which they are 
responsible. DOD and VA developed VTA improvements that will allow DOD 
board liaisons and VA case managers--and their supervisors--to track 
the status of their cases. VA included these operational reporting 
improvements in its June 2012 VTA upgrade. In the meantime, staff at 
many IDES sites have been using their own local systems to track cases 
and alleviate limitations in VTA. Further, the military services have 
been moving ahead with their own solutions. For instance, the Army has 
deployed its own information system for MEBs and PEBs Army-wide. In 
addition, DOD has also been piloting its own tracking system at 9 IDES 
sites.[Footnote 23] As a result, staff at IDES sites we visited 
reported having to enter the same data into multiple systems. For 
example, board liaisons at Army MEBs Fort Meade and Joint Base Lewis-
McChord reported entering data into VTA and the Army's new system, 
while board liaisons at Andrews Air Force Base reported entering data 
into VTA and DOD's pilot data system. 

Improving IDES data quality: DOD is taking steps to improve the 
quality of data in VTA. Our analysis of VTA data identified erroneous 
or missing dates in at least 4 percent of the cases reviewed. 
Officials told us that VTA lacks adequate controls to prevent 
erroneous data entry, and that incorrect dates may be entered, or 
dates may not be entered at all, which can result in inaccurate 
timeliness data. For example, Army officials noted that some cases 
shown in VTA as very old were actually closed, but were missing key 
dates. In September 2011, DOD began a focused effort with the services 
to correct erroneous and missing case data in VTA. Officials noted 
that the Air Force and Navy completed substantial efforts to correct 
the issues identified at that time, but Army efforts continue. DOD and 
Army officials noted that additional staff resources are being devoted 
to cleaning up Army VTA data. While improved local tracking and 
reporting capabilities will help facilities identify and correct 
erroneous data, keeping VTA data accurate will be an ongoing challenge 
due to a lack of data entry controls. While DOD is currently assisting 
the services, DOD officials said they expect that eventually the 
services will be responsible for identifying and fixing data errors. 

DOD and VA are also pursuing options to allow them to save time by 
replacing the shipping of paper case files among facilities with 
electronic file transfers. Requirements for an electronic case file 
transfer solution have been completed and DOD and VA officials expect 
to begin piloting it in August 2012. As a short-term solution, the 
Army and VA began using an Army file transfer Web site to move IDES 
records between the Army's PEBs and the Seattle VA rating site in 
March 2012.[Footnote 24] According to VA officials, this could save 
several days currently spent shipping paper files between these 
offices. VA officials noted that the same Web site is being used for 
transfers between the Navy PEB and Providence rating site. Meanwhile, 
the secretaries of Defense and Veterans Affairs tasked their staffs to 
develop standards for electronic IDES case files by July 2012. 

DOD and VA are Pursuing Broader Solutions to Improving IDES 
Performance: 

Based on concerns of the Secretaries of DOD and VA about IDES delays, 
the departments have undertaken additional initiatives to achieve time 
savings for servicemembers. For example, in response to the 
secretaries' February 2011 directive to streamline the process, DOD 
and VA officials proposed a remodeled IDES process. In December 2011, 
senior agency leadership decided to postpone the pilot of a remodeled 
IDES process, and instead tasked the agencies to explore other ways to 
streamline the process.[Footnote 25] As a result, DOD, with VA's 
assistance, began a business process review to better understand how 
IDES is operating and identify best practices for possible 
implementation. This review incorporates several efforts, including 
visits to 8 IDES sites to examine how the process was operating and 
identify best practices.[Footnote 26] This review also includes the 
following: 

* Process simulation model: Using data from site visits and VTA, DOD 
is developing a simulation model of the IDES process. According to a 
DOD official, this process model will allow the agencies to assess the 
impact of potential situations or changes on IDES processing times, 
such as surges in workloads or changes in staffing. 

* Fusion diagram: DOD is developing this diagram to identify the 
various sources of IDES data--including VA claim forms and narrative 
summaries--and different information technology systems that play a 
role in supporting the IDES process. Officials said this diagram would 
allow them to better understand and identify overlaps and gaps in data 
systems. 

Ultimately, according to DOD officials, this business process review 
could lead to short-and long-term recommendations to improve IDES 
performance, potentially including changes to the different steps in 
the IDES process, performance goals, and staffing levels; and possibly 
the procurement of a new information system to support process 
improvements. However, a DOD official noted that these efforts are in 
their early stages, and thus there is no timetable yet for completing 
the review or providing recommendations to senior DOD and VA 
leadership. DOD officials indicated that they expect this to be a 
continuous IDES improvement process, including further site visits. 

Finally, DOD is also developing guidance to expand implementation of 
an expedited disability evaluation process for servicemembers with 
catastrophic, combat-related conditions by allowing it to be operated 
at more military treatment facilities. DOD created this expedited 
process in January 2009 for servicemembers who suffer catastrophic, 
combat-related disabilities. Under an agreement with VA, the services 
can rate such members as 100 percent disabled without the need to use 
VA's rating schedule. However, according to DOD officials, the 
services report that no eligible servicemembers are using this 
process. Instead, servicemembers are having their cases expedited 
through the IDES informally. The revisions to DOD's policy would allow 
the expedited process to be used at additional military treatment 
facilities beyond the original 4 facilities.[Footnote 27] According to 
DOD officials, this guidance will be part of a rewrite of DOD's key 
guidance documents, and was undergoing review at the time of our 
review. 

Conclusions: 

By merging two duplicative disability evaluation systems, IDES shows 
promise for expediting the delivery of DOD and VA benefits to injured 
servicemembers and is considered by many to be an improvement over the 
legacy process it replaced. However, nearly 5 years after its 
inception as a pilot, delays continue to affect the system and the 
contribution of various, complex factors to timeliness is not fully 
understood. Recent efforts by DOD and VA to better understand how 
different IDES processes contribute to timeliness are promising and 
may provide the departments with an opportunity to reassess resource 
levels and timeframes, and to make adjustments if needed. This 
information will also help to ensure that DOD and VA are making the 
best use of limited resources to improve IDES performance. However, it 
is not clear when these efforts will be complete or if any recommended 
actions will be implemented. DOD has also begun rethinking its 
approach to determining servicemember satisfaction with IDES. Our 
analysis of customer satisfaction data suggests that there are 
opportunities for improving the representativeness of the survey 
information collected and reconsidering the cost-effectiveness of the 
current lengthy surveys. Finally, providing local facilities the 
capability to track and generate reports on the status of their cases 
is long overdue and may empower local staff to better address 
challenges. However, tracking reports are only as good as the data 
that are entered into VTA, and DOD and VA can ensure the quality of 
these data through continuous monitoring. Meanwhile, the DOD-led 
business process review should identify and ultimately eliminate any 
redundant or inefficient information systems for tracking cases as 
well as for other IDES purposes. 

Recommendations for Executive Action: 

1) To ensure that servicemember cases are processed and are awarded 
benefits in a timely manner, we recommend that the Secretaries of 
Defense and Veterans Affairs work together to develop timeframes for 
completing the IDES business process review and implementing any 
resulting recommendations. 

2) To improve DOD's ability to measure servicemembers' satisfaction 
with the IDES process, we recommend that the Secretary of Defense 
develop alternative approaches for collecting more meaningful and 
representative information in a cost effective manner. 

3) To ensure that IDES management decisions continue to be based upon 
reliable and accurate data, we recommend that the Secretaries of 
Defense and Veterans Affairs work together to develop a strategy to 
continuously monitor and remedy issues with VTA timeliness 
information. This could include issuing guidance to facilities or 
developing best practices on preventing and correcting data entry 
errors; and developing reporting capabilities in VTA to alert 
facilities to potential issues with their data. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to DOD and VA for review and 
comment. In their written comments, which are reproduced in appendixes 
IV and V, DOD and VA both concurred with our recommendations. VA also 
provided technical comments, which we incorporated as appropriate. 

While concurring with our recommendations, DOD also commented that our 
discussion of IDES surveys contained inaccuracies, but did not specify 
the inaccurate information in our draft report. In a subsequent 
communication, DOD officials noted that our draft inaccurately 
described DOD's decision to not survey veterans. We corrected this 
information accordingly. Further, while VA concurred with our 
recommendation that it work with DOD to develop timeframes for 
completing the IDES business process review and implementing any 
resulting recommendations, VA stated that DOD is leading the business 
process review, and therefore should develop the timeframes for 
completing the review. We have revised this report to clarify that DOD 
is leading the business process review, but we did not alter the 
recommendation because we believe that it is important for VA to work 
closely with DOD, including in developing review timeframes. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the appropriate congressional committees, the Secretary of Defense, 
the Secretary of Veterans Affairs, and other interested parties. The 
report is also available at no charge on the GAO Web site at 
[hyperlink, http://www.gao.gov]. 

If you or your staff members have any questions about this report, 
please contact me at (202) 512-7215 or at bertonid@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. Staff members who made 
key contributions to this report are listed in appendix VI.  

Signed by:  

Daniel Bertoni: 
Director, Education, Workforce, and Income Security Issues:  

[End of section]  

Appendix I: Objectives, Scope, and Methodology: 

In conducting our review of the Integrated Disability Evaluation 
System (IDES), our objectives were to examine (1) the extent to which 
the Departments of Defense (DOD) and Veterans Affairs (VA) are meeting 
IDES performance goals, and (2) steps DOD and VA are taking to improve 
IDES performance. We conducted this performance audit from May 2011 to 
August 2012, in accordance with generally accepted government auditing 
standards. These standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Review of IDES Timeliness Data: 

To determine the extent to which IDES is meeting established 
timeliness goals, we analyzed data collected through VA's Veterans 
Tracking Application (VTA) database. While VA manages VTA, both DOD 
and VA staff enter data into VTA, and the evaluation of IDES data is 
primarily conducted by staff at DOD's Office of Warrior Care Policy 
(WCP).[Footnote 28] WCP provided us with a dataset that was current as 
of January 1, 2012 and contained data spanning back to the inception 
of IDES in late 2007. This data export included data on a total of 
39,260 cases. Of these cases, 34,185 were active duty servicemembers 
and 5,068 were Reserve/Guard servicemembers.[Footnote 29] This VTA 
data set contained demographic data for each individual IDES case as 
well as a record of dates for when servicemembers reached various 
milestones in IDES.[Footnote 30] Overall and interim IDES timeliness 
calculations are based on computing the number of days elapsed between 
appropriate milestone dates. For example, overall timeliness for 
servicemembers that receive benefits is calculated as the number of 
days between the individual being referred into the IDES and the date 
on which his or her VA benefits letter is issued. We met with staff at 
WCP to ensure we used appropriate variables when calculating 
timeliness. We also met with officials at VA to discuss the 
calculations used to determine the timeliness of cases. 

We took a number of steps to assess the reliability of VTA data and 
ultimately found the data to be sufficiently reliable for the purposes 
of our audit. Past GAO work[Footnote 31] relied on VTA data, and 
therefore we took a number of steps to follow up on past assessments 
of VTA. 

* We interviewed DOD and VA and determined that internal controls on 
VTA data had not changed substantially since our past review. 

* We conducted electronic testing of the VTA data and generally found 
low rates of missing data or erroneous dates pertinent to our 
analysis--approximately 4 percent of cases. [Footnote 32] For IDES 
cases in which we found missing dates or dates out of sequence, we 
excluded those cases from all of our analyses. While there were some 
instances in which the erroneous dates may be justified, we excluded 
the entire case from our analysis if any such dates appeared at any 
point in the VTA database. Such data included cases in which (1) there 
was no MEB referral date signifying the start of IDES process, and (2) 
the ending date preceded the beginning date of the IDES phase 
(resulting in timeliness calculations appearing as a negative amount 
of time). 

* We also conducted a limited trace-to-file process to determine 
whether date fields in VTA were an accurate reflection of the 
information in IDES case files. Specifically, we compared VTA dates in 
15 IDES cases completed in fiscal year 2011 against the dates in the 
corresponding paper files. In comparing dates, we allowed for a 
discrepancy of 5 days in dates to allow for the possibility that dates 
may have been entered into the database after an event took place. 
Ninety-three percent of the dates we traced back to the original file 
documents were found to be accurate, that is falling within our 5 day 
allowance. 

Analysis Conducted Using VTA Data: 

For the cases meeting our criteria for reliability, we analyzed 
timeliness data for those cases that had completed the entire IDES 
process or had completed each of the four IDES phases. We specifically: 

* Identified the total number of cases enrolled each fiscal year 
[Footnote 33] from FY 2008 through 2011, by active as well as National 
Guard and reserve servicemembers, and by military branch of service. 
[Footnote 34] 

* Identified the number of cases that completed the entire IDES 
process for each fiscal year from fiscal year 2008 through fiscal year 
2011. We analyzed completed cases in two different ways: (1) those who 
completed the process and received VA benefits and (2) those who 
completed the IDES with any outcome (such as permanent retirement, 
Temporary Disability Retirement List, return to duty, etc.). In order 
to be able to make comparisons across cases with different outcomes 
for a given point in time, we defined fiscal year by using the VTA 
variable "final disposition date". We did this because most completed 
cases--regardless of outcome--have a final disposition date in VTA. In 
contrast to our approach, VA use the "VA benefit date" variable to 
determine fiscal year of completion for cases resulting in benefits. 
As such, their number of cases and timeliness calculations by fiscal 
year differed from ours, although overall trends are similar. 

* Identified the number of cases that completed each phase of IDES and 
the interim stages within each phase, again by fiscal year (fiscal 
years 2008 through 2011). 

* Computed timeliness statistics for the completion of the IDES 
process, phases, and stages against the performance goals set by DOD 
and VA, such as average days and percent meeting goals. 

* Computed number and percent of cases where a servicemember appealed 
a decision made during the IDES process, by fiscal year. 

For the purposes of this report, GAO opted to not include reserve 
component time spent in the VA benefit phase in our calculations for 
overall time because the 30 days allotted for this phase is not 
included in the 305-day overall goal for the reserve component. 

GAO also performed analyses similar to those above, except that we 
grouped cases according to the year in which they were enrolled in 
IDES. (See appendix II for more detail on these analyses.) 
Additionally, we analyzed timeliness for cases that had not yet 
completed the MEB stage as of the date we received the VTA data. 

Review of IDES Customer Satisfaction Data: 

To determine the extent to which IDES is meeting its customer 
satisfaction goals, we analyzed data collected from IDES customer 
satisfaction surveys conducted at the end of three phases: MEB, PEB 
and Transition. These surveys are administered by telephone by 
contractors hired by DOD. The dataset we received contained survey 
responses for individual servicemembers from the beginning of the IDES 
pilot to December 2011, at which time administration of the survey was 
suspended. Additionally, we matched individual survey responses with 
information from VTA to gain additional understanding into how 
customer satisfaction varied according to different factors such as 
timeliness and case outcome. We matched survey and VTA data using the 
unique case identifier attached to each IDES case, maintaining the 
anonymity of the servicemembers. See appendix II for the results of 
additional analyses we conducted using survey data and survey data 
matched with VTA data. 

In the course of our review we concluded that the survey data were 
sufficiently reliable for our purposes. We interviewed relevant 
officials at DOD and their contractors about eligibility requirements 
and the administration of the surveys. Further, we met with DOD and 
their contractors on multiple occasions to discuss the calculations 
used to determine response rates for the survey and servicemembers' 
level of satisfaction. See appendix II for more details on GAO's 
review of response rates. 

Identifying Challenges and Actions Taken to Improve Performance: 

To identify challenges in implementing IDES as well as steps taken to 
improve performance, we visited six military treatment facilities. 
During the site visits, we interviewed officials involved in 
implementing IDES from both DOD and VA, including military facility 
commanders and administrators, DOD board liaisons, military physicians 
involved in MEB determinations, DOD legal staff, VA case workers, VA 
or contract examiners, and administrators at VA medical clinics and 
regional offices. Additionally, we interviewed servicemembers who were 
currently enrolled in the IDES process. We selected the six facilities 
to obtain perspectives from sites in different military services, 
geographic areas, and their ability to meet timeliness goals for 
different phases of the process (see table 7). In addition, we visited 
the Air Force's Formal Physical Evaluation Board at Randolph Air Force 
Base, Texas. During this visit we observed a hearing and met with 
board members to obtain a better understanding of the process. 

Table 7: Selected Characteristics of IDES Pilot Sites Visited as of 
May 2011: 

Military treatment facility: Joint Base Andrews; 
Military service: Air Force; 
State: Maryland; 
IDES caseload: 511; 
Average exam times (goal = 45 days): 24; 
Average MEB stage times (goal = 35 days): 56; 
Average Total IDES time (goal = 295 days): 472. 

Military treatment facility: Fort Hood; 
Military service: Army; 
State: Texas; 
IDES caseload: 1,976; 
Average exam times (goal = 45 days): 31; 
Average MEB stage times (goal = 35 days): 107; 
Average Total IDES time (goal = 295 days): 334. 

Military treatment facility: Joint Base Lewis-McChord; 
Military service: Army; 
State: Washington; 
IDES caseload: 1,069; 
Average exam times (goal = 45 days): 46; 
Average MEB stage times (goal = 35 days): 102; 
Average Total IDES time (goal = 295 days): 339. 

Military treatment facility: Fort Meade; 
Military service: Army; 
State: Maryland; 
IDES caseload: 543; 
Average exam times (goal = 45 days): 35; 
Average MEB stage times (goal = 35 days): 140; 
Average Total IDES time (goal = 295 days): 556. 

Military treatment facility: Fort Sam Houston; 
Military service: Army; 
State: Texas; 
IDES caseload: 1,336; 
Average exam times (goal = 45 days): 65; 
Average MEB stage times (goal = 35 days): 72; 
Average Total IDES time (goal = 295 days): 412. 

Military treatment facility: Naval Hospital Bremerton; 
Military service: Navy; 
State: Washington; 
IDES caseload: 199; 
Average exam times (goal = 45 days): 52; 
Average MEB stage times (goal = 35 days): 14; 
Average Total IDES time (goal = 295 days): 348. 

Source: GAO analysis of DOD's June 2011 IDES Monthly Report. 

Note: Processing times listed in table are for active duty cases.  

[End of table]  

We also interviewed officials in various offices at DOD and VA 
involved in implementing IDES. At DOD, these offices included Warrior 
Care Policy; Office of the Assistant Secretary of Defense (Reserve 
Affairs); Office of the Deputy Chief Management Officer; Air Force 
Physical Disability Division; Army Physical Disability Agency; Navy 
Physical Evaluation Board; Air Force Surgeon General; Army Medical 
Command; and Navy Bureau of Medicine and Surgery. At VA, we 
interviewed officials in the Veterans Benefits Administration, 
Veterans Health Administration, and the VA/DOD Collaboration Service. 
Furthermore, we reviewed relevant documents, including DOD and VA 
policies, federal laws, regulations, directives[Footnote 35] and 
guidance, a study produced for DOD on diagnostic variances, and plans 
to streamline IDES or improve performance.  

[End of section]  

Appendix II: Additional Timeliness and Satisfaction Analyses: 

This appendix provides additional information on the timeliness of the 
IDES process and servicemember satisfaction with it. First, we use 
timeliness data to examine whether changes over time in processing 
times and the percentage of cases meeting timeliness goals look any 
different when cases are grouped according to the fiscal year in which 
the cases were first enrolled rather than the fiscal year in which the 
cases were completed.[Footnote 36] Second, we use survey data to 
examine different measurements of servicemember satisfaction with 
IDES, how satisfaction varied according to various servicemember 
characteristics, response and coverage rates for the servicemembers 
surveyed, and how the survey respondents differed from nonrespondents. 
[Footnote 37] 

With respect to timeliness, we find generally similar trends for cases 
grouped by fiscal year of enrollment versus fiscal year of completion, 
with some key differences. Organizing cases by completion date results 
in shorter average processing times in 2008, since only those cases 
that are processed quickly could be completed in the first year of 
IDES. As such, organizing cases by enrollment date provides a better 
estimate of the processing times for the early IDES cases. However, 
this approach results in shorter processing times in 2011, the most 
recent full year of the program, since only cases that finish quickly 
can be analyzed. 

With respect to satisfaction, we find that the particular index used 
to summarize servicemembers' responses can affect the proportion 
reported as being "satisfied" or "dissatisfied" with IDES overall. 
DOD's index suggests that 67 percent of servicemembers have been 
satisfied since the IDES program began, but a reasonable alternative 
measure we developed suggests that only 24 percent of servicemembers 
have been satisfied. Using this measure, satisfaction varies only 
slightly across many important groups of servicemembers, such as by 
disenrollment outcome, suggesting that available program data cannot 
precisely explain satisfaction outcomes. Also, servicemembers surveyed 
may not represent the servicemembers who completed the different 
phases of IDES well enough to generalize to them, given the low 
response rates to the MEB survey and fact that being selected for 
latter (PEB and Transition) surveys were conditional on completing the 
MEB survey. 

Overall IDES Timeliness: Processing Times and Percent of Cases Meeting 
Timeliness Goals: 

Average IDES processing times for completed cases resulting in 
benefits generally worsened since 2008, especially for active duty 
cases, regardless of whether cases are grouped by the fiscal year in 
which they were completed (figure 10) or by the fiscal year in which 
they were enrolled (figure 11). The notable exception is when fiscal 
year 2011 is the year of enrollment. However, caution must be used 
when examining cases enrolled in 2011 because over 15,600 service 
members of the 18,651 (or at least 84 percent) who entered IDES in 
fiscal year 2011 did not have an outcome in 2011 and were enrolled in 
IDES as of January 1, 2012,[Footnote 38] potentially changing the 
distribution of processing times as they proceed through IDES. 

Figure 10: Average IDES Processing Times by Year of Completion, for 
Completed Cases Resulting in Benefits: 

[Refer to PDF for image: horizontal bar graph]  

Total time in IDES:  

Reserves/National Guard goal: 305; 
Active forces goal: 295.  

FY 2008: 
Completed cases: 154; 
Reserves/National Guard: 297; 
Active forces: 283.  

FY 2009: 
Completed cases: 956; 
Reserves/National Guard: 316; 
Active forces: 313.  

FY 2010: 
Completed cases: 3,878; 
Reserves/National Guard: 370; 
Active forces: 357.  

FY 2011: 
Completed cases: 6,312; 
Reserves/National Guard: 420; 
Active forces: 394.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

Figure 11: Average IDES Processing Times by Fiscal Year of Enrollment 
for Completed Cases Resulting in Benefits: 

[Refer to PDF for image: horizontal bar graph]  

Total time in IDES:  

Reserves/National Guard goal: 305; 
Active forces goal: 295.  

FY 2008: 
Cases enrolled: 481; 
Reserves/National Guard: 381; 
Active forces: 367.  

FY 2009: 
Cases enrolled: 3,066; 
Reserves/National Guard: 398; 
Active forces: 378.  

FY 2010: 
Cases enrolled: 6,108; 
Reserves/National Guard: 398; 
Active forces: 387.  

FY 2011: 
Cases enrolled: 1,372; 
Reserves/National Guard: 301; 
Active forces: 301.  

Source: GAO analysis of DOD and VA data.  

Note: Enrolled cases reflects those cases that were enrolled for each 
year with cases with erroneous or questionable dates removed. This 
number does not match the number of enrolled cases previously reported 
for this reason.  

[End of figure]  

We also examine average IDES processing times according to year of 
completion (see figure 12) and year of enrollment for cases (see 
figure 13) for all completed cases regardless of outcome. As with 
cases that resulted in benefits, for cases resulting in any outcome we 
find that average processing times increased since 2008--again with 
the exception of fiscal year 2011 for reasons discussed earlier--
although average processing times are somewhat shorter than when only 
servicemembers receiving benefits are included (figure 11). 

Figure 12: Average IDES Processing Times by Year of Completion, for 
All Completed Cases Regardless of Outcome: 

[Refer to PDF for image: horizontal bar graph]  

Total time in IDES:  

Reserves/National Guard goal: 305; 
Active forces goal: 295.  

FY 2008: 
Completed cases: 210; 
Reserves/National Guard: 263; 
Active forces: 246.  

FY 2009: 
Completed cases: 1,183; 
Reserves/National Guard: 295; 
Active forces: 288.  

FY 2010: 
Completed cases: 4,448; 
Reserves/National Guard: 341; 
Active forces: 338.  

FY 2011: 
Completed cases: 7,106; 
Reserves/National Guard: 390; 
Active forces: 378.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

Figure 13: Average IDES Processing Times by Fiscal Year of Enrollment 
for All Completed Cases Regardless of Outcome: 

Refer to PDF for image: horizontal bar graph]  

Total time in IDES:  

Reserves/National Guard goal: 305; 
Active forces goal: 295.  

FY 2008: 
Cases enrolled: 602; 
Reserves/National Guard: 357; 
Active forces: 339.  

FY 2009: 
Cases enrolled: 3,506; 
Reserves/National Guard: 365; 
Active forces: 358.  

FY 2010: 
Cases enrolled: 6,858; 
Reserves/National Guard: 370; 
Active forces: 372.  

FY 2011: 
Cases enrolled: 1,851; 
Reserves/National Guard: 253; 
Active forces: 268.  

Source: GAO analysis of DOD and VA data.  

Note: Enrolled cases reflects those cases that were enrolled for each 
year with cases with erroneous or questionable dates removed. This 
number does not match the number of enrolled cases previously reported 
for this reason.  

[End of figure]  

Figures 14 and 15 show that regardless of whether cases are organized 
by year of completion or enrollment, the percent of completed cases 
resulting in benefits that were not timely increased between fiscal 
year 2008 and 2010 for both active duty servicemembers and members in 
the Reserves or National Guard. As with the average processing times, 
caution must be used when examining cases enrolled in fiscal year 2011 
(figure 15), since only those cases that are processed quickly are 
observed in the last year. Similarly, caution also must be used when 
examining cases in 2008 (figure 14), since the only cases that are 
included in the first year are those that completed IDES quickly. 

Figure 14: Percentage of Completed Cases Meeting and Missing the 
Overall Processing Time Goals, by Fiscal Year of IDES Completion, for 
Servicemembers Receiving Benefits: 

[Refer to PDF for image: horizontal bar graph]  

Percentage of IDES cases:  

Active forces:  

Fiscal year 2008: 
Not meeting service goals: 37%; 
Meeting service goals: 63%.  

Fiscal year 2009: 
Not meeting service goals: 50%; 
Meeting service goals: 50%.  

Fiscal year 2010: 
Not meeting service goals: 68%; 
Meeting service goals: 32%.  

Fiscal year 2011: 
Not meeting service goals: 81%; 
Meeting service goals: 19%.  

Reserves/National Guard:  

Fiscal year 2008: 
Not meeting service goals: 35%; 
Meeting service goals: 65%.  

Fiscal year 2009: 
Not meeting service goals: 48%; 
Meeting service goals: 52%.  

Fiscal year 2010: 
Not meeting service goals: 63%; 
Meeting service goals: 37%.  

Fiscal year 2011: 
Not meeting service goals: 82%; 
Meeting service goals: 18%.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

Figure 15: Percentage of Completed Cases Meeting and Missing the 
Overall Processing Time Goals, by Fiscal Year of IDES Enrollment, for 
Servicemembers Receiving Benefits: 

[Refer to PDF for image: horizontal bar graph]  

Percentage of IDES cases:  

Active forces:  

Fiscal year 2008: 
Not meeting service goals: 66%; 
Meeting service goals: 34%.  

Fiscal year 2009: 
Not meeting service goals: 69%; 
Meeting service goals: 31%.  

Fiscal year 2010: 
Not meeting service goals: 81%; 
Meeting service goals: 19%.  

Fiscal year 2011: 
Not meeting service goals: 54%; 
Meeting service goals: 46%.  

Reserves/National Guard:  

Fiscal year 2008: 
Not meeting service goals: 64%; 
Meeting service goals: 37%.  

Fiscal year 2009: 
Not meeting service goals: 67%; 
Meeting service goals: 33%.  

Fiscal year 2010: 
Not meeting service goals: 77%; 
Meeting service goals: 23%.  

Fiscal year 2011: 
Not meeting service goals: 46%; 
Meeting service goals: 54%.  

Source: GAO analysis of DOD and VA data.  

[End of figure] 

Figures 16 and 17 show how average processing times for each of the 
four phases of IDES have changed over the four fiscal years when cases 
are grouped by the fiscal year in which they completed a given phase 
and when cases are grouped by the fiscal year in which they were 
enrolled or started a given phase.[Footnote 39] Figure 16 shows that 
when cases are grouped according to the fiscal year in which the 
different phases were completed, processing times increased for all 
phases except the Transition phase. Figure 17 shows a roughly similar 
pattern of increases in processing times in all but the Transition 
phase, though processing times in 2011 are skewed for the reason 
mentioned above. Figures 18 and 19 show how the percentages meeting 
the timeliness goals for each of the four phases of IDES have changed 
over the four fiscal years when cases are grouped by the fiscal year 
in which they completed a given phase and when cases are grouped by 
the fiscal year in which they were enrolled or started a given phase. 
Figure 18 shows that the percent of cases meeting timeliness goals 
decreased over the four years for the MEB and PEB phases, although a 
high percent of cases met PEB goals. However, the Transition and 
Benefits phases fluctuated up and down and both were favorable across 
some years. Figure 19 also shows decreases in percentages of cases 
meeting timeliness goals at the MEB and PEB phases when cases are 
grouped by fiscal year of starting a given phase. The fluctuations in 
the timeliness of the Transition and Benefits phases were more 
prevalent when cases were grouped in this manner. 

Figure 16: Average Processing Time for Each IDES Phase, by Fiscal Year 
in Which the Phase Was Completed: 

[Refer to PDF for image: 4 vertical bar graphs]  

MEB phase:  

Phase goal for Reserves/National Guard: 100; 
Phase goal for Active forces: 140.  

Fiscal year: 2008; 
Reserves/National Guard: 128; 
Active forces: 114.  

Fiscal year: 2009; 
Reserves/National Guard: 153; 
Active forces: 126.  

Fiscal year: 2010; 
Reserves/National Guard: 158; 
Active forces: 141.  

Fiscal year: 2011; 
Reserves/National Guard: 188; 
Active forces: 181.  

PEB phase:  

Phase goal for Reserves/National Guard: 120; 
Phase goal for Active forces: 120.  

Fiscal year: 2008; 
Reserves/National Guard: 41; 
Active forces: 42.  

Fiscal year: 2009; 
Reserves/National Guard: 60; 
Active forces: 58.  

Fiscal year: 2010; 
Reserves/National Guard: 83; 
Active forces: 80.  

Fiscal year: 2011; 
Reserves/National Guard: 116; 
Active forces: 93.  

Transition phase:  

Phase goal for Reserves/National Guard: 45; 
Phase goal for Active forces: 45.  

Fiscal year: 2008; 
Reserves/National Guard: 81; 
Active forces: 88.  

Fiscal year: 2009; 
Reserves/National Guard: 73; 
Active forces: 80.  

Fiscal year: 2010; 
Reserves/National Guard: 82; 
Active forces: 79.  

Fiscal year: 2011; 
Reserves/National Guard: 75; 
Active forces: 76.  

Benefits phase:  

Phase goal for Active forces: 30.  

Fiscal year: 2008; 
Reserves/National Guard: N/A; 
Active forces: 29.  

Fiscal year: 2009; 
Reserves/National Guard: N/A; 
Active forces: 43.  

Fiscal year: 2010; 
Reserves/National Guard: N/A; 
Active forces: 48.  

Fiscal year: 2011; 
Reserves/National Guard: N/A; 
Active forces: 38.  

Source: GAO analysis of DOD and VA data.  

Note: Data shown in figures 16, 17, 18, and 19 are for servicemember 
cases that either started or completed a phase in a particular year, 
according to the figure title. For purposes of this report, we opted 
to not include reserve component time spent in the VA benefit phase in 
our calculations phase because this goal applies to some but not all 
reservists, depending on their active duty status. Any time spent 
within the VA benefit phase is reflected within the overall processing 
time calculations for such reserve component servicemembers.  

[End of figure]  

Figure 17: Average Processing Time for Each IDES Phase, by Fiscal Year 
of Enrollment in Each Phase: 

[Refer to PDF for image: 4 vertical bar graphs]  

MEB phase:  

Phase goal for Reserves/National Guard: 100; 
Phase goal for Active forces: 140.  

Fiscal year: 2008; 
Reserves/National Guard: 185; 
Active forces: 160.  

Fiscal year: 2009; 
Reserves/National Guard: 174; 
Active forces: 160.  

Fiscal year: 2010; 
Reserves/National Guard: 206; 
Active forces: 193.  

Fiscal year: 2011; 
Reserves/National Guard: 143; 
Active forces: 132.  

PEB phase:  

Phase goal for Reserves/National Guard: 120; 
Phase goal for Active forces: 120.  

Fiscal year: 2008; 
Reserves/National Guard: 66; 
Active forces: 65.  

Fiscal year: 2009; 
Reserves/National Guard: 87; 
Active forces: 82.  

Fiscal year: 2010; 
Reserves/National Guard: 116; 
Active forces: 89.  

Fiscal year: 2011; 
Reserves/National Guard: 118; 
Active forces: 91.  

Transition phase:  

Phase goal for Reserves/National Guard: 45; 
Phase goal for Active forces: 45.  

Fiscal year: 2008; 
Reserves/National Guard: 81; 
Active forces: 88.  

Fiscal year: 2009; 
Reserves/National Guard: 76; 
Active forces: 81.  

Fiscal year: 2010; 
Reserves/National Guard: 84; 
Active forces: 82.  

Fiscal year: 2011; 
Reserves/National Guard: 76; 
Active forces: 76.  

Benefits phase:  

Phase goal for Active forces: 30.  

Fiscal year: 2008; 
Reserves/National Guard: N/A; 
Active forces: 27.  

Fiscal year: 2009; 
Reserves/National Guard: N/A; 
Active forces: 42.  

Fiscal year: 2010; 
Reserves/National Guard: N/A; 
Active forces: 44.  

Fiscal year: 2011; 
Reserves/National Guard: N/A; 
Active forces: 43.  

Source: GAO analysis of DOD and VA data.  

Note: Data shown in figures 16, 17, 18, and 19 are for servicemember 
cases that either started or completed a phase in a particular year, 
according to the figure title. For purposes of this report, we opted 
to not include reserve component time spent in the VA benefit phase in 
our calculations phase because this goal applies to some but not all 
reservists, depending on their active duty status. Any time spent 
within the VA benefit phase is reflected within the overall processing 
time calculations for such reserve component servicemembers.  

[End of figure]  

Figure 18: Percent of Cases Meeting Timeliness Goals for each Phase of 
IDES, by Fiscal Year in Which the Phase Was Completed: 

[Refer to PDF for image: 4 vertical bar graphs]  

MEB phase:  

Fiscal year: 2008; 
Reserves/National Guard: 69%; 
Active forces: 40%.  

Fiscal year: 2009; 
Reserves/National Guard: 46%; 
Active forces: 41%.  

Fiscal year: 2010; 
Reserves/National Guard: 50%; 
Active forces: 34%.  

Fiscal year: 2011; 
Reserves/National Guard: 33%; 
Active forces: 18%.  

PEB phase:  

Fiscal year: 2008; 
Reserves/National Guard: 92%; 
Active forces: 97%.  

Fiscal year: 2009; 
Reserves/National Guard: 92%; 
Active forces: 90%.  

Fiscal year: 2010; 
Reserves/National Guard: 79%; 
Active forces: 81%.  

Fiscal year: 2011; 
Reserves/National Guard: 62%; 
Active forces: 78%.  

Transition phase:  

Fiscal year: 2008; 
Reserves/National Guard: 13%; 
Active forces: 10%.  

Fiscal year: 2009; 
Reserves/National Guard: 29%; 
Active forces: 17%.  

Fiscal year: 2010; 
Reserves/National Guard: 34%; 
Active forces: 19%.  

Fiscal year: 2011; 
Reserves/National Guard: 25%; 
Active forces: 17%.  

Benefits phase:  

Fiscal year: 2008; 
Reserves/National Guard: N/A; 
Active forces: 64%.  

Fiscal year: 2009; 
Reserves/National Guard: N/A; 
Active forces: 52%.  

Fiscal year: 2010; 
Reserves/National Guard: N/A; 
Active forces: 46%.  

Fiscal year: 2011; 
Reserves/National Guard: N/A; 
Active forces: 54%.  

Source: GAO analysis of DOD and VA data.  

Note: Data shown in figures 16, 17, 18, and 19 are for servicemember 
cases that either started or completed a phase in a particular year, 
according to the figure title. For purposes of this report, we opted 
to not include reserve component time spent in the VA benefit phase in 
our calculations phase because this goal applies to some but not all 
reservists, depending on their active duty status. Any time spent 
within the VA benefit phase is reflected within the overall processing 
time calculations for such reserve component servicemembers.  

[End of figure]  

Figure 19: Percent of Cases Meeting Timeliness Goals for each Phase of 
IDES, by Fiscal Year of Enrollment in Each Phase: 

[Refer to PDF for image: 4 vertical bar graphs]  

MEB phase:  

Fiscal year: 2008; 
Reserves/National Guard: 35%; 
Active forces: 21%.  

Fiscal year: 2009; 
Reserves/National Guard: 46%; 
Active forces: 31%.  

Fiscal year: 2010; 
Reserves/National Guard: 28%; 
Active forces: 17%.  

Fiscal year: 2011; 
Reserves/National Guard: 56%; 
Active forces: 38%.  

PEB phase:  

Fiscal year: 2008; 
Reserves/National Guard: 78%; 
Active forces: 89%.  

Fiscal year: 2009; 
Reserves/National Guard: 80%; 
Active forces: 79%.  

Fiscal year: 2010; 
Reserves/National Guard: 65%; 
Active forces: 76%.  

Fiscal year: 2011; 
Reserves/National Guard: 56%; 
Active forces: 79%.  

Transition phase:  

Fiscal year: 2008; 
Reserves/National Guard: 13%; 
Active forces: 10%.  

Fiscal year: 2009; 
Reserves/National Guard: 28%; 
Active forces: 17%.  

Fiscal year: 2010; 
Reserves/National Guard: 33%; 
Active forces: 18%.  

Fiscal year: 2011; 
Reserves/National Guard: 24%; 
Active forces: 17%.  

Benefits phase:  

Fiscal year: 2008; 
Reserves/National Guard: N/A; 
Active forces: 75%.  

Fiscal year: 2009; 
Reserves/National Guard: N/A; 
Active forces: 51%.  

Fiscal year: 2010; 
Reserves/National Guard: N/A; 
Active forces: 50%.  

Fiscal year: 2011; 
Reserves/National Guard: N/A; 
Active forces: 49%.  

Source: GAO analysis of DOD and VA data.  

Note: Data shown in figures 16, 17, 18, and 19 are for servicemember 
cases that either started or completed a phase in a particular year, 
according to the figure title. For purposes of this report, we opted 
to not include reserve component time spent in the VA benefit phase in 
our calculations phase because this goal applies to some but not all 
reservists, depending on their active duty status. Any time spent 
within the VA benefit phase is reflected within the overall processing 
time calculations for such reserve component servicemembers.  

[End of figure]  

Servicemember Satisfaction Survey: Response and Coverage Rates: 

Low response and coverage rates for servicemember satisfaction surveys 
administered after each phase of IDES raise concerns about how well 
the satisfaction survey results represented the larger population of 
servicemembers who completed one or more phases. 

DOD surveys servicemembers after they complete the MEB, PEB, and 
Transition phases of IDES. The department attempts to survey all 
servicemembers who complete each phase, but only if they completed the 
prior surveys. For example, the MEB survey must be completed before a 
servicemember is eligible to complete the PEB survey. 

Using the data available to us, and as table 8 below shows, we found 
that 9,604 of the 25,212 servicemembers who completed the MEB phase 
were surveyed, for a 38 percent response and coverage rate.[Footnote 
40] Of the 18,296 servicemembers who completed the PEB phase, only 
8,968 of them completed the prior MEB survey and were eligible for the 
PEB survey and of these only 4,795 were surveyed. Using DOD's 
eligibility criteria, the response rate for the PEB survey was roughly 
54 percent (4,795 of 8,968). However, the coverage rate for all 
servicemembers who completed the PEB phase (regardless of whether they 
completed the prior survey) was only 26 percent (4,795 of 18,296). 
Similarly, the response rate for the Transition survey was 72 percent 
while the coverage rate was only 23 percent (See table 8). 

Table 8: Survey phase response and coverage rates: 

Phase: MEB; 
Servicemembers completing phase: 25,212; 
Servicemembers eligible for survey (completed prior phase survey): all; 
Servicemembers surveyed: 9,604; 
Response rate (No. surveyed/No. eligible) (%): 38.1%; 
Coverage rate (No. surveyed/No. completing phase) (%): 38.1%. 

Phase: PEB; 
Servicemembers completing phase: 18,296; 
Servicemembers eligible for survey (completed prior phase survey): 
8,968; 
Servicemembers surveyed: 4,795; 
Response rate (No. surveyed/No. eligible) (%): 53.5%; 
Coverage rate (No. surveyed/No. completing phase) (%): 26.2%. 

Phase: Transition; 
Servicemembers completing phase: 12,352; 
Servicemembers eligible for survey (completed prior phase survey): 
3,996; 
Servicemembers surveyed: 2,893; 
Response rate (No. surveyed/No. eligible) (%): 72.4%; 
Coverage rate (No. surveyed/No. completing phase) (%): 23.4%. 

Source: GAO analysis of DOD and VA data. 

Note: This table excludes 558 cases (or approximately 1.4 percent) of 
VTA data with logical errors in IDES phase start and end days for any 
IDES phase (e.g. exclude cases where MEB start date occurs after MEB 
end date). This table includes the approximately 4 percent of cases we 
found to have inconsistent start and end dates across phases.  

[End of table]  

As table 9 below shows, there were some sizable differences between 
respondents and nonrespondents, especially for the PEB and Transition 
surveys. For example, respondents to the transition survey spent more 
time than nonrespondents in the Transition phase, were less likely to 
be separated with benefits, and were more likely to be placed on the 
Permanent Disability Retired List. These differences, combined with 
low response and coverage rates, raise the possibility of biased 
responses. 

Table 9: Selected Servicemember Characteristics of Respondents and 
Nonrespondents to the MEB, PEB and Transition Surveys: 

Survey: MEB survey;  

Variable: Timeliness;  

Total days in MEB; 
Respondents: N: 9,604; 
Respondents: Mean or percent: 158.8; 
Nonrespondents: N: 15,608; 
Nonrespondents: Mean or percent: 156.6. 

Met MEB goal; 
Respondents: N: 2,854; 
Respondents: Mean or percent: 29.7%; 
Nonrespondents: N: 5,187; 
Nonrespondents: Mean or percent: 33.2%. 

Variable: Branch;  

Army; 
Respondents: N: 5,191; 
Respondents: Mean or percent: 54.1%; 
Nonrespondents: N: 8,910; 
Nonrespondents: Mean or percent: 57.1%. 

Marine Corps; 
Respondents: N: 2,177; 
Respondents: Mean or percent: 22.7%; 
Nonrespondents: N: 2,802; 
Nonrespondents: Mean or percent: 18.0%. 

Navy; 
Respondents: N: 1,417; 
Respondents: Mean or percent: 14.8%; 
Nonrespondents: N: 2,168; 
Nonrespondents: Mean or percent: 13.9%. 

US Air Force; 
Respondents: N: 819; 
Respondents: Mean or percent: 8.5%; 
Nonrespondents: N: 1,728; 
Nonrespondents: Mean or percent: 11.1%. 

Variable: Component;  

Active duty; 
Respondents: N: 8,628; 
Respondents: Mean or percent: 89.8%; 
Nonrespondents: N: 14,111; 
Nonrespondents: Mean or percent: 90.4%. 

Reserve/National Guard; 
Respondents: N: 976; 
Respondents: Mean or percent: 10.2%; 
Nonrespondents: N: 1,497; 
Nonrespondents: Mean or percent: 9.6%. 

Variable: Outcome of case;  

Separated with benefits; 
Respondents: N: 2,778; 
Respondents: Mean or percent: 31.4%; 
Nonrespondents: N: 2,678; 
Nonrespondents: Mean or percent: 31.0%. 

Separated without benefits; 
Respondents: N: 152; 
Respondents: Mean or percent: 1.7%; 
Nonrespondents: N: 134; 
Nonrespondents: Mean or percent: 1.6%. 

Permanent disability retirement list; 
Respondents: N: 2,191; 
Respondents: Mean or percent: 24.7%; 
Nonrespondents: N: 2,192; 
Nonrespondents: Mean or percent: 25.3%. 

Temporary disability retirement list; 
Respondents: N: 2,704; 
Respondents: Mean or percent: 30.5%; 
Nonrespondents: N: 2,730; 
Nonrespondents: Mean or percent: 31.6%. 

Found fit and returned to duty; 
Respondents: N: 959; 
Respondents: Mean or percent: 10.8%; 
Nonrespondents: N: 856; 
Nonrespondents: Mean or percent: 9.9%. 

Found unfit and returned to duty; 
Respondents: N: 73; 
Respondents: Mean or percent: 0.8%; 
Nonrespondents: N: 61; 
Nonrespondents: Mean or percent: 0.7%. 

Survey: PEB Survey;  

Variable: Timeliness;  

Total days in PEB; 
Respondents: N: 4,764; 
Respondents: Mean or percent: 85.0; 
Nonrespondents: N: 13,364; 
Nonrespondents: Mean or percent: 91.2. 

Met PEB goal; 
Respondents: N: 3,762; 
Respondents: Mean or percent: 79.0%; 
Nonrespondents: N: 10,424; 
Nonrespondents: Mean or percent: 78.0%. 

Variable: Branch;  

Army; 
Respondents: N: 2,646; 
Respondents: Mean or percent: 55.5%; 
Nonrespondents: N: 8,298; 
Nonrespondents: Mean or percent: 62.1%. 

Marine Corps; 
Respondents: N: 1,061; 
Respondents: Mean or percent: 22.3%; 
Nonrespondents: N: 2,347; 
Nonrespondents: Mean or percent: 17.6%. 

Navy; 
Respondents: N: 708; 
Respondents: Mean or percent: 14.9%; 
Nonrespondents: N: 1,640; 
Nonrespondents: Mean or percent: 12.3%. 

US Air Force; 
Respondents: N: 349; 
Respondents: Mean or percent: 7.3%; 
Nonrespondents: N: 1,079; 
Nonrespondents: Mean or percent: 8.1%. 

Variable: Component;  

Active duty; 
Respondents: N: 4,262; 
Respondents: Mean or percent: 89.5%; 
Nonrespondents: N: 12,228; 
Nonrespondents: Mean or percent: 91.5%. 

Reserve/National Guard; 
Respondents: N: 502; 
Respondents: Mean or percent: 10.54; 
Nonrespondents: N: 1,136; 
Nonrespondents: Mean or percent: 8.50. 

Variable: Outcome of case;  

Separated with benefits; 
Respondents: N: 1,408; 
Respondents: Mean or percent: 29.6%; 
Nonrespondents: N: 4,035; 
Nonrespondents: Mean or percent: 31.9%. 

Separated without benefits; 
Respondents: N: 72; 
Respondents: Mean or percent: 1.51; 
Nonrespondents: N: 211; 
Nonrespondents: Mean or percent: 1.7%. 

Permanent disability retirement list; 
Respondents: N: 1,222; 
Respondents: Mean or percent: 25.7%; 
Nonrespondents: N: 3,147; 
Nonrespondents: Mean or percent: 24.9%. 

Temporary disability retirement list; 
Respondents: N: 1,440; 
Respondents: Mean or percent: 30.2%; 
Nonrespondents: N: 3,962; 
Nonrespondents: Mean or percent: 31.4%. 

Found fit and returned to duty; 
Respondents: N: 584; 
Respondents: Mean or percent: 12.3%; 
Nonrespondents: N: 1,181; 
Nonrespondents: Mean or percent: 9.4%. 

Found unfit and returned to duty; 
Respondents: N: 38; 
Respondents: Mean or percent: 0.8%; 
Nonrespondents: N: 94; 
Nonrespondents: Mean or percent: 0.7%. 

Survey: Transition survey;  

Variable: Timeliness;  

Total days in transition; 
Respondents: N: 2,894; 
Respondents: Mean or percent: 82.3; 
Nonrespondents: N: 9,259; 
Nonrespondents: Mean or percent: 75.1. 

Met transition goal; 
Respondents: N: 455; 
Respondents: Mean or percent: 15.7%; 
Nonrespondents: N: 1,856; 
Nonrespondents: Mean or percent: 20.0%. 

Variable: Branch;  

Army; 
Respondents: N: 1,744; 
Respondents: Mean or percent: 60.3%; 
Nonrespondents: N: 5,972; 
Nonrespondents: Mean or percent: 64.5%. 

Marine Corps; 
Respondents: N: 597; 
Respondents: Mean or percent: 20.6%; 
Nonrespondents: N: 1,837; 
Nonrespondents: Mean or percent: 19.8%. 

Navy; 
Respondents: N: 344; 
Respondents: Mean or percent: 11.9%; 
Nonrespondents: N: 924; 
Nonrespondents: Mean or percent: 10.0%. 

US Air Force; 
Respondents: N: 209; 
Respondents: Mean or percent: 7.2%; 
Nonrespondents: N: 526; 
Nonrespondents: Mean or percent: 5.7%. 

Variable: Component;  

Active Duty; 
Respondents: N: 2,587; 
Respondents: Mean or percent: 89.4%; 
Nonrespondents: N: 8,562; 
Nonrespondents: Mean or percent: 92.5%. 

Reserve/National Guard; 
Respondents: N: 307; 
Respondents: Mean or percent: 10.6%; 
Nonrespondents: N: 697; 
Nonrespondents: Mean or percent: 7.5%. 

Variable: Outcome of Case;  

Variable: Separated with benefits; 
Respondents: N: 783; 
Respondents: Mean or percent: 27.1%; 
[Empty]; 
Nonrespondents: N: 3,496; 
Nonrespondents: Mean or percent: 38.1%. 

Separated without benefits; 
Respondents: N: 47; 
Respondents: Mean or percent: 1.6%; 
Nonrespondents: N: 173; 
Nonrespondents: Mean or percent: 1.9%. 

Permanent disability retirement list; 
Respondents: N: 955; 
Respondents: Mean or percent: 33.0%; 
Nonrespondents: N: 2,405; 
Nonrespondents: Mean or percent: 26.2%. 

Temporary disability retirement list; 
Respondents: N: 1,102; 
Respondents: Mean or percent: 38.1%; 
Nonrespondents: N: 3,086; 
Nonrespondents: Mean or percent: 33.7%. 

Found fit and returned to duty; 
Respondents: N: 5; 
Respondents: Mean or percent: 0.2%; 
Nonrespondents: N: 4; 
Nonrespondents: Mean or percent: 0.0%. 

Found unfit and returned to duty; 
Respondents: N: 2; 
Respondents: Mean or percent: 0.1%; 
Nonrespondents: N: 3; 
Nonrespondents: Mean or percent: 0.0%. 

Source: GAO analysis of DOD and VA data. 

Note: This table excludes 558 cases (or approximately 1.4 percent) of 
VTA data with logical errors in IDES phase start and end days, for any 
IDES phase (e.g. exclude cases where MEB start date occurs after MEB 
end date). This table includes the approximately 4 percent of cases we 
found to have inconsistent start and end dates across phases.  

[End of table]  

Measuring Servicemember Satisfaction: 

The particular measure used to assess servicemember satisfaction can 
affect the proportion reported as "satisfied" with the IDES program. 
Depending on the measure used, satisfaction is about 2.8 times lower 
than what DOD has reported, and many servicemembers classified as 
"satisfied" express moderate dissatisfaction with some aspects of the 
process. 

DOD has reported average servicemember satisfaction with IDES overall 
and with three phases of the process, i.e., MEB, PEB, and Transition 
phases. In so doing, DOD has developed indices of satisfaction on 
several broad dimensions, such as satisfaction with the overall 
experience and fairness, which combine responses to selected survey 
questions. Although the number of questions used in each index vary 
depending on the number of phases completed, each index classifies 
servicemembers as "satisfied" or "dissatisfied" using the average of 
their responses across all questions in the index. Each question's 
scale ranges from 1 to 5, with 1 denoting "very dissatisfied" (or a 
similar negative response), 5 denoting "very satisfied," and 3 
denoting "neither satisfied nor dissatisfied." DOD reports that a 
servicemember is "satisfied" if his or her average response across all 
items in the scale exceeds 3. Table 10 summarizes the responses to 
each question that DOD uses in its overall satisfaction index at each 
phase (as of August 2011). 

Table 10: Questions Used in DOD Indices of Overall Servicemember 
Satisfaction: 

How satisfied or dissatisfied were you with the management of your 
case during the [MEB/PEB/Transition] phase of the pilot process?  

Very dissatisfied; 
MEB phase: Number: 813; 
MEB phase: Percent: 8.4; 
PEB phase: Number: 388; 
PEB phase: Percent: 7.9; 
Transition phase: Number: 223; 
Transition phase: Percent: 6.4. 

Dissatisfied; 
MEB phase: Number: 1,369; 
MEB phase: Percent: 14.1; 
PEB phase: Number: 607; 
PEB phase: Percent: 12.4; 
Transition phase: Number: 406; 
Transition phase: Percent: 11.6. 

Neither; 
MEB phase: Number: 1,259; 
MEB phase: 13; 
PEB phase: Number: 578; 
PEB phase: Percent: 11.8; 
Transition phase: Number: 397; 
Transition phase: Percent: 11.3. 

Satisfied; 
MEB phase: Number: 4,310; 
MEB phase: Percent: 44.5; 
PEB phase: Number: 2,306; 
PEB phase: Percent: 47.1; 
Transition phase: Number: 1,737; 
Transition phase: Percent: 49.6. 

Very satisfied; 
MEB phase: Number: 1,926; 
MEB phase: Percent: 19.9; 
PEB phase: Number: 1,019; 
PEB phase: Percent: 20.8; 
Transition phase: Number: 741; 
Transition phase: Percent: 21.1. 

Total; 
MEB phase: Number: 9,677; 
MEB phase: Percent: 100; 
PEB phase: Number: 4,898; 
PEB phase: Percent: 100; 
Transition phase: Number: 3,504; 
Transition phase: Percent: 100. 

How satisfied or dissatisfied were you with the overall [MEB/PEB/ 
Transition] phase of determining your retention status?  

Very dissatisfied; 
MEB phase: Number: 559; 
MEB phase: Percent: 6.7; 
PEB phase: Number: 269; 
PEB phase: Percent: 5.5; 
Transition phase: Number: 224; 
Transition phase: Percent: 6.4. 

Dissatisfied; 
MEB phase: Number: 1,131; 
MEB phase: Percent: 13.6; 
PEB phase: Number: 548; 
PEB phase: Percent: 11.2; 
[Empty]; 
Transition phase: Number: 401; 
Transition phase: Percent: 11.5. 

Neither; 
MEB phase: Number: 1,007; 
MEB phase: Percent: 12.1; 
PEB phase: Number: 510; 
PEB phase: Percent: 10.4; 
Transition phase: Number: 345; 
Transition phase: Percent: 9.9. 

Satisfied; 
MEB phase: Number: 4,048; 
MEB phase: Percent: 48.6; 
PEB phase: Number: 2,562; 
PEB phase: Percent: 52.3; 
Transition phase: Number: 1,798; 
Transition phase: Percent: 51.6. 

Very satisfied; 
MEB phase: Number: 1,578; 
MEB phase: Percent: 19; 
PEB phase: Number: 1014; 
PEB phase: Percent: 20.7; 
Transition phase: Number: 715; 
Transition phase: Percent: 20.5. 

Total; 
MEB phase: Number: 8,323; 
MEB phase: Percent: 100; 
PEB phase: Number: 4,903; 
PEB phase: Percent: 100; 
Transition phase: Number: 3,483; 
Transition phase: Percent: 100. 

How would you evaluate the timeliness of the process since entering 
the Disability Evaluation Pilot process?  

Very poor; 
MEB phase: Number: 1,685; 
MEB phase: Percent: 17.4; 
PEB phase: Number: 759; 
PEB phase: Percent: 15.4; 
Transition phase: Number: 500; 
Transition phase: Percent: 14.1. 

Poor; 
MEB phase: Number: 1,594; 
MEB phase: Percent: 16.5; 
PEB phase: Number: 681; 
PEB phase: Percent: 13.9; 
Transition phase: 468; 
Transition phase: Percent: 13.2. 

Mix of poor/good; 
MEB phase: Number: 2,397; 
MEB phase: Percent: 24.7; 
PEB phase: Number: 1,262; 
PEB phase: Percent: 25.7; 
Transition phase: Number: 967; 
Transition phase: Percent: 27.3. 

Good; 
MEB phase: Number: 2,811; 
MEB phase: Percent: 29; 
PEB phase: Number: 1,470; 
PEB phase: Percent: 29.9; 
Transition phase: Number: 1,065; 
Transition phase: Percent: 30.1. 

Very good; 
MEB phase: Number: 1,200; 
MEB phase: Percent: 12.4; 
PEB phase: Number: 743; 
PEB phase: Percent: 15.1; 
Transition phase: Number: 543; 
Transition phase: Percent: 15.3. 

Total; 
MEB phase: Number: 9,687; 
MEB phase: Percent: 100; 
PEB phase: Number: 4,915; 
PEB phase: Percent: 100; 
Transition phase: Number: 3,543; 
Transition phase: Percent: 100. 

How would you evaluate your overall experience since entering the 
Disability Evaluation Pilot process?  

Very poor; 
MEB phase: Number: 686; 
MEB phase: Percent: 7; 
PEB phase: Number: 356; 
PEB phase: Percent: 7.2; 
Transition phase: Number: 235; 
Transition phase: Percent: 6.6. 

Poor; 
MEB phase: Number: 921; 
MEB phase: Percent: 9.4; 
PEB phase: Number: 413; 
PEB phase: Percent: 8.4; 
Transition phase: Number: 285; 
Transition phase: Percent: 8. 

Mix of poor/good; 
MEB phase: Number: 2,875; 
MEB phase: Percent: 29.4; 
PEB phase: Number: 1,477; 
PEB phase: Percent: 29.9; 
Transition phase: Number: 1,115; 
Transition phase: Percent: 31.4. 

Good; 
MEB phase: Number: 3,891; 
MEB phase: Percent: 39.8; 
PEB phase: Number: 1,917; 
PEB phase: Percent: 38.8; 
Transition phase: Number: 1,333; 
Transition phase: Percent: 37.5. 

Very good; 
MEB phase: Number: 1,402; 
MEB phase: Percent: 14.3; 
PEB phase: Number: 772; 
PEB phase: Percent: 15.6; 
Transition phase: Number: 587; 
Transition phase: Percent: 16.5. 

Total; 
MEB phase: Number: 9,775; 
MEB phase: Percent: 100; 
PEB phase: Number: 4,935; 
PEB phase: Percent: 100; 
Transition phase: Number: 3,555; 
Transition phase: Percent: 100. 

Source: GAO analysis of DOD data. 

Note: Entries are responses to four survey questions (rows) asked 
after servicemembers complete the MEB, PEB, or Transition phase 
(columns).  

[End of table]  

DOD's indices are one reasonable method of summarizing servicemember 
opinions. In quarterly performance reports, DOD notes that it has used 
factor analysis, a form of latent variable statistical models to 
assess the reliability of its scales. While we did not review DOD's 
models, we independently found that DOD's overall index of 
satisfaction with IDES was highly reliable. (Specifically, using 
Cronbach's alpha, the index was highly correlated with a single latent 
dimension at = 0.92.) This supports DOD's choice to measure the single 
concept of "satisfaction" by averaging the ordinal servicemember 
responses. 

Nevertheless, the average survey response can obscure variation in the 
responses that make up the index. For example, suppose that a 
servicemember said she was "very satisfied" (response of 5) on two of 
the four questions in the index, "dissatisfied" on one (response of 2) 
and "very dissatisfied" on the last one (response of 1). With an 
average response over 3, the DOD measure would classify her as 
"satisfied," despite the fact that she was "somewhat dissatisfied" or 
"very dissatisfied" with two of the four aspects of IDES that DOD 
considers important. The grouping rule considers this servicemember 
equally happy with IDES as someone who says they are "satisfied" with 
all four aspects in the index. 

To assess the extent to which DOD's index might mask dissatisfaction, 
we calculated the proportion of questions in the scale on which 
servicemembers whom DOD classified as "satisfied" gave neutral or 
negative responses (1, 2, or 3). We found that half of these 
servicemembers gave neutral or negative answers to at least 25 percent 
of the items in the index, and a quarter gave such answers to at least 
41 percent of the items. For these servicemembers, the DOD index may 
suggest more satisfaction than the underlying survey questions would 
support. 

We further assessed the sensitivity of DOD's index by comparing it 
against a different (i.e., GAO's) measure of satisfaction: whether a 
servicemember is "somewhat" or "very satisfied" (or gives a similarly 
positive response) on all items in DOD's scale of overall IDES 
satisfaction. Our measure is more conservative than DOD's, because 
ours only includes positive responses and uses a broader cutpoint (two 
response categories) to distinguish between "satisfied" and "not 
satisfied" servicemembers. (In contrast, DOD calculates average 
satisfaction on an ordinal scale of 1 to 5, and then uses a cutpoint 
at 3.) Our measure is not inherently more valid, however, and has its 
own weaknesses. In particular, we classify a servicemember as "not 
satisfied" if she gives a neutral or negative response to just one of 
the four items in DOD's scale.[Footnote 41] 

When we analyzed overall satisfaction using both measures, we found 
that overall, servicemembers are 2.8 times less satisfied on our 
measure than on DOD's (i.e., 23.8 versus 67 percent). Further, only 
about 20 to 30 percent of servicemembers are "satisfied" with each 
aspect of the IDES process that DOD considers important across most of 
the subgroups we analyzed, while DOD classifies about 60 to 70 percent 
of such servicemembers as "satisfied" on average. In the next section, 
we present further information on variation in satisfaction across 
servicemember groups. 

Explaining Servicemember Satisfaction Outcomes: 

Although the servicemember survey provides numerous measures of 
satisfaction, it is also important to explain variation in 
satisfaction outcomes--i.e. why some servicemembers are more satisfied 
than others. Explaining variation can connect dissatisfaction with 
poor program performance and help identify specific reforms to improve 
the experiences of servicemembers who typically have been less 
satisfied. However, the available program data cannot precisely 
explain outcomes when used in this type of explanatory analysis. Using 
the available data, we could predict satisfaction only 1.9 percentage 
points better after controlling for multiple factors than what we 
would have achieved by chance (65.5 percent vs. 63.7 percent of 
satisfied responses predicted correctly). 

In order to further explain variation in satisfaction, we matched the 
survey responses to the data that DOD and VA maintain on the 
processing of each servicemember's case, known as the VTA data. This 
database primarily measures the time it took servicemembers to 
complete each phase of the IDES process. A small number of other 
program and demographic variables are also available, such as service 
branch, component, and the number of conditions claimed and referred. 
Using the matched survey and VTA data, we estimated the association 
between satisfaction and observable factors that could potentially 
explain variation in servicemembers' experiences. 

Table 11 below (columns 2-4) presents these associations for both 
DOD's and GAO's overall measures of satisfaction. The "raw data" 
estimates are simply the proportion of servicemembers in a particular 
group who were satisfied according to either measure. In the fourth 
column ("model estimates"), we estimate this proportion holding 
constant all of the other factors listed, using a statistical model. 
Specifically, the estimates are in-sample mean predicted probabilities 
of giving a satisfied response on the GAO satisfaction index from a 
logistic model of satisfaction. The covariates are given by indicators 
of whether the servicemember belonged to each group in column 4. The 
maximum likelihood estimators allowed the probability of satisfaction, 
given the covariates, to be dependent across observations within the 
26 cross-classified groups of PEB location and MEB medical treatment 
facility. This adjusted for the possibility that servicemembers were 
similarly satisfied if they were processed in the same locations, 
given similar values on the observed covariates.[Footnote 42] 

Table 11: Overall IDES Satisfaction by Subgroups: 

All respondents; 
DOD Measure: Average answer exceeded neutral: Raw data: Percentage: 67; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.8; 
Model estimates: Percentage: [Empty]; 
N: 9,865. 

Component: Active; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.4; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.7; 
Model estimates: Percentage: 23.7; 
N: 8,692. 

Component: Reserves/National Guard; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 63.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24.6; 
Model estimates: Percentage: 23.7; 
N: 1,173. 

Branch: Air Force; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 61.8; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.6; 
Model estimates: Percentage: 23.7; 
N: 838. 

Branch: Army; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 70.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 26.5; 
Model estimates: Percentage: 24.5; 
N: 5,398. 

Branch: Marine Corps; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 61.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 19; 
Model estimates: Percentage: 23.8; 
N: 2,198. 

Branch: Navy; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.2; 
Model estimates: Percentage: 23; 
N: 1,431. 

Rollout phase[A]: 1; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 63.7; 
GAO Measure: All answers positive: 
Raw data: Percentage: 20.2; 
Model estimates: Percentage: [Empty]; 
N: 1,430. 

Rollout phase[A]: 1.1; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.8; 
Model estimates: Percentage: [Empty]; 
N: 4,827. 

Rollout phase[A]: 2; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 72.7; 
GAO Measure: All answers positive: 
Raw data: Percentage: 30.4; 
Model estimates: Percentage: [Empty]; 
N: 1,601. 

Rollout phase[A]: 3.1; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 69.6; 
GAO Measure: All answers positive: 
Raw data: Percentage: 34.6; 
Model estimates: Percentage: [Empty]; 
N: 257. 

Rollout phase[A]: 3.2; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.7; 
GAO Measure: All answers positive: 
Raw data: Percentage: 38.5; 
Model estimates: Percentage: [Empty]; 
N: 65. 

Rollout phase[A]: 3.3; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 0; 
GAO Measure: All answers positive: 
Raw data: Percentage: 0; 
Model estimates: Percentage: [Empty]; 
N: 1. 

Final disposition: Fit and Returned to Duty; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 70.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 27.4; 
Model estimates: Percentage: 21.7; 
N: 963. 

Final disposition: Permanent Disability Retirement List; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 69.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 25.4; 
Model estimates: Percentage: 25.6; 
N: 2,286. 

Final disposition: Separated with Benefits; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.1; 
Model estimates: Percentage: 22.5; 
N: 2,848. 

Final disposition: Separated without Benefits; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 63.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 20.4; 
Model estimates: Percentage: 21.5; 
N: 157. 

Final disposition: Temporary Disability Retirement List; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.8; 
Model estimates: Percentage: 24.4; 
N: 2,786. 

Final disposition: Unfit and Returned to Duty; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 63; 
GAO Measure: All answers positive: 
Raw data: Percentage: 16.4; 
Model estimates: Percentage: 17; 
N: 73. 

Fiscal Year started MEB: 2008; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 60.4; 
GAO Measure: All answers positive: 
Raw data: Percentage: 15.1; 
Model estimates: Percentage: 13; 
N: 530. 

Fiscal Year started MEB: 2009; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 68.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.4; 
Model estimates: Percentage: 21.2; 
N: 2,627. 

Fiscal Year started MEB: 2010; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 22.9; 
Model estimates: Percentage: 24.5; 
N: 4,721. 

Fiscal Year started MEB: 2011; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 69.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 31.6; 
Model estimates: Percentage: 28.3; 
N: 1,986. 

Claimed conditions: 0-4; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 71.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 27.7; 
Model estimates: Percentage: 24.2; 
N: 2,309. 

Claimed conditions: 5-7; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 69.4; 
GAO Measure: All answers positive: 
Raw data: Percentage: 25.1; 
Model estimates: Percentage: 23.9; 
N: 2,028. 

Claimed conditions: 8-12; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 22.5; 
Model estimates: Percentage: 23.1; 
N: 2,771. 

Claimed conditions: 13-19; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 62.6; 
GAO Measure: All answers positive: 
Raw data: Percentage: 22.6; 
Model estimates: Percentage: 25.1; 
N: 1,882. 

Claimed conditions: 20+; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 61.7; 
GAO Measure: All answers positive: 
Raw data: Percentage: 16.6; 
Model estimates: Percentage: 20.2; 
N: 825. 

Referred conditions: 0-1; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.7; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24.9; 
Model estimates: Percentage: 24.1; 
N: 4,686. 

Referred conditions: 2; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24; 
Model estimates: Percentage: 23.7; 
N: 2,482. 

Referred conditions: 3; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.8; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.6; 
Model estimates: Percentage: 23.4; 
N: 1,679. 

Referred conditions: 4+; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.8; 
GAO Measure: All answers positive: 
Raw data: Percentage: 22.5; 
Model estimates: Percentage: 22; 
N: 953. 

DOD Percentage Rating: 0; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 20.8; 
Model estimates: Percentage: [Empty]; 
N: 236. 

DOD Percentage Rating: 10; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24.6; 
Model estimates: Percentage: [Empty]; 
N: 1,287. 

DOD Percentage Rating: 20; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.6; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.1; 
Model estimates: Percentage: [Empty]; 
N: 978. 

DOD Percentage Rating: 30; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.7; 
Model estimates: Percentage: [Empty]; 
N: 680. 

DOD Percentage Rating: 40; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 22.2; 
Model estimates: Percentage: [Empty]; 
N: 531. 

DOD Percentage Rating: 50; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.4; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.5; 
Model estimates: Percentage: [Empty]; 
N: 786. 

DOD Percentage Rating: 60; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.7; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23; 
Model estimates: Percentage: [Empty]; 
N: 609. 

DOD Percentage Rating: 70; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 68.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.2; 
Model estimates: Percentage: [Empty]; 
N: 556. 

DOD Percentage Rating: 80; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 68.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 30; 
Model estimates: Percentage: [Empty]; 
N: 280. 

DOD Percentage Rating: 90; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 63.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.3; 
Model estimates: Percentage: [Empty]; 
N: 122. 

DOD Percentage Rating: 100; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 75.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 35.5; 
Model estimates: Percentage: [Empty]; 
N: 282. 

MEB treatment facility: Andrews; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 57.4; 
GAO Measure: All answers positive: 
Raw data: Percentage: 18.5; 
Model estimates: Percentage: 19.8; 
N: 319. 

MEB treatment facility: Lejeune; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 56.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 18; 
Model estimates: Percentage: 18.4; 
N: 1,046. 

MEB treatment facility: Pendleton; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 62.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 17; 
Model estimates: Percentage: 19.2; 
N: 372. 

MEB treatment facility: Belvoir; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 25; 
Model estimates: Percentage: 30.1; 
N: 252. 

MEB treatment facility: Benning; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 77.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 34; 
Model estimates: Percentage: 25.6; 
N: 260. 

MEB treatment facility: Bliss; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: [Empty]; 
GAO Measure: All answers positive: 
Raw data: Percentage: [Empty]; 
Model estimates: Percentage: [Empty]; 
N: 0. 

MEB treatment facility: Bragg; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 72.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 36.8; 
Model estimates: Percentage: 34.3; 
N: 288. 

MEB treatment facility: Campbell; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 90; 
GAO Measure: All answers positive: 
Raw data: Percentage: 50; 
Model estimates: Percentage: 47; 
N: 10. 

MEB treatment facility: Carson; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 64.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 19; 
Model estimates: Percentage: 23.7; 
N: 736. 

MEB treatment facility: Drum; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 30; 
Model estimates: Percentage: 27.9; 
N: 397. 

MEB treatment facility: Gordon; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 50; 
GAO Measure: All answers positive: 
Raw data: Percentage: 50; 
Model estimates: Percentage: [Empty]; 
N: 2. 

MEB treatment facility: Hood; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 81.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 33.3; 
Model estimates: Percentage: 30.7; 
N: 519. 

MEB treatment facility: Lewis; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 59.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21; 
Model estimates: Percentage: 20.2; 
N: 252. 

MEB treatment facility: Meade; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 55.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 15.6; 
Model estimates: Percentage: 14.7; 
N: 218. 

MEB treatment facility: Polk; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 81; 
GAO Measure: All answers positive: 
Raw data: Percentage: 33.3; 
Model estimates: Percentage: 28.1; 
N: 400. 

MEB treatment facility: Riley; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 59; 
GAO Measure: All answers positive: 
Raw data: Percentage: 20; 
Model estimates: Percentage: 17.4; 
N: 105. 

MEB treatment facility: Stewart; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 74.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 26.7; 
Model estimates: Percentage: 27; 
N: 718. 

MEB treatment facility: Other; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 68; 
GAO Measure: All answers positive: 
Raw data: Percentage: 25.6; 
Model estimates: Percentage: 20.3; 
N: 961. 

MEB treatment facility: Portsmouth; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23; 
Model estimates: Percentage: 21.8; 
N: 283. 

MEB treatment facility: San Antonio; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 71.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 26.3; 
Model estimates: Percentage: 24.2; 
N: 505. 

MEB treatment facility: San Diego; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 20.5; 
Model estimates: Percentage: 21.3; 
N: 1,127. 

MEB treatment facility: Tripler; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 83.3; 
GAO Measure: All answers positive: 
Raw data: Percentage: 53; 
Model estimates: Percentage: 37.8; 
N: 30. 

MEB treatment facility: Walter Reed (Army); 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 65.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.7; 
Model estimates: Percentage: 26.4; 
N: 511. 

MEB treatment facility: Walter Reed (Navy/Marine Corps); 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.4; 
GAO Measure: All answers positive: 
Raw data: Percentage: 19.9; 
Model estimates: Percentage: 24.1; 
N: 554. 

Time in IDES (GAO calculation) at disenrollment or December 31, 2011): 
1st quartile; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 81.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 38.3; 
Model estimates: Percentage: 41.1; 
N: 313. 

Time in IDES (GAO calculation) at disenrollment or December 31, 2011): 
2nd quartile; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 78.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 35; 
Model estimates: Percentage: 33.8; 
N: 1,658. 

Time in IDES (GAO calculation) at disenrollment or December 31, 2011): 
3rd quartile; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 70.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24.8; 
Model estimates: Percentage: 23.7; 
N: 3,667. 

Time in IDES (GAO calculation) at disenrollment or December 31, 2011): 
4th quartile; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 58.8; 
GAO Measure: All answers positive: 
Raw data: Percentage: 17.4; 
Model estimates: Percentage: 17.8; 
N: 4,226. 

Sex: Female; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 25.4; 
Model estimates: Percentage: 25.2; 
N: 1,799. 

Sex: Male; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.5; 
Model estimates: Percentage: 23.3; 
N: 8,063. 

Personnel Class: Enlisted; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.9; 
Model estimates: Percentage: 23.9; 
N: 9,126. 

Personnel Class: Officer; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 64.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.7; 
Model estimates: Percentage: 21; 
N: 691. 

Personnel Class: Warrant; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 77; 
GAO Measure: All answers positive: 
Raw data: Percentage: 29.2; 
Model estimates: Percentage: 27.6; 
N: 48. 

PEB Location: Lewis; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 64.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 20.4; 
Model estimates: Percentage: [Empty]; 
N: 1,283. 

PEB Location: National Capital Region; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66; 
GAO Measure: All answers positive: 
Raw data: Percentage: 26.1; 
Model estimates: Percentage: [Empty]; 
N: 1,698. 

PEB Location: San Antonio (Randolph); 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 61.8; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.6; 
Model estimates: Percentage: [Empty]; 
N: 838. 

PEB Location: San Antonio (Sam Houston); 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 76.5; 
GAO Measure: All answers positive: 
Raw data: Percentage: 30; 
Model estimates: Percentage: [Empty]; 
N: 2,417. 

PEB Location: Washington Navy Yard; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 63.1; 
GAO Measure: All answers positive: 
Raw data: Percentage: 19.9; 
Model estimates: Percentage: [Empty]; 
N: 3,629. 

MEB Exam Location: Military Treatment Facility; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67.2; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24.1; 
Model estimates: Percentage: 22.1; 
N: 868. 

MEB Exam Location: Contractor Facility; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 67; 
GAO Measure: All answers positive: 
Raw data: Percentage: 23.9; 
Model estimates: Percentage: 23.7; 
N: 1,991. 

MEB Exam Location: VA Medical Center; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 24.2; 
Model estimates: Percentage: 23.4; 
N: 5,976. 

MEB Exam Location: Not Available; 
DOD Measure: Average answer exceeded neutral: 
Raw data: Percentage: 66.9; 
GAO Measure: All answers positive: 
Raw data: Percentage: 21.2; 
Model estimates: Percentage: 27.2; 
N: 1,030. 

Source: GAO analysis of DOD and VA data. 

[A] Rollout phases correspond with the order in which IDES deployed at 
various facilities.  

[End of table]  

Regardless of which measure is used (DOD's or GAO's), satisfaction 
varied only modestly across many important groups of servicemembers. 
Our model estimates that the GAO measure of satisfaction varied by no 
more than approximately five percentage points across branch, 
component, disenrollment outcome, sex, MEB exam provider, enlisted and 
officer personnel classes, and the number of claimed and referred 
conditions, although differences across MEB treatment facilities and 
PEB locations were larger.[Footnote 43] This can be seen as a positive 
outcome, if this correlation implies that DOD and VA administer the 
program consistently across servicemembers and locations. However, the 
lack of variation also could suggest that the survey items do not 
measure opinions in enough detail to discriminate among 
servicemembers' experiences. 

Also shown in table 11, satisfaction had a stronger association with 
case processing time (time spent in IDES) than some of the other 
factors we examined. Servicemembers whose case processing times were 
among the quickest 25 percent were about 2.3 times as likely to be 
satisfied (on the GAO scale) than those whose times were among the 25 
percent of cases with the longest overall timeframes (i.e., 41 versus 
18 percent). Nevertheless, only 41 percent of those servicemembers 
whose cases were processed most quickly were satisfied (holding 
constant the other factors). This suggests that servicemembers' 
opinions about IDES may be only loosely related to the amount of time 
they spent in IDES, as discussed in the next section below. 

Although the average case processing time has generally increased 
since 2008, when we look at satisfaction by fiscal year, servicemember 
satisfaction shows evidence of improvement since fiscal year 2008. 
Specifically, our measure of satisfaction from the model increased by 
15 percentage points since 2008, roughly doubling from 13 to 28. 
Because the model estimates control for various other factors, these 
results suggest that servicemember views of the IDES process have 
improved over time, rather than the possibility that IDES has simply 
processed different types of cases. 

Satisfaction does not vary by a large amount across many MEB treatment 
facilities, but there are exceptions. Our model estimates that about 
18 to 26 percent of servicemembers were satisfied at most facilities. 
However, there were pockets of greater satisfaction. Specifically, 
servicemembers had more positive experiences at Forts Belvoir, Bragg, 
Campbell, Drum, Hood, and Polk, with satisfaction estimated to have 
ranged from 28 to 45 percentage points. Fort Meade had the lowest 
satisfaction at 15 percent. These estimates hold constant time spent 
in IDES and other factors in column 4 and, thus, partially account for 
the types of cases each facility processes. 

Satisfaction with Timeliness: 

DOD and VA measure IDES timeliness directly in VTA and as part of the 
overall servicemember satisfaction scale. These overlapping measures 
let us compare servicemembers' opinions to their actual experiences in 
the program. To do this, we calculated processing times at each phase 
of IDES for servicemembers who expressed varying degrees of 
satisfaction with the timeliness of their case processing at that 
phase. In addition, we analyzed whether servicemembers who were 
satisfied with the overall IDES process were more or less likely to 
meet timeliness goals. Table 12 provides these statistics. 

Table 12: Perceived Timeliness of IDES by Actual Processing Times: 

Overall satisfaction with IDES (GAO Measure):  

Not satisfied; 
Days in IDES (by phase): 
10th Percentile: 198; 
Days in IDES Median: 348; 
Days in IDES 90th Percentile: 544; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 2.57; 
Percent in 2nd Quartile: 14.33; 
Percent in 3rd Quartile: 336.68; 
Percent in 4th Quartile: 46.42; 
Percent Met DOD Goal: 33.8; 
N: 7,516. 

Satisfied; 
Days in IDES (by phase): 
10th Percentile: 159; 
Days in IDES Median: 297; 
Days in IDES 90th Percentile: 479; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 5.11; 
Percent in 2nd Quartile: 24.74; 
Percent in 3rd Quartile: 38.76; 
Percent in 4th Quartile: 31.39; 
Percent Met DOD Goal: 50.2; 
N: 2,349. 

Perceived MEB timeliness:  

Very poor; 
Days in IDES (by phase): 
10th Percentile: 88; 
Days in IDES Median: 175; 
Days in IDES 90th Percentile: 323; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 7.9; 
Percent in 2nd Quartile: 22.5; 
Percent in 3rd Quartile: 30.9; 
Percent in 4th Quartile: 38.7; 
Percent Met DOD Goal: 16.8; 
N: 1,685. 

Poor; 
Days in IDES (by phase): 
10th Percentile: 78; 
Days in IDES Median: 156; 
Days in IDES 90th Percentile: 290; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 11.6; 
Percent in 2nd Quartile: 25.7; 
Percent in 3rd Quartile: 35.1; 
Percent in 4th Quartile: 27.6; 
Percent Met DOD Goal: 23.4; 
N: 1,594. 

Mix of poor/good; 
Days in IDES (by phase): 
10th Percentile: 72; 
Days in IDES Median: 142; 
Days in IDES 90th Percentile: 273; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 15.1; 
Percent in 2nd Quartile: 28.6; 
Percent in 3rd Quartile: 34.7; 
Percent in 4th Quartile: 21.6; 
Percent Met DOD Goal: 27.9; 
N: 2,397. 

Good; 
Days in IDES (by phase): 
10th Percentile: 66; 
Days in IDES Median: 128; 
Days in IDES 90th Percentile: 254; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 19.8; 
Percent in 2nd Quartile: 31.9; 
Percent in 3rd Quartile: 29.8; 
Percent in 4th Quartile: 18.6; 
Percent Met DOD Goal: 35.6; 
N: 2,811. 

Very good; 
Days in IDES (by phase): 
10th Percentile: 62; 
Days in IDES Median: 114; 
Days in IDES 90th Percentile: 223; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 24.9; 
Percent in 2nd Quartile: 38.2; 
Percent in 3rd Quartile: 24.3; 
Percent in 4th Quartile: 12.6; 
Percent Met DOD Goal: 45.3; 
N: 1,200. 

Perceived PEB timeliness satisfaction:  

Very poor; 
Days in IDES (by phase): 
10th Percentile: 28; 
Days in IDES Median: 88; 
Days in IDES 90th Percentile: 203; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 22.8; 
Percent in 2nd Quartile: 18.1; 
Percent in 3rd Quartile: 23.7; 
Percent in 4th Quartile: 35.4; 
Percent Met DOD Goal: 67.8; 
N: 759. 

Poor; 
Days in IDES (by phase): 
10th Percentile: 28; 
Days in IDES Median: 89; 
Days in IDES 90th Percentile: 186; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 20.8; 
Percent in 2nd Quartile: 18.9; 
Percent in 3rd Quartile: 24.7; 
Percent in 4th Quartile: 35.7; 
Percent Met DOD Goal: 69.7; 
N: 680. 

Mix of poor/good; 
Days in IDES (by phase): 
10th Percentile: 23; 
Days in IDES Median: 70; 
Days in IDES 90th Percentile: 155; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 28.3; 
Percent in 2nd Quartile: 23.4; 
Percent in 3rd Quartile: 24; 
Percent in 4th Quartile: 24.3; 
Percent Met DOD Goal: 79.5; 
N: 1,262. 

Good; 
Days in IDES (by phase): 
10th Percentile: 23; 
Days in IDES Median: 65; 
Days in IDES 90th Percentile: 138; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 30.7; 
Percent in 2nd Quartile: 25.5; 
Percent in 3rd Quartile: 24.4; 
Percent in 4th Quartile: 19.4; 
Percent Met DOD Goal: 84.3; 
N: 1,470. 

Very good; 
Days in IDES (by phase): 
10th Percentile: 17; 
Days in IDES Median: 55; 
Days in IDES 90th Percentile: 128; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 40.2; 
Percent in 2nd Quartile: 26.2; 
Percent in 3rd Quartile: 18; 
Percent in 4th Quartile: 15.6; 
Percent Met DOD Goal: 88.1; 
N: 740. 

Perceived Transition timeliness:  

Very poor; 
Days in IDES (by phase): 
10th Percentile: 33; 
Days in IDES Median: 87; 
Days in IDES 90th Percentile: 461; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 7.9; 
Percent in 2nd Quartile: 22.5; 
Percent in 3rd Quartile: 30.9; 
Percent in 4th Quartile: 38.7; 
Percent Met DOD Goal: 16.1; 
N: 500. 

Poor; 
Days in IDES (by phase): 
10th Percentile: 43; 
Days in IDES Median: 90; 
Days in IDES 90th Percentile: 459; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 11.6; 
Percent in 2nd Quartile: 25.7; 
Percent in 3rd Quartile: 35.1; 
Percent in 4th Quartile: 27.6; 
Percent Met DOD Goal: 12.6; 
N: 468. 

Mix of poor/good; 
Days in IDES (by phase): 
10th Percentile: 40; 
Days in IDES Median: 90; 
Days in IDES 90th Percentile: 481; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 15.1; 
Percent in 2nd Quartile: 28.6; 
Percent in 3rd Quartile: 34.7; 
Percent in 4th Quartile: 21.6; 
Percent Met DOD Goal: 13.3; 
N: 967. 

Good; 
Days in IDES (by phase): 
10th Percentile: 37; 
Days in IDES Median: 91; 
Days in IDES 90th Percentile: 496; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 19.8; 
Percent in 2nd Quartile: 31.9; 
Percent in 3rd Quartile: 29.8; 
Percent in 4th Quartile: 18.6; 
Percent Met DOD Goal: 12.8; 
N: 1,065. 

Very good; 
Days in IDES (by phase): 
10th Percentile: 39; 
Days in IDES Median: 91; 
Days in IDES 90th Percentile: 657; 
Actual processing time (by Phase): 
Percent in 1st Quartile: 24.9; 
Percent in 2nd Quartile: 38.2; 
Percent in 3rd Quartile: 24.3; 
Percent in 4th Quartile: 12.6; 
Percent Met DOD Goal: 12.8; 
N: 541. 

Source: GAO analysis of DOD and VA data.  

[End of table]  

As shown in table 12, satisfaction generally stayed the same or 
decreased as processing times increased. The median days spent in the 
MEB and PEB phases were 35 and 38 percent lower, respectively, among 
those servicemembers who said that MEB and PEB timeliness was "very 
good" as compared to those who said it was "very poor." The former 
group was 170 percent more likely to have met the MEB timeliness goal 
and 30 percent more likely to have met the PEB timeliness goal. 
Similarly, the case for a median servicemember--whom we classified as 
"satisfied" with the overall IDES process--was completed 15 percent 
more quickly and was 49 percent more likely to have met the timeliness 
goal than the median servicemember who was "dissatisfied." The model 
estimates in table 11 confirm that the GAO measure of satisfaction and 
timeliness (time spent in IDES) are negatively related even when 
holding constant several other variables. 

Perceived and actual timeliness had little association at the 
Transition phase. Across all levels of satisfaction with timeliness, 
the median processing time varied by no more than 4 days, and the 
proportion meeting the timeliness goal varied by no more than 4 
percentage points. The use of personal leave is one plausible 
explanation for the unresponsiveness of servicemember satisfaction to 
actual processing times in the Transition phase. A servicemember might 
not have been dissatisfied with delays if taking leave was the reason, 
rather than the IDES process itself. 

Despite the associations between actual and perceived timeliness at 
the MEB and PEB phases, there were many servicemembers who were 
satisfied or dissatisfied with timeliness that spent similar amounts 
of time in the program. For example, 68 percent of those who said that 
PEB timeliness was "very poor" completed the phase on time, and 55 
percent of those who said that MEB timeliness was "very good" did not 
complete on time. Among servicemembers who said that MEB timeliness 
was "very good," the middle 80 percent of processing times ranged from 
62 days to 223 days. The same range for servicemembers who said MEB 
timeliness was "very poor" was 88 to 323 days. As table 12 shows, a 
similar pattern holds for the PEB phase. Although servicemembers tend 
to be more satisfied in MEB and PEB when their cases take less time, 
many of them are highly dissatisfied even when their cases take an 
unusually short amount of time (and vice versa). In the Transition 
phase, however, 40 percent of servicemembers who said that timeliness 
was "very good" were processed in 91 to 657 days--a more lengthy range 
than at the other phases. The large range and relationship with 
satisfaction may reflect the use of servicemember leave. 

The fact that many servicemembers are similarly satisfied with 
timeliness, even though they can have widely different processing 
times, has broader implications for measuring the performance of IDES. 
DOD's timeliness goals may not be meaningful to servicemembers or 
necessarily reflect high-quality service. Alternatively, 
servicemembers may not use reasonable standards to assess the time 
required to process their cases, or they may not accurately perceive 
the time they have spent in the program. In these scenarios, the value 
of servicemember satisfaction as a performance measure becomes less 
certain. 

The relationship between perceived and actual timeliness may simply 
reflect a large amount of unobserved heterogeneity across 
servicemembers. For example, a servicemember whose case has been in 
IDES for an extremely long time might still be highly satisfied with 
timeliness if the case was complex or personal leave was taken during 
the process. Neither the survey nor the VTA data measure these or 
other such characteristics that might affect the program's key 
performance measures. 

Factors Affecting Survey Results and Implications for Program 
Evaluation: 

The lack of variation in satisfaction across servicemember groups and 
according case timeliness might be seen as a positive outcome, and may 
suggest that DOD and VA administer the program consistently across 
servicemembers and locations. However, the lack of variation also 
could suggest shortcomings in the design and administration of the 
survey, or in data limitations that, alone or together, may reduce the 
usefulness of survey data for program evaluation. For example: 

* Survey questions: The survey questions may not be sufficiently 
detailed to measure important differences among servicemembers' 
experiences. For example, the survey includes 12 questions (4 per 
survey) that measure broad opinions about IDES, and DOD subsequently 
averages these responses together. This approach may limit the 
survey's capacity to describe IDES experiences in sufficient detail. 

* Precision of DOD indices: DOD reports measures of overall 
satisfaction with IDES for each phase, using the questions in table 9. 
However, these measures include one question that asks respondents to 
"evaluate their overall experience since entering the IDES process," 
which could be influenced by experiences in prior phases. 
Consequently, the satisfaction measures reported for each phase could 
represent a combination of servicemembers' experiences in that phase 
and prior phases. 

* Completing two surveys at once: DOD officials told us that a 
servicemember may be surveyed for the PEB and Transition phases in one 
session. In these instances a large amount of time may have passed 
since the servicemember completed the PEB phase and it may be more 
difficult for the servicemember to isolate his or her satisfaction 
with a particular phase. 

* Survey design: The satisfaction survey is primarily designed to 
measure performance, not explain it. The survey includes many highly 
correlated questions measuring satisfaction with the overall process 
or broad components of it, such as DOD board liaisons, VA case 
managers, or timeliness. While multiple questions can improve the 
statistical reliability and validity of DOD's performance measures, 
they require costly survey administration time that could be used for 
other purposes, such as to measure a larger number of variables that 
could explain servicemember satisfaction or case processing times. 

* VTA data limitations: The VTA administrative data that we matched to 
survey data primarily measure processing times and basic servicemember 
demographics, such as service branch, component, and treatment 
facility. The data support detailed reporting of performance measures, 
but they do not measure similarly detailed information on the nature 
of each case that might allow DOD and VA to understand the reasons for 
lengthy case processing times or to identify cases that might become 
delayed and ensure that they remain on schedule. For example, the 
database does not measure the type or severity of referred medical 
conditions in detail, the nature of delays experienced early in the 
process, or the use of servicemember leave. In addition, little 
information is available on staffing at or caseloads for MEB and PEB 
locations, DOD board liaisons, or VA case managers, which might help 
to explain or predict performance. 

* Low response and coverage rates: The response and coverage rates of 
the satisfaction survey further limit the degree to which DOD can 
generalize the data obtained to the population of servicemembers who 
participate in IDES. In particular, the survey does not assess the 
views of servicemembers who disenroll from the process before 
finishing a stage or those who do not complete prior waves of the 
survey. Including servicemembers who do not complete all waves would 
complicate longitudinal analysis, however.  

[End of section]  

Appendix III: Monthly DOD Timeliness Data for Active Duty Cases in 
Fiscal Year 2012: 

Table 13 presents data reported by DOD on average processing time for 
active duty cases completed during part of fiscal year 2012--Oct. 2011 
to June 2012. DOD's data are provided as a supplement the analyses GAO 
conducted for fiscal years 2008 through 2011.[Footnote 44] We did not 
evaluate the reliability of these data and cannot predict the extent 
to which any trends will continue for the rest of the fiscal year. 

Table 13: DOD Reported Monthly Average Processing Times for Active 
Component Servicemembers in Fiscal Year 2012 (in days): 

Time spent in: IDES overall (only cases resulting in receipt of 
benefits); 
Goal (days): 295; 
Month that IDES process, phase or stage was completed: 
October 2011: 395; 
November 2011: 391; 
December 2011: 404; 
January 2012: 395; 
February 2012: 392; 
March 2012: 394; 
April 2012: 400; 
May 2012: 409; 
June 2012: 395. 

Time spent in: MEB phase; 
Goal (days): 100; 
Month that IDES process, phase or stage was completed: 
October 2011: 136; 
November 2011: 130; 
December 2011: 128; 
January 2012: 132; 
February 2012: 130; 
March 2012: 124; 
April 2012: 114; 
May 2012: 116; 
June 2012: 122. 

Time spent in: Medical Exam Stage; 
Goal (days): 45; 
Month that IDES process, phase or stage was completed: 
October 2011: 40; 
November 2011: 39; 
December 2011: 41; 
January 2012: 44; 
February 2012: 37; 
March 2012: 37; 
April 2012: 39; 
May 2012: 39; 
June 2012: 39. 

Time spent in: MEB Stage; 
Goal (days): 35; 
Month that IDES process, phase or stage was completed: 
October 2011: 76; 
November 2011: 71; 
December 2011: 73; 
January 2012: 76; 
February 2012: 77; 
March 2012: 72; 
April 2012: 64; 
May 2012: 66; 
June 2012: 69. 

Time spent in: PEB phase; 
Goal (days): 120; 
Month that IDES process, phase or stage was completed: 
October 2011: 95; 
November 2011: 100; 
December 2011: 104; 
January 2012: 112; 
February 2012: 116; 
March 2012: 111; 
April 2012: 114; 
May 2012: 120; 
June 2012: 122. 

Time spent in: Informal PEB stage; 
Goal (days): 15; 
Month that IDES process, phase or stage was completed: 
October 2011: 23; 
November 2011: 28; 
December 2011: 34; 
January 2012: 34; 
February 2012: 28; 
March 2012: 23; 
April 2012: 22; 
May 2012: 19; 
June 2012: 24. 

Time spent in: VA Preliminary Rating stage; 
Goal (days): 15; 
Month that IDES process, phase or stage was completed: 
October 2011: 26; 
November 2011: 27; 
December 2011: 31; 
January 2012: 34; 
February 2012: 37; 
March 2012: 51; 
April 2012: 58; 
May 2012: 45; 
June 2012: 35. 

Time spent in: Transition phase; 
Goal (days): 45; 
Month that IDES process, phase or stage was completed: 
October 2011: 71; 
November 2011: 70; 
December 2011: 69; 
January 2012: 75; 
February 2012: 77; 
March 2012: 76; 
April 2012: 74; 
May 2012: 77; 
June 2012: 74. 

Time spent in: VA Benefits phase; 
Goal (days): 30; 
Month that IDES process, phase or stage was completed: 
October 2011: 38; 
November 2011: 36; 
December 2011: 47; 
January 2012: 56; 
February 2012: 49; 
March 2012: 50; 
April 2012: 57; 
May 2012: 66; 
June 2012: 62. 

Source: DOD IDES Monthly Reports for April, May June , and July 2012. 

Note: This table does not reflect data on all phases or stages of the 
IDES process. Instead, it presents data on those phases and stages 
that GAO also presented data on in the body of the report.  

[End of table]  

[End of section]  

Appendix IV: Comments from the Department of Defense:  

Office of The Assistant Secretary Of Defense: 
Health Affairs: 
Washington, DC 20301-1200:  

Mr. Daniel Bertoni: 
Director, Education, Workforce, and Income Security Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548:  

Dear Mr. Bertoni,  

This is the Department of Defense (DoD) response to the Government 
Accountability Office (GAO) Draft Report 12-676, "Military Disability 
System: Improved Monitoring Needed to Better Track and Manage 
Performance," July 18, 2012 (GAO Code 131089).  

While the Department notes inaccuracy in some content of the report in 
regard to Veteran surveys, we concur with the GAO's three 
recommendations. The DoD response is attached to this letter.  

Sincerely,  

Signed by:  

John R. Campbell: 
Deputy Assistance Secretary of Defense for Warrior Care Policy:  

[End of letter]  

GAO Draft Report Dated July 18, 2012: 
GAO-12-676 (GAO Code 131089):  

"Military Disability System: Improved Monitoring Needed to Better 
Track and Manage Performance"  

Department Of Defense Comments To The GAO Recommendations:  

Recommendation 1: To ensure that Service member cases are processed 
and are awarded benefits in a timely manner, we recommend that the 
Secretaries of Defense and Veterans Affairs work together to develop 
timeframes for completing the IDES business process review and 
implementing any resulting recommendations.  

DoD Response: Concur.  

Recommendation 2: To improve DOD's ability to measure Service members' 
satisfaction with the IDES process, we recommend that the Secretary of 
Defense develop alternative approaches for collecting more meaningful 
and representative information in a cost effective manner.  

DoD Response: Concur.  

Recommendation 3: To ensure that IDES management decisions continue to 
be based upon reliable and accurate data, we recommend that the 
Secretaries of Defense and Veterans Affairs work together to develop a 
strategy to continuously monitor and remedy issues with VTA timeliness 
information. This could include issuing guidance to facilities or 
developing best practices on preventing and correcting data entry 
errors; and developing reporting capabilities in VTA to alert 
facilities to potential issues with their data.  

DoD Response: Concur.  

[End of section]  

Appendix V: Comments from the Department of Veterans Affairs:  

Department of Veterans Affairs: 
Washington DC 20420 

August 20, 2012:  

Mr. Daniel B. Bertoni: 
Director: 
Education, Workforce, and Income Security Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548:  

Dear Mr. Bertoni:  

The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, "Military Disability System:
Improved Monitoring Needed to Better Track and Manage Performance"
(GAO-12-676) and concurs with Recommendations 1 and 3 made to the 
Department. The enclosure contains additional details on VA's planned 
actions to address both recommendations, and technical comments to the 
draft report.  

VA appreciates the opportunity to comment on your draft report.  

Sincerely,  

Signed by:  

John R. Gingrich: 
Chief of Staff:  

Enclosure:  

[End of letter]  

Enclosure:  

Department of Veterans Affairs (VA) Comments to Government 
Accountability Office (GAO) Draft Report "Military Disability System: 
Improved Monitoring Needed to Better Track and Manage Performance" 
(GA0-12-676):  

GAO Recommendation 1: To ensure that servicemember cases are processed
and are awarded benefits in a timely manner, we recommend that the 
Secretaries of Defense and Veterans Affairs work together to develop 
timeframes for completing the IDES business process review and 
implementing any resulting recommendations.  

VA Response: Concur. Although the Department of Defense (DoD) has been 
leading the business process review efforts described in this report, 
the Department of Veterans Affairs (VA) has provided input and support 
to promote these efforts and will continue to do so to the extent 
possible. At this time, the full scope or current status of these 
efforts has not been disclosed to VA. As such, VA recommends that 
developing timeframes for completion of these efforts should be 
deferred to DoD. If, as a result of these business process reviews, 
recommendations are set forth and approved by VA and DoD, VA will 
contribute to the development of timeframes for implementation of 
these recommendations.  

GAO Recommendation 3: To ensure that IDES management decisions continue
to be based upon reliable and accurate data, we recommend that the 
Secretaries and Defense and Veterans Affairs work together to develop 
a strategy to continuously monitor and remedy issues with VTA 
timeliness information. This could include issuing guidance to 
facilities or developing best practices on preventing and correcting 
data entry errors; and developing reporting capabilities in VTA to 
alert facilities to potential issues with their data.  

VA Response: Concur. VA is currently engaged in several efforts to 
improve the quality of Veterans Tracking Application (VTA) data. VA 
now leads bi-monthly VTA training sessions, which are offered to both 
VA and DoD users. The training serves as an introduction for new users 
and refresher training for experienced users. As VA identifies error 
trends in VTA data, the training is modified to address those trends.
Further, VA holds a monthly Integrated Disability Evaluation System 
(IDES) teleconference call for IDES field personnel. As part of this 
call, VTA updates are shared each month, including information about 
system updates, reminders (to address error trends), and best 
practices (to facilitate compliance). Finally, during routine
oversight visits of regional offices and intake sites, VA conducts 
IDES program reviews that include verifying the timeliness and 
accuracy of VTA data entered by VA users.  

VA continues to identify VTA reporting capability needs and will 
examine the feasibility of alerting local users of potential outlier 
data. Expected date of completion, to include VTA programming, is 
September 30, 2014.  

[End of section]  

Appendix VI: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Daniel Bertoni, (202) 512-7215, bertonid@gao.gov: 

Staff Acknowledgments: 

Michele Grgich (Assistant Director), Daniel Concepcion, Melissa 
Jaynes, and Greg Whitney made significant contributions to all aspects 
of this report. Also contributing to this report were Bonnie Anderson, 
James Bennett, Mark Bird, Joanna Chan, Brenda Farrell, Jamila Jones 
Kennedy, Douglas Sloane, Almeta Spencer, Vanessa Taylor, Jeffrey 
Tessin, Roger Thomas, Walter Vance, Kathleen van Gelder, and Sonya 
Vartivarian.  

[End of section]  

Related GAO Products: 

Military Disability System: Preliminary Observations on Efforts to 
Improve Performance. [hyperlink, 
http://www.gao.gov/products/GAO-12-718T] (Washington, D.C.: May 23, 
2012). 

Military and Veterans Disability System: Worldwide Deployment of 
Integrated System Warrants Careful Monitoring. [hyperlink, 
http://www.gao.gov/products/GAO-11-633T] (Washington, D.C.: May 4, 
2011). 

Military and Veterans Disability System: Pilot Has Achieved Some 
Goals, but Further Planning and Monitoring Needed. [hyperlink, 
http://www.gao.gov/products/GAO-11-69] (Washington, D.C.: December 6, 
2010). 

Military and Veterans Disability System: Preliminary Observations on 
Evaluation and Planned Expansion of DOD/VA Pilot. [hyperlink, 
http://www.gao.gov/products/GAO-11-191T] (Washington, D.C.: November 
18, 2010). 

Veterans' Disability Benefits: Further Evaluation of Ongoing 
Initiatives Could Help Identify Effective Approaches for Improving 
Claims Processing. [hyperlink, http://www.gao.gov/products/GAO-10-213] 
(Washington, D.C.: January 29, 2010). 

Recovering Servicemembers: DOD and VA Have Jointly Developed the 
Majority of Required Policies but Challenges Remain. [hyperlink, 
http://www.gao.gov/products/GAO-09-728] (Washington, D.C.: July 8, 
2009). 

Recovering Servicemembers: DOD and VA Have Made Progress to Jointly 
Develop Required Policies but Additional Challenges Remain. 
[hyperlink, http://www.gao.gov/products/GAO-09-540T] (Washington, 
D.C.: April 29, 2009). 

Military Disability System: Increased Supports for Servicemembers and 
Better Pilot Planning Could Improve the Disability Evaluation Process. 
[hyperlink, http://www.gao.gov/products/GAO-08-1137] (Washington, 
D.C.: September 24, 2008). 

DOD and VA: Preliminary Observations on Efforts to Improve Care 
Management and Disability Evaluations for Servicemembers. [hyperlink, 
http://www.gao.gov/products/GAO-08-514T] (Washington, D.C.: February 
27, 2008). 

DOD and VA: Preliminary Observations on Efforts to Improve Health Care 
and Disability Evaluations for Returning Servicemembers. [hyperlink, 
http://www.gao.gov/products/GAO-07-1256T] (Washington, D.C.: September 
26, 2007). 

Military Disability System: Improved Oversight Needed to Ensure 
Consistent and Timely Outcomes for Reserve and Active Duty Service 
Members. [hyperlink, http://www.gao.gov/products/GAO-06-362] 
(Washington, D.C.: March 31, 2006). 

[End of section]  

Footnotes:  

[1] GAO, Military and Veterans Disability System: Pilot Has Achieved 
Some Goals, but Further Planning and Monitoring Needed, [hyperlink, 
http://www.gao.gov/products/GAO-11-69] (Washington, D.C.: Dec. 6, 
2010).  

[2] We visited the facilities at Joint Base Andrews and Fort Meade, 
Maryland; Naval Hospital Bremerton and Joint Base Lewis-McChord 
(Madigan Army Medical Center), Washington; and Forts Hood and Sam 
Houston, Texas. 

[3] A physician is required to identify a condition that may cause the 
member to fall below retention standards after the member has received 
the maximum benefit of medical care. 

[4] In this report, "DOD board liaisons" refers to DOD Physical 
Evaluation Board Liaison Officers, while "VA case managers" refers to 
VA Military Service Coordinators. 

[5] This evaluation is based on the results of the medical exams, the 
member's medical records, and input from the member's commanding 
officer. 

[6] VA determines the degree to which veterans are disabled in 10 
percent increments on a scale of 0 to 100 percent. If VA finds that a 
veteran has one or more service-connected disabilities with a combined 
rating of at least 10 percent, the agency will pay monthly 
compensation. 

[7] The reviewing authorities of PEBs in the respective services are 
the Air Force Personnel Council, the Army Physical Disability Agency, 
and the Navy Council of Review Boards. 

[8] The three original pilot sites were Walter Reed Army Medical 
Center, Washington, D.C.; National Naval Medical Center, Bethesda, 
Maryland; and Malcolm Grow Air Force Medical Center, Andrews Air Force 
Base.  

[9] When processing times are broken down by the year of completion, 
as in Figure 4, the average processing times in the first few years 
are lower since cases with longer processing times after the program's 
inception in fiscal year 2008 would not show up in the data until 
fiscal year 2009 or later.  

[10] Our data were for cases in the MEB phase as of December 31, 2011. 
For cases that had not yet been in the MEB for 100 days (the MEB 
goal), we cannot predict whether they will be timely cases or not. For 
instance some of these cases entered the MEB phase just days before 
the cutoff date GAO chose. 

[11] As previously noted, the narrative summary documents the medical 
conditions and the impact of these conditions on the servicemembers' 
ability to perform their military duties. 

[12] VA's goal is 1 case manager for every 30 new cases. 

[13] DOD did not provide monthly data on the percentage of reservist 
cases meeting these stage goals. See appendix III for more information 
on monthly processing times. 

[14] According to VA officials, reservists who are not on active duty 
do not have a transition phase. 

[15] DOD and VA did not set a goal for reserve component 
servicemembers. As noted above, for purposes of this report, we opted 
to not include reserve component time spent in the VA benefit phase in 
our calculations phase because this goal applies to some but not all 
reservists, depending on their active duty status. Any time spent 
within the VA benefit phase is reflected within the overall processing 
time calculations for such reserve component servicemembers. 

[16] DOD officials also told us that a servicemember may be surveyed 
simultaneously for a prior phase along with the phase just completed 
(e.g. the MEB and PEB phases). However, in some cases a significant 
amount of time may have passed since the servicemember completed the 
prior phase and, thus, it may be more difficult for the servicemember 
to isolate his or her satisfaction with a particular phase. 

[17] DOD could interview a probability sample of servicemembers to 
accurately assess satisfaction with IDES in the population as a whole. 
A sample could decrease the cost of obtaining this information, 
because DOD would need to contact fewer servicemembers and the cost of 
surveys generally increases with the number of people interviewed. If 
properly designed and executed, a probability sample would provide 
estimates that would be equally valid as interviewing all 
servicemembers.  

[18] Using DOD's satisfaction measure, we found less than expected 
variation in satisfaction over time and across key case 
characteristics, such as component, military branch, final rating and 
final disposition. To better understand factors that may drive 
servicemember satisfaction, we included neutral responses in the 
category of "not satisfied" rather than counting such responses as 
"satisfied" in the manner that DOD did thus arriving at a measure that 
more strongly reflects satisfaction and might be a more sensitive 
indicator of factors affecting satisfaction for performance management 
purposes. 

[19] Military Disability System: Increased Supports for Servicemembers 
and Better Pilot Planning Could Improve the Disability Evaluation 
Process, [hyperlink, http://www.gao.gov/products/GAO-08-1137] 
(Washington, D.C.: Sept. 24, 2008). 

[20] In January 2012, senior agency officials decided to merge the DOD-
VA Senior Oversight Committee into the DOD-VA Joint Executive Council, 
co-chaired by the Under Secretary of Defense (Personnel and Readiness) 
and the Deputy Secretary of Veterans Affairs. DOD and VA plan to 
establish a joint IDES working group under the Joint Executive 
Council. Meanwhile, the IDES has been incorporated into the agencies' 
Joint Strategic Plan, and agency stakeholders meet weekly to discuss 
IDES issues. 

[21] The Army has physicians dedicated to MEB cases. In contrast, Navy 
and Air Force MEB determinations are prepared by physicians who 
perform other responsibilities, such as clinical treatment and 
supervision. 

[22] Originally, reserve component members determined to require IDES 
referral were referred to the MEB at Fort Gordon, Georgia. According 
to Army officials, these members are referred to one of 10 Army MEBs. 

[23] A DOD official told us that based on recent negative feedback, 
including from site visits, DOD is considering canceling this pilot 
project. 

[24] As of March 2012, the Seattle rating site is responsible for 
cases from all 3 Army PEBs.  

[25] Under this proposed process, VA would not begin work on a case 
until a servicemember was found to be unfit for military service by an 
informal PEB. The pilot was postponed in response to concerns raised 
by the military services and VA about specific changes to the existing 
process. For example, VA officials expressed concerns that the 
preliminary VA rating could occur long after the servicemember was 
found to be unfit for duty, which could cause the servicemember to 
seek a formal PEB hearing. 

[26] In January and February 2012, teams visited Little Rock Air Force 
Base, Arkansas; Naval Medical Center San Diego, California; Fort 
Carson, Colorado; Wright-Patterson Air Force Base, Ohio; and Naval 
Medical Center Portsmouth, Virginia. In March 2012, a team visited 
Joint Base Andrews, Maryland, and Fort Eustis and Naval Health Clinic 
Quantico, Virginia, to examine the piloting of a new DOD-wide IDES 
information system. 

[27] The original facilities were Walter Reed Army Medical Center, 
National Naval Medical Center, Brooke Army Medical Center (Fort Sam 
Houston), and Naval Medical Center San Diego. 

[28] DOD's Office of Warrior Care Policy changed its name from the 
Office of Wounded Warrior Care and Transition Policy (WWCTP) in 2012.  

[29] The number of active and Reserve components cases do not add up 
to the number of total cases because seven cases did not have a 
component code in the data.  

[30] Demographic data included such information as gender, personnel 
class, service branch, component, and the number of conditions claimed 
and referred. 

[31] GAO, Military and Veterans Disability System: Pilot Has Achieved 
Some Goals, but Further Planning and Monitoring Needed, [hyperlink, 
http://www.gao.gov/products/GAO-11-69] (Washington, D.C.: Dec. 6, 
2010). 

[32] Our analysis found an additional 11 percent of cases with dates 
missing for stages in the IDES program--dates that were not critical 
to our analyses. We believe that many if not most of these cases of 
missing data might be explainable. Servicemembers do not always pass 
through every step of the process and we believe this may explain some 
portion of missing dates. However, without looking at case files for 
all of these cases or the VTA data in more detail we cannot determine 
what portion of those 11 percent have a reasonable explanation for 
being missing.  

[33] We did not review the data on a monthly basis because this is 
done in DOD's monthly reports and we wanted to provide a longer term 
perspective of the evolution of the IDES program.  

[34] While we have data through December for fiscal year 2012, we did 
not include these data in our analysis because we sought to compare 
completed fiscal years.  

[35] Including Directive-Type Memorandum 11-015, which establishes 
policies, assigns responsibilities, and prescribes procedures for the 
IDES. 

[36] We consider processing times overall, for the four phases of IDES 
(MEB, PEB, Transition, and VA Benefits), Active Duty, Reserve, or 
National Guard status, and for servicemembers receiving benefits 
versus all servicemembers regardless of the outcome of their cases. 

[37] Response rates refer to the percentages of servicemen who 
responded to the surveys among those who DOD deemed eligible. For the 
MEB survey, eligibles included all servicemen who completed the MEB 
phase, but for the PEB and Transition surveys eligibles included only 
those who completed those phases and also completed the prior surveys. 
Coverage rates refer the percentages of servicemen that responded to 
each survey among those who completed each phase, regardless of 
whether they completed prior surveys. 

[38] GAO's latest data export from the VTA database included data 
through January 1, 2012. 

[39] The servicemembers included in these figures have not necessarily 
completed the entire IDES process, but did complete a given phase to 
be included in the figure for that phase.  

[40] Though there are different ways to calculate response and 
coverage rates, which can result in different estimates of these 
quantities, we chose our response rate and coverage estimates in order 
to better understand what fraction of the intended group was actually 
surveyed. 

[41] Both measures of satisfaction combine responses from 
servicemembers who have finished various phases of IDES. For example, 
a servicemember who completed only the MEB would provide at most 4 
responses from that wave's survey, while a servicemember who completed 
all phases would provide at most 12 responses from all three surveys. 
Each measure calculates overall satisfaction using all responses 
available, even though the number of responses varies among 
servicemembers (e.g., 4 vs. 12).  

[42] Because the "cluster-robust" variance estimator is only 
consistent as the number of clusters becomes large, the relatively 
small number of MEB-PEB locations in our data raises the possibility 
of finite-sample bias. The practical effect of this problem should be 
minimal, however, since the model controls for MEB and PEB location 
and, thus, residual intraclass correlation should be small. 

[43] In addition, servicemembers who were found unfit and returned to 
duty--a rare outcome--were approximately 4 to 10 percentage points 
less likely to be satisfied than servicemembers who had any of the 
other outcomes. 

[44] DOD calculates timeliness separately for active, reserve, and 
guard components. Because our report includes analysis for active and 
reserve/guard combined, we are only including active servicemembers in 
this appendix for easy comparison. Active duty cases reflect the bulk 
of cases in fiscal year 2012, as with other fiscal years. 

[End of section]  

GAO’s Mission:  

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability.  

Obtaining Copies of GAO Reports and Testimony:  

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.”  

Order by Phone:  

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm].  

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537.  

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information.  

Connect with GAO:  

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov].  

To Report Fraud, Waste, and Abuse in Federal Programs:  

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470.  

[End of document]  

Congressional Relations:  

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548.  

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548.