VA Disability Benefits: Board of Veterans' Appeals Should Address Gaps in Its Quality Assurance Process
Fast Facts
If the VA denies a veteran's benefit request, the veteran can bring the case to the Board of Veterans' Appeals. Congress has raised questions about the quality of the Board's decisions.
We testified that as the Board faces large workloads and time pressure, quality assurance activities such as case reviews and decision monitoring are key tools for minimizing rework.
But the Board could better use evidence to improve these activities. For example, it hasn't fully assessed the underlying causes of common errors, which could lead to better solutions. We recommended doing so—and using other evidence—to address gaps in the Board's quality assurance.
Highlights
What GAO Found
The Board of Veterans' Appeals (Board) has a quality assurance (QA) process and a related accuracy measure for its decisions. Specifically, its QA process involves: 1) checking a random sample of draft decisions for certain types of Board-defined errors each month through its case review process; and 2) monitoring outcomes of Board decisions that were further appealed to the U.S. Court of Appeals for Veterans Claims (CAVC). The Board uses results from these activities to provide various interventions, such as individual feedback to veteran law judges (VLJs) or training. The Board also calculates and publishes an accuracy rate that represents error-free adjudications. However, GAO found shortfalls in the Board's process for calculating this measure. Contrary to federal internal control standards, GAO found that the Board did not have:
(1) written policies or procedures for calculating its accuracy rate or managing case review error data, or
(2) checks on its accuracy rate calculation.
As a result, the Board lost data and GAO could not verify accuracy rates provided by the Board in 2 of 4 fiscal years. Until the Board develops written policies and procedures, the Board will likely continue to have difficulty supporting the accuracy of its publicly reported measure.
GAO found gaps in the Board's efforts to build and use evidence—such as a lack of data and analysis—to assess its QA process and related activities. GAO analyses of CAVC and Board data show that over the past 3 fiscal years, CAVC remanded (sent back) about 80 percent of appealed Board decisions, often because CAVC found the Board's explanation of its findings to be inadequate. However, GAO found that the Board lacked evidence to better understand and address these and other issues and set priorities to help improve its QA process. Specifically:
- Board officials told GAO it has no comprehensive, written plan outlining how it will accomplish the mission and goals of its QA process.
- Board officials told GAO they had not fully analyzed trends or underlying causes of the most common Board-identified errors or CAVC remands.
- The Board has not systematically or comprehensively built or used evidence to better understand and improve its interventions, such as collecting feedback about training.
- The Board does not assess VLJ decisions for consistency, such as whether common misunderstandings of policy or law exist in decisions.
GAO's prior work has identified key practices for evidence-based decision-making. These practices involve building and assessing evidence and using it to foster a culture of continuous improvement. Absent such a process and a plan to guide it, the Board is not positioned to fully understand and address underlying causes of the most common errors and remands or understand consistency—evidence which is needed to target and implement effective interventions and foster continuous improvement of its QA efforts.
Why GAO Did This Study
Each year, the Board adjudicates thousands of cases in which a veteran was dissatisfied with the Department of Veterans Affairs' (VA) initial decision on their claim for benefits. However, researchers and Congress have raised questions about how the Board ensures the quality of its adjudications.
GAO was asked to review the Board's QA process. This testimony examines (1) how the Board assures and measures the quality of its adjudicative decisions, and (2) the extent to which the Board builds and uses evidence to assess its QA process.
GAO reviewed relevant federal laws and Board documents. GAO analyzed QA data provided by the Board for fiscal years 2019 through 2022 and assessed its reliability. GAO also reviewed relevant quality assurance literature and interviewed VA officials, attorneys, VLJs, and subject matter experts, including researchers and veterans service organizations. GAO also conducted group discussions with randomly selected Board decision-writing attorneys and VLJs.
Recommendations
GAO is making four recommendations, including that the Board develop policies for calculating its accuracy rate; develop and implement an evidence-based decision-making process for its QA efforts; and study decision-making consistency. The Board generally agreed with three recommendations and disagreed with the fourth about consistency. GAO continues to believe that the recommendation is valid as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Board of Veterans Appeals | The Chairman of the Board of Veterans' Appeals should develop written policies and procedures related to its accuracy rate measure, to require that OAI (1) involves more than one official in the calculation process; (2) documents its calculation of monthly and fiscal year accuracy rates; and (3) manages related error data. (Recommendation 1) |
VA's Board of Veterans' Appeals concurred in principle with this recommendation and acknowledged that its policy specifically related to its accuracy rate measure could be improved. According to Board officials, they are drafting a more comprehensive set of written guidance and procedures. Specifically, the Board reports that it modified its quality assurance processes in January 2024 so that more than one official is involved in calculating the accuracy rate; the calculation of the accuracy rate is documented and reviewed by the Board's Budget Office; and related error data are stored both electronically and in a secure, physical location. We will consider closing this recommendation when the Board provides us with (1) a copy of its new written guidance and procedures related to its accuracy rate measure, (2) documentation related to the modifications of its quality assurance processes, including details about which officials are involved in calculating the accuracy rate, examples of the documentation of the accuracy rate, and (3) details about the storing of the error data in an electronic and physical space and the security protocols in place.
|
| Board of Veterans Appeals | The Chairman of the Board of Veterans' Appeals should monitor how veteran law judges choose to incorporate the feedback they receive from the case review process—including whether errors are corrected—and use this data to inform decision making related to the case review process. (Recommendation 2) |
VA's Board of Veterans' Appeals concurred with this recommendation and stated it has modified its internal processes to monitor whether a Veteran Law Judge (VLJ) accepts or rejects the feedback offered through the Board's quality review process. In June 2024, Board officials stated that they had implemented a policy that requires VLJs to respond in writing whether they agree with the quality review feedback and whether they are addressing the identified error. Further, officials said the error memorandum has been modified to include a section where the VLJ must document their decision regarding whether to accept the feedback. They also said they collected data that revealed that between October 2023 and April 2024 VLJs rejected the quality review feedback at a rate of 0.35 percent and that the agency will use this data to inform decision-making related to the case review process. We will consider closing this recommendation when the Board provides (1) a copy of the Board's updated policy for monitoring how VLJs choose to incorporate OAI feedback, (2) the modified error memorandum, and (3) evidence showing that the Board has implemented the monitoring procedures and are using the information gleaned from the process.
|
| Board of Veterans Appeals | The Chairman of the Board of Veterans' Appeals should develop and implement an evidence-based decision-making process that includes a plan outlining how it will build evidence to assess the underlying causes for the most common errors identified by the case review process and the most common reasons for CAVC remands. The Board should use this evidence to better target its interventions and assess their results. One option is to fold the development of this process into the Board's planned evaluation. (Recommendation 3) |
VA's Board of Veterans' Appeals concurred in principle with this recommendation, and stated that since the release of our report, it has engaged with the Social Security Administration to discuss their use of causal "heat maps," and how the Board may design tools to better assess any need for targeted intervention to reduce U.S. Court of Appeals for Veterans Claims (CAVC) remands. Also, they reported that the Board has been attempting to engage outside contractor assistance to help improve its historical appeals data collection, analysis, and sampling methodologies. Further, Board officials stated that it is currently creating a comprehensive database of Appeals Modernization Act decisions that identifies the number of claims that are returned to the Board by CAVC, and the specific statute, regulation, or Court precedent, that led to the remand. It is developing programming that will generate a monthly report that will be given to Veteran Law Judges highlighting their decisions that were remanded by CAVC. We will consider closing this recommendation once the Board provides us with (1) examples of the programming and monthly reports given to VLJs highlighting their decisions that were remanded by CAVC, (2) a well-defined written plan for applying an evidence-based approach to assess the underlying causes for common errors and remands, and (3) examples of how it uses evidence to better target its interventions.
|
| Board of Veterans Appeals | The Chairman of the Board of Veterans' Appeals should study how to evaluate VLJ adjudicative decisional consistency. One option is to fold the development of this study into the Board's planned evaluation. (Recommendation 4) |
The Board disagreed with this recommendation. Board officials stated that it would be inappropriate to force consistency in VLJ decisions in a way that is inconsistent with codes of judicial conduct and other standards applicable to VLJs. They stated that there will and should be variance in how legal authority is applied to the individual facts and circumstances of each case appealed to the Board. Board officials further stated that VLJs should be independent and not subject to pressure and influence. They also stated that, as part of evaluating individual VLJs' adherence to performance standards, the Board evaluates the total number of decisions each VLJ has adjudicated and the manner in which they have adjudicated them, among other things. In addition, Board officials noted that the number of decisions remanded does not demonstrate whether such remands were warranted, and that the number of CAVC remands or reversals does not necessarily correlate with productivity, legal acumen, or even with performance. Finally, they said that the concept of "consistency" of decision-making among individual judges evaluating sets of facts and circumstances of each individual case is a difficult one to address at all levels of adjudication, not only at the Board, but also at CAVC. We acknowledge the importance of judicial independence, that some degree of variance is expected given that independence, and that variation is not necessarily an indicator of poor quality decision-making. In addition, the Board acknowledges that consistency among VLJs in the use of appropriate legal authority is appropriate. However, without studying consistency, the Board will remain unaware of whether systemic inconsistencies in VLJ decision-making exist, and the Board will be unable to target interventions, as appropriate. We continue to believe that the results of systematic study of VLJ adjudicative decisions for consistency could provide a basis for targeting interventions, such as training, to assist VLJs. As such, this recommendation is not intended to "force" consistency in any VLJ decisions, but rather is meant to help the Board assist VLJs. Our recommendation is worded to allow the Board the necessary latitude to implement it in a way that allows for the retention of judicial independence while meeting other goals it deems appropriate.
|