Social Security Disability: Additional Measures and Evaluation Needed to Enhance Accuracy and Consistency of Hearings Decisions
Highlights
What GAO Found
Allowance rates—the rate at which Social Security Administration (SSA) administrative law judges allowed disability benefits to be paid when claimants appealed—varied across judges, even after holding constant certain characteristics of claimants, judges, hearing offices, and other factors that could otherwise explain differences in allowance rates. Specifically, GAO estimated that the allowance rate could vary by as much as 46 percentage points if different judges heard a typical claim (one that was average in all other factors GAO analyzed). SSA officials said that this level of variation is not surprising, given the complexity of appeals and judicial discretion. Nonetheless, the variation declined by 5 percentage points between fiscal years 2007 and 2015 (see figure), a change officials attributed to enhanced quality assurance efforts and training for judges. GAO also identified various factors that were associated with a greater chance that a claimant would be allowed benefits. In addition to characteristics related to disability criteria, such as the claimant's impairment and age, GAO found that claimants who had representatives, such as an attorney or family member, were allowed benefits at a rate nearly 3 times higher than those without representatives. Other factors did not appear related to allowance rates, such as the percentage of backlogged claims in a hearing office.
Estimated Allowance Rates across Social Security Administrative Law Judges for Typical Disability Claims, Fiscal Years 2007-2015
Note: The range is the difference in allowance rates for judges at the 95th and 5th percentiles. We also identify the 75th and 25th percentiles. A typical claim had average values on all other factors we analyzed.
SSA has various reviews to monitor the accuracy and consistency of hearings decisions by administrative law judges, but some of these reviews may overlap and SSA has not systematically evaluated them. Specifically, SSA conducts five types of quality assurance reviews of hearings decisions, several of which have similar goals and may look at similar claims. SSA has not evaluated the efficiency or effectiveness of these reviews, despite spending at least $11 million on them in fiscal year 2016. Moreover, the agency has struggled to sustain all of its quality reviews due to competing priorities—two of the five reviews were curtailed in 2016 because SSA reassigned staff to help expedite claims decisions. By evaluating which quality assurance reviews are most effective and efficient in improving accuracy and consistency, SSA would be better positioned to meet its goals within its resources.
Why GAO Did This Study
Individuals who do not agree with the initial decision on a claim for Social Security disability benefits can ultimately appeal the decision by requesting a hearing before one of SSA's approximately 1,500 administrative law judges. However, the rate at which these judges have allowed benefits has varied, raising questions about the reasons for this variation. GAO was asked to review aspects of SSA's oversight of judges' decisions.
This report examines (1) to what extent allowance rates vary across administrative law judges, and factors associated with this variation; and (2) the extent to which SSA has processes to monitor the accuracy and consistency of hearings decisions.
GAO developed a statistical model to analyze SSA data on adult disability decisions made by administrative law judges from fiscal years 2007 through 2015, the most current data available at the time of GAO's analysis; reviewed relevant federal laws, regulations, and agency documents; and interviewed SSA officials and chief judges in SSA's 10 regions, as well as officials from organizations representing judges, disability claimants, and claimant representatives.
Recommendations
GAO is making two recommendations, including that SSA systematically evaluate its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them. SSA concurred and plans to address them through a comprehensive assessment of its oversight.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Social Security Administration | The Commissioner of SSA should develop a set of public performance measures, to include accuracy and consistency, as well as timeliness, of administrative law judges' (ALJ) disability decisions. SSA could consider whether existing quality review or monitoring efforts could provide suitable data for such measures. (Recommendation 1) |
SSA agreed with this recommendation and stated that the agency would address it as part of a comprehensive assessment and refinement of its oversight roles and processes. As of August 2024, SSA officials reported that the agency has expectations, measures, and processes in place to ensure administrative law judges comply with policy and adjudicate a sufficient number of cases in a quality and timely manner. Further, officials noted that they continuously evaluate whether there are other metrics and methodologies that can help them manage administrative law judges' performance. However, the agency has decided not to make those performance expectations and measures available to the public and did not provide additional information on the types of measures that are in place. By not sharing performance information on the accuracy and consistency of hearings-level decisions, SSA may not be providing the public with adequate information on progress toward its objective to improve the quality, consistency, and timeliness of disability decisions.
|
| Social Security Administration | The Commissioner of SSA should systematically evaluate the efficiency and effectiveness of its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them to ensure strategic use of resources. Such steps could include enhancing collaboration where reviews overlap or only conducting the reviews that are most efficient and effective in achieving agency goals for improving accuracy and consistency of ALJ disability decisions. (Recommendation 2) |
SSA systematically evaluated quality assurance reviews over a period of several years and took steps to ensure more strategic use of resources, including eliminating one duplicative review of hearings-level decisions. As of August 2022, the agency conducted an inventory of existing reviews and studies, including an in-depth look at the methodologies, review standards employed, and required resources. As a first step in this effort, in fiscal year 2019, SSA conducted a review of favorable and unfavorable hearing decisions to provide statistically valid results on the quality of decisions on appealed disability claims at the national level. Officials said they compared the results of analyzing this national sample to those from other existing ALJ quality assurance reviews, and the comparison showed that the results were consistent across reviews. As a result, they eliminated the Office of Quality Review's review of hearings decisions. That office will instead focus on reviewing the accuracy of decisions on initial disability claims as well as reconsiderations, whereas the Division of Quality, within the Office of Appellate Operations, will continue to review hearings level decisions. Although officials said that eliminating this hearings-level review would not generate direct cost savings, they were able to redirect staff resources to other projects. In addition, officials reported that they created a singular quality metric that focuses on "performance accuracy," as opposed to metrics, such as error rates, that differed across reviews. SSA will use this metric across offices and levels of adjudication. Such efforts should result in more efficient and effective use of SSA's resources in achieving the agency's goals for improving accuracy and consistency of ALJ decisions.
|