SSA Disability Decision Making: Additional Steps Needed to Ensure Accuracy and Fairness of Decisions at the Hearing Level
GAO-04-14
Published: Nov 12, 2003. Publicly Released: Nov 12, 2003.
Skip to Highlights
Highlights
Historically, the proportion of the Social Security Administration's (SSA) disability benefits claims that were approved has been lower for African-Americans than for whites. In 1992, GAO found that racial differences, largely at the Administrative Law Judge (ALJ) level, could not be completely explained by factors related to the decision-making process. This report examines how race and other factors influence ALJ decisions and assesses SSA's ability to ensure the accuracy and fairness of ALJ decisions.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Social Security Administration | To improve SSA's ability to ensure the accuracy and fairness of ALJ decisions, the agency should conduct ongoing analyses of ALJ decisions by race/ethnicity, as well as by other claimant groups (such as claimants with attorneys and nonattorneys, with translators, with low incomes, from certain regions and claimants who are men). In doing so, it should collect data on the types and sources of medical evidence in the claimant's file to better understand the agency's and attorney's role in the development of evidence. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
| Social Security Administration | To improve SSA's ability to ensure the accuracy and fairness of ALJ decisions, the agency should conduct ongoing analyses of ALJ decisions by race/ethnicity, as well as by other claimant groups (such as claimants with attorneys and nonattorneys, with translators, with low incomes, from certain regions and claimants who are men). In doing so, it should analyze differences in support (accuracy) rates, in addition to differences in allowance decisions. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
| Social Security Administration | To improve SSA's ability to ensure the accuracy and fairness of ALJ decisions, the agency should conduct ongoing analyses of ALJ decisions by race/ethnicity, as well as by other claimant groups (such as claimants with attorneys and nonattorneys, with translators, with low incomes, from certain regions and claimants who are men). In doing so, it should over-sample the selection of ALJ decisions by African-American claimants and, to the extent possible, other racial/ethnic groups to ensure that SSA has a sufficient number of cases to conduct analyses of ALJ decisions by race. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
| Social Security Administration | To improve SSA's ability to ensure the accuracy and fairness of ALJ decisions, the agency should conduct ongoing analyses of ALJ decisions by race/ethnicity, as well as by other claimant groups (such as claimants with attorneys and nonattorneys, with translators, with low incomes, from certain regions and claimants who are men). In doing so, it should publish methods used and results as part of its biennial reporting on the findings of its disability hearings quality review process. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
| Social Security Administration | To improve SSA's ability to ensure the accuracy and fairness of ALJ decisions, the agency should conduct ongoing analyses of ALJ decisions by race/ethnicity, as well as by other claimant groups (such as claimants with attorneys and nonattorneys, with translators, with low incomes, from certain regions and claimants who are men). In doing so, it should, if needed, take actions to correct and prevent any unwarranted differences in allowance and support rates among racial/ethnic and other claimant groups. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
| Social Security Administration | To further ensure the accuracy and fairness of ALJ decisions for various claimant groups, SSA should conduct in-depth investigations of cases (e.g., case studies) to better understand differences in ALJ allowances for certain claimant groups, including claimants with and without an attorney. The results of these investigations should also be published in the biennial reports. If needed, SSA should take actions to correct and prevent any unwarranted differences in allowance rates among these claimant groups. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
| Social Security Administration | To ensure that SSA uses a sample that is representative of all ALJ decisions in its quality assurance review, the agency should restructure its sampling process to incorporate cases that are appealed to SSA's Appeals Council in the quality assurance review sample. These appealed cases should be analyzed together with, rather than separate from, the rest of SSA's quality assurance sample. |
SSA addressed the shortcomings in its ongoing quality assurance process for ALJs, thereby improving its assessment of ALJ decision-making accuracy, by making appropriate changes to its data collection to ensure a representative sample. Specifically, the agency improved its collection of cases that are denied by ALJs and subsequently appealed to the Appeals Council--a subset that was formerly excluded from their sample. It then conducted statistical analyses of these cases, comparing the decision support rates for denied appealed cases with the support rates for cases that were not appealed to the Appeals Council. They also included the appealed cases with the other cases in the computation of ALJ accuracy rates. The results, as published in its biennial report, showed that the support rates of each group did not differ significantly.
|
| Social Security Administration | In light of the methodological complexities associated with analyzing ALJ decisions, SSA should establish an advisory panel comprised of external experts in a range of disciplines--including statistics/econometrics, design methodology, law, medicine, vocational training, and disability--to provide leadership, oversight, and technical assistance with respect to conducting these and other quality assurance reviews of ALJ decisions. |
In June 2003, SSA convened an Agency workgroup tasked with developing recommendations on how we can collect meaningful data on race and ethnicity so we will have the information needed to analyze any adverse effects of our program policies and rules. SSA considered this a very complicated project that affects SSA's major programmatic systems and expects this will be a multi-year project. Upon completion of this systems project, SSA noted that it should be in a better position to respond to the GAO recommendations. At the same time, SSA has indicated that it discontinued its biennial evaluation of a representative sample of ALJ decisions for accuracy, which may diminish SSA's ability to conduct future tests for fairness of ALJ decisions by race and other factors. Despite the merit of this recommendation, as of September 2008, SSA reported that it had not taken action on this recommendation.
|
Full Report
GAO Contacts
Topics
African AmericansData collectionData integrityDecision makingDisability benefitsFederal social security programsInternal controlsMinoritiesQuality assuranceRacial discriminationSex discriminationSocial security benefitsStatistical dataSupplemental security income