GAO-04-656, Social Security Administration: More Effort Needed to Assess Consistency of Disability Decisions


This is the accessible text file for GAO report number GAO-04-656 
entitled 'Social Security Administration: More Effort Needed to Assess 
Consistency of Disability Decisions' which was released on July 02, 
2004.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Report to the Chairman, Subcommittee on Social Security, Committee on 
Ways and Means, House of Representatives:

United States General Accounting Office:

GAO:

July 2004:

SOCIAL SECURITY ADMINISTRATION:

More Effort Needed to Assess Consistency of Disability Decisions:

GAO-04-656: 

GAO Highlights:

Highlights of GAO-04-656, a report to the Chairman, Subcommittee on 
Social Security, Committee on Ways and Means, House of Representatives: 

Why GAO Did This Study:

Each year, about 2.5 million people file claims with the Social 
Security Administration (SSA) for disability benefits. If the claim is 
denied at the initial level, the claimant may appeal to the hearings 
level. The hearings level has allowed more than half of all appealed 
claims, an allowance rate that has raised concerns about the 
consistency of decisions made at the two levels. To help ensure 
consistency, SSA began a “process unification” initiative in 1994 and 
recently announced a new proposal to strengthen its disability 
programs. This report examines (1) the status of SSA’s process 
unification initiative, (2) SSA’s assessments of possible 
inconsistencies in decisions between adjudication levels, and (3) 
whether SSA’s new proposal incorporates changes to improve consistency 
in decisions between adjudication levels.

What GAO Found:

SSA has only partially implemented its process unification initiative. 
Although the agency initially made improvements in its policies and 
training intended to address inconsistency in decisions made at the 
two adjudication levels, it has not continued to actively pursue these 
efforts. Further, as part of this initiative, the agency implemented a 
review of hearings level decisions to identify ways to improve 
training and policies, but no new improvements were made as a result of 
the review. Finally, the agency began tests of two process changes 
intended to improve the consistency of decision making between the two 
adjudication levels. One test, which is ongoing, was not well designed 
and therefore will not provide conclusive results. The other test was 
abandoned because of implementation difficulties. 

SSA’s assessments have not provided a clear understanding of the 
extent and causes of possible inconsistencies in decisions between 
adjudication levels. The two measures SSA uses to monitor 
inconsistency of decisions have weaknesses, such as not accounting for 
the many factors that can affect decision outcomes, and therefore do 
not provide a true picture of the changes in consistency. Furthermore, 
SSA has not sufficiently assessed the causes of possible inconsistency. 
For example, SSA conducted an analysis in 1994 that identified 
potential areas of inconsistency, but it did not employ more 
sophisticated techniques—such as multivariate analyses, followed by in-
depth case studies—that would allow the agency to identify and address 
the key areas and leading causes of possible inconsistency. SSA has yet 
to repeat or expand upon this 10-year-old study. 

SSA’s new proposal incorporates changes intended to improve 
consistency in decisions between levels. However, challenges may hinder 
its implementation. Most stakeholder groups for adjudicators and 
claimant representatives told us that a number of aspects of the 
proposal hold promise for improving consistency. These included one 
change, being tested as part of the process unification initiative, 
that requires state adjudicators to more fully develop and document 
their decisions, as well as several new changes, such as providing both 
adjudication levels with equal access to medical expertise. However, 
stakeholder groups also told us that insufficient resources and other 
obstacles might hinder the implementation of some changes. Adding to 
uncertainties about the proposal’s overall success is its dependence 
on a new electronic folder system that would allow cases to be easily 
accessed by various adjudicators across the country. However, this 
technically complex project has not been fully tested.

What GAO Recommends:

To build an effective strategy to address possible inconsistencies in 
its decisions, we recommend that SSA quickly expand its assessment of 
inconsistency by implementing several specific enhancements. In its 
comments, SSA had some reservations concerning our findings, 
conclusions, and recommendations but agreed to pilot one 
recommendation and consider the others as it refines its new proposal. 
We continue to believe that SSA should implement our recommendations 
without delay to ensure the effectiveness of its new proposal.

www.gao.gov/cgi-bin/getrpt?GAO-04-656.

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Robert Robertson, 202-
512-7215, robertsonr@gao.gov.

[End of section]

Contents:

Letter:

Results in Brief:

Background:

SSA Has Partially Implemented Its Process Unification Initiative:

SSA Lacks a Clear Understanding of the Extent and Causes of 
Inconsistency between Levels:

SSA's New Proposal Incorporates Efforts to Improve the Consistency of 
Decisions, but Challenges May Impede Successful Implementation:

Conclusions:

Recommendations:

Agency Comments and Our Evaluation:

Appendix I: Excerpt of SSA's Testimony Announcing Its New Proposal to 
Improve Its Disability Decision-Making Process:

Appendix II: Comments from the Social Security Administration:

GAO Comments:

Appendix III: GAO Contacts and Staff Acknowledgments:

GAO Contacts:

Staff Acknowledgments:

Tables:

Table 1: SSA's Process Unification Efforts:

Table 2: Newly Proposed Changes to the Disability Decision-Making 
Process:

Figures:

Figure 1: SSA's Disability Decision-Making Process and Outcomes for 
Fiscal Year 2003:

Figure 2: SSA Allowances at Initial Level versus Hearings Level, by 
Proportion of Allowances:

Abbreviations:

AeDib: Accelerated Electronic Disability System:

ALJ: Administrative Law Judge:

DDS: Disability Determination Services:

DI: Disability Insurance:

OASDI: Old-Age, Survivors and Disability Insurance:

OHA: Office of Hearings and Appeals:

OQA: Office of Quality Assurance and Performance Assessment:

RO: Reviewing Official:

SSA: Social Security Administration:

SSI: Supplemental Security Income:

United States General Accounting Office:

Washington, DC 20548:

July 2, 2004:

The Honorable E. Clay Shaw, Jr.: 
Chairman, Subcommittee on Social Security:
Committee on Ways and Means: 
House of Representatives:

Dear Mr. Chairman:

The Social Security Administration (SSA) is the nation's largest 
provider of income assistance to individuals with disabilities, paying 
$91 billion in federal benefits to 11.4 million beneficiaries with a 
disability and their families in 2003.[Footnote 1] Each year, about 2.5 
million people file claims with SSA for disability benefits. State 
agencies called Disability Determination Services (DDS) decide whether 
claimants meet SSA's definition of disability by applying SSA's 
decision-making criteria. If a DDS ultimately decides, after an initial 
determination and then a reconsideration of this decision, that a 
claimant does not meet SSA's definition for disability, the claimant 
may appeal to the hearings level, where an SSA Administrative Law Judge 
(ALJ) reviews the claim to decide if the claimant should be allowed 
benefits. About one-third of disability claims denied at the state 
level were appealed to the hearings level; of these, SSA's ALJs have 
allowed over one-half, with annual allowance rates fluctuating between 
58 percent and 72 percent since 1985. While it is appropriate that some 
appealed claims, such as those in which a claimant's impairment has 
worsened and prohibits work, be allowed benefits, representatives from 
SSA, the Congress, and interest groups have long been concerned that 
the high rate of claims allowed at the hearings level may indicate that 
decision makers at the two levels are interpreting and applying SSA's 
criteria differently. If this is the case, adjudicators at the two 
levels may be making inconsistent decisions that result in similar 
cases receiving dissimilar decisions.

Concerned about the possibility that adjudicators are making 
inconsistent decisions, SSA embarked on a "process unification" 
initiative in 1994 with the goal of ensuring that adjudicators at both 
levels consistently apply SSA's policy guidance and make similar 
decisions on similar cases. Partly on the basis of early studies of 
potential causes of inconsistent decisions, SSA included in its process 
unification initiative efforts to provide consistent guidance to all 
adjudicators, clarify policy, provide training, test potential process 
changes, and perform a new quality review of allowances decided by 
ALJs. However, SSA continues to face challenges in its efforts to 
provide consistent disability decisions. These challenges, and others 
associated with modernizing its disability programs, contributed to our 
decision to include federal disability programs on our list of high-
risk government programs.[Footnote 2] In September 2003, SSA's 
Commissioner unveiled a new proposal that laid out the vision of a 
long-term strategy for improving the disability decision-making process 
and helping people with disabilities return to work. Several of the 
changes in the new proposal are intended to improve the accuracy, 
timeliness, and consistency of decisions, such as having the DDS 
decision makers more fully develop and document their decisions, 
providing for centralized quality review of all decisions, and 
providing both adjudication levels equal access to medical expertise.

In response to your interest in the effectiveness of SSA's past and 
future efforts to improve and assess the consistency of decisions 
between levels, we evaluated these agency efforts. Specifically, we 
examined (1) the status of SSA's process unification initiative, (2) 
SSA's assessments of possible inconsistencies in decisions between 
adjudication levels, and (3) whether SSA's new proposal incorporates 
changes to improve consistency in decisions between adjudication 
levels.

To assess the extent to which SSA has implemented its planned 
activities under the initiative, we evaluated agency documentation 
describing SSA's process unification efforts. To evaluate SSA's efforts 
to assess consistency in decisions between levels, we interviewed 
officials from SSA's Office of Quality Assurance and Performance 
Assessment (OQA) and reviewed summary data and reports from SSA's 
quality assurance and performance management systems, including 
findings from SSA's Disability Hearings Quality Review, which is a 
quality review of DDS adjudicators' and ALJs' decisions and the 
associated case files. While we evaluated SSA's methods and approaches 
for assessing the consistency of decisions, we generally did not trace 
figures cited by SSA back to their original source documents.[Footnote 
3] To further evaluate SSA's process unification efforts and its new 
proposal, we interviewed selected officials who have firsthand 
knowledge about these issues, including SSA officials, staff of the 
Social Security Advisory Board, and leaders of SSA stakeholder groups 
representing state and hearings office adjudicative staff and claimant 
attorneys.[Footnote 4] We also reviewed recent testimony and other 
documents on the consistency of SSA's decisions from the Social 
Security Advisory Board and other stakeholder groups. We conducted our 
work between February 2003 and March 2004 in accordance with generally 
accepted government auditing standards. Our ability to evaluate SSA's 
new proposal has been limited by a lack of detailed information, such 
as specific information on how changes will be implemented and their 
costs, because the agency is still in the process of developing and 
refining its proposal.

Results in Brief:

SSA has partially implemented its process unification initiative. At 
the beginning of its process unification efforts, the agency took 
decisive steps by issuing clarifying guidance in a number of key policy 
areas to all adjudicators. However, SSA ultimately abandoned its plans 
to update older policy guidance and to provide a single policy guide 
for both initial-and hearings-level adjudicators, concluding that these 
efforts would not be cost-effective. Similarly, while SSA initially 
provided extensive process unification training to adjudicators at both 
levels, the scope of SSA's training efforts on this issue has since 
diminished. Also as part of this initiative, SSA implemented a quality 
review of ALJ allowances, in part to identify the need for new policies 
and training to improve consistency, but no new improvements were made 
as a result of findings from the review. Finally, the agency began two 
tests to determine if process changes would improve the consistency of 
decisions. However, the test of having DDS adjudicators more fully 
document decisions--which is ongoing--will, because of design flaws, be 
unable to provide conclusive information on how this change might 
affect the consistency of decisions. SSA abandoned its other test of 
having the initial level reevaluate appealed cases for which new 
medical information was submitted prior to the hearing, because of 
several major difficulties encountered during testing, such as 
difficulty identifying cases to be sent back to the initial level.

SSA's assessments have not provided the agency with a clear 
understanding of the extent and causes of possible inconsistencies in 
decisions between adjudication levels. The two measures SSA uses to 
monitor changes in the extent of inconsistency of decisions have 
weaknesses, such as not accounting for the many factors that can affect 
decision outcomes, and therefore do not provide a true picture of 
changes in consistency. SSA has also not sufficiently assessed causes 
of possible inconsistency. For example, 10 years ago SSA analyzed cases 
in which reviewers representing the initial and hearings level 
disagreed over the final decision and identified two prevalent areas of 
disagreement: assessments of claimants' mental impairments and 
assessments of claimants' ability to work. Although SSA continues to 
collect information that would support this analysis, it has not 
repeated this initial effort, nor has it expanded on it by employing 
more sophisticated techniques--such as multivariate analyses, followed 
by in-depth case studies--that would allow the agency to identify and 
address the key areas and leading causes of possible inconsistency.

SSA's recent proposal to improve the disability decision-making process 
incorporates efforts intended to address inconsistencies in decisions 
between levels. However, challenges may hinder the implementation of 
the proposal. Most stakeholder groups for SSA adjudicators and claimant 
representatives told us that several aspects of the proposal hold 
promise for improving consistency between adjudication levels. For 
example, they thought that requiring state adjudicators to more fully 
develop and document their decisions--a process change that SSA is 
still testing under its process unification initiative--might improve 
consistency. In addition, they said that other new ideas proposed by 
SSA, such as centralizing medical expertise to give both adjudication 
levels equal access to it, may also improve consistency. However, 
stakeholders told us that insufficient resources and other obstacles 
might hinder the implementation of some changes. For example, they were 
concerned that, as has happened in the past, limited resources would 
hinder state adjudicators' ability to more fully document their 
decisions. Stakeholders also questioned SSA's ability to attract and 
retain sufficient medical expertise. Adding to uncertainties about the 
proposal is its dependence upon the successful implementation of a new 
electronic folder system that would allow cases to be easily accessed 
by adjudicators across the country. However, this technically complex 
project has not been fully tested.

GAO is making several recommendations in this report to the 
Commissioner of Social Security to improve SSA's methods for assessing 
the inconsistency between DDS and ALJ decisions and thereby build a 
more effective strategy to address potential inconsistencies. In 
commenting on the draft of this report, SSA had several reservations 
concerning the report's findings, conclusions, and recommendations. 
Although SSA agreed to pilot one of our recommendations, the agency 
believes our other recommendations need to be reevaluated as the design 
of its Commissioner's new approach to disability decision making 
matures. We continue to believe that SSA should begin implementing our 
recommendations without delay so that it has the critical information 
needed to build a new approach to decision making that will improve the 
consistency of decisions between adjudication levels.

Background:

SSA operates the Disability Insurance (DI) and Supplemental Security 
Income (SSI) programs--the two largest programs providing cash benefits 
to people with disabilities. The law defines disability for both 
programs as the inability to engage in any substantial gainful activity 
by reason of a severe physical or mental impairment that is medically 
determinable and is expected to last at least 12 months or result in 
death. The programs have grown substantially, from 10.7 million 
beneficiaries and $61 billion in benefits in 1995 to 11.4 million 
beneficiaries and $91 billion in federal benefits to individuals with 
disabilities in 2003.[Footnote 5] While disability benefits account for 
only 15 percent of SSA's total benefit payments for its Old-Age, 
Survivors and Disability Insurance (OASDI) programs, administering the 
disability benefits accounted for 45 percent of the agency's annual 
administrative expenses.[Footnote 6] The relatively high cost of 
administering the DI program reflects the complex and demanding nature 
of making disability decisions. SSA estimates that the cost of the 
disability programs will rise substantially in the near future as the 
baby boom generation reaches its disability-prone years.

The disability determination process begins at a field office, where an 
SSA representative determines whether a claimant meets the programs' 
non-medical eligibility criteria. Claims meeting these criteria are 
forwarded to the state DDS to determine if a claimant meets the 
agency's definition of disability. At the DDS, the disability examiner 
takes the lead, or works as a team with the medical or psychological 
consultants, to analyze a claimant's documentation, gather additional 
evidence as appropriate, and approve or deny the claim. A denied 
claimant may ask the DDS to reconsider its finding, at which point a 
different DDS team reviews the claim. If the claim is denied again, the 
claimant may appeal the determination to SSA's Office of Hearings and 
Appeals (OHA), where it will be reviewed by an ALJ. The ALJ usually 
conducts a hearing in which the claimant and others may testify and 
present new evidence.[Footnote 7] In making the disability decision, 
the ALJ uses information from the hearing and from the state DDS, 
including the findings of the DDS medical consultant. A claimant whose 
appeal is denied may request a review by SSA's Appeals Council and, if 
denied again, may file suit in federal court. Figure 1 provides an 
overview of SSA's disability decision-making process and outcomes for 
2003.

Figure 1: SSA's Disability Decision-Making Process and Outcomes for 
Fiscal Year 2003:

[See PDF for image]

Notes:

The data provided by SSA did not include the number or rate of claims 
appealed to the next decision step. Without this information, we cannot 
determine the extent to which claimants appealed or abandoned their 
denied claims.

Twenty-five percent of the initial DDS determinations are subject to an 
alternative process that does not include the reconsideration step.

Under certain specified circumstances, ALJs and Appeals Council judges 
can dismiss a claim. For example, an ALJ can dismiss a claim if the 
claimant's request for a hearing is not timely and lacks a good cause 
for the delay.

The Appeals Council can remand a claim by returning it to an ALJ for 
further proceedings and a new hearing decision.

Because of rounding, decisional outcomes may not equal 100 percent.

[End of figure]

SSA uses a sequential evaluation process when determining disability. 
First, SSA field office representatives determine whether a claimant is 
performing substantial gainful work.[Footnote 8] If not, DDS or ALJ 
adjudicators will assess the severity of a claimant's medical 
condition(s) to determine whether it meets or equals the criteria in 
SSA's regulations (commonly referred to as the medical listings). For a 
claimant whose conditions do not meet or equal the listings, 
adjudicators then focus on the functional consequences of the 
claimant's medically determined impairments--that is, whether the 
claimant can perform work he or she has done in the past, and, if not, 
whether the claimant can perform other work in the national economy.

Concerns about the rate of appeals for hearings, ALJs' allowance rates, 
and the accuracy and consistency of ALJ decisions led the Congress to 
direct SSA to conduct a study in 1980 to determine the extent to which 
hearings decisions conformed to legal requirements and binding SSA 
policy.[Footnote 9] Since the allowance rates at the hearings level 
could be influenced by many factors, such as the introduction of new 
evidence, the purpose of the 1980 study was to present the same 
evidence on cases to different reviewers representing different 
adjudication levels. In determining the extent to which decision makers 
agreed on whether to allow or deny benefits, the study concluded that 
different levels of decision makers had significantly different 
allowance rates. Specifically, the ALJs decided to allow 64 percent of 
the cases, whereas the SSA's central office quality assurance 
reviewers, comprising medical consultants and disability examiners, 
decided that only 13 percent of cases should be allowed. The study 
identified several possible causes of the disparity, including 
inconsistency in the standards and procedures, interpretation of the 
standards, and weight given to the evidence. The study also found that 
disability decisions are complex and necessarily involve some degree of 
subjectivity by adjudicators.

To help address concerns raised by this and other studies,[Footnote 10] 
SSA began its process unification efforts to ensure that both levels 
more consistently interpreted and applied SSA's policy guidance. SSA's 
plans for its process unification initiative were part of SSA's larger 
effort to redesign its disability claims process and were modified over 
time. SSA's process unification plans included six major efforts, as 
described in table 1.

Table 1: SSA's Process Unification Efforts:

Process unification effort: Develop a single presentation of policy; 
Planned activities: Publish all new guidance in the same wording to all 
adjudicators; Revise older guidance; Provide one policy manual for 
adjudicators.

Process unification effort: Create additional policy guidance; 
Planned activities: Publish nine process unification rulings to clarify 
policy areas contributing to inconsistent DDS and ALJ decisions that 
are binding on all SSA adjudicators; Publish regulations and 
instructions to clarify selected process unification rulings, including 
(1) the weight to be given to the treating physician's opinion when 
evaluating a claim, (2) the weight to be given to the DDS medical 
consultant's opinion, and (3) the evaluation of the residual functional 
capacity for claimants who are limited to performing less than a full 
range of sedentary work; Publish a regulation to clarify the agency's 
process for acquiescing to court decisions and an action plan to 
implement the regulation.

Process unification effort: Provide training; 
Planned activities: Provide ongoing training to all adjudicators to 
increase their understanding of the three most complex disability 
areas--assessing (1) the opinion evidence from physicians and others, 
(2) the claimant's symptoms, and (3) the claimant's remaining capacity 
to work (i.e., residual functional capacity).

Process unification effort: Fully document DDS decisions; 
Planned activities: Test procedures for fully developing and 
documenting DDS decisions to determine their impact on DDS accuracy, 
allowance rates, and other aspects of the claims process.

Process unification effort: Remand cases awaiting a hearing to a DDS 
when new medical evidence is received; 
Planned activities: Test the impact of returning cases to a DDS that 
are awaiting a hearing and have received new medical information to 
determine if a reevaluation of the cases by medical consultants 
residing at a DDS will help to improve the consistency of decisions 
between levels, with a 1-year goal of remanding 100,000 cases.

Process unification effort: Review selected ALJ allowance decisions; 
Planned activities: Conduct a joint Appeals Council and OQA review of 
ALJ allowances that have not yet received a final decision (i.e., the 
claimant has not been awarded benefits) to identify policy areas 
leading to inconsistent decisions between levels. 

Source: SSA documents and prior GAO reports.

[End of table]

In 1997, we reported on the possible reasons for the inconsistency of 
decisions between the initial and hearings levels. Our report found 
that differences in state DDSs' and ALJs' views on the claimants' 
functional abilities was a key factor in explaining why ALJs allowed 
benefits on appealed cases.[Footnote 11] We also reported that poorly 
documented state DDS evaluations of the claims were of limited use to 
ALJs and SSA quality reviews did not focus on identifying inconsistency 
in decisions. To support SSA's process unification efforts, the report 
recommended that SSA, using available systems and data collected so 
far, move quickly ahead to implement its quality assurance initiative 
to provide consistent feedback to DDS and ALJ adjudicators as soon as 
possible. In addition, we recommended that SSA expand its effort to 
return cases to a DDS for review when new evidence is introduced on 
appeal. Last, we recommended that SSA set goals for measuring the 
effectiveness of process unification in reducing inconsistent 
decisions.

More recently, the Social Security Advisory Board issued a 2001 report 
that identified many factors that could potentially affect the overall 
consistency of disability decision making between adjudication 
levels.[Footnote 12] Some of the factors the board suggested as 
potentially affecting consistency included:

* the fact that most claims are decided based on a paper review of case 
evidence without face-to-face contact with an adjudicator until a 
claimant has an ALJ hearing,

* involvement of attorneys and other claimant representatives at the 
ALJ hearing,

* the fact that claimants are allowed to introduce new evidence and 
allegations at each stage of the appeals process,

* differences in quality assurance procedures applied to initial-and 
hearings-level decisions,

* differences in the training given to ALJs and state examiners, and:

* lack of clear and unified policy guidance from SSA.

Despite SSA's process unification efforts and related studies to 
improve the consistency of decisions, recent ALJ allowance rates--which 
declined after process unification began, but started increasing in 
1999 to reach 61 percent in fiscal year 2003--still raise questions as 
to whether initial-and hearings-level decision makers are consistently 
applying the agency's guidance. In addition to inconsistent application 
of SSA's policy guidance, there are several other reasons why a large 
number of ALJ allowances are made. For example, some ALJ allowances 
should be expected because, by law, cases can remain open throughout 
the hearings process, allowing new evidence to be submitted that may 
not have been available to the state adjudicators. Such new evidence 
could show that the claimant's condition has worsened and prohibits 
work. Also, SSA's decision-making criteria require that a great deal of 
professional judgment be applied. As a result, some allowances at the 
hearings level could simply reflect the differing judgments of two 
adjudicators reviewing a case. While a claimant's deteriorating health, 
changes in the characteristics of a claim over time, and the complexity 
of disability decisions may help to explain some of the ALJ allowances, 
studies have not sufficiently explained why consistently over half the 
cases appealed to the hearings level are allowed. Instead, studies 
indicate that systemic differences in the assessment of claims at both 
adjudication levels are contributing to the ALJ allowance rate. For 
example, our 1997 report noted a difference in state DDSs' and ALJs' 
views on the claimant's functional abilities was a key factor in 
explaining why ALJs allowed cases on appeal.

Inconsistency in decisions may create several problems. High hearings 
allowance rates may create the perception that the hearings level is 
applying SSA's criteria less strictly than the initial level and create 
an incentive for claimants to appeal to an ALJ for a more favorable 
decision.[Footnote 13] If deserving claimants must appeal to the 
hearings level for benefits, this situation increases the burden on 
claimants, who must wait, on average, almost a year for a hearing 
decision and frequently incur extra costs to pay for legal 
representation.[Footnote 14] In addition, to the extent that the ALJ 
allowance rates include inappropriate allowances, SSA could be 
incurring unwarranted program costs. Although SSA has tried to address 
these problems, its inability to resolve them has contributed to our 
decision to include federal disability programs on our list of high-
risk government programs.[Footnote 15]

Renewing its effort to address long-standing and critical problems with 
the disability programs, SSA's Commissioner recently announced a new 
proposal to improve these programs. (See app. I for an excerpt of the 
announcement that describes the newly proposed decision-making 
process.) In addition to proposing demonstration projects that provide 
work incentives and supports to help people with disabilities return to 
work, SSA has proposed significant changes to both the process of 
adjudicating disability claims and the structure and management of the 
agency's quality management system to improve the timeliness, accuracy, 
and consistency of the disability decision-making process. The agency 
believes that several of these changes will help to improve consistency 
between DDS and ALJ decisions. For example, SSA plans to provide more 
centralized end-of-line quality reviews. According to SSA, the proposed 
quality reviews should help to hold adjudicators more accountable for 
their decisions and ensure that they consistently apply SSA's policies 
as well as help the agency detect and amend those policy areas leading 
to inconsistent decisions. Table 2 provides a description of SSA's 
proposed changes to improve the disability decision-making process.

Table 2: Newly Proposed Changes to the Disability Decision-Making 
Process:

Proposed change: Centralize medical expertise from the states into 
regional offices and organize experts by medical specialty; 
Purpose: To make quick decisions on initial claims for individuals who 
are obviously disabled (e.g., those with aggressive cancers and end-
stage renal disease) and to provide equal access to medical expertise 
for all adjudicators and more consistent medical review of claims.

Proposed change: Require DDSs to fully document and explain the basis 
for their decisions; 
Purpose: To hold DDS adjudicators accountable for providing higher-
quality and more consistent decisions and to have better information 
to identify and correct problem areas leading to incorrect decisions.

Proposed change: Eliminate DDS reconsiderations; 
Purpose: To reduce time taken to process claims and avoid having 
claimants who are disabled drop out of the disability claims process.

Proposed change: Create a reviewing official position; 
Purpose: To evaluate all appealed DDS decisions and prepare either (1) 
an on-the- record allowance, (2) a recommended disallowance detailing 
reasons for a denial, or (3) a prehearing report outlining the 
evidence needed to fully support the claim.

Proposed change: Require ALJs to address the reviewing officials' 
reports; 
Purpose: To improve accountability and consistency by having ALJs 
either (1) describe in detail the basis for rejecting the reviewing 
official's recommended disallowance or (2) provide detailed information 
on the evidence used to support allowances made in response to the 
reviewing official's prehearing report.

Proposed change: Eliminate the claimant's ability to appeal ALJ 
decisions to the Appeals Council; 
Purpose: To reduce processing time for claims and use resources more 
effectively.

Proposed change: Use of in-line quality control; 
Purpose: To build quality into each level of the decision-making 
process rather than rely too heavily on case reviews performed by end-
of-line quality reviewers or by the next adjudication level in response 
to claimants appealing denied decisions.

Proposed change: Use of centralized quality control unit; 
Purpose: To perform end-of-line reviews for all decisions, thereby 
replacing regional reviews of DDS decisions, and to provide a more 
balanced review of both DDS and ALJ decisions to ensure that all 
adjudicators are consistently applying SSA's policies and to detect 
and amend those policy areas leading to inconsistent decisions.

Proposed change: Create oversight panels that include two ALJs and one 
Administrative Appeals Judge from the Appeals Council; 
Purpose: To review and either affirm or reverse an ALJ decision 
referred by the centralized quality control unit when the unit 
disagrees with the ALJ decision. 

Sources: SSA documents and agency interviews.

Note: Under the new proposal, when the agency's centralized quality 
control unit and oversight panel review an ALJ decision, a claimant 
cannot submit any new information to be considered by the agency.

[End of table]

SSA does not plan to implement its proposed changes before it has 
successfully implemented its Accelerated Electronic Disability (AeDib) 
system. This major initiative should allow adjudication staff in states 
and throughout the agency, regardless of geographic location, to access 
case information electronically through the use of an electronic 
disability folder. The initiative is intended to reduce delays that 
result from mailing, locating, and organizing paper folders. SSA also 
expects this new system to provide critical management information for 
analyzing and reducing inconsistencies in disability decisions. SSA is 
implementing the new system and plans to give adjudicators time to 
adjust to this change before implementing its new proposal. SSA's 
implementation of the new proposal will therefore be no earlier than 
October 2005. In the meantime, SSA continues to discuss the proposal 
with stakeholders and plans to further refine it before implementation.

SSA Has Partially Implemented Its Process Unification Initiative:

SSA has partially implemented its process unification initiative. 
Although the agency initially made improvements in its policies and 
training intended to improve the consistency of decisions between 
adjudication levels, it has not continued to actively pursue these 
efforts. As part of the initiative, the agency also implemented a 
review of ALJs' allowance decisions to identify additional ways to 
improve training and policies, but no new changes were made as a result 
of findings from the review. Finally, the agency also began two tests 
of process changes to help improve the consistency of decisions, but 
one ongoing test with design problems is not likely to lead to any 
conclusive results and the other test has been abandoned.

SSA Made Early Progress Improving Policies and Training, but Has Not 
Actively Pursued These Efforts:

While SSA initially made progress carrying out efforts to improve 
policies and training to better ensure the consistency of decisions, 
the agency has not continued to actively pursue these efforts. SSA 
quickly accomplished most of its planned efforts to clarify policy 
guidance. In 1996, SSA issued nine process unification rulings to 
clarify policy areas it found to be contributing to inconsistent 
decisions. For example, one ruling provided all adjudicators with 
guidance on how to weigh and document their evaluation of the treating 
physician's opinions when making a disability decision.[Footnote 16] 
SSA successfully went through the regulatory process several years 
later and published three new regulations to strengthen its process 
unification rulings, but was unable to agree on a fourth regulation 
regarding the weight to be given to the treating physician's opinion 
when evaluating a claim.[Footnote 17]

SSA planned to develop a single presentation of policy guidance to 
replace the different sources used by each level, but has since 
abandoned full implementation of these plans in favor of a more limited 
approach. DDS adjudicators currently follow a detailed set of policy 
and procedural guidelines,[Footnote 18] whereas ALJs rely directly on 
statutes, regulations, and rulings for guidance in making disability 
decisions. To help ensure that inconsistent guidance was not 
contributing to inconsistent DDS and ALJ decisions, SSA began issuing 
guidance in the same wording to all adjudicators in 1996. Although SSA 
had also planned to address differences in policy guidance issued 
before 1996 and to eventually combine existing adjudication policy 
documents into a single document, it ultimately decided not to take 
these additional steps. According to SSA, further efforts to unify the 
policy guidance used by both levels would be a massive undertaking and 
not worth the cost because the guidance issued since 1996 had already 
addressed important policy areas that were leading to inconsistent 
decisions. While some stakeholder groups representing adjudicators 
tended to agree with SSA's position, the Social Security Advisory Board 
and other groups still believe the agency should take additional steps 
to provide a unified policy guide to all adjudicators. Instead of 
creating one policy manual for all adjudicators, SSA told us that it 
plans to undertake a comprehensive effort to evaluate and improve its 
disability policies to make them less susceptible to differing 
interpretations and to ensure they are up to date. A more comprehensive 
approach could address key weaknesses in SSA's disability program that 
we previously highlighted in our performance and accountability series, 
and thereby help to modernize federal disability programs to better 
meet the needs of Americans with disabilities.

Early on, SSA also provided extensive cross-training of DDS and ALJ 
adjudicators, although the scope of its efforts has since diminished. 
To help all adjudicators understand how to appropriately apply process 
unification rulings, SSA provided extensive and mandatory training in 
1996 and 1997 to 15,000 disability adjudicators (including DDS 
examiners, physicians, ALJs, and quality assurance staff). The training 
was provided to adjudicators at all levels of the process in three of 
the most complex disability areas--assessment of symptoms, treatment of 
expert opinions, and assessment of claimants' remaining capacity to 
work (i.e., residual functional capacity). While this training was 
intended to be ongoing, SSA's training efforts have diminished 
significantly since 1997. Stakeholder groups representing DDS 
adjudicators told us that SSA's training does not sufficiently cover 
process unification issues. In addition, our review of DDS and OHA 
participation in video training revealed inconsistent participation in 
training by adjudicators. To provide ongoing training to both 
adjudication levels and other components involved in the claims 
process, SSA has used interactive video technology. Almost all the 
state DDS sites and about 85 percent of OHA offices have this 
technology. However, in reviewing participation for two recent courses, 
we found for those sites with this interactive technology only 31 
percent of DDS sites and 16 percent of OHA sites logged on for a course 
on the role of consultative examinations, and 18 percent of DDS sites 
and 4 percent of OHA sites logged on for a monthly disability hour 
training class.[Footnote 19] According to SSA, neither DDS nor OHA 
adjudicators are generally required to attend courses. In line with 
these findings, our recent report on the human capital challenges 
facing DDSs found gaps in the key knowledge and skills of their 
adjudicators in the same areas SSA had earlier identified as critical 
to making consistent decisions, and we recommended that SSA work with 
DDSs to close these gaps.[Footnote 20]

Despite SSA's early efforts to improve policy guidance and provide 
training, stakeholder groups representing state adjudicators told us 
that many states are not performing the additional development and 
documentation of decisions required by the process unification rulings. 
They also told us that the rulings have added significantly to the 
time, complexity, and subjectivity of the decision-making process, 
while insufficient resources have limited their ability to fully 
implement the rulings' requirements. In addition, claimant lawsuits 
against three state DDSs have alleged that DDS adjudicators were not 
following SSA's rulings or other decision-making guidance. In settling 
these lawsuits, SSA agreed to have these states fully develop and 
document cases. However, according to DDS stakeholder groups, SSA has 
not ensured that states have sufficient resources to meet ruling 
requirements, which they believe may lead to inconsistency in decisions 
among states. Furthermore, SSA's quality assurance process does not 
help ensure compliance because reviewers of DDS decisions are not 
required to identify and return to the DDSs cases that are not fully 
documented in accordance with the rulings. SSA's procedures require 
only that the reviewers return cases that have a deficiency that could 
result in an incorrect decision.

SSA's Review of ALJ Allowances Has Not Resulted in Improvements to 
Policy and Training:

As part of its initiative, the agency has also implemented a quality 
review of ALJ decisions, but the review has not proved useful for 
identifying any new changes to SSA's policies or training that would 
help to address the inconsistency of decisions. This review--referred 
to as the ALJ Pre-effectuation Review--involves a sequential review by 
SSA's OQA and the Appeals Council of certain ALJ allowances that have 
not yet been finalized (i.e., the claimant has not yet been awarded 
benefits). In selecting allowances for review, OQA uses an error-prone 
profile developed from its analysis of errors detected when reviewing 
DDS allowances. SSA began testing the new review of ALJs' decisions in 
1996 and implemented it as an annual review in 1998. From fiscal years 
1998 through 2002, OQA reviewed 27,148 ALJ allowances and of these, OQA 
found fault with about 35 percent and referred them to the Appeals 
Council.[Footnote 21] The Appeals Council screens the allowances for 
its own review and selects those in which the prior actions may not 
have been proper, fair, or in accordance with the law or the ALJ's 
decision was not supported by substantial evidence.[Footnote 22] If the 
council finds fault with the ALJ's decision, it will deny the claimant 
benefits or return the claim to the ALJ to have the identified problems 
corrected. If the council does not find fault with the ALJ's decision, 
the claimant will be awarded benefits.

In addition to identifying inappropriate ALJ allowances, SSA intended 
to use the new quality review to identify areas of inconsistency 
between adjudication levels and ways to improve policies and training 
to address those inconsistencies. Specifically, OQA identified cases 
where it found fault with the ALJ decision, but the Appeals Council, 
after screening them, did not accept them for review. OQA then 
forwarded these cases to a panel of staff from the various components 
involved in SSA's claims process to determine whether the inconsistent 
assessment of these cases by OQA and the Appeals Council indicated the 
need to clarify policies, issue new policies, or provide training to 
improve the consistency of decisions. However, according to a SSA 
official, this review did not identify any new areas of inconsistency 
that required improvements to policy and training. Weaknesses in the 
design of the review may have contributed to SSA's inability to 
identify new policy areas contributing to inconsistency. For example, 
rather than reviewing a random sample of all ALJ decisions, this review 
focused on allowances. Further, the review looked only at ALJ 
allowances that were selected using a DDS error-prone profile, i.e., a 
profile that is based upon cases in which quality reviewers did not 
agree with the DDS adjudicators' decisions. As a result, SSA selected 
and reviewed nonrandom allowance decisions with case characteristics 
that the agency may have already suspected were associated with 
inconsistent decisions. In 1999, the panel was disbanded because 
members had other priorities needing attention. OQA told us that it 
continued to perform a limited review of cases viewed differently by 
OQA and the Appeals Council. More recently, OQA began an effort to 
summarize the results of its review and expected to issue a report of 
its findings in April 2004. As of April 2004, this report had not been 
issued.

SSA Began Two Tests of Process Changes to Improve the Consistency of 
Decisions, but Neither Test Was Successfully Completed:

SSA began two tests of potential changes to the process to help improve 
the consistency of decisions, but neither test was successfully 
completed. The changes tested were (1) more fully developing and 
documenting decisions made at the initial level and (2) sending 
appealed cases that involve new medical information back to the initial 
level to be reevaluated.

SSA wanted to test having DDSs more fully develop and document 
decisions because it believed that DDS decisions, especially denials, 
are often not well documented. SSA wanted to test whether better 
explanations of why benefits were denied would improve the accuracy of 
DDS decisions and consistency of decisions between adjudication levels. 
SSA first implemented a pilot of this change to explore alternatives 
for developing and documenting decisions. Then SSA tested this change, 
along with other process changes, in a larger test, called the 
prototype initiative. Concurrently, SSA tested other process changes, 
such as the elimination of a reconsideration step and a predecision DDS 
interview with the claimant.

The prototype test had limitations for predicting the impact of 
documented decisions. For example, SSA's decision to test several 
changes together left the agency without clear information on what 
impact fully developed decisions would have on the decision-making 
process without the other process changes. SSA's test design also did 
not build in an ALJ feedback mechanism to provide sufficient 
information on the usefulness of more fully documented decisions. SSA 
continues to test this change along with other changes and, despite 
limited information on the best approach for and impact of this change, 
currently plans to implement more fully documented decisions as part of 
the Commissioner's new proposal to improve SSA's disability programs.

SSA also began, but ultimately abandoned, a test in which appealed 
cases with new medical information submitted prior to the hearing were 
to be sent back to the initial level so that the evidence could be 
evaluated by medical consultants residing at the DDSs. Since medical 
expertise resides in the DDS and not at the hearings level,[Footnote 
23] SSA decided to test whether "remanding," or sending cases to the 
DDS for evaluation, might result in a more consistent review of medical 
evidence. SSA believed that this change, in turn, could help improve 
the consistency of decisions because the new medical information might 
be contributing to ALJ allowances. However, the change also had the 
potential to increase the time claimants with remanded claims would 
have to wait for final decisions because claims that were not allowed 
by the DDSs had to be returned to OHA for hearings. SSA began remanding 
cases in July 1997, with a 1-year goal of remanding 100,000 cases, but 
after 10 months, it had remanded fewer than 9,000. In implementing this 
test, SSA encountered several difficulties. For example, it had 
difficulty identifying the claims to be remanded and ensuring the ALJs, 
who had authority over the claims, would remand the claims to the DDSs. 
The ALJs' resistance to remanding claims to the DDSs may be due in part 
to concerns that remanding would not lead to many allowances by the 
DDSs and would result in many claims being returned to OHA, thereby 
increasing the time many claimants would have to wait for a final 
decision from OHA. Realizing that the agency would not be able to reach 
its remanding goal, the agency decided to discontinue this test.

SSA Lacks a Clear Understanding of the Extent and Causes of 
Inconsistency between Levels:

SSA's assessments have not provided the agency with a clear 
understanding of the extent and causes of possible inconsistencies in 
decisions between adjudication levels. The two measures SSA uses to 
monitor changes in the extent of inconsistency of decisions have 
weaknesses and therefore do not provide a true picture of the changes 
in consistency. In addition, SSA has not sufficiently assessed the 
causes of possible inconsistency. The agency conducted an analysis in 
1994 that identified some potential areas of inconsistency. However, 
although SSA continues to collect information that would support this 
analysis, it has not repeated this initial effort, nor has it expanded 
on it by employing more sophisticated assessment techniques.

SSA Attempts to Monitor Changes in the Extent of Inconsistency, but the 
Measures It Uses Provide an Incomplete Picture:

SSA has made some efforts to monitor changes in the extent of 
inconsistency between the initial and hearings levels, including 
tracking trends in allowance rates at different levels and conducting 
special reviews of ALJ decisions. Together, according to SSA, these 
measures and assessments suggest that the consistency between levels 
has improved since the agency began implementing its process 
unification initiative. However, because of methodological weaknesses, 
these measures provide, at best, a partial picture of trends in the 
consistency of decisions between adjudication levels.

SSA tracks trends in the proportion of all allowances decided at each 
level to assess the consistency of decisions between levels. The agency 
collects information on the number of allowances granted to claimants 
at each level of the process, tracks the proportion of claims allowed 
at the initial level relative to the hearings level, and looks at 
trends in these proportions over a period of several years. According 
to data from SSA, the proportion of overall allowances that occurred at 
the initial level has increased since process unification was 
implemented. As shown in figure 2, in fiscal year 1996, 72 percent of 
all allowances were granted at the initial level. This proportion 
increased in most subsequent years, and by fiscal year 2003, 77 percent 
of all allowances were granted at the initial level. Officials from 
OQA, the office responsible for reviewing, evaluating, and assessing 
the integrity and quality of the administration of SSA's programs, view 
the relative shift toward earlier allowances as an indicator that 
consistency between adjudication levels has improved, and they believe 
that process unification efforts have contributed to these results.

Figure 2: SSA Allowances at Initial Level versus Hearings Level, by 
Proportion of Allowances:

[See PDF for image]

Note: Hearings level allowances include allowances made by ALJs and 
judges from the Appeals Council.

[End of figure]

However, SSA's measure of tracking yearly changes in the proportion of 
allowances at each level is a simplistic and inconclusive indicator of 
trends in the consistency of decisions because it does not control for 
the multitude of factors that can affect allowance rates at either 
adjudication level in any given year and over time. For example, SSA 
uses "snapshot" data in looking at the proportion of allowances granted 
at each level, meaning that it looks at the number of claimants and 
allowances at each level during a given year, rather than following a 
1-year cohort of initial claimants through the entire process and 
capturing the proportion of allowances for that cohort decided at each 
level. Because SSA uses data that illustrate allowance rates at a given 
moment in time, it captures a different pool of claimants in the 
process at each level, and the resulting allowance rates are subject to 
a different set of demographic and case characteristics. Over time, the 
pool of claimants may change because of factors such as a downturn in 
the economy, which can cause more people with less severe impairments 
to claim benefits or appeal initial denial decisions. In addition, 
snapshot data may be significantly affected by fluctuations in 
productivity at either adjudication level caused by process changes 
that are unrelated to process unification and that affect only one 
level.

SSA has collected other data to further assess trends in the 
consistency of decisions. Since 1993, the agency has conducted a 
biennial case review as part of its Disability Hearing Quality Review 
process.[Footnote 24] This review consists of medical consultants and 
disability examiners in SSA's central office evaluating a sample of 
ALJs' decisions plus supporting documentation to determine whether the 
ALJ has adequately supported his or her decision. In evaluating the ALJ 
decisions, these medical consultants and disability examiners use the 
same standards as those used by initial-level adjudicators to 
adjudicate claims, which are from the official SSA program policy and 
operations guidance found in POMS. To some degree, therefore, the 
medical examiners and disability reviewers serve as a proxy for 
initial-level adjudicators, and their decisions are representative of 
how initial-level examiners should be deciding claims.

While unpublished results from the biennial case reviews indicate an 
increase in supportable ALJ allowances, such findings focus on the ALJ 
level and therefore provide only a partial picture of trends in 
consistency. The reviews indicated that medical consultants and 
disability examiners have found that supportable ALJ allowances 
increased from 36 percent in fiscal year 1993-94 to 57 percent in 
fiscal year 1999-2000. OQA officials told us that this increase 
suggests an improvement in consistency between adjudication levels 
because it indicates that disability examiners using initial-level 
standards and ALJs increasingly agree on how like cases should be 
decided.[Footnote 25] However, SSA's assessment provides only a partial 
picture because it does not reflect trend information on the extent to 
which ALJs have found DDS decisions to be supportable, to ensure that 
both levels are making more consistent decisions. Although the 1994 
report of findings from the initial biennial case review included the 
results of a special probe in which ALJs reviewed 165 DDS 
reconsideration denial decisions, the sample was not representative, 
and therefore results could not serve as a baseline for developing 
trend information. In 2003, SSA began another probe, in which ALJs 
reviewed 400 DDS reconsideration denial determinations, but the agency 
does not plan to release its findings until summer 2004.

Although SSA has limited information on how ALJs view DDS decisions, 
other information collected by the agency suggests that consistency of 
decision making at the initial level might not be improving. For 
example, OQA reviewers routinely assess the accuracy and supportability 
of DDS decisions. A recent SSA study of these data shows that the 
accuracy of DDS denial decisions--those decisions most likely to be 
appealed to the hearings level--has declined by 4 percentage points 
over a 1-year period.[Footnote 26] Another review of DDS decisions by 
OQA reviewers also suggests a lack of improvement at the initial level. 
Specifically, the extent to which quality reviewers found that DDS 
reconsideration denials appealed to the hearings level were supported 
declined from 71 percent in fiscal year 1993-94 to 68 percent in fiscal 
year 1999-2000.

SSA's Assessments Have Not Identified the Areas and Causes of Possible 
Inconsistency in Decisions between Adjudication Levels:

Despite some efforts to assess inconsistency in decisions, shortcomings 
in SSA's analyses also limit its ability to identify areas and causes 
of possible inconsistency. Most notably, over the last 10 years, SSA 
has not updated its prior analyses of information from its initial 
biennial case review that helped identify problem areas. In addition, 
SSA has not improved on its case review and analysis by ensuring that 
reviewers assess all relevant case evidence used to make decisions, or 
performed more sophisticated analysis to identify the areas and causes 
of inconsistency in decisions. Other efforts--including the review of 
ALJ allowances and a probe of DDS reconsideration denials--have yet to 
yield useful information.

In 1994, for its initial biennial case review report, the agency took 
its first step in identifying areas of possible inconsistency by 
identifying two characteristics about the claimants and their cases 
over which initial-level reviewers tended to disagree with ALJs. 
Specifically, the 1994 report concluded that teams of reviewing medical 
consultants and disability examiners sometimes viewed cases involving 
mental impairments differently than the reviewing ALJs. In addition, 
these two sets of reviewers tended to have different views on the 
severity of claimants' impairments and their resulting capacity to 
work. According to the official responsible for overseeing the review, 
the findings in this initial report provided important support for 
SSA's process unification efforts as well as the agency's efforts to 
redesign the disability claims process.

SSA continues to conduct the biennial case reviews; however, the agency 
has not continued to analyze and identify areas that are viewed 
differently by different adjudication levels. Specifically, SSA no 
longer identifies the particular case characteristics over which 
reviewers from the two levels tend to disagree. As a result, SSA does 
not know whether previously identified problem areas are still present. 
Moreover, SSA no longer publishes any information from the medical 
consultant and disability examiner biennial case reviews, even though 
it has performed some limited analysis of the supportability of 
decisions made by adjudicators. By not continuing to publish its 
analysis and findings, the agency makes it difficult to ensure the 
reliability of its methods and results, and leaves stakeholders outside 
the agency, including disability groups, without a means for 
understanding SSA's assessment efforts and progress in improving the 
consistency of decisions. The SSA office conducting the study has told 
us that, because of downsizing and competing priorities, it has no 
current plans to further analyze and publish these data.

Further, in its ongoing biennial case reviews, SSA does not make full 
use of available case information that would be useful in identifying 
areas and causes of inconsistency. Specifically, medical consultants 
and disability reviewers do not listen to tapes of the hearings and 
therefore do not review the entire case as presented to the original 
ALJ. Although reviewing medical consultants and disability examiners 
read the ALJs' explanations for their original decisions, which should 
include the most important factors behind the ALJs' decisions, the 
reviewers do not evaluate the oral evidence independently. An SSA 
official with whom we spoke indicated that some evidence entered by 
witnesses at the hearing might not be accompanied by other hard copy 
sources of the same information. Therefore, reviewers would not 
consider information potentially relevant to the ALJ's decision that 
could be used to identify areas and causes of inconsistency.

SSA also does not make full use of the information it collects because 
it has not employed analytical tools that would improve its ability to 
identify areas and causes of inconsistency. For example, SSA's biennial 
case reviews provide a rich dataset that lends itself to regression 
analysis to identify areas and possible causes of inconsistency between 
levels. Regression analysis would allow the agency to better pinpoint 
any significant case characteristics affecting decisions and to more 
clearly identify the underlying causes of inconsistency. Specifically, 
among the data collected in this review are such variables as the types 
of impairments the claimant has, the types of relevant medical 
evidence, and additional impairments presented at the hearing. 
Multivariate analysis, such as a multiple regression model, could allow 
SSA to assess how these and many other factors, relative to one 
another, contribute to whether a case results in a similar outcome at 
both levels. However, SSA has not employed this more sophisticated 
multivariate technique, citing resource constraints and competing 
priorities. We recognize the methodological complexities of analyzing 
disability decisions, and we previously recommended that SSA establish 
an advisory panel of external experts from a range of disciplines to 
provide leadership, oversight, and technical assistance to the 
agency.[Footnote 27] Otherwise, in forgoing such analysis, the agency 
will continue to miss an opportunity to better pinpoint areas and some 
possible causes of inconsistency in decisions between the two 
adjudication levels, and to lay the foundation for further 
investigation.

Another tool SSA has not sufficiently employed for identifying areas 
and causes of inconsistency is in-depth case studies involving both 
levels of adjudication. Case studies, in which different adjudicators 
review the same test case, can be a means for unearthing causes for 
inconsistency by getting adjudicators from both levels to acknowledge 
and address discrepancies in the ways they view cases. SSA has 
performed case studies in the past to ascertain differences in policy 
interpretation between DDS examiners and quality reviewers. However, 
SSA does not routinely have both DDS examiners and ALJs perform in-
depth review of the same sample of cases, despite this method's 
potential for helping identify causes of inconsistency between the two 
adjudication levels. OQA officials told us that case studies are a very 
resource-intensive tool because they need a sufficient number of cases 
from which to generalize. Therefore, the agency is reluctant to use 
this approach to help it understand the causes of inconsistency between 
adjudication levels. However, using multivariate analyses of the 
biennial case review data could help the agency to more effectively 
target its in-depth case studies on those areas found to be leading to 
inconsistent decisions and thereby increase its success at identifying 
the causes of inconsistency.

SSA conducts other analyses of inconsistency between levels, but to 
date these efforts have yielded limited information concerning areas 
and possible causes of inconsistency. For example, as part of SSA's ALJ 
Pre-Effectuation Review, two different levels of reviewers have 
evaluated thousands of cases. However, limitations in the review 
methodology, such as not using a random sample of ALJ decisions, do not 
allow the agency to use this review to identify the leading causes of 
inconsistency. SSA recently began an evaluation of this effort and 
plans to publish its findings and recommendations in April 2004. 
Another analysis currently under way, a special 400-case review, might 
help identify areas of inconsistency at the initial level, but it has 
yet to be completed. Begun in 2003, this review by ALJs of DDS 
reconsideration denial determinations is expressly aimed at assessing 
inconsistency between adjudication levels. SSA expects to gain some 
understanding of why about 60 percent of cases denied by the initial 
level and appealed to the hearings level are allowed. The agency plans 
to publish its findings in summer 2004.

SSA's New Proposal Incorporates Efforts to Improve the Consistency of 
Decisions, but Challenges May Impede Successful Implementation:

Some changes included in SSA's new proposal to overhaul its disability 
claims process may improve the consistency of DDS and ALJ decisions, 
but challenges may hinder the implementation of the proposal. The new 
proposal includes several changes to the disability claims process that 
the agency and stakeholder groups representing adjudicators and 
claimant representatives believe offer promise for improving the 
consistency of DDS and ALJ decisions. However, past difficulties in 
improving the process, as well as stakeholder concerns about limited 
resources and other obstacles, indicate that some difficulties may 
arise in the development and implementation of SSA's new proposal.

The Agency and Most Stakeholders Viewed Several Aspects of the New 
Proposal as Offering Promise for Improving Consistency:

SSA told us that several aspects of the new proposal may improve the 
consistency of decisions, and although opinions varied among 
stakeholder groups, most thought the following four proposed changes 
have the potential to improve the consistency of decisions between 
adjudication levels: (1) requiring state adjudicators to more fully 
develop and document their decisions, (2) centralizing the agency's 
approach to quality control, (3) providing both adjudication levels 
with equal access to more centralized medical expertise, and (4) 
requiring ALJs to address agency reports that either recommend denying 
the claim or outline the evidence needed to fully support the claim.

Representatives from most stakeholder groups with whom we spoke told us 
that having state adjudicators more fully develop and document their 
decisions may help to improve the consistency of DDS and ALJ decisions. 
Specifically, stakeholders said that more developed decisions may 
provide ALJs with a better understanding of the DDS decision and enable 
them to more fully consider this information when evaluating a case. 
According to the agency and stakeholders, this change may contribute to 
a more consistent interpretation and application of SSA's decision-
making criteria. They also mentioned that well-developed decisions by 
DDS examiners could assist SSA in holding adjudicators accountable for 
case development and decisions, such as enabling quality reviewers to 
more effectively assess the appropriateness of the DDSs decisions. 
Unlike SSA's earlier attempt at more fully developing decisions as part 
of process unification, SSA plans to incorporate a reviewing official 
into the process whose assessment of all appealed DDS decisions can 
provide feedback on the extent to which cases are being fully 
developed.

In addition, the agency and many stakeholders told us that they believe 
centralizing the agency's quality control system may help resolve some 
problems contributing to inconsistent decisions between the two levels. 
For example, they believed that it may help ensure a more consistent 
review of cases across the country and between adjudication levels. 
According to both stakeholders and other experts within and outside of 
SSA (including SSA's Deputy Commissioner of Disability and Income 
Security and a consulting group that reviewed SSA's quality assurance 
system),[Footnote 28] the current quality control and case review 
process encourages adjudicators at the initial level to inappropriately 
deny cases, while encouraging adjudicators at the hearings level to 
inappropriately allow cases. Specifically, by overemphasizing a review 
of DDS allowances to help control the cost of benefits, the agency has 
unintentionally encouraged DDS examiners to deny cases. Conversely, 
SSA's review of ALJ decisions consists mostly of SSA's Appeals Council 
reviewing cases denied by ALJs, thereby providing an incentive for ALJs 
to allow cases. By centralizing the quality control system and making 
other changes to the process, SSA believes that it can remove the 
current incentives that contribute to inconsistency.

The third proposed change that the agency and most stakeholder groups 
believe may improve consistency is SSA's plan to provide both 
adjudication levels with equal access to more centralized medical 
experts, organized by clinical specialty. Although located in the 
regions, these experts should be able to review cases from across the 
country with the successful completion of SSA's AeDib initiative--an 
electronic folder initiative for exchanging case information currently 
being implemented by SSA. By making experts in a range of specialties 
available to assist both levels of adjudicators in their decision 
making, SSA and stakeholders believe that adjudicators could more 
consistently apply SSA's decision-making criteria, in addition to 
acquiring better medical evidence.

Finally, the agency and most stakeholder groups told us that the 
requirement to have an ALJ's decision address the recommendations from 
a reviewing official's report to either deny or more fully develop the 
claim may increase consistency between levels. Under the new proposal, 
SSA plans to introduce a reviewing official into the process to 
evaluate all appealed DDS claims. The official will allow claims that 
meet SSA's definition of disability and, for the remaining claims, will 
develop a report that either (1) contains reasons for denying the claim 
or (2) outlines the evidence needed to fully support the claim. The 
ALJ's decision must address issues raised in the reviewing official's 
report. Stakeholders believed that this change could, as intended by 
SSA, hold adjudicators more accountable for their decisions and provide 
adjudicators with feedback on the reasons decisions tend to differ 
between levels to improve the quality and consistency of their 
decisions.

Although there was less agreement among stakeholder groups on the 
potential effect that other aspects of the new proposal may have on the 
consistency of decisions, some groups thought that other changes could 
result in improved consistency between DDS and ALJ decisions. For 
example, the Social Security Advisory Board and two groups representing 
the DDSs thought that the proposed in-line quality control, if 
implemented effectively at all levels, could have a positive impact on 
consistency by ensuring that adjudicators adhere to the rulings and 
regulations throughout the decision-making process. One stakeholder 
group added that in-line quality control could also help the agency 
identify problem areas, including areas in which policy is applied 
inconsistently or where more training is needed.

Resource and Other Constraints May Limit SSA's Ability to Successfully 
Implement Some Changes in the New Proposal:

According to stakeholder groups--and based on SSA's prior experience 
with making significant changes to its claims process--insufficient 
resources and other obstacles may prove to be major challenges for the 
agency in developing and implementing aspects of its new proposal. For 
example, experience with the process unification initiative has shown 
that limited state resources have hindered the agency's ability to have 
state adjudicators fully document decisions. To address this issue, SSA 
plans to reduce the states' workloads by decreasing the number of 
claims to be decided by the DDSs. Specifically, SSA expects that 
establishing regional expert review units to make quick decisions for 
claimants who are obviously disabled will substantially decrease the 
states' workloads. However, SSA has not developed and provided 
stakeholders with estimates of the administrative cost for more fully 
documenting decisions and other planned changes, and stakeholder groups 
were not convinced that the reduction in claims was sufficient to 
offset resources needed to fully document their decisions. Although the 
agency has had some recent success in increasing its 2004 
administrative budget, and is confident that it will be successful in 
acquiring the resources it needs to implement the proposal, the 
significance of stakeholders' concerns about funding cannot be assessed 
until SSA fully develops its proposal and associated cost estimates.

Experience has also shown that another proposed change, developing a 
centralized quality control system for both adjudication levels, could 
be a major challenge for the agency. In 1994, SSA began efforts to 
create a unified and comprehensive quality control system as part of 
its redesign efforts, but made little progress, in part because of 
considerable disagreement among internal and external stakeholders on 
how to accomplish this difficult objective. To get external assistance 
in developing an effective quality assurance system, SSA contracted 
with an independent consulting firm to assess SSA's quality assurance 
practices used in the disability claims process. In 2001, concluding 
that SSA could achieve its quality objectives for the disability 
program only by adopting a broad, modern view of quality management, 
the consulting firm recommended SSA abandon its current system and 
design a new quality management system focused on building quality into 
the process. The agency agreed that it was appropriate to transform the 
existing quality assurance system and established an executive work 
group to decide a future course of action. The agency is working with 
another consulting group to further develop the changes recently 
proposed by the Commissioner. However, after 10 years of efforts to 
develop a more unified quality review system, SSA has not yet 
formulated changes to its quality review system, beyond the brief and 
general descriptions provided in the Commissioner's new proposal.

Other obstacles also add to the complexity and difficulty of 
implementing the proposal. For example, stakeholder groups have raised 
concerns about SSA's ability to successfully implement its proposed 
change to provide equal access for all adjudicators to more centralized 
medical expertise by removing medical expertise from the state DDSs and 
providing expertise in regional offices. Stakeholder groups were 
concerned that SSA would not be able to attract and retain sufficient 
medical experts to meet the agency's needs. They told us that states 
are currently experiencing problems attracting medical experts because 
SSA's compensation rates are too low. State adjudicators, who currently 
work with medical experts directly at DDS offices, were also concerned 
that removing these experts and placing experts in SSA regional offices 
would impair the states' effectiveness and efficiency. By placing 
experts in regional offices, state disability examiners would no longer 
have on-site access to these experts who help facilitate the states' 
adjudication of claims and provide on-the-job training and mentoring to 
DDS examiners.

Stakeholders have also raised questions about SSA's ability to ensure 
that ALJs' decisions fully respond to the reviewing officials' reports 
and the ultimate effectiveness of this change. Stakeholder groups 
representing ALJs and claimant representatives believed that the 
requirements may have the potential to impinge on an ALJ's legal 
responsibility to ensure a claimant receives a fair hearing and an 
independent decision. Other groups have raised concerns about SSA's 
ability to ensure that ALJs will adequately address recommendations in 
the reviewing officials' reports to help ensure that this requirement 
leads to more consistent decisions. Although these concerns have been 
raised, the Commissioner has clearly stated that the intent of the 
proposal is to improve service to claimants, including providing fair 
and accurate decisions, and that changes will not impinge on the 
independence of ALJs.

In addition, several stakeholder groups also told us that staffing the 
new reviewing official positions with attorneys, as SSA intends to do, 
would be expensive. To the extent that SSA has difficulty filling these 
positions, the agency could create a slowdown or bottleneck in the 
process that could increase the time claimants must wait for a 
decision. Furthermore, according to one stakeholder group, SSA's new 
quality assurance process will need to ensure that this new position 
does not create another source of inconsistent interpretation and 
application of SSA's decision-making criteria.

Several groups representing hearings level adjudicators and claimant 
representatives were also concerned about other aspects of the 
Commissioner's new proposal, such as the proposed elimination of the 
Appeals Council and the claimants' loss of the right to appeal an ALJ 
decision to the council. The Appeals Council currently reviews about 
100,000 appealed ALJ decisions annually. For these claims, the council 
provides an additional appellate step for addressing claimants' 
objections to the ALJs' decisions, reviewing new medical information on 
the claims and reducing the number of claims appealed directly to the 
federal courts. According to one stakeholder group, the council also 
performs other important functions, such as reviewing claims for 
surviving children or spouses of workers who were insured under the 
disability insurance and retirement program. The council also reviews 
cases remanded from federal courts. This stakeholder group also told us 
that as SSA refines its proposal it will need to articulate how all of 
the council's functions will be handled under the new process.

Adding to uncertainties about the proposal's success is its dependence 
on the successful development and implementation of the AeDib system--
a highly complex and as yet unproven system using electronic folders to 
share information with all entities involved in disability 
determinations. SSA does not plan to implement its newly proposed 
changes before it has completed a national rollout of its electronic 
disability system, scheduled to be completed by October 2005. The new 
electronic disability system represents an important step toward a 
paperless and more efficient sharing of information by multiple 
partners involved in the disability claims process, including SSA and 
state officials, as well as physicians and other members of the medical 
community who provide needed medical evidence. SSA also expects this 
new system to provide critical management information for analyzing and 
reducing inconsistencies in disability decisions. As we previously 
reported, SSA has made progress developing the new system. However, its 
approach involves risks that could jeopardize the agency's successful 
transition to an electronic disability claims process.[Footnote 29] For 
example, SSA recently began a national rollout of the electronic 
disability system without fully evaluating pilot test results or 
ensuring the resolution of all critical problems. Skipping such 
important steps in development and implementation leaves the new system 
vulnerable to problems in its performance and reliability. In addition, 
problems with implementation of this system could delay the 
implementation of SSA's new proposal.

SSA recognizes that transforming its massive and complex disability 
programs and achieving the benefits envisioned by the Commissioner will 
be a challenging undertaking. The agency is refining its proposal and, 
as part of this process, is actively seeking input from stakeholder 
groups. The Commissioner and her staff have met directly with 
stakeholder groups to understand and begin to address their concerns. 
As the agency refines its proposal, the significance of both 
stakeholder concerns and previous problems SSA has experienced 
improving its programs should become clearer.

Conclusions:

When SSA's Commissioner announced her new proposal to overhaul the 
disability programs, the agency acknowledged the importance of making 
similar decisions on similar cases and making the right decision as 
early in the process as possible. SSA has good cause to focus on the 
consistency of decisions between adjudication levels. Incorrect denials 
at the initial level that are appealed increase both the time claimants 
must wait for a decision and the cost of deciding cases. Incorrect 
denials that are not appealed may leave needy individuals without a 
financial or medical safety net. Conversely, incorrect allowances at 
any adjudication level could substantially increase the cost of 
providing disability benefits.

While the agency has made some effort to assess the inconsistency in 
decisions between levels, its efforts have not provided the agency with 
a clear understanding of the extent and leading causes of possible 
inconsistencies in the interpretation and application of disability 
guidance. For example, SSA's assessment of ALJ error-prone allowances 
has not proven to be effective at identifying new areas and causes of 
inconsistency. SSA also has not updated its more effective approach of 
analyzing its Disability Hearings Quality Review data to identify 
problem areas and help improve its understanding of the factors that 
may be contributing to inconsistency. Further, SSA's analysis lacked 
sophisticated statistical techniques and in-depth analysis of cases by 
adjudicators at both levels, which together would have allowed SSA to 
better identify and address the areas and leading causes of 
inconsistency. Moreover, by not having examiners and medical 
consultants perform a complete review of all relevant information 
before an ALJ, SSA has limited its ability to understand the areas and 
causes of possible inconsistency.

Without better information on the areas and causes of possible 
inconsistency, the agency cannot ensure that the Commissioner's new 
proposal will help to resolve this complex and long-standing concern. 
By taking immediate actions to improve its understanding of the leading 
causes of possible inconsistency in decisions, the agency will have 
information needed to evaluate and possibly refine its new proposal, 
including its plans to build an effective quality assurance system that 
can both detect and prevent inconsistencies in decisions. This 
information will help the agency to target its limited resources and 
take decisive steps to build a claims process that provides claimants 
with the accurate, consistent, and timely decisions they deserve, as 
envisioned in the Commissioner's proposal.

Recommendations:

To move successfully forward with agency efforts to make more 
consistent decisions, including efforts incorporated in the 
Commissioner's proposal for an improved disability claims process and 
quality assurance system, we recommend that SSA quickly expand its 
assessment of the areas and causes of inconsistency in decisions 
between adjudication levels. In doing so, SSA should consider making 
near-term and cost-effective enhancements to its current approach for 
assessing the consistency of decisions, including:

1. Reestablish ongoing analyses of case characteristics as part of its 
biennial case review, in line with efforts undertaken for the review 
report published in 1994.

2. Perform more sophisticated multivariate analysis on the biennial 
case review data in order to pinpoint the most significant case 
characteristics influencing allowance decisions and to distinguish 
factors that might be contributing either appropriately or 
inappropriately to allowance decisions.

3. Expand the biennial case review by requiring disability examiners 
and medical consultants to review the hearing tapes to ensure that 
reviewers have the complete case before them (including the types and 
sources of testimonial evidence provided during the hearings) when 
evaluating the ALJs' decisions.

4. Have adjudicators and reviewers from each level study cases in depth 
to help pinpoint the causes of inconsistency, once potential areas of 
inconsistency between levels are identified.

5. Publish the methods and findings of all analyses, to keep internal 
and external stakeholders aware of the agency's efforts to assess 
consistency and demonstrate improvement over time.

6. Use the information from these improved analyses to develop a more 
focused and effective strategy for ensuring uniform application of 
SSA's guidance and to improve the consistency of decisions. To 
accomplish this, SSA should clarify guidance for making disability 
decisions and develop mandatory training for adjudicators on issues 
identified as contributing to inconsistency.

Agency Comments and Our Evaluation:

We provided a draft of this report to SSA for comment. SSA expressed 
several reservations about the recommendations, findings, and 
conclusions of our report. Primarily, SSA took issue with: (1) our 
characterization of the agency's progress over the past several years 
in analyzing and reducing the inconsistency of decisions, (2) our 
recommendation that the agency incorporate multivariate analysis into 
its assessments, and (3) our finding that the agency has not acted on 
the results of its reviews of decisions. SSA indicated that it would 
reevaluate our recommendations as the design of its Commissioner's new 
approach to disability decision making evolves. However, the agency did 
agree to pilot one recommendation--that quality reviewers assess 
hearing tapes when evaluating the ALJs' decisions--as part of a quality 
review.

One of SSA's main concerns was that our report did not fully discuss 
the progress SSA had achieved in analyzing and reducing the 
inconsistency in decision making between adjudication levels. For 
example, SSA commented that our report dismissed the 21-percentage 
point increase in the quality reviewers' support rate of ALJ decisions, 
conducted as part of SSA's biennial case reviews over the last 10 
years. SSA also pointed to findings from its ALJ peer reviews as 
additional evidence that the quality and consistency of SSA's decisions 
had improved. In addition, SSA asserted that its comparison of the 
relative proportion of allowances at the DDS and ALJ levels, along with 
high accuracy rates, indicated that adjudicators were making the right 
decisions sooner in the process--a goal of both process unification and 
the Commissioner's new disability approach. Although our report 
incorporates results from the analyses cited by SSA, our conclusion 
about the improvement in consistency between levels is not as 
optimistic as SSA's because of weaknesses in SSA's assessments. As we 
reported, SSA's analysis of the quality reviewers' assessment of ALJ 
cases has been limited for 10 years to calculating ALJ support rates. 
SSA has not used available data to determine the potential areas of 
inconsistency between levels or the extent to which changes in the ALJ 
support rate is related to improvements in consistency of decisions 
between adjudication levels. SSA's assessment also lacks a reliable 
method for determining whether DDS decisions are more consistent with 
ALJ decisions, for example, by having ALJs regularly review a 
statistical sample of DDS decisions. Lastly, as we pointed out, changes 
in the proportion of overall allowances made by the DDS and ALJ levels 
cannot serve as a reliable indicator for measuring the consistency of 
decisions between levels, because many factors can affect these 
proportions, such as significant fluctuations in the number of 
decisions made at each adjudication level.

SSA also expressed its reservations about the benefits of multivariate 
analysis in its evaluation of decision making. SSA asserted that its 
analyses over the past 10 years have provided the agency with a solid 
understanding of how certain variables influence disability decision 
making and that the multivariate analyses we recommended would not 
identify the causes and effects of inconsistent decision making at 
different levels of this complex process. We agree with SSA that the 
disability decision-making process is complex and that multivariate 
analysis alone cannot establish all the causes and effects of 
inconsistent decision making. However, because multivariate analysis 
takes into account the influence of a number of relevant variables for 
each decision, this analytical technique can provide a more accurate 
understanding of areas and causes of inconsistency in decisions than 
methods previously employed by SSA. Such analyses, followed by in-depth 
case studies by adjudicators at both levels, which we also recommended, 
would bring SSA closer to understanding and resolving the inconsistency 
of decisions between adjudication levels. Therefore, we continue to 
believe that by performing the analyses we recommend, the agency will 
have a better understanding of the extent and causes of inconsistency, 
and that SSA's Commissioner should quickly implement our 
recommendations to ensure that her new approach effectively addresses 
the consistency of decisions between adjudication levels.

Finally, SSA disagreed with our finding that it has not acted on the 
results of its reviews of decisions. SSA noted that it has made changes 
to address training needs that have been identified by its reviews. 
Specifically, SSA indicated that it has provided a series of 
interactive video training (IVT) sessions focusing on problematic areas 
noted in the ALJ peer review reports. We acknowledge that SSA has 
conducted ALJ peer reviews and used findings from its reviews to 
develop and provide training to ALJs. However, we did not include these 
findings in our report, because our objectives were limited to 
reporting efforts undertaken by SSA to assess or improve the 
consistency of decisions between adjudication levels or to implement 
its process unification initiative. SSA's ALJ peer review is conducted 
to identify problems with the quality of ALJ hearing process and 
decisions, not to identify problems with the inconsistency of decisions 
between levels. Conversely, our report included information on SSA's 
ALJ pre-effectuation review, because it was part of SSA's process 
unification initiative. According to information provided to us by SSA 
during our audit, although this review was intended to help identify 
policy and training areas that were associated with inconsistent 
decisions between adjudication levels, it was not effective at 
identifying any new areas to be pursued by the agency. This finding, 
along with those provided throughout the report, supports our 
recommendations to SSA that the agency perform additional analysis to 
determine the causes of potential inconsistency between adjudication 
levels and to clarify guidance and provide mandatory training to 
address any identified causes.

In addition, SSA provided several other general and technical comments 
about the draft report. These additional comments, as well as our 
response to them, are provided in appendix II.

Copies of this report are being sent to the Commissioner of SSA, 
appropriate congressional committees, and other interested parties. The 
report is also available at no charge on GAO's Web site at 
http://www.gao.gov. If you have any questions about this report, please 
contact me at (202) 512-7215. Other contacts and staff acknowledgments 
are listed in appendix III.

Sincerely yours,

Signed by: 

Robert E. Robertson: 
Director, Education, Workforce, and Income Security Issues:

[End of section]

Appendix I: Excerpt of SSA's Testimony Announcing Its New Proposal to 
Improve Its Disability Decision-Making Process:

In designing my approach to improve the overall disability 
determination process, I was guided by three questions the President 
posed during our first meeting to discuss the disability 
programs.[Footnote 30]

* Why does it take so long to make a disability decision?

* Why can't people who are obviously disabled get a decision 
immediately?

* Why would anyone want to go back to work after going through such a 
long process to receive benefits?

I realized that designing an approach to fully address the central and 
important issues raised by the President required a focus on two over-
arching operational goals: (1) to make the right decision as early in 
the process as possible; and (2) to foster return to work at all stages 
of the process. I also decided to focus on improvements that could be 
effectuated by regulation and to ensure that no SSA employee would be 
adversely affected by my approach. My reference to SSA employees 
includes State Disability Determination Service employees and 
Administrative Law Judges (ALJs).

As I developed my approach for improvement, I met with and talked to 
many people--SSA employees and other interested organizations, 
individually and in small and large groups--to listen to their concerns 
about the current process at both the initial and appeals levels and 
their recommendations for improvement. I became convinced that 
improvements must be looked at from a system-wide perspective and, to 
be successful, perspectives from all parts of the system must be 
considered. I believe an open and collaborative process is critically 
important to the development of disability process improvements. To 
that end, members of my staff and I visited our regional offices, field 
offices, hearing offices, and State Disability Determination Services, 
and private disability insurers to identify and discuss possible 
improvements to the current process.

Finally, a number of organizations provided written recommendations for 
changing the disability process. Most recently, the Social Security 
Advisory Board issued a report prepared by outside experts making 
recommendations for process change. My approach for changing the 
disability process was developed after a careful review of these 
discussions and written recommendations. As we move ahead, I look 
forward to working within the Administration and with Congress, as well 
as interested organizations and advocacy groups. I would now like to 
highlight some of the major and recurring recommendations made by these 
various parties.

The need for additional resources to eliminate the backlog and reduce 
the lengthy processing time was a common theme. This important issue is 
being addressed through my Service Delivery Plan, starting with the 
President's FY 2004 budget submission which is currently before 
Congress. Another important and often heard concern was the necessity 
of improving the quality of the administrative record. DDSs expressed 
concerns about receiving incomplete applications from the field office; 
ALJs expressed concerns about the quality of the adjudicated record 
they receive and emphasized the extensive pre-hearing work required to 
thoroughly and adequately present the case for their consideration. In 
addition, the number of remands by the Appeals Council and the Federal 
Courts make clear the need for fully documenting the administrative 
hearing record.

Applying policy consistently in terms of: 1) the DDS decision and ALJ 
decision; 2) variations among state DDSs; and 3) variations among 
individual ALJs--was of great concern. Concerns related to the 
effectiveness of the existing regional quality control reviews and ALJ 
peer review were also expressed. Staff from the Judicial Conference 
expressed strong concern that the process assure quality prior to the 
appeal of cases to the Federal Courts.

ALJs and claimant advocacy and claimant representative organizations 
strongly recommended retaining the de novo hearing before an ALJ. 
Department of Justice litigators and the Judicial Conference stressed 
the importance of timely case retrieval, transcription, and 
transmission. Early screening and analysis of cases to make expedited 
decisions for clear cases of disability was emphasized time and again 
as was the need to remove barriers to returning to work.

My approach for disability process improvement is designed to address 
these concerns. It incorporates some of the significant features of the 
current disability process. For example, initial claims for disability 
will continue to be handled by SSA's field offices. The State 
Disability Determination Services will continue to adjudicate claims 
for benefits, and Administrative Law Judges will continue to conduct 
hearings and issue decisions. My approach envisions some significant 
differences.

I intend to propose a quick decision step at the very earliest stages 
of the claims process for people who are obviously disabled. Cases will 
be sorted based on disabling conditions for early identification and 
expedited action.

Examples of such claimants would be those with ALS, aggressive cancers, 
and end-stage renal disease. Once a disability claim has been completed 
at an SSA field office, these Quick Decision claims would be 
adjudicated in Regional Expert Review Units across the country, without 
going to a State Disability Determination Service. This approach would 
have the two-fold benefit of allowing the claimant to receive a 
decision as soon as possible, and allowing the State DDSs to devote 
resources to more complex claims.

Centralized medical expertise within the Regional Expert Review Units 
would be available to disability decision makers at all levels, 
including the DDSs and the Office of Hearings and Appeals (OHA). These 
units would be organized around clinical specialties such as 
musculoskeletal, neurological, cardiac, and psychiatric. Most of these 
units would be established in SSA's regional offices.

The initial claims not adjudicated through the Quick Decision process 
would be decided by the DDSs. However, I would also propose some 
changes in the initial claims process that would require changes in the 
way DDSs are operating. An in-line quality review process managed by 
the DDSs and a centralized quality control unit would replace the 
current SSA quality control system. I believe a shift to in-line 
quality review would provide greater opportunities for identifying 
problem areas and implementing corrective actions and related training. 
The Disability Prototype would be terminated and the DDS 
Reconsideration step would be eliminated. Medical expertise would be 
provided to the DDSs by the Regional Expert Review units that I 
described earlier.

State DDS examiners would be required to fully document and explain the 
basis for their determination. More complete documentation should 
result in more accurate initial decisions. The increased time required 
to accomplish this would be supported by redirecting DDS resources 
freed up by the Quick Decision cases being handled by the expert units, 
the elimination of the Reconsideration step, and the shift in medical 
expertise responsibilities to the regional units.

A Reviewing Official (RO) position would be created to evaluate claims 
at the next stage of the process. If a claimant files a request for 
review of the DDS determination, the claim would be reviewed by an SSA 
Reviewing Official. The RO, who would be an attorney, would be 
authorized to issue an allowance decision or to concur in the DDS 
denial of the claim. If the claim is not allowed by the RO, the RO will 
prepare either a Recommended Disallowance or a Pre-Hearing Report. A 
Recommended Disallowance would be prepared if the RO believes that the 
evidence in the record shows that the claimant is ineligible for 
benefits. It would set forth in detail the reasons the claim should be 
denied. A Pre-Hearing Report would be prepared if the RO believes that 
the evidence in the record is insufficient to show that the claimant is 
eligible for benefits but also fails to show that the claimant is 
ineligible for benefits. The report would outline the evidence needed 
to fully support the claim. Disparity in decisions at the DDS level has 
been a long-standing issue and the SSA Reviewing Official and creation 
of Regional Expert Medical Units would promote consistency of decisions 
at an earlier stage in the process.

If requested by a claimant whose claim has been denied by an RO, an ALJ 
would conduct a de novo administrative hearing. The record would be 
closed following the ALJ hearing. If, following the conclusion of the 
hearing, the ALJ determines that a claim accompanied by a Recommended 
Disallowance should be allowed, the ALJ would describe in detail in the 
written opinion the basis for rejecting the RO's Recommended 
Disallowance. If, following the conclusion of the hearing, the ALJ 
determines that a claim accompanied by a Pre-Hearing Report should be 
allowed, the ALJ would describe the evidence gathered during the 
hearing that responds to the description of the evidence needed to 
successfully support the claim contained in the Pre-Hearing Report.

Because of the consistent finding that the Appeals Council review adds 
processing time and generally supports the ALJ decision, the Appeals 
Council stage of the current process would be eliminated. Quality 
control for disability claims would be centralized with end-of-line 
reviews and ALJ oversight. If an ALJ decision is not reviewed by the 
centralized quality control staff, the decision of the ALJ will become 
a final agency action. If the centralized quality control review 
disagrees with an allowance or disallowance determination made by an 
ALJ, the claim would be referred to an Oversight Panel for 
determination of the claim. The Oversight Panel would consist of two 
Administrative Law Judges and one Administrative Appeals Judge. If the 
Oversight Panel affirms the ALJ's decision, it becomes the final agency 
action. If the Panel reverses the ALJ's decision, the oversight Panel 
decision becomes the final agency action. As is currently the case, 
claimants would be able to appeal any final agency action to a Federal 
Court.

At the same time these changes are being implemented to improve the 
process, we plan to conduct several demonstration projects aimed at 
helping people with disabilities return to work. These projects would 
support the President's New Freedom Initiative and provide work 
incentives and opportunities earlier in the process.

Early Intervention demonstration projects will provide medical and cash 
benefits and employment supports to Disability Insurance (DI) 
applicants who have impairments reasonably presumed to be disabling and 
elect to pursue work rather than proceeding through the disability 
determination process. Temporary Allowance demonstration projects will 
provide immediate cash and medical benefits for a specified period (12-
24 months) to applicants who are highly likely to benefit from 
aggressive medical care. Interim Medical Benefits demonstration 
projects will provide health insurance coverage to certain applicants 
throughout the disability determination process. Eligible applicants 
will be those without such insurance whose medical condition is likely 
to improve with medical treatment or where consistent, treating source 
evidence will be necessary to enable SSA to make a benefit eligibility 
determination. Ongoing Employment Supports to assist beneficiaries to 
obtain and sustain employment will be tested, including a Benefit 
Offset demonstration to test to effects of allowing DI beneficiaries to 
work without total loss of benefits by reducing their monthly benefit 
$1 for every $2 of earnings above a specified level and Ongoing Medical 
Benefits demonstration to test the effects of providing ongoing health 
insurance coverage to beneficiaries who wish to work but have no other 
affordable access to health insurance.

I believe these changes and demonstrations will address the major 
concerns I highlighted earlier. I also believe they offer a number of 
important improvements:

* People who are obviously disabled will receive quick decisions.

* Adjudicative accountability will be reinforced at every step in the 
process.

* Processing time will be reduced by at least 25%.

* Decisional consistency and accuracy will be increased.

* Barriers for those who can and want to work would be removed.

Describing my approach for improving the process is the first step of 
what I believe must be--and will work to make--a collaborative process. 
I will work within the Administration, with Congress, the State 
Disability Determination Services and interested organizations and 
advocacy groups before putting pen to paper to write regulations. As I 
said earlier, and I say again that to be successful, perspectives from 
all parts of the system must be considered.

Later today, I will conduct a briefing for Congressional staff of the 
Ways and Means and Senate Finance Committees. I will also brief SSA and 
DDS management. In addition, next week I will provide a video tape of 
the management briefing describing my approach for improvement to all 
SSA regional, field, and hearing offices, State Disability 
Determination Services, and headquarters and regional office employees 
involved in the disability program. Tomorrow, I will be conducting 
briefings for representatives of SSA employee unions and interested 
organizations and advocacy groups, and I will schedule meetings to 
provide an opportunity for those representatives to express their views 
and provide assistance in working through details, as the final package 
of process improvements is fully developed.

I believe that if we work together, we will create a disability system 
that responds to the challenge inherent in the President's questions. 
We will look beyond the status quo to the possibility of what can be. 
We will achieve our ultimate goal of providing accurate, timely service 
for the American people.

[End of section]

Appendix II: Comments from the Social Security Administration:

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix.

SOCIAL SECURITY:

The Commissioner:

June 4, 2004:

Mr. Robert E. Robertson: 
Director, Education, Workforce and Income Security Issues: 
U.S. General Accounting Office: 
Room 5-T-57: 
441 G Street, NW: 
Washington, D.C. 20548:

Dear Mr. Robertson:

Thank you for the opportunity to review and comment on the draft report 
"Social Security Administration --More Effort Needed to Assess 
Consistency of Disability Decisions" (GAO-04-656). Our comments on the 
report are enclosed.

If you have any questions, please have your staff contact Candace 
Skumik, Director, Audit Management and Liaison Staff, at (410) 965-
4636.

Sincerely,

Signed by: 

Jo Anne B. Barnhart: 

Enclosure:

SOCIAL SECURITY ADMINISTRATION	BALTIMORE MD 21235-0001:

COMMENTS ON THE GENERAL ACCOUNTING OFFICE (GAO) DRAFT REPORT "SOCIAL 
SECURITY ADMINISTRATION --MORE EFFORT NEEDED TO ASSESS CONSISTENCY OF 
DISABILITY DECISIONS" (GAO-04-656):

Thank you for sharing the draft report with us.

We have several reservations about the findings, conclusions and 
recommendations of this report. Our overall impression is that GAO has 
focused on the shortcomings of "process unification," but is hesitant 
to fully discuss the progress that the Social Security Administration 
(SSA) has achieved in not only investigating and analyzing the 
underlying causes of inconsistent decision-making between the State 
disability determination service (DDS) examiners and Administrative Law 
Judges (ALJs), but also the Agency efforts to deal with the problem.

For example, the draft report frequently notes that "SSA's assessments 
have not provided a clear understanding on the extent and causes of 
possible inconsistencies in decisions between adjudication levels." We 
believe that the data obtained and analyzed over the last 10 years have 
supplemented and further added to the findings and conclusions of the 
original 1994 Disability Hearings Quality Review Process (DHQRP) 
Report.

When discussing the data provided by SSA's reports, the GAO report 
seems to have overlooked some of the key findings in these reports. For 
example, although GAO acknowledged that the medical consultant/
disability examiner (MC/DE) review showed an improved hearing allowance 
support rate of 57 percent in fiscal years (FY) 2001-2002 over the FYs 
1993-1994 support rate of 36 percent, it dismisses this improvement as 
portraying only a partial picture. However, a 21-percentage point 
increase over this period of time is tremendous - especially given the 
fact that it occurred during a period of time in which the hearing-
level allowance rate increased. Moreover, findings of the peer review 
process conducted by detailed ALJs, i.e., reviewing judges (RJ), 
confirm that allowance quality is on the increase. More importantly, 
for process unification purposes, the "gap" between these MC/DE and RJ 
support rates is narrowing, indicating that the process unification 
initiatives have resulted in improved decisional accuracy and increased 
compliance with Agency program policy.

Disability decision-making is a complex process at the DDS level, and 
even more so at the hearing level given the potential impact of 
testimony and a de novo hearing. The process takes into account both 
objective and subjective factors-and all cannot be quantified. As such, 
running the multivariate analyses recommended by GAO would not 
establish the total cause and effect they seek in trying to solve the 
enigma of inconsistent decision-making at different levels. Over the 
past 10 years, the Agency has conducted several analyses that take into 
account case characteristics. While additional studies can be 
conducted, we believe that these analyses have provided the Agency with 
a solid understanding of how certain variables influence disability 
decision-making.

We are disappointed in GAO's view that our approach to tracking annual 
allowance rates and trends is "simplistic and inconclusive" because it 
uses "snapshot" data rather than tracking cohorts of cases through the 
appeals process. Although tracking a cohort of cases is a viable 
alternative, we disagree with the GAO assessment of our approach. We 
believe our data are representative of each year analyzed; i.e., about 
6 years of disability dispositions. As such, there were no DDS and 
Office of Hearings and Appeals (OHA) disability dispositions 
representing a different set of demographics and case characteristics 
during that period of time that we failed to include in our analyses. 
Given the sheer volume of cases adjudicated by both the DDSs and OHA 
during a relatively stable economic period, we believe our findings 
provided an accurate portrayal of the allowance rate dynamic (see table 
below). Not only were DDSs allowing more cases, but their accuracy was 
high as well. This is indicative of adjudicators making the right 
decision sooner in the process-a goal of both process unification and 
the Commissioner's new disability approach.

[See PDF for table]

[End of table]

One of GAO's recommendations calls for expanding the DHQRP to permit 
the MC/DEs to review the hearing tape as part of their ongoing review 
process. Earlier, we had indicated to GAO that resources would be a 
concern as review time could be increased by about 45 minutes per 
reviewer (e.g., if one DE and three MCs review a case, the additional 
time would equal 3 hours). However, as a compromise, we propose 
conducting a test in which the DEs audit the hearing tape. As such, we 
would gain the benefit of the DE's assessment at a much smaller 
resource cost. We would then report our findings and conclusions 
concerning whether the DE audit would impact the outcome of the review 
process.

We are also concerned about comments in the report regarding the 
failure to make changes as a result of findings from reviews of the 
decisions. On page 15, the report states, "As part of the initiative, 
the agency also implemented a review of judges' allowances decisions to 
identify additional ways to improve training and policies, but no new 
changes were made as a result of findings from the review." On page 19, 
the report states, "However, according to a SSA official, this review 
did not identify any new areas of inconsistency that required 
improvements to policy and training." And the final recommendation of 
the report suggests that SSA should "clarify guidance for making 
disability decisions and develop mandatory training for adjudicators on 
issues identified as contributing to inconsistency." We disagree with 
the assertion that SSA has not addressed training needs that have been 
identified as the result of these reviews. In fact, SSA provides a 
series of Interactive Video Training (IVT) sessions focusing on 
problematic areas noted in the ALJ Peer Review Report. These broadcasts 
have occurred bi-monthly, cover a broad range of topics and all ALJs 
are required to view them, either during a live broadcast or by viewing 
a videotape of the broadcast at a later date.

On September 25, 2003, the Commissioner testified before the House Ways 
and Means Subcommittee on Social Security and presented her approach to 
improve the disability determination process. She explained that in 
designing the approach, she listened to a number of interested parties 
who expressed concerns about applying policy consistently in terms of 
(1) the DDS decision and ALJ decision; (2) variations among the state 
DDSs; and (3) variations among individual ALJs. A main theme of the new 
approach is to make the right decision as early in the process as 
possible. We are pleased with GAO's recognition that several aspects of 
the new approach are intended to improve the accuracy, timeliness, and 
consistency of decisions, such as having the DDS decision-makers more 
fully develop and document their decisions, providing for centralized 
quality review of all decisions, and providing all adjudication levels 
equal access to medical expertise. The new approach takes a 
comprehensive look at quality at each step of the disability 
adjudication process. At its core, the approach includes an in-line 
quality assurance process and a centralized quality control review that 
will replace the current SSA quality assurance system. This shift to an 
in-line quality assurance review will provide greater opportunities for 
identifying problem areas and implementing corrective actions and 
related training. Likewise, a centralized quality control review will 
provide end-of-line reviews and timely feedback to disability decision-
makers.

The main theme of the GAO report is that in the current disability 
process, the DDS examiners and the ALJ's may be interpreting SSA's 
criteria differently. This inconsistency was also a major theme in the 
comments that were considered in designing the new approach. A key 
aspect of the approach is the creation of feedback loops between each 
stage of the disability process that are intended to increase 
accountability between decisional levels and reduce decisional 
inconsistency. For example, ALJ's will be expected to provide feedback 
to reviewing officials, and reviewing officials will likewise provide 
feedback to the DDSs. Meanwhile, the DDSs will have their own in-line 
quality assurance process that will provide consistency within each 
State entity. In short, we will be ensuring quality and decisional 
consistency at all levels of the disability determination process by 
having "downstream" adjudicators provide essential feedback and 
management information to earlier "upstream" adjudicators.

However, we would like to clarify several aspects of the new approach 
that are described in the GAO report. The report notes that "providing 
both adjudication levels with equal access to more centralized medical 
expertise" could improve decisional consistency. The new approach 
relies on ensuring availability of enhanced medical expertise to all 
levels of adjudication - DDS examiners, Reviewing Officials, ALJs and 
Oversight Panels. It is critically important that a medical expert with 
the appropriate expertise be available to assist the adjudicators.	
However, no final decision has been made concerning the locations of 
these experts. With e-Dib in place, access to the appropriate experts 
will be available, regardless of location.

The GAO report also notes that decisional consistency will be improved 
by "requiring ALJs to address agency reports that either recommend 
denying the claim or outline the evidence needed to fully support the 
claim." We believe that decisional consistency and improved quality 
will be achieved only if greater accountability is established at each 
stage of the disability process. As noted earlier, the new approach to 
disability determinations provides for feedback at all levels of 
adjudication. For example, although we expect that e-Dib, and 
especially implementation of the Electronic Disability Collect System 
(EDCS), will greatly enhance the information received by the DDS 
examiners from the SSA field office (FO), if the quality of this 
information is not adequate, we expect feedback from the DDS to the 
SSA. FO. Similarly, if the Reviewing Official (RO) does not find that 
the DDS examiner's explanation of the determination is adequate, the RO 
will remand the case to the DDS to ensure that the DDS provides 
adequate documentation for the basis of the decision. If the RO 
disagrees with the determination of the DDS examiner, the RO will be 
expected to explain to the DDS examiner the reasons that the RO reached 
a different decision. Of course, if the claimant requests a hearing, 
the ALJ will be issuing a de novo decision. While the RO decision is 
not controlling and will be accorded no weight by the ALJ, if the ALJ 
disagrees with the RO decision, the ALJ will be expected to give 
feedback to the RO by addressing the matter in the decision. This 
feedback will be crucial to ensuring that the RO understands the basis 
for the ALJ disagreement so that subsequent evaluations by the RO will 
be more responsive. The improved documentation requirements and the 
feedback loops at each stage of the process are designed to ensure that 
each decision maker is consistently interpreting and applying SSA 
criteria. This enhanced accountability at all stages of the process 
will be critical to ensure improved decisional consistency.

And while the GAO report portrays the implementation of an electronic 
disability system as a vulnerability to the implementation of the new 
disability approach, it is also important to recognize the value of the 
system as an essential tool that will allow SSA to capture critical 
management information needed to reduce decisional variance. Analyzing 
this management information will help determine whether decisional 
variance results from human error, overly complex policies, or faulty 
training. Thus, either critical training resources can be applied where 
needed or complex policies that might be driving inconsistent decision 
making can be simplified. In short, this electronic management 
information model will provide a wealth of opportunities to highlight 
and remedy pockets of decisional variance at all levels.

As we move forward, we will consider GAO's recommendations vis-a-vis 
methodology changes for quality measures, as appropriate. We are deeply 
committed to improving both the quality of and the consistency within 
disability decision-making across SSA's adjudicatory process. However, 
the recommendations in the report will need to be revaluated as the 
design of the new disability approach matures.

We have the following technical comments on the report:

* To avoid any confusion, we suggest using the term ALJ instead of 
judge throughout the report, unless referring to a district court 
judge.

* On page 7, Figure 1, the chart apparently was derived from an SSA 
document; the fiscal year (FY) 2003 Waterfall Chart prepared by Agency 
staff for GAO. It omits the footnote that explains that the data is not 
longitudinal. Because the chart is not longitudinal, using a cohort of 
claims, the GAO methodology produces results that are very different 
from those experienced in the actual disability determination process 
and subjects the results to the same critique GAO makes of our failure 
to follow a cohort. We would suggest replacing the chart with the cross 
sectional waterfall we provide Congress for their "Green Book."

* Also, the chart (page 7, Figure 1) fails to note that about 25 
percent of the initial claims workload is subject to an alternate 
process that does not include the reconsideration step, which affects 
the appellate level data in the chart. Footnote #1 from the FY 2003 
Waterfall Chart would address those issues.

* The first full sentence at the top of page 12 is unclear. We suggest 
using two sentences: "Instead, studies indicate that systemic 
differences in the assessment of claims at both adjudication levels are 
contributing to the ALJ allowance rate. An example of one such study is 
our 1997 report that noted that differences in the views held by state 
DDSs and ALJs regarding a claimant's functional abilities was a key 
factor explaining why ALJs allowed cases on appeal."

* On page 12, footnote #11, is misleading. We believe it is more 
appropriate to read as follows: "The overall Agency cost, both Federal 
and DDS costs, for an average initial claim decision was about $812, 
while...."

GAO Comments:

1. We maintain that our report fully and fairly describes SSA's 
progress in analyzing and addressing the underlying causes of 
inconsistent decisions between state DDS examiners and ALJs. Our 
research included an extensive review of agency documentation and 
interviews with SSA officials, as well as stakeholder groups for 
adjudicators and claimant representatives, to develop a complete 
understanding of the agency's efforts to assess and improve the 
consistency of decisions between adjudication levels. Also, in 
agreement with our requestor, we sought to expand the review to include 
SSA's new approach to improving its disability programs, so that we 
could provide the Congress with an understanding of how SSA's future 
plans may help to address this issue.

2. We provided information on the various reviews and analyses of 
disability decisions to assess the consistency of decisions between 
adjudication levels conducted by SSA over the last 10 years, but none 
of these reviews have clearly identified the causes of inconsistency in 
decisions between adjudication levels.

3. Our report has not overlooked the data cited by SSA. Nevertheless, 
our conclusion about the improvement in consistency between levels 
indicated by the data is not as optimistic as SSA's because of 
weaknesses in SSA's assessments. As we reported, for 10 years SSA's 
analysis of the quality reviewers' assessment of ALJ cases has been 
limited to calculating ALJ support rates. SSA has not used available 
data to determine the potential areas of inconsistency between levels 
nor the extent to which changes in the ALJ support rate are related to 
improvements in consistency of decisions between adjudication levels. 
SSA's assessment only provides a general indication of overall changes 
in consistency at one adjudication level.

4. Our report recognizes that SSA's disability decision-making process 
is complex. Because of this complexity, we believe that multivariate 
analysis is an appropriate assessment tool that would allow SSA to 
assess the effect of multiple factors. In recommending this 
sophisticated tool, we were careful not to imply that causes and 
effects of inconsistent decision making can be established with 
certainty. However, we believe that such an analysis will help SSA 
understand the relative importance of the variety of factors that 
affect its decision-making process. After identifying areas of 
inconsistency, SSA can target these areas with in-depth case analyses 
to pinpoint the causes of inconsistency and develop a more effective 
strategy for addressing inconsistency. On the basis of our review of 
SSA's analyses to date, we do not agree with the implications of SSA's 
comments that it has a solid understanding of how certain variables 
influence disability decision making, and therefore does not need to 
conduct additional, more sophisticated analyses.

5. We agree with SSA that the proportion of allowances made at each 
level can provide some insight into the allowance rate dynamic. 
However, as we reported, we do not believe that it can serve as a 
reliable indicator of the agency's progress in achieving more 
consistent decisions between the DDS and OHA levels. The allowance data 
provided by SSA simply show that the relative proportion of allowances 
made at the DDS level increased in comparison with the OHA level, but 
SSA has not performed any additional analysis to show that these 
changes have any relationship to improved consistency in decision 
making between the two adjudication levels. Additional analysis is 
needed because a myriad of factors, such as changes in the economy, can 
affect allowance rates. Although SSA claims that over this period of 
time the economy has been "relatively stable," without performing any 
additional analysis it cannot eliminate changes in the economy or 
demographics of claimants as an influence on the allowance rates at 
each level. In addition, SSA has not analyzed how other factors, such 
as changes in productivity and total number of decisions made at each 
level, may be influencing the allowance data.

6. The allowance rate data provided by SSA in its comments is very 
similar to that provided by SSA earlier to us and included in our 
report in figure 2. The figures we reported for the proportion of 
allowances made by the DDS and OHA levels for fiscal years 1997 and 
1998 vary in comparison with those provided by SSA by one percentage 
point. We have not changed the figures in our report because we believe 
that these slight differences simply reflect that we reported data 
based upon fiscal, not calendar, years.

7. In our report, our statements that SSA has not made changes as a 
result of findings from its reviews were specifically related to SSA's 
ALJ pre-effectuation review. We included information on this review 
because it was part of SSA's process unification initiative and was 
intended to identify policy and training areas associated with 
inconsistent decisions between adjudication levels. During our review, 
we were told by an SSA official that the ALJ pre-effectuation review 
was not successful at identifying new areas of inconsistency to be 
addressed by SSA. In its comments, SSA cites a review unrelated to 
assessing the inconsistency of decisions between levels, the ALJ peer 
review, to assert that it has used reviews to identify training issues 
to improve the quality of decisions. The lack of success with the ALJ 
pre-effectuation review--along with other findings showing a limited 
understanding of the cause of inconsistency--supports our 
recommendations to SSA to perform additional analysis and to clarify 
guidance and provide mandatory training to address any identified 
causes of inconsistency between adjudication levels.

8. We applaud SSA's plans to use the electronic disability system to 
capture critical management information to address decisional variance 
or inconsistency, which could provide a wealth of useful information 
for the agency. We have adjusted our report's text to reflect this 
additional purpose. We continue to believe that SSA should not wait for 
the development of this system, but should proceed to perform 
multivariate analysis, using available data from its biennial case 
reviews, to start identifying areas of potential inconsistency between 
adjudication levels.

9. We applaud SSA's deep commitment to improving the disability 
decision-making process, but believe that additional efforts to 
understand the causes of potential inconsistencies in decision making 
would help to inform the design of the Commissioner's new approach and 
should, therefore, be undertaken immediately.

10. We generally agree with the technical comments provided and changed 
the text accordingly.

[End of section]

Appendix III: GAO Contacts and Staff Acknowledgments:

GAO Contacts:

Michele Grgich, Assistant Director (415) 904-2183 Julie DeVault, 
Analyst in Charge (415) 904-2269:

Staff Acknowledgments:

In addition to the individuals mentioned above, the following staff 
members made major contributions to this report: Michael Morris, 
Corinna Nicolaou, Walter Vance, and Rebecca Woiwode. Douglas Sloane 
provided assistance with methodological issues, and Daniel Schwimer 
provided legal support.

FOOTNOTES

[1] The figures include federal payments for the Disability Insurance 
and the Supplemental Security Income programs to beneficiaries who have 
a disability or are blind and their families. 

[2] For additional information on the major problems facing federal 
disability programs and our decision to designate them as high-risk, 
see U.S. General Accounting Office, High-Risk Series: An Update, 
GAO-03-119, (Washington, D.C.: January 2003).

[3] We took additional steps to assess the reliability of data included 
in our report. For the proportion of initial and hearings levels 
allowances, we verified SSA's calculations of summary data from its 
workload reporting systems. For the information from the Disability 
Hearings Quality Review, we reviewed the weights and calculations used 
by SSA to determine national support rates for reviewed decisions and 
found them to be correctly calculated and reliable. 

[4] These groups included the National Association of Disability 
Examiners, the National Council of Disability Determination Directors, 
the Association of Attorney Advisors, the Association of Administrative 
Law Judges, the National Treasury Employee Union, the Public Employees 
Federation, the Social Security Section of the Federal Bar Association, 
and the National Organization of Social Security Claimants' 
Representatives. 

[5] The figures include federal payments for the Disability Insurance 
and the Supplemental Security Income programs to beneficiaries who have 
a disability or are blind and their families. The figures were 
calculated based on statistical information from SSA's web site.

[6] These figures are based on information provided on SSA's web site 
from the 2004 OASDI Trustees Report, Part III. Financial Operations of 
the Trusts Funds and Legislative Changes in the Last Year.

[7] Others who may testify at ALJ hearings frequently include 
vocational and medical experts.

[8] Substantial gainful work is a level of work activity that involves 
doing significant physical or mental work, or a combination of both, 
that is productive. SSA has established earning criteria as a 
reasonable indication of whether claimants are able to engage in 
substantial gainful activity. In 2004, SSA generally considered 
claimants to be engaging in substantial gainful activity if their 
earnings averaged over $810 a month.

[9] See Implementation of Section 304 (g) Public Law 96-265, Social 
Security Disability Amendments of 1980 (the Bellmon Report), Secretary 
of Health and Human Services (Washington, D.C.: January 1982).

[10] Findings from SSA's 1994 Disability Hearings Quality Review Process 
report provided the agency with additional information on potential 
causes of inconsistency. The report identified two assessment areas 
associated with inconsistent decisions. In addition, quality reviewers 
found that when applying standards used by the initial level to 
adjudicate claims, 29 percent of the appealed DDS reconsideration 
decisions and 51 percent of ALJ decisions were not supported by the 
decision makers. These findings help to support SSA's decision to 
include efforts to have DDSs more fully develop and document their 
decisions and to assess ALJ decisions as part of its process 
unification initiative. 

[11] U.S. General Accounting Office, Social Security Disability: SSA 
Must Hold Itself Accountable for Continued Improvement in Decision-
Making, GAO/HEHS-97-102 (Washington, D.C.: Aug. 12, 1997). 

[12] The board also identified factors that could affect consistency 
within adjudication levels, as opposed to strictly between adjudication 
levels. For more information see the Social Security Advisory Board, 
Disability Decision Making: Data and Materials, January 2001, pp. 5-6.

[13] An appeal adds significantly to costs associated with making a 
decision. According to SSA's Performance and Accountability Report for 
fiscal year 2001, the average cost per claim for an initial DDS 
disability decision was about $583, while the average cost per claim of 
an ALJ decision was an estimated additional $2,157. 

[14] An appeal also significantly increases the time required to reach 
a decision. According to SSA's Performance and Accountability Report 
for fiscal year 2003, the average number of days that claimants waited 
for an initial decision was 97 days, while the number of days they 
waited for an appealed decision was 344 days. However, the time a 
claimant waits for a decision should not impact the amount of benefits 
received, as benefits are based on the date the claimant becomes 
disabled. 

[15] U.S. General Accounting Office, Performance and Accountability 
Series: Major Management Challenges and Program Risks: Social Security 
Administration, GAO-03-117 (Washington, D.C.: January 2003). 

[16] Social Security Ruling 96-2: Policy Interpretation Ruling Titles 
II and XVI: Giving Controlling Weight to Treating Source Medical 
Opinions, effective July 2, 1996.

[17] Since SSA rulings are binding only on SSA adjudicators and do not 
have to be followed by the courts, SSA planned to strengthen the impact 
of several rulings by creating regulations that would be followed by 
the courts. 

[18] These guidelines--called the Program Operations Manual System 
(POMS)--contain, within an estimated 30,000 pages, interpretations of 
relevant statutes, regulations, and rulings and procedural information.

[19] Additional DDS and OHA sites may have taped and viewed this 
training, but SSA's monitoring of training provides only the number of 
sites logged on to the interactive video training. 

[20] U.S. General Accounting Office, Social Security Administration: 
Strategic Workforce Planning Needed to Address Human Capital Challenges 
Facing the Disability Determination Services, GAO-04-121 (Washington, 
D.C.: Jan. 27, 2004). 

[21] When reviewing an ALJ allowance, OQA uses the preponderance of 
evidence standard, which requires that the reviewer fairly consider all 
evidence and decide whether the weight of that evidence supports the 
allowance. 

[22] The Appeals Council uses a substantial evidence standard of review 
that requires the reviewer to determine that the evidence in the case 
is sufficient to convince a reasonable mind of the credibility of the 
allowance decision, and that there is no opposing evidence that clearly 
compels another finding or conclusion. 

[23] At the hearings level, ALJs can purchase medical evidence, at 
their discretion. 

[24] See the Social Security Administration, Office of Program and 
Integrity Reviews, Findings of the Disability Hearings Quality Review 
Process: An Assessment of the Quality of Hearing Decisions and Appealed 
Reconsideration Denial Determinations, September 1994.

[25] An OQA official also said that results from another review bolster 
the conclusion that the quality of ALJ decisions has improved. 
Specifically, in the ALJ peer review, in which ALJs evaluate a random 
sample of other ALJs' decisions, the reviewing ALJs have found an 
increasing percentage of ALJ decisions to be supportable. The percent 
of decisions found to be supportable increased from 81 percent to 90 
percent from the reviews of decisions issued from fiscal years 1993 
through 1994 compared with decisions from fiscal years 2001 through 
2002.

[26] See the Social Security Administration, Office of Quality 
Assurance and Performance Assessment, Quality Assurance Group I Initial 
Denial Accuracy Report: August-October 2002, July 2003.

[27] In addition, the report recommended that SSA include cases 
appealed to its Appeals Council in the sample for its biennial case 
review to eliminate the systematic bias in that sample and make it 
representative of all cases that receive a decision from the hearings 
level. For more information on this recommendation and others, see U.S. 
General Accounting Office, SSA Disability Decision Making: Additional 
Steps Needed to Ensure Accuracy and Fairness of Decisions at the 
Hearings Level, GAO-04-14 (Washington, D.C.: Nov. 12, 2003).

[28] See the Lewin Group, Inc., et al., Evaluation of SSA's Disability 
Quality Assurance (QA) Processes and Development of QA Options That 
Will Support the Long-Term Management of the Disability Program, (Falls 
Church, VA: Mar. 16, 2001), pp. 168 and E-9. 

[29] For additional information on SSA's progress and its risky 
strategy for implementing its accelerated disability claims system, see 
U.S. General Accounting Office, Electronic Disability Claims 
Processing: Social Security Administration's Accelerated Strategy 
Faces Significant Risks, GAO-03-984T (Washington, D.C.: July 24, 2003); 
Social Security Administration: Subcommittee Questions Concerning 
Efforts to Automate the Disability Claims Process, GAO-03-1113R 
(Washington, D.C.: Sept. 5, 2003); and Electronic Disability Claims 
Processing: SSA Needs to Address Risks Associated with Its Accelerated 
Systems Development Strategy, GAO-04-466 (Washington, D.C.: Mar. 26, 
2004).

[30] This excerpt is taken from a statement by the Honorable Jo Anne B. 
Barnhart, Commissioner, Social Security Administration, Testimony 
before the Subcommittee on Social Security of the House Committee on 
Ways and Means, September 25, 2003.

GAO's Mission:

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. General Accounting Office
441 G Street NW,
Room LM Washington, D.C. 20548:

To order by Phone: 	

Voice: (202) 512-6000:

TDD: (202) 512-2537:

Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm 
E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director, 
NelliganJ@gao.gov 
(202) 512-4800 

U.S. General Accounting Office, 
441 G Street NW, Room 7149 
Washington, D.C. 20548: