Social Security Disability:
Management Controls Needed to Strengthen Demonstration Projects
GAO-08-1053: Published: Sep 26, 2008. Publicly Released: Sep 26, 2008.
Since 1980, Congress has required the Social Security Administration (SSA) to conduct demonstration projects to test the effectiveness of possible changes to its Social Security Disability Insurance (DI) and Supplemental Security Income (SSI) programs that could decrease individuals' dependence on benefits or improve program administration. However, in 2004, GAO reported that SSA had not used its demonstration authority effectively. This follow-up report assesses (1) how SSA has used its demonstration authority to test DI and SSI program changes and what information these efforts have yielded and (2) what steps SSA has taken to improve the planning and management of its demonstration projects. To do this, GAO reviewed documents related to SSA's demonstration project management and the steps it took to implement the recommendations in the 2004 report, as well as the projects' designs, evaluations, and costs. GAO also interviewed officials from SSA, its contractors and project sites, and disability experts.
Over the last decade, SSA has initiated 14 demonstration projects under its authority to test possible DI and SSI policy and program changes; however, these projects have yielded limited information for influencing program and policy decisions. Of the 14 projects, SSA has completed 4, cancelled 5, and had 5 projects in progress as of June 2008. In total, SSA spent about $155 million on its projects as of April 2008, and officials anticipate spending another $220 million in the coming years on those projects currently under way. Yet, these projects have yielded limited information on the impacts of the program and policy changes they were testing. SSA did not conduct impact evaluations for two of its completed projects, and intended to evaluate five other projects, but could not do so because significant challenges led SSA to cancel them. SSA officials believe the five projects currently under way will yield useful information, but it is too early to tell. SSA has taken steps to improve its demonstration projects but continues to lack management controls to ensure that the projects yield reliable information for making disability policy decisions. SSA has used methodological designs that GAO determined were strong or reasonable when assessed against professional research standards for 11 of its 14 projects. SSA has also used external research professionals to work with the agency on the design, implementation, or evaluation of 12 of the projects, and appointed new program management to oversee its demonstration program. However, as of August 2008, SSA had not fully implemented the recommendations GAO made in 2004 and did not have written policies and procedures governing how it should review and operate its demonstration project program. Specifically, SSA does not have written policies and procedures for its managers and project officers to follow as they design, implement, and evaluate its demonstration projects. Absent such protocols, SSA did not always apply standard research practices, such as conducting pilot phases or obtaining sufficient stakeholder input, which led to data limitations and project cancellations.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: SSA noted existing processes and written procedures for managing and reviewing its programs, including the demonstration project program. The agency agreed with the need to develop a guidebook to assist staff in the design, implementation, and evaluation phases of demonstration projects and the value of piloting demonstration projects before proceeding with full implementation. In May 2011, SSA provided GAO with a copy of its revised "Demonstration Project Guidebook," which outlines the agency's policies, procedures, and mechanisms for managing and operating its demonstrations projects. GAO determined that it was consistent with research practices and GAO standards and that the guidebook addressed our recommendation regarding coordinating with internal and external stakeholders.
Recommendation: To improve SSA's management of its demonstration projects the Commissioner of Social Security should direct the Deputy Commissioner for the Office of Retirement and Disability Policy to establish written policies, procedures, and mechanisms for managing and operating its demonstration projects that are consistent with standard research practices and internal control standards in the federal government, including those for coordinating with internal and external stakeholders and sharing information with Congress.
Agency Affected: Social Security Administration