Medicaid Demonstrations: Evaluations Yielded Limited Results, Underscoring Need for Changes to Federal Policies and Procedures
Fast Facts
About one-third of Medicaid's spending goes toward demonstrations, which allow states to test new approaches to delivering Medicaid services. Do they save money? Improve care?
The short answer is that states and the federal government don't fully know. We found that the federal government did not require complete and timely evaluations from the states, so conclusive results were not available. Moreover, the federal government wasn't making its evaluation results public—missing opportunities to inform federal and state Medicaid policy discussions.
We recommended ways for the Centers for Medicare & Medicaid Services to address these issues.
Percentage of federal Medicaid spending that is under demonstration projects in each state for fiscal year 2015
Map showing 11 states with no spending on demonstrations, 10 states where 75% or more of Medicaid spending is on demonstrations.
Highlights
What GAO Found
Under section 1115 of the Social Security Act, the Secretary of Health and Human Services (HHS) may approve Medicaid demonstrations to allow states to test new approaches to providing coverage and for delivering services that can transform large portions of states' programs. However, GAO found that selected states' evaluations of these demonstrations often had significant limitations that affected their usefulness in informing policy decisions. The limitations included gaps in reported evaluation results for important parts of the demonstrations. (See table.) These gaps resulted, in part, from HHS's Centers for Medicare & Medicaid Services (CMS) requiring final, comprehensive evaluation reports after the expiration of the demonstrations rather than at the end of each 3- to 5-year demonstration cycle. CMS has taken a number of steps since 2014 to improve the quality of state-led evaluations, and in October 2017, officials stated that the agency planned to require final reports at the end of each demonstration cycle for all demonstrations. However, the agency has not established written procedures for implementing such requirements, which could allow for gaps to continue. CMS also plans to allow states to conduct less rigorous evaluations for certain types of demonstrations but has not established criteria defining under what conditions limited evaluations would be allowed.
Examples of Gaps in States' Evaluations of Medicaid Section 1115 Demonstrations
|
Arizona |
The state was required to evaluate whether providing long-term services and supports under a managed care delivery model improved access and quality of care. The evaluation report lacked information on important measures of access and quality. |
|
Arkansas |
The state was required to evaluate the effects of using Medicaid funds to purchase private insurance for more than 200,000 beneficiaries. The evaluation did not address a key hypothesis that using private insurance would improve continuity of coverage for these beneficiaries, who were expected to have frequent changes in income that could lead to coverage gaps. |
|
Massachusetts |
The state was required to evaluate the effectiveness of its approach of providing up to $690 million in incentive payments to seven hospitals to improve quality of care and reduce per capita costs. Evaluation reports submitted after 5 years provided no conclusions on the impact of the payments in these areas. |
Source: GAO. | GAO-18-220
Federal evaluations led by CMS have also been limited due to data challenges that have affected the progress and scope of the work. For example, delays obtaining data directly from states, among other things, led CMS to considerably reduce the scope of a large, multi-state evaluation, which was initiated in 2014 to examine the impact of state demonstrations in four policy areas deemed to be federal priorities. Though CMS has made progress in obtaining needed data, it is uncertain when results from the multi-state and other federal evaluations will be available to policymakers because CMS has no policy for making results public. By not making these results public in a timely manner, CMS is missing an opportunity to inform important federal and state policy discussions.
Why GAO Did This Study
Demonstrations—which represented roughly a third of the more than $300 billion in federal Medicaid spending in 2015—are a powerful tool to test new approaches to providing coverage and delivering Medicaid services that could reduce costs and improve beneficiaries' outcomes. Evaluations are essential to determining whether demonstrations are having their intended effects. States are required to evaluate their demonstrations and CMS can initiate its own federal evaluations of demonstrations.
GAO was asked to examine evaluations of demonstrations, including how the results have been used to inform Medicaid policy. This report examines (1) state-led evaluations and (2) federal evaluations. GAO reviewed evaluation documentation for eight states with high demonstration expenditures that varied in the number of years their demonstrations had been in effect and by geography. GAO also reviewed documentation for the ongoing federal evaluations and interviewed state and federal Medicaid officials. GAO assessed evaluation practices against federal standards for internal control and leading evaluation guidelines.
Recommendations
GAO recommends that CMS: (1) establish written procedures for requiring final evaluation reports at the end of each demonstration cycle, (2) issue criteria for when it will allow limited evaluations of demonstrations, and (3) establish a policy for publicly releasing findings from federal evaluations of demonstrations. HHS concurred with these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Centers for Medicare & Medicaid Services | The Administrator of CMS should establish written procedures for implementing the agency's policy that requires all states to submit a final evaluation report after the end of each demonstration cycle, regardless of renewal status. (Recommendation 1) |
In April 2018, HHS officials reported that the Centers for Medicare & Medicaid Services (CMS) had developed an enhanced set of special terms and conditions (STCs) related to evaluation and monitoring. These STCs specify that a final evaluation report must be submitted after the end of each demonstration approval period regardless of renewal status. In January 2020, CMS reported that it had completed standard operating procedures for ensuring that approved STCs consistently include all applicable standard agency provisions, including the requirement that final evaluation reports are submitted after the end of each demonstration period. As such, we consider this recommendation closed and implemented.
|
| Centers for Medicare & Medicaid Services | The Administrator of CMS should issue written criteria for when CMS will allow limited evaluation of a demonstration or a portion of a demonstration, including defining conditions, such as what it means for a demonstration to be longstanding or noncomplex, as applicable. (Recommendation 2) |
HHS agreed with this recommendation. In November 2019, HHS reported that it was applying limited evaluation requirements to certain demonstration types, including routine family planning demonstrations. In April 2021, CMS stated that it continues to work with states as they apply for new or extensions of approved demonstrations to determine whether the demonstrations as a whole or certain components would qualify for limited evaluation. The agency, however, reiterated that it needs more experience before developing generalized guidance and that was not a priority for 2021. As of February 2024, HHS officials have not informed us of any additional actions taken to implement this recommendation. We will continue to monitor CMS's progress and will review whether to close the recommendation when the criteria are issued.
|
| Centers for Medicare & Medicaid Services |
Priority Rec.
The Administrator of CMS should establish and implement a policy for publicly releasing findings from federal evaluations of demonstrations, including findings from rapid cycle, interim, and final reports; and this policy should include standards for timely release. (Recommendation 3) |
HHS concurred with this recommendation. In March 2020, CMS issued a standard policy for its review and clearance of federal evaluation findings for release to the public. The process entails three levels of internal review and is estimated to take a minimum of 16 weeks from the date a final evaluation is submitted to CMS. The policy notes that any level of review and final clearance may take additional time based on the extent of revisions to the draft evaluation. With the issuance of CMS's policy, we consider this recommendation closed and implemented.
|