Opioid Use Disorder: Opportunities to Improve Assessments of State Opioid Response Grant Program
Fast Facts
The Substance Abuse and Mental Health Services Administration (SAMHSA) administers the State Opioid Response Grant program for opioid use disorder prevention, treatment, and recovery services.
SAMHSA assesses the program through 2 annual performance reports. We recommended making such reports more useful by:
- Including assessment limitations: Knowing about any limitations—such as incomplete data—helps decision makers understand and draw reasonable conclusions from the reports
- Further assessing existing data: For example, a look at performance variations across states and demographics could help program managers improve client outcomes
Highlights
What GAO Found
The State Opioid Response (SOR) grant program provides grants to help address the negative effects of the opioid crisis, including overdose deaths. Through the program, the Substance Abuse and Mental Health Services Administration (SAMHSA) has awarded nearly $5.2 billion since 2018 to states, the District of Columbia, and U.S. territories for opioid-use disorder (OUD) prevention, treatment, and recovery support services.
Examples of Prevention, Treatment, and Recovery Support Services Grantees Provide Using State Opioid Response Grant Program Funding
SAMHSA primarily assesses the SOR grant program through its annual SOR program profile and report to Congress, according to SAMHSA officials. These reports present a high-level national snapshot of SOR program performance, such as nationwide changes in drug abstinence and housing stability among program participants. The report to Congress also describes how grantees are implementing the program, such as describing evidence-based treatments and practices used by grantees.
However, neither the program profile nor the report to Congress provide information on potential limitations associated with the assessments. For example, SAMHSA does not make it clear that the data used for the 2020 profile were incomplete for two-thirds of people participating in the program, which could potentially affect conclusions that could be drawn from this information. SAMHSA is working to improve the completeness of these data, but also identifying potential limitations and their effects on conclusions could help ensure that Congress and others can correctly interpret the data and make more fully informed decisions, such as if changes are necessary to the SOR program.
In addition, the program profile and report to Congress do not fully leverage information available to provide a more in-depth assessment of the SOR program. SAMHSA has the potential to use its existing data to gain further insights into how well the program is working and why. For example, looking for variation in program performance across states and demographic groups could help identify best practices and areas for improvement. Such insights could help identify opportunities to improve program effectiveness and client outcomes, which may help reduce opioid-related overdose deaths and improve the lives of the clients the program serves.
Why GAO Did This Study
The misuse of illicit and prescription drugs, including opioids, has been a long-standing and persistent problem in the U.S., representing a serious risk to public health that has become even greater during the COVID-19 pandemic. Provisional data estimate that drug overdose deaths increased 29 percent in the year ending in April 2021—to a record high of 100,306—with opioid-related overdose deaths making up three-quarters of the total.
SAMHSA leads federal public health efforts to address the opioid crisis, which include administering the SOR grant program, the agency's largest such program since the grant began in 2018.
GAO was asked to review SAMHSA's SOR grant program. This report examines how SAMHSA assesses the grant program, among other things.
GAO reviewed documents relevant to SAMHSA's SOR monitoring and program assessment efforts; interviewed SAMHSA officials; and analyzed documentation for 10 SOR grantees, selected to reflect a range of award amounts.
Recommendations
GAO is making two recommendations, that SAMHSA ensure its SOR grant program assessment efforts (1) identify potential limitations and how those limitations may affect the conclusions that can and cannot be drawn; and (2) further analyze existing program information to provide a more comprehensive, in-depth assessment of the program to identify opportunities for improvement. The Department of Health and Human Services concurred with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Substance Abuse and Mental Health Services Administration | The Assistant Secretary for Mental Health and Substance Use should ensure that SAMHSA's SOR grant program assessment reports, such as its annual SOR program profile and report to Congress, identify potential limitations in their findings, and describe how any such limitations may affect the conclusions that can and cannot be drawn about the effectiveness of the program. (Recommendation 1) |
SAMHSA concurred with this recommendation. In December 2022, SAMHSA reported that it was working to better leverage programmatic data to assess SOR grantee performance and ensure effectiveness of the program. SAMHSA expected that this approach would address GAO's recommendation by developing a limitations section for inclusion in SOR grant program assessment reports, profiles, and reports to Congress. According to SAMHSA, the limitations may include (but are not limited to) selection bias, incomplete data, and differences in grantee service delivery models. In November 2023, SAMHSA informed GAO that it had released its 2022 SOR Report to Congress. The report includes a limitations section and, according to SAMHSA, the new report format will be used for future SOR Reports to Congress. Consistent with GAO's recommendation, the limitations section describes potential limitations in data collected and report findings.
|
| Substance Abuse and Mental Health Services Administration | The Assistant Secretary for Mental Health and Substance Use should further analyze existing SOR grant program information, such as by disaggregating data by client groups, to provide a more comprehensive, in-depth assessment of program performance and use such information to identify opportunities for program improvement. (Recommendation 2) |
SAMHSA concurred with this recommendation. In December 2022, SAMHSA reported that it was working to better leverage programmatic data to assess SOR grantee performance and ensure effectiveness of the program. SAMHSA said this involves modifying data collection tools to better assess performance on key programmatic objectives as well as conducting additional analyses of program data. Specifically, SAMHSA said it was modifying the two Government Performance and Results Act (GPRA) data collection tools that SOR grantees must use to report performance measures, in order to better enable SAMHSA to analyze the impact the SOR program has on clients and populations served by more accurately capturing client demographics, services rendered, and client outcomes. In addition to revising its data collection tools, SAMHSA said it was conducting a more robust analysis of its current SOR data using data disaggregated by client characteristics to provide a more comprehensive, in-depth assessment of program performance. In June 2023, SAMHSA reported that it had begun using the new GPRA data collection tools. In November 2023, SAMHSA informed GAO that it had released its 2022 State Opioid Response Grants (SOR) Report to Congress. Consistent with GAO's recommendation, the 2022 report includes a new statistical analysis of program data disaggregated by client demographic groups, which identified disparities in outcomes across different client groups. The report also describes efforts SAMHSA is undertaking to address the identified disparities. According to SAMHSA, it will continue to conduct similar analyses as part of future SOR Reports to Congress.
|