Nursing Home Quality: CMS Should Improve Efforts to Monitor Implementation of the Quality Indicator Survey [Reissued on March 9, 2012]
Highlights
What GAO Found
CMS has commissioned three studies to evaluate the QIS-based survey process but does not routinely monitor the extent to which the objectives established for the QIS are being met. The first two studies, completed in 2006 and 2007, were commissioned to determine whether the new survey process could be used in real-world settings and whether the QIS was helping the agency meet several of the objectives established for the process. The studies’ findings suggested that surveyors could use the survey process and recommended changes intended to help improve the QIS. CMS officials reported taking steps to address the studies’ findings and recommendations. In 2009, CMS commissioned a third study that was completed in 2011 and identified aspects of the QIS process that could affect the consistency with which surveyors identify quality problems. For example, the study found that during resident interviews, surveyors did not consistently probe for further information when provided with incomplete responses to interview questions. However, CMS does not have the means to routinely monitor the extent to which the QIS is helping improve the survey process as intended. Such routine, ongoing monitoring would be consistent with federal internal control standards and could include the use of performance goals and measures. CMS does have access to some data, such as the amount of time surveyors have spent inspecting facilities, that could be used to help develop performance goals and measures.
CMS has taken some steps to monitor and facilitate states’ implementation of the QIS-based routine survey, but CMS’s efforts are not systematic. As part of the agency’s efforts to monitor states’ implementation, CMS primarily uses quarterly teleconferences with state survey agency officials to obtain information on the extent to which each state has completed training all its surveyors to use the QIS. However, states may not always participate in the teleconferences, and those that do may not provide complete information on their progress. As a result, the information CMS obtains through its monitoring of states’ progress may be incomplete. For example, CMS was unable to provide GAO with information on training completion dates of all state surveyors in states that completed training. Although CMS reported it plans to develop a more systematic process for monitoring states’ implementation progress, it has not established a time line for doing this. To help facilitate states’ implementation of the QIS, CMS provides states with guidance, gives presentations, and offers states opportunities to share their implementation experiences through quarterly teleconferences. However, CMS does not have a systematic method for obtaining, compiling and sharing information on state experiences, especially information on approaches states have taken to help facilitate implementation of the QIS. Systematically sharing such information—for example, through CMS’s annual conference in which all state survey agencies participate—could help the agency facilitate implementation in states that have not begun QIS implementation.
Why GAO Did This Study
The Centers for Medicare & Medicaid Services (CMS) is responsible for ensuring that nursing homes receiving federal funding meet federal quality standards. CMS contracts with state survey agencies to conduct periodic inspections of nursing homes using a CMS-developed survey process. Weaknesses in the survey process led CMS to develop and begin implementing the Quality Indicator Survey (QIS), a new electronic survey. CMS developed the QIS to achieve several objectives, including improving the efficiency, accuracy, and consistency of the survey process. CMS expects the QIS to be implemented nationally by 2018. We examined the extent to which CMS (1) evaluates whether progress is being made in meeting the objectives of the QIS and (2) monitors and facilitates states’ implementation of the QIS. To do this, GAO reviewed relevant CMS documents, including CMS-commissioned studies of the QIS, policies, and guidance. GAO also interviewed officials from CMS’s Central and all 10 Regional Offices as well as officials from a judgmental sample of nine state survey agencies.
Reissued on March 9, 2012
Recommendations
GAO recommends that CMS develop a means to routinely monitor the extent to which progress is being made in meeting the objectives of the QIS and systematic methods for monitoring and facilitating states’ efforts to implement the QIS. The Department of Health and Human Services fully concurred with all of the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Centers for Medicare & Medicaid Services | As CMS continues to implement the QIS, the Administrator of CMS should develop a means--such as performance goals and measures--to routinely monitor the extent to which CMS is making progress in meeting the objectives established for the QIS. |
Consistent with our recommendation, CMS has taken steps to monitor QIS performance, including preparing quarterly reports that measure QIS outcomes. We note that CMS has indicated that due to challenges with QIS it is refraining from implementing QIS in additional states while it focuses on improving and adjusting the system. Moving forward, it is important for CMS to develop the means to monitor whether any future survey methodology, including QIS, is meeting its intended objectives. CMS provided information to support closing this recommendation in February 2016.
|
| Centers for Medicare & Medicaid Services | As CMS continues to implement the QIS, the Administrator of CMS should develop and implement a systematic methodology to track state survey agencies' progress with implementation activities. |
Consistent with our recommendation, CMS has begun providing survey frequency and completion rates, as well as information on other aspects of state survey agency performance, in quarterly reports to state survey agencies. In addition, CMS provides monthly reports to CMS regional offices to describe state survey agency performance. CMS has indicated that it is refraining from implementing the QIS in additional states, and thus training surveyors in those states, until certain improvements have been made. CMS provided information to support closing this recommendation in February 2016.
|
| Centers for Medicare & Medicaid Services | As CMS continues to implement the QIS, the Administrator of CMS should develop and implement a systematic method for obtaining, compiling, and sharing information from state survey agencies about their implementation experiences. |
Consistent with our recommendation, CMS has provided information about the QIS, including relevant guidance and updates for states and surveyors, on the CMS website. In addition, CMS has begun inviting all states to participate in their quarterly teleconferences, regardless of whether the states are currently implementing the QIS or still using the traditional survey process. CMS reported implementing these methods as of April 2013.
|