Hospital Quality Data: CMS Needs More Rigorous Methods to Ensure Reliability of Publicly Released Data
GAO-06-54
Published: Jan 31, 2006. Publicly Released: Feb 24, 2006.
Skip to Highlights
Highlights
The Medicare Modernization Act of 2003 directed that hospitals lose 0.4 percent of their Medicare payment update if they do not submit clinical data for both Medicare and non-Medicare patients needed to calculate hospital performance on 10 quality measures. The Centers for Medicare & Medicaid Services (CMS) instituted the Annual Payment Update (APU) program to collect these data from hospitals and report their rates on the measures on its Hospital Compare Web site. For hospital quality data to be useful to patients and other users, they need to be reliable, that is, accurate and complete. GAO was asked to (1) describe the processes CMS uses to ensure the accuracy and completeness of data submitted for the APU program, (2) analyze the results of CMS's audit of the accuracy of data from the program's first two calendar quarters, and (3) describe processes used by seven other organizations that assess the accuracy and completeness of clinical performance data.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Centers for Medicare & Medicaid Services | In order for CMS to help ensure the reliability of the quality data it uses to produce information on hospital performance, the CMS Administrator should, focusing on the subset of hospitals for which it is statistically uncertain if they met CMS's accuracy threshold in one or more previous quarters, increase the number of patient records reabstracted by the Clinical Data Abstraction Center in a subsequent quarter so that the proportion of hospitals with statistically uncertain results is reduced. |
In its August 2009 final rule to implement FY 2010 changes in the Medicare Inpatient Prospective Payment System, CMS alerted hospitals that starting in 2010 it will change its procedure for checking the accuracy of quality data that hospitals submit quarterly in order to receive their full annual payment update. Rather than reviewing 5 patient records per quarter from every hospital, CMS will henceforth review 12 patient records from a randomly selected sample of 800 hospitals. We had recommended that CMS increase the number of patient records it reviewed for hospitals with statistically uncertain results. Although the procedure that CMS adopted differs somewhat from the approach that we proposed, it addresses the problem we identified through an alternative mechanism that accomplishes the same result -- reducing an excessive degree of statistical uncertainty that arose from insufficiently large samples by focusing the data validation effort on a selected subset of hospitals in any given year. CMS noted that this increase in sample size would provide more reliable estimates of the accuracy of the hospital submitted data.
|
| Centers for Medicare & Medicaid Services | In order for CMS to help ensure the reliability of the quality data it uses to produce information on hospital performance, the CMS Administrator should require hospitals to certify that they took steps to ensure that they submitted data on all eligible patients, or a representative sample thereof. |
In its August 2009 final rule to implement FY 2010 changes in the Medicare Inpatient Prospective Payment System, CMS alerted hospitals that starting in 2010 they would be required to electronically acknowledge once a year the completeness and accuracy of the quality data that they submit quarterly in order to receive their full annual payment update. The rationale CMS provided for this requirement included a reference to our recommendation that CMS require hospitals to certify that the quality data they submitted were complete. CMS stated that this requirement would help to ensure that hospitals implemented the steps needed to guarantee the completeness and accuracy of the quality data that they submitted.
|
| Centers for Medicare & Medicaid Services | In order for CMS to help ensure the reliability of the quality data it uses to produce information on hospital performance, the CMS Administrator should assess the level of incomplete data submitted by hospitals for the APU program to determine the magnitude of underreporting, if any, in order to refine how completeness assessments may be done in future reporting efforts. |
A CMS official reported in a an email sent 7/26/2007 that CMS has addressed our recommendation on assessing the level of incomplete data through analyses that compared hospital quality data submissions with each hospital's Medicare claims submissions. In an email on 1/5/2009, the same official reported that CMS plans to conduct similar analyses in 2009. The official acknowledged that Medicare claims do not include all the patients for whom hospitals are required to submit data. CMS therefore plans to ask hospitals to self report their total number of Medicare and non-Medicare cases.
|
Full Report
Public Inquiries
Topics
Data collectionData integrityHospitalsMedical recordsPerformance measuresQuality assuranceQuality controlReporting requirementsStatistical dataProgram implementation