Health Care Quality Measurement: HHS Should Address Contractor Performance and Plan for Needed Measures
Highlights
What GAO Found
NQF has made progress on projects under its contract activities, as of August 2011. Specifically, NQF has completed or made progress on 60 of 63 projects. For example, NQF has completed projects to endorse measures related to various topics, including nursing homes. However, for more than half of the projects, NQF did not meet or did not expect to meet the initial time frames approved by HHS. For example, NQF completed one project to retool measures—that is, convert previously endorsed quality measures to an electronic format. While the retooling project was expected to be completed by September 2010, its completion was delayed by 3 months. NQF and HHS officials identified various reasons that contributed to this delay, including an expansion of the project’s scope and complexity. As a result of the delay, HHS did not have all the retooled measures it expected to include in its Electronic Health Records (EHR) Incentive Program. The delay of this project was also a contributing factor to NQF exceeding its estimated cost for its entire contract activity related to EHR by about $560,000 in the second contract year— January 14, 2010, through January 13, 2011. While HHS monitored NQF’s progress through monthly progress reports and approved changes to time frames and costs, HHS did not use all of the tools for monitoring that are required under the contract. Specifically, HHS did not conduct an annual performance evaluation to assess timeliness and cost issues that could have helped to inform NQF’s future scope of work. Until August 2011, HHS did not enforce the provision for NQF to submit a financial graph to compare monthly costs for each contract activity with cost estimates, which is information not included in monthly progress reports. These tools could have provided additional, more detailed information to help identify instances in which NQF might have been at risk of not meeting time frames or exceeding cost estimates, which could have provided HHS an opportunity to make any appropriate changes to NQF’s activities.
HHS had used or planned to use about half of the measures—164 of 344—that it received from NQF under the contract, as of August 2011. For example, HHS used 44 measures that NQF retooled under the contract in its EHR Incentive Program. HHS officials stated that the 44 measures used in the program contained errors, which required corrections. HHS officials also have not yet tested the retooled measures to assess the feasibility of implementing them in the electronic format; therefore, HHS runs the risk that some of these measures may not work as intended when implemented. HHS officials told GAO they expect to evaluate if and how they could use all of the remaining measures HHS received under the contract. However, HHS has not determined how PPACA requirements for quality measurement may have changed its needs for endorsed quality measures. As a result, HHS has not established a comprehensive plan that identifies its measurement needs and time frames for obtaining endorsed measures and that accounts for relevant PPACA requirements. Without such a plan, HHS may be limited in its efforts to prioritize which specific measures it needs to develop and to have endorsed by NQF during the remainder of the NQF contract. As a result, HHS may be unable to ensure that the agency receives the quality measures needed to meet PPACA requirements, including time frames for implementing quality measurement programs.
Why GAO Did This Study
The Medicare Improvements for Patients and Providers Act of 2008 (MIPPA) directed the Department of Health and Human Services (HHS) to enter into a 4-year contract with an entity to perform various activities related to health care quality measurement. In January 2009, HHS awarded a contract to the National Quality Forum (NQF), a nonprofit organization that endorses health care quality measures—that is, recognizes certain ones as national standards. In 2010, the Patient Protection and Affordable Care Act (PPACA) established additional duties for NQF. This is the second of two reports MIPPA required GAO to submit on NQF’s contract with HHS. In this report—which covers NQF’s performance under the contract from January 14, 2010, through August 31, 2011—GAO examines (1) the status of projects under NQF’s required contract activities and (2) the extent to which HHS used or planned to use the measures it has received from NQF under the contract to meet its quality measurement needs, as of August 2011. GAO interviewed NQF and HHS officials, reviewed relevant laws, and reviewed HHS and NQF documents.
Recommendations
GAO recommends HHS: (1) use all monitoring tools required under the contract to help address NQFs performance, (2) complete testing of retooled measures, and (3) comprehensively plan for its quality measurement needs. HHS neither agreed nor disagreed with these recommendations. NQF concurred with many of the findings in the report and provided additional context.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Health and Human Services | To help ensure that HHS receives the quality measures it needs to effectively implement its quality measurement programs and initiatives within required time frames, the Secretary of HHS should use monitoring tools required under the NQF contract to obtain detailed and timely information on NQF's performance and use that information to inform any appropriate changes to time frames, projects, and cost estimates for the remaining contract years. |
HHS reported that it began to use monitoring tools required under the NQF contract, including the performance evaluation and financial graphs. HHS completed an interim performance evaluation of NQF for the period covering January 14, 2010 through January 13, 2013. HHS also noted that it began to receive financial information as part of the monthly progress reports and NQF's invoices began to include charts that compared the direct costs, as well as actual versus budgeted costs. To make the data more useful, HHS requested that the format be revised to provide more data in a spreadsheet.
|
| Department of Health and Human Services | To help ensure that HHS receives the quality measures it needs to effectively implement its quality measurement programs and initiatives within required time frames, the Secretary of HHS should ensure that testing of the electronic versions of the measures retooled by NQF that are being used or are planned for use in the Medicare and Medicaid EHR Incentive programs is completed in a timely manner to help identify potential errors and address issues of implementation. |
HHS reported that feasibility testing for those measures already used in Stage 1 of the Medicare and Medicaid EHR Incentive programs was not needed, as the agency found that their use and reporting demonstrated feasibility for use in Stage 2. For newly introduced stage 2 measures for eligible professionals, HHS implemented the recommendation by having various Centers for Medicare and Medicaid Services (CMS) contractors complete feasibility testing. Feasibility testing of measures for eligible hospitals was completed by another CMS contractor. In addition, HHS established plans for conducting feasibility testing for measures considered in the future.
|
| Department of Health and Human Services |
Priority Rec.
To help ensure that HHS receives the quality measures it needs to effectively implement its quality measurement programs and initiatives within required time frames, the Secretary of HHS should develop a comprehensive plan that identifies the quality measurement needs of HHS programs and initiatives, including PPACA requirements, and provides a strategy for using the work NQF performs under the contract to help meet these needs. |
The Department of Health and Human Services (HHS) provided information in April 2017 indicating that the agency uses a comprehensive planning process to identify the quality measurement needs of HHS programs and initiatives. For example, the Centers for Medicare & Medicaid Services' (CMS) Measurement Needs and Priorities documents, published annually since 2015, identify priority needs for quality measures for its various quality programs, such as its quality reporting and value-based purchasing programs. Further, HHS provided information on how it uses the work that the National Quality Forum (NQF) performs to meet identified measurement needs. Specifically, HHS reported that it leverages the expertise of the NQF's Measure Applications Partnership, which annually reviews measures CMS identifies as candidates to meet identified needs and provides recommendations to the HHS Secretary on measures to be used in CMS programs.
|