This is the accessible text file for GAO report number GAO-11-403R 
entitled 'Nursing Home Quality: Implementation of the Quality 
Indicator Survey' which was released on May 6, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

GAO-11-403R: 

United States Government Accountability Office: 
Washington, DC 20548: 

April 6, 2011: 

The Honorable Herb Kohl: 
Chairman: 
Special Committee on Aging: 
United States Senate: 

The Honorable Charles E. Grassley: 
Ranking Member: 
Committee on the Judiciary: 
United States Senate: 

Subject: Nursing Home Quality: Implementation of the Quality Indicator 
Survey: 

This report formally transmits our briefing slides that describe the 
approach that the Centers for Medicare & Medicaid Services (CMS) is 
taking to implement the Quality Indicator Survey (QIS) nationally (see 
enclosure I). CMS developed the QIS--a new electronic process used to 
conduct nursing home inspections--to provide a more reliable 
assessment of the quality of care in nursing homes. You asked us to 
review the implementation of the QIS. As you requested, these slides 
provide background information on the nursing home inspection process 
and describe the approach CMS is taking to implement the QIS. These 
slides were used to brief your staff on March 28, 2011. We will 
address other issues related to implementation of the QIS in a 
subsequent report. We provided a draft of this report to the 
Department of Health and Human Services (HHS) and the agency provided 
technical comments, which we incorporated as appropriate. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the Secretary of HHS, the Administrator of CMS, and relevant 
congressional committees. In addition, the report will be available at 
no charge on the GAO Website at [hyperlink, http://www.gao.gov]. 

If you or your staffs have any questions regarding this report, please 
contact me at (202) 512-7114 or kohnl@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. Key contributors to this report were 
Anne Laffoon, Assistant Director; George Bogart; Krister Friday; Seta 
Hovagimian; Samantha Poppe; and Priyanka Sethi. 

Signed by: 

Linda T. Kohn: 
Director, Health Care: 

Enclosure: 

[End of section] 

Enclosure I: Nursing Home Quality: Implementation of the Quality 
Indicator Survey: 

Briefing for staff of: 

Special Committee on Aging: 
United States Senate, and: 

Committee on the Judiciary: 
United States Senate: 

March 28, 2011: 

Overview: 
* Introduction; 
* Objective; 
* Scope and Methodology; 
* Background; 
* Results; 
* Agency Comments. 

Introduction: 

Nursing homes provide skilled nursing, rehabilitation, and/or 
custodial care to elderly and disabled individuals. 

Federal and state governments share responsibility for ensuring that 
nursing homes provide quality care in a safe environment for the 
nation’s 1.5 million residents dependent on such care. 

The Centers for Medicare & Medicaid Services (CMS) is responsible for 
ensuring that nursing homes participating in the Medicare and Medicaid 
programs meet federal quality standards. To do so, among other 
activities, CMS contracts with state survey agencies to conduct 
periodic inspections of nursing homes. 

Over the last decade, we have reported significant weaknesses in these 
inspections, which include poor measurement of serious care problems 
in nursing homes.[Footnote 1] 

To improve the inspection process, CMS developed the Quality Indicator 
Survey (QIS),a new electronic method for conducting nursing home 
inspections that is designed to provide a more reliable assessment of 
the quality of nursing homes. 

In 2005, CMS launched a demonstration of the QIS in five states: 
California, Connecticut, Kansas, Louisiana, and Ohio. 

CMS is in the process of implementing the QIS in all 50 states, the 
District of Columbia, and two territories.[Footnote 2] 

[End of Introduction section] 

Objective: 

Our briefing describes the approach that CMS is taking to implement 
the QIS nationally. 

[End of Objective section] 

Scope and Methodology: 

This briefing will describe CMS’s efforts to implement the QIS 
nationally from August 2009, when CMS announced its implementation 
plan, to February 2011. 

To describe the agency’s implementation approach, we: 

* Reviewed CMS policies, procedures and other relevant documents, 
including CMS’s QIS implementation plan. 

* Interviewed CMS officials regarding the implementation plan. 

* Conducted interviews with subject matter experts outside of CMS,such 
as the CMS contractor who led the development of the new QIS process, 
to further our understanding of the QIS and how CMS plans to implement 
it. 

* Reviewed CMS data on the number of states that have implemented the 
QIS and the number of nursing home inspections conducted using the QIS. 

Limitations to our research are as follows: 

* We focus on steps taken by CMS at the federal level to facilitate 
implementation of the QIS; state-level perspectives are not included 
in this product. 

* CMS data on the number of nursing home inspections conducted in 
fiscal year (FY) 2010 are not yet complete; the numbers we present are 
the most current as of February 2011 but may not reflect all QIS 
surveys conducted in FY 2010. 

To ensure reliability of the nursing home inspection data collected, 
we reviewed key documents and checked data for obvious errors. Based 
on these activities, we determined that the data are sufficiently 
reliable for our purposes. 

We conducted this audit from October 2010 through March 2011 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objective. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objective. 

[End of Scope and Methodology section] 

Background: 

Medicare and Medicaid Coverage for Nursing Home Care: 

Medicare covers up to 100 days of skilled nursing home care following 
a hospital stay. 

Medicaid covers nursing home stays for certain low-income individuals. 

Approximately 15,700 nursing homes receive payment from Medicare 
and/or Medicaid programs. According to a CMS official, combined 
Medicare and Medicaid payments for nursing home care in 2009 were 
about $89 billion.[Footnote 3] 

Oversight of Nursing Homes: 

Oversight of nursing homes is a shared federal-state responsibility. 

To help ensure quality of care in nursing homes, CMS, among other 
tasks, sets federal quality standards that focus on the delivery of 
care, resident outcomes, and facility conditions. 

* For example, some of these standards focus on areas such as the 
prevention of pressure sore development and keeping the resident 
environment as free from accident hazards as possible. 

Nursing Home Inspection Process: 

To determine whether nursing homes meet federal standards, CMS 
contracts with state survey agencies to conduct on-site nursing home 
inspections, which consist of routine surveys and complaint 
investigations. State surveyors are to conduct routine surveys, on 
average, every 12 months, and complaint investigations are conducted 
in response to allegations of quality problems. 

During a routine survey, teams of state surveyors evaluate nursing 
homes’ compliance with federal quality standards, in part by measuring 
resident outcomes such as the incidence of preventable pressure sores, 
weight loss, and accidents. 

During both routine surveys and complaint investigations, state 
surveyors may cite deficiencies—-that is, areas in which facilities 
fail to meet federal quality standards.[Footnote 4] 

* Nursing homes must prepare a correction plan to address most 
deficiencies. 

* State surveyors may conduct revisit surveys to ensure that the homes 
implemented their plans and corrected the deficiencies. 

CMS generally imposes sanctions for serious deficiencies which may 
result in the termination of a nursing home’s participation in 
Medicare and Medicaid programs. 

CMS is required to ensure the reliability of state survey agencies’ 
routine surveys by conducting federal monitoring surveys. According to 
CMS, these consist of two types of reviews—-comparative and 
observational surveys. 

* In a comparative survey, a federal survey team conducts an 
independent inspection of a home recently surveyed by a state survey 
agency in order to compare and contrast its findings with those of the 
state survey team. 

* In an observational survey, federal surveyors accompany a state 
survey team to a nursing home to evaluate how well the state surveyors 
identify and investigate concerns and document deficiencies. 

Traditional Routine Survey Process: 

The traditional process for routine surveys involves offsite 
preparation as well as an on-site inspection. 

* During offsite survey preparation, surveyors identify at-risk 
residents and facility-related areas of concern to focus on during the 
on-site inspection by reviewing various sources of information and 
relying on their professional judgment.[Footnote 5] 

* During on-site inspections, surveyors review the quality of care 
provided to select residents by interviewing and observing residents; 
interviewing residents’ family members; reviewing medical records; and 
observing staff and nursing home conditions and practices. Throughout 
this process, surveyors manually complete a paper-based questionnaire 
to document findings. 

Development of the QIS: 

In 1998, CMS began to revise the traditional process used to conduct 
routine surveys. According to the agency, the new process—-the QIS--
was developed to: 

* Improve consistency and accuracy of surveys. 

* Improve documentation of survey findings. 

* Systematically review critical aspects of nursing home care. 

* During an inspection, focus survey resources on areas where the 
greatest quality concerns exist. 

Comparison of the QIS & Traditional Process: 

The approach used to conduct routine surveys largely remains the same 
with the introduction of the QIS.[Footnote 6] Both the QIS and the 
traditional survey process involve: 

* analyzing information available about the facilities and their 
residents offsite to identify areas of care that have been 
historically problematic and residents who may be at higher risk for 
poor care; 

* interviewing residents, family members, and staff; 

* observing staff and nursing home practices; 

* reviewing medical records; and; 

* citing deficiencies when necessary. 

According to CMS officials, the QIS differs from the traditional 
survey process in two key areas: 

* The method of collecting, recording, and analyzing information and 
documenting findings during the onsite inspection. 

* The method of selecting residents to interview and observe and 
medical records to review during the onsite inspection. 

Conducting the survey with QIS’s specialized software on personal 
tablet computers is expected to enhance the survey process by: 

* Guiding surveyors through the survey process to ensure that they ask 
all applicable questions and consider all applicable quality standards. 

* Allowing surveyors to electronically record observations and view 
guidance on their computers as they conduct the inspection. 

* Automatically synthesizing and organizing findings; for example, the 
software compares surveyors’ preliminary findings with national 
quality thresholds and flags areas that exceed these thresholds for 
further inspection. 

Using the traditional approach, surveyors select residents for the 
review using various sources of information combined with professional 
judgment. Using the QIS software, surveyors select a random sample of 
residents that is representative of the home’s population. 

* The traditional process allows surveyors to review a group of 
residents—-20 percent of the current facility census-—and relies on 
the surveyors’ use of various reports, data sets, guidance and 
professional judgment to select residents who may be at-risk for poor 
quality of care. 

* The QIS software allows surveyors to draw a larger, random sample of 
up to 70 residents.[Footnote 7] 

According to CMS, the change in how residents are selected as part of 
the inspection allows surveyors to: 

* Ensure that the number of residents interviewed and observed during 
the on-site inspection is sufficient to draw conclusions about the 
overall facility population. 

* Help ensure that a sufficient number of at-risk residents are 
included in the inspection. For example, the process helps ensure that 
newly admitted residents, who may be at higher risk for 
rehospitalization or functional decline, are included in the sample. 

[End of Background section] 

Results: 

CMS’s approach to implementing the QIS nationally relies on three main 
activities implemented concurrently over the span of several years: 

* Adopting a train-the-trainer approach to train state surveyors to 
use the QIS for routine surveys. 

* Gradual roll-out of the QIS with the goal of all routine surveys 
being conducted using the QIS. 

* Developing QIS-based tools to conduct federal monitoring surveys, 
revisit surveys, and complaint investigations. 

Train-the-Trainer Approach: 

In August 2009, CMS issued a national plan for training surveyors in 
states that had not yet begun QIS training:[Footnote 8] 

* Under this plan, CMS grouped states into training cohorts and then 
assigned each cohort to one of six training periods. Each cohort is 
comprised of surveyors from five to eight states. The first training 
period began in 2009; the last begins between June 2014 to June 2015. 

* In each training period, CMS will train a small number of surveyors 
to (1) conduct routine surveys using the QIS and (2) train other 
surveyors in their states to use the QIS. 

According to CMS officials, time needed to train all surveyors within 
a state could vary from one to three years depending on factors such 
as the number and availability of surveyors in any given state. 

The last CMS training period is scheduled to begin between June 2014 
and June 2015. Therefore, training of all surveyors nationwide may not 
be completed until 2018. 

Budgetary concerns and other state-specific issues may cause states to 
postpone their training. For example, officials in one state requested 
to postpone training because they lacked funding to purchase the 
necessary tablet computers. CMS officials indicated that QIS 
implementation is mandatory and that this state has been added to the 
last scheduled training period. 

Gradual Roll-out of the QIS: 

QIS implementation involves the gradual roll-out of the QIS with the 
goal of conducting all routine surveys of nursing homes across the 
country using the QIS. 

* Until all surveyors in a state have been trained in the QIS, states 
may use both the QIS and the traditional survey processes for routine 
surveys. According to CMS, once all surveyors in a state have been 
trained in the QIS, all routine surveys should be conducted using the 
QIS. 

* States that have not yet begun to train surveyors in the QIS will 
continue to conduct routine surveys using the traditional survey 
process. 

As of February 2011: 

* 7 state survey agencies have completed training of all surveyors 
within their states. 

* 14 state survey agencies have started training surveyors within 
their states. 

* 32 state survey agencies are scheduled to start training surveyors 
between now and 2015. 

Figure: States’ QIS Training Status as of February 2011: 

[Refer to PDF for image: illustrated U.S. Map] 

Training of surveyors completed: 
Connecticut: 
Florida: 
Kansas: 
Maine: 
Minnesota: 
New Mexico: 
West Virginia: 

Training of surveyors begun: 
Arizona: 
Colorado: 
Georgia: 
Indiana: 
Louisiana: 
Maryland: 
Nebraska: 
New York: 
North Carolina: 
Ohio: 
Utah: 
Vermont: 
Washington: 

Training of surveyors scheduled to begin between February 2011 and 
June 2015: 
Alabama: 
Alaska: 
Arkansas: 
California: 
Delaware: 
District of Columbia: 
Hawaii: 
Idaho: 
Illinois: 
Iowa: 
Kentucky: 
Massachusetts: 
Michigan: 
Mississippi: 
Missouri: 
Montana: 
Nevada: 
New Hampshire: 
New Jersey: 
North Dakota: 
Oklahoma: 
Oregon: 
Pennsylvania: 
Puerto Rico: 
Rhode Island: 
South Carolina: 
South Dakota: 
Tennessee: 
Texas: 
Virginia: 
Wisconsin: 
Wyoming: 

Note: Figure does not portray the U.S. Virgin Islands, which plans for 
training of surveyors to begin between February 2011 and June 2015. 

Source: GAO analysis of CMS data. 

[End of figure] 

Table: Routine Surveys Using the QIS, FY 2009-2010[A]: 

State: Connecticut; 
Date that Training of Surveyors Started: 9/2005; 
Number of Routine Surveys, FY09[B]: 232; 
Number of Routine Surveys, FY10[B]: 223; 
Percent QIS-Based Routine Surveys, FY09[B]: 98.3%; 
Percent QIS-Based Routine Surveys, FY10[B]: 98.7%. 

State: Kansas; 
Date that Training of Surveyors Started: 9/2005; 
Number of Routine Surveys, FY09[B]: 298; 
Number of Routine Surveys, FY10[B]: 301; 
Percent QIS-Based Routine Surveys, FY09[B]: 52.7%[C]; 
Percent QIS-Based Routine Surveys, FY10[B]: 97.0%. 

State: Florida; 
Date that Training of Surveyors Started: 10/2006; 
Number of Routine Surveys, FY09[B]: 665; 
Number of Routine Surveys, FY10[B]: 690; 
Percent QIS-Based Routine Surveys, FY09[B]: 79.5%[C]; 
Percent QIS-Based Routine Surveys, FY10[B]: 99.1%. 

State: Minnesota; 
Date that Training of Surveyors Started: 1/2008; 
Number of Routine Surveys, FY09[B]: 391; 
Number of Routine Surveys, FY10[B]: 390; 
Percent QIS-Based Routine Surveys, FY09[B]: 60.4%[C]; 
Percent QIS-Based Routine Surveys, FY10[B]: 93.3%. 

State: New Mexico; 
Date that Training of Surveyors Started: 8/2008; 
Number of Routine Surveys, FY09[B]: 72; 
Number of Routine Surveys, FY10[B]: 65; 
Percent QIS-Based Routine Surveys, FY09[B]: 47.2%[C]; 
Percent QIS-Based Routine Surveys, FY10[B]: 72.3%[D]. 

State: West Virginia; 
Date that Training of Surveyors Started: 2/2009; 
Number of Routine Surveys, FY09[B]: 120; 
Number of Routine Surveys, FY10[B]: 71; 
Percent QIS-Based Routine Surveys, FY09[B]: 14.2%[C]; 
Percent QIS-Based Routine Surveys, FY10[B]: 35.2%[D]. 

State: Maine; 
Date that Training of Surveyors Started: 7/2009; 
Number of Routine Surveys, FY09[B]: 106; 
Number of Routine Surveys, FY10[B]: 107; 
Percent QIS-Based Routine Surveys, FY09[B]: 10.4%[C]; 
Percent QIS-Based Routine Surveys, FY10[B]: 96.3%. 

State: Total; 
Number of Routine Surveys, FY09[B]: 1,884; 
Number of Routine Surveys, FY10[B]: 1,848; 
Percent QIS-Based Routine Surveys, FY09[B]: 64.3%; 
Percent QIS-Based Routine Surveys, FY10[B]: 93.9%. 

Source: GAO analysis of CMS data. 

[A] This table reflects data for those states that have completed 
training of all surveyors. 

[B] As of February 2011, CMS data for FY10 are not complete, and these 
numbers are subject to change. 

[C] For these states, the percentage of QIS-based routine surveys has 
increased between FY 2009 and FY 2010. 

[D] Two of the states that have completed training of surveyors are 
not using the QIS-based routine survey at or near 100%. 

[End of table] 

Development of other QIS-based Tools: 

According to CMS officials, the agency is also developing QIS-based 
tools to conduct: 

* revisit surveys, 

* complaint investigations, and: 

* federal monitoring surveys. 

As of February 2011, the QIS-based tools are still under development. 

[End of Results section] 

Agency Comments: 

We provided a draft of these briefing slides to the Department of 
Health and Human Services (HHS) for comment. HHS provided technical 
comments, which we incorporated as appropriate. 

[End of Enclosure I] 

Footnotes: 

[1] See, for example, GAO, California Nursing Homes: Care Problems 
Persist Despite Federal and State Oversight. [hyperlink, 
http://www.gao.gov/products/GAO/HEHS-98-202] (Washington, D.C.: July 
27, 1998) and GAO, Nursing Homes: Federal Monitoring Surveys 
Demonstrate Continued Understatement of Serious Care Problems and CMS 
Oversight Weaknesses, [hyperlink, 
http://www.gao.gov/products/GAO-08-517] (Washington, D.C.: May 9, 
2008). 

[2] In this briefing, we use the term “states” to refer to the 50 
states, the District of Columbia, and territories. 

[3] Calendar year 2009 data were the most recent data available at the 
time we conducted our study. 

[4] Deficiencies are classified according to scope (i.e., the number 
of residents potentially or actually affected) and severity (i.e., the 
degree of relative harm involved). 

[5] Information reviewed includes reports containing demographic 
information about residents, previous survey findings regarding 
facilities and their residents, and the facility’s status on various 
quality measures as compared to state and national averages. 

[6] Additionally, according to CMS officials, the underlying federal 
quality standards for nursing homes, as well as surveyor guidance on 
identifying deficiencies, were not changed due to the implementation 
of the QIS. 

[7] Additional residents may be added to the sample at the surveyors’ 
discretion. 

[8] Eleven states began QIS training prior to the issuance of this 
plan: Connecticut, Kansas, Ohio, Louisiana, Florida, Minnesota, North 
Carolina, New Mexico, West Virginia, Maryland, and Washington. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: