Background

This document presents the responses from two Web-based questionnaires on the process for identifying and citing nursing home deficiencies. We collected this information to study and report on the factors that contribute to the understatement of serious deficiencies on nursing home surveys. We developed two Web-based questionnaires—one for the nursing home surveyors and one for the state agency directors. Both questionnaires asked about the Centers for Medicare & Medicaid Services’ (CMS) survey process, the surveyor work force, surveyor training, supervisory review, and state policies and practices.¹ This document presents results from all closed-ended survey questions on both questionnaires, but does not include narrative responses that we received.

Questionnaire for Nursing Home Surveyors

We fielded the questionnaire from May through July 2008 to 3,819 eligible nursing home surveyors. To identify the eligible population, we downloaded a list of identification numbers for surveyors who had conducted at least one health survey of a nursing home in 2006 or 2007 from CMS’s On-Line Survey, Certification, and Reporting (OSCAR) system database and we obtained surveyors’ e-mail addresses from state survey agencies.² We received complete responses from 2,340 state surveyors, for a 61 percent response rate.³ The state-level response rates were above 40 percent for all but three states--Connecticut, Illinois, and Pennsylvania.⁴ We excluded Pennsylvania from our analysis because Pennsylvania’s Deputy Secretary for Quality Assurance instructed the state’s surveyors not to respond to our survey and few responded.

Questionnaire for State Agency Directors

We fielded the second questionnaire from September to November 2008 to all 50 state survey agency directors and the survey agency director for the District of Columbia. We received completed responses from 50 of 51 survey agency directors, for a 98 percent response rate. The District of Columbia survey agency director did not respond. This questionnaire was designed to gather information on the nursing home survey process in each state. State agency directors were asked many of the same questions as the surveyors, but the directors’ questionnaire contained additional questions on the overall organization of the survey agency, resource and staffing issues, CMS’s Quality Indicator Survey (QIS), and the agency’s experience with CMS’s federal monitoring surveys.⁵ In addition, the questionnaire for state agency directors asked them to rank the degree to which several factors, derived from our previous work, contributed to understatement.⁶

Question-Level Responses

With few exceptions, questionnaire respondents entered their responses directly into the Web-based database.⁷ We performed computer analyses to identify illogical or inconsistent responses and other indications of possible error. We also conducted follow-up interviews with select respondents to clarify and gain a contextual understanding of their responses. Although not all respondents answered every intended question, in most cases the question-level response was comparable and representative of all respondents. Both questionnaires included questions that only a subset of respondents were expected to answer, such as those directed at nursing home surveyors who had complaints about the survey process, or state agency directors whose states had participated in the QIS. For such questions, the low question-level responses are appropriate. In addition, we note that not all directors responded when discussing close-ended questionnaire responses based on fewer than 48 state agency directors.

***

We administered these Web-based questionnaires in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain appropriate, sufficient evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. Based on the strength of our systematic survey processes and the follow-up procedures we used, we determined that the questionnaire responses were representative of the experience and perceptions of nursing home surveyors and state agency directors nationally and at the state level, with the exception of Pennsylvania surveyors and the survey agency director of the District of Columbia. In addition, we determined that the data were sufficiently reliable for our purposes. A more detailed discussion of our scope and methodology is contained in our report Nursing Homes: Addressing the Factors Underlying Understatement of Serious Care Problems Requires Sustained CMS and State Commitment (GAO-10-70).

Instructions for Viewing the Questionnaires

How to View the Questionnaires

• Click on the To Access the Questionnaires link located at the top of this screen. To read to the bottom of the screen, you may need to use your scroll bar on the right side of this screen.
• The first screen in the questionnaire is an introduction and general information that was sent to and viewed by recipients of the questionnaire. There are no results to view on this screen. This screen is for information only and you may bypass it by clicking on Next located at the bottom of the screen in the lower right.
• The questionnaire may have links to allow respondents to bypass inapplicable questions (skip patterns). While these were active links during the data collection period, they have now been disabled.
• We do not present any information about narrative responses to questions.

How to View the Responses for Each Question

• To view the responses to each question, click on the question number (Links to questions will look like this: 1., etc.).
• After viewing the responses to each question, click on the "x" in the upper right corner of your screen to close that window and return to the questionnaire.

How to Return to a Page That You Previously Visited

• To return to the last screen you viewed, click the Previous button on the lower right corner of the screen.
• Click the Next button to advance to the next screen.

How to Make the Font Larger on Your Screen

• You can make the font larger by changing your browser setting. For example, on Internet Explorer you can change the font size by going to View and selecting Text Size.

Contact Information?

• If you have questions concerning these data, please contact John E. Dicken, Director, Health Care at (202) 512-7114 or dickenj@gao.gov.