Clément Dassa

Fall

Interrater Reliability in Content Analysis of Healthcare Service Quality Using Montreal's Conceptual Framework

Authors:
Pages:
81-102

This study examines the usefulness of the Montreal Service Concept framework of service quality measurement, when it was used as a predefined set of codes in content analysis of patients' responses. As well, the study quantifies the interrater agreement of coded data. Two raters independently reviewed each of the responses from a mail survey of ambulatory patients about the quality of care and recorded whether or not a patient expressed each concern. Interrater agreement was measured in three ways: the percent crude agreement, Cohen's kappa, and the coefficient of the generalizability theory. We found all levels of interrater code-specific agreement to be over 96%. All kappa values were above 0.80, except four codes associated with rarely observed characteristics. A coefficient of generalizability equal to 0.93 was obtained. All indices consistently revealed substantial agreement. We empirically showed that the content categories of the Montreal Service Concept were exhaustive and reliable in a well-defined content-analysis procedure.

Fall

Effects of a neutral answer choice on the reliability and validity of attitude and opinion items

Authors:
Pages:
61-80

The objective of this study is to determine whether a middle alternative in the response choices to a questionnaire influences the reliability and validity of survey responses. A 32-item questionnaire regarding maternity care was administered to a sample of 1,390 persons consisting of 597 physicians, 723 nurses, and 70 midwives. The sample was randomly divided into two groups, one receiving a questionnaire with five answer choices, including a middle option, and the other receiving a similar questionnaire but without the middle option. Results showed that including a neutral option had little effect on overall reliability and validity; however, it did allow better psychometric coherence when the items were considered globally. The effect of the middle option also depends on the opinions the surveyed persons hold about the study issue.