HOW TO
MEASURE
SURVEY
RELIABILITY
AND VALIDITY
THE SURVEY KIT
Purpose: The purposes of this 9-volume Kit are to enable readers to prepare and conduct surveys and become better users of survey results. Surveys are conducted to collect information by asking questions of people on the telephone, face-to-face, and by mail. The questions can be about attitudes, beliefs, and behavior as well as socioeconomic and health status. To do a good survey also means knowing how to ask questions, design the survey (research) project, sample respondents, collect reliable and valid information, and analyze and report the results. You also need to know how to plan and budget for your survey.
Users: The Kit is for students in undergraduate and graduate classes in the social and health sciences and for individuals in the public and private sectors who are responsible for conducting and using surveys. Its primary goal is to enable users to prepare surveys, and collect data that are accurate and useful for primarily practical purposes. Sometimes, these practical purposes overlap the objectives of scientific research and so survey researchers will also find the Kit useful.
Format of the Kit: All books in the series contain instructional objectives, exercises and answers, examples of surveys in use and illustrations of survey questions, guidelines for action, checklists of dos and donts, and annotated references.
Volumes in The Survey Kit:
1. The Survey Handbook
Arlene Fink
2. How to Ask Survey Questions
Arlene Fink
3. How to Conduct Self-Administered and Mail Surveys
Linda B. Bourque and Eve P. Fielder
4. How to Conduct Interviews by Telephone and in Person
James H. Frey and Sabine Mertens Oishi
5. How to Design Surveys
Arlene Fink
6. How to Sample in Surveys
Arlene Fink
7. How to Measure Survey Reliability and Validity
Mark S. Litwin
8. How to Analyze Survey Data
Arlene Fink
9. How to Report on Surveys
Arlene Fink
HOW TO
MEASURE
SURVEY
RELIABILITY
AND VALIDITY
Copyright 1995 by Sage Publications, Inc.
All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher.
For information address:
| SAGE Publications, Inc. 2455 Teller Road Thousand Oaks, California 91320 E-mail: SAGE Publications Ltd. 6 Bonhill Street London EC2A 4PU United Kingdom SAGE Publications India Pvt. Ltd. M-32 Market Greater Kailash I New Delhi 110 048 India |
Printed in the United States of America
Library of Congress Cataloging-in-Publication Data
Main entry under title:
The survey kit.
p. cm.
Includes bibliographical references.
Contents: v. 1. The survey handbook / Arlene Fink v. 2. How to ask survey questions / Arlene Fink v. 3. How to conduct self-administered and mail surveys / Linda B. Bourque, Eve P. Fielder v. 4. How to conduct interviews by telephone and in person / James H. Frey, Sabine Mertens Oishi v. 5. How to design surveys / Arlene Fink v. 6. How to sample in surveys / Arlene Fink v. 7. How to measure survey reliability and validity / Mark S. Litwin v. 8. How to analyze survey data / Arlene Fink v. 9. How to report on surveys / Arlene Fink.
ISBN 0-8039-7388-8 (pbk. : The survey kit : alk. paper)
1. Social surveys. 2. Health surveys. I. Fink, Arlene. HN29.S724 1995
300. 723dc20 95-12712
This book is printed on acid-free paper.
00 10 9 8 7
Sage Production Editor: Diane S. Foster
Sage Copy Editor: Joyce Kuhn
Sage Typesetter: Janelle LeMaster
How to Measure Survey Reliability and Validity: Learning Objectives
The aim of this book is to guide the reader in assessing and interpreting the quality of collected survey data by thoroughly examining the survey instrument used. Also presented are important considerations in coding and pilot-testing surveys. Specific objectives are to:
Select and apply reliability criteria, including: |
Test-retest reliability |
Alternate-form reliability |
Internal consistency reliability |
Interobserver reliability |
Intraobserver reliability |
Select and apply validity criteria, including: |
Content validity |
Criterion validity |
Construct validity |
Understand the fundamental principles of scaling and scoring |
Create and use a codebook for survey data |
Pilot-test new and established surveys |
Address cross-cultural issues in survey research |
| Overview of Psychometrics |
A successful data collection survey is not simply a set of well-designed questions that are written down and administered to a sample population. There are good surveys and bad ones. Bad surveys produce bad data, that is, data that are unreliable, irreproducible, or invalid or that waste resources. Conversely, good surveys yield critical information and provide important windows into the heart of the topic of interest.
guest satisfaction. It is not obvious whether one or the other is better, but they are clearly different. During the course of this book, you will learn how to evaluate their differences.
EXAMPLE 1.1
Two Surveys of Guest Satisfaction
Purpose of the Survey: To assess guest satisfaction at a downtown hotel with a three-item survey at the time of checkout
Survey 1: Circle one number for each item
1. Did you enjoy your stay?
2. How was the service at the hotel?
3. Would you stay here again?
Survey 2: Circle one number for each item
1. Overall, considering the service, food, and all other aspects of our hotel, how would you describe your stay here?
Very enjoyable | |
Somewhat enjoyable | |
Neither enjoyable nor unenjoyable | |
Somewhat unenjoyable | |
Very unenjoyable | |
2. How would you describe our service?