Michael D. Ornstein - A Companion to Survey Research

SAGE has been part of the global academic community since 1965, supporting high quality research and learning that transforms society and our understanding of individuals, groups, and cultures. SAGE is the independent, innovative, natural home for authors, editors and societies who share our commitment and passion for the social sciences.

Find out more at: www.sagepublications.com

Connect, Debate, Engage on Methodspace

A Companion to
Survey Research

Michael
Ornstein

SAGE Publications Ltd
1 Olivers Yard
55 City Road
London EC1Y 1SP

SAGE Publications Inc.
2455 Teller Road
Thousand Oaks, California 91320

SAGE Publications India Pvt Ltd
B 1/I 1 Mohan Cooperative Industrial Area
Mathura Road
New Delhi 110 044

SAGE Publications Asia-Pacific Pte Ltd
3 Church Street
#10-04 Samsung Hub
Singapore 049483

Editor: Katie Metzler
Assistant editor: Anna Horvai
Production editor: Ian Antcliff
Copyeditor: Neville Hankins
Proofreader: Jonathan Hopkins
Indexer: Silvia Benvenuto
Marketing manager: Ben Griffin-Sherwood
Cover design: Jennifer Crisp
Typeset by: C&M Digitals (P) Ltd, Chennai, India
Printed and bound by: CPI Group (UK) Ltd,
Croydon, CR0 4YY

Michael Ornstein 2013

First published 2013

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act, 1988, this publication may be reproduced, stored or transmitted in any form, or by any means, only with the prior permission in writing of the publishers, or in the case of reprographic reproduction, in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers.

Library of Congress Control Number: 2012947040

British Library Cataloguing in Publication data

A catalogue record for this book is available from the British Library

ISBN 978-1-4462-0908-0
ISBN 978-1-4462-0909-7 (pbk)

Michael Ornstein teaches sociology at York University in Toronto and since 2000 has been the Director of the Institute for Social Research, the largest Canadian university-based survey research organization. He has been active in the development, design and execution of numerous large scale research projects at the Institute, including the Quality of Life project from 1977 to 1981 and the first Canadian study on knowledge, behaviour and attitudes about AIDS. His recent research is on ethno-racial inequality in Toronto, Montreal and Vancouver, occupational wage differences, age and cohort trajectories of earnings, promotion at Canadian universities and the family strategies for employment and childcare. Dr Ornsteins Politics and Ideology in Canada: Elite and Public Opinion in the Transformation of a Welfare State, co-authored with H. Michael Stevenson, was the 2001 winner of the Harold Adams Innis Prize for the best Social Science Federation of Canada supported book in the Social Sciences and English. From 2005 to 2010 he led Yorks SSHRC-Statistics Canada sponsored Summer Program in Data Analysis.

In succession, survey researchers and methodologists have described themselves as practising an art, a craft and a science. In this book, I try to capture all three dimensions.

Quite scientifically, survey researchers define and measure a huge range of material and subjective phenomena and analyse them with statistical models. But even the most factual question involves the indirect measurement of a mental construct and the answer incorporates some measurement error due to respondents, the wording of a question, and the process of data collection.

The first survey researchers thought of question design as an art, first because there were only limited generalizations about how to write good questions and, second, because they understood that their questions incorporated assumptions about how respondents thought about a topic. To a degree, this is still true. The art of survey research also involves subtle judgements about which research questions are worth pursuing and can be answered, as well as technical ingenuity in the design of questions and samples.

Each element of survey research writing questions, combining the questions in a survey that makes sense to respondents and encourages them to answer, improving the questionnaire by testing, designing strategies to maximize respondents, and accurately and cost effectively collecting data requires the experience, skill and attention to detail that characterizes a craft. The craft of applied sample design begins with statistical theory, but often it is necessary to compromise between multiple analytical goals and make judgements about spending limited funds.

addresses the practical concerns in writing individual questions and combining them into a questionnaire that is inviting to respondents and encourages them to answer carefully. Because many more researchers expect to design and pretest questionnaires themselves, these chapters offer quite detailed advice.

deals with practical sample design, such as what kind of sample and how large a sample is needed, in the context of cost and information constraints. Because every sample design can benefit from the advice of an expert, these two chapters are more a primer on the principles of sample design and a catalogue of survey strategies than a practical guide to sample design.

discusses the decline in survey response rates over the last 20 years and what it means for the future of surveys.

To learn about the power and limits of survey research there is no substitute for working with a group to design a sample and questionnaire and for using the resulting data to address the research questions that motivated the design. For many such opportunities I thank my colleagues at the Institute for Social Research at York University in Toronto.

For their sage advice and editing prowess I thank Penni Stewart and Jacob and Noah Stewart-Ornstein. Very helpful comments were provided by two anonymous reviewers. At Sage, I thank Katie Metzler for shepherding this entire project, Ian Antcliff for expertly shepherding the manuscript through production and Anna Horvai for much helpful assistance. Any errors, and the likely more numerous failures in judgement, are the responsibility of the author alone.

1Respectively, for example, see Lazarsfeld (1935) and Payne (1951); Converse and Presser (1986); and Weisberg (2005) and Groves et al. (2009).

Survey research has roots in centuries of census taking, intelligence and psychological testing beginning in the late nineteenth century, research on attitudes from the 1920s, and social surveys of the conditions of the urban poor, pioneered in England by Charles Booth in the 1880s and Joseph Rowntree in the 1890s, and extended to many other countries in the first third of the twentieth century (Bulmer et al., 1991). Modern surveys did not evolve directly from these ancestors, however. Instead, aspiring to represent entire populations, to ask questions on almost any topic, to gather data in a timely manner, and to yield quantitative results, surveys developed out of the public opinion polls initiated in the mid-1930s by American market researchers, notably Archibald Crossley, Elmo Roper and George Gallup (Converse, 1987: 87ff.). Surveys successful arrival was signalled by their correct prediction of the 1936 US presidential election. In Ropers words, advertising men are to be credited with the early development of the technique which has been evolved for sampling public opinion (1940: 325).