Gerald Tindal - Large-scale Assessment Programs for All Students: Validity, Technical Adequacy, and Implementation

Large-Scale Assessment
Programs for All Students:

Validity, Technical Adequacy,
and Implementation

Large-Scale Assessment
Programs for All Students:

Validity, Technical Adequacy,
and Implementation

Edited by

Gerald Tindal

University of Oregon

Thomas M.Haladyna

Arizona State University

This edition published in the Taylor & Francis e-Library, 2009.

To purchase your own copy of this or any of Taylor & Francis or Routledges collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk.

Copyright 2002 by Lawrence Erlbaum Associates, Inc.

All rights reserved. No part of this book may be reproduced in any form, by photostat, microform, retrieval system, or any other means, without the prior written permission of the publisher.

Lawrence Erlbaum Associates, Inc., Publishers

10 Industrial Avenue

Mahwah, New Jersey 07430

Cover design by Kathryn Houghtaling Lacey

Library of Congress Cataloging-in-Publication Data

Large-scale assessment programs for all students: validity, technical adequacy, and implementation/edited by Gerald Tindal, Thomas M.Haladyna.

p. cm.

Includes bibliographical references and index.

ISBN 0-8058-3709-4

1. ExaminationsValidity. 2. Educational tests and measurements. I. Tindal, Gerald. II. Haladyna, Thomas M.

LB3060.68 .L37 2002

371.26'01'3dc21 2001033453

CIP

ISBN 1-4106-0511-6 Master e-book ISBN

Contents

Gerald Tindal

Robert L.Linn

Russell Gersten and Scott Baker

Joseph M.Ryan and Sarah DeMark

Thomas M.Haladyna

S.E.Phillips

William A.Mehrens

Richard Tate

Michael C .Rodriguez

Catherine S.Taylor

George Engelhard, Jr.

Joseph M.Ryan

Seung W.Choi and Marty McCall

Patricia J.Almond, Camilla Lehr, Martha L.Thurlow, and Rachel Quenemoen

Richard P.Durn, Catherine Brown, and Marty McCall

Keith Hollenbeck

Robert Helwig

Martha L.Thurlow, John Bielinski, Jane Minnema, and James Scott

Jim Ysseldyke and J.Ruth Nelson

Thomas M.Haladyna

Preface

As our title suggests, this book deals with three issues we believe are critical to large-scale assessments of student achievement. Validity is the most important consideration in achievement testing. Technical adequacy continues to challenge us. We constantly face technical challenges in designing, administering, scoring, and reporting test results. Implementation of best practices and adherence to testing standards are much needed in all assessments of student achievement.

The emphasis on all students in our title addresses the need for large-scale assessments to be inclusive with regard to populations who are often underserved both in terms of their educational programs and assessment of their achievement. Specifically, we refer to students with disabilities and English-language learners.

This book includes five perspectives regarding large-scale assessments:

1. The Standards for Educational and Psychological Testing published by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education in 1999 provide a context for the many topics discussed in this volume. We believe these standards are very important in guiding us in the development of tests and the interpretation and use of test scores.

2. Validity has been cast as part of the decision-making process and not simply a function of the measures. This view is articulated clearly in all chapters throughout the book. The authors have used validity consistently in relation to many of the issues addressed in this book. In some states, large-scale assessments are being used for instructional program evaluation. Increasingly, states also are using achievement tests to make promotion, certification, and graduation decisions. We think that validity is a paramount concern in large-scale assessment, but validation applies uniquely to one test score interpretation or use, not all test score interpretations and uses. Test sponsors and the public are increasingly finding new uses for tests. For each of these new uses, we need to address validation.

3. This book focuses on inclusion of all students in large-scale assessment programs. No longer can students with disabilities or non-English native language backgrounds be excluded from mainstream schools and large-scale assessments. To the degree that students participate in classroom instruction, they also must be included in our assessments. If we are to include all students in our assessments and if these outcomes are to be used in improving our programs, then the information must be accurate and unbiased. The measurement of student learning should not be influenced by student disability or language proficiency.

4. This book focuses on many complex technical issues associated with large-scale assessments. We have approached the validation process by suggesting research that helps inform educators about more effective ways to measure and evaluate student achievement. A number of different strategies are used to ensure that our measurement tools work well in helping us make decisions. Judgments need to be rendered in the sampling of content and scaling of items and problems, ensuring appropriate depth and breadth. On occasion, this technical rigor also is related to the actual items and the manner in which they are calibrated and included on multiplechoice tests. On other occasions, the focus needs to be on production responses in which students construct an answer and solve problems.

5. Implementation issues are addressed to ensure that the practices and principles supporting our assessments are indeed anchored to the context in which they are used. Testing should not be done just to students. Rather, tests should be conducted with teachers, students, and parents. They are implemented in classrooms and provide outcomes that are used by principals, superintendents, and directors of testing in making statements about the effects of our teaching and extent of student learning, the quality of our schools, and the directions that we need to move to improve student learning. By including the social applications of large-scale assessments, we believe students, teachers, and schools can be held accountable not just to the larger public, but also to effective practices. The real goal of all assessments is the improvement of student learning. This goal, however, requires a socially navigated process in which we all develop a common language to anchor the focus of our teaching, assessing, and reporting.

Audience for this Book

This book is designed for educators who are responsible for implementing large-scale assessment programs at the district or state level. Although the contents may have some bearing on classroom assessment, the book is not aimed for the classroom teacher, although clearly some chapters have great relevance for them. For example, opportunity to learn or delivery standards often are important issues in judging outcomes from any largescale assessment program, and teachers need to be part of this process. Many classroom assessments may be relevant for students with disabilities or those who are English-language learners. Finally, with the stakes of large-scale assessment increasing, we have maintained a rigorous approach to the issues, again making the tone of the volume oriented to those with a minimum of technical training. We hope, however, that this book is useful in helping prepare educators to begin leadership roles and therefore would be useful for graduate students in many types of programs, including educational testing, special education, English-language learners, educational leadership, and policy.