Robert J. Mislevy - Sociocognitive Foundations of Educational Measurement

Laying out an articulation between socially situated human learning and action on the one hand, and the models and machinery of psychometrics on the other, covers a lot of territory. Among the topics that appear are evidentiary arguments, connectionist models, automated scoring, complex adaptive systems, model-based reasoning, assessment design, cognitive models of cultural meaning, and a subjectivist-Bayesian probability. No single treatment can hope to present the nuances or the latest results of any of these topics, let alone their interrelationships. I draw selectively from each, to identify ideas that are fundamental to the articulation. The book is not as up to date today as it might be, but because the fundamental ideas and the articulation they ground are apt to remain relevant, neither will it go out of date as quickly.

The emphasis is on understanding measurement models from a sociocognitive psychological perspective. We want to understand the entities and relationships in the models and the ways people can design them and use them in practical settings, as viewed from a sociocognitive perspective. To the extent I succeed, the work will help educators and researchers use familiar kinds of assessment more wisely and apply psychometric tools to new forms of assessment such as games and simulations. A situative perspective is necessary to think through the design challenges that an artifact like a game-based assessment, which must at once serve purposes that come from the disparate worlds of games, learning, and assessment.

The book is not an exploration of all the ways assessment can be designed and used to foster learning that arise from a sociocognitive perspective. This is the interest of several of my IOT colleagues, and their chapters in Assessment, Equity, and Opportunity to Learn hold insights to this end (e.g., Gee, 2008; Greeno & Gresalfi, 2008; Lee, 2008; Moss, 2008). They, and others whose expertise lies along these lines, write wonderful books and articles about this. I hope they can use this book to see where and how tools from the educational measurement paradigm, reconceived through a sociocognitive lens, can help improve assessment as they see it.

And aside from seeing how certain assessments might meet or fail their intended purposes, the book does not delve into what those purposes should be, how social systems ought to be structured, and the roles that various assessments play, for better or worse, in the lives of individuals and societies. These issues are particularly salient in language testing, because language tests can be pivotal in matters of opportunity and access, and of privilege and discrimination. Tim McNamara and Carsten Roever (2006) correctly noted in Language Testing: The Social Dimension that in my investigations into the structure of educational assessment,

consideration of the social dimension of assessment remains implicit and limited to issues of fairness. For example, Mislevy does not consider the context in which tests are commissioned and, thus, cannot problematize the determination of test constructs as a function of their role in the social and policy environment . Nor does Mislevy deal directly with the uses of test scores, the decisions for which they form the basis, except insofar as they determine the formulation of relevant claims, which in any case, is taken as a given and is not problematized.

(p. 24)

Tim and Carsten were kind enough to include my response from our correspondence:

The main reason I focus on technical and epistemological issues is that by coincidences of training, experiences, and interest, I find myself in a position to make what I think are useful contributions along these linesneeded, I believe, to take assessment to the next level in response to advances in technology and psychology. So many problems I see in assessment, socially as well as technically, arise from inadequate conceptualizations of the assessment enterprise.

(Personal communication to Tim McNamara, April 2, 2006; quoted in McNamara & Roever, 2006, p. 42, in footnote 3 of .)

This volume is similarly circumscribed, focusing on the meanings and methods of between-person measurement models in educational assessment. However, I believe the extensions to the sociocognitive perspective and the expanded analyses of assessment arguments provide valuable conceptual tools for understanding social and policy aspects of assessment (Mislevy, 2012).