Michael Planty - Understanding Education Indicators: A Practical Primer for Research and Policy

Why a book on education indicators and why one now? Our decision to undertake this book is driven by the current focus on accountability at all levels and sectors within the education system. This focus has resulted in our education system facing several new and unique pressures. For example, only in the last decade have elementary and secondary schools become accountable not only for achieving specific performance goals and preventing dropouts, but also for maintaining safe and healthy environments, employing a cadre of highly qualified teachers, and providing a challenging educational curriculum. Similarly, there is a nascent movement to increase accountability in postsecondary education. These pressures are fostered by the labor markets concern about Americas position as an international competitor.

The pressures facing our education system are wide-ranging and diverse, but they share one commonality. Specifically, these pressures have increased the demand for, and emphasis placed on, indicators that purport to describe the state and progress of our education system. The increased reliance on indicators has occurred with the noblest of intentions and is, on balance, a positive development. It is often said that if something is not measured, you can never hope to change it. That being said, in this push to bring data to bear on all of the important education issues of the day, one simple, yet vitally important, fact is often overlooked: Not all indicators are created equal. Indicators employ data sources, measurement methods, and presentation styles that are of varying levels of quality.

The differing quality of indicators represents a potential problem because many consumers of education indicators are unaware of this variation. Most individuals take the information provided by indicators at face value, accepting or dismissing the information wholesale; rarely do people question or take a critical look at the process that produced the indicator. We hope this book will prompt individuals to take an informed look at indicator quality, and we provide a comprehensive framework for doing so. In particular, we introduce readers to the idea of an indicators error structure, based on the concept of total survey error developed by Groves et al. (2004). The basic idea of indicator error is that any number of potential factors can cause an indicator to be imperfect. It is important to determine whether the imperfections undermine the purpose and intent of the indicator. Introducing readers to the issues and language of measurement, data, and basic statistics allows them to become educated consumers of the types and importance of indicator error. By drawing on the fields of social indicatorssurvey research, statistics, and educationwe hope the reader walks away with a comprehensive interdisciplinary approach to understanding and evaluating the value and quality of education indicators released every year.

Many people are responsible for this book, and we must first recognize the support we have received from our families. Deven dedicates this book to Nina, who has gracefully put up with many evenings and weekends when Deven was holed up working on it. Mike dedicates this book to his wife, Michelle, and his sons, Sammy and Charlie.

The idea for this book is a product of both education and experience. Both authors have benefited greatly from the influence and ideas of colleagues and mentors who have expressed encouragement, support, and criticism. Mike would like to express thanks to Tom Snyder, Val Plisko, Marilyn Seastrom, and James P. Lynch as well as his former team at ESSI/AIR, including Mary Ann Fox, Grace Kena, Kevin Bianchi, Jana Kemp, and Katie Ferguson for their conversations, thoughts, and assistance. Others too numerous to mention at the National Center for Education Statistics and the American Institutes for Research have provided valuable insight to the workings we describe in this book. Deven would like to thank John Witte, Bob Haveman, Bobbi Wolfe, and Tom Kaplan, whose guidance, assistance, and support during his graduate studies have been invaluable. He would also like to thank his colleagues and faculty in the Interdisciplinary Training Program in Education Sciences and the Department of Political Science who have been the source of innumerable valuable conversations and suggestions. Finally, Deven would like to express gratitude to his former colleagues at the American Institutes for Research, especially his coauthor. This book would not have come together if not for Mikes ideas and efforts.

Finally, we would like to express our gratitude to our editors at Teachers College Press, Brian Ellerbeck and Lori Tate, and two anonymous reviewers who assessed the initial proposal for this book. Brian and Lori have provided wonderful guidance, suggestions, and enthusiasm for the project. We owe them for believing in our ideas and recognizing the value of this book.

As we go about our daily lives we are regularly inundated with facts, figures, and statistics that describe all aspects of society. We consistently absorb nuggets of information that describe the status of the economy, government, health, crime, and leisure in America. Within minutes of opening the newspaper, turning on the television news, or browsing the Internet we will likely encounter statements similar to the following:

• Unemployment is rising.
• Crime is at record lows.
• The 85 and older population is expected to more than triple between 2008 and 2050.
• About 4 million adolescents smoke.
• An estimated 55 million children and teens were treated in emergency departments for unintentional injuries between 2001 and 2006.
• Civic engagement is declining in society.

Statistically based statements such as those presented above possess the virtues of concision and clarity. As such, they can provide clean, powerful, and convincing statements about a group or situation. The definitive nature of these statements does not invite dissension or debate about their accuracy. As a result, these simple statements can prove to be very compelling and often serve to ignite and drive policy debates.