Peter Hernon - Getting Started with Evaluation

ALA Editions purchases fund advocacy, awareness, and accreditation programs for library professionals worldwide.

2014 by the American Library Association. Any claim of copyright is subject to applicable limitations and exceptions, such as rights of fair use and library copying pursuant to Sections 107 and 108 of the U.S. Copyright Act. No copyright is claimed for content in the public domain, such as works of the U.S. government.

Extensive effort has gone into ensuring the reliability of the information in this book; however, the publisher makes no warranty, express or implied, with respect to the material contained herein.

ISBNs: 978-0-8389-1195-2 (paper); 978-0-8389-9682-9 (PDF); 978-0-8389-9683-6 (ePub); 978-0-8389-9684-3 (Kindle). For more information on digital formats, visit the ALA Store at alastore.ala.org and select eEditions.

Library of Congress Cataloging-in-Publication Data

Hernon, Peter.

Getting started with evaluation / Peter Hernon, Robert E. Dugan, and Joseph R. Matthews.

pages cm

Includes .

ISBN 978-0-8389-1195-2 (alk. paper)

1. LibrariesEvaluation. 2. Public services (Libraries)Evaluation. 3. Library administrationDecision making. 4. Library statistics. 5. LibrariesEvaluationProblems, exercises, etc. I. Dugan, Robert E., 1952 II. Matthews, Joseph R. III. Title.

Z678.85.H473 2014

027.0029dc23 2013017137

Cover design by Kimberly Thornton; image Shutterstock, Inc.

Contents

Figures

Tables

Text Boxes

With all of the developments occurring today in evaluation and the linkage of that topic with accountability, evidence-based decision making, and strategic planning, the average library manager, especially if working in small organizations, may be overwhelmed and unsure where to start and how to gather and use the resulting data. Further, most likely, if that person graduated from an accredited masters program in library and information science, he or she did not take a course, required or elective, in research methods or evaluation research. Today, library managers can hear and read about topics such as metrics, accountability, return on investment, and value, but without a background in evaluation research, they may feel overwhelmed and unsure how to proceed and what the staff should be doing.

Getting Started with Evaluation guides library managers through assorted topics: planning and decision making ( sections at the end of the book offer answers relevant to the exercises and key readings, respectively.

For the evaluation topics covered in this book, we advise readers on relevant source material they can use, and we alert them about how to collect evidence. The goal is the presentation of strategies that do not require a sophisticated understanding and application of a complex research process. As librarians review what they want or need to know and sources of evidence, they should remember that they want to portray the library in the life of customers and communities, and not others in the life of the library. In other words, libraries need to view themselves from the perspectives of others.

Accountability focuses on planning, evidence-based decision making, and demonstrating to stakeholders the effective and efficient use of resources, financial and other, to meet the organizational mission and that of the parent institution or organization. Service or program improvementbetter service to constituent groupsadds another dimension to evaluation as managers engage in benchmarking and apply best practices. Clearly, a focus on both accountability and service improvement is part of managing organizations in the twenty-first century. To do both effectively more academic and public libraries need to invest in management information systems that enable their managers to communicate with stakeholders important to the library.

In summary, Getting Started with Evaluation addresses two questions:

1. What might library managers do that they are not currently doing?

2. How do they do those things?

To help address both questions, this guide relies on the knowledge and experiences of three writers on the application of evaluation to the management of academic and public libraries. It is our expectation that our approach will make this book of value to library managers in academic and public libraries, be they directors, members of the senior management team, or at other levels of management, as well as members of governing boards and others wanting to adopt any of the concepts presented in a given library. We also think management and research courses in schools of library and information science will find the content useful.

L ibrarians today are somewhat familiar with terms such as evidence-based decision making, accountability, change management, learning organizations, and strategic planning. Such terms assume added importance as library managers cope with decreased resources but elevated customer and community expectations, view technology as a change agent, and create sustainability, the capacity to endure and add value to the lives of customers, stakeholders, and communities served. Stakeholders demand evidence that the resources allocated to libraries and their parent institution or organization are managed in a cost-effective or cost-efficient way and that libraries contribute to vital outcomes set by the institution or broader organization such as those dealing with student success and faculty research productivity. Further, as many library directors and others realize, most libraries are undergoing a transition and, it is hoped, moving from surviving to thriving. Rush Miller adds,

If our vision of the future is imperfect, it is better to be moving forward helping to define the future than to sit back, pat ourselves on the backs for how valuable we always were, and let that future move on without us. We cannot allow anything to deter us from creating the future for libraries that will maintain our relevance to the... mission of... [the parent institution or organization]. Even in a recession, we should seize the opportunities it affords us to question our traditions in light of the needs of our users in the digital age.

Within this context, library managers need to monitor the organizations progress in making a transition to a vision of the future as laid out in planning documents while reviewing services and making adjustments as evidence dictates. They also need to go beyond anecdotes and assumptions while providing evidence of accountability, the obligation to be responsible for their actions and for the appropriate management of resources with which they have been entrusted. Accountability, therefore, should be a part of the organizational culture and the planning process: setting the direction (what does the organization want to achieve?), developing strategies to achieve that direction (how will the organization achieve what it wants?), monitoring performance (how will the organization know how well it is doing?), and, finally, repeating the cycle. Evaluation involves monitoring performance and using the evidence gathered to review and modify, as necessary, the direction set.

The Concept

Simply stated, evaluation is the process of identifying and collecting data about specific services or activities, establishing criteria by which their success can be measured, and determining the quality of the service or activitythe degree to which it accomplishes stated goals and objectives. As Peter H. Rossi, Mark W. Lipsey, and Howard E. Freeman explain, The role of evaluation is to provide answers to questions about As such, evaluation is a decision-making tool that is intended to assist library staff in allocating necessary resources to those activities and services that best enable the organization to accomplish its mission, goals, and objectives.