Jason Vaughan - Web Scale Discovery Services: A Library Technology Report

Web scale discovery services for the library environment have the capacity to more easily connect researchers with the library's vast information repository. This includes locally held and hosted content, such as physical holdings, digital collections, and local institutional repositories. Perhaps more signifcantly, web scale discovery also accesses a huge array of remotely hosted content, often purchased or licensed by the library, such as publisher and aggregator content for tens of thousands of full-text journals, additional content from abstracting and indexing resources, and content from open access repositories.

This chapter defines web scale discovery and highlights a few key concepts essential for understanding these services. For anyone who has worked a reference interview and heard a student utter, I couldn't find an article in the library catalog, web scale discovery services hold tremendous potential. Extensive research on user expectations in the discovery arena, and the tools used by those seeking informationtools often disassociated from the library and often overlooking much of what the library holds and licensesprovide ample rationale for why web scale discovery is important for the library environment.

Connecting users with the information they seek is one of the central pillars of our profession. Succinctly put, Web scale discovery can be considered as deep discovery within a vast ocean of content. The mechanics behind Web scale discovery are not necessarily new, though a commercial application of this approach within the library environmentefciently and, it's hoped, efectivelyis very new. While there are various approaches to Web scale discovery, this issue of Library Technology Reports will focus on what today appears to be the most common approach, which, at its heart, involves huge, centralized, preaggregated indexes searched by the end user.

Expanded further, Web scale discovery isor certainly holds the potential to bethe evolution that libraries have long sought for information discovery. As information professionals, we all have at least a general awareness of the evolution of discovery tools within the library context. Such tools initially were print-based, such as bound handwritten catalogs, the card catalog, and works such as Poole's Index to Periodical Literature and the Reader's Guide to Periodical Literature. For the past several generations, such tools gradually transferred into the automated, electronic realm, with an obvious example the development and evolution of the online integrated library system (ILS) with a front-end catalog accessible to librarians and end users. These catalogs were initially available within the local library's physical building, often through a menu-driven, text-based interface. With the development in the 1990s of the Web's physical infrastructure and the empowerment of many to access this new environment, library vendors created HTTP Web-based online catalogs. Other evolutions included early pioneers in broad online information systems, including Dialog and LexisNexis. The 1990s ushered in growth in publisher-based electronic journal content, e-text and e-book content, abstracting and indexing databases, and full-text content aggregators looking to pull related information together within an easily accessible and searchable electronic medium. Many of these services or products were initially provided on CD-ROM or through text-based, menu-driven networked systems, which all eventually evolved into information search and delivery through a Web-based environment.