Irina Shamaeva - Custom Search – Discover more:: A Complete Guide to Google Programmable Search Engines

First Edition published 2021
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

2021 Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, LLC

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, access

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification and explanation without intent to infringe.

Library of Congress CataloginginPublication Data
Names: Shamaeva, Irina, author. | Galley, David, author.
Title: Custom search - discover more: a complete guide to Google programmable search engines / Irina Shamaeva, David Galley.
Other titles: Google.
Description: First edition. | Boca Raton : CRC Press, 2021. | Includes bibliographical references and index.
Identifiers: LCCN 2020048129 | ISBN 9780367567392 (paperback) | ISBN 9780367569686 (hardback) | ISBN 9781003100133 (ebook)
Subjects: LCSH: Search engines.
Classification: LCC TK5105.885.G66 S53 2021 | DDC 025.04252dc23
LC record available at https://lccn.loc.gov/2020048129

ISBN: 978-0-367-56968-6 (hbk)
ISBN: 978-0-367-56739-2 (pbk)
ISBN: 978-1-003-10013-3 (ebk)

Typeset in Minion
by SPi Global, India

GOOGLE'S CUSTOM SEARCH ENGINES (CSEs) offer search opportunities that are unavailable with any other tool. CSEs have advanced settings and search operators that do not work in regular Google searches. Anyone can create CSEs for themselves and others to use, uncover extra results, and boost research productivity.

With this book, we hope to popularize this fun and powerful tool so that many more people become aware of it and include CSEs in their research.

To our disappointment, Google has recently renamed "Custom Search Engines" to "Programmable Engines." But we like the old, long-lived name better and will continue using it.

You might be wondering why the word more: in the title is not capitalized and followed by a colon. It is intentional and has a double meaning (!). You will find out what we meant in the second half of the book.

We want to thank several colleagues who generously spent their time reviewing the manuscript draft: Julia Tverskaya, Elena Pavlovskaia, Pierre-Andr Fortin, Glenn Gutmacher, and Guillaume Alexandre. Our sincere thanks for your honest, constructive, and encouraging feedback.

"Google it!" Most people search Google with a few keywords looking to find one piece of information and find what they need as a first or second result.

As a researcher, however, you need to get some control over results and are often looking for as many results as possible, not just one. As a practical example, you might be looking to find social site profiles that fit additional requirements, such as a location, employer, or job title. Or you might be interested in top management moves in manufacturing. You can find the target information on Google, but you would need to run a series of advanced searches that are unlike "simple searching with keywords."

Google allows you to control results with advanced search operators, advanced search dialog, and settings. You can narrow the search to a given site, file format, or words in the page title or URL. For example, you can search for pages on the site slideshare.net with the word "CV" in the title. But you cannot filter results by the type of content pages represent, such as people or company profiles.

CSEs are a software layer Google built on top of its search. CSEs get results by filtering search results from Google to match the CSE settings. They allow you to use the same operators and settings as Google's. They also have additional parameters and search operators.

CSEs can do many things that Google cannot. You can narrow down to a country or language. You can set "soft" parameters that would make your CSE rank pages higher if they have given keywords or are on a given site. You can run long OR searches, overcoming Google's 32 keywords limitation.

What is of particular interest to us is that CSEs can search for standard types of structured information and values. You can search for people, restricting results to a given employer, job title, location, or school. You can search for Microsoft as an employer or director as a job title, not just a keyword. You can achieve the filtered search writing out advanced CSE operators. We will present detailed explanations of the operators' syntax, along with examples for you to recreate.

Lots of professionals use existing CSEs, for instance, in talent acquisition to source for "passive" job candidates, in digital journalism or with other research goals. But few people have experience creating them. Even fewer know about advanced CSE-only search operators.

We think that the main reason CSEs are not as widespread as they should be is that it is not easy to get educated on CSE creation There is little information online and no books (other than the one you are reading). Only sketchy information is available on the "structured" operators, which are by far, the best CSE feature.

Another reason for the lack of popularity of CSEs is that Google's help and posts are oriented towards website owners. What is of interest to us are CSEs that search on sites that we did not build.

Help documentation is also not aligned 1-to-1 with what you see. Many links in Google's CSE support documentation refer to guides for programmers. It is as if they do not believe anyone will create or edit CSEs via the regular user interface (UI, called the Control Panel).

Yet another reason is that the terminology is too technical, and the UI is archaic. Context, Annotations, Refinements, Knowledge Graph, and Schema.org Objects on the front "basic" page in the Control Panel, as well as "JSON API," do not communicate anything to non-coders. Beginners get discouraged.

You can find some educational videos about CSEs, but none of them go in-depth.

We wrote the book to cover the gap.

There are two ways to engage with CSEs: as a creator and as an end-user.

• The creator can set up CSEs for themself and others (e.g., less technically savvy colleagues or clients). Even simple CSEs can be quite useful. But to take advantage of more advanced functionality, the creator needs a deeper understanding of how Google search and CSEs work.

• CSEs allow end-users to perform advanced searches without the need to use (or even understand) complex operators. For the more sophisticated end-user, CSEs' unique search operators allow for a high-precision web search unavailable anywhere else.