Seiersen Richard - The Metrics Manifesto : Confronting Security with Data

1. Chapter 2
2. Chapter 4
3. Chapter 9
4. Chapter 10

RICHARD SEIERSEN

Copyright 2022 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our website at www.wiley.com.

Library of Congress Cataloging-in-Publication Data

Names: Seiersen, Richard, author.

Title: The metrics manifesto : confronting security with data / Richard Seiersen.

Description: First edition. | Hoboken, New Jersey : Wiley, [2022] | Includes index.

Identifiers: LCCN 2021062802 (print) | LCCN 2021062803 (ebook) | ISBN 9781119515364 (cloth) | ISBN 9781119515401 (adobe pdf) | ISBN 9781119515418 (epub)

Subjects: LCSH: Risk assessmentData processing. | PerformanceMeasurementData processing. | Quantitative researchData processing. | Security systemsData processing. | Hazard mitigationData processing.

Classification: LCC TA169.55.R57 S45 2022 (print) | LCC TA169.55.R57 (ebook) | DDC 363.1/02dc23/eng/20220302

LC record available at https://lccn.loc.gov/2021062802

LC ebook record available at https://lccn.loc.gov/2021062803

Cover Design: Wiley
Cover Image: Sandipkumar Patel/Getty Images

To Helena. If patience is a virtue, then you are the most virtuous person I know. Thank you for your unending support, understanding, caring, and occasionally awe-inspiring motivation. I love you.

Richard

By Doug Hubbard

When Richard and I co-authored How to Measure Anything in Cybersecurity Risk, I saw firsthand how he applies his energy, wit, and real-world experience to one of the most important issues in twenty-first-century management. Now, he has brought his insights and style of delivery to this critical issue again in The Metrics Manifesto.

At one point in this book, Richard suggests that The Metrics Manifesto may be a lofty title, but he does everything an author could do to justify the name. It is an ambitious book in the breadth of solutions described in comprehensive examples. The chapters of his book avoid the hand-waving common to so many books for managers and, instead, focus on delivering understanding as well as the details of doing real-world risk analysis. Whether the reader is just starting to explore these topics or is already an expert looking for the finer points of risk analysis, this book has the goods.

He starts with primers on basic subjects like the logic of uncertainty. The reader gets plenty of help with numerous illustrations and examples throughout the book. He includes extensive discussions of everything needed to apply these ideas, including hundreds of lines of R code for those who want to be more hands-on. The book could even be a standalone explanation of Monte Carlo simulations and Bayesian methods by itself.

Like me, Richard is a skeptic by habit. He questions many common assumptions about what works and what doesn't in cybersecurity. After we implement cybersecurity capabilities, he challenges us to ask, What would I see occurring that would let me know if it does (work)? He refers to this as productive skepticism (a term I think I will use more often). While his professional experience as both a hands-on cybersecurity expert and a CISO is an excellent basis for accepting many of his claims, he doesn't ask the reader to rely entirely on that. He supports his claims with numerous specific references in each chapter.

As a cybersecurity professional who rose through the ranks, Richard has direct experience with what data should inform the decision-making in cyber. He introduces the reader to a set of critical rate metrics: burndown, arrival, wait-time, and escape rates. Each has a dedicated chapter where he justifies the need for such measurements, as well as giving complete instructions and code that is nearly copy/paste ready. These metrics and others are combined into a dashboard that actually supports decisions, which is refreshing compared to so many that I see.

Richard Seiersen has a knack for entertaining delivery while never compromising on practical, useful information. The balance he has found ensures that he covers basic concepts without fluff while managing to cover detail with context. Rarely will a professional in any field find a book that combines textbook-like examples, detailed code like a how-to manual, the extensive citations of a scientific publication, easy-to-understand illustrations for the visually oriented, interesting historical backgrounds for the methods, and the humor and excitement of a passionate writer. The Metrics Manifesto should be on the short list of required reading for cybersecurity professionals everywhere.

Retain uncertainty without obscuring certainty.

A lot has changed since the publication of How to Measure Anything in Cybersecurity Risk. At the time, I was getting pulled into consulting side hustles, keynotes, and numerous meetups, on top of my former day gig as a serial CISO. It seemed that Fortune-level enterprises, leading startups, and nonprofits had a growing interest in quantitatively measuring risk. In the span of a year, what started out as informally helping a few fellow CISOs grew into what I call