Contents
Guide
Pagebreaks of the print version
Technical Debt in Practice
How to Find It and Fix It
Neil Ernst, Rick Kazman, and Julien Delange
The MIT Press
Cambridge, Massachusetts
London, England
2021 The Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.
The MIT Press would like to thank the anonymous peer reviewers who provided comments on drafts of this book. The generous work of academic experts is essential for establishing the authority and quality of our publications. We acknowledge with gratitude the contributions of these otherwise uncredited readers.
Library of Congress Cataloging-in-Publication Data
Names: Ernst, Neil, author. | Kazman, Rick, author. | Delange, Julien, author.
Title: Technical debt in practice : how to find it and fix it / Neil Ernst, Rick Kazman, and Julien Delange.
Description: Cambridge, Massachusetts : The MIT Press, 2021. | Includes bibliographical references and index.
Identifiers: LCCN 2020041281 | ISBN 9780262542111 (paperback)
Subjects: LCSH: Computer softwareDevelopmentQuality control. | Software failuresPrevention. | Project management. | Maintainability (Engineering)
Classification: LCC QA76.76.Q35 E76 2021 | DDC 005.3028/7dc23
LC record available at https://lccn.loc.gov/2020041281
d_r0
For Kieran, Elliott, and Kambria, who toured our old hometown while we labored on this. N.E.
For Hong-Mei, who humored our rants and cheered us on. R.K.
For Alejandra and Chewie. J.D.
Contents
- List of Figures
- Technical debt quadrants.
- Fishbone diagram of the causes of technical debt, per software lifecycle phase. A fishbone (or Ishikawa) diagram represents a problem on the right (head) and causes of that problem on the left.
- Technical debt and the software value curve.
- A value stream model, with sample tools. The arrows show how the idea moves from an issue tracker all the way to an A/B test result. This visualization can highlight bottlenecks, where a feature takes the most time to progress.
- A DSM of an industrial project showing structural dependencies.
- A DSM of the project in figure 4.1, now showing evolutionary dependencies.
- Design complexity and design flaws in the real world.
- Source: left image: https://pxhere.com/en/photo/996703 ; right image: Rick Kazman.
- Example of a cyclic dependency.
- Architecture antipatterns in Apache Hadoop.
- Data collection and analysis for the SS1 project.
- Debt calculation for the SS1 project.
- Architecture debt calculation.
- Good commit messages explain the change and include other details.
- Bad commit messages dont explain the change.
- Dashboard from code-inspector showing trends in a codebase. (a) Metrics and trends; (b) categorized violations; (c) duplicated code.
- Example dashboard offered by coveralls.io: coverage metrics are showing which files have appropriate levels of testing and which files require more testing.
- Example of metrics trends (CPU and Memory utilization) over a day.
- Winston Churchill giving the V (for Victory) sign in 1943.
- The machine learning ecosystem. Note the small size of the machine learning codethe modelrelative to the entire system.
- Exponential growth in communication networks.
- Technical debt and the software value curve.
- List of Tables
- Industrial benchmark data. Higher DL and lower PC are better.
- Architectural measures for the platform with and without co-change information.
- Architecture root analysis
- Design flaws in Brightsquids platform
- Architecture flaws in the platform before and after refactoring
- Productivity measures for Brightsquids platform
- Project context and documentation approaches
- Approaches to documentation
- Community smells (adapted from [Tamburri 2016] and [Tamburri 2019])
- Community smell mitigations
- List of Boxes
- Financial Literacy Test
- Voice of the Practitioner: Andriy Shapochka
- Types of Requirements
- Requirements Traceability in Chromium
- Lean and Agile and Requirements Debt
- Coupling and Cohesion: A Primer
- Design Principles
- Finding Security Design Flaws in Chromium
- Software Metrics and Lines of Code
- Voice of the Practitioner: Marco Bartolini
- To Use or Not to Use a Security Framework
- Code and Bug Duplication
- Voice of the Practitioner: Nicolas Devillard
- Voice of the Practitioner: Andriy Shapochka
- Voice of the Practitioner: Marco Bartolini
- Voice of the Practitioner: Julien Danjou
- Cost Estimation and Technical Debt
- Voice of the Practitioner: Andriy Shapochka
Acknowledgments
This book would not have happened without the help and inspiration of a lot of different people. The three of us met while working at the Carnegie Mellon University Software Engineering Institute in Pittsburgh. Thanks to all our coworkers at the SEI, including Robert Nord, Peter Feiler, Ipek Ozkaya, James Ivers, Felix Bachmann, John Klein, Stephanie Bellomo, Phil Bianco, and Chuck Weinstock. Linda Northrop remains an inspiration.
Rick Kazman would especially like to thank his research collaborators, including Yuanfang Cai, Damian Tamburri, Humberto Cervantes, and the group at SoftServe, including Serge Haziyev and Andriy Shapochka.
Julien would like to thank Julien Danjou and Nicolas Devillard for their time.
Neil would like to thank his collaborators, as well as the anonymous reviewers of this manuscript, who although causing more late nights, doubtless improved the final version.
1Introduction
There is scarcely anything that drags a person down like debt.
P. T. Barnum
1.1What Is Technical Debt?
If you have a credit card or a mortgage or a car loan, you already know a bit about debt. You are also probably familiar with the many metaphors that surround the ways we think about and talk about debt: crushing debt, drowning in debt, buried in debt. We talk of debt in relationships, or the social debt that you feel after your neighbor invited you to dinner and you have not reciprocated. Metaphors are pervasive in our lives. So it should come as no surprise that we also apply this metaphor to the technical world. The fact that you cracked the cover of this book suggests that this metaphor is already speaking to you. The metaphor of technical debt is something that every software developer has at least heard of. But metaphors only take you so far. Our purpose in writing this book is to move us all beyond just the metaphor into actionable insights, methods, and tools that allow us to deal with technical debt as software engineers.
Every organization that creates nontrivial software systems has technical debt, and the software development world is becoming increasingly conscious of this debt. Take Facebook, for example: the company originally used the PHP language to prototype and deliver their product quickly as a young company, but then as they grew they had to face the limitations of their early technical decisions. PHP simply was not able to scale and provide the kind of performance that they needed to support their ever-growing userbase, and so Facebook had to find solutions to pay this debt back; in their case, the solution was to create a PHP transpiler. Almost all companies face such issues, and this book provides many concrete examples of technical debtfrom Boeing and Airbus to Twitter and many others. But we do not just provide horror stories (although there are plenty). We provide a concrete set of practical solutionsmethods, tools, and techniquesfor dealing with technical debt. An important message of this book is not that you should never incur debt; it is that you should not