Jeroen Janssens - Data Science at the Command Line: Obtain, Scrub, Explore, and Model Data with Unix Power Tools

by Jeroen Janssens

Copyright 2021 Jeroen Janssens. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Acquisitions Editor: Jessica Haberman
• Development Editor: Sarah Grey
• Production Editor: Kate Galloway
• Copyeditor: Arthur Johnson
• Proofreader: Shannon Turlington
• Indexer: nSight, Inc.
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Kate Dullea

• October 2014: First Edition
• August 2021: Second Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781492087915 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Data Science at the Command Line, the cover image, and related trade dress are trademarks of OReilly Media, Inc.

The views expressed in this work are those of the author, and do not represent the publishers views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

Data Science at the Command Line is available under the Creative Commons Attribution NonCommercial-No Derivatives 4.0 International License. The author maintains an online version at https://github.com/jeroenjanssens/data-science-at-the-command-line.

978-1-492-08791-5

[LSI]

It was love at first sight.

It must have been around 1981 or 1982 that I got my first taste of Unix.Its command-line shell, which uses the same language for single commands and complex programs, changed my world, and I never looked back.

I was a writer who had discovered the joys of computing, and regular expressions were my gateway drug.Id first tried them in the text editor in HPs RTE operating system, but it was only when I came to Unix and its philosophy of small cooperating tools with the command-line shell as the glue that tied them together that I fully understood their power.Regular expressions in ed, ex, vi (now vim), and emacs were powerful, sure, but it wasnt until I saw how ex scripts unbound became sed, the Unix stream editor, and then AWK, which allowed you to bind programmed actions to regular expressions, and how shell scripts let you build pipelines not only out of the existing tools but out of new ones youd written yourself, that I really got it.Programming is how you speak with computers, how you tell them what you want them to do, not just once, but in ways that persist, in ways that can be varied like human language, with repeatable structure but different verbs and objects.

As a beginner, other forms of programming seemed more like recipes to be followed exactlycareful incantations where you had to get everything rightor like waiting for a teacher to grade an essay youd written.With shell programming, there was no compilation and waiting.It was more like a conversation with a friend.When the friend didnt understand, you could easily try again.Whats more, if you had something simple to say, you could just say it with one word.And there were already words for a whole lot of the things you might want to say.But if there werent, you could easily make up new words.And you could string together the words you learned and the words you made up into gradually more complex sentences, paragraphs, and eventually get to persuasive essays.