Ken Youens-Clark - Reproducible Bioinformatics with Python

by Ken Youens-Clark

Copyright 2021 Ken Youens-Clark. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://oreilly.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Editors: Michelle Smith and Corbin Collins
• Production Editor: FILL IN PRODUCTION EDITOR
• Copyeditor: FILL IN COPYEDITOR
• Proofreader: FILL IN PROOFREADER
• Indexer: FILL IN INDEXER
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Rebecca Demarest

• February 2022: First Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781098100889 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Reproducible Bioinformatics with Python, the cover image, and related trade dress are trademarks of OReilly Media, Inc.

The views expressed in this work are those of the author(s), and do not represent the publishers views. While the publisher and the author(s) have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author(s) disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-098-10088-9

[FILL IN]

Programming is a force multiplier.We can write computer programs to free ourselves from tedious manual tasks and to accelerate research.Programming in any language will likely improve your productivity, but each language has different learning curves and tools that improve or impede the process of coding.

There is an old adage in business that says you have three choices:

1. Fast

2. Good

3. Cheap

Pick any two.

When it comes to programming languages, Python hits a sweet spot in that its fast because its fairly easy to learn and to write a working prototype of an idea its pretty much always the first language Ill use to write any program.I find Python to be cheap because my programs will usually run well enough on commodity hardware like my laptop or a tiny AWS instance.However, I would contend that its not necessarily easy to make good programs using Python because the language itself is fairly lax.For instance, it allows one to mix characters and numbers in operations that will crash the program.

This book has been written for the aspiring bioinformatics programmer who wants to learn Pythons best practices and tools such as the following:

• Since Python 3.6, you can add type hints to indicate, for instance, that a variable should be a type like a number or a list, and you can use the mypy tool to ensure the types are used correctly.

• Testing frameworks like pytest can exercise your code with both good and bad data to ensure that it reacts in some predictable way.

• Tools like pylint and flake8 can find potential errors and stylistic problems that would make your programs more difficult to understand.

• The argparse module can document and validate the arguments to your programs.

• The Python ecosystem allows you to leverage hundreds of existing modules like Biopython to shorten programs and make them more reliable.

Using these practices will improve your programs, but combining them all will improve your code in compounding ways.This book is not a textbook on bioinformatics per se.The focus is on what Python offers that makes it suitable for writing scientific programs that are reproducible.That is, Ill show you how to design and test programs that will always produce the same outputs given the same inputs.Bioinformatics is saturated with poorly written, undocumented programs, and my goal is to reverse this trend, one program at a time.

The criteria for program reproducibility includes:

• Parameters: All program parameters can be set as runtime arguments. This means no hard-coded values which would require changing the source code to change the programs behavior.

• Documentation: A program should respond to a --help argument by printing the parameters and usage.

• Testing: You should be able to run a test suite that proves the code meets some specification.

You might expect that this would logically lead to programs which are perhaps correct, but, alas:

Program testing can be used to show the presence of bugs, but never to show their absence!

Edsger Dijkstra

Bioinformatics is composed mostly of scientists whove learned programming or programmers whove learned biology (or someone like me who had to learn both).No matter how youve come to this, I want to show you the practical programming techniques that will help you write reproducible programs.Ill start with how to write programs that document and validate their arguments.Then Ill show how to write and run tests to ensure the programs actually do what they purport.

For instance, the first chapter shows you how to report the tetranucleotide frequency from a string of DNA.Sounds pretty simple, right?Its a trivial idea, but Ill take about 40 pages to show how to structure, document, and test this program.Ill spend a lot of time on how to write and test the 7 different versions of the program so that I can explore many aspects of Python data structures, syntax, modules, and tools.