To my daughter, Claire, who enables me to see the world anew, and to my wife, Elizabeth, partner in the adventure of life.
Introduction
To err is human; to really foul things up requires a computer.
Bill Vaughan
I started programming with Pythonin 2000,at the very tail end of The Bubble.In that time,Ivedone things.Things Im not proud of.Some of them simple,some of them profound,all with good intentions.Mistakes, as they say, have been made.Some have been costly,many of them embarrassing.By talking about them,by investigating them,by peeling them back layer by layer,I hope tosave you some of thetoe-stubbing and face-palmingthat Ive caused myself.
As Ive reflected onthe kinds of errors Ive madeas a Python programmer,Ive observed that theyfall more or less intothe categories thatare presented here:
Setup
How an incautiously prepared environment has hampered me.
Silly things
The trivial mistakes that waste a disproportionate amount of my energy.
Style
Poor stylistic decisions that impede readability.
Structure
Assembling code in ways that make change more difficult.
Surprises
Those sudden shocking mysteries that only time can turn from OMG to LOL.
There are a couple of quick thingsthat should be addressedbefore we get started.
First, this work does not aimto be an exhaustive referenceon potential programming pitfallsitwould have to be much, much longer,and would probably never be completebutstrives insteadto be a meaningful tourof the greatest hitsof my sins.
My experiences are largely basedon working with real-world butclosed-source code;though authentic examplesare used where possible,code samples that appear heremay be abstracted and hyperbolizedfor effect,with variable names changedto protect the innocent.They may also refer toundefined variables or functions.Code samples make liberal useof the ellipsis ()to gloss over reams of codethat would otherwise obscurethe point of the discussion.Examples from real-world codemay contain more flawsthan those under direct examination.
Due to formatting constraints,some sample code thatsdescribed as one linemay appear on more than one line;I humbly ask the useof your imagination in such cases.
Code examples in this bookare written for Python 2,though the concepts under considerationare relevant to Python 3and likely far beyond.
Thanks are due toHeather Scherer,who coordinated this project;to Leonardo Alemeida,Allen Downey,and Stuart Williams,who provided valuable feedback; to Kristen Brown and Sonia Saruba, who helped tidy everything up; and especially to editorMeghan Blanchette,who picked my weird ideaover all of the safe onesand encouraged me to run with it.
Finally,though the material discussed hereis rooted in my professional life,it should not be construedas representing the current stateof the applications I work with.Rather, its drawn from over 15 years(an eternity on the web!)and much has changedin that time.Im deeply grateful to my workplacefor the opportunityto make mistakes,to grow as a programmer,and to share what Ive learnedalong the way.
With any luck,after reading thisyou will be in a positionto make a more interestingcaliber of mistake:with an awarenessof what can go wrong,and how to avoid it,you will be freedto make the exciting,messy, significantsorts of mistakesthat push the artof programming,or the domain of your work,forward.
Im eager to seewhat kind of troubleyoull get up to.
Chapter 1. Setup
Mise-en-place is the religionof all good line cooks The universe is in orderwhen your station is set upthe way you like it:you know where to find everythingwith your eyes closed,everything you needduring the course of the shiftis at the ready at arms reach, your defenses are deployed.
Anthony Bourdain
There are a couple of waysIve gotten off on the wrong footby not starting a projectwith the right tooling,resulting in lost timeand plenty of frustration.In particular,Ive made a proper hashof several computers byinstalling packages willy-nilly,rendering my system Pythonenvironment a toxic wasteland,and Ive continued to usethe default Python shelleven though better alternativesare available.Modest up-front investmentsof time and effortto avoid these issueswill pay huge dividendsover your career as a Pythonista.
Polluting the System Python
One of Pythons great strengthsis the vibrant communityof developers producinguseful third-party packagesthat you can quicklyand easily install.But its not a good ideato just go wild installingeverything that looks interesting,because you can quickly end upwith a tangled messwhere nothing works right.
By default, when youpip install
(or in days of yore, easy_install
)a package,it goes into your computerssystem-wide site-packages
directory.Any time you fire upa Python shell or a Python program,youll be able to importand use that package.
That may feel okay at first,but once you start developingor working with multipleprojects on that computer,youre going to eventuallyhave conflicts over package dependencies.Suppose project P1 depends onversion 1.0 of library L,and project P2 usesversion 4.2 of library L.If both projects have to bedeveloped or deployedon the same machine,youre practically guaranteedto have a bad daydue to changes to thelibrarys interface or behavior;if both projectsuse the same site-packages
,they cannot coexist!Even worse,on many Linux distributions,important system toolingis written in Python,so getting into thisdependency management hellmeans you can breakcritical pieces of your OS.
The solution for thisis to use so-called virtual environments.When you create a virtual environment(or virtual env),you have a separate Pythonenvironment outside of the system Python:the virtual environment has its own