Adam Johnson - Speed Up Your Django Tests

The measure of success most often is speed.Doing things better is synonymous with doing things faster so that we can do even more things efficiently and effectively.In buying into this premise, we enter a spiral of acceleration that we can never hope to master.

Diana Scharf Hunt and Pam Hait

Welcome, dear reader.

Oh, tests.

Tests are great.Tests show us where bugs are at least some of them.Tests help us to not add new bugs.

But tests.Tests are also a frequent source of frustration.When you first started writing tests, you probably found them an extra nuisance to write.You probably still think the same, at least sometimes.

Even if you love writing tests, running them can be boring.Slow tests can make you lose concentration, juggle tasks, and end up getting less stuff done.They can suck all the fun out of programming.And programming is meant to be fun!

Theres a phase shift when you speed up your tests, restoring that joy.If your tests take 1000 seconds and you reduce them to 600 seconds, youll feel it.Your whole team will feel it.The same happens if you can reduce a test run from 30 seconds down to 10 seconds.

And theres no limit to fast enough.Reducing test run time is the easiest and safest way to increase your speed of delivery.And your organization wants to deliver as fast as possible, to get valuable features and fixes in front of users.

Fast tests also tend to be more accurate tests.Theyre well targeted, so they only fail when a feature is broken.And with less overhead, you can exercise features more completely.

The content is based on my eight years of experience with tests on Django projects.Ive sped up many projects test suites, improved Djangos own testing framework, and created several pytest plugins.

In your hands, or at least on your screen, is my best guide for speeding up tests on your project.Ive tried to cover most situations, so you should be able to find something relevant to your project.

May it reduce your test speed and your test pain.Enjoy!

Adam

P.S. Please send me any and all feedback through the contact details on my site.

If you are a Django developer working with tests, this is for you.No matter your level.

My aim is to make the book readable by a junior developer who has been given the vague task of improving the tests, but also to include material that even long time Django users may not have come across.Ive tried to be opinionated enough to be actionable, but provide advice thats applicable to all projects.

The first two chapters, , explain the various tools you have at hand for modifying and measuring your tests respectively.

The next six chapters cover ways of speeding up your tests without the bother of rewriting your test code (much). cover ways of speeding up your tests in configuration for those environments.

The three chapters that follow cover how to write fast tests. shows some focussed techniques for replacing components during testing that can help you write better, faster tests.

Finally, the gives some closing remarks and links to further reading.

To get the 80% of improvements that take 20% of the effort:

• Record your current test runtime with a shell timing command as in .

• Skim to understand the pieces youre changing.

• If you dont yet run your tests in parallel, read .

• Learn how to reuse your database and squash your migrations in .

• Do as many of the upgrades described in as possible.

• Compare your test runtime!

If after this, your tests are still too slow, profile them with one of the tools described in the second half of .Then when you know where to focus, read the later chapters to determine ways of speeding up the slowest tests.

This book has a lot of code examples.I hope that is not a surprise!

Heres some information on their layout.

Commands to run in your shell look like this:

The $ represents the prompt - dont type that.Im using macOS but hope to have removed any macOS or *nix bias.Commands should work cross-platform, but where thats not possible Ive added a separate Windows example with a PowerShell prompt:

If Ive written something incompatible with your platform (most likely Windows), please let me know.

Examples have been prepared with Python 3.9 and Django 3.2.Im only using the official Python distribution, CPython, since Django or its dependencies often dont work on other Python interpreters.

Python code is formatted with Black.(Like Django itself will be one day.)Import statements are sorted and grouped with isort.And all code is also linted with Flake8.

Throughout, I refer to installation commands with:

Im using python -m pip as per pip developers recommendations - see my blog post on the subject.

Please read these commands as install using your dependency management solution.There are several tools for this in the Python ecosystem, and I cant cover them all.

If you arent using any dependency management tool (beyond pip), pip-compile is a great start as its a small layer on top of pip.Its also my preference.

The example projects Ive used are based on my simple-core startproject template.This has directory structure like this:

This is slightly different to Djangos normal startproject template.The whole project lives inside a single Python package, example . example.core is an app, inside which there are models and tests.This is counter to the Django default of having apps in modules that are peers to the project app.

I prefer this layout because it makes it clear when youre importing something belonging to your project.Owned names always start with example (or whatever your project is called).

This PDF also came with a resources.zip .This includes all the example projects.Theyre structured with one directory per chapter, within which there are several individual projects.

When a file would be particularly useful to open, I mention the file name to open.Otherwise, the code examples dont refer to their file names, since they should stand alone.If you do find yourself wanting to refer to some code, the files should be named well enough to figure out which one you want.

Quite often in examples, I snip the code or output to keep it short and discard irrelevant details.You can spot this whenever theres a ... .For example, here Ive snipped the list of subcommands:

In Python, ... is also syntactically valid - its the rarely needed Ellipsis object .This never has any meaning in my examples.

Thank you to everyone who helped me put this book together.

Thanks to my partner Mafalda for her love, encouraging me to carry on when I needed it, and encouraging me to stop at the end of the day.Thanks to my parents - my mum for sharing her experience as an author, and my dad for proofreading this book with the same care he applied to every homework essay and CV.And thanks to my brother and sister-in-law for their support.