Matthew Wilkes - Advanced Python Development: Using Powerful Language Features in Real-World Applications

Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the books product page, located at www.apress.com/978-1-4842-5792-0 . For more detailed information, please visit http://www.apress.com/source-code .

Python is a very successful programming language. In the three decades that it has existed, it has become very widely used. It ships by default with major operating systems; some of the largest websites in the world use Python for their back ends, and scientists are using Python every day to advance our collective knowledge. As so many people are working on and with Python daily, improvements come thick and fast. Not all Python developers have the chance to attend conferences, or the time to follow the work done by different parts of the community, so its inevitable that some features of the language and ecosystem are not as well known as they deserve to be.

The objective of this book is to examine parts of the language and Python tooling that may not be known to everyone. If youre an experienced Python developer, you may well know many of these tools, but a good many more may be on your to-do list of things to try when you have time. This is especially true if youre working on established systems, where rearchitecting a component to take advantage of new language features isnt something that can be done frequently.

If youve been using Python for a shorter period, you may be more familiar with recent additions to the language but less aware of some of the libraries available in the wider ecosystem. A large part of the benefit of attending events like Python conferences is the chance to notice minor quality-of-life improvements fellow programmers have made and integrate them into your workflow.

This is not a reference book with stand-alone sections covering different features of Python: the flow from chapter to chapter is dictated by how we would build a real piece of software.

With many pieces of technical documentation, there is a tendency to provide simple examples. Simple examples are great for explaining how something works, but not so useful for understanding when to use it. They can also be tricky to build on, as complex code is often architected quite differently to simple code.

By following this one example, we are able to consider technology choices in context. You will learn what considerations to bear in mind when choosing if a particular approach is suitable. Topics that are related by how theyre used will be covered together, rather than topics that are related by how they work.

My objective in writing this book is to share knowledge from different parts of the community and lessons learned over 15 years of writing Python code for a living. It will help you to be productive, both with the core language and add-on libraries. You will learn how to effectively use features of the language that are not strictly essential to be a productive programmer, such as asynchronous programming, packaging, and testing.

However, this book is aimed at people who want to write code, not people who are looking to understand deep magics. I will not delve too far into subjects that involve implementation details of Python. You will not be expected to grok Python C extensions, metaclasses, or algorithms to benefit from this book.

Substantive code samples are shown as numbered listings, and the accompanying code for this book includes electronic versions of these listings. Some of these listings also have output shown directly beneath, rather than separately as a numbered figure.

The accompanying code for this book is also where youll find copies of the full codebase for the example on a chapter-by-chapter basis, as well as helper code for the exercises. In general, I would recommend that you follow along with the code by checking out the Git repository from the books website or the code distribution and changing to the relevant branch for the chapter youre reading.

As well as listings, I show some console sessions. When lines which are formatted like code begin with > , that indicates that a shell session is being shown. These sections cover commands to be run from your operating systems terminal. Any that involve >>> are demonstrating a Python console session and should be run from within a Python interpreter.

This books example is that of a general-purpose data aggregator. If you work in DevOps, then it is very likely you use a program of this sort to track the resource utilization of servers. Alternatively, as a web developer, you may use something like this for statistics aggregation from different deployments of the same system. Some scientists use similar methods, for example, for aggregating the findings of air-quality sensors distributed across a city. It isnt something that every developer will need to build, but it is a problem space that is familiar to many developers.

It has been picked not just because its a common task, but because it allows us to explore many of the subjects we want to cover in a natural, unified way. You will be able to follow the complete example perfectly well using any modern computer running any modern operating system, without purchasing any additional hardware. You may find you get more out of some of the examples if you have additional computers to act as remote data sources.

I will be using a Raspberry Pi Zero equipped with some aftermarket sensors for my examples. This platform is widely available for approximately 5 US dollars and provides lots of interesting data. There are commercial sensor add-ons available from many Raspberry Pi stockists.

Although Ill be recommending things specific to the Raspberry Pi to make following the examples easier, this book is not about the Internet of Things or the Raspberry Pi itself. Its a means to an end; you should feel comfortable to adapt the examples to fit tasks that are more relevant to your interests if you like. Any of the similar problems mentioned earlier would follow the same design process.