Jessica McKellar - Twisted Network Programming Essentials

Beijing Cambridge Farnham Kln Sebastopol Tokyo

As the Twisted projects originator and nominal leaderand as someone who is not being paid for writing thisI can very honestly say that this is a fine book, and it has made me proud of what Ive started. You now hold in your hands a wondrous key that contains the knowledge to unlock a very powerful software systema software system borne of a consistent, methodical vision; a vision half a decade in realization and hundreds of man-years in implementation; a vision for a video game that has yet to be written, called Divunal.

I have been lauded many times for my role in Twisteds creation, and in this foreword I will attempt to disabuse you of the notion that any of it was on purpose. Not only was it an accident, but neither I, nor anyone else, has made one iota of progress towards my original goal of writing a game.

When I was eight years old, I decided I wanted to be a writer. I was going to write video games just like my favorite ones, the text-based games from Infocom. They were like books, but better. I knew how to write alreadyat a fourth-grade level, or so Im toldand all I needed to figure out was the part where the computer wrote back. Lucky for you nobody thought to tell me how hard that step between the input and the output was, or Twisted would be a series of detective novels instead of a Python program.

Tolkien said it best: The tale grew in the telling, and Ill say it worse: the code grew in the hacking. Twisted began over a decade after my aforementioned first plunge into the netherworld of software, as a solitary attempt to create a networking subsystem for a small online fantasy world. Since then, it has become an ongoing community quest to unify all manner of asynchronous communications. This book will take you on an adventure through Twisted for the Web, Twisted for email, Twisted for chat, and of course, Twisted for whatever new kind of networked application you want to dream upmaybe even an online video game.

Much as the tale of Twisted has grown and changed, its origins still have a profound effect on its nature, and on its future. Having origins in an eclectic[] audience. The community in the online support forum engages in discussions that are often funny. To put it more directly: were weird.

Weird is a badge I have long worn with pride, dear reader, so please take it as a compliment that I bestow it upon you. Youre not simply non-average, youre better than average. Almost by definition, Twisted hackers are the ones for whom good enough isnt good enough. You are the web programmers who cant use their operating systems stock HTTP daemon because you need more power and more control over how its run; the chat developers who arent content with chatting on a perfectly working network just because it doesnt support some cool new features you want; the (dare I say it?) gamers who arent content with the markets offerings of online games. You want to create something newer, different, better. To build higher than those who have come before, because you are building not merely upon the shoulders of giants, but upon the apex of an acrobatic balancing act of giants, or more literally an interlocking network of frameworks and libraries for different tasks, rather than just one at a time.

Twisted will let you do that, by letting you leverage code written by far more and far better programmers than I. Twisted provides a common method for that code to cooperate, which means you can use all of that code without performing a complex integration pass. In this spirit, Id like to invite you to release your Twisted-based projects, or the infrastructure components of them, as open source software, so that we might together build a Twisted commons upon which many more fantastic applications will be built.

Dont mistake this friendly vision for altruism, however. I didnt have anything to do with the start of the Free Software or Open Source movements, respectively, but they came along at a convenient time for me. This feeling of share-and-share-alike has been a feature of the Twisted community since day one, but not because I care about sharing.[

You see, the majority of Twisteds core developers, including myself, suffer from Attention Deficit Disorder. This malady is the grease that makes the magic wheels of integration turn. While most developerssane developerswould be content to write a perfectly good web server that could work only as a web server and leave it at that, we are always afraid well suddenly lose interest and need a chat application insteador maybe it should be a mail server? Hey, theres a squirrel! I dont like this song.

What was I saying? Oh yes. The essence of Twisted is apparently paradoxical. Created on a whim by crazed eccentrics, designed to be a toy, and yet powerful enough to drive massive email systems, high-traffic web sites, transaction-processing systems, and inventory management applications.

However, the paradox is an illusion. People produce the best work when they are working and having fun at the same time. It takes a sense of humor to call yourself a crazed eccentric (whether its true or not). You have to have a sense of fun to try and build a toy. In enjoying ourselves, we have brought to life a system that many of us have tried and been unable to create in more serious surroundings.

So, when I look out upon the lone and level sands of Divunal, a game whose incarnation today is little more than its name, I am not concerned. I am having a good time with Twisted. With this book in hand, I have no doubt that you will, too.

Matthew the Glyph Lefkowitz
CTO at Divmod, Inc.
(not a game company)
(yet)
August 2005

This book is about Twisted, an open source, event-driven networking engine written in Python.

Twisted supports many common transport and application layer protocols, including TCP, UDP, SSL/TLS, HTTP, IMAP, SSH, IRC, and FTP. Like the language in which it is written, it is batteries-included; Twisted comes with client and server implementations for all of its protocols, as well as utilities that make it easy to configure and deploy production-grade Twisted applications from the command line.

Twisted includes both high- and low-level tools for building performant, cross-platform applications. You can deploy a web or mail server with just a few lines of code, or you can write your own protocol from scratch. At every level, Twisted provides a tested, RFC-conforming, extensible API that makes it possible to rapidly develop powerful network software.

In keeping with the spirit of the OReilly Essentials series, this book is not about torturing you with the nitty-gritty details of specific networking protocols, or with exhaustively documenting Twisteds APIs. For a comprehensive treatment of how to use Twisted to build applications for a particular protocol, please get your footing with this book and then consult the official documentation.