Danjou - The Hackers Guide to Scaling Python

• Write a single-threaded application. This should be your first pick, and indeed it implies no distribution. They are the simplest of all applications. They are easy to understand and therefore easier to maintain. However, they are limited by the power of using a single processor.
• Write a multi-threaded application. Most computers even your smartphone are now equipped with multiple processing units. If an application can overload an entire CPUs, it needs to spread its workload over other processors by spawning new threads (or new processes). Multi-threading applications are more error-prone than single-threaded applications, but they offer fewer failure scenarios than multi-nodes applications, as no network is involved.
• Write network distributed applications. This is your last resort when your application needs to scale significantly, and not even one big computer with plenty of CPUs is enough. Those are the most complicated applications to write as they use a network. It means they should handle a lot of scenarios, such as a total or partial failure of a node or the network, high latency, messages being lost, and any other terrible property related to the unreliability of networks.

Scaling across processors is usually done using multithreading. Multithreadingis the ability to run code in parallel using threads . Threads are usuallyprovided by the operating system and are contained in a single process. Theoperating system is responsible to schedule their execution.

Since they run in parallel, that means they can be executed on separateprocessors even if they are contained in a single process. However, if only oneCPU is available, the code is split up and run sequentially.

Therefore, when writing a multithreaded application, the code always runsconcurrently but runs in parallel only if there is more than one CPU available.

This means that multithreading looks like a good way to scale and parallelizeyour application on one computer. When you want to spread the workload, youstart a new thread for each new request instead of handling them one at a time.

However, this does have several drawbacks in Python. If you have been in thePython world for a long time, you have probably encountered the word GIL , andknow how hated it is. The GIL is the Python global interpreter lock , a lockthat must be acquired each time CPython needs to execute byte-code.Unfortunately, this means that if you try to scale your application by makingit run multiple threads, this global lock always limits the performance of yourcode, as there are many conflicting demands. All your threads try to grab it assoon as they need to execute Python instructions.