Making Multiprocessing Easy for All

Speed up your applications by doing the heavy lifting in parallel

Photo by Artiom Vallat on Unsplash

What’s Multiprocessing?

Multiprocessing is the use of two or more central processing units (CPU) within a single computer system.

The term also refers to the ability of a system to support more than one processor or the ability to allocate tasks between them.

What Are the Advantages of Multiprocessing?

The advantages of the multiprocessing system are:

• Increased throughput: By increasing the number of processors, more work can be completed in a unit time.
• Cost saving: The parallel system shares memory, buses, peripherals, etc. A multiprocessor system thus saves money as compared to multiple single systems. Also, if a number of programs are to operate on the same data, it’s cheaper to store that data on one single disk shared by all processors instead of using many copies of the same data.
• Increased reliability: In this system, the workload is distributed among several processors, which results in increased reliability. If one processor fails, then its failure may slightly slow down the speed of the system, but the system will work smoothly.

What Are We Trying to Achieve?

I’ll be showing how we can reduce the time taken in the execution of code with the help of multiprocessing. Since Python is the most common language used for multiprocessing and to make the code simpler, we’ll be using Python.

Below is a simple program demonstrating calling a function two times, each time taking 1 second for its execution. I’ve used the time library to calculate the total execution time of the code.

The code execution will look like this

Now with the help of the multiprocessing library available in Python, we can make the code run in parallel.

As you can see, the execution time is reduced to 1.07 seconds.

Code execution will look like this

Using the ‘concurrent.futures’ Module

The concurrent.futures module provides a high-level interface for asynchronously executing the callable. The asynchronous execution can be performed with separate processes by using ProcessPoolExecutor. It implements an interface, which is defined by the abstract Executor class. Let’s modify the code to use ProcessPoolExecutor and run 10 processes.

The time is increased because our PoolExecutor has made a decision on the basis of hardware not to allot as many processes to run in parallel. The hardware here refers to the number of cores available on your machine.

Conclusion

That’s enough for this article. Stay tuned — in the next article, I’ll be discussing multithreading and how it’s different from multiprocessing.

If you’re facing difficulties connecting things together, here’s the repository demonstrating the same work:

GOVINDDIXIT/python

Thanks for reading!