Introduction

While forecasting time series data there comes a time when you need to fine tune the various parameters of your algorithm to get better speed and accuracy in results. Using multiprocessing the process time is decreased drastically. Here is a step-by-step guide on how to do this in python.

Time Series Model - SARIMA (Seasonal Autoregressive Integrated Moving Average)

If the time series data has seasonality then we have to use SARIMAX model which uses seasonal differencing. Seasonal differencing is similar to regular differencing expect for the regular differencing we have to subtract consecutive term whereas for seasonal differencing we subtract the value from the previous season.

The model is represented as SARIMAX(p,d,q)(P,D,Q)m

where p,d,q represents,

p is the order of the AR term, q is the order of the MA term, d is the number of differencing to make the time series stationary.

where P,D,Q,m represents,

P is Seasonal AR term, Q is seasonal MA term, D is seasonal difference order, m is the number of time steps for a single seasonal period

Let’s get started with the example -

Python with a multiprocessing module allows your code to run functions in parallel by offloading calls to available processors.

Step 1:

We need to import all the necessary libraries required for multiprocessing.

Step 2:

We will use Air Passengers Dataset for this example. This dataset contains the number of air travel passengers from the start of 1949 to the end of 1960. This dataset has a positive trend and annual seasonality.

As soon as the dataset is read, the index is set to the date. This is standard practice when working with time-series data in Pandas and makes it easier to implement time series models.

Step 3:

The line chart shows the number of Airline Passengers by date. At first glance, the plot makes it abundantly evident that the dataset exhibits an upward trend as well as seasonality or cyclicity and there does not appear to be any major irregularities or noise in the data.

Step 4:

Seasonal decomposition allows you to break (or “decompose”) time series data into its seasonal, trend, and residual components. For this , we exploit the seasonal_decompose() function provided by the statsmodels library. Among the input parameters, we can specify the decomposition model (additive or multiplicative) and if we want to extrapolate the trend or not .By analyzing these components, we can identify some pieces of our SARIMA model to include.

Step 5:

Output

Here, last 3 months will be used for testing the seasonal ARIMA model and everything else will be used for training.

Step 6:

Autocorrelation Function (ACF)-

Correlation between time series with a lagged version of itself. The correlation between the observation at the current time spot and the observations at previous time spots. We will be using the plot_acf function from the statsmodels.graphics.tsaplots library.

Partial Autocorrelation Function (PACF)-

Additional correlation explained by each successive lagged term. The correlation between observations at two time spots given that we consider both observations are correlated to observations at other time spots. We will be using the plot_pacf function from the statsmodels.graphics.tsaplots library.

Step 7:

In this, the custom parameter grid search has the ARIMA terms d=[1,2], p =[1,2] and q =[1,2]. The seasonal terms are: D =[1,2], P =[1,2] and Q =[1,2]. Finally, m is the number of terms in each season i.e. [6,12].

With all the different hyperparameters permutations, a total of 128 models are validated. If the number of hyperparameters is increased, the number of permutations increase exponentially.

Step 8:

Here, I have defined a function Sarima() which is predicting the test values and on basis of the predictions we can find the error percentage for each parameter obtained in param_grid_models_ls using MAPE (Mean Absolute Percentage Error) metric. Then I have created a process using multiprocessing module and run iterations on the whole dataset so that the work is divided into processes which leads to significant speedup in tasks. Multiprocessing enables the computer to utilize multiple cores of a CPU to run tasks/processes in parallel.

Step 9:

In the above code, we are doing out-of-sample forecast for the next year i.e. 1961 using forecast() function. It took 0.3 seconds to execute it.

The forecast() function has an argument called steps that allows you to specify the number of time steps to forecast.

Conclusion

In this article, I have demonstrated how to adapt a multiprocessing framework to forecasting model from SARIMA on the dataset along with the out-of-sample forecast.

Further, we can use the same approach to boost the forecasting for the other time series models as well.

For more information you can refer this Link.

References

https://statskernel.ca/sarima-models-hyperparameter-tuning-in-parallel/

https://medium.com/analytics-vidhya/fine-tune-sarima-hyperparams-using-parallel-processing-with-joblib-step-by-step-python-code-2037fec1659

https://ao.ms/multiprocessing-pools-in-python/

https://towardsdatascience.com/time-series-forecasting-with-arima-sarima-and-sarimax-ee61099e78f6

Introduction

In this tutorial we will grasp an understanding of multi-threading and multi-processing and see in practice how these techniques can be implemented in Python. Before discussing about threading and multi-processing it’s important to understand the multitasking process.

Multitasking

Multitasking is performing multiple tasks at the same time in the operating system, for example, our computer runs multiple applications at the same time. Multitasking programming is like eating breakfast and listening to music over and over in our lives. Multitasking involves often CPU switching between the tasks, so that users can collaborate with each program together. The biggest benefit of multitasking is to improve efficiency and improve resource utilization.

Types of multitasking-

• Process-based Multitasking
• Thread-based Multitasking

Process-based Multitasking

• In process-based multitasking, two or more processes and programs can be run concurrently.
• It is unable to gain access over the idle time of the CPU.
• It has a faster data rate multi-tasking because two or more processes/programs can be run simultaneously.
• Example: Downloading, listening to songs and playing a game.

Thread-based Multitasking

• In thread-based multitasking, two or more threads can be run concurrently.
• It allows taking gain access over idle time taken by the CPU.
• Threads are lighter and cause less overhead. Also, because they share the same memory inside a process, it is easier, faster, and safer to share data.
• Example: In a word-processing application like MS Word, we can type text in one thread, and spell checker checks for mistakes in another thread.

Thread vs Process

A thread is a unit of execution in a process. Threads can execute individually while sharing their process resources but a process can have multiple threads running concurrently, taking on different parts of the task.

A process is basically the program in execution. When you start an application in your computer (like a browser or text editor), the operating system creates a process. Multiple processes can be running the same program, but they can use different data and compute resources.

The Basics

Threading and multi-processing are two of the most fundamental concepts in programming. Python supports various mechanisms that enable various tasks to be executed at almost the same time. The key differences between multi-threading and multi-processing are:

1. A process is an independent instance executed in a processor core. Threads is an entity that resides within a process and run concurrently (inside that process).

2. True parallelism can ONLY be achieved using multiprocessing by taking advantage of multi-core machines since processes can run on different CPU cores.

3. Only one thread can be executed at a given time inside a process time-space due to Python’s global interpreter lock (GIL)

4. Threads share the same memory and can write to and read from shared variables while Processes has its own memory space

Concurrency and Parallelism

Concurrent execution means that two or more can start, execute and complete at the same time. Therefore, these tasks don’t necessarily have to run simultaneously they just need to make progress in an overlapping manner. One of the main goals of concurrency is to prevent tasks from blocking each other by switching back and forth, when one of the tasks is forced to wait

An application consisting of two tasks that are being executed concurrently in a single core is illustrated in the diagram below.

Parallel execution implies that two or more jobs are being executed simultaneously. Therefore, it is not possible to have parallelism on machines with a single processor and single core. With parallelism we can maximize the use of hardware resources.

In multi-core environments, each core can execute one task at exactly the same time, as illustrated in the diagram below:

Multi-threading implements concurrency whereas Multi-processing implements parallelism.

The Global Interpreter Lock (GIL)

We know that threads share the same memory space, so special precautions must be taken so that two threads don’t write to the same memory location. The CPython interpreter handles this using a mechanism called GIL or the Global Interpreter Lock.

The Global Interpreter Lock (GIL) is a process lock that prevents multiple threads from executing simultaneously in a Python process. Even though multiple threads can be running concurrently in a process, only one thread can be executing code at any given time, and the rest must be waiting. This lock is necessary mainly because CPython’s memory management is not thread-safe. Therefore, Python cannot use multiprocessing automatically. However, the multiprocessing module solves this problem by bypassing the GIL.

Multithreading: I/O bound tasks

Multithreading is a program execution technique that allows a single process to have multiple code segments (like threads) sharing the same CPU and memory. However, because of the GIL in Python, not all tasks can be executed faster by using multithreading. Multiple threads cannot execute code simultaneous, but when one thread is idly waiting, another thread can start executing code.

Multithreading in Python is perfect for I/O bound tasks, which are tasks whose execution time is primarily bound by the time spent waiting for input and output. Examples of these includes downloading data from the Internet and writing data to files.

Let’s do some hands-on practice-

Multiprocessing: CPU Bound tasks

Multiprocessing is when multiple processes are spawn from the main process, each having its own CPU and memory. These additional CPUs help increase the computing speed of the system. Processors share processes and resources amongst themselves dynamically so that no processor may sit idle or get overloaded.

Multiprocessing in Python is ideal for CPU-bound tasks, whose execution time is essentially constrained by the speed of the CPU. Because the workload is distributed among several CPUs, multiprocessing helps speed up tasks that have a high CPU utilization.

Let’s do some hands-on practice-

concurrent.futures

Python introduced the concurrent.futures module that provides a simpler interface to bring together both the threading and multiprocessing modules. It makes use of the ThreadPoolExecutor and ProcessPoolExecutor classes to manage thread and process pools, which share much of the same interface to make switching between multithreading and multiprocessing easier.

Conclusion

In today’s article we introduced about the multitasking process and two of the most fundamental concepts in programming, namely concurrency and parallelism and how they differ when it comes to execution. Furthermore, we discussed about threading and multi-processing and explored their main differences as well. Finally, we showcased how to implement threaded and multi-processing applications with Python.

References

https://towardsdatascience.com/multithreading-multiprocessing-python-180d0975ab29

https://machinelearningmastery.com/multiprocessing-in-python/

https://towardsdatascience.com/multithreading-vs-multiprocessing-in-python-3afeb73e105f

https://www.guru99.com/difference-between-multiprocessing-and-multithreading.html

https://www.youtube.com/watch?v=fKl2JW_qrso&list=WL&index=3&t=1657s