Flask and Gunicorn are Python packages that are used together to serve various services at scale. Flask is a light-weight web application framework which in combination with Gunicorn, a WSGI server, becomes capable of serving larger loads with minimum development and configuration. Upon running a Gunicorn server, multiple processes, a.k.a ‘workers’, are spawned-up to handle individual requests that the application receives. With any multi-process application, there are needs of sharing data that can not only be read but also updated to be reflected across all sub-processes. This could range from using variables set through initialization to periodic configuration updates for the application from external sources.
Hence, I wanted to share simple but effective ways of sharing common data across workers in a Gunicorn+Flask application which can be applied to various situations.
The three methods I wanted to share utilize the following:
1. Shared Variables using Value, Array
2. Multiprocess Manager
3. Hangup(HUP) signals
I have written a sample application to demonstrate a working example that can be referenced from my Github repository.
Before going into the details, it is beneficial to know about the fundamentals of how processes work because essentially, the Gunicorn workers are sub-processes.
When a process is ‘forked’, data within the process is copied to a sub-process and stored in memory separate from the parent process. As Gunicorn adopts a pre-fork model, the workers are forked before requests are handled from the application. Thus, any global variables or data created upon start-up is made available for each worker.However, because each process is allocated their own address space in memory, changing a global variable from one process does not affect the same variable in other processes.
By understanding that each worker of a Gunicorn application is a process, there are a couple of ways to get and update information with minimum help of other storage systems like file systems, databases or external systems.
Example Application Setup
i) App Initialization
To be able to setup data and the references upon deploying a Gunicorn server, I create a Custom Application by using the gunicorn.app.base.BaseApplication. As for the initialization, it covers the creation of variables stored in a dictionary data:
- Global variable
multiprocessing.Valuemultiprocessing.Arraymultiprocessing.Manager- Variable for SIGHUP example + background thread to update data on the main thread
which are referenced throughout the article.
ii) Test script
I have written a test script which will execute HTTP requests to fetch data from corresponding to the examples that are shown below. First it queries the application to get the current state of each worker and prints the responses. Next, a new value is sent to the application to update the appropriate type of variable and again retrieves the state of each worker. Each response from a worker is prepended with their process ID (pid).
0. Problem with using Global Variables
As mentioned earlier, once the process has been forked, any updates to it’s state is independent of the other sub-processes that had been created earlier. Therefore, even if we have a global variable global_data defined in initialization from the code above, updating a specific variable only updates for the specific worker that received the update request.
$ ./run.sh test_global_variable
Getting data from workers...
[Worker 45172] This is global data
[Worker 45173] This is global data
[Worker 45171] This is global data
[Worker 45170] This is global data
[Worker 45174] This is global data
Updated! # Update to 'new_data'
Getting data from workers...
[Worker 45170] This is global data
[Worker 45173] This is global data
[Worker 45174] new_data
[Worker 45172] This is global data
[Worker 45171] This is global dataAs shown from the logs of the test, only worker 45174 has the updated value.
1. Shared Variables
The simplest way is to create use the Value or Array shared memory objects from the multiprocessing package. These data types can store a single or multiple values in a declared variable and fundamentally the data is shared across processes that have a reference to the object. However, the types of data that can be stored in the two shared objects are limited to int or double. More flexibility can be found in the sharedctypes module which support a wider variety of ctypes to be stored in the shared memory map.
In the example application that I have provided, we can solve the problem that we saw when data was managed through a global variable. Upon initialization of the app, the reference to the shared variable gets forked into the workers and because the memory is shared across the processes, all workers are in sync. This is shown in tests using Value:
$ ./run.sh test_mp_value
Getting data from workers...
[Worker 45465] <Synchronized wrapper for c_double(0.0)>
[Worker 45463] <Synchronized wrapper for c_double(0.0)>
[Worker 45464] <Synchronized wrapper for c_double(0.0)>
[Worker 45462] <Synchronized wrapper for c_double(0.0)>
[Worker 45461] <Synchronized wrapper for c_double(0.0)>
Updated! # Update to 1.0
Getting data from workers...
[Worker 45462] <Synchronized wrapper for c_double(1.0)>
[Worker 45465] <Synchronized wrapper for c_double(1.0)>
[Worker 45461] <Synchronized wrapper for c_double(1.0)>
[Worker 45463] <Synchronized wrapper for c_double(1.0)>
[Worker 45464] <Synchronized wrapper for c_double(1.0)>and Array:
$ ./run.sh test_mp_array
Getting data from workers...
[Worker 48736] 0,1,2,3,4
[Worker 48732] 0,1,2,3,4
[Worker 48733] 0,1,2,3,4
[Worker 48735] 0,1,2,3,4
[Worker 48738] 0,1,2,3,4
Updated! # Update index 0 to 100
Getting data from workers...
[Worker 48732] 100,1,2,3,4
[Worker 48735] 100,1,2,3,4
[Worker 48736] 100,1,2,3,4
[Worker 48733] 100,1,2,3,4
[Worker 48738] 100,1,2,3,4where both provide consistency in data across all workers.
As a side note, the shared memory objects have synchronization built-in so there is no need to maintain a separate Lock when updating the variables. Simply use the .get_lock().
2. Manager
Another method that can be used from the same multiprocessing package is the Manager class. This is a server process that manages the storage of shared data. As compared to the previous method, it accommodates for a wider variety of data structures, such as a dictionary. This is shown in the example application where the value within the dictionary is successfully updated and available to all workers.
$ ./run.sh test_mp_manager
Getting data from workers...
[Worker 45625] {'manager_key': 'manager_value'}
[Worker 45623] {'manager_key': 'manager_value'}
[Worker 45624] {'manager_key': 'manager_value'}
[Worker 45627] {'manager_key': 'manager_value'}
[Worker 45626] {'manager_key': 'manager_value'}
Updated! # Update 'manager_key' to 'new_manager_data'
Getting data from workers...
[Worker 45625] {'manager_key': 'new_manager_data'}
[Worker 45623] {'manager_key': 'new_manager_data'}
[Worker 45626] {'manager_key': 'new_manager_data'}
[Worker 45627] {'manager_key': 'new_manager_data'}
[Worker 45624] {'manager_key': 'new_manager_data'}While using this option, the documentation states that this method is slower than using Value or Array. Having that said, it would be wise to evaluate the structure and amount of data that needs to be shared before using the Manager.
3. HUP signals
Lastly, the HUP or the Hangup signal can be sent to the main process to force the workers of the Gunicorn application to be re-spawned. As per Gunicorn’s signal handling documentation, upon a hangup, the new configuration file is loaded and all workers are gracefully shutdown as new worker processes are created. This also means that whatever the current state of the main process holds, at the moment of handling the HUP signal, it will be copied over to the newly created workers.
To demonstrate this, imagine a situation where the application needs to pull configuration updates on a daily basis. By using the signal handling and introducing a background thread to act periodically update the state of the application, workers will be refreshed to have the up-to-date information from the master process. For the sake of a simplicity, the example pulls updated information from a local file every 5 seconds.
$ ./run.sh test_signal_handling
Getting data from workers...
[Worker 49170] Init
[Worker 49167] Init
[Worker 49168] Init
[Worker 49171] Init
[Worker 49169] Init
Updated! # Update file with "data:{date}"
Getting data from workers...
[Worker 49196] new_sighup_data:Mon Sep 13 22:18:23 KST 2021
[Worker 49193] new_sighup_data:Mon Sep 13 22:18:23 KST 2021
[Worker 49249] new_sighup_data:Mon Sep 13 22:18:23 KST 2021
[Worker 49252] new_sighup_data:Mon Sep 13 22:18:23 KST 2021
[Worker 49251] new_sighup_data:Mon Sep 13 22:18:23 KST 2021You might wonder about what happens when the application is at a state of refreshing the workers — a valid aspect to consider. While the duration of forking and killing processes are relatively fast, the application will have a state where there are discrepancies of data amongst the workers because each process is killed ‘gracefully’. If a worker is in the middle of handling a request, only after successfully responding to the request will the process be killed. This allows for slight delays in the application as a whole to have fully gone through the life cycle of restarting all workers. This eventual consistent characteristic should be evaluated with the requirements and importance of data the application is expected to handle.
Conclusion
We have looked at different ways of how data can be shared throughout the workers of a Gunicorn application for various situations. The fundamentals of how processes share data and how signals are handled when using Gunicorn make these options possible. I have personally used all the methods described in this article in various scenarios to push services in production. There is no right or wrong, but you may want to consider factors like the architecture of the whole system (is it distributed?) or the acceptable latency for your application.
