Symfony Lock: dealing with shared resources, concurrency and parallelism
Take control over shared resources across PHP processes with this simple, yet powerful tool
Asynchronous tasks in PHP
You certainly know it and I don’t teach you anything by telling you this: pure multithreading and parallelism do not exist with PHP. Although some solutions like Tornado from the French company M6 Web try to compensate for this lack, it is not a native solution as we could find it with some languages more or less low level, such as C, C++, Java or C#.
So there you have it, obviously we don’t have a way to do real parallelism, but we can get close. That’s what I’m referring to when I talk about asynchronous tasks.
You will surely encounter (or have already encountered) in your life as a PHP developer performance issues with heavy tasks. The problem with these tasks is that they are disastrous for the user experience, especially in the web. The first risk is that the user will go into timeout, or even that your site will not respond.
I recently worked for a company where these performance issues were at the heart of our thinking. We were dealing with huge amounts of external API calls, and the idea of asynchronous tasks came up quite quickly and quite naturally.
To make it simple, and because it’s not the purpose of this article, we have set up a system that regularly checks if tasks (whose details are persisted in the database) need to be executed, and if so, unpack them from the task queue. It works well, and this system allows us to be very flexible in terms of scaling. We just need to add a watcher on top of the first one so that the tasks can be processed twice as fast (roughly).
And this is where the concurrency problems start: if two workers (or more, maybe 10, 15 or even hundreds?) are processing data at the same time, how can we be sure that they are not modifying the same data at the same time? Or more simply, how can we be sure that they are not processing the same task at the same time, because they would have taken it at the exact same moment, before one of the workers could warn that the task was being processed?
This is when we enter the wonderful world of locks and mutual exclusions (also called mutex).
Symfony locks: concurrency and shared resources
We’ll take a very simple example of a worker retrieving the next asynchronous task to be processed from the database (pay attention to lines where
taskLocker is called):
As you can see, there is a time lapse between the moment when the task details are retrieved and the moment when the task status (which we change to “processing”) is flushed to the database (line 30 to 40). Although this time can be only a few microseconds, we are working today on multicore machines, which can execute several instructions at the same time. And this can cause real problems when we have a lot of workers. Moreover, we have to try to minimize the database flush calls as much as possible, for obvious performance reasons. A hard disk access is the worst you can do in terms of speed.
If you look closely, we’re dealing with this problem with a lock, thanks to Symfony’s Lock component and Lock Factory. We are acquiring a blocking lock just before fetching the next task from database, and releasing it as soon as we can to let other workers fetch tasks.
Because we’re dealing with just one lock named
async_task_lock, and because we declared it as blocking (the true parameter we’re passing to
acquire at line 28), anyone trying to acquire this precise lock will “wait” at this line of code for the lock to be released. And because we want this lock to be shared across all workers (across different PHP processes, and maybe even across different servers!), we have several options to configure it. Symfony Lock component allows you to choose where to store the lock:
- In database with
- In a file with
FlockStore(not the best one for performances and to share locks across servers though, because we’re storing the lock on the filesystem) ;
- In a cache system like Redis with
- And many more!
My choice goes naturally to Redis. If you don’t know about it and to be short, Redis is a key-value cache system. Written in C and storing data in RAM, its performances are mind-blowing. You can search a key in a set of million ones in just a few milliseconds, on a mid-end laptop. Symfony can create the lock in Redis for you. This way, all workers will search for locks in Redis, allowing them to “share” these locks.
Once the lock is released (by calling the
release method in the worker that currently holds the lock), another worker will acquire the lock while the asynchronous task is running by the first one. And you’re sure your workers won’t process a task twice!
Even if it might not be the very best solution for this problem, this practical case demonstrates concretely the usefulness of locks.
Mutual exclusion and locks are absolutely necessary when dealing with some sort of parallelism. They allow you to control resources access. More than being super useful, locks will definitely make your debugging easier. You don’t want to investigate on shared resources bugs with some parallelism without locks. Trust me.
More info here: https://symfony.com/doc/current/components/lock.html