Celery Tutorial: A Must-Learn Technology for Python Developers
When you work on data-intensive applications, long-running tasks can seriously slow down your users.
Modern users expect pages to load instantaneously, but data-heavy tasks may take many seconds or even minutes to complete. How can we make sure users have a fast experience while still completing complicated tasks?
Enter Workers + Message Queues
If we want users to experience fast load times in our application, we’ll need to offload some of the work from our web server.
Workers
One way we do this is with asynchronicity. While the webserver loads the next page, a second server is doing the computations that we need in the background.
We call these background, task-based servers “workers.” While you typically only have one or a handful of web servers responding to user requests, you can have many worker servers that process tasks in the background.
These workers can then make changes in the database, update the UI via webhooks or callbacks, add items to the cache, process files, send emails, queue…
