Hacking Celery Task

how to assign periodic task to celery worker in any app

This is a quick example of how I got Periodic Tasks to work using Celery in my Python Flask app. There are lots of examples out there for creating tasks and loose documentation on how to start Celery and Celery Beat, but most of them but I didn’t found any basic example of periodic task. For example “How to send mail every 24 hour to subscriber”. I just wanted to run a simple example and I spent way too long trying to fill in the gaps to get even this simple task to run periodically. So hopefully this quick example will help somebody else out there save some time.

Some settings to add

Add this to your config.py.

CELERY_BROKER_URL = os.environ.get(‘REDISCLOUD_URL’) or ‘redis://127.0.0.1:6379'
CELERY_BROKER_BACKEND = os.environ.get(‘REDISCLOUD_URL’) or ‘redis://127.0.0.1:6379'

Celery instance

The first thing you need is a Celery instance, this is called the celery application. It serves the same purpose as the Flask object in Flask, just for Celery. Since this instance is used as the entry-point for everything you want to do in Celery, like creating tasks and managing workers, it must be possible for other modules to import it.

https://gist.github.com/rahulrrixe/8a131f0adffe05bd7dd3.js

The function creates a new Celery object, configures it with the broker from the application config, updates the rest of the Celery config from the Flask config and then creates a subclass of the task that wraps the task execution in an application context.

Periodic Email example

Now create your tasks (generally they will be saved into a file called tasks.py). This is what my tasks.py looks like:

https://gist.github.com/rahulrrixe/15ce3c75d6e1f81a9cb2.js

It have two tasks one is send_event_notification which is executed when we call the function and another one i.e. email_example is a periodic task which runs every minute, every hour and every week (‘*’ is for every).

Start Celery

Now we just need to start a Celery worker with the —beat flag and our task will run every minute, hour and week. This part was not so obvious, but the —beat flag needs to appear after worker, otherwise nothing will happen.

$ celery -A tasks worker --loglevel=info --beat

You can execute this on another terminal tab.

Be happy result is here

You should see Celery output a bunch of info as tasks are ran. Among them you should see a line that looks like this

[2014–10–01 23:22:00,041: WARNING/Worker-3] sent the mail to subscriber
[2014–10–01 23:23:00,038: WARNING/Worker-5] sent the mail to subscriber
[2014–10–01 23:24:00,004: WARNING/Worker-4] sent the mail to subscriber

Thanks ☺

Show your support

Clapping shows how much you appreciated Rahul Ranjan’s story.