Deploy celery and celery beat in production with Django (Ubuntu)

Celery

We all have been in this situation, we try some new tool in development and it works wonder. Then we wish to deploy it in production and that’s where the nightmare begins.

In this article, I will talk about how to implement celery and celery beat into production. Celery daemonizing is a well talked topic on stackoverflow but then clear and actual implementation is missing.

Use Case : Use celery to perform task and use celery beat in conjunction with celery to perform periodic task which executes every second to aggregate data and perform analytics on it.

Requirements : A django based web application where celery is already being used.

Solution :

There are in total 3 files we need to deploy the celery and celery beat.

1. Celery configuration file which contains the variables and details which are ought to be used in celery and celerybeat service files.
2. Celery service file which will invoke the workers to perform the tasks.
3. Celery beat service file which is used to send out periodic tasks to celery which in turn will execute those task.

P.S. In case you started celerybeat service and are wondering why the tasks are not being executed, that’s because sometime, we miss to start the celery service. While starting celerybeat service, one has to make sure that atleast one worker is up and running.

Now the 3 files.

1. Celery (Celery configuration file)

# Names of nodes to start
# most people will only start one node:
CELERYD_NODES="worker1"
# Absolute or relative path to the 'celery' command:
CELERY_BIN="/opt/anaconda/envs/django_env/bin/celery"
# App instance to use
CELERY_APP="inlog_p_v2"
# How to call manage.py
CELERYD_MULTI="multi"
# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"
# Set logging level to DEBUG
CELERYD_LOG_LEVEL="DEBUG"
# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"
# Workers should run as an unprivileged user.
# You need to create this user manually (or you can choose
# a user/group combination that already exists (e.g., nobody).
CELERYD_USER="ubuntu"
CELERYD_GROUP="ubuntu"
# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

Point to be noted here is that, the documentation assumes celery as username but that will generally cause issues as you probably wouldn’t have made the user named celery whilst you deployed your django application. Although if you use celery user you would run into permission issues at lot of places. Hence, make sure you use the same user which owns the django application and definitely not root.

2. Celery service file (celery.service) :

[Unit]
Description=Celery Service
After=network.target
[Service]
Type=forking
User=ubuntu
Group=ubuntu
EnvironmentFile=/etc/default/celery
WorkingDirectory=/opt/inlog_p_v2
ExecStart=/bin/sh -c '${CELERY_BIN} -A $CELERY_APP multi start $CELERYD_NODES \
--pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
--loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait $CELERYD_NODES \
--pidfile=${CELERYD_PID_FILE} --loglevel="${CELERYD_LOG_LEVEL}"'
ExecReload=/bin/sh -c '${CELERY_BIN} -A $CELERY_APP multi restart $CELERYD_NODES \
--pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} \
--loglevel="${CELERYD_LOG_LEVEL}" $CELERYD_OPTS'
Restart=always
[Install]
WantedBy=multi-user.target

There are 3 main variables to this service file which have to be considered. Type User and Group . Make sure that user and group are same as defined in config file. Then Type has to be forking.

3. Celery beat service file (celerybeat.service) :

[Unit]
Description=Celery beat service
After=network.target celery.service
[Service]
Type=simple
User=ubuntu
Group=ubuntu
EnvironmentFile=/etc/default/celery
WorkingDirectory=/opt/inlog_p_v2
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat  \
--pidfile=${CELERYBEAT_PID_FILE} \
--logfile=${CELERYBEAT_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}'
ExecStop=/bin/systemctl kill celerybeat.service
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target

Same like celery.service file, Type User and Group variable has to be properly defined. Make sure that the Type is set to simple.

Now let’s start the celery service and then we shall start the celerybeat service.

Reload the systemctl daemon so that it can acknowledge the two services which we made.

systemctl daemon-reload

Now we will enable the services and then start them.

systemctl enable celery.service
systemctl enable celerybeat.service
systemctl start celery.service
systemctl start celerybeat.service

You can see the status of the services using the command

systemctl status celery.service
systemctl status celerybeat.service

I hope this article would help someone who is trying to deploy celery and celery beat onto their server and having a hard time.

Thanks