DOKKU up and running with Django projects, postgresql, Celery, ssl, syslog for logging

got tired of scanning through dokku doc ?

(and lots of google search)

assuming you have

  • ubuntu or debian server(vps)
  • with root access(to install dokku)
  • or without root access(dokku already installed)
  • having your domain pointed to the vps: yourdomain.tld
  • or the vps is barebone with ip address, go to the last of this post
  • your local django project running with virtualenv
  • git in your local machine(even if you use mercurial or else)

required pip packages:

  • gunicorn
  • dj_database_url

in a terminal in your project root directory

source env/bin/activate
pip install dj_database_url
pip install gunicorn

install dokku in the server:

you may skip this step if dokku is up on server already

connect to your server:

ssh root@yourdomain.tld

If ssh server is running in a non standard port:

ssh -p <port> root@yourdomain.tld

now in a terminal(server):

https://raw.githubusercontent.com/dokku/dokku/v0.10.5/bootstrap.sh;
sudo DOKKU_TAG=v0.10.5 bash bootstrap.sh

grant other unix users dokku access, to prevent some mess when adding public keys via ssh: check this post

add ssh public key to server:

generate ssh keys:

ssh-keygen -t rsa -C “your_email@example.com”

leave the passphrase empty (just press enter key when asked for passphrase)

send the public key from your local machine to the vps:

cat /path/to/your/key.pub | ssh dokku@mydomain.tld dokku ssh-keys:add <an_arbitrary_unique_name>

Now verify with ssh connection to your vps with your private key, ssh password will not be asked

ssh -i /path/to/your/private.key dokku@mydomain.tld

I am still asked for ssh password

ssh key doesn’t appear to get added/have effect

append your public key to the server ~/.ssh/authorized_keys

  • copy your public key content
  • login to server
  • open ~/.ssh/authorized_keys
  • append the content(paste to bottom)

or with ssh-copy-id package from your local machine

ssh-copy-id -i /path/to/your/key.pub root@yourdomain.tld
pointing the ssh private key everytime is tiring if you are a human

point the ssh private key for the server, in your local ssh config

add these lines to your local ~/.ssh/config

Host mydomain.tld
IdentityFile /path/to/your/private.key

Now verify with ssh login again

If you face some error like too many authentication failure

put this entry too

IdentitiesOnly yes

So now the entry in your local ~/.ssh/config is finally

Host yourdomain.tld
IdentityFile /path/to/your/private.key
IdentitiesOnly yes

Procfile, runtime.txt, app.json:

in your local project root directory(where manage.py is):

remove/rename any Dockerfile in project root

create three text files named

  • Procfile
  • runtime.txt
  • app.json

a minimal Procfile contains:

web: gunicorn core.wsgi — log-file -

to specify python version for project, like python 2.7, add in runtime.txt

python-2.7.13

app.json minimal entries are:

{
"scripts": {
"dokku": {
"postdeploy": "./manage.py migrate"
}
}
}

assuming virtualenv set up and running for your project, export pip packages:

cd /path/to/your/project
source /path/to/your/project/env/bin/activate
pip freeze > requirements.txt
git add .

your dokku app

# in server
dokku apps:create yourappname
dokku apps:list

database inside dokku container:

install the postgres plugin in server(if not already installed):

dokku plugin:install https://github.com/dokku/dokku-postgres.git

create a postgres service:

dokku postgres:create yourappdb

link the dokku postgres service to your dokku app:

dokku postgres:link yourappdb yourappname

verify with

dokku postgres:info yourappdb

output like:

=====> Container Information
Config dir: /var/lib/dokku/services/postgres/yourappname/config
Data dir: /var/lib/dokku/services/postgres/yourappname/data
Dsn: postgres://postgres:b539b90bbe7ce37b08716970e1d232f6@dokku-postgres-yourappdb:5432/yourappdb
Exposed ports: — 
Id: 1b383687b155817556887520238aac2e235c59d74690bd3140c65d14fc361406
Internal ip: 172.17.0.7
Links: yourappname
Service root: /var/lib/dokku/services/postgres/yourappdb
Status: running
Version: postgres:9.6.1

Now verify

dokku config myappname

output like:

=====> myappname config vars
DATABASE_URL: postgres://postgres:b539b90bbe7ce37b08716970e1d232f6@dokku-postgres-myappdb:5432/myappdb

it means dokku linked the database service to the dokku app via a environment variable named DATABASE_URL,

so, when the app runs in dokku, the environment variable is available
this environment variable contains the postgresql database link Dsn, formatted as:

postgres://<database user><database password>@<host>:<port>/<database name>

now its your projects duty to utilize this environment variable, using the Dsn to connect to database

database settings in django project settings.py

use a python package dj-database-url
to specify the Dsn in settings.py(which eventually parses the Dsn for database, user, host, port, password and sets the config variables)

entry in your project settings.py

# Database
# https://docs.djangoproject.com/en/1.9/ref/settings/#databases

import dj_database_url
# to facilitate on dokku environment | by resgef
if os.environ.get('DATABASE_URL', ''):
DATABASES = {
'default': dj_database_url.config()
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'dbname',
'USER': 'dbuser',
'HOST': '127.0.0.1',
'PORT': '5432',
'PASSWORD': 'yourdbpass',
}
}

you may dump your local project sql and import it in server:

# in your local machine
pg_dump your_db > db.sql
scp db.sql root@yourdomain.tld:~

# in server
dokku:postgress connect yourappname < db.sql

importing the latest dump means you don’t need to run django migrate

project git remote

# in local project root directory
git remote add dokku dokku@yourdomain.tld:yourappname
git add .
git push dokku master

takes some time, gives lots of output about how its building and deploying

in the end you should get the application url like:

http://yourappname.yourdomain.tld

how to run manage.py from dokku, in vps:

dokku run appname python manage.py

Congrates! Now everytime you do some code and wanna pulish it, just git push to the dokku repo

Hey! I use mercurial for my project, not git!

just create a git repo in your project, create .gitignore file, add .git to your .hgignore file, with entry

.git/

.gitignore entry is like

env
.idea
*.pyc

so now just

  • git init
  • git add .
  • git commit -m ‘message’
  • git push …

I want https (letsencrypt)

thanking in https://medium.com/@pimterry/effortlessly-add-https-to-dokku-with-lets-encrypt-900696366890

Use Celery to process jobs, not supervisor.

I had lots of problems in processing jobs with supervisor.

Will be discussing in a later edit(sorry! extremely busy here with project loads)

I want to daemonize a script with supervisor:
(in the server)
Lets say you want to run myscript.py in background all the time(a replacement for cron or celery tasks in server)
In my case I wanted an email sender daemon
create supervisor config, save it as my_daemon.conf
[program:my_email_sender]
command=dokku run yourdokkuappname python scripts/daemon.email.py
autostart=true
autorestart=true
stderr_logfile=/var/log/myproject/err.log
stdout_logfile=/var/log/myproject/out.log
now daemonize
# in server
mkdir /var/log/myproject/
sudo touch /var/log/myproject/err.log
sudo touch /var/log/myproject/out.log

apt-get install supervisor

cp my_daemon.conf /etc/supervisor/conf.d/

supervisorctl reread

supervisorctl update

Celery:

(coming sooner)

Syslog for logging:

(coming sooner)

dokku success

My vps is not connected with a domain, only ip address available now:

I will write on this, later, in hurry at this moment

Troubleshooting:

pkg_resources

If you face problem(as shown in deployment) with pkg_resources(which is due to an ubuntu bug),

simply remove pkg_resources entry form requirements.txt, and then git-add, git-commit

https://stackoverflow.com/questions/7446187/no-module-named-pkg-resources

static files

if deployment(git-push) face problems with manage.py collectstatic:

check for STATIC_ROOT in settings.py

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static")

now in project core urls.py

# at top
from
django.contrib.staticfiles.urls import staticfiles_urlpatterns
# at bottom
urlpatterns += staticfiles_urlpatterns()

dont forget to git-add, git-commit, git-push

you may see this too:
https://stackoverflow.com/questions/19323513/heroku-django-oserror-no-such-file-or-directory-app-myappname-static

problems in gunicorn dokku web container

error: failed to push some refs to ‘git_remote’

ImportError: No module named ‘leads’

you may have modified project core directory and apps directory

but didnt include that in core wsgi file

problems in pip install

your runtime.txt python version is not correct

— — — — — — — — — — — — — — — — — — — — — — — — — — — -

Thanking in:

https://confluence.atlassian.com/bitbucketserver/creating-ssh-keys-776639788.html
http://dokku.viewdocs.io/dokku/deployment/application-deployment/
https://stackoverflow.com/questions/7446187/no-module-named-pkg-resources
https://alexperry.io/node/2015/05/05/how-to-host-your-node-app-with-dokku.html
https://github.com/dokku/dokku-postgres
http://dokku.viewdocs.io/dokku/community/plugins/
https://github.com/kennethreitz/dj-database-url
https://www.digitalocean.com/community/tutorials/how-to-read-and-set-environmental-and-shell-variables-on-a-linux-vps
https://stackoverflow.com/questions/34851335/accessing-postgres-database-inside-dokku-container-from-outside
https://blog.candlewaster.co/how-to-deploy-a-wagtail-site-to-dokku/
https://support.healthcareblocks.com/support/solutions/articles/1000237147-python-quickstart-guide
https://devcenter.heroku.com/articles/python-runtimes
https://realpython.com/blog/python/deploying-a-django-app-on-dokku/
https://www.digitalocean.com/community/tutorials/how-to-backup-postgresql-databases-on-an-ubuntu-vps