Automate boring stuffs, Python Fabric

Everyday we have to do many boring stuffs when it comes to DevOps. Everytime we have to deploy new version of our application, we need to run branch of codes. If we are using complex architecture like AWS ECS or GCP Kubernetes etc, deployment can be automated and it saves lots of our time. But when it comes to a small node management (AWS EC2, Linode, DigitalOcean Droplets etc) we need to connect via SSH and run our commands. That’s a boring stuff and I don’t like it. Also when we are deploying to development server, it has to be done many times a day. So, if deployment takes 5 mins and we need to do it 20 times just imagine I have to spend one hour for deployment.

Yup, automation makes the life easy

Python Fabric (www.fabfile.org) is a library or package which can run shell commands remotely via SSH. It saves million people’s time just by automating the tasks we needed to run by ourselves. Today, we will see how we can connect to an AWS EC2 instance and executing our task. The amazing part of using Python Fabric is, we don’t need to install or configure anything of the server side.

Let’s say we have an EC2 instance on IP x.x.x.x or DNS ec2-x-x-x-x-region.compute.amazonaws.com. And also we have a certificate file named x-x-x-x.pem for user ec2-user which we will be needed to connect with this instance. Now let’s install python3 and fabric if you don’t have it already. Let’s create a file named fabfile.py on our project root. To connect to our EC2 instance:

from fabric import Connection
try:
    print("Connecting to server")
    conn = Connection(
        host="ec2-x-x-x-x-region.compute.amazonaws.com",
        user="ec2-user",
        connect_kwargs={
            "key_filename": "x-x-x-x.pem",
        }
    )
    print("Successfully connected")
    # TODO: write your commands here
    print("Deployment was successful")
except Exception as e:
    print("Deployment was unsuccessful.\nError:\n")
    print(e)

You can see that we have wrapped the whole connection in try block to avoid any uncertain error like connection issue, network issue etc.

Also carefully write the path of the *.pem file. We assumed it is on the same directory for now.

Now, we can run any command on the conn variable like:

# to run simple shell commands
conn.run("pwd")
# to run with sudo (if no password is needed for sudo)
conn.run("ls", pty=True)
# to run with sudo (is password is needed)
sudoer = Responder(
    pattern=r'\[sudo\] password:',
    response="your-password\n"
)
conn.run('sudo ls', pty=True, watchers=[sudoer])
# to change directory and run something
with conn.cd('project'):
    conn.run('pwd')

Now, run your file like:

$ python3 fabfile.py
or
$ fab

You can also write tasks in fabfile.py like:

env.hosts = ['host1', 'host2']
def taskA():
    run('ls')
def taskB():
    run('whoami')

And we can run like:

$ fab --hosts host1,host2 taskA taskB
Output:
Running taskA on host1!
Running taskA on host2!
Running taskB on host1!
Running taskB on host2!

That’s all you need to know to jump into Python Fabric and automate your daily tasks for running shell commands remotely over SSH.

In the next article we will use Python QT to make GUI for our deployment scripts.