Clone, build and deploy a React project from Github, with an Ansible Playbook

Ross Bulat
Jul 22 · 14 min read

From Github to Deployment — Automatically

Ansible, an IT automation framework (and my automation framework of choice) opens the doors for automated deployment of applications, whether that be on a single VPS or throughout an entire datacenter.

The collection of features Ansible offers makes it suitable for a wide range of automated tasks, from provisioning, server configuration management, and of course, application deployment. The built-in modules Ansible offers, and overall requirements for Javascript based apps — React, Angular, and Webpack based apps that require a building (or bundling) processes — make Ansible an elegant solution for automating the entire process.

This article will walk through the process of writing an Ansible Playbook for fetching a repository (public or private) from Github, installing dependencies, and building it on your server with yarn build. From here the build will be moved to your live production directory, replacing the previous build in the process. A Playbook is a series of YAML formatted commands that will execute for each of the hosts the Playbook connects to. In our case we will connect to a singular host — a server that will host your production build.

The techniques talked about here will be applied to a Create React App project, but in reality they can be applied to any framework / build process. This process will serve as an introduction to Ansible and its capabilities, as well as how to use it in conjunction with Github. We will cover:

  • How to install Ansible on your local machine
  • How to set up Github SSH keys and switch from HTTPS to SSH repository URLs. Making the switch to SSH and SSH keys will allow Ansible to fetch your private repositories without being prompted with credentials
  • How to configure Ansible to support SSH agent forwarding, which is necessary to make authenticated Github SSH requests
  • How to store sensitive data (your SSH password for your VPS, for example) using an encryption mechanism called Ansible Vault
  • How to write a simple deployment Playbook (written in YAML), and how to use modules such as the git, yarn and command to make the deployment happen.
  • How to run a Playbook, with useful flags

We will firstly install Ansible within a Python virtual environment and set up a VS Code Workspace, before setting up Github SSH access and writing our Ansible playbook to automate deployment.


Installing Ansible

Although the full installation documentation can be found here, I am opting for a Python pip installation of Ansible within a virtual environment. If you have not installed pip already, do so with:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python get-pip.py --user

Note: This article assumes the reader is familiar with Python and pip package manager. Check out PyPI for more information on pip.

The following commands will set up a virtual environment, named github-deployment, and install Ansible within:

# setup virtualenv with python 3.6
virtualenv --system-site-packages -p python3.6 github-deployment
# activate environment
cd github-deployment && source bin/activate
# install ansible
pip install ansible

Although totally optional, you may wish to install Ansible globally too, with the benefits of having global Ansible configurations that will be applied to all your projects. Globally install with:

# install ansible globally - requires sudo priviledges
sudo pip install ansible

We will continue by using the virtual environment installation of Ansible, ensuring we will not encounter permission issues down the line.

VS Code Extension

I recommend using VS Code for Ansible development; the editor is well suited to developing Ansible Playbooks and running them from the command line built into the editor. An Ansible extension is also available from the VS Marketplace that offers syntax highlighting, code snippets, auto completion, YAML validation, and more.

Set up a new Workspace in the editor and add the github-deployment folder. We will be writing the deployment Playbook within this folder, and running it from the terminal. CTRL +` will toggle the terminal within VS Code, and be sure to activate the virtual environment with source bin/activate within this terminal editor too.

Next, let’s set up Github SSH keys.

Github SSH Keys

Our Ansible Playbook will be making Github requests via SSH, and will also utilise SSH agent forwarding so we can use our local keys to make those requests on server. This is particularly useful for larger projects with clusters of servers, perhaps as a load balancing solution, where one SSH key can be used throughout.

Navigate to your SSH and GPG Keys page within your Github Settings. From here we can add and remove keys, apply labels to them, and see when they were last used — Github automatically remove keys that have been inactive for 12 months.

Note: If you would like to read up on how Github uses SSH, check out their documentation on the subject.

Now, at this point you may wish to generate a new SSH Key or use an existing one. Let’s create a new set of keys for the purpose of this talk.

Generating an SSH Key

Github provide instructions for a range of platforms here for generating new SSH keys and adding them to an agent. I will be using a Mac. To generate a new key pair, run the following commands:

ssh-keygen -t rsa -b 4096 -C "<your_github_email_address>"

Press enter to confirm a filename, and create a passphrase for the key for added security. Github also notify us that for MacOS 10.12.2 or later, we must modify ~/.ssh/config to automatically load keys into the ssh-agent and store passphrases in your keychain:

# ~/.ssh/configHost *
AddKeysToAgent yes
UseKeychain yes
IdentityFile ~/.ssh/id_rsa

Now run the SSH agent and add the key to it:

eval "$(ssh-agent -s)"
ssh-add -K ~/.ssh/id_rsa

Adding the SSH Key to Github

Github provide instructions for the major platforms for adding SSH keys to your Github account here.

Copy the SSH key just generated and paste it as a new key in your Github SSH and GPG settings:

# copy to clipboard utility
pbcopy < ~/.ssh/id_rsa.pub
# or open file itself to copy contents
less ~/.ssh/id_rsa.pub

Your key title usually coincides with the machine the key is stored on. For example, I would opt for a name like “Ross’s MacBook”.

Optional: HTTPS to SSH Remote URLs

The last task for our Github setup is to move from HTTPS remote URLs to SSH. This is optional as our Ansible Playbook will clone your Github repositories using SSH initially anyway, but if you have HTTPS URLs at present and you wish to move to SSH, this is useful!

Changing to SSH URLs simply requires us to change the remote URL of your repositories. Github have written a page about it here. Navigate to your repository and run the following commands to make the switch:

# list remotes to get repo name
git remote -v
# change url
git remote set-url origin git@github.com:USERNAME/REPOSITORY.git
# verify change took place
git remote -v

Note: Do not change the git user from the repository URL. Only amend the USERNAME and REPOSITORY section to reflect the name of the repository to connect to.


Configuring Ansible

Time now to turn to Ansible. In order for our Ansible Playbook to work, we need to add a few configurations to the environment:

  • Enable SSH agent forwarding. As explained above, we will utilise one key to establish Github authentication
  • An Inventory file. A major concept in Ansible, an Inventory file is used to configure hosts and groups (of hosts) for the Playbook to connect to. Inventory files can be written as an INI file or YAML file. This talk opts for YAML, as we will be utilising another feature, Ansible Vault, to encrypt your SSH login password to be plugged into the Inventory file via a host variable. We will visit Ansible Vault in the next section

Enabling SSH Agent Forwarding

Agent forwarding with Ansible is simple to achieve — we just need to enable the feature in an Ansible configuration file.

By default, Ansible looks for a configuration file at /etc/ansible/ansible.cfg. This is an INI formatted file that provides a default configurations for the framework. Again, this is practical for global configuration, but not so practical when configuring on a per-project basis.

Note: Beyond SSH agent forwarding, I have not had the need to amend other defaults in my personal Ansible projects (yet!).

What we can also do is include an ansible.cfg file within our project directory — the same directory as the Playbook itself. Within the github-deployment folder, add the file:

# ansible.cfg[defaults]
transport = ssh
[ssh_connection]
ssh_args = -o ForwardAgent=yes

We are ensuring that SSH will be used as the method of connection, and are plugging in configuration to an ssh_args field, not dissimilar to the way we would in the command line. No surprises here.

Inventories File

Now for the required Inventory file that defines our hosts. In the same github-deployment directory, create a inventory.yml file. Within this file we will supply the connection options for one host only — this will be your remote server we will deploy the application on.

Let’s take a look at the file:

# inventory.yml with password placeholderwebservers:
hosts:
server1:
ansible_connection: ssh
ansible_host: "123.123.123.123"
ansible_user: ross
ansible_password: "<password_here>"
  • We are grouping our hosts field within a group named webservers. This is required. hosts is a reserved field name that cannot also be a group name. webservers can be changed to anything you see fit, perhaps production, or static to reflect these are servers that hold your static app builds
  • server1 is the name of a host, that can also be changed depending on your requirements. If you have a domain, myapp.com, perhaps you’d name this myapp_com_1
  • Within server1 we give the SSH connection details needed for Ansible to connect to the remote server, that would mirror the same details required for logging in via Terminal SSH:
ssh ross@123.123.123.123
> password: <password_here>

However, the password is currently in plain text. This is a huge security risk. It makes our Playbook project not viable for source control, and therefore not able to be worked on as a team effort. If your local machine is compromised, your server will also be compromised.

Ansible is aware of this issue, and provide a solution in the form of Ansible Vault. Ansible Vault is an encryption mechanism whereby we provide a passphrase to encrypt passwords, which in turn returns an encrypted string to plug into Playbooks. Let’s briefly explore how to set this up, before coding the Playbook itself.


Ansible Vault

Here we will use Ansible’s Vault (full documentation here) to store the SSH password of your remote host. A Vault requires a passphrase to access, and Ansible primarily direct us to store that passphrase in a plaintext file. In order to use the Vault, we will include the--vault-id flag when running our Playbook, pointing to a particular vault.

Generating an encrypted SSH password with ansible-vault

To set a new encrypted data file, run the following command in your virtual environment to provide the vault passphrase:

# store passphrase as a fileecho "my-ansible-vault-pw" > ~/.ansible-vault-pw

I have deliberately stored the password in a hidden file for security purposes. Replace my-ansible-vault-pw with a random string of your choosing.

Now we can create an encrypted SSH password for our remote host using that passphrase. The following command will do just that:

# create encrypted passwordansible-vault encrypt_string \
--vault-id user@~/.ansible-vault-pw \
'ssh-password-here' \
--name 'remote_host_password'
  • We are using the ansible-vault utility to output an encrypted string representation of our SSH password
  • A vault-id has been provided, in the form of user@<passphrase_file>. If you would rather not store the passphrase in a file, use user@prompt instead. user can also vary depending on who is using the vault. I would use ross as the user, for example
  • The SSH password follows the --vault-id flag, the string that will be encrypted
  • A --name flag is significant; this will be the variable we refer to in our inventory file

After running this command, en encrypted string will be output into the terminal, which will resemble something similar to the following:

 remote_host_password: !vault |
$ANSIBLE_VAULT;1.1;AES256
...

Storing the encrypted password in host_vars

This string needs to be stored as a host variable, which can be done in a host_vars folder in our virtual environment directory:

ansible.cfg
host_vars/
vault.yml

inventory.yml
...

Note: host_vars and group_vars directories can be used to define variables to be used either in the Playbook directory or inventory file directory. In our case, both these files are in the same directory. Group vars by convention are defined to all hosts, whereas host vars are defined for individual hosts.
See the
Using Variables documentation for more information on them.

Within our vault.yml file, copy and paste the encrypted string from your terminal:

# host_vars/vault.ymlremote_host_password: !vault |
$ANSIBLE_VAULT;1.1;AES256
...

Now we can utilise this variable in inventory.yml as the value for ansible_password:

# completed inventory.ymlwebservers:
hosts:
server1:
ansible_connection: ssh
ansible_host: "123.123.123.123"
ansible_user: ross
ansible_password: remote_host_password

With this configuration in place we are ready to write the Playbook itself. Let’s get to it.

Deployment Playbook

The playbook is perhaps the simplest task in this entire process, where we define the tasks to carry out when the Playbook is executed.

To recap, our Playbook, playbook.yml, will run the following tasks:

  • Connect via SSH to the remote hosts we defined in inventory.yml
  • Clone the Github repository we wish to build and deploy
  • Run a build of the project
  • Copy the final build into your live production directory

playbook.yml sits in your virtual environment directory. The file, formatted entirely in YAML, connects to our hosts and carries out specific tasks in the order they are defined.

We firstly define a name, connection details and remote hosts of the playbook, as well as variables we wish to use throughout the Playbook tasks:

---- name: Deploy React App Front-end Build
connection: ssh
gather_facts: false
hosts: all
vars:
repo_folder: /var/www/github-deployment/repos/
live_folder: /var/www/my_app/build
repo_name: USERNAME/REPOSITORY
  • The — name of the Playbook is defined first, and the entirety of what follows is indented under — name
  • Directly below — name are our connection configurations. Our connection method is ssh, and we have used all as our hosts value, specifying that we wish to connect to all the hosts defined in our inventory.yml
  • gather_facts, as the name suggests (documentation here), gathers information on the remote host as the Playbook is being executed. This can be very useful when debugging or analysing how the Playbook executes. For this example we have disabled gather_facts
  • We have also defined variables under vars, defining a folder to clone your repository, the app live directory, and repository name. Be sure to replace the bold values with your own

Note: YAML uses three dashes (---) to separate directives from document content. This also serves to signal the start of a document if no directives are present. These are included in all Playbooks. Comments can be included above them.

Note 2: Be sure to make the user your Playbook is logging in as the owner of your repository and live app directories to prevent permission issues when the Playbook attempts to clone the repository or move the build. If you wish your user to run with sudo privileges, refer to the become option within the Privilege Escalation documentation.

The second part of our Playbook are the tasks themselves. We have utilised three modules (no installation needed) to achieve our automatic deployment: git, yarn and command modules.

This is how they have been used:

# tasks, indented under playbook name  tasks:
- git:
repo: ssh://git@github.com/{{ repo_name }}.git
dest: "{{ repo_folder }}"
update: yes
- name: Install dependencies
yarn:
path: "{{ repo_folder }}"
- name: Build project
command: yarn build
args:
chdir: "{{ repo_folder }}"

- name: Copy build to live directory
command: cp -TRv build {{ live_folder }}
args:
chdir: "{{ repo_folder }}"
# end of Playbook

And this is our Playbook in its entirety. Let’s break down what is happening, task by task.

Git module: clone repository

The — git task clones a git repository to a directory. We provide the repository URL in SSH format, that includes the repo_name variable in double curly braces. dest is the directory we wish to clone to, and update allows the Playbook to update a repository if it already exists.

Note: A string variable as a value must be wrapped in quotation marks. E.g. "{{ variable_name }}"

Yarn module: Install dependencies

Our next task is embedded under another — name of Install Dependencies, that will execute once the repository clones. The yarn module allows us to install dependencies with ease:

# yarn module to install dependenciesyarn:
path: "{{ repo_folder }}"

This is the Ansible Playbook equivalent of running yarn install (or just yarn) in a directory. The only additional value needed here is path, being the directory of the cloned repository.

Command module: build project

The yarn module documentation does not outline a shortcut for the build command, so I have resorted to use the command module, allowing us to carry out vanilla commands, just like we would in a terminal.

The first task is to build the project, and is embedded under another — name:

- name: Build project
command: yarn build
args:
chdir: "{{ repo_folder }}"

Arguments to the command are indented under args, where the chdir argument allows us to change directory before executing it. We will simply run yarn build to produce the final build of our app.

Command module: copy build to HTTP directory

Finally, the command module has been used again to copy the final build folder into the live production folder, where the app is served under HTTP:

- name: Copy build to live directory
command: cp -TRv build {{ live_folder }}
args:
chdir: "{{ repo_folder }}"

We have made sure to replace the current build directory, treating it as a file with the -TRv flags of the cp command. We have also utilised our live_folder variable to make the Playbook more readable.


Running the Playbook

Okay — all that is needed now is to run the Playbook. Do so with the ansible-playbook command within your virtual environment:

ansible-playbook -i inventory.yml --vault-id user@~/.ansible-vault-pw playbook.yml -vv
  • We have provided the inventory file with the -i flag, so Ansible knows which remote hosts to connect to
  • --vault-id allows us to supply the passphrase and vault needed for the encrypted passwords used in the Playbook, that we set up earlier
  • playbook.yml is the Playbook file we are running
  • -vv is the second level of verbosity of the command output, and is my generally preferred verbosity. Play with -vvv and even -vvvv for very granular output. Alternatively, remove the flag altogether to only get task and name feedback as the Playbook executes.
  • The Playbook ends with a summary of the jobs run, where we can see if there were errors during execution:
Playbook results, breaking down which tasks were successful, and if there were changes made on the server as a result of a task

Enjoy following your terminal feedback as your automated deployment executes! There should be no errors provided all previous steps of the article were taken. If there were errors, take note of the Playbook feedback to see what went wrong — Ansible also does a great job at pointing out Playbook syntax errors.


In Summary

This has been an introductory of Ansible, used in conjunction with Github to solve the tedious task — if done manually — of deploying your apps from Github to remote hosts.

This of course only scratches the surface of what Ansible can do. We could have set up the entire remote host from scratch, provisioning the server and installing the necessary software to host your project, before cloning and building it on the server — or gone even further.

Github Webhooks

In regards to Github, we could use Webhooks to notify one of your servers when a repository (or a particular branch of a repository) has new commits pushed to it. Upon receiving the Webhook, an Ansible script could be triggered to automatically deploy that updated project. I have published a dedicated article to do exactly this:

Dynamic Inventory

Ansible also supports a dynamic inventory, whereby hosts are automatically generated depending on your cluster setup. You could spin up or shut down servers as and when your traffic fluctuates. Services like AWS and Digital Ocean have APIs that allow us to do this, opening and closing instances or droplets when needed.

I hope this has served well as a fun yet useful introductory to Ansible, Playbooks, the Vault, and connecting to remote hosts through SSH.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade