From 5 minutes to 5 seconds: Customize your checkout step in CircleCI 2.0

Published in

Equify | {tech:’blog’}

5 min readNov 7, 2018

Continuous Integration was designed to make sure engineering teams could work more smoothly. And, for the most part, that’s true.

At Equify, we use the CircleCI Github integration to keep the main branch clean while each engineer works on new features. If you don’t use it yet, here’s a tutorial on how to add your project to CircleCI.

If you already use it, then there’s a good chance you’re passing through the checkout step in your jobs. A step that can take some time to complete. Especially if your Github repository is dense. Let’s see why.

Check it out 🤙

When you run the checkout step in CircleCI 2.0, it clones your project from GitHub in order to let CircleCI instantiate it, run tests, lint, and much more. The problem is that this step checks the project in his globality (every branches and all the commit history), and this can take a while… 😅

The reason for this is pretty self-explanatory: CircleCI created the checkout step to be easy to use and sufficiently generalist to serve the needs of everyone. But easiness comes at a cost: a very long checkout.

For example, when we would run it on our Equify repository, it would take over 5 minutes to complete. A long time to wait when you’re running through it several times a day.

If you’re familiar with Circle CI, this is what I’m talking about:

version: 2
jobs:
  build:
    docker:
      - image: circleci/<language>:<version TAG>
    steps:
      - checkout # 👈 Here’s what I’m talking about
      - run: <command>

The above code snippet is an example of a .circleci/config.yml. For more examples see here.

Customize the process 🔧

So by default CircleCI checkout step clone all branches and the entire commit history of your GitHub repository. However, this can easily be changed to meet your specific use case. Most of the time we only need the branch we’re working on and the latest commit.

By manually connecting to our GitHub project via SSH and git clone it with a depth of 1 (more info on git clone depth here), we can actually create a shallow clone with a history truncated to the specified number of commits.

With this custom checkout, our checkout process went from lasting 5 minutes to lasting 5 seconds. 🎉

Please note: this method was officially recommended by a CircleCI employee on the official CircleCI forum!

Now here’s how it works:

SSH Configuration 📡

The first step is to create a fresh SSH key. We’ll need it in order to connect to our GitHub repository from our CircleCI build. When asked to enter a passphrase, enter nothing, just press enter.

ssh-keygen -t rsa -b 4096 -C “your_email@example.com”

Next, add this new SSH key to the deploy keys of your GitHub repository. Go to: GitHub > Your repository > Settings > Deploy keys > Add deploy key. Now enter a title (Example: “CircleCI Read & Write Key”) and copy paste your public SSH key in the text area.

PRO TIP: To add your PUBLIC SSH key to your clipoard :

pbcopy < ~/.ssh/sshkeyfile.pub

IMPORTANT: When adding your new deploy key don’t forget to tick “Allow write access”.

Then we’ll need to add your private SSH key on CircleCI. To do so, head over to: CircleCI > Settings > Projects. Then click on the gear icon at the top right of your project.

This will lead you to your project settings. Once you’re there, go to: Permissions > SSH permissions > Add SSH key. Enter the hostname: github.com. Then copy paste your private SSH key.

PRO TIP: Here’s the shortcut to add your private SSH key to your clipboard :

pbcopy < ~/.ssh/sshkeyfile

⚠️ Note that we’re not appending the .pub extension at the end of the file. This will gives us the private key. Copy paste it and click on Add SSH key.

CircleCI configuration ⚙️

Now that CircleCI can connect to your GitHub project via SSH, it’s time to create our custom checkout step in our .circleci/config.yml. In order to make that work, start by replacing the checkout step with the following code:

- add_ssh_keys:
  fingerprints:
    - YOUR_SSH_KEY_FINGERPRINT- run:
  name: Avoid hosts unknown for github
  command: echo -e “Host github.com\n\tStrictHostKeyChecking no\n” > ~/.ssh/config- run:
  name: Clone GitHub repository
  command: git clone -b ${CIRCLE_BRANCH} — single-branch YOUR_REPO_SSH_URL — depth=1

Then, you’ll need to replace YOUR_SSH_KEY_FINGERPRINT with the fingerprint of your SSH key. You’ll find it there: CircleCI > Settings > Projects > Your project > Permissions > SSH permissions.

Finally, replace YOUR_REPO_SSH_URL with the SSH URL of your GitHub repository. Which you can find here: GitHub > Your repository > Clone or download > Use SSH.

Well done 🎉

That’s it, you should be all set. You can now push your new CircleCI configuration to GitHub and compare your new checkout time! Impressive right? 😄

If you like this article don’t forget to clap 👏 ! And if you have any question don’t hesitate to reach out on Twitter (pierrericgarcia).