Integrating TeamCity and GitHub with SSH is actually as easy as you would think it is

At RiskLens we have been using TeamCity with Mercurial and Kiln for a while now. This year we have decided to make the move to GitHub for our source control. Very exciting! In our conversion, we ran into some frustrating bumps with getting TeamCity integrated. Our GitHub company organization was setup with private repositories for security reasons. It was decided that we would integrate over SSH, again for security reasons.

Sounds easy enough, right? Unfortunately, this took us far longer to work through the details of than we would like to admit. Too many instructions and blog posts walked through doing this with public repos or with a specific user’s repositories, not repositories for an organization. All the blogs, StackOverflow posts, and forum posts did (technically) contain all the information we needed, but unfortunately, the information we needed was scattered across all of them! After traversing blind alleys, following red-herrings, and copy pasting .ssh into almost every folder we could find (don’t ask) we finally got what should be a days worth of work done after a week of overworking our Google-fu muscles and shouting ‘THERE IS NO WAY THIS IS THIS HARD!’.

Here is all the information you need in one location. We did all the hard work, so you don’t have to.

Suggested Tools

  • Latest version of TeamCity
    There have been reports of some SSH issues with previous versions of TeamCity which should now be resolved in 2017.2. Upgrading our instance to this version solved half our integration issues.
  • Secure password keeper
    The SSH key process will generate both a public and private key for you to save. You’ll also need to have and securely store/remember a passphrase so we recommend utilizing a secure password keeper for this. See 10 Best Password Managers of 2018 for some ideas.

Configure Build Server

Install Git

First things first. You need to be sure Git is installed on your TeamCity build server. Download and install it from here. This will include GitBash which is what is needed for setting up the SSH key on the server.

Set your system PATH environment variable

The PATH variable doesn’t automatically get set correctly so you will want to check it and confirm the following is present or add it if not.

C:\Program Files\Git\cmd;C:\Program Files\Git;C:\Program Files\Git\usr\bin

The default install location is what our example uses for Git. But be sure to have your PATH reflect the install location of Git on your system. The ~\Git\usr\bin is very important as this is the location of the ssh-agent which TeamCity will be looking for when it runs.

Setup GitHub

There are a few different ways to configure TeamCity to securely connect to private repositories on GitHub. We chose to use the SSH Deploy Key option as it fit our needs well and didn’t require us to use login credentials or create a special GitHub user account for TeamCity. See all the available options here:

The only real down side to the Deploy Key method is that you will need to create a new Deploy Key for each repository. So if you have a lot of repositories this could be tiresome.

Create the SSH Deploy Key for your private repository

Using GitBash on any machine (doesn’t need to be the build server), follow these steps to create a new SSH key. Full details can be found here. Learn more about ssh-keygen here.

$ ssh-keygen -t rsa -b 4096 -C "key label"

Quick command breakdown:

  • -t is the key type and rsa is the algorithm used to generate the key. This is preferred and recommended by GitHub.
  • -b is the bits specifying the key size. To an extent the higher the number the greater the security level. RSA defaults to 2048 but 4096 is recommended by GitHub.
  • -C is for comments and is saved in the comment field of the key. This can be any string you want to help identify the key.

Follow the prompts given. It’s recommended you pick a non-default name for the key if you have multiple repositories you will be creating keys for. The last prompt will be for a passphrase. Be sure to securely save this somewhere for later use (as suggested… keep this in a secure password keeper).

This process will generate a private and public key for you to use.

Add Deploy Key to GitHub private repository

You will need to copy the SSH key you just created to the clipboard via the following command within GitBash:

$ clip < ~/.ssh/{key_name}.pub

Replace {key_name} with what you named your key. Be sure to copy the public key.

Now within GitHub, we will add the public key to the list of repository deploy keys.

  1. Go to your private repository within your organization
  2. Click Settings
  3. Click Deploy keys
  4. Click Add deploy key
  5. Give the new deploy key a title (recommend calling it “TeamCity” since it’s for that integration)
  6. Paste the key in from the clipboard
  7. Check Allow write access (this allows TeamCity to tag the repository)
  8. Click Add key

Setup TeamCity

Add the SSH folder to TeamCity build server

With our setup we have the TeamCity server and agents running on the same machine but under different users. Under the user used by the TeamCity Agent, you need to be sure the following folder exists, and if not, create it or copy it in from another user.

C:\Users\{TeamCity Agent Username}\.ssh

Windows will not allow you to create this folder yourself via the GUI. You can use GitBash to create the folder.

$ mkdir .ssh

Setup the known_hosts file on TeamCity build server

Inside the new .ssh folder, you will need to copy in or create a known_hosts file.

Populate the file by running the following command from GitBash. It will collect the necessary public SSH host keys for the host we’ve selected — github.com. We are porting the results out into our known_hosts file which the Agent will use when it connects.

$ ssh-keyscan -t rsa github.com >> /c/users/{TeamCityAgentUser}/.ssh/known_hosts

Quick command breakdown:

  • -t again specifies the key type and that RSA is the algorithm being used with SSH.
  • github.com is the host we are scanning for SSH host keys.
  • The last piece of the command is the location of the known_hosts file for the Agent we are updating with the results of running the command.

Run this command once on the server. It’ll prime the known_hosts file so that the first few runs of the builds from TeamCity will be sure it gets updated as needed with host information.

Add SSH key to TeamCity

Next we’ll add the SSH key to the TeamCity project connected to that repository.

  1. Go to Administration
  2. Click on Projects
  3. Select the project for your repository from the list
  4. Click SSH Keys from the left side of the page
  5. Click Upload SSH Key
  6. Give the key a name that will be unique (suggested: “TeamCity_GitHub_RepoName” where “RepoName” is the name of your repository)
  7. Browse to and select the private key (this may be on your local computer or you may need to move your private key somewhere accessible via this prompt)
  8. Click Save

Setup TeamCity Project

Set TeamCity Project VCS to use GitHub with SSH key

Still in your project in TeamCity, click VCS Roots from the left side of the page. If you already have a VCS Root to GitHub, you can update it to the below settings to work with SSH. If you don’t have a GitHub VCS Root setup, then click Create VCS Root to get started.

  1. Set the Type of VCS to “Git”
  2. VCS Root Name & ID can be whatever you like (suggested is “RepoName_Git”)
  3. Fetch URL will need to be copied from GitHub
  4. Go to your repository’s main page in GitHub
  5. Click Clone or download
  6. Click “Use SSH” which should update the URL to something that starts with git@github.com:
  7. Click the Copy to Clipboard button
  8. Paste this into the Fetch URL back in TeamCity for your project
  9. Default branch can be whichever branch you prefer to work with the most, for us that was refs/heads/develop
  10. Set the Authentication Method to “Uploaded Key”
  11. Leave the Username blank
  12. Select the key you added in the last step from the dropdown for the Uploaded Key

Setup TeamCity Build Configuration

With our build configurations, we wanted TeamCity to run our tests and deploy the code if the tests pass. Following successful deployment we wanted to tag the repository with a marker.

Add VCS to build config with SSH key

Inside your build configuration of your project, click Version Control Settings from the left side of the page. Click Attach VCS Root and select the Git VCS Root from the last step.

Add SSH agent build feature

Still inside the build configuration, click on Build Features on the left side of the page. Click Add Build Feature and select SSH Agent. In the prompt, select the uploaded key for your repository and copy in your saved passphrase for it. Click Save.

You should now be all setup to run your desired build steps! The build may fail the first two or three times as the known_hosts file gets populated. Don’t panic but do monitor the file to confirm it’s being updated with each run attempt.

There ya go folks, feel free to commend with your own experiences, errors and successes. That way, maybe the process will become easier for the next person to set this up.

intersectional feminist, antiracist, servant leader, Future Ada founder/president, qa & devops, social engineer, SpoQuality co-founder - opinions are my own