Migrate to Google Cloud DNS Using OctoDNS: Part 1

5 min readJul 20, 2020

Introduction

In this multi-part series, we will install, configure, test and run OctoDNS to transfer on-prem DNS zones/records to Google Cloud DNS.

Part 1: Installing OctoDNS for Google Cloud (This Post)Part 2: Migrate BIND Zone File to Google Cloud DNS using OctoDNS

For this series, we are assuming that the readers have basic DNS, Linux administration and Google Cloud knowledge.

What is OctoDNS?

In the age where we are managing Infrastructure as code, OctoDNS lets us manage DNS as code. It is an open source tool which is maintained by Github. You can read more about OctoDNS on their Github page.

Why Use OctoDNS?

Google Cloud DNS supports the import of zone files in BIND or YAML format. As of July 2020, the only Google native option is to use the “gcloud dns record-sets import” command.

For a one time import of a small zone file, the gcloud command is straightforward and easy to use, but it has its limits. OctoDNS supports multiple DNS providers including Google Cloud DNS. OctoDNS works with Cloud DNS right out of the box, making it really easy to import, manage and sync multiple DNS zones across multiple DNS providers.

OctoDNS can come in handy for the following use cases:

One Time Bulk/Large Data Imports: Whether you are importing a single zone with thousands of records or multiple zones with tens of thousands of records. OctoDNS can do it using a single config file.
Sync From On-Prem to Cloud DNS: In a hybrid DNS setup, there might be a need to continuously sync records between Cloud DNS and the on-prem BIND server or even between multiple DNS providers. OctoDNS supports migration using both BIND zone file and AXFR.

Good to Know:

As we mentioned earlier, in this series, we will install, configure, test and use OctoDNS to migrate zones/records from on-prem BIND server to Google Cloud DNS. Before we jump into the actual hands on, we would like to highlight a few things:

While installing OctoDNS, we need to provide a special flag. This is to make sure that all the Google Cloud related packages are installed. We will discuss this further in the later sections.
If a zone does not exist in Cloud DNS, OctoDNS will create the zone for you (subjected to the IAM permissions of the service account, more on this later). Please note that OctoDNS will always create a public zone. If you want to use a private zone, then you need to create the zone manually before-hand.

Preparing the Virtual Machine (VM):

We will be installing OctoDNS on a VM running in GCP. For this tutorial, we are using the “Debian GNU/Linux 10” image provided by Google. Step by step instructions for creating a VM can be found here.

OctoDNS assumes permissions either by using the provided credentials file (optional configuration) or by using the VM’s service account. In this section, we will make sure that the service account associated with our VM has the required permissions.

You can set up the access scopes during the VM creation process. If you have an existing VM, then you need to stop the instance to make changes to the associated service account. Please note that the default access does not have the required permissions to work with Cloud DNS. Because of that, we are giving full access to our VM’s service account. If you don’t want to allow full access, create a custom service account with the dns.admin role. You can learn more about creating and enabling service accounts here.

Setting up access scope to full access while creating the VM

Confirming the access scope in the VM’s details section

Once the VM is up and running, we can run the “gcloud config list” command on the VM to see the service account associated with the VM.

paarthm@octodns-test:~/octodns$ gcloud config listOutput:
[core]
account = 273892260752-compute@developer.gserviceaccount.com
disable_usage_reporting = True
project = paarthm-****-playground
Your active configuration is: [default]

Installing OctoDNS:

Before we begin, please make sure that python and the virtualenv package is installed on your system. Click the link for step by step instructions to install python and virtualenv.

paarthm@octodns-test:~$ python — versionOutput:
Python 3.5.3paarthm@octodns-test:~$ virtualenv — versionOutput:
virtualenv 20.0.20 from /home/paarthm/.local/lib/python2.7/site-packages/virtualenv/__init__.pyc

Once we make sure that the required packages are installed, we can proceed with the actual installation.

Create a directory to contain the virtual environment and all the OctoDNS related files. We are being creative and naming it “octodns”.

paarthm@octodns-test~$ pwd
/home/paarthmpaarthm@octodns-test:~$ mkdir octodns

2. Let’s do a list to make sure that the new directory is empty.

paarthm@octodns-test:~$ ls octodns/ -lh
total 0

3. Move to the directory that we created in step 1 and use the “virtualenv” command to create a virtual environment.

paarthm@octodns-test:~$ cd octodns/

We are creating a python3 based virtual environment, hence the -p flag. We are naming the environment “env”.

paarthm@octodns-test:~/octodns$ virtualenv -p /usr/bin/python3 envOutput:
created virtual environment CPython3.5.3.final.0–64 in 1273ms
creator CPython3Posix(dest=/home/paarthm/octodns/env, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/paarthm/.local/share/virtualenv/seed-app-data/v1.0.1)activators PythonActivator,FishActivator,XonshActivator,CShellActivator,PowerShellActivator,BashActivator

Now if we run the “ls” command we will see a directory named “env”.

paarthm@octodns-test:~/octodns$ ls -lh
total 4.0K
drwxr-xr-x 5 paarthm paarthm 4.0K May 17 00:02 env

4. Activate the virtual environment that we created in the previous step.

paarthm@octodns-test:~/octodns$ source env/bin/activate

Once activated, the terminal prompt will change from

paarthm@octodns-test:~/octodns$to(env) paarthm@octodns-test:~/octodns$

5. Let’s install Octodns now. Make sure to pass the google-cloud-dns flag as it installs Google specific packages. We are writing the output to “output.txt” file.

(env) paarthm@octodns-test:~/octodns$ pip install octodns google-cloud-dns >> output.txt

Tail output.txt to see the Google specific packages

(env) paarthm@octodns-test:~/octodns$ tail output.txt | grep “Successfully installed”

Output will look something like the one below. The Google specific packages are in bold font.

Successfully installed PyYaml-5.3.1 attrs-19.3.0 cachetools-4.1.0 certifi-2020.4.5.1 chardet-3.0.4 coverage-5.1 dnspython-1.16.0 google-api-core-1.17.0 google-auth-1.14.3 google-cloud-core-1.3.0 google-cloud-dns-0.32.0 googleapis-common-protos-1.51.0 idna-2.9 importlib-metadata-1.6.0 ipaddress-1.0.23 more-itertools-8.3.0 natsort-7.0.1 octodns-0.9.10 packaging-20.3 pathlib2–2.3.5 pluggy-0.13.1 pprintpp-0.4.0 protobuf-3.12.0 py-1.8.1 pyasn1–0.4.8 pyasn1-modules-0.2.8 pycountry-19.8.18 pycountry-convert-0.7.2 pyparsing-2.4.7 pytest-5.4.2 pytest-cov-2.8.1 pytest-mock-3.1.0 python-dateutil-2.8.1 pytz-2020.1 repoze.lru-0.7 requests-2.23.0 rsa-4.0 six-1.14.0 urllib3–1.25.9 wcwidth-0.1.9 zipp-1.2.0

6. Check if OctoDNS is installed

(env) paarthm@octodns-test:~/octodns$ octodns-sync --versionOutput:
octoDNS 0.9.10

We have now successfully installed OctoDNS with all the Google specific packages. In the next part, we will configure and test OctoDNS.