Preparing your Mac OS dev environment for use with the Syntropy Stack

Taking the time to properly set up your local dev environment pays dividends down the road. It maximizes efficiency and reduces the number of repetitive tasks you need to perform . I’ve now built several projects using the Syntropy Stack, most of which share the same initial setup, so I’ll go through the steps for getting yourself squared away. Here’s a list of what we’ll cover:

• Creating an agent token through your Syntropy Stack account
• Installing Python ≥ 3.6
• Installing Syntropy CLI & Syntropy NAC
• Authentication
• Installing Ansible and the Syntropy Ansible Collection
• Creating Virtual Machines
• SSH Keys

Create an Agent Token

Your Agent Token is the API key you’ll use with your Syntropy Agents so that they can authenticate and join the Syntropy Ecosystem. You can generate an Agent Token using the Syntropy UI. If you’d don’t already have a Syntropy Stack account, you can request access via their website. Once you’re able to login, click the dropdown next to your username and select Agent tokens.

Creating your Agent Token

Click [+] New Agent Token to create your token. I like to create one token per project. This way, if your credentials are somehow leaked, you can destroy that project token without it affecting any other networks that might be online. Remember to change the Expiration date as it defaults to the current day, meaning that if you leave it as is, the token will be invalid the following day.

Installing Python ≥ 3.6

The dependencies for the Syntropy CLI require a minimum Python version of 3.6. Mac OS comes with Python 2.7 out of the box, so we may need to install Python3.

Before doing anything, check that you have python3 installed:

$ which python3
/usr/local/bin/python3
$ python3 --version
Python 3.8.5

If it’s not installed, you’ll see this:

$ python3 --version
-bash: python3: command not found

We’ll be installing Python3 using Homebrew (a package manager for Mac OS). If you find yourself experiencing problems, you may need to install Xcode utilities, you can do so following this guide. If you don’t have Homebrew installed, you can install it by copying and pasting the following command into your terminal:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Okay, now let’s install Python3.

brew install Python

Now try:

$ which python3
usr/local/bin/python3

To install packages for python3, we’ll use pip3 . You can check the version of pip3 if you want to see it.

$pip3 --version
 pip 19.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

Installing the Syntropy CLI

We’ll mostly be using two separate utilities that form part of the CLI, syntropyctl (Computational Topology Library) and syntropynac (Network As Code) utilities. There’s more information in the Syntropy Stack documentation should you need it. We can install them using:

pip3 install syntropycli && pip3 install syntropynac

Before you can generate an API token, you’ll need to add the SYNTROPY_API_SERVER to your ENV. You can copy+paste the command into your terminal and hit return , but I recommend adding it to your ~/.bashrc or ~/.zshrc (depending on which shell you use). Remember to source the file after adding it so that the ENV variable is available in your current session.

export SYNTROPY_API_SERVER=https://controller-prod-server.syntropystack.com

Authentication

We need to generate an API token that allows us to authenticate our CLI commands. Doesn’t confuse the API Token (generated using the CLI) with the Agent Token (An API key generated using the Syntropy UI).

Generate an API Token by logging in using the CLI:

syntropyctl login <your_syntropystack_username> <your_syntropystack_password>

This command will output an alphanumeric string (ie. your API token). Copy the API token this command outputs and add it to your system’s ENV variables. I’ve placed mine in my .zshrc.(because I use zshell over bash, but you can also place it in your .bashrc). Additionally, you’ll need to add your Syntropy Stack username in password alongside the SYNTROPY_API_SERVERto your system’s environment variables.

Edit your .zshrc or .bashrc . Eg. vim ~/.zshrc and add the following to it:

export SYNTROPY_API_TOKEN=”your_syntropy_api_token”
export SYNTROPY_PASSWORD=”your_syntropy_password”
export SYNTROPY_USERNAME=”your_syntropy_username”

Remember to source the file from your current terminal window so that the variables are available in your ENV.

source ~/.bashrc or source ~/.zshrc .

You can check your env variables by simply typing env in your terminal window and hitting return .

Installing Ansible and the Syntropy Ansible Galaxy Collection

Ansible is the simplest way to automate apps and IT infrastructure. Application Deployment + Configuration Management + Continuous Delivery.

We’ll use Ansible to configure our host VMs, install necessary dependencies, deploy our agents and services, and even create our networks. Installing Ansible is simple and can be done using Python’s pip3 package manager. Ansible has great documentation, so definitely check it out if you’re curious about what it’s capable of, or if you aren’t sure about what something’s doing. We’ll be using the Syntropy Ansible Galaxy Collection. Ansible Galaxy houses content created by the Ansible community. Collections contain useful playbooks, roles and modules for us to include in our own playbooks.

First, let’s install Ansible.

pip3 install ansible

Check that the installation completed successfully.

$ ansible --version
ansible 2.10.4
  configured module search path = ['/Users/craig/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.9.1 (default, Nov  1 2019, 02:16:32) [Clang 11.0.0 (clang-1100.0.33.8)]

The next thing we want to do is create a global config file. To make the log output from your Ansible CLI easier to read, create an Ansible config file and place it in your home directory.

$ vim ~/.ansible.cfg

And add the following to it (make sure the ansible_python_interpreter is set to the result of your which python3 command output):

[defaults]
stdout_callback=yaml
# use stdout_callback when running adhoc commands too
bin_ansible_callbacks = True
interpreter_python = auto_silent
remote_tmp = /tmp/ansible-$USER
ansible_python_interpreter = usr/local/bin/python3

Without the config, your log output looks like this:

TASK [create_docker_network : Create Docker network] *****************************************************************
ok: [broker] => {"ansible_facts": {"docker_network": {"Attachable": false, "ConfigFrom": {"Network": ""}, "ConfigOnly": false, "Containers": {"1893dcb898d5299c5dd9cf5a2219f2e2f08956507afc56d86bfe64cb96e18837": {"EndpointID": "7b9ec7d78666c965795c34db03465562e4bb7f5bdfa21de71d5f69824c3b48a7", "IPv4Address": "172.20.0.2/24", "IPv6Address": "", "MacAddress": "02:42:ac:14:00:02", "Name": "mosquitto"}}, "Created": "2021-01-11T17:10:29.613448381Z", "Driver": "bridge", "EnableIPv6": false, "IPAM": {"Config": [{"Subnet": "172.20.0.0/24"}], "Driver": "default", "Options": null}, "Id": "9e6daec0c1bb385fc2d6459655602ab5ce127505e7e6eaea091c9e5af7b5a1f0", "Ingress": false, "Internal": false, "Labels": {}, "Name": "syntropynet", "Options": {}, "Scope": "local"}}, "changed": false, "network": {"Attachable": false, "ConfigFrom": {"Network": ""}, "ConfigOnly": false, "Containers": {"1893dcb898d5299c5dd9cf5a2219f2e2f08956507afc56d86bfe64cb96e18837": {"EndpointID": "7b9ec7d78666c965795c34db03465562e4bb7f5bdfa21de71d5f69824c3b48a7", "IPv4Address": "172.20.0.2/24", "IPv6Address": "", "MacAddress": "02:42:ac:14:00:02", "Name": "mosquitto"}}, "Created": "2021-01-11T17:10:29.613448381Z", "Driver": "bridge", "EnableIPv6": false, "IPAM": {"Config": [{"Subnet": "172.20.0.0/24"}], "Driver": "default", "Options": null}, "Id": "9e6daec0c1bb385fc2d6459655602ab5ce127505e7e6eaea091c9e5af7b5a1f0", "Ingress": false, "Internal": false, "Labels": {}, "Name": "syntropynet", "Options": {}, "Scope": "local"}}

After adding your config, it’ll look like this:

TASK [create_docker_network : Create Docker network] *****************************************************************
ok: [broker] => changed=false
  ansible_facts:
    docker_network:
      Attachable: false
      ConfigFrom:
        Network: ''
      ConfigOnly: false
      Containers:
        1893dcb898d5299c5dd9cf5a2219f2e2f08956507afc56d86bfe64cb96e18837:
          EndpointID: 7b9ec7d78666c965795c34db03465562e4bb7f5bdfa21de71d5f69824c3b48a7
          IPv4Address: 172.20.0.2/24
          IPv6Address: ''
          MacAddress: 02:42:ac:14:00:02
          Name: mosquitto
      Created: '2021-01-11T17:10:29.613448381Z'
      Driver: bridge
      EnableIPv6: false
      IPAM:
        Config:
        - Subnet: 172.20.0.0/24
        Driver: default
        Options: null
      Id: 9e6daec0c1bb385fc2d6459655602ab5ce127505e7e6eaea091c9e5af7b5a1f0
      Ingress: false
      Internal: false
      Labels: {}
      Name: syntropynet
      Options: {}
      Scope: local
  network:
    Attachable: false
    ConfigFrom:
      Network: ''
    ConfigOnly: false
    Containers:
      1893dcb898d5299c5dd9cf5a2219f2e2f08956507afc56d86bfe64cb96e18837:
        EndpointID: 7b9ec7d78666c965795c34db03465562e4bb7f5bdfa21de71d5f69824c3b48a7
        IPv4Address: 172.20.0.2/24
        IPv6Address: ''
        MacAddress: 02:42:ac:14:00:02
        Name: mosquitto
    Created: '2021-01-11T17:10:29.613448381Z'
    Driver: bridge
    EnableIPv6: false
    IPAM:
      Config:
      - Subnet: 172.20.0.0/24
      Driver: default
      Options: null
    Id: 9e6daec0c1bb385fc2d6459655602ab5ce127505e7e6eaea091c9e5af7b5a1f0
    Ingress: false
    Internal: false
    Labels: {}
    Name: syntropynet
    Options: {}
    Scope: local

That induces much less anxiety! By placing the .ansible.cfg file in your home directory ( ~/ ), you’re making it global. You can place an additional .ansible.cfg in your project root and it will override any settings that overlap with the global config.

Next, let’s install the galaxy collection:

ansible-galaxy collection install git@github.com:SyntropyNet/syntropy-ansible-collection.git

Navigate to your local ansible directory, for example on Mac OS:

cd /Users/{user}/.ansible/collections/ansible_collections/syntropynet/syntropy

Install the Python dependencies.

pip3 install -U -r requirements.txt

Creating Virtual Machines

Each VM you run should be on a separate cloud provider’s infrastructure. You can use any cloud provider currently supported by the Syntropy Stack. Check their documentation for a complete list. Using different providers isn’t an absolute requirement, but we do this to help illustrate the flexibility, interoperability, and ease of use when working with the Syntropy Stack across the expanse of the internet. The most important thing is that your VMs don’t share a public IP address. So, if you don’t want to use three separate cloud providers, try placing your VMs in different geographic regions. The most important thing is that they don’t share a public IP address. I have 3 basic VMs from Digital Ocean, GCP, and AWS. I used these providers as I already have accounts with them and am familiar with their services. that I chose to set up my VMs as follows:

FluentD: Digital Ocean, Type: Basic, 1vCPU, 1GB memory, OS: Ubuntu 20.04

Elasticsearch: Google Cloud Platform, Type: e2-micro, 2vCPU, 1GB memory, OS: Ubuntu 20.04

Kibana: AWS, Type: t2-micro, 1vCPU, 1GB memory, OS: Ubuntu 18.04

All three of these options are in the same price range (around $5–6/month) and are sufficient for the purposes of this example.

Setting up each VM on a particular cloud provider is beyond the scope of this post, but I’ve included some links to guides below to help get you started if you’re not sure how to go about creating the VMs. You’ll want to add your SSH public key so you can access your VMs from the terminal using SSH. I’d recommend installing Ubuntu as your OS to ensure the commands I share translate directly. But of course, if you’re comfortable with other flavours of Linux, by all means, have at it. The fact that the Syntropy Stack leverages docker means that we’re distro agnostic!

Creating an AWS EC2 instance

Quickstart using a Linux VM | Compute Engine Documentation

How to Create a Droplet from the DigitalOcean Control Panel | DigitalOcean Product Documentation

Quickstart - Create a Linux VM in the Azure portal - Azure Virtual Machines

SSH Keys

As a personal preference, I prefer to create separate SSH key-pairs for different projects or tasks, depending on the scope. For example, I have a dedicated syntropy_rsa keypair that I use with all of my Syntropy VMs.

If you want to create a new key-pair, you can use the following command. The -N flag just creates a key-pair without a passphrase, which is useful for certain automated tasks (like GitHub workflows, for example). The "user@host" is just a way of identifying the keypair. Feel free to use something like syntropy@local .

$ ssh-keygen -t rsa -b 4096 -C "user@host" -q -N ""
Enter file in which to save the key (/Users/craig/.ssh/id_rsa): /Users/<user>/.ssh/syntropy_rsa

Next you’ll want to authorize your new key on your VMs. Add your new public key to each of your hosts authorized_keys file. You can do this from your local dev machine’s command line. The <host> will be your VM’s public IP and the <user> will be the Linux username for that VM. For example, on AWS the EC2’s default user is always ubuntu .

Eg.

ssh-copy-id -i ~/.ssh/syntropy_rsa.pub <user>@<host> -f

You can also do this manually if you don’t have ssh-copy-id available. Just SSH into your VM and add the public key to the ~/.ssh/authorized_keys file. You can copy your public key to the clipboard using:

pbcopy < ~/.ssh/syntropy_rsa.pub

Just bear in mind that, depending on your cloud provider, you’ll probably have to add your SSH public key via their UI/console before you’ll have SSH access. For example, on Digital Ocean, you need to add your public key via Settings > Security.

Adding an SSH public key to Digital Ocean

The final thing you may need to check is that port 22 is open to inbound traffic. Again, depending on your cloud provider, you may have to manually set this or configure a firewall to allow inbound traffic on port 22. Here’s an example of configuring my network firewall on Digtial Ocean under Networking > Firewalls > Rules.

Setting inbound rules on a Digital Ocean firewall

Bonus

Someone who watched a screen recording of mine asked about my IDE and terminal setup. I use iTerm2 as a replacement for my Mac OS terminal and use zshell as my default shell. I’ve customized iTerm2 using Martin Seeler’s Google Material Design Color theme (linked below).

MartinSeeler/iterm2-material-design

As far as IDEs go, there is a slew of development IDEs out there, but Microsoft VS code is hands down the best (in my opinion, obviously).

Visual Studio Code - Code Editing. Redefined