Deploying on Cycle Using Git Repositories and Stacks

Cycle v2 is live, and with it comes a plethora of new features and capabilities. In this post, I’ll be highlighting our stacks feature, along with some of the other image importing and management techniques available in the latest version.

What is a Stack?

Those of you familiar with Docker have probably heard the term “stack” before. It describes a series of container images as well as meta information such as networks, volumes, and other deployment configurations in a YAML file. Stacks enable deployment information to be committed with the code, keeping the source for the images and their orchestration together.

When developing Cycle, we made a choice to focus on OCI compatibility, and didn’t limit ourselves by running Docker or Kubernetes under the hood. By writing everything from scratch, we’re able to provide features that would be impossible otherwise. As a result we created a more robust stack specification that takes full advantage of Cycle. It is not backward compatible with a docker-compose.yml file, but offers much of the same functionality with tons of perks.

Creating a Cycle Stack

All you need is a cycle.yml file in the root of your git repository. That’s it, you’re done…!

Well, it won’t DO anything, but technically you’ve created your stack. To be useful, we’ll need to describe our deployment. Check out this sample file below:

version: "1.0"
description: "Cycle v2 Stack File"
containers:
  db:
    name: "Database"
    image:
      name: "MongoDB"
      source: 
        docker_hub:
          target: "mongo:3.6"
      preload: false
    volumes:
      - local: 
          max_size: "2G"
        destination: "/data/db"
        read_only: false
        remote_access:
          enable: true
          password:
            raw: "test123"
    config:
      runtime:
        command:
          path: "/entrypoint.sh"
          args: "mongod --auth"
        privileged: false
        capabilities: []
        namespaces: ["network", "ipc", "uts", "mount", "pid"]
      network:
        public: "enable"
        hostname: "db"
        ports: ["27017"]
        dns:
          nameservers: ["8.8.4.4", "8.8.8.8"]
      deployment:
        instances: 1
        tags:
          require: ["db"]
          pool: ["sjc", "nyc"]
      resources:
        cpu:
          limit: "0.5cores"
          reserve: "0.1cores"
        ram:
          limit: "2G"
          reserve: "250M"
      options:
        events:
          start:
            webhooks: []
          stop:
            webhooks: []
          deploy:
            webhooks: []
        monitor:
          auto_restart: true
          max_restarts: 3
          restart_delay_secs: 10
          notify:
            emails: ["alex@alexmattoni.com"]

Whew, that’s a lot to digest. Let’s break it down. Immediately you’ll notice quite a few differences from a Docker compose file.

Containers

Our specification wouldn’t be very useful if you could only describe a single container. For simplicity, we’ll only be working with one for this example, but you’re able to list as many containers as your application needs to function here. A common split would be between front-end, back-end, database, and auxillary services.

Images and their sources

This is where things start to get interesting.

image:
      name: "MongoDB"
      source: 
        docker_hub:
          target: "mongo:3.6"
      preload: false

Since Cycle is able to import and build your images for you, you are not limited to just Dockerfiles or Docker Hub source images. While both are supported, Cycle is also able to import directly from other repos (enabling you to keep separate repos for all your services), and will eventually support exported container images, as well as other, non-docker formats. If you’ve got some Dockerfiles in this repo, go ahead and set up a local image source like so:

image:
      name: "MongoDB"
      source: 
        docker_file:
          path: "path/to/Dockerfile"
      preload: false

Describing Volumes

Volumes provides a place to keep data that persists across container restarts and reimaging. Simply put, it’s the equivalent to an external hard drive. If your cycle.yml file containers volumes, Cycle will automatically build and mount these volumes when your start your container. We’ve also added some additional tools into the platform to make managing them even easier.

volumes:
      - local: 
          max_size: "2G"
        destination: "/data/db"
        read_only: false
        remote_access:
          enable: true
          password:
            raw: "test123"

First, we describe the type of volume. As of writing this, only “local” (i.e. on your server) types are supported. Down the road, we’ll be offering support for SANs and other types of storage devices. Each type has its own settings, but for local there is only max_size , with M for megabytes, G for gigabytes and so on. The volume is thin-pooled on Cycle, meaning that the storage is not allocated ahead of time. You could set a max_size of 10G but if you’re only using 50MB, that will be the size of the volume.

Destination describes where the volume is mounted inside of the container. For this example, my volume will be located at /data/db and any files written there will be stored on the volume. You can set it to read-only if you don’t want the container to modify it (since this is a database, we need to be able to write to it).

Remote access is one of the unique features provided by Cycle. Traditionally, it is a pain to add and remove files in a volume outside the context of the container, but by providing remote access it’s easier than ever. You don’t need to install anything or change your container whatsoever, just set enable: true and Cycle will mount an SFTP server into your container.

When enabling remote access, you’re able to specify a default password using either raw , sha512, or md5. Using either sha512 or md5, you have the ability to input a password without ever telling Cycle what your actual password is. Although md5 should not be used for storing passwords, we know a number of legacy applications rely on MD5 for passwords.

If you’re on a Mac and would like to use sha512 to hide your password, open your terminal and run echo -n "yourpassword" | shasum -a 512 to generate the appropriate hash for your password.

Cycle also supports access control lists (ACLs) where you can authorize explicit IP addresses, and apply a password to it using the method mentioned above. Each authorized IP address can also be set to read-only, which as I’m sure you guessed, means that particular IP won’t be able to write to the disk.

To set up an ACL, add this to your volume’s remote_access section, replacing the fake IP for one you want to grant access to:

ips: 
  - ip: 192.168.1.1
    read_only: false
    default_password: null

Configuration — Managing Deployments and Resources

Up until this point, we’ve been managing meta information about how the container is created, but now it’s time to dive into configuring how the container runs. You’ll notice several sections under config, so let’s run through them.

Runtime

The runtime section describes how your container starts and what it’s capabilities are. Your command/entrypoint settings will be automatically inferred from your image, but you’re able to override them.

runtime:
        command:
          path: "/entrypoint.sh"
          args: "mongod --auth"
        privileged: false
        capabilities: []
        namespaces: ["network", "ipc", "uts", "mount", "pid"]

Since your containers are running on your own infrastructure instead of a shared public cloud, there is a lot more freedom in what capabilities your containers can have. Setting privileged: true will give the container essentially all of the same capabilities as the host machine, so use it wisely. If you require more granular controls, you can specify the exact capabilities or list of namespaces your container needs.

If you’re interested in learning more about capabilities, namespaces, and other ways to harden your container, click here.

Network

Perhaps the area Cycle excels most at is simplifying container networking. It’s been a notoriously difficult problem to tackle, and until recently there weren’t any “decent” solutions. We’re changing that by making network configuration a breeze.

network:
        public: "enable"
        hostname: "db"
        ports: ["27017"]
        dns:
          nameservers: ["8.8.4.4", "8.8.8.8"]

There are three modes of network access, specified by the public key. “disable”, “egress-only”, and “enable”. Disable and enable are pretty self explanatory, but what is “egress-only” ?

Imagine you’re building some data analytics tools, and you need to scrape the web for some data. There’s no reason for people to connect to your tool, but you still want it to be able to get information off the internet. This is where “egress-only” comes in. All inbound connections will be blocked, but your container will be free to make requests to the web, providing an extra level of security by keeping unwanted onlookers out.

Your hostname is important, because any other container within the same environment (more on that later) will automatically be networked together privately and encrypted, using the hostname to resolve each other. In this example, this database could be set to public: disable and another container, such as an API, could still connect by referring to it as db in your code. Effortless!

Of course you also have standard port mapping, same as Docker. But unlike Docker, Cycle also provides managed DNS, enabling you to attach records to containers directly. As a bonus, you can even specify nameservers, so you can even have custom lookups.

Deployments

Under deployment, specifying instances will tell Cycle how many instances you’d like to create on first start.

deployment:
        instances: 1
        tags:
          require: ["db"]
          pool: ["sjc", "nyc"]

Additionally, tags help Cycle determine which servers to deploy containers to, and can be completely customized. Each server you deploy will automatically be assigned its respective geographic tags. The require key means a server MUST have those tags for the container to be deployed, while the pool key means it MAY have those tags. in our example below, the server is required to have the tag “db”, but may be deployed into both servers tagged db, sjc and db, nyc .

More advanced scaling options will be available in the near future. Cycle already creates an HAProxy load balancer container and routes traffic to your instances for you. Soon, through your spec, you will be able to configure your load balancer as well.

Resources

This section is simple, but powerful (kind of our theme).

resources:
        cpu:
          limit: "0.5cores"
          reserve: "0.1cores"
        ram:
          limit: "2G"
          reserve: "250M"

The only part that might be confusing is limit vs. reserve. Limit means no matter what, the container can never use more than this amount. Under CPU, we set 0.5cores, but you could also say 2seconds if you’d rather calculate it by CPU time instead of per core. When working with CPU time, Cycle uses a 10 second base. By specifying 2seconds, the container could utilize 100% of the entire CPU for two of every 10 seconds.

Reserve means that Cycle sets aside that amount of resources for this instance, preventing other instances from using them. Here we’ve set 250M of RAM to be saved for this instance, but it can scale up to 2G if it needs to.

What makes this approach great is the ability to over commit. Theoretically, you could set 100 containers to use 4 processing cores. As long as you aren’t expecting them to constantly max out, you’ll be able to fit more ‘stuff’ onto a single box.

Options

Finally, we have our options section.

options:
        events:
          start:
            webhooks: []
          stop:
            webhooks: []
          deploy:
            webhooks: []
        monitor:
          auto_restart: true
          max_restarts: 3
          restart_delay_secs: 10
          notify:
            emails: ["alex@alexmattoni.com"]

Here you’re able to set up monitoring and event hooks. In this example, we’ve described an auto_restart to happen on a deploy event. When the container is redeployed, Cycle will automatically start it for you. Cycle also supports webhooks, allowing the platform to call out to one of your endpoints when a container is stopped, started or deployed (more info on webhooks coming soon).

Cycle also provides some monitoring options. If you set auto_restart to true and your container shuts down unexpectedly (for example if there is an issue with the code and the container crashes), Cycle will automatically restart it to minimize downtime.

Deploying the Stack To Cycle

Once your cycle.yml file is committed to the root of your git repo, it’s time to pull it onto the platform.

1: Log into the portal:

2: Navigate to the Stacks section and click “Import Stack” in the top right hand corner:

You can also paste in a stack file by selecting “Import from File”

3: Select the “Import from Git Repo” option, and put in your repository information. Then watch the magic happen!

If your repo is private, use the SSH URL and paste in the private key.

Cool, now what?

Alright so Cycle has your stack imported, and your images have been built. Now let’s deploy them to an environment.

A what?

I touched on it a bit earlier, but environments are collections of containers that can communicate with each other, via hostname, over a private encrypted network. Really makes all that complex network configuration stuff look silly.

They pair perfectly with stacks, because the stack is deployed directly to an environment. Your entire application configuration, deployment, and networking is automated by Cycle.

Let’s deploy some containers!

1: Create an environment:

2: Select “Deploy Containers”, select our stack (and the build…you can rebuild the stack whenever you make a change), and click deploy!

3: Hit “Start!”

How about that? Depending on how many containers were in your stack file, and how aggressively you’re scaling, you potentially orchestrated hundreds of containers across dozens of nodes!

Keep in mind, the first start of a container will take longer. Cycle first copies the image to the servers needed. Subsequent starts should be less than 1/10 of a second.

You just deployed on Cycle!

Cycle aims to make this whole process as easy as possible. Without writing any code or touching a command line (minus git), we’ve orchestrated a complex container deployment from our code base. Not only that, but we’ve put it onto bare-metal infrastructure, ensuring top performance of anything we build, while physically isolating our critical infrastructure from anyone else.

A Note About Documentation

While hopefully helpful, this article is not a replacement for our documentation. Our team is hard at work on the docs as you read this. There will be much more detail there once complete, so don’t fret if you need some more info.

If you have any questions, be sure to contact our staff through the live chat in the portal. We’ll be around and more than happy to help. We’ve also got a public slack channel up, so feel free to leave a message in there as well.

Feature Requests

If there’s something you feel is missing from the platform, check out our public roadmap over on Trello. Send an email to support@cycle.io with the subject “Feature Request” to have our staff review it. If it’s a common request or makes sense for us to implement, we’ll put it on the board!

Learn More + Get Started Today

Thanks for checking out our blog! If you like what you read, please subscribe as we’ll have lots to share over time.

To learn more about Cycle, please visit https://cycle.io.

To create a free account, check out our portal!

About Cycle: ​Cycle deploys and manages bare-metal private clouds built for containers, revolutionizing container orchestration by making the process simple, secure, and fast. Learn more at cycle.io. Based in Reno, Nevada, Cycle.io is a subsidiary of Petrichor, Inc.