Winning hackathons devops style!

Recently I joined up for a hackathon (Spoiler — we won the NetApp challenge) after moving to NYC. We found ourselves in an interesting scenario.

THE PROBLEM — Have you ever lived or worked in a building where the buzzer is just a pain to change so you can forward calls to coworkers/roommates? Imagine this; A friend of yours get’s to the apartment and buzzes your #. However it’s set to forward to your roommate who’s in South Africa visiting his girlfriends parents (in other words NO CELL SERVICE). Now you’re running late to get home and your friend is just stuck outside instead of being able to have the building call you instead.

THE PRODUCT — What we ended up building is the solution to that problem. What if the buzz could forward to multiple numbers, or event better, your friend could enter a pin to gain entry? Essentially we built a PBX style app that integrates with Zang to enable this.

THE CONSTRAINT — For the sake of efficiency we had to use hardware that was pre-setup and couldn’t use anything we wanted. Also, we wanted to have our builds scalable from the start and each deployment would snapshot our entire data environment pre-deployment using NetApp ONTAP (A sponsor of the Hackathon and really nice group of folks) so if any deployment had an issue we could quickly & simply roll back.

Unfortunately, or Fortunately for me, I drank the stateless, docker, rebuildable architecture a la terraform Kool-aid and I hate going back to the crappy world that existed before. You know that world I’m talking about; no real CI/CD, lots of ssh’ing into other systems, scp’ing this to there. Signing up for Heroku and using products like Travis or Circle CI can go a long way to solve the speed issues, however they can’t be self-hosted and there are limits/complications on some integrations the more complex things get. Thankfully we now have tools and strategies that can help us. Many of these tools and strategies resolve the issues when in-between our ideal world and what we’re currently stuck with. Especially in large corporations that have not or cannot take the full plunge due to legacy issues.

Let’s look at what we used: Rancher — Awesome tool for deploying Orchestration (K8s, Cattle, Swarm, etc) in-cloud, on-prem, and really just about anywhere. Drone-ci — A great simple CI tool with a stateless build mindset built on top of docker.

Steps to take.

  1. Turn the NetApp server into a Rancher host
  2. Create a NetApp snapshot plugin for Drone to allow easy roll-back in case of an error
  3. Create the CI/CD pipepline with Snapshots baked in

Turn the NetApp Server into A Rancher host.

Create a new environment in Rancher by going to the host section and getting the custom host commands and running that command on the NetApp server. Looks something like:

sudo docker run — rm — privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.2 http://rancher.somedomain.com/v1/scripts/somekeyvalues

After a few minutes you should see the host show up and you can now have the freedom to deploy any containers to it.

Creating a NetApp snapshot plugin for Drone

Plugins for Drone are just simple Docker containers that get environment variables injected in. This means you could use it in any build system you have. In this case running a simple command like:

docker run --rm                       \
-e PLUGIN_URL=<source> \
-e PLUGIN_USERNAME=<user_name> \
-e PLUGIN_USER_NAME=<user_password> \
-e PLUGIN_STORAGE_VM_KEY=<vm_key> \
-e PLUGIN_VOLUME_KEY=<volume_key> \
-e PLUGIN_NAME_PREFIX=<name_prefix>
-v $(pwd):$(pwd) \
-w $(pwd) \
peloton/drone-netapp-snapshot

will create a snapshot of the data volume. You can see all the code at https://github.com/josmo/drone-netapp-snapshot

Create the CI/CD pipepline with Snapshots baked in

Drone pipelines are out of the scope of this post but adding the following.

pipeline:
build:
...
docker_publish:
...
snapshot:
image: peloton/drone-netapp-snapshot
url: https://netappapiserver:8443/api/2.0/ontap/snapshots
storage_vm_key: 082eb55c-53b0-11e7-b33e-633e98aa9f79:type=vserver,uuid=6141d5cc-53b0-11e7-b33e-633e98aa9f79
volume_key: 082eb55c-53b0-11e7-b33e-633e98aa9f79:type=volume,uuid=2b8a5865-8def-4dd1-a7b1-c2fbe52921f1
name_prefix: mySnapshot
secrets: [ NETAPP_USER_NAME, NETAPP_USER_PASSWORD ]
deploy:
image: peloton/drone-rancher
url: http://rancher.domainname.com
access_key: accesskeyvalue
secrets: [ rancher_secret_key ]
service: hack/web
docker_image: classis/dockerimage:${DRONE_BUILD_NUMBER}

Watch it fly!

In this case you get a pretty rad full snapshot prior to any deployments. This then makes restoring quick and easy. Rancher makes it simple to build all these kinds of practices in a mixed environment. I can’t even imagine living in a world where it’s not around. Drone helps create simple build and delivery pipelines, and NetApp helps with simple snapshots. This all translates beautifully to corporate environments where most things need to be hosted internally.

With the LIFFFT/Pelo.tech @pelotechnology crew, we’re working on the next versions of these concepts which allow for complete self-hosted apps, deployments which are based on high-availability, high-scalability, and simple blue/green style deployments. Hit me up if you’re interested in more info about it!

Joachim Hill-Grannec @lindyblues is a Partner at http://www.pelo.tech, a group that helps organizations improve their dev practices and culture. These days you’ll also find him traveling around the world dancing at West Coast Swing events.

)
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade