Building a progress indicator for the Comm utility
Today I found myself needing to use the comm utility to compare two 20GB files. This ended up taking about 10 minutes and while it was running I got curious at how much time was left. Knowing that comm must read through both files before it finishes, I decided to see if I could build a simple progress indicator based on the read offset of one of it’s file descriptors.
I jumped straight into the /proc filesystem and discovered the fdinfo directory. This directory contains a file for each of the process’ file descriptors indicating the offset, file mode and mount ID. …
One parser to rule them all
Bash is a wonderful and terrible language. It can provide extremely elegant solutions to common text processing and system management tasks, but it can also drag you into the depths of convoluted workarounds to accomplish menial jobs.
Recently I ran into one of these situations when trying to parse varying inputs in a control script. …
Use Git hooks to create your own push deployment pipeline
I recently got interested in supporting Heroku style push deployments in the AWS stack at Fullscreen, Inc. There are a few solutions for this in the wild like Dokku, but I wanted to use something flexible enough to grow with our deployment infrastructure that wasn’t dependent on container technology. The solution I came up with involves a deploy server, a Git hook and a bit of Bash.
Configuring the Server
Before I could have engineers start pushing code, I needed to setup a deployment server. The most basic setup involved a new instance with a “git” user for authenticated pushes over ssh. …
About
