291 Followers
·
Follow

Building a progress indicator for the Comm utility

Today I found myself needing to use the comm utility to compare two 20GB files. This ended up taking about 10 minutes and while it was running I got curious at how much time was left. Knowing that comm must read through both files before it finishes, I decided to see if I could build a simple progress indicator based on the read offset of one of it’s file descriptors.

I jumped straight into the /proc filesystem and discovered the fdinfo directory. This directory contains a file for each of the process’ file descriptors indicating the offset, file mode and mount ID. …


One parser to rule them all

Bash is a wonderful and terrible language. It can provide extremely elegant solutions to common text processing and system management tasks, but it can also drag you into the depths of convoluted workarounds to accomplish menial jobs.

Recently I ran into one of these situations when trying to parse varying inputs in a control script. …


Use Git hooks to create your own push deployment pipeline

Image for post
Image for post

I recently got interested in supporting Heroku style push deployments in the AWS stack at Fullscreen, Inc. There are a few solutions for this in the wild like Dokku, but I wanted to use something flexible enough to grow with our deployment infrastructure that wasn’t dependent on container technology. The solution I came up with involves a deploy server, a Git hook and a bit of Bash.

Before I could have engineers start pushing code, I needed to setup a deployment server. The most basic setup involved a new instance with a “git” user for authenticated pushes over ssh. …

About

Drew Stokes

I turn beer into software. Be good humans.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store