Embrace the Bullshittery or: How I Learned to Stop Worrying and Love the Fu

A few weeks ago I read, with great enjoyment, Philip Guo’s loving paen to Command Line Bullshittery. While there were lots of follow up posts and comments that implied that Philip was belittling command line chops, I read it just the opposite. I think that Philip is correctly pointing out that to be a good developer, you gotta learn to love the command line.

Here at POPSUGAR, I’ve had the opportunity to work with a lot of developers who are fresh out of school. Some have way better command line fu than I do, but some have learned to write solid code without any mastery of the command line. When I’m doing a code review or helping debug an environment setup issue, it’s pretty easy to figure out which camp the developer falls into. After enough time in the field, any developer who loves her craft will eventually learn the ins and outs of the command line, but I never miss an opportunity to try and share my command line love and encourage a lifelong pursuit of getting more command line fu.

So if you’re an aspiring developer, here’s a quick tour of my tool belt.

The Basics

Know your environment: Aliases & Environment Variables.

Know how to create aliases for frequently used, long incantations. I use my git branches as a sort of todo list, so when my code is merged, I like to remove the merged branch. It’s a pretty long incantation, so I have a line in my ~/.zshrc that looks like this:

alias branchpurge=”git branch — merged | grep -v \”\*\” | grep -v master | grep -v development | xargs -n 1 git branch -d”

Your $PATH is an important environment variable.

$ echo $PATH

Understand the output of that command and how and why you would change it. I use AWS a lot. I use the ‘aws’ command to get files from S3 and spawn EC2 instances. I’ve set my AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables so I don’t ever have to worry about my AWS credentials again.

With our environment understood, we can start doing stuff, and there are some basics we should know: What’s stdin? What’s stdout? Putting commands together in a pipeline (see the branchpurge alias above) is the unifying principle of the unix command line: putting little discreet commands together to do big powerful things. In addition to using the output of one command as the input for another through a pipe, the command ‘xargs’ allows you to use the output of one command as arguments for another command.

You should be able to edit a file from the command line. Pick vi or emacs and learn the very basics of how to open a file, change something, save it, and exit. That’s good enough.

Know what your machine is doing. I’m guessing that ‘ps’ and ‘kill’ were some of the earliest unix utilities. Is your ssh tunnel still up? Use ps to find out.

Know how to read a man page and how to use ‘apropos’ to find commands that you don’t know about (maybe check out the xargs command I mentiond above with the command “man xargs”). It takes a few tries to start getting value out of reading manpages, but its worth it, and there’s stupid unix lore in em too

File Manipulation

We’re making websites here at POPSUGAR. Our code is made up of source files, our servers spew logs. Files play an important role in what we do. If I’m debugging a problem, I may want to look at the logs to find specific requests or look for clumps of requests at a certain time. Being able to pick out a particular column, find strings that match a particular pattern, or sort on a column are all useful tools that can help when investigating problems. I use ‘awk’ for manipulating lines and columns, ‘ack’ for finding strings in files, ‘find’ for finding files by name or type. There are a number of tools for each of these functions, pick one and know how to use it.

Networking

We’ve got hundreds of hosts out there in the cloud somewhere. We have dev machines, prod machines, qa machines. We’ve got db servers, batch servers, web servers. From time to time, you’ll need to either get on a machine to look at something, or connect to a machine to do something. Our machines are fire-walled and we need to use bastion hosts to get at them. Understanding how to connect to remote machines using tunnels through proxies can make all this possible. How to create tunnels using ssh and how to move files using scp are important tools to know how to use quickly and easily. Building a useful ~/.ssh/config for tunnels and hosts that you use often can make getting around to any host you need nice and easy.

Installing Stuff

Package managers have made installing software much easier than it used to be. I used to use port. Now I use homebrew. Know how to search for packages, install, upgrade, remove. Beyond that, some useful more advanced skills you should have: get a specific version; or patch a package and then use the package manager to install your patched package.

Working

I still use git on the command line. Being quick to branch and commit and being comfortable cherry-picking, reverting, and branching will give you the confidence to try things and throw them out. While git took me a little while to get a good command of, there is no doubt that using it has improved my coding practice through cheap branches, frequent commits and the ease of reverting to known working states.

If you use a build tool like gulp, maven or gradle, understand what the configuration files are doing.

One of my favorite habits is always tailing my logs with ‘tail -f’ so I can see errors, warnings and info that other developers have left to help me. tail -f together with grep can help you filter for stuff you’re looking for.

curl or wget can help you see exactly what a web request and response looks like. The chome inspector has largely made this skill a thing of the past, but still something I use when I want to set specific headers (like Origin) to see what I get back from a server.

Conclusion and Some Things to Try

If you’ve got your new javascript skills from a bootcamp or you’ve spent the last four years in an IDE doing coding homework and you now want to get ready for producing real products that get used, but you don’t have much time spent on the command line, boning up on your skills is good way to make sure that you can hit the ground running. Here’s a couple of things you can try:

1. Install the aws command line tool. Set your credentials so you can see them with the command “echo $AWS_ACCESS_KEY_ID” and upload a file to an s3 bucket.
2. Find a web log file. Count the number of requests that resulted in something other than a 200. What’s the most requested uri that results in a 404?
3. Kill the shell you’re working in (hint: use ps to find all the processes that you’re running).
4. From the command line, figure out what the “x-frame-options” header is for google.com