Living in the command line — tips n tricks
TL;DR — if you like the command line, in the below post I enumerate a few of my favorite CLI apps and commands that have made me productive.
I first became interested in computers in high school. I had the good fortune to go to a small school where everyone was already fairly, well let’s be honest, nerdy. So of course, when I got a chance to take a class of my choice that wasn’t just to round out my college application or GPA, I signed up for the Introduction to Programming class. This class ignited what would eventually become a life-long passion for all things programming, but unfortunately treated the command line as a mystery box where arcane incantations sometimes made our programs run, and sometimes spat out scary errors and warnings.
During my year abroad between high school and college, I began to experiment more with the guts of computers. I installed a Linux Operating System in a virtual machine, so that I could mess around with it with no fears of damaging my normal computer (not that I didn’t have many gripping moments of “OH CRAP! Did I do it wrong?!”). I slowly learned how to navigate file systems, or how to load drivers for my wireless card using various tutorials and blogs.
However, it wasn’t until my freshman year of college that I truly learned to appreciate the power of the command line. One of my professors would bring up programs, distribute homework assignments, and even ‘compile’ and present lectures to us — all from the command line! I knew that the scant bits of knowledge I had gleaned up to that point would not be enough, I needed to dive deeper and to be able to feel ‘at home’ right in that black screen with its blinking green cursor.
Over the years I have been able to put together a pretty useful arsenal of bashisms, and various command line programs. I believe that if you get good enough and comfortable enough with the command line, the only two programs you need to ever install on your computer are a terminal and a web browser. The topic of bash and command line introductions has been covered time and time again (some good examples: http://www.tldp.org/LDP/Bash-Beginners-Guide/html/ or https://ryanstutorials.net/bash-scripting-tutorial/). In this blog post, I want to go over some of the more interesting and useful commands and programs that I have discovered over the years
We pretty much all work (or should) in some form of version control software to keep our codebases in a sane, accountable, and distributable state. But sometimes
git log just is not good enough for browsing through commits, diffs, authors, etc. If you don’t want to jump over to your Github site, just launch tig! It can show you all the merging info, and allow you to jump around and look at specific commits, branches, etc.
Tig is an ncurses-based text-mode interface for git. It functions mainly as a Git repository browser, but can also…jonas.github.io
We all try to write good unit tests around our code, but sometimes behavior is hard to capture, especially if it touches downstream code or depends on production data. Let’s say you discover some buggy behavior, but it’s not clear when it was introduced or even what bit of code causes it. You could do a deep and detailed investigation, maybe launch a debugger, and spend tons of time tracking down the behavior. Or, you could use git bisect to more optimally hunt down the changeset when buggy behavior was introduced. Bisect works by establishing ‘good’ and ‘bad’ points in commit-time, and uses a binary search to discover the faulty commit. The basic commands to know are:
git bisect startthis will start a bisect session, at the current commit
git bisect goodwill mark the current commit as good
git bisect badwill mark the current commit as bad
The basic workflow is to start with a bad commit, find a point in time where you know things are good, and then git will use a binary search to move you around commits, marking them as good or bad until the first bad commit is identified. An example session might look like:
$ git bisect start
$ git bisect bad # Current version is bad
$ git bisect good v2.6.13-rc2 # v2.6.13-rc2 is known to be good
Bisecting: 675 revisions left to test after this (roughly 10 steps)
$ git bisect good
Bisecting: 337 revisions left to test after this (roughly 9 steps)
This example, and much more in depth documentation, can be found at:
Sometimes you are not looking for the commit that introduced a breakage, but rather for a commit that caused a change…git-scm.com
One of the most annoying things that happens to me somewhat regularly, is that I will be halfway through typing some long command, and can’t remember either what some flag or file is called. What are the options at this point? Well, you could hit Enter, and just let the command error out — but sometimes this will just run with the wrong commands! You could hit ctrl-c, but then you have to retype the whole command over again.
Readline to the rescue! Every modern shell is built with GNU readline built in, which gives all sorts of controls over how you input and recall commands. By default, readline is configured to use Emacs commands for editing, which may make them familiar to some readers. The most important ones here, are killing, yanking, and history:
unix-line-discardwill ‘kill’ (aka delete and put into the readline buffer) everything on the line to the left of the cursor
yankwill ‘yank’ (aka paste from the readline buffer) whatever was previously stored
reverse-search-historywill begin searching the command history for something previously run. Pressing Ctrl-r once again will keep looking further back in time
For a quick demo, check out this video
Whenever I am writing Python code, I like to have a REPL open in a nearby terminal, just so that I can sanity check that what I am doing is correct. The built in Python one is relatively good, but it can definitely be improved. Enter bpython — ‘a fancy interface to the Python interpreter for Linux, BSD, OS X and Windows’. Some of the most useful features of bpython include:
- Live syntax highlighting
- “Rewind” functionality, letting you undo a line of code
- Autocomplete with access to bpython history (inluding readline support), and access to all installed packages
- In-line documentation windows for library packages with docstrings
- Auto-indentation (because who wants to worry about that)
bpython is a fancy interface to the Python interpreter for Linux, BSD, OS X and Windows (with some work). bpython is…bpython-interpreter.org
I end up using the awscli pretty often. Unfortunately, it is not the worlds most intuitive tool. Enter saws, which is a ‘supercharged AWS CLI’. Saws reads your shells AWS credentials, and is able to autocomplete based on commands as well as your actual resources! Check it out in action:
When you run a command, sometimes you want to capture the output, to either run through a different command later, or perhaps to try to capture some erroneous output. Standard shell redirection such as
>> works great for that. But what if you want to also be able to watch the output in real-time? For that you can use the tee command — think of it conceptually as a splitter in your shell! Even the shape conjures this,
T, it has one input on the left, and two outputs down and right. Here’s an example:
$ run_some_command | tee my_output.log
Starting your command
round 1 complete....
round 2 complete...
Find + Exec
Sometimes I will find myself needing to execute a command over a large amount of files. Let’s say I had a ton of data files I had generated, but needed to replace every occurrence of ‘apple’ with ‘banana’ in files whose name started with ‘today’. Command line to the rescue! I could run the following command:
$ find . -type f -name "today*" -exec sed -i 's/apple/banana/g' \;
So what exactly is going on here?
-type ftells find that I only want to find files, not directories
-name "today*"tells find to match on files starting with that string, then anything
-execnow this is where the magic happens — find will run this command on any file that it finds
\;this last bit here tells find that i want to run the command on each file individually. If i had used
\+instead, find would have concatenated all the files as a big argument string
Looking for useful one-liners to accomplish all sorts of stuff on the command line? One of the most useful resources I’ve found down the years is the commandlinefu website:
A repository for the most elegant and useful UNIX commands. Great commands can be shared, discussed and voted on to…www.commandlinefu.com
Tmux/Tmuxinator + Mosh
I find that tmux sessions are a very powerful tool to use, but it can be a a pain to have to reconstruct my state after I reboot my computer. Tmuxinator allows you to define ‘projects’, and enumerate the state of windows, panes, what directory they should open in, what commands should automatically be run in each, and more! Check it out:
If you are using tmux sessions on a remote machine, then installing mosh and using it instead of ssh will make your life easier yet. Mosh is a layer on top of ssh that does its absolute best to keep your connection alive, no matter what. Switch wireless networks? Put your laptop to sleep? Experience network disconnects? Mosh has your back. In the vast majority of cases, it will be able to reconnect, so it feels like your remote session is really local! And in the rare case that mosh is not able to reconnect (if you reboot your computer, for example) your tmux session is still alive on the remote destination, so you can mosh back in, reconnect, and stay productive!
Mobile shell that supports roaming and intelligent local echo. Like SSH secure shell, but allows mobility and more…mosh.org
So much more…
You can really do almost everything you need in the command line. Want to read your email? Check out NeoMutt. Text / code editing on the command line works great via Vim or Emacs (or if you like both, maybe try Spacemacs). Want to listen to Pandora? There’s an app for that. You can even have a browser in the command line. If you want to view a talk I gave about the command line, on the command line, check out mdp.
There are almost limitless things you can do on the command line. The only question is, what will you try next?