Terminal Magic 101

This is the second post in a series about how to tweet cool programs. See the previous post, So you want to tweet a program?, and stay tuned for the next one, coming soon!

A quick example of what we’re learning to do:

Premise: A Unix terminal (emulator)

If you are unsure of what a terminal is, then I recommend this wonderful video. While there are many types of terminals, the terminal VT100 became super popular and then the xterm emulator supported a superset of its commands and most terminals implement its interface. This interface is very standard these days.

To write a serious program, you would need to look up the actual codes for the terminal that the user is using (eg man terminfo), but for these tweetable programs, I simply assume xterm.

Premise: Ruby

These examples will be done using Ruby. Any Ruby will do, even much older ones. You can use any language you like, but Ruby has an ability to express sophisticated programs succinctly, so it’s what I usually use.

How to do magic

Generally, the way to do terminal magic is to print specific character sequences that the terminal will recognize as commands instead of text. These are usually called “escape codes” or “control sequences” or several other similar sounding names.

For example, in the program below, the terminal will recognize \e[31m as a command to turn the foreground colour to red and \e[0m to remove all text format settings (eg the red):

$ ruby -e 'puts "normal \e[31m red \e[0m normal again"'

Anatomy of an escape code

For most of the commands that you will want to do, they will have this structure:

Code: CSI + Arguments+ Command
CSI: \e[ the control sequence introducer
Arguments: decimal numbers delimited by semicolons
Command: a letter

So, for example, above we used \e[31m, \e[ is the CSI, an escape character (ASCII value 27, hex 0x1b, octal 033) followed by an opening bracket, followed by the argument 31, followed by the command m.

The opening bracket seems strange, but I think they were just iterating through printable characters, and the one that we find most relevant just happened to be the opening bracket. If you look at the docs, you’ll see that there are other things you can do with other characters, and \e[ just happens to be the Control Sequence Introducer.

Foreground vs background colours

The foreground (the colour of the text) and background colours use the same numbers for the same colours, the only difference is that the foreground colours start at the offset 30 and the background colours start at 40. So we saw earlier that 31 set the foreground to red, thus 41 would set the background to red:

$ ruby -e '
puts "normal \e[31m red \e[0m normal again"
puts "normal \e[41m red \e[0m normal again"
'

Colour Swatches

There are several ways you can tell the terminal what colour to use. Almost all terminals will support an 8 swatch palette (0–7). Some fancy terminals even allow “bright” variants of these swatches. The exact colours used for the swatches come from the terminal theme.

$ ruby -e '40.upto(47) { |n| puts "#{n}: \e[#{n}m  \e[0m" }'

Multi-argument commands

Say we want to set both the foreground and the background colours. To do this, we can pass 2 arguments with a ; between them. Here, we will use \e[35;44m, the 35 tells it the foreground should be magenta, the ; tells it there’s another argument coming, the 44 tells it the background should be blue, and then the m tells it to interpret these numbers as colours.

$ ruby -e 'puts "\e[35;44m magenta on blue \e[0m"'

Full RGB

If you want an exact colour, there are several other colour spaces you can use. The simplest one allows allows full RGB. Note that it is not supported in all terminals, for example Terminal.app doesn’t support it.

Since there are 8 swatches in the palette, 0–7 are already spoken for, so the next available number is 8, which doesn’t map directly to a colour, but instead tells it we’re going to use a different colour space. Thus the first argument is 38 Then it needs us to tell it which one we want. The RGB colour space is the second one, so we give it a 2. Now it knows we are using RGB, so it’s waiting for the red value, the green value, and the blue value. In this program, I interpolate them just to make it more obvious what those positions are for. If we wanted, we could add another semicolon and provide additional arguments for other things (eg the background colour)

$ ruby -e '
red, green, blue = 255, 100, 0
puts "\e[38;2;#{red};#{green};#{blue}m sorta orangey"
'

$ ruby -e '
3.times do |channel|
  rgb = [0, 0, 0]
  256.times do |i|
    rgb[channel] = i
    print "\e[48;2;#{rgb.join ";"}m \e[0m"
  end
  puts
end'

Screen size

There are many ways to access the screen size, but for tweetable programs, the shortest is to use the shell’s $COLUMNS and $LINES variables:

$ echo width: $COLUMNS, height: $LINES

Setting the cursor location

The command to move the cursor is H, it takes two optional arguments, the row and the column. These are indexed from 1, not 0. If omitted, they default to 1. Here I wait to quote the program until after I’ve interpolated the window sizes into the program string. Because they are not quoted, the shell replaces them with their values before passing the program to Ruby. To see what is actually evaluated by ruby, you can put echo in front of the command, which will print all the arguments instead of evaluating them.

$ ruby -e width,height=$COLUMNS,$LINES'
print "\e[37;46m" # white on green
print "\e[Htop-left"
print "\e[#{height}Hbot-left"
print "\e[1;#{width-8}Htop-right"
print "\e[#{height};#{width-8}Hbot-right"
sleep 1
puts "\e[0m"'

Relative cursor movement

Sometimes it’s more useful to be able to say “move the cursor up 1 line” instead of tracking the absolute coordinates. In these cases, we can use:

• \e[A to go up
• \e[B to go down
• \e[C to go right
• \e[D to go left

These can also take arguments, eg \e[3A means “go up 3 lines”

$ ruby -e width,height=$COLUMNS,$LINES'
print "\e[46m"                     # background green
print "\e[#{height/2};#{width/2}H" # middle of the screen
length = 0
3.times do
  length.times { print " ";         sleep 0.1 }; length += 1 # right
  length.times { print " \e[D\e[A"; sleep 0.1 }; length += 1 # up
  length.times { print " \e[2D";    sleep 0.1 }; length += 1 # left
  length.times { print " \e[D\e[B"; sleep 0.1 }; length += 1 # down
end'

Clearing the screen

We can clear the screen with \e[2J there are three arguments you can give it, but in practice, we pretty much always want 2, which clears the entire screen. Usually we want to move the cursor to the first line, also, so we can do \e[H\e[2J to accomplish both goals. You’ll notice that this is actually how the clear program works, and this is what the terminal database suggests (though it represents escape with \E instead of \e)

$ infocmp -L1g | grep clear_screen
$ clear | ruby -e 'p $stdin.read'

Hiding and showing the cursor

You can hide the cursor with \e[?25l and show it with \e[?25h I know that seems very peculiar, my mental model is that the 25th wire somewhere in the terminal’s hardware is either high (eg on) or low (eg off). The 25th wire just happens to dictate whether it displays the cursor or not, so we set it to l (low) to turn off the cursor. For these, I like to also use an at_exit block to set it back on. This way, even if there’s an exception that kills the program, it will turn the cursor back on (because it’s extremely disorienting to wind up back in the shell with the cursor turned off) If that does happen, though, there’s a program called reset that will fix it.

$ ruby -e '
puts "on"; sleep 1
print "\e[?25l"
at_exit { print "\e[?25h" }
puts "off"; sleep 1'

Other commands

The official xterm docs are somewhat difficult to understand, so here are a few others that are pretty good: 1, 2, 3

My favourites, which I won’t demonstrate as they aren’t very useful for making tweetable programs, are swapping to the alternate buffer (eg this is how console vim can restore your terminal when you background it), mouse reporting (eg this is how we made the terminal photoshop), save and load cursor position, and alternative character sets (eg useful for drawing boxes). You can also draw images in the terminal using sixel or ReGIS graphics. Some consoles, such as iTerm can display images, which I’ve used to create and print actual images:

Other magic

In addition to escape sequences, there are some commands that operate at a much lower level, issuing a syscall called an ioctl (input / output control) on the file descriptor. Using this, you can do things like receive individual keystrokes instead of post-processed lines (in Ruby, see IO#raw).

Putting it all together

Lets use the things we learned to make an interesting animation!

$ ruby -e width,height=$COLUMNS,$LINES'
print "\e[?25l"
at_exit { print "\e[?25h" }
def channel(value, offset: 0)
  ((Math.sin(value/5.0 + offset) * 127) + 128).to_i
end
0.step do |i|
  print "\e[H"
  1.upto height  do |y|
    print "\e[#{y}H"
    1.upto width do |x|
      r = channel i+x
      g = channel i+y
      b = channel i+x+y, offset: Math::PI/2
      print "\e[48;2;#{r};#{g};#{b}m "
    end
  end
  sleep 0.05
end'

But how do I make these programs tweetable?

Stay tuned, we’ll talk about that in a future post!