Getting your head around Emacs Lisp macros

There are two rites of passage for a blogging developer: a macro tutorial and a monad tutorial. The world doesn’t need any more of either, but it also doesn’t need wifi juice machines or Nectar cards so it’s clearly used to compromise.

I find it easiest to understand macros as a templating system, similar to what you’d use to generate dynamic HTML in the ancient past before everything was a JavaScript SPA.

That would look something like this:

<select name="club-members">
{% for member in club %}
<option value="{{ member }}" />
{% endfor %}
</select>

You essentially write something that is a mix of raw HTML and templating code. The whole file is passed to a templating engine (in this example Python’s Jinja2) that will output raw HTML as-is and evaluate the bits escaped with curly braces. The output is pure HTML, which is then passed to a rendering engine (your browser).

Lisp macros are conceptually the same:

  • Raw lisp and templated lisp (macros) written in code file
  • Pre-processor evaluates (“expands”) the macros
  • Lisp interpreter evaluates the final output

To distinguish macro evaluation from regular Lisp evaluation, the term “expansion” is used. There are a few nuances to the evaluation/expansion order here, and you can use templates outside of macros — you might even want to use macros without templates — but for now this is a useful model.

So here’s a simple example. This is the when macro, which is just a sugary version of an if without an ‘else’ clause.

(defmacro when (condition action) 
`(if ,condition ,action nil))
;; e.g. 
(when (has-meat bone)
(message "Baby, you got a stew goin’!"))

In Emacs Lisp macros, the back quote (`) is saying “what follows is a template” and the comma is saying “insert contents here”. Those contents are evaluated as normal. The above would be expanded to:

(if (has-meat bone) 
(message "Baby, you got a stew goin’!")
nil)

That would then evaluate to an excited message about stew.

Here’s a key point: the arguments to the macro were not evaluated before it was expanded. If they were, we’d get this garbage:

(if t nil nil)

and the stew message would get printed regardless of how much meat is on the bone.

Now we could have achieved this functionality with lambdas:

(defun when (condition action) 
(if (condition)
(action)
nil)))
;; e.g.
(when (λ () (has-meat bone)
(λ () (message "Baby…")))

but that is much less readable than the original if expression.
In this case we could have made the condition just a value rather than converting both arguments to lambdas, which would be more readable, but still less than the raw

So what have we achieved? A couple fewer parentheses! And the crowd goes wild!

It gets better, honest. Because don’t have to just pass the macro arguments straight through to the templated code. We can manipulate them first. This is the bit where the beardy unix types get excited: because Lisp code is just a list of symbols, literals and other lists, you can manipulate it easily. The code has the same structure as normal Lisp data. The code is data. Data is code. Code is data. (Paul Graham, I get it now. I am enlightened. Please will you fund my startup?)

Let’s prove it by implementing the cool thread-last macro from Emacs 25,which allows you to pipe data into a flat list of partial functions. This makes for more readable code than if you were to compose the functions by hand.

Here’s an example from my init file. It takes an AST element from org-element-parse-buffer and performs some transformations on it.

(thread-last headline-element 
(org-element-property :raw-value)
(downcase)
(regexp-replace-in-string "\\s-+" "-")
(format "init-module-%s"))
;; e.g. * Proxy Setup -> init-module-proxy-setup

You can see the pipeline of transformations to get from the headline Org element to a formatted string. The initial argument headline-element is passed through in the last position of the four functions in the body.

Written without the macro, it would be:

(format "init-module-%s" 
(regexp-replace-in-string "\\s-+" "-"
(downcase
(org-element-property :raw-value headline))))

which you have to read from the inside out, rather than top to bottom. So thread-last is a nice readability improvement. It was nicked from Clojure, where it is called ->>.

It’s also dead simple to implement:

(defmacro ->> (arg &rest functions) 
"…documentation…"
(if functions
`(->>
,(seq-concatenate 'list (car functions) (list arg))
,@(cdr functions))
arg))

What we’re doing here is recursively calling ->> with the initial argument wrapped in the head of the functions list each time. When the function list is empty, we return the initial argument from the macro. At that point we’ve built the nested structure above and it can be evaluated.

There’s a bit of macro syntax here I haven’t mentioned yet, which is splicing: the expression escaped by ,@ returns a list, and its results are “spliced” directly into the template at that point, rather than as a list. That saves you having to call apply or otherwise unpack the lists.

It’s not the most performance implementation, but we’re shooting for readability here. The point is that there’s no magic here, just normal list manipulation. The quoted code is just another list. Code is data. Om….

(To be fair, the actual implementation of thread-last in Emacs 25 is similar and quite readable, as long as you already know the pcase macro.)

This generates the nested code that you’d have had to write by hand. To prove it you can pass an example call to macroexpand and see what you get.

(macroexpand 
'(->> headline
(org-element-property :raw-value)
(downcase)
(regexp-replace-in-string "\\s-+" "-")
(format "init-module-%s")))
;; expands to the handwritten nested code above:
(format "init-module-%s" (regexp-replace-in-string "\\s-+" "-" (downcase (org-element-property :raw-value headline))))

Note the quote in front of the argument. macroexpand is a regular function and its arguments will be evaluated, so you have to quote the code you pass in.

Whatever the flaws of Lisp, you try building that feature in a language without macros. Even in non-Lisps that do have macros I’d wager you won’t get it in under 10 lines and be anywhere near as readable.

Now for the responsible kung-fu master part: use this power wisely. Macro-heavy code is harder to understand because you have to think about the expansion-time context and the evaluation-time context. It will drive you nuts. And let’s face it, you’re spending your time reading my ravings about Emacs Lisp, so you’re already dangerously low on good sense.

Now please ignore my warnings and go have fun with macros!

Show your support

Clapping shows how much you appreciated Chris Bowdon’s story.