Why I (still) want to learn Lisp

A couple years ago, I read Hackers and Painters and drank the Paul Graham kool-aid. My background is in English (Creative writing for the win! Oh, wait, now I’m broke. Damn.) and my programming experience up to that point was effectively nil. I took a class on Java in high-school (and hated it), back when Windows 98 was still a thing, and the real geeks were excited because soon we’d have 32-bit computing. But hell, I’m a smart guy (At least, I’ve always thought so…) and Lisp seemed different from Java, so I figured I’d give it a go. It went pretty much like this:

• Google some stuff. Read more articles about how great Lisp is.
• Download Emacs. Open Emacs. Cry, uninstall.
• Google some more. Lisp is so cool! Discover SLIME.
• Download Emacs. Download SLIME. Google. Try what it says. Cry, uninstall.
• Discover Lispbox. Download. Run. WOOO! Finally something works!
• Google. Start working through Practical Common Lisp.
• None of this is applicable to anything I want to do. God, I hate Emacs. Get bored around Chapter 18.
• Uninstall.

But the itch never went away, and that saga eventually replayed itself to one extent or another over a not-insignificant (for an English major) list of other languages:

• JavaScript (Poorly documented, and holy crap, so much fragmentation. Every four days or so there’s a new thing.js, and I don’t understand any of them.)
• Scala (Awkward to write, incomplete macros, tries too hard to be all things to all people. Runs on the JVM.)
• Clojure (Very interesting, but so restrictive! And it runs on the JVM.)
• Racket (Hard to find real-world applications. Enough like Lisp that I find myself looking for things that aren’t there. Given the choice, I’d rather just write Lisp.)
• Ruby (Too sugary for me. Has a reputation for being slow as molasses. Probably all the sugar. Har. Rails was nice, though.)
• Rust (Actually, I really like Rust, but it’s still young and I’m still inexperienced enough that I’m not sure what to do with it yet. Plus, systems programming is intimidating.)
• Plus small peeks at Go, Io, Prolog, Erlang, Haskell, and Elixir. I’ll probably come back to each of them, or maybe others, over the next couple years.

But I came back to Lisp at roughly the same time a year later (a little before Clojure, I think), this time with Graham’s own ANSI Common Lisp in-hand. Reinstalled Emacs. Hated it more. Tried a few alternatives (Notepad++, LispWorks, and AllegroCL, and btw, let me offer a grand fuck-you to LispWorks and AllegroCL, who charge utterly outrageous fees for utterly mediocre software) and went back to Emacs, grumbling the whole way. I hit a brick wall on one of the exercises in Chapter 4, and, unsure how to ask the internet for help without just being given the answer, and feeling that this would be cheating, and eventually that if I couldn’t figure it out on my own I probably just wasn’t cut out for this, I uninstalled yet again.

In the mean time, I have actually done some real programming (aside from my hobbyistic exploration of languages). I wrote some Batch and a small application in C# to allow me to script some of the things I do regularly at work, and these have continued to evolve over time as new errors show up, or I think of better ways to do the things I want to do, or as I just tinker with them. I now have a rudimentary command-line interface to our software, and I keep working on making it better and more efficient. Eventually I’ll have actually contributed something useful to the company, besides my acknowledged ability to move boxes of computer equipment with grace and efficiency. So every time I get bored and start toying around with a new language I come at it with a cumulative perspective.

But I keep coming back to Lisp. Why? I keep trying to explain that, to myself as much as anyone else. My dad has been a software engineer for most of his life, has written like a billion lines of Java and a few million of various other things (JavaScript, numerous markup languages, and probably a few other broadly typical things. Some Ruby, I think.) and every time I come back to Lisp (or one of its children) he looks at me a little sideways and says something along the lines of “wouldn’t it be a better idea to pick up something that has at least a little market power?” He’s a fan of Scala, citing its concurrency model and Twitter’s recent migration to the language. And it’s true; Scala is in demand. Ruby is in demand, and seems easy enough to write, and it has really good library management. I get the underlying concepts of programming well enough that I can confidently be a beginner in effectively any language I want, because at this point the only differences are syntax, and I can just look that up. So why not stick my head into Scala or Ruby and run from there?

The short answer? I don’t like them. Or rather, more problematically, I don’t like them as much as I like Lisp. I love Lisp. And when I sat down to write Scala and Ruby I mostly found myself wanting to do things that those languages made difficult, and which would have been trivially easy in Lisp. I had that experience with everything I tried. Once you get used to prefix notation and learn to stop going bug-eyed at the parentheses, almost everything is easy to do in Lisp, and it’s easy to do in whatever way you want.

For example:

Scala, like Java, tries to make sure that you, the programmer, don’t fuck up when passing different types of variables around. It does this by forcing a type onto a variable when you declare it, either implicitly or by just saying “declare a type or I’ll shit on you,” and by then forcing you to declare what types of variables your functions accept and return. Scala wants you to be safe, and to make sure that your code runs before you try running it. Lisp doesn’t do this. Lisp wants you to write code. Whether it runs is entirely up to you, and as a result you don’t have to declare a type for anything. A thing just is what it is. Why does this matter? Safety is good, right? Here are two different functions for generating the nth element of a Fibonacci sequence (not because the Fibonacci sequence is in and of itself special, but because it’s a fairly common beginner exercise and because short solutions are available on Rosetta Code).

Scala:
def fib(i:Int):Int = i match{
    case 0 => 0
    case 1 => 1
    case _ => fib(i-1) + fib(i-2)
}
Common Lisp:
(defun fib (i)
  (if (< i 2)
      i
     (+ (fib (- i 2)) (fib (- i 1)))))

Here’s my point. In order to define a function in Lisp, I write “(defun function-name (parameter-name),” followed by the body of my function: “(if blah blah persnickety blah))”. In order to define the same function in Scala, I have to write “def FunctionName(ParameterName:Parameter-Type) : ReturnType = ParameterName,” and then the body, “match {herp derp flerpy derp}”. I’ve made the names of things longer to emphasize the difference, but there’s a lot more typing involved when defining things in Scala, and as a result Lisp code is, in general, quite a bit easier and smoother to write. Moreover, because types are dynamic and almost entirely implicit, Lisp functions will often accept a much wider range of types as arguments than their equivalent functions in more statically typed languages, which means you can often make them do really cool and interesting things.

Or:

Rust, like fellow systems-programming languages C and C++, and like many other languages in popular use today (JAVA and C#, in particular) has a compiler, and in order to find out if your program works, you send it to the compiler and see what happens. Rust has the advantage of a very well-designed compiler, and when it gives you an error it works really hard to tell you where and why that error occurred in your code. But C and C++ are notoriously bad at this, and C# is nearly as useless, so when you send anything of any size at them, if it hits a snag, there’s a long and tedious process of going through your code line by line to try and figure out what the hell just happened. Lisp, by contrast, has what’s called an interpreter (or REPL, pronounced however you like), which you run as an active window next to your code editor, and you can throw anything you want at the interpreter, from a single statement:

> (+ 4 6)
10
>

to an entire function:

> (defun fib (i)
    (if (< i 2)
        i
       (+ (fib (- i 2)) (fib (- i 1)))))
FIB
>

which you can then call and pass arguments:

> (fib 9)
34
>

Just as if it had been built into the language in the first place. That is a powerful thing to have; it means that testing a piece of your code happens almost as fast as you can think about it. Not sure about a big data-mapping function? Try it and see what happens. Didn’t work? Fix it (or do it another way) and try again. It’s that fast, and it enables a software development paradigm that is utterly different from programming in other languages. Programming Lisp and its derivatives (Clojure, Racket, Scheme) is exploratory in a way that no other language really pulls off, and that makes it incredibly rewarding for tasks that are creative or whose goals and means are otherwise vaguely defined.

And then there are macros. Actually, I won’t wax poetic about macros because I still don’t entirely understand how they work, but based on what I do understand, it seems like no other language (again excepting other Lisps, like Clojure, Racket, and Scheme) offer anything remotely close to the power and flexibility that Lisp macros give you.

Ultimately, my experience with Lisp, as compared to my experience with other languages, can be summed up as follows: In anything else, you have to contort your brain to fit the language. In Lisp, the language contorts itself to fit your brain. It is that different from the rest of the playing field. And as I said, I love it.

But. And it’s a big but. There are problems. Nothing is perfect, and Lisp’s particular foibles have now broken the game two times running for me.

First among equals: Emacs. Oh yes. I went there.

For those who haven’t had the displeasure, Emacs, and it’s near-twin XEmacs, are extensible text-editors whose flexibility, customization, and sheer power boggle the mind. I’ve seen it written, and I believe it, and an appropriately configured Emacs can be an operating system. They have their own web-browsers, game software, calculators, databases, you name it, and somebody’s probably written an add-on to do it in Emacs. They also suck to use. The GUI hasn’t seen an update since 1990 or so, the keybindings are totally unique to Emacs and will confuse you for at least the first five years of your Emacs experience, and most importantly, getting anything to work properly requires you to first learn how to write ELisp.

Wait, what? Yes, the Emacsen are (not coincidentally) written in Lisp, but because they were written by people who are so astronomically intelligent that they can’t even imagine how you or I think, they have built their own dialect of the language just for customizing this stupid, stupid piece of software. What this means, for those of you who are keeping count, is that if you want to learn Lisp, and you want to do it in Emacs (for which there currently is no really viable alternative; I reiterate my grand fuck-you to Allegro and LispWorks) you must first learn two (Two!) other whole languages before you can even get started doing what you originally set out to do. First you have to learn the incredibly powerful but also heinously complicated language of hotkeys that the Emacsen have pretty much all to themselves, and then when your Emacs still isn’t doing what you wanted it to do, you have to go and learn a different dialect of the language you started this whole process for in the first place.

That, right there, is the very definition of bullshit, and since nobody else seems to have done so, I hereby call out the Lisp and Emacs communities on it. Yes, Emacs is the one of the best tools for writing code that has ever been built, but in order to use it you have to first spend a decade sitting on a mountaintop and meditating, isolated from everything you love or care about, until you attain enlightenment. I don’t want to sit on a mountain for ten years, I just want to learn Lisp. Please, for the love of god, build something friendlier and let the rest of the world in. We’re not all bad, I promise.

Next: Individualism and Independence.

These are good qualities; they power much of western society and they’re part of what make most first-world countries the cultural and economic powerhouses that they are. The Lisp community is full of them too, but to such an extreme that it almost feels fragmented. Do you know how many active versions of Ruby there are? One. It’s called Ruby. Here’s a list of Lisps off the top of my head, excluding the dialects like Clojure and Scheme:

Armed Bear Common Lisp, CLisp, Corman Common Lisp, Clozure Common Lisp, Embeddable Common Lisp, Steel-Bank Common Lisp, CMU Common Lisp.

If I actually went and looked, the list would probably be close to a hundred strong. And they’re all at the same time compliant with the established standard for Common Lisp, and also all different! Right now, the generally accepted standard is Steel Bank, or SBCL, for its stability and the speed it grants to the executables it compiles. But ten years (and more) ago, when all the Common Lisp books seem to have been written, the standard was CLisp, and in the mid-level to advanced topics, the sample programs start to break down. It’s hard to learn geometry with a broken protractor. Not impossible, no, but harder than it needs to be. And if you change platforms, as I find I have to do regularly, bits of the same Lisp start to be different, because they’re stable at different versions on different platforms. When I first tried to learn Lisp, SBCL ran on Linux and Mac, but wasn’t available yet for Windows. The CLisp mirror was broken, or the site had just moved or something, and Clozure Common Lisp (which I really like for its informative error messages) ran on Mac and Windows (one of the few that runs on Windows), but not Linux. Sample problems I’d completed in one had to be rewritten to work in another. Who wouldn’t get tired of doing that?

Variety and flexibility, individuality and independence are all great, and I don’t begrudge any community its ability to sprout new seeds and innovate. But Lisp could use a little less innovation and a little more standardizing. Not a lot. Just enough to lower the bar of entry, and to trim down the bewildering array of choices that confront any newcomer to the scene.

Ultimately, none of this will stop me learning Lisp; it won’t stop anybody who really wants it. But it will cause a lot of unnecessary failures, and it will greatly lengthen the time I have to spend learning before I can start being a productive member of the community, which, in the end, is really all I want. I do want to learn Lisp. In spite of Emacs’ stupidity, and in spite of Lisp’s fragmented and not particularly well supported diaspora, and in spite of the weird looks I’ll keep getting from my dad, and from the CS/Math professor I play D&D with, who said “Lost In Silly Parentheses” and then didn’t seem to be paying attention when I tried to explain. In spite of the frustration and the lack of good guides for how to do anything, and in spite of the lack of books that suit my pace and style of learning, I still want to learn Lisp.

I’ve just started attempt three, and I’ve managed to get SBCL running tolerably inside a Cloud9 workspace, thanks to this guide by Lisper ThePhoeron, who coincidentally is now writing the next book I’d like to try in order to learn some more Lisp. It’s called Learn Lisp The Hard Way. Wish me luck.

As I tried to explain to my D&D group, for my own benefit as much as theirs, the parentheses, intimidating though they are at first, are actually a window into a great and powerful beauty, an almost fractal elegance that lies at the heart and in every corner I’ve so far seen of the language. It is truly breathtaking.