CLOJURESCRIPT REBOOTED

Build tools are great. They’re the part of the process of software development that only developers see–The Aristocrats of computing, a secret handshake we don’t generally trot out for clients or bosses. Here at Adzerk we’ve been polishing our Clojure workflow and I’d like to show you some of the tools we’ve built.

See, the thing is… well… the thing is we’re not using Leiningen. If this blows your mind too much you may want to get your earmuffs ready, because it’s going to get weirder. We’re using boot, a new Clojure build tool, to manage our build and deployment, and so far it’s been great.

Disclaimer: Boot is alpha software. The API may change. It may explode.

CLOJURESCRIPT

For this demonstration we chose to tackle the ClojureScript development workflow because historically it’s been an especially tricky thing to get right, and because the kind of flexibility ClojureScript workflows need are ideally suited to showing off what boot can do.

Everyone has a different idea of the perfect development environment, but there seem to be certain affordances that almost every ClojureScript web developer wants:

• CLJS compiler
• incremental builds
• browser REPL
• live-reload
• lein cljsbuild clean

The current state of the art in this area is things like chestnut: Leiningen templates that generate code and emit configuration files that pull together all the parts you need and set you up with a working skeleton project. This approach is effective, but it just doesn’t make us feel good about ourselves when we do it. We don’t want to generate a tower of boilerplate code, we want to understand what our tools are doing!

You can’t get to the moon by piling up chairs. –Stanislav

Let’s try piling up some other kinds of furniture and see how we feel about it, shall we?

GETTING A TASTE

The easiest way to see how things work is to just get in there and try it. We’ve prepared a GitHub repository with a “hello world” project: adzerk/boot-cljs-example. Go ahead, clone the repo and install boot, we’ll wait. When you’re ready you can start the build machinery with the following command line:

$ boot watch speak cljs-repl cljs -usO none reload

This starts a CLJS incremental build, a REPL server, and live reload via websocket. You can open the target/index.html file in a browser to see a friendly greeting (and a cat). You can now connect to the REPL server in your editor or via the command line:

$ boot repl --client

and start a ClojureScript REPL in the browser

boot.user=> (start-repl)

The browser will connect on its own and evaluate your expressions. Sweet! Ready to build a war file for production? Just add a Clojure namespace with your ring stack in there (at server/app, perhaps). Then,

$ boot cljs -O advanced web -s server/app add-src war

That’s it, your application was compiled with advanced optimizations and wrapped up as a war file in the target directory.

UM. SO?

If you’re like me you probably didn’t clone any repos or install new build tools while reading this blog. You can’t see the cat. That’s okay, you’re not missing much. The repo contains the following files:

boot-cljs-example/
├── README.md
├── build.boot
├── html/
│   ├── css/
│   │   └── main.css
│   ├── img/
│   │   ├── bg.jpg
│   │   └── face.jpg
│   └── index.html
└── src/
    └── app.cljs

The thing to notice here is that the only file related to boot or the build process in any way is the build.boot file. The rest is static HTML, assets, and a CLJS namespace. There is also no boilerplate code in the project related to connecting CLJS REPLs or live reloading. Basically, this project had no idea what was about to be done to it.

Even the build.boot file is surprisingly austere:

(set-env!
  :src-paths    #{"src"}
  :rsc-paths    #{"html"}
  :dependencies '[[adzerk/boot-cljs "0.0-2371-20"]
                  [adzerk/boot-cljs-repl  "0.1.5"]
                  [adzerk/boot-reload     "0.1.3"]])
(require
  '[adzerk.boot-cljs      :refer :all]
  '[adzerk.boot-cljs-repl :refer :all]
  '[adzerk.boot-reload    :refer :all])

We are building a ClojureScript project with almost no configuration!

BOOT IS DIFFERENT

Boot is different from other build tools in that it is not “declarative”, except in the sense of LAMBDA, the Ultimate Declarative. Boot has no innate understanding of project structure or build phases. There is no dependency tracking between tasks, and there is nothing analogous to lein’s :prep-tasks in boot. There is also no boot clean.

What boot does provide is an environment for running standalone, self-contained Clojure scripts. The actual build process is delegated to library code pulled in by the script (the build.boot script in this example).

DESIGN

In general, a build tool must perform three main functions:

1. Manage resolution and loading of project dependencies.
2. Provide a framework in which the build process can be specified, with suitable means of abstraction and combination.
3. Mediate the flow of files through the build process, from source to target.

We’ve already seen how boot handles dependencies in the build.boot file (via the set-env! function), so let's take a look under the hood at the boot build process. (We'll talk about how boot manages artifacts and files in another post.)

BOOT BUILD PROCESS

The central abstraction in boot is the task. Tasks are the modular building block of the build process.

• Where other tools have a menagerie of abstractions (plugins, middleware, profiles, etc.), boot has only one; the others can be trivially implemented as tasks.
• Boot tasks are uniformly composable. Tasks compose to form build pipelines, and pipelines are closed under composition.
• Boot tasks take only keyword arguments. This establishes a correspondence between calling tasks in the REPL or in code and from the command line. This also gives tasks the interesting property that partial application is idempotent (and the last setting wins).
• Tasks are normally defined by the deftask macro, which provides a simple DSL for creating self-documenting tasks that can be used effectively from the command line, the REPL, or from your Clojure code.

To get a feel for what this all means let’s start a REPL in the project and rebuild the project there. The build pipeline we constructed on the command line corresponds directly to expressions you can evaluate at the REPL.

REPL COMMAND LINE SYMMETRY

Consider the command line above that started the CLJS compiler process. The arguments we used correspond to tasks, and tasks can themselves take options:

# individual tasks separated by -- for clarity
$ boot -- watch -- cljs-repl -- cljs -usO none -- reload
# the same as above, long options, one task per line
$ boot watch \
-      cljs-repl \
-      cljs --unified --source-map --optimizations=none \
-      reload

We can do this in the REPL, too:

$ boot repl
boot.user=> (boot (watch)
       #_=>       (cljs-repl)
       #_=>       (cljs :unified true
       #_=>             :source-map true
       #_=>             :optimizations :none)
       #_=>       (reload))

You probably noticed that the keyword arguments in the REPL correspond to long options on the command line. This is true of any task defined with deftask.

ONLINE HELP

One of the things you get for free with this REPL/command line symmetry is great help documentation. You can see help info for the cljs task on the command line:

or in the REPL:

This documentation is generated from the task definition, and formatted for REPL or command line modes automatically.

DEFINING NEW TASKS

The final demonstration will be to define a new task. We can do this in the REPL, but for now let’s add a task definition to the build.boot file:

(set-env!
  :src-paths    #{"src"}
  :rsc-paths    #{"html"}
  :dependencies '[[adzerk/boot-cljs "0.0-2371-20"]
                  [adzerk/boot-cljs-repl  "0.1.5"]
                  [adzerk/boot-reload     "0.1.3"]])
(require
  '[adzerk.boot-cljs      :refer :all]
  '[adzerk.boot-cljs-repl :refer :all]
  '[adzerk.boot-reload    :refer :all])
(deftask dev
  "Build cljs example for development."
  []
  (comp (watch)
        (speak)
        (cljs-repl)
        (cljs :unified true
              :source-map true
              :optimizations :none)
        (reload)))

We define the dev task as simply the composition of other tasks. This is similar to how you would define a new transducer as the composition of other transducers. We can now call this task from the command line:

$ boot dev

or in the REPL:

boot.user=> (boot (dev))

CONCLUSION

Build tooling is a hard problem. Those of us who build mostly applications need tools that are flexible and modular. The current state of the art just doesn’t provide what we really need. We need to stop generating boilerplate, eliminate the tangled web of redundant configuration and start creating reusable, modular build tasks. Boot is our attempt to bridge this gap.

We’ll be posting regularly about boot and how we’re using it here at Adzerk to build a worldclass Ad Server. I hope you have fun playing with it and remember, “Lisp Can Do It!”

This post was originally published by Micha Niskin on Adzerk’s Tech Blog in November, 2014. You can learn more about Adzerk here.