A DASHBOARD WITH HOPLON

Last month our own Alex Carl and Ron Nelson attended CED Tech Venture Conference, a regional networking event for North Carolina startups and investors.

Ron, our Director of Sales and resident Mario Kart prodigy, also gave a short presentation about Adzerk’s rapid growth over the past year.

While I didn’t personally attend the conference, some software I wrote did. That software, the “Impressions Dashboard”, is a single-page application (or SPA) showcasing our technology and some of our customers. Alex and Ron set it up to run on a Macbook connected to a TV at our booth.

The purpose of the Dashboard was to present live data in a way that gave passers-by a sense for our technology and business, and to help start conversations.

Here’s what it looked like set up at the conference:

You can also run a client-only version of the Dashboard to get a better sense for what it does. For expedience, the Dashboard was styled specifically for the TV it was presented on — it may not display properly on your screen.

In this post, I’ll share some of the tools, services, and practices behind this small but successful application.

THE DASHBOARD APPLICATION

Early last month, I had only been working at Adzerk for a few days when the Dashboard came up as a potential first project. Tech Venture was right around the corner, and projects similar to the Dashboard had been successful at previous conferences. We needed a Dashboard, and we needed one quickly.

Specifically, we needed an application that:

• showed real, production metrics and updated continuously
• would be easy for Alex and Ron to set up
• fell back to historical data during wifi outage, resumed with live data once connectivity restored
• was aesthetically pleasing
• could be built and tested in 3 days, tops

Ideally, the application would also be easy to improve by a designer, and would contain pieces we could employ in future conference and analytic dashboards, without necessarily involving a programmer.

DAY 1: DECISIONS, DECISIONS…

The only thing I knew for certain at the beginning of Day 1 was where the data I was supposed to show lived, and that place was Datadog.

We are happy and extensive Datadog users. We operate hundreds of AWS EC2 instances, and metrics are a big part of what makes operation at our scale possible. Our services send event data to Datadog, which we can then view in near-realtime using Datadog’s in-browser query and charting tools.

The tools Datadog provides through their web interface to build graphs and dashboards are phenomenal, and close to what we needed for our own Impressions Dashboard. However, we were interested in showing something Adzerk-branded and simpler.

Fortunately, in addition to the dashboard and query tools, Datadog has a nice HTTP API we could poll with a metric query. We could then use this query result as the value behind the dashboard gauges. With our own services, Datadog, and the Dashboard in play, the picture looks something like this:

MORE ABOUT HOPLON

The start of a project is the most expensive time to make a poor technology decision, so it’s generally a good idea to entertain and assess a variety of tools and approaches before committing yourself to any one of them.

That said, if you need to get to work on time, it doesn’t make much sense to waste time debating the merits of one horse breed over another when you have keys to a Cadillac in your hand.

The Cadillac in this case is Hoplon, a Clojure and ClojureScript framework for building SPA’s like our Dashboard.

Beyond the merits of functional programming Clojure brings, Hoplon brings also a treatment of HTML markup — as an alternate ClojureScript syntax — that can help satisfy our stretch goal of the application being designer-maintainable.

The unification of program and markup is what saves us from being a shadow world and will unlock for us the time-honored tools of black-box abstractions in a browser context.

Perhaps the most important Hoplon benefit in the case of this application, considering the hard due date, is the ability to consume and modularize existing pieces, like jQuery plugins and grid frameworks, for rendering interface elements like gauges and doing fluid positioning.

While browsers claim various levels of support for a slew of committee-designed and failed-but-supported/legacy APIs, only one — which has never even been in the browser — has stood the test of time and robustness. That API is jQuery, and through it and it and its plugin ecosystem we can save ourselves tons of time.

Oh, and Hoplon also rules because I made it (with Micha Niskin), and have been honing it specifically for SPAs like this one over the past two years. Yes, this blog post is rigged :-)

DAY 2.0: MOCK-DATA DEVELOPMENT

After establishing a Hoplon project scaffold, I got to work looking for the pieces I’d need to make something aesthetically pleasing.

Because of the way Hoplon works, I wasn’t concerned at all with details of obtaining data from Datadog for the purposes of interface development. I had a pretty good idea what I needed and that Datadog would support my data needs.

Helping me not care about Datadog was Javelin, a ClojureScript library Hoplon brought in, which cleanly separates presentation from computation using a spreadsheet-like dataflow model.

In the same way a chart in a spreadsheet doesn’t care where the values it’s derived from came from, my User Interface (UI) components won’t care about the provenance of values they’re to display. The process of both faking data and fetching it over the network is isolated from the process of displaying it, thanks to Javelin and the spreadsheet model.

The way most SPA frameworks work, you might think this isn’t possible. Trust me, it is.

A UI can display only values, not processes. How the values came to be, we don’t know or care.

The first piece I looked for was a grid framework to ease the pain of element positioning. While I’ve had good results in the past with Zurb Foundation, I tend to prefer Twitter Bootstrap these days, if only because there is a tremendous Bootstrap presence on StackOverflow that will be extremely helpful when I inevitably get stuck.

Another benefit of Bootstrap is that it’s available in Hoplon Vendor, a bank of JavaScript and jQuery libraries packaged for easy consumption by Hoplon projects. I had only to add the Hoplon Vendor bootstrap package to my project’s set of Maven dependencies, and Bootstrap was in, assets and all.

Interactive development using Hoplon is primarily via “live reload” — as I saved ClojureScript, HTML, and CSS files, the application in my browser automatically refreshed itself.

Because I was concerned only with how the application would look on the particular display Ron and Alex were bringing to the conference, I set up the TV on my desk so I could develop directly on it, live-reloading as I went:

DAY 2.5: COMPONENTS

My Hoplon introduction concluded with the idea that Hoplon makes accessible, and easy to modularize, the wealth of jQuery and JavaScript plugins and libraries already widely available and battle-tested.

Once I had established a skeletal Hoplon/Bootstrap project around day 2.5, it came time to prove this again to myself. I began to search for a “gauge” plugin of some kind.

I tried gauge.js, jQuery gage, and JustGage, and settled on JustGage because I liked the way it looked most. JustGage looked beautiful on the TV because it uses SVG and animations, and it was clearly the kind of thing I could not quickly cook up myself. Using it instead of an alternative, or rolling my own gauge, was a no-brainer, especially considering my time constraint.

The JustGage API

To create a JustGage, one first writes a little HTML placeholder:

<div id="gauge" class="200x160px"></div>

Then, a little JavaScript:

var g = new JustGage({
    id: "gauge",
    value: 67,
    min: 0,
    max: 100,
    title: "Visitors"
  });

The JustGage API is typical of browser UI element libraries, in that:

• instances are created using a constructor, in this case JustGage(), that takes a JavaScript object of configuration key/value pairs
• the constructor takes an id of the HTML node it should append the dynamically generated markup to. For JustGage, one specifies an "id" key in the configuration object. jQuery plugins usually "chain" to the jQuery object and operate on whatever selected elements it represents
• methods on the element instance provide for runtime modification of internal state. JustGage instances support a .refresh method that takes a new value to display.

Whether or not an API like this suits you is a matter of taste, as is just about everything about programming beyond the mathematical foundations of computing. For my own part though, I’m not a huge fan.

To start on a positive note, It’s nice that JustGage() takes an object instead of positional parameters, considering there are a dozen or more possible keys in the configuration object.

What’s less nice is that a required argument is the id of the target element. This is like hard-coding a pointer as an argument to a function. Beyond the fact that this id lives in a namespace other than our JavaScript program's, the way the DOM works we've also just established a temporal dependency between our JustGage instance and the DOM, because the element we specified is only available after the DOM has loaded.

So, we need to use jQuery’s $(document).ready(...) to manage this temporal dependency, adding immmediately to our boilerplate.

Finally, in order to update the JustGage instance after the application has loaded, we must retain a handle to it so we can call its .refresh method. With the need for a callback, the circle of confusion is complete. Adding more instances and other kinds of UI elements that need to be updated similarly (and in particular orders) will make the callbacks harder to orchestrate and debug as the application grows.

Modularizing with defelem

Fortunately, with Hoplon, we have tools to abstract away the details of owning and operating JustGage instances. Let's play with the most important tool, the idea that HTML can be seen as a Lisp, before we apply it to JustGage.

HTML as ClojureScript

Hoplon supplies a framework in which the JavaScript and HTML worlds are unified. In Hoplon, markup and code are expressed the same way — as a program. For instance, consider the following HTML:

<h1>Lisp is awesome!<h1>

Under conventional browser semantics, this fragment exists in a static HTML file, is interpreted by the browser’s HTML renderer, and becomes accessible in the DOM as a HTMLHeadingElement object after the HTML file has been processed.

In Hoplon, HTMLHeadingElement objects can be created like this:

(h1 "Lisp is awesome!")

h1 in this case is a ClojureScript function, and its calling conventions follow the rest of the ClojureScript language. Here is a more advanced example that also evaluates to a HTMLHeadingElement, but demonstrates some advantages inherent in considering HTML an alternate Lisp:

(let [language "Lisp"
      adjective "awesome"]
  (h1 (str language " is " adjective "!")))

Functional HTML Composition

Because h1 and all other DOM object constructors are normal functions, we can create and compose markup in cool ways. Consider this HTML fragment:

<ol>
  <li>first</li>
  <li>second</li>
  <li>third</li>
</ol>

In Hoplon, we could express it like this:

(ol (li "first")
    (li "second")
    (li "third"))

While syntactically the Hoplon version is marginally more concise, the real power is in the fact that because HTML is a programming language, we can eliminate the liduplication by functional programming means:

(ol (map li ["first" "second" "third"]))

Of note here are two things:

1. DOM constructor functions like li can be passed to higher-order functions like mapwithout issue
2. If a DOM constructor function like ol is passed a collection, it "splices" the collection into its children

Elements are Functions

Perhaps we find ourselves often creating ol lists with a bunch of strings. We might consider creating a function:

(defn quicklist [& strs]
  (ol (map li strs)))

Now, we can make lists like this:

(quicklist "first" "second" "third")

A custom element! Since elements are functions, and we can trivially create named functions, we can just as trivially create “custom elements” like quicklist here.

Back to JustGage

So, what I really wanted was a “custom element” called gauge that took some parameters and that I could use in markup as if it shipped with the browser. Here's what it looked like when I was done:

(defelem gauge [{:keys [value min max title label]
                 :or   {max 100}
                 :as   attrs}]
  (let [div-id     (str (gensym))
        elem-attrs (dissoc attrs :value :max :title :label)]
    (with-let [elem (div (assoc elem-attrs :id div-id))]
      (with-init!
        (let [params (js-obj "id" div-id, "max" max, "title" title, "label" label)
              gauge (js/JustGage. params)]
          (cell= (.refresh gauge value)))))))

This is intermediate ClojureScript, and also uses some functions and macros particular to Hoplon, so don’t worry if it doesn’t make total sense to you. Things to note about it include:

• it uses defelem to define a function, gauge, that takes a number of attributes as attrs using ClojureScript's map destructuring.
• it generates a randomly-named div called (str (gensym)) to point the JustGage instance at
• it instantiates a JustGage by passing a params object built with js-obj, a ClojureScript function for creating generic JavaScript objects
• cell= attaches a Javelin "formula cell" to the value argument. value here is an "input cell", which is a reference type similar to an Atom. The idea behind cells and cell= are that expressions should be run when any dependent cells change value. Here, cell= is used to run the expression (.refresh gauge value) whenever the value of value changes.

The Query Gauge

With a general gauge element handy, I was able to create another custom element, query-gauge. The idea with query-gauge was that I could instantiate gauges in markup and pass the Datadog query to visualize as a string attribute.

Internally, query-gage polls our Clojure server, which in turn polls the Datadog API, which returns a metric back to the gauge for display. Because this post is already a bit meaty I'll spare the details for another post.

In the end, in a index.html.hl file, I was able to create gauges like this:

<dash.query-gauge
  q="sum:eventserver.events.impression{*}"
  max="15000"
  title="Impressions"
  label="per second">
</dash.query-gauge>

DAY 3: FIDDLING AND TESTING

I didn’t actually do anything very interesting on Day 3 because Day 2 was so productive! My time was mostly spent tweaking CSS and ensuring the code worked under various connectivity scenarios. In the afternoon I deployed the application to a machine in AWS and showed Alex how to set it up in Chrome.

GROUP HUG

This post was a shameless Hoplon romp, but I hope at the very least it gave you a sense for how excited I am about Hoplon and the kind of potential I see in it. I encourage you totry the short tutorial on hoplon.io to see for yourself.

If YouTube is more your speed, you might be interested in a talk and demonstration I gave recently to Clojure Toronto.

Have a question or comment? Ping me on Twitter, I’m @alandipert. Thanks for reading!

This post was originally published by Alan Dipert on Adzerk’s Tech Blog in October, 2014. You can learn more about Adzerk here.