Little steps to big data
A fragmented path to automating creativity
There’s nothing that makes datasets more accessible than good visualisation.
I’ve been a fan of David McCandless work in visualisation for some time and I wanted to imitate him, but balked at the idea of drawing artwork from spreadsheets in Adobe Illustrator (I know just enough Illustrator to get into trouble). I knew there must be a better way, pixels on the screen are after all, just representations of data. So the question that plagued me was; why isn’t there a software tool that takes tabular data and creative direction as inputs and generates beautiful visualisations as output? (MS Excel’s chart tool does not count).
Is this kind of work just too creative to automate? Is there something inherently human about good design that just cant be programmed into a machine? Maybe, but that wouln’t stop me from trying.
Processing
My first attempts at machine generated visualisations started when I discovered Processing, a visually oriented Java like programming language and IDE. Processing was conceived at the MIT Media Lab as a teaching aid to give Computer Science students immediate visual feedback on objects and their properties in an on-screen canvas. But it grew to bring a greater understanding of the visual arts to software engineers and data geeks everywhere.
I had a lot of fun with Processing and managed to build a simple model for traffic patterns on a motorway. However sharing my work and getting feedback was, at the time, limited to people who I could convince to install the Processing browser plugin. So I forgot about visualisations for a while and got bogged down in less interesting projects.
In hindsight my traffic pattern sketch might have been a lot more engaging had I had access to real data instead of generating random vehicle size and speed.
Lesson learned: It’s not OK to play with random data.
There’s really no excuse with the wealth of open APIs available to developers.
D3.js
The next foray into visualisation was after attending a recent O’Reilly webinar “From Scattered to Scatterplot in 2 hours” by Scott Murray. It gave a whirlwind introduction to D3.js, a JavaScript framework similar in concept to jQuery, that allows rows in datasets to be bound to visual elements in the browser.
Fortunately, the webinar coincided with a request from a client to add a simple climate chart to their website. I took the opportunity to experiment with D3 on my own before attending the webinar in the evening. This worked out well, my experiments produced an almost complete (if a little uninspired) climate chart based on real average temperature and rainfall data for the Malaga province.
After the webinar I had learned enough to solve the axis and scale problems I was having and add a little animation.
You can see the animated chart along with my source code in a jsfiddle.
So far, I have found no magic tool to create stunning visualisations of raw data, but I have learned a lot about computer graphics and I’m better equiped to illuminate data in the browser.
