Steal Like a Data Visualiser
How I recreated a particle orbit in Clojure and improved my animation skills in the process
Improving your data visualisation skills through theft may sound vicious at first, but it is a very effective learning method and hopefully a great tribute to the original creator.
Why did I steal?
I was born in Germany, but my parents immigrated from Poland. In Germany there’s a stereotype that Eastern Europeans, and especially people from Poland, tend to steal a lot. But confirming this stereotype was not my main motivation … I was inspired by the book Steal Like an Artist by Austin Kleon. This book is full of great advice. It shows how stealing can help you become a better artist — here’s one of my favourite quotes from the book:
If we’re free from the burden of trying to be completely original, we can stop trying to make something out of nothing, and we can embrace influence instead of running away from it.
So I thought, if that works for art, then it can work for data visualisation as well. I checked my list of favourite data visualisations and decided to steal one. This article is about how I stole a viz from ‘The Unlikely Odds Of Making It Big’ by The Pudding.
Note: I’m not actually encouraging you to steal someone else’s work. This article explains how useful recreating other people’s ideas can be, to practise and enhance your own skills.
What did I steal?
The Unlikely Odds Of Making It Big shows the trajectory of 3000 small bands who tried to make it big in New York City from 2013–2016. Most headlined at a small venue with a capacity of less than 700 people. Only 400 successfully headlined a show with a capacity of over 700 people and, of those, 21 bands had made it big by 2016, i.e. they had headlined a show with an audience capacity of 3000+. This was one of the visualisations that inspired me to learn as much as possible about dataviz. My plan is to skill up so that one day I can create my own stories like this.
The visualisation is split into three parts: a particle orbit introducing all bands, a bubble chart about the bands who made it big and a filterable list with information about all bands who tried.
The particle orbit is the most impressive part of the article, and what I wanted to recreate. The journey of each band plays out in front of you as you scroll through the article. Initially you see 3000 white particles moving in a circle; each represents a band. While scrolling, you’re effectively applying filters on those bands as you move through time from 2013–2016. The more success a band had, the closer it moves to the middle. When looking at this visualisation, many details stand out e.g. the colour-coding and the embedded music players - these enable the reader to listen to a sample of all bands who made it big. Ultimately, the sheer number of particles illustrates really well just how few bands make it big in New York City.
What did I learn?
The particle orbit doesn’t just look awesome. I chose to recreate it because it looked like a good challenge for me. Thanks to the excellent book The Nature of Code I did not start from scratch. I had a rough idea of how to steal the particle orbit concept.
I think the “sweet spot” in learning is finding a task that builds on your existing skills. Don’t try to build on too many skills at once, or you’ll just get frustrated. Try to find a task that will challenge you just enough. Recreating the particle orbit was a “sweet spot” task for me as I could build upon the following skills: Processing, Trigonometry, Vector math, Particles, Flow fields and use of Autonomous agents.
I also learned some new skills: Creating dashed circles and curved text, and Managing multiple transition stages with Multi-dispatch and ad-hoc inheritance.
Now you have a good overview about what I stole, let’s get into the nitty-gritty detail and see how I stole the particle orbit.
How did I steal?
I recreated the particle orbit using Clojure, a functional LISP which can run either on the JVM or the browser. I split my learning journey into three blocks: Data, Design and Visualisation. Let’s take a look into each one separately.
Data
When I am not stealing data visualisations my day job is developing business applications at Comsysto Reply, and I am mostly specialised in front-end Development. In my job I spent a lot of time with loading and transforming data from servers to clients. Since I already have some experience in this area I did not invest time recreating the original visualisation data. Instead, I randomly generated some dummy data, only as much as I needed to create a similar particle orbit.
In the orbit you can see three major groups of bands, those who gigged at small, medium or big capacity venues in NYC from 2013-2016. Each medium band used to be a small band and each big band used to be a medium one. This creates a hierarchy within the data, and in the “Visualisation” section below you’ll see how this hierarchy minimises the amount of code we have to write. But first, let’s take a deeper look at the design.
Design
Studying the design of the visualisation was something I was very excited about, since my design skills definitely need improvement. Stealing is a really helpful process here because to create an authentic copy you are forced to observe every detail in the original. This may sound pretty easy and generic, but for me it’s hard, and I tend to overlook many details at first glance. Here are some specific things I overlooked when I saw the visualisation for the first time:
- In the initial orbit there are 15 lanes of particles.
- It seems that the particles randomly switch lanes but actually each particle stays at the same lane. The switching effect is created by using different speeds for the particles.
- When you pass a scroll point a short transition kicks off to get into the next state.
- It’s possible to go backwards and the particles will slowly find their old position in the outer orbits.
Writing down those details helped me to gather all requirements for the visualisation, so I could start working.
Visualisation
Programming the visualisation turned out to be a really challenging road with many dead ends. Here’s how I found my way through:
Creating my circular flow fields
My first idea for recreating the particle orbit was to use a flow field. This means that you think about your screen as a grid and for each grid cell you create an arrow. Each arrow represents a force pointing in the direction of the arrow. Whenever an object (known as an ‘autonomous agent’ is above the cell of the flow field this force is applied to the agent and it moves in the direction of the arrow. Here you can see an example:
My idea was to create a flow field where the arrow directions form a circle. Each particle in the orbit would be represented by an autonomous agent. In order to do this I had to learn how to calculate the correct angle for each arrow. I have recently improved my knowledge about trigonometry from reading The Nature of Code and working with Khan Academy. But there is a big difference between learning about different math concepts and applying them when needed.
I did not make any progress in rotating the force arrows for my circular flow field for many hours, which was pretty frustrating. In order to improve my understanding of the problem I created a debugging tool which showed me how trigonometric functions change when the angle of one agent is adjusted. I was able to find a solution to re-compute arccosine based on the y-position of an agent when it is moved.
Being able to interact with my problem helped me a lot. However, this approach is a really complicated solution! Some days later I learned about the trig. function atan2 which solved everything in one line of code. So on the one hand I spent several hours coming up with a solution I would later throw away, but on the other hand I improved my understanding of trigonometric functions along the way. I had learnt something, so it wasn’t a total waste of time.
After the flow field was in place I was thrilled to place my particles in the flow field and let them loose. By now I had added the second and third circle fields and the whole orbit looked like this:
Adding a steering behaviour to the autonomous agents
Unfortunately, as soon as the particles started to accelerate they quickly went off the flow field and were gone forever. Hmm… My first idea for tackling this issue was to add more forces to restrict the particle movements. But this seemed really complicated again. Instead, I tried a different solution.
I remembered reading about autonomous agents with steering behaviours in The Nature of Code: I knew it was possible to move an agent to a specified target point, so the solution I ended up with was to constantly update the specified target point to move the particles along a circular path.
It was great to see my particles now moving in a circle and not escaping. But once again it meant I had to delete everything I’d done so far with the flow fields. On the plus side, at least taking a wrong turn in the beginning turned into a great opportunity to improve my flow field knowledge :-)
Animating transitions
After the first particle orbit was in place I created a scroll transition so that the small venue circle is displayed after you start to scroll through the visualisation.
This involved learning how to draw a dashed circle and curved text, since none of those functionalities are provided out of the box by Quil, the library I used to draw the visualisation. In addition I marked the band Sylvan Esso, just as in the original visualisation.
The next step was to add another scroll transition so that medium bands move into a second orbit after a certain scroll position is passed. Once again, this is done by updating the steering behaviour. In addition, the colour and opacity of the particles change based on the scroll position, and the “MEDIUM” label appears.
Once the first and second particle orbits were done, creating the third orbit was straightforward. All bands who made it big now change their colour to pink and steer towards a predefined position in the middle of the “BIG” orbit circle.
I still remember when I saw this visualisation for the first time and thought how one day I want to create something great like that, so you might guess how immensely proud I was after my recreation was finally done :D
Tidying up
At this point the code was a bit of a mess and I decided it was time for some refactoring. My code was cluttered with ifs, which controlled the transition stages of the different venue sizes. Clojure’s multimethod was the perfect tool to clean this up. I could specify the hierarchy which occurred in the data (big bands → medium bands → small bands) and create a multimethod which dispatched a particle dependent upon the band size and the venue size. This way, no code was duplicated, each function is responsible for one band/venue pair and depending on the state of a particle (e.g. the position of a medium band particle while the reader scrolls through and starts the big band transition) the correct code runs without any additional ifs.
Conclusion
Stealing this visualisation was a massive journey for me and I learned a lot. I definitely plan to steal more, and am currently looking for new challenges, so please share any worthwhile visualisations you discover with me. To put it in perspective it took me about 30 hours to complete this project, including deciding what I actually wanted to steal, and later writing up and editing this article.
The complete code for the visualisation can be found here: https://github.com/rollacaster/steal-like-a-data-visualizer
Bloopers
As a bonus, here are some inbetween states of my visualisations which I thought looked awesome as well!
