That is the story of Electrified North. A journey I started at the beginning of this year. I commonly worked only for 30 mins per day on this project. I am proud I finally finished it after 70 hours of work.
This post will show the technical backstory of this project. I learned a lot, faced some deadends though I am happy with the final result. So let’s dive into the details.
A colleague of mine told me about the NOBIL database. The database includes information about charging stations for Scandinavia. That alone is very interesting though the data I was most excited about was the real-time data provided by NOBIL.
It’s possible to receive live events whenever someone plugs his in car somewhere in Norway. In the last few months, I spent quite some time waiting for people to do precisely that.
The NOBIL database offers two APIs a REST endpoint to load basic information about each charging station and a WebSocket connection pushing the Live Events. While the REST endpoint includes data for Scandinavia, live events are sent only for Norway. Therefore my visualization focuses only on Norway.
To wrap my head around the available data I started to visualize it. Those first tries are not beautiful though it’s fun comparing the start with the final result.
The next step was to understand the real-time changes. For each change, NOBIL sends the new state of a charging station after something changed, but it does not explicitly tell you which attributes of this particular charging station changed. For instance, which station connector was plugged or unplugged. I decided to use Clojure since its powerful standard library could help me here. I found a function called diff and combined it with a multimethod and reduce. This way, I was able to compute a list of changes with little effort.
Technical Detours Part I
Before working on this project, I have studied 3D modeling and development intensively. I decided that I wanted to create a 3D visualization due to that. In general, this is a terrible idea. Creating a 3D visualization only because you happen to be excited about 3D is quite likely to end as a failure. But somehow, I got lucky, and displaying the data in 3D worked well. Still, I advise against this approach you better check on your data and then decide what’s the best representation for it.
I wanted to include 3D objects into the visualization that I modeled by myself. That was a lot of extra work. Some effects look like a total beginner built them (because that was the case) though I am proud that I modeled the effects myself. Only for the complex charging station, I decided to buy a model to save time.
From Human Terrain I adopted the approach to show an intro before loading all 3D models.
Technical detours: part II
While deck.gl has pre-build capabilities to cluster location data. Unfortunately, they do not work with custom 3D models. So I needed to use another library called supercluster to extend deck.gl. That created a new challenge. The wrong live events were assigned if the cluster changed during zooming. So I needed to reassign events while zooming. Searching for the correct clusters turned out to be an expensive operation, so I had to make sure I’ll do as little as possible.
Analyzing Human Terrain again, I decided to add an intro tour that explains the visualization.
I also checked on shipmap once more and liked how they incorporated the notion of time by offering the feature to jump between different points in time. I decided to add a time slider which helps to see how the charging network developed from 2011-now.
There’s a helpful mantra for data viz coined by Ben Shneiderman: overview first, zoom & filter, then details-on-demand. I already provided an overview of all charging stations and allowed to filter them with the time slider. So my last step was to show details. A click on a station cluster shows every connector of a station in a modal.
My last steps were various optimizations. Such as making sure that the visualization is usable on any screen size, omitting some features on small screens, and adjusting the intro.
Besides, there are several techniques to optimize a web application in general. For instance, making sure there are no unnecessary libraries packaged in your bundle. shadow-cljs was a great help with that because it provides an analyzer. Using this tool, I discovered duplicated instances of the three.js library. Avoiding this saved me many KBs. Another optimization is to turn on gzip to reduce the amount of data.
After I finished the first version of the visualization, it was time to post it at ComSysto Reply’s internal Slack. There, I received tons of helpful feedback. I polished the first version. Now I am happy with the visualization.
I learned a ton of new skills while working at Electrified North. I learned to create 3d models, optimize a shadow-cljs app, work with deck.gl, and handling WebSockets in Clojure. To summarize, it was a worthwhile journey to build this data visualization. I hope you enjoy exploring the charging network of Norway. You can see the live version here: https://vizards.herokuapp.com/
This visualization was made possible by Comsysto’s concept of ‘Lab Time’ where employers can use part of their working hours to explore new technologies and build what excites them the most. If you want to do that as well, it’s time you join Comsysto.
This blogpost is published by Comsysto Reply GmbH