Electrified North — Visualizing the Norwegian charging network
That is the story of Electrified North. A journey I started at the beginning of this year. I commonly worked only for 30 mins per day on this project. I am proud I finally finished it after 70 hours of work.
This post will show the technical backstory of this project. I learned a lot, faced some deadends though I am happy with the final result. So let’s dive into the details.
Data
A colleague of mine told me about the NOBIL database. The database includes information about charging stations for Scandinavia. That alone is very interesting though the data I was most excited about was the real-time data provided by NOBIL.
It’s possible to receive live events whenever someone plugs his in car somewhere in Norway. In the last few months, I spent quite some time waiting for people to do precisely that.
The NOBIL database offers two APIs a REST endpoint to load basic information about each charging station and a WebSocket connection pushing the Live Events. While the REST endpoint includes data for Scandinavia, live events are sent only for Norway. Therefore my visualization focuses only on Norway.
To wrap my head around the available data I started to visualize it. Those first tries are not beautiful though it’s fun comparing the start with the final result.
The next step was to understand the real-time changes. For each change, NOBIL sends the new state of a charging station after something changed, but it does not explicitly tell you which attributes of this particular charging station changed. For instance, which station connector was plugged or unplugged. I decided to use Clojure since its powerful standard library could help me here. I found a function called diff and combined it with a multimethod and reduce. This way, I was able to compute a list of changes with little effort.
Technical Detours Part I
Besides Clojure, I used its JS-Compiler ClojureScript to build the visualization. To facilitate interop with npm packages, I decided to use shadow-cljs. I planned to visualize the data using kepler.gl. With the help of the impressively helpful creator of shadow-cljs, I learned that due to uncommon usage of JavaScript within kepler.gl, it is not compatible with shadow-cljs. So I decided to work with fighweel-main that serves a similar purpose as shadow-cljs but allows integrating any JavaScript with some additional configuration. I was able to compile kepler.gl with fighweel-main only to find out that kepler.gl was too high-level to use for my purpose. Instead, I needed to use its underlying library deck.gl. deck.gl works perfectly fine with shadow-cljs and since I’ve got more experience with shadow-cljs, I switched once more. This whole technical detour cost me 6 hours, I almost gave up on the project, but luckily it went much better after that.
Design
Before working on this project, I have studied 3D modeling and development intensively. I decided that I wanted to create a 3D visualization due to that. In general, this is a terrible idea. Creating a 3D visualization only because you happen to be excited about 3D is quite likely to end as a failure. But somehow, I got lucky, and displaying the data in 3D worked well. Still, I advise against this approach you better check on your data and then decide what’s the best representation for it.
I wanted to include 3D objects into the visualization that I modeled by myself. That was a lot of extra work. Some effects look like a total beginner built them (because that was the case) though I am proud that I modeled the effects myself. Only for the complex charging station, I decided to buy a model to save time.
I got my inspiration for the final design mostly from the visualizations Human Terrain & shipmap. Those two incredible data visualizations that you should check out as well.
From Human Terrain I adopted the approach to show an intro before loading all 3D models.
Technical detours: part II
While deck.gl has pre-build capabilities to cluster location data. Unfortunately, they do not work with custom 3D models. So I needed to use another library called supercluster to extend deck.gl. That created a new challenge. The wrong live events were assigned if the cluster changed during zooming. So I needed to reassign events while zooming. Searching for the correct clusters turned out to be an expensive operation, so I had to make sure I’ll do as little as possible.
Interactions
Analyzing Human Terrain again, I decided to add an intro tour that explains the visualization.
I also checked on shipmap once more and liked how they incorporated the notion of time by offering the feature to jump between different points in time. I decided to add a time slider which helps to see how the charging network developed from 2011-now.
There’s a helpful mantra for data viz coined by Ben Shneiderman: overview first, zoom & filter, then details-on-demand. I already provided an overview of all charging stations and allowed to filter them with the time slider. So my last step was to show details. A click on a station cluster shows every connector of a station in a modal.
Optimizations
My last steps were various optimizations. Such as making sure that the visualization is usable on any screen size, omitting some features on small screens, and adjusting the intro.
Besides, there are several techniques to optimize a web application in general. For instance, making sure there are no unnecessary libraries packaged in your bundle. shadow-cljs was a great help with that because it provides an analyzer. Using this tool, I discovered duplicated instances of the three.js library. Avoiding this saved me many KBs. Another optimization is to turn on gzip to reduce the amount of data.
Feedback
After I finished the first version of the visualization, it was time to post it at ComSysto Reply’s internal Slack. There, I received tons of helpful feedback. I polished the first version. Now I am happy with the visualization.
Conclusion
I learned a ton of new skills while working at Electrified North. I learned to create 3d models, optimize a shadow-cljs app, work with deck.gl, and handling WebSockets in Clojure. To summarize, it was a worthwhile journey to build this data visualization. I hope you enjoy exploring the charging network of Norway. You can see the live version here: https://vizards.herokuapp.com/
This visualization was made possible by Comsysto’s concept of ‘Lab Time’ where employers can use part of their working hours to explore new technologies and build what excites them the most. If you want to do that as well, it’s time you join Comsysto.
This blogpost is published by Comsysto Reply GmbH
