Crafting a custom, mobile-friendly
data visualization
The design process, from science paper to responsive interactive
Mobile- and touch-optimized data visualizations are still hard to find. Especially more complex approaches such as network visualizations. The data visualization of free trade agreements FTAVis tries to tackle this issue. FTAVis (www.ftavis.com) is a tool to make the complexity of the world’s trade agreements visible, accessible and understandable. The project visualizes 789 trade agreements from 205 countries over the last 66 years.
It is a valuable tool both for economists, as well as journalists and freely accessible to the public. The project was carried out by the Global Economic Dynamics Project Team of the Bertelsmann Foundation with the help of data visualization designer Sebastian Sadowski and data analyst Tobias Pfaff.
The project was awarded with the Design Award 2016 of the Federal Republic of Germany for it’s excellence in Communication Design.
This article is the story of the design process starting with a scientific paper and the data set. Some early sketches will be presented and the path to the final visual language pictured. Some further steps like the design of the user interface and user experience are also discussed as well as the optimization for mobile, touch devices.
The final visualization is available at ftavis.com and the iPad app in the App Store.
The Task
In 2014, senior project manager Dr. Ulrich Schoof of the Bertelsmann Foundation thought about presenting the linked, globalized world through trade agreements. “Trade agreement” is a term that appears more and more in the press recently, for instance the TTIP (Transatlantic Trade and Investment Partnership) which is a proposed Free Trade Agreement between the European Union and the United States of America.
However, the idea was on the one hand to present important trends and stories of trade agreements for the general public but also to create on the other hand a tool which could help experts to gain some new insights. Therefore, it became clear that it would be necessary to present the development of trade agreements over a longer timespan such as the last 60 years and more, thus getting a better impression of “the evolution”.
Inspired by the publication of a new dataset about the “Design of Preferential Trade Agreements” published by the World Trade Institute (University of Bern), the Bertelsmann Foundation asked interaction designer Sebastian Sadowski about crafting a data visualization.
1. Starting with a Scientific Paper
Data acquiring, parsing and mining
The used data was collected by an international team of researchers working for the World Trade Institute, an interdisciplinary institute based at the University of Bern and is freely available at designoftradeagreements.org. It was originally published in the paper: Dür, A., Baccini L., and Elsig M. 2014. The Design of International Trade Agreements: Introducing a New Dataset. The Review of International Organizations.
Data scientist Dr. Tobias Pfaff helped with bridging the economic and data perspectives. His first rough analysis of the data set made clear: the data contains 789 free trade arrangement (FTA), which are signed between 1949 and 2014 for around 200 countries. He figured out that one of the most interesting and revolutionary fact of the data set is the provision of a depth value for each trade agreement. It is a simple additive index of seven key statements that can be part of a FTA (e.g., statements about tariff reduction or intellectual property rights protection). The statements have been found by researchers of the World Trade Institute who searched for them in the FTA’s contracts.
The paper already contained some very basic, static visualizations which highlighted the main findings and helped with getting a first overview of the FTAs over time or per country. Even though the approach was to develop a more attractive, visually appealing and complex, interactive data visualization, the simple graphs and highlighted findings were a good start for the visualization process.
Preparing the data
In the next step, we extracted the most important information with a custom web-based data parsing tool from the data set and transformed it into an JSON-structure for further use. During the process we had to make some decisions about which data we want to include or exclude. For instance, the data set provided two measuring methods for the depth value of agreements, namely the depth index and what’s called the Rasch model. We decided to go with the depth index because the calculation is easier to understand and has also been used by colleagues of the economic research group Ifo Institute.
Besides choosing which data should be used, we also had to extend the data set. For instance, data for each single country had to be extracted (following the ISO 3166 standard) and it had to be figured out if and when each country joined the European Union so that European trade agreements could be assigned accordingly.
2. Finding a Visual Language
After analysing and working on the data set, the next step was to work on the visual output.
Two main aspects of the data set should be presented: First of all, every country and their data and secondly the connections between them resulting in a “network”. Therefore, Sebastian started with a research to find already existing visualization projects which are either geo-based or network visualizations.
A good tool to keep an overview of the visual research is Pinterest.
Here is a collection of inspirational projects for the project:
After the research, three fast sketches had been programmed with D3.js to figure out how the data set could work with different visualization techniques.
D3.js is a javascript visualization library which is becoming the standard tool in today’s web-based data visualization projects and the growing community and existing examples of different techniques helped with getting fast forward.
Geo-based visualization
The first sketch was a simple geo-based visualization with a bubble for each country on the rough position of the geo-location. The background color identifies one specific FTA and the color radius highlights the amount of partners. Even though it became clear which areas worldwide are economically more active than others, the connections between countries could only hardly been seen and the visualization technique failed on presenting many FTAs. There is also a basic visualization by the WTO that tries to visualize some FTAs on a map.
The code is based on the D3.js example “D3.js world map with force layout” which had been created for the presentation of Olympic medals at an article for the Huffington Post.
Sankey diagram
To get a better overall picture of the network and connections between countries another sketch with the visualization technique of a sankey diagram had been programmed. The diagram highlights the FTAs with the most countries and also made clear that many countries are engaged. Unfortunately, the diagram is not very “sexy” and it is hard to follow some countries if the data set for another year had been chosen. Therefor another sketch had been created to present the network but also consider the geo-based position.
The code is more or less based on the D3.js example “Sankey diagram using names in JSON”.
Radial dendrogram
The third sketch has been inspired by the first two methods and focused on presenting the network of FTAs. It also became clear that with that technique it would be possible to position the country items on their rough geo-position (north/east/south/west).
Additionally, the visual aesthetic of the visualization technique was quite interesting and after pitching it to the Bertelsmann Foundation, we all agreed on developing the idea of the radial dendrogram further.
3. How to Implement a Radial Dendrogram Made with D3.js
Using hierarchical edge bundling
The aesthetics of the Eigenfactor Project by Moritz Stefaner was an important visual inspiration for further development. The diagram visualizes citation patterns among branches of science. Stefaner did a great job with creating the bezier curves using the hierarchical edge bundling technique. This technique is based on a standard tree visualization method and extended with a curve connecting one node to another which results of a hierarchical clustering calculation. This visualization method is also known as radial dendrogram.
Here is a visual example:
In 2009, Flash was a popular tool thus the Eigenfactor visualizations were built with flare.
The requirements for the Bertelsmann visualization were not only that it should work on all major browsers but also on mobile devices (Flash doesn’t work on iPads).
Mike Bostock — one of the creators of D3.js and former editor in The New York Times’ graphics department — created already a good example of a hierarchical edge bundling visualization made with D3.js.
Starting with that sketch, the data had to be adapted to follow the Flare class hierarchy – courtesy Jeffrey Heer. Therefore, the custom data parsing tool was extended to output a useable Flare file.
Experimenting with the visualization technique of a radial dendrogram
The data set provides different kinds of country connections (e.g., between two countries, one country and a group of countries, etc.). So one of the first tasks was to figure out how the hierarchical edge bundling technique could help with visualizing the complex hierarchy.
The results were quite satisfying and even many FTAs with heaps of connections could be visualized. By grouping countries around continents the overall structure became clearer and some promising patterns could already been seen. The results were also visually appealing:
The visualization should show FTAs over the last 66 years so it was clear that all countries had to be visible, even though they had only one or a few FTA(s).
Unfortunately, if every country in the world (around 250) would be shown, it would be pretty hard to figure out which countries had some FTAs with others. Thus some further data processing removed all countries which had never been involved in any FTAs, at all.
Under these 44 countries are mostly island countries such as New Caledonia and Puerto Rico or remote parts of the world such as the Antarctica.
Defining visual attributes
The radial dendrogram can visualize at least six variables:
line color/opacity/weight, text size/weight/position.
The total amount of FTAs for each country should also have been visible therefore various visual attributes have been tested resulting in introducing another visual element: a circle in front of every text item.
Circles
The circle in front of each country text item visualizes the total amount of FTAs with the circle size. In earlier versions the fill color represented the average depth index of the country but due to user feedback that was later removed so the user could focus and is not overstrained.
Lines
Every FTA is presented as a collection of lines bundled through the hierarchical edge bundling technique and highlighted with a color for visualizing the depth of that FTA. If a FTA has been signed in the past it will be drawn in the back and dimmed. Only the FTAs which are signed at the selected year are visually highlighted.
Line Colors
A great tool for finding some colors is chroma.js by Gregor Aisch which is there to help mastering multi-hued and multi-stops color scales.
The above color scale has been replaced by the one beneath due to user feedback.
Providing only two color areas such as blue for a minor value and red for a major value seemed to be easier understandable than three color areas. Only one colour area has already been tested, such as light blue to dark blue, but didn’t manage to highlight the FTAs with a high depth index accordingly if there are many agreements/lines visualized.
Line positions
Another challenge was the amount of lines and to identify each FTA. We solved that issue by placing the intra-continental agreements outside of the circle. Credits go to the intern of the Bertelsmann Foundation: Quentin Dumont.
Handling performance issues
We tested the visualization on several devices — including tablets — and figured out that the performance of redrawing many lines were bad. One of the ideas was reduce the drawn lines, e.g. remove the lines of the FTAs which were signed before the current selected year.
However, we had a call with one of the data set publishers Andreas Dür — from the Department of Political Science and Sociology, University of Salzburg — and he argued that it is definitely necessary to present all signed FTAs in the background. It would help to understand that FTAs will last over a long period of time.
However, drawing thousand of (SVG) lines in a browser resulted in some serious performance issues. Additionally, for some years there are FTAs with over 100 member countries resulting in drawing heaps of lines for one agreement only. Finally, we had to find a better solution for drawing so many lines at once.
- Solution: Drawing many lines faster with Canvas
Another rendering technique in the web which is normally used for drawing many items in the browser is the Canvas technology. There are a lot of discussions about that topic.
At the end, we decided to go with a mixture: Drawing all lines with Canvas and the Nodes (country text and circle items) with SVG so they are easily selectable. There are three layers with different Canvas elements: One for the total amount of FTAs over the whole time period, one for the selected year and one for the selected FTA(s). Therefore, not all Canvas elements had to be redrawn at every stage and some layers can be faded out easily.
There is also an example of a hierarchical edge bundling done with Canvas by Bostock. The transformation from SVG lines to Canvas has been done with the library pathg. It would be better to calculate the lines in Canvas in the first place with libraries e.g. like d3-canvas but pathg is very fast and due to the complex visualisation technique not worth the trouble. - Solution: Outsource performance-heavy calculations to Web Workers
Web Workers spawning background scripts in your web application thus the User Interface is still usable during calculations. Here is a nice article about that topic. It has been used especially for the “Total”-Feature and export function.
Intermediate results without the user interface
4. Designing the User Interface
After processing the data and designing the visualization, an interface was needed to offer the user some interaction possibilities to explore the data in more detail.
To keep things simple we decided that the user should filter the data through selecting a specific country and year to find some new patterns and insights. Finally, the visualization should not provide a total overview of all years to the past but rather highlight the “evolution” of the economic development over the last 66 years worldwide and for each country.
Time slider
The user should be able to select a specific year as well as start an animation to obey the development over time. Due to the mobile-optimization we decided to provide three methods of interaction: 1. Draggable time slider for a quick jump to a time range, 2. Forward/backward button to go through each year slowly and 3. Play/pause button for the animation. In an earlier version, the slider and trend highlighter were two separated parts but after some user feedback merged into one. The range slider is based on the Javascript library noUiSlider which is great for mobile use.
Country selector
The user should be able to select a country in an easy way. Select2 has been chosen for that task. It is a jQuery-based replacement for select boxes and is usable on touch devices. Besides the interaction at the sidebar, the user can also hover over a country item to get a preview or click / touch on a country item to select it.
Trend highlighter
It became clear that it was quite a challenge to browse through countries and years to find, e.g., the most FTAs or the highest depth index of a country in one year. Inspired by the paper of Dür, Baccini, and Elsig (2014) we decided to include another small, interactive bar chart for each country to highlight trends and find interesting results or data stories faster. For instance the years with the maximum or minimum amount of FTAs or the deepest FTAs.
In an earlier version, two line charts represented the amount of FTAs and average depth over time but it turned out that the interpolated line was hardly readable for countries which had only a few FTAs over the last 66 years. Therefore, the two charts were replaced with a bar chart so it became clearer in which exact years were the max or min amount of FTAs.
Finally, the two charts were merged into one stacked bar chart and were combined with the time slider. This redesign is due to some user feedback, obeying that users were looking first on the charts to find some interesting years to look at and use the time slider next. Another reason was the creation of a cleaner user interface and to free up some space for the list of FTAs below. The stacked bar chart has been chosen to easily find some interesting patterns due to the fact that every FTA with their color of the depth index are previewed in the graph.
List of trade agreements
In some scenarios there are too many lines so that it is hard to figure out how many and which FTAs are actually shown.
We decided to not include an interaction method to select a path by hovering/touching it but rather by presenting all FTAs in a separate list.
The FTAs are sorted by the highest depth index and amount of member states thus the most important FTAs are likely to be shown on top.
By selecting a list item additional information are presented such as the agreement type (e.g., Plurilateral and Third Country) or the reason for the specific depth index.
Due to the amount of items a resize/shrink button has been added below the list making it possible to focus on the shown FTAs. Unfortunately, a deep link to the FTAs for even more information is not provided, yet. Even though there is a list of a few agreements by the WTO.
Including a legend
If the user sees the visualization for the first time, some elements are not self-explanatory so some kind of legend is needed.
Interactive color scale
The definition of the depth index is among the most interesting information in the data set. Therefore, we included an interactive color scale: a bar chart which highlights (higher position) all depth values which are currently visible. Additionally, by hovering over the color all FTAs with that index are highlighted.
Circle size
The circle in front of each country label has been added to provide another dimension of information: the total amount of FTAs until the selected time. The circle should highlight countries which are more or less involved in the “economic exchange”. By hovering over a circle all countries with up to that amount are visually highlighted. Therefore, countries who are not engaged until a specific time frame can be found as well as countries which signed the most FTAs.
5. Providing a Better User Experience
Until that point the data has been processed, the data visualization designed and a user interface to manipulate the data has been created. As one of the final steps, the experience of the visualization had to be improved so the user can have some fun browsing through the data. In the best case, the user would be even more engaged with the data visualization and would recommend it to others.
Loading time
The data for the visualization is around 2MB in size and for users with a slow internet connection a preloader has been included implemented with d3.js, too. The preloader should keep the user on the page and provide some information how much longer he or she has to wait.
Mike Bostock created a nice example that demonstrates how to display the progress of an asynchronous request.
Animation
Some simple animations should convey the feeling of using an application instead of a website. After the data has been loaded, the middle radial visualization as well as the sidebar on the left and right are faded in. Some unimpressive animations has also been included such as fading effects by hovering over countries or by changing the selected year.
Responsive design
The optimization of web-based data visualizations for different devices and above all touch devices are still complicated. Therefore only some basic optimization has been implemented.
First of all, the visualization needs a specific amount of CPU/GPU power and device dimension to visualize the data visualization properly, e.g. tablets or desktop computers. Thus the user experience at small devices such as smartphones would not have been amazing. Therefore, smartphone users are getting notified about the minimum device requirements and can browser through the “Selection of Findings and Data Stories”.
Secondly, the data visualization is basically optimized for tablet devices. The filtering of countries or the selection of a specific year can be handled with a touch interaction as well as the selection of a FTA. The project is even accessible offline for iPad users with the native iPad app which can be downloaded at the Apple App Store.
Thirdly, the project is easily accessible on desktop computers with different screen resolutions as well as browsers. Due to the fact that on bigger screens the data visualization would be drawn in an accordingly big Canvas element, the performance would be problematically. Thus the maximum width of the project has been limited to 1300px and positioned in the center.
Providing more information
User feedback highlighted that some of the most interesting data stories were hardly found by themselves. Therefore, some additional information are provided thus user can get even more into the topic and gain some knowledge.
First of all, a “Selection of Findings and Data Stories” section has been included below the visualisation which focuses on six findings and should enable the user to find more stories.
The second section “About” is more about a general introduction into the topic, a further explanation of the depth index and who the people are behind the project.
6. Promoting the Project
What is a visualization worth if it is not seen by people? Besides spreading the word about the project, the Bertelsmann Foundation decide to create a video about the project to further highlight some findings and tell some stories with the visualization.
A press kit has also been packed for the media.
The Final Version
Here is the final version available at ftavis.com.
The data visualization on a desktop computer
Evaluation
As already mentioned at the introduction the data visualization should not only be a presentation for consumption but also a tool for experts. It will be interesting to find out if and what kind of new findings will be found with the help of the project.
A brief evaluation with insight about some new findings of experts will be collected and hopefully published soon. Some new data stories can also be submitted to hello@ftavis.com.
Enjoy!
