Storytelling with Enterprise Data
How I designed a data storytelling platform
Intro
Each enterprise possesses a unique dataset.
First of all, every enterprise business collects data about its relationship with customers, such as its sales figures, KPIs and user metrics, to be fed back into business strategy and product development. For example, your UX research team looks at task success rates to help make design decisions. Furthermore, especially in B2B world, some enterprises are helping others manage their data. For example, Salesforce manages your sales data, while Splunk integrates all of your machine-generated data.
All of this data isn’t going to be useful until business can extract and communicate insights from it, and this is getting more and more challenging as data volume and complexity rapidly increase.
Why storytelling with visualization?
Data Storytelling is an effective technique to approach the challenge, because it combines the power of visualizations and stories, and provides narrative and context around data to really help people understand a complex dataset.
There are many inspiring articles about this topic that indicate the challenge is often times not technical, but about user experience and communication. I want to share my own experience building a data storytelling enterprise platform from scratch.
There are two main categories for data visualizations: explanatory and exploratory. This case study I focus on here is mainly explanatory because the goal is to explain some complex data to the audience.
However, unlike most other explanatory storytelling pieces, this application deals with dynamic datasets. Essentially, we built a data viz platform that can take in different datasets, so that each user can engage with their own data story.
The Problem
At work, we process security data for customer network traffic, and use that data to build products for customers to secure their company. At the same time, we need to communicate security data to help customers uncover their network risks and understand the product value. So the design question for me is:
“How might we communicate security data to our customers to help them understand their network risks?”
My process to design for data storytelling
My process consists of four key elements.
User
For general storytelling purpose, where your visualization lives will have a big impact on your design decisions. Is your user reading a New York times article, walking around a museum exhibition, or scanning their annual shopping summary on a Sunday night? This determines the level of engagement and interactivity they commit. This also leads to various actionable items you can provide after yours users leave your design. The questions to ask are:
- What is the user’s goal?
- Why are they listening to your story?
- What can they learn and act on from the story?
As for enterprise users, their goal is to extract information needed for business decisions and then take actions to get their job done.
In my case study, I am designing a web application for potential customers who are trying out our security products. They are curious to see what security events happened in their network, learn more about the threats, and further understand the efficacy of our security solution.
Data
Enterprise dataset is usually complex with multiple variables. Figuring out the data part should happen at the same time, if not earlier, than the user research. Some key questions to ask are like:
- What’s the type and meaning of each attribute?
- What is the main characteristic of the dataset: graph, geographical or time-series?
- Does any attribute have patterns or outliers worth calling out?
Now the tricky part is that with a fixed dataset you can have every detail of the dataset knocked out including the scale and range. But with a dynamic dataset, you should be looking at as many samples as possible to get a sense of the range and trend.
In my case study, the raw data is consisted of individual logs from both customer local and global database. Each log entry has a timestamp, location and network type of attribute.
Story
You want to make the user the hero of the story and help them flow through it to reach their goal. My approach here is to start with writing narratives while imagining myself telling the story in front of the audience, almost as a tour guide. From there I identify main plot points. A common narrative structure includes a starting point setting up the scene, rising actions leading to the crisis and the climax, and falling action to the ideal happy ending. Donna Lichaw has an inspiring book of story mapping.
Below is the storyline from my project. Our user starts with an uncertain and curious feeling, and then gets a little interested when they see their network results. User interest rises when more threat details are revealed. The climax is the moment when a customer sees a high-risk threat but also realizes that they are safely protected with their network security deployments. The happy ending is the user feeling confident about their product purchase decision.
Visuals: Map and encode data
With the data and the story set, you can start filling the data into your narrative structure and meanwhile come up with the visual code. While there are a lot of charting libraries offering comprehensive charting options, custom visualizations will make a difference, because you can tailor it to fit your data story perfectly.
In my case study, the users want to know when and how all the events happen in their network. Our solution has custom 3D-style line charts, with aggregated hourly information represented as bubbles.
Another example is a customized graph. The story here is: starting with two seed nodes, you can mine the graph to unveil the full attack infrastructure. After tinkering with tree, force-directed and radial layout, I designed this graph layout that fit that particular story better.
Let the story flow: Navigation design
A common used pattern is to use scrollytelling, where the user can see the data change and story progress as they scroll down the page. Another pattern is referred as stepper, which provides clear navigation for use to go back and forth. Zan has a thorough discussion about the difference.
No matter which pattern you choose, animation is critical between different stages, because it helps maintain context, drives attention to changing elements, and overall contributes to the perception of the dataset.
In my case, dealing with a dynamic dataset proved a bit challenging to craft a universal transition that works every time. I decided to go with the stepper, where a user clicks to view the next/previous state (bonus point: the whole app is made keyboard friendly). Animation design is something I want to further improve on in the next iteration to make sure they serve my storytelling purposes.
Key Challenges
Rapid prototyping is not rapid for data viz
Data is a difficult medium to work with for designers, because a dataset can easily become overwhelming to ‘sketch’ out or ‘animate’. Yet it’s important to prototype with real data as early as possible. Knowing how to code visualizations and build quick prototypes certainly help in the process. You can also work closely with your engineers who are willing to experiment and try together.
Design visualization for data edge cases
The biggest challenge I encountered is designing for dynamic datasets with unexpected edge cases. Since we can never be sure what every future customer’s dataset is going to look like, there is a risk that our carefully crafted storytelling experience may break in some situations.
For example, one of my original designs had a bubble chart arranged on a timeline. But later on, a user had a large amount of suspicious events within a short time period and we couldn’t arrange those bubbles any more, so I had to make changes to it. We definitely encountered ‘worst-case’ datasets in the wild where it broke the front-end app from rendering. The solution here is really to consider and collect edge cases as much as you can, but adapt to the unexpected through fast iteration.
Automation v.s. customization
Like many tech products, the more you automate, they more issues you will have with control and trust. As for our project, we built this web tool to facilitate data storytelling and eliminate manual tasks, yet it’s difficult for the code to capture and highlight the best story. The feedback I collected was that people want to customize the results by emphasizing certain views. In general, I think it’s a balancing act between automation and providing customization capability, and is definitely something we also learn from users as we go.
Conclusion
I’d admit that I was drawn to the world of data visualization by the aesthetics, but then became fascinated by its power. Visualization makes it possible for us to interpret a very complex dataset and further draw insights from it. When putting visualization in a compelling narrative, it will unlock huge potential and help us achieve better human communication. For me, there are certainly encouraging moments when people adopted our tool and recognized the value immediately, and that makes me a even stronger believer.
I hope you find this article useful or interesting. I’d love to hear any feedback!
