Visualizing, The Field: Finding Our Way

There are numerous routes through the profession of Data Visualization, and most of them are still uncharted. However, we in the field have found that many of these lead us away from the work of visualization itself and into other related territories.

Elijah Meeks dropped the gauntlet on Twitter recently, with a blunt assessment of “the state of dataviz” as more likely to turn practitioners away from the field than not:

This spark set off a tinderbox of responses within the community, on Twitter and in the D3 Slack team. We took the opportunity to harness the energy around these conversations to push the conversation to a more public and persistent platform, in an effort to bring the profession of Visualization to a higher level, one in which we determine the value of our work and present our findings to those who would benefit.

The following is a collection of anecdotes and insights from, well, anyone who stuck their hand up when we asked for them. We hope this post is the beginning of a publication that evolves along with the profession, and we invite you to ride along with us.

Contributions by:

• Elijah Meeks (Netflix)
• Sebastian Gutierrez (DashingD3Js.com)
• Kerry Rodden (Google)
• Shirley Wu (DataSketch.es)
• Moritz Stefaner (Truth and Beauty)
• Eric Socolofsky (Stamen)
• Jon Sadka (Uber)
• Nadieh Bremer (DataSketch.es)
• Susie Lu (Netflix)
• Lars Verspohl (Datamake.io)

Elijah Meeks

When I was at Stanford, I built data visualization products to support analysis and publication for research. At Netflix, I do similar work but for business intelligence. While the products look very similar, the goals and the measures of impact are not. At Stanford the implementation of novel techniques was itself considered a valuable achievement but the lack of emphasis on measurable success made it feel indulgent. At Netflix, arguments for untraditional data visualization are viewed skeptically and custom data visualization is constantly under pressure to be overly simplistic.

During my time in both positions, the number of self-identified practitioners has dramatically increased and publications like the Information Is Beautiful awards have raised the visibility of novel, complex work. Dedicated professional data visualization roles have emerged, and data visualization responsibilities in other roles have become the norm. Despite these changes, we in industry are still operating on archaic precepts of data visualization developed when it was an elite skill used to make static charts for busy executives or promotional material. When we find ourselves unable to find professional success building the kinds of data visualization lauded as cutting edge, we leave data visualization roles for other positions where we can be rewarded for innovation.

Sebastian Gutierrez

Data Visualization is older than you think. Small multiples were introduced by Galileo in 1610 with Sidereus Nuncius and then shown closer to modern use by Christopher Scheiner in 1630 referencing data from 1611. Michael Florent van Langren, in 1644, created the first visual representation of statistical data. In 1801, William Playfair (creator of line graphs, pie charts, and others) used a “bespoke” visualization created with circles, pies, and lines to show taxes per capita in various nations in order to argue that the British were overtaxed, compared with others. Statistical graphics and data visualization have been used for centuries to communicate important data insights to those whom are less data-literate than the creator of the graphic.

You or someone you work with needs to understand data that has come into contact with your organization. The organization that better understands and meets the needs, wants, desires, and goals of the people whom they serve will win in the short run, the medium run, and the long run. It is therefore imperative that the organization focuses as much as possible in understanding what the data says about those needs, wants, desires, and goals.

Which brings us to today — where data visualization professionals (professionals focused on communicating things about data to further the organization’s goals) are not valued by the organizations that employ them. This then drives data visualization professionals to the data science profession or the UI/UX professions. Isn’t this surprising? That a valuable field that is centuries old is being shunned by the people whom it would most benefit?

Kerry Rodden

Data visualization requires a hybrid set of skills that tend to be scattered across a set of more conventional job roles (UX, data science, frontend engineering). That can be tricky in a large organization, because your performance tends to be evaluated against the expectations of whichever is your primary role, creating some pressure to stick to it.

My personal experience: at Google in 2007, I helped to create a hybrid role (quantitative UX researcher) that is a cross between UX research and data science, and successfully made the case for incorporating data science into the more typical UX research skillset. I’ve mostly stayed in that crossover, keeping data visualization as a thing I use sometimes, to create engaging and interactive deliverables from analysis. I’m mostly happy with that, but I do keep wishing I had more time to improve my web development skills, so I could make more usable and robust visualizations. In practice I get there through collaboration with others who have different subsets of the hybrid skillset.

Shirley Wu

So maybe it’s because I freelance now and don’t actually talk to that many people (if any) on a day to day, but I hadn’t even been aware of any problems until Elijah had tweeted it. I’m a software engineer by training, and I fell into data visualization accidentally when my first job asked me to learn D3.js. I stuck with it because the Bay Area D3 User Group was by far the friendliest and least bro-y meetup I went to my first year in SF. And I love the fact that data visualization in the web is such a young field; it’s not something we go to school for, so I’ve met practitioners from many different backgrounds — biologists, astronomers, programmers, data scientists, designers, journalists, statisticians — and each of them approach their visualizations from a perspective I would have never even considered. I find that beautifully exciting.

But being a young field does have its challenges, and I think the biggest is education. Many companies are now becoming aware of the importance of visualizing their data, but they aren’t quite sure yet what it can do for them. When I was at Illumio, my job function was almost exclusively to build out the data visualization part of our front-end solution. At one point, we had five front-end people (and a whole scrum team) dedicated to the effort — so this was a company that realized the value of visualization in their product. And a lot of customers realized the value too, when they could see where traffic was coming from and going to in their data centers; it certainly helped to sell the product. But the hard part came when my teammates and I wanted to go beyond the bar graph — or even the network graphs that most IT and ops professionals were so familiar with — because suddenly there was a learning curve to the more esoteric visualizations. It was harder to get everyone on board with the ideas, because there was a risk to investing in them, and the rewards weren’t as clear-cut.

I firmly believe that once we have a few successful use cases of bespoke visualizations in product, that the path forward will become easier. And I hope that as data visualization roles become more widely accepted in industry, that our primary responsibilities will shift from just building the visualizations, to recommending the most effective visualizations for the use case — esoteric or not.

Moritz Stefaner

I started being serious about data visualization around 2005, when the field was still extremely niche and have seen a massive evolution of the field over the last 12 years, with many people from all kinds of background streaming into data visualization. Why? Well, the equation is simple — there’s solid agreement that data is kind of a thing right now, and also that data visualisation makes data more human (whatever that means) — so… that seems to work.

All these years, I’ve been working as a freelance Truth & Beauty Operator on ambitious projects to bring data explorations into communication contexts, and I must say, I never really had a shortage of projects. There is still a huge need for skilled designers who can take a data source, and a brief on information needs and / or communication goals, and turn them into an intriguing, truthful, unique and aesthetic data experience. Most of my projects are public facing, communicative in nature, but I’ll also do internal, more utilitarian work, such as helping identify fraud networks for a large tech company or help the German railway system optimize passenger load in trains.

So, things look pretty good on this end, but as Elijah points put, there’s one nut we have not been able to crack: anchoring bespoke data visualisation as a full fledged profession in industry.

Often, when I speak at companies, the audience loves creative approaches to data visualisation, but at the same time, I am confronted with the same questions:

- “Are these custom interfaces really useful or just a gimmick?”,
- “How can we make this scale?”,
- “Will this last, or is there something new and fancy next year?”.

So, yes, I do see there is a gap and a problem to be tackled. But — I don’t necessarily see this as due to deficiencies of the data visualization community. Keep in mind — it’s a young community in a very fresh field, and it’s only natural that we are having a hard time between much more established fields like UI/UX design and computer science and engineering. We all know that there are many different flavors of data visualization, and we also know we bring something unique to the table — so let’s work together to convince the rest :)

Eric Socolofsky

My first forays into visualization were as an artist, at New York’s Eyebeam. From there it was into exhibit design at San Francisco’s Exploratorium. We all pretty much bootstrapped ourselves at the Exploratorium, so I was forced to learn and use tools like MongoDB and Java (don’t ask) to process data, to complement my design knowledge and front-end skills in Processing and D3.

At Stamen, there’s more of an established pipeline: teams generally comprise one Design Technologist (visualization + tech + UX) and one Designer (UX + visual), with support staff as needed; when possible we lean on our clients to do heavy lifting with the data. As part of the “discovery” (prototyping) phase, we often have to massage data, but due to the speed of most client services projects, that’s usually a block of D3 code and you’re off and running. Positioning Stamen as a capital-D(ata) capital-V(isualization) firm means that we can concentrate our efforts on visualization, user experience, and data stories, with analysis and data management on an as-needed basis. Still, understanding how to handle and manipulate data is a skill fundamental to getting the data visible and legible.

Jon Sadka

Four years ago, I was so moved by a visualization that I took a leap of faith and switched professions; structural engineer to software engineer. I aspired to create stunning visualizations with fanciful transitions and stunning color gradients. One could stitch all my visualizations together and get lost walking across fields of curves, stacks, and bars I created. Creation of this landscape drove me to where I am today.

Today I work as part of the Visualization team at Uber. At Uber, dynamic marketplace information and movement data fill the databases every fraction of a second. Desks are filled with backend teams dedicated to curating, cleaning, and serving up perfectly packaged datasets. And best part of all, teams receiving us visualization engineers will almost always be staffed with a product designer focused on the visualization component itself.

However, even with all these perfect pieces in place, I spend a good chunk of my time as a front-end engineer wiring the visualization to the surrounding components, adding filters, saving user preferences, etc. While the front end component of my work is not data visualization per se, it is essential for production level services and actually enhances the importance of the visualization. It also means my barrier to entry for adding new and experimental visualizations is much lower. I see this as an advantage because the best way to convince someone they need a visualization is to show them a working prototype they can’t unsee.

Nadieh Bremer

I actually sort-of took the reverse route mentioned in Elijah’s tweet. Starting out as a data scientist/consultant/astronomer I slowly came to love the visualization of data analyses more than doing the analysis itself. Teaching myself the best practices, web languages & D3 in my free time I started to take every opportunity during my day job to add in more “unusual” (and hopefully more effective) charts. Clients were intrigued & enlightened by the (interactive) charts, making my managers happy, resulting in me getting more time to create extensive visualizations during the day as well.

Two years later and I’m working part-time as the (only) data visualization designer at Adyen (located in Amsterdam) while also freelancing under Visual Cinnamon. Yes, I do still work hands-on with data, making initial data-dumps to build the dashboards with. But preparing data has always been a great way for me to understand the intricacies of it, which in turn informs my design. Yes, I still have to convince the occasional colleague that visuals are better than tables, and sometimes an unusual chart even works better than bars and lines (typically using a case, showing a comparison and pointing out the improvement works like a charm). But overall Adyen has been very good to me, inviting me to find new ways to visualize their data and I’m helping out wherever there is a need to visualize; from dashboards, to marketing to client presentations needing some charts.

But if we could figure out a way to attach some form of “improvement measurement” to our visuals, like UX/UI can do for example with number of clicks/increase in views or orders, that would be the most important thing to make my professional life easier I feel…

Susie Lu

I first learned how to program right out of college at Accenture’s Technology Labs. After learning the basics of distributed systems and shell scripts I found myself in a position to refocus my career on front-end work, specifically data visualization. Today, I am a Senior Data Visualization Engineer at Netflix. It’s my first job with data visualization as my main role, and working on a team with other vis engineers and web developers.

Recently I’ve focused in on design, from application architecture to visual design; I now see data visualization as simply a subset of design. Design is an area of growth and of learning for the data community as a whole. In my public work, my most impactful public projects have been d3-legend and d3-annotation built with design at the forefront, to remove the load of making repeatable components (machine work) so we can focus on communicating the data (human work).

Lars Verspohl

I’ve been working in research for a good while trying to get people to understand sometimes complex models. Having lost some audiences with long and static power-pointed sittings we started to build interactive, animated and scrollytold visuals. The main result was an audience leaning forward to discover rather than leaning back to absorb. These effects — not measured but clear — could have turned this visualisation skill into a full time role. They didn’t, and I think this is why: for many companies data visualisation is still a hygiene factor; necessary but not crucial. The key interest is in data engineering and analysis to produce results rather than communicating them. As data visualisation is a required job spec skill (when I started, it wasn’t even that) it’s covered. Tick.

However, this appears to be changing. As data itself is creeping up in rank and attractiveness within the average company, visualisation is as well. A few years back, budget and decision makers enlisted data scientists to run departments. Recently, we saw some of them surfacing from their rabbit holes with tangible results in data engineering, synthesising and analysing. They will have chats with their Presidents and CEO’s about how best to communicate the data — internally and probably more importantly externally. I’m not saying it’s a given that they will turn the visualisation skill into a must-have department, but datavis will hopefully get more of a foot to stand on.

Some of us couldn’t wait and are now working as independent data visualisation designers and developers. I believe we’re filling in the gap between companies’ growing visualisation needs and their resources. While that might be naive, it works for now. But equally, I hope discussions like this can help to argue the case for data visualisation beyond the skill…