What we gain when we think beyond spike maps, choropleths, and curvy case charts
Since the start of the pandemic, data visualization has taken center stage in the effort to educate the public. Data has been used as a means of warning, informing, and educating. To be sure, this is important work; but in reporting the pandemic data, we also need to reinforce the humanity of the data. These positive cases are entire lives uprooted. With rapidly evolving aggregated datasets such as case counts, deaths, hotspots, and hospitalizations, it can be easy to forget these stories and focus instead on precision.
This disconnect goes beyond the emotional; it also hinders our ability to persuade…
Creating custom charts can help us better understand, validate, and improve imputation tasks in data science and machine learning.
TL;DR
Imputation is a useful tool for machine learning, but validating results can be difficult. We can improve imputation tuning by applying more advanced data visualization techniques as shown in this article.
Imputation in Data Science
Data imputation is a common practice in machine learning. At a basic level, imputation is the practice of replacing a missing value with an estimated value, usually through mathematical inference. Within machine learning, there are many useful applications for imputation, including:
- Validating an existing model
- Filling in missing values in raw data (data cleaning)
- Using small amounts of data to generate unlimited amounts of data like it
- Smoothing of…
Missing data is a common problem. Here’s how imputation can help.
For many data scientists, analysts, or engineers, dealing with missing or bad data is an everyday experience. A dataset could have missing values for a key period of time, or perhaps the dataset contains outlier values that need to be corrected.
Often, you may look for new data or work with small subsets of the dataset. However, the complete data set, after correcting for its limitations, can hold real insights. What can you do to preserve the integrity of the data while still mining it for useful signal?
Imputation can help solve this problem. Over the coming weeks, the Tagup…
We have a responsibility to visualize, communicate, and translate data accurately. Here’s where to start.
Originally published on March 16th. Updates included below, since the story is rapidly evolving…
- March 17: added National Geographic maps and an epidemic simulator.
- March 23: added NYT story about how the virus spread. Also a Medium article on 13 things to visualize about COCID-19 besides caseloads. Also Our World in Data’s COVID-19 research and statistics page.
This is a special edition of my weekly newsletter, Data Curious. Each week I share links for data visualization professionals; this week, I felt the responsibility to put my time towards specifically focusing on resources for the coronavirus, and to share it beyond…
I tracked my activities for a year. Then I visualized them. Here’s what I learned.
If you lurk in the shadows of the data viz world, you’re no doubt familiar with this kind of exercise. Health and productivity apps offer more ways to track daily data than ever before. As people naturally obsessed with quantifying, many data visualization practitioners have used it as an exercise in reflection.
This was my 2019. It was the first year I decided to intentionally track activities like working out, computer productivity, and listening to music. Most notably, I was inspired by the legendary Feltron reports. But at the time of starting this project, I was also still riding high…
Go easy on yourself. Try something new. Learn in public
Around two months ago, I decided to take part in an experiment: for 30 days, I would program a new creative sketch using only code based on a single word prompt. Every single day.
Some of you may be familiar by now, but I’m talking about the annual #Codevember challenge. I first discovered this challenge in 2018 but couldn’t muster up the courage to actually take part (crippling new-developer imposter syndrome). But 2019 was my time. For a while, I had been lurking in the shadows of the p5.js community, so I thought this would be a good opportunity to…
Two filters + one interactive area chart in roughly 25 lines of code.
I’ve been using Altair for over a year now, and it has quickly become my go-to charting library in Python. I love the built-in interactivity of the plots and the fact that the syntax is built on the Grammar of Graphics.
Altair even has some built-in interactivity through using Vega widgets. However, I have found this to be limiting at times and it doesn’t really allow me to create layouts the way I would want to for a dashboard.
Then I found Panel. Panel calls itself a “high-level app and dashboarding solution for Python” and it’s part of the HoloViz…
You’ve heard of user personas, and maybe even Jobs to Be Done. But what about QTBA? No? Good. I made it up.
When building data visualization products, we need a better way of understanding the people using it. While previous standard web UX methods can help in this process, I think that building a data visualization product with the purpose of delivering insight is fundamentally different than most web builds.
Let’s dive right in with an example from a previous design workshop. My team and I were starting to sketch out possible user flows for a client’s data-driven microsite (not to be confused with user journeys — a user flow is basically how a person gets from point A, to B, to…
We know data literacy matters. But visual literacy matters too. Here’s why.
Data is all around us, and the way people work has changed because of it. Companies are now investing more in roles like Chief Data Officer, building their data science teams and talking about things like “data literacy” in organizations. Data literacy has even recently emerged an important part of preserving democracy for future generations.
But what about visual literacy? Do we understand how visual mechanisms work with data? Some might argue these are one and the same, but I think there’s an important distinction to be made here. …
Here’s the best data-related stuff I found on the web last month: things to read, explore, analyse and learn.
Last year I began an experiment. Each week I found and posted my favourite data-driven stories, datasets and visualisations that I found online (usually via Twitter). My thought was this: if I spend half my day looking for visual inspiration online, then maybe other people would benefit from it too.
To some degree, I think this hunch was correct. I got some great feedback from Medium readers over the 20+ editions I published, and stored up an extensive archive of creative references in the process. Then I took a break…
And now it’s back! But in a different format. Data…
Benjamin Cooley
Visualization Software Engineer @ Pattern (Broad Institute). I design and develop charts, graphics, and data interfaces. Portfolio: bendoesdataviz.com
