Visualizing with Filters and Blazes

Customization and highlighting in EpochShift

Tradespaces are big.

OK, maybe that should say “tradespaces are (often) big”. But all engineers and analysts have been there before: staring down a dataset with hundreds, thousands, even millions or more datapoints. Information overload is a threat to our ability to extract useful insights from data, and the more data we have, the more unwieldy it becomes to explore in detail. How does EpochShift help you manage your data so that you can always create clear and concise visualizations, as well as calculate metrics and statistics when and where you need them?

Let’s introduce two core tools in the EpochShift toolkit that do exactly this: filters and blazes. To illustrate, I’ll use an example decision problem of “buying a used car”. This dataset has 274 cars in it — not too many, but certainly enough that I don’t want to spend the time to examine each car in detail. How can I go about highlighting the data that is relevant to me?

We’ll start with filters, because many people will already be familiar with this concept from other analysis software. In general, a filter takes a large quantity of data and reduces it to a smaller quantity. Maybe I want to look at only the subset of cars that have at least 40 inches of headroom, because I’m tall and need my space. In EpochShift this is an alternative filter, because the individual cars are the alternatives of my dataset (what I am making a decision between) and I want to filter down to a subset.

To make an alternative filter, I open the Filter Manager, select the Alternatives tab, and click one of the “New Filter” buttons at the bottom. In this case the filter I want is a logical statement (“at least 40 inches”), so I’ll use the “New Filter (from logic)” option.

The Filter Manager (before creating any filters)

After giving my new filter a name that describes the subset of alternatives it will allow through (“Good headroom”), the Filter Logic window enables me to define my logical condition (Front Head Room >= 40). Immediately, I receive heads-up information about the yield of this condition: 74 cars will “pass” the filter, which is about 27% of the cars in my dataset. I’ll save this filter as-is, and leave a deep dive into the more detailed settings of the Filter Logic window for another time — but there is a lot more power in this tool, including the ability to combine multiple conditions with logical operations as well as create conditions comparing multiple fields in my dataset.

Using Filter Logic to isolate the subset of cars with Front Head Room of at least 40 inches

Having saved, my new filter appears in the Filter Manager, where by default it is set to active. Other EpochShift windows will now have access to this filter via their Alternative Filters menu, and will grab the data only for those alternatives that “pass” all active filters. Perhaps I open an Epoch Scatterplot to check out the result: I see from the yield indicator on this plot that only 74 cars are being shown.

A scatterplot with my new filter active. The yield of the plot matches the yield of the filter since it is currently the only thing reducing the dataset. I could toggle my filter (and any others I create) on/off in the Alternative Filters menu located at the top of the window.

Quick recap: filters offer a way to reduce the amount of data a given visualization/analysis is using, the Filter Manager allows us to create and save any number of different filters, and then we can toggle them on/off for any EpochShift window using the filters menu. Pretty neat!

But what if I have some brand loyalty to Toyota? I want them to be highlighted in my scatterplot, but I don’t want to create a filter such that I only see Toyotas, because I want to see where they are relative to the rest of the dataset — after all, I’m not locked in. And I can’t simply color this scatterplot by Make, because there are ~30 different manufacturers in the dataset and Toyota won’t stand out from the clutter.

Hmmm… this wasn’t a very effective way of highlighting Toyotas

Now it’s time to shine for a blaze! A blaze is sort of like a reverse-filter: where filters take things that we normally do to the full dataset (e.g., points on a scatterplot) and reduces to a smaller subset, blazes allow us to mark a subset of the data to be given a special treatment that normally is given to none of the dataset.

Many EpochShift widgets allow you to create blazes as you explore: for example, I can easily create this single-manufacturer blaze from the Alternatives Overview widget by clicking on Toyota in the side panel and then clicking the “Create Blaze” button over the resulting bar charts (which helpfully shows me that 14 alternatives were selected). I could also select multiple Makes here to highlight them together, if I wanted, or restrict myself further to Toyota SUVs by selecting that value for the Type variable.

Going through Alternatives Overview is one way to quickly create a blaze on a subset of alternatives

EpochShift prompts me to give this blaze a name (“Toyotas”), optionally provide some notes as to why I am saving it, and to define a preferred visual style for these alternatives including color, shape, and size. Once I save, all open windows that use Alternative Blazes update to show this new special marker, including my original scatterplot! The Toyotas are now marked with a blue triangle, regardless of the color/shape settings of the scatterplot.

My scatterplot, now with a blaze highlighting Toyotas using blue triangles

Look at that: now I can clearly see where the Toyotas are in my car tradespace! Blaze markers are plotted in order of the number of alternatives in the blaze — rarer blazes on top, so blazes that highlight just a few alternatives don’t get covered up by larger ones. And just like with filters, we can create any number of blazes and turn them on/off for each individual EpochShift window with an Alternative Blazes menu: giving us full customizability for each plot we generate.

Plus, EpochShift always has a special blaze named “all” which refers to the whole dataset: those many circles I was seeing plotted before creating the Toyota blaze was EpochShift plotting “all” for me. If I turn the “all” blaze off, I get a plot with only my manually-created blazes and nothing else — a useful way to quickly toggle between viewing the entire tradespace and only known points of interest.

The same scatterplot with “all” toggled off — now showing only cars I have already called out as important using a blaze. And if you are thinking “wait didn’t you say there were 14 Toyotas but I see only 7 here?”, don’t forget that I still have a filter on: there are 7 Toyotas with at least 40 inches of headroom.

Notice how EpochShift colored the Alternative Blazes menu? That’s a reminder for me that I have modified the settings in this window away from the global default: I toggled off “all” for this plot only, and I don’t want to forget that it may not match any other plots I have open at the same time.

Quick recap number 2: blazes offer a way to highlight subsets of data with custom markers, we can create and save blazes as we explore in a variety of different ways, and then we can toggle them on/off for any EpochShift window using the blazes menu. Also pretty neat!

EpochShift is here to do the heavy lifting for you: remembering all the different ways you want to markup your plots and filter your data, all while letting you individually customize which are active for any given plot via the menus located at the top of the window. The Blaze and Filter Manager widgets store this information: you can go there to create more or edit existing blazes/filters, as well as to update global settings for which are on by default when a new window is opened. And there are even more ways that blazes and filters can interact with different EpochShift widgets, including creating Epoch and Era blazes/filters — so look around and see what you can find. With a little practice, you’ll be making fancy plots like this in no time:

An example scatterplot with blazes customized to (1) fade “all”, (2) call out the Pareto set, and (3) highlight the comparison between Toyota vs Chevrolet (blue vs red fill) and SUVs vs other types (cyan vs green edge). That’s a lot of information! Looks like Toyotas are generally cheaper and less powerful than Chevys regardless of type — with the exception of that one small Chevy SUV with <200 horsepower. Maybe I should look at that outlier in detail next?

Good luck exploring!