Why data is easier than gardening: using Kibana to understand art
For my birthday last year, a friend bought me an herb garden.
Fresh herbs! Tired of going to the supermarket to buy a gallon of basil when you only need a tablespoon? Just reach over to your own personal herb garden and take what you need.
There’s just one small problem: I don’t cook. I’d like to, but instead I buy the occasional cookbook, promise myself I’m going to cook more and then break my promise by ordering dumplings on the internet. And so the herb garden, instead of transporting me to rapturous heights of culinary excellence, becomes an increasingly dry and brown reminder of my inability to start cooking.
In this series, we’re writing about working with the Barnes Foundation to build a new way to explore their collection online. Previously we’ve talked about getting our data out of TMS and pushing that data into Elasticsearch. But now that it’s in Elasticsearch, what do we do with it? Or is the database doomed to become like my herb garden, an amazing resource that we don’t have the time to learn how to use?
Fortunately there’s Kibana. One of the main reasons that we decided to go with Elasticsearch is that the plugins and support tools are so great. Chief among those tools is Kibana, useful for monitoring, visualizing, and basically getting a sense of what our data actually looks like.
Painting by Queries
Once you’ve got your data indexed, the first thing you’ll want to do is ask some questions about it. “Who are all the painters in the collection?” for example, is something I wanted to know. Kibana has a tab called “Discover” that lets us explore the data in the index by sending search queries and adding filters.
To the left of this view there’s a pane titled “Available Fields,” which lists all of the fields in the Elasticsearch index. I’m interested in paintings, so I can click on the “classification” field and then add a filter for paintings by clicking on the magnifying class with the plus sign in it.
With this filter added, the number of hits, displayed in the top-left of the window, drops from 2217 to 961—the number of items in the collection classified as paintings. I may want to use this search later, to create an aggregation for example, so I’ll click on “Save” at the top of the window and save this search as “Paintings.”
Now I’d like to create a visualization, so I can group paintings by the person who painted them. I click on “Visualize” in the lefthand navigation menu, and then on the blue plus button to create a new visualization. There’s a lot of choices for the type of visualization to create here. I’ve heard somewhere that every chart has a purpose, except the pie chart which is always wrong. Whatever, I’m picking the pie chart. In the next view, I can click on the “Paintings” search that I saved before to create a pie chart from that search.
And lo, welcome to the least informative pie chart of all time.
If we want to make this pie chart a bit more useful, then we have to click on “Split Slices” to create some pie slices. Now under the word Aggregation, there’s a drop-down that lets us pick what we’d like to aggregate over. I’ll pick “Terms”, since I want to aggregate over the maker of the painting. Now the “Field” drop-down appears, and I can select “people.keyword.” Finally, I can press the blue “play” button at the top of the view to load the data into the pie. And then I get something like this:
Nice! Never has it been more clear how strongly the Barnes Collection favors Renoir. One thing I’d like to change, however, is the presence of “Unidentified artist” as an artist. Fortunately, it’s super easy to add a filter. We can just go back to the saved “Paintings” search, click on “people,” and then on the magnifying glass with the “minus” sign in it to add a filter that excludes this value for the “people” field.
Now when I save the search and return to the visualization, the pie updates automatically. Charles Demuth has joined the party.
Alright, so that’s pretty cool, but let’s get a little more advanced. What if I want to see a breakdown of how many paintings by each painter appear in each room? It’s time for a sub-aggregation, or an aggregation within an aggregation. This time, I’ll pick a bar chart, and I’ll start with the same saved “Paintings” search.
Again, this bar chart is not currently very useful. All it tells us is that all 900 or so paintings are indeed paintings. Let’s start by adding a bucket along the x-axis, aggregating on the “room” field.
Alright, so now this is showing us that Room 17 has the most paintings, with almost twice as many paintings as Room 13. But now for the really cool part. Press the button that says “Add sub buckets,” and then click on “Split Series.” Let’s add a sub-aggregation over the “people.keyword” field, limiting it to the top 5 painters per room.
How cool is that? We can see that even though Renoir has the most paintings in the collection, he’s totally absent from Room 17, which has more paintings than any other room. The king of Room 17 is actually our old friend Demuth. Let’s have a Demuth to celebrate.
The important takeaway here is that I was able to make these quick visualizations without having to know anything about Elasticsearch. I didn’t have to learn the query syntax, or write code for an Elasticsearch client. All I had to do was load up Kibana, click a few buttons, and I was able to make an interactive graph that taught me something interesting about the structure of the Barnes Collection.
If only using my herb garden were so easy.
The Barnes Foundation collection online project is funded by the Knight Foundation and our code is open source. Follow the Barnes Foundation on Medium.
