Coffee Data Science
Correlating Taste to Extraction Yield in Espresso
Examining Years of Shot Data
Often when I publish results related to espresso, I get a similar criticism: your taste is subjective (therefore just an opinion) and extraction yield (the most quantifiable metric for coffee) doesn’t correlate to taste; therefore you did a bunch of nothing.
I wanted to address this using data, and I have collected quite a bit over the years. Over 4 years of data collection, I have data on 3,000 espresso shots, and if there is a correlation between extraction yield and taste, I should be able to see it.
I will start with data cleaning and then discuss how I normalized the data based on roast. I have data across multiple machines, roasts, and techniques.
Metrics of Performance
I use two sets of metrics for evaluating the differences between techniques: Final Score and Coffee Extraction.
Final score is the average of a scorecard of 7 metrics (Sharp, Rich, Syrup, Sweet, Sour, Bitter, and Aftertaste). These scores were subjective, of course, but they were calibrated to my tastes and helped me improve my shots. There is some variation in the scores. My aim was to be consistent for each metric, but some times the granularity was difficult.
Total Dissolved Solids (TDS) is measured using a refractometer, and this number combined with the output weight of the shot and the input weight of the coffee is used to determine the percentage of coffee extracted into the cup, called Extraction Yield (EY).
Intensity Radius (IR) is defined as the radius from the origin on a control chart for TDS vs EY, so IR = sqrt( TDS² + EY²). This metric helps normalize shot performance across output yield or brew ratio.
Data Cleaning
I pulled all my data into a simplified sheet. The aim was for simplicity, so I didn’t keep all the rows. I started with just over 3,000 shots.
First, I removed all shots without extraction yield information or without taste scores. Then I removed any roast with less than 10 shots as well as a few other metrics (see table below) leaving me with 934 shots over 51 roasts.
Data Analysis
We can plot the raw data, unnormalized, and there is a general trend of increasing taste score with EY.
For each roast, I normalized the EY and Taste using Z-normalization:
I then converted these numbers back to their regular domains using average and std of all of the scores.
At a high level, I did a correlation. For most metrics, they seem to be correlated to taste at around 50%. 100% means they are identical, 0% means they aren’t correlated, and -100% means that are inversely correlated.
We can plot all the normalized values as a scatter plot as well as a bar plot per roast.
TDS seems better correlated with a smaller R² value for a best fit.
I particularly like Intensity Radius (IR) because it combines TDS and EY in a way that can normalize for output yield.
Most roasts have a higher correlation than 50%. We can also plot the TDS correlation vs the EY correlation to taste. A few roasts really go low or even negative.
Failure Analysis
Six of the roasts had a negative correlation to taste, and when I looked at each individually, they seemed to buck the trend. I didn’t see anything in this data aside from a few points that seemed like a good explanation.
Five of the six roasts were before the pandemic, so it is possible there was a connection to coffee roasting or some espresso process at the time. I didn’t remove them though because I think they are an interesting part of the story. Not all roasts have a good correlation between extraction yield and taste, but most do.
For the 2019 roasts, I had been doing a lot of paper filter experiments, so the variables were a bit conflated.
Overall, I found a correlation between my taste and extraction yield. Some roasts had a higher correlation than others. Everyone’s experiences are different, but most of the publicly available data on this topic are small samples across a few shots, not a larger exploration. This is why larger scale data is important.
At the same time, EY fluctuates quite a bit with respect to taste because taste can be noisy especially over multiples roasts, years, and techniques. There is definitely signal separable from the noise, and I’m hopeful others will collect similar data to help speak more broadly to extraction yield as a useful tool in the field of coffee.
If you like, follow me on Twitter, YouTube, and Instagram where I post videos of espresso shots on different machines and espresso related stuff. You can also find me on LinkedIn. You can also follow me on Medium and Subscribe.
