Fashion Science takes on Seasonal Color Analysis
Turns out ‘seasons’ just aren’t found in the data
Wear the right color clothes and be more attractive!
That’s the allure of seasonal color analysis. By appropriately placing you into one of four seasons — spring, summer, autumn, and winter — each has its own palette of colors appropriate for you.
This paper applies Fashion Science to explore a simple concept — do people really break down into four segments that map to the traditional seasonal assignments? Fashion science for my purposes includes image processing, machine learning, and data analytics but also traditional artistic and psychological concepts.
Using over 100,000 ‘portrait selfies’ collected from our current iPhone application (All Eyes on Hue) we break each selfie into key body parts — skin, hair, eyes, and lips. (By ‘portrait selfie’ I explicitly mean a portrait at high resolution — we need lots of pixels on the iris!) We then apply cluster analysis techniques to group selfies together based on the similarity of their body part colors. While there is clearly nice clustering of two clusters and four clusters — these don’t correlate at all to the seasonal descriptions.
With a new, science-based segmentation there is a solid base on which to better view a data approach to market segmentation for fashion design, marketing, and retailing. Further work is also taking us to revisit personal color analysis — the creation of personalized palettes based on color harmonies from an individual.
Background
Seasonal Color Analysis (SCA) is based on the concept that harmonious colors will enhance the natural beauty of the individual. The seasonal aspect relates to placing all humans into one of four segments, named after the seasons. There is guidance for users to self identify, though historically users have been placed into their season by expert practitioners. Interestingly, these practitioners often disagree into which season an image is to place, especially in edge cases.
Given the prevalence of smartphones with cameras — isn’t this a natural area to attempt to automatically identify a person’s season?
So a group of us researched, designed and deployed an iOS app that did just that, i.e., take a selfie portrait and identify that person’s season. Further, we showed that person their seasonal palette and also provided them augmented reality capabilities — such as a live video stream from their camera’s phone showing good, and bad colors highlighted. This is useful in a location such as a clothing store.
There are challenges in this version of the app, most importantly a lack of repeatability in seasonal assignment due to variations in the lighting and composition of the portrait selfie. We have addressed this in the analysis shown below and plan on releasing a more effective app in the near future.
Through the usage of our app, we have over 100K portrait selfies of humans. This set is not representative of the distribution of humans across ages, gender, geographies, and races. One useful way to think of the portraits is as ‘fashion interested individuals with an iPhone.’ These portrait selfies were collected only for analytics. The users agreed to this provision prior to providing their selfie portrait and these are not available or shown at the individual level.
Given this data set of portrait selfies, what can we learn? Let’s test the hypothesis that there are indeed seasons to classify individuals in our data.
Classifying a Portrait
The essence of evaluating a portrait lies in breaking the portrait image into body parts — skin, hair, eyes, and lips. The colors in each body part are representative of this individual and guide us to their color assignment. This is a crucial assumption — that a seasonal color assignment is based on key body part colors.
In short, this process is a simple pipeline:
- Filter out poor images — those overly bright/dark, those with deep shadows, wearing glasses/hats, and more.
- Automatically white balance the image — through extensive training of a machine learning network we achieve good results in showing a ‘real’ human. As an exercise, look through say a set of Facebook selfies and see the wide variation in apparently human skin tones!
- Identify body parts — use image processing and additional machine learning to identify the skin, hair, eyes (iris), eyes (sclera — the whites), and lips.
- Classify the colors of these body parts — both the mean color and a ranking of the representative colors so as to grab the richness of more subtle secondary colors.
That’s it — we now have a numerical representation of a selfie portrait ready for analysis.
Cluster Analysis
Cluster analysis is meant to group portraits so that similar portraits are in clusters. This is a common task in data analysis. There are many approaches and algorithms available for cluster analysis. The input to our cluster analysis is a set of colors for a set of body parts. The output is a set of clusters and the identification of each portrait to its cluster.
The following result uses three body parts — the skin, hair, and iris. These were clustered based on their mean color in the Lab color space rather than RGB. As Wikipedia says, Lab “was designed to be perceptually uniform with respect to human color vision, meaning that the same amount of numerical change in these values corresponds to about the same amount of visually perceived change.” That said, we’ve evaluated with multiple body parts and various color spaces to achieve similar results.
The first approach to identify clustering is known as the elbow approach. Clusters from 1 to 30 are evaluated as to how well they cluster. The best representation of the number of clusters is where there is an elbow in the graph.
Seasonal color analysis would have us believe that there should be clear and distinct clusters of two — warm/cool — and four — the seasons. In this graph, it really doesn’t pop out as to where the ‘elbow’ is — 2, 3, 4, or 5 clusters?
Another approach is hierarchical clustering. This identifies clusters based on the distances between each model’s body part colors. The output is shown as a dendrogram — a tree structure. In the chart following the Y-axis is a measure of closeness between the portraits. The way to interpret this chart to identify clusters is to find large Y lengths where the clusters are consistent.
Using this method one sees a clear interpretation for two clusters followed by four clusters. That’s keeping with a seasonal color interpretation.
One concept in seasonal color analysis is that the clustering to two segments could/should be based on the warm/cool colors of the skin. On the color wheel warm is often related to the top of the wheel — reds to yellows — while cool is the other side — or blues to purples. Traditionally ‘warm’ covers Spring and Autumn while ‘cool’ contains Summer and Winter.
In a representative set of our data, the hues of skin seem to be all warm, i.e., (surprise!) flesh-colored, and the clustering seems to be evenly dispersed as the hue changes. In other words, the two cluster model doesn’t help with skin colors/hues. Though note that this association of temperature with Hue is often claimed to be psychological rather than physical, referring to an ‘undertone’ to ascertain the temperature. This undertone does not appear in the data.
Four Clusters — are they Seasons? The big test
If we take the four cluster model from the dendrogram clustering graph previously, we can test if those map to seasons. The actual mean colors for each of the four clusters are shown pictorially at the bottom of that chart.
To test our clustering we mapped our clustering assignments against the assignments of a popular seasonal color expert. Since we don’t know how the seasons map to these newly found clusters we need to try all combinations, e.g., try cluster one as Spring, then cluster one as Summer, etc. All in all, there are 24 possible combinations of our clusters to possible seasons. A perfect mapping would be 100%, while a completely random mapping is 25%. From experience, I would hope for an 80%-90% mapping to claim success.
For the best mapping of our clusters to the experts season classifications, there is a matching percentage of 36%. This is not a credible mapping.
Ignoring the total mapping — where each season is mapped to one cluster, but focusing only on one season at a time — the best mapping is Winter to cluster two — which by the way looks somewhat appropriate to the eye. This is followed by Summer to cluster three. However, while a perfect mapping would again be 100%, the mapping percentage is still only 47% for Winter to cluster two, and 45% for Summer to cluster three — still a poor, not credible result.
Summary
We have shown that there is, at best a weak linkage between our over 100K selfie portraits and the seasonal color analysis model — we are seeing above random assignment of 25% per class but nowhere near an expected 80–90%. This is shown by various clustering techniques, the application of a warm/cool assessment, and the application of an experts assignment of seasonal colors. There can be multiple ways to disagree with our interpretation, ranging from poor extraction of colors from selfie portraits to issues with our data set to others. We respect all challenges but reserve the caveat that this investigation is beginning — but with the precept of Fashion Science —that is a strong preference for fact-based analysis when possible.
This work contains strong implications for the fashion industry. Given a more precise and factual identification of market segments as defined by their coloration can allow finer tuning of design, marketing, and even retail.
We are continuing this work in various ways. An interesting way, as an offshoot of this work, is the creation of dynamic palettes — or palettes personalized to the individual as opposed to seasons or clusters. This further offers the advantage of tuning the palette to particular body part coloration, e.g., hair tints, lip gloss, etc. as well as varying lighting conditions, e.g., sun, ballroom, office, etc.
Stay tuned as we continue our research and sharing of datasets and code!