Fashion E-Commerce: Using Computer Vision to Find Clothing that Fits Like a Glove
It was October 2019, and my cousin’s wedding was taking place in about a week. Like everyone else, I was faced with the dilemma — what to wear? The struggle was real.
So there I was, in the mall with my brother picking out shirts, because everyone knows you can’t go wrong with a great shirt right? I picked a few and was searching for a place to try them on before deciding which ones to buy. Just because it looks good on the hanger didn’t mean it would look good on me.
Into the fitting room
I tried them on, and after checking myself out (and getting my brother’s nod of approval) I bought the ones that fit just right. The following week, I wore one of them to the wedding. I looked great. Just call me GQ.
Like many guys, I don’t particularly like shopping, but I tolerate the fitting room because it gives me the opportunity to see how everything looks before going out in public. Getting the right clothes the first time means not having to head back to the store for a return and round 2. I’m efficient like that.
It goes by different names — changing room, dressing room, trial room. You know, those small cubicles with the big mirrors and lighting that makes you look like an extra in The Walking Dead? The place where before you enter, the staff stops you and barks “Sorry, you can only bring in 4 pieces at a time”, and you’re like — “Can I please bring in all 7? I won’t take long!” — just so that you can play dress-up for the next 45 minutes?
They were a necessary evil. But then 2020 happened. With quarantines, social distancing protocols, and strict sanitation rules, shopping for clothes just got a lot harder.
A Problem of Proportions
It’s not like 2020 forced us to spin our own cloth. It was simple enough to order clothes online, but it created a whole new set of issues.
Most of us are used to shopping online. Electronics, household appliances, books, or kitchenware is all pretty standard. It’s pretty straightforward to purchase a smartphone, laptop, or other gadget— you only need to worry if the specs meet your needs, and if it’s got decent reviews. Ditto with books, handbags, or even jewelry.
But clothes? Reviews alone won’t cut it.
Shopping for clothes gets old — even for the most avid fashionistas. Finding something you like is the hardest part. Once you find it, things get simple. Grab your size, try it on, check yourself out.
With online shopping; however, everything gets flipped. Searching for just the right thing is easy — thanks to the ever-improving search features from all the mainstream e-commerce platforms. But getting the right fit? It’s not as easy as it looks.
It’s no wonder that the average return rate for online apparel shopping is a whopping 32%, and it burns a hole in the pocket of e-commerce retailers. Here are some wrinkle-inducing stats:
- Almost 1 out of every 3 online apparel purchases gets returned.
- In 2019, about $260 Billion worth of retail goods were returned, out of which $30 Billion were from clothing.
- The cost of returns is estimated to be nearly $1.5 Billion. The cost of each return is between $5-$15.
What are the top 3 reasons people return clothes bought online?
- Incorrect sizing is the biggest factor, with 43% of total online shoppers citing poor fit.
- Quality and color mismatch accounts for 29%.
- 22% of shoppers want to recreate the fitting room at home.
Despite these stats, online shopping isn’t going away.
More users are shopping online post-COVID:
- 27% of customers switched to online shopping due to COVID-19 in USA.
- 12% of customers would prefer to continue purchasing online even after COVID .
So needless to say, online shopping is here to stay, leaving apparel retailers with a big problem. Sizing.
Incorrect sizing contributes $1 Billion in returns annually and affects inventory planning leading to heavy losses for e-retailers.
Sizing variance is not a mere wardrobe malfunction, it is a huge problem.
Egen decided to iron this one out.
Introducing SmartTry
How do you determine fit when you can’t physically handle the product? The next best thing is to compare it to clothing you already know fits well.
That’s what SmartTry does — it enables users to compare their existing best fitting clothes to new ones they want to purchase online.
How does SmartTry work?
Say you’re trying to buy a T-shirt online from Amazon, or from mango.com like the below screenshot —
You find the ultimate white T-shirt for the ultimate in effortless cool in your next Zoom meeting. But you’ve never bought a T-shirt from Mango before, so how will you know how it fits. (We’ll spare you the rant on inconsistent sizing. You know it. We all know it.)
Returning things during COVID is a whole new level of inconvenient. So what can you do to make sure you get the right fit the first time?
Here’s where SmartTry’s user experience is guaranteed to knock your socks off!
Features
Clicking on the green SmartTry button opens the SmartTry plugin in a popup on your screen:
- The SmartTry plugin shows you the T-shirt or the outfit that you want to buy, along with your previous best fits for you to compare.
- Clicking on the Add new apparel to compare icon lets you upload a picture of an outfit that you currently own, into the SmartTry platform. You’ll use this to upload a photo of the T-Shirt that is your current best-fitting favorite.
- Once uploaded, it gets added to the Recommended Items section and to your Collection, with a button underneath that says Compare. Clicking on the Collection tab on the left will also show you all your previous uploads neatly organized within categories:
Finally, when you click on Compare, SmartTry runs its magic in the background and shows you how good of a fit your selection is!
- The blue T-shirt border represents your current best-fit that you used for the comparison, while the black border is the T-shirt that you want to buy.
- On the right, you’ll be able to see precise measurements for the T-shirt you selected and your current best-fit. This helps you easily compare the differences between the two. (Let’s face it. Every site has a size chart. No one is whipping out a measuring tape before clicking add to cart. Problem solved.)
- In this example, the Small size seems a little too small — a 67% match. SmartTry makes it easy to compare sizes:
Medium size is a 94% fit.
You can now buy the right size, knowing fit’ll fit you to a T!
Life ain’t perfect, but your outfit can be!
Unmasking the Magic
If you’re a programmer, I’m sure you’re wondering by now —
How does SmartTry really work behind the scenes, and how do you build it?
Well roll up your sleeves, because we put together a tutorial for you to follow along.
What you will learn
Whatever the mode of user-interaction may be, the power of SmartTry comes from Computer Vision techniques, which allows it to compare two different outfits and analyze the difference in measurements. I’ll walk you through how to build this algorithm.
- Use OpenCV, one of the most popular and widely used computer vision and machine learning libraries.
- For writing code, use a Jupyter-Notebook, an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.
A word to the wise
Some knowledge of Python is essential in order to understand this tutorial. If you haven’t worked on Python before, I highly recommend this free course on Introduction to Python, before proceeding.
Why Python? Because it is fairly easy to learn and the Python ecosystem has a lot of tools and libraries for computer-vision, machine learning, and more. Once you’ve built the algorithm, Python also has several web-frameworks that allow you to build a service around this feature.
Tutorial
Go to this github link, and follow the instructions to download and install the the project dependencies. Then, run jupyter-lab in your project root and create a new Python3.x notebook file. Next,
Head over to the Jupyter tutorial and follow the steps outlined, to build your very own program that calculates accurate measurements for a given T-Shirt!
Alternatively
Head over to the below Binder link and execute the code right within your browser without the hassles of installation.
The smarttry.ipynb notebook has the complete source code with instructions for you to read. You could choose to run this directly, or write code on your own in a new notebook file as you follow the tutorial.
Slides
If you prefer taking just a quick peek into the logic, here’s a short slide-deck:
So go ahead, put on your thinking cap and have fun!
The first version of SmartTry that we developed at Egen achieved an accuracy of 91.7% for clothing measurements! Hopefully, you’ll be able to use this as a performance benchmark when following the tutorial and writing the program. I’m curious to know what your accuracy score looks like!
Next Steps
Once you’ve developed your program, you can integrate this into a web-service and expose it via an API. Here’s the architecture implementation that we designed at Egen:
It doesn’t stop here! The future scope for such a feature is immense:
- On websites, SmartTry can be easily integrated into any existing e-commerce platform — as a popup, or an installable plugin, or a platform integration, or even as a browser extension.
- Integrate SmartTry into mobile applications, which will allow you to upload photos of your outfit more easily.
- You could maintain a common SmartTry profile that’ll exist across multiple e-commerce websites.
- A history page to quickly review past comparisons.
and more!
Potential Revenue Models
Being an intermediary between different parties that interact with a product is a model that a lot of businesses are exploring. SmartTry being designed as an easy-to-add plugin with a high accuracy makes it a really good contender for adoption in new businesses and existing e-commerce platforms. Here are some revenue models that can be quite viable (although they’re by no means an exhaustive list):
- At-cost SaaS model for online shopping. The pricing could be based on the number of API calls made to the SmartTry platform, for example — $0.001 per 1 API call.
- Size and shopping data of users and their different demographics
- Sending traffic from our platform to “preferred” e-commerce websites
- In-store enterprise yearly licensing model
- A freemium model with basic features accessible for free, and upgrades or more advanced features available at a monthly subscription.
Summary
Convenience is king, especially in fashion e-commerce. Solving the sizing problem can help businesses reduce nearly $1 Billion USD in returns. Reduced apparel returns would also translate into reduced carbon emissions, logistics, and waste.
Thanks for reading!
If you’d like to know how SmartTry can provide value to your business, reach out at digital@egen.solutions or ashwin01h@gmail.com.
All credits for developing the idea and concept goes to the SmartTry team at Egen, consisting of Arif, Gagan, Anmol and Aatman.🙌🏼
