Why AI is Not a Magic Wand

A recommender system use case POC

Dev Shah
Dev Shah
Nov 13 · 4 min read

The Task: Proof of Concept (Treat with caution)

I worked on an ML project recently where the ask was to build a recommender system (Remember, it’s a proof of concept). A system that recommends users’ materials finishes & eventually furniture too based on their social media data like Pinterest, Instagram, etc. The targeted user base was mainly the A&D community (architects and interior designers).

Basic use case: User goes online, uploads an image of a room, furniture, or space of whose material they liked and find similar finishes offered by the company.

Typical recommender system for furniture

Ah, how I wish the ask were as simple as the image above. Well, it wasn’t, because it wasn’t just furniture that needed to be recommended but the finishes that are on top of furniture, walls, objects, etc. For example, the “yellow material” on top of the sofa in the above image.

Resources & Timeline

Deliverable: Prototype

Resources: 1 Developer

Timeline: 3 weeks


To build a recommender model, the first thing that you require is a polished dataset. The data layer which the client had was not very sophisticated, and much of the data was not clearly tagged with relevant features and attributes.

Considering the dataset, timeline, and the resources at hand, what one would expect from the AI-powered model would be that it shows some promise in terms of doing a similarity search based on straightforward input images.

Initial Prototype

Given it was a 3-week ordeal to build the proof of concept, with me as the sole developer, I decided to use MLaaS (Machine Learning as a service) models instead of building models from scratch (Duh! because it takes much longer to build models from scratch). It was a cumbersome task, given the complexity of input types.

Input types:

  1. Material/Finish Images (typical for Interior designers, consumers)

2. Mood boards (one image which consists of multiple finishes) for rooms or spaces

3. A room full of objects having materials/finishes on them

The model’s workflow — the input (x) was fed to a model that would run image recognition to isolate ’n’ objects from the image and then feed each corresponding object image (x’, x’’) into another model, which would run a visual similarity search on the database (which consisted of thousands of finishes). The output would be all finishes that matched the input image based on their similarity scores or accuracy.

AI — Magic wand, isn’t it?

To summarize, the client was happy with the model’s results and said it was ‘smart.’

However, in spite of warning them about how the model would fare for complex input types (like input #3), they still had some expectations that the model would be capable of doing justice for those inputs. Even after knowing that the data they provided was full of images that looked like input #1.

Takeaways for building AI-powered products

  1. AI or machine learning powered models are not magic wands where you would expect them to give good accuracy for custom scenarios
  2. Datasets for recommender systems should be well polished and have tons of images for each product taken in different angles, lighting conditions & environments in this case
  3. Building ML models for takeaway #1 take a lot of time, computing resources, and testing to have something even close to being production-ready. Make sure you have enough resources to build it in the first place
  4. We need to scale down our expectations drastically from AI
  5. Lastly, if I, as a human, can’t understand input #3 and what the required output (finishes/materials) could be, it would be naive for us to expect that a basic model could do that.

Feel free to drop your experiences in the comments section or if you want to know more about the workflow I designed and built for this prototype.

Towards AI

Towards AI, is the world’s fastest-growing AI community for learning, programming, building and implementing AI.

Dev Shah

Written by

Dev Shah

Full Stack QA & DevOps Practitioner. I write about tech, finance & life. Love to share knowledge. Writer @ NoteWorthy—The Journal Blog, The Ascent, Towards AI.

Towards AI

Towards AI, is the world’s fastest-growing AI community for learning, programming, building and implementing AI.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade