Fashion Product Recommendation System Using Resnet 50
Fashion is an ever-evolving industry that requires constant adaptation and innovation to stay relevant and staying up-to-date with the latest trends can be a daunting task. In this blog post, I explore how ResNet50 can be leveraged to build robust fashion recommendation systems that can provide personalized fashion choices to customers, making their shopping experience seamless and enjoyable. It is captivating realm and one of the latest technological advancements in the industry is the use of deep learning algorithms for fashion recommendation systems. In this blog, we will explore how to use the ResNet50 model for building a fashion recommendation system.
For this point of time, we create one streamlit webpage on localsystem to see the 10 recommended fashion product images which looks similar to query image.
What is Resnet50?
The ResNet50 is a deep convolutional neural network that was introduced by Microsoft Research in 2015. It is a powerful model that has been trained on millions of images and can recognize a wide range of objects, including fashion items. This makes it an ideal choice for building a fashion recommendation system. For Detailed explanation check out my another blog on Resnet50.
We will not fully train resnet50 model from scratch(as it is already trained on imagenet dataset), we use transfer learning concept here, and just simply extract the features by passing dataset images(credits:Param aggarwal), dataset consists 44k images of fashion products.
We will pass 44k images to resnet50 model and take output of last conv. block before avg. pooling, resultant output would be 2d array of size 44k*2048 where each image has 2048 dimensional vector called features.
Why Streamlit?
Streamlit provides a simple and intuitive API that allows to create web applications using only a few lines of Python code.
Streamlit is an open-source Python library that enables developers to quickly and easily create web applications for data science and machine learning projects. With Streamlit, developers can build interactive web-based applications that allow users to visualize, explore, and analyze data in real-time.
Streamlit provides a simple and intuitive API that allows developers to create web applications using only a few lines of Python code. It includes built-in support for a wide range of data visualization and analysis tools, including charts, tables, maps, and machine learning models.
For further details check their official website.
Plan of Action
- Import Model: Import model resnet50 using keras with preprocessing_input to just process the image.
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input
############################ Defining Model##############################################
model=ResNet50(weights='imagenet',include_top=False, input_shape=(224,224,3))
model.trainable=False
model=tf.keras.Sequential([model,GlobalMaxPool2D()])
model.summary()2. Extract Features: Pass 44k images to restnet50 model and extract features and then flatten the features to convert into 1d vector. Extracted features and images has to be stored into pickle files so that while recommendation we pass index of resultant features will get images in return.
############### One time Code: need to extract features of 44k images, U can run this ######
def image_preprocess(path,model):
img=image.load_img(path, target_size=(224,224))
img_arr=image.img_to_array(img)
ex_img_arr=np.expand_dims(img_arr,axis=0)
pre_pr_img=preprocess_input(ex_img_arr)
result=model.predict(pre_pr_img).flatten()
normal_result=result/norm(result)
return normal_result
path=r'C:\Users\TanishSharm\Desktop\Fashion_recom_sys\myntradataset\images'
images=[os.path.join(path,files) for files in os.listdir(path)]
pickle.dump(images,open('images.pkl','wb'))
feature_list=[]
for file in tqdm(images):
feature_list.append(image_preprocess(file, model))
pickle.dump(feature_list,open('fetaures.pkl','wb'))3. Load Extracted features: To get recommendation for given query point using distance metrics.
################################Loading Stored Features and images##################################
file_img=pickle.load(open(r'C:\Users\TanishSharma\Desktop\Fashion_recom_sys\images.pkl','rb'))
feature_list=(pickle.load(open(r'C:\Users\TanishSharma\Desktop\Fashion_recom_sys\fetaures.pkl','rb')))4. Save Images: We use streamlit to make this system. Create one upload button, that will provide browse option then click on upload image will bed saved it into uploads folder in your local.
st.title('Fashion Recommender system')## title of the webpage
###################### Method to Save Uploaded Image into local############################
def Save_img(upload_img):
try:
with open(os.path.join('uploads',upload_img.name),'wb') as f:
f.write(upload_img.getbuffer())
return 1
except:
return 0
########### To display upload button onto screen######################
upload_img=st.file_uploader("Choose an image")
5. Generate Recommendation: Now features got stored and will pass new query image to resnet50 model, it will generate features which will flatten further. Here we use Euclidean distance metrics to find out recommended 10 Nearest Neighbors or look similar images. Below are two methods to extract feature and prod recommendation.
######################## Method to Extract features of new query image#######################
def feature_extraction(path,model):
img=image.load_img(path, target_size=(224,224))# Load image in size of 224,224,3
img_arr=image.img_to_array(img)# storing into array
ex_img_arr=np.expand_dims(img_arr,axis=0)## Expanding the dimension of image
pre_pr_img=preprocess_input(ex_img_arr)## preprocessing the image
result=model.predict(pre_pr_img).flatten()### to make 1d vector
normal_result=result/norm(result)## Normalize the result using norm func from linalg(numpy)
return normal_result
def prod_recom(features, feature_list):
neb=NearestNeighbors(n_neighbors=10,algorithm='brute',metric='euclidean') #using brute force algo here as data is not too big
neb.fit(feature_list)## fit with feature list
dist, ind=neb.kneighbors([features])# return distance and index but we use index to find out nearest images from stored features vector
return indWill call this functions to extract feature, get 10 recommendation and create sections of recommended images into streamlit app. Find below code for the same.
### Condition to check if image got uploaded then call save_img method to save and preprocess image followed by extract features and recommendation
if upload_img is not None:
if Save_img(upload_img):
st.image(Image.open(upload_img))
st.header("file uploaded successfully")
features=feature_extraction(os.path.join("uploads",upload_img.name),model)
progress_text = "Hold on! Result will shown below."
my_bar = st.progress(0, text=progress_text)
for percent_complete in range(100):
time.sleep(0.02)
my_bar.progress(percent_complete + 1, text=progress_text) ## to add progress bar untill feature got extracted
ind=prod_recom(features, feature_list)# calling recom. func to get 10 recommendation
### to create 10 section of images into the screen
col1,col2,col3,col4,col5,col6,col7,col8,col9,col10=st.columns(10)
##for each section image shown by below code
with col1:
st.image(Image.open(file_img[ind[0][0]]))
with col2:
st.image(Image.open(file_img[ind[0][1]]))
with col3:
st.image(Image.open(file_img[ind[0][2]]))
with col4:
st.image(Image.open(file_img[ind[0][3]]))
with col5:
st.image(Image.open(file_img[ind[0][4]]))
with col6:
st.image(Image.open(file_img[ind[0][5]]))
with col7:
st.image(Image.open(file_img[ind[0][6]]))
with col8:
st.image(Image.open(file_img[ind[0][7]]))
with col9:
st.image(Image.open(file_img[ind[0][8]]))
with col10:
st.image(Image.open(file_img[ind[0][9]]))
# st.text("Using Spotify ANNoy")
# df = pd.DataFrame({'img_id':file_img, 'img_repr': feature_list})
# f=len(df['img_repr'][0])
# ai=AnnoyIndex(f,'angular')
# for i in tqdm(range(len(feature_list))):
# v=feature_list[i]
# ai.add_item(i,v)
# ai.build(10) # no of binary tress want to build more number of tree more accuracy
# neigh=(ai.get_nns_by_item(0,5))
# with col1:
# st.image(Image.open(file_img[neigh[0]]))
# with col2:
# st.image(Image.open(file_img[neigh[1]]))
# with col3:
# st.image(Image.open(file_img[neigh[2]]))
# with col4:
# st.image(Image.open(file_img[neigh[3]]))
# for i in range(len(neigh)):
# with st.columns(i):
# st.image(Image.open(file_img[neigh[i]]))
else:
st.header("Some error occured")find full code of .py file in my github profile.
STEPS
- Create new project in your IDE(pycharm, spyder, Vscode etc.) paste your downloaded Dataset from given resource inside the same folder.
- Execute given 2 steps in above Plan of Action section and store features into local machine inside same folder.
- Open cmd-> activate python env-> streamlit run main.py
4. Browse any fashion product image and wait for couple of seconds 10 look alike products will shown below.
Recommendation can be improved based on distance metrics you can try different metrics like Manhattan Distance, Minkowski Distance, and Hamming Distance and see the results or annoy index can be tried(by spotify used in music recommendation) to find out best recommended products.
Conclusion
Fashion recommendation system using ResNet50 can provide a personalized shopping experience for users and help fashion retailers increase sales. By analyzing the features of fashion items, the system can recommend similar items that are more likely to be of interest to the user. As deep learning algorithms continue to advance, we can expect to see more sophisticated fashion recommendation systems in the future.