How does MeaLeon use NLP? Part 3: Some Results Comparing One Hot Encoding and TF-IDF
Hey folks!
This is part 3 in a walkthrough tutorial series on how my full-stack, machine learning webapp MeaLeon uses natural language processing (NLP) to provide suggestions on new dishes to cook from different cuisines, based on one you enter as a search query! Check it out in the link above, and the repo here.
In the first part of this series, I discussed how I took a collection of documents (recipes) and converted the raw text ingredients into tokens of root words by combining Natural Language Toolkit (NLTK), WordNet lemmatization, and “expert” knowledge (my familiarity with cooking).
In the previous part, I then discuss how to convert the tokenized ingredient lists into vectors in a many-dimensional space and why using cosine similarity for MeaLeon was a better method to provide suggestions than Euclidean distance between vectors.
In this article, I’ll show you the results obtained from MeaLeon for some sample recipes, as well as some comparisons between One Hot Encode (OHE) and term frequency, inverse document frequency (TF-IDF).
As a reminder, because these numbers will likely come up later, MeaLeon uses a database containing ~18,000 recipes from 24 cuisines spanning an ingredient space of just over 2,000 ingredient axes. This, honestly, is not meant to be a brag! As much as I enjoy working on and using MeaLeon, it is in a state of constant improvement:
- There are other recipe sources that have many more dishes; one source I was considering has nearly half a million.
- 24 cuisines is a very small number and the current database is both disproportionately weighted and labeled to a less than satisfactory quality. At least for me.
If you have suggestions for recipes from under-represented cuisines, please let me know!
3. Over 2,000 ingredients sounds like a lot…if you’ve never made heavily spiced items. I remember years ago that Sheila Dillon tried to make a curry using a BBC recipe and, when she went into a specialty store to ask for ingredients and about the quality of the recipe, the store owner gave her a family recipe for the same dish with over 50 ingredients in it.
Pick 10, Chews 5: MeaLeon Searches and Displays
To start with, MeaLeon asks the user for a dish name and the cuisine it is from. The dish name is simply provided via text box on the homepage and the cuisine names are chosen from a dropdown list. MeaLeon uses the Edamam API to find dishes with the same dish name, takes the 10 top hits from the API, and extracts the ingredient lists from all 10 to make an “averaged version”. This “averaged version” is what is used for the cosine similarity analysis. MeaLeon’s database is scraped from Epicurious.
I’m going to show some comparisons between One Hot Encoded and TF-IDF results. These screenshots are NOT what I have deployed on the Heroku app as I find the presentation to be unattractive and unhelpful for the user. They display the weights of each ingredient vector and are essentially a lot of numbers. However, they are helpful for me since I’m trying to iteratively improve the app.
Anyway, here are some test recipes! In each of this photos, the OHE results will be on the left and the TF-IDF results on the right.
First up, Italian carbonara
Second, Chinese mapo tofu
Third, English fish and chips
Fourth, Middle Eastern kefta kabobs
Last, American buffalo wings
How do we measure performance here? Well, notice that none of the recipes have a perfect cosine similarity of 1. That would require that all of the tokenized ingredients be the same between the search query and the database…which is possible, but highly unlikely. But a higher cosine similarity would reflect a closer relationship between the dishes’ ingredients.
Then which was better, OHE or TF-IDF? Logically, I would say TF-IDF is better because it reduces the significance of the most common words in the corpora. With these five recipes, I personally think that TF-IDF produces suggestions that are most similar to the search queries. However, I felt that OHE returned results more surprising than what TF-IDF gave!
In fact, I think I may refactor this to display the top five most similar (TF-IDF) and the top five stretch/adventurous similar (OHE)!
But what about model/computer performance? I think save that for next time. In the meantime, feel free to give MeaLeon a try and let me know what suggested recipes have been the most intriguing for you!
