Recommender Systems: What Long-Tail tells ?
Long-tail graph shows the distribution of ratings or popularity among items or products in marketplace. On the X-column items are ordered by their popularity or rating frequencies, whereas y-column shows the popularity in terms of ratings, demand etc. This graph basically points 3 important facts for recommender systems;
Popularity
Products on left side (or in blue area) are called as popular because their popularity is higher then those in green or long-tail area. Moreover popular products are generally competitive products. On the other hand, products in green long-tail area are thought to be unpopular or new products in market. The threshold which discriminates the popular and unpopular items in market is an hyper-parameter for merchant. We will talk about it in later sentences and articles.
Some researches show that even though popular products mean to be sold a lot, unpopular products or those in long-tail generally returns in better profit for merchants. See below.
Diversity
Recommender algorithms are generally designed to give recommendations for popular items because they are popular :) However, a good recommendation system should provide diversity. Same and known items can make the customers bored. Therefore adjusting the threshold, starting point of long-tail, in recommendation system is an important research to take into account. Moving it right in the graph can increase the diversity in recommendations made.
Sparsity
Items in the right side of graph are less rated than the those in left side. This means that there are much more sparsity or unobserved areas for those items in ratings matrix. This can cause a recommender system which relies on neighborhood algorithms produce bad results. The more we move the threshold to right side, The worse recommendation system results.
As a result, sparsity and long-tail are 2 important properties of a recommender system to take into account in design and process.
