This is very interesting work. I do have a couple of questions for you though.
Tiny Eval


  1. It would be easier to sort the references with only one score for recommendations purposes. For example, if you have a vector of probability of ratings for (5,4,3,2,1) how do you compare (0.50,0.1,0.1,0.1,0.2) and (0.30,0.7,0.,0.,0.) ? (Unsure the model can actually output theses cases). You would have to draw some rules here to have an 1 dimensional score. Taking the max proba class and sort based on that can be one choice. It’s not obvious that it could bring more performance in the recommender. A good question is then how do you compare these models. It’s not really possible offline.
  2. I don’t have the code and output anymore so I won’t create this graph. Would have been nice to show I concur.

Thanks for reading!