Updates to “A Metric Learning Reality Check”
I recently uploaded a new version of A Metric Learning Reality Check to arXiv. Here are the highlights:
Unfair Comparisons: Concrete Examples
People were asking for examples to back up my claims about unfair comparisons, so I created a list of examples. See the screenshots below.
Bayesian Optimization Plots
The supplementary material comes with bayesian optimization plots, like the one below. Each plot shows how the validation accuracy changes with respect to hyperparameters. Click here to view the plots.
Comprehensive “Papers vs Reality” Figure
We updated the “trend according to papers” to include more algorithms.
Large Batch Experiment Results on CUB200
We added results for CUB200 with a batch size of 256. The increase in batch size gives FastAP a significant boost in accuracy, and as a result, it performs on par with the rest of the methods, rather than underperforming.
Better Explanation of MAP@R and its Benefits
We explained in more detail why MAP@R is preferable to Recall@1.
Related links
Finally, here are some links you might find interesting:
- Powerful Benchmarker (the code used to run the experiments)
- Powerpoint Slides (summarizes the paper)
- Supplementary material (contains lots of goodies)
- PyTorch Metric Learning (the algorithm implementations)
