Stan KriventsovinDeep Learning ReviewsSwitch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity…Review of paper by William Fedus, Barret Zoph, and Noam Shazeer, Google Brain, 2021.Feb 11, 20211Feb 11, 20211
Stan KriventsovinDeep Learning ReviewsAutoDropout: Learning Dropout Patterns to Regularize Deep Networks (paper review)Review of paper by Hieu Pham¹ ² and Quoc V. Le¹, ¹Google Research and ²Carnegie Mellon University, 2021.Jan 21, 2021Jan 21, 2021
Stan KriventsovinDeep Learning ReviewsPoint Transformer (paper review)Review of paper by Hengshuang Zhao¹, Li Jiang², Jiaya Jia², et al, ¹University of Oxford, ²The Chinese University of Hong Kong, 2020.Jan 4, 2021Jan 4, 2021
Stan KriventsovinDeep Learning ReviewsEvery Model Learned by Gradient Descent Is Approximately a Kernel Machine (paper review)Review of paper by Pedro Domingos, University of Washington, 2020Dec 15, 20201Dec 15, 20201
Stan KriventsovinDeep Learning ReviewsScaling *down* Deep Learning (paper review)Review of paper by Sam Greydanus, Oregon State University and the ML Collective, 2020Dec 7, 2020Dec 7, 2020
Stan KriventsovinDeep Learning ReviewsGradient Starvation: A Learning Proclivity in Neural Networks (paper review)Review of paper by Mohammad Pezeshki¹ ², Sekou-Oumar Kaba¹ ³, Yoshua Bengio¹ ², et al, ¹Mila, ²Université de Montréal, ³McGill University…Dec 1, 20203Dec 1, 20203
Stan KriventsovinDeep Learning ReviewsAdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients (paper review)By Juntang Zhuang¹, Tommy Tang², Yifan Ding³, et al, ¹Yale University, ²University of Illinois at Urbana-Champaign, and ³University of…Nov 24, 2020Nov 24, 2020
Stan KriventsovinDeep Learning ReviewsAttention Augmented Differentiable Forest for Tabular Data (paper review)Review of paper by Yingshi Chen, Xiamen University, 2020Nov 11, 2020Nov 11, 2020
Stan KriventsovinDeep Learning ReviewsDeep Learning for Symbolic Mathematics (paper review)Review of paper by Guillaume Lample and François Charton, Facebook AI Research, 2019.Oct 21, 2020Oct 21, 2020
Stan KriventsovinDeep Learning ReviewsCompounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network…Review of paper by Jungkyu Lee, Taeryun Won, and Kiho Hong, Clova Vision, NAVER Corp, 2019Oct 21, 2020Oct 21, 2020