Stan KriventsovinDeep Learning ReviewsSwitch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity…Review of paper by William Fedus, Barret Zoph, and Noam Shazeer, Google Brain, 2021.·5 min read·Feb 11, 2021--1--1
Stan KriventsovinDeep Learning ReviewsAutoDropout: Learning Dropout Patterns to Regularize Deep Networks (paper review)Review of paper by Hieu Pham¹ ² and Quoc V. Le¹, ¹Google Research and ²Carnegie Mellon University, 2021.·4 min read·Jan 21, 2021----
Stan KriventsovinDeep Learning ReviewsPoint Transformer (paper review)Review of paper by Hengshuang Zhao¹, Li Jiang², Jiaya Jia², et al, ¹University of Oxford, ²The Chinese University of Hong Kong, 2020.·5 min read·Jan 4, 2021----
Stan KriventsovinDeep Learning ReviewsEvery Model Learned by Gradient Descent Is Approximately a Kernel Machine (paper review)Review of paper by Pedro Domingos, University of Washington, 2020·4 min read·Dec 15, 2020--1--1
Stan KriventsovinDeep Learning ReviewsScaling *down* Deep Learning (paper review)Review of paper by Sam Greydanus, Oregon State University and the ML Collective, 2020·5 min read·Dec 7, 2020----
Stan KriventsovinDeep Learning ReviewsGradient Starvation: A Learning Proclivity in Neural Networks (paper review)Review of paper by Mohammad Pezeshki¹ ², Sekou-Oumar Kaba¹ ³, Yoshua Bengio¹ ², et al, ¹Mila, ²Université de Montréal, ³McGill University…·4 min read·Dec 1, 2020--3--3
Stan KriventsovinDeep Learning ReviewsAdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients (paper review)By Juntang Zhuang¹, Tommy Tang², Yifan Ding³, et al, ¹Yale University, ²University of Illinois at Urbana-Champaign, and ³University of…·4 min read·Nov 24, 2020----
Stan KriventsovinDeep Learning ReviewsAttention Augmented Differentiable Forest for Tabular Data (paper review)Review of paper by Yingshi Chen, Xiamen University, 2020·4 min read·Nov 11, 2020----
Stan KriventsovinDeep Learning ReviewsDeep Learning for Symbolic Mathematics (paper review)Review of paper by Guillaume Lample and François Charton, Facebook AI Research, 2019.·3 min read·Oct 21, 2020----
Stan KriventsovinDeep Learning ReviewsCompounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network…Review of paper by Jungkyu Lee, Taeryun Won, and Kiho Hong, Clova Vision, NAVER Corp, 2019·4 min read·Oct 21, 2020----
