ICLR 2020
A few highlights and takeaways
I recently had the pleasure to attend #ICLR2020, sort of a formal academic conference gathering of machine learning researchers sharing their work. The experience was quite ‘with the times’, as the original meeting space in Ethiopia was only a few short weeks ago set aside in the interest of social distancing, and conference halls and meeting spaces were traded in aggregate for chat sessions, Zoom meetings, pre-recorded poster summaries, oh and not too mention this sort of 8bit avatar space with built in proximity video conferencing — pretty neat stuff.
I’d like to use this forum to quickly highlight a few of the papers I found of interest. I tried to approach the explorations with a novelty filter, mostly turning attention towards unrecognized points within proximity of my interests, which are admittedly somewhat varied. I’ll structure this as a power point presentation of sorts, loosely inspired by seeing Warren Buffett’s understated approach to slide presentation at the recent shareholder’s meeting. Yeah so without further ado.
References
Ba, J., Erdogdu, M., Suzuki, T., Wu, D., and Zhang, T. Generalization of two-layer neural networks: An asymptotic viewpoint. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=H1gBsgBYwH.
Bengio, Y., Deleu, T., Rahaman, N., Ke, R., Lachapelle, S., Bilaniuk, O., Goyal, A., and Pal, C. A meta-transfer objective for learning to disentangle causal mechanisms, 2019. URL https://arxiv.org/abs/1901.10912.
Dyer, E. and Gur-Ari, G. Asymptotics of wide networks from feynman diagrams. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=S1gFvANKDS.
Ermon, S. Measuring economic development from space with machine learning, April 2020.
Flatter, G. C. Why the climate change ai community should care about weather: A new approach for africa, April 2020.
Gissin, D., Shalev-Shwartz, S., and Daniely, A. The implicit bias of depth: How incremental learning drives generalization, 2019. URL https://arxiv.org/abs/1909.12051.
Goyal, A., Sodhani, S., Binas, J., Peng, X. B., Levine, S., and Bengio, Y. Reinforcement learning with competitive ensembles of information-constrained primitives, 2019. URL https://arxiv.org/abs/1906.10667.
Jia, J., Cao, X., Wang, B., and Gong, N. Z. Certified robustness for top-k predictions against adversarial perturbations via randomized smoothing, 2019. URL https://arxiv.org/abs/1912.09899.
Jiang, Y., Neyshabur, B., Mobahi, H., Krishnan, D., and Bengio, S. Fantastic generalization measures and where to find them, 2019. URL https://arxiv.org/abs/1912.02178.
Kong, L., d’Autume, C. d. M., Ling, W., Yu, L., Dai, Z., and Yogatama, D. A mutual information maximization perspective of language representation learning, 2019. URL https://arxiv.org/abs/1910.08350.
Lee, C., Cho, K., and Kang, W. Mixout: Effective regularization to finetune large-scale pretrained language models, 2019. URL https://arxiv.org/abs/1909.11299.
Li, H., Chaudhari, P., Yang, H., Lam, M., Ravichandran, A., Bhotika, R., and Soatto, S. Rethinking the hyperparameters for fine-tuning, 2020a. URL https://arxiv.org/abs/2002.11770.
Li, J., Socher, R., and Hoi, S. C. H. Dividemix: Learning with noisy labels as semi-supervised learning. 2020b. doi: 10.48550/ARXIV.2002.07394. URL https://arxiv.org/abs/2002.07394.
Long, Q., Zhou, Z., Gupta, A., Fang, F., Wu, Y., and Wang, X. Evolutionary population curriculum for scaling multi-agent reinforcement learning, 2020. URL https://arxiv.org/abs/2003.10423.
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., and Sutskever, I. Deep double descent: Where bigger models and more data hurt, 2019. URL https://arxiv.org/abs/1912.02292.
Oktay, D., Balle ́, J., Singh, S., and Shrivastava, A. Scalable model compression by entropy penalized reparameterization. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=HkgxW0EYDS.
Oner, M. U., Lee, H. K., and Sung, W.-K. Weakly supervised clustering by exploiting unique class count. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=B1xIj3VYvr.
Park, S., Kim, K., Kim, S., Lee, J., Lee, J., Lee, J., and Choo, J. Hurricane nowcasting with irregular time-step using neural-ode and video prediction. In ICLR 2020 Workshop on Tackling Climate Change with Machine Learning, 2020. URL https://www.climatechange.ai/papers/iclr2020/21.
Popov, S., Morozov, S., and Babenko, A. Neural oblivious decision ensembles for deep learning on tabular data, 2019. URL https://arxiv.org/abs/1909.06312.
Rahaman, N., Wolf, S., Goyal, A., Remme, R., and Bengio, Y. Learning the arrow of time for problems in reinforcement learning. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=rylJkpEtwS.
Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., and Babenko, A. Editable neural networks, 2020. URL https: //arxiv.org/abs/2004.00345.
Toth, P., Rezende, D. J., Jaegle, A., Racaniére, S., Botev, A., and Higgins, I. Hamiltonian generative networks, 2019. URL https://arxiv.org/abs/1909.13789.
Wei, J., Goyal, M., Durrett, G., and Dillig, I. Lambdanet: Probabilistic type inference using graph neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Hkx6hANtwH.
Wu, Z., Liu, Z., Lin, J., Lin, Y., and Han, S. Lite transformer with long-short range attention, 2020. URL https: //arxiv.org/abs/2004.11886.
Xiong, W., Du, J., Wang, W. Y., and Stoyanov, V. Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model, 2019. URL https://arxiv.org/abs/1912.09637.
For further readings please check out the Table of Contents, Book Recommendations, and Music Recommendations. For more on Automunge: automunge.com
