Effect of Batch Size on Neural Net Training

co-authored with Apurva Pathak

Welcome to the first installment in our Deep Learning Experiments series, where we run experiments to evaluate commonly-held assumptions about training neural networks. Our goal is to better understand the different design choices that affect model training and evaluation. To do so, we come up with questions about each design choice and then run experiments to answer them.

In this article, we seek to better understand the impact of batch size on training neural networks. In particular, we will cover…