Backpropagation process of neural networks explained — In another article, we addressed the problem of batch size being limited by GPU memory, and how gradient accumulation helps in overcoming this. In this post, we will first examine the backpropagation process of a neural network and then go through the technical and algorithmic details of gradient accumulation. …