Convolutional Neural Network — Lesson 5: Strides
The stride is a parameter of the convolution operation that refers to the number of pixels by which the filter matrix moves across the input matrix. When the stride is 1, the filter moves across the input matrix 1 pixel at a time. When the stride is 2, the filter jumps 2 pixels at a time as we slide it around. And so on.
To illustrate, let’s say we have a 5x5 input matrix (representing a part of an image), and we’re applying a 3x3 filter using a stride of 1. The filter starts from the top left corner of the image, and then moves one pixel to the right at each step until it hits the edge of the image, at which point it moves back to the left edge and one pixel down.
But, if the stride were 2, the filter would move two pixels to the right at each step instead of one. This means the filter would be applied fewer times, and the resulting output (often referred to as a feature map) would be smaller.
The stride’s value impacts the model in a few ways:
- Dimensionality Reduction: Larger stride values result in smaller output dimensions, effectively performing a form of dimensionality reduction.
- Computation Speed: Larger strides can also speed up computation, as the filter needs to be applied fewer times.
- Model Capacity: However, larger strides may result in the model losing some detailed information because the filter doesn’t cover every single pixel; it’s skipping over some. This could potentially reduce model accuracy, particularly in tasks that require capturing fine-grained details.
Choosing the appropriate stride value requires a balance between preserving information (smaller strides) and computational efficiency or dimensionality reduction (larger strides). It’s a hyperparameter that you may need to tune during the model training process.
