A quick journey through Conv1D functions from TensorFlow to PyTorch, passing via SciPy. Part 2

4 min readFeb 25, 2023

Convolution, Same Mode

We continue the journey from the first part, talking about the same modality in convolution operations.

From the TensorFlow library documentation we read that the same modality results in padding evenly to the left/right of the input such that output has the same size dimension as the input.

Remembering what we used in the last article, the input signal x[n] and the filter w[n] are defined as :

N = 4, M = 3
x[n] = [1, 2, 3, 4]
w[n] = [1, 1, 1]

With the same mode we expect an output signal y[n] of length O = 4. If we insert in the valid mode output size calculation O = N — M + 1 a new term P, O = P + N — M +1, and require that O = N (the output size is equal to the input size) we find that P = M — 1 = 2 ( in this case). This means that we have to pad the input signal x[n] with M — 1 zeros, evenly distributed to the left and to the right:

N = 4, M = 3
x_pad[n] = [0, 1, 2, 3, 4, 0]

TensorFlow

In TensorFlow, we just need to set padding= “same” in the constructor of Conv1d layer.

x = np.array([1, 2, 3, 4]).astype(np.float32)
x = x.reshape([1, 4, 1])
w = np.array([1, 1, 1]).astype(np.float32)

conv1d = tf.keras.layers.Conv1D(1,3,
            kernel_initializer=tf.keras.initializers.constant(w),
            padding="same",
            use_bias=False) 
y = conv1d(x)
print(y)

tf.Tensor(
[[[3.]
  [6.]
  [9.]
  [7.]]], shape=(1, 4, 1), dtype=float32)

and we have y [n] = [3, 6, 9, 7]

PyTorch:

Also PyTorch gives the possibility to set the same modality:

x = np.array([1, 2, 3, 4]).astype(np.float32)
x = x.reshape([1, 4, 1])
w = np.array([1, 1, 1]).astype(np.float32)
x_torch = torch.from_numpy(x)
x_torch = x_torch.permute(0,2,1)
wt = w.reshape([3, 1])
conv1d = nn.Conv1d(1, 1, 3, bias=False, padding="same") 
conv1d.weight.data.fill_(1)
y = conv1d(x_torch)
print(y)

tensor([[[3., 6., 9., 7.]]]

And again we have y [n] = [3, 6, 9, 7]

Scipy :

Also SciPy gives the same result, by setting mode= “same”:

x = np.array([1, 2, 3, 4]).astype(np.float32)
w = np.array([1, 1, 1]).astype(np.float32)
y_scipy = scipy.signal.correlate(x, w, mode="same")
print(y_scipy)
[3. 6. 9. 7.]

Calculation by hand

Lastly, we see now how to implement the same mode convolution,

Basically we have to:

Take the input signal x[n]
Pad x with P = M — 1 zeros:
pad with P_left = (M — 1) / 2 zeros to the left, P_right = P — P_left zeros to the right. In the case M is even, M — 1 is odd, we need to use a better formula:
Calculate P_left = ceil((M — 1) / 2) (rounds up to the next integer) and then calculate P_right = P — P_left
Use the convolution formula (we have already seen) on the padded signal:

x[n] = [1, 2, 3, 4]  # Take the input signal
x_pad[n] = [0, 1, 2, 3, 4, 0] 
#Pad x with O = M - 1 = 2 zeros, one 0 on the left, one 0 on the right side
w[n] = [1, 1, 1]

y[0] = x_pad[0] * w[0] + x_pad[1] * w[1] + x_pad[2] * w[2] = 3 
y[1] = x_pad[1] * w[0] + x_pad[2] * w[1] + x_pad[3] * w[2] = 6
y[2] = x_pad[2] * w[0] + x_pad[3] * w[1] + x_pad[4] * w[2] = 9
y[3] = x_pad[3] * w[0] + x_pad[4] * w[1] + x_pad[5] * w[2] = 7

If the case in which M is even, ( for example M = 4, and w[n] = [1, 1, 1, 1]) both TensorFlow and Scipy give y[n] = [3, 6, 10, 9] as result .

x[n] = [1, 2, 3, 4]
w[n] = [1, 1, 1, 1]
x_pad[n] = [0, 0, 1, 2, 3, 4, 0]
y[n] = [3, 6, 10, 9]

This because x is padded with P_left = 2 zeros at left, P_right = P — P_left = 1 zero at right.

In Pytorch we have: y[n] = [6, 10, 9, 7] because x is padded in the opposite way:

P_right= ceil((M — 1) / 2)
P_left = P — P_right

x[n] = [1, 2, 3, 4]
w[n] = [1, 1, 1, 1]
x_pad[n] = [0, 1, 2, 3, 4, 0, 0]
y[n] = [6, 10, 9, 7]

Next time we will see other convolution modalities tipically used in dsp and ML and we will compare the results.

Happy reading and see you next time,
Stefano