Hi RAIN,

I looked at the picture you shared. Yes, the CP decomposition involves 3D convolutions.

First, on input: [1 x 1] x In x Rank
Mid1, 1st spatial dim: [d x 1] x 1 x Rank
Mid2, 2nd spatial dim: [1 x d] x 1 x Rank
Last, on output: [1x1] Rank x Out

First and last operations are 2D convolutions. The reason is that we move the kernel in height and width dimensions (2 dims only excluding batch dim). Mid operations are 3D convolutions. Here we move the kernel across height, width and depth dimensions.

For 3D convolutions, you do not have to split tensors and operate on sub parts. For example in Keras, the CP decomposition can be applied as follows,

First: Conv2D(Rank, kernel_size=(1, 1), strides=(1, 1))
Mid1: Conv3D(Rank, kernel_size=(d, 1, 1), strides=(1, 1, 1))
Mid2: Conv3D(Rank, kernel_size=(1, d, 1), strides=(1, 1, 1))
Last: Conv2D(out, kernel_size=(1, 1), strides=(1, 1))

Reference https://keras.io/layers/convolutional/#conv3d

I Hope this helps. I haven’t read the papers you shared, I will read once I get time and add my comments here.