CNN for Deeppipe2
In 2020, I have been concentrated in developing CNN. Previously I introduced a Deep-Learning library called DeepPipe2. I was adding CNN features such as convolution and pooling to this. They are written in CUDA and use GPU. It is parallel processing, and it calculates CNN at high speed. It’s still incomplete. I will officially release it until the end of June 2020.
CNN is difficult
It is the hardest one I’ve ever developed. It handles 4D tensors. It handles GPU parallelism. Backpropagation calculations are complicated. I felt frustration. The basic functions of CNN are now working. I am relieved and enjoyed a little satisfaction.
CNN example
The following code is an example of CNN. It is for processing a dataset called Fashion-MNIST. MNIST is a handwritten letters dataset. Fashion-MNIST is monotone image dataset. It is a set of fashion images such as shoes and T-shirts. I’m testing the functionality of CNN with this.
Neural networks mimic Elixir’s pipe operators. f (…) represents the filter. It takes the size of the filter, the number of input channels, the number of output channels, the initial value, the size of the stride and padding, and the dropout rate as arguments. If you omit those arguments, they will take default values.
Doesn’t work well with CIFAR10
I also tried a dataset called CIFAR10. It is a 32 * 32 image. A set of 10 class images such as airplanes and frogs. This learning requires deep neural networks and multi-channel convolution. It takes a lot of time. It takes half a day using GPU. I spent many nights doing computational experiments. In the process I fixed some bugs. Unfortunately, so far DeepPipe2 doesn’t work well. It is a 3-channel color image. Image pre-processing may be required to make it easier to learn.
Hex registration
I wanted Elixir users to use it easily. So I registered DeepPipe2 with Hex. Initially, the NIFs mechanism did not work well. I hurriedly fixed the bug. It works normally from ver1.1.2.
Further improvement
I spent a lot of time and patience debugging and testing. But there will still be bugs. I will repeat the verification further. I will officially release it by the end of June 2020.