GSoC’19 with Aimacode

Most of us usually get introduced to programming in our freshman year. Then we often hear words like “open source contributions”, “Google Summer of Code”, etc. But initially, any of us don’t have much clue about what they exactly are? How the programs like GSoC actually work? How we can participate? and most importantly how to get started? Therefore, through this blog, I’ll be sharing my journey from being a newbie in programming to getting selected as a Student developer in GSoC’19. Later in this blog, I’ll be covering my GSoC project and the tasks I’ve accomplished so far.

What is GSoC?

For those who don’t know what Google Summer of Code is, it’s a program (popularly known as GSoC) organized by Google to promote the open source development among the student developers. As the name suggests, it takes place every summer, during the months of May to August. A list of selected organizations (also called orgs) is announced around late February. Every organization contains an idea page which lists various projects of that organization. Students can browse through these projects to select the one which suits their interests and passion. Each student can then submit up to 3 proposals to these organizations describing his/her exact plans about the work to be done throughout the summers, a detailed timeline explaining these plans and why he/she should be the person to be selected. Org admins then review these proposals and a list of selected students is announced in the first week of May.

Flashback

As mentioned, I was introduced to coding and open source contributions in my freshman year. Since then I’ve been excited and enthusiastic about contributing to the open source community. So, in my free time, I used to surf across various projects on GitHub. I often tried searching for a platform, where I can begin my contributions, an organization whose goals suits my interests. But I was really having a hard time figuring out how to get started? Where to contribute? What my interests are? What I’m passionate about? etc.

It was around April last year when two of my seniors took a lecture on Machine Learning and Artificial Intelligence. They introduced us with the various basic concepts, told us about the recent achievements and various ongoing projects and researches in this field. I’ve always loved reading new algorithms and solving the mathematics involved behind them. So that day, I was literally enjoying learning those new algorithms. Moreover, I found myself attracted to this field and wanted to explore it further. One of them suggested us a book for AI named Artificial Intelligence: a Modern Approach (also popular as AIMA) by Stuart J. Russell and Peter Norvig. I bought and started reading this book. Soon I realized, I was getting addicted to this book!!! The way it explains the algorithms, the way it demonstrates them in a real-world situation was really amazing. I was loving reading the new algorithms every day and solving the problems mentioned in the textbook. I also started to implement the pseudo codes given in the textbook.

Someday in late June, when I was randomly exploring more about this book over the internet, I came across an organization on GitHub named aimacode. It contained the implementations of the pseudo codes mentioned in the book. But it was a work in progress. I also came to know that this organization appears in GSoC every year and there’s plenty of work still left to be done. That day itself I decided to try for GSoC in this org. Therefore, I cloned the aima-java repo and started trying to understand its codebase. It was just after a day or two, I received a notification about a new issue created in the repo. It was about the initialization of the Jupyter notebooks in the project which should contain a brief explanation of each algorithm, the way they are implemented in the code repository and their demonstrations on various problems mentioned in the book. As I’ve just completed reading the first part of the textbook, what could be a better issue for me to begin with :D Hence, I claimed this issue and started contributing. Within a week, I submitted my first pull request and Yippee! this was my first ever contribution to the open source community. I then continued reading this awesome book and contributing to the organization. I worked on some of the old issues, created some new ones and submitted PRs solving these issues.

Pre GSoC involvements

January-February 2019

As GSoC was near, everyone was gearing up for it. Students had begun searching for the projects from the list of organizations selected last year hoping that the same organizations will appear again for the GSoC this year. Meanwhile, I had got a headstart as I’d already decided my project and I also had two of my pull request merged in the code repository by then. Hence, I kept contributing by adding more notebooks for the next chapters throughout this period.

Finally, on February 27th, the list of selected organizations was announced. I was glad to see that “aimacode” was selected again this year. I went through its idea page and found the project I’ve already been contributing to, so far. I then approached my potential mentor for enquiring more about the project and the further tasks to be done. I got some issues assigned and thus I spent most of this period trying to solve these issues.

March-April 2019

This was the time, I started working on my proposal. In one of my discussions with the potential mentor, I got to know that the community will soon be publishing the fourth edition of the textbook and therefore all the changes and new additions were required to be implemented in a new branch. So by this time, I’ve pretty much learned about what the community demands from this GSoC period. I had some further discussions with my mentor regarding the project and the community coding practices and thus made a proposal giving a detailed description of my plans for the summers. I knew that a proposal plays the most crucial role in one’s selection, so I tried to keep it as much detailed as I could. I then drafted a tentative proposal to my mentor, got a few suggestions, made some necessary changes and thus finalized my proposal and finally submitted it on April 8.

Pheewww! I had finished my part, the deadline of proposal submission was now over and all I could do was to wait for the results. I had my end semester examinations in the month of April but the thought of my GSoC result was always in my mind. This period of proposal review (April 9 to May 5) seemed to be the longest period ever :-P.

May 2019

Finally, on May 6, Google rolled out the list of selected students. Excited and a bit nervous at the same time, I checked my dashboard on GSoC’s website…and …Oh Yeahh! I made it! I was selected in aimacode for the aima-java project. Within an hour or so, I received a mail:

This was really a dream come true moment for me. May 6, 2019, will always be one of the dates that I’ll always cherish.

And thus began my journey of GSoC with aimacode!!!

Now I’ll be discussing my GSoC project in detail and the tasks I’ve accomplished so far…

GSoC Coding Phase

Initial weeks were the community bonding period. I contacted my mentor thanking him for the selections. We had our first meeting regarding the project and he assigned me some tasks to get started with. I was supposed to read the chapter “Deep Learning” from the new edition. My initial tasks were:

• Explore the usecases of ND4J and DL4J libraries.
• Create a Jupyter notebook demonstrating the usecases of these libraries in various scenarios.

I’ve never worked before with any of these libraries so I spent about a week understanding the working of these libraries and exploring their usecases. Simultaneously, I also finished reading the Deep Learning chapter.

Let’s have a quick look at what these libraries are for and how I’ve used them in my project.

ND4J (N-Dimensions For Java) or — as its creators present it — the “numpy” for the JVM, is a tensor & N-dimensional array scientific computing library written in Java for the Java Virtual Machine(JVM) and is part of the Deeplearning4j suite of software.

DL4J (Deep Learning For Java) is a deep learning programming library, which provides a computing framework with wide support for deep learning algorithms. This library too is written in Java for the Java Virtual Machine(JVM). Since all deep learning is based on vectors and tensors, DL4J relies on the above-mentioned tensor library “ND4J” for all the computations involving n-dimensional arrays (also called tensors).

In order to explain their usage, I created a deep neural network using DeepLearning4J and trained the model capable of classifying random handwritten digits. Let’s briefly look at the implementation part.

DL4J library uses the NeuralNetConfiguration class for building a neural network. It is the class, where we pass all the required parameters that define the architecture and how the algorithm learns.

int seed = 123;
int batchSize = 100;
int numEpochs = 1;
int height = 28;
int width = 28;
int channels = 1;
int numInput = height * width;
int numHidden = 1000;
int numOutput = 10;
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
        .seed(seed)
        .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
        .updater(Updater.ADAM)
        .list()
        .layer(0, new DenseLayer.Builder()
                .nIn(numInput)
                .nOut(numHidden)
                .activation(Activation.RELU)
                .weightInit(WeightInit.XAVIER)
                .build())
        .layer(1, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                .nIn(numHidden)
                .nOut(numOutput)
                .activation(Activation.SOFTMAX)
                .weightInit(WeightInit.XAVIER)
                .build())
        .setInputType(InputType.convolutional(height, width, channels))
        .build();

Now that we’ve built a NeuralNetConfiguration, we can use the configuration to instantiate a MultiLayerNetwork. When we call the init() method on the network, it applies the chosen weight initialization across the network and allows us to pass data to train. An instantiated model has a fit() method that accepts a dataset iterator.

MultiLayerNetwork model = new MultiLayerNetwork(conf);
model.init();
model.fit(trainIter);

A MultiLayerNetwork has a few built-in methods to help us perform the evaluation. You can pass a dataset iterator with your testing/validation data to an evaluate() method.

Evaluation eval = model.evaluate(testIter);
    System.out.println(eval.stats());

This was how I trained my first ever model on MNIST dataset using the DeepLearning4J library and the results obtained were quite amazing.

==========================Scores====================================
 # of classes:    10
 Accuracy:        0.9672
 Precision:       0.9674
 Recall:          0.9669
 F1 Score:        0.9670
Precision, recall & F1: macro-averaged (equally weighted avg. of 10 classes)
====================================================================

My next subtask was to create a Recurrent neural network using the DeepLearning4j library. As I’ve already learned to create and train a neural network using DL4J, so I proceeded the same way and created a model capable of generating text character by character.

This completed all the stuff I’ve planned for the first milestone. I then again had a meeting with my mentor for discussing further milestones. The next task I was assigned was:

• Explore the usecases of TensorFlow Java API and include the same in the Deep Learning notebook.

This was again something new for me. I had no experience working with the TensorFlow library. Therefore I spent some time reading various blogs, following tutorials for understanding the use of this library. Soon I learned that TensorFlow is an open source library for dataflow programming, but the TensorFlow Java API does not have feature parity with the Python API. Hence, the Java API is only suitable for inference using pre-trained models and for training pre-defined models.

I explained the use of TensorFlow Java API with a couple of example of image classification and object detection and the results obtained were really awesome. I’ve pasted one of the results of object detection below.

This concluded my Deep Learning notebook and the tasks for the second milestone as well. The complete notebook is now merged in the code repository and can be viewed here.

This week is my first evaluation of the work done so far and I’m eagerly waiting for the results:’)

Thanks a lot for reading this blog. I’ll be publishing more blogs soon, discussing my further work and the overall GSoC experience with Aimacode. Stay tuned!!!