Introduction of Graph Convolutional Network (GCN) & Quick Implementation

With modern machine learning methods, tasks like object detection, machine translation, and speech recognition, have been given new life with state-of-the-art deep learning, which is good at capturing hidden patterns of Euclidean data (images, text, videos). However, what about applications where data is generated from non-Euclidean domains, usually composed of graphs with complex relationships and interdependencies between objects?

That’s why we need Graph Neural Networks (GNN)! Among different kinds of GNN, we introduce one of the most popular model, Graph Convolutional Network (GCN), here — convolutional, because filter parameters are typically shared over all locations in the graph.

Graph Convolutional Network (GCN)

Visualization of the zackary karate club network

For GCN, the goal is to learn a function of feature from a graph G=(V, E) and take as input:

• N×D feature matrix X (N: number of nodes, D: number of input features), represents feature description for every node.
• An adjacency matrix A, represents description of the graph structure.

and return an output:

(where F is the number of output features per node)

Thus, every neural network layer can then be written as a non-linear function:

(“L” being the number of layers)

where W(l) is a weight matrix for the l-th neural network layer and σ( ) is a non-linear activation function like the ReLU.

At this point, we need to deal with limitations in A by:

• Add the identity matrix to A to fix this by enforcing self-loops in the graph.

Visualization of A for the zackary karate club network

• Normalizing A such that all rows sum to one in order to avoid changing scale of the feature vectors. We usually used symmetric normalization.

Visualization of A after normalization

Code for dealing with A provided here:

import numpy as np
import scipy as sp
degrees = list(dict(graph.degree).values())
degrees = [d+1 for d in degrees]
D = np.matrix(np.diag(degrees))
D_normed = np.matrix(sp.linalg.fractional_matrix_power(D, -0.5))
A_normed = D_normed*AplusI*D_normed

After all these steps, we can now define our GCN layer as:

Next, we can use the same algorithm such as backpropagation and gradient descent to optimize our weight W. We are not gonna include the algorithm here. Once you know the basic knowledge of GCN, we’ll want to know how to apply it directly. So, let’s dive into how to impelement GCN for our data.

Implementation of GCN

Tools you need:

1. PyTorch Geometric

PyG (PyTorch Geometric) is a library built upon PyTorch to easily write and train Graph Neural Networks (GNNs) for a wide range of applications related to structured data.

This library help us quickly download some open source data and build our model.

2. PPI Dataset

This is protein-protein interaction network that contains physical interactions between proteins. Nodes represent human proteins and edges represent physical interaction between proteins in a human cell. This data contains positional gene sets, motif gene sets and immunological signatures as features (50 in total) and gene ontology sets as labels (121 in total).

Example of PPI network

3. Weights & Biases

Weights & Biases is the MLOps platform for developers to build better models faster. With W&B, you can quickly track experiments, version and iterate on datasets, evaluate model performance, reproduce models, visualize results and spot regressions, and share findings with colleagues.

I’ll then use this tools to track our training log, you guys can see the dashboard right here!

Implementation:

For implementation, please check out my GitHub repository for detail guideline. Here, we briefly mention some steps:

1. Install W&B and PyTorch Geometric

Create an environment and install all the required packages. Note that you need to preinstall PyTorch before installing PyG.

2. Run the code

Start training and view our result on W&B dashboard. Make sure that you got a W&B account with your login key.

The end

That’s the end of this article. Please hit the clap button, leave a comment, and follow me if you’re interested about my content!!!

You may also want to check out some reference below for further information.

See you guys next time!!!

Reference

How to do Deep Learning on Graphs with Graph Convolutional Networks

How powerful are Graph Convolutional Networks?

Human protein-protein interaction network

GitHub - pyg-team/pytorch_geometric: Graph Neural Network Library for PyTorch