Introduction of Graph Convolutional Network (GCN) & Quick Implementation
With modern machine learning methods, tasks like object detection, machine translation, and speech recognition, have been given new life with state-of-the-art deep learning, which is good at capturing hidden patterns of Euclidean data (images, text, videos). However, what about applications where data is generated from non-Euclidean domains, usually composed of graphs with complex relationships and interdependencies between objects?
That’s why we need Graph Neural Networks (GNN)! Among different kinds of GNN, we introduce one of the most popular model, Graph Convolutional Network (GCN), here — convolutional, because filter parameters are typically shared over all locations in the graph.
Graph Convolutional Network (GCN)
For GCN, the goal is to learn a function of feature from a graph G=(V, E) and take as input:
- N×D feature matrix X (N: number of nodes, D: number of input features), represents feature description for every node.
- An adjacency matrix A, represents description of the graph structure.
and return an output:
Thus, every neural network layer can then be written as a non-linear function:
where W(l) is a weight matrix for the l-th neural network layer and σ( ) is a non-linear activation function like the ReLU.
At this point, we need to deal with limitations in A by:
- Add the identity matrix to A to fix this by enforcing self-loops in the graph.
- Normalizing A such that all rows sum to one in order to avoid changing scale of the feature vectors. We usually used symmetric normalization.
Code for dealing with A provided here:
import numpy as np
import scipy as spdegrees = list(dict(graph.degree).values())
degrees = [d+1 for d in degrees]
D = np.matrix(np.diag(degrees))
D_normed = np.matrix(sp.linalg.fractional_matrix_power(D, -0.5))
A_normed = D_normed*AplusI*D_normed
After all these steps, we can now define our GCN layer as:
Next, we can use the same algorithm such as backpropagation and gradient descent to optimize our weight W. We are not gonna include the algorithm here. Once you know the basic knowledge of GCN, we’ll want to know how to apply it directly. So, let’s dive into how to impelement GCN for our data.
Implementation of GCN
Tools you need:
PyG (PyTorch Geometric) is a library built upon PyTorch to easily write and train Graph Neural Networks (GNNs) for a wide range of applications related to structured data.
This library help us quickly download some open source data and build our model.
2. PPI Dataset
This is protein-protein interaction network that contains physical interactions between proteins. Nodes represent human proteins and edges represent physical interaction between proteins in a human cell. This data contains positional gene sets, motif gene sets and immunological signatures as features (50 in total) and gene ontology sets as labels (121 in total).
Weights & Biases is the MLOps platform for developers to build better models faster. With W&B, you can quickly track experiments, version and iterate on datasets, evaluate model performance, reproduce models, visualize results and spot regressions, and share findings with colleagues.
I’ll then use this tools to track our training log, you guys can see the dashboard right here!
Implementation:
For implementation, please check out my GitHub repository for detail guideline. Here, we briefly mention some steps:
- Install W&B and PyTorch Geometric
Create an environment and install all the required packages. Note that you need to preinstall PyTorch before installing PyG.
2. Run the code
Start training and view our result on W&B dashboard. Make sure that you got a W&B account with your login key.
The end
That’s the end of this article. Please hit the clap button, leave a comment, and follow me if you’re interested about my content!!!
You may also want to check out some reference below for further information.
See you guys next time!!!