Topological Generalisation in GNNs

Topological Generalisation with Advective Diffusion Transformers

One of the key open questions in the study of graph neural networks (GNNs) is their generalisation capabilities, in particular, under changes in the topology of the graph. In this post, we study this problem from the perspective of graph diffusion equations, which are intimately related to GNNs and have been used in the past as a framework for analysing GNN dynamics, expressive power, and justifying architectural choices. We describe a new architecture based on advective diffusion that combines the computational structure of message-passing neural networks (MPNNs) and Transformers and shows superior topological generalisation capabilities.

Image: Unsplash

This post was co-authored with Qitian Wu and Chenxiao Yang and is based on the paper by Q. Wu et al., Advective Diffusion Transformer for Topological Generalization in Graph Learning (2023)…