Origins of Geometric Deep Learning

Towards Geometric Deep Learning III: First Geometric Architectures

Geometric Deep Learning approaches a broad class of ML problems from the perspectives of symmetry and invariance, providing a common blueprint for neural network architectures as diverse as CNNs, GNNs, and Transformers. In a new series of posts, we study how these ideas have taken us from ancient Greece to convolutional neural networks.

Image: Shutterstock.

In the third post from the “Towards Geometric Deep Learning series,” we discuss the first “geometric” neural networks: the Neocognitron and CNNs. This post is based on the introduction chapter of the book M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric Deep Learning (to appear with MIT Press upon completion) and accompanies our course in the African Masters in Machine Intelligence (AMMI). See Part I discussing symmetry, Part II on the early history of neural networks and the first “AI Winter,” and Part IV dedicated to early GNNs.

The inspiration for the first neural network architectures of the new ‘geometric’ type came from neuroscience. In a series of…