This article serves to briefly outline the design of the Neural Turing Machine (NTM), a backpropogatable architecture that can (among many possibilities) learn to dynamically execute programs.

The original paper.

I’ve added some specifications about the NTM’s architecture that the paper excludes for the sake of generality. These will be discussed upon presentation.

The Neural Turing Machine was proposed by Graves et al. as a Turing-Complete network capable of learning (rather complex) programs. Inspired by the sequential nature of the brain, and the large, addressable memory of the computer.

The NTM is composed of five modules:

  • The controller
  • The addressing…

Let’s do this…

We all know LSTM’s are super powerful; So, we should know how they work and how to use them.

Image for post
Image for post

Syntactic notes

  • Above ⨀ is the element-wise product or Hadamard product.
  • Inner products will be represented as ⋅
  • Outer products will be respresented as ⨂
  • σ represents the sigmoid function

The forward components

The gates are defined as:

Aidan Gomez

Machine Learning at Oxford •

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store