The magic of LSTM neural networks

LSTM Neural Networks, which stand for Long Short-Term Memory, are a particular type of recurrent neural networks that got lot of attention recently within the machine learning community.

In a simple way, LSTM networks have some internal contextual state cells that act as long-term or short-term memory cells.
The output of the LSTM network is modulated by the state of these cells. This is a very important property when we need the prediction of the neural network to depend on the historical context of inputs, rather than only on the very last input.

As a simple example, consider that we want to predict the next number of the following sequence: 6 -> 7 -> 8 -> ?. We would like to have the next output to be 9 (x+1). However, if we provide this sequence: 2 -> 4 -> 8 -> ?, we would like to get 16 (2x).
Although in both cases, the current last input was number 8, the prediction outcome should be different (when we take into account the contextual information of previous values and not only the last one).

https://www.captionbot.ai/How they work

LSTM networks manage to keep contextual information of inputs by integrating a loop that allows information to flow from one step to the next. These loops make recurrent neural networks seem magical…