Positional Embeddings
Transformer has already become one of the most common model in deep learning, which was first introduced in “Attention Is All You Need”. Before that, the most common model for sequence modelling was RNN. However, in Transformer, input sequence is not fed into the model one by one, while…