

Digital Marketing strategist, Speaker, Trainer, Search Awards Judge, Lecturer, SEO consultant MD of Move It Marketing https://www.move-it-marketing.co.uk
As opposed to directional models, which read the text input sequentially (left-to-right or right-to-left), the Transformer encoder reads the entire sequence of words at once. Therefore it is considered bidirectional, though it would be more accurate to say that it’s non-directional. This characteristic allows the model to learn the context of a word based on all of its surroundings (left and right of the word).