A Light Introduction to Transformer-XL

Summary of a novel technique for attentive language modeling that supports longer-term dependency.

Elvis
Elvis
Jan 11 · 5 min read

Background

Transformer-XL

Transformer-XL — training and evaluation phase (figure source)

Results

Other Benefits

Further Readings


dair.ai

Diverse Artificial Intelligence Research & Communication

Elvis

Written by

Elvis

ML-NLP Research Scientist | Ph.D. | Educator | Speaker | Find me on Twitter (https://twitter.com/omarsar0) and LinkedIn (https://www.linkedin.com/in/omarsar/)

dair.ai

dair.ai

Diverse Artificial Intelligence Research & Communication