Part 1 : Input Embedding and Positional Embedding
Part 2 : Encoder Stage
Part 3 : Decoder Stage
Part 4 : Pros and Cons, Resources
Part 5 : Research Papers using Transformers
Machine Translation
- Paper: “Attention is All You Need” (2017) by Vaswani et al.
- Application: Translating text from one language to another.
- Link : https://arxiv.org/abs/1706.03762
Text Summarization
- Paper: “BERTSUM: Extensively Pretrained Encoder-Decoder Architecture for Text Summarization” (2019) by Liu and Lapata.
- Application: Generating concise summaries of longer texts.
- Link : https://aclanthology.org/D19-1387.pdf
Text Generation
- Paper: “Language Models are Few-Shot Learners” (2020) by Brown et al.
- Application: Generating human-like text based on a given prompt.
- Link : https://arxiv.org/abs/2005.14165
Question Answering
- Paper: “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” (2019) by Devlin et al.
- Application: Answering questions based on provided context.
- Link Paper : https://arxiv.org/abs/1810.04805
Text Classification
- Paper: “XLNet: Generalized Autoregressive Pretraining for Language Understanding” (2019) by Yang et al.
- Application: Categorizing text into predefined classes.
- Link : https://arxiv.org/abs/1906.08237
Image Captioning
- Paper: “Show, Attend and Tell: Neural Image Caption Generation with Visual Attention” (2015) by Xu et al.
- Application: Generating descriptive captions for images.
- Paper Link : https://arxiv.org/pdf/1502.03044
Speech Recognition
- Paper: “Transformer-based Acoustic Modeling for Hybrid Speech Recognition” (2020) by Zhou et al.
- Application: Converting spoken language into written text.
Document Understanding
- Paper: “Longformer: The Long-Document Transformer” (2020) by Beltagy et al.
- Application: Extracting and summarizing key information from documents.
Conversational AI (Chatbots)
- Paper: “DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation” (2020) by Zhang et al.
- Application: Developing systems that can carry on a natural conversation with users.
Code Generation
- Paper: “Codex: A Large-Scale Neural Network for Code Generation” (2021) by Chen et al.
- Application: Automatically generating code snippets or entire programs from natural language descriptions.
Happy Learning
