Weekly ML Resources — June 28
It has been 6 months since I started the NLP paper discussion group at Aggregate Intellect, and it has been a wonderful experience so far, with a lot of interesting discussions spanning different subfields of NLP. We started off with some foundational papers in attention mechanisms and Transformers and moved on to cover more recent papers that are fresh off the arXiv oven. In the latter months, we covered papers that I felt didn’t get as much attention on social media as it deserved.
Starting last week, I pivoted the group to a new format. Henceforth, I will be doing code-walkthroughs of foundational and recent papers during our weekly sessions, rather than be limited to just the paper. I am excited for the new format — if you are interested, join the #dg-nlp channel on the Aggregate Intellect Slack and request a calendar invite; All are welcome!
Looking back, here is a list of papers that we covered at the group during the past several months —
- Neural Machine Translation by Jointly Learning to Align and Translate
- Effective Approaches to Attention-based Neural Machine Translation
- Pointer Networks
- Neural Turing Machines
- Attention is all you Need
- Adaptive Attention Span in Transformers
- Reformer: The Efficient Transformer
- Longformer: The Long-document Transformer
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Language Models are Few-Shot Learners
- Do Transformer Modifications Transfer Across Implementations and Applications?
- CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
- Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
- Smoothing and Shrinking the Sparse Seq2Seq Search Space
- On Hallucination and Predictive Uncertainty in Conditional Language Generation
- FeTaQa: Free-Form Table Question Answering
- Causal Effects of Linguistic Properties
- Disentangling Representations of Text by Masking Transformers
- How Individuals Change Language
- Recipes for Building an Open-Domain Chatbot
- We Need to Talk About Random Splits
- ZmBART: An Unsupervised Cross-Lingual Transfer Framework for Language Generation
It was great fun discussing the excellent contributions made by the authors of the above papers. Looking forward to conducting deep dives into more papers as we move into the new format for the paper discussion group!
Aggregate Intellect
Aggregate Intellect is a Global Marketplace where ML Developers Connect, Collaborate, and Build. Connect with peers & experts at https://ai.science or Join our Slack Community.
- Check out the user generated Recipes that provide step by step, and bite sized guides on how to do various tasks
- Join our ML Product Challenges to build AI-based products for a chance to win cash prizes
- Connect with peers & experts through the ML Discussion Groups or Expert Office Hours