Weekly ML Resources — June 28

It has been 6 months since I started the NLP paper discussion group at Aggregate Intellect, and it has been a wonderful experience so far, with a lot of interesting discussions spanning different subfields of NLP. We started off with some foundational papers in attention mechanisms and Transformers and moved on to cover more recent papers that are fresh off the arXiv oven. In the latter months, we covered papers that I felt didn’t get as much attention on social media as it deserved.

Starting last week, I pivoted the group to a new format. Henceforth, I will be doing code-walkthroughs of foundational and recent papers during our weekly sessions, rather than be limited to just the paper. I am excited for the new format — if you are interested, join the #dg-nlp channel on the Aggregate Intellect Slack and request a calendar invite; All are welcome!

Looking back, here is a list of papers that we covered at the group during the past several months —

1. Neural Machine Translation by Jointly Learning to Align and Translate
2. Effective Approaches to Attention-based Neural Machine Translation
3. Pointer Networks
4. Neural Turing Machines
5. Attention is all you Need
6. Adaptive Attention Span in Transformers
7. Reformer: The Efficient Transformer
8. Longformer: The Long-document Transformer
9. RoBERTa: A Robustly Optimized BERT Pretraining Approach
10. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
11. Language Models are Few-Shot Learners
12. Do Transformer Modifications Transfer Across Implementations and Applications?
13. CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
14. Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
15. Smoothing and Shrinking the Sparse Seq2Seq Search Space
16. On Hallucination and Predictive Uncertainty in Conditional Language Generation
17. FeTaQa: Free-Form Table Question Answering
18. Causal Effects of Linguistic Properties
19. Disentangling Representations of Text by Masking Transformers
20. How Individuals Change Language
21. Recipes for Building an Open-Domain Chatbot
22. We Need to Talk About Random Splits
23. ZmBART: An Unsupervised Cross-Lingual Transfer Framework for Language Generation

It was great fun discussing the excellent contributions made by the authors of the above papers. Looking forward to conducting deep dives into more papers as we move into the new format for the paper discussion group!

Aggregate Intellect

Aggregate Intellect

Aggregate Intellect is a Global Marketplace where ML Developers Connect, Collaborate, and Build. Connect with peers & experts at https://ai.science or Join our Slack Community.

• Check out the user generated Recipes that provide step by step, and bite sized guides on how to do various tasks
• Join our ML Product Challenges to build AI-based products for a chance to win cash prizes
• Connect with peers & experts through the ML Discussion Groups or Expert Office Hours