Zero To One For NLP

All the best NLP resources out there

Photo by Thomas Schweighofer on Unsplash

Come join Maxpool — A Data Science community to discuss real ML problems!

NLP has grown into a field of wide applications ranging from chatbots to news generation. I started my journey into NLP with standard courses but a majority of my practical learnings came from personal blogs of researchers. This metablog is a collection of my favourite blogs and courses I go through every once in a while to refresh myself.

As they say “Finding the right learning material is half the work.”

Basics

• Intro to NLP(Theory) — Stanford OR Jurafsky & Manning book
• Spacy 101
• Text Mining
• Coursera sequence models
• CS224n — Natural Language Processing with Deep Learning — 2019
• BigO for Data Scientists

fastai

1. Fastai ML
2. Fastai DL
3. Fastai NLP

Embeddings

https://github.com/graykode/nlp-roadmap

Word2Vec

• Jalammar
• Lilian
• McCormik — Part 1
• McCormik — Part 2

GLOVE

Attention

• Attention by Lilian

Language models

ELMO

• ELMO by mlexplained
• ELMO by Edward Ma
• ELMO by Eric

Transformer

• Transformer by Ashish Vaswani
• Transformer by Peter
• Transformer by Jalammar
• Transformer by MLexplained

BERT

• BERT by Jalammar
• BERT by mlexplained
• Łukasz Kaiser’s talk

GPT-2 by Jalammar

BERT Vs GPT-2

XLNet by MLexplained

DistilBERT

ALBERT

Embeddings summary by Lilian Weng

Transfer learning

ULMFiT (Universal Language Model For Fine Tuning)

• RNN theory
• Seq2Seq theory
• Neural machine translation(NMT)
• AWD-LSTM
• ULMFiT

Ruder on Multi-task learning

Transfer learning

• NAACL
• INRIA
• Notebook with all transfer learning examples

NN concepts

• Improving NNs
• Why momentum works?
• Learning-rate-tuning
• Cyclical Learning Rates
• Pros and cons of activation functions

Normalisation

• Why batch normalisation works
• Weight and layer normalisation
• Overview of normalisation techniques

4 Sequence Encoding Blocks You Must Know Besides RNN/LSTM

Conversational AI

Neural Approaches review paper

What makes a good conversation?

Denny’s

• DL for chatbots Part 1
• DL for chatbots Part 2

Han’s

• Question answering systems Part 1
• Question answering systems Part 2
• Neural Information Retrieval

Rasa

• Bot levels
• Level 3 bot
• 13 rules for chatbot design
• Rasa NLU in Depth: Part 1 — Intent Classification
• Rasa NLU in Depth: Part 2 — Entity Recognition
• Rasa NLU in Depth: Part 3 — Hyperparameter Tuning

Huggingface

• A Transfer Learning Approach for Neural Network Based Conversational Agents

Putting ML to production

• Full Stack Deep Learning
• Rules of ML by Google
• Deploying ML models
• Deploying NLP models
• ML system design
• ML Ops Day — Oscon 2019
• Deploying transformers
• Technical debt in ML
• Model management

Airbnb

• Zipline — Data management
• Bighead — ML infra

Uber

• Michelangelo
• Scaling Michelangelo
• Big Data Infrastructure
• Uber engineering

Databricks Mlflow

Extra tips

• Training NN on GPUs
• pytorch-lightning
• Distributed training in pytorch
• Faster training with large batches

Libraries

• Custom training in Spacy
• Spacy transformers
• Spacy crash course
• Serving Google BERT in Production

Semantic search

On Semantic Search

• Generic Neural Elastic Search
• Sentence-transformers
• Approximate nearest neighbors
• Sentence-embedding

PyTorch

Get Pro in PyTorch For NLP

Random

• Tricks in NLP
• Tranformers time benchmarking
• Text data augmentation
• Topic modelling by Nanonets
• How to detect fake text?
• An Embarrassingly Simple Approach for Transfer Learning
• Machine Learning Fairness

Hackathons

• Reflecting back on one year of Kaggle
• Kaggle ensemble guide

AutoML

• Automl for predictive modeling

Interview

• Reverse interview
• How to build your project portfolio?

Papers and code

• Papers on language models
• Tasks in NLP
• NLPprogress
• State of the art NLP papers

Interview question

NLP Interview Questions