Understanding emotions — from Keras to pyTorch

Introducing torchMoji, a PyTorch implementation of DeepMoji

Thomas Wolf
Oct 4, 2017 · 6 min read
torchMoji/DeepMoji model

How to build a custom pyTorch LSTM module

Keras default LSTM VS pyTorch default LSTM

Attention layer: side-by-side Keras & pyTorch

pyTorch attention module
Keras attention layer

When it comes to writing and debugging custom modules and layers, pyTorch is a faster option while Keras is clearly the fastest track when you need to quickly train and test a model built from standard layers.

How the PackedSequence object works

A typical NLP batch with five sequences and a total of 18 tokens
Packing a batch in a PackedSequence object
How to construct a PackedSequence object (with batch_first=True)

Smart data loading in pyTorch: DataSets & Batches

From Keras to pyTorch: don’t forget the initialization

Conclusion

HuggingFace

Stories @ Hugging Face

Thanks to Clément Delangue.

Thomas Wolf

Written by

Natural Language Processing, Deep learning and Computational Linguistics – Science Lead @Huggingface | thomwolf.io

HuggingFace

Stories @ Hugging Face