Open in app
Sign inGet started
[Notes] Gradient Checkpointing with BERT

[Notes] Gradient Checkpointing with BERT

A brief analysis of huggingface’s implementation
Go to the profile of Ceshine Lee
Ceshine Lee
[Paper] Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

[Paper] Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

Essential for fine-tuning T5 v1.1 and mT5 models
Go to the profile of Ceshine Lee
Ceshine Lee
[Paper] Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control

[Paper] Rethinking Cooperative Rationalization: Introspective Extra...

Building competitive self-explaining NLP models
Go to the profile of Ceshine Lee
Ceshine Lee
Reducing the SentencePiece Vocabulary Size of Pretrained NLP Models

Reducing the SentencePiece Vocabulary Size of Pretrained NLP Models

Useful for fine-tuning on a subset of available languages
Go to the profile of Ceshine Lee
Ceshine Lee
[PyTorch Lightning] Log Training Losses when Accumulating Gradients

[PyTorch Lightning] Log Training Losses when Accumulating Gradients

The global step is not what you think it is
Go to the profile of Ceshine Lee
Ceshine Lee
About VeritableLatest StoriesArchiveAbout MediumTermsPrivacyTeams