Evaluate Camembert, Flaubert and DistilCamemBERT on a sentiment analysis dataset

In the world of Natural Language Processing (NLP), the French language is getting a lot of attention lately, particularly with models like Camembert and Flaubert. These models are a big step forward in how we understand and generate language, thanks to their transformer architecture. However, there’s still a shortage of easy-to-follow resources and guides, especially when it comes to practical uses like sentiment analysis.

I’m working on a package called ‘nlpbaselines’, aimed at simplifying the evaluation of these models. My goal is to make it easier to fine-tune and understand how these models perform and to show that evaluating them can be straightforward and accessible.

In this tutorial, I’ll fine-tune 14 models in a few lines and compare them on several dimensions, featuring models derived from Camembert, Flaubert and Bert. A special guest would be DistilCamemBERT: a distillation version of CamemBERT (and it works remarkably well).

Install the package

pip install nlpbaselines

Prepare the dataset

These lines load a training and validation set from this dataset. It’s scraped from Allociné.fr user reviews with 2 labels: negative and positive. I’ll use…