Using RoBERTa with fast.ai for SuperGLUE Task CB

Finetuning state of the art RoBERTa with fast.ai on Commitment Bank NLP task.

This tutorial will heavily rely and build on my previous work on using Roberta with fast.ai for IMDB sentiment analysis. We will simply be modifying our code from earlier to parse the data and customize the appropriate Roberta model for the SuperGLUE task, CB. This tutorial can also be applied to other tasks BoolQ and RTE with slight modifications.

Update 2020.11: fast.ai has upgraded to v2 since the release of this article. For compatibility with the steps below, v2 remains untested. Therefore, usage of v1 is recommended for following along with this article.

Commitment Bank or CB is an NLP task where given a premise and a hypothesis, we classify how committed the author of the premise is towards the truth of the hypothesis. The three possible labels, then, are entailment, contradiction or neutral. Take a look at this example of entailment:

• Premise: It was a complex language. Not written down but handed down. One might say it was peeled down.
• Hypothesis: the language was peeled down
• Label: Entailment

Example of contradiction:

• Premise: A: I, that would have…