How to Create and Train a Multi-Task Transformer Model

A step-by-step tutorial on how to create and train a multi-task transformer model using Huggingface Transformers.

Photo by Vlad Zaytsev on Unsplash

While working on an AI chatbot project, I did a short review of the available companies that offer NLP models as a service. I was surprised by the cost that some providers charge for basic intent classification models and even more for those that offer both intent classification and a token classification task(i.g. part-of-speed tagging).

In my opinion, the only added value of these services is the deployment and maintenance. Since the open-source Huggingface Transformer library (the current standard for training such models) reached maturity and is usable by anyone without extensive knowledge of Transformer models.

I understand that deployment and maintenance are critical aspects of any production application (knowing that maintenance takes 70% on average of the cost of any software). However, there are some disadvantages to going this route:

• You have less control over the model used under the hood. You treat the model as a black box that you access via an API. Furthermore, you are always bound by the release cycles to get the latest models architectures (if they ever get added).