[Hands-On] Head-based Text Classification with BERT
This code is written for educational purposes.
(You can find the Korean version of the post at this link.)
Classification techniques are incredibly diverse. In this post, we’ll delve into one such technique within the realm of neural networks, specifically focusing on transformer-based classification methods known as “Head-based” classification. This article marks the first installment in our Head-based classification tutorial series, starting with text classification using BERT. The subsequent post will explore image classification based on ViT.
What is Head-Based Classification?
Head-based classification involves appending a “head” (a layer or set of layers of a neural network) to a pre-trained model to carry out a specific task, such as classifying the topics of texts. In this scenario, the “head” is optimized to output predictions for the classification task, effectively leveraging the rich representations learned by the network from ample data to adeptly address the problem at hand — in this case, classification.
By employing a head-based approach, we can fine-tune the pre-trained model on a relatively…
