Member-only story
How to Train a Joint Entities and Relation Extraction Classifier using BERT Transformer with spaCy 3
A step-by-step guide on how to train a relation extraction classifier using Transformer and spaCy3
Introduction
One of the most useful applications of NLP technology is information extraction from unstructured texts — contracts, financial documents, healthcare records, etc. — that enables automatic data query to derive new insights. Traditionally, named entity recognition has been widely used to identify entities inside a text and store the data for advanced querying and filtering. However, if we want to semantically understand the unstructured text, NER alone is not enough since we don’t know how the entities are related to each other. Performing joint NER and relation extraction will open up a whole new way of information retrieval through knowledge graphs where you can navigate across different nodes to discover hidden relationships. Therefore, performing these tasks jointly will be beneficial.
Building on my previous article where we fine-tuned a BERT model for NER using spaCy3 and UbiAI, we will now add relation extraction to the pipeline using the new Thinc library from spaCy. We train the relation extraction model following the steps outlined in spaCy’s documentation. We will compare the performance of the relation classifier using transformers and tok2vec algorithms. Finally, we will test the model on a job description found online.
Relation Classification:
At its core, the relation extraction model is a classifier that predicts a relation r for a given pair of entity {e1, e2}. In case of transformers, this classifier is added on top of the output hidden states. For more information about relation extraction, please read this excellent article outlining the theory of fine tuning transformer model for relation classification.
The pre-trained model that we are going to fine-tune is the roberta-base model but you can use any pre-trained model available in huggingface library by simply inputting the name in the config file (see below).
In this tutorial we are going to extract the relationship between the two…
