Custom Named Entity Recognition (NER) model with spaCy 3 in Four Steps

If you found this article useful. Kindly subscribe and support me to write more articles.

Named-entity recognition is a subtask of information extraction that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages,etc.,

Link to Code: https://github.com/Vijayvj1/Custom_NER_Spacy3

Step 1: Installation

Check the spaCy Version

Step 2: Creating training data

Now, the major part is to create your custom entity data for the input text where the named entity is to be identified by the model during the testing period.

Note: spaCy v3.1, however, no longer takes .json format and this has to be converted to their .spacy

Code to Convert .jsonto .spacy

To run in CL(Command Line) mode:

.spacy file generated

Step 3: Creating the configuartion and Training the model

1. Creating the configuartion:

Before doing the training you need to create the config file. To create custom config file follow here

Run in CL(Command Line) mode:

This will config file that you downloaded from Spacy’s widget with the defaults. You can play around with the defaults and tweak it as you see fit but let’s just go with the default for now.

2. Training the model:

Once that’s done, you’re ready to train your model! At this point, you should have three files on hand: (1) the config.cfg file, (2) your training data in the .spacy format and (3) an evaluation dataset. In this case, I didn’t create another evaluation dataset and simply used my training data as the evaluation dataset (not a good idea but just for this article!). Make sure all three files are in the folder that you’re running the CLI in. Here, I also set --gpu-id to 1 in order to select my GPU.

Run in CL(comman line) mode:

1. E — Epochs
2. # — optimization steps
3. LOSS NER — model loss
4. ENTS_F, ENTS_P, and ENTS_R — precision, recall and fscore of the model

Step 4: Prediction

Load the model

Yes!! We did it!!

Conclusion

I hope you have now understood how to train your own NER model with the spaCy 3 NER model. Thanks for reading!

If you found this article useful. Kindly subscribe and support me to write more articles.

If you want to learn more about machine learning, continue reading my blogs:

1. Audio Data Augmentation: https://vijay-anandan.medium.com/lets-augment-a-audio-data-part-1-5ab5f6a87bae
2. Sentiment Analysis On Voice Data: https://vijay-anandan.medium.com/sentiment-analysis-of-voice-data-64533a952617
3. Resample an extremely imbalanced datasets: https://vijay-anandan.medium.com/how-to-resample-an-imbalanced-datasets-8e413dabbc21
4. How Do Neural Networks Really Work in the Deep Learning:https://medium.com/analytics-vidhya/how-do-neural-networks-really-work-in-the-deep-learning-72f0e8c4c419
5. Sentiment Analysis in Five Steps using AutoML: https://medium.com/analytics-vidhya/sentiment-analysis-in-five-steps-using-automl-d16feeab2a36

https://www.linkedin.com/in/vijay-anadan/