Week 6 — This is the way!
by Baran Orhan and Erdem Korhan Erdem
Hola, AI geek. It’s nice to see you again.
In the last week, we tried to extract the skills out from raw texts using a bidirectional LSTM model, however could not end up with a satisfactory result. This week, we have decided to change our model choice and made a significant amount of progress.
Transformers!!
After getting partially correct results by using biLSTMs, we started to think whether we do not have enough data to train a robust model, or the model we are creating is not complex enough. To handle this dilemma, we started to search for the bigger ready-to-use datasets and more complex model options. In fact, we even ended up with a public dataset that we thought it might help us grow our data amount. At the same time, we encountered the beautiful, one and only, the state of art: Transformers.
For our Named Entity Recognition task, we decided to use bert-base-uncased pretrained model by Hugging Face. Our model has been trained for 12 epochs, with a learning rate of 0.0001. Here are the metrics we have obtained finally:
And, here are some of the predictions that the model has provided:
As can be seen, we are finally able to get satisfactory results. That is, we have almost finished the NER part of our whole recommendation system. In the next step, we will need to compute the similarities between skills of engineers and the extracted skills. For this purpose, we are planning to use cosine similarity. Below, you can see a general diagram of our system.
Under these conditions, we are looking forward to compute the similarities between course outcomes and engineer skills to finally recommend related courses. Until next blog, keep yourself up and get ready for a very unique course recommendation system, au revoir.
