In my previous article I covered some of the fundamental concepts in NLP. This time, I will introduce ELMo, BERT and MT-DNN, architectures which leverage these concepts and have recently achieved some State Of The Art (SOTA) results on NLP tasks.

Embeddings from Language Models (ELMo)

An introduction to ELMo can be found in this paper. ELMo aims to provide an improved word representation for NLP tasks in different contexts by producing multiple word embeddings per single word, across different scenarios. In the example below, the word “minute” has multiple meanings (homonyms) so gets represented by multiple embeddings with ELMo. …

Badoo is the largest dating network in the world, with over millions of registered users worldwide. Users of our site are prompted to write an “About me” section in order to give any prospective love interests a little bit more information about themselves.

In my previous blog, I introduced the concept of combining a CNN with an LSTM to generate a caption of an image, and talked about how we are using this at Badoo to help our users find love. This time, we will explore how the incorporation of attention networks can help us improve on and enrich the captions that are generated by our model.

Why attention networks?

To improve the quality of our captions, we often need the decoder to look at different parts of an image when generating a new word in the sequence. Incorporating an attention mechanism essentially allows the model…

Badoo is the largest dating network in the world, with over millions users across 190 countries who upload over 10 million photos per day to our platform. These images provide us with a rich data set we can derive a wealth of insights from.

Our Data Science team use image captioning to describe what is in these images. Image captioning is essentially the process of generating a textual description of a picture’s content.