Summary: Learning Sequence Encoders for Temporal Knowledge Graph Completion
Authors: Alberto Garcıa-Duran, Sebastijan Dumancic, Mathias Niepert
Link prediction in knowledge graphs (KG) is a task of predicting the actuality of a fact, which usually consists of learning a representation for each component of that fact and then calculate a scoring function based on those representations. The underlying idea in this work is doing link prediction for a KG that has temporal information, i.e., there are relations in the graph that may hold for a time interval or a specific point in the time. As an example of this temporal information, the facts can be in the form of (Barack Obama, born, US, 1961) or (Barack Obama, president, US, occursSince, 2009–01).
To capture the temporal information, the paper firstly encodes the relation and the time information in each fact into a sequence of tokens (they call it predicate sequence). Accordingly, for each digit corresponding to the year they add a “y”, for a month they add an “m”, and for a day they add a “d” after that digit. Examples of this encoding process are provided below.
Then, after mapping each token to its d-dimensional embedding via using a linear layer they apply an LSTM layer to the embeddings to find a vector representation for the predicate sequence. Their model is presented in the following figure.
Finally, to calculate the score of any fact, they use TransE and DistMult scoring function on entities and predicate sequence embeddings. They evaluate their method on 4 datasets, improving the MRR and Hits@ metrics in all of them. Personally, I found this work really interesting firstly because of the fact that they address a less studied aspect of KGs, i.e., temporal property. And secondly, because of their unique way of handling this information which is completely different than previous methods with numerical information like this work and our own approach in this paper.
