Paper accepted at INLG2019
We are pleased to announce that a paper from OMRON SINIC X (in collaboration with Kyoto University) was accepted at International Conference on Natural Language Generation (INLG) 2019!
Taichi Nishimura, Atsushi Hashimoto, and Shinsuke Mori, “Procedural Text Generation from a Photo Sequence”
Overview
Multimedia procedural texts, such as instructions and manuals with pictures, support people to share how-to knowledge. In this paper, we propose a method for generating a procedural text given a photo sequence allowing users to obtain a multimedia procedural text. We propose a single embedding space both for image and text enabling to interconnect them and to select appropriate words to describe a photo. We implemented our method and tested it on cooking instructions, i.e., recipes. Various experimental results showed that our method outperforms standard baselines.
