NAACL 2016 : 3 papers

At the panel on Tuesday, Pascale Fung made the comment that our goal as a scientific community is to find the right problems to ask. As an applied field of machine learning, often what this means in practice is to find tasks that push our sense of what’s possible. I always like these kinds of paper, and there were three papers this year that especially caught my attention in this regard.

• Stating the Obvious: Extracting Visual Common Sense Knowledge
• Conversational Flow in Oxford-Style Debates
• Parsing Algerbraic Word Problems into Equations

Stating the Obvious: Extracting Visual Common Sense Knowledge

It has been incredibly exciting to see the attention that grounded semantics has seen in the past few years in the NLP community. It wasn’t too long ago that language and vision seemed like an oddball topic — and work from people like Deb Roy and Michael Fleischman seemed completely irrelevant to mainstream work in the field. This always seemed like a loss to me. The grounded semantics work probes the notion that there might be insights on language meaning that are inaccessible simply from observing text and that we can gain leverage into understanding the meaning of text by resorting to extra-textual means.

This paper demonstrates a very clean example of this. By observing natural images, and leveraging the high quality object segmentation and recognition the paper demonstrated induction of common sense propositions like “dishes go on table, but cars do not”. Information like this is difficult to induce directly from text, since it is never explicitly stated, but exists as a common set of knowledge that authors apply when constructing a narrative. It is almost necessary, from a pragmatic perspective, to omit these details — because their inclusion would suggest a relevance to this information that doesn’t exist.

This paper was one of the first, in my mind, to show that this kind of wide coverage common sense can be induced from images at a large scale — hundreds of thousands of propositions. This is in contrast to the failed research paradigm of manual common sense construction as explored by the Cyc project.

Clearly, there is more work that must be done in this line. These common sense directives weren’t connected back to a language understanding task. The generalization of this work to common sense theorems beyond spatial relationships is unclear. But it seemed like a very significant paper to me.

Parsing Algerbraic Word Problems into Equations

There has been a flourishing of work on reading comprehension from the AI2 institute over the past few years, and it has been wonderful to see. Lynette Hirshmann proposed reading comprehension as a task in the early 2000s, but it was quickly overshadowed by open domain question answering as a task. This always seemed like a bit of a loss to me — there was something very clean in Lynette’s phrasing of the problem as being able to demonstrate understanding of a text by being able to answer questions after reading it that you wouldn’t be able to answer before having read it.

This paper comes from a subset of work in reading comprehension that focuses on math problems. There have been a few phenomenal papers on solving the math word problems, but they always felt quite constrained — for example mapping a word problem into one of several specific templates. This paper does something interesting by constructing a semantic grammar on top of a sentence parse and then using that to do inference. This feels like it will scale and generalize much better than the template-ish methods of the past few years. It is still unclear, however, how this method will scale to domains beyond mathematics — where the set of possible semantic propositions is much larger then the four mathematic operators.

Conversational Flow in Oxford-Style Debates

The field of computational social science is undergoing a really interesting maturation right now, with a ton of interest work applying nlp techniques to understanding not just what people communicate but how and why. As an example, the work on framing is a very nice example. But to my recollection, there still hasn’t been a ton done on the nature of argumentation — why are some arguments convincing while others are not.

This paper was interesting from two perspectives. First, it presented a new corpus of debates that will serve as useful material for significant future work — the set of debates from the “Intelligence Squared Debates”. Debates are interesting for a number of reasons that the paper elucidates including that the audience explicitly votes on the winning argument.

The other interesting thing about the paper was what they demonstrate — that the argument that is more responsive to its opponent is more likely to be convincing. Of course, this ignores (as an audience member questioned) the underlying factual support for the various sides, but I think this kind of phenomena is absolutely real, and very difficult to study in any other way.