NLP-Overview, Facebook’s XNLI, ShaRC, AutoAugment, pair2vec, Iterated Amplification,…
Welcome to the 32nd Issue of the NLP Newsletter! Here is an overview of this week’s notable NLP and AI news: enhancing conversational and dialogue AI systems; an AI safety technique that encourages humans and machines working together; an overview of modern deep learning techniques applied to NLP, reinforcement learning and deep learning course materials, and much more.
On People…
Amazon researchers will be hosting their first workshop on fact extraction and verification at EMNLP 2018. Earlier this year, the team released a dataset called FEVER which contains 185,000 assertions of facts together with sentences from Wikipedia that either substantiate or refute those facts. Results from several teams will be discussed at the conference in the coming days — Link
ShaRC is a new dataset that focuses on building end to end conversational question answering systems that support the addition of background knowledge, especially when the system tries to answer a more difficult question, where the answer is not directly found in the text source — Link
Materials for the deep learning and reinforcement learning summer school carried out in Toronto are now available (video format). The materials include topics such as auto differentiation, interpretability, and language understanding — Link
OpenAI proposes a new AI safety technique called iterated amplification. The idea is that we can specify complicated behaviors and goals and be able to generate training signals that go beyond the traditional rewards and labels used in supervised and reinforcement learning, respectively. The interesting part is that humans are able to coordinate with an AI system in a framework that is able to scale up to higher level tasks based on a human-specified sample of smaller subtasks — Link
On Education and Research…
Elvis Saravia and Soujanya Poria release a project called NLP-Overview that is intended to help students and practitioners to get a condensed overview of modern deep learning techniques applied to NLP, including theory, algorithms, applications, and state of the art results — Link
François Fleuret makes available the materials used to teach the introduction to deep learning course given in the African Master’s of Machine Intelligence at AIMS. The materials include topics such as computer vision, optimization, and generative models — Link
Facebook and New York University released a dataset called XNLI which was created for evaluating cross-lingual approaches to natural language understanding (NLU). XNLI also includes baselines that can assist researchers to create systems that understand multiple languages — Link
pair2vec is a method for learning word embedding pairs that represent background knowledge about their implied relationships. The method is able to obtain gains on the SQuaD 2.0 dataset and can also be incorporated into a cross-sentence layer of existing inference models such as BiDAF — Link
On Code and Data…
Priya Dwivedi covers the building blocks of a question answering system built on top of the Stanford Question Answering Dataset (SQuAD) dataset. She explains all the main components that are needed to build an effective QA system — from the embedding layer to the output layer — Link
Mac Brennan provides a detailed post on how to build a neural translation model with attention. He discusses the conceptual understanding of the model and also provides a brief summary of how the model performed — Link
The Facebook AI research team releases a more efficient implementation of R-CNN and Mask R-CNN using PyTorch 1.0. The modular implementations can be used for instance segmentation and object detection — Link
William Wang and colleagues release code and paper for tackling a cross-lingual dialog state tracking problem using a simple but very efficient framework. The framework is based on a transfer learning technique that automatically generates semantic annotations in different languages and is able to track user’s beliefs (this paper is published in EMNLP 2018) — Link
If you want to conduct computer vision (CV) projects and research here is a detailed list of reliable and rare CV datasets used in the field — Link
The TensorFlow team releases AutoAugment which consists of a series of modules provided on TFHub that allows researchers to train better image models with fewer data using image augmentation tricks — Link
On Industry…
IBM releases a dataset which includes information that can be leveraged to build more comprehensive QA systems based on knowledge and reasoning — Link
Duolingo announces their new AI website which contains research publications, open datasets, and AI careers — Link
Baidu releases a new machine translation algorithm called Simultaneous Translation with Anticipation and Controllable Latency (STACL), which performs simultaneous translation using a technique called controllable latency. Controllable latency works by using machine learning to anticipate speaker’s next words instead of having to wait for a speaker to make a pause in their speech to carry out the translation — Link
Worthy Mentions…
An in-depth machine learning tutorial that introduces readers to a machine learning pipeline from scratch using PyTorch — Link
How to implement a recursive neural network using TensorFlow — Link
A research group from Maastricht University develop a GAN-based algorithm to generate logos — Link
If you spot any errors or inaccuracies in this newsletter please comment below.
One last quick thing: any sort of engagement (like follows, shares, 👏👏👏, and feedback) will make a huge difference for the future and sustainability of the dair.ai publication. So I will deeply appreciate any of that in advance.
