Robustness of Modern Deep Learning Systems with a special focus on NLP

Modern neural network models have achieved outstanding performance on many tasks ranging from Computer Vision to Language Processing. This high performance should entail that they have become exceptionally good at solving such tasks but recent works have shown that “these models often solve a dataset, not the task” as they make surprising failures when given inputs that are slightly different from the ones seen during training or when the inputs are perturbed (adversarial inputs). Researchers in NLP will readily concede that while we…