13 Deep Learning Frameworks for Natural Language Processing in Python

7 min readFeb 5, 2018

by Olga Davydova

In this paper, we discuss the most popular neural network frameworks and libraries that can be utilized for natural language processing (NLP) in the Python programming language. We also look at existing examples of these tools.

A comparative table was specially created. Every cell with a plus sign contains a link to a framework usage example in NLP task and network type perspectives.

Chainer

Chainer, developed by the Japanese company Preferred Networks founded in 2014, is a powerful, flexible, and intuitive Python-based framework for neural networks that adopts a “define-by-run” scheme [1]. It stores the history of computation instead of programming logic. Chainer supports CUDA computation and multi-GPU. The framework released under the MIT License and is already applied for sentiment analysis, machine translation, speech recognition, question answering, and so on using different types of neural networks like convolutional networks, recurrent networks, and sequence to sequence models [2].

Deeplearning4j

Deeplearning4j is a deep learning Java programming library, but it also has a Python API, Keras that will be described below. Distributed CPUs and GPUs, parallel training via iterative reduce, and micro-service architecture adaptation are its main features [3]. Vector space modeling enables the tool to solve text-mining problems. Parts of speech (PoS) tagging, dependency parsing, and word2vec for creating word embedding are discussed in the documentation.

Deepnl

Deepnl is another neural network Python library especially created for natural language processing by Giuseppe Attardi. It provides tools for part-of-speech tagging, named entity recognition, semantic role labeling (using convolutional neural networks [4]), and word embedding creation [5].

Dynet

Dynet is a tool developed by Carnegie Mellon University and many others. It supports C++ and Python languages, runs on either CPU or GPU [6]. Dynet is based on the dynamic declaration of network structure [7]. This tool was used for creating outstanding systems for NLP problems including syntactic parsing, machine translation, morphological inflection, and many others.

Keras

Keras is a high-level neural-network based Python API that runs on CPU or GPU. It supports convolutional and recurrent networks and may run on top of TensorFlow, CNTK, or Theano. The main focus is to enable users fast experimentation [8]. There are many examples of Keras usage in the comparative table: classification, text generation and summarization, tagging, parsing, machine translation, speech recognition, and others.

Nlpnet

Erick Rocha Fonseca’s nlpnet is also a Python library for NLP tasks based on neural networks. Convolutional networks enable users to perform part-of-speech tagging, semantic role labeling, and dependency parsing [9]. Most of the architecture is language independent [10].

OpenNMT

OpenNMT is a Python machine translation tool that works under the MIT license and relies on the PyTorch library. The system demonstrates efficiency and state-of-the-art translation accuracy and is used by many translation providers [11]. It also incorporates text summarization, speech recognition, and image-to-text conversion blocks [12].

PyTorch

PyTorch is a fast and flexible neural network framework with an imperative paradigm. It builds neural networks on a tape-based autograd system and provides tensor computation with strong GPU acceleration [13]. Recurrent neural networks are mostly used in PyTorch for machine translation, classification, text generation, tagging, and other NLP tasks.

SpaCy

Developers called spaCy the fastest system in the world. They also affirm that their tool is the best way to prepare text for deep learning. Spacy works excellent with well-known Python libraries like gensim, Keras, TensorFlow, and scikit-learn. Matthew Honnibal, the author of the library, says that spaCy’s mission is to make cutting-edge NLP practical and commonly available [14]. Text classification, named entity recognition, part of speech tagging, dependency parsing, and other examples are presented in the comparative table.

Stanford’s CoreNLP

Stanford’s CoreNLP is a flexible, fast, and modern grammatical analysis tool that provides APIs for most common programming languages including Python. It also has an ability to run as a simple web service. As mentioned on the official website, the framework has a part-of-speech (POS) tagger, named entity recognizer (NER), parser, coreference resolution system, sentiment analysis, bootstrapped pattern learning, and open information extraction tools [15].

Tensorflow

The Google Brain Team developed TensorFlow and released it in 2015 for research purposes. Now many companies like Airbus, Intel, IBM, Twitter and others use TensorFlow at production scale. The system architecture is flexible, so it is possible to perform computations on CPUs or GPUs. The main concept is flow graphs usage. Nodes of the graph reflect mathematical operations, while the edges represent multidimensional data arrays (tensors) communicated between them [16]. One of the most known of TensorFlow’s NLP application is Google Translate. Other applications are text classification and summarization, speech recognition, tagging, and so on.

TFLearn

As Tensorflow is a low-level API, many high-level APIs were created to run on top of it to make the user experience faster and more understandable. TFLearn is one of these tools that runs on CPU and GPU. It has a special graph visualization tool with details about weights, gradients, activations, and so on [17]. The library is already used for sentiment analysis, text generation, and named entity recognition. It lets users work with convolutional neural networks and recurrent neural networks (LSTM).

Theano

Theano is a numerical computation Python library that enables users to create their own machine learning models [18]. Many frameworks like Keras are built on top of Theano. There are tools for machine translation, speech recognition, word embedding, and text classification. Look at Theano’s applications in the table.

Summary

In this paper, we described neural network supporting Python tools for natural language processing. These tools are Chainer, Deeplearning4j, Deepnl, Dynet, Keras, Nlpnet, OpenNMT, PyTorch, SpaCy, Stanford’s CoreNLP, TensorFlow, TFLearn, and Theano. A table lets readers easily compare the frameworks discussed above.