Natural Language Processing (NLP), JavaScript, NPM packages, and a Chatbot
Intro to NLP
What is Natural Language Processing or NLP? According to the wiki, NLP is:
A subfield of computer science, information engineering, and artificial intelligence concerned with the interactions between computers and human (natural) languages, in particular how to program computers to process and analyze large amounts of natural language data. Challenges in natural language processing frequently involve speech recognition, natural language understanding, and natural language generation.
Now, NLP relies heavily on machine learning, according to the wiki. In other words, right now it is easy to handle human language data with the help of machine learning. As someone who has been using some digital tools to analyze some literary texts and who has been learning JavaScript as a tool for web development, here I would like to explore options to see if there is any possibility for us to perform NLP tasks by using JavaScript and Node.js.
What are the basic tasks for NLP? According to the wiki and this website, some top basic tasks are: stemming, part-of-speech(POS) tagging, parsing, sentence breaking, stemming, word segmentation, semantic text similarity, language identification, and so on. More advanced tasks would include sentiment analysis, machine translation, topic modeling, and so on. In other words, if we know how to perform those basic and advanced tasks, it is possible for us to develop an application that could, say, identify a language, do machine translation, or render a dictionary. It is also possible to build something interesting such as a chatbot who can talk to a user, who can identify the intent of the user, who can help the user to search something on a website, or who can detect one’s negative emotion and reply with a positive message.
3 Useful NPM Packages for NLP
While Python is always the tool for doing almost all NLP tasks because of the availability of Natural Language Toolkit, JavaScript could also do almost everything, with the help from NPM packages. As of today, if you put “NLP” as the search term on npmjs.com, you can find 557 packages. Some of the packages can do NLP for many different languages. For example, the NLP.js package can deal with 29 languages, while “HanziJS” is particularly working for the Chinese language. That means, it is still possible to do NLP in JavaScript if we could npm install <correct-package-for-nlp>. I have tried out several packages, while for some, I may not figure out the correct way of using them, for some, I find them very easy to use.
For example, “Natural” and “HanziJS” are two perfect tools for performing basic NLP tasks for English and Chinese languages. Both have offered people easy-to-use tools as well as clear documentation for performing tasks such as stemming, tokenization, calculating string distances, part-of-speech tagging, checking definitions of a word or a phrase, as well as some tasks unique to a particular language such as getting the radicals of a Chinese character. Both are perfect tools for not only basic tasks, but also advanced stuff such as performing sentiment analysis or building an online dictionary.
Another package that is particularly useful for building a chatbot is “NLP.js”, “an NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more” according to their GitHub page. I tried it out by git cloning the script on their GitHub page. It worked perfectly fine. You can even customize the training model by importing excel files according to the official documentation.
An NLP Example: Building a Chatbot with NLP
Build better voice apps. Get more articles & interviews from voice technology experts at voicetechpodcast.com
Inspired by these NPM packages, I then tried to explore the ways of building a basic chatbot with NLP incorporated, since to build a chatbot appears to be the perfect way to show how NLP could be applied to software development. Currently, it is possible to build a basic chatbot by using Microsoft Bot Framework service and Node.js. Also, according to the Microsoft official document, to add NLP to a basic chatbot can also be done with Microsoft’s Language Understanding service (LUIS). I thus followed the official tutorial and built a very basic chatbot which could ask for user’s input.
How does this chatbot work? First, we need a back-end server. Microsoft’s tutorial is using Restify, a Node.js web service framework as a back-end server. Alternatively, NLP.js package’s official tutorial is using Express.js as the back-end server. Then, each message sent by the user is in fact an object while each message sent out by the bot is an object as well. Messages are in fact POSTing to the back-end server.
After developing the bot, it is also possible to put it on a website by using <iframe> tag, or connect a bot to Slack, or convert it into a Facebook Messenger bot. It seems that there are many options for one to put the bot online!
But as of now, I’m still in the process of exploring the ways of developing a chatbot with NLP, and hopefully, in the future, a chatbot which could naturally understand people’s messages would be a part of my project!
