It is easy to confuse common terminology in the fast-moving world of machine learning. For example, the term NLU is often believed to be interchangeable with the term NLP. But NLU is actually a subset of the wider world of NLP (albeit an important and challenging subset).
Natural Language Processing (NLP) refers to all systems that work together to handle end-to-end interactions between machines and humans in the preferred language of the human. In other words, NLP lets people and machines talk to each other “naturally.”
NLP is a critical piece of any human-facing artificial intelligence. An effective NLP system is able to ingest what is said to it, break it down, comprehend its meaning, determine appropriate action, and respond back in language the user will understand.
Meanwhile, Natural Language Understanding (NLU) encompasses one of the more narrow but especially complex challenges of AI: how to best handle unstructured inputs that are governed by poorly defined and flexible rules and convert them into a structured form that a machine can understand and act upon. While humans are able to effortlessly handle mispronunciations, swapped words, contractions, colloquialisms, and other quirks, machines are less adept at handling unpredictable inputs.
A good rule of thumb is to use the term NLU if you’re just talking about a machine’s ability to understand what we say.
To build machines that understand natural language, we must distill speech into a structured ontology using a combination of rules, statistical modeling, or other techniques. Entities must be extracted, identified, and resolved, and semantic meaning must be derived within context, and be used for identifying intents. For example, a simple phrase such as: “I need a flight and hotel in Miami from October 4 to 10” must be parsed and given structure:
need:flight {intent} / need:hotel {intent} / Miami {city} / Oct 4 {date} / Oct 10 {date} / sentiment: 0.5723 (neutral)
Computational linguistics has become a critical area of interest in recent years, as companies work to build systems capable of effortless, unsupervised, and socially acceptable direct interaction with customers. Everyone from small tech startups (like Lola) to the major technology companies like Amazon (Alexa) and Apple (Siri) are investing in efforts to make their systems feel more human. New and exciting things are happening in this field everyday, and I’m excited to be a part of bringing these advances to life.
Interested in how we use Artificial Intelligence in Corporate Travel? Visit Lola.com to learn more.
-Bryan (Director of AI)
