NLP. What is Natural Language Processing?
NLP is a crucial part of artificial intelligence. It’s getting computer systems to recognize, understand and respond intelligently to human languages to be able to communicate with people.
Applications of natural language processing are spreading today wide and fast. They range from simple tasks like spell checking and keyword search to very complex tasks, that require a really high level of human language understanding, like machine translation, dialog systems, question answering, stock trading and so on. Today many applications can already communicate with the user and help him to solve different tasks: like talking to Siri or booking a restaurant. These extremely compelling things were made possible thanks to NLP technologies. Most of us are dealing with NLP applications every day and it is becoming an integral part of modern life.
This sheer speed of NLP adoption creates an erroneous impression that the natural language processing is an easy and solved task, but it’s not. NLP is a very special kind of artificial intelligence because language is a distinctive property of human being. Humans use and understand natural language mostly intuitively. We think in terms of the language, we use words and sentences to express our thoughts, we are lying and assuming with words. Understanding human language is far more than simply knowing the meaning of every word in a language.
One of the main factors that make NLP difficult for machines is ambiguity.
Words can have different meaning depending on the context, they can change their meaning over time. How would you understand the word “JAGUAR”? Is it an animal, a car or somebody’s name?
A phrase or a sentence can be given two or more different interpretations as a result of the words arrangement: I met tall boys and girls. Are we talking about tall boys and tall girls or only boys who are tall?
Some things can be intuitively understood by a human, but are really hard to understand for a computer. For example, look at the sentence: As I reached the bank at closing time, the bank clerk helpfully shut the door in my face. Even if the computer would correctly understand every word, it might fail to understand the meaning of the whole sentence.
While communicating with each other we are relying on common sense and knowledge about how the world works. Computers don’t have that.
Take these two examples:
I ate the spaghetti with meatballs. — I ate the spaghetti with chopsticks.
Did I eat meatballs or chopsticks? Did I use meatball or chopsticks to eat?
As you see, NLP is a difficult, but exciting task. It already has a long history, many great algorithms were developed to enable computers to process and understand language. But today we are facing a new era in NLP. Tremendous amounts of spoken and written material on the web, fast computers, and advanced machine learning techniques have enabled impressive advances in all AI fields, including NLP. They yield new state-of-the-art results for most NLP tasks and make all the recent magic NLP applications possible.
Okay, that’s it for today. Next time we’ll talk about one of the most important techniques in the modern NLP called word2vec. See you ;))
