Levels of Natural Language Processing — NLP Part I

As humans, listening and comprehending our native language comes naturally to us at this age. We were definitely not born with the understanding but with years passing we steadily gained expertise in our respective languages. But our “chip” friends are not natural at it. We have to put some extra effort to make them understand our languages. This is where Natural Language Processing comes into the picture.

Natural Language Processing or NLP is about making computers better at understanding and working with human language. By using various techniques and tools, researchers and developers work to teach machines how to analyze and comprehend language, which has countless applications in different fields like healthcare, finance, education, and entertainment. As technology advances, the potential for NLP to make a significant impact in the world only continues to grow.

NLP employs a variety of methods from computer science, linguistics, and artificial intelligence to examine and grasp the structure, significance, and context of natural language. Examples of such techniques include machine learning algorithms, statistical models, and deep learning neural networks.

Levels/Steps of NLP

As mentioned earlier, teaching our language to computers is an extensive task. Hence, in order to make the process more understandable to us, we have divided it into some distinct segments. These segments work as one to perform the ultimate task. Let’s see each one in detail.

1. Morphological

This level of analysis deals with the formation and structure of words. The focus here is on the individual components of the words called morphemes. A morpheme is the smallest meaningful unit of meaning. Any word can be almost divided into three morphemes, which are prefix, stem and suffix. For example in the above text, ‘unimaginable’ can be broken down into un, imagine and able. All three components have individual meanings and together form the word.

2. Lexical

Lexicons are the vocabulary of a language. These are the collection of individual lexemes. Sometimes the form of words is changed to express a grammatical function. For example, if you are referring to more than one cat, you use the word cats. Here you add an additional -s to satisfy the grammatical rules. The extra -s is called an inflectional ending. When a word is stripped of its inflectional endings, it becomes a lexeme.

This level of analysis deals with the form a word takes in the context of its surrounding words. For example, in the above text, ‘goal’ is a lexeme with an inflectional ending -s. However, the word goal can have two meanings:

1. “(in soccer, rugby, hockey, and some other games) a pair of posts linked by a crossbar and typically with a net between, forming a space into or over which the ball has to be sent in order to score.”
2. “the object of a person’s ambition or effort; an aim or desired result.”

It's only when we consider the whole phrase, “scored some ridiculous goals”, we can conclude that the meaning of the word is more aligned with the first definition. All these ambiguities are dealt with in this level of analysis.

3. Syntactic

In this level of analysis, we start dealing with the syntax of the phrases and sentences. Here a phrase is parsed (resolved into its component parts and then the individual role is described as per the syntax) and it is ensured that it is correct in structure. For example in the above text, the sentence “scored some ridiculous goals” has the correct syntax whereas if it was “goals ridiculous scored some” it would have been an incorrect syntax.

4. Semantic

The semantic analysis deals with the determination of the real-world meaning of a sentence. A sentence might be syntactically correct but may not convey any real-world meaning. For example, “Goals have scored some ridiculous Ronaldos in his career” might be syntactically correct but doesn’t make any sense.

5. Discourse

At this level of processing, we start dealing with multiple sentences. Sometimes words in a sentence exist which may not hold a context unless you are given the sentence prior to the current one. For example, in “He has scored some ridiculous goals in his career”, who are we referring to? Who is this he? It's only when you read the sentence prior to it, you understand that we are talking about Ronaldo here.

6. Pragmatic

The pragmatic analysis deals with the outside world knowledge. In the previous steps, we were more focused towards what is being described in the text. In this analysis, however, we deal with what was intended from the text.

The pragmatic analysis emphasizes understanding the true meaning of a text by interpreting what it actually implies, and identifying the different aspects of language that need real-world knowledge. For example, “turn off the AC?” should be interpreted as a request instead of an order.

Summary

Natural Language Processing (NLP) analysis involves breaking down a piece of text into smaller units to understand its meaning. The first step in NLP analysis is morphological analysis, where words are broken down into their smallest units to identify their structure and form. The lexical analysis follows, which involves understanding the meanings of individual words in a text. The next step, syntactic analysis, helps understand the grammatical structure and rules of a sentence. Semantic analysis interprets the meaning of words and phrases in a text based on their context. Discourse analysis is used to understand larger segments of language, such as conversations or paragraphs, to comprehend their meaning and function. Finally, pragmatic analysis is used to understand the social context and implications of language use in a particular situation or culture. By using these six levels of NLP analysis, we can gain a deeper understanding of the meaning and implications of a text.