What is NLP?

NLP: Natural Language Processing.

As the name suggests NLP on natural language processing in walls, computational processing of natural or human languages. Analysis of written or spoken languages is helpful in various tasks like spam detection, your emails, translation of text from one language to the other, grammar correction. Or even something as simple as search, where the machine has to understand your search query and present relevant results to you.

One of the key components of NLP is language modelling. Language modelling is a fundamental task in modern NLP applications. It enables machines to understand qualitative information and enables people to communicate with machines in natural languages that humans used to communicate with each other.

Language modelling is used directly in a variety of industries including tech, finance, healthcare, transportation, legal, military, government and more. Actually, you probably have just interacted with the language model today, whether be through search, engaging with the voice assistant, or using text auto complete features.

So how does language modelling work?

The roots of modern language modelling can be traced back to 1948 when Claude Shannon published a paper title, a mathematical theory of communication. Laying the foundation for information theory and language modelling. In the paper, Shannon detail the use of stochastic model called Markov chains to create a statistical model for the sequences of letters in English text.

The Markov models along with n-grams are still among the most popular statistical language models today. However, simple statistic language models has serious drawbacks in scalability and fluency, because of the sparse representation of language. A way to overcome these shortcomings is by representing language units such as words or characters in a continuous vector space,. Neural language models use this technique and are even able to find representation for rare or unknown words. So these are the two key types of language models, the statistical language models that use traditional statistical techniques. Like anagrams, hidden Markov models and certain linguistic rules to learn the probability distribution of words. And the neural language models that use different kinds of neural networks to model the language.

In recent years, new language models have surpassed the performance of statistical language models. Hence in this module will focus on the new language model and more specifically, a language model called bird. It has delivered exceptional results for natural language processing tasks.

Key Discoveries:

• Computational Processing of Natural or Human Languages
• NLP is useful in tasks like Search, Spam Detection, Language Translation, Grammer Correction etc.
• Key component of NLP: Language Modelling: Probability of given sequence of words occuring in a sentence.
• Origin of Modern Language Processing — “A Mathematical Theory of Communication”, “laying the foundation for information theory and language modeling” by Claude Shannon.
• Types of language modelling: Statistical Language Models & Neural Language Models.

Some common tasks that are used to evaluating NLP models: