MSc Data Science in the UK — Applied NLP — Week2

Hi there everyone, welcome to my week 2 Master’s in the UK session.

For this category, we will be looking at the course Applied NLP at Sussex.

The professor will be using the book shown below as our main material. I will extract the important points from the book and share them with you here. Doing this will also give me a chance to review what I’ve been taught and will definitely consolidate my understanding of the field.

Hope you guys enjoy and let’s dive in.

First Chapter:

1. KNOWLEDGE IN SPEECH AND LANGUAGE PROCESSING

Phonetics and Phonology

Explanation: These are the fields of knowledge concerned with the physical sounds in speech. Phonetics deals with the articulation and acoustic properties of these sounds, while phonology addresses the way sounds function within a particular language or language.

Example: For instance, a speech recognition system must be able to distinguish between the phonetic sounds of ‘s’ and ‘sh’ to accurately convert speech to text. Understanding the role of intonation, stress, and pitch also falls under this category.

Morphology

Explanation: This refers to the structure of words, breaking them down into their smallest units of meaning, known as morphemes.

Example: For example, “unhappiness” can be broken down into three morphemes: ‘un-’, ‘happy’, and ‘-ness’. A sophisticated language processing system should be able to identify these components to understand that “unhappiness” is the state (‘-ness’) of not (‘un-’) being happy.

Syntax

Explanation: Syntax deals with how words are ordered to form grammatically correct sentences, taking into account the role and relationship of each word in a sentence.

Example: For instance, in the sentence “The cat sat on the mat,” syntactic rules dictate the order of subject (“The cat”), verb (“sat”), and object (“on the mat”).

Semantics

Explanation: Semantics refers to the meaning of words and sentences. It involves understanding definitions, as well as the relationships between different words and how their meanings can change in different contexts.

Example: A question-answering system dealing with “How much Chinese silk was exported to Western Europe by the end of the 18th century?” would need to understand the semantic nuances in terms like “exported,” “Western Europe,” and “end of the 18th century.”

Pragmatics

Explanation: This relates to how language is used in practice and how context influences the meaning of utterances. It includes understanding the speaker’s intentions and the situation in which the communication is occurring.

Example: In the request “HAL, open the pod bay door,” the use of “HAL” specifies the intended recipient, and the phrase “open the pod bay door” is a directive. Understanding this request-action relationship is a part of pragmatics.

Discourse

Explanation: Discourse knowledge is required to understand and interpret larger units of text, beyond individual sentences. It involves skills like co-reference resolution, which is identifying what a pronoun or a phrase refers back to in a given context.

Example: In a question like “How many states were in the United States that year?” the phrase “that year” refers back to a previously mentioned context, perhaps the year Lincoln was born. Understanding this relationship is crucial for correctly answering the question.

Each of these dimensions of linguistic knowledge is crucial for building sophisticated language processing systems, allowing them to interact in a manner that is contextually appropriate, syntactically accurate, and semantically meaningful.

2. Types of Ambiguities:

Morphological or Syntactic Ambiguity

Explanation: This type of ambiguity pertains to the role or function of words within a sentence. For instance, a word may serve as a noun in one context and as a verb in another.

• Example: In the sentence “I made her duck,” the word “duck” can either be a noun (representing the bird) or a verb (to lower oneself quickly).
• Resolution Method: Part-of-speech tagging is a standard technique for resolving this type of ambiguity.

Semantic Ambiguity

Explanation: This involves multiple meanings for a single word, based on its context.

• Example: The word “make” can mean “to cook” or “to create,” depending on the context.
• Resolution Method: Word sense disambiguation techniques, often leveraging machine learning models or rule-based algorithms, can resolve semantic ambiguities.

Syntactic Ambiguity in Sentence Structure

Explanation: Here, the ambiguity lies in the arrangement of words and how they relate to one another to convey different meanings.

• Example: The phrase “I made her duck” could imply that “her” and “duck” are distinct entities (she owns the duck), or it could mean that the speaker caused “her” to perform the action of ducking.
• Resolution Method: Probabilistic parsing, often employing context-free grammars and statistical models, can be used to resolve such ambiguities.

Speech and Phonetic Ambiguity

Explanation: In spoken language, words may sound similar but have different meanings.

• Example: The spoken words “eye” and “I” or “maid” and “made” sound similar but have different meanings.
• Resolution Method: Advanced speech recognition systems that consider context and employ deep learning techniques can resolve these ambiguities.

Pragmatic and Discourse Ambiguity

Explanation: This extends beyond the sentence level and involves understanding the intent behind the speaker’s utterances and how sentences relate to a discourse.

• Example: Deciding if a sentence is a statement or a question based on intonation and context.
• Resolution Method: Speech act interpretation and co-reference resolution algorithms can often resolve these types of ambiguities.

Ambiguities exist not just in isolated examples but are rampant in complex NLP tasks. For instance, a text-to-speech system needs to resolve lexical ambiguities to correctly pronounce words like “lead,” depending on the context. Similarly, question-answering systems must perform multiple levels of disambiguation to provide accurate responses.

By considering these facets of ambiguity, the algorithms and models used in NLP aim to interpret human language in a manner that is as nuanced and context-sensitive as possible. Resolving ambiguities is therefore not just a challenge but a necessity for effective language processing systems.

3. Models and Algorithms in Language Processing

The domain of Natural Language Processing (NLP) has seen a proliferation of theoretical models and algorithms aimed at comprehending and generating human language. These models are primarily derived from established domains such as computer science, mathematics, and linguistics.

State Machines and Rule Systems:

Finite-state machines, deterministic or non-deterministic, and their variants like finite-state transducers, are employed for phonological, morphological, and syntactic aspects of language. Formal rule systems such as regular grammar and context-free grammar serve a similar purpose and can be augmented with probabilistic variants for more nuanced analyses.

Logic:

Particularly first-order logic and its related formalisms like lambda-calculus, play a critical role in capturing semantics and pragmatics of language. While they offer a structured approach, they are not inherently well-equipped to deal with the probabilistic nature of language.

Probabilistic Models:

These models address the ambiguity inherent in language. Hidden Markov Models (HMMs), for instance, find applications across various NLP tasks such as part-of-speech tagging, speech recognition, and machine translation. These models inherently answer the question: “Given N choices for some ambiguous input, which is the most probable?”

Vector-Space Models:

Originating from linear algebra, these models are instrumental in information retrieval and semantic analysis. They provide a mathematical framework for semantic relations among words or documents.

Algorithms:

Computational tractability is often achieved via state space search algorithms like dynamic programming for probabilistic models and heuristic searches like A* for non-probabilistic tasks.

Machine Learning Tools:

Classifiers and sequence models are ubiquitously employed. Decision trees, Support Vector Machines, and Gaussian Mixture Models are examples of classifiers, while Maximum Entropy Markov Models and Conditional Random Fields exemplify sequence models.

The application of these models usually involves traversing through a state space representing possible interpretations of the input, making search algorithms an integral part of NLP.

4. Current Deployments in Speech and Language Processing: An Analytical Overview

The advent of advanced computational capabilities, the ubiquity of the Internet, and the rise of mobile technologies have catalyzed significant strides in the field of speech and language processing. Applications leveraging Natural Language Processing (NLP) and Automatic Speech Recognition (ASR) have increasingly percolated through diverse sectors. Below are some sectors and specific implementations that exemplify the current state of the art in this area:

Travel and Tourism

Companies like Amtrak and United Airlines utilize conversational agents to assist customers in real time. These automated systems can navigate customers through reservation processes and disseminate timely information regarding arrivals and departures.

Technological Stack: Such implementations commonly use rule-based dialog systems or more advanced machine learning models trained on vast datasets of human-agent interactions. They often incorporate ASR and Text-to-Speech (TTS) functionalities.

Automotive and Aerospace Industries

Luxury car brands like Mercedes-Benz have integrated automatic speech recognition systems that allow voice-controlled manipulation of various vehicle functionalities. Similarly, astronauts on the International Space Station employ spoken dialogue systems for specific tasks.

Technological Stack: These systems usually employ robust ASR algorithms that are optimized for noisy environments and different accents. They may also use Contextual Bandit algorithms to optimize system responses based on the user’s past interactions.

Multimedia Search Services

Companies like Blinkx offer video search functionalities, capitalizing on speech recognition technology to index and retrieve spoken content in videos.

Technological Stack: Such services use ASR technology optimized for recognizing speech in noisy and diverse acoustic conditions, often supported by neural network-based models for improved accuracy.

Multilingual Information Retrieval

Google’s translation services exemplify advanced NLP capabilities, allowing users to search for information across language barriers.

Technological Stack: These services commonly employ machine translation models, likely sequence-to-sequence neural networks trained on large bilingual or multilingual corpora. They often use Information Retrieval algorithms to find the most relevant translated content.

Education and Healthcare

Automated systems are being deployed for grading student essays or for assisting in educational or therapeutic scenarios. For example, lifelike animated tutors help children learn to read or assist individuals dealing with aphasia and Parkinson’s disease.

Technological Stack: These systems might incorporate complex NLP pipelines including Named Entity Recognition, Sentiment Analysis, and even Generative Models to simulate human-like responses.

Marketing and Business Intelligence

Companies like Nielsen Buzzmetrics and Collective Intellect provide valuable market intelligence through the automated analysis of social media, discussion forums, and other online platforms.

Technological Stack: These services employ a myriad of NLP algorithms such as sentiment analysis, topic modeling, and named entity recognition to analyze and interpret human language on a large scale.

Conclusion and Future Directions

While these advancements are promising, they also underscore the ongoing challenges and ethical considerations that the field must address, ranging from system interpretability to data privacy. Furthermore, as computational models continue to grow in complexity and capability, the need for rigorous evaluation metrics becomes increasingly imperative. Alan Turing’s assertion that

“we can only see a short distance ahead, but we can see plenty there that needs to be done”

remains apt — signifying a research landscape abundant with opportunities and challenges.

Feel free to drop me a question or comment below.

Cheers, happy learning. I will see you in chapter 2.

The data journey is not a sprint but a marathon.

Medium: MattYuChang

LinkedIn: matt-chang

Facebook: Taichung English Meetup

(I created this group four years ago for people who want to hone their English skills. Events are held regularly by our awesome hosts every week. Follow the FB group link for more information!)