Natural Language Processing (Part 46)-Markov Chains
📚Chapter 7: Part of Speech Tagging and Hidden Markov Models
Introduction
This blog will teach you about Markov chains. Markov chains are crucial as they are employed in speech recognition and also utilized for parts of speech tagging. This blog will explore transition probabilities and also delve into the concept of states.
Sections
Concept
What is Markov Chains
Components of a Markov Chain
Practical Applications
Text Generation
Part-of-Speech Tagging
Conclusion
Section 1- Concept
Before jumping in let’s start with a small example, to show what you want to accomplish.
And then you’ll observe how Markov chains can be utilized to achieve the task. In the sentence “Why not learn,” the word “learn” functions as a verb. The subsequent task is to determine if the following word in the sentence acts as a noun, verb, or another part of speech.
If you have a good grasp of the English language, you might predict that a verb in a sentence is typically followed by a noun Rather than another verb. The concept is that the probability of a word’s part of speech tag in a sentence is often influenced by the part of speech tag of the preceding word. Makes sense
Different likelihoods can be represented visually. Here is a circle that represents a verb, and next to it is a circle that represents a noun. Draw an arrow leading from the circle labeled ‘verb’ to the circle labeled ‘noun’. So the following arrow. You can draw another arrow that goes from the verb circle, and loops around back to point at itself.You can associate numbers with each of these arrows. Where a larger number means that there is a higher likelihood, you’re moving from one circle to the other.
In this example, the arrow that goes from the verb so the noun maybe 0.6. Whereas the arrow that goes from the verb, back to the verb has a 0.2. The higher number of 0.6, means that it’s more likely to go from a verb to a noun. As opposed to from a verb, to another verb. This is a great example of how Markov chains work on a very small scale.
Section 2- What is Markov Chains
Def: A Markov Chain is a stochastic model that describes a sequence of possible events, where the probability of each event depends solely on the state attained in the previous event. This “memoryless” property, known as the Markov Property, simplifies the modeling of complex systems by focusing only on the present state to predict the next.
Def: They’re a type of stochastic model that describes a sequence of possible events. To get the probability for each event, it needs only the states of the previous events. The word stochastic just means random or randomness.
Section 3- Components of a Markov Chain
So a stochastic model, incorporates and models processes does have a random component to them. A Markov chain, can be depicted as a directed graph. So in the context of Computer Science, a graph is a kind of data structure that is visually represented, as a set of circles connected by lines. When the lines that connect the circles have arrows that indicates a certain direction, this is called a directed graph.
1- States
Def: These represent the possible events or conditions in the process.
The circles of the graph, represents states of our model. A state refers to a certain condition of the present moment. For example, if you are using a graph to model whether water is in a frozen state, a liquid state, or a gas state. Then you would draw a circle, for each of these states to represent the three possible states that water can be at the present moment. I’m labeling each state as q1, q2, q3 etc. To give them each a unique name. Then referring to the set of all states with a capital letter Q. For this graph there are three states, q1, q2, and q3.
2- Transition Probabilities:
These are the probabilities of moving from one state to another.
3- Initial State:
The state where the process starts.
4- Transition Matrix:
A matrix that represents the transition probabilities between all pairs of states.
Section 4- Practical Applications
- Speech Recognition: Markov Chains are used to model phonemes and their transitions, improving the accuracy of speech-to-text systems.
- Text Prediction: Many predictive text systems, such as those used in keyboards and search engines, leverage Markov Chains to suggest the next word based on previous inputs.
- Language Modeling: In machine translation and other NLP tasks, Markov Chains help in modeling the probabilities of word sequences, aiding in generating more fluent translations.
Section 5- Text Generation
One of the most straightforward applications of Markov Chains in NLP is text generation. By analyzing a large corpus of text, we can build a Markov Chain model that captures the probabilities of word transitions. For example, if we have a sentence “The cat sat on the mat,” we can use it to estimate the probability of “sat” following “cat” and “on” following “sat.”
import random
# Example text
text = "The cat sat on the mat. The mat was soft."
# Create a Markov Chain model
def build_markov_chain(text):
words = text.split()
model = {}
for i in range(len(words) - 1):
if words[i] not in model:
model[words[i]] = []
model[words[i]].append(words[i + 1])
return model
# Generate text using the Markov Chain model
def generate_text(model, start_word, length=10):
current_word = start_word
result = [current_word]
for _ in range(length - 1):
next_word = random.choice(model.get(current_word, [start_word]))
result.append(next_word)
current_word = next_word
return ' '.join(result)
# Build the model and generate text
model = build_markov_chain(text)
generated_text = generate_text(model, start_word="The", length=10)
print(generated_text)This simple example demonstrates how a Markov Chain can be used to generate text by selecting the next word based on the current word’s transition probabilities.
Section 6- Part-of-Speech Tagging
Markov Chains are also used in Part-of-Speech (POS) tagging, where the goal is to assign POS tags (like noun, verb, adjective) to each word in a sentence. The Hidden Markov Model (HMM), an extension of Markov Chains, is particularly useful for this task. In HMM, we consider the sequence of hidden states (POS tags) and observable events (words).
Conclusion
Next up, get ready to use Markov chains to tag parts of speech. You now learned about the states in your model. And we said that the states represent the condition, or the present moments. These states, could be thought of as part of speech tags. Maybe one states could correspond to the verbs. Another one could correspond to the nouns and so forth.
Please Follow and 👏 Clap for the story courses teach to see latest updates on this story
🚀 Elevate Your Data Skills with Coursesteach! 🚀
Ready to dive into Python, Machine Learning, Data Science, Statistics, Linear Algebra, Computer Vision, and Research? Coursesteach has you covered!
🔍 Python, 🤖 ML, 📊 Stats, ➕ Linear Algebra, 👁️🗨️ Computer Vision, 🔬 Research — all in one place!
Don’t Miss Out on This Exclusive Opportunity to Enhance Your Skill Set! Enroll Today 🌟 at
Natural Language Processing with Probabilistic models Course
Natural Language Processing course
🔍 Explore cutting-edge tools and Python libraries, access insightful slides and source code, and tap into a wealth of free online courses from top universities and organizations. Connect with like-minded individuals on Reddit, Facebook, and beyond, and stay updated with our YouTube channel and GitHub repository. Don’t wait — enroll now and unleash your NLP potential!”
Stay tuned for our upcoming articles where we will explore specific topics related to NLP in more detail!
Remember, learning is a continuous process. So keep learning and keep creating and sharing with others!💻✌️
Note:if you are a NLP export and have some good suggestions to improve this blog to share, you write comments and contribute.
Ready to dive into data science and AI but unsure how to start? I’m here to help! Offering personalized research supervision and long-term mentoring. Let’s chat on Skype: themushtaq48 or email me at mushtaqmsit@gmail.com. Let’s kickstart your journey together!
Contribution: We would love your help in making coursesteach community even better! If you want to contribute in some courses , or if you have any suggestions for improvement in any coursesteach content, feel free to contact and follow.
Together, let’s make this the best AI learning Community! 🚀
To Do List
1- Collects Keys points from the blogs
Source
1- Natural Language Processing with Probabilistic Models (Coursera)
