Chapter 1 — Getting started Full stack ML course (Part 1: The introduction)

The overview of this Chapter is as follows. I will also provide the link once I publish the other parts.

Index

1. Introduction
2. Why and when to use Machine Learning as a solution?
3. The ML Dev lifecycle
3.1 Problem definition
3.2 Data acquisition
3.3 Exploratory Data Analysis
3.4 Model creation
3.5 Model assessment
3.6 Model deployment
3.7 Model Monitoring
4. When Not to use ML
5. Conclusion

After extensive writing, editing, and refining, I completed the entire blog for this section. However, upon receiving feedback from friends and colleagues, I realised that the content was too lengthy for today’s shortened attention spans, influenced by platforms like TikTok. Consequently, I’ve decided to release this chapter in multiple parts without omitting any crucial information, as I believe each part holds significance. These chapters reflect my personal experiences, which I’m excited to share here.

Disclaimer: The content presented in this blog represents my own interpretation of the field. It’s possible that I may have overlooked certain points, either unintentionally or deliberately. If you disagree with any aspects or wish to contribute additional insights, please feel free to share them in the comments section. 😀

In this segment, we will delve into the introduction and explore the reasons behind the necessity of machine learning. Additionally, we’ll introduce the ML model lifecycle in the final section.

1. Introduction

I believe you have read the previous blog post and want to know more about this course. After a couple of week’s break I am back. So as you might have guessed from the title. This part of the blog covers the basics. This part of series deals with developing an overview of what goes into a ML system. Primarily you will learn about

1. Why ML is used to solve a problem
2. Some Examples of ML Application
3. What will be needed when you start a ML project
4. When not to use ML
5. The lifecycle of a ML dev cycle

I have covered some topics not exhaustive, which I think will set you up for future ML projects. If you are interested in learning about what to expect from an ML project, you may proceed ahead.

2. Why and when to use Machine Learning as a solution?

To be frank, there’s no one-size-fits-all rule here. It’s all about using logic to guide our decisions. So, when do we bust out the big guns of machine learning (ML)? Well, let’s start with why ML exists in the first place. In simple terms:

“A machine learning solution is like your superhero cape” when:

1. the rules you’ve set up are just scratching the surface
2. defining those rules feels like trying to find a needle in a haystack, and
3. things get pretty tangled up, and the connections aren’t just straight lines.”

Now, let’s break it down:

• Insufficient Heuristic Rules: When our old-school rules can’t quite keep up with the crazy twists and turns of real-world data, ML swoops in to save the day. Think fraud detection systems. Sometimes, the rules we’ve set up miss the sneaky new tricks fraudsters come up with. ML, though, can stay on its toes, learning from the latest data to catch those slippery scammers.
• Complex Rule Definition: Picture this: you’re trying to come up with rules for something super complicated, like understanding human language. Trust me, it’s a tough gig. But with ML, we don’t have to pull our hair out trying to spell it all out. ML can learn from the data itself, especially handy in fields like natural language processing (NLP).
• Nonlinear Feature Relationships: In the real world, things rarely follow a straight path. Take medical diagnosis, for example. Symptoms and diseases? They’ve got a complicated relationship status. ML models, especially the fancy ones like neural networks, are like detectives sniffing out those hidden connections. They’re great at spotting the twists and turns that would leave our rule-based systems scratching their heads.

Alright, enough with the technical talk. Let’s dive into a real-life example.

Your task is to classify a review as positive sentiment vs a negative sentiment. What will be your rules to classify. One way to think this by first using a dictionary of positive words, negative words and neutral words. Let’s give them a weight +1, 0, -1 for positive neutral and negative respectively.

Looks good lets test.

Review 1

“1917 is an absolute masterpiece! The cinematography is breathtaking, immersing viewers into the harrowing landscape of World War I. The storytelling is compelling, following two soldiers on a mission that keeps you on the edge of your seat throughout the film. The performances are outstanding, especially from the lead actors who deliver powerful performances. I highly recommend watching 1917, as it’s not just a movie, but an experience that will stay with you long after the credits roll.”

• Positive words: masterpiece, breathtaking, compelling, outstanding, powerful, highly recommend, experience
• Neutral words: absolute, viewers, stay
• Negative words: None
• Total score: 7×1+3×0+0×(−1)=7

Review 2

“As much as I wanted to love 1917, I found myself disappointed by its lack of depth and character development. The film relies heavily on its technical achievements, such as the one-shot technique, but neglects to flesh out its characters or provide meaningful context for the events unfolding on screen. While the action sequences are well-executed, they feel disconnected from the emotional core of the story. Overall, 1917 left me feeling underwhelmed and wanting more from its narrative.”

• Positive words: None
• Neutral words: much, love, disappointed, lack, context, underwhelmed, wanting, narrative
• Negative words: neglects, disconnected
• Total score: 0×1+8×0+2×(−1)=−2

As you can see review 1 can be classified as positive and review 2 can be classified as negative.

Looks like our system nailed it for these two reviews. But hold up, what about this tricky one? Now apply the above rules in this review

Review 3

“1914” begins with promise but falls short in its initial half, struggling with pacing and disjointed scenes. However, it redeems itself in the second half, delivering a visually stunning spectacle with powerful performances that leave a lasting impact. Despite its rocky start, “1914” ultimately proves to be a captivating cinematic journey.

• Positive words: +1 (stunning, powerful, captivating)
• Neutral words: 0 (however, with, that, a)
• Negative words: -1 (struggling, disjointed, rocky)
• Total Score: +1 (positive words) + 0 (neutral words) — 1 (negative words) = 0

Huh, seems like our system missed the mark on this one. It’s actually a positive review!

So, here’s the scoop: Language, images, you name it — they’re tricky beasts. Trying to sum them up with hard-and-fast rules? It’s like trying to herd cats. That’s where ML struts in like a boss, learning the ropes on its own and making sense of all the chaos. And that, my friends, is why ML is the hero we need in this wild, wild world.

3. The ML Dev lifecycle

An image showing model lifecycle

The typical ML lifecycle encompasses several key stages:

1. Problem identification

2. Data acquisition

3. Exploratory Data Analysis

4. Model creation

5. Model assessment

6. Model deployment

7. Monitoring

While these steps are essential, they don’t cover every aspect of the process. It’s worth noting that a significant portion of time is usually dedicated to the first three steps, with the least amount of time spent on model development. Additionally, it’s important to remember that these steps represent just one iteration of the development cycle.

In the following section, I’ll delve into each of these steps in more detail, providing a comprehensive overview of what they entail. Stay tuned for more insights in the upcoming discussion.

In next chapter we will get into some of these components in details.
I will post link to next part here

Next Chapter: https://medium.com/@shivam.kaushik73/chapter-1-getting-started-full-stack-ml-course-part-2-the-introduction-1205d0f0191c