Will ChatGPT Replace Online Learning Platforms?

This question has been top of mind for me lately after seeing what happened to Chegg last week.

Chegg, a public online education company, saw it’s market cap get crushed by ChatGPT, down 48% in a single day.

Chegg’s Stock Price

So what exactly happened to Chegg?

How did ChatGPT cause the company’s stock price to drop from May 1st’s close at $17.60 to $9.08 by the end of trading on May 2nd?

And what does that mean for other EdTech companies?

The short of it is, in Chegg’s Q1 earnings release, CEO Dan Rosenweig told investors that “…since March we saw a significant spike in student interest in ChatGPT. We now believe it’s having an impact on our new customer growth rate.”

Despite Chegg beating expectations on both top line revenue and bottom line earnings per share, the market panicked on the discussion of ChatGPT hurting customer acquisition, and what that might mean for the need for Chegg’s services long-term.

It’s not like Chegg is ignoring the AI threat either. They’re running straight at it, building their own AI solution, CheggMate. They’re billing it as ‘the power of GPT-4’s advanced AI systems with Chegg’s extensive content library and subject-matter experts’.

Clearly investors weren’t buying it. I’ll give my personal take in due time.

Is using ChatGPT a good way to learn?

Backing up from Chegg for a moment and looking at education more broadly, how good do we think ChatGPT is at teaching?

Before answering the question, I should put my own motivations and biases out there transparently. I am an owner and operator of an online education company, Maven Analytics. I ask the question out of concern for my business. Right now, our training solution is best of breed. But could we be displaced by ChatGPT and other AI?

Despite my concerns and biases, I’m going to try to be as objective as I can in analyzing ChatGPT’s performance as a teacher. Please, call me out and note any blind spots if you think I’m not being fair and objective.

To evaluate whether ChatGPT seems like a good teacher, I asked it to perform as a data analyst and an instructor who helps people learn the skills they need to launch a data career. I prompted it to do things like the example below:

ChatGPT Prompt #1

Throughout this article I will share ChatGPT’s response, and then evaluate that response myself (from my perspective as a 15 year data analyst and a full-time instructor and operator of a business that does what I’m asking ChatGPT to do).

Can ChatGPT build a good roadmap for someone to learn data analysis from scratch?

Here’s the prompt I gave ChatGPT:

ChatGPT Prompt #1

*Note that ChatGPT crashed mid-response (I’m on the free version, this happens sometimes). I’ll include the first partial response from the crash, and the complete response below. The comparison is interesting.

Here’s ChatGPT’s first response, the incomplete one from the :
Sure! Here’s a detailed learning roadmap to help someone with no knowledge become a data analyst and land their first job in the field:

1. Introduction to Data Analysis:
   - Understand the role and responsibilities of a data analyst.
   - Learn the basic concepts and terminology of data analysis, such as data types, variables, and basic statistics.
2. Foundational Mathematics and Statistics:
   - Review essential mathematical concepts like algebra, calculus, and linear algebra.
   - Study statistics, including probability, distributions, hypothesis testing, and regression analysis.
3. Database Fundamentals:
   - Learn about relational databases and SQL (Structured Query Language).
   - Understand how to write basic SQL queries to retrieve and manipulate data.
4. Data Manipulation and Cleaning:
   - Gain proficiency in data manipulation using tools like Python (Pandas) or R (dplyr).
   - Learn techniques to clean and preprocess data, handle missing values, and address outliers.
5. Data Visualization:
   - Understand the principles of effective data visualization.
   - Explore tools like Tableau, Power BI, or Python libraries like Matplotlib and Seaborn.
   - Practice creating various visualizations to communicate insights effectively.
6. Exploratory Data Analysis (EDA):
   - Master techniques to explore and analyze data.
   - Learn how to identify patterns, relationships, and anomalies in datasets.
   - Use statistical methods and visualizations to derive insights.
7. Introduction to Machine Learning:

Here’s ChatGPT’s second response, the one it completed:

Sure, here’s a detailed learning roadmap for someone who has no knowledge of data analysis but wants to become a data analyst:

1. Learn the basics of statistics and probability: To become a data analyst, you need to have a strong foundation in statistics and probability theory. This will help you understand the concepts and methods used in data analysis. Some important topics to learn include probability distributions, hypothesis testing, regression analysis, and data visualization.
2. Learn programming skills: Data analysts often use programming languages like Python or R to clean, analyze and visualize data. Learn the basics of programming in one of these languages, including data types, loops, conditional statements, functions, and libraries.
3. Learn SQL: SQL is a language used to manage and manipulate relational databases, and it’s a critical skill for data analysts. Learn SQL syntax, database design, and common SQL queries to extract and manipulate data.
4. Practice with real-world datasets: Start working with real-world datasets to develop your data analysis skills. You can find datasets online or in your field of interest. Practice cleaning and preprocessing data, analyzing data, and visualizing data.
5. Learn data visualization: Data visualization is an essential skill for data analysts. Learn the principles of data visualization, including chart types, design principles, and best practices. Practice creating visualizations using Python or R libraries like Matplotlib, Seaborn, or ggplot2.
6. Learn machine learning: Machine learning is a growing field in data analysis. Learn the basics of machine learning algorithms, including linear regression, logistic regression, decision trees, and clustering. Practice implementing these algorithms on real-world datasets using Python or R libraries like scikit-learn.
7. Build a portfolio: Build a portfolio of your data analysis projects. Include a variety of projects that demonstrate your skills in data cleaning, analysis, and visualization. You can showcase your portfolio on platforms like GitHub, Kaggle, or your personal website.
8. Apply for jobs: Apply for data analyst jobs that match your skills and interests. Use your portfolio to demonstrate your skills and experience to potential employers. Attend networking events and connect with other data analysts to learn about job opportunities and gain insights into the industry.
9. Continue learning: The field of data analysis is constantly evolving, and it’s essential to keep learning and developing your skills. Attend conferences, read industry publications, and take online courses to stay up-to-date with the latest tools and techniques in data analysis.

What I liked about ChatGPT’s responses:

1. The topics chosen were pretty on the money, with only a couple of exceptions. A- for topic selection (we’ll talk about sequence later).
2. LOVED response #1 starting with Introduction to data analysis. Getting an overview of the role and the space is where everyone should start. A+ here.
3. Again, LOVED that response #2 included Build a portfolio. This is the best way to put your skills on display and is a must for anyone trying to break into the space. A+ for this bullet point.
4. It was awesome that response #2 included Practice with real-world datasets. Again, this is really solid advice. Every analyst needs to get their hands dirty as early as possible to really start building skills. A+ for this one too.
5. Across both responses, the language was very clear. It was easy to follow, and anyone could pick this up and understand it. A+ for clarity of communication.

What ChatGPT could have done better:

1. It’s interesting to see how different the two responses were. I loved how response #1 started with the Introduction to data analysis, but #2 missed that first critical bullet point entirely. Clearly ChatGPT doesn’t have a strong point of view on the sequencing for this roadmap (that’s not how LLM’s work). The inconsistency is problematic. There are objectively better ways to learn. Following a tried and true roadmap seems more effective than randomly sequencing topics each time.
2. My next biggest concern is the sequencing, which was really off in some cases. In response #2, putting Learn programming skills as the second thing in the sequence is horrible advice. Talk about a way to scare people off with technical learning before they’ve learned how an Analyst is supposed to think or gotten their hands dirty with data through less intimidating tools.
3. Both responses completely missed discussing Excel, which is both the most widely used data analysis tool on the planet, and a very non-intimidating way to get your hands dirty with data. I would recommend practicing data analysis with Excel using formulas, pivot tables, and learning how to spin up charts and graphs should be near the very top of every aspiring Analyst’s learning plan. There’s no easier way to build momentum and start learning about data.
4. Both responses included machine learning, which really is not a requirement for a Data Analyst role. If the aim was to become a Data Scientist, that’s a different story. But for those looking to become Data Analyst’s, this isn’t needed, and will consume a ton of time and effort. Skip this one and consider it later if you want to pivot to Data Science.

Can ChatGPT build a daily learning plan?

Next, I asked ChatGPT the following question to try to get at a detailed learning plan (you can see I keep repeating the first paragraph so it continues to play the instructor role):

Here’s ChatGPT’s response, an attempt at a daily learning plan:

Certainly! Here’s a day-by-day learning plan for an aspiring data analyst to spend 1 hour per day over a span of 20 days:

Day 1:

• Spend some time researching the role of a data analyst and familiarize yourself with the skills and responsibilities.
• Explore various industries and sectors where data analysts are in demand.
• Read articles or watch videos about successful data analysts and their career journeys.

Day 2–3:

• Learn the basics of statistics and probability. Familiarize yourself with terms like mean, median, mode, standard deviation, and probability distributions.
• Watch online tutorials or read beginner-friendly books on statistics to solidify your understanding.

Day 4–5:

• Start learning a programming language commonly used in data analysis, such as Python or R. Begin with the basics, such as data types, variables, and basic operations.
• Complete interactive coding exercises or follow online tutorials to practice coding.

Day 6–7:

• Dive deeper into programming by learning about control flow, loops, and conditional statements.
• Explore different Python or R libraries commonly used in data analysis, such as Pandas, NumPy, or dplyr.

Day 8–9:

• Begin learning SQL. Familiarize yourself with basic SQL syntax, including querying and manipulating databases.
• Practice writing SQL queries using online platforms or software like MySQL or SQLite.

Day 10–12:

• Start working with real-world datasets. Look for publicly available datasets online or on platforms like Kaggle.
• Practice data cleaning and preprocessing techniques using Python or R. Learn how to handle missing values, outliers, and data transformations.

Day 13–14:

• Focus on data visualization. Learn about different chart types, color schemes, and best practices for effective data visualization.
• Practice creating visualizations using libraries like Matplotlib, Seaborn, or ggplot2. Experiment with different types of plots and customize them.

Day 15–17:

• Deepen your understanding of statistics by learning about hypothesis testing, correlation, and regression analysis.
• Implement statistical concepts using Python or R, and practice analyzing relationships between variables.

Day 18–19:

• Familiarize yourself with the basics of machine learning. Learn about common algorithms such as linear regression, logistic regression, decision trees, or clustering.
• Use libraries like scikit-learn in Python or caret in R to implement these algorithms on sample datasets.

Day 20:

• Review and reinforce your knowledge by working on a small data analysis project. Choose a dataset, clean it, perform exploratory analysis, create visualizations, and draw insights from the data.
• Document your project and add it to your portfolio.

Remember, this learning plan is just a starting point. Data analysis is a vast field, so continue to learn, explore new concepts, and practice regularly to enhance your skills as a data analyst.

What I liked about ChatGPT’s response:

1. Like with our first set of questions, the topic selection was solid again (not commenting on sequence yet).
2. Again, the communication was crystal clear. Very easy to understand.

What ChatGPT could do better:

1. Like the first time around, I’ve got some concerns about the sequencing. Loved the Day 1 topic list. Day 2’s topics were okay. And then after that it got off the rails and into programming way too quickly. Bad idea if you want someone to build momentum early and not get frustrated.
2. This time the prompt specified having 1 hour per day to learn. While I liked the topics, it was extremely unrealistic what ChatGPT thought a person could cover in one hour. Go through the list and ask yourself if you think an hour or two is enough to cover any of the topics above. From my perspective, there’s no chance.

Can ChatGPT give me more detail about some of the bullet points on their high level learning plan to make learning actionable?

This time, I wanted ChatGPT to expand the first bullet point from it’s Day 1 plan from the roadmap, to see if it could give me enough information to actually start learning.

ChatGPT’s Day 1 Learning Plan

So I asked it to expand specifically on that one data point from Day 1 (the idea is we could do this for each of the bullet points in the plan).

Here’s ChatGPT’s response:

Certainly! Here’s a detailed learning plan for someone brand new to learn about the role of a data analyst and familiarize themselves with the skills and responsibilities:

Day 1:

• Start by researching the role of a data analyst. Look for reliable sources such as articles, blogs, or videos that provide an overview of what data analysts do.
• Read about the key responsibilities and tasks typically associated with the role. This may include tasks like data cleaning, analysis, visualization, and reporting.

Day 2–3:

• Dive deeper into the skills required for a data analyst. Begin by understanding the foundational skills such as statistics, programming, and data manipulation.
• Research the technical skills commonly used by data analysts, such as proficiency in programming languages like Python or R, data querying using SQL, and data visualization using tools like Tableau or Power BI.

Day 4–5:

• Explore the analytical skills needed to be a successful data analyst. This may include skills like exploratory data analysis, hypothesis testing, regression analysis, and basic machine learning concepts.
• Look for online resources, tutorials, or books that provide a comprehensive understanding of these analytical skills.

Day 6–7:

• Familiarize yourself with the tools and software commonly used in data analysis. Research and learn about popular programming environments like Jupyter Notebook or RStudio, as well as relevant data analysis libraries such as Pandas, NumPy, or dplyr.

Day 8–9:

• Understand the importance of data management and data governance. Learn about data quality assurance, data privacy regulations, and best practices for data handling.
• Research different database management systems (DBMS) and become familiar with concepts like data modeling, data normalization, and database querying using SQL.

Day 10–12:

• Explore the role of data visualization in data analysis. Learn about different chart types, design principles, and effective data communication.
• Look for examples of data visualizations used in various industries and understand how they can be used to convey insights and tell a story.

Day 13–14:

• Research industry-specific applications of data analysis. Understand how data analysts contribute to sectors like finance, marketing, healthcare, or e-commerce.
• Look for case studies or articles that showcase real-world examples of data analysis projects in different industries.

Day 15–16:

• Gain an understanding of the business side of data analysis. Learn how data analysts collaborate with stakeholders, translate business problems into data questions, and provide actionable insights.
• Research the importance of effective communication and presentation skills in the context of data analysis.

Day 17–19:

• Engage with the data analyst community. Join online forums, communities, or social media groups dedicated to data analysis. Participate in discussions and seek advice from experienced data analysts.
• Attend webinars, conferences, or workshops related to data analysis to stay updated with industry trends and developments.

Day 20:

• Reflect on what you have learned about the role of a data analyst and the skills and responsibilities involved.
• Summarize your key takeaways and create a personal development plan to acquire the necessary skills for becoming a data analyst.
• Identify areas where you may need further study or practice, and set goals for your learning journey.

Remember, this learning plan serves as a starting point. Continuous learning, practice, and hands-on experience will be essential to becoming a proficient data analyst.

What I liked about ChatGPT’s response:

1. Like all other times, I liked the topics selected. Maybe even more this time than the last couple of attempts.
2. Unlike other questions, for this question ChatGPT did a pretty solid job with the sequencing, especially the first handful of topics. Following this order is a smart way to tackle learning about the role of an analyst.

What ChatGPT could do better:

1. The biggest problem for me is that ChatGPT broke out this one bullet point, which was previously a part of the learning plan for 1 hour available on Day 1, and made it a 20 day plan. Perhaps my reusing the phrase ‘detailed learning plan’ was poor prompt engineering, but it’s concerning that ChatGPT would recommend this topic as 20 minutes worth of learning, and then later expand it to 20 hours. The lack of a recommendation and lack of consistency here seems problematic. ChatGPT has no point of view here. The output will vary wildly based on what the user asks for. And how does the user know what to ask for if they are just getting started?
2. It is probably far too much detail and time spent on this one specific topic. In my experience, it is quite valuable to do some of this overview learning up front, but then shift into more hands-on learning earlier to start to build momentum.

Can ChatGPT recommend how long learning to become a data analyst should take when asked directly?

Given my concerns about the wide variance in timelines proposed over the last two questions, I wanted to be more direct with ChatGPT. I was also specific to tell it not to just say “it depends”, to try and force it to give a concrete answer.

Here’s ChatGPT’s response:

As someone brand new to data analysis, if you can consistently spend 1 hour studying data analysis per day, it is reasonable to expect that it may take you approximately 6 to 12 months to learn the skills necessary to land your first data analyst job and start a career in data.

Here’s the rationale behind this estimate:

1. Complexity of the field: Data analysis is a multifaceted field that encompasses various skills, including statistics, programming, data manipulation, visualization, and more. Acquiring proficiency in these areas takes time and practice.
2. Learning curve: As a beginner, you’ll need time to understand and grasp the foundational concepts, tools, and techniques used in data analysis. This initial learning curve may require a bit more time and effort.
3. Depth of knowledge: While consistent daily study will contribute to your progress, becoming proficient in data analysis requires depth of knowledge and hands-on experience. It involves working with real-world datasets, tackling complex problems, and gaining practical skills that go beyond theoretical understanding.
4. Individual learning pace: Learning speed can vary depending on your prior knowledge, aptitude, and the quality of your study materials. Some individuals may grasp concepts faster, while others may require additional time to fully understand and apply them.

However, it’s important to note that this estimate is a rough guideline and individual results may vary. Your progress will depend on factors such as your dedication, the quality of your study materials, the support you receive, and opportunities for practical application of your skills.

To optimize your learning process and accelerate your progress, consider the following tips:

• Be consistent: Stick to your 1-hour daily study routine to ensure continuous learning and avoid gaps in your knowledge.
• Practice hands-on: Work with real-world datasets and practice implementing data analysis techniques to gain practical experience.
• Seek guidance: Consider enrolling in online courses, attending workshops, or finding a mentor who can provide guidance and support throughout your learning journey.
• Build a portfolio: As you develop your skills, create a portfolio of data analysis projects to showcase your abilities to potential employers.
• Network and seek feedback: Engage with the data analysis community, attend meetups, participate in online forums, and seek feedback on your work to enhance your skills and gain insights from experienced professionals.

Remember, the key is to maintain consistent effort, remain motivated, and actively apply your knowledge to real-world scenarios. This will increase your chances of securing a data analyst job and starting your career in data analysis within the estimated timeframe.

What I liked about ChatGPT’s response:

1. The timeframe: 6–12 months is a reasonable expectation. Some will do it quicker, some will take longer. But this is pretty solid guidance.
2. I also liked that it backed up it’s estimate with a rationale, which was very reasonable and easy to understand.
3. Was impressed that even though I told it not to say “it depends”, it still said “it depends” and gave a range, with some caveats which were solid.
4. Absolutely LOVED the advice given at the end in the tips section. These tips are gold. If you’re trying to become a data analyst, I would note them down, explore them, and build them into your action plan.

What ChatGPT could do better:

NOTHING :)

I think ChatGPT rocked this answer. And I wouldn’t change a thing. Well played, old sport!

Does ChatGPT think it would make a good tutor and learning guide for someone learning data analysis?

To wrap this one up, I put the question that got this whole thing started in front of ChatGPT. What does it think about being used as a tutor to learning data analysis?

Here’s ChatGPT’s response:

While ChatGPT can provide some guidance and information on data analysis concepts, using it as the sole resource to learn all the skills needed to become a data analyst may not be the most effective approach. Here’s why:

1. Limitations of ChatGPT: ChatGPT is a powerful language model that can provide general information and answer questions based on pre-existing knowledge. However, it has limitations in terms of providing interactive and tailored learning experiences, practical application, and personalized feedback that are crucial for learning data analysis skills.
2. Interactive and hands-on learning: Learning data analysis requires active engagement, hands-on practice, and interactive learning experiences. This includes working with real datasets, solving problems, and receiving feedback on your work. ChatGPT may not be able to provide the same level of interactive learning as other resources or platforms designed specifically for data analysis education.
3. Structured learning resources: There are numerous online courses, tutorials, and learning platforms that are specifically designed to teach data analysis skills. These resources often provide a structured curriculum, comprehensive learning materials, practical exercises, and assessments that facilitate progressive learning.
4. Community and networking opportunities: Engaging with a community of learners and professionals in the field of data analysis is highly beneficial. Online forums, meetups, and networking events allow you to connect with others, exchange ideas, seek guidance, and gain valuable insights. ChatGPT cannot replicate the community aspect of learning and networking.

While ChatGPT can be a helpful tool to supplement your learning journey, I recommend combining it with other resources and strategies for a more comprehensive approach:

1. Online courses and tutorials: Enroll in reputable online courses or tutorials specifically tailored to data analysis. Platforms like Coursera, edX, and DataCamp offer a variety of courses taught by experts in the field.
2. Interactive learning platforms: Utilize interactive platforms that provide hands-on exercises and projects to strengthen your skills. Platforms like Kaggle, DataQuest, and Mode Analytics offer practical learning experiences.
3. Books and textbooks: Explore recommended books and textbooks on data analysis to gain a deeper understanding of fundamental concepts and methodologies.
4. Practical projects: Undertake personal projects to apply your knowledge and build a portfolio. Working with real-world datasets will enhance your skills and demonstrate your abilities to potential employers.
5. Seek mentorship or join study groups: Find mentors or join study groups, either online or offline, where you can learn from experienced professionals and receive feedback on your work.
6. Attend workshops and conferences: Participate in workshops or conferences focused on data analysis to stay updated on industry trends, gain insights, and expand your professional network.

Remember, learning data analysis is a journey that requires consistent effort, practice, and exposure to a variety of resources and experiences. By combining different learning methods and utilizing the appropriate resources, you can maximize your chances of acquiring the skills needed to land your first data analyst job.

What I liked about ChatGPT’s response:

1. The high level self awareness (no AI pun intended) from ChatGPT to admit that using it alone is not the best strategy to learn data analysis was well delivered.
2. The reasons given were clear, and they are the actual reasons that a potential learner should be considering other means of learning, while using ChatGPT as a supplement.
3. The sentence, ‘While ChatGPT can be a helpful tool to supplement your learning journey, I recommend combining it with other resources and strategies for a more comprehensive approach’ as well as the types of resources recommended after was completely on the money.

What ChatGPT could do better:

I would change very little about this final response, which felt accurate, well-explained, and offered a lot of value in terms of pointing out specific strategies that would be more effective. This was great advice.

Maybe the only thing I would change would be adding a mention of Maven Analytics in the recommended approach at the end (joking, but not joking).

In ChatGPT’s defense:

We didn’t include it in this article, but there are LOTS of pointed questions ChatGPT is amazing at answering. This one was just getting long already because of the format. We’ll include a future look at a number of ways Analysts can use ChatGPT on the job, and also some specific ways you can use AI to augment your learning experience.

Wrapping up:

After completing this exercise, I feel like I’ve got a pretty good idea for how using ChatGPT as your sole learning guide would go in any industry.

While it’s an impressive and tremendously valuable tool good for lots of tasks, I wouldn’t recommend anyone put their learning entirely in ChatGPT’s hands. Instead, use it just as ChatGPT itself suggests, to augment your learning journey, where it can certainly add lots of value.

I do think I was able to put my own biases and concerns aside and give ChatGPT a fair assessment as a learning tutor for aspiring analysts. It did some things quite well, and it whiffed on some others. It was good to hear ChatGPT come to the same conclusion in the end too.

The main feeling I have is relief, after seeing how the AI performed when my learning journey was in its hands. I don’t think AI will be replacing the role of industry-specific experts in educating the next wave of analysts any time soon.

I also feel intrigued by the possibility of potentially combining the current learning experience at Maven Analytics with some sort of on-demand AI tutor in the future. Pairing something this user friendly with a strong learning plan developed by experts, niche industry-specific content, networking channels, and rich hands-on projects seems like it could be a really valuable evolution.

Next I’m going to write up my thoughts on ChatGPT and it’s impact

What’s your take?

Am I thinking about this the right way? Or am I missing something important? Call me out!

-John