ChatGPT and the future of Machine Learning models

I got my first PC in the early 2000s. It was an IBM machine with Microsoft Windows OS on it. It was very special to me and it had a profound impact on my childhood (mostly in a good way). I remember spending time on the PC listening to songs, drawing paintings in MS paint, writing HTML code, and searching the Britannica encyclopedia among other fun things. There was no internet, so I would rely on software loaded on the computer.

One such software was a ‘chatbot’ that my cousin gave me on a ‘floppy disk’. I don’t remember the name of it but it had a black background with green font color.

I didn’t believe that this software could “chat with me” or answer my questions. I remember spending many hours with it asking stupid questions, trying to beat it, and getting into circular loops.

Fast forward to December 2022, I started hearing stories about a magical chatbot in mainstream media and it got my attention. Yes, it is Open AI’s ChatGPT.

What is ChatGPT?

ChaptGPT is a chat interface based on Generative Pre-trained Transformer 3 (GPT-3) developed by OpenAI. GPT-3 is an autoregressive language model that uses deep learning to produce human-like text. Given an initial text as a prompt, it will produce text that continues the chat with the user. In layman’s terms, it is a language model (an ML model which understands language nuances) which has been trained on a massive amount of data (~800GB) to generate text(answers) in different styles and formats.

Open AI: https://openai.com/
ChatGPT app: https://chat.openai.com/chat
GPT-3 model: https://en.wikipedia.org/wiki/GPT-3

My interaction with ChatGPT

Below are the screenshots of my chat with OpenAI’s ChatGPT. I will not comment on the correctness of the responses but it was fun interacting with ChatGPT. I strongly suggest everyone go to Open AI’s website and explore ChatGPT.

What does it mean for the future?

ChatGPT is not the first of its kind. Artificial Intelligence based chatbots are not a new thing. These have been in existence for a long time. What makes it interesting is the underlying model, data set and the approach used to train the model.

• ChatGPT is impressive in the way it interacts with the users and generates responses. We have been doing conversational searches (text-based or voice-based) for some time now, it takes the bar much higher.
• We will continue to see improvements in these language models (and other models) every year and the user experience of the products based on these models will continue to improve. We will also see a few exceptional improvements along the way but we are still many years away from true Artificial General Intelligence (it takes multiple decades to get the last 2–5% right).
• Models will eventually become a commodity and proprietary data sources will become more important than the models. Companies that are able to get exclusive access to these data sources will become the champions in this race.
• Getting access to proprietary data sources (e.g. healthcare data) will not be a simple task. It will involve regulatory hurdles, and personal data protection rules, among other challenges. Users will become more aware and will expect a service/reward for sharing the data. Big tech is positioned to win here.
• We might see walled gardens emerge out of this trend with users sharing their data, and getting the services/rewards for it. This might also result in the self-selection of users and more bias in these AI systems. Regulators have to do a good job to help the users stay informed and have the right to choose/migrate.
• Conversational AI-based chatbots have the potential to (and they are) disrupt the way humans interact with the machines (e.g. search will become more conversational). We will see network effects kick in and companies trying to get control of the distribution channels (like in the case of search).
• Specific models vs General models- In the near future, we will continue to see task-specific models emerge for solving specific needs. These models will not “talk” to each other but eventually, pipes/channels will be built to let the users interact with these models via a unified interface.
• We will see a jump in jobs related to data tagging, model training, and ML ops.

Note: This article was first published on vinaysachdeva.com on Dec 15, 2022.

Disclaimer: https://vinaysachdeva.com/disclaimer/