Moonshot Meetup #3 — Large Language Model and ChatGPT
February 2nd marks the 3rd round of SCB 10X’s moonshot meetup with special guests Dr.Sarana Nutanong (Dean, School of IST at VISTEC and CEO & Co-founder, VISAI), Mr.Dipen Mehta(General Manager, Financial Services, Asia Pacific at Microsoft) and Dr. Satya Nitta (CEO & Co-founder of Merlyn Mind (Former Global Head of AI Solutions at IBM)) accompanied and moderated by Tanwa Arpornthip, Technical Advisor at hosted at SCB 10X’s newest innovative space called district X.
The meetup started with a warm welcoming speech by
of SCB 10X stating “Today we are here to explore the exciting world of large language models and their impact on the field of artificial intelligence. As many of you know, the breakthroughs in language models such as GPT-3 have revolutionalized the way we understand and interact with natural language. The ability to process a vast amount of text data and generate humanlike responses has opened up new possibilities for a range of applications, from language translation to conversation-based AI systems. Large language models like GPT-3 have also pushed the boundaries of what’s possible in terms of generating high-quality content, from writing articles and composing music to generating code, and even creating new scientific theories. So today we are here to discuss the potential and challenges of large language models and how they are shaping the future of AI. Let’s talk about the implication of these models for the fields of natural language processing and machine learning and explore their impact on society's ethics. So let’s dive in and discover the exciting world of large language models. Let’s discuss and exchange ideas and be part of shaping the future of AI together” later mentioning that this speech was actually scripted by Chat GPT itself and stated the wonders of modern technological advancement that was deemed impossible last year” here are a few recaps of what went on at the event.Q: Chat GPT is very prominent in recent news articles within the span of the past few months. This amazing AI technology has been around for a while but how come, people, are just starting to pay a lot more attention to Chat GPT and generative AI in the most recent time? Like, what happened there? Is there a big innovation or it’s just that people got more exposure to AI?
Dr. Satya: If you look back at the last decade in AI, right, we’ve had IBM’s Watson, we’ve had AlphaGo. It’s all in the World Go Championships. And we’ve had broadly GPT-3 that came out about a year ago that captured people’s imagination. But what happened with Chat GPT was a little bit viral. Of course, along with Chad GPT, you had Dall-E and various generative AI-based image translators that popped up as well. As I think about this, I think what really happened is we went from a time when you could talk to Alexa through single shot commands, maybe have some chain dialogue. You can only do chain dialogue with Alexa when you’re playing something like 20 questions or something, even there, you can only kind of speak in restrictive phrases. You’re not really talking to these things in very natural language. And so people went from a world where you could basically do limited numbers of things with a computer to Chat GPT, where you’re seemingly conversing in very fluid, very natural colloquial language and having multiple terms and having a long dialogue session and going back and referencing something and disagreeing with it, pulling up all kinds of information. It’s composing poems like “Neruda”. It’s creating code, and that captured people’s imagination. Some of it has to go back to people thinking, okay, I can now basically see the future where I might be able to speak to a robot and ask it to wash my dishes, right? It’s essentially allowing people to bridge the gap between Chat GPT and the world of tomorrow because of the ability of a machine to understand very human-like language and perform commands like creating images or creating essays, poetry, summarizing things, right? This really bridged the gap for people in a way that say, Alexa Siri Watson AlphaGo never did. But in reality, this didn’t come out of anywhere. It was built on top of a series of innovations that have happened over the last 25 years in AI. This was an example of brilliant engineering, putting it all together and kind of reaching the tipping point. So that’s kind of my perspective.
Mr.Dipen: I think just building on what my colleagues said, I think it’s literally that we’re talking to another human, right? So it’s a natural interaction, talking to technology and actually blurring the line between what is a technology and what is a human conversation. So I think that was ultimately what I think led to the hype behind it. Obviously, from building on the advancements from an AI perspective, I think obviously, if you’d like to break through, but I would call it more evolution rather than a complete kind of revolution, was really this idea of going from very singular task-oriented models to having one that’s kind of multitask oriented and more generalized in that sense. And so once you paired that with the ability to talk to it like a human, and (I came from Amazon, so I was laughing at your Alexa example, by the way.) I spent a lot of time in that space as well. I think that’s ultimately what’s captured the hearts and minds. It’s really just kind of almost what’s the right word? It’s dumbed it down in a way, in the sense that it’s accessible to everyone. And that’s the main piece. I would say from a more technical perspective, I would argue that maybe some of the breakthroughs are obviously just the sheer size of data that these models are being trained on and in particular GPT-3 and potentially whatever GPT Four is going to be looking like. But if you look at Chat GPT alone, I think it’s almost a remarkable outcome of what’s come out, the capabilities that have come out of this model based on so much data, I think we wouldn’t have been able to imagine that before. And being able to train that model against so much data has allowed us to really see almost the art of possible. Now, obviously, then, there are things around that that’s quite generalized and where we’re going to go when we start kind of fine-tuning that into more specialized use cases. So that’s pretty exciting. I think certainly as people are looking to adopt the technology and actually build on it, there’s going to be a lot of excitement coming forward and we’ll touch on that in a second. I also think the last is just really a performance, right? The whole performance of this thing is remarkably accurate, surprisingly, and not being trained on any particular data set.
Q: There are a lot of costs and development and operating costs that go into using it. So as a company that relies on large language models of big AI to power your business, what are the benefits and drawbacks of using this kind of huge model that a normal startup wouldn’t be able to develop by themselves?
Dr.Sarana: I look at this, all right, as a researcher and as a business person. As a researcher is very exciting. It’s extremely interesting that we have big companies doing big things, pushing the boundary. When we look at language modeling, we look at it as a scientific problem. And a lot of times in science, you want to kind of like remove one constraint and see what fantastic stuff you can do. In terms of large language models, the constraint you remove is the number of parameters or the model size. Okay? And of course, what GTP has shifted is very impressive. All right? You have a language model that’s also a database and they can do like, reinforcement learning as well. So that’s really cool. Okay. Now, as a business person, as a CEO of Visa, I think that we need to put that constraint back on. If you want this language model or you want this kind of language model to be beneficial to a country like Thailand where we cannot afford compute power at the same scale as all these countries.
To answer the question, we should not rely on these prominent players to build language models for us because whatever they build is going to fit the market. It’s not going to fit Thailand. But basically, the global AI community is pretty open, companies like Microsoft, and Google, when they have their inventions and build new models, they release open-source versions as well. As an academic person, I’m very grateful that we also have that option so that we can take those technologies. Let me ask you one quick question, right? Has anyone used Chat GPT in Thai yet? Okay. In comparison to English, is it like same speed or much slower? Or do you know why? That’s because it is the input encoding that differs. In Thai, it is done at a character level, but in English, it’s done at almost a word level. This is one of the reasons why we can’t rely on other countries to build models for us. This is a technical issue that can be fixed very quickly if you have the code, if you have the source code, we can fix that in a day and a half and to close off the answer to this question, I think we should definitely join the global game and be more deliberate in responding to our local needs and take control.
Q: As you might have known, Microsoft invested in OpenAI and Chat, GPT as such, making them considered a big player in this space. there is a concern about having the power of this foundation model centralized in a big player’s hand such as you, Mr.Diphen, is there any other way that smaller players can compete with people like you or Google or Amazon?
Mr.Diphen: I think I would argue, obviously as a vendor in the room and obviously a big investor in OpenAI, that it’s unfair to say that we’re trying to control the technology. If anything, actually the reason why Microsoft’s invested in OpenAI is that we actually want to deploy those models onto our platform to allow others to then consume them. And so I think when you think about the power being in the big companies, it may be true for some other technology vendors who have built their own models and then are protecting them. Obviously, not everything is open source yet and Open AI is for various reasons. But as we get there, the idea here is all of that training, all of the mass compute power required for training the model is then marginalized because you can reuse all of our investment. So actually our view is that it’s democratizing the power of this AI by actually kind of exposing the investment through our platform and through our APIs to allow a smaller company to then fine-tune the model to a specific need. The Thai language is a really interesting one to take offline.
But I think when you look at specific, say, business use cases or processes, particularly if you’re starting an AI-based company, why wouldn’t you take the power of this model, and fine-tune it to a specific use case? And potentially even build around it in terms of workflows for making an end-to-end solution. Then you have the power of that outcome. From a business perspective, that’s actually what you’re building. Right, so in a way, yes, it can be seen as a detriment that only the big players, particularly in the case of cloud providers, have the infrastructure to really do a lot of this training. But if we’re exposing the trained models to allow you to fine-tune them actually, what we’ve done is we’ve just taken away all the heavy lifting. That isn’t necessary except for some edge cases. It’s going to become undifferentiated heavy lifting in terms of the core language model. And then you can then fine-tune it against your specific data sets to meet a particular need.
Additionally, for those who aren’t familiar with OpenAI more broadly, it’s not just Chat GPT, there are quite a lot of other models and large language models. It’s going to be large vision models as well. And so there are actually sort of four models that we’ve kind of co-invested with OpenAI on. The first one is obviously GPT is a broad-based language model and GPT-3 in this instance, and potentially as we go forward in our partnership. The other one is Dall-E. This is a visual model that takes natural language input and then generates images off the back end of that, which is if you haven’t played with it, it’s wildly exciting and scary at the same time. And then the third one is called Codex. This one is a really powerful one, particularly for people who might want to enter into the space and aren’t hardcore developers. Codex is obviously the ability to take natural language and then develop code or produce code on the back end. And so we’ve obviously taken that technology and implemented it into some of our properties, like GitHub, which is a technology platform that we own, and something called “Copilot”.You can actually have paired programming, in a way through AI. So again, it’s about empowering the developer to focus more on that kind of core algorithm or logic that you’re trying to build and let Copilot look at. Are you writing insecure code or you’re going to run into this problem and all these sorts of pieces that it can kind of infer? And then the last is Chat GPT. So that’s obviously the newest. And we’re going through where that’s going to go. But I would add that I think when you look at it from Microsoft’s perspective, we’re not in the business of actually building kind of those AI outcomes. So, yes, we will use this technology where we see it applicable as I talked about here with GitHub and Copilot. You’ll see it in Microsoft office. The ability to generate text, that same word, or emails, and Outlook, of course, are areas where we’re going to invest. And we’re going to use the same technology, by the way, that’s being deployed through our open AI stack on Azure. But ultimately, we’re not trying to build these outcomes. We’re actually trying to enable others to do more with what we have. So it’s not about competing with us. In fact, it’s actually about partnering with us.
Q: AI will enable a lot of application use cases. A question that typically comes up from a founder is, well, what am I building with an AI? A good example would be Merlyn mind which is specifically building something AI-powered education. I would love for you to tell us a little bit more about what was your process in narrowing down the market, to education AI. Of all the problems to tackle, why do you pick to attack this one?
Dr. Satya: I think we see the AI landscape the same way, which is what I don’t think the big technology providers are super interested in doing, and please correct me if I’m wrong, depending is going and building all the domain-specific instantiations of AI. In some cases, their applications. In some cases, the broader, the deeper their applications. In fact, the way we see it, as well as these large language models, are broadly built for general-purpose capabilities, but you have to fine-tune them for domain-specific applications. And the world is very large. Even though Microsoft, Google, et cetera are huge companies, the way AI will eventually proliferate and fill the whole world is through a number of companies like us that build on top of the platforms that the big tech providers will create. And an additional interesting part about AI before I jump into how we picked education and the particular use cases. So an additional interesting part about this whole field is virtually everything’s published, and everything is open source. It’s really about computing power, data, and training. All the algorithms are pretty well known, and it’s about clever architectures. But even the size of the models is shrinking -Things like transfer learning are coming in. People are beginning to use various compression techniques to collapse the size of the models and reduce the footprint, et cetera, which starts to make even the core technology accessible to smaller players and eventually to academics. Leaving aside all that, so coming back to kind of an application domain like education, why education? Why large language models here? So we’ve always so our particular company, again, we come out of IBM Research. I spent 18 years there. And when we founded Merlyn Mind about five years ago, we basically were broadly interested in taking AI and applying it to domains where we could actually impact humanity. We weren’t especially interested in going after domains like Defense, but we were super interested in domains like education and healthcare and places where AI could have a very large beneficial effect and where broadly when I look at fields like education, teaching, and learning, et cetera, these are places where failures are necessarily going to have very bad outcomes. By failures, I mean an AI algorithm. These are all stochastic machines. If it goes wrong three, four, or 5% of the time, it’s not necessarily dramatically bad for you. What we are focusing on is specifically the way we look at
AI really can’t do things that people do really well. — If you look at a teacher broadly or a professor, their job primarily is not just to impart wisdom and knowledge to you, but also to fill you with motivation. Do all the human things that people do so well, that the best teachers are role models. We look at them, and we remember them for a very long time. Even as late into my career as now, I still remember the best teachers I’ve had. These are things humans do really well. If a student is struggling, a human teacher will say they’ll use metaphors, analogies, a shared background, understanding of the world to communicate to them. All of these are well beyond the ability of a computer to do, to understand human affection, and so on. Whether you’re ready to learn whether a particular analogy is landing on you, or maybe I’ll switch seamlessly into something else, these are very difficult for a computer to do.
Whereas a human does it effortlessly. However, there are some things that people do like grading papers, and creating assessments that are very laborious, but these are things that computers can do pretty well. So when we pick an application domain to go after, what we’re trying to figure out is what is the computer really good at, right? That a human might be reasonable at, but the computer can do this really well at scale and save a human a lot of right hassle and leave the human to do the human things that they can only uniquely do so they can focus on them and elevate their level of productivity, their level of consciousness at some level if you will. We are picking problems that are well within the wheelhouse of computing where computers are basically being positioned to reduce the mundane workloads from humans and the human is then left to basically do the human things really well. An example is we are basically building using things like GPT-3 and T-Five and building our own transformers to do things like question generation. So professors, and teachers, in the course of their teaching, rely on assessments quite a bit.
For instance, I’m teaching, I want to insert a question halfway through the class or give a small pop quiz to the class. This quiz is very topical, it’s derived from my own source material. It’s not something that I just grabbed from some random textbook. It comes from the textbook I’m teaching from or the PowerPoint slides that I showed the class over the past few semesters or the past few weeks. Rather than having a professor or a teacher spend a lot of time creating questions, could we basically get a computer to do it? These are the kinds of things that we’re super interested in. Similarly, when students respond to these kinds of questions, if it’s a short answer, could a computer score? This is where some of the fascinating advances with large language models and their ability to, because the sentence embeddings, their ability to understand, pass and summarize and create comes in super handy for us. It’s a great example of an application domain where we have to train it on topical data, we have to tune it to the specific instance, we have to localize it to the context of teaching and learning. It’s the kind of thing that companies like us are super interested in doing. Companies like Google and Microsoft are less interested. They’re more interested in being the broad application providers. And that’s kind of how AI will proliferate in the world and that’s kind of how we see the lay of the land.
Q: Earlier, when talking about localizing AI models for specific use cases or specific regions, if someone wants to attack that problem, smaller players, as you said, have an edge there. What are the processes there? What is it that we have to consider? What are the limitations?
Dr. Satana: First of all, when I heard about these advanced NLP applications that are achievable in English, Chinese, or Japanese, all right, I listened with great envy because we don’t have the same kind of luxury in Thai yet at this moment. There is quite a lot for us to catch up on. This is quite problematic if there is someone who wants to follow the passion and build advanced NLP applications.
Let me give you an example. In English, let’s say that you want to build a meeting summary model that reads the meeting minutes and gets the key point, and summarize in one or two sentences, these sorts of things. If you want to do that in English, you find a text-to-text model and join it. You can do it in a short period of time. It could be like a weekend project if you already have the data. But in Thai, all right, we don’t have that luxury yet. Of course, if you really want to build it, you can still do it. But the cost of developing a supervised machine-learning model from scratch is really high. This is something that we need to catch up a lot with.
On a more positive note, I have to say that there are also new up-and-coming open-source initiatives in Thai. Have you heard of “Pi Thai NLP”? This is one of the projects that we are trying to build like a toolkit, that’s something similar to NLTK in other languages. So this is probably one of the things that we want to look at in order to catch up and have a bit of luck. My research group, okay, we also built a language model called Wang Janberta, which is something that we also released on Path in LP as well. So, yes, at this point we are at this level, all right. We’re going to move forward a lot quicker if you have more effort put in at the foundation model level.
Q: We have talked about the immersion of the AI model and how we apply the application. So I want to take the conversation from the past and present to the future. Let’s start with the near future, Dr.Diphen. What is your vision of what is going to happen in the next few years, the short future? What is the current barrier to innovation and what kind of breakthrough do we need so that we can unlock further growth in the AI model?
Dr.Diphen: First off, I think let’s talk about the breakthroughs. I think in many ways this is a bit subjective, but I think the breakthroughs already happened. I think we talk about what we’re doing now with these LLMs as a kind of hype cycle, but it’s actually just the beginning. We really don’t know where this is going to go. So I think this is actually the breakthrough that happens and we’re just at the beginning of it. So over the next two to three years, we’re just going to see how this evolves even more. So there’s a race for a number of parameters, there’s a race for the data sets, the size of the data sets and then there’ll be a race for fine-tuning them to specific domains and specific outcomes. So I think the breakthrough has already happened. I would say in terms of where the opportunities are by definition as an AGI is a generalized AI. It doesn’t actually solve a particular outcome. I think what we’re going to see a lot of, I would say kind of maybe in a hypercycle sort of way really soon, is a bunch of people trying to find the kind of the “hammer looking for a nail” as people say, right, so what can they use this for? I think what we’re going to see is what you do with the outcomes of these models.
So if you give an example, you can point to Chat GPT to summarize 100 blog posts and 30 are positives. You can do sentiment analysis. 30 or positive, 30 or negative, maybe 30 or neutral, or somewhere around there. What do you do with that information? Today it’s a textual outcome. You can then get it programmatically in JSON or some other kind of machine-readable format, but what are you going to do with it? And so I think that’s where we’re going to see quite a lot of rapid innovation off the back of this is really going to be how are people trying to consume this? And then they don’t tie it into an end-to-end process. And similar to what we talked about earlier, the opportunity is for smaller players, if you’d like, if you want to use that terminology, to start thinking about where that is. And then can they partner with somebody who’s already doing an end ten process or do they want to start a business that’s around the end ten processes?
I think those are kind of where we are in terms of the kind of breakthrough and then what we’re going to see in the short term.
Q: Let’s say you are a founder that wants to use AI in your business. But you found out that the current AI model is actually not powerful enough. Not the open AI, not like any other AI. The largest model doesn’t even serve your purpose as an application developer. So the top layer that’s user-facing, what can you do? Are you just stuck waiting for a better AI model or is there something that you can do from the application side to make your life easier?
Dr. Satya: It’s basically right. It just comes down to the type of company you are in. I’ll use a couple of examples, not necessarily in education, but I could use education examples as well. So one example is I mentioned question generation recently. So in fact, part of what we are building is an AI assistant similar to Alexa, but domain specific for the education industry. But broadly, question generation is an example of an LLM output that we are super interested in. So what we have found, just to give you to answer the question very concretely, there are other people trying to do question generation. For the most part, these are small application development shops that hit chat GPT or GPT-3 in most cases and basically give it a prompt and say, okay, so if I’m an application developer who rocks up and says I want to build a company based on question generation, I’m pretty much stuck, It’s good 50% of the time, It doesn’t give me great distractors; Distractors are let’s say I generate a multiple choice question. There’s the right answer. There are three other answers that are not right but that are plausible.
They have to be plausible. For a good multiple-choice question. So we looked at that problem, it was a fascinating problem. Because we have AI smarts and we have a quarter of our company is AI Ph.D. So we basically said, okay, a classic example of a domain-specific deep problem that requires us to go build with we start with T-five, we basically tune T-five. Then we say we need to actually go and build our own transformers to answer the question, generate distractors, and create an ensemble solution that actually goes from the machine 50% to about 85% to 90%. At which point it’s actually usable for professors like you. If you’re basically saying, okay, I want to generate questions, and half of the computer output is garbage, you’re not going to use this very much. But if most of it is pretty good, I can use it as it is with a few tweaks here and there, then you’ll adopt it and embrace it. It depends on the kind of company you are in. If all you’re doing is hitting an API, you’re basically limited to what the core technology can do.
If, on the other hand, I know what it can do, I know what the core technology can do, and I know what the large language models in general are capable of. I have the ability to go in and tweak and tune and create ensemble methods and write a stitch a few things together. Then you can actually extend well beyond what the core technology itself out of the box provides. That’s solution development.
But broadly, just to answer the question in a more generic way, what I’ve always been a huge believer in is really understanding the limits of technology. So after you put all these various pieces of the puzzle together, there are things AI cannot do like tutoring for example.
I don’t believe we’re at a point where we can have AI driving a car or we should have AI driving a car. It’s an example of a mission-critical application where AI getting it wrong will kill a lot of people. As a founder and as a startup, we are super interested in working within the limitations of AI and doing something super imaginative with it, as opposed to starting a company and imagining a capability that doesn’t exist, because that’s not a path to success.
Dr. Sarana: I would build a company that’s very specialized in transfer learning, and that’s pretty much what we did last year. To define transfer learning is essentially taking an existing model and adapting it to your data set. So you can perform the same task but in a different domain. If you can do transfer learning efficiently, then you can save a lot of data annotation effort. That’s deep learning in general. One great thing about these large language models or foundation model is that not only the domain that you can adapt the existing model to you can adapt it to different tasks as well. If I were to start a company, we would really utilize these large language models or foundation models as a base for a lot of NLP tasks in a lot of domains so that our customers don’t have to do a lot of data annotation themselves and can have an AI model of their own.
Q: In the far future, what kind of opportunities or trends should we be looking for? Should we be looking forward to having and I’m not just talking about from a technology perspective, what are the trends that you think are important to keep an eye out for? AI Adoption AI Acceptance AI Ethics commercialization of AI Anything that you think we should really look for as an AI.
Dr. Satya: A couple of things that have been super surprising to us in the field have been the success of deep learning. We’ve been in the field a long time, and when deep learning first came out, people said, okay, these are really interesting. Then they said they’ll plateau at some point, and it doesn’t seem to be plateauing, in fact, if anything, the ability of machines to do remarkable things like what GPT-3 and Chat GP have just shown as examples of Dall-Ei has been stunning. We are very interested in a class of AI, in a class of transformers called action transformers, which we think will have amazing consequences for the world. For instance, what are action transformers? Broadly taking natural language and performing some action either in the virtual world or in the physical world. Right. In the virtual world, I’m basically controlling my browser, say, opening new tabs or playing videos or so on, we think there’s some pretty significant ramifications for this.
For instance, if I just say, let’s focus on computing and the user interface. The evolution of the user interface doesn’t happen very often. The last one that’s significant is, touch computing came along and it’s been broadly very transformative. Voice has been long before touch. Alexa gave the whole field a big Filip, but the voice has not broadly been as transformative as touch computing. We suspect that natural language interfaces — I’m going to be talking to my computer and doing things like booking flights or shopping for things in totally different ways because the ability for AI or Action Transformers to perform actions for you will change how I interface with computing. As an example, I’d like to say, book me a flight to Bangkok. I want to leave tomorrow night. I want to come back a week from now. I want to fly in an aisle seat. So I’ll just give some natural language commands, and the thing will just come back and give me a few choices, as opposed to me pulling down menus, clicking things, searching, refreshing a session, and starting all over again. There’s a ton of friction associated with using computing now with a mouse and keyboard and so on.
Now, Action Transformers are also interesting because they’re not restricted to controlling just a virtual machine like a computer. I could also control robots with Action Transformers. There are some pretty interesting implications. It’s also very hard to predict where things go, because really, this whole field is evolving super rapidly, one. Two, several fields are evolving at the same time, and so the nonlinear effects come in, and so you’d be really foolish to to hazard a guess, like something will happen in five years or ten years.
As an example, back in 1957 at the Dartmouth Conference of AI, where in fact, the term AI was coined by John McCarthy, if you were to ask those early founders, they used to say, McCarthy, Minsky, Simon, and Newell, they all used to say, you’d ask them, why would you build an intelligent computer? They’d say, Well, I want a computer to tell jokes and to teach people. And they’d made this bold prediction in 1957 that within ten years, a computer will be the world chess champion. They were actually wrong. It took 40 years to get there, but they were right directionally, and that computing was evolving to do certain things so they couldn’t get it right. There’s zero chance that any prediction that, say, I’ll make will be right. I want to just paint it with a broad brush and say, it’s a fascinating field. Things are evolving very rapidly. Nonlinear effects, multiple things developing at the same time as the algorithms, you still have brand new chip architectures that are coming out, as well as Analog and AI are coming out, which is reducing the footprint for how much power you consume, and how quickly you can train. These application domains are evolving. So if you put it all together, we’re in for a fascinating decade.
Dr. Sarana: From my perspective, first of all, we want to solve the foundation model problem probably within three years; Defined the foundation module problem because we do have a foundation module now, as well as a more complete pipeline.
All these Roberta Albert-based sorts of models, sequence to sequence models or GPT. Okay. That’s also one of them we’re looking at. So in the first year, starting from this year, we will build these foundations first and then we’ll also support the transfer learning pipeline so that it can be adapted to different domains, and to different tasks. Assume that we have solved these problems within five years or within three years.
Another threat that we are working on is machine learning operations. We are looking at the machine learning problem as a solution to a system problem. We look at how we continue to develop a machine-learning model, test it, and deploy it. Monitoring, looking at the data distribution, update the model when we need to. This sort of thing is kind of like it’s kind of underrated. The non-sexy part of AI essentially..but it’s extremely impactful. This is going to be something that’s going to help us utilize the foundation models a lot easier. When you can test, we can develop your transfer model faster. So I would say that within a time frame of five years, I want Thailand to be very mature in terms of machine learning operations and after that, I don’t dare to predict.
Dr.Diphen: I was hoping to pull out my phone and ask chat GPT what the answer to that would be, but instead I’ve decided I’ll answer a lot, would be an answer around the question and hopefully you’ll get to the answer you’re hoping for. I would say instead of talking about the opportunities, I’d love to talk a little bit about where the challenges are.
I think that would be just a counterpoint to what we’ve just heard. First and foremost, I think when you think about Microsoft’s position and this is fundamentally about responsible AI. One of the reasons why there’s a commercial reason why we’ve obviously invested in OpenAI is that’s a known outcome. But the way we deploy OpenAI is obviously we wrap OpenAI in our APIs. And the reason we’re doing that is across a number of dimensions.
The first is to make it large-scale and enterprise-grade so that people can build businesses off the back of it, as you have tried public Chat GPT today, although it does run on Azure, it’s not designed for large-scale consumption, therefore it’s always busy. So that’s obviously the first piece. The second piece is all about. Then again, from an enterprise perspective is making it role-based and access control and all the security around that, which is obviously baked in. But then the most important part is our responsible AI framework that sits on top. We actually proxy every request through our own framework around responsible AI. I think when you think about the challenges going forward, or the opportunity in one sense that challenge is going to be how we’re going to govern training data, how we’re going to govern the outcomes of these models. So large, large training data sets, the bias that can occur from that just basically the unwanted effects of what has come out of these large language models. And so our position is all pretty much around how can we make this much more responsible way of consuming AI.
From a training test perspective, you would argue what is the valid outcome in trying to prove that? Because actually, that could then leads you very much into a very gray area about what is responsible. And to your point, Dr. Satya, around whether should AI drive cars and so on and so forth. So we have spent an enormous amount of investment in not just defining what our views of a responsible AI are, but also then creating the kind of governance policies and then the tooling on top.
I think the long-term opportunity is going to be other organizations and maybe other startups that are creating governance frameworks for data for generating training data or consuming others' data for training. I think that’s going to be a really interesting space. Then we were just talking about I didn’t want to miss what we were talking about before we started, which was the perception of AI. I think that’s going to be a massive challenge. There are probably twofold. With Merlin, it’s impressive to hear about what the project is itself, but we were talking about learning. And if you look at it, even in a short amount of time, the amount of impact Chat GPT has made in the education kind of community is bad in one way, but it’s strikingly interesting as well.
For example, in New York, the Board of Education has obviously tried to ban Chat GPT from schools. Although, like many educators, I’m not sure how that solves a problem because people use their phones and log in from home or use a VPN, so they’ve done that. Obviously, OpenAI just recently released an AI detector. It’s going to thwart that off. And I think that the big challenge or the opportunity, again, let’s say from an education perspective, is this isn’t going to replace education. This is going to allow us to think about how we should be teaching. I think we were talking about prompts, and it’s really all about how you generate the correct prompts. There’s a lot of money in it as a prompt developer, but should we be teaching how to ask the questions, and how to write the prompts, rather than what is the answer? that’s going to be a really interesting space in education, and it’s maybe near term now from the Hype cycle, but it’s really going to be a challenge globally because people have to really rethink the way they’re going to run education.
And then I would take it to the more commercial side in terms of if I look at things like Codex or even our own products like GitHub Copilot, then there’s the view that my job is going to be taken away if I’m a developer. Again, that’s going to change how we think about what a programmer or a software developer is doing for a living. And is it worried about kind of the framework of the code, the security of the code, and kind of how it’s managed? Or is it about what the outcome is you’re trying to drive? Again, so we have to start teaching software development in a slightly different way, but I guess it kind of relates back to education in that sense as well. But I think that’s going to be a really interesting challenge ahead. And that’s not just one, two, three years. That’s going to be ongoing for five to ten.
Key Takeaways
1. AI advancements, such as deep learning and action transformers, will reshape user interfaces and enable new methods of communication and control.
2. Responsible AI and governance of training data are critical challenges to address for the successful adoption of AI technology.
3. AI’s impact on education will require a shift in focus, teaching students how to ask the right questions and generate effective prompts.
4. The role of software developers may evolve as AI-driven tools like Codex and GitHub Copilot become more prevalent.
5. Addressing AI challenges and adapting to new methods will unlock its full potential and lead to more effective integration across various industries.
Bottomline, all three speakers have different perspectives on the outcome of the near future but what they all have in common is that none of them mentioned that they’re looking forward to more technological breakthroughs- Dr. Satya is looking for more ways of communicating with AI. Dr. Sarana is looking for more tooling for AI developers honing in specific regions as well as markets and Mr. Diphen is looking for basically responsible usage of AI, how to change the culture, how to change the way we think about AI and usage using the AI responsibly. Some challenges must be addressed, including responsible AI, governance of training data, and managing the unwanted effects of large language models. The perception of AI and its impact on education and software development will also play a significant role in shaping the future. The integration of AI in education and commercial applications will require rethinking traditional approaches and adapting to new methods, focusing on teaching how to ask the right questions and generate prompts rather than simply providing answers. These challenges and opportunities will shape the AI landscape over the next several years, with implications that could last well into the future. Take notes, AI builders or aspiring AI builders in this room take that, like the experts in the field, they’re not looking for more technological evolution.
Check out the full video at MOONSHOT MEETUP#3 Large Language Model and ChatGPT LIVE
📲Visit SCB 10X at
Twitter | Medium | Youtube | LinkedIn | Discord | Facebook | Website
