In the following series of articles, we’ll discuss several noteworthy practices we’ve gained in the past few months — integrating GPT (Large Language Models in a “larger” sense) to solve unique business challenges, while further automating and optimizing data science workflows.
Since ChatGPT was introduced last fall, our team has been bursting with creativity. New models arrive every week, inspiring us to come up with new ideas on how to make use of them simultaneously.
We conducted experiments of recent models using various data for different use cases, gaining valuable first-hand experience along with some surprising insights. In the meantime, we inevitably ran into unique problems as well, giving us real-time opportunities to re-think and learn.
We’re excited to share our learnings with you, keying on product integration, model selection, as well as prompting, performance/monitoring, and more. Given the evolving speed of the technology, we encourage you to prime yourself now!
Part 1: Why we are pro-GPT for the future of technology
To start our series, it is important to address why we think the technology of LLMs is highly relevant to the future. This seems to be a no-brainer given the fact that ChatGPT set records for both fastest growing user base and business adoption rate. But there are also voices against it. Some question the legitimacy of GPT answers, while others question the ability of LLMs to perform serious tasks (rather than telling jokes or writing poems).
It is key to note: There is no perfect AI, just as there are no perfect human beings. LLMs all have their pros and cons, similar to humans in the workplace. There are tasks that people are not great at, or only expensive people are good at. Some of these tasks can be easily, systematically replicated through the use of LLM agents. At the end of the day, it’s not a question of whether AI can do the job perfectly; rather whether AI can serve as an effective supplement or, in some cases alternative, to human beings — offering new solutions to businesses given performance and investment considerations.
For those of you who know how GPT works, you may wonder how a model of simply predicting the next word can fundamentally change the way we work. In actuality what we’re seeing is the process of LLMs uncovering (and then repacking) the relevant language abilities of humans — the specific capacities we apply to many of our tasks in workplaces and business settings. What this means is the possibility to leverage a machine that can do these tasks for you, which potentially opens up time and resource savings. Language is also one of the cornerstones of all other human abilities. It enables logic and reasoning to happen, expanding the work that LLMs can achieve.
LLMs also have other unique advantages, opening up solutions to a number of different business challenges.
- The ability to hold and sort through enormous amounts of text. It’s not always easy for us humans to process information from a large set of contextual data, especially when the data is across different domains or the text comes from different languages. However this task is easier for LLMs, since they read and write extremely fast, and they have a common representation trained in different languages. With LLMs, we can set up tasks like summarizations and question & answering to more efficiently tap into the value of these data assets.
- Programming languages are just different kinds of languages that machines can understand. GPT is trained with the goal of predicting of the next word. And if you think of computer programs as language, the original task is actually perfectly for coding. Having been exposed to and trained on millions of lines of code, LLMs can confidently write good codes to solve a wide range of business problems. Many surprising use cases, which at first glance seem to have no relation to language, can actually be handled by the coding capability of LLMs such as “5 Movies” by Morten Just — an iphone app made out of conversations with GPT4.
- Language abilities are leading to certain levels of reasoning (and planning!) faculty . By organizing your thoughts in a logical way, LLMs can mimic your sentences and therefore (to an extent) mimic your thinking processes — this can in turn lead to more success in generating desired results (Example how Chain-of-Thought Prompting improves the LLM’s capability in solving math problems. There are other variations of CoT Prompting which can help even further!)
As we get further into our series, we’ll find that examples of useful LLM business use cases often adhere with these key wins.
A caveat: even though the power of LLMs can extend to more complex capabilities, current performance and consistency of LLMs for the above tasks still vary considerably. (GPT4 provided the wrong solution for the game which takes multi-steps planning)
What we’ve found out is that tasks closer to pure language abilities — like question/answering, summarization, categorization, etc. — are more likely to succeed vs. more complex tasks such as reasoning and planning. For tasks that are a bit more complex, we need to put more effort into breaking up the steps in the workflow and it requires more exquisite techniques in prompt engineering (we will cover some of them in later articles) to achieve desired and reliable results.
You may already have a million ideas on how to utilize the magic of LLMs in your business. But how can you turn the hype into reality…and quickly find out if LLMs can actually help you with the use cases you’re envisioning? And what could you realistically expect in terms of the quality and consistency of the results?
We’ll share our ideas (and maybe some answers) to all of these important questions in the rest of our series. Stay tuned!
Coming up Part 2: Try It Yourself: The first steps to set up LLMs for your business
Bibliography
[1] Wei, Jason, et al. “Chain of thought prompting elicits reasoning in large language models.” arXiv preprint arXiv:2201.11903 (2022).
[2] Bubeck, Chandrasekaran, et al. “Sparks of Artificial General Intelligence: Early experiments with GPT-4” arXiv preprint arXiv:2303.12712 (2023).
