To Use or Not to Use, Is That Really the Question?
Whether to embrace Generative Artificial Intelligence (GAI) is still a question, particularly within the legal profession. While we all recognize GAI’s extraordinary capability to craft human-like text, legal practitioners continue to grapple with the question of whether to use this technology at all, given the concern that GAI may sometimes generate inaccurate content.
If you are still deliberating over the use of a tool that relies on a Large Language Model (LLM), or are still contemplating a policy implementation to prohibit its use by others, it is crucial that you first understand how LLMs actually work. This understanding is essential to ensuring that you do not curtail the use of applications that could significantly streamline processes, saving both your time and that of others, while maintaining a manageable level of risk. Accordingly, this blog post aims to discuss LLMs in a somewhat simplified manner, equipping you with the knowledge needed to assess the potential risks associated with relying on an LLM’s output.
How it Works… Technically (Simplified)
Machine learning generally leverages historical data, known as a training set, to learn patterns and make predictions. For LLMs, the training set consists of a vast corpus of text, and the prediction involves estimating the likelihood of the next word based on the context or preceding words. The LLM does this by transforming words into numerical representations, where these numerical values help predict the probability of the subsequent word’s occurrence. It does so not just by reference to the exact sequence of words, but by understanding their semantic connection. To do this, words are translated into a number sequence known as word vectors (numerical representations of words that capture their semantic meaning). These vectors coexist within a “space,” so to speak, connected by their inherent meaning or semantic relationships.
All of these word numbers require an extensive amount of computing power. So, to be more efficient, LLMs generalize text to make it more manageable. Unfortunately, this streamlining or compression inevitably leads to information loss. Moreover, while LLMs “understand” relationships between words, they lack the ability to “understand” the difference between factual and non-factual information or what is grounded in truth and what is not. Consequently, due to both this generalization and the LLM’s general lack of “truthiness,” it can produce output text that is contextually relevant but false. In other words, it produces content that sounds good but is complete bullshit.
Why This Matters When Considering Whether to Use GAI
When using a chatbot such as ChatGPT, it’s essential to grasp these underlying mechanics to understand what the tool is doing: generating responses based on predictions regarding what the next piece of text should be. It’s crucial to recognize that the LLMs powering the chatbot do not furnish you with “true” text; instead, they provide predicted text. Whether to use this tool, therefore, hinges on your ability to discern whether the predicted text is true or not, the effort to figure that out, and the risk associated with the text being false.
For example, if you use ChatGPT to edit written content, you can readily review the edits and determine their accuracy. If your purpose is to generate creative ideas or to summarize text where you just need to know the gist, whether the output is 100% true is less significant. The risk with these use cases is low. If instead, you want to rely on ChatGPT for content in unfamiliar domains, the risk increases dramatically. Here, the challenge lies in either accepting the output at face value, whether it is true or not, or expending significant effort trying to verify its accuracy, potentially defeating the purpose of using the tool altogether. Finally, if you use ChatGPT to create output you mostly understand at face value but need to partially verify, the feasibility and efficiency of verification become pivotal factors in determining whether the tool can or should be used.
Use When the Risk Is Low or Where Veracity is Clear
As the transformative potential of GAI continues to shape industries and professions, the creation of a well-considered GAI policy becomes imperative. By understanding the technology, evaluating use cases, and focusing on risk assessment, organizations can harness the power of GAI while safeguarding against its known shortcomings. Thus, it is not whether to use or not to use, but rather, to use when it saves you time and the risk is relatively low or the output is easily verifiable.
