Efficiency at Your Fingertips: Simplify Email Workflow with Large Language Models

Photo by Glenn Carstens-Peters on Unsplash

Effective communication plays a pivotal role in the success and sustainability of any company. Building and maintaining strong relationships with clients and partners is a complex undertaking that requires dedicated effort. As an insurance broker, CoverWallet understands the importance of effective communication in fostering robust connections with both clients and carriers.

With the advent of advanced Large Language Models (LLMs), such as the Falcon LLM developed by the Technology Innovation Institute, insurance brokers like CoverWallet now have a powerful tool at their disposal to streamline their email communication process.

At our insurance brokerage, we regularly engage with clients and carriers to address inquiries, negotiate terms, and manage policy renewals. Constructing personalized emails for each interaction can be time-consuming and resource-intensive, potentially impeding operational efficiency.

In this article, I will explore how our insurance brokerage can leverage the Falcon LLM to generate client emails, the benefits it offers, and the considerations to keep in mind. Additionally, I have developed a simple user-friendly interface, allowing anyone to utilize an Email Generator seamlessly.

Email Generator: Let the LLM write for you

This tutorial serves as a compelling Proof of Concept (PoC), showcasing how anyone can construct an Email Generator to streamline their communication processes effectively.

The code provided below is derived from comprehensive guides available that walk through the deployment process of Falcon LLMs on Amazon Sagemaker. I encourage you to explore the guide written by Philipp Schmid as well as the HuggingFace official repository.

Deploy Falcon 7B & 40B on Amazon SageMaker

notebooks/sagemaker/27_deploy_large_language_models/sagemaker-notebook.ipynb at main ·…

In this case, I have chosen to utilize the Falcon-7b-instruct model hosted by HuggingFace. This choice is driven by several compelling factors: good performance, minimal hardware requirements, open-source availability, and suitability for commercial use. However, it’s worth noting that there are numerous other LLMs available in the market. One such example is the big brother Falcon-40b-instruct, which currently holds the top position in the Open LLM leaderboard.

To make the Email Generator easily accessible, I have created a simple and intuitive user interface (UI) using Gradio. With Gradio, users can input their prompts and receive generated email responses instantly. This UI then simplifies the interaction with the Falcon LLM, allowing users without technical expertise to leverage the benefits of automated email generation.

Programming the tool

To deploy the Falcon-7b-instruct Model on Amazon SageMaker, I begin by creating a HuggingFaceModel class. However, compared to deploying regular Hugging Face models, there is an additional step involved. I first need to obtain the container URI and provide it to the HuggingFaceModelclass using the image_uri parameter. To retrieve the new Hugging Face LLM Deep Learning Container (DLC) on Amazon SageMaker, we can leverage the get_huggingface_llm_image_uri method provided by the SageMaker SDK.

The get_huggingface_llm_image_uri method plays a crucial role in obtaining the URI for the desired Hugging Face LLM DLC based on specific backend, session, region, and version parameters. You can find the available versions here.

Once I have created the HuggingFaceModel, I can deploy it to Amazon SageMaker using the deploy method. The deployment is made with the ml.g5.12xlarge instance type, which has 4 NVIDIA A10G GPUs and 96GB of GPU memory.

import json
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri

try:
    role = sagemaker.get_execution_role()
except ValueError:
    iam = boto3.client('iam')
    role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']

hub = {
    'HF_MODEL_ID': 'tiiuae/falcon-7b-instruct',
    'SM_NUM_GPUS': json.dumps(1)
}

huggingface_model = HuggingFaceModel(
    image_uri=get_huggingface_llm_image_uri("huggingface", version="0.8.2"),
    env=hub,
    role=role,
)

predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g5.8xlarge",
    container_startup_health_check_timeout=300,
)

Now the endpoint is ready and accessible from the Endpoint configurations section inside your Amazon SageMaker console. By making use of the HuggingFacePredictor class, I can now create the Large Language Model that our tool will use to generate emails.

from sagemaker.huggingface import HuggingFacePredictor

llm = HuggingFacePredictor(endpoint_name="huggingface-pytorch-tgi-inference-[timestamp]")  # <-- update endpoint_name

Now it is time to create the UI so that our agents and teammates are able to call the LLM and generate new emails. In this case, I provide the ability to generate up to 3 different types of emails:

• Renewal: an email informing the client about the upcoming renewal of a certain policy
• Cancellation: an email informing the client about the upcoming cancellation of a certain policy
• Custom: a particular case that allows the user to input a custom message for the client

Additionally, I have implemented a dynamic button generation feature for each type of email. This automation is achieved by parsing the prompts.yaml file, which serves as a repository for storing various prompts. By introducing a new prompt to this file, the system automatically generates a corresponding button, seamlessly integrating it into the user interface. This process eliminates the need for manual button creation, ensuring that any updates or additions to the prompt file are reflected instantly in the user interface.

Lastly, I have incorporated a user-friendly slider within the UI, providing the ability to control the temperature parameter passed as input to the .predict() method of the language model. This parameter plays a crucial role in regulating the model’s “creativity” and acts as a safeguard against excessive hallucinations.

In essence, the temperature parameter determines the extent to which the model introduces imaginative elements. Higher temperature values increase the likelihood of the model generating more creative outputs, including potential hallucinations. Conversely, lower temperature values promote more conservative and focused responses from the model. By leveraging the slider, users can fine-tune the temperature parameter to strike the desired balance between creativity and maintaining the desired level of coherence in the generated emails.

The code for the UI looks like this:

import yaml
from yaml.loader import SafeLoader
import gradio as gr

with open('prompts.yaml') as f:
    prompts_file = yaml.load(f, Loader=SafeLoader)

with gr.Blocks() as demo:
    gr.Markdown("## Falcon-7b | Email Generator")
    with gr.Row():
        temperature = gr.Slider(0.1,
                                1,
                                value=0.7,
                                step=.1,
                                label="Temperature")
        email_type = gr.CheckboxGroup(
            [key.capitalize() for key in prompts_file['prompts'].keys()],
            label="Select type of email (default: Renewal)")

    with gr.Column():
        with gr.Row():
            with gr.Column():
                client_name = gr.Textbox(label="Client name",
                                         placeholder="",
                                         show_label=True)
                agent_name = gr.Textbox(label="Agent name",
                                        placeholder="",
                                        show_label=True)
                policy_premium = gr.Textbox(label="Policy premium ($)",
                                            placeholder="500",
                                            show_label=True)
                custom_message = gr.Textbox(label="Custom message",
                                            placeholder="",
                                            show_label=True)
            with gr.Column():
                with gr.Row():
                    submit = gr.Button("Submit")
                    clear = gr.Button("Clear")

            with gr.Column():
                output = gr.Textbox(value="", label="Generated email")

    def respond(client_name, agent_name, policy_premium, custom_message,
                temperature, email_type):
        if not client_name:
            client_name = "George"
        if not agent_name:
            agent_name = "Martin"
        if not policy_premium:
            policy_premium = "500"
        if len(email_type) > 0:
            email_type = email_type[0]
        else:
            email_type = 'Renewal'  # default

        prompt = prompts_file['prompts'][email_type.lower()].format(
            client_name=client_name,
            agent_name=agent_name,
            policy_premium=policy_premium,
            custom_message=custom_message)
        parameters = {
            "do_sample": True,
            "top_p": 0.7,
            "temperature": temperature,
            "top_k": 50,
            "max_new_tokens": 256,
            "repetition_penalty": 1.03,
        }

        llm_response = llm.predict({
            "inputs": prompt,
            "parameters": parameters
        })

        output = llm_response[0]['generated_text']
        processed_output = output.split(prompt)[1].lstrip()

        return processed_output

    inputs = [
        client_name, agent_name, policy_premium, custom_message, temperature,
        email_type
    ]
    outputs = [output]

    submit.click(fn=respond, inputs=inputs, outputs=outputs, queue=False)
    clear.click(lambda: None, outputs=outputs, queue=False)

demo.launch(share=True)

Testing the Email Generator

Now the UI is accessible at a certain URL, so it is time to test it out! Imagine one of our agents, Thomas, needs to notify a client, Susan, about a new coverage included in her policy, with a cost of $450. In this scenario, Thomas can make use of the “Custom” type of email, providing a concise message like this one:

Inform about a new coverage included in the policy, with a cost of $450

With this prompt, the LLM comprehends the intent and generates a fully formed email ready to be sent directly to our client. To visualize the outcome, I invite you to watch the video showcasing the generated email.

Email Generator demo

Not bad, right? Now, Thomas could effortlessly copy/paste the email generated by Falcon-7b-instruct and send it as is or make any necessary modifications as required.

Pros and Cons of Using LLMs for Email Generation

Let’s summarize now some of the main advantages of using these kinds of models for the task of generating emails.

• Time Efficiency: The automated email generation process reduces the time and effort required to compose personalized emails, allowing companies like insurance brokers to allocate their resources more effectively.
• Personalization: LLMs enable customization based on the input message, ensuring that the generated emails align with individual client preferences and requirements.
• Consistency and Accuracy: By relying on a powerful language model, brokers can maintain consistent quality across email communication, reducing the risk of errors and misinterpretation.
• Data Privacy and Security: Ensuring the privacy and security of sensitive client data is paramount. In the context of this tutorial, deploying the models on your own servers (like in this tutorial with Amazon SageMaker), offers robust protection for Personally Identifiable Information (PII). This approach stands in stark contrast to utilizing other tools like ChatGPT via API. By leveraging a self-hosted solution like Amazon SageMaker, you have greater control over data privacy and can implement stringent security measures tailored to your specific requirements. This level of control minimizes the risk of unauthorized access and safeguards sensitive client information from potential breaches. Comparatively, relying on external APIs introduces additional layers of data transfer and potential exposure to security vulnerabilities.

However, all that glitters is not gold, and deploying large language models to production faces a lot of challenges. Here I point out some of them, but I encourage you to grab a coffee and read this detailed post by Chip Huyen.

• Human Oversight: While the Falcon Large Language Model is highly advanced, there is still the potential for misinterpretation or generating responses that may not accurately address the client’s concerns. Human oversight (man-in-the-middle) is necessary to ensure the generated emails meet professional standards and accurately convey the intended message.
• Regular Model Updates: Language models fastly evolve over time with updates and improvements. It is important to stay informed about these updates and adapt the code accordingly to maintain the desired email generation quality.
• Unforeseen expenses: Maintaining Language Models (LLMs) can be quite costly in terms of hardware requirements. It is crucial to monitor the expenses associated with running LLMs in production and carefully evaluate the cost-benefit ratio when considering the deployment of such applications. LLMs demand substantial computational resources, which can contribute significantly to operational costs. It is essential to assess the long-term financial implications and weigh them against the anticipated benefits before embarking on deploying LLMs. By proactively tracking and analyzing the costs incurred by LLMs, organizations can make informed decisions about their feasibility and potential return on investment. Striking the right balance between the benefits derived from LLM-powered applications and the associated expenses is pivotal to ensure sustainable and cost-effective implementation.

Conclusion

This tutorial has demonstrated the remarkable ease and speed with which one can develop and deploy an application using cutting-edge, large language models. The presented user interface (UI) may appear simplistic, but it represents merely the surface of what can be achieved. Even seemingly straightforward applications possess the potential to create a profound impact within an organization.

The true power lies in our ability to harness these advanced language models and unlock their potential to add significant value to our businesses. By embracing the possibilities they offer, we have the opportunity to elevate our operations, enhance customer experiences, and drive innovation.