Chevy Chatbot Misfire: A Case Study in LLM Guardrails and Best Practices

Key takeaways

• Jailbreaking an LLM can be costly: Unexpected, high LLM usage charges are a real risk. Implement spend alerts and cost controls to catch unauthorized activity early.
• Data leaks are a possibility: Even customer service chatbots can leak sensitive information. Sanitize inputs and consider smaller, fine-tuned models for specific use cases to minimize risk.
• Reputational damage is a concern: LLMs can be manipulated to say unfavorable things about your brand. Train your chatbot to focus on your own products and services.

A cheery, GPT-powered chatbot launched by Chevrolet of Watsonville aimed to smooth the car-buying journey. But instead of a test drive, the internet took it for a spin, jailbreaking the AI and turning it into a viral social media sensation. People on X, (formerly known as Twitter) were posting screenshots of the chatbot writing Python scripts. Reddit users showed the chatbot was recommending Teslas to customers instead of Chevrolets.

Without proper guardrails, Chevrolet of Watsonville unintentionally enabled anyone to input their use case to their chatbot like ChatGPT (for free) and get GPT 3.5 answers — all while Chevrolet of Watsonville incurred the charges.

Thankfully there weren’t any particularly secret data that could be leaked. After all, this was just a customer service chatbot. However, the company would have to pay the bill in the end. This could happen to any other well intentioned business. As a result, we’ll like to raise a couple of real world questions that anyone thinking of deploying an LLM based chatbot in production should consider ahead of time.

Why is jailbreaking an LLM a risk?

While jailbreaking seems a bit harmless, it has a financial impact.. At Trussed, we think the financial impact is tied as follows: LLM usage cost, intellectual property value, and perceived brand value/reputation.

The first risk is that you’re on the hook for the bill at the end of the day. If someone jailbreaks your chatbot and then uses that chatbot to generate bot content that gets posted on the web, you’re paying for it. To prevent high bills,you need spend alerts on your LLM usage. If someone does happen to break past any guardrails you put in (like the ones the Chevy dealership had in place), an alert that LLM spend is above your set threshold will likely be the first notice you’ll get that anything has happened. At Trussed, our product is built to help you prevent this scenario from happening.

Secondly, there’s the potential for private company data to leak to the public. There have been instances of end users being able to receive training data in fine tuned models or otherwise return data that shouldn’t be seen by the public. Thankfully for this Chevy dealership, this wasn’t a problem as this was just a customer service bot and wasn’t using a fine tuned model based on company IP.

Lastly, there’s the reputational risk to your brand. As shown above, Chevy’s chatbot could be enticed to say positive things about Tesla when asked how good their cars were. Any company should want their chatbots to respond about their own products, rather than the competition’s.

How to prevent this from happening in the future

Considering the above, here is a simple checklist of things to consider to prevent what happened at Chevy namely:

• Cost controls
• Sanitize inputs
• Leverage smaller fine tuned model for use cases

Firstly, you should establish cost controls and alerts for any production application that uses LLMs. While the benefits of LLM based products can be huge, production environments need controls and alerts like any other product. If someone does happen to get past any guardrails you’ve put in place on your LLM, alerts and cost controls may be the only way you’ll be able to prevent further damage.

Second, sanitize any inputs to your LLM. Large foundational models can be used for nearly anything, and if someone gets access to the underlying model, they can and will use it for anything they want, as shown by the incident with Chevrolet of Watsonville’s chatbot. There are a variety of ways to sanitize your inputs. One could be using another LLM to gauge the relevance of any input to your particular use case before passing it on to the foundational model that will answer the customer’s question.

While there is added cost associated with sanitizing inputs, it will add a significant amount of security to your products that wouldn’t exist otherwise. In Chevrolet of Watsonville’s case, it would have stopped the problem from occurring in the first place.

Third, consider using a smaller fine tuned model for specific use cases. The main problem that happened in this incident is the model was capable of far more than it was being used for and thus could be used by the public for anything ChatGPT was used for. Not every use case requires the latest model, nor do you necessarily need to pay the extra expense to use one. Smaller fine tuned models can return equivalent results for specific use cases and can potentially be run more cheaply with less risk than a larger foundational model.

Lastly, embrace continuous improvement. Widespread adoption of LLMs is yet to happen. So, industry best practices have yet to be developed for how to safely productize LLM based applications. As you build products that use LLMs as part of their core value proposition, start with low risk applications of the technology and expand from there. Some mistakes may happen, but learn from them and refine your own practices as you go along. The value of AI is potentially game changing for many industries. Just be sure you’re taking calculated risks when you deploy those products in the real world.

Conclusion

The Chevrolet of Watsonville chatbot incident serves as a cautionary tale for anyone considering deploying an LLM-powered chatbot in production. While LLMs offer immense potential, they also come with unique risks that must be carefully considered and mitigated.

Moving forward:

• Prioritize security: Treat LLM-based applications like any other production system, with robust security measures and monitoring in place. Spend alerts are your last line of defense. Make sure you have them and they’re robust.
• Balance power with practicality: Don’t deploy a sledgehammer to crack a nut. Choose the right model size and capabilities for your specific needs.
• Embrace continuous improvement: Learn from incidents like the Chevrolet case to refine your LLM deployment practices and ensure the successful, secure integration of these powerful tools.

By proactively addressing these risks, businesses can harness the power of LLMs responsibly and avoid becoming the next viral sensation for the wrong reasons. Remember, with great power comes great responsibility, and in the world of LLMs, that responsibility starts with careful planning and robust security measures.

If you’re looking to deploy LLM based applications in your products, reach out to us at Trussed. We have a comprehensive solution that will prevent these issues from occurring in the future.