Innovate Safely with GPTs
In light of OpenAI’s latest announcements on Custom GPTs, we dive into the security and privacy considerations they entail. Our discussion explores potential risks as well as offers practical guidelines to mitigate these issues. As we acknowledge the transformative potential of GPTs, we underscore the imperative of navigating their adoption responsibly, ensuring that innovation progresses hand-in-hand with security.
Co-authored by Dan Klein and Itamar Golan
Prologue
Although it is not directly related to the focus of this article, we could not overlook the circus surrounding OpenAI since DevDay. Essentially, OpenAI had almost vanished from the public eye after Sam Altman’s dismissal by the board. However, in a surprising turn of events, he returned to the helm, backed by a much friendlier board. In the context of our discussion, this highlights the pressing need for security and privacy awareness and measures when engaging with such powerful tools shrouded in secrecy.
Introduction
In an era where AI is reshaping the technological landscape daily, OpenAI’s DevDay stood as a beacon pointing to advancements are to be made, and what role will Generative Pre-trained Transformers (GPTs) play in research and productivity purposes.
During the DevDay, OpenAI introduced GPTs, a capability that allows individuals and organizations to create customized versions of ChatGPT for specific purposes. For instance, a custom GPT could help learn the rules of a new board game, assist in teaching math to kids, or contribute to creative projects like designing stickers. Building a GPT requires no coding skills or previous knowledge. Users can create their GPT by simply initiating a conversation, providing instructions and additional information, and selecting its functionalities, which may include web searching, generating images, or analyzing data.
While this advancement represents yet another step towards more personalized and versatile AI tools, enabling a broader range of users to experience and share the benefits of tailored AI solutions, it also brings to the forefront critical considerations regarding security.
This article analyses GPTs announcement, highlighting scenarios where caution is paramount and aims to equip users with the knowledge to not only harness the potential of GPTs but also to navigate their challenges in a secure manner.
Key Security Considerations
Similar to any newly announced technology and in effort to incentivise adoption, customizable GPTs, while offering flexibility and few-clicks setup, introduce pitfalls in terms of data privacy, misuse potential and other concerns.
GPTs Accessibility
By default, OpenAI’s custom GPTs are accessible to anyone with a link (and ChatGPT Plus account), as there is currently no built-in option for user authentication. Alternatives are public GPT or a private one. This openness, while facilitating ease of use, can pose significant security risks, such as unauthorized access to sensitive information. This is especially important as there is no moderation or feedback on who is using the link and the GPT created.
Data Privacy and Confidentiality
One of the foremost concerns with GPTs is data privacy. When GPTs are tailored for specific tasks, they require, unsurprisingly, access to sensitive or personal data and how the data is structured (a.k.a. data model). This concern was validated repeatedly, often within hours of the official GPTs announcement.
Ensuring that this data is protected and used ethically becomes paramount. It’s essential to implement robust data protection measures, including encryption and strict access controls, to safeguard user privacy.
Consider this scenario: a user tricks a custom GPT into sharing datasets, information, and metadata that are meant to remain confidential.
For instance, the company Levels.fyi rapidly launched a custom GPT to its audience, initially garnering significant positive feedback.
However, it was soon discovered that their GPT could be easily compromised (AKA “jailbreaked”) by a simple prompt, practically leaking the company’s uploaded dataset. Fortunately, they claimed to have uploaded only a snippet of their data — which means less comprehensive and complete application. For a more established company with a more comprehensive dataset, this could be devastating.
Although these examples may be resolved over time, new issues are likely to emerge and you should assume that at some point, this risk is inevitable and we encourage you to step into GPTs with caution and not onboard critical information.
Third Party Connectivity and Data Sharing
Similar to plugins, OpenAI allows to define custom actions by making one or more APIs available to the GPT. It allows GPT to connect with external data and/or the real world. Particularly when interfacing with sensitive live data, third-party connectivity in GPTs raises significant security and ethical concerns.
More specifically, any access granted to GPT ‘actions’ (which is OpenAI’s terminology for Retrieval Augmented Generation) should be carefully considered and granulated to prevent excessive, undesired access for either read, write or modify actions. Where relevant, set multiple actions, each with appropriate and accurate permissions and paths to prevent excessive agency.
Additionally, linking your GPT to knowledge bases or remote data sources, combined with enabling web browsing capabilities (which is enabled by default) in your application introduces another attack vector of data leakage of your private data towards the web, into a potentially malicious website.
Mapping Attack Vectors to OWASP Top 10 For LLM Applications
The OWASP Top 10 for LLM Applications provides a comprehensive framework for understanding the most critical security risks to Large Language Models (LLMs) applications. This framework is particularly relevant in assessing the security of the latest GPTs introduced by OpenAI.
The key risks mentioned in this article correspond to the following OWASP Top 10 categories for LLM applications:
- Prompt Injection (LLM01) — more specifically, indirect prompt injection risks remote yet connected data sources not only from data leakage aspects but also from further damage such as denial of service.
- Sensitive Information Disclosure (LLM06)
- Insecure Plugin Design (LLM07) — when connecting the GPT to an external application / downstream services.
- Excessive Agency (LLM08) — by providing over functionality, excessive permissions or too much autonomy.
The below diagram outlines the attack vectors presence over the diagram of OWASP Top 10 for LLM Applications.
Mitigations
Whether you are a personal ChatGPT Plus user or working for a corporate, you should be aware of the risks explained above.
We also recommend taking the following actions before your GPT goes live:
- Carefully think which capabilities and data sources should be enabled to allow your GPT to perform. Anything that is not required should be reduced (by config, by knowledge or by context).
Remember that not all functionality behaves as expected, often leading to bigger attack surface comparing to initial analysis. An example of such excessive context abuse found in ChatGPT’s plugins. - Build a policy and instruct your organization to be cautious, especially when uploading proprietary/sensitive data and/or connecting to downstream services/APIs with GPTs. This is big.
- Enforce the organization’s usage policies for these AI tools. It’s impossible to block them completely because they are significant productivity enhancers, but you can technologically ensure they are monitored and comply with policies that prevent or sanitize the uploading of sensitive data.
Summary
OpenAI’s announcements in its DevDay of GPTs marked yet another progress of democratizing the creating and activation of LLM applications. While GPTs present remarkable opportunities for personalization and efficiency across various domains, they also entail significant responsibilities, notably in safeguarding data privacy, ensuring secure access, and mitigating risks associated with third-party access. We aligned the described attack vectors with the OWASP Top 10 for LLM Applications, providing a clearer understanding of their impact on the general architecture of LLM applications.
Thank you for reading. Stay tuned for more insights and analyses on this fast evolving landscape of AI innovation and security.
About the authors:
Itamar Golan is the CEO & Co-founder @ Prompt Security (a Generative AI security platform)
Dan Klein is the Cyber Research Lead @ Accenture Labs Israel
Both Itamar and Dan are OWASP Top 10 for LLM Applications Core Team Members.
