Prompt Engineering & Injection: Stealing the new Intellectual Property
Hidden Prompts of billion-dollar companies — Notion AI & Bing-ChatGPT
What is a Prompt?
A prompt is a text, question, or image that serves as input to an AI model. The model processes the Prompt and produces an output, which can be a text response, a classification, or an image generation. The Prompt defines the context, tone, and content of the generated text and can vary from a simple one-word prompt to a more complex, multi-sentence description of the desired output. The quality and specificity of the Prompt determine the quality of the output generated by the model. Thus, it is vital to crafting a prompt that is clear, concise, and specific to the task at hand to ensure the AI system can accurately provide the desire.
An effective supervisor in a company is like a well-designed engineering prompt. A good supervisor can clearly and succinctly convey their expectations to their subordinates, just as a clear and precise engineering prompt provides a roadmap for the project. On the other hand, a poor supervisor, much like a vague and indefinite prompt, can lead to confusion and subpar results from the employees they oversee. Specificity and clarity are crucial to success for supervisors and engineering prompts. A well-defined task, whether set by a supervisor or a prompt, allows the person responsible to fully understand what is expected of them and deliver the desired outcome.
Prompt Engineering
Prompt engineering is the practice of crafting a prompt that is specific and relevant to the task at hand to ensure that the AI system can accurately provide the desired response.
Prompt engineering for Large Language Models involves designing and crafting effective prompts that enable the model to produce high-quality, coherent, and relevant text outputs. Prompt engineering aims to provide clear and concise inputs that maximize the model’s ability to understand the desired output and generate text that meets the requirements. The quality of the generated text output is mainly dependent on the quality of the Prompt, making prompt engineering a critical component in developing and deploying Large Language Models based services.
The importance of prompt design is increasingly being recognized as a critical factor in the success of AI systems. As AI grows in sophistication and capability, the importance of well-designed prompts will only continue to increase.
The value of such prompts can be glimpsed at when looking at an existing prompt marketplace. Promptbase.com is a marketplace of prompts used to create beautiful images, using Image Generation Models <Large Language Models> such as Midjourney, DALL E2, GPT-3, and Stable Diffusion.
The recognition of value in Prompts has led to the creation of specialized roles dedicated to prompt design, such as Anthropic’s job opening in January 2023, for a Prompt Engineer. Anthropic is looking to pay a yearly salary of $250K-335K for this position, offering a competitive compensation package for a job and talent that only started to exist a few years.
Sidenote: Anthropic is part of the OpenAI Mafia
Anthropic has recently (a few days ago) received a significant investment of $300 million from Google, demonstrating their upcoming Chatbot, Claude, and positioning the relationship similarly to OpenAI’s ChatGPT and Microsoft (Feb 4th, 2023).
Stealing the Secret Sause
Just as the key idea within the dream world in the movie Inception, by Christopher Nolan, is the driving force behind the heist’s success, the Prompt in an AI-native product is the driving force behind its success. In the movie, the critical idea is hidden deep within the subconscious mind and must be implanted with precision and care.
Similarly, the Prompt in an AI-native product is the magic crafted and designed to guide the AI toward the desired outcome. Just as a poorly implanted vital idea in the dream world can lead to disastrous consequences, a poorly designed prompt in an AI-native product can result in subpar or irrelevant outputs. The Prompt is the seed that drives the AI’s creativity and decision-making, just as the key idea is the seed that goes the characters’ actions within the dream world. A well-designed prompt is like the perfect key that unlocks the full potential of the AI, leading to impressive and valuable outputs.
When you look at the features of Jasper, a company valued at $1.5 Billion in 2022, all these templates and features are created by adding “the perfect prompt” for that specific function. These products use the API of LLMs such as GPT-3 or other language models, and product makers add their own “prompt” and parameter selection to create a competitive edge.
Simply put, AI products are a mix of a prompt and a specific parameter on which the function was tweaked.
I used the movie Inception as the medium of an analogy because prompt engineering can be reversed so that the Prompt can be “stolen.” The “Intellectual Property” that defines the competitive edge can be extracted by asking the right questions. Competitors can easily reconstruct similar functions if these “prompts for templates” can be stolen.
Gaslighting an AI
Prompt Injection (named similarly to SQL injection) is where an attacker attempts to inject malicious inputs into an AI system to manipulate the system’s output. The outcomes can be categorized into two types: prompt takeovers and leaks.
Prompt Takeovers look like having a GPT3 product say something other than what the feature or function intended, such as “LOL” or the famous “haha pwned.” Users have gotten Microsoft Tay (a chatbot) to say racist comments in the past, which eventually caused the service to be taken down. It would create big problems if a user could a bot of a large enterprise or a bot designed to resemble a politician (as an AI bot has been utilized in South Korea in the recent presidential election).
Remoteli.io is a Twitter bot of a website that promotes remote jobs and companies that allow for remote work. The Twitter bot uses the GPT-3 model of OpenAI. The screenshot above shows how easy it was for users to instruct the bot to say controversial and problematic responses that were shared with the public.
Prompt Leaking the Notion AI
Prompt Leaks are when an AI spits out the “proprietary prompt prefix” that differentiates the products of separate AI-native companies, building on the same foundation model.
The problem is that there are no 100% leak-proof methods yet.
When the famous Notion AI Alpha was released, one of the top posts in Hackernews — a community run by Y-Combinator, was about Prompt Injecting the Notion AI. There are ways to make the leak process harder, but I myself can easily replicate the leak.
Due to possible legal issues that may arise from sharing what I ended up with, I will share what others have shared on GitHub with a big legal disclaimer for lawyers: LINK.
Code Name: Sydney — the Microsoft Bing x ChatGPT
It has been less than 2 days since the CEO of Microsoft, Nedella presented the world with its new Bing, incorporated with ChatGPT. Funny enough, the prompt was immediately “leaked.”
Clearly, there are efforts of hiding the precious prompt behind the chatbot within Bing, but the efforts were easily beaten. The codename of Bing Chat is ‘Sydney’ as well as the suspected prompts that were entered to create the new baby of Microsoft.
I love Microsoft and do not wish for any trouble. ❤
Conclusion
At the end of the day, a critical defining feature of an AI-Native Product happens in the inference, using the prompt and the parameters set within the Large Language Model produced by a conglomerate. The Prompt’s value is one of two critical defining components of an AI-Native product. As seen above, it is not complicated enough for Prompt itself to become the barrier to creating a lasting competitive edge in a product.
Although prompt Injection is less dangerous and detrimental than it sounds, solving it is a task that must be dealt with for the size of the AI-native market to grow even faster.
