What is Prompt Engineering?
Hey everyone. Today I’m going to talk about a little bit different topic. What is prompt engineering?
But before I dig deeper, I would like to explain some fundamental terms.
What is Generative AI?
As you know, a standard AI aims to make some tasks autonomous, analyze data and solve some complex problems, but when it comes to mimic human creativity, it falls short. This is exactly where we encountered Generative AI. Generative AI is a kind of AI sub-branch, and it is a solution to this lack of creativity.
Generative AI refers to AI systems capable of generating text, images, music, and various types of data. These systems can create new contents by using learning model based on a pre-defined pattern or sample.
ChatGPT
As you all know, a chatbot application such as ChatGPT has recently entered our lives. It’s a really different tool that we’ve used so far. When we ask a question, it can answer like a human. It can create completely unique contents such as cover letter, code samples etc.
Actually, ChatGPT is a kind of generative AI model. That’s why it has these abilities. Such models are artificial intelligence systems that can generate text-based data and often have natural language processing (NLP) capabilities.
NLP (Natural Language Processing)
NLP is a sub-branch of artificial intelligence. In this approach, natural human language is tried to be understood and processed. This technology is used in many areas such as analyzing texts, making inferences from texts, and asking text-based questions.
LLM (Large Language Model)
LLM is a term used to represent large-scale and complex NLP models. LLMs represent current technological advances in this field, while NLP focuses on processing and understanding natural language. These models are artificial intelligence systems that understand natural language by training and processing large amounts of data. They can be very successful in text-based tasks.
Prompt
A prompt is any input we can give to such models. An initial input that specifies how the model will run or respond. Especially in generative AI, a model tries to produce content according to a given prompt. For example, “Can you write me a cover letter to apply to X university and X department?”, a prompt like this will trigger the model to write an cover letter about the university and department you are providing.
When you write an accurate and engaging prompt, you help the model generate the answers you want. That’s why it’s really important to write prompts in AI-based applications. In this context, we came across a new profession called Prompt Engineering.
What is prompt engineering?
Prompt engineering is the process of preparing and designing prompts for an NLP model to elicit desired responses or behaviours from the model. It involves formulating prompts that will help the model generate more accurate, meaningful and specific outputs. This arrangement is done by tuning wording and content of a prompt.
How can we be more carefull when doing prompt engineering, or at least using ChatGPT? Let’s examine it in a few steps.
Clarity and Specificity: Preparing prompts with clear and specific information can help ensure that the model correctly understands the client’s wishes. In this way, the model can generate more relevant answers.
This is a really simple example, I know, but it makes sense in our case here. The tool we’re trying to ask is not a human, just an AI model. Don’t forget it :)
This model tries to convert our prompts into some numerical values in the background and then respond by making a predictions based on these values. So writing something randomly and not giving details prevents us from getting better answers. It’s actually about tokenization. We will talk about this term later in the article.
Contextual Information: Adding contextual information or related data to a prompt is really important. Thus, the model can generate more relational contents.
Here I asked for a line graph where I specified the coordinates, then gave a small dataset to support my prompt.
Multi-Turn Conversations: Doing multi-turn conversations means that the conversation between the human and the AI model involves many turns. In these types of interactions, the client typically starts the conversation with an initial message or a question, then the AI model responds and it goes on like this. Thus, the model can provide interrelated answers based on previous questions or answers.
For example, you asked something and received an answer. From now on, you can ask “Can you add my new value here?” or “Can you change last sentence you gave me because I don’t understand it?”. In this way, you can ensure that the model outputs more accurate.
In this example, I gave a part of code and asked the model to convert this code for StringBuilder format. After that, I asked the model to compare these two code samples. Here, I asked a new question in the same context. According to my previous prompt, the model knew what to compare, and the model made a performance comparison between the code I first wrote and the code with StringBuilder.
Chain of Thought (CoT): In such an approach, we try to guide the model by giving some instructions and telling the model how to think so that it can create a logical response chain. The main purpose here is to shape the model’s outputs according to the instructions we will give. That’s why, each instruction should be design according to the previous instruction so that the model can generate logical outputs. Thus, the model can create a flow of thought and generate coherent responses for a task.
Here I asked the model a question and asked it to think about it step by step. So I gave the model an instruction so that it could think step by step.
And here I opened a new chat and asked the model to use the steps I took in the previous chat. So in a new prompt, I used the information I got from the model and told the model to use these steps.
Prompt Chunking: It is an approach to separate complicated prompts into more manageable chunks. We can make the model provide more consistent and context-sensitive responses. This approach helps improve the outputs and enables the model understand the question comprehensively. In fact, you can give the entire text to the model at the same time but token size is the critical here.
Token & Tokenization
Let’s explain what a token is.
A token is the smallest part of a text that can be meaningful. The token can be a word, number, punctuation mark or several characters. Tokenization is the process of splitting a text into tokens.
When dividing the text into tokens, some distinctions are made according to factors such as spacing, punctuation marks, and grammar rules. Each token is assigned a weight during the tokenization process. This weight value is determined based on how often that token appears in the text. In such models, the aim is to predict the next tokens according to your prompt. While making this estimation, a calculation is made based on the weight values given to these tokens.
The more tokens in the prompt you give are seen in the model, the greater the weight value is given to it. Therefore, the more likely the model will use this token or related tokens when responding.
Closely related words and other contextually relevant tokens can also be considered by the model and used in responses. For example, if the word “beautiful” has a high weight and appears frequently in the model’s training data, the model is more likely to use the word “beautiful” when responding to “the weather is very nice today.” However, the model may also use other words or tokens, taking into account the context. For example, it can also generate similar meaningful expressions such as “the weather is wonderful” or “the weather is perfect.”
In summary, tokens with higher weights will have more influence in determining the response of the model. If we consider again to the prompt chunking as I mentioned above, the longer our prompt is, the more tokens there will be and the weight value of each token will be smaller when determining the response of the model. Therefore, the model will give us a less accurate or less logical prediction. It means that long prompts may be less effective in responding.
In this article, I tried to touch upon the subject of Prompt Engineering. I hope it will be a useful content. Enjoy reading :)
Additionally, while researching for this article, dear Daron Yondem’s Azure OpenAI from Scratch training video was very useful for me. I highly recommend anyone interested in the subject to watch it.
References
