Prompt Engineering: Get the Best out of LLM Using These 5 Simple Techniques
Prompt engineering is like giving clear directions to a smart but sometimes confused friend. You’re figuring out the best way to ask questions or give instructions so they understand exactly what you want and can give you the best answer or help possible. It’s all about finding the right words to get the right results from an AI.
With simple techniques, you can provide these directions to LLM and get better output. As I spend a lot of time using GitHub Copilot, I will use that as LLM example to explain these techniques.
Techniques:
- Zero-Shot Prompts vs Few-Shot Prompts
Zero-shot prompts are also called direct prompts. They are the simplest prompts that you can use on LLM. Zero-shot prompts are single prompts that have no context or additional information for the model. The model is expected to understand the prompt and answer without any context.
Let's say I want to generate a function that uses the new switch and null functionality released in Java 21 [https://openjdk.org/jeps/441].
Example of zero-shot prompt in GitHub Copilot:
can you generate me a function in java that use switch statement to match a string.
For “foo” and “bar” string case it should print “great”.
For default case it should print “ok”.
For Null it should print “oops”
With the above prompt, GitHub Copilot does provide an answer that explains how you can create a function in Java to use the switch statement. It adds an if condition at the beginning of the function to check for null rather than handling it as part of the switch case in newly java release.
// As of Java 21
static void testFooBarNew(String s) {
switch (s) {
case null -> System.out.println("Oops");
case "Foo", "Bar" -> System.out.println("Great");
default -> System.out.println("Ok");
}
}This might not be relevant for you as you are interested in newly released functionality in Java 21, where null is handled in the switch.
Now, this can be resolved by updating the model or with fine-tuning. Both of those methods are expensive and time-consuming. Let's try to solve this with a few shots of prompting.
A few-shot prompt is a technique of providing prompts to LLM with one or more examples that guide the model in generating specific output.
Let's take the above zero-shot prompt and convert it into a few-shot prompt. I copied the text from Oracle's webpage about the switch and null handle release, which provides an example in Java 21, and made it part of my prompt.
“Traditionally, switch statements and expressions throw NullPointerException if the selector expression evaluates to null, so testing for null must be done outside of the switch:
// Prior to Java 21 static void testFooBarOld(String s) { if (s == null) { System.out.println(“Oops!”); return; } switch (s) { case “Foo”, “Bar” -> System.out.println(“Great”); default -> System.out.println(“Ok”); } } This was reasonable when switch supported only a few reference types. However, if switch allows a selector expression of any reference type, and case labels can have type patterns, then the standalone null test feels like an arbitrary distinction which invites needless boilerplate and opportunities for error. It would be better to integrate the null test into the switch by allowing a new null case label:
// As of Java 21 static void testFooBarNew(String s) { switch (s) { case null -> System.out.println(“Oops”); case “Foo”, “Bar” -> System.out.println(“Great”); default -> System.out.println(“Ok”); } } The behavior of the switch when the value of the selector expression is null is always determined by its case labels. With a case null, the switch executes the code associated with that label; without a case null, the switch throws NullPointerException, just as before. (To maintain backward compatibility with the current semantics of switch, the default label does not match a null selector.)
Now using the above reasoning , can you generate me a function that use switch statement to match “foo” and “bar” string case it should print “great”. For default case it should print “ok”.for null it should print oops. I am not interested in java 17 or earlier version code. I am writing my application in java 21”
With these few-shot prompts, we can get the desired output in Java 21 format.
So, depending on your tasks, if zero-shot prompts are not fetching the correct information, try using few-shot prompts by providing relevant examples. You will definitely see an improvement in suggestions ;).
2. Chain of thought
Chain of thought prompting technique was published in this paper https://arxiv.org/abs/2201.11903.
Chain of thought” prompting techniques involve guiding an LLM response by providing related prompts. It’s like giving the model a series of cues to follow, much like how one thought leads to another in our minds.
As developers, we often employ the chain of thought approach. When faced with a large, complex task, we break it into smaller, manageable tasks and sections. Later, these smaller tasks combine to achieve the larger goal. Chain of thought prompting techniques operate similarly. Instead of directly querying the large task from the LLM, you can prompt it to solve the smaller tasks that collectively contribute to the final solution.
Let's understand this with an example from GitHub Copilot:
In a simple spring boot java app: https://github.com/spring-projects/spring-petclinic, I would like to add a new page prompt.html that can be viewed when users reach endpoint “/prompt” on the app. Also when user reaches this page, i would like to show a picture of GitHub Copilot.
When I try to do this with a single prompt, even providing all relevant information about the task, Copilot gets a bit lost and gives me a generic answer, which I must refactor.
Instead, let's break it down. How will i resolve this without an AI.
- Create a new controller “PromptController.java”, that returns “prompt.”
2. Create a new HTML file, “prompt.html”, that shows the image “prompt.png” from static/resources folder
3. Add the image to the resources folder
4. Add unit tests for “PromptController.java” [ Sorry for all TDD lovers that I am writing unit tests later :D :D ].
5. Finish
Now, if I use the same approach working with GitHub Copilot, then I would have responses that are built on top of each other and with the exact solution that I am looking for
With the following prompt, I was able to create a controller:
// Create a controller than returns the view “/prompt when the url “/prompt” is accessed
No refactoring is required :)
Now, I create the HTML file with the below prompt:
// create a view that corresponds to html template in welcome.html. Add an image to the view that is displayed on the page. Image is stored in src/main/resources/static/resources/images/githubcopilot.png
Again, no refactoring was required. If you notice, I used a few-shot prompting technique to follow the same template that I have in the welcome.html file by mentioning it in the prompt.
Add githubcopilot.png and add unit tests for PromptControllerTests.java using the following prompt:
Create unit tests for promptcontroller.java
I was lazy and used Copilot for chat to generate the tests in one go instead of accepting function by function.
Let's Run the application:
Voila!!!! With a 100% acceptance rate and zero refactoring, I was able to implement this feature. Chain of thought really helps with step-by-step implementing smaller tasks and building on top of every answer provided by LLM.
3. Soft Prompts vs Hard Prompts
Before we look at soft and hard prompts, it's important to know the concept of prompt tuning.
Imagine you’re coaching a soccer team with a talented but sometimes unpredictable player. Let’s call them the AI player. As a coach, you want to get the best performance out of this player, so you adjust your instructions during the game.
Prompt tuning is like giving specific directions or tactics to your player during a match. For example, if you notice they’re struggling to pass accurately, you might prompt them, "Look for open teammates on your left before making a pass.” This prompt guides their actions and helps them perform better on the field.
Similarly, in prompt tuning, you adjust how you communicate with the AI to get more accurate or relevant responses.
Hard Prompts:
Hard prompts in prompt tuning are tough challenges we give to the AI. They’re specific instructions or questions that demand a precise response.
Think of it like giving a clear set of directions: “Turn left at the stop sign, go straight for two blocks, then turn right at the blue house.” That’s a hard prompt — it’s very specific about what needs to be done.
Hard prompts are also referred to as human-generated prompts. Let's take an example of a hard prompt with GitHub Copilot.
I hard-prompted GitHub Copilot with the following:
Create python code to query facebook apis to get information about a page.Please use
- python3
- use python-facebook-api library
- Use descriptive and meaningful names for variables, functions,
- Use comments to explain code
The result was precise to my prompt, and every detail was fulfilled. The prompt is transparent and contains all the information, and we can tweak it to prompt it again, depending on the result.
Soft Prompts:
Soft prompts in prompt tuning are like gentle nudges we give the AI. Instead of giving strict instructions, we provide hints or suggestions to guide its behaviour. Imagine giving directions to a friend, but instead of saying, “Turn left at the corner,” you might say, “Head in that direction, and you’ll find it eventually.” That’s a soft prompt — it gives a general idea without being too specific.
AI itself generally generates soft prompts, and that's the case for GitHub Copilot as well. They are unrecognizable to the human eye and consist of embeddings and numerous strings.
An example of a soft prompt is when you have existing code in the IDE, and without writing a prompt, GitHub copilot shows you a suggestion.
In the above example, GitHub Copilot suggests the delete book method without a prompt. Soft prompts can be a combination of multiple prompts connected in some way. This information is opaque to the user, and they cannot edit this prompt.
If you are not getting good results with soft prompts, try using hard prompts for complex problems by providing detailed information as part of the prompt.
4. RAG: Retrieval Augmented Generation
RAG is one of the most popular and efficient prompt engineering techniques. But what exactly is RAG?
Think of a soccer team preparing for a big game. First, they watch videos of their opponents’ past games to gather (retrieve) all the strategies and moves (information). Then, during their game, they combine what they learned with their skills, and develop new plays (generate) to win the match.
For LLMs, retrieval-augmented generation works similarly. The AI “watches” (retrieves information from a data source) and then “plays” (generates a response) by mixing the retrieved info with what it knows, crafting a new and informed answer, just like a soccer team devising winning strategies based on past games.
Let's look at how LLM works without RAG.
In the above picture, when we query the LLM, it generates a response without checking any additional knowledge base or data source.
With RAG approach :
When a query is sent to LLM in RAG, it first retrieves relevant information from the knowledge base and uses that information to create a more informed and accurate response.
Retrieval-Augmented Generation ( RAG ) is being used in GitHub’s latest AI offering, GitHub Copilot Enterprise. With this offering, you can index your GitHub repo and documentation as a data source along with GitHub Copilot LLM. The answers will be more accurate based on your indexed code and documentation.
GitHub Copilot Enterprise provides a framework to index repos and documentation in it. You can read more about it here.
5. ReAct
ReAct is inspired by the synergies between “acting” and “reasoning” which allow humans to learn new tasks and make decisions or reasoning.
The ReAct framework can allow LLMs to interact with external tools to retrieve additional information, leading to more reliable and factual responses.
Imagine you’re chatting with a friend and want to ensure the information you share is accurate. With the ReAct framework, it’s like having access to a library in the middle of your conversation. If you’re unsure about something, quickly look it up using external tools (like search engines or databases) and then share that reliable information with your friend. This helps ensure that your responses are factual and trustworthy.
It sounds very similar to RAG, where you provide your private data source, but in ReAct you provide external sources/tools along with private sources. The model gathers information from both these sources and then reaches a response.
In GitHub Copilot Enterprise, you have a setting on the org and enterprise level to activate Bing search. Copilot can provide results using your private data and Bing search results.
Here is an example from my chat interaction with my demo repo on github.com
I asked GitHub Copilot about the package version in my repo and requested that it check the latest available version. Copilot checked the private indexed repo and used Bing to provide the result.
Summary
Prompt engineering is crucial when working with LLMs because it allows users to guide the model’s behaviour and tailor responses to specific needs or tasks. Sometimes, if you cannot get adequate results from the model, the problem is not the model, but maybe you are :P.
Get creative with your prompts, provide detailed context, and hopefully, the above-stated techniques will help you get the most out of LLMs ;)
There are many more techniques available for you to explore. You can read more on https://www.promptingguide.ai/techniques about these techniques.
My journey into the fascinating world of prompt engineering is ongoing, and every day offers new insights and discoveries. This blog is a reflection of my current understanding and experiences. If you’re well-versed in AI, I’d love for you to view this as an opportunity for dialogue and knowledge exchange. Your expertise could provide invaluable perspectives and help enrich our collective learning journey. Let’s explore the potential of AI together!
Reach out to me on Linkedin or comment below :)
About Me
My name is Tajinder Singh, but most of my friends and colleagues call me TJ. Currently, I am working at GitHub as a Solutions Engineer based in the beautiful city of Zurich, Switzerland.
Linkedin Profile: https://www.linkedin.com/in/tajinder-singh-74740115b/
Note: Opinions are my own and not the views of my employer
