Iterative Prompt Development
Any prompt to the LLM(Large Language Model) cannot be a perfect prompt in one shot. A good prompt engineer does not write a perfect prompt in the first shot. We always start with a base prompt that defines the overall idea of the task and then check the results from this base prompt. If the results are not satisfactory then we refine our prompt to get closer to the result that we need. We must be able to iteratively make our prompt better so that it works for our task. This process is called Iterative Prompt Development.
It is similar to developing any Machine Learning Algorithm. In ML the steps include:
1) Think of an Idea or changes in the idea.
2) Import the data and write the code of the ML algorithm
3) Get the experimental results
4) Do some error analysis
5) Repeat
However, in prompt engineering, the steps go as follows:
1) Think of the task or changes in the task to be achieved.
2) Write a clear and specific base prompt or Refine the prompt for the task.
3) Get the experimental results
4) Do some error analysis
5) Repeat
Let’s look at an example:
This description is lengthy. We can create a more structured format to make it more readable. So we further refine our prompt.
The tabular format here does not consist of the ‘time limit’ and ‘score’ of each section. We try to be specific here.
This table is still very confusing. If we want to incorporate it into a website we have to use HTML elements. Refining the prompt again.
As we see here the model has eliminated the Experimental, Research and Unscored Sections. But we want a complete description.
Displaying it in a table format.
Here we have seen how iteratively we refine our prompt and convert a textual description of GRE sections into a nice tabular format that is more readable.
This is useful in small applications but for more mature applications we must test our prompt on multiple examples i.e. In our case the prompt must be able to give such text-to-table conversion for any exam. Any prompt must be checked on 50 to 100 test cases but this is only when our application is mature.
Originally published at https://aanshsavla.hashnode.dev.
