Introducing PromptML, a modern markup language for Prompt Engineering [TLDR]
Hi everyone, I am very happy to introduce PromptML (open-source Prompt Markup Language from Vidura Labs). It is a new language with a simple, declarative syntax that comes along with a language parser.
Prompt Engineering is an emerging field in AI whose objective is to get the best possible outputs from Generative AI. But, as with every new field, it has its challenges. PromptML is designed to navigate these challenges by standardizing prompt creation to enable the “AI Prompt As a Code (APaC)” paradigm for individuals and teams.
You can find the project here: https://github.com/narenaryan/promptml
Challenges with Prompt Engineering
There are a few challenges in the space:
- How to standardize writing prompts?
- How to version control prompts in systems like Git?
- How to make a prompt verbose enough for AI?
- How to obtain consistent results from AI?
We asked these questions ourselves, and there were multiple attempts already to tackle them:
- `ChatML` by OpenAI
- `prompt-engine` library by Microsoft
- Awesome ChatGPT prompts community collection
- Guardrails prompting
All these solutions deviate in their ways, and previous prompt engineering libraries require a level of coding knowledge in Python (scripts and Jupyter Notebooks).
There are two problems with existing solutions:
- There is still no standard for writing an AI prompt
- Building prompts with opinionated object-oriented libraries is complex for most of the prompt engineers
There comes PromptML by Vidura. A refreshing new language to define prompts in one language and target multiple destinations based on the preferred template of choice. What it means is you write one prompt and generate multiple variations in natural language targeted for various LLMs.
It means the PromptML source file is the single source of truth for a prompt.
In the next section, we see what are the core tenets of PromptML and how they improve prompt engineering in general.
Core tenets of PromptML
The core tenets of PromptML come from inheriting all the best practices of the software industry. They are:
- Standardization
- Version Control
- High signal-to-noise ratio (verbosity)
- Collaboration
Let’s talk about each of these in detail.
Standardization
HTML and XML are very popular because they are the web standards that everybody accepts. They are simple to use, in text format, and version-controlled. Without standardization, we build silos in the industry, and the same applies to Prompt Engineering. Writing PromptML is target-agnostic with its declarative nature, and it is specially built for the AI era.
Version Control
Version controlling prompts will provide a way to processes like Refining prompts, reviewing prompts, auditing, and automation prompt execution. Infrastructure-as-Code has already proven that automation is a way to go in building reliable systems. The same is applicable for prompts. A prompt can be written once and carefully tracked changes over its lifetime.
Natural language makes version controlling tough with paraphrasing and flexibility. PromptML keeps the flexibility but embraces version control.
Signal to noise ration (verbosity)
All prompt engineers immediately agree higher signal strength inputs given to AI systems will lead to better quality outputs.
Keeping an LLM focused on structured natural language queries is one of the main goals of PromptML. The input to LLM is already broken down into helpful sections prompted in PromptML.
Collaboration
Prompt Engineers often should work with each other in designing and refining prompts. For a common understanding, two parties need to speak the same language. The same is applicable for prompts.
If we use natural language as a shared prompting language, engineers will often spend time understanding the motivation & context of a shared prompt by others. This costs a lot of human cognitive cycles that are a result of using a natural language (which is highly context-sensitive) for prompting.
PromptML instead forces prompt engineers to express their prompt systematically with clear semantics defined for different sections. We believe this makes collaboration easy.
PromptML is destined to be the language and ecosystem that allows these core tenets.
How does the PromptML language look ?
The language syntax is inspired by XML & Ruby with begin (@section) and (@end) blocks.
Let’s see an example prompt for writing a melancholic poem:
The prompt `poem.pml` defines information like:
- Context
- Objective
- Examples
- Constraints
- Metadata
in the form of annotations.
See all available prompt language annotations here: https://github.com/narenaryan/promptml/blob/main/prompt.pml
Annotations act as section blocks of a given prompt. These hand-picked annotations are specific to the prompt engineering domain.
This type of prompt we call “Original Prompt”.
Parsing a PromptML file
A `.pml` file can be easily parsed with a promptml parser. See the package installation details here: https://github.com/narenaryan/promptml/blob/main/BUILD.md
After installing the package, you can import the parser and parse the file `poem.pml`
from promptml.parser import PromptParserFromFile
prompt_parser = PromptParserFromFile('poem.pml')
prompt = prompt_parser.parse()In the above code, the promptml (.pml) is parsed into a Python dictionary with all the definitions as keys and lists. This dictionary can then be used as a base context for any templating languages like Jinja2 or Django templating language. For the above poem.pml parsed content will be this:
{'context': 'You are a master poem writer who is creative and imaginative. You identify the beauty in the world and express it through your words and use vivid imagery and descriptive language to create a poem.', 'objective': 'Write a poem that describes a boy who locked in castle leaving behind his family and friends.', 'examples': [{'input': 'Requiem', 'output': 'Under the wide and starry sky,\n Dig the grave and let me lie.\n Glad did I live and gladly die,\n And I laid me down with a will.\n\n This be the verse you grave for me:\n Here he lies where he longed to be;\n Home is the sailor, home from sea,\n And the hunter home from the hill.'}, {'input': 'My Heart Leaps Up', 'output': 'My heart leaps up when I behold \n A rainbow in the sky:\n So was it when my life began; \n So is it now I am a man; \n So be it when I shall grow old, \n Or let me die!\n The Child is father of the Man;\n And I could wish my days to be\n Bound each to each by natural piety.'}], 'constraints': {'length': {'min': 1, 'max': 10}, 'tone': 'Melancholic'}, 'metadata': {'domain': 'Arts', 'difficulty': 'Beginner'}}Now, we can inject the prompt variable into a Jinja2 template which looks something like this:
Note: We are committing details about Python code for brevity.
and I will see a nice natural language prompt that looks like this:
Context: You are a master poem writer who is creative and imaginative. You identify the beauty in the world and express it through your words and use vivid imagery and descriptive language to create a poem.
Examples:
input: Requiem
output: Under the wide and starry sky,
Dig the grave and let me lie.
Glad did I live and gladly die,
And I laid me down with a will.
This be the verse you grave for me:
Here he lies where he longed to be;
Home is the sailor, home from sea,
And the hunter home from the hill.
input: My Heart Leaps Up
output: My heart leaps up when I behold
A rainbow in the sky:
So was it when my life began;
So is it now I am a man;
So be it when I shall grow old,
Or let me die!
The Child is father of the Man;
And I could wish my days to be
Bound each to each by natural piety.
Audience: Arts
Task Difficulty: Beginner
Reply in tone: Melancholic
Objective: Write a poem that describes a boy who locked in castle leaving behind his family and friends.Now, this type of prompt is a “Derived Prompt” which is ready to be executed against an LLM. With the Cohere chat model, here is the output generated from the above prompt.
As you can see adding context and examples to the AI prompt helps in getting better-quality output. You can modify the original prompt and re-run the whole process and it will lead to a slightly changed derived prompt with most other sections intact.
You can see the full example here: https://github.com/narenaryan/promptml/tree/main/examples/poem-writer
How can you help the cause?
Clone the repository, have a look at the language spec, and start coding some prompts in PromptML. Project contributions are very welcome too!
Please don’t forget to give us a star ⭐️ on our repo!
