Prompt Management with Vertex AI

Prompts are everywhere, yet this Google Cloud offering is still one of the lesser-known ones.

Managing these prompts becomes a significant challenge. How do you version them? Share them across teams? Maintain consistency? Prompt management is all about organizing, versioning, and centralizing the prompts used with LLMs.

This isn’t a super technical or very buzzwordy article. Still, I believe this is one of those Vertex AI services that require more attention.

Why Prompt Management Matters

Without structured prompt management, organizations quickly face a range of challenges:

• Teams recreate similar prompts with subtle differences, introducing inconsistencies
• It’s hard to track prompt versions over time or roll back safely
• There is no simple way to share well-performing prompts across teams

Vertex AI Prompt Management tackles these challenges by providing a repository for prompts deeply integrated into the UI and SDK.

Managing Prompts in Vertex AI

The vertexai.preview.prompts module provides the foundation for prompt management.

Creating and Saving Prompts

The first step is to create a local prompt. In this example, we’ll build a prompt that helps match artist names to their correct spelling from a reference list.

Prompt management allows us to store and version the following attributes of our prompt.

• A clear name for identification
• The primary prompt text with a variable placeholder, if needed
• Variables to substitute into the prompt
• A detailed system instruction that guides the model’s behavior
• Model to use
  If you don't define the model name, the default one currently used is gemini-1.5-flash-002. I am a big fan of digging into the Vertex AI SDK. Fancy doing the same? Here you go if you want to take the red pill 🔴.
• Generation config (your model parameters)

import vertexai
from vertexai.preview import prompts
from vertexai.preview.prompts import Prompt

prompt = Prompt(
    prompt_name="artist-matcher",
    prompt_data="correct the following artist {artist}",
    variables=[
        {"artist": "acdc"},
        {"artist": "Pink Floy"},
    ],
    model_name="gemini-2.0-flash-001",
    system_instruction="""
 you are a artist matching service based on a list of artists provided.
 you suggest the correct naming. users might use slightly different names
 you need to ensure they are exactly as in the artist list:
 
 The Rolling Stones
 The Beatles
 Led Zeppelin
 Pink Floyd
 The Who
 The Doors
 Queen
 Aerosmith
 The Eagles
 Fleetwood Mac
 David Bowie
 Jimi Hendrix
 Bob Dylan
 AC/DC
 Guns N' Roses
 The Clash
 Nirvana
 U2
 Bruce Springsteen
 The Kinks
 The Beach Boys
 """,
)

At this point, the prompt only exists locally. To save it to Vertex AI’s prompt management system, we use:

prompt = prompts.create_version(prompt=prompt)

This creates a persistent, versioned prompt resource that can be shared and reused using the Vertex AI SDK or the Vertex AI UI.

Retrieving Saved Prompts

To access a prompt that was previously saved:

# Retrieve a prompt by its ID
prompt = prompts.get("your-prompt-id")

Listing Available Prompts

To discover all prompt templates available in your project:

all_prompts = prompts.list()
for p in all_prompts:
  print(f"Name: {p.display_name}, ID: {p.prompt_id}")

Updating Prompts

One of the most powerful features of prompt management is version control. This lets you iterate on prompts while maintaining a history of changes.

When you want to update a prompt, you modify its properties locally and then save a new version. Every change creates a new version, maintaining full history for auditing or rollback.

# Update a prompt property
prompt.system_instruction = "New and improved system instruction"

# Save as a new version
updated_prompt = prompts.create_version(prompt=prompt)
print(f"Created new version: {updated_prompt.version_id}")

Viewing Version History

You can list all versions of a prompt:

# List versions of a specific prompt
versions = prompts.list_versions(prompt_id="your-prompt-id")
for v in versions:
 print(f"Version: {v.version_id}, Name: {v.display_name}")

Retrieving Specific Versions

You can retrieve a specific version for comparison or use:

# Get a specific version
specific_version = prompts.get(
 prompt_id="your-prompt-id",
 version_id="version-id"
)

Using Prompts with Gemini Models

The real power of managed prompts comes when generating content. You can use a prompt directly with its associated model:

# Generate content using a prompt
response = prompt.generate_content("acdc")
print(response.candidates[0].content.parts[0].text) # Outputs: AC/DC

Restoring Previous Prompt Versions

One of the features I love the most about Vertex AI Prompt Management is version restoring.

Every time you save a prompt, it creates a new immutable version behind the scenes. If you ever need to return to a previous state, you can restore an older version without losing the entire history.

restored_version = prompts.restore_version(prompt_id="your-prompt-id", version_id="1")
# Now the latest version is the restored one
prompt = prompts.get(prompt_id=restored_version.prompt_id)

Restoring a prompt is like Git revert: it creates a new version automatically without losing history.

Best Practices for Prompt Management

Based on my experience with Vertex AI Prompt Management, here are some recommendations:

1. Tag Prompts by Purpose (Naming Convention)

Use consistent naming conventions so people can quickly find prompts.

Examples:

• prod-user-support-faq
• test-customer-onboarding
• research-longform-summary-experiment

✅ Good names help large teams collaborate better without confusion.

2. Use Variables Extensively

Avoid hardcoding content inside prompts. Instead, parameterize inputs with variables whenever possible.

It improves flexibility and reduces the number of prompt versions you need to maintain.

Example:

prompt_data="Summarize the following article: {article_text}"

Instead of:

prompt_data="Summarize the article about the power outtage in Spain."

3. Bundle GenerationConfig with Prompts

Always explicitly set parameters like temperature, top_p, max_output_tokens) inside the prompt object.

✅ This ensures reproducibility. You always know what settings were used with a particular prompt version.

4. Prepare for Prompt Migration Between Models

When moving prompts from one model, like Gemini 1.5 Flas,h to another,r like Gemini 2.0 Flash:

• Assume small prompt adjustments will be necessary
• Version separately for each target model if needed

✅ Different models can interpret the same prompt slightly differently

The full code for this article is available on GitHub

gen-ai-livestream/prompts/prompt-management/management.ipynb at main ·…

Thanks for reading and listening

I appreciate your feedback and hope you enjoyed the article.
You can find me on LinkedIn. Even better, subscribe to my YouTube channel ❤️.