Getting Started with PaLM II and Python: Your Ultimate Guide to Turbocharged GenAI!

Introduction

PaLM II is the second iteration of Google’s Pathways Language Models. Google’s strategy is to focus on these pathways models which aims for a single model that could generalize across domains and tasks while being highly efficient.

According to Google, PaLM II contains improved:

• Multilinguality: PaLM II is more heavily trained on multilingual text, spanning more than 100 languages
• Reasoning: PaLM II’s wide-ranging dataset includes scientific papers and web pages that contain mathematical expressions.
• Coding: PaLM II was pre-trained on a large quantity of publicly available source code datasets. This means that it excels at popular programming languages like Python and JavaScript, but can also generate specialized code in languages like Prolog, Fortran, and Verilog.

And generally can be used for summarization, question answering, classification, sentiment analysis, entity extraction,…

In this demo, we use the “text-bison-32k” model which was released in public preview on 2023–08–29. This model can support up to 32k input tokens, this is GCP its largest PaLM model. In order to unlock this model’s full potential, you’ll want to bring its capabilities to your own tools, software, embed it into your workflows,… As this is a common pain point among starters, I decided to write a blog about calling GCP its PaLM II Models via API.

Prerequisites

Before we begin, make sure you have the following:

1. A Google Cloud Platform (GCP) account with billing enabled.
2. Python is installed on your machine.

Let’s dive into the steps to set up PaLM II with Python.

Step 1: Create a new GCP project

A project is a way to organize your resources and services in GCP. To create a new project, follow these steps: Click on the project dropdown and select “New Project.” Next, give your project a name (e.g., “python-palm2”) and click “Create.” Location can be no organization.

Step 2: Enable the PaLM II API

To use PaLM II via API, you’ll need to enable its Vertex AI API for your project. Here’s how: In the GCP Console, navigate to the “APIs & Services” pane and select “Library”. Search for the Vertex AI API and click “Enable”.

Step 3: Create a service account and generate a key

Next in order to call the PaLM II services in other services and tools, you’ll need a service account. To this, search “Service Accounts” and create a service account. I’ll call it the “genai-test “ service account. Of course, we’ll need to grant this service account the needed permissions. To keep things simple, I’ll give it the “owner” permissions.

Finally, generate access keys and download the JSON file with the key. This will allow you to authenticate to the Vertex AI API.

Step 4: Code and call the Python Function

Now that we have completed the prerequisites and steps 1 to 3. We can move ahead in our Python environment. Where we’ll import the necessary packages, load the JSON key file, and run the PaLM II “prompt model” function. In this function, I’ll ask the model to “Tell me more about GCP?”.

# Pip install the following packages 
!pip install google-cloud-aiplatform
!pip install protobuf

# Import the required packages 
import vertexai
from vertexai.preview.language_models import ChatModel, InputOutputTextPair, TextGenerationModel
from google.oauth2 import service_account
from IPython.display import Markdown

# Load the JSON Key File
mycredentials = service_account.Credentials.from_service_account_file('[your_key_file.json]')
vertexai.init(project='[your_project_name]', location='us-central1', credentials=mycredentials)

# Construct the prompting defintion 
def prompt_model(
    project_id: str,
    model_name: str,
    temperature: float,
    max_decode_steps: int,
    top_p: float,
    top_k: int,
    content: str,
    location: str = "us-central1",
    tuned_model_name: str = "",) :
    
    """Predict using a Large Language Model."""
    vertexai.init(project=project_id, location=location)
    model = TextGenerationModel.from_pretrained(model_name)
    
    if tuned_model_name:
        model = model.get_tuned_model(tuned_model_name)
    response = model.predict(
        content,
        temperature=temperature,
        max_output_tokens=max_decode_steps,
        top_k=top_k,
        top_p=top_p,)
    
    print(display(Markdown(response.text)))
    
# Call the function     
prompt_model("[your_project_name]", "text-bison-32k", 0.2, 256, 0.8, 40, "Tell me more about GCP?", "us-central1")

Voila! 🤩 You made it! We’ve prompted the model with the question to tell more about GCP and we’ve displayed the response in markdown for a more enjoyable output. As you can see below we get the expected output. Now you are equipped with the required building block code to start building your next applications.

❤️ If you found this article helpful, I’d be grateful if you could follow me on Medium and give it a clap or two. Your support means a lot to me. Thank you!

Check out how you can use Azure OpenAI in my next blog below. Azure OpenAI has solid Data Usage Policies which makes it very suitable for enterprise use cases. Although, OpenAI has made the explicit statement that prompts that are sent to their API aren’t used for retraining anymore.🎉

Azure OpenAI ChatGPT: The Ultimate AI-Powered Conversational Chatbot Solution. A Beginner Guide.

Enjoy!

Alexandre t’Kint