How to Use LLMs to Generate Concise Summaries

Photo by Aaron Burden on Unsplash

Large language models (LLMs) are a type of artificial intelligence (AI) that can be used to generate text. They are trained on massive datasets of text, which allows them to learn the nuances of human language and generate text that is both accurate and natural-sounding.

In recent years, LLMs have been used to generate summaries of text. This is a very useful application, as it can help people to quickly and easily understand the main points of a long piece of text.

In this blog post, we will discuss how to use LLMs to generate concise summaries of text. We will be using Vertex SDK to access the LLM models from Google Cloud. Python will be the programming language of choice.

If you are not technical or not familiar with Python, please check out my other blog to get started through the user interface. It is as simple as typing a few sentences and clicking a button!

Setting up the environment

To get started, you need to have a Google Cloud project. You can create a new project by following these instructions or simply use the existing one.

Once the project is created, you can either use Vertex Workbench or Google Colab to get started. Using your own environment is also an option, but you will need to setup authentication part yourself.

Don’t forget to enable the required Vertex AI APIs!

In a Jupyter Notebook environment, install the require libraries

! pip install google-cloud-aiplatform --upgrade --user

If you are using Google Colab, you need an additional step to authenticate yourself. It can be easily done by running these 2 blocks of code.

from google.colab import auth

# choose a google account the associated with the cloud project
auth.authenticate_user()

from google.cloud import aiplatform

aiplatform.init(
    project="your-project-id", 
    location="your-project-location", 
)

Now we just need to import the model and initialize it. The following code does that.

from vertexai.preview.language_models import TextGenerationModel

text_generation_model = TextGenerationModel.from_pretrained("text-bison@001")

If you can successfully run the code above, you are good to proceed with the rest of this tutorial.

Summarizing a paragraph

In order to summarize a paragraph using a LLMs, we need to cover a few basic concepts about LLMs. LLMs takes in something called prompt, an input text (e.g. a question), and produces responses (output text) based on the structure of the prompt.

To test whether the text generation model is working as intended, try running this code.

prompt = "What is prompt design?"
answer = text_generation_model.predict(prompt, max_output_tokens=1024).text

print(answer)

You should receive a response from the model that answers your question about prompt design. Now, let’s try to summarize the answer.

To summarize a paragraph, you simply need to add “Summarize this text: ” as a prefix.

prompt = "Summarize this text: " + answer
summary = text_generation_model.predict(prompt, max_output_tokens=1024).text

print(summary)

How is the summary? If you are not pleased with the summary, you can try editing the prefix. Let’s try “Provide a concise summary of this text: ” instead.

prompt = "Provide a concise summary of this text: " + answer
summary = text_generation_model.predict(prompt, max_output_tokens=1024).text

print(summary)

Noticed any changes? You can also try asking the model to provide the summary in bullet points format. Just add “Provide the summary in bullet points.”

prompt = "Provide a concise summary of this text: " + answer + "Provide the summary in bullet points"
summary = text_generation_model.predict(prompt, max_output_tokens=1024).text

print(summary)

Take a look at the prompt variable, it’s getting messier as you add additional information to it. It can cause undesired outcome such as mixing up instructions and the input. In order to prevent it from happening, we can make use of prompt template.

A prompt template is a pre-defined structure that you can use to provide instructions to the LLM. Let’s create one and use it right now!

prompt_template = """
    Provide a concise summary of the triple backquoted text.
    
    ```{text}```

    Provide the summary in bullet points.
"""

prompt = prompt_template.format(text=answer)
summary = text_generation_model.predict(prompt, max_output_tokens=1024).text
    
print(summary)

So far, so good? We can reuse the same template to summarize other paragraphs too. While doing that, if you try to summarize a large document, you might encounter a token limit imposed by the model.

To summarize a large document, we need to be use LLMs creatively. Well, that calls for a sequel blog.

If you want to learn more about text summarization using LLMs on Google Cloud, visit our GitHub repository for a comprehensive list of resources.

generative-ai/text_summarization.ipynb at main · GoogleCloudPlatform/generative-ai

About Writer

Thu Ya is a Developer Advocate for Google Cloud and a curious person who enjoys sci-fi, fantasy, and documentaries. He is fascinated by the unknown and the possibilities of other worlds. He often wonders if aliens built the pyramids, or if we live in a simulated world.

Get connected with Thu Ya on LinkedIn and Twitter!