The [SITE] Framework for LLMs — Part 2: Inferring

Table of Contents

1. [S]ummarizing
2. [I]nferring
3. [T]ransforming
4. [E]xpanding

Introduction

As previously discussed, understanding the potential of Large Language Models (LLMs) is important for Product Managers because we can leverage this technology to drive new features and products.

Understanding how to optimize prompts and how to identify primary use cases for integrating API calls into LLMs becomes an invaluable skill set for any Product Manager leading an AI product. It can help us better solve user needs and provide competitive advantage in crowded marketplaces.

In a previous post we looked at First Principles of Prompt Engineering. With First Principles covered, we’re diving deeper into the four primary capabilities of LLMs. Those capabilities are 1) Summarizing 2) Inferring 3) Transforming 4) Expanding.

Knowing how to effectively leverage these capabilities can be a really powerful framework for integrating and leveraging LLMs into your apps. I refer to this as the “SITE” framework for LLMs.

Today’s post focuses on the second capability: Inferring.

[I]nferring

In this post, we’ll explore the potential of LLMs when it comes to inferring key data points from text-based content — whether it’s recognizing sentiment, extracting product information, or setting up alerts based on inferred topics.

One model, one API

When we talk about inferring, we’re referring to the model’s ability to take a text as input and perform analysis — such as extracting labels, names, or understanding the sentiment of the text.

The beauty of large language models is their ability to perform these tasks without the need to create and train separate models for each specific task, saving precious development time and resources. For many inference tasks, you can just write a prompt and have the model start generating results pretty much right away. And that gives tremendous speed in terms of application development. And you can also just use one model, one API, to do many different tasks rather than needingto figure out how to train and deploy a lot of different models.

Inferring Sentiment

Consider, for example, the task of discerning a sentiment — positive or negative — from a product review. In a traditional machine learning workflow, this would involve collecting a labeled dataset, training a model, and deploying it for inferences. With large language models, however, we can write a prompt to kickstart the generation of results almost instantaneously.

Examples: The [I]nference Prompt

Let’s illustrate with a simple example. Suppose we have a review for a lamp.

We can easily construct a prompt to classify the sentiment of this review, with the result being a concise and clear sentiment label — ‘positive’ or ‘negative.’

positive

And with this prompt, the model not only extracts sentiment but can also detect specific emotions.

happy, satisfied, grateful, impressed, appreciative

Or you could hone in on a specific emotion like anger to quickly batch process all the angry customers into one group for a customer outreach program.

No

Information Extraction

Beyond sentiment analysis, large language models can extract more granular data from customer reviews. We can ask the model to identify specific things like the item purchased and the name of the company that made it. This can be invaluable for tracking trends and sentiment about specific product lines or manufacturers.

{
  "Item": "lamp",
  "Brand": "Lumina"
}

Perhaps more impressive, we can compile a single prompt to extract multiple fields from a piece of text and streamline the process further — all with one API call.

{
  "Sentiment": "positive",
  "Anger": false,
  "Item": "lamp with additional storage",
  "Brand": "Lumina"
}

Inferring Topics

Another powerful application of large language models is topic inference. Given a lengthy piece of text, such as a newspaper article, we can prompt the model to identify the key topics being discussed. This becomes especially useful for indexing articles and tracking specific themes.

This simple technique gives us a more robust way to figure out if a particular article covers a specific topic like “NASA.”

government survey, job satisfaction, NASA, Social Security Administration, employee concerns

Let’s say we want to know whether a news article discusses specific topics such as “NASA,” “employee satisfaction,” or “federal government.” By constructing a prompt, we can determine which of these topics are covered in the news article and assign appropriate booleans for subsequent processing within a workflow.

nasa: 1
local government: 0
engineering: 0
employee satisfaction: 1
federal government: 1

Conclusion

In the span of a few minutes, we can construct multiple systems for making inferences about text that would have previously taken days or even weeks for a skilled machine learning developer. This capability opens up a ton of valuable use cases from sentiment analysis to topic inference which can be incorporated into your app and provide value to users.

In Part 3 of the [SITE] framework, we’ll explore how LLMs can be used for Transorming text, such as translating content to a different language.

If you enjoyed this post please check more in my blog series AI For Product People.