Super-Fast Inference of LLMs with Groq

Imagine artificial intelligence that understands language with blazing speed and efficiency. That’s the future Groq is building with its revolutionary Language Processing Units (LPUs).

Forget clunky processors — LPUs are tailor-made for AI tasks, accelerating everything from real-time translation to in-depth sentiment analysis. Groq’s innovation promises to unlock a new era of AI potential.

Here are some of the key benefits of Groq’s LPU:

• Faster inference: Groq’s LPU is designed to be significantly faster than traditional processors for AI tasks. This can lead to faster results and improved performance.
• More efficient: Groq’s LPU is also more energy efficient than other AI processors. This can help to reduce costs and improve the environmental impact of AI applications.

What are Language Processing Units (LPUs)?

They are a new type of processing system called an LPU™ inference engine. This system is designed to improve the performance of large language models (LLMs), which are known to be computationally demanding and require a lot of memory bandwidth. LPU™ inference engines address these challenges by providing exceptional sequential performance, a single-core architecture, and instant memory access. This allows them to generate text sequences much faster than GPUs.

GROQ API:

We will be utilizing Groq API to test open-source models available now like Llama2, Gemma and Mistral. These models are made available on Groq, because of the processing unit called LPUs they made the models ultra fast while inferring.

To create an API key: https://console.groq.com/keys

You can also check for the rate limit.

Data:

The Data we are going to use is about location reviews and ratings of Starbucks stores in the USA region.

The data has been collected from ScrapeHero, one of the leading web-scraping companies in the world. Here’s the Data Source that we used for analysis!

Columns:

ID, Name, Address, Street, Zip_Code, State, City, Author, Review, Rating

Installation

Create a Virtual Environment on Python or Anaconda:

python -m venv myenv

Activate the Virtual Environment:

On Windows:

myenv\Scripts\activate

On macOS and Linux:

source myenv/bin/activate

Install dependency libraries:

pip install -q langchain langchain-groq openai pandas

Keyword Extraction on Starbucks Reviews:

Now let’s look at an example on how we can extract the keywords from the Starbucks reviews

from langchain_core.prompts import ChatPromptTemplate
from langchain_groq import ChatGroq

chat = ChatGroq(temperature=0.3, groq_api_key=”YOUR_API_KEY”, model_name=”mixtral-8x7b-32768")

human_prompt = “Give me only the keywords from the review: {review}”
system_prompt = “You are a helpful assistant and you are an extract at extract keywords from the text”

prompt = ChatPromptTemplate.from_messages([(“system”, system_prompt), (“human”, human_prompt)])
chain = prompt | chat

review = “Perfection! Always friendly, knowledgeable about great options to try when I can’t think of what would like, clean and comfy to sit and sip if you choose”

print(chain.invoke({“review”: review}))

Answer :

Keywords:

• Friendly
• Knowledgeable
• Great options
• Clean
• Comfy

Now let’s try to find the negative comments of Starbucks in New York city:

from langchain_core.prompts import ChatPromptTemplate
from langchain_groq import ChatGroq
import pandas as pd

df = pd.read_csv(“Starbucks_reviews.csv”)
new_york_starbucks_reviews = list(df[df.city == ‘New York’].review)

chat = ChatGroq(temperature=0.3, groq_api_key=”YOUR_API_KEY”, model_name=”llama2–70b-4096")

human_prompt = “Give me only the negative keywords from list of review: {review}”
system_prompt = “You are a helpful assistant and you are an extract at extract keywords from the list of reviews”

prompt = ChatPromptTemplate.from_messages([(“system”, system_prompt), (“human”, human_prompt)])
chain = prompt | chat
print(chain.invoke({“review”: new_york_starbucks_reviews}))

It’s super fast. It is able to extract these keywords within 2 seconds from a list of 1000 reviews which is amazing!!

Answer:

Here are the negative keywords from the list of reviews:\n\n

• Disappointment
• Slow service
• Overpriced
• Poor service
• Unfriendly staff
• Incorrect orders
• Unsatisfactory food
• Unclean bathrooms
• Unwelcoming atmosphere
• Unsatisfactory experience
• Unhelpful staff
• Unorganized
• Rude employees
• Limited selection
• Poor refund experience
• Inconvenient location
• Unhappy staff
• Unsatisfactory drinks
• Unsatisfactory food
• Unclean facility
• Unfriendly staff
• Unsatisfactory overall experience

Don’t worry, here are the positive keywords as well:

Answer:

Sure, here are the positive keywords from the list of reviews:

• Friendly
• Helpful
• Great
• Excellent
• Delicious
• Fancy
• High-end
• Relaxed
• Quiet
• Convenient
• Reliable
• Coffee
• High-quality
• Plenty of counter space
• Very welcoming
• Fresh
• Recommend

Conclusion:

In this blog, we’ve explored the potential of Groq’s LPU which is a promising new technology that has the potential to revolutionize the field of artificial intelligence.

Hope you learned something new today, Happy Learning!