Deploying a Chatbot Using AWS Bedrock Simplified

Overview

In this article, we walk through setting up a simple yet powerful AI chatbot using AWS Bedrock and the Llama3.1 405b Instruct model. You’ll learn how to create an AWS Bedrock Runtime client, configure the chatbot’s behavior, and manage a conversation loop that keeps track of both user and bot interactions. We also cover key concepts like model authentication, inference configuration, and error handling to ensure smooth communication with the model. Whether you’re building an intelligent assistant or experimenting with AI, this guide provides a clear and concise starting point.

View the appendix for the resources used in this article.

Create an IAM user

We need to create an IAM user to securely interact with the AWS Bedrock API because the IAM user provides the necessary credentials (access key and secret key) to authenticate requests to the API.

Enable Generative AI model

To use the Llama3.1 405b Instruct model with AWS Bedrock, we need to request access to it because access to all models are blocked by default.

Install proper dependencies

These libraries are essential for interacting with AWS services from a Python environment. “boto3” and “botocore” are the core libraries for creating, configuring, and managing AWS resources, with “botocore” providing the low-level components that “boto3" builds on. The “awscli” allows you to manage and automate AWS services from the command line, making it a powerful tool for deployment and management tasks.

To create new coding cells, click on the “+ Code” button at the top right of colab.

%pip install --quiet \
  boto3 \
  botocore \
  awscli

import boto3
from botocore.exceptions import ClientError

import os
import sys
from getpass import getpass

Setup AWS access

Create two new code cells and run them independently to set AWS credentials.

os.environ["AWS_ACCESS_KEY_ID"] = getpass("Your AWS Access Key ID:")

os.environ["AWS_SECRET_ACCESS_KEY"] = getpass("Your AWS Secret Access Key:")

Write chatbot code

Create a new code cell and run the the following code.

This Python script interacts with the AWS Bedrock API using the “boto3” library to create a Bedrock Runtime client. It sets up a conversational loop where the user can input prompts, which are sent to the Llama3.1 405b Instruct model (specified by the “model_id”).

The model processes the input with predefined characteristics (set in the “system_message”) and returns a response. Additionally, the conversation is maintained by appending each user and assistant message to the “conversation” list, allowing for context to be preserved.

# Create a Bedrock Runtime client in the AWS Region you want to use.
client = boto3.client("bedrock-runtime", region_name="us-west-2")

# Set the maximum length of the response.
max_tokens = 512

# Set the model ID, e.g., Titan Text Premier.
model_id = "meta.llama3-1-405b-instruct-v1:0"

# Set the characteristics of the bot.
system_message = """[INST]You are a a very intelligent bot with exceptional critical thinking.[/INST]"""

conversation = []

while True:
  try:
      user_message = input("\nEnter a prompt (empty to quit):").strip()
      if user_message:
          conversation.append({
              "role": "user",
              "content": [{"text": system_message + user_message}],
          })
    
          # Send the message to the model, using a basic inference configuration.
          response = client.converse(
              modelId=model_id,
              messages=conversation,
              inferenceConfig={"maxTokens":max_tokens,"temperature":0.5,"topP":0.9},
              additionalModelRequestFields={}
          )

          # Extract and print the response text.
          response_text = response["output"]["message"]["content"][0]["text"]
          print(response_text + "\n")

          conversation.append({
              "role": "assistant",
              "content": [{"text": response_text}],
          })
          
      else:
          print("[Exited]")
          break

  except (ClientError, Exception) as e:
      print(f"ERROR: Can't invoke '{model_id}'. Reason: {e}")
      exit(1)

Appendix

Full notebook: https://github.com/1solomonwakhungu/bedrock-chatbot

AWS: https://console.aws.amazon.com/