How to get started with AWS Bedrock Agents as code
A prime example
📣 Update [September 2024] 🔽
Since this article CDKLabs release Generative AI constructs! I’ve made a new article with an updated Bedrock-Agents-Starter repository!
Introduction
An AI agent is a computer program or system that autonomously perceives its environment, processes information, and takes rational actions to achieve specific objectives. In the context of LLMs, an agent engage in conversations, and reason, demonstrating a degree of autonomy akin to that of digital assistants but with enhanced capabilities in written communication and task execution.
At AWS re:Invent 2023, Amazon Web Services (AWS) announced the general availability of the fully managed Agents for Amazon Bedrock. These Agents enable generative AI applications to execute multi-step tasks across various systems and data sources. The Agents use the reasoning capabilities of foundation models to break down user requests into logical sequences, determine necessary information, and decide on the APIs to call and the execution sequence. They securely perform these tasks in the background, encrypting data in transit and at rest, thereby relieving customers from the need to engineer prompts, train models, or manually connect systems.
Bedrock Agents can be created via the AWS console, CLI or API. Sadly, CDK support is not yet available. In this article we will explore how to create a Bedrock Agent with actions groups in CDK using custom resources.
Bedrock agents
High level AWS Bedrock agents break down in two sets of APIs. A build API to create agents and a run-time API to interact with previously defined agents.
An agents is build with following components:
- A foundational model — Currently only
Claude InstantandClaude v2are supported. these models are invoked to understand user requests, break down complex tasks into multiple steps, carry on a conversation to collect additional information and take actions to fulfil the request. - Instructions — Instructions that describe what the agent is designed to do. A lot of the prompt engineering work will be covered by AWS. However, with advanced prompts, you can further customise instructions for the agent at every step of orchestration.
- Action groups — (optional) these define the actions the agents can execute. Action groups are defined by an openAPI schema and associated lambda method.
- Knowledge bases — (optional) contain information than agents can query for extra context and/or augment the responses.
Base prompt templates extend the foundational models to process the user requests, break down complex tasks into multiple steps, carry on a conversation to collect additional information and take actions to fulfil the request.
Bedrock agents solve user-requested tasks with a reasoning technique called ReAct (synergising reasoning and acting). You do not have to worry about implementing a ReAct prompt flow, Amazon Bedrock automates the prompt engineering and orchestration of the tasks. The base prompts include a sequence of question-thought-action-observation examples. Bast on interactions the Agent will collect observations. The actions that the FM is able to choose from are made available as options when the model decides the next step.
Action groups
Actions involve the agent’s ability to use external services and APIs for tasks beyond language generation, like executing operations or retrieving additional information. Actions allow allowing an agent to interact with and manipulate its environment in more pre-determined manners.
Action groups define the tasks that you want your agent to help users carry out. An action group requires two main components.
- An OpenAPI schema that define the APIs that your action group should call. Your agent uses the API schema to determine the fields it needs to elicit from the customer to populate for the API request.
- A Lambda function that defines the business logic for the action that your agent will carry out.
There are two options to define the openAPI specification the API the agent can use. You can host the specification on AWS S3 or you can set it by a string when creating the action group via the API. The agent can use the response from the Lambda function for further orchestration or to help it return a response to the user.
Bedrock agents as code
AWS Cloud Development Kit (CDK) is an open-source software development framework that enables developers to define cloud application resources using familiar programming languages such as TypeScript, JavaScript, Python, Java, C#/.Net, and Go. CDK provides cloud resource constructs with proven defaults, design and share reusable components that meet their security, compliance, and governance requirements. By defining our infrastructure as code, we can use the same tools and practices for managing our infrastructure as we use for managing our application code. This allows to deploy our solution to multiple environments without having to manually configure our infrastructure for each environment.
There are a few resources that need to be created:
- A deployment for the openAPI specification
- A lambda method that implements the Bedrock Agents requests, responses
- A role for the Bedrock Agent
- The Agent together with the associated action groups
In this example we’re not going to add knowledge base.
While there are no constructs available for Bedrock Agents at the moment, we can use the CDK Custom resources to create a Bedrock Agent. Following code example are inspired by the work in this repository (Bedrock Agent and Bedrock Knowledge Base Constructs).
The code
We’ll create a BedrockAgent that extends a Construct
export class BedrockAgent extends Construct {
readonly region: string;
readonly agentName: string;
readonly instruction: string;
readonly foundationModel: string;
readonly agentResourceRoleArn: string;
readonly description: string | undefined;
readonly idleSessionTTLInSeconds: number;
readonly bedrockAgentCustomResourceRole: Role;
readonly actionGroups: ActionGroup[];
}This constructs includes an AWS lambda method (Python) that will use the AWS Bedrock build APIs.
const onEvent = new PythonFunction(this, 'BedrockAgentCustomResourceFunction', {
runtime: Runtime.PYTHON_3_10,
index: 'index.py',
handler: 'on_event',
description: 'Custom resource to create a Bedrock agent.',
entry: path.join(__dirname, '..', '..', 'lambda', 'BedrockAgentCustomResource'),
timeout: Duration.seconds(600),
environment: {
AGENT_NAME: this.agentName,
INSTRUCTION: this.instruction,
FOUNDATION_MODEL: this.foundationModel,
AGENT_RESOURCE_ROLE_ARN: this.agentResourceRoleArn,
DESCRIPTION: this.description ?? 'Undefined',
IDLE_SESSION_TTL_IN_SECONDS: this.idleSessionTTLInSeconds.toString(),
ACTION_GROUPS: JSON.stringify(this.actionGroups),
},
role: this.bedrockAgentCustomResourceRole
});This Lambda method will be triggered on the CDK events (create, update or delete) of a BedrockAgent construct (each of these methods need to be implemented).
agent_client = boto3.client("bedrock-agent", region_name=region)
def create_agent(agent_name,
agent_resource_role_arn,
foundation_model,
agent_description,
agent_session_timeout,
instruction):
# creation of a Bedrock agent via the Bedrock agent build api
args = {
'agentName': agent_name, # unique name of an agent
'agentResourceRoleArn': agent_resource_role_arn,
'foundationModel': foundation_model,
'idleSessionTTLInSeconds': int(agent_session_timeout),
'instruction': instruction,
'description': agent_description
}
if args['description'] == 'Undefined':
args.pop('description')
response = agent_client.create_agent(**args)
return response['agent']['agentId']An ActionGroup has following structure:
export interface ActionGroup {
readonly actionGroupName: string;
readonly actionGroupExecutor: string; // lambda method ARN
readonly s3BucketName: string; // Bucket where the openAPI spec is stored
readonly s3ObjectKey: string; // Key of the openAPI spec
readonly desription?: string;
}For each action group (an action group is created for an existing agent, you can re-use the same lambda en openAPI spec in other agents.)
The following code will run for each defined action group.
def create_agent_action_group(agent_id,
lambda_arn,
bucket_name,
key,
action_group_description,
action_group_name):
args = {
'agentId': agent_id,
'actionGroupExecutor': {
'lambda': lambda_arn,
},
'actionGroupName': action_group_name,
'agentVersion': 'DRAFT',
'apiSchema': {
's3': {
's3BucketName': bucket_name,
's3ObjectKey': key
}
},
'description': action_group_description
}
if args['description'] == 'Undefined':
args.pop('description')
return agent_client.create_agent_action_group(**args)Next up, creating an action that our agent can use.
The action group lambda and openAPI specification
For the purpose of this demo we’re going to create an action group to perform a very simple task, but one a large language model struggles with, checking primes (try that for yourself)
First, we’ll create the openAPI specification for a prime-checking action.
{
"openapi": "3.0.0",
"info": {
"title": "Prime checker API",
"version": "1.0.0",
"description": "Prime checker API"
},
"paths": {
"/is_prime": {
"post": {
"summary": "Checks if a number is prime",
"description": "Checks if a number is prime",
"operationId": "isPrime",
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"number": {
"type": "string",
"description": "number to check"
}
}
}
}
}
},
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"isPrime": {
"type": "boolean",
"description": "true if the number is prime, false otherwise"
}
}
}
}
}
},
"400": {
"description": "Bad request. One or more required fields are missing or invalid."
}
}
}
}
}
}Amazon Bedrock expects a the following response from your Lambda method.
Checking for a prime is just a few lines (this approach works great, but will become slow for very big primes — again, demo purposes; extending the capabilities of LLMs).
def is_prime(n):
n = int(n)
if n <= 1:
return False
if n <= 3:
return True
if n % 2 == 0 or n % 3 == 0:
return False
i = 5
while i * i <= n:
if n % i == 0 or n % (i + 2) == 0:
return False
i += 6
return TrueThe Stack
Everything is brought together in a single CDK stack (AgentStack) that will; upload an OpenAPI specification to AWS S3, create a Lambda method for the action group, create the role for the agent and create the bedrock agent itself by using the custom resource defined above. The stack looks like this:
export class AgentStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
// Bucket to store the action specifications
const actiongroupSpecBucket = new Bucket(this, 'ActiongroupSpecBucket', {});
// openAPI specification
const openAPISpecJson = fs.readFileSync(path.join(__dirname, '..', 'actions', 'demo-action', 'spec.json'), 'utf8');
// upload the openAPI specification to AWS S3
new BucketDeployment(this, 'DeployActiongroup', {
sources: [Source.jsonData('spec.json', JSON.parse(openAPISpecJson))],
destinationBucket: actiongroupSpecBucket,
destinationKeyPrefix: 'actions/demo-action/'
});
// Create an AWS Lambda method (no roles are defined here because the demo action does not need any additional permissions)
const demoActionFunction = new PythonFunction(this, 'ActiongroupFunction', {
entry: path.join(__dirname, '..', 'actions', 'demo-action'),
index: 'index.py',
handler: 'lambda_handler',
runtime: Runtime.PYTHON_3_10,
timeout: Duration.seconds(30),
});
// Create a role for the agent.
const agentRole = new Role(this, 'AgentIamRole', {
roleName: 'AmazonBedrockExecutionRoleForAgents_' + 'demo-agent',
assumedBy: new ServicePrincipal('bedrock.amazonaws.com'),
description: 'Agent role.',
})
// Give the agent role the permission to use AWS Bedrock (in a real application you should be more strict regarding the permissions provided here)
agentRole.addToPolicy(new PolicyStatement({
actions: ['*'],
resources: ['arn:aws:bedrock:*'],
}));
actiongroupSpecBucket.grantRead(agentRole);
// Bedrock Agent properties
const bedrockAgentProps: BedrockAgentProps = {
agentName: 'demo-agent',
instruction: 'you are an agent that helps a user to check if numbers have certain properties. e.g if a number is prime. You use action groups for these tasks',
foundationModel: 'anthropic.claude-v2',
agentResourceRoleArn: agentRole.roleArn,
description: 'Example agent',
idleSessionTTLInSeconds: 3600,
actionGroups: [{
actionGroupName: 'demo-action',
actionGroupExecutor: demoActionFunction.functionArn,
s3BucketName: actiongroupSpecBucket.bucketName,
s3ObjectKey: 'actions/demo-action/spec.json',
desription: 'Demo action',
}]
};
// Create the Bedrock agent in this stack
new BedrockAgent(this, 'BedrockAgent', bedrockAgentProps);
}
}Deployment
At the moment Bedrock agents are available in us-east-1 and us-west-1. Bedrock does not support cross-region Lambda invocations, so we’ll deploy all resources in us-east-1.
To deploy this stack to an AWS account, you’l need to define this stack in the index.ts file of your CDK project:
const app = new cdk.App();
const props = {
env : {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION || 'us-east-1',
}
};
const agentStack = new AgentStack(app, 'AgentStack', props);cdk deploy AgentStack --profile=YOUR-AWS-PROFILEThis will create a new stack called AgentStack in your AWS account with the Bedrock Agent, an API specification in an S3 bucket and an action group Lambda method.
Testing the Agent
After creating an agent, you can find it in the Agents section of the AWS console under the AWS Bedrock service. When you first create an agent, you will have a working draft version and a TestAlias alias that points to it. If the agents prompts you to prepare, do so, this will package the agent with the latest changes. At this point you can interact with the agent and inspect the trace of the agent’s reasoning; The prompts, the observations, the planning actions, etc… in a json format. This article in the AWS Bedrock documentation will guide you through all the steps.
With regard to the agents deployed with the AgentStack , I’ve asked it if 32647 is a prime (it is).
In the first step (pre-processing), the agents is prompted with options on how to classify the input of the user. e.g. harmful inputs, attempts to retrieve the prompts, input that is not relevant to the objective of the agent, input that is relevant to the objective of the agent or answers to possible questions of the agents to the user during the conversation.
Based on the input is 32647 a prime number? the input is classified as a question that can likely be answered. It reasons to use the POST::demo-action::isPrime with the correct request body based on the openAPI specification. After receiving the response from the lambda, that 32647 is indeed a prime, the final observation is returned to the user as answer.
Updating the stack
Some work in progress; while working with AWS Bedrock agents, I’ve implemented an on_update() method in the python lambda that triggers on CDK changes to the custom resource used to create the agent. This allows for faster prototyping in my opinion! Looking forward for full CDK support for agents!
START RequestId: Version: $LATEST
Action groups to create: set()
Action groups to delete: set()
Action groups to check for update: {'demo-action'}
Action group demo-action is up to date
END RequestId:
REPORT RequestId: Duration: 822.90 ms Billed Duration: 823 ms Memory Size: 128 MB Max Memory Used: 71 MB Conclusions
This article goes over my first experience with Bedrock agents and how to create and update them via code (CDK, custom resources, AWS Bedrock APIs (python)). That takes some initial time, but it speeds up development and iterations on the agent and action groups significantly.
The prompts generated based on the objective, provided actions, base prompts are interesting! They seem to provide a good-enough barrier between the user’s input and the action groups. I did not manage to get passed them. The fact that you can see the reasoning trace and the InvokeModel inputs are really nice and useful.
Actions via lambda methods and openAPI specifications are a really strong combination in my opinion. This concepts will scale very will and allows developer teams with strong knowledge creating APIs to create actions for LLM agents without a lot of hands on experience with generative AI technology.
The default prompts in combination with ClaudeV2 (on demand) were not that fast for my setup. The agent took 10 to 15 seconds for most responses. Remember, cross region calls are not possible (yet?).
Hope you enjoy it!
