GenAI with Stock Trading Website (Module 3)

Module 3: Create Lambda

In this module, we will create an AWS Lambda function to process user queries and interact with the Kendra index. AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, making it ideal for handling query processing in a scalable manner.

Step 1: Set Up Lambda with Docker

3.1 Introduction to AWS Lambda with Docker

AWS Lambda allows you to run code in response to events and automatically manages the compute resources for you. By using Docker, you can package your Lambda function with all its dependencies, ensuring consistency and ease of deployment. Please create lambda in US-EAST-1 region.

3.2 Create a Lambda Function with Docker

To create a Lambda function with Docker, follow these steps:

a. Install Docker: Ensure Docker is installed on your local machine. You can download Docker from the official website.

b. Set Up a Dockerfile:

• Create a new directory for your Lambda function.
• Inside this directory, create a Dockerfile with the following content:

FROM public.ecr.aws/lambda/python:3.9

# Set the working directory
WORKDIR ${LAMBDA_TASK_ROOT}

# Copy requirements.txt
COPY lambda/requirements.txt .

# Install the specified packages
RUN pip install --no-cache-dir -r requirements.txt

# Copy function code
COPY lambda/app.py .

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.lambda_handler" ]

c. Write Your Lambda Function Code:

• Create a file named app.py in the new directory call lambda with the following content:

import os
import sys 
import json

if "LAMBDA_TASK_ROOT" in os.environ:
    envLambdaTaskRoot = os.environ["LAMBDA_TASK_ROOT"]
    sys.path.insert(0, "/var/lang/lib/python3.9/site-packages")

from langchain.retrievers import AmazonKendraRetriever
from langchain.chains import ConversationalRetrievalChain
from langchain.llms.bedrock import Bedrock
from langchain.prompts import PromptTemplate

REGION_NAME = os.environ['aws_region']

MODEL_TYPE = "CLAUDE"

retriever = AmazonKendraRetriever(
    index_id=os.environ['kendra_index_id'],
    region_name=REGION_NAME
)

if(MODEL_TYPE == "CLAUDE"):
    llm = Bedrock(
        model_id="anthropic.claude-v2:1",
        endpoint_url="https://bedrock-runtime." + REGION_NAME + ".amazonaws.com",
        model_kwargs={"temperature": 0.7, "max_tokens_to_sample": 500}
    )

    condense_question_llm = Bedrock(
        model_id="anthropic.claude-instant-v1",
        endpoint_url="https://bedrock-runtime." + REGION_NAME + ".amazonaws.com",
        model_kwargs={"temperature": 0.7, "max_tokens_to_sample": 300}
    )
else:
    llm = Bedrock(
        model_id="ai21.j2-ultra-v1",
        endpoint_url="https://bedrock-runtime." + REGION_NAME + ".amazonaws.com",
        model_kwargs={"temperature": 0.7, "maxTokens": 500, "numResults": 1}
    )

    condense_question_llm = Bedrock(
        model_id="ai21.j2-mid-v1",
        endpoint_url="https://bedrock-runtime." + REGION_NAME + ".amazonaws.com",
        model_kwargs={"temperature": 0.7, "maxTokens": 300, "numResults": 1}
    )

#Create template for combining chat history and follow up question into a standalone question.
question_generator_chain_template = """
Human: Here is some chat history contained in the <chat_history> tags. If relevant, add context from the Human's previous questions to the new question. Return only the question. No preamble. If unsure, ask the Human to clarify. Think step by step.

Assistant: Ok

<chat_history>
{chat_history}

Human: {question}
</chat_history>

Assistant:
"""

question_generator_chain_prompt = PromptTemplate.from_template(question_generator_chain_template)

#Create template for asking the question of the given context.
combine_docs_chain_template = """
Human: You are a friendly, concise chatbot. Here is some context, contained in <context> tags. Answer this question as concisely as possible with no tags. Say I don't know if the answer isn't given in the context: {question}

<context>
{context}
</context>

Assistant:
"""
combine_docs_chain_prompt = PromptTemplate.from_template(combine_docs_chain_template)

# RetrievalQA instance with custom prompt template
qa = ConversationalRetrievalChain.from_llm(
    llm=llm,
    condense_question_llm=condense_question_llm,
    retriever=retriever,
    return_source_documents=True,
    condense_question_prompt=question_generator_chain_prompt,
    combine_docs_chain_kwargs={"prompt": combine_docs_chain_prompt}
)

# This function handles formatting responses back to Lex.
def lex_format_response(event, response_text, chat_history):
    event['sessionState']['intent']['state'] = "Fulfilled"

    return {
        'sessionState': {
            'sessionAttributes': {'chat_history': chat_history},
            'dialogAction': {
                'type': 'Close'
            },
            'intent': event['sessionState']['intent']
        },
        'messages': [{'contentType': 'PlainText','content': response_text}],
        'sessionId': event['sessionId'],
        'requestAttributes': event['requestAttributes'] if 'requestAttributes' in event else None
    }

def lambda_handler(event, context):
    if(event['inputTranscript']):
        user_input = event['inputTranscript']
        prev_session = event['sessionState']['sessionAttributes']

        print(prev_session)

        # Load chat history from previous session.
        if 'chat_history' in prev_session:
            chat_history = list(tuple(pair) for pair in json.loads(prev_session['chat_history']))
        else:
            chat_history = []

        if user_input.strip() == "":
            result = {"answer": "Please provide a question."}
        else:
            input_variables = {
                "question": user_input,
                "chat_history": chat_history
            }

            print(f"Input variables: {input_variables}")

            result = qa(input_variables)

        # If Kendra doesn't return any relevant documents, then hard code the response 
        # as an added protection from hallucinations.
        if(len(result['source_documents']) > 0):
            response_text = result["answer"].strip() 
        else:
            response_text = "I don't know"

        # Append user input and response to chat history. Then only retain last 3 message histories.
        # It seemed to work better with AI responses removed, but try adding them back in. {response_text}
        chat_history.append((f"{user_input}", f"..."))
        chat_history = chat_history[-3:]

        return lex_format_response(event, response_text, json.dumps(chat_history))

Code breakdown
This Lambda function uses Docker to integrate Amazon Kendra and AWS Bedrock for creating a conversational retrieval chatbot. It initializes environment variables, sets up a Kendra retriever, and configures Bedrock models based on the specified type (CLAUDE or AI21). The function uses langchain to create a ConversationalRetrievalChain with custom prompt templates for question generation and document combination. The main lambda_handler processes user queries, retrieves relevant information using Kendra, and formats responses for simple chatbot maintaining a chat history across sessions.

d. Create a requirements.txt File:

• Add langchain and boto3 python dependencies in a requirements.txt file.

boto3 == 1.29.3
langchain == 0.0.338

e. Push the Docker Image to Amazon ECR:

• Create a repository in Amazon Elastic Container Registry (ECR).

• Tag your Docker image and push it to the ECR repository. You can get the push command by selecting “View Push Commands”

f. Create a Lambda Function Using the Docker Image:

• Go to the AWS Lambda console.

• Click “Create function.”
• Choose “Container image.”
• Select the repository and image you pushed to ECR.
• Configure the function name and role and click “Create function.”

g. Setup environment variables for lambda and increase the lambda performance.

• Kendra index ID can be fetch from Kendra console you created earlier.

• We need to increase the timeout to 15 minutes and allocate higher CPU and memory resources due to the high utilization of memory and CPU.

3.3 Test Lambda Function

1. Create Test Events:

• Create multiple test events with different queries related to stock trading.
• Ensure the function handles various scenarios, including valid queries, invalid queries, and edge cases.
• Example test event json payload we can use:

{
  "inputTranscript": "Tell me about New Zealand-based logistics company",
  "sessionState": {
    "sessionAttributes": {},
    "intent": {
      "state": "InProgress"
    }
  },
  "sessionId": "test-session",
  "requestAttributes": {}
}

2. Verify Responses:

• Verify that the Lambda function returns the correct search results from Kendra.
• Check the format and content of the responses to ensure they meet your requirements.

1. Debugging and Logging:

• Use the CloudWatch logs to debug any issues that arise during testing.

• Add logging statements in your code to help trace and diagnose problems. (Optional)

Summary

This module guides you through creating and testing an AWS Lambda function using Docker to process queries and interact with the Kendra index. In the next module, we will set up an API Gateway to expose the Lambda function to the chatbot.

Next Module: GenAI with Stock Trading Website (Module 4)