The Crow Flies at Midnight — Exploring Red Team Persistence via AWS Lex Chatbots
Today, we will look at using an AWS Lex Service chatbot as a persistence method for a red teamer. This entire blog post is more of a fun exercise in creativity than it is a practical technique. However, it never hurts to get hands-on experience with a service that is being increasingly used by companies in the AI explosion.
For those unfamiliar with the differences between a red team engagement and a penetration test — allow me to briefly explain.
A penetration test will attempt to uncover as many vulnerabilities/security risks in an environment as possible within a window of time.
While a red team will generally have a higher-level goal, such as:
- Attempt to access specific sensitive data (customer data, trade secrets, etc.)
- Attempt to evade detection from defenders while working in the environment
- Attempt to establish persistence
Why persistence is valuable
Before diving into the technical aspects, let’s describe persistence and why it’s valuable to an adversary.
Once an adversary has access to an environment, they generally want to maintain said access. They might want to take a lovely vacation from December 1st until December 25th, to resume hacking when fewer personnel look at logs. They may wish to routinely exfiltrate future data yet to enter the environment. Additionally, real-world adversaries may also sell their access to others as an initial access broker.
These are some reasons why persistence is valuable and the mindset you must be aware of as a red teamer. Thus, demonstrating persistence might appear in the high-level engagement goals of a red team assessment. So the challenge becomes — how do we ensure we can access the environment at a future date?
There are quite a few persistence methods that are publicly known for AWS. I’m including them here as they will generally be more practical than using Lex, in all honesty.
https://hackingthe.cloud/aws/post_exploitation/iam_persistence/
- IAM user login profile
- IAM Role Assume Role Policy
- STS (federation token)
- EC2 Instance Persistence
https://hackingthe.cloud/aws/post_exploitation/lambda_persistence/
- Lambda Persistence
https://hackingthe.cloud/aws/post_exploitation/user_data_script_persistence/
- User Data Script Persistence
AWS Lex Use-cases
Amazon Lex is an AWS service that allows developers to build conversation chatbots with which their customers can interact. These days, it’s also often hooked up to AI, so the company doesn’t have to pay a human to provide customer support. I assert that anyone who has been on the internet long enough has seen a chat window pop on a website with something equivalent to “looks like you’ve been browsing our website, need some help?” and is familiar with chatbots.
You can also build a voice chat agent using Lex, but I’ll leave it for someone else to alter this persistence concept to utilize a slide whistle to relive their glory days of phreaking.
The hypothetical situation
For our persistence, we’re going to imagine we’ve been hired by an air travel company. This company worries about cyberattacks after several doors have flown off planes midflight and videos have gone viral (negatively). An utterly unrealistic scenario that would never occur. Either way, they’ve hired a red team to come in and demonstrate if it’s possible to obtain sensitive data, evade detection, and establish persistence. They also have a wonderful sense of humor, as all security professionals do.
They use AWS Lex as a chat agent to help customers book their flights, check their status, and otherwise use their services. After wrapping up the first two goals, you have extra time and intend to demonstrate persistence and your sense of humor to make sure they keep coming back.
The Game Plan
While it’s possible to build a Lex chatbot from scratch and do everything in a custom manner. We’ll be using the most low-effort method as a proof of concept. We will use the Lex Bot Templates, pick a business vertical, run the CloudFormation that is so helpfully provided by the Lex team, and modify the business logic functions in Lambda to demonstrate.
Specifically, we will modify a Lambda function from the Lex bot template. If our sleeper phrase is used, it will trigger our malicious business logic to provide credentials for its Lambda role.
Modifying the Lambda Function
Once we’ve deployed the CloudFormation stack, it will create a Lambda Application in our account with associated Lambda functions. Viewing the Lambda functions that are now in our account shows there is one function named “AirlinesBusinessLogic.”
We must test our function, which requires sending an “event” as input to the Lambda function. An event is the input that would be made in the API call to invoke the Lambda.
Unfortunately, there’s no template event for our Airplane company, but its syntax is similar enough to where it works fine.
We can view the function logs after the Lambda was invoked using the test event button with our event. We can also send test events to the function and the logger.debug() output will display. This is also available via CloudWatch logs.
We’re now interested in taking the user input and checking if it contains our sleeper phrase; we’ll be using “the crow flies at midnight.” The most promising function in the Python code that’s being used is the “dispatch” function, which is also defined within the same lambda_function.py file.
We can see the function is essentially just routing the event to the correct “intent” module. An intent is a Lex-specific terminology and is essentially the goal the chatter is attempting, such as “Book a flight.”
From the logger.debug, we can see the event being passed to the dispatch() function and stored in a variable now called intent_request. We’re looking for the raw user input; the closest guess we have without reading through the documentation to confirm would be the inputTranscript.
So we’ll modify the event JSON to include that sleeper phrase.
We must then test for that phrase from the raw user input. I wrote a quick check for that with logger.debug() messages.
To exploit this now requires writing code to steal the Lambda’s role credentials. Luckily, I’ve written that code previously and can reuse it. One of the benefits of saving previous work and tossing it on GitHub.
https://github.com/lmoratti/moratti-cloud-toolkit/blob/main/index.py
We must use the same imports from the GitHub repo (boto3, base64) to create a one-liner base64 encoded. The base64 encoding is for workflow reasons, so it’s easier to copy and paste into a ~/.aws/credentials file and then use. Optionally, you can return these in clear text by modifying the Lambda response JSON later in this post to return an array of messages.
From the logging we can see that it does indeed find credentials and then creates a base64 blob.
I pasted that blob into output.txt and confirmed it worked by decoding it on WSL. You can also paste the blob into an online base64 decoder, though I caution to only use a client-side decoder as these will be valid credentials.
This format is for appending the decoded text to a ~/.aws/credentials file as part of my workflow optimization. This makes it easy to confirm the credentials are valid.
As you can see, we’ve confirmed the Lambda role credentials are usable. If you have the permissions to do so in a red team engagement, you could also elevate that Lambda role permissions to whatever suits your needs.
Now, we must ensure this output is returned to the chatter no matter what the intent dialogue tree they take. To make this easier, we will move the credential exfil code out of the dispatch() function and into the lambda_handler() function.
We had to change the variable “intent_request” to “event” because of where we moved the code. Now that the code is working and returning a response to the bot, we can build it and test it with our user input.
On the console, this is where the build button is located. If you have a giant screen, it’ll likely be off to the right and easy to miss.
We’ll use the Lex console to test the draft bot version.
Unsurprisingly, the format is off. To return a valid response, we have to follow the syntax that is dictated by the Lex Service.
https://docs.aws.amazon.com/lexv2/latest/dg/lambda-response-format.html
According to the documentation, a valid response syntax will always require “sessionState” which is the state of the conversation between the user and the LexV2 Bot. Within sessionState, required subfields will depend on the “dialogueAction” taken.
I am closing the session as a dialogueAction because it requires the minimal fields. The intent I am using is BookAFlight since it’s in our Bot Template.
What our modified function now looks like with the correct syntax for the response.
Now let’s give it a try by talking to the bot.
When we decode the output
We can see we have valid credentials for the role. If this bot was built and deployed externally to customers, we would have a valid persistence mechanism.
I hope this blog post was helpful or, at the very least, interesting.
If you enjoyed this post, please follow my blog or connect with me on Medium, X (formerly Twitter), Mastodon, or LinkedIN.
