AWS Observability: Monitoring a Sample Web Application with AWS X-Ray

8 min readDec 29, 2023

Introduction

In today’s digital world, monitoring the performance and behavior of web applications is crucial for ensuring optimal user experiences and identifying areas for improvement. That’s where AWS Observability services come into play, providing developers with powerful tools to gain deep insights into their applications and infrastructure.

In this blog post, we will focus on one of the key AWS Observability services: AWS X-Ray. We will walk you through the process of monitoring a sample web application using AWS X-Ray and showcase how it empowers developers to achieve comprehensive application visibility and performance insights.

Observability

Observability refers to the ability to understand and gain insights into the internal workings of a system or application based on its external outputs. In the context of software development and operations, observability involves collecting and analyzing various data points such as metrics, logs and traces to gain visibility and understanding of how a system behaves.

The three key components of observability are:

Metrics: These are quantitative measurements that provide information about the performance, behavior, and usage of a system or application. Metrics can include data such as CPU usage, memory utilization, request latency, error rates, and many other relevant indicators.
Logs: Logs are textual records of events, actions, and messages generated by a system. They capture valuable information about the execution flow, errors, warnings, and other relevant activities within an application. Logs are typically stored in a centralized location for easy access and analysis.
Traces: Traces provide a detailed view of the interactions and dependencies between different components of a distributed system. They track the path of a request as it flows through various services and capture information about the time taken at each step, enabling performance analysis and troubleshooting.

AWS definition of observability

Amazon Web Services (AWS) defines observability as how well you can understand what is happening in a system, often by instrumenting it to collect metrics, logs, or traces. You must understand how your systems perform to achieve operational excellence and meet business objectives.

AWS provides services that can help customers address the requirements of the three main pillars of Observability.

CloudWatch metrics

CloudWatch monitoring helps you track metrics. Metrics are a numeric representation of data measured over intervals of time or time series data. They help identify trends, mathematical modeling, and prediction.

CloudWatch logs

Logs are immutable, timestamped records of discrete events that happen over time. They help uncover emergent and unpredictable behavior.

AWS X-Ray traces

Traces represent a series of related distributed events that encode the end-to-end request flow through a distributed system. They provide visibility, such as latency, into the path traversed by a request and the structure of the request.

AWS X-Ray

X-Ray is a service that collects data about requests the application serves. It provides tools to view, filter, and gain insights into the data to identify issues and opportunities for optimization.

X-Ray helps developers analyze and debug production and distributed applications. X-Ray helps analyze and debug modern applications built using microservices and serverless architecture.

Features of AWS X-Ray

Debugging

X-ray helps you locate the sources of bugs by aggregating errors, faults, and exceptions. Powerful search capabilities help you pinpoint issues

Tracing

X-ray collects timing and response data needed to identify high-latency services, bottlenecks, and throttling in a microservices architecture.

Service map

X-ray creates a map of services used by your application to provide you with a view of connections among clients, front-end services, and downstream services.

How AWS X-Ray works

AWS X-Ray provides a complete view of requests as they travel through your application and filters visual data across payloads, functions, traces, services, APIs, and more with no-code and low-code motions.

The following diagram depicts the X-Ray process.

Hands-on

We are going to deploy a sample web application using CloudFormation, and monitor the traffic and its performance, using AWS X-Ray.

Download the files aws-xray-node-sample-app.zip and xray-cf.yml from the GitHub repo: https://github.com/Sudarkodi-Muthiah-repo/12weeksAWSworkshopchallenge/tree/main/Week12 and save it on your local machine.

Step 1: Create an S3 bucket

Go to the S3 service console, click on Create bucket button, and fill in the bucket details.

Bucket name: Enter mys3bucket<Random_Numbers>
Region: Select US East (N. Virginia) us-east-1
Leave other settings as default.
Click on Create bucket button.

To upload the aws-xray-node-sample-app.zip file to our S3 bucket,
Click on Upload button.
Click on Add files button.
Select the zip file.
Click on the Upload button.

Copy the Bucket name and Object name for later use.

Step 2: Deploy a sample X-Ray application using CloudFormation

The sample application uses AWS CloudFormation to create an Elastic Beanstalk application that generates sample data for you to view in the AWS X-Ray console.

The CloudFormation template will create below resources:

Elastic Beanstalk application: host sample X-Ray application.
IAM Role: used by the EC2 in Elastic Beanstalk.
DynamoDB: store signup details.
S3 bucket: store the template.
SQS and SQS: decouple the application.

On the CloudFormation dashboard, click on Create stack button.

Prepare Template: Select Template is ready
Specify template: Select Upload a template file
Click on Choose file and choose the xray-cf.yml from your local machine
Click on the Next button.

On the Specify stack details page, provide the following details:

Stack Name: Enter a name of your choice.
NodejsPlatformVersion: Enter the latest version of Amazon Linux 2 Node.js 18 (Use this link to get the latest version)

S3BucketName : Enter S3 Bucket Name
S3ObjectKey: Enter Object Name
Subnet: Select any subnet from the dropdown (any default subnet)
VPC: Select the default VPC from the dropdown
Click on the Next button.

On the Configure Stack options page, leave it as default and click on Next button in the bottom right corner.
Review the Stack, Check the acknowledgment at the bottom of the page, and then click on Submit button.
Once you click on Create Stack, your stack (xray-cf.yml) will start deploying the sample application.
Initially, the stack status will be CREATE_IN_PROGRESS and it will create all the resources.
Wait for 10 minutes till the stack status changes to CREATE_COMPLETE.

Note: It will create an additional nested stack.

Step 3: Test the Application

Go to EC2 Service in the console, and click on the Instances from the left side.

Select the myappstack instance and Copy the Elastic IP Address ( it will be the application Public IPv4 that you will use to access the application).

Open the IP in the new tab. It will open the sample X-ray application.
Let’s generate some traffic by signing up for the application.
For this step, you can manually sign up for the application. But you will automate the process by clicking on the Start button.
Wait for a few minutes and then click on the Stop button

Hence, you have successfully created a few signup accounts in the sample application.

To view the signup details, navigate to DynamoDB by clicking on the Services menu at the top and search for DynamoDB in the search bar.
Click on the Explore table items button from the left navigation panel and Select the table created.
You can view the signup details.

Step 4: Monitor the traffic in AWS X-Ray

Go to the CloudWatch service in the console.
Now click on Service Map from the left navigation menu under the X-Ray traces.
In the service map, you will get insight into the traffic that flows through our application.

Now click on any of the components to view the response distribution.
It will show the traffic response status along with the graph.
Click on the Analyze Traces to view detailed information.

It will show detailed information about the HTTP Method, URL, User-Agent, HTTP Method, Trace List, etc.

You can click on any of the Trace List to view the traces.

you can see detailed information not only about the request and response, but also about calls that your application makes to downstream AWS resources, microservices, databases, and web APIs.

We have successfully deployed a sample X-ray application and monitored its traffic in AWS X-Ray.

Step 5: Clean up AWS resources

Delete the created CloudFormation stack and S3 bucket.

Conclusion

In this blog post, we delved into the world of AWS Observability, focusing on the powerful capabilities of AWS X-Ray for monitoring web applications.

We walked through a step-by-step process of building a sample web application and instrumenting it with AWS X-Ray. Using the AWS X-Ray console, we explored various features and visualizations that helped us analyze the application’s performance. We learned how to navigate trace visualizations, understand service maps, and identify performance bottlenecks.

#awsobservability #awsxray #monitoring #cloudformation #dynamodb