AWS X-Ray: Distributed Tracing in minutes

AWS X-Ray helps developers analyze and debug production, distributed applications, such as those built using a microservices architecture. With X-Ray, you can understand how your application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors. X-Ray provides an end-to-end view of requests as they travel through your application, and shows a map of your application’s underlying components. You can use X-Ray to analyze both applications in development and in production, from simple three-tier applications to complex microservices applications consisting of thousands of services.

How does X-Ray work?

X-Ray captures trace data from code running on EC2 instances (including ECS containers), AWS Elastic Beanstalk, Amazon API Gateway, and more. It implements follow-the-thread tracing by adding an HTTP header (including a unique ID) to requests that do not already have one, and passing the header along to additional tiers of request handlers. The data collected at each point is called a segment, and is stored as a chunk of JSON data. A segment represents a unit of work, and includes request and response timing, along with optional sub-segments that represent smaller work units (down to lines of code, if you supply the proper instrumentation). A statistically meaningful sample of the segments are routed to X-Ray (a daemon process handles this on EC2 instances and inside of containers) where it is assembled into traces (groups of segments that share a common ID). The traces are segments are further processed to create service maps that visually depict the relationship of services to each other.

Supported programing languages

You can instrument the following programming languages using X-Ray:

• Node.js
• Java
• C# .Net

Walkthrough

I will walkthrough how you can instrument your existing Node.js application with AWS X-Ray to capture http calls and all AWS SDK calls. I will also walkthrough how to setup X-Ray daemon in a docker image so that it can capture the data and send to AWS X-Ray

Instrument and run your application

The AWS X-Ray SDK for Node.js provides middleware that you can use to instrument incoming HTTP requests. This walkthrough uses Express.

• Add aws-xray-sdk to your package.json using npm

npm install --save aws-xray-sdk

• In your application initialize the x-ray SDK and use it prior to declaring your routes by calling openSegment()

// Initialize X-ray SDK
const XRay = require('aws-xray-sdk');
// Initialize an instance of Express
const app = express();
// Start capturing the calls in the applications
app.use(XRay.express.openSegment());

• After the all the routes have been defined, stop capturing the calls in your application by calling the closeSegment()

app.use('/', routes);
app.use(XRay.express.closeSegment());

• You can instruct X-Ray to capture all http/https wrapping captureHTTPs around http package

const http = require('http');
XRay.captureHTTPs(http);

• You can instruct X-Ray to capture all AWS SDK calls by wrapping captureAWS around aws-sdk package

const AWS = captureAWS(require('aws-sdk'));

• If you are running in AWS either on ECS (EC2 Container Service) or EC2 instance, X-ray can capture the ECS container ID or EC2 instance data. You can capture it by enabling X-Ray plugin

const XRay = require('aws-xray-sdk');
// Enable EC2 plugin
XRay.config([AWSXRay.plugins.EC2]);
// Enable ECS plugin
XRay.config([AWSXRay.plugins.ECS]);

This is all the work that is needed on the application side to start instrumenting the application using X-Ray.

The below page has lot more examples on how you can configure your Node.js application to instrument further:

aws-xray-sdk

Let’s walkthrough what needs to be done so that you can capture the traces and send to AWS X-Ray

Run the AWS X-Ray Daemon

The AWS X-Ray SDK does not send trace data directly to AWS X-Ray. To avoid calling the service every time your application serves a request, the SDK sends the trace data to a daemon, which collects segments for multiple requests and uploads in batches. Run the daemon alongside your application.

Let’s walkthrough what you need to do to setup and run the x-ray daemon in a docker image.

• Add the below instructions to your Dockerfile

# Download the x-ray daemon from s3
RUN curl https://s3.dualstack.us-west-2.amazonaws.com/aws-xray-assets.us-west-2/xray-daemon/aws-xray-daemon-linux-1.x.zip -o ./aws-xray-daemon-linux-1.x.zip
# unzip the archive file
RUN unzip -o aws-xray-daemon-linux-1.x.zip -d .

• As you need to run the daemon as a background process, I suggest you add running the x-ray daemon in background process in the Docker entrypoint script
• Add the below instructions to your docker entrypoint script

xray -f /var/log/xray.log &

• Now build the docker image and push the image to your docker registry so that ECS can pick up your service

X-Ray Console

X-Ray is still in preview, make sure you sign up for preview so that AWS can send you the link to X-Ray console when they have selected to participate in the preview.

Once you have access to the console you can navigate to the X-Ray console in that region.

e.g: For us-west-2 https://us-west-2.console.aws.amazon.com/xray will be the console.

Once you are in the console you can view your service in the Service Map section.

Service Map will look like:

AWS X-Ray Service Map in the console

Selecting a particular Service will lead you to Traces section or you can directly go to the Traces section

AWS X-Ray Traces in the console

After spending 15 minutes, I have distributed tracing for my Node.js application.

You can find more information about AWS X-Ray at:

AWS X-Ray — Distributed Tracing System

AWS X-Ray — See Inside of Your Distributed Application | AWS Blog