AWS XRay: Check the internal health of your application

What is an X-ray?

According to Google, the definition is:

“ X-ray is a picture taken of the inside of something using high energy electromagnetic radiation with short wavelengths that can pass through items”

Note the words “ picture taken of the inside of something” which is the same thing that AWS Xray does to your application. It helps you to diagnose, analyze and debug applications (mostly distributed applications built on microservices architecture)

With X-Ray, developers can understand how their applications and services are performing and troubleshoot any errors encountered in those applications. It provides a snapshot of how a request flows through an application along with a view of all the internal components of the application.

AWS X-Ray provides tools that developers can use to view, filter, and gain insights into that data to identify issues and opportunities for optimization. For any traced request to your application, you can see detailed information not only about the request and response but also about calls that your application makes to downstream AWS resources, microservices, databases, and HTTP web APIs.

To start with you can use the X-Ray SDK to instrument your application code. You can use this SDK application locally or on application deployed on AWS EC2, Lambda, or ECS.

The SDK records data about incoming and outgoing requests and sends it to the X-Ray daemon, which relays the data in batches to X-Ray. For example, when your application calls S3 to retrieve a file from a bucket, the X-Ray SDK records data from both the client request and the downstream call to S3.

The X-Ray SDK provides:

• Interceptors to add to your code to trace incoming HTTP requests
• Client handlers to instrument AWS SDK clients that your application uses to call other AWS services
• An HTTP client to use to instrument calls to other internal and external HTTP web services

Enough of theory lets now have a hands-on session!!!!

Prerequisites

• AWS Console login and configured on the local desktop.
• NPM and Node 10+ installed
• npm install aws-xray-sdk
• MySQL DB connection string or locally installed MySQL DB

We will be instrumenting a Node JS Express application using the XRay SDK to get insights from the application.

To use the Express middleware, initialize the SDK client and use the middleware returned by the express.openSegment function before you define your routes.

Example app.js — Express

var app = express();
var AWSXRay = require('aws-xray-sdk');
app.use(AWSXRay.express.openSegment('MyApp'));
app.get('/', function (req, res) {
  res.render('index');
});
app.use(AWSXRay.express.closeSegment());

After you define your routes, use the output of express.closeSegment as shown to handle any errors returned by the X-Ray SDK for Node.js.

We are going to develop a simple Express application with 4 routes:

1. Calling a MySQL DB (Use the captureMySql function of AWSXRay module)

const mysql = AWSXRay.captureMySQL(require('mysql'));
app.get('/mysql/', (req, res) => {
 const mysqlConfig = require('./mysql-config.json');
 const config = mysqlConfig.config;
 const table = mysqlConfig.table;
 if (!config.user || !config.database || !config.password || !config.host || !table) {
   res.send('Please correctly populate mysql-config.json');
   return;
 }
 const connection = mysql.createConnection(config);
 connection.query(`SELECT count(*) as count FROM ${table}`, (err, results, fields) => {
 if (err) {
   res.send(`Encountered error while querying ${table}: ${err}`);
   return;
 }
 console.log(JSON.stringify(results));
 res.send(`Retrieved the following results from  ${table}:\n${results[0].count}`);
 });
 connection.end();
 });
));

2. Calling an AWS S3 Bucket (All AWS service calls are taken care by AWSXRay SDK)

app.get('/aws-sdk/', (req, res) => {
try {
 var s3 = new AWS.S3({ accessKeyId: AWS_S3_ACCESS_KEY_ID, secretAccessKey: AWS_S3_SECRET_ACCESS_KEY });
 var getParams = { Bucket: 's3BucketName', Key: 'xyz.json' };
//Fetch or read data from aws s3
 s3.getObject(getParams, function (err, data) {
 if (err) {
   res.send(`Encountered error while reading file: ${err}`);
 } else {
   res.send(`result:\n ${data.Body.toString()}`);
 }});
} catch (error) {
  logger('error', 'searchHandler', 'search_entity_handler', 'error  while getting config files from S3 bucket', ' Error: ' + JSON.stringify(error));
  throw error;
}});

3. Calling a public website (Use captureHTTPsGlobal from AWSXRay SDK)

AWSXRay.captureHTTPsGlobal(require('https'));
const https = require('https');
app.get('/http-request/', (req, res) => {
  const endpoint = 'https://google.com/';
  https.get(endpoint, (response) => {
   response.on('data', () => {});
   response.on('error', (err) => {
     res.send(`Encountered error while making HTTPS request: ${err}`);
  });
  response.on('end', () => {
  res.send(`Successfully reached ${endpoint}.`);
  });
 });
});

4. Calling another service that has XRay instrumented and hosted on the same ECS cluster like this one. (Use captureHTTPsGlobal and capturePromise() from AWSXRay SDK)

const axios = require('axios');
AWSXRay.captureHTTPsGlobal(require('http'));
const http = require('http');
AWSXRay.capturePromise(); // We should capture promies
const instance = axios.create({
  httpAgent: new http.Agent(),
  httpsAgent: new https.Agent(),
}); // Instrument axious instance
app.get('/callECSAPI/', async (req, res) => {
 let url = 'http://xray-Public.us-east-1.elb.amazonaws.com/callecs';
 let body = await instance.get(url);
 console.log(body);
 res.send(JSON.stringify(body.data));
});

Go ahead and deploy this application on ECS Fargate Cluster with ALB (Application Load Balancer) with XRay enabled on the cluster.

Now once we deployed the ECS, below is the ALB link generated.

When we call the various routes of the endpoints, the traces are created via the various components in Xray.

Login to your AWS Console, go to AWS XRay Services, and check the traces of the requests that we made.

Note: I gave wrong S3 access key and id to show errors and success both. Click on a specific trace and click on Analyze trace to get more details of the error.

References:

AWS X-Ray SDK for Node.js

Tracing AWS SDK calls with the X-Ray SDK for Node.js