SenseDeep 2.3 — Correlated Logs

If you are using AWS Lambda or Serverless and you want to correlate logs from multiple lambdas and AWS services, then this release is for you.

When using AWS Lambda, you typically have multiple Lambda functions or services that cooperate to respond to a single client request. This means that the log data for a single request is scattered over multiple AWS CloudWatch log groups. Diagnosing a single request by tracing the log events for that request in all the relevant log groups can be difficult and time consuming.

SenseDeep 2.3 addresses this problem by supporting meta logs to provide a unified, correlated view over multiple log groups.

SenseDeep combines multiple AWS CloudWatch log groups into unified, correlated log views. Perfect for Lambda to quickly pinpoint errors in serverless architectures.

Features

I’m happy to announce SenseDeep 2.3 which has some great new features including:

• Meta logs that merge multiple log groups into a unified, correlated log view.
• Log view activation control to temporarily deactivate views while keeping configuration.
• Greatly optimized cloud and log synchronization. Faster, increased parallelism and more responsive.
• Many minor UI improvements.
• Improved Marketplace subscription dialog with option to rerun the IAM Template.
• Icon to copy log events to the clipboard.
• Option to reload logs or clouds of logs.
• Improved Viewer first time render of a log view.
• Reduced default log max age to 7 days to focus on most recent data.

A SenseDeep meta log will combine multiple AWS CloudWatch log groups for different services into a unified and correlated log view. This single log can then be viewed and queried like any normal SenseDeep log view.

To create a meta log, click on the Logs icon in the left sidebar and then click “Add”. Enter your log name, maximum amount of data to store and the maximum age of data.

You can explicitly select the individual contributing logs by clicking the checkbox next to each dependent log. Alternatively you can enter a regular expression pattern that will be used to dynamically match contributing logs. Using a regular expression pattern is preferable if you have a changing set of log group names.

High Cardinality IDs

To get the most out of meta logs, you should define a unique request ID that is passed to all your Lambda functions and then emitted in all log events. This is sometimes called a High Cardinality ID. Using this ID, you can filter log events on this ID and display only those events. A single double click in the SenseDeep Viewer on the request ID will isolate and display the entire request over all participating services.

Match Logs

When creating a meta log you can explicitly select the individual contributing dependent logs. Alternatively you can select the logs using a dynamic regular expression pattern. Using a regular expression pattern is preferable if you have a changing set of log group names.

Fixes

This release also fixes:

• Double click on Lambda report events to filter.
• Log unwanted log pruning when changing organization.
• Automatic upgrading to new versions.

Summary

SenseDeep is the most popular and fastest CloudWatch Logs viewer. It is 100% browser based and rather than duplicating log event data in a 3rd party solution, SenseDeep transparently downloads log events and caches them locally in your browser. As you scroll forward or backward, SenseDeep fetches new log events in the background to anticipate your need. With SenseDeep, events can be displayed, queried, and scrolled with lightning speed.

Getting Started

There is nothing to install. Just navigate your browser to: https://app.sensedeep.com/

To learn more about SenseDeep and how to use the app, please read the documentation at: https://www.sensedeep.com/doc/.

Please let us know what you think, we thrive on feedback. dev@sensedeep.com.

Originally published at https://www.sensedeep.com on April 5, 2020.