26 Resources That Will Help You Master Microservices Monitoring

Disclaimer: I published this article first on TechBeacon, you can read the original article here.

Microservices are solving many architectural problems that have plagued developers and operations for years. But while the benefits are now clear, many professionals aren’t fully aware of the many challenges that come with microservices. A significant portion of those center around monitoring and observability.

Microservices add complexity to your system, communication, and network, which makes monitoring more complicated. Thankfully, you can find lots of tutorials, books, tools, and case studies from organizations that have successfully implemented effective monitoring and observability in a microservices environment.

Here is our list of the most popular and useful tools and resources shared within the software engineering, site-reliability engineer (SRE), and DevOps communities.

Tutorials and guides for monitoring microservices

How Hootsuite Manages Its Growing Microservices Landscape

A software engineer from Hootsuite explains how service relationships, dependencies, and status are paramount to keeping his team, the build pipeline, and the application running smoothly. This talk includes a section on the API that’s baked into Hootsuite’s service-oriented architecture, enabling them to monitor and explore application service dependency graphs in real time.

Q&A James Turnbull: The Art of Monitoring in the Age of Microservices

James Turnbull is the author of The Art of Monitoring. This conversation with him provides a good overview of the state of microservices monitoring.

Monitoring Microservices Architectures: Enterprise Best Practices

This roundup talks to engineers from IBM, the Financial Times, and Netflix about the pain points of microservices monitoring, then moves on to describe each firm’s best practices for maintaining and monitoring microservices.

Monitoring for Distributed and Microservices Deployments

This guide explains the challenges of building distributed architectures and the required changes to scale the monitoring of microservices. It emphasizes the ways in which various system architectures can influence how you respond to incidents and resolve them.

Monitoring a Microservices-Based Application

Despite the benefits of using a microservices-based architecture, they add a lot of “noise.” Monitoring a microservices-based environment without taking this noise into account can be very confusing. Here’s how you can eliminate at least some of it.

Future-proof Docker and Microservices with Monitoring

This post explains the relationship between containers (Docker containers in this example) and microservices. It also covers the advantages of using these technologies and the challenges of monitoring Docker. “What should you look for in a Docker monitoring solution?” Three things, the author says.

The Future of Microservices Monitoring & Instrumentation

This article explains how containerization and microservices changed the way developers write and run their applications. The author predicts what new monitoring and instrumentation practices will be utilized in 2018.

Monitoring as a Service: a Modular System for Microservice Architecture

This blog post offers up a real-world example of a monitoring architecture for microservices that uses a variety of open-source tools, including Graphana, Collectd, Heapster, and Brubeck.

Monitoring a Production-Ready Microservice

This excerpt from the book Production-Ready Microservices, by Susan Fowler, provides a list of useful questions to assess the production-readiness of microservices and a microservice ecosystem. The questions are related to key metrics, logging, dashboards, alerting, and on-call rotations to give you an idea of what kinds of tools and processes you need to have in place to effectively monitor microservices.

Communication Between Microservices: How to Avoid Common Problems

This pitfalls-style post explains the performance, fault tolerance, and monitoring challenges when running microservices. Then it gives the reader some tools and suggestions to overcome those challenges.

Node.js Performance Monitoring with Prometheus

This detailed guide on Node.js monitoring with the Prometheus monitoring framework will help you understand Node.js-based microservices monitoring by covering code instrumentation (and its risks), data collection, data visualization, queries, alerting, and Kubernetes integration.

Monitoring Envoy and Ambassador on Kubernetes with the Prometheus Operator

This tutorial shows you how to set up monitoring in a cloud-native microservices environment using Prometheus Operator to monitor Envoy proxy. As the author explains, one reason for Envoy’s growing popularity is its emphasis on observability. Instead of using Envoy directly, the tutorial uses Ambassador.

Distributed Tracing with Zipkin, Stackdriver Trace in Kubernetes

You can use another proxy, Zipkin, with Kubernetes (and Google Container Engine) to send trace information when monitoring microservices. Learn how it works in this article.

In the Land of Microservices, the Network is the King(maker)

Networking knowledge is critical in microservices monitoring. Here’s why.

Planning Your Docker-based Microservice Stack

Here are several considerations for deploying a Docker-based microservices architecture — and some best practices for externalizing, monitoring, and managing microservices.

Monitoring & observability tools

A Curated List of Microservice Architecture Related Principles and Technologies

This article offers up a great list of tools for monitoring and debugging microservices.

Prometheus

Prometheus is an open-source service-monitoring system and time-series database.

Spigo

Spigo is a tool that simulates protocol interactions in the Go language. You can use it to run microservices architecture simulations and generate new visualizations and metrics.

Collectd

This daemon collects system and application performance metrics over time, and provides mechanisms to store the values in a variety of ways. It gathers metrics from many sources, including the operating system, applications, log files, and external devices. It can then store this information or make it available over the network.

Riemann

A tool for monitoring distributed systems, Riemann can forward any event stream to Graphite, a popular monitoring framework and time-series database. It also provides phone, SMS, and email alerts.

Heapster

Use Heapster to monitor Kubernetes container clusters. It collects and interprets various signals such as compute resource usage and lifecycle events, supporting multiple sources of data.

cAdvisor

This container-monitoring tool, developed by Google, natively supports Docker containers. The cAdvisor daemon, which collects, aggregates, processes, and exports information about containers, is available in a graphical version.

Sysdig

Sysdig is a monitoring and troubleshooting tool that captures system calls and events from the Linux kernel, and lets you to save, filter, and analyze the data from the command line or from within a desktop app. The open-source developers of Sysdig describe it as “strace + tcpdump + htop + iftop + lsof + wireshark for your entire system.”

Grafana

A metrics dashboard, Grafana integrates with many data sources, including Graphite, CloudWatch, Prometheus, Elsaticsearch, and InfluxDB.

Trace

The Trace visualized distributed tracing platform is designed for Node.js microservices.

Sensu

Sensu helps developers monitor servers, services, application health, and business KPIs. It collects and analyzes custom metrics and can send failure notifications. You can use Sensu with containers such as Docker, RKT, and LXC.

Are there any other resources that you find particularly useful in the field of microservices monitoring and observability? Share them in the comments below.