Image for post
Image for post
A Command-and-Control Center image taken from www.appliedglobal.com

I work as an SRE at iZettle and, at times, during my on-call it can be challenging to understand the contributing factors of incidents when they occur or the nature of failures as they happen. As someone running a Microservices Architecture, it’s not an easy task setting up some decent level of observability into the system to quickly understand what’s going on.

In this article I’d like to share my experiences on how one can improve the observability of services through the use of the Consul Connect Service Mesh and later I’ll explain my justification for using it compared to other alternatives. Just for a recap, a service mesh allows you to route HTTP traffic through proxies that are deployed as sidecar containers to a service. These proxies can then expose metrics, logs and traces that allow you to gain rich visibility into the system thus making it more efficient to debug applications quickly and easily. Consul Connect, in addition provides service-to-service authorization and encryption using mutual TLS and uses Envoy as the data plane for the data transmission. …


Image for post
Image for post
Beautiful image of logs taken from keremeosreview.com

TL;DR
Logspout is a great tool for log routing where as Fluentd is a great tool for log processing. However, if you’re using docker’s built-in Fluentd Log Driver, you might have come across a few limitations in the docker-ce edition such as the log splitting issue and being disallowed from viewing your logs locally via docker logs. This article explains how to combine the strengths of Logspout and also route your logs directly into Fluentd using a custom built Logspout-Fluentd module thus surpassing these limitations and giving you much more control over your logs.

From the official documentation:

Logspout is a log router for Docker containers that runs inside Docker. It attaches to all containers on a host, then routes their logs where you want. It also has an extensible module system. …


Image for post
Image for post

TL;DR: Having HAProxy configurations reloaded almost constantly due to dynamic backends is bad and can cause unreliability in the system. However this can be avoided if backend configs are “close-to-static”. The good news is this can be achieved even if you have 100+ microservices at scale.

If you’ve been working closely with a microservices architecture or are generally interested in the topic, this post is just for you. We at Wrapp have adopted such an architecture and we host 100+ microservices over 50+ hosts. …

About

Jude Dsouza

SRE @ iZettle — A PayPal Service | Cloud Architect | DevOps Enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store