I’ve been following the news about istio since it’s first alpha release in 2017. I think this project has a great future, because it solves a lot of pain points in the microservice based architecture, like auth, observability, fault-injection, etc.

But the fact is, I never actually tried it. Prior to v1.0 there were too many bugs and limitations to put it into production. So, when they finally released “production-ready” version I got quite exciting. It still took me a few months to find free time, but I finally gave it a try.​​

I decided to start my experimentations with…

I recently bought 3 ODROID-HC1 devices to add a dedicated storage cluster to my home Kubernetes. I thought that it’s a good excuse to spend some time redeploying the cluster. Usually, I would’ve gone with CoreOS, since I’m a big fan of their immutable OS. Unfortunately, that is not an option if you have ARM nodes. So I had to choose between manual provisioning and Ansible. I chose Ansible.

I knew about Kubespray, but still, I decided to spend some time looking for alternatives. I found a few other projects: some targeted towards ARMs, some with a “simplified” approach. In…

In this part of the series, I’m going to add Sentry to the monitoring stack. For those who do not know, Sentry is an open-source cross-platform crash reporting and aggregation platform written in Python. I know about it for several years now, since the time it supported only a few languages. A lot has changed since my last use of it: new domain, new languages, new integrations. Today it has integrations with most of the modern languages and frameworks like Elixir, React, React-Native, Go. Not to mention olds like Java, PHP or Python. …

Generation of sitemaps is usually not a problem, at least not until you have several millions of items in the database to iterate through. Currently, I have slightly more than 4M documents, so it takes time to regenerate. Previously all my sitemaps were stored on the disk and were served using simple Nginx container. It worked like this:

• Sitemap-related containers have nodeSelector to assign to particular node with hostPath volume
• sitemaps are served from /data/sitemaps folder
• periodic task generates new sitemaps to /data/sitemaps-temp
• on completion, folders is atomically switched

It was fine in my previous VM-based cluster, which was on…

There are dozens of ready dashboards available on grafana.net/dashboards and Github. Many of them just work, but many do not. In most cases, you just need to fix template variables, but some require deeper involvement. For example dashboard for PostgreSQL monitoring. After import, it welcomes you with the error message “Datasource named ${DS_PROMETHEUS} was not found”. Setting the correct datasource name in dashboard settings does not help because it has an error in __inputs declaration. The easiest way to fix it is to edit JSON before import.

It takes time to find useful dashboards, make them work for you and…

This post is one of a series of posts about monitoring of infrastructure and services. Other posts in the series:

0. Intro

1.Deploying Prometheus and Grafana to Kubernetes

2. Creating the first dashboard in Grafana (this article)

3. 10 most useful Grafana dashboards to monitor Kubernetes and services

4. Configuring alerts in Prometheus and Grafana

5. Collecting errors from production using Sentry

6. Making sense of logs with ELK stack

7. Replacing commercial APM monitoring

8. SLA, SLO, SLI and other useful abstractions

Having grafana.net/dashboards with dozens of dashboards to choose from is awesome and helps get started with monitoring…

I’ve been keeping my eye on Prometheus for some time. What I like about it — is its pull nature and great integration with service discovery and Kubernetes.

With pull system, you do not have to tell every service you have where the monitoring system is located. You just need to expose your metrics as a plain text using HTTP and Prometheus will access this endpoint and pull the data. The system will never fail because your monitoring service ran out of space or overloaded.

Among other advantages, for me, are: it is written in Go, easy to deploy and…

This post is one of a series of posts about monitoring of infrastructure and services. Other posts in the series:

0. Intro (this article)

1.Deploying Prometheus and Grafana to Kubernetes

2. Creating the first dashboard in Grafana

3. 10 most useful Grafana dashboards to monitor Kubernetes and services

4. Configuring alerts in Prometheus and Grafana

5. Collecting errors from production using Sentry

6. Making sense of logs with ELK stack

7. Replacing commercial APM monitoring

8. SLA, SLO, SLI and other useful abstractions

Monitoring of the infrastructure is an essential part of any product. But it’s not uncommon for companies…