We ran a poll on Twitter

“Do you care about the quality of your infrastructure code?”

And on Reddit

Wikipedia defines Root Cause Analysis (RCA) as “a method of problem-solving used for identifying the root causes of faults or problems.”

Essentially, root cause analysis means to dive deeper into an issue to find what caused a non-conformance. What’s important to understand here is that Root Cause Analysis does not mean just looking at superficial causes of a problem. Rather, it means finding the highest-level cause- the thing that started a chain of cause-effect reactions and ultimately led to the issue at hand.

Root cause analysis methodology is widely used in IT operations, telecommunications, healthcare industry, etc. …

In an earlier Post, I talked about how SLOs can be misleading, and the Service Level Indicator in consideration was Uptime. There is another SLI which is almost impossible to be accurate about, Latency.

Like Uptime is measured as % and aggregated over a month/year/week, based on time window choice, Latency is for a unit of time (ms and s.), and the preferred aggregate is percentile.

The purpose of this post is to debunk common mistakes that I did while dealing with Percentiles.

Why is it important to understand percentiles in depth? Because one of the critical Indicators of software…

SLO is an acronym for Service Level Objective. But before I explain SLO, you need one more acronym SLI (Service Level Indicator)

An SLI is a quantitative measurement of a (and not the) quality of a Service. It may be unique to each use-case, but there are certain standard qualities of services that practitioners tend to follow.

• Availability The amount of time that a service was available to respond to a request. Referred to as Uptime
• Speed How fast does a service responds to a request. Referred to as Latency
• Correctness Response alone isn’t good enough. It also matters whether…

Engineering teams like to move fast. Multiple products with a variety of projects, deployments happen almost every day. Ensuring that the chaos does not take over sanity and lead to multiple failures requires carefully thought out processes. And while most of the teams have processes in place — right from the dev stage to the production stage — ensuring that the processes are followed becomes a challenge.

Site Reliability Engineering is the new fad. It’s not the Docker that you don’t need, it’s not the Kubernetes that you don’t need. It’s also not the Blockchain that you don’t need. Or well, maybe it is.

Question on running operations at scale, like the manpower or cost involved. Or what is the cost associated with each 9 in the Five Nines are being talked about more often. What does it take to run things smoothly.

Here are a few pillars, or keys to success, of running things reliably:

1. Have no silos of information
2. Measure everything
3. Culture, not tools
4. Accidents…

I was recently writing an application in Golang which required some Database interaction. The db library I was using had inbuilt Pooling so I didn’t have to bother about connection recycling and reusing, as long as I could initialise a DbPool and continue to call DbPool.new(). Having a module level Singleton object of DbPool would do this trick. However the problem with Singletons is that, in a multi threaded environment, the initialisation must be protected to prevent re-initialisation. I will discuss a few common ways to achieve this, along-with the shortcomings of each approach.

Module init()

Most common approach I have come…

Recently, I have been trying to bring up virtual machines in Microsoft Azure but ran into this interesting & annoying problem of not being able to upload SSH keys via the terraform DSL. There is a provision to provide a ssh_key_thumbprint but sadly no way to upload what you would call a KeyPair in AWS jargon.

While terraform does not support this operation via its DSL, It is possible to achieve this using some less-explored features of terraform.

Solution

I am using OS X, so my code samples might include some OS X specific commands. …