This post describes how we improved the query performance for our OpenTSDB cluster and enabled queries that previously were impossible by reducing the resolution of historic data.

Skyscanner’s focus is to drive every decision in Skyscanner by complete, timely and accurate data. As part of this, we’re operating a large metrics and logging platform that enables all engineers in Skyscanner to monitor their service 24 hours a day. We provide application logs and any metrics that our engineers would like to record; for instance, business and operational metrics for their services. We store and serve the data that powers Grafana dashboards and our alerting infrastructure which comprises Bosun and VictorOps. …

Introduction

Skyscanner’s products are powered by hundreds of services hosted on AWS. In order to deploy changes and new services to production with zero clicks, we have an automated pipeline that´s responsible for building, testing and deploying new code, and provisioning and configuring new infrastructure. Developers perform these changes by writing CloudFormation templates that model their service’s Infrastructure as Code (IaC).

CloudFormation is an AWS service that essentially allows developers to programmatically provision AWS resources. CloudFormation templates could have security issues similar to regular source code, such as hardcoded secrets, overly permissive permissions, and many more.

Our goal at the Security Automation team is to inject security into the pipeline as early as possible, and to make sure that the relevant security scans and audits are performed at every step, minimising the risk of any vulnerable code getting into production. …

Cristiano Balducci is one of our squad leaders. A problem-solver at heart, Cristiano shares valuable insights on how you can identify issues and achieve your goals as a team. Read this short blog to discover his top tips on how to achieve some of the most impressive sprint goals. If you want to find out about how we troubleshooting our process and what we have learnt doing this click ​here​.

How do we choose impactful sprint goals?

Here at Skyscanner we are arranged into squads and tribes; I’m a squad lead, and my squad — like many of the other engineering squads at the company — uses an ‘agile’ methodology called ‘Scrum’ for the purposes of effective software development. …

Two years ago in Skyscanner we made the decision to start moving our workloads to Kubernetes. Today, even though this transition is far from complete, our infrastructure is using >2000 nodes spread across 30 different clusters to power a fleet of >160 services.

As part of the transition to Kubernetes, the Security team had to come up with a way to detect malicious activity in Skyscanner’s Kubernetes clusters. Due to the sheer size of the target we want to monitor, the solution we chose needed to be able to scale as much as our most demanding services, without hindering their performance while allowing us to be immediately alerted if one of the machines in any cluster is compromised. Also, not only do we need the ability to scale to all the Kubernetes clusters, we also need an automated way of mapping and contacting the owners of the affected services if anything were to happen. …

In June I (re)joined Skyscanner as a thirty-plus year-old graduate software engineer. How did this come about? The answer to this question has two parts…

Part one — my not-so-uncommon story of career change

A few years ago, I decided to pivot from an almost ten-year career in marketing and address a question that was echoing more and more loudly in my head: would I enjoy working as a software engineer more? So I decided to woman-up, quit my job, and spend a couple of months on full-time home learning — and sooner than I ever dreamed possible, I got my first job as a web developer.

I remember being anxious at first about how awkward and difficult it would be to explain to others how and why I made this change. Yet to my very pleasant surprise, the more I spoke about it, the more I understood how common this story is becoming nowadays. Our professional careers are now more fluid and flexible than even before — and it’s no wonder: we live in a day and age where high quality learning resources are very accessible — and either free, or very inexpensive. Changing career is no longer such an uncommon story, and it’s becoming a choice bravely made by more and more people. …

Over the last few weeks Skyscanner has been looking for some of the best talent to join its engineering team. In this video, our Chief Technology Officer, George Goodyer talks about some of reasons he works at Skyscanner and what inspires the team day to day.

Skyscanner is growing and our Engineers don’t just love working here — they love telling people about it. Over the next few months some of our most passionate engineers will be writing to showcase our London hotbed of software engineering talent, and how we’re impacting millions of travellers, all over the world.

Introduction

In my recent blog post, Finding the steps on the individual contributor ladder, I wrote about the problems that exist in the industry around defining and finding senior individual contributor (IC) roles like that of principal engineer (my own role). As I observed in my blog post, you can tell if a company is committed to helping their individual contributors grow by asking if they have job specs prepared for IC roles — if they can provide you with a spec it is a good indicator that they can and will support individual contributors. …

Skyscanner is growing and our Engineers don’t just love working here — they love telling people about it. Over the next few months some of our most passionate engineers will be writing to showcase our London hotbed of software engineering talent, and how we’re impacting millions of travellers, all over the world. Nicky Wrightson has also written a follow-up to this blog on what it’s like to be a principal engineer at Skyscanner, which you can find right here.

Introduction

I rose through the ranks to become principal engineer at the Financial Times. Since leaving there I have had two principal engineer roles, with the second of those taking me to where I am at the moment: Skyscanner. During this journey of promotion and seeking a new role I have had to get what a principal engineer actually is really clear in my mind. I didn’t get it right initially, and found myself in a role at a company that was not yet ready for a principal, so having me was not good for them — or for me. I found a few useful resources which helped me to understand what it means to be a principal engineer, but it’s fair to say that there’s not much information out there. So I thought I would pull together my experiences, learnings and questions and write something to help people decide if the individual contributor (IC) route is for them. …