Relax and put your seat belt on… You’re about to enter my journey of an intense 2 weeks sprint I recently led, swapping my DevOps hat for a front-end one to build a full JAMStack website from the ground up deployed on AWS!
Foreword & Context
The GumGum Verity team applies machine learning techniques to the text and imagery on web pages to identify the context of the advertising inventory and brand safety threats. It then delivers content-level data over a Rest API that help publishers, DSPs and advertisers place ads in safe and relevant contexts.
This API not only supports page level…
What is Terraform ?
Terraform is a powerful software from Hashicorp that allows you to write infrastructure as code in a declarative way to provision and manage any cloud, infrastructure, or service.
The terraform plan command (similar to git diff output) is used to create an execution plan. Terraform performs a refresh, and then determines what actions are necessary to achieve the desired state specified in the source code.
Kafka is a really powerful distributed publish / subscribe software that helps you build complex asynchronous applications. GumGum was an early adopter of this technology and is nowadays running hundreds of brokers across multiple clusters.
Kafka monitoring is a must
Kafka cluster operations are a thing on their own (scaling clusters in and out, recovering from a dead broker, reassigning partitions across the cluster…) but if you want to build performant client applications based on Kafka you want to pay close attention to your consumers:
- Are they able to keep up with the incoming traffic (consumer lag) ?
Kafka Consumer Lag is an indicator of…
Florian Dambrine
Principal Engineer & DevOps enthusiast, building modern distributed Machine-Learning eco-systems @scale
