106 Followers
·
Follow

Image for post
Image for post
Go + GCP = 🚀

Distributed State of Mind

When creating a distributed system, photo handling becomes less trivial. At the first signs of scale it can become a real pain point in the application. Why?

  • Mounting multiple running nodes to the same persistent storage is impossible — for good reason. If it isn’t with some providers/stores, you will have to worry about data corruption during writes.
  • Storing and serving (bill|mill|trill)ions of images on a disk and serving them yourself isn’t fun and is error prone. You also probably won’t be able to serve it as quickly as Google.
  • Image manipulation (resizing, cropping, auto-orientation) is resource intensive. In a distributed system, you want your nodes to have maximum capacity for throughput. …


Over the past few weeks I’ve been working extensively with all three technologies. Here are some of my notes:

  • Terraform’s kubernetes provider is great for initializing the k8s cluster (cloud agnostically), but that’s about as far as you can reasonably go with it. In other words, provision with Terraform, orchestrate app with K8s via kubectl and yaml files.
  • An auto-generating/renewing certbot flow is painful. No great solutions AFIK. +1 for AWS handling certs for users out-of-the-box — would love to see GCP offer something similar for it’s L7 load balancer.
  • The non-immutable parts: Configuring stateful application volumes and secrets within k8s and terraform is not trivial. Especially, GCP’s awesome managed Postgres instance. Due to this, using a standard postgres/deployment/service configuration is much more straight forward but fully managed would be preferable for critical web apps. …


Image for post
Image for post
A deep metaphor describing cloud-based resources.

Ephemeral cloud-based functions like AWS Lambda and GCE Functions are gaining traction quickly. If done right, they can be economically beneficial and enable infinite scalability™ without forcing you to wade into swarms of containers. If applied incorrectly, they can result in a much more expensive, over-engineered, and difficult to manage application. It’s important to understand that it isn’t all or nothing. Serverless is in our future, but it isn’t our exclusive future.

Functions as a service (FaaS) are functions configured in a cloud provider with a single entry point which lay dormant until triggered. Planning for FaaS is similar to the way you would approach learning about immutable objects, functional programming paradigms, or migrating from monolithic to distributed architectures. …

About

Pete Saia

Engineer & Co-Founder @ WorkTaps

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store