Q&A — In the Cloud: Serverless on GCP
Hey everyone, Harizuan and Weiyuan here. It was great meeting all of you last week, in the online event organised by Google Developers Space.
There were several questions asked during the livestream, that we were not able to answer due to the time constraints. As such, we would like to answer those questions that we could not get to answer during this event in this blogpost.
Before jumping into the Q&A, let’s recap on the actual livestream itself:
Q&A
Q1: I have an App Engine — Nodejs application that uses the Firebase Auth library for authentication. To secure my Firebase API key, I store it in Secrets Manager and retrieve it as I start the app. What do you think of this approach?
Harizuan
This approach does sound pretty decent. This approach is already way better approach as compared to past approaches (e.g. embedded the encrypted version of the secret within application code base)
For people unfamiliar with secrets manager, you might want to check the following content:
Weiyuan
Secret Manager is awesome! I really love the fact that versioning for secrets and integrating with modern-day applications is seamless when using this tool.
Check out the following library for interacting with Secrets Manager in Nodejs, https://github.com/googleapis/nodejs-secret-manager. Using this library, you can also opt to retrieve your secrets when you need it or update it with a cron job. This will allow you to rotate secrets easily, without the need to restart your application.
Q2: What would be the best learning path to learn serverless with GCP, if I am completely new to writings apps to run in the cloud?
Hairizuan
If you need Google Cloud Platform (GCP) to try out — you could try and see if you can get Qwiklab credits from upcoming events? We’ll continue advertising such events in the future once we hear about them.
Either that, or you could go for cloud functions where if you use languages like Python, Nodejs. This would allow you to focus on specific on application development.
Building applications on the serverless options on GCP usually tend to be on the cheaper side of things. Cloud Functions and Cloud Run have a pretty generous free tier which can use; if you’re not exactly having production level traffics, it is pretty hard to use it all up. Personally, I’ve been using Cloud Functions for some of my little side projects on the side and the bill amounts to under 5 cents per month. The function generally runs for like a minute per day at most; the bill is mostly for storing data in GCS, Google Cloud datastore
Generally, I would recommend cloud run at this stage but that would require Docker as pre-requisite knowledge.
Weiyuan
Use Qwiklabs! → https://google.qwiklabs.com/ . It's an awesome platform for learning all about Google Cloud Platform, not just serverless. Time to time, meetup groups, Qwiklabs (the provider), and Google will release codes that you can use to use Qwiklabs for free.
Check out the following quests in Qwiklabs for Serverless concepts in GCP:
Cloud Run: https://www.qwiklabs.com/quests/98
Deploying App Engine and Cloud Functions: https://www.qwiklabs.com/quests/26
Previously, as a student, I also benefited a lot from using GCP’s free tier for learning and experimentation. The smallest configuration on App Engine is available for no cost, up to 28 instance hours a day (essentially free!). Cloud Run and Cloud Functions also have high free quota per day, making it really attractive. See here for more details: https://cloud.google.com/free
Q3: Not sure if cloud function now supporting Java?
Hairizuan
Yes. Cloud Function has supported Java for quite some time. Recently, Java 11 is also being supported; see the announcement here: https://cloud.google.com/blog/products/application-development/introducing-java-11-on-google-cloud-functions
Q4: With monoliths, you have one thing to take care of. With microservices, you may have hundreds of things to manage, with cloud functions you may have many thousands. How would you manage such a beast?
Hairizuan
Handling all those code and deployments just mean that one/the company involved need to automate as much as possible. Each service going to production should ideally go through the deployment process (e.g. deployment review, load etc). The more of development that you can automate out and move away — the easier it would be in order to attempt to control such a situation (and yes, it’s also important to look at the pitfalls of such approaches)
You can automate it to the point where on a new service being made available — you can have scripts to create the required resources on the GCP project. E.g. Cloud Pubsub publishers and subscribers? GCS Buckets?
It comes with other nice benefits of being repeatable (if done well) — in the case where you lost access to the project etc, you can repeat this to a new GCS project)
And with more automation needed — you will need to ensure your CI/CD deployment pipelines can take it. Some people just assume one can use Jenkins to do this work, but they forgot to realize that using another workload to manage your workloads would mean now, you have to make sure you can scale it in order to manage it. Plenty of effort is needed to make sure that this doesn’t go down — so that is another thing you need to consider. My personal opinion is that if you can afford to use a managed build system — use it. The amount of effort it takes to make sure the build system stays up is quite expensive (https://www.youtube.com/watch?v=zS-MAVMjfQI)
It’s the same with logging and monitoring systems; if you can afford to utilize managed services, use it. These systems are extremely critical to understand applications. If such systems ever go down, you can surely guess that it’s gonna be an extremely painful affair.
Let’s say you managed to do all this and you’re deploying applications at very high rates. You can lead to an issue where a new application update can break a whole chunk of the functionality of the product. Now, it boils down to good computing practices to make sure developers are not delivering things that would results in issues in the downstream application. In the microservice world — applications become dependent on one another — it becomes extremely important to be the gracious neighbour — make sure you’re not the dude that keep changing api changes, forcing downstream applications to change it with you (this is a tiring game to play — like a never-ending process of cat and mouse)
One way in order to attempt to avoid this would be to do up API contracts between components: https://microservices.io/patterns/testing/service-integration-contract-test.html
The services should be tested to make sure that it doesn’t break on the API layer.
TLDR version:
- Automate as much as possible
- Use managed services as much as possible
- Build system — Google Cloud Build
- Artifact Delivery — Google Cloud Registry, Google Cloud Storage
- Logging/Monitoring — Stackdriver
- Automated resource management (Ansible, Terraform, Chef)
- Don’t break interfaces with other applications (Be gracious)
- Try to have a common way to do retries and ensure applications have proper degraded ways to respond to requests. Bad retry policies can lead to thundering herd situation (a retry that keeps retrying forever result in the number of requests to increase infinitely within the system)
Weiyuan
I would put it this way — while maintaining 1000 unique microservices sounds like a daunting task, having to debug a single error in a monolith that is the combination of these 1000 microservices is no simple task as well.
Let’s simulate an error occurring across both of these configurations. We will assume that the basic telemetry of metrics and logs, and alerting, are integrated for both configurations.
For the monolith, there might be lesser metrics and logs to observe, in addition to them being categorized together for different functionalities. This can make debugging or fixing a bug much harder than a microservice. For a microservice, if an alert is triggered, it is very easy to pinpoint factors leading to the error in metrics and logs that reflects its own status.
The crucial part is the initial setup — every new microservice should be integrated with the proper tools — for metrics dashboard, alerting, and logs. If not, it will be hard to manage each service individually.
Automation is also important when you reach the scale of 100s and 1000s of services, to keep things manageable. Look into self-healing for microservices which can explain more on this.
That said, my personal opinion is that you should look into if you have enough manpower to support microservices throughout. Microservices is just one of the solutions out there, like a monolith and Service-Oriented Architecture (SOA). There is no solution that is more “correct” than the other, only the solution that is the most suitable for a team or organization.
Q5: So is it fair to say — one doesn’t need to use Google AppEngine anymore?
Hairizuan
Yes and No. If you have already been used to app engine, e.g. your app in build upon app engine, it wouldn’t make sense to rewrite such apps yet to utilize other serverless tools (e.g. cloud run). As of now, it seems unlikely that Google would stop supporting AppEngine — even if they do, it’ll take a while before any such features are ever depreciated. In one of my previous roles, I got to see the whole depreciation process — it takes a really long time for the depreciation of critical features. Initially, GCP would blast users of the feature with continuous notifications warning about depreciation. The depreciation would start appearing in the logs. Eventually, they start announcements of reduced availability of the service (which would then kind of force people to move away — notifications are not that strong of a factor). A reduced availability from 99.9% to 90% is apparently very very obvious — any engineer seeing the sudden spike in latencies/failed requests would freak out and begin to refactor/rework his application accordingly.
It would be said that AppEngine is just another choice in the GCP world. There are still features in Appengine that appeal to some groups of appeal (e.g. centralized dashboard that holds meta information of the application.) Control of application deployments via a central YAML file etc. Ease of deployment. Lack of need to deal with docker (for some, docker is actually a turn-off -> I can’t explain this point seeing that I kind of grew up with Docker in my computing career)
Weiyuan
In some sense, App Engine Flex — Custom Runtime can be directly replaced by Cloud Run. Both tools support a Dockerfile as the base configuration for the application.
When comparing the two tools, we can immediately see one obvious difference: Cloud Run’s instances are based on gVisor, a container sandbox runtime, and built with Knative, while App Engine Flexible’s instances started as Compute Engine’s VMs. This translates into Cloud Run generally being able to deploy and scale faster than the App Engine Flex counterpart. See here for a side by side comparison: https://youtu.be/j6da-Kq8TYU
However, the above only applies to App Engine Flex — Custom Runtime. If you’re simply looking to deploy an application (e.g. a Nodejs application) and not be concerned with maintaining dependencies in a Dockerfile, App Engine Standard is going to be your answer.
Another benefit for App Engine is the numerous use cases and support in GCP built up over the years. For example, being able to interface with a managed Redis store with MemoryStore. That said, Cloud Run is looking at catching up.
Q6: Does it make sense to go with microservices in a non-Cloud world? The orchestration, monitoring, testing, etc might be much tougher if it is not in Cloud environment? Or are there ways around it?
Hairizuan
It kind of boils down to company
Actually, I would worry if you’re not doing monitoring and testing -> these are needed even if you’re on the “monolith” — just that testing it is slightly harder.
Monitoring is not a “microservice” thing — it’s needed for you to kind of reason about production workloads. On the cloud, certain metrics may pose less importances as some of it becomes the cloud vendor’s responsibility to take care of it. Most of the time, you can kind of assume that “things just work” — e.g. IO throughput to your hardware devices, network throughput — the cloud vendors have pretty stable infrastructure to guarantee certain numbers. Unfortunately, in the non-cloud world — these numbers are extremely critical to the ones that maintain the infrastructure/applications. You can have your “noisy neighbour” effect being way more pronounced — and this noisy neighbour can easily impact the whole “datacentre” worth of infrastructure and applications.
Testing is kind of a minimum of minimum — it provides a level of safety of developers. This would probably a wrong place to debate on whether testing is necessary for applications but instead, I’ll refer you to watch the following video — although its from a Golang talk, it can be applied broadly to the development process. https://www.youtube.com/watch?v=a6oP24CSdUg
Before going deeper into orchestration, it might be good to look into the definition of it — https://en.wikipedia.org/wiki/Orchestration_(computing) — so, if you’re not orchestrating your systems, that would mean that you’re doing things manually — naturally, microservices would be a really hard game to handle. For each application that is to deployed to production, it should ideally go through all development pipeline processes. If you have 2 or even more, and each of them requiring to go through the pipeline -> you can imagine the pain and stress the people managing the applications/deployment would go through, it would want to make one stop all deployments. Which would naturally reduce frequency of release of services. And this leads to a completely different outcome.
Weiyuan
Each organization has its own requirements and needs. A one-man developer in a startup may resort to building the product as a monolith for ease of development and maintenance, while a larger organization can turn to services or microservices where the level of granularity is properly matched with the different teams. Technology and tools (i.e. the cloud provider) can be a contributing factor on whether to go with microservices, but should not be the deciding factor.
As with microservices in a non-cloud environment, the same concerns for deploying and monitoring to a non-microservice model apply as well. For example, you can continue to use the same monitoring tools when you transit from a monolith, as the core principles still apply.
However, the value of using cloud providers is to make it easier for developers like you and me to implement our ideas. You can utilize GCP to allow your teams to manage your applications with greater ease, microservices or not, by using the various managed tools.
Q7: What stack should we prepare to be able to implement microservices? is Docker or Kubernetes is a must to learn?
Hairizuan
For microservice — docker and kubernetes are optional (dependent on companies). Potentially, one can deploy each microservice in VMs
Microservices is a concept and is not tied to specific technologies. Even the definition of microservices is very vague — any strict definition would generally not make sense in the real world (e.g. one service should strictly have one endpoint -> then you will have an explosion of services. One service should only be a page long -> then you’ll have an explosion of services etc)
You can easily have a situation where you have a whole bunch of microservices but the services are each in their own VM (random note — node you even have micro-VMs, so what exactly is VM anyway? Everything is not exactly in a black and white situation)
But if you’re talking about the generic sense of things then yes; many companies that are entering microservices territory might decide that Kubernetes is a good platform to work off from. One advice is to check the company directly; in some cases, some of the developers of the companies would give public talks, sharing about the tech stack that their company utilizes (e.g. self promotion of Acronis — the DevOps lead mentioned that the company switched to k8s, a very obvious sign — https://www.youtube.com/watch?v=P6ECIj21JX0 )
Weiyuan
Similar to application development, there is some liberty on what you want to use to implement microservices. However, in my opinion, Docker is an amazing and easy choice for the deployment of your applications.
If your organization doesn’t have the manpower to manage their own machine clusters (for example, via Kubernetes), try going serverless! You can go as lean as using Dockerfiles in Cloud Run and App Engine Flex, to even not having to use Docker in App Engine Standard and Cloud Function (entirely application code!).
Q8: While transitting towards a micro-services model, from a full monolith, how much “DevOps” (e.g. Cloud Formation/Kubernetes in GCP) did you have to pick up and where would you start?
Weiyuan
For migrating from a monolith to a model like microservices, it does, yet does not require “DevOps” knowledge at the same time.
At its core, a monolith is at the level of “product” for the business (the entire product), SOA is more of “enterprise” (singular categories within the product), and a microservice is scoped down to a “single specialization”. If an application is rewritten from one to the other, that’s more on the application code, where DevOps expertise is not directly required.
However, the crucial part of any service architecture, in terms of DevOps, is figuring out the pieces of the puzzle to solve — service deployment, telemetry, and proper maintenance guidelines (e.g. documentation, playbooks and auto-healing for dealing with system failures)
On the end of my “DevOps journey”, it is a continuous learning journey for me. I would recommend Qwiklabs very heavily, as it provides a sandbox environment for hands-on learning, unlike just reading plainly from the documentation.
We have provided out answers as per the questions above. If you have any other questions, do add them in the comment section as well!
Ciao~
