How we built a Service Skeleton instead of a framework
Table of Contents
- Introduction
- Building the Skeleton
- HTTP Endpoints
- Swagger
- Prometheus & Grafana
- Logging
- CICD Integration - Conclusion
Introduction
Custom frameworks are built and used in many organizations to make life easier for engineers. Any such framework is supposed to deliver a bundle of necessary features that we expect in every application, or as the organization demands, to make it standardized and reliable. This can range from specific logging format, using or restricting certain open source software, to enforcing versions, build tools or even deployment strategies. Frameworks (and libraries) in general come as a blackbox, and the engineer is not expected to change its internal working. Rather to simply use it by fitting the application being built into it (if it’s a framework) or import the packages/classes into the code and invoke them (if it’s a library).
While this sounds nice, more often, if the framework is not designed properly, it can get unstructured, or outdated. An example would be a custom framework which might work with Tomcat but not with Spring Boot, or something that works with Tomcat 7.x but not with Tomcat 9.x. To make things worse, if the framework keeps on adding new features without focusing on backward compatibility, or enforces new standards, then you have made a beast out of it which is no longer in control. At this point the cost for an engineer to fit her code into this is more than implementing her own minimal framework. There is a line we need to draw when building such frameworks and I often ponder on the question, how much is too much?
To begin with, a framework needs to be maintainable. We might be tempted to include a ton of features and that obviously with good intentions. However, do all my users need all the features? Am I enforcing a specific feature? What if one of the applications have a specific need which is blocked by this enforcement? Also, is it a better idea to provide a template, from which engineers can pick and choose, rather than provide a black box of one size fits all.
More often, we come across internal frameworks which force an engineer to use log4J (instead of slf4j) and spits out the logs at a specific location and format which can’t be overridden. Or we see that a core feature of the framework is using a deprecated dependency, or a shaded dependency which can’t be upgraded.
Pros of using a custom framework
- Get out of the box support without the need to understand how it works
- No need to customize and add dependencies. Just integrating the framework is enough
Cons of using a custom framework
- One size fits all means, you also pull in features that you do not need
- In case, a dependency is deprecated, you have no way to override that unless you dig in, and even then, you might not be able to fix a shaded dependency
- If the framework enforces restrictions, you are out of luck. You need to reach out to the team who is maintaining it, to get it working
- Unless it’s properly designed, maintainability would be an issue, and the more features it incorporates, the harder it gets to maintain. The ‘How much is too much?’ problem
- There is no way an engineer can exclude items she does not need, or incorporate things she needs without working with the team that supports the framework
Hence, unless there are specific needs, may a times, the cons outweigh the pros of a custom framework. When mulled upon carefully, we also observe that if building a template, the above pros become cons and vice versa.
Building the Skeleton
Faced with all these questions, we embarked on an effort to build something that is just right. One that has all the things we need, and yet engineers can pick and choose. This shouldn’t be a library or a framework but rather a template. Teams can clone this template and remove things that are not relevant to their use cases. People should feel free to use, contribute, and let other know about this template. For now, we want to keep it Inner-Sourced (if not Open-Sourced in the future).
We decided to call it the Service Skeleton and build it in the language most commonly used in the ‘Search Organization’ — Java. This is a simple collection of nifty and useful tools that integrate nicely with each other. For that purpose, we decided to use Spring Boot. The goal here is not to reinvent the wheel. Rather provide easy integration points to spin off a new micro service quickly. Spring boot seemed to be the obvious choice due to its offering of almost every thing we needed to get started.
Spring boot works easily with Rest/HTTP service with the need of zero boiler plate code. It comes packaged with logback for logging purposes, and actuator-prometheus integration for metrics exposure. It also integrates very easily with Swagger. The application runs inside a docker container, which in turn is managed and orchestrated by Kubernetes. For smoother operation, the application needs to expose logs and metrics to external systems like Splunk and Grafana, along with its own HTTP endpoints. We will go over these now.
HTTP Endpoints
Since most of the applications are micro services exposing HTTP endpoints, it gives a template of Controller-DTO-Service integration using Spring boot and sample test cases, something most commonly used in any micro services.
It is important to keep in mind that a micro service is already broken down based on domain driven design. There should be no further need to have sub-domains inside the micro service. The code should simply be grouped based on its responsibility (controller/service/repository/DTO). The external facing DTOs might be worth being a separate artifact, but that is a separate discussion and beyond our scope. It’s okay to have other logical groupings as you need. However, unless there is a specific need, you should not create sub-modules in your repo.
Swagger
With HTTP endpoints, there is a need to define contracts and communicate that with teams who will be using this service. That means documenting these on Confluence, Github or some other location. Over time, these docs seem to get out of sync and even fall through the cracks. It would be nice if the documentation is integrated right with the code and there is also an easy way to test those out. Enter Swagger for contract documentation.
Prometheus and Grafana
It is important to understand the traffic volume, latency, CPU/memory usage, scalability, throughput and availability among other things to ensure the application is operational. Our devOps teams have Grafana infrastructure which expect applications to provide these metrics which can be displayed as graphs. The common and standard format supported is that of Prometheus. The Service Skeleton exposes metrics in Prometheus format and provides examples to implement custom metrics.
Logging
Another operational aspect is to trouble shoot issues and go back in time to see what happened. For this we need logging. Unlike old days, we can’t have logs written to physical files on the same ‘hardware’ where the application is running. We are on the cloud and inside a container, the logs from all the instances of the container need to go to a central log management system. We use Splunk for that. The application simply writes logs to console, and the devOps team takes care of ensuring that the logs end up in Splunk. We are not going to focus on how that is done, rather, we ensure that the Service Skeleton writes the logs in the right format. We use logback, that comes bundled with Spring Boot. In this context, we are also leveraging spring profiles, where local profile generates standard single color-coded log lines for easy debugging should you need to run it on your local computer/IDE. While the deployments on Kubernetes, that generate logs for Splunk, do so in json format for easy filtering.
CICD Integration
Since all the deployments would happen on pods running on Kubernetes, we even fitted a template of the Docker file and a sample CICD pipeline for the Service Skeleton.
So what we built is not a framework or library, rather, is a collaboration of things an engineer needs in the organization to kick start a project, without digging out the integration pain-points. Just like with Spring, you don’t need to worry about parsing the HTTP Request anymore, with this, you do not need to lay down the bones. Go ahead and add the muscles, the bones will start to move, since they are already there. Exactly the reason, we call it a Service Skeleton.
Conclusion
Every time we build something new, we need to draw our boundaries. While it is tempting to add new flavors into the broth, adding too many of them may overwhelm the taste, and make your dish less appealing. It is also important to understand the use case of what we are building.
As we can see above, the Service Skeleton, while providing the tooling for integration with Swagger, Prometheus, Grafana, Logging and CICD pipeline, also gives an engineer the option to include or remove features. If someone has a specific need to use a different logger, or not use Metrics endpoint, they can do so easily.
We want to empower the engineer to quickly spin off a new service, but at the same time ensure there is enough flexibility for her to fit her needs and customize. The best thing you can do to achieve this is not to build a library or a framework, but rather a skeleton and let the engineer add muscles to it.
