The sweetest fruit salad recipe for energy-saving services
Chapter 0. Launching an energy-saving rocket to the stars
Welcome to the first chapter of a series of articles describing the decisions and challenges we faced (suffered and enjoyed) during our path towards the delivery of a brand-new service.
In this new challenge, we explored modern and mature technologies in the market to serve as productive examples which would allow us to define our standards for future services. To be the seed for our service development template.
The main goal is to end up with a service made by the best of the best in every aspect. Notice there is clearly a subjective approach to this. What is the best? Does best really exist? If so, how long will it be the best? In our sector maybe months.
At DEXMA we are not designing rockets, but this is how we felt during the development process. We are focused on energy management and efficiency improvement and everything started with the mission to build a new service to handle, enrich and deliver messages from unknown and diverse IoT devices to our ingestion platform
As an initial brainstorming of our preferences, we could summarize our technical desires as follows:
An asynchronous service with outbound pressure control built with Kotlin, Spring, Webflux, Caffeine, and Cucumber, running on Kubernetes
How can a single line give you so many headaches and joys? Let me tell you the whole story:
Define the Goals and Challenges
As I explained before, the team’s goal was getting the best of modern technologies by choosing the most suitable for each of the needs we faced. This is our stack selection:
Technical Summary
- Kotlin and Spring
- WebFlux for asynchronous communication:
- Fire & Forget pattern → Receive the request, return an acknowledgement to the client (releasing the connection) and process it in the background
- Outbound pressure control at the client level to avoid overflowing the consumed services
- Caffeine as our client-side cache
- Cucumber + Junit + Mockito for testing framework
- Docker + Kubernetes
- Maven for managing dependencies
- Swagger for service documentation
- Postman for functional test Suite and Newman for load/stress test (both published to corporate’s postman team workspace)
You might agree or disagree that these guys compose a good stack, it was in early 2020. Nowadays, we might have changed some components or even chosen a completely different stack. We could enter a long and wide discussion about it, which is not the scope of this post, so let’s focus on the main goals for the new service:
- Asynchronous: Avoid blocking the consumer while the request is being processed. This design pattern was called Fire&Forget during SOA times 😉 Going down to technical details, we want it returning an HTTP 202 (Accepted) response as soon as the request is received (and syntactically validated) and let the service process it in the background.
- Outbound pressure control: Protect the consumed services based on their SLA to avoid overflowing them
- Retry mechanism: Properly react when the consumed services are unavailable or have issues (either retryable or non-retryable errors).
- HealthCheck: Provide a complete health check summary and monitoring of the service.
- Profiling: Provide necessary elements to supervise the internals of our components.
- Caching: Avoid unnecessary and repeated calls.
- Human-friendly software lifecycle: Define the required behaviour and seamlessly develop the code according to it (BDD will help us with this)
- Discoverable/Testable: Facilitate the understanding of service functionalities and easily test each of them
We are developers, so documentation is not in our veins :-D I will explain each of the above features in a concise and separate article…
The trip continues in Chapter 1: Asynchronous
