Outbox Pattern Implementation on Couchbase DB
Hi everyone, In this article, we introduce how to implement an outbox pattern to couchbase and deliver your messages more reliable.
Outbox pattern is mostly applied when our system generates changes by its domain and we use it when our changes require a delivery guarantee for external systems, for example (messaging systems, or async callbacks).
Implementation in Couchbase
After a piece of quick information about the goal of the outbox pattern. We are starting to inform about the infrastructure that powers the solution and the solution and the parts that make up the solution. Finally, we will complete by giving the pros and cons.
Couchbase Eventing
Eventing is a service of Couchbase like other services to data or indexing, and it’s responsible for tracking document changes. Our solution is empowered by the Eventing Service. It captures changes in our business model and saves them to the outbox bucket.
The business bucket is to store your information, it should also store the event of that change if any change occurs because most NoSQL systems do not have acid processing but document level changes bring consistency and this is a simple transaction boundary.
For example, Our business bucket is storing stock data and it has stock quantity and also includes events if any changes have occurred.
When the business model is successfully updated and the changes in the event are created, the eventing service catches this change and records it to the outbox bucket with success.
This process is managed by the eventing service and, its eventing function is a collection of JavaScript functions together react to a class of events. This javascript function was created by us for our business needs.
The function required for our solution is as follows:
The function is running after our business document is changed and mutation is successfully created. The twelfth line of code is referring to your outbox bucket where the events are stored. The seventh line of code is referring to our business document events that are created after changed occurred.
Other service-specific terminologies are available in this document.
After, we create and store our changes now require to publish or notify required systems. Our challenge was the need to publish to Kafka, now that we have fully saved the changes.
Couchbase Kafka Connector
Couchbase, have several connectors to send changes to other systems. For example, Kafka, Elasticsearch, etc. Kafka connector is the second part of our outbox solution. It’s delivering our stored messages with at least one delivery to Kafka. The connector is sporting extensibility and you also modify your delivery requirements, more detail and a setup guide are here.
Observability
Observability is also the main requirement of this solution because if message delivery time is increased or any long-running incident is not handled in the system tolerated time, consistency issues are occurring and invalid decisions are created by message consuming systems. Couchbase Server comes with Prometheus exporter, so it’s exposed system metrics and you are able to create your alerts. For example, outbox bucket filling up alert, and eventing processing error threshold alert are some examples of server-side alerts.
The Kafka Connector also has a rest API for its application runtime alert for example you create a stream queue alert or Kafka publishes rate alerts are some examples of connector side alerts.
Pros and Cons
This architecture had some pros and cons when we started building it, so we worked on the cons and provided more realistic observability and performance contributions to our SLA metrics.
Example of Pros
- We prevented message loss and increased the system consistency rate.
- Business application is independent of message infrastructure and the availability rate is increased.
- We have more control over application messages, so back pressure is more controllable.
Example of Cons
- Couchbase Kafka Connector doesn’t expose much more metrics for our SLA requirements, so we forked the connector project and implement our observability stack components.
Conclusion
Modern microservice architectures most of the time focus on stability and availability, but consistency is another part of reliable systems. In this article, we have described one of the ways, when we use the couchbase our database system and ensure consistency in our microservice systems.
Thank you for reading.
