Project Dashboard — Polymorphic model in DynamoDB
How to dynamically map DynamoDB model to Java in Spring Boot.
In the first article about Azimo’s Project Dashboard, we explained how we made the Azimo app’s home screen load 10x faster by dynamically computing content in the back end.
We achieved this by using an architecture based on event sourcing and by using Amazon’s DynamoDB to store data.
In this article we will look more closely at the storing and retrieving of data from Amazon DynamoDB.
I will explain:
- The home screen aggregator service.
- Why we selected DynamoDB as storage.
- How we implemented a polymorphic model in DynamoDB.
- How we implemented the logic for dynamic mapping to Java objects (POJO’s).
- The benefits of this solution.
Service homescreen aggregator
Project Dashboard is responsible for serving smart content to our users. Various components are shown to the user on the home screen of Azimo’s mobile app. Depending on conditions, this can be a component showing active transfers, recent transfers, various banners, customer support conversations or, in the near future, required documents.
The data required to compute and display of this content is gathered from various domains.
Our microservices infrastructure relies on event sourcing and our message broker is Apache Kafka. This allowed us to build a single service (homescreen-aggregator) that consumes various Kafka topics, computes the data and places it in storage that the clients can easily consume and display. All of this happens in the background.
The results is that whenever a user opens their app, we just need one call to the underlying storage to retrieve the components that should be shown.
In the image above, the user is being shown three different components:
Customer Support — showing their open conversation with one of our agents.
Active transfer — showing a transfer in progress.
Recent transfers — showing their two most recent completed transfers.
The app creates a request to the back end for a front end service which retrieves data via REST from the homescreen aggregator. BFF then enriches the data with UI enhancements and translations and returns it to the mobile app.
No on-the-fly computation is required. This allows us to be super fast.
Choosing the right storage
While designing Project Dashboard and the homescreen aggregator we had to choose the right storage that would power the idea we had here. The following requirements had to be met:
- Fast
- Scalable
- Highly Available
- Preferably managed with good SLA
Based on the nature of our data we knew we would only need to retrieve data for one user, which allowed us to select a key-value store. We wanted something scalable and managed so we looked mainly at Google Datastore and Amazon DynamoDB. But since most of our microservices infrastructure is situated in AWS we went for DynamoDB. This brought the following benefits:
- Retrieving row(s) based on partition key is fast.
- We are able to store nested documents in it, which was a perfect fit for our home screen components.
- It is horizontally scalable.
- It is managed, which doesn’t put more work on our already busy DevOps team.
- The SLA promises 99,99% up-time.
Database model
One requirement for the model was to be able to easily add new types of components. Each component contains different data, so we needed to be able to retrieve single components, but also retrieve all the components for one user.
Because all the UI components are aggregated per user, for the partition key we used UserId. The same id is used through our whole system to store different types of data and to group our Kafka events together. Partition key is a main look-up key in DynamoDB.
Since we wanted to group all the components for one user together for easy and fast retrieval, we took advantage of “sort key”. Based on the requirement for easy extension of components, we decided to use the component’s java class name (POJO) as a sort key. This allowed us to dynamically map a stored JSON object to the equivalent Java object.
Dynamic retrieval of data
To be able to dynamically map our component model to Java classes, we have to store it accordingly.
As explained in the previous section, we store component data as a JSON object. Using UserId as a partition key and components class name as a sort key.
This allows us to easily retrieve all of the components for a user with dynamic mapping:
Conversion of the components to classes looks like this:
Deleting and retrieving a single component is also very easy. All the methods plus the configuration of DynamoDB for Spring Boot can be found on git here. In the README in the section related to this article, you will find direct links to the code.
Benefits
As I already mentioned, the computation of components happens in the background in response to Kafka events. This allows us to serve components to clients blazingly fast. Below you can see the response time of REST requests to the homescreen aggregator. The 95th percentile rarely exceeds 20 ms.
The whole roundtrip from the mobile app through the homescreen BFF and the homescreen aggregator is around 200ms. You can read more about this in our previous article about Project Dashboard.
To sum up, the new project dashboard powering the home screen of the Azimo app is:
- Fast (the home screen loads 10x faster).
- Highly available thanks to DynamoDB and multiple instances of our services.
- Easily extendible — the dynamic polymorphic model makes adding new components simple.
- Creates a great user experience by serving smart, personalised content.
Towards financial services available to all
We’re working throughout the company to create faster, cheaper, and more available financial services all over the world, and here are some of the techniques that we’re utilizing. There’s still a long way ahead of us, and if you’d like to be part of that journey, check out our careers page.
