We constantly strive to improve the user experience by delivering features that empower self-service. This time, we wanted to equip users with the ability to create copies (non-live) of their data environment. This allows them to test changes through simulations before applying them to the live environment.
This article dives into our implementation of versioned entities within a Spring Boot application. We’ll explore how we optimized code modifications while prioritizing a maintainable and well-understood solution. We’ll explore the implementation details, covering the database layer all the way to the frontend.
The concept
In this chapter, we’ll explore the initial approach we took for implementing entity versioning in our Spring Boot application. We aimed to minimize code changes while ensuring a maintainable solution.
Identifying Versionable Data
Our first step was to determine which data would benefit most from versioning for simulations. We analyzed our data model and identified approximately 6 out of 150 tables as relevant for this feature. We named this feature “ConfigVersion.”
Why not Hibernate Envers?
While Hibernate Envers is a powerful tool for auditing individual entity changes, it wasn’t the ideal fit for our specific needs. Envers excels at tracking field-level modifications and creates separate audit tables for each audited entity.
However, our requirement was to manage entire configuration versions as a whole, including relationships between entities. These configurations could be built and modified by users over time, requiring consistent snapshots at specific points. The user is then also able to compare two config version against each other.
Adding a Config Version ID
We opted to add a new column named config_version_id to the 6 chosen tables. This new column, along with the existing primary key identifier, would form a composite primary key for these tables.
Sample Database Schema
A quick cutout of our database schema looks like this:
The blue boxes highlight the newly added config_version_id column.
Foreign Keys and Cascade Deletes
The foreign key from the CalendarEntry table to the CleaningTour table is now a composite foreign key, referencing both cleaning_tour_id and config_version_id. However, the foreign key from the MobileDevice table to CleaningTour remains a simple foreign key using only cleaning_tour_id.We’ll discuss the handling of cascade deletes in a later chapter.
Runtime Data View
With this approach, our database can hold multiple entities with the same original ID but differentiated by their unique config_version_id values.
For instance, we can maintain the same cleaning_tour_id value (e.g., 10) in the MobileDevice table even when referencing different configuration versions. When fetching the associated CleaningTour based on the cleaning_tour_id from the MobileDevice entity, we can choose to return the CleaningTour from either the active configuration version or another version based on the user’s context.
This allows users to interact with the MobileDevice defined in the active ConfigVersion while viewing a different configuration version. Consequently, the UI for the MobileDevice view will need to be read-only. The technical details of implementing this filter for CleaningTour queries will be covered in the next chapter.
Filter implementation with Hibernate Filters
We aimed to avoid manually modifying existing queries with custom filter logic for each config-versioned entity. This approach would become cumbersome as the number of queries grows. Therefore, we opted to leverage Hibernate Filters for a more centralized and maintainable solution. The following code snippet demonstrates how we implemented a filter for the CleaningTour entity:
Architecture for Setting Config Version
The value for the filter is set via a Spring Aspect once a repository of a config-versioned entity is called. The following diagram shows the architecture of setting the config version:
We created two custom annotations:
@ConfigVersionRelatedEntity: This annotation serves as a reminder that the entity is config-versioned but has no direct impact on the code.@ConfigVersionRelatedRepository: This annotation is applied to the repository. The aspect will intercept all calls to a repository method where the repository has this annotation.
This aspect intercepts calls to repositories managing config-versioned entities. This aspect sets the relevant configVersionId for the corresponding filter value on the current Hibernate session. This ensures that all queries operating on config-versioned entities within the transaction consider the appropriate config version.
To be able to use the session object, an active transaction is required. Therefore, we annotate our repositories with @Transactional. This ensures the aspect can intercept calls and set the filter on the Hibernate session within a transactional context.
If you now call cleaningTourRepository.findByEconomicEntityId(71L);, the following query will be executed:
Hibernate: select ... from cleaning_tour cleaningto0_ where cleaningto0_.config_version_id = ? and economicen1_.id=?As expected, Hibernate automatically adds the cleaningto0_.config_version_id = ? clause to the query based on the active filter. This filtering approach also applies to custom JPQL queries you might write.
Handling Many-to-Many Relationships with Versioning
Our data model included several @ManyToMany relationships between entities. One such example is the relationship between CleaningTourUpdateGroup and CleaningTour. However, directly mapping the CleaningTour entity within the @ManyToMany relationship becomes incompatible with our versioning approach.
Challenge and Solution
Since we need to track the specific config version of each related CleaningTour, directly mapping the entity object is no longer suitable. To address this, we opted to store only the IDs of the related CleaningTour entities within the join table.
Code Before:
@ManyToMany
@JoinTable(name = "cleaning_tour_update_group_member",
joinColumns = @JoinColumn(name = "cleaning_tour_update_group_id", referencedColumnName = "id"),
inverseJoinColumns = @JoinColumn(name = "cleaning_tour_id_id", referencedColumnName = "id"))
private Set<CleaningTour> cleaningTourUpdateGroupMembers = new HashSet<>();Code After:
@ElementCollection(fetch = FetchType.EAGER)
@CollectionTable(name="cleaning_tour_update_group_member", joinColumns=@JoinColumn(name="cleaning_tour_update_group_id"))
@Column(name = "cleaning_tour_id")
private Set<Long> cleaningTourUpdateGroupMembers = new HashSet<>();Explanation of the Solution
By using @ElementCollection and @CollectionTable, we store a collection of Long values representing the IDs of the related CleaningTour entities. This approach allows us to maintain versioning information for each related entity through the config version ID within the main entity.
Fetching Additional Information
When we need to retrieve the complete CleaningTour object for a specific ID, we can leverage the injected CleaningTourRepository and filter the results based on the desired config version (which is automatically handled by the previously described filter implementation).
Obtaining the Config Version ID
The config version ID originates from the frontend application. Whenever a user wants to view a specific configuration version, the frontend dynamically transmits the desired config version ID as a header parameter with each subsequent request.
Backend Implementation
On the backend, we implemented a Spring Filter to intercept incoming requests. This filter retrieves the config version ID from the request header and stores it within a request-scoped bean named ConfigVersionContext.
Request-Scoped Bean
The @Bean annotation with scope = RequestScoped in the code snippet configures ConfigVersionContext to have a lifespan limited to the current (Web) request. This ensures the retrieved config version ID is accessible within the scope of that particular request.
The bean configuration looks like this:
Aspect Accessing ConfigVersionContext
The aspect responsible for setting the configVersionId in filters retrieves the ID from the ConfigVersionContext bean. This retrieved ID is then used to dynamically filter queries based on the desired configuration version.
Full Flow
A simplified view of the data retrieval process for a config-versioned entity is depicted in the following diagram: https://www.mermaidchart.com/raw/9a22117f-55fb-4c00-bf7f-2a4a1f1e5b97?theme=light&version=v0.1&format=svg
Pitfalls to Avoid
As you implement versioning for your entities, it’s crucial to consider potential pitfalls. Here’s one key point to remember:
Foreign Keys for Relationships Between Config-Versioned Entities
When you have a relationship between two config-versioned entities, remember to adjust the foreign key constraints. This is because a foreign key referencing only the entity ID wouldn’t be able to uniquely identify the intended related entity within a specific configuration version.
To ensure proper foreign key references, modify the foreign key to include both the entity ID and the config_version_id as a composite key. This way, the foreign key can accurately reference the specific version of the related entity within the context of a particular configuration.
Challenges
Implementing our versioning approach involved overcoming a few challenges. Here, we’ll discuss two key aspects that required careful consideration:
1. Request-Scoped Beans and Web Context
One initial hurdle we encountered involved using request-scoped beans outside of a web context. Since our ConfigVersionContext bean was scoped to the request, attempting to access it in non-web contexts resulted in exceptions. To address this, we devised a custom solution that's described in this blog post.
2. Cascade Deletes and Data Integrity
Another challenge we faced was managing cascade deletes at the database level. Cascade deletes typically function as expected when the foreign key includes the config_version_id in the referenced columns.
Scenario: MobileDevice and CleaningTour
However, consider the scenario described earlier with MobileDevice and CleaningTour. If the foreign key from MobileDevice only references cleaning_tour_id, deleting a CleaningTour while viewing a non-active config version could unintentionally impact the active config version by deleting the associated MobileDevice.
Our Approach
To mitigate this issue, we opted to drop all foreign keys on tables referencing config-versioned entities that lacked the config_version_id column. We then implemented custom logic within our code to handle the database cascades. This approach ensures data integrity by managing deletes programmatically. The delete itself is handled by using another Aspect that intersects on the delete call to a repository.
We needed to have a similar logic if the user wants to set a config version to the new active version. There we need to check if the references are satisfied on the new active version. If not, we delete the referenced entity.
Conclusion
Implementing entity versioning was a significant undertaking, requiring several months of development effort. While we encountered various challenges, particularly related to removing entity mapping, we created creative workarounds to overcome them.
Unique Feature and Benefits
As a result, we’ve established a unique feature within our industry, and early customer feedback has expressed positive reactions. From a developer perspective, we’re particularly pleased with maintaining original IDs and avoiding modifications to existing queries for config-versioned entities. This approach simplifies development and maintenance.
Phased Rollout and Future
We decided to go for a phased rollout strategy, introducing the feature on a per-entity basis. This allowed us to identify and address any issues effectively during the initial stages. We’re happy with the final outcome and excited for our customers to leverage the benefits of this feature. We’ll continue to monitor and improve the feature based on user feedback and evolving needs.
