From Memcache to Static Variables: Firestore Realtime Database

Recently, I upgraded a Java 8 App Engine application to Java 17. This change required a shift in how the application handled real-time configurations, which were critical for each API call. In the Java 8 environment, we leveraged App Engine’s Datastore and Memcache, but this approach was no longer viable with the upgrade.

The Challenge: Replacing Memcache in Java 17

Upgrading to Java 17 presented several key challenges:

1. App Engine Memcache Deprecation: App Engine Memcache was no longer available in the new environment.
2. Cost and Security Concerns: Google’s alternative Memcache solutions were either not cost-effective or raised security concerns due to the banking nature of the application.
3. External Dependency Risk: We sought to minimize the reliance on external dependencies to ensure robustness and stability.

The Solution: Static Variables and Firestore Realtime Database

After evaluating various options, we devised a solution that combined the power of static variables with Firestore’s Realtime Database:

1. Static Variables for Speed: We chose static variables to hold the configuration data in memory, offering the fastest possible access times.
2. Firestore for Real-Time Updates: We utilized Firestore’s Realtime Database to mirror the configuration data from the primary Datastore. This allowed us to push real-time updates to all application instances seamlessly.
3. Firestore Listeners: Each application instance had a Firestore listener that actively monitored changes in the Firestore Node. Upon detecting an update, the listener would refresh the corresponding static variables.

Technical Implementation

The solution was built using:

• Spring Boot + Java 17
• Firestore Java SDK
• Google Cloud Datastore Library
• GCP Services (Datastore, Firestore Native Mode, App Engine)

Considerations and Mitigation

Following two concerns should be handled:

1. Fail-Safe Mechanisms: Implement fail-safe mechanisms to handle cases where updates might not reach the Firestore listeners due to network issues or other disruptions.
2. Contention Management: Put strategies in place to manage potential contention issues that could arise when multiple threads access and update static variables simultaneously.

Conclusion

By replacing Memcache with a combination of static variables and Firestore’s Realtime Database, we successfully migrated our Java 8 application to Java 17. This solution met our requirements for speed, cost-effectiveness, and minimal external dependencies. We were able to overcome the challenges posed by the upgrade and ensure the continued smooth operation of our banking application.

This solution can run on most cloud provider services, including GKE, Cloud Run, VM instances, Docker, and on-premises servers.