CQRS/ES and Firebase Cloud Functions
Or how I failed to put these two together
The application (at the relevant commit) has commands that write events to the Firebase Realtime Database to “/event_store”, and the apply function applies the events on top of the current state of the aggregate in “/state_store” path. To elaborate, apply triggers a once()listener to fetch the state of all the aggregates¹ in memory, and the promise-chained on() listener on “/event_store” will apply any new events, mutating the in-memory state and updating the “/state_store”.
All works well, even when events coming in from multiple sources (such as from the admin commands in the repo or from the iOS app). I figured that a cloud server could be spared to handle events when apply is reimplemented as a cloud function. Distribution is not easy though…
Carrying state
There are (probable) ways to try to work around the stateless nature of cloud functions [1, 2], it seemed better to accept the truth that I won’t be able to do this.
Before admitting (temporary?) defeat, I tried to rewrite apply (see cloud_apply ) to mimic cloud functions² by invoking the “/state_store” once() listener inside the callback of the on() “/event_store” listener. That is, each new event would trigger fetching the state from the database, work on that state and write the mutated state back to the “/state_store”.
Unfortunately, if events trigger cloud_apply fast enough, all callbacks end up with the same state to work with, and for some reason events get handled in a reversed order, newest to oldest. (In this implementation, events have a sequence number property to deal with ordering.³)
$ node
> var f = require('./admin-functions.js');
> f.public_commands.add_user({first_name: "Attila", last_name: "Gulyas", username: "", email: "toraritte@gmail.com", account_types: ["admin", "reader"]})// 4 events in the bank.
> f.cloud_apply()Stream: -LLf1MW_BjNdImtFLpl8 event: -LLf1MXo7XODvaHYXxll added_to_group
Action: replay (event_seq: 4, state_seq: 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Stream: -LLf1MW_BjNdImtFLpl8 event: -LLf1MXnv_wxX6yl7wwo added_to_group
Action: replay (event_seq: 3, state_seq: 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Stream: -LLf1MW_BjNdImtFLpl8 event: -LLf1MXknyGts_5jIHq9 email_added
Action: replay (event_seq: 2, state_seq: 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Stream: -LLf1MW_BjNdImtFLpl8 event: -LLf1MWaV2o95AUg7VbN person_added
Action: replay (event_seq: 1, state_seq: 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Documenting a peculiar behaviour
Interestingly enough, when issuing
f.ADMIN_APP.database().ref("state_store").on('value', function() {})(note the empty callback), the ordering issue went away. Didn’t look into the cause of this and it wouldn’t be safe to rely on such undocumented(?) behaviours anyway.
Out of order event handling (or how do you remember the future?)
Cloud Functions are not guaranteed to fire in order or only once. With state out of the picture, how does one deal with out of order events when order matters? Imagine the email_deleted event applied before email_added ; even if idempotency is ensured, there needs to be a way to remember that a future property is being deleted.
As the quoted SO answer below suggests, a state table can (probably) be the solution, but app becomes really complex.
The reverse-order issue may not hit the cloud function implementation, but the functions would probably be triggered out of order nonetheless. “Cloud Functions are not guaranteed to fire in order or only once” (see question also).
Conclusion
It was a nice thought experiment, and may pursue this further when time permits, but the safest would be to just run a virtual machine with a daemonized Node app.
- [^] Not ideal, hence issue #5 to fetch on a per stream basis.
- [^] Firebase Cloud Functions also have Realtime Database triggers, similar to the ones of the Admin and client SDKs.
- [^] Far from perfect, but timestamp-based ordering on Firebase did not work out.
