The virtual glue for distributed programming
When I first started coding, I remember how you had first to spend at least a full day setting up your machine (if you were a bad-ass), then a good chunk of time downloading the application you were going to work on, and finally the rest of the week getting it configured and ready to give you its default behavior. At this point, you haven’t written a single line of code yet. Instead, you are just trying to make sure that whatever tribal knowledge and complicated configuration steps have made its way to your installation of the product so that you can start doing what you need to do.
Great, finally got this set up. Let’s make some code changes!
Boom! You kick off a build and go to lunch, or sometimes you go home for the day. Hopefully, when you’re back, the build and tests would be over. Finally, when that’s all done you would lock your file on whichever source control repo the elders before you had picked and commit your change. Of course, you’re hoping the file wasn’t already locked, and there aren’t that many conflicts if that’s not the case, its a simple case of “rinse-and-repeat” until you get to the promised land.
Boy am I glad we don’t do that anymore; we now have the Cloud, micro-services, server-less and <insert buzzwords here> that break us free of the shackles of the Monolith! I can focus on my sliver of functionality and not worry about every other part of the product!
Now, I need to know about when something I care about happens and take action on it. Cool, I’ll grab that reference to that object and…
Oh, I don’t have that anymore.
No problem, I’ll expose an API method so other parts of the system can tell me when they change those things…
oh, no one is going to change that code anytime time soon?
I guess I’ll frequently poll to see if things have changed… for all the things… all the time… no, that destroys performance.
Well, how do others solve this problem?
Event-driven architecture (EDA), is a software architecture pattern promoting the production, detection, consumption of, and reaction to events. An event can be defined as “a significant change in state”. For example, when a consumer purchases a car, the car’s state changes from “for sale” to “sold”
There is quite a bit to unpack there; its a notable change in mentality from traditional development in the sense that you can’t just raise an event with the “sender” or the whole object in the args. Let’s discuss how to think in a more distributed manner from the perspective of the publisher, the consumer and the observer. Finally, I’ll shamelessly plug a library we put together to simplify the concepts we discuss here and show you how seamlessly decouple and detach your code from the Monolith, all while still feeling like you’re part of the whole solution.
Only send references to your object not the whole thing:
When you live on an island made of code, you need to think a little differently about what actions you are making and whether anyone else could care about it. Furthermore, you need to consider that whatever object you’re eventing on may not be easily de-serializable by the consumer. As a matter of fact, in some cases, the observer of the event may not know what your objects look like nor do they care. Moreover, you wouldn’t want to send an object graph to everyone if they don’t care, do you?
Consider your object life cycle and event on each transition:
This concept seems pretty straightforward, but I see many people fall into the mistake of omitting intermediate states. Let’s consider the car example from the Wikipedia article: “when a consumer purchases a car, the car’s state changes from ‘for sale’ to ‘sold.’” Simplistically speaking, those are the 2 major states that car object goes through in its life cycle. However, what if there was an observer system that wanted to know how close deals were to being closed so that it may offer incentives or so it can project out an end of day revenue. Simply waiting until the car is “sold” is not sufficient.
Furthermore, what if there is another consumer that is trying to sell the same car? Seeing the car as available might prompt them to begin processing their sale only to be beaten to it by someone else.
In this case, a more robust state list would look like this: “for sale,” “sale pending,” “pending financing,” “sold,” “pending delivery”, “delivered.”
Don’t assume delivery or timeliness, always include some metadata:
As the publisher, you must assume that whoever is listening on the other side of the wall has no idea about you or what you’re event represents. That’s why additional metadata in the event is crucial to help others know what they should do with this event.
I usually like to add the source of the event concerning the name of the micro service, the machine that raised it and even the process id. That metadata goes a long way regarding introspection and lets the receiver quickly decide if this message is of any importance to them.
Another critical element is a message identifier; this is a unique identifier that allows the receiver to keep track of the received messages so as not to reprocess a single message.
Finally, its useful to include a timestamp on the message. The stamp helps the receiver determine message validity in the case messages are delivered out of order, or they become stale due to network issues.
Of course, many more things could be included as metadata, and adding a handful of them is always wise, but be careful not to flood the message with more metadata than payload.
Next time, I’ll dive into the Event Consumer and highlight some important things to consider from that perspective.