Self Contained Systems Part 5— Ownership of Writes

Don’t miss the other posts in the series:

• Part 1 — An architecture for scaling dev organizations
• Part 2 — Data Follows UI
• Part 3 — Data Replication
• Part 4 — Data SLAs

By now you are well on your journey to splitting up you system into multiple Self Contained Systems (SCS). You are replicating your data and enjoying the freedom of unfettered data manipulation, reduced team communication costs, and are serving lightning fast UIs to your customers. But what about data writes? How do you avoid cycles between SCSs?

Our industry has tended to create one “true” objects that we pass around throughout our system. Domain Driven Design (DDD), which SCS has been heavily influenced by, teaches us to create a new object (I will use the word object, but do not confuse it with the object-oriented concept of an object, feel free to implement these concepts as structs, records, or any other programming construct that makes sense in your paradigm) in each domain for your concept. DDD refers to these concepts as an Entities. An Entity is defined by its thread of continuity (usually tracked by a unique id) instead of attributes like an object is. The attributes live in the multiple manifestations of the Entity.

Trip Entity

Our life at Veyo revolves around providing no-cost Non Emergency Medical Transportation (NEMT) trips to aging and low income populations through the Medicare and Medicaid programs. Suffice it to say that the Trip entity is one of the most important concepts in our system.

The Veyo system contains a very comprehensive model for transporting patients and it is out of the scope of this post to explore it fully, so we will make up a far simpler transportation service for the purposes of illustration.

Let’s say a trip doesn’t really start out as a trip — it starts out as a schedule. A schedule will eventually turn into one or more trips in the future. Let’s say we have a patient who needs to go to dialysis three times a week for the rest of her life. We will create a schedule that encapsulates the details of the trip: she is ambulatory (can walk without assistance), she doesn’t need any medical equipment (like oxygen tanks), the address of her dialysis facility, and a representation of her repeating appointments.

At some point closer to the date of the transportation need we can schedule a trip. This is a process of taking a single appointment from the schedule and creating a trip from it. Once we have a trip we can start to reason about how we are going to fulfill it. Are we going to use public transportation, do we need to do a mileage reimbursement, or are we going to dispatch a vehicle?

Let’s assume that for this trip we are going to dispatch a vehicle. The trip will then go through a dispatch process where we will assign the trip to a transportation provider.

Lastly we will track the trip to make sure that it completes successfully. This tracking will involve the inputs such as the GPS coordinates from the driver, and also tools such as the web app that agents will use to provide patient and driver support during the trip.

Breaking into SCSs

At this point we want to look at the seams of the system. There is a pretty obvious scheduling seam. We even have a different entity than we have in downstream parts of the system.

A second seam can be dispatching the trip. This will include trip creation, choosing the transportation mode, and preparing for the trip according to the needs of the chosen transportation mode.

The final seam could be fulfilling the trip. This part handles everything that needs to happen while the trip is in process such as tracking locations to providing support for those involved in the transportation.

For the sake of the example we will call them:

• Scheduling SCS
• Dispatching SCS
• Fulfillment SCS

We can think of our complete system as a state machine that is going to perform a data transformation such that:

Schedule -> Trip -> Trip’

Scheduling

As we enter some schedules into our system we are concerned with a start date with some kind of repeating schedule and possibly an end date. We are interested in the pickup and the drop off address. We are also interested in some basic attributes that will transfer to the trip such as the need for a lift on the vehicle to accommodate a patient with a wheel chair. Thus the schedule can be thought of as a template for creating a trip along with some meta data the dispatcher will use to create trips.

Dispatching

In dispatching we create the trip. It is tempting in a monolithic system to spend our time reasoning about the one true trip that we will use throughout the system. In this case we are only interested in the trip details that have to do with Dispatching the trip and we will intentionally leave out the additional information that will come into play during Fulfillment.

On Trip object creation we are going to copy a lot of the data from the Schedule into a Trip object. We are also going to add the transportation mode along with some additional data that pertains to the mode. For instance if we are using a transportation provider we are going to add their information to the trip. Since trip is going to be a SCS crossing concern we are going to add a unique persistent id to the Trip which is going to follow the trip’s journey through different manifestations in different SCSs.

Fulfillment

In fulfillment we will create a new trip object with the same unique persistent id as our Dispatch Trip object had. This id lets us reason about the trip as it moves through our system. Whereas The Dispatch Trip was our plan about how the trip was going to happen. The Fulfillment Trip is a record of everything that really happened.

In the Fulfillment Trip we don’t care what time someone was supposed to be picked up, we care about when they were actually picked up. We don’t care where they were supposed to be picked up at, we care about where they were actually picked up at. We can start attaching estimated times of arrival, and timestamped GPS coordinates. This is a much different view of the trip than the Dispatch Trip had.

And yet some things will be the same. We have a patient that we are picking up. Individual patients are tied to trips because there is an eligibility check for the trip and by changing the patient we would have to rerun eligibility. We also may want to deal in time windows. If the fulfillment time is outside the time window it is a different trip. So if you don’t get trasported today we are going to cancel the trip and create a new trip for tomorrow.

Ownership of Writes

By splitting up the ownership of objects between SCSs we have decluttered our data model and made it easier to locally reason about it. By giving each SCS it’s own data storage we have guaranteed that another SCS can’t become coupled to your database and thus have granted you data freedom. The last question comes down to the fields in your object. How do we reason about those?

There are really two types of data in your manifestation of an Entity. The first is data another SCS created that you have copied. The second is data that you have created in your SCS. An example of the former is the patient in the Trip entity. The Dispatch SCS created it and the Fulfillment SCS copied it (you may also argue that the patient is owned by the Schedule SCS and both the Dispatch SCS and the Fulfillment SCS copied it). An example of the latter timestamped GPS data which the Fulfillment SCS created for itself.

For data that your SCS has created you have read and write access. You will want to message that data upon write if you have downstream dependencies, but you can both read and write with impunity.

For data that another SCS created you have read access. If you need to change it then you should go back to the SCS that created that data and tell it about your proposed change. You can do this through a messaging command, or an API. This will guarantee uniform data processing and propogation.

Conclusion

By creating new manifestations of an Entity for each SCS you can reason about your system in a more localized way. This method will produce a much more flexible system. By being clear about ownership of data at the field level and delegating mutation to the SCS that created the data you will guarantee uniform data processing and propogation.

Continue Reading in Part 6 — Data Tricks