CAP Theorem notes
… skipping definition…
This applies to read and write operations — so can be best explained with a data store.
Terms of CAP
- A set of reads and writes on all the nodes, at the same point of time would lead to same results.
Atomic vs Eventual Consistency
- Above is atomic consistency
- Eventual consistency relaxes this condition by allowing a certain lag.
- Messages take an unbounded amount of time to return
- We don’t get to know weather they failed or are pending
- If we write data or read data, it should return a result
- a message is lost between nodes (not delayed).
- total partition — a period of time when no message is delivered
Why can messages get delayed between nodes?
- GC, retries,
Mongodb and CAP theorem
- Mongodb is a single master
- C1: All reads go through primary by default, so it is strongly consistent by default
- C2: If we enable reading on secondaries, the system becomes eventually consistent.
- Discuss: There are 2 data centers and each data center has one mongodb master. The main master is at DC1. The connection between DC1 and DC2 breaks — what will happen?
Oracle and financial systems
- Oracle for financial systems have been run on a single lag with very little latency between nodes
- This allows them to have ACID properties required from debit credit transactions
List out the doubts with above system
- Rack has N1, N2, … Nk nodes
- A debit goes to N1 and another debit goes to N2 almost at the same time
- Can both succeed?
- How can we ensure that only one succeeds?
- Users are sharded among nodes
- If a node goes down, then the transaction fails until is it brought back up — not Available
- If we want availability, we will need backup racks (failover) or spare nodes that takeover… however we will have to induce latency such that the write ahead logs of both the nodes are at sync always
Primary with hot failover
- Explain below
- ~ Mediocre at catastrophic failover
- ~ Window of loss
- ~ Failover data may be in-consistent
- ~ banks, brokerages, etc
- Simultaneous writes in different DCs
- Higher latency
X axis —
(e) Data loss
(b) Master Slave
Transactions across data-centers
- Usually not required
- 2-phase commit and Paxos solve this problem
- How is eventual consistency used above?
- The first commit does a prepare — it removes the available resource and puts it into a shared area
- This operation is synced across both the datacenters
- What is this operation is done in both the datacenters at the same time?
- Slide 14: http://snarfed.org/transactions_across_datacenters_io.html
- Discuss: https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
- 2 Phase commit in mongodb