Write-Ahead Log — WAL

Write Ahead Log Sample Struct

Strong durability is a default, that we assume out of all our data stores. But have you wondered, how this high durability is achieved with all the high performance/throughput requirements, even in the face of server crashes(power failures/os failures/hardware failures). Well, the answer is WAL.

WAL — Write Ahead Log

WAL isn’t truly a distributed design pattern, but a widely used pattern that is prevalent in most distributed systems guaranteeing durability.

WAL has been a common theme across traditional RDBMS systems and is used to help with atomicity & durability(A & D of ACID) guarantees. All mutations to a table are written first to the WAL(Transaction Log/Bin Log) & then applied to the actual data files of the tables asynchronously.

Sample WAL and WALEntry structure:

type WAL struct {
    dir       string // directory under which WAL files are stored.

    file      *os.File // reference to the file

    metadata  []byte           // metadata recorded at the head of each WAL
    decoder   *decoder       // decoder to decode records
    encoder   *encoder // encoder to encode records
    
    mutex          sync.Mutex // To ensure single update per writer
    lastIndex      uint64   // index of the last entry saved to the WAL
}

type WALEntry struct {
    lsn      uint64 // unique…