Replication?

Keeping a copy of the same data on multiple machines. Each node that stores the copy is called Replica.
Advantages of Replication
  1. Lower latency: Put a replica near the user(in India). He donot need to wait for reading from Virginia.
  2. Availability: If one node crashes, replica still serves data.
  3. High throughput: Writes go to the leader; many followers/replicas share the read load. 10 replicas can serve roughly ten times the reads of one node.
  4. Parallel processing: Leader handles writes. Followers serve read queries (reports, product listing, “my orders”).

Types of Replication

1. Master Slave, Multiple Replica/Followers
2. Multi Master/leader
3. Leaderless

1. Master/Slave / Leader/Follower

Clients write only to the leader. The leader writes locally, then ships a replication log (change stream) to followers.
Followers apply those changes in the same order as the leader. Clients usually read from any follower.
Examples: Built in Postgres, MySQL, MongoDB, Kafka partition replicas

                                                                        |------> |Replica|  ---- Read ----> Client
                                        |-----------|                   |
Application Server --- write ---------> | Master DB | --replication log-|------> |Replica|
                                        |-----------|                   |
                                                                        |------> |Replica|
        

2. Master/Master / Active/Active / Leader/Leader

Two or more leader nodes. A client can write to any one of them. The other leaders act as followers for those writes and pull the change stream. Useful for multi-datacenter writes; the cost is write conflicts (two leaders update the same row).

Implementation of Replication Logs

1. Statement-based replication

Leader logs every DB request (INSERT, UPDATE, or DELETE) that it executes to its disk in a file
And sends this log file to its followers which it applies to its DB
Issues:
1. Any statement that calls a nondeterministic function(eg: NOW(), RAND()) will generate different results on master and replica
  Fix: leader can replace any nondeterministic function calls with a fixed value
2. if statement depends on some condition. Eg: (UPDATE … WHERE some condition) will break since condition might not have been written on replica
3. Many other edge cases
Other replication methods are now generally preferred now

2. Write-ahead log (WAL)

Write to disk 1st then DB. Then send log to replicas
Suppose node crashes while writing, so when it recovers it can check Log file and actual Disk content so see what's completed and not.
RAFT also uses WAL

Format WAL (Binary file)?

WAL is stored as binary file
Benefits of Storing as Binary wrt txt:
1. File Header: Every binary entry starts with a fixed-size header detailing exactly how many bytes long the entry is. The operating system can read this header and instantly know where the next log entry begins without parsing lines of text.
2. Appends Only: The file is strictly append-only. Random edits or insertions in the middle of the file are forbidden, making it fast to write
3. Checksums (CRC): Every entry includes a checksum. If a power outage or disk failure corrupts even a single bit of data, the system detects the broken checksum on reboot and prevents corrupted data from entering your database.
2.