System Design
Notes on how data systems actually work underneath: what a storage engine does on one node, what breaks when you put several of them behind a network, and what each fix costs. The ground is roughly that of Designing Data-Intensive Applications, written up as mechanisms and trade-offs rather than tutorials.
Every claim that comes from outside carries a footnote back to the paper or the documentation it came from.
- 01Storage and RetrievalHash indexes, B-trees, LSM-trees with SSTables, the WAL, column-oriented storage.→
- 02Transactions and IsolationACID, read committed through serialisable, and the three routes to serialisability.→
- 03ReplicationSingle-leader, multi-leader and leaderless, plus CRDTs, read repair and quorums.→
- 04PartitioningRange and hash partitioning, secondary indexes, consistent hashing, 2PC.→
- 05Consensus and LinearizabilityLamport clocks, total order broadcast, Raft, and coordination services.→
- 06Database ComparisonsRelational against document, MySQL against PostgreSQL, VoltDB, Spanner, Cassandra.→
- 07Batch ProcessingHDFS, HBase, MapReduce, the three batch join strategies, and Spark.→
- 08Stream ProcessingTime windows, CDC, event sourcing, delivery semantics, brokers and Flink.→
- 09Search and Specialised StoresInverted indexes, Lucene, Elasticsearch, time-series, graph and geospatial.→
- 10CachingWhere a cache sits, the three write strategies and what each one costs.→