KafkaIQ

section 7 · 16 modules

Internal Mechanics

Under the hood: log structure, replication, ISR, leader election, the controller and KRaft metadata.

  1. End to End Message Flow and Core Architecture
  2. Topics, Partitions & Log Structure
    1. Topic and Partition Internals
    2. Log Segments, Indexes and Storage Layout
    3. Compaction and Retention Mechanics
  3. Replication & Durability Protocol
    1. Replication Protocol Explained
    2. ISR and High Watermark Mechanics
    3. Follower Fetching and Data Consistency
  4. Leader Election & Failover Mechanics
    1. Partition Leadership
    2. Leader Election Algorithms
    3. Failover Scenarios and Edge Cases
  5. Cluster Metadata & Coordination
    1. Metadata Model and Cluster State
    2. ZooKeeper Coordination (Legacy)
    3. KRaft Metadata Quorum
  6. The Kafka Controller Internals
  7. Broker Lifecycle and State Transitions
  8. Transaction and Exactly Once Internals (Broker Side)