10.3.3 – KRaft vs ZooKeeper Resilience

Module: M10 – Advanced Topics & Architecture
Section: 10.3 – High Availability & Disaster Recovery
Goal: Compare resilience of KRaft vs ZooKeeper modes

Architecture Comparison

ZooKeeper Mode

Kafka Brokers ← depend on → ZooKeeper Ensemble (3-5 nodes)
  - Metadata stored in ZK
  - Controller election via ZK
  - Broker coordination via ZK

Dependencies: Kafka + ZooKeeper (two systems to manage)

KRaft Mode

Kafka Cluster
  ├── Controller Nodes (3-5, run Raft consensus)
  └── Broker Nodes (data plane only)

Dependencies: Kafka only (self-contained)

Resilience Comparison

Failure Scenarios

Controller Failure

ZooKeeper Mode:

1. Controller broker crashes
2. ZooKeeper detects loss (heartbeat timeout)
3. Remaining brokers compete for controller role (write to ZK)
4. First to write wins → new controller
5. New controller reads state from ZK

Time: 10-30 seconds

KRaft Mode:

1. Controller node crashes
2. Remaining controllers detect loss (Raft heartbeat)
3. Raft elects new leader (automatic, no competition)
4. New leader has full metadata (already replicated via Raft)

Time: 3-10 seconds (faster)

Metadata Quorum Loss

ZooKeeper Mode:

• Quorum: Requires majority of ZK nodes (2 of 3, 3 of 5)
• Loss: If quorum lost, cluster frozen (no metadata updates, no leader elections)
• Recovery: Restore ZK quorum → Kafka recovers automatically

KRaft Mode:

• Quorum: Requires majority of controller nodes
• Loss: Same behavior (cluster frozen)
• Recovery: Restore controller quorum → Kafka recovers automatically

Winner: Tie (both use quorum-based consensus)

Metadata Corruption

ZooKeeper Mode:

• Corruption: If ZK data corrupted, entire cluster affected
• Recovery: Restore from ZK backup or rebuild metadata

KRaft Mode:

• Corruption: If controller log corrupted, same impact
• Recovery: Restore from controller log backup

Winner: Tie (both need backups)

Performance Under Failure

Metadata Operations During Failover

ZooKeeper Mode:

• Metadata operations (create topic, leader election) blocked during controller failover
• Latency spike: 10-30 seconds

KRaft Mode:

• Faster failover = shorter metadata operation block
• Latency spike: 3-10 seconds

Winner: KRaft (faster recovery)

Operational Differences

Monitoring

ZooKeeper Mode:

• Monitor Kafka + ZooKeeper separately
• Tools: JMX (Kafka), ZK 4-letter commands (ruok, stat)

KRaft Mode:

• Monitor Kafka only
• Tools: JMX, Kafka CLI tools

Winner: KRaft (fewer systems)

Backup and Restore

ZooKeeper Mode:

# Backup ZK data
tar -czf zk-backup.tar.gz /var/zookeeper/data

# Restore
tar -xzf zk-backup.tar.gz -C /var/zookeeper/data

KRaft Mode:

# Backup controller logs
tar -czf controller-backup.tar.gz /var/kafka/controller-logs

# Restore
tar -xzf controller-backup.tar.gz -C /var/kafka/controller-logs

Winner: Similar process

Migration Path

ZooKeeper → KRaft Migration:

1. Upgrade to Kafka 3.3+ (supports KRaft)
2. Run migration tool: kafka-metadata.sh --migrate
3. Start controllers
4. Migrate brokers one by one
5. Decommission ZooKeeper

Recommendation: New clusters should use KRaft (ZooKeeper deprecated in Kafka 4.0)

Key Takeaways

1. KRaft Faster: Controller failover 3-10 seconds (vs 10-30 seconds with ZK).
2. KRaft Simpler: One system to manage (vs Kafka + ZK).
3. Quorum Math Same: Both require majority for consensus.
4. KRaft Scales Better: Supports millions of partitions (vs 200k for ZK).
5. Migration Available: Can migrate from ZK to KRaft.
6. ZooKeeper Deprecated: Removed in Kafka 4.0 (use KRaft for new deployments).

What's Next?

Next Module: 10.3.4 – Backup and Restore Strategies

References:

• Controller details → M7: Kafka Internals
• Operations → M6: Operations