KafkaIQ

10.2.3 – Active-Passive vs Active-Active

Module: M10 – Advanced Topics & Architecture
Section: 10.2 – Multi-Datacenter Replication & Geo Architecture
Goal: Master active-passive and active-active replication patterns


Active-Passive (Primary-Backup)

Architecture

Primary Cluster (US-East) - All writes/reads
  ↓ one-way replication
Passive Cluster (US-West) - Standby only

Characteristics

  • Single active cluster: All traffic goes to primary
  • Passive replica: Disaster recovery only
  • One-way replication: Primary → Passive
  • Manual failover: Switch to passive if primary fails

Use Cases

  • Disaster recovery
  • Regulatory backup requirements
  • Simple DR strategy

Pros

✅ Simple to understand and implement
✅ No conflict resolution needed
✅ Clear failover procedure

Cons

❌ Passive cluster idle (wasted resources)
❌ Manual failover (RTO in minutes)
❌ Replication lag = data loss window (RPO > 0)

Configuration

# Primary → Passive replication only
clusters = primary, passive

primary->passive.enabled = true
passive->primary.enabled = false  # No reverse replication

# Failover: Switch applications to passive cluster

Active-Active (Multi-Master)

Architecture

Cluster A (US-East) ←→ Bidirectional ←→ Cluster B (US-West)
  ↑                                        ↑
Producers/Consumers                  Producers/Consumers

Characteristics

  • Both clusters active: Reads and writes in both regions
  • Bidirectional replication: A ↔ B
  • Local latency: Producers/consumers connect to nearest cluster
  • Conflict resolution: Required for writes to same key

Use Cases

  • Global applications
  • High availability
  • Low latency for distributed users

Pros

✅ Low latency (local writes)
✅ High availability (no single point of failure)
✅ Resource utilization (both clusters used)
✅ Automatic load distribution

Cons

❌ Complex (conflict resolution, loop prevention)
❌ Eventual consistency (replication lag)
❌ Higher operational burden

Configuration

clusters = us-east, us-west

# Bidirectional replication
us-east->us-west.enabled = true
us-west->us-east.enabled = true

# Loop prevention
us-east->us-west.topics = (?!us-west\\.).*
us-west->us-east.topics = (?!us-east\\.).*

Conflict Resolution Strategies

Conflict Scenario

Time T0:
  Cluster A: User updates profile: name="Alice"
  Cluster B: User updates profile: name="Bob"

Time T1 (after replication):
  Cluster A has: name="Bob" (from B)
  Cluster B has: name="Alice" (from A)

Conflict: Which value wins?

Strategy 1: Last-Write-Wins (LWW)

Rule: Message with latest timestamp wins.

Implementation:

if (messageA.timestamp() > messageB.timestamp()) {
    use(messageA);
} else {
    use(messageB);
}

Pros: Simple
Cons: Clock skew can cause incorrect results


Strategy 2: Cluster Priority

Rule: One cluster always wins conflicts.

Example: US-East priority > US-West

If conflict, US-East write wins

Pros: Deterministic
Cons: One cluster effectively primary (defeats active-active benefit)


Strategy 3: Application-Level Resolution

Rule: Application merges conflicting writes.

Example: Shopping cart merge

Cluster A: Cart has [item1, item2]
Cluster B: Cart has [item2, item3]

Merge: Cart has [item1, item2, item3]  # Union

Pros: Application-specific logic
Cons: Complex implementation


Strategy 4: Avoid Conflicts (Partitioning)

Rule: Route users to specific cluster, no cross-cluster writes for same key.

Example:

US users → US-East cluster
EU users → EU-West cluster

No US user writes to EU cluster → no conflicts

Pros: No conflicts
Cons: Loses active-active benefits (users tied to region)


Failover Scenarios

Active-Passive Failover

Scenario: Primary cluster fails

Steps:

  1. Detect primary failure (healthcheck, monitoring)
  2. Stop MM2 replication (primary → passive)
  3. Promote passive to primary (update DNS/load balancer)
  4. Redirect producers/consumers to new primary
  5. Start reverse replication (new primary → old primary, when recovered)

RTO: 5-15 minutes
RPO: Replication lag (seconds to minutes)


Active-Active Failover

Scenario: Cluster A fails

Steps:

  1. Detect Cluster A failure
  2. Redirect Cluster A traffic to Cluster B (DNS/load balancer)
  3. Cluster B handles all traffic (already active)
  4. When Cluster A recovers, resume bidirectional replication

RTO: Seconds (traffic already handled by B)
RPO: 0 (both clusters have data, eventual consistency)


Performance Comparison

Metric Active-Passive Active-Active
Write Latency Low (single cluster) Low (local cluster)
Read Latency Low (single cluster) Low (local cluster)
Failover RTO Minutes (manual) Seconds (automatic)
Failover RPO Replication lag 0 (eventual consistency)
Resource Utilization 50% (passive idle) 100% (both active)
Complexity Low High

When to Use Each

Use Active-Passive If:

  • Simple DR requirement (backup only)
  • Limited operational resources
  • No global user base (all users in one region)
  • Cost-sensitive (don't want to pay for active resources)

Use Active-Active If:

  • Global user base (users in multiple regions)
  • High availability requirement (no downtime tolerated)
  • Low latency requirement (< 50ms for all users)
  • Resource efficiency (utilize all clusters)

Key Takeaways

  1. Active-Passive: Simple DR, passive cluster idle, manual failover (RTO: minutes).

  2. Active-Active: Both clusters active, low latency, complex (conflict resolution).

  3. Conflict Resolution: LWW, cluster priority, app-level merge, or partitioning.

  4. Failover: Active-passive = manual (minutes), active-active = automatic (seconds).

  5. Resource Utilization: Active-passive wastes 50%, active-active uses 100%.

  6. Complexity: Active-passive simpler, active-active requires careful design.


What's Next?

Next Module: 10.2.4 – Replication Consistency Challenges

References:

  • Multi-cluster patterns → 10.2.1
  • MirrorMaker 2 → 10.2.2