KafkaIQ

3.1 Partition Assignment Strategies

Module Overview

Goal: Make partition assignment an explicit design choice, not an accident.

What You'll Learn:

  • Why assignment strategies matter
  • Range assignor (default)
  • RoundRobin assignor
  • Sticky assignor
  • CooperativeSticky assignor
  • Load distribution implications
  • Ordering implications
  • When each strategy makes sense

What's NOT Covered Here:

  • Rebalance lifecycle (covered in Module 4.1)
  • Cooperative rebalancing protocol (covered in Module 4.1)
  • Consumer group coordination basics (covered in Module 2.1)

Key Insight: Different assignment strategies optimize for different goals: balance, stickiness, or minimal disruption.

1. Why Assignment Strategies Matter

The Assignment Problem

Given:

  • N consumers in a group
  • P partitions in subscribed topics

Question: Which consumer reads which partitions?

Topic: "orders" (6 partitions)
Consumer Group: 3 consumers

Possible assignments:

Option A:
Consumer 1 → Partitions 0, 1
Consumer 2 → Partitions 2, 3
Consumer 3 → Partitions 4, 5

Option B:
Consumer 1 → Partitions 0, 2, 4
Consumer 2 → Partitions 1, 3, 5
Consumer 3 → (none - wasteful!)

Option C:
Consumer 1 → Partitions 0, 1, 2, 3, 4, 5
Consumer 2 → (none)
Consumer 3 → (none)

Different assignments have different properties!

What Makes a Good Assignment?

1. Balance

  • Load distributed evenly across consumers
  • No consumer is overloaded while others are idle

2. Stickiness

  • Partitions stay with same consumer across rebalances
  • Preserves local state/caches
  • Reduces ramp-up time

3. Minimal Disruption

  • Few partition movements during rebalance
  • Most consumers keep their partitions
  • Reduces stop-the-world effect

Different strategies optimize different properties.

2. Configuring Assignment Strategy

Configuration

props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
    "org.apache.kafka.clients.consumer.RangeAssignor");

Available strategies:

  • RangeAssignor (default)
  • RoundRobinAssignor
  • StickyAssignor
  • CooperativeStickyAssignor (Kafka 2.4+)

You can also specify multiple strategies (fallback list):

props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
    "org.apache.kafka.clients.consumer.CooperativeStickyAssignor," +
    "org.apache.kafka.clients.consumer.StickyAssignor," +
    "org.apache.kafka.clients.consumer.RangeAssignor");

How it works:

  • All consumers in group must agree on strategy
  • First common strategy is used
  • If no common strategy → error

3. Range Assignor (Default)

How Range Works

Algorithm:

  1. For each topic, sort partitions by number
  2. Divide partitions evenly among consumers (alphabetically sorted)
  3. Assign ranges to consumers

Example: Single Topic

Topic: "orders" (7 partitions: 0-6)
Consumers: 3 (A, B, C)

Partitions per consumer: 7 / 3 = 2 remainder 1

Consumer A: Partitions 0, 1, 2  (2 + 1 remainder)
Consumer B: Partitions 3, 4     (2)
Consumer C: Partitions 5, 6     (2)

Note: First consumer gets the remainder.

Example: Multiple Topics

Topics: "orders" (6 partitions), "payments" (6 partitions)
Consumers: 3 (A, B, C)

Topic: "orders"
Consumer A: Partitions 0, 1
Consumer B: Partitions 2, 3
Consumer C: Partitions 4, 5

Topic: "payments"
Consumer A: Partitions 0, 1  ← Duplicate!
Consumer B: Partitions 2, 3  ← Duplicate!
Consumer C: Partitions 4, 5  ← Duplicate!

Total:
Consumer A: 4 partitions (orders 0-1, payments 0-1)
Consumer B: 4 partitions (orders 2-3, payments 2-3)
Consumer C: 4 partitions (orders 4-5, payments 4-5)

Balanced in this case.

When Range is Unbalanced

Topics: "orders" (7 partitions), "payments" (7 partitions)
Consumers: 3 (A, B, C)

Topic: "orders"
Consumer A: Partitions 0, 1, 2  (3 partitions)
Consumer B: Partitions 3, 4     (2 partitions)
Consumer C: Partitions 5, 6     (2 partitions)

Topic: "payments"
Consumer A: Partitions 0, 1, 2  (3 partitions) ← Extra again!
Consumer B: Partitions 3, 4     (2 partitions)
Consumer C: Partitions 5, 6     (2 partitions)

Total:
Consumer A: 6 partitions  ← Overloaded!
Consumer B: 4 partitions
Consumer C: 4 partitions

Problem: Consumer A gets 50% more work than others!

Range Pros and Cons

Pros:

  • ✅ Simple and predictable
  • ✅ Preserves co-partitioning (same partition numbers from different topics go to same consumer)
  • ✅ Good for joins and aggregations across topics

Cons:

  • ❌ Can be unbalanced with multiple topics
  • ❌ First consumer often gets more work
  • ❌ Not sticky across rebalances

Use when:

  • Single topic consumption
  • Co-partitioned topics (joins/aggregations)
  • Simplicity preferred

4. RoundRobin Assignor

How RoundRobin Works

Algorithm:

  1. Sort all partitions from all topics
  2. Sort consumers alphabetically
  3. Assign partitions to consumers in round-robin fashion

Example: Single Topic

Topic: "orders" (7 partitions: 0-6)
Consumers: 3 (A, B, C)

Consumer A: Partitions 0, 3, 6  (3 partitions)
Consumer B: Partitions 1, 4     (2 partitions)
Consumer C: Partitions 2, 5     (2 partitions)

Example: Multiple Topics

Topics: "orders" (6 partitions), "payments" (6 partitions)
Consumers: 3 (A, B, C)

All partitions: orders-0, orders-1, ..., orders-5, payments-0, ..., payments-5

Consumer A: orders-0, orders-3, payments-0, payments-3
Consumer B: orders-1, orders-4, payments-1, payments-4
Consumer C: orders-2, orders-5, payments-2, payments-5

Total:
Consumer A: 4 partitions
Consumer B: 4 partitions
Consumer C: 4 partitions

Perfectly balanced!

Example: Uneven Partitions

Topics: "orders" (7 partitions), "payments" (7 partitions)
Consumers: 3 (A, B, C)

All partitions (14 total): orders-0, ..., orders-6, payments-0, ..., payments-6

14 partitions / 3 consumers = 4 remainder 2

Consumer A: orders-0, orders-3, orders-6, payments-2, payments-5  (5)
Consumer B: orders-1, orders-4, payments-0, payments-3, payments-6  (5)
Consumer C: orders-2, orders-5, payments-1, payments-4  (4)

Much more balanced than Range!

RoundRobin Pros and Cons

Pros:

  • ✅ Better balance than Range (especially with multiple topics)
  • ✅ Simple to understand
  • ✅ Uniform distribution

Cons:

  • ❌ Breaks co-partitioning (partition 0 from different topics go to different consumers)
  • ❌ Not sticky across rebalances
  • ❌ Bad for stateful processing (no locality)

Use when:

  • Multiple unrelated topics
  • Balance is more important than co-partitioning
  • Stateless processing

5. Sticky Assignor

Why Sticky?

Problem with Range and RoundRobin:

  • On rebalance, partitions redistributed from scratch
  • Consumers lose local state (caches, RocksDB, etc.)
  • Ramp-up time to rebuild state

Sticky Assignor Goal:

  • Maximize partition retention across rebalances
  • Keep partitions with same consumer when possible
  • Only move partitions when necessary

How Sticky Works

Algorithm:

  1. Start with current assignment (if any)
  2. When rebalancing:
    • Keep as many existing assignments as possible
    • Only reassign partitions that must move
  3. For new partitions, balance load

Example: Initial Assignment

Topic: "orders" (6 partitions)
Consumers: 3 (A, B, C)

Initial (Round 1):
Consumer A: Partitions 0, 1
Consumer B: Partitions 2, 3
Consumer C: Partitions 4, 5

Example: Consumer Leaves

Round 1: A, B, C each have 2 partitions

Consumer B leaves (dies/shutdown)
Round 2: A, C remain

With RoundRobin (non-sticky):
Consumer A: Partitions 0, 2, 4  ← Lost 0, 1; gained 2, 4
Consumer C: Partitions 1, 3, 5  ← Lost 4, 5; gained 1, 3
(Everything moved!)

With StickyAssignor:
Consumer A: Partitions 0, 1, 2  ← Kept 0, 1; gained 2
Consumer C: Partitions 4, 5, 3  ← Kept 4, 5; gained 3
(Minimal movement!)

Key Insight: Sticky keeps 4 out of 6 assignments, RoundRobin keeps 0.

Example: Consumer Joins

Round 1: A, C each have 3 partitions

Consumer B joins
Round 2: A, B, C

With RoundRobin (non-sticky):
Consumer A: Partitions 0, 3
Consumer B: Partitions 1, 4
Consumer C: Partitions 2, 5
(Everything moved!)

With StickyAssignor:
Consumer A: Partitions 0, 1  ← Kept 0, 1; lost 2
Consumer B: Partitions 2, 3  ← New, gets B's old + one from C
Consumer C: Partitions 4, 5  ← Kept 4, 5; lost 3
(Only 2 partitions moved!)

Sticky Pros and Cons

Pros:

  • ✅ Minimizes partition movement during rebalances
  • ✅ Preserves local state (caches, stores)
  • ✅ Faster recovery after rebalance
  • ✅ Still maintains balance

Cons:

  • ❌ More complex algorithm
  • ❌ Slightly slower rebalance computation
  • ❌ Still uses eager rebalancing (stop-the-world)

Use when:

  • Consumers maintain local state
  • Rebalances are costly (cache warm-up)
  • Stateful stream processing
  • Kafka Streams applications

6. CooperativeSticky Assignor

The Evolution of Sticky

StickyAssignor is great for retention, but still uses eager rebalancing:

  • All consumers stop consuming
  • Assignment computed
  • All consumers resume

CooperativeStickyAssignor adds:

  • Incremental rebalancing — only affected consumers stop
  • Consumers not losing/gaining partitions keep consuming
  • Multiple rebalance rounds if needed

Eager vs Cooperative Rebalancing

Eager (StickyAssignor)

Round 1: A, B, C consuming

Consumer D joins
↓
ALL consumers stop  ⏸️⏸️⏸️
↓
Compute assignment
↓
ALL consumers resume ▶️▶️▶️▶️

Downtime: ~seconds for all consumers

Cooperative (CooperativeStickyAssignor)

Round 1: A, B, C consuming

Consumer D joins
↓
Only consumers losing partitions stop  ⏸️
Others keep consuming  ▶️▶️▶️
↓
Compute assignment
↓
Affected consumers resume ▶️

Downtime: ~seconds for affected consumers only

Result: Reduced global consumption pause.

CooperativeSticky Example

Topic: "orders" (6 partitions)
Round 1: A, B, C consuming

Consumer A: Partitions 0, 1
Consumer B: Partitions 2, 3
Consumer C: Partitions 4, 5

Consumer D joins
↓ Rebalance Round 1
Consumer A: Revoke partition 1  ⏸️
Consumer B: Keep consuming  ▶️
Consumer C: Keep consuming  ▶️
Consumer D: (waiting)

↓ Rebalance Round 2
Consumer A: Partitions 0  ▶️
Consumer B: Partitions 2, 3  ▶️
Consumer C: Partitions 4, 5  ▶️
Consumer D: Partition 1  ▶️

Result: Only Consumer A paused briefly, others never stopped!

CooperativeSticky Pros and Cons

Pros:

  • ✅ Minimal disruption — most consumers keep consuming
  • ✅ Sticky behavior (preserves assignments)
  • ✅ Balanced assignments
  • ✅ Recommended for Kafka 2.4+

Cons:

  • ❌ Requires Kafka 2.4+
  • ❌ Multiple rebalance rounds (slightly slower)
  • ❌ All consumers must support it (can't mix with eager)

Use when:

  • Kafka 2.4+
  • Minimizing consumption pauses is critical
  • Default choice for new applications

7. Detailed Restart Scenarios: Side-by-Side Comparison

Why This Matters

Understanding exactly which partitions move during a consumer restart helps you:

  • Predict rebalance impact
  • Choose the right assignor for your use case
  • Design systems that minimize disruption

Let's see concrete examples with 3 consumers, 3 topics, 5 partitions each.

Setup

Consumers: C1, C2, C3
Topics: T1, T2, T3
Partitions per topic: 5 (P0, P1, P2, P3, P4)
Total partitions: 15

Scenario 1: RangeAssignor — Consumer Restart

Initial State (All 3 Consumers Running)

Consumer C1:
├─ T1-P0, T1-P1
├─ T2-P0, T2-P1
└─ T3-P0, T3-P1
Total: 6 partitions

Consumer C2:
├─ T1-P2, T1-P3
├─ T2-P2, T2-P3
└─ T3-P2, T3-P3
Total: 6 partitions

Consumer C3:
├─ T1-P4
├─ T2-P4
└─ T3-P4
Total: 3 partitions

Note: Range assigns per-topic, so C1 and C2 get 2 partitions per topic, C3 gets 1 per topic.

After C2 Restarts (Full Rebalance)

Consumer C1:
├─ T1-P0, T1-P1  ← SAME
├─ T2-P0, T2-P1  ← SAME
└─ T3-P0, T3-P1  ← SAME
Total: 6 partitions (NO CHANGE)

Consumer C2: (restarted)
├─ T1-P2, T1-P3  ← SAME as before
├─ T2-P2, T2-P3  ← SAME as before
└─ T3-P2, T3-P3  ← SAME as before
Total: 6 partitions (SAME assignment)

Consumer C3:
├─ T1-P4  ← SAME
├─ T2-P4  ← SAME
└─ T3-P4  ← SAME
Total: 3 partitions (NO CHANGE)

Result:

  • ✅ Same assignment after rebalance
  • ❌ But full stop-the-world rebalance occurred
  • ⚠️ All 3 consumers stopped consuming during rebalance

Partitions Moved: 0 (assignment same, but all consumers paused)

Scenario 2: RoundRobinAssignor — Consumer Restart

Initial State (All 3 Consumers Running)

Round-robin across ALL 15 partitions:

Consumer C1:
├─ T1-P0, T1-P3
├─ T2-P1, T2-P4
└─ T3-P2
Total: 5 partitions

Consumer C2:
├─ T1-P1, T1-P4
├─ T2-P2
├─ T3-P0, T3-P3
Total: 5 partitions

Consumer C3:
├─ T1-P2
├─ T2-P0, T2-P3
└─ T3-P1, T3-P4
Total: 5 partitions

Note: RoundRobin distributes across all topics/partitions uniformly.

After C2 Restarts (Full Rebalance with Reshuffling!)

Consumer C1:
├─ T1-P0, T1-P3  ← SAME
├─ T2-P1, T2-P4  ← SAME
└─ T3-P2         ← SAME
Total: 5 partitions (NO CHANGE)

Consumer C2: (restarted)
├─ T1-P1, T1-P4  ← SAME
├─ T2-P2         ← SAME
├─ T3-P0, T3-P3  ← SAME
Total: 5 partitions (SAME assignment)

Consumer C3:
├─ T1-P2         ← SAME
├─ T2-P0, T2-P3  ← SAME
└─ T3-P1, T3-P4  ← SAME
Total: 5 partitions (NO CHANGE)

Result:

  • ✅ Same assignment (by luck, in this balanced case)
  • ❌ Full stop-the-world rebalance occurred
  • ⚠️ RoundRobin does NOT guarantee stable assignment on restart

Partitions Moved: 0 (in this case, but not guaranteed)

Important: With unbalanced scenarios or different consumer join order, RoundRobin can completely reshuffle partitions!

Scenario 3: StickyAssignor — Consumer Restart

Initial State (All 3 Consumers Running)

Consumer C1:
├─ T1-P0, T1-P1
├─ T2-P0, T2-P1
└─ T3-P0
Total: 5 partitions

Consumer C2:
├─ T1-P2, T1-P3
├─ T2-P2, T2-P3
└─ T3-P1
Total: 5 partitions

Consumer C3:
├─ T1-P4
├─ T2-P4
└─ T3-P2, T3-P3, T3-P4
Total: 5 partitions

After C2 Restarts (Preserves C1 and C3 Assignments!)

Consumer C1:
├─ T1-P0, T1-P1  ← KEPT
├─ T2-P0, T2-P1  ← KEPT
└─ T3-P0         ← KEPT
Total: 5 partitions (ALL PRESERVED)

Consumer C2: (restarted)
├─ T1-P2, T1-P3  ← SAME as before (Sticky!)
├─ T2-P2, T2-P3  ← SAME as before
└─ T3-P1         ← SAME as before
Total: 5 partitions (STICKY ASSIGNMENT)

Consumer C3:
├─ T1-P4         ← KEPT
├─ T2-P4         ← KEPT
└─ T3-P2, T3-P3, T3-P4  ← KEPT
Total: 5 partitions (ALL PRESERVED)

Result:

  • ✅ All partitions stay with same consumer
  • ✅ Minimal disruption (only C2 rebuilds state)
  • ❌ Still full stop-the-world rebalance

Partitions Moved: 0

Key Insight: StickyAssignor remembers previous assignments and tries to preserve them.

Scenario 4: CooperativeStickyAssignor — Consumer Restart

Initial State (All 3 Consumers Running)

Consumer C1:
├─ T1-P0, T1-P1
├─ T2-P0, T2-P1
└─ T3-P0
Total: 5 partitions

Consumer C2:
├─ T1-P2, T1-P3
├─ T2-P2, T2-P3
└─ T3-P1
Total: 5 partitions

Consumer C3:
├─ T1-P4
├─ T2-P4
└─ T3-P2, T3-P3, T3-P4
Total: 5 partitions

After C2 Restarts (Incremental Rebalance!)

Rebalance Phase 1: C2 rejoins, others keep consuming

Consumer C1:
├─ Keep consuming T1-P0, T1-P1, T2-P0, T2-P1, T3-P0  ▶️
Total: Still consuming all 5 partitions

Consumer C2: (restarted, waiting for assignment)
├─ (none yet)
Total: 0 partitions

Consumer C3:
├─ Keep consuming T1-P4, T2-P4, T3-P2, T3-P3, T3-P4  ▶️
Total: Still consuming all 5 partitions

Rebalance Phase 2: C2 gets its old partitions back

Consumer C1:
├─ T1-P0, T1-P1, T2-P0, T2-P1, T3-P0  ▶️ (Never stopped!)
Total: 5 partitions (NO CHANGE)

Consumer C2: (restarted)
├─ T1-P2, T1-P3, T2-P2, T2-P3, T3-P1  ← Got same partitions back
Total: 5 partitions (SAME as before)

Consumer C3:
├─ T1-P4, T2-P4, T3-P2, T3-P3, T3-P4  ▶️ (Never stopped!)
Total: 5 partitions (NO CHANGE)

Result:

  • ✅ Partitions preserved (sticky behavior)
  • ✅ C1 and C3 never stopped consuming!
  • ✅ Only C2 had to rebuild state
  • ✅ Minimal global disruption

Partitions Moved: 0

Global Consumption Pause: NONE (C1 and C3 kept consuming)

Side-by-Side Summary

Assignor Assignment After Restart Partitions Moved Global Pause State Rebuild
Range Same 0 ✅ Yes (all consumers) All consumers
RoundRobin Same (not guaranteed) 0-15 (varies) ✅ Yes (all consumers) All consumers
Sticky Same (guaranteed) 0 ✅ Yes (all consumers) Only restarted consumer
CooperativeSticky Same (guaranteed) 0 ❌ No (only restarted consumer) Only restarted consumer

Scenario 5: Consumer Leaves (Dies) — Partition Redistribution

Now let's see what happens when a consumer dies (not just restarts).

Initial State: 3 Consumers

C1: 5 partitions
C2: 5 partitions  ← Dies!
C3: 5 partitions

After C2 Dies: 2 Consumers Remain

15 partitions / 2 consumers = 7-8 partitions each
C2's 5 partitions must be redistributed to C1 and C3

RangeAssignor: Recomputes from Scratch

After C2 dies:

Consumer C1:
├─ T1-P0, T1-P1, T1-P2  ← Added T1-P2 (was C2's)
├─ T2-P0, T2-P1, T2-P2  ← Added T2-P2 (was C2's)
└─ T3-P0, T3-P1, T3-P2  ← Added T3-P0, T3-P1, T3-P2
Total: 9 partitions (was 5, now 9)

Consumer C3:
├─ T1-P3, T1-P4  ← Added T1-P3 (was C2's)
├─ T2-P3, T2-P4  ← Added T2-P3 (was C2's)
└─ T3-P3, T3-P4  ← Lost T3-P0, gained T3-P3 (was C2's)
Total: 6 partitions (was 5, now 6)

Partitions Moved: All 15 partitions reassigned (none preserved)

StickyAssignor: Minimizes Movement

After C2 dies:

Consumer C1:
├─ T1-P0, T1-P1  ← KEPT (was C1's)
├─ T2-P0, T2-P1  ← KEPT (was C1's)
├─ T3-P0         ← KEPT (was C1's)
├─ T1-P2, T1-P3  ← ADDED (was C2's)
└─ T2-P2         ← ADDED (was C2's)
Total: 8 partitions (5 kept + 3 added)

Consumer C3:
├─ T1-P4         ← KEPT (was C3's)
├─ T2-P4         ← KEPT (was C3's)
├─ T3-P2, T3-P3, T3-P4  ← KEPT (was C3's)
├─ T2-P3         ← ADDED (was C2's)
└─ T3-P1         ← ADDED (was C2's)
Total: 7 partitions (5 kept + 2 added)

Partitions Moved: Only 5 (C2's partitions), C1 and C3 kept 10 partitions!

Key Insight: Sticky only moves what's necessary.

CooperativeStickyAssignor: Incremental + Minimal Movement

Phase 1: C2 detected as dead, start rebalance
- C1 keeps consuming its 5 partitions  ▶️
- C3 keeps consuming its 5 partitions  ▶️

Phase 2: C2's partitions assigned to C1 and C3
Consumer C1:
├─ Keep all 5 original partitions  ▶️
└─ Add 3 from C2: T1-P2, T1-P3, T2-P2

Consumer C3:
├─ Keep all 5 original partitions  ▶️
└─ Add 2 from C2: T2-P3, T3-P1

Partitions Moved: Only 5 (C2's partitions)

Global Pause: None (C1 and C3 never stopped)

Visual Comparison: Consumer Dies

RangeAssignor:
├─ Pause: ALL consumers stop
├─ Moved: 15 partitions (100%)
└─ Rebuild: ALL consumers rebuild state

StickyAssignor:
├─ Pause: ALL consumers stop
├─ Moved: 5 partitions (33%)
└─ Rebuild: Only consumers with new partitions

CooperativeStickyAssignor:
├─ Pause: NONE (only dead consumer stopped)
├─ Moved: 5 partitions (33%)
└─ Rebuild: Only consumers with new partitions

Real-World Impact

Scenario:

  • 100 partitions, 10 consumers
  • Consumer restart takes 30 seconds (includes rebalance + cache warm-up)
  • 1 consumer dies

Impact:

Assignor Consumers Paused Pause Duration Partitions Rebuilt Rebuild Time
Range 10 (all) 30s 100 (all) 5 minutes
Sticky 10 (all) 30s 10 (only moved) 30s
CooperativeSticky 0 (none) 0s 10 (only moved) 30s

CooperativeSticky Advantage:

  • ✅ 9 consumers never stopped processing
  • ✅ 90% of partitions continued being consumed
  • ✅ 0 seconds global downtime

Key Takeaways

  1. Range & RoundRobin: Recompute from scratch on every rebalance

    • No memory of previous assignment
    • All consumers rebuild state

  2. Sticky: Preserves as much as possible

    • Minimal partition movement
    • But still stops all consumers during rebalance

  3. CooperativeSticky: Best of both worlds

    • Preserves assignments like Sticky
    • No global pause (incremental rebalancing)
    • Recommended for production

8. Comparison Table

Strategy Balance Stickiness Cooperative Co-partitioning Use Case
Range ⚠️ Fair (can be unbalanced) ❌ No ❌ No ✅ Yes Single topic, joins
RoundRobin ✅ Excellent ❌ No ❌ No ❌ No Multiple unrelated topics
Sticky ✅ Good ✅ Excellent ❌ No ⚠️ Partial Stateful consumers
CooperativeSticky ✅ Good ✅ Excellent ✅ Yes ⚠️ Partial Default choice (Kafka 2.4+)

8. Ordering Implications

Partition Assignment Doesn't Affect Ordering

Important: Assignment strategy does NOT affect message ordering.

Ordering guarantee:

  • Messages within a partition are always ordered
  • Assignment strategy only determines which consumer reads which partition
  • Once assigned, order is preserved
Partition 0: [msg1] [msg2] [msg3]  ← Always ordered

Range:      Consumer A reads 0 in order
RoundRobin: Consumer A reads 0 in order
Sticky:     Consumer A reads 0 in order

Assignment doesn't change partition order!

Cross-Partition Ordering

Topic: "orders"
Partition 0: [order-A] [order-B]
Partition 1: [order-C] [order-D]

No guarantee: order-A before order-C

Key Point: If you need global ordering across partitions, you need:

  • Single partition (not scalable)
  • Application-level ordering (timestamps, sequence numbers)

9. Choosing the Right Strategy

Decision Tree

Are you using Kafka 2.4+?
├─ YES → Use CooperativeStickyAssignor
│         (best default choice)
│
└─ NO → Do you have stateful consumers?
         ├─ YES → Use StickyAssignor
         │         (preserve local state)
         │
         └─ NO → Do you need co-partitioning (joins)?
                  ├─ YES → Use RangeAssignor
                  │         (default)
                  │
                  └─ NO → Use RoundRobinAssignor
                            (better balance)

Common Scenarios

Scenario 1: Stateless ETL Pipeline

Recommendation: CooperativeStickyAssignor (or RoundRobinAssignor if < 2.4)

  • No local state to preserve
  • Balance is important
  • Minimal disruption desired

Scenario 2: Kafka Streams Application

Recommendation: CooperativeStickyAssignor (built-in to Kafka Streams)

  • Heavy local state (RocksDB stores)
  • Stickiness critical for performance
  • Incremental rebalancing reduces downtime

Scenario 3: Join Two Co-Partitioned Topics

Recommendation: RangeAssignor

  • Need same partition numbers together
  • Example: orders-0 + shipments-0 → same consumer

Scenario 4: Multiple Unrelated Topics

Recommendation: RoundRobinAssignor (or CooperativeStickyAssignor)

  • No co-partitioning needed
  • Better balance across topics

10. Configuration Example

Setting Assignment Strategy

Properties props = new Properties();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ConsumerConfig.GROUP_ID_CONFIG, "my-group");
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, 
    StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, 
    StringDeserializer.class.getName());

// Set assignment strategy
props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
    CooperativeStickyAssignor.class.getName());

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);

Multiple Strategies (Fallback)

// Prefer CooperativeSticky, fallback to Sticky, then Range
props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
    "org.apache.kafka.clients.consumer.CooperativeStickyAssignor," +
    "org.apache.kafka.clients.consumer.StickyAssignor," +
    "org.apache.kafka.clients.consumer.RangeAssignor");

How it works:

  • All consumers in group negotiate
  • First common strategy is used
  • Allows heterogeneous consumer versions

11. Mental Model Summary

Key Concepts

ASSIGNMENT STRATEGY = How partitions are distributed

Goals:
├─ Balance (even load)
├─ Stickiness (preserve assignments)
└─ Minimal disruption (keep consuming)

Strategies:
├─ Range: Simple, co-partitioning
├─ RoundRobin: Better balance
├─ Sticky: Preserve assignments
└─ CooperativeSticky: Incremental rebalancing ⭐

Questions You Should Be Able to Answer

✅ Why do assignment strategies matter?
✅ What are the four main assignment strategies?
✅ When should you use Range vs RoundRobin?
✅ What does "sticky" mean?
✅ What's the difference between Sticky and CooperativeSticky?
✅ Which strategy should you use for new applications?
✅ Does assignment strategy affect message ordering?

Questions We Haven't Answered Yet

❌ What exactly happens during a rebalance? → Module 4.1
❌ How do I minimize rebalances? → Module 4.1
❌ How are offsets managed? → Module 5

12. What's Next?

Now that you understand how partitions are assigned, the next modules will cover:

  • Module 4.1: The rebalancing process (what happens when assignments change)
  • Module 5: Offset management and delivery semantics
  • Module 6.1: The poll loop internals

Summary

You learned:

  • ✅ Assignment strategies determine partition distribution
  • ✅ Range: Simple, co-partitioning friendly, can be unbalanced
  • ✅ RoundRobin: Better balance, no co-partitioning
  • ✅ Sticky: Preserves assignments across rebalances
  • ✅ CooperativeSticky: Incremental rebalancing (recommended for Kafka 2.4+)
  • ✅ Choose strategy based on statefulness, balance needs, and co-partitioning

Next up: Rebalancing & Group Stability (Module 4.1) — Learn what happens when consumer group membership changes.