8.2 Latency vs Throughput Tuning

Module Overview

Goal: Understand the latency/throughput trade-off and optimize for your use case.

What You'll Learn:

• The fundamental latency/throughput trade-off
• Configurations that impact latency
• Configurations that impact throughput
• Tuning for different scenarios
• Measuring and monitoring performance
• Finding the sweet spot for your workload

Prerequisites:

• Module 7.1 (Producer Internals)
• Module 8.1 (Batching & Buffering)

1. The Fundamental Trade-Off

Latency vs Throughput

        Low Latency              High Throughput
            │                          │
            │                          │
   Send immediately           Wait and batch
   Small batches             Large batches
   No compression            Compress data
   acks=1                    acks=all
            │                          │
            ↓                          ↓
   Few msgs per request      Many msgs per request
   More network overhead     Less network overhead

Key Insight: You cannot maximize both simultaneously. Every optimization for one impacts the other.

Why the Trade-Off Exists

Latency optimization requires:

• Sending immediately (no batching delay)
• Small batches (less waiting to fill)
• Minimal processing (no compression)

Throughput optimization requires:

• Batching (introduces delay)
• Large batches (more waiting)
• Compression (CPU overhead, but network savings)

2. Latency-Focused Tuning

Goal: Minimize Time from Send to Broker Acknowledgment

Target: < 10ms p99 latency

Configuration for Low Latency

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

// Latency-optimized settings
props.put("linger.ms", 0);                   // Send immediately
props.put("batch.size", 8192);               // 8 KB (small batches)
props.put("acks", "1");                      // Leader ack only
props.put("compression.type", "none");       // No compression overhead
props.put("buffer.memory", 16777216);        // 16 MB (sufficient)
props.put("max.in.flight.requests.per.connection", 5); // Allow pipelining

Latency Impact of Each Setting

Expected Performance

Configuration: Low latency
Message size: 1 KB
Message rate: 1,000/sec

Results:
  Throughput: ~50 MB/sec
  Latency p50: 3ms
  Latency p99: 8ms
  Requests/sec: ~500 (many small requests)

3. Throughput-Focused Tuning

Goal: Maximize Messages per Second

Target: > 100,000 msgs/sec

Configuration for High Throughput

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

// Throughput-optimized settings
props.put("linger.ms", 20);                  // Wait to batch more
props.put("batch.size", 65536);              // 64 KB (large batches)
props.put("acks", "1");                      // Balance (not "all")
props.put("compression.type", "lz4");        // Efficient compression
props.put("buffer.memory", 67108864);        // 64 MB (absorb spikes)
props.put("max.in.flight.requests.per.connection", 5);

Throughput Impact of Each Setting

Expected Performance

Configuration: High throughput
Message size: 1 KB
Message rate: 100,000/sec

Results:
  Throughput: ~100 MB/sec
  Latency p50: 15ms
  Latency p99: 35ms
  Requests/sec: ~2,000 (large batches)

4. Configuration Impact Analysis

linger.ms Impact

batch.size Impact

Assumes 1 KB average message size

acks Impact

compression.type Impact

Varies by data type (text compresses better)

5. Balanced Configuration

Goal: Optimize for Both Latency and Throughput

Target: < 20ms p99 latency, > 50,000 msgs/sec

Balanced Configuration

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

// Balanced settings
props.put("linger.ms", 10);                  // Small batching delay
props.put("batch.size", 16384);              // 16 KB (default)
props.put("acks", "1");                      // Leader ack
props.put("compression.type", "lz4");        // Efficient compression
props.put("buffer.memory", 33554432);        // 32 MB (default)
props.put("enable.idempotence", "true");     // No duplicates
props.put("max.in.flight.requests.per.connection", 5);

Rationale:

• linger.ms=10: Brief delay for batching without excessive latency
• batch.size=16KB: Default works well for most message sizes
• acks=1: Balance durability and speed
• compression.type=lz4: Fast compression, good ratio
• enable.idempotence=true: Reliability without major overhead

Expected Performance

Configuration: Balanced
Message size: 1 KB
Message rate: 50,000/sec

Results:
  Throughput: ~75 MB/sec
  Latency p50: 8ms
  Latency p99: 18ms
  Requests/sec: ~3,000

6. Scenario-Based Tuning

Scenario 1: Real-Time User Events

Requirements:

• Ultra-low latency (< 5ms p99)
• Moderate volume (5,000 msgs/sec)
• User-facing application

Configuration:

props.put("linger.ms", 0);
props.put("batch.size", 8192);
props.put("acks", "1");
props.put("compression.type", "none");

Trade-off: Accept lower throughput for minimal latency.

Scenario 2: IoT Sensor Data

Requirements:

• High volume (100,000 msgs/sec)
• Small messages (100 bytes)
• Latency acceptable (< 50ms)

Configuration:

props.put("linger.ms", 20);
props.put("batch.size", 32768);        // 32 KB
props.put("acks", "1");
props.put("compression.type", "lz4");

Trade-off: Brief latency for maximum throughput.

Scenario 3: Financial Transactions

Requirements:

• Maximum durability (no data loss)
• Low volume (1,000 msgs/sec)
• Latency acceptable (< 100ms)

Configuration:

props.put("linger.ms", 0);
props.put("batch.size", 16384);
props.put("acks", "all");              // Max durability
props.put("enable.idempotence", "true");
props.put("compression.type", "lz4");

Trade-off: Higher latency for guaranteed delivery.

Scenario 4: Log Aggregation

Requirements:

• Maximum throughput (500,000 msgs/sec)
• Large messages (1 KB average)
• Latency not critical (< 200ms OK)

Configuration:

props.put("linger.ms", 50);
props.put("batch.size", 131072);       // 128 KB
props.put("acks", "1");
props.put("compression.type", "lz4");
props.put("buffer.memory", 134217728); // 128 MB

Trade-off: Higher latency for maximum throughput.

7. Measuring Performance

Key Metrics to Track

1. Latency Metrics

kafka.producer:type=producer-metrics,client-id=<id>,name=request-latency-avg
  → Average request latency

kafka.producer:type=producer-metrics,client-id=<id>,name=request-latency-max
  → Maximum request latency

// Application-side (measure end-to-end)
long start = System.currentTimeMillis();
producer.send(record, (metadata, exception) -> {
    long latency = System.currentTimeMillis() - start;
    // Track latency histogram
});

2. Throughput Metrics

kafka.producer:type=producer-metrics,client-id=<id>,name=record-send-rate
  → Records per second

kafka.producer:type=producer-metrics,client-id=<id>,name=outgoing-byte-rate
  → Bytes per second

3. Efficiency Metrics

kafka.producer:type=producer-metrics,client-id=<id>,name=records-per-request-avg
  → Batching effectiveness

kafka.producer:type=producer-metrics,client-id=<id>,name=compression-rate-avg
  → Compression effectiveness

Benchmarking Example

public class ProducerBenchmark {
    
    public static void main(String[] args) {
        KafkaProducer<String, String> producer = createProducer();
        
        int totalRecords = 100000;
        long startTime = System.currentTimeMillis();
        
        List<Long> latencies = new ArrayList<>();
        
        for (int i = 0; i < totalRecords; i++) {
            long sendStart = System.currentTimeMillis();
            
            producer.send(new ProducerRecord<>("test-topic", "key-" + i, "value-" + i),
                (metadata, exception) -> {
                    long latency = System.currentTimeMillis() - sendStart;
                    latencies.add(latency);
                });
        }
        
        producer.flush();
        long endTime = System.currentTimeMillis();
        
        // Calculate metrics
        long totalTime = endTime - startTime;
        double throughput = (totalRecords * 1000.0) / totalTime;
        
        Collections.sort(latencies);
        long p50 = latencies.get((int) (latencies.size() * 0.50));
        long p99 = latencies.get((int) (latencies.size() * 0.99));
        
        System.out.println("Throughput: " + throughput + " msgs/sec");
        System.out.println("Latency p50: " + p50 + " ms");
        System.out.println("Latency p99: " + p99 + " ms");
        
        producer.close();
    }
}

8. Advanced Techniques

Technique 1: Adaptive Batching

Adjust linger.ms based on load:

public class AdaptiveProducer {
    private KafkaProducer<String, String> producer;
    private AtomicInteger pendingRecords = new AtomicInteger(0);
    
    public void send(ProducerRecord<String, String> record) {
        pendingRecords.incrementAndGet();
        
        producer.send(record, (metadata, exception) -> {
            int pending = pendingRecords.decrementAndGet();
            
            // Adjust batching based on load
            if (pending > 1000) {
                // High load: Increase batching
                adjustLingerMs(20);
            } else if (pending < 100) {
                // Low load: Reduce latency
                adjustLingerMs(0);
            }
        });
    }
    
    private void adjustLingerMs(int value) {
        // Note: Can't change dynamically, requires restart
        // This is conceptual — use separate producers or queues
    }
}

Technique 2: Multiple Producers

Use separate producers for different priority levels:

// Low-latency producer for critical messages
KafkaProducer<String, String> criticalProducer = new KafkaProducer<>(
    latencyOptimizedConfig());

// High-throughput producer for bulk messages
KafkaProducer<String, String> bulkProducer = new KafkaProducer<>(
    throughputOptimizedConfig());

// Route based on priority
if (message.isCritical()) {
    criticalProducer.send(record);
} else {
    bulkProducer.send(record);
}

9. Common Tuning Mistakes

❌ Mistake 1: Over-Optimizing for Latency

props.put("linger.ms", 0);
props.put("batch.size", 1024);  // 1 KB
props.put("compression.type", "none");

Problem: Throughput suffers dramatically (10x worse).

Fix: Accept 5-10ms latency for 2-3x throughput improvement.

❌ Mistake 2: Over-Optimizing for Throughput

props.put("linger.ms", 500);    // 500ms!
props.put("batch.size", 1048576); // 1 MB

Problem: 500ms latency unacceptable for most applications.

Fix: Use 10-50ms linger for good balance.

❌ Mistake 3: Ignoring Workload Characteristics

// Same config for all use cases
props.put("linger.ms", 10);
props.put("batch.size", 16384);

Problem: One size doesn't fit all.

Fix: Tune based on actual requirements (latency SLA, message rate, size).

Summary

Key Takeaways

✅ Latency and throughput are inversely related — optimize for your priority
✅ Low latency: linger.ms=0, small batch.size, no compression
✅ High throughput: linger.ms=10-50ms, large batch.size, compression
✅ Balanced: linger.ms=10, batch.size=16KB, lz4 compression
✅ Measure actual performance — don't rely on assumptions
✅ Tune iteratively — adjust one parameter at a time
✅ Consider workload — message rate, size, latency requirements

Decision Matrix

Priority: Latency → Throughput
          │               │
          ↓               ↓
linger.ms: 0 ───→ 1 ───→ 10 ───→ 50 ───→ 100
batch.size: 8KB ───→ 16KB ───→ 64KB ───→ 128KB
compression: none ───→ lz4 ───→ snappy ───→ zstd
acks: 1 ───→ 1 ───→ 1 ───→ all

Tuning Workflow

1. Define requirements (latency SLA, throughput target)
2. Start with balanced config
3. Measure baseline (latency p50/p99, throughput)
4. Identify bottleneck (latency too high? throughput too low?)
5. Adjust one parameter
6. Measure again
7. Repeat until targets met

What's Next

• Module 8.3: Compression Strategies
  Deep dive into compression types and tuning

• Module 9.1: Practical Producer Patterns
  Apply tuning knowledge to real-world patterns

Previous Module: ← 8.1 - Batching & Buffering
Next Module: 8.3 - Compression Strategies →