10.6.4 – Stateful Processing in ksqlDB

Module: Section 10 – Kafka Ecosystem & Tools
Section: 10.6 – Querying & Processing with ksqlDB
Goal: Understand stateful processing, state stores, and materialized views

State Stores

Purpose: Store aggregated state locally (RocksDB)

CREATE TABLE user_totals AS
SELECT
  user_id,
  SUM(amount) AS total_spent
FROM purchases
GROUP BY user_id
EMIT CHANGES;

Result: State store with running totals per user

Pull Queries

Query current state:

SELECT total_spent
FROM user_totals
WHERE user_id = 123;

Response: Current total for user 123 (point-in-time)

Changelog Topics

Automatic state replication:

State store on Server 1 → user_totals-changelog topic → Server 2 (standby)

Fault tolerance: If Server 1 crashes, Server 2 takes over with replicated state

Materialized Views

CREATE TABLE hourly_revenue AS
SELECT
  TIMESTAMPTOSTRING(WINDOWSTART, 'yyyy-MM-dd HH:00') AS hour,
  SUM(amount) AS revenue
FROM orders
WINDOW TUMBLING (SIZE 1 HOUR)
GROUP BY 1
EMIT CHANGES;

Query:

SELECT revenue FROM hourly_revenue WHERE hour = '2025-02-01 14:00';

Key Takeaways

State Stores: RocksDB for local state
Pull Queries: Point-in-time state lookup
Changelog Topics: Automatic state replication
Materialized Views: Queryable aggregations

What's Next?

Next Module: 10.7 – Joins & Enrichment with ksqlDB