Start learning

Kafka: Cluster Defense

Build a Kafka cluster and keep it alive through 36 levels of rising traffic, broker failures and cost ceilings.

Start level 1

First chapter free, no account. Runs in your browser. Pricing

levels
36
chapters
6
exams
6
lines of code
0
Kafka: Cluster Defense Level 09 · Rack blackout · wave running

01 / Why play it

Would you approve this?

An AI will propose each of these with total confidence. Each is a level you can play.

  1. Kafka 02 The topic has two partitions. Consumers were lagging, so I added a third.

    Right.House of cards. Within a group each partition has one owner, so the third consumer sits idle. Play Kafka

  2. Kafka 09 Three replicas, so the data is safe.

    Right.House of cards. Not if they share a rack: a rack fails as one and takes all three copies with it. Play Kafka

  3. Kafka 10 Replication factor 3, min.insync.replicas=2, acks=all.

    Right.This one holds. A write needs two copies before it is acknowledged, and the topic stays writable with one broker down. Play Kafka

02 / Syllabus

36 levels, 6 chapters.

Each level is a service contract. You read what the service owes, build for it, run the traffic and read the debrief. Every chapter ends in an exam that mixes what came before.

Chapter 01

Foundations

Why a broker is a log and not a queue, what an offset is, and why a third consumer does nothing for a topic with two partitions.

  1. 01 First events Kafka stores records; applications remember their position Intro
  2. 02 Parallel lanes Parallel work needs separate partition assignments Experiment
  3. 03 Checkout integration Larger records change the demand on each broker Challenge
  4. 04 Payload delivery Count the work inside each event Challenge
  5. 05 Two streams Separate topics still share physical machines Challenge
  6. 06 Launch-day exam A burst needs capacity to prevent or clear its queues Exam

Chapter 02

Durability

What a replica protects and what it does not. How the ISR and min.insync.replicas decide whether a write is accepted, and why three copies in one rack fail together.

  1. 07 A second copy A copied log protects data without adding a new work lane Intro
  2. 08 Broker evacuation A replacement leader needs the acknowledged history Experiment
  3. 09 Rack blackout Copies in one rack can fail together Challenge
  4. 10 Admission floor Minimum ISR controls whether writes may proceed Challenge
  5. 11 Catch-up race A returning broker must repair its missing log Challenge
  6. 12 Resilience exam Surviving copies and recovery capacity must work together Exam

Chapter 03

Producer engineering

How batching, linger and compression trade latency and CPU for throughput, and how one busy key ends up overloading a single partition.

  1. 13 Small-event swarm Batching shares request overhead across records Intro
  2. 14 Compressible sensors Compression trades computation for fewer bytes Experiment
  3. 15 Opaque payloads Compression only helps as much as the payload allows Challenge
  4. 16 One noisy sensor One busy key can concentrate a whole workload Challenge
  5. 17 Latency contract A smoother stream can still contain delayed events Challenge
  6. 18 Producer exam Different producer pressures need different remedies Exam

Chapter 04

Consumer applications

Why two consumer groups read the same log independently, what a restart replays, and how retention can delete records nobody has read yet.

  1. 19 Two applications Each application keeps its own progress Intro
  2. 20 A slow dashboard Find the application that is falling behind Experiment
  3. 21 Commit checkpoint A restart uses the saved checkpoint Challenge
  4. 22 Replay storm Smaller replay windows still need a recovery budget Challenge
  5. 23 Retention deadline Unread records can expire while an application recovers Challenge
  6. 24 Application exam A restart and a surge compete for the same capacity Exam

Chapter 05

Operations

What a KRaft quorum needs before it can elect a leader, why reading a record never frees its disk, and what a reassignment costs while it runs.

  1. 25 Metadata majority Metadata decisions need controller votes Intro
  2. 26 Two controller outages A majority is measured against configured membership Experiment
  3. 27 Election under siege Elections need both metadata agreement and a data copy Challenge
  4. 28 Long memory Reading history does not release its storage Challenge
  5. 29 Live expansion Moving a log temporarily creates more work Challenge
  6. 30 Operations exam Use the signals to locate the limiting path Exam

Chapter 06

Production

How to size for the loss of a broker, stay under a cost ceiling, and reason about two failures that overlap.

  1. 31 Mixed workload Fit the hardware to the workload placed on it Intro
  2. 32 N-1 capacity Useful capacity must remain after a broker fails Experiment
  3. 33 Cost envelope Serve the workload within the running-cost envelope Challenge
  4. 34 Overlapping failures Overlapping outages test separate protections Challenge
  5. 35 Incident chain Explain the incident sequence rather than one average Challenge
  6. 36 Production graduation A successful system satisfies every declared contract Exam

03 / The model

What the simulation does, and what it skips.

This is a deterministic teaching model based on the Apache Kafka 4.3 documentation. It is not a Apache Kafka server and it does not provision one.

Modelled

  • Leader append, follower fetch, high watermark and acknowledgement are separate steps, so you can watch a write be stored before it is safe.
  • acks=all waits for every in-sync replica, with min.insync.replicas as the admission floor.
  • Each broker has separate CPU, network, disk I/O and storage budgets. Replication and consumer reads spend from the same budgets as writes.
  • Producer batching uses a 64 KB threshold and a linger window. Compression costs CPU at the producer, the broker and the consumer.
  • Consumer groups keep independent committed offsets. A restart rewinds only the group that restarted.
  • Retention removes records on schedule whether or not they were read.
  • Rack incidents take out every broker in the zone. Three or five KRaft controllers tolerate one or two voter losses.

Simplified or left out

  • Transactions, share groups and eligible leader replicas.
  • Log-segment files and the client wire protocol.
  • Permanent disk loss.

Capacities, prices, timings and compression ratios are game units chosen to make the trade-offs visible. They are not benchmarks.

Primary sources

Level 1 takes a few minutes.

Kafka stores records; applications remember their position.

Start level 1

Also in the library: Redis: Voxel Defense