fivenines
35/39
Lesson 4.3

The millisecond rung: architecture-level latency

What you'll learn

Why the architecture you already built is most of the way to 5 ms — and the three remaining architecture-level costs: hops, encoding, and the quorum.

Building on

The 1.1/2.9 architecture and the 3.2 quorum gate. This rung is won or lost in diagrams, not in code.

Here is the part's most reassuring fact: the biggest latency decision was made in lesson 1.1, for availability reasons, and it was the right one for speed too. Compare the path an order could have taken with the one ours takes:

flowchart TB
    subgraph chatty["The path NOT built — synchronous service chain"]
        a["gateway"] --> b["auth svc"] --> c["account svc"] --> d["risk svc"] --> e["order svc"] --> f["matching svc"]
        f -.responses retrace all five hops.-> a
    end
    subgraph ours["The path Parts 1–3 built — one sequenced hot path"]
        g["gateway"] --> h["sequencer + risk + engine — one process boundary (1.1, 1.9)"]
        h --> i["outputs fan out once"]
    end
    
Hop arithmetic. Each synchronous network hop costs 50–500 µs in the kernel-networking world and multiplies tail risk — the chain is slow whenever any link is slow, so p99s compound across hops. Five hops each 99% fast are jointly slow 5% of the time. The single hot path pays once.

With the shape right, three costs remain at this rung:

  • Encoding. A JSON order — parse, allocate, validate strings, garbage-collect later — costs single-digit microseconds and unpredictable allocation behavior. A fixed-layout binary message (every field at a known offset, prices as scaled integers per 1.3's tick rule) costs nanoseconds: reading it is pointer arithmetic. Real venues' native protocols are all fixed-layout binary; keep JSON for the account APIs (1.1) where humans and web apps live and latency is irrelevant.
  • The quorum round trip. The 3.2 gate — outputs wait for 2-of-3 zone acks — is now the single largest line item: 0.5–2 ms of the 5 ms budget. At this rung, pay it happily; RPO 0 is worth more than the milliseconds to this rung's users. (4.7 revisits when it isn't.)
  • Micro-batching the journal. One quorum round trip per order would cap throughput at a few thousand orders/sec. The sequencer instead groups events that arrive within a tiny window — a few microseconds or until a batch fills — into one journal append. The elegance: under load the batch fills in nanoseconds (no added latency precisely when latency matters), and when idle the bounded wait adds microseconds nobody notices. Batching done right is a free lunch; batching done wrong (fixed timers, unbounded windows) is where p99s go to die — check it with 4.2's histograms.

Budget check for the rung, per 4.2's discipline: gateway parse + validate ~100 µs (kernel networking, binary protocol), hop to sequencer ~100 µs, quorum gate ~1.5 ms, match ~20 µs, publish + feed egress ~300 µs — comfortably inside 5 ms wire-to-wire at p99, with the remainder absorbing load spikes (2.7's bounded queues keep the "absorbing" honest). Everything at this rung was won by counting hops and round trips — no exotic engineering yet. That is the next rung's job.

Key ideas
  • The availability architecture was already the low-latency architecture: one hot path, hops counted.
  • Fixed-layout binary on the trading path; JSON only where humans live.
  • Pay the quorum at this rung; amortize it with bounded micro-batching that costs nothing under load.
Next step

See what actually stuck.

Take the practice scenarios now.