The microsecond rung: mechanical sympathy

The kernel is a general-purpose compromise, and the hot path can't afford compromises. Every conventional network receive crosses it: interrupt, kernel-buffer copy, protocol stack, syscall wakeup, scheduler decision, context switch — tens of microseconds each way, with variance from everything else the machine is doing. The sub-millisecond rung removes the kernel from the data plane:

flowchart TB
    subgraph K["Kernel path — fine for the account APIs"]
        K1["NIC raises interrupt"] --> K2["kernel copies into buffers, runs stack"] --> K3["syscall boundary"] --> K4["scheduler wakes app thread — eventually"]
    end
    subgraph B["Bypass path — the trading hot path"]
        B1["NIC writes packet directly into application memory"] --> B2["dedicated core, pinned, busy-polling — never sleeps, never scheduled"] --> B3["message read in place: zero copies, zero syscalls, zero context switches"]
    end
    K4 -."tens of µs, variable".-> OUT1["order enters app"]
    B3 -."single-digit µs, steady".-> OUT2["order enters engine loop"]
    

Then aim the same sympathy at memory. A cache miss to main memory costs ~100 ns — at this rung, a budget item. Three consequences for the engine 1.3 designed:

• The book is arrays, not trees. Textbooks suggest balanced trees for sorted price levels; trees chase pointers, and every hop is a potential cache miss. Real engines exploit domain facts: prices move on a tick grid (1.5) and activity clusters violently near the top of book (1.3) — so a contiguous array of price levels indexed by tick, plus intrusive FIFO queues per level, makes the hot operations (match at best, add near best) touch one or two warm cache lines. The data structure is chosen by the memory hierarchy, not by big-O alone.
• No allocation on the hot path. Every order, level, and message object is pre-allocated in pools at startup. Allocation cost isn't the malloc — it's the page faults, the cache pollution, and (in managed languages) the GC pauses that 4.5 will show ruin the tail.
• Single-threading pays again. The 1.3 decision — one core owns the book — now delivers its third dividend: no locks, no cache lines ping-ponging between cores, and a branch predictor that learns the matching loop cold. The availability design (1.7 determinism) and the latency design keep turning out to be the same design.

Budget at this rung: NIC-to-engine ~2 µs (bypass), risk + match ~1–3 µs (warm cache, no allocation), engine-to-wire ~2 µs, journal quorum on metro-close zones ~200–400 µs — wire-to-wire p99 ≈ 500 µs, and now one line item is 80% of the budget. Stare at that: the quorum. The entire remaining ladder is a negotiation with it, and 4.7 conducts the negotiation.