fivenines
24/39
Lesson 2.9

The four-nines architecture, assembled

What you'll learn

Nothing new — on purpose. This is an integration lesson: the complete 99.99% exchange in one diagram, every functional requirement from lesson 0.2 placed in it, and its availability story told per requirement. CLT calls this schema assembly: eight mechanisms only become one design when you've seen them carry the load together.

Building on

Compare directly against the 1.1 diagram — same functional boxes, different reliability skeleton. Reading it as a diff against 1.1 is the point: your Part 1 schema does the heavy lifting, and only the changes cost you working memory.

flowchart TB
    T["Traders — 500k concurrent sessions"]
    T --> LB["Load balancers"]
    subgraph EDGE["EDGE — stateless, spread ⅓ per zone (2.8), rolling deploys (2.5)"]
        OG["Order gateways ×25 — any gateway resumes any session (2.4)"]
        FS["Feed servers — L1/L2/L3 tiers + conflation (1.8), NAK repair (2.3)"]
        AG["Account and admin APIs"]
    end
    LB --> OG
    T --> AG
    subgraph CORE["TRADING CORE — one symbol shard shown, ×N (1.11), spread across zones A · B · C"]
        EL["Election group ×3 — quorum votes, epoch fencing (2.6)"]
        SEQ["Sequencer leader + pre-trade risk (1.9) + session state (2.4)"]
        ME1["Engine PRIMARY — zone A, publishes"]
        ME2["Engine standby — zone B, suppressed (2.2)"]
        ME3["Engine standby — zone C, suppressed"]
        J[("Journals ×3 zones — quorum append (2.3)")]
        RTX["Retransmit servers (2.3)"]
    end
    OG --> SEQ
    EL -.-> SEQ
    SEQ --> J
    SEQ --> ME1 & ME2 & ME3
    ME1 --> MDP["Market data publisher"] --> FS --> T
    ME1 -- acks and fills --> OG
    RTX -.-> FS
    J --> PT["POST-TRADE — clearing, netting, settlement, surveillance, audit (1.10) — async consumers, pause and catch up"]
    
The four-nines exchange. Same four jobs as lesson 0.1, same tiers as 1.1 — but every box that was red in the 2.1 SPOF audit is now a quorum, a fleet, or a suppressed twin. Trace any functional arrow and there is no longer a single machine whose death interrupts it for more than seconds.

Read the diagram as the 2.1 audit, answered. The sequencer is no longer "one process" but a leadership role that a three-node election group reassigns in under a second, with epochs making deposed leaders harmless (2.6). The engine is three deterministic replicas consuming one stream, exactly one of which publishes (2.2). The journal is three zone-separated replicas acknowledging by quorum (2.3). The gateways were always a fleet (1.1); what's new is that sessions survive any member's death because session truth lives in the core (2.4). The human is gone from the failover path entirely — the 2.8 walk-through demonstrated a zone loss handled in seconds with the on-call merely watching.

Now place every functional requirement from the 0.2 map into this skeleton and ask lesson 2.1's question of each — the table is the whole part in one view:

Functional requirementWhere it livesInstance diesZone dies
Order entry, cancel, modifyGateways → sequenced coreSession resumes on another gateway, replay proves state (2.4)Seconds of pause; reconnect to surviving zones (2.8)
Matching + order lifecycle (1.3–1.5)Engine replicas ×3Standby promoted, fenced, resumes at next event (2.2, 2.6)Same — replicas span zones by design
Pre-trade risk (1.9)In the sequenced path, replicated with itTravels with the engine failover — balances are core stateSame
Auctions, halts, phases (1.6)Events in the logNothing special — replay and replicas carry phase state like any other stateSame
Market data L1/L2/L3 (1.8)Publisher + feed fleetConsumers NAK-repair from retransmit servers (2.3)Feed capacity at ⅔ headroom absorbs it (2.8)
Clearing, settlement, audit (1.10)Async journal consumersPause, restart, catch up — invisible to tradingSame, from surviving journal replicas
Deploys of all of the above (2.5)The redundancy itselfEdge rolls; core upgrades by failover; post-trade pauses — zero scheduled downtime

One honest asterisk survives, and it matters: the engine publishes outputs the instant it processes an event, while journal quorum acknowledgment completes microseconds-to-milliseconds later. There is a hair's-width window in which a published fill could outrun its own durability. At four nines we accept it; closing it is the first act of Part 3 (3.2).

Key ideas
  • Four nines = the 1.1 functional architecture with every SPOF replaced by quorum, fleet, or twin.
  • Every functional requirement now has a per-failure story measured in seconds, humans excluded.
  • Remaining gap: outputs can outrun durability by a hair — Part 3's opening move.
Next step

See what actually stuck.

Take the practice scenarios now.