fivenines
19/39
Lesson 2.4

Sessions that survive failure

What you'll learn

The client's view of a failover — and the session protocol that turns "did my order go through?" from a support ticket into a solved problem.

Lessons 2.2–2.3 made the core immortal, but a trader whose gateway died mid-order doesn't care: their question is "is my order live or not?" Guessing wrong in either direction costs money — resend and you may be doubled; don't and you may be flat while the market moves. Four nines includes the client experience, so the session protocol must make reconnection provably lossless. Three mechanisms:

  • Client order IDs (idempotency). Every order carries a client-chosen unique ID, and the core treats "new order with an ID it has seen" as a duplicate to be ignored-and-reported, not executed. Resending after any ambiguity becomes safe — the cornerstone.
  • Per-session sequence numbers, both directions. The client numbers its requests; the exchange numbers its reports to that session. Each side always knows exactly what the other has and hasn't seen — the 2.3 gap trick, applied to the private conversation.
  • Session state lives in the core, not the gateway. The gateway is a translator (1.1). Which orders are live, what reports are unacknowledged — all core state, journaled like everything else. Any gateway can therefore resume any session.
sequenceDiagram
    autonumber
    participant T as Trader
    participant GA as Gateway A
    participant GB as Gateway B
    participant C as Core
    T->>GA: order, client ID X-7001, session seq 3502
    GA--xC: gateway A dies — delivery unknown
    T->>GB: reconnect — session token, last report seen: 8804
    GB->>C: resume session, replay reports after 8804
    C->>GB: reports 8805, 8806 — includes ACK of X-7001
    GB->>T: replayed — X-7001 is live
    T->>GB: (had it been missing) resend X-7001 — duplicate-safe
    
Reconnect protocol. The trader learns the exact fate of every in-flight order from the replayed report stream, and can blindly resend anything unacknowledged because client IDs make duplicates inert. Ambiguity is eliminated, not managed.

One policy question remains: what happens to live orders while their owner is disconnected? Per-session choice, declared at logon. Cancel-on-disconnect — the core pulls all your resting orders when your session drops (market makers choose this; unattended quotes are free money for others). Persist — orders remain live (retail default; a phone dropping into a tunnel shouldn't cancel a week-old limit order). Note the pleasing symmetry with 1.2: the ACK is a promise that survives your failure, and now also survives ours.

Key ideas
  • Idempotent client IDs make resends safe; bidirectional sequence numbers make gaps provable.
  • Sessions are core state — any gateway can resume any session.
  • Disconnect policy (cancel vs persist) is the client's declared choice, executed by the core.
Next step

See what actually stuck.

Take the practice scenarios now.