Scaling to millions of users
Where each tier's scaling ceiling lies, and the two moves — horizontal edge, partitioned core — that lift them.
One writer per symbol gives deterministic matching and replay. Scaling must preserve that ownership rule instead of weakening it.
A natural first design is to add worker threads inside one order book or make every shard synchronously consult one global balance service. But each symbol has one core owner, core shards share nothing on the hot path, and cross-shard account capacity is allocated or rebalanced asynchronously.
Design targets, so we're scaling toward something concrete:
| Dimension | Target | Scaling tool |
|---|---|---|
| Registered users | 10,000,000 | Ordinary horizontally-scaled web/account services — off the trading path |
| Concurrent sessions | 500,000 | More gateways (stateless, 1.1) |
| Peak order messages | 150,000/sec | Partition the core by symbol (below) |
| Listed symbols | 5,000 | Same partitioning |
| Market data delivery | Millions of endpoints | The 1.8 fan-out tree — already built |
The edge scales by addition. 500k sessions ÷ 20k per gateway ≈ 25 gateways behind load balancers. Nothing new to invent.
The core scales by partitioning on the one boundary that requires no coordination: the symbol. Price-time priority is a per-symbol rule — matching ACME never needs to know anything about a trade in ZETA. So we run several complete sequencer + risk + engine shards, each owning a set of symbols end to end:
flowchart TB
LB["Load balancers"] --> G1["Gateway"] & G2["Gateway"] & G3["Gateway ×25"]
G1 & G2 & G3 --> RT{"Route by symbol"}
RT --> S1 & S2 & S3
subgraph S1["Shard 1 — symbols A–F"]
Q1["Sequencer + risk + engine + journal"]
end
subgraph S2["Shard 2 — symbols G–O"]
Q2["Sequencer + risk + engine + journal"]
end
subgraph S3["Shard 3 — symbols P–Z"]
Q3["Sequencer + risk + engine + journal"]
end
Partitioning has one honest cost: cross-shard account risk. If Alice trades on shards 1 and 3, who owns her balance? Options: pin each account's funds to per-shard sub-balances that a slower background process rebalances (simple, slightly capital-inefficient — our choice at this tier), or coordinate reservations across shards on the hot path (a distributed transaction in the microsecond path — we decline). Note what we did: moved the coordination problem off the latency-critical path rather than solving it there. That maneuver recurs throughout Parts 2 and 3.
GULL traffic spikes by 20 times. If shards share CPU queues or a live balance service, GULL can starve unrelated KELP trading. Full hot-path isolation confines the spike to the owning shard and its explicit allocation.
Edge scales by adding boxes; core scales by symbol partitioning. Shards share nothing on the hot path; account funds rebalance asynchronously. One symbol's throughput = one engine's throughput. That's the hard ceiling.