Scaling to millions of users
Where each tier's scaling ceiling lies, and the two moves — horizontal edge, partitioned core — that lift them.
The tier split (1.1), the single-threaded engine (1.3), and market data tiering (1.8). Now we size them for real load.
Design targets, so we're scaling toward something concrete:
| Dimension | Target | Scaling tool |
|---|---|---|
| Registered users | 10,000,000 | Ordinary horizontally-scaled web/account services — off the trading path |
| Concurrent sessions | 500,000 | More gateways (stateless, 1.1) |
| Peak order messages | 150,000/sec | Partition the core by symbol (below) |
| Listed symbols | 5,000 | Same partitioning |
| Market data delivery | Millions of endpoints | The 1.8 fan-out tree — already built |
The edge scales by addition. 500k sessions ÷ 20k per gateway ≈ 25 gateways behind load balancers. Nothing new to invent.
The core scales by partitioning on the one boundary that requires no coordination: the symbol. Price-time priority is a per-symbol rule — matching ACME never needs to know anything about a trade in ZETA. So we run several complete sequencer + risk + engine shards, each owning a set of symbols end to end:
flowchart TB
LB["Load balancers"] --> G1["Gateway"] & G2["Gateway"] & G3["Gateway ×25"]
G1 & G2 & G3 --> RT{"Route by symbol"}
RT --> S1 & S2 & S3
subgraph S1["Shard 1 — symbols A–F"]
Q1["Sequencer + risk + engine + journal"]
end
subgraph S2["Shard 2 — symbols G–O"]
Q2["Sequencer + risk + engine + journal"]
end
subgraph S3["Shard 3 — symbols P–Z"]
Q3["Sequencer + risk + engine + journal"]
end
Partitioning has one honest cost: cross-shard account risk. If Alice trades on shards 1 and 3, who owns her balance? Options: pin each account's funds to per-shard sub-balances that a slower background process rebalances (simple, slightly capital-inefficient — our choice at this tier), or coordinate reservations across shards on the hot path (a distributed transaction in the microsecond path — we decline). Note what we did: moved the coordination problem off the latency-critical path rather than solving it there. That maneuver recurs throughout Parts 2 and 3.
- Edge scales by adding boxes; core scales by symbol partitioning.
- Shards share nothing on the hot path; account funds rebalance asynchronously.
- One symbol's throughput = one engine's throughput. That's the hard ceiling.