Driver presence and location ingestion
- Design the highest-throughput write path in the system: 250 K location pings/s into a queryable geo index.
- Choose the right consistency model for location data — and defend why it's the weakest one in the system.
From 1.1: the streaming plane exists for exactly this. From 1.3: hex cells quantize space. From 0.3: pings are 250:1 versus trip requests — this data is huge but each datum is nearly worthless, since it's superseded in 4 seconds.
The core design decision: location is ephemeral state, so it lives in memory, not in the system-of-record database. Writing 250 K rows/s into PostgreSQL would melt it for zero benefit — nobody needs last Tuesday's ping to dispatch a car now. The live index is Redis: for each hex cell, the set of online drivers with their latest coordinates, heading, and vehicle tier; each entry expiring after ~15 s so crashed phones vanish from dispatch automatically. The "nearby drivers" query is then: expand rings of cells around the pickup point until enough candidates are found.
flowchart TB
DA["🚗 Driver app — ping every 4 s"] --> WS["Realtime gateway — sticky WebSocket"]
WS --> ING["Location ingest workers — validate, snap to cell"]
ING --> RD[("Redis geo index — cell to driver set, 15 s TTL")]
ING --> Q[["Queue — ping firehose"]]
Q --> ARCH["Archiver — batch to object store for analytics 1.13"]
MATCH["Matching service"] -- "nearby query: ring-expand cells" --> RD
RA["📱 Rider app"] -.-> WS
WS -. "throttled nearby-cars for live map" .-> RA
ING -- "go online / offline" --> PG[("PostgreSQL — shift records only")]
Worked example: one ping's journey
sequenceDiagram
autonumber
participant D as Driver app
participant WS as Realtime gateway
participant I as Ingest worker
participant R as Redis geo index
participant RA as Rider app
D->>WS: ping {lat, lng, heading, ts}
WS->>I: forward with driver id from session
I->>I: validate — reject stale ts, impossible speed
I->>R: update cell membership, reset 15 s TTL
Note over R: driver moved cells — remove from old cell, add to new
RA->>WS: subscribed: nearby cars for map
WS->>R: read cells around rider (throttled to 1/s per rider)
R-->>WS: driver positions
WS-->>RA: animated car positions
This is the one place in the system where we choose at-most-once delivery on purpose. A lost ping costs a 4-second-stale position; an ingestion pipeline with acks, retries, and ordering would cost multiples in capacity and complexity to protect data with a 4-second half-life. Contrast deliberately with payments (1.8), where we'll pay full price for exactly-once effects. Matching delivery guarantees to data value is a core system-design schema — this pair of lessons is its worked contrast.
Don't let the rider map read path fan out to the raw index unthrottled: 5 M riders polling nearby cars can out-traffic the drivers writing them. The realtime gateway throttles map updates to ~1/s per viewer and serves them from the same cell reads dispatch uses.