Driver presence and location ingestion
From 1.1: the streaming plane exists for exactly this. From 1.3: hex cells quantize space. From 0.3: pings are 250:1 versus trip requests — this data is huge but each datum is nearly worthless, since it's superseded in 4 seconds.
A first pass might persist every coordinate because durable storage sounds safer than memory. That works only while the failure case is absent. Matching must see only fresh driver presence, while durable ride truth remains independent of rapidly superseded coordinates.
The core design decision: location is ephemeral state, so it lives in memory, not in the system-of-record database. Writing 250 K rows/s into PostgreSQL would melt it for zero benefit — nobody needs last Tuesday's ping to dispatch a car now. The live index is Redis: for each hex cell, the set of online drivers with their latest coordinates, heading, and vehicle tier; each entry expiring after ~15 s so crashed phones vanish from dispatch automatically. The "nearby drivers" query is then: expand rings of cells around the pickup point until enough candidates are found.
flowchart TB
DA["🚗 Driver app — ping every 4 s"] --> WS["Realtime gateway — sticky WebSocket"]
WS --> ING["Location ingest workers — validate, snap to cell"]
ING --> RD[("Redis geo index — cell to driver set, 15 s TTL")]
ING --> Q[["Queue — ping firehose"]]
Q --> ARCH["Archiver — batch to object store for analytics 1.13"]
MATCH["Matching service"] -- "nearby query: ring-expand cells" --> RD
RA["📱 Rider app"] -.-> WS
WS -. "throttled nearby-cars for live map" .-> RA
ING -- "go online / offline" --> PG[("PostgreSQL — shift records only")]
Worked example: one ping's journey
sequenceDiagram
autonumber
participant D as Driver app
participant WS as Realtime gateway
participant I as Ingest worker
participant R as Redis geo index
participant RA as Rider app
D->>WS: ping {lat, lng, heading, ts}
WS->>I: forward with driver id from session
I->>I: validate — reject stale ts, impossible speed
I->>R: update cell membership, reset 15 s TTL
Note over R: driver moved cells — remove from old cell, add to new
RA->>WS: subscribed: nearby cars for map
WS->>R: read cells around rider (throttled to 1/s per rider)
R-->>WS: driver positions
WS-->>RA: animated car positions
This is the one place in the system where we choose at-most-once delivery on purpose. A lost ping costs a 4-second-stale position; an ingestion pipeline with acks, retries, and ordering would cost multiples in capacity and complexity to protect data with a 4-second half-life. Contrast deliberately with payments (1.8), where we'll pay full price for exactly-once effects. Matching delivery guarantees to data value is a core system-design schema — this pair of lessons is its worked contrast.
Don't let the rider map read path fan out to the raw index unthrottled: 5 M riders polling nearby cars can out-traffic the drivers writing them. The realtime gateway throttles map updates to ~1/s per viewer and serves them from the same cell reads dispatch uses.