fivenines
36/38
Lesson 3.8 99.999%

Observability v3: probes, client truth, and auto-remediation

🎯 Objectives
  • Measure from outside-in: synthetic probes and client telemetry become the primary SLIs.
  • Wire detection to remediation for the failures whose response is mechanical.
🔗 Connect

From 2.9: burn-rate alerts on edge SLIs. At five nines the math turns brutal: a 26-second monthly budget means a one-minute detection delay is double the budget. And 3.6 created a blind spot: outages the client masks never reach server SLIs at full strength.

Detection moves outside the system. Synthetic probes — headless rider and driver clients running from every region and major mobile network — continuously execute the true T0 journey against production: go online, stream pings, request (a flagged, unmatched test ride in a reserved test city cell), receive offer, transition states. They measure what users experience, including everything server metrics can't see: DNS, TLS, edge routing, app-layer serialization. Client telemetry is the second outside-in source: apps report their own SLIs (time-to-offer, reconnect counts, hedge rates, offline-queue flushes — 3.6's check answered), aggregated with care for battery and privacy. Server-side burn rates (2.9) remain for diagnosis; probes and clients own the pager for T0.

Then close the loop where the response is mechanical. The 2.9 incident machine assumed a human mitigates; at five nines, the failures with known mitigations get auto-remediation: probe failures localized to one cell → auto-drain the cell (3.3 makes this cheap — siblings absorb its cities); localized to one region → auto re-home (3.2); localized to one deploy → auto-rollback (2.8 already does this). Humans are paged simultaneously — to supervise, and to handle the case automation misjudged. The design constraint on every auto-remediator: its worst-case action must be safe (draining a healthy cell = mild capacity loss), it must act on corroborated signals (multiple probes, multiple vantage points — one flaky probe network must not drain cells), and it must rate-limit itself (a remediator in a loop is a new failure mode; the 3.2 damping rule, generalized).

sequenceDiagram
    autonumber
    participant PR as Probes (multi-vantage)
    participant CT as Client telemetry
    participant COR as Correlator
    participant REM as Auto-remediator
    participant CELL as Cell A2
    participant OC as On-call (supervising)
    PR->>COR: cell A2 probes: offer latency 8× baseline, 3 vantage points
    CT->>COR: reconnect spikes from A2's cities — corroborated
    COR->>REM: verdict: cell-local degradation, high confidence
    REM->>REM: guardrails — action safe? rate limit ok? scope bounded?
    REM->>CELL: drain — re-home A2's cities to siblings
    REM->>OC: page: acted, here's why, here's the undo
    Note over CELL: users: ~15 s of reconnects. Budget spent: seconds, not minutes.
    OC->>CELL: diagnose offline — cell out of rotation, no time pressure
Detection to mitigation in seconds, with the human moved from the critical path to the review seat. The diagnosis — the slow, thoughtful part — happens after users are safe, on a drained cell.
⚠️ Pitfall

Auto-remediation inherits the false-positive problem that made 2.3 demand quorum detection — but with bigger levers. Every remediator needs a global concurrency brake: if three "independent" cell-drain decisions fire within minutes, something systemic is being misread (probe infrastructure down? correlator bug?) — freeze, page, and let 2.9's humans take it. The machine handles the failures it was taught; the compound and the novel stay human.

Next step

See what actually stuck.

Take the practice scenarios now.