Static stability: the data plane outlives the control plane

Split every component's traffic in two. The data plane touches users: routing pings, making offers, committing transitions. The control plane configures the data plane: deploys, flag changes, rate-card updates, autoscaling decisions, lease assignment, the city→home map. The five-nines rule: the data plane must keep serving — at last-known-good configuration — through total control-plane failure. Concretely: every service caches its flags, rate cards, routing maps, and shard leases locally with generous validity; when the flag service, config store, or coordination service goes dark, nothing changes — no deploys, no re-balancing, no surge updates, and no user impact. The system freezes in its last valid shape, which is a perfectly good shape, and pages humans to fix the control plane at T2 urgency.

Why this ranks so high: mature systems rarely die of hardware — they die of their own machinery. The config push that NULLs a routing table, the coordination service election storm that revokes every lease at once, the autoscaler that scales to zero on bad metrics. Static stability inverts the dependency: control planes propose, data planes adopt — validated, versioned, rollback-able — and adoption failure means keep-what-you-have. Note the deliberate echo of CLT: just as a lesson shouldn't collapse when one prerequisite is shaky, the data plane shouldn't collapse when its instructor goes silent. Both are designs for graceful degradation of guidance.

flowchart TB
  subgraph CP ["Control plane — may fail for hours"]
    DEP["Deploy pipeline"]
    FLAG["Flag service"]
    CFG["Config and rate cards"]
    COORD["Coordination — leases, home map"]
  end
  subgraph DP ["Data plane — must never stop"]
    GWD["Gateways"]
    DISPD["Dispatch"]
    TRIPD["Trip state"]
    LOCD["Location"]
    CACHE[("Local last-known-good: flags, rates, maps, leases — versioned, validated")]
  end
  CP -- "propose new config (signed, versioned)" --> CACHE
  CACHE -- "validate, adopt or keep current" --> DP
  CPX["💥 Entire control plane dark"] -.-> CP
  DP --> OK["Users unaffected — system frozen in last good shape, humans paged at leisure"]