fivenines
22/38
Lesson 2.6 99.99%

Resilience patterns: timeouts, breakers, shedding, degradation

🎯 Objectives
  • Give every synchronous edge a budget: timeout, bounded jittered retries, circuit breaker, bulkhead.
  • Design overload behavior on purpose: shed by tier, degrade features before failing flows.
🔗 Connect

From 1.14: vendor brownouts and traffic spikes are unhealed. From 2.1: the tier table says who gets sacrificed first. From 1.8's pitfall you've seen degradation once — complete the trip, capture later. This lesson makes that instinct a system-wide policy.

Four mechanical rules on every remaining synchronous edge. Timeouts: every call has one, shorter than the caller's own deadline, so waiting can't stack up the chain. Retries: only on idempotent operations, exponential backoff with jitter, and a retry budget (e.g. retries ≤ 10% of calls) — because a 3× retry storm is how a 30-second brownout becomes a self-inflicted outage. Circuit breakers: after enough failures, stop calling for a while and fail fast to the fallback; half-open probes test recovery. Bulkheads: per-dependency connection/thread pools, so a drowning maps vendor can't consume the threads that payments needs.

stateDiagram-v2
    [*] --> Closed
    Closed --> Open : failure rate over threshold in window
    Open --> HalfOpen : cool-down elapsed
    HalfOpen --> Closed : probe requests succeed
    HalfOpen --> Open : probe fails
    note right of Closed : normal — calls flow, failures counted
    note right of Open : fail fast to fallback — no calls, dependency gets air
    note right of HalfOpen : few real requests test recovery
The circuit breaker state machine — same formalism as trips (1.7) and onboarding (1.2), now applied to a dependency edge. Fail-fast plus fallback converts "hang" into "degrade".

Overload is the other half. When demand exceeds capacity (concert lets out, autoscaler still reacting), the worst response is treating all requests equally — everything times out, goodput hits zero, and retries pile on. Instead: load shedding by tier at the gateway and in each service — under pressure, reject T3 and T2 work with fast, honest errors and keep T0/T1 goodput high; and graceful degradation inside features — the rider map shows fewer animated cars (1.4's throttle tightens), quotes fall back to cached cell ETAs (1.3), surge holds its last-known value (1.5), receipts render late (2.4). Each flow declares its degraded mode in advance; improvising degradation during an incident is how incidents get worse.

Worked example: the maps vendor browns out

sequenceDiagram
    autonumber
    participant R as Riders (5,000 quotes/s)
    participant P as Pricing service
    participant G as Geo facade
    participant B as Breaker (geo to vendor)
    participant V as Maps vendor
    R->>P: quote requests
    P->>G: ETA lookups
    G->>V: cache misses only (~10%)
    V--xG: p99 30 s, errors 40%
    G->>G: timeouts fire at 800 ms — no thread pileup (bulkhead holds)
    B->>B: failure threshold crossed — OPEN
    G->>G: fallback: serve stale cell ETAs + historical speed model
    G-->>P: ETA (degraded, flagged)
    P-->>R: quotes flow — slightly wider ETA ranges shown
    Note over B,V: 30 s later — half-open probe: still failing → stay open
    B->>V: probe succeeds after vendor recovers
    B->>B: CLOSED — fresh ETAs resume
    Note over R,V: rider impact: ETA precision dipped. Ride requests: 100% served. Vendor SLA decoupled from ours.
A dependency failure absorbed as feature degradation instead of flow failure. Compare 1.14's version of this row: "requests hang, manual mitigation."
⚠️ Pitfall

Retries multiply through layers: gateway ×3 → service ×3 → client ×3 = 27 attempts per user action. Retry at one layer (usually the caller closest to the user), budget it, and propagate deadlines so a request abandoned upstream isn't still being lovingly retried downstream.

Next step

See what actually stuck.

Take the practice scenarios now.