Reading the nines: availability as a budget
How to translate an availability percentage into a downtime budget, and why chaining components eats that budget alarmingly fast.
"99.9% available" is a contract expressed as a percentage of time the system works. Invert it and it becomes something you can design against: a downtime budget.
| Target | Downtime / year | Downtime / month | What recovery must look like |
|---|---|---|---|
| 99.9% (three nines) | 8 h 46 m | 43.8 min | A human can be woken up, diagnose, restart, and replay. A handful of 30–60 minute incidents a year fits. |
| 99.99% (four nines) | 52.6 min | 4.4 min | No time for humans in the loop. Failover must be automatic; maintenance can no longer mean downtime. |
| 99.999% (five nines) | 5.26 min | 26 s | Failover in seconds, invisible to most users. Whole-datacenter loss must be survivable. Change itself becomes the main risk. |
We build a 24/7-capable venue (think a global crypto-style exchange, or a traditional one that refuses to rely on overnight maintenance windows). That makes the math honest: there is no free "market closed" time to hide repairs in.
One more piece of arithmetic shapes everything that follows. Availability composes multiplicatively across components a request must pass through in series:
flowchart LR
A["Gateway 99.95%"] --> B["Risk 99.95%"] --> C["Matching 99.95%"] --> D["Market data 99.95%"]
D --> R["End to end: 0.9995 x 0.9995 x 0.9995 x 0.9995 = 99.80%"]
style R fill:none,stroke-dasharray: 5 5
The reverse is also true and is the engine of Parts 2 and 3: redundancy composes in your favor. Two independent 99.9% replicas fail together only if both are down at once — theoretically 99.9999%. The word doing the heavy lifting is independent: shared disks, shared bugs, and shared operators quietly re-correlate your failures. Much of this course is a war on hidden correlation.
- Each extra nine divides your downtime budget by ten — and changes who can perform recovery (human → machine).
- Serial hops multiply against you; independent replicas multiply for you.
- Hidden correlation is the enemy of redundancy.