fivenines
38/40
Lesson 38 · Production

Events, Logs, and Metrics

In one sentence: The engine emits three observability streams with different shapes — an event bus for state changes, pluggable log drivers for app output, and cgroup-derived metrics for resource usage — and consumers subscribe rather than poll.

You already know

  • From Lessons 18, 21, 24: lifecycle transitions, exits, and health flips all surface as events.
  • From Lesson 22: app output already flows through the shim to a log driver.
  • From Lesson 5: cgroups meter every container's CPU and memory — the accounting is free.

Three streams, three questions

Events answer "what happened?" — create, start, die, OOM, health-flip: discrete, structured, low-volume facts on one daemon-wide bus that any client can subscribe to. You've been generating them since Lesson 18; here they get an audience. Logs answer "what did the app say?" — high-volume text owned by the app, routed per-container to a pluggable driver (local files, journald, a remote shipper). Metrics answer "what does it cost?" — numeric time-series the engine reads straight from each container's cgroup, since the kernel was already counting. Keeping the three separate keeps each simple: nobody greps logs to learn a container died — that's an event; nobody emits an event per request — that's the app's logs.

Architecture — MiniDock's observability plumbing
flowchart LR
  subgraph SRC["sources"]
    LC["lifecycle + health
(state changes)"] APP["app stdout/stderr
(via shim)"] CG["cgroup counters
(cpu, mem, io)"] end BUS["event bus"] LD["log driver (pluggable)"] MET["metrics endpoint"] LC --> BUS APP --> LD CG --> MET BUS --> S1["subscriber: CI waiting on a build"] BUS --> S2["subscriber: audit trail"] LD --> S3["log storage / search"] MET --> S4["dashboards + alerts"]
Sequence — one crash, seen by all three streams
sequenceDiagram
  participant C as container
  participant E as engine
  participant B as event bus
  participant L as log driver
  participant M as metrics
  C->>L: "FATAL: out of memory allocating buffer"
  C->>E: exit (OOM-killed — L5)
  E->>B: event: die (id, exit 137, oom=true)
  B-->>B: all subscribers notified at once
  M-->>M: memory series for c-42 flatlines at the ceiling
  note over B,M: the event says WHAT, the log says WHY,
the metric shows IT COMING — you need all three

Anchor these

  • Events = discrete facts (push, subscribe); logs = app's text (pluggable sinks); metrics = kernel's counters.
  • Don't cross the streams: state changes never hide in logs; request chatter never floods the bus.
  • Metrics are nearly free — cgroups were already counting.
Next step

See what actually stuck.

Take the practice scenarios now.