MiniDock Build 01: Survive a Daemon Upgrade

The Layered Runtime Architecture

Design claim: Runtime responsibilities should be split by lifespan and knowledge so control-plane upgrades do not become workload failures.

Lifetimes force architecture

Running ParcelFlow processes must survive a daemon restart while the engine remains able to collect streams and exit status. Begin with the guarantee visible to a workload or operator. Kernel machinery matters only after a simpler arrangement can no longer supply that guarantee.

The tempting shortcut is straightforward: let the daemon spawn and directly parent every container process. The shortcut is plausible on a quiet development host, where its missing boundary has not yet been tested. A daemon restart would then sever process custody, and a reusable low-level runtime would be entangled with Docker-specific state. The constraint reveals which responsibility must move before the design can survive isolation or process failure.

Split policy, custody, and kernel work

Early Docker was a monolith, and that caused two problems: restarting the daemon killed every container, and nobody else could reuse the container-running machinery. The fix was to split by lifespan and knowledge. The daemon keeps the high-level features (build, API, networks, volumes). A separate container manager (containerd in real Docker) owns container lifecycles. A tiny OCI runtime (runc) knows only how to talk the kernel into creating one container — then exits. And because someone must hold the container's stdio and exit code after runc leaves, a per-container shim process sticks around as the container's foster parent.

The daemon delegates lifecycle management to containerd, a per-container shim retains custody, and an OCI runtime performs kernel setup and exits. The named mechanism is the consequence of that separation: one layer owns policy, another enforces it, and callers receive a stable promise.

Read the topology as a map of authority. Durable knowledge, transient setup, and kernel enforcement should not blur into one box.

  flowchart TB
    CLI["CLI
parses commands"] -- "REST API" --> D["Daemon
images · build · net · volumes"]
    D -- "gRPC" --> CD["Container manager
lifecycle · supervision"]
    CD -- "spawns per container" --> SHIM["Shim
holds stdio + exit code"]
    SHIM -- "invokes" --> RUNC["OCI runtime
creates container, then exits"]
    RUNC -- "prepare isolation + limits, exec" --> K["Kernel"]
    P["Container process"] --- SHIM
    K -. "gives birth to" .-> P
      

It assigns Docker knowledge near the top and kernel-specific work near the bottom. Its central claim is that the component that must outlive an event cannot be owned by a component allowed to disappear during that event; the labels therefore describe authority rather than decorative grouping.

Creation is a relay, not a single call

The strangest part of the chain is that the component that actually creates containers is alive for under a second. The sequence makes the hand-offs visible — notice who remains standing at the end:

The manager creates a shim, the shim invokes the runtime with a bundle, the runtime configures the kernel and executes the workload, then the shim remains. Trace the order carefully; each step establishes a fact the next step is entitled to use.

The second figure tests the same model in motion: it shows the OCI runtime leaving while the shim continues to own the process relationship.

  sequenceDiagram
    participant D as Daemon
    participant CD as Container manager
    participant S as Shim
    participant R as OCI runtime
    participant K as Kernel
    D->>CD: create container (bundle: rootfs + config)
    CD->>S: spawn shim for this container
    S->>R: run
    R->>K: prepare isolated process environment
    R->>K: exec the app process
    R-->>S: done — runtime exits
    note over S: shim now parents the container,
holding stdio and waiting for exit
    S-->>CD: running (process handle)
    CD-->>D: container started
      

It shows the OCI runtime leaving while the shim continues to own the process relationship. The ordering is valid only when it continues to preserve the stated invariant under retries and interruption.

Upgrade the brain without killing the body

The daemon is upgraded while a route worker is active, so the workload and its stream custody must remain intact. A sound boundary either preserves its promise or refuses the operation where the promise becomes impossible.

The component that must outlive an event cannot be owned by a component allowed to disappear during that event. This rule survives changes in implementation names because it describes ownership rather than a particular process tree.

Give MiniDock a lower durable seam

MiniDock needs durable custody below its replaceable API process. MiniDock needs the property now, but its learner must still decide where the responsibility belongs.