MiniDock Build 12: Publish Content Before Its Name

Push and Dedup

Design claim: Publishing must make content durable before making a manifest visible to readers.

Visibility is the commit point

A client should upload only missing content while ensuring no observer can resolve a manifest whose blobs are absent. Distribution adds concurrency and remote failure to an immutable object graph. The useful question is when another client may safely observe a release.

The tempting shortcut is straightforward: publish the manifest first and fill its blobs afterward. The happy path makes the shortcut appear atomic even though metadata and large byte transfers complete at different times. Readers can observe a visible manifest immediately, and deduplication means some blobs exist while others still require upload. Once a reader or retry can interleave with publication, visibility and durability need an explicit order.

Discover shared content before uploading

Push order is pull order reversed, for the same crash-safety reason: a manifest must never exist on the registry while blobs it references are missing — anyone pulling it would fail. So the engine uploads all blobs first, and the manifest only when every referenced digest is safely stored (registries enforce this: a manifest referencing a missing blob is rejected). Before each upload the engine sends a cheap HEAD: registries are content-addressed too, so if any image ever uploaded that blob — even to another repository — the bytes can be skipped, or linked across repos with a cross-repo mount request.

The client probes blob existence, uploads missing blobs through resumable sessions, then publishes the manifest as the final visibility step. The protocol separates authorization, object identity, transfer, and publication so each response has a precise meaning.

Read the architecture as a trust boundary around metadata and bytes, then identify the one operation that makes a graph visible.

  flowchart TB
    B["next blob in manifest"] --> H{"HEAD: registry
has this digest?"}
    H -- "yes (any repo)" --> SKIP["skip or cross-repo mount
(zero bytes transferred)"]
    H -- no --> UP["chunked upload
POST session + PUT chunks"]
    UP --> V{"registry re-hashes:
digest matches?"}
    V -- yes --> OK["blob stored"]
    V -- no --> REJ["rejected — retry"]
    SKIP --> DONE{"all blobs present?"}
    OK --> DONE
    DONE -- yes --> M["PUT manifest (last!)"]
    style M fill:#f0faf2,stroke:#1f7a34
      

It distinguishes missing and already-present blobs beneath one manifest. Its central claim is that a visible manifest references only blobs the registry can already serve by digest; the labels therefore describe authority rather than decorative grouping.

Close the graph last

Push order is pull order reversed, for the same crash-safety reason: a manifest must never exist on the registry while blobs it references are missing — anyone pulling it would fail. So the engine uploads all blobs first, and the manifest only when every referenced digest is safely stored (registries enforce this: a manifest referencing a missing blob is rejected). Before each upload the engine sends a cheap HEAD: registries are content-addressed too, so if any image ever uploaded that blob — even to another repository — the bytes can be skipped, or linked across repos with a cross-repo mount request.

Existence checks reuse shared layers, upload commits verify digests, and the manifest PUT closes the graph only after all descriptors resolve. At each message, ask what the client and registry can now prove about the object graph.

The second figure tests the same model in motion: it orders existence checks, resumable upload, digest commit, and manifest publication.

  sequenceDiagram
    participant E as Engine
    participant R as Registry
    E->>R: HEAD blobs/sha256:aa (base layer)
    R-->>E: 200 — exists (someone pushed it before)
    E->>R: HEAD blobs/sha256:cc (app layer)
    R-->>E: 404 — missing
    E->>R: POST upload session for cc
    E->>R: PUT chunks … done (digest cc)
    R->>R: re-hash ✓ — stored
    E->>R: PUT manifest (references aa, cc)
    R->>R: verify all referenced blobs exist ✓
    R-->>E: 201 — image published
      

It orders existence checks, resumable upload, digest commit, and manifest publication. The ordering is valid only when it continues to preserve the stated invariant under retries and interruption.

Interrupted transfer must remain invisible

The client loses its connection halfway through a large blob, so it resumes or restarts that blob without exposing an incomplete release. Interruption may leave resumable work, but it must not leave a visible release whose referenced content cannot be served.

A visible manifest references only blobs the registry can already serve by digest. The rule binds remote visibility to verified graph completeness rather than to connection success.

Give MiniDock an ordered publish protocol

MiniDock can publish atomically at the metadata edge while transferring large content incrementally. MiniDock gains a distribution contract while placement remains a design decision.