fivenines
12/25

Guided Problem

MiniPG Build 07: Acknowledge Only Durable Writes

Time
30m
Level
advanced
Artifacts
not specified
Progress0%

Build Your Own PostgreSQL

Write-Ahead Logging and the Durability Contract

Time
6 min
Prerequisites
Buffer Pools And The Memory Disk Border

What You Will Learn

  • State the WAL-before-data and WAL-before-commit ordering rules
  • Trace a change from shared buffers through a durable commit record
  • Compare synchronous, asynchronous, and grouped commit promises
WALWAL recordlog sequence numbercommit recordflushgroup commitfull-page imagedurability

The commit promise and why the simple version is too slow

A bookstore confirms an order, then loses power before its modified table page reaches disk. If commit meant 'the page happened to be cached,' the confirmed order could vanish. Requiring every changed page at its final location before success would be safer but would scatter commit-time writes across many files. Write-ahead logging separates the durable promise from later page placement.

The write-ahead log, or WAL, is an ordered stream of records. Each record describes enough about a change to redo it during recovery. There are records for tuple inserts, updates, deletes, page changes, transaction commits, aborts, checkpoints, and many structural operations. The exact format can vary, but the central rule is stable. Before a dirty data page containing a change is flushed, the corresponding WAL must already be durable.

Commit is a log position

Commit is the moment the promise tightens. A transaction can modify buffers in memory, but those modifications are not enough on their own. To commit, the system must ensure that the commit record and all WAL needed by the transaction have reached durable storage according to the configured durability policy. Once that happens, the server can report success even if the data pages themselves remain dirty in memory. If a crash follows, recovery can replay the log.

Commit path

Rendering diagram…

Trace the two ordering rules

The backend changes an orders page in shared buffers and appends a WAL record describing the storage change. The page records the associated log sequence number. It later appends a transaction commit record. Before synchronous commit can be acknowledged, WAL must be flushed through that commit record. The table page may remain dirty because recovery now has a durable recipe for reconstructing the committed effect.

A second rule applies when the buffer manager wants to write that page: WAL through the page's recorded position must reach durable storage first. Otherwise a crash could leave a data page containing a change that recovery's log does not explain. These are ordering invariants, not timing promises. WAL need not wait for the page; the page must wait for WAL, and successful commit must wait for the chosen durability boundary.

The alternatives are painful

Force-at-commit storage pays the durability bill immediately by writing every changed data page before success. That design makes recovery easier because committed data is already in place, but it makes commit latency depend on scattered page writes. At the other extreme, no logging with periodic snapshots can be fast until a crash loses recent commits or leaves pages half-updated. WAL falls between these two designs. It gives sequential durability now and page placement later.

The log has positions, often called log sequence numbers. A page can record the latest WAL position whose change is reflected in that page. When the buffer manager wants to flush the page, it checks whether WAL through that position has been flushed. This creates a precise dependency between page state and log durability. Without it, recovery might see a data page with a change that the log cannot explain, or it might miss a committed change whose page was not written.

WAL records have to be designed with care. Some records can describe logical actions, such as inserting a tuple. Others need lower-level physical detail, especially for page structure changes. Full-page images may be written after checkpoints so recovery can repair a page that was torn by a crash during write. The log is not a list of SQL statements. Replaying SQL would depend on the state of catalogs, functions, snapshots, and the data itself. Recovery needs records that can deterministically restore storage-level effects.

Throughput comes from grouping

Group commit is one of WAL's most useful performance properties. Many backends may commit at around the same time. Rather than each forcing a separate disk flush, the system can flush WAL once through a position that covers several commits. Every transaction whose commit record is included can then be acknowledged. This turns many small synchronous operations into fewer larger ones, which improves throughput while keeping the contract intact.

There are still tradeoffs. Synchronous commit waits for local WAL durability before success. Asynchronous commit may acknowledge earlier, accepting that recent transactions can be lost if the server crashes before WAL reaches disk. Synchronous replication can extend the boundary by waiting for selected standbys to reach configured WAL positions. Durability is a specific promise, not a single universal setting.

WAL also powers more than crash recovery. Replication streams WAL to standbys. Point-in-time recovery replays WAL to a chosen moment. Logical decoding can interpret changes for downstream systems. Backup tools rely on the relationship between base data files and later WAL. Once the database has an ordered change stream, many operational features become possible.

Name the durability boundary

Whenever a system says committed, ask which WAL position was forced, which storage or standby acknowledged it, and which later work remains asynchronous. That vocabulary prevents 'written,' 'flushed,' and 'applied' from collapsing into one vague success. The ordering rule can then be tested directly even when the chosen durability policy deliberately accepts a wider failure window.

Into transactions

The write-ahead log is the database's record of what was intended and what followed. It lets the system tell users that a transaction is safe before the final pages settle. But a transaction is about more than durability. It is also about grouping changes so they appear atomic, isolated, and reversible. That is the next promise to cover.

This is why WAL sits below so many higher-level features. It does not know business meaning, yet it records the storage consequences that make business meaning survive failure. A commit message to the client depends on that lower promise being kept with precision.

When the user hears "done," the log has already made that word recoverable.

Next step

See what actually stuck.

Take the practice scenarios now.