fivenines
1/25

Build Your Own PostgreSQL

What a database is really promising

Time
6 min
Prerequisites
None

What You Will Learn

  • Understand the promise under pressure in a PostgreSQL-like database
  • Understand three promises, not one feature in a PostgreSQL-like database
  • Understand map of the machine in a PostgreSQL-like database
  • Understand the cheap version and the real one in a PostgreSQL-like database
databasereallypromisingpromisepressurethreepromisesfeaturemapmachine

The promise under pressure

Consider the worst possible moment for a system to improvise. Money has moved, two users press save at once, the power flickers, and a report asks a question nobody imagined when the rows were written. A database begins with a strange promise: hand it your facts, and it will make them feel stable in a world that is not. Your data will have a shape. Your changes will have an order. Your questions will not require rereading everything you have stored.

The easiest way to underestimate a Postgre-like system is to think of it as a file with a nicer interface. A file can hold bytes, but it does not know whether a row is half-written, whether two changes belong together, whether old readers should see new updates, or whether a later query can reuse the structure you stored yesterday. A serious database is closer to a small operating system devoted to data. It owns memory, schedules work, keeps a log of intent, manages names and permissions, and translates human declarations into physical movement.

Three promises, not one feature

The first promise is durability. When the server says a transaction committed, it must mean that the change can survive a crash. This does not require writing every final data page immediately. Doing so would make the system painfully slow. Instead, the database records enough information in a write-ahead log to reconstruct the change later. The permanent state is not just the heap files on disk. It is the combination of data pages, log records, checkpoints, and recovery rules.

The second promise is consistency. People often speak of consistency as if it were a single feature, but it is really a bundle of agreements. Tables have columns and types. Constraints protect relationships. Transactions move the system from one valid state to another. Internal structures such as indexes and catalogs must agree with the rows they describe. A database cannot merely store data; it must maintain a world where stored facts stay interpretable.

The third promise is queryability. Retrieving rows by an exact key is not enough. A relational database accepts a declarative question: describe the result, not the route. The engine must parse the text, bind names to real objects, choose a plan, execute operators, and produce tuples. That pipeline is why SQL is powerful. It separates what the user wants from how the system finds it.

Map of the machine

The cheap version and the real one

Compare two designs. In the first, each query opens a table file, scans bytes, rewrites the file on updates, and returns matching lines. This design is simple. You can build it quickly and understand it in an afternoon. It also falls apart as soon as several users write together, a crash lands between two writes, or a query needs to join large tables. The system has no durable intent, no stable concurrency model, and no independent way to reason about cost.

In the second design, the database treats every user action as a request moving through layers. The session layer owns the conversation. The parser owns syntax. The binder owns meaning. The planner owns possible strategies. The executor owns tuple production. The storage layer owns pages and indexes. The log owns reconstruction. Each layer limits what the next layer must worry about. The executor does not need to understand the wire protocol. The recovery process does not need to parse SQL. This separation is not about aesthetics. It is how the system stays understandable under pressure.

The tradeoff is that every layer introduces machinery. A tiny query now passes through several transformations before it touches a row. That can seem wasteful until the same machinery lets the system handle joins, indexes, transactions, failures, and extensions without becoming a pile of special cases. The complexity pays off because it makes harder behavior composable.

Three truths to protect

One useful mental model is to see the database as protecting three versions of truth at once. There is the truth the user states, written as SQL. There is the truth the system understands, represented by plans, catalogs, snapshots, and locks. There is the truth the machine can preserve, represented by pages, bytes, and log records. Much of database architecture is translation between those truths without losing meaning.

That translation is where Postgre-like design gets interesting. A user says "insert that row" or "find these accounts." The system turns that statement into a transaction, maps names to catalog entries, chooses a physical route, moves data through memory, writes log records before data pages, and eventually presents a result as if the whole thing were straightforward. No single part is mysterious. What matters is that many modest mechanisms cooperate to keep the promise.

Into the server body

The next layer to look at is the living shape of the server. Before SQL can be parsed or data can be written, a client must connect to something, work must be isolated, and memory must be shared without turning into chaos. A Postgre-like database begins as a collection of processes that behave like one machine.

Next step

See what actually stuck.

Take the practice scenarios now.