Build Your Own PostgreSQL
Processes, memory, and the shared system
What You Will Learn
- Explain why PostgreSQL gives each client connection its own backend process
- Place state in private memory, shared memory, or durable storage according to who needs it and how long it must live
- Distinguish an orderly backend exit from a crash that may have damaged shared state
The illusion of one database
Suppose a checkout service updates an order while a reporting job reads yesterday's totals. Both connect to the same host and port. Both name the same database. From either client's point of view, there is one server with one body of data. That is a useful interface, but it is a poor description of what is running.
The tempting implementation is one long-lived program with one large memory space. Accept a request, do the work, return a result, and repeat. That model is easy to picture. It also hides the first hard question: when several clients overlap, which state belongs to one client's work, and which state must they all observe? PostgreSQL answers by combining separate processes with deliberately shared state.
One connection, one backend
A main server process waits for connections. For each connection, PostgreSQL starts a backend process that acts on that client's behalf. The backend receives commands, performs database work, and returns results. Two clients therefore have two backends, even though both backends present the same database.
A process has a private address space enforced by the operating system. Backend A cannot casually overwrite Backend B's stack, temporary buffers, or bookkeeping. This is a valuable default: local work starts private and must cross an explicit boundary to become shared. It is not the same as giving every backend a private database. The processes are separate workers, not separate owners of the data.
Put state where its lifetime belongs
The useful rule is to classify state by ownership and lifetime. If a value exists only to help one backend finish its current work, it belongs in private memory. Temporary calculations and per-connection bookkeeping fit here. When the connection ends normally, the backend exits and the operating system can reclaim that memory without consulting every other backend.
If several backends must observe or coordinate through the same live state, private memory is the wrong home. PostgreSQL uses shared memory for structures such as the buffer pool and server-wide coordination data. Shared does not mean unprotected: because many processes can reach these structures, access must follow strict synchronization rules. Shared also does not mean durable. Shared memory belongs to the running instance and disappears when that instance stops.
State that must survive the processes themselves belongs in durable files. Data files and WAL files outlive a backend and can outlive the running server. This gives us the central invariant: one backend owns its temporary work, all backends coordinate through one shared system, and durable state does not depend on any process remaining alive.
Process map
Read the map as an ownership model
Follow Client A first. Its connection leads to Backend A; the private memory used along the way remains inside that backend and is therefore not drawn as a shared node. Client B follows the same shape through Backend B. The two paths meet only where the work must become visible or coordinated: shared memory, its buffer pool, and the other shared structures.
The lower half of the map has a different lifetime. Buffers mirror data that ultimately belongs in data files, while the WAL writer moves log data toward WAL files. The checkpointer and vacuum worker have no client of their own in this picture. They are background processes that reach the shared system because their work matters to the server as a whole. The diagram shows ownership and access paths; it does not imply that every arrow has the same synchronization or durability rules.
Why not choose one simpler model?
A single request loop really is simpler. It can be an excellent design for an embedded database or a deliberately serialized workload. For a server, however, a request may wait for storage or for another request. If that request owns the only loop, unrelated clients wait with it. Avoiding concurrent mutation also avoids useful concurrency.
Giving every backend its own copy of the database moves the problem rather than solving it. Reads may be local, but two updates now produce two answers to the question "what is the database?" Reconciliation would require another shared authority, and private caches would duplicate memory. The design eventually recreates shared state, only with less explicit rules.
Threads offer another reasonable tradeoff. Because threads share one address space, communication can be cheaper and sharing is natural. But natural sharing is not always desirable: temporary state is reachable unless the program carefully keeps it private, and a memory error can damage the whole process. PostgreSQL's process model makes privacy the default and sharing an explicit architectural choice. It pays for that choice with process startup cost, connection overhead, and the need to coordinate through shared memory. Neither model removes concurrency; each chooses where its discipline will live.
Clean exits and suspicious crashes
Background work follows the same ownership rule as client work. A checkpointer, vacuum worker, or WAL writer has private memory for its own calculations and reaches shared or durable state only for its assigned server-wide job. Moving this work out of client backends prevents its lifetime from being tied to whichever client happened to trigger it.
Now distinguish two events that are easy to blur together. When a client disconnects cleanly, its backend can release its shared registrations, discard private memory, and exit. Other backends can continue because the departing process followed the coordination rules. An orderly exit is a local lifetime event.
An unexpected backend crash is different. Process isolation protects the private memory of sibling backends, but the failed process may have stopped while changing shared memory. The main server cannot prove that the shared structures are still safe. PostgreSQL therefore treats such a crash as an instance-wide integrity problem: it terminates sibling processes, reestablishes a coherent shared system, and recovers before accepting normal work again. The process boundary limits accidental reach; it does not make shared-state failure local.
Three questions for every piece of state
When placing state in a PostgreSQL-like server, ask three questions: who owns it, who must observe it, and how long must it live? One backend plus a short lifetime points toward private memory. Several processes observing it while the server runs points toward shared memory. Survival across server restarts points toward durable storage. Real mechanisms often cross these boundaries, but each crossing should be deliberate.
That compact test explains the architecture better than a list of process names. Many workers can behave like one database only when private work stays private, shared coordination remains singular, and lasting state outlives every worker.
From a connection to a request
We now have a place for client work to run and a rule for where its state belongs. The next question begins at the connection boundary: how do client and backend establish identity, exchange ordered messages, and agree that the server is ready for more work?