TCP (Transmission Control Protocol) provides a reliable, ordered, byte-stream connection between two applications. Before any data can flow, both sides must agree on initial parameters through a three-way handshake. This handshake synchronizes sequence numbers, confirms both sides are reachable, and establishes the connection state on both hosts.

The Three-Way Handshake

The connection establishment involves three segments, each identified by flags in the TCP header:

Step 1 — SYN (Client to Server): The client sends a TCP segment with the SYN (Synchronize) flag set. This segment carries the client's Initial Sequence Number (ISN) — a 32-bit value that will be the starting point for numbering all bytes the client sends. The segment has no payload. Example: SYN, Seq=1000.

Step 2 — SYN-ACK (Server to Client): The server responds with both the SYN and ACK flags set. The SYN carries the server's own ISN. The ACK acknowledges the client's ISN by setting the acknowledgment number to client's ISN + 1. Example: SYN-ACK, Seq=5000, Ack=1001.

Step 3 — ACK (Client to Server): The client sends a segment with the ACK flag set, acknowledging the server's ISN (Ack = server's ISN + 1). At this point, the connection is ESTABLISHED on both sides. This segment can optionally carry data. Example: ACK, Seq=1001, Ack=5001.

After these three segments, both sides know each other's starting sequence numbers and can begin sending data.

Initial Sequence Numbers (ISN)

The ISN is not zero or one — it is randomized for security. Early TCP implementations used a predictable ISN (incrementing by a fixed amount per connection), which enabled TCP sequence prediction attacks: an attacker could forge packets with guessed sequence numbers, hijacking connections. Modern operating systems generate ISNs using cryptographic random number generators, making prediction computationally infeasible.

Connection Teardown (4-Way Close)

TCP uses a four-way handshake to close a connection gracefully, because each direction is closed independently:

1. FIN — the initiator sends a segment with the FIN flag, signaling "I have no more data to send."
2. ACK — the receiver acknowledges the FIN.
3. FIN — the receiver sends its own FIN when it is also done sending.
4. ACK — the initiator acknowledges the receiver's FIN.

A simultaneous close occurs when both sides send FIN at roughly the same time. Both transition through the CLOSING state and converge to TIME_WAIT.

TIME_WAIT State

After sending the final ACK, the connection enters the TIME_WAIT state for a duration of 2 * MSL (Maximum Segment Lifetime), typically 60-120 seconds. This serves two purposes:

1. Ensures the final ACK is received. If the remote side's FIN was retransmitted (because the ACK was lost), the local side can resend the ACK.
2. Prevents old segments from a prior connection from being confused with a new one. By waiting 2*MSL, all segments from the old connection will have expired.

On busy servers, thousands of connections in TIME_WAIT can consume ports and memory. The SO_REUSEADDR socket option mitigates this by allowing a new connection to bind to a port still in TIME_WAIT.

SYN Flood Attack and SYN Cookies

A SYN flood is a denial-of-service attack where the attacker sends thousands of SYN segments with spoofed source IPs. The server allocates state for each half-open connection (the SYN queue), eventually exhausting memory and preventing legitimate connections.

The defense is SYN cookies: instead of storing state for the half-open connection, the server encodes the connection parameters (ISN, MSS, timestamp) into the SYN-ACK's sequence number using a cryptographic hash. When the client's ACK returns, the server reconstructs the state from the acknowledged sequence number. This allows the server to handle SYN floods without maintaining any per-connection state until the handshake completes.