Back to DAG

OS Kernel

os

The Kernel: Core of the Operating System

The kernel is the central component of an operating system. It runs in privileged mode (ring 0 on x86) with full access to hardware, and it mediates every interaction between user programs and the underlying hardware. When you open a file, allocate memory, or send a network packet, your program asks the kernel to do it on its behalf.

Kernel Responsibilities

The kernel handles four major areas:

  1. Process management: creating, scheduling, and terminating processes. The kernel maintains the process table, performs context switches, and implements scheduling algorithms (round-robin, CFS, etc.).
  2. Memory management: managing virtual address spaces, page tables, and physical frame allocation. The kernel handles page faults, implements demand paging, and manages swap space.
  3. Device drivers: providing a uniform interface to diverse hardware (disks, network cards, GPUs, USB devices). Drivers translate generic OS requests into device-specific commands.
  4. File systems: organizing data on storage devices. The kernel implements file system logic (ext4, XFS, NTFS), the VFS (Virtual File System) abstraction layer, and the page cache for buffering disk I/O.

Monolithic Kernel

A monolithic kernel runs all OS services -- process management, file systems, networking, device drivers -- in a single large binary in kernel space. Linux is the most prominent example.

Advantages: fast communication between subsystems (direct function calls, no message passing overhead), simpler to optimize for performance. Disadvantages: a bug in any driver or subsystem can crash the entire kernel. The codebase is large and complex (Linux is over 30 million lines of code).

Microkernel

A microkernel keeps only the absolute minimum in kernel space: basic scheduling, inter-process communication (IPC), and memory management. Everything else -- file systems, device drivers, networking -- runs as user-space servers that communicate via message passing.

Examples include Mach, QNX, MINIX 3, and seL4. QNX is widely used in safety-critical systems (automotive, medical devices) because a crashed driver can be restarted without rebooting.

Advantages: fault isolation (a crashed driver doesn't take down the kernel), cleaner architecture, easier to formally verify (seL4). Disadvantages: IPC overhead for every operation that crosses address space boundaries, potentially lower throughput.

Hybrid Kernels

Hybrid kernels blend both approaches. Windows NT runs some services (like the window manager) in kernel space for performance while keeping the architecture modular. macOS XNU combines a Mach microkernel with BSD monolithic components in a single kernel address space.

Kernel Modules

Modern monolithic kernels like Linux support loadable kernel modules (LKMs). Device drivers and file systems can be compiled as modules and loaded/unloaded at runtime without rebooting. This gives monolithic kernels some of the flexibility of microkernels. Run lsmod on a Linux system to see currently loaded modules.

Monolithic vs. Microkernel in Practice

Real-World Example

Linux (monolithic): when you plug in a USB drive, the kernel's USB subsystem detects the device, loads the appropriate driver module (usb-storage), and the file system code (e.g., vfat) mounts the volume -- all within kernel space. If the USB driver has a bug that corrupts memory, it can crash the entire system (kernel panic).

QNX (microkernel): the same USB plug-in event is handled by a user-space USB driver server. The file system is another user-space server. They communicate through IPC messages routed by the microkernel. If the USB driver crashes, QNX restarts it automatically. The rest of the system continues running. This is why QNX is trusted in car infotainment systems and nuclear power plant controllers.

macOS XNU (hybrid): XNU's Mach layer provides IPC primitives and virtual memory management. The BSD layer (running in the same address space) provides the POSIX API, file systems, and networking. I/O Kit handles device drivers in a C++ framework. This lets macOS combine Mach's clean abstractions with BSD's mature networking and file system code.

Monolithic vs. Microkernel Architecture

Monolithic (Linux) User Space App 1 App 2 Shell Ring 0 boundary Kernel Space Process Mgmt Memory Mgmt File Systems Networking Device Drivers Scheduler Loadable Kernel Modules Bug in any module can crash whole kernel Microkernel (QNX) User Space App 1 App 2 Shell FS Server Net Server USB Drv Disk Drv GPU Drv Display Servers communicate via IPC Ring 0 boundary Microkernel IPC Scheduler Mem Mgmt Crashed driver restarts safely
Step 1 of 2