Docker Engine Architecture

Docker Engine is the core runtime that builds and runs containers. Understanding its architecture helps you make better decisions about configuration, performance, and troubleshooting.

Architecture Overview

Docker Engine is a client-server application with these major components:

┌────────────────────────────────────────────────────────────────┐
│                         Docker Client                          │
│                     (docker CLI / API calls)                   │
└──────────────────────────┬─────────────────────────────────────┘
                           │  REST API (Unix socket / TCP)
                           ▼
┌────────────────────────────────────────────────────────────────┐
│                        Docker Daemon                           │
│                          (dockerd)                             │
│                                                                │
│  ┌─────────────┐  ┌──────────────┐  ┌───────────────────┐    │
│  │   Image      │  │  Network     │  │   Volume           │    │
│  │   Management │  │  Management  │  │   Management       │    │
│  └─────────────┘  └──────────────┘  └───────────────────┘    │
│                                                                │
│  ┌─────────────────────────────────────────────────────────┐  │
│  │                    containerd                            │  │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────────────────┐  │  │
│  │  │ Image    │  │ Snapshot │  │   Task Management    │  │  │
│  │  │ Service  │  │ Service  │  │                      │  │  │
│  │  └──────────┘  └──────────┘  └──────────────────────┘  │  │
│  │                                                         │  │
│  │  ┌──────────────────────────────────────────────────┐  │  │
│  │  │                containerd-shim                    │  │  │
│  │  │  ┌──────────────────────────────────────────┐   │  │  │
│  │  │  │                  runc                     │   │  │  │
│  │  │  │  ┌──────────┐  ┌──────────┐             │   │  │  │
│  │  │  │  │Container │  │Container │  ...        │   │  │  │
│  │  │  │  │   1      │  │   2      │             │   │  │  │
│  │  │  │  └──────────┘  └──────────┘             │   │  │  │
│  │  │  └──────────────────────────────────────────┘   │  │  │
│  │  └──────────────────────────────────────────────────┘  │  │
│  └─────────────────────────────────────────────────────────┘  │
└────────────────────────────────────────────────────────────────┘

Core Components

Docker Client

The Docker client (docker) is the primary interface for interacting with Docker. It sends commands to the Docker daemon via the REST API.

# The client communicates with the daemon
docker run nginx          # Client sends request to daemon
docker build -t app .     # Client sends build context to daemon
docker ps                 # Client queries daemon for container list

Communication channels:

Channel	Default	Description
Unix socket	/var/run/docker.sock	Default on Linux/macOS
TCP	tcp://localhost:2376	Remote access (TLS required)
Named pipe	npipe:////./pipe/docker_engine	Default on Windows

# Connect to a remote Docker daemon
export DOCKER_HOST=tcp://192.168.1.10:2376
export DOCKER_TLS_VERIFY=1
docker info

Docker Daemon (dockerd)

The Docker daemon (dockerd) is the background process that manages Docker objects (images, containers, networks, volumes).

# Start the daemon
sudo dockerd

# Start with specific options
sudo dockerd --debug --storage-driver overlay2

# Check daemon status
sudo systemctl status docker

# View daemon logs
sudo journalctl -u docker.service -f

Daemon responsibilities:

• Listening for Docker API requests
• Managing images, containers, networks, and volumes
• Communicating with other daemons (Swarm mode)
• Delegating container operations to containerd

containerd

containerd is the high-level container runtime that manages the complete container lifecycle:

┌─────────────────────────────────────────┐
│              containerd                  │
│                                         │
│  Image pull/push    ──▶  Registry       │
│  Image management   ──▶  Storage        │
│  Container creation ──▶  runc           │
│  Task execution     ──▶  shim           │
│  Snapshot management──▶  Filesystem     │
│  Event monitoring   ──▶  Clients        │
└─────────────────────────────────────────┘

# Interact with containerd directly (advanced)
sudo ctr containers list
sudo ctr images list
sudo ctr tasks list

runc

runc is the low-level OCI-compliant container runtime that creates and runs containers:

• Creates Linux namespaces (pid, net, mnt, user, ipc, uts)
• Configures cgroups for resource limits
• Sets up the root filesystem
• Starts the container process

containerd-shim

The shim process allows containers to survive daemon restarts:

dockerd ──▶ containerd ──▶ containerd-shim ──▶ runc ──▶ container process
                                    │
                          (shim keeps running)
                          (daemon can restart)

Key benefits of the shim:

• Container continues running if dockerd restarts
• Reports container exit status back to containerd
• Manages STDIO and file descriptors
• Enables daemonless containers

Container Lifecycle in Detail

When you run docker run nginx, here's what happens:

1. docker CLI ────────────────────▶ Parse command and options
       │
2.     └──── REST API call ───────▶ POST /containers/create
       │
3. dockerd ──────────────────────▶ Pull image (if needed)
       │                            Prepare container config
       │                            Create container object
       │
4.     └──── gRPC call ──────────▶ containerd: Create container
       │
5. containerd ───────────────────▶ Prepare snapshot (rootfs)
       │                            Create task
       │
6.     └──── Start shim ─────────▶ containerd-shim: Started
       │
7. shim ─────────────────────────▶ Call runc create
       │
8. runc ─────────────────────────▶ Create namespaces
       │                            Set up cgroups
       │                            Mount rootfs
       │                            Start container process
       │
9. Container process running ─────▶ PID 1 inside container

Linux Kernel Features

Docker leverages several Linux kernel features for container isolation:

Namespaces

Namespace	Isolates	Description
pid	Process IDs	Container has its own PID tree
net	Networking	Container has its own network stack
mnt	Mount points	Container has its own filesystem
uts	Hostname	Container has its own hostname
ipc	IPC resources	Container has its own shared memory
user	User IDs	Container can map user IDs
cgroup	Cgroup views	Container has its own cgroup hierarchy

Control Groups (cgroups)

Cgroups limit and account for resource usage:

# View cgroup for a container
docker inspect --format '{{.HostConfig.CgroupParent}}' my-container

# View container resource usage
docker stats my-container

# View cgroup v2 hierarchy
ls /sys/fs/cgroup/system.slice/docker-<container-id>.scope/

Resource	Cgroup Controller	Docker Flag
Memory	memory	--memory, --memory-swap
CPU	cpu, cpuset	--cpus, --cpu-shares
Block I/O	blkio	--blkio-weight
PIDs	pids	--pids-limit

Union Filesystem (OverlayFS)

Docker uses OverlayFS to efficiently layer filesystems:

┌──────────────────────────────────┐
│         Container Layer          │  Read-Write (thin layer)
│         (writable)               │
├──────────────────────────────────┤
│         Image Layer 4            │  Read-Only
├──────────────────────────────────┤
│         Image Layer 3            │  Read-Only
├──────────────────────────────────┤
│         Image Layer 2            │  Read-Only
├──────────────────────────────────┤
│         Image Layer 1            │  Read-Only
│         (base image)             │
└──────────────────────────────────┘

# Check the storage driver in use
docker info | grep "Storage Driver"

# Inspect overlay mounts for a container
docker inspect --format '{{.GraphDriver.Data}}' my-container

Swarm Mode Architecture

When Docker Engine runs in Swarm mode, additional components are activated:

┌───────────────────────────────────────┐
│            Manager Node               │
│                                       │
│  ┌─────────┐  ┌──────────────────┐  │
│  │  Raft    │  │  Orchestrator    │  │
│  │Consensus │  │                  │  │
│  └─────────┘  └──────────────────┘  │
│  ┌─────────┐  ┌──────────────────┐  │
│  │  API     │  │  Scheduler       │  │
│  │  Server  │  │                  │  │
│  └─────────┘  └──────────────────┘  │
│  ┌─────────┐  ┌──────────────────┐  │
│  │Dispatcher│  │  Allocator       │  │
│  │          │  │  (IP, ports)     │  │
│  └─────────┘  └──────────────────┘  │
└───────────────────────────────────────┘

Component	Description
Raft Consensus	Distributed consensus for cluster state
Orchestrator	Reconciles desired state with actual state
Scheduler	Assigns tasks to nodes
Dispatcher	Communicates with worker nodes
Allocator	Allocates IPs, ports, and other resources

Daemon Configuration

The Docker daemon can be configured via /etc/docker/daemon.json:

{
  "debug": false,
  "storage-driver": "overlay2",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "default-runtime": "runc",
  "containerd": "/run/containerd/containerd.sock",
  "dns": ["8.8.8.8", "8.8.4.4"],
  "max-concurrent-downloads": 10,
  "max-concurrent-uploads": 5
}

See the Configuration Management guide for full details.

Next Steps

• CLI Commands Reference — Master Docker CLI commands
• Configuration Management — Configure the Docker daemon
• API Reference — Interact with Docker programmatically
• Docker Networking Guide — Deep dive into Docker networking
• Security Best Practices — Secure the Docker Engine