Somewhere around service fifteen, hand-maintained nginx configs stop
being configuration and become a part-time job. I did the nginx
years: include files, renewal cron jobs, the ritual nginx -t, the
one vhost nobody remembered the purpose of. Traefik’s pitch is that
the proxy should configure itself from what’s actually running, and
it delivers: a service declares its routing as labels on its own
container, the proxy watches Docker, and
https://app.hermes.zt.example.dev exists with a valid certificate
the moment the container starts.
One proxy per host, defined once
Every host runs its own Traefik, but it’s the same Traefik: one
definition in compose/_common/traefik-internal.yml, included by
each host’s infra stack (the manually-deployed layer, post 3). Host
differences ride the variables from post 2: ${INTERNAL_DOMAIN} is
hermes.zt.example.dev on hermes and bender.zt.example.dev on
bender. One file to upgrade, three boxes that can’t drift.
The static config, trimmed to the decisions that matter:
providers:
docker:
endpoint: "tcp://socket-proxy:2375" # not the raw socket, see below
exposedByDefault: false # opt-IN routing
file:
directory: /dynamic # middlewares + shared routes
watch: true
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint: {to: websecure, scheme: https}
websecure:
address: ":443"
http:
tls:
certResolver: dns
domains:
- main: "*.hermes.zt.example.dev"
certificatesResolvers:
dns:
acme:
email: certs@example.dev
storage: /acme/acme.json
dnsChallenge:
provider: cloudflare # reads the scoped INTERNAL token from env
exposedByDefault: false is non-negotiable. Containers get routes
only by asking (traefik.enable=true). The default-on alternative
publishes every half-finished experiment you docker run at 1 a.m.,
and you discover which ones when the SSO middleware isn’t on them.
About that socket-proxy endpoint: Traefik needs Docker API access
to read labels, and the raw socket is root-equivalent on the host. A
small filtering proxy that exposes read-only container info (and
nothing like /containers/create) turns “proxy compromised” from a
host takeover into an information leak. One extra container, much
smaller blast radius.
Wildcard certs with zero open ports
Internal hosts can’t answer Let’s Encrypt’s HTTP-01 challenge. There’s no path from the internet to them, which is the entire design. DNS-01 proves domain control by writing a TXT record through the provider’s API instead, which means real, browser-trusted wildcards on machines with no inbound exposure at all. No self-signed warnings to train your household to click through (a habit they will happily apply to actual attacks someday), no internal CA to distribute to every device.
The entrypoint-level domains: block requests
*.hermes.zt.example.dev once and serves every service on the host
from it. No per-router issuance, no rate-limit roulette when you
deploy eight services in a day.
Check it worked:
docker logs traefik 2>&1 | grep -i 'certificate obtained'
curl -vI https://anything.hermes.zt.example.dev 2>&1 | grep 'issuer'
A service declares itself
services:
app:
image: ...
networks: [proxy]
labels:
- traefik.enable=true
- traefik.http.routers.app.rule=Host(`app.${INTERNAL_DOMAIN}`)
- traefik.http.routers.app.entrypoints=websecure
- traefik.http.routers.app.tls=true
- traefik.http.routers.app.middlewares=sso@file
- traefik.http.services.app.loadbalancer.server.port=8080
Five labels: route, port, auth. That sso@file reference is post 8’s
payoff; the middleware is defined once in the dynamic directory, and
protecting any app forever is that one label. This is also the
argument for a proxy per host instead of one central proxy: the
labels live with the service, the proxy lives with the containers,
and nothing needs to know what’s running on another box.
Three footguns, ranked by hours lost
Footgun 1: Traefik runs Go templating on the ENTIRE dynamic file,
comments included. The file provider renders {{ }} before parsing
YAML, on every byte. I once documented a middleware with a
commented-out example containing {{ env "X" }}. The template engine
executed my comment, failed, and took down every middleware and route
in that file, SSO included. Every protected app on the host returned
500, and the log blamed a “template” I didn’t think of as code.
Rule. Never put
{{ }}inside a dynamic-file comment. Write “env X” in prose. And validate the honest way, render then parse:
python3 -c "
import re, yaml
raw = open('traefik/dynamic/middlewares.yml').read()
yaml.safe_load(re.sub(r'{{[^}]*}}', 'DUMMY', raw)); print('ok')"
Footgun 2: default router priority is the rule’s character
length. Not declaration order. Not specificity. String length. The
day two routers can match one request (a shared host with a path
carve-out, say), the longer rule text wins, silently, and requests
arrive at the wrong backend with no error anywhere. A sprawling
Host(a) || Host(b) || Host(c) beats your surgical
Host(a) && PathPrefix(/api) because it has more characters. Set
explicit priority= on every overlapping router. I treat a missing
priority on overlapping rules as a review-blocking bug.
Footgun 3: mount the dynamic DIRECTORY, not individual files. A
single-file bind mount pins an inode. Editors and git pull replace
files (write-temp, rename-over), so the container keeps watching the
old inode while you edit the new file. No error, no reload, your
changes just don’t exist. Mount the directory and the watcher sees
replacements. Cheap lesson here, half an hour of confusion otherwise.
Milestone: every internal service on every host now gets HTTPS, a real certificate, and optional SSO, for the price of five labels.
Next: the other Traefik, the one the internet actually talks to.