selfhostedops-stack/caddy
Maximilian Grimm 88670f028f Episode 12/13: CrowdSec next to fail2ban, and an AppSec WAF in Caddy
Measured on the demo box against the same 24h of real traffic.

Episode 12 (detection, blocklist, firewall):
- crowdsec/ with install notes and the traps: port 8080 clash, empty first
  blocklist pull, replay forensic-mode dropping events (FIFO workaround),
  ssh-time-based-bf shipping remediation:false, the AppSec collections that
  refuse to start without extra rules.
- firewall bouncer in nftables mode (input + forward), which blocks
  Docker-published ports and survives ufw reload; iptables/DOCKER-USER does not.
- HARDENING.md episode 12 section with the numbers: fail2ban 4 banned IPs vs
  CrowdSec 23 detected / 1 blocked, blocklist overlap 50% of IPs but 19% of
  attempts, and the mode=normal blind spot for key-only root attempts.

Episode 13 (AppSec WAF):
- caddy/Dockerfile now builds the CrowdSec bouncer and AppSec handler alongside
  rate-limit, pinned to v0.14.1.
- Caddyfile imports a crowdsec snippet (bouncer + appsec) on every site except
  vpn; compose passes CROWDSEC_BOUNCER_KEY and host.docker.internal.
- Measured: virtual patching blocks known CVEs only (7 of 41 probes, near-zero
  false positives); OWASP CRS raises that to 27 of 41 but breaks git push and
  wiki editing at default paranoia. Default stays vpatch+generic.

fail2ban config unchanged. No secrets or real domains/IPs in tracked files.
2026-09-16 20:34:18 +02:00
..
Dockerfile Episode 12/13: CrowdSec next to fail2ban, and an AppSec WAF in Caddy 2026-09-16 20:34:18 +02:00
README.md Episode 12/13: CrowdSec next to fail2ban, and an AppSec WAF in Caddy 2026-09-16 20:34:18 +02:00

caddy/ (episode 10, extended in episode 12)

Custom Caddy image with the rate limiting plugin, the CrowdSec bouncer and the CrowdSec AppSec handler compiled in (see crowdsec/README.md).

The stock caddy:2-alpine has no rate_limit directive; it is not a runtime flag, the module must be built into the binary with xcaddy. compose.yml carries both image: and build: ./caddy, so docker compose build caddy produces selfhostedops/caddy:crowdsec locally and docker compose up -d uses it.

Do not build on a small box while the stack is running - a Go build can push a 2 GB machine into OOM and take the live services down with it. Build it elsewhere (a bigger machine, or a throwaway host) and move it over:

docker build -t selfhostedops/caddy:crowdsec ./caddy
docker save selfhostedops/caddy:crowdsec | ssh root@box docker load

Measured in episode 12: the build with all three modules took 60 s on a 12 GB build host, the image is 158 MB (the rate-limit-only image was 154 MB).

The modules are pinned to the release tag (@v0.14.1), not @main as the upstream README suggests, so a rebuild next month gives the same binary.