Episode 13: correct the CRS exclusion mechanism (ruleRemoveById, not ruleEngine)

Follow-up measurements showed the first exclusion approach was too broad.
ctl:ruleEngine=Off disables the whole Coraza engine for the route, which also
turns off the native virtual-patching rules: a .env probe on such a route
returned 404 (no WAF) instead of 403.

The exclusion now uses ctl:ruleRemoveById=949110,959100 (the CRS inbound and
outbound anomaly-score blockers) so the engine stays on and virtual patching
keeps firing on the excluded routes. Measured: .env / PHPUnit / .git/config
still 403 on those routes, generic SQLi/XSS pass, non-excluded hosts keep full
CRS (27 of 41). Rule order relative to the CRS rule no longer matters.

Also measured and documented: RemoveInBandRuleByName in a pre_eval hook does
nothing to CRS (SQLi stayed 403 globally and per route); a full authenticated
Vaultwarden flow (register, login, sync, cipher create) passes under full CRS
on the vault host. README traps and numbers updated accordingly.

crowdsec/README.md

@@ -141,14 +141,26 @@ SQL, Outline documents with a code block, ntfy publish bodies, n8n webhook
 bodies. Six normal workflows, all 403. For a stack where people push git and
 write wiki pages, unusable as-is.
 
-The fix is per-route exclusion, not turning CRS off globally.
-`appsec-rules/crs-route-exclusions.yaml` is a small SecLang rule that runs in
-phase 1 and sets `ctl:ruleEngine=Off` for exactly the legitimate application
-routes (Forgejo git smart-HTTP and REST API, Outline API, ntfy publish, n8n
-webhooks), matched by `REQUEST_HEADERS:Host` + `REQUEST_URI`. `ctl:ruleEngine`
-only affects the Coraza/SecLang engine, so the native virtual-patching and
-generic rules still run on those routes; only the generic CRS body/URL
-inspection is skipped there.
+The fix is a per-route exclusion. `appsec-rules/crs-route-exclusions.yaml` is a
+small SecLang rule that runs in phase 1 and, for exactly the legitimate
+application routes (Forgejo git smart-HTTP and REST API, Outline API, ntfy
+publish, n8n webhooks), matched by `REQUEST_HEADERS:Host` + `REQUEST_URI`,
+removes the CRS anomaly blocking rules with
+`ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100` (949110 is the CRS
+inbound anomaly-score blocker, 959100 the outbound one). CRS still scores the
+request but no longer blocks it on those routes.
+
+Do **not** use `ctl:ruleEngine=Off` for this. It turns the whole Coraza engine
+off for the request, and measured on the box that also disables the native
+CrowdSec virtual-patching rules: on a route excluded with `ruleEngine=Off`, a
+`.env` probe returned 404 (no WAF at all) instead of 403. With
+`ctl:ruleRemoveById` the engine stays on, so virtual patching keeps firing:
+measured, `.env`, the PHPUnit RCE and `.git/config` still return 403 on the
+excluded routes, while generic SQLi/XSS now pass (that is the point).
+
+Rule order does not matter for `ctl:ruleRemoveById` (a phase-1 ctl removes the
+phase-2 blocker regardless of file position); the exclusion sits first only for
+readability. It would matter for `ruleEngine=Off`, one more reason to avoid it.
 
 `appsec-configs/appsec-tuned.yaml` wires it up: base-config, the exclusion rule
 (must come **before** crs so the ctl takes effect in phase 1), crs, then
@@ -161,26 +173,26 @@ cp appsec-configs/appsec-tuned.yaml /etc/crowdsec/appsec-configs/
 crowdsec -t && systemctl restart crowdsec
 ```
 
-Measured with the tuned config: **0 false positives** across all the workflows
-above, still **27 of 41** probes blocked on the non-excluded routes. The cost is
-precise and worth stating out loud: on an excluded route (say the Forgejo API),
-generic SQLi/XSS now passes to the app (virtual patching still covers known
-CVEs); on a non-excluded host (vault, status) the full CRS applies. Latency
-added over vpatch: about +2 ms time-to-first-byte per request. RAM: CRS adds
-roughly 15 to 40 MB over vpatch (RSS settles near 185 MB, peaks higher right
-after loading the rule data).
+Measured with the tuned config: **0 false positives** across every workflow
+above (git clone/push, Forgejo issues and file contents, Outline documents with
+an attack-looking code block, ntfy publish, n8n webhooks, and a full
+authenticated Vaultwarden register/login/sync/cipher-create), still **27 of 41**
+probes blocked on the non-excluded routes. The cost is precise: on an excluded
+route (say the Forgejo API) generic SQLi/XSS now passes to the app, while
+virtual patching still catches known CVEs there, and on a non-excluded host
+(vault, status) the full CRS applies. Latency added over vpatch: about +2 ms
+time-to-first-byte per request. RAM: see the episode notes; CRS settles around
+the same order as vpatch once loaded, roughly +15 to 40 MB.
 
 Traps found while tuning:
 - `RemoveInBandRuleByName("crowdsecurity/crs")` in a `pre_eval` hook does **not**
-  disable CRS: the bouncer name does not match the compiled SecLang rules. Use
-  `ctl:ruleEngine` inside SecLang instead.
-- The exclusion rule must be listed before `crowdsecurity/crs` in the config, or
-  the phase-1 `ctl` runs too late.
+  disable CRS: the bouncer name does not match the compiled SecLang rules
+  (measured, globally and per route, SQLi stayed 403). Use `ctl:` inside SecLang.
+  Docs: https://docs.crowdsec.net/docs/appsec/hooks
+- `ctl:ruleEngine=Off` is too broad: it disables virtual patching on the route
+  too. Use `ctl:ruleRemoveById` for the CRS blocking rules and keep the engine on.
 - Every new app or new public path needs its own exclusion. A missed one blocks
   a real workflow, and it fails only after login, so it is easy to miss.
-- Authenticated Vaultwarden and Outline sync with real encrypted payloads were
-  not exercised against CRS here; test your own before trusting CRS on those
-  hosts.
 
 **What this repo runs by default: vpatch + generic** (the `appsec-default`
 config, zero false positives, no per-app tuning). The CRS files above are the