selfhostedops/selfhostedops-stack
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Episode 13: correct the CRS exclusion mechanism (ruleRemoveById, not ruleEngine)

Browse source 
Follow-up measurements showed the first exclusion approach was too broad.
ctl:ruleEngine=Off disables the whole Coraza engine for the route, which also
turns off the native virtual-patching rules: a .env probe on such a route
returned 404 (no WAF) instead of 403.

The exclusion now uses ctl:ruleRemoveById=949110,959100 (the CRS inbound and
outbound anomaly-score blockers) so the engine stays on and virtual patching
keeps firing on the excluded routes. Measured: .env / PHPUnit / .git/config
still 403 on those routes, generic SQLi/XSS pass, non-excluded hosts keep full
CRS (27 of 41). Rule order relative to the CRS rule no longer matters.

Also measured and documented: RemoveInBandRuleByName in a pre_eval hook does
nothing to CRS (SQLi stayed 403 globally and per route); a full authenticated
Vaultwarden flow (register, login, sync, cipher create) passes under full CRS
on the vault host. README traps and numbers updated accordingly.
This commit is contained in:
Maximilian Grimm 2026-09-16 21:06:32 +02:00
parent 26a7da0495
commit 22300d7a28
3 changed files with 60 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

58
crowdsec/README.md
View file

@ -141,14 +141,26 @@ SQL, Outline documents with a code block, ntfy publish bodies, n8n webhook
bodies. Six normal workflows, all 403. For a stack where people push git and bodies. Six normal workflows, all 403. For a stack where people push git and
write wiki pages, unusable as-is. write wiki pages, unusable as-is.
The fix is per-route exclusion, not turning CRS off globally. The fix is a per-route exclusion. `appsec-rules/crs-route-exclusions.yaml` is a
`appsec-rules/crs-route-exclusions.yaml` is a small SecLang rule that runs in small SecLang rule that runs in phase 1 and, for exactly the legitimate
phase 1 and sets `ctl:ruleEngine=Off` for exactly the legitimate application application routes (Forgejo git smart-HTTP and REST API, Outline API, ntfy
routes (Forgejo git smart-HTTP and REST API, Outline API, ntfy publish, n8n publish, n8n webhooks), matched by `REQUEST_HEADERS:Host` + `REQUEST_URI`,
webhooks), matched by `REQUEST_HEADERS:Host` + `REQUEST_URI`. `ctl:ruleEngine` removes the CRS anomaly blocking rules with
only affects the Coraza/SecLang engine, so the native virtual-patching and `ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100` (949110 is the CRS
generic rules still run on those routes; only the generic CRS body/URL inbound anomaly-score blocker, 959100 the outbound one). CRS still scores the
inspection is skipped there. request but no longer blocks it on those routes.
Do **not** use `ctl:ruleEngine=Off` for this. It turns the whole Coraza engine
off for the request, and measured on the box that also disables the native
CrowdSec virtual-patching rules: on a route excluded with `ruleEngine=Off`, a
`.env` probe returned 404 (no WAF at all) instead of 403. With
`ctl:ruleRemoveById` the engine stays on, so virtual patching keeps firing:
measured, `.env`, the PHPUnit RCE and `.git/config` still return 403 on the
excluded routes, while generic SQLi/XSS now pass (that is the point).
Rule order does not matter for `ctl:ruleRemoveById` (a phase-1 ctl removes the
phase-2 blocker regardless of file position); the exclusion sits first only for
readability. It would matter for `ruleEngine=Off`, one more reason to avoid it.
`appsec-configs/appsec-tuned.yaml` wires it up: base-config, the exclusion rule `appsec-configs/appsec-tuned.yaml` wires it up: base-config, the exclusion rule
(must come **before** crs so the ctl takes effect in phase 1), crs, then (must come **before** crs so the ctl takes effect in phase 1), crs, then
@ -161,26 +173,26 @@ cp appsec-configs/appsec-tuned.yaml /etc/crowdsec/appsec-configs/
crowdsec -t && systemctl restart crowdsec crowdsec -t && systemctl restart crowdsec
``` ```
Measured with the tuned config: **0 false positives** across all the workflows Measured with the tuned config: **0 false positives** across every workflow
above, still **27 of 41** probes blocked on the non-excluded routes. The cost is above (git clone/push, Forgejo issues and file contents, Outline documents with
precise and worth stating out loud: on an excluded route (say the Forgejo API), an attack-looking code block, ntfy publish, n8n webhooks, and a full
generic SQLi/XSS now passes to the app (virtual patching still covers known authenticated Vaultwarden register/login/sync/cipher-create), still **27 of 41**
CVEs); on a non-excluded host (vault, status) the full CRS applies. Latency probes blocked on the non-excluded routes. The cost is precise: on an excluded
added over vpatch: about +2 ms time-to-first-byte per request. RAM: CRS adds route (say the Forgejo API) generic SQLi/XSS now passes to the app, while
roughly 15 to 40 MB over vpatch (RSS settles near 185 MB, peaks higher right virtual patching still catches known CVEs there, and on a non-excluded host
after loading the rule data). (vault, status) the full CRS applies. Latency added over vpatch: about +2 ms
time-to-first-byte per request. RAM: see the episode notes; CRS settles around
the same order as vpatch once loaded, roughly +15 to 40 MB.
Traps found while tuning: Traps found while tuning:
- `RemoveInBandRuleByName("crowdsecurity/crs")` in a `pre_eval` hook does **not** - `RemoveInBandRuleByName("crowdsecurity/crs")` in a `pre_eval` hook does **not**
disable CRS: the bouncer name does not match the compiled SecLang rules. Use disable CRS: the bouncer name does not match the compiled SecLang rules
`ctl:ruleEngine` inside SecLang instead. (measured, globally and per route, SQLi stayed 403). Use `ctl:` inside SecLang.
- The exclusion rule must be listed before `crowdsecurity/crs` in the config, or Docs: https://docs.crowdsec.net/docs/appsec/hooks
the phase-1 `ctl` runs too late. - `ctl:ruleEngine=Off` is too broad: it disables virtual patching on the route
too. Use `ctl:ruleRemoveById` for the CRS blocking rules and keep the engine on.
- Every new app or new public path needs its own exclusion. A missed one blocks - Every new app or new public path needs its own exclusion. A missed one blocks
a real workflow, and it fails only after login, so it is easy to miss. a real workflow, and it fails only after login, so it is easy to miss.
- Authenticated Vaultwarden and Outline sync with real encrypted payloads were
not exercised against CRS here; test your own before trusting CRS on those
hosts.
**What this repo runs by default: vpatch + generic** (the `appsec-default` **What this repo runs by default: vpatch + generic** (the `appsec-default`
config, zero false positives, no per-app tuning). The CRS files above are the config, zero false positives, no per-app tuning). The CRS files above are the

9
crowdsec/appsec-configs/appsec-tuned.yaml
View file

@ -1,9 +1,10 @@
name: selfhostedops/appsec-tuned name: selfhostedops/appsec-tuned
default_remediation: ban default_remediation: ban
# ep13: vpatch + generic (nativ) plus OWASP CRS (SecLang). Eine lokale # ep13: vpatch + generic (nativ) plus OWASP CRS (SecLang). Die lokale Regel
# SecLang-Regel schaltet die CRS-Engine je legitimer Route ab (ctl:ruleEngine), # crs-route-exclusions entfernt je legitimer Route nur die CRS-Anomalie-
# die native vpatch-Abdeckung bleibt dort erhalten. Die Exclusion-Regel muss # Blockregeln (ctl:ruleRemoveById), die native vpatch-Abdeckung bleibt dort
# VOR crowdsecurity/crs stehen, damit ctl in Phase 1 greift. # erhalten. Reihenfolge zu crowdsecurity/crs ist egal (Phase 1 vor Phase 2),
# der Uebersicht halber steht die Ausnahme oben.
inband_rules: inband_rules:
- crowdsecurity/base-config - crowdsecurity/base-config
- selfhostedops/crs-route-exclusions - selfhostedops/crs-route-exclusions

38
crowdsec/appsec-rules/crs-route-exclusions.yaml
View file

@ -1,26 +1,28 @@
name: selfhostedops/crs-route-exclusions name: selfhostedops/crs-route-exclusions
# ep13: schaltet die CRS-Coraza-Engine (SecLang) fuer genau die legitimen # ep13: entfernt je legitimer Route NUR die CRS-Anomalie-Blockregeln (949110
# Anwendungs-Routen ab, auf denen CRS sonst Fehlalarme wirft. ctl:ruleEngine=Off # inbound, 959100 outbound), sodass das generische CRS dort nicht mehr sperrt.
# betrifft nur die SecLang-Regeln (CRS + base-config), die nativen vpatch- und # Die Coraza-Engine bleibt an (ruleEngine On), deshalb laufen die nativen
# generic-Regeln von CrowdSec laufen weiter, auch auf diesen Routen. # CrowdSec-vpatch-Regeln auf diesen Routen weiter (bekannte CVEs bleiben
# geschuetzt). NICHT ctl:ruleEngine=Off benutzen, das schaltet auch vpatch ab.
# Die Regel muss VOR crowdsecurity/crs stehen (Phase 1).
seclang_rules: seclang_rules:
# Forgejo git smart-HTTP (clone/push), binaerer Pack-Body # Forgejo git smart-HTTP (clone/push), binaerer Pack-Body
- | - |
SecRule REQUEST_HEADERS:Host "@beginsWith git." "id:900101,phase:1,pass,nolog,chain,msg:'ep13 exclude git smart-http'" SecRule REQUEST_HEADERS:Host "@beginsWith git." "id:900101,phase:1,pass,nolog,chain,msg:'ep13 crs-off git smart-http'"
SecRule REQUEST_URI "@rx (git-upload-pack|git-receive-pack|/info/refs\?service=)" "ctl:ruleEngine=Off" SecRule REQUEST_URI "@rx (git-upload-pack|git-receive-pack|/info/refs\?service=)" "ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100"
# Forgejo REST-API: Issues, Datei-Inhalte (Nutzer schreiben Code/SQL/.env) # Forgejo REST-API: Issues, Datei-Inhalte
- | - |
SecRule REQUEST_HEADERS:Host "@beginsWith git." "id:900102,phase:1,pass,nolog,chain,msg:'ep13 exclude forgejo api'" SecRule REQUEST_HEADERS:Host "@beginsWith git." "id:900102,phase:1,pass,nolog,chain,msg:'ep13 crs-off forgejo api'"
SecRule REQUEST_URI "@beginsWith /api/v1/" "ctl:ruleEngine=Off" SecRule REQUEST_URI "@beginsWith /api/v1/" "ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100"
# Outline Dokument-API (Codebloecke) # Outline Dokument-API
- | - |
SecRule REQUEST_HEADERS:Host "@beginsWith wiki." "id:900103,phase:1,pass,nolog,chain,msg:'ep13 exclude outline api'" SecRule REQUEST_HEADERS:Host "@beginsWith wiki." "id:900103,phase:1,pass,nolog,chain,msg:'ep13 crs-off outline api'"
SecRule REQUEST_URI "@beginsWith /api/" "ctl:ruleEngine=Off" SecRule REQUEST_URI "@beginsWith /api/" "ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100"
# ntfy Publish (POST /<topic>, freier Text) # ntfy Publish (POST /<topic>)
- | - |
SecRule REQUEST_HEADERS:Host "@beginsWith ntfy." "id:900104,phase:1,pass,nolog,chain,msg:'ep13 exclude ntfy publish'" SecRule REQUEST_HEADERS:Host "@beginsWith ntfy." "id:900104,phase:1,pass,nolog,chain,msg:'ep13 crs-off ntfy publish'"
SecRule REQUEST_METHOD "@streq POST" "ctl:ruleEngine=Off" SecRule REQUEST_METHOD "@streq POST" "ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100"
# n8n Webhook-Pfade (beliebiges JSON) # n8n Webhook-Pfade
- | - |
SecRule REQUEST_HEADERS:Host "@beginsWith n8n." "id:900105,phase:1,pass,nolog,chain,msg:'ep13 exclude n8n webhook'" SecRule REQUEST_HEADERS:Host "@beginsWith n8n." "id:900105,phase:1,pass,nolog,chain,msg:'ep13 crs-off n8n webhook'"
SecRule REQUEST_URI "@beginsWith /webhook" "ctl:ruleEngine=Off" SecRule REQUEST_URI "@beginsWith /webhook" "ctl:ruleRemoveById=949110,ctl:ruleRemoveById=959100"