Episode 15: offboarding waits out the Vaultwarden admin rate limit, fail2ban exception only on the provider host

Measured clean run: Pocket ID disable alone left 12 of 14 ways open, the
script (0.9 s, 9 of 9 steps) left 1 of 14.

HARDENING.md

@@ -587,11 +587,12 @@ before the eth0 lines:
 **fail2ban locked out people logging in.** The `caddy-probes` jail counts
 401/403/404. Pocket ID's login page asks `/api/users/me` (401 without a session)
 and the logo (404) on every load. Ten page loads in a minute and the employee is
-banned for an hour. Narrow exception in `filter.d/caddy-probes.conf`:
+banned for an hour. Narrow exception in `filter.d/caddy-probes.conf`, only for
+the provider's own host, so the same paths elsewhere still count:
 
 ```
-ignoreregex = "uri":"/api/users/me"
-              "uri":"/api/application-images/
+ignoreregex = "host":"id\.[^"]*","uri":"/api/users/me"
+              "host":"id\.[^"]*","uri":"/api/application-images/
 ```
 
 **Pocket ID's `/setup` belongs to whoever opens it first.** A scanner fetched

offboarding/README.md

@@ -7,7 +7,7 @@ Measured on the demo box, with a test employee who had the access a real one has
 | --- | --- |
 | Forgejo login prohibited (Forgejo was the SSO provider) | **10 of 13**, unchanged for 45 minutes |
 | Pocket ID account disabled, nothing else | **12 of 14** |
-| `./offboard.py <user>` (about one second) | **1 of 14**: the vault copy on their laptop |
+| `./offboard.py <user>` (0.9 s, all 9 steps ok) | **1 of 14**: the vault copy on their laptop |
 
 What stayed open with only the SSO switched off: the wiki session (valid for
 three months) and its API key, the vault sync (Forgejo kept handing out fresh
@@ -65,7 +65,8 @@ give every key an expiry, and renew them on purpose.
   without a JSON content type is not a 403, it is a 404, and it looks exactly
   like "user not found".
 - **Vaultwarden rate-limits the admin login** (a handful per five minutes). Two
-  test runs back to back and the third one gets 429. The script says so.
+  test runs back to back and the third one gets 429. The script now waits 60 s
+  and retries, up to six times, and prints each wait.
 - **An old n8n API key cannot delete users.** Keys created before scopes
   existed, or with the wrong ones, get 403. Create one with `user:delete`.
 - **Forgejo tokens:** an admin can list a user's tokens (`Sudo:` header) but

offboarding/offboard.py

@@ -125,9 +125,15 @@ def outline(email):
 def vaultwarden(email):
     jar = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()))
     base = f"http://{ip('vaultwarden')}"
+    # The admin login is rate-limited (a few tries per 5 minutes). On 429 wait and retry.
+    for attempt in range(6):
         s, _ = call("POST", f"{base}/admin", {"token": CFG["VAULTWARDEN_ADMIN_TOKEN"]}, form=True, opener=jar)
-    if s != 200:  # 429: the admin login allows only a few tries per 5 minutes
-        return step("vaultwarden", f"admin login refused (HTTP {s}), run again in 5 minutes", False)
+        if s != 429:
+            break
+        step("vaultwarden", f"admin login rate-limited (429), waiting 60 s, try {attempt + 2} of 6", True)
+        time.sleep(60)
+    if s != 200:
+        return step("vaultwarden", f"admin login refused (HTTP {s})", False)
     s, users = call("GET", f"{base}/admin/users", opener=jar)
     hits = [u for u in (users if s == 200 else []) if u["email"] == email]
     if not hits: