Summary

On May 25, 2026 at around 10:02 PM (local time), Sonarr and Radarr became completely inaccessible from the home IP (82.XX.XX.XX), returning 403 on every URL including /login. The service was fully operational. Initial suspicion fell on the day’s crowdsec-cf-sync refactor deployment — the real cause was a CrowdSec AppSec heuristic false positive.

Timeline

Root Cause

CrowdSec AppSec maintains an in-memory heuristic state, separate from LAPI decisions. When the browser simultaneously tries to load many resources and receives 302/403 from the upstream application (Sonarr), AppSec interprets the burst of failures as aggressive probing (http-probing) and blocks the source IP.

TL;DR

The LoveIt theme’s typewriter animation (TypeIt) stopped working on the home after adding Mermaid diagrams to posts. Cause: a #id-1 DOM selector shared between both libraries. When Mermaid finds an orphan block in a home summary, its initialization crashes, and the JS init chain stops before reaching TypeIt. Fix: add `

The symptom

I just finished a webhook endpoint in hugo-mcp-proxy that will receive notifications from GitHub on every push to the arleo.eu repo. Clean implementation: HMAC-SHA256, rate limiting, IPAddressAllow GitHub ranges in systemd.

Functional test from an external client:

$ curl -X POST https://mcp-hugo.arleo.eu/webhook/test \
    -H "Content-Type: application/json" \
    -d '{"test": true}'

Response: 403 Forbidden.

Strange. The service is running, my source IP is whitelisted, the HMAC is correct. Why 403?

Server-side investigation

NUC nginx logs:

$ sudo tail -100 /var/log/nginx/mcp-hugo.access.log | grep webhook

Empty. No request reaches nginx.

mcp-oauth-proxy logs:

$ sudo journalctl -u mcp-oauth-proxy -n 100 | grep webhook

Empty too. The request doesn’t even reach the service.

Either it’s blocked by the firewall before nginx (CrowdSec or ufw), or upstream by Cloudflare.

The truth at Cloudflare

The bug

mcp-installer is a bash script I wrote to automate the installation of mcp-oauth-proxy (FastAPI + nginx + systemd) on a new host. Standard workflow: clone, run, done.

Except when re-running the script on an already installed host (e.g. to update the version), I discovered an idempotence bug: all secrets were regenerated.

$ sudo ./install.sh
[+] Generating MCP_TOKEN...
[+] Generating CLIENT_ID...
[+] Generating CLIENT_SECRET...
[+] Generating TOKEN_SECRET...
[+] Writing /etc/mcp-oauth-proxy/.env...

If you already have a .env with tokens in service, the installer overwrites them. All already-registered OAuth clients (Claude.ai in my case) end up with invalid credentials. Connection breaks.

Why it happened

The script used this logic:

MCP_TOKEN=$(openssl rand -hex 32)
CLIENT_ID=$(openssl rand -hex 16)
CLIENT_SECRET=$(openssl rand -hex 32)
TOKEN_SECRET=$(openssl rand -hex 32)

cat > /etc/mcp-oauth-proxy/.env <<EOF
MCP_TOKEN=$MCP_TOKEN
CLIENT_ID=$CLIENT_ID
CLIENT_SECRET=$CLIENT_SECRET
TOKEN_SECRET=$TOKEN_SECRET
EOF

The “generate then write” pattern is correct for first install. But on rerun, it completely ignores the existing .env.

It’s the classic mistake: the author (me) tested the script only on a fresh host. The “reinstall on existing host” mode was never tested.

Bug reproduction

$ sudo ./install.sh
[OK] Installation complete
$ cat /etc/mcp-oauth-proxy/.env | head -1
MCP_TOKEN=a7f3e9d2c4b8a1...

$ sudo ./install.sh
[OK] Installation complete
$ cat /etc/mcp-oauth-proxy/.env | head -1
MCP_TOKEN=8c2f6a1b9e4d7c...   ← DIFFERENT

Reproducible test in 30 seconds. Almost comical I didn’t catch it sooner.

The lesson: “idempotent” is a contract, not a hope

An installation script must be idempotent by default. Meaning: ./install.sh once, twice, five times → system in the same stable state.

I had this contract in my head. I didn’t have it in the code.

The fix: pre-flight checks + explicit force flag

Context

I deployed a Hugo MCP Server (FastAPI, 7 tools) that lets me edit arleo.eu from Claude.ai. Architecture: claude.ai → mcp-oauth-proxy NUC → hugo-mcp-proxy NUC → MCP server VM.