Infrastructure Documentation

In short

Runbook for arleo.eu’s infrastructure: security architecture, sync scripts, OpenResty configuration, and the Hugo deployment pipeline.

Scope: an Intel NUC running Linux (host), a KVM VM for site generation, Cloudflare in front. Prerequisites to reproduce: root access on the host, a Cloudflare account with an API token, a CrowdSec account. Not covered here: initial installation of CrowdSec and Cloudflare (see each tool’s official docs) and secrets, stored outside the repo.

Each section is self-contained: file paths, systemd services, and verification commands are given at the top of the section.

1. Security Architecture (10 layers)

The local WAF migrated from ModSecurity to CrowdSec AppSec on OpenResty: see CrowdSec AppSec + OpenResty for the motivation and bouncer configuration.

2. Security Automation Go — CrowdSec ↔ Cloudflare orchestration

Repository: github.com/jmrGrav/security-automation-go (Apache-2.0, active)

Downloadable, ready-to-use script examples (including the full Python history) are in Infrastructure Scripts.

Replaces the former Python script crowdsec-cf-sync.py (and its V2/V3 variants) with a modular Go daemon, with a built-in web operator console. The Python → Go cutover was done service by service, in parallel (the old service kept running while the new one was validated), then disabled once behavior was confirmed equivalent in production.

How it works

The core is a 5-stage pipeline (internal/orchestrator/pipeline): admission → discovery → planning → execution → reporting. Each stage is an independently testable component, with typed clients for Cloudflare (REST + GraphQL), CrowdSec (LAPI/cscli), and AbuseIPDB.

Notable design points:

• Auto-ban requires corroboration: an AbuseIPDB confidence score of 100 alone is no longer enough to trigger a ban — it now also requires a corroborating local signal (a WAF/HTTP-error burst actually observed on this server). This avoids banning purely on an IP’s external reputation when it’s never been seen here.
• Trusted Networks registry: a single source of truth for trusted ASNs/CIDRs (legitimate bots, known ranges), consumed by every ban/report decision — no more exception logic scattered across the script.
• Cloudflare ban lifecycle: bans expire and are lifted automatically (no more manually cleaning up stale CF rules).
• Encrypted SQLite state (runtime.db, AES-GCM), replacing the old version’s flat JSON files (recidivists.json, cidr-banned.json, etc.) — one transactional store instead.
• Cloudflare WAF replay: the daemon periodically polls Cloudflare’s WAF event history (GraphQL), classifies each hit (protected target / benign signal / suspect), and only reports/bans what crosses the configured thresholds.

Active services

Operator console: http://127.0.0.1:9091 (loopback only — never exposed publicly). SOC dashboard, live event timeline, “Providers” view (integrations + Trusted Networks), per-IP investigation (“Focus Incident”), audit notes. Password authentication (bcrypt, first-run setup wizard); credentials and secrets encrypted at rest, never stored in plaintext. Prometheus metrics: http://127.0.0.1:9092/metrics (loopback only).

Check

# Service status
systemctl status cf-sync
systemctl status cf-allowlist-sync.timer

# Real-time logs (journald — no more flat /var/log/crowdsec/cf-sync.log file)
journalctl -fu cf-sync
journalctl -fu cf-allowlist-sync

# Built-in diagnostics (config, CrowdSec/Cloudflare connectivity, DB state)
cf-sync -mode doctor
cf-sync -mode status

Former Python version — historical, retired

Service stopped since June 9, 2026 (crowdsec-cf-sync.service, disabled/inactive). The files remain on disk for reference — do not re-enable or edit them:

The JSON state files they produced (/var/log/crowdsec/recidivists.json, cidr-banned.json, modsec-banned.json, abuseipdb-reported.json) are no longer updated; state now lives in runtime.db (SQLite).

3. CrowdSec Allowlist

Name: my_allowlist Sync: cf-allowlist-sync (systemd timer, every 15 min — see section 2) Sources: BetterStack IPs + Cloudflare IPv4/IPv6 ranges

This allowlist is the single source of truth: it’s pushed to Cloudflare, never edited on the Cloudflare side — any change made there gets overwritten.

# Inspect contents
cscli allowlists inspect my_allowlist

# Count entries
cscli allowlists list

4. OpenResty — CSP (Content Security Policy)

File: /usr/local/openresty/nginx/conf/nginx.conf ($csp_header map)

The public CSP has run on dynamic nonces since v2. See From 46 hashes to zero for the migration details and why hashes were dropped.

httpoxy fix (April 2026)

httpoxy is a CGI/FastCGI vulnerability (CVE-2016-5385 and related CVEs): a client-supplied Proxy: header gets mapped by the server into the HTTP_PROXY environment variable, which many application-side HTTP libraries then trust as their outbound proxy setting — opening the door to a man-in-the-middle attack. The fix strips that variable before it reaches the application.

Added in /usr/local/openresty/nginx/conf/fastcgi.conf:

fastcgi_param  HTTP_PROXY  "";

5. Hugo CMS

Root: /home/jm/hugo-site/ (KVM VM 192.168.122.69) Theme: LoveIt (git submodule) Build: hugo --minify via deploy.sh Deployment: rsync to /var/www/hugo/ on the NUC host

Important files

Publishing a change

# 1. Check the render locally
cd ~/hugo-site && hugo server -D --bind 0.0.0.0

# 2. Control build
hugo --minify

# 3. Full deployment (build + rsync + CF purge)
~/deploy.sh

6. Cloudflare

Security rules (order matters)

GOOD BITS — conditions

• BetterStack Uptime Bot
• Legitimate bot categories (Search Engine, Monitoring, Security…)
• CloudflareBrowserRenderingCrawler
• ip.src in $allowed_ip

AI Crawl Control — blocked bots

GPTBot, ClaudeBot, Bytespider, CCBot, ChatGPT-User, FacebookBot, Meta-ExternalAgent, Perplexity, MistralAI, OAI-SearchBot, AmazonBot…

Two exceptions worth knowing: BingBot passes through GOOD BITS (Search Engine Crawler category), and DeepSeekBot is blocked upstream by Filter NON EU/US since it’s hosted in China.

IP access rules (dynamic)

Managed automatically by cf-sync (see section 2), not to be edited by hand:

• crowdsec-local-ban — local CrowdSec bans
• modsec-ban — 2h WAF bans
• crowdsec-cidr-ban — /24 blocks

7. Monitoring & Alerts

Status page: status.arleo.eu Footer badge: https://status.arleo.eu/en/badge Cloudflare Analytics token: /etc/secrets/ (not published)

8. SSL Certificates

Auto-renewed by Cloudflare. Qualys SSL Labs grade: A+.

9. Points of attention

• The Cloudflare challenge inline script (__CF$cv$params) changes on every request: the resulting CSP error is unavoidable and non-blocking.
• The CrowdSec allowlist (my_allowlist) is the single source of truth, synced to Cloudflare. Any change made on the Cloudflare side gets overwritten.
• cscli bans (repeat-offender escalation) don’t show up in the CrowdSec REST API with ?limit=1000: use cscli decisions list --origin cscli.
• The Cloudflare token and AbuseIPDB credentials are stored outside the repo in /etc/secrets/, at chmod 600.