NUC Security Audit: ModSecurity Removed, 500 MB Recovered

⚡ TL;DR

A security stack audit on the homelab NUC reveals redundant double WAF inspection: ModSecurity + OWASP CRS load 11,872 rules into memory despite SecRuleEngine Off, running in parallel with CrowdSec AppSec which already covers the same surface. After removing the ModSecurity nginx module and five other targeted fixes, nginx drops from ~520 MB to ~27 MB PSS. Same security, memory footprint divided by 20.

🏗️ Architecture Before the Audit

The security stack had six stacked layers:

Internet
    ↓
Cloudflare WAF  (edge WAF + DDoS + CrowdSec auto-ban)
    ↓
nginx  (CrowdSec Lua bouncer)
    ↓
ModSecurity + OWASP CRS   ←── double inspection
CrowdSec AppSec            ←── double inspection
    ↓
CrowdSec agent v1.7.8  (scenarios: http-cve, nginx, linux, sshd)
    ↓
Bouncers: firewall-bouncer (nftables) + nginx-bouncer (Lua) + cf-sync
    ↓
Backend (Hugo VM 192.168.122.69 / PHP-FPM)

The core issue: ModSecurity and CrowdSec AppSec each inspect every request independently. The CRS loads its regex into memory even when SecRuleEngine Off is set — the module is loaded, rules compiled, nothing is inspected, but RAM is consumed.

Initial memory footprint (smem PSS)

🔍 6 Issues Identified

1. ModSecurity / AppSec double inspection

ModSecurity was configured with SecRuleEngine Off — functionally inactive — but the nginx module was still loading 11,872 OWASP CRS rules on every startup. CrowdSec AppSec (appsec-generic-rules + appsec-virtual-patching) covers the same surface with community-maintained rules.

Decision: ModSecurity removed, AppSec kept.

2. APPSEC_FAILURE_ACTION=allow

The nginx bouncer config had APPSEC_FAILURE_ACTION=allow — if AppSec fails to respond within its timeout (timeout, oversized body), the request passes through. Changed to passthrough: on AppSec failure, the request goes to the backend without blocking but also without inspection. This bug was causing the Hugo 502s: AppSec was returning request body exceeded limit and the allow logic created an inconsistent state.

3. mcp-hugo.access.log parsed with the wrong parser

The Hugo MCP JSON log was being read by the standard nginx parser (combined format). 126 out of 132 entries showed up as “unparsed” in CrowdSec metrics. Fix: dedicated acquisition with the mcp-oauth parser.

4. http-sensitive-files duplicate — false alarm

The warning multiple scenarios named crowdsecurity/http-sensitive-files was misleading: a local override (capacity 2, leakspeed 60s) intentionally replaces the hub scenario (capacity 4, leakspeed 5s) to reduce false positives. One active scenario, nothing to fix.

5. Oversized nginx proxy_buffers

Three vhosts (radarr, sonarr, plex) used proxy_buffers 32 512k or 16 256k — up to 16 MB of buffer per peak connection. For near-zero homelab traffic, this is direct PSS waste.

6. nginx-bouncer: 0 non-empty decisions

Metrics showed crowdsec-nginx-bouncer: Empty answers = 11 / Non-empty = 0. This is normal behavior: banned IPs are blocked by Cloudflare before reaching nginx. The bouncer queries the LAPI for each legitimate visitor → empty response is correct. No malfunction.

🔧 Fixes Applied

Fix 1 — ModSecurity removal

# Comment out module loading in nginx.conf
#load_module /etc/nginx/modules-enabled/ngx_http_modsecurity_module.so;

# Disable in all vhosts (sed across sites-available/)
#modsecurity on;        # DISABLED — using CrowdSec AppSec only
#modsecurity_rules_file ...;

Applied to 10 active vhosts via a single sed pass. nginx -t + systemctl reload nginx.

Fix 2 — APPSEC_FAILURE_ACTION

# /etc/crowdsec/bouncers/crowdsec-nginx-bouncer.conf
APPSEC_FAILURE_ACTION=passthrough   # was: allow

Fix 3 — Dedicated acquisition for mcp-hugo

# /etc/crowdsec/acquis.d/mcp-hugo.yaml
filenames:
  - /var/log/nginx/mcp-hugo.access.log
labels:
  type: mcp-oauth

Fix 5 — Reducing proxy_buffers

# Before (radarr, sonarr)
proxy_buffers 32 512k;
proxy_buffer_size 512k;

# After
proxy_buffers 8 64k;
proxy_buffer_size 16k;
proxy_busy_buffers_size 128k;

📊 Results

nginx memory after full restart

nginx: master    →  PSS  2.3 MB
nginx: worker ×3 →  PSS  6.4 MB each
nginx: worker ×1 →  PSS  6.9 MB
Total            →  PSS ~27 MB

Before / After comparison

-493 MB freed. The gain comes entirely from unloading the ModSecurity module and reducing proxy_buffers.

Security: zero regression

🧠 The Lesson

The stack was described as “over-secured” — multiple WAF layers stacked, each added at a different time for a valid reason. The problem: a functionally disabled layer is still expensive if the module is loaded. ModSecurity with SecRuleEngine Off consumed as much RAM as in active mode — 11,872 regex rules compiled at nginx startup, whether requests were inspected or not.

The rule to remember: an inactive security layer is not free if the module is in memory.

🏁 Final Stack

Internet
    ↓
Cloudflare WAF + DDoS
    ↓
nginx (Lua bouncer — stream mode)
    ↓
CrowdSec AppSec  (appsec-generic-rules + appsec-virtual-patching)
    ↓
CrowdSec agent v1.7.8
    ↓
firewall-bouncer (nftables) / nginx-bouncer / cf-sync
    ↓
Backend

Four active layers, zero redundancy, ~500 MB of RAM recovered.