Weekly Cron Failure Exposed a Bug in check_sri_versions

In short

This site pins every CDN-hosted library (Font Awesome, Mermaid, Lunr, and others) with Subresource Integrity hashes, and a weekly cron job on the Hugo VM checks those hashes are still valid and auto-bumps minor/patch versions when a newer release is available. That job had been failing every single run for weeks, silently, and today’s investigation turned up two distinct bugs — one in operator infrastructure, one in mcp-hugo-server-go itself.

The sequence

check-sri-versions.sh detects an outdated minor/patch version, fetches the new asset, computes its SRI hash, then rebuilds the site with hugo --minify --cleanDestinationDir before deploying and purging Cloudflare. The rebuild step had been failing with chtimes: operation not permitted every week since the MCP server’s own build_site tool started managing public/ — that tool runs as a dedicated mcp-hugo-server-go system account, and once it has built the site once, every file in public/ is owned by that account. chtimes (used by Hugo to preserve mtimes when copying static files) specifically requires the calling process to own the destination file — write permission on the containing directory is not enough. The cron job ran as a different user, so it could no longer touch its own output directory.

Because the rebuild failed, the script’s own rollback path also failed the exact same way, and every run ended in a BetterStack incident that nothing ever auto-resolved — nine of them had piled up since mid-August with the actual cause never surfacing, partly because the script’s diagnostic log file was deleted immediately after the line telling you where to find it.

The fix

The simplest correct fix: run the rebuild and deploy steps as the same account that owns public/, instead of granting the cron job’s user broader access to a tree it doesn’t need to own. No new privilege escalation was needed — the operator account already had unrestricted sudo. A live run afterward applied the two pending version bumps, rebuilt, deployed, purged Cloudflare, verified the new hashes live, and auto-resolved all nine backlogged incidents in one pass.

What that fix exposed

Once the cron job’s hugo --minify rebuild actually succeeded for the first time, check_sri_versions — the MCP server’s own built-in tool for this exact check — started reporting a sharp, reproducible drop in one of its diagnostic counters: files_with_sri_attributes went from 276 to 2.

The cause turned out to be a second, independent, dormant bug. build_site (the MCP tool most operators actually use) never passes --minify to Hugo, so its HTML output always keeps quoted attributes exactly as authored: href="https://cdn.jsdelivr.net/...". Hugo’s own --minify flag — used only by this deployment’s external cron/deploy scripts, never by the MCP server — enables an HTML minifier that strips quotes from attribute values that don’t need them, which is entirely valid HTML5: href=https://cdn.jsdelivr.net/... parses identically in every browser. But check_sri_versions’s own URL-extraction regex hardcoded quotes:

var sriURLRe = regexp.MustCompile(`(?:src|href)="(https?://[^"]+)"`)

Against an unquoted tag, that pattern matches nothing, so the tool silently drops the file from its count. The security-relevant part of the tool — the actual hash comparison against data/sri.yaml — reads from a completely separate code path and stayed correct throughout; this was a cosmetic undercount in a descriptive stat, not a broken integrity check. But it’s exactly the kind of thing that erodes trust in a tool’s own reporting if it goes unexplained: a 276-to-2 drop reads like a serious regression until you know where to look.

This bug had presumably existed since the tool was written, but had never been triggered before, because the only build path that ever produced --minify output had been failing on the permission bug above for as long as anyone had been watching. Fixing one bug was the only reason the second one became visible at all.

Takeaways

Two build paths producing subtly different HTML for the same site is worth knowing about on its own, independent of either bug — an operator debugging a discovery or rendering issue should check which mechanism last touched public/ before assuming the templates themselves changed. And a diagnostic script that deletes its own error log right after telling you to go read it will cost you real time the next time something breaks in a way you haven’t seen before.

Both issues are tracked: the regex fix is filed as mcp-hugo-server-go#1252; the cron/permission fix and this whole incident are documented on the project’s wiki.