Source-of-truth had drifted from /etc/systemd/system/breakglass-sync.service
on the deployed VM (installed had 8h timeout, source had 1h). My hot-patch
on 2026-05-15 raised the running unit to 16h + 4G after the service kept
timing out at 8h during full-fleet sync runs (large LFS objects + GitHub
intermittent failures across ~hundreds of repos).
Realigning source to match installed values + headroom for slow days.
Current run after the bump: Memory peak 1.0G against the new 4G ceiling
(vs would-have-OOMed at the old 1G limit).
- Wiki: clones {repo}.wiki.git, pushes to Gitea (renders natively)
- Releases: downloads latest N releases per repo (RELEASE_KEEP=3)
including all binary assets + source archives (tar.gz, zip)
- Release metadata (name, prerelease flag) preserved
- Assets uploaded to matching Gitea releases via API
- Both features toggleable via SYNC_WIKIS and SYNC_RELEASES env vars
- Skips repos with no wiki/releases silently
seedsigner/buildroot LFS push was taking hours, hitting the systemd
timeout, causing restart loop that never reached seedhammer.
- LFS fetch/push now wrapped in timeout (LFS_TIMEOUT=600s default)
- Systemd timeout raised to 8h for safety
- Git refs always push first (the critical data), LFS is best-effort
- Sync timer: daily at 02:00 AEST + on boot
- Healthcheck timer: daily at 08:00 AEST
- Service timeout raised to 4h for large repos
- Restart on failure with 5 min backoff
- ntfy.sh push notifications with priority levels:
- urgent: wipe detection (red alert)
- high: sync errors
- low: successful sync summary
- healthcheck results with stethoscope tag
- All notifications work via free ntfy.sh (no SMTP needed)
