monitoring: retire Garage scrape, add Hetzner backup-endpoint alert #26

Merged
k.renner merged 1 commits from finish-line-drop-garage into master 2026-06-25 14:10:39 +00:00
1 Commits
Author SHA1 Message Date
k.rennerandClaude Opus 4.8 fe23f18c86 monitoring: retire Garage scrape, add Hetzner backup-endpoint alert
Finish line of the backup migration — nas-1's Garage is no longer a
cluster-backup dependency, so:
- drop the prometheus `garage` scrape job (s3.nas-1.renner.uno:39033) +
  the garage-metrics-token secret (now orphaned)
- delete prometheusrule-garage.yaml (GarageDown / GarageScrapeFailing) —
  it would fire every night once nas-1 powers off

Replace it with outage visibility for the new offsite target:
- add https://fsn1.your-objectstorage.com to the blackbox-http probe
  (bare endpoint returns 200)
- new HetznerBackupEndpointDown alert on probe_success == 0 for 10m

Velero outages are already covered by VeleroBackupValidationFailure /
VeleroBackupStale. A granular CNPG "WAL archiving behind" alert needs
CNPG PodMonitors (enablePodMonitor=false today, metrics not scraped) —
left as a follow-up rather than shipping an alert on an absent metric.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 16:09:06 +02:00