Files
homelab/kubernetes/apps/monitoring-rules
claude-botandClaude Opus 5 2131c7b5f5
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
feat(monitoring): probe that the LAN names still resolve
external-dns metrics say what the publisher intends. This says what a client
actually gets, and catches causes the publisher cannot see — a record removed
on the UDM directly, for one.

Two findings shaped it.

The probe asks the UDM directly rather than the cluster resolver. k8s-gateway
caches these names with `serve_stale 1h immediate`, so a deleted record would
keep answering from cache for an hour before the probe noticed.

And rcode alone is not a usable signal here. Measured against 10.2.40.1:

  does-not-exist-xyz.renner.uno            -> NOERROR, CNAME proxy.renner.uno
  does-not-exist-xyz.paperless.renner.uno  -> no answer

The public *.renner.uno wildcard matches one label deep, which is exactly
where ha, mqtt, ldap and syslog live: delete their records and the UDM still
answers NOERROR, pointing at the WAN proxy. A probe checking only the rcode
would have stayed green through the outage for four of the six names. So the
modules require an A record in the answer and reject a CNAME, which separates
a real record from the wildcard without encoding any address.

One module per name because the dns prober reads query_name from the module
and takes the resolver as its target; both families are probed per name and
the alert wants both to fail, so the UDM's bursty connection refusals do not
read as a missing record. That is also why the window is 30m rather than the
10m the HTTP probes use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:20:04 +02:00
..