From 2219c928e055cfaa5828279c8bb2113f43d694ea Mon Sep 17 00:00:00 2001 From: svc-atlas Date: Fri, 25 Sep 2026 18:25:58 +0000 Subject: [PATCH] feat(monitoring): in-cluster app-health probes for navidrome and lidarr MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit navidrome.renner.uno and lidarr.renner.uno are served entirely through Authentik outposts. An external probe of either name therefore measures DNS, TLS, Traefik and the gate outpost — and stays green while the app behind a live gate is dead. This adds the second signal the monitoring-vantage-point-rule asks for: an in-cluster probe of the ClusterIP, which goes red on the app dying regardless of the gate. Targets are /ping on both, the same path the pods' own kubelet liveness/readiness probes already use. That it is ungated is not an assumption: lidarr's reconcile script curls /ping in a loop before it has read the API key out of config.xml (lidarr.yaml:29). No CiliumNetworkPolicy change, and no edit to the navidrome namespace at all. The wiki gotcha cnp-cross-namespace-allow-apps-gatus-to-monitored-services is about `apps`, where gatus runs; the influxdb allow-list it describes already contained `monitoring`. navidrome-ns.yaml:25-27 likewise already admits the whole monitoring namespace, which is where blackbox-exporter runs. No PrometheusRule change either: both targets are covered by the existing BlackboxProbeFailing catch-all and are correctly absent from its exclusion list, which per prometheusrule-blackbox.yaml:18-33 holds only instances that have a dedicated rule elsewhere. Refs REN-21. Co-Authored-By: Paperclip --- .../monitoring-rules/probes-blackbox.yaml | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/kubernetes/apps/monitoring-rules/probes-blackbox.yaml b/kubernetes/apps/monitoring-rules/probes-blackbox.yaml index 90bc23fa..771a210b 100644 --- a/kubernetes/apps/monitoring-rules/probes-blackbox.yaml +++ b/kubernetes/apps/monitoring-rules/probes-blackbox.yaml @@ -88,6 +88,35 @@ spec: # /ready answers 200 unauthenticated and is in this namespace, so the # same intra-namespace allowance the two above rely on covers it. - http://loki.monitoring.svc.cluster.local:3100/ready + # APP-HEALTH probes for the two gated navidrome-namespace apps. Same + # rule as the block above (monitoring-vantage-point-rule), but these + # are the first targets here that leave the monitoring namespace. + # + # Both sit entirely behind an Authentik outpost — navidrome.renner.uno + # on ak-outpost-navidrome-gate (navidrome/ingress.yaml:21), + # lidarr.renner.uno on ak-outpost-lidarr-gate + # (navidrome/lidarr.yaml:218) — so an EXTERNAL probe of either name + # measures the gate, not the app, and stays green while the app behind + # a live gate is dead. That external signal is still wanted, and REN-2 + # adds it from UptimeRobot; these are the other half of the pair. + # + # NO CiliumNetworkPolicy change is needed, unlike what a reading of + # cnp-cross-namespace-allow-apps-gatus-to-monitored-services suggests. + # That gotcha is about `apps`, where GATUS runs; the influxdb + # allow-list it describes already contained `monitoring`. The + # navidrome baseline CNP likewise already admits the whole monitoring + # namespace on every port + # (infrastructure/network-policies/navidrome-ns.yaml:25-27), which is + # where blackbox-exporter runs. Nothing in the navidrome namespace is + # edited by this change. + # + # /ping on both: unauthenticated, and already the pods' own kubelet + # probe path (navidrome/deployment.yaml:61-74, + # navidrome/lidarr.yaml:114-127). For lidarr that it is ungated is + # proven independently of the gate — lidarr.yaml:29 curls /ping in a + # loop BEFORE it has read the API key out of config.xml. + - http://navidrome.navidrome.svc.cluster.local:80/ping + - http://lidarr.navidrome.svc.cluster.local:80/ping # WAN-path probes of the public edge BY PROXY IP. The name-based # home.renner.uno probe above resolves in-cluster (CoreDNS/k8s_gateway) # and never leaves the LAN — the vantage-point blind spot that let the -- 2.54.0