diff --git a/kubernetes/apps/gatus/configmap.yaml b/kubernetes/apps/gatus/configmap.yaml index 266856c9..a1df68c2 100644 --- a/kubernetes/apps/gatus/configmap.yaml +++ b/kubernetes/apps/gatus/configmap.yaml @@ -30,6 +30,35 @@ metadata: # Web Service 9 = Music Assistant # Web Service 10 = TRMNL # Web Service 11 = Paperless (gate-level only, see its entry) +# +# CORRECTION (REN-21, 2026-09-25) to the Web Service 11 comment further down, +# which claims that an in-cluster probe of the paperless ClusterIP "needs a +# CiliumNetworkPolicy first — the monitoring probes get away without one only +# because their targets share a namespace with the prober". +# +# That is wrong, and it was wrong when written. It is stated here rather than +# fixed in place because the sentence lives inside the `config.yaml: |` block +# scalar below: any edit to it changes ConfigMap DATA, and +# `configmap.reloader.stakater.com/reload: gatus-config` +# (gatus/deployment.yaml:13) would then roll the gatus Deployment — a +# production restart for a comment. This header is outside `data:`, so it is +# inert; the served config is byte-identical. +# +# * infrastructure/network-policies/paperless-ns.yaml:24-26 ALREADY admits +# the whole monitoring namespace on every port, and monitoring is where +# blackbox-exporter runs. Cilium unions the allow rules of every CNP +# selecting an endpoint, so no extra policy is required. +# * The in-cluster paperless probe therefore exists, added with no change +# to the paperless namespace: +# monitoring-rules/probes-blackbox.yaml, target +# http://paperless.paperless.svc.cluster.local:80/accounts/login/ +# * The claim does NOT generalise to gatus itself. Gatus runs in `apps`, +# which is a different source namespace; a gatus-sourced in-cluster check +# would need its own allow, exactly as +# cnp-cross-namespace-allow-apps-gatus-to-monitored-services describes. +# +# The Web Service 11 entry below stays a reachability-only check on purpose. +# The app-health half of the pair is the blackbox probe named above. data: config.yaml: | storage: diff --git a/kubernetes/apps/monitoring-rules/probes-blackbox.yaml b/kubernetes/apps/monitoring-rules/probes-blackbox.yaml index abef5736..85b63bc8 100644 --- a/kubernetes/apps/monitoring-rules/probes-blackbox.yaml +++ b/kubernetes/apps/monitoring-rules/probes-blackbox.yaml @@ -100,21 +100,58 @@ spec: # a live gate is dead. That external signal is still wanted, and REN-2 # adds it from UptimeRobot; these are the other half of the pair. # - # NO CiliumNetworkPolicy change is needed, unlike what a reading of - # cnp-cross-namespace-allow-apps-gatus-to-monitored-services suggests. - # That gotcha is about `apps`, where GATUS runs; the influxdb - # allow-list it describes already contained `monitoring`. The - # navidrome baseline CNP likewise already admits the whole monitoring - # namespace on every port + # NO CiliumNetworkPolicy change is needed — but NOT because + # cnp-cross-namespace-allow-apps-gatus-to-monitored-services fails to + # apply here. It applies: its Generalization section names + # blackbox-exporter by hand. The reason we are safe is that the allow + # that note demands is ALREADY PRESENT — the navidrome baseline CNP + # admits the whole monitoring namespace on every port # (infrastructure/network-policies/navidrome-ns.yaml:25-27), which is - # where blackbox-exporter runs. Nothing in the navidrome namespace is - # edited by this change. + # where blackbox-exporter runs. Run that check for the next namespace + # you add; do not skip it on the grounds that the note "is about + # gatus". Nothing in the navidrome namespace is edited by this change. + # + # EGRESS — the half the wiki note is silent on, and the half that is + # actually load-bearing here. Cilium enforces at BOTH ends, so the + # destination ingress allow above is worthless if the prober itself + # is in egress enforcement without a matching rule. It is not: an + # endpoint only enters egress enforcement once some policy selecting + # it declares an `egress:` block, and none does for monitoring. + # Verified at 0a82d61f across the whole repo — 73 Cilium policies, 38 + # with an egress block, none of them in the monitoring namespace, and + # the single CiliumClusterwideNetworkPolicy is scoped to + # `reserved.health` so it cannot select a pod. 17 other namespaces + # here already have an egress block. If a routine hardening pass ever + # adds one to monitoring, it MUST list the navidrome and paperless + # namespaces or these three cross-namespace targets go red while the + # *.monitoring.svc ones above keep working — a failure mode nobody + # will connect back to the hardening commit. # # /ping on both: unauthenticated, and already the pods' own kubelet # probe path (navidrome/deployment.yaml:61-74, # navidrome/lidarr.yaml:114-127). For lidarr that it is ungated is # proven independently of the gate — lidarr.yaml:29 curls /ping in a # loop BEFORE it has read the API key out of config.xml. + # + # STATUS CODE, unproven for all three of the new targets: module + # http_2xx follows redirects and passes only on a FINAL 2xx, while + # every piece of repo evidence behind these paths establishes only + # `< 400` — a kubelet httpGet accepts 200-399, and lidarr.yaml:29's + # `curl -f` without -L exits 0 on a 302. That evidence proves the + # paths are UNGATED, not that they answer 200. Check + # probe_http_status_code on the first scrapes; a target that settles + # on a 3xx gets REVERTED, not widened to accept 3xx. + # + # REVERT (recipe tested by review at 0a82d61f): delete everything + # from the "APP-HEALTH probes for the two gated navidrome-namespace + # apps" line above through the lidarr target below. Do NOT use + # `git revert -m 1 16238d39` — it conflicts, and resolving the + # conflict the natural way also drops the paperless target below, + # silently leaving no app-health signal at all. The paperless block + # below reverts on its own with `git revert -m 1 0a82d61f`, but only + # against a tree where this comment block is absent — this commit + # edits that block, so expect one conflict covering the whole + # paperless block, and resolve it by taking the deletion. - http://navidrome.navidrome.svc.cluster.local:80/ping - http://lidarr.navidrome.svc.cluster.local:80/ping # APP-HEALTH probe for paperless. Same rule as the block above @@ -127,13 +164,21 @@ spec: # this is the other half of the pair. # # That gatus comment says the in-cluster probe "needs a - # CiliumNetworkPolicy first". It does not, for THIS prober — the - # sentence was written for gatus, which runs in `apps`. - # paperless-ns.yaml:24-26 already admits the whole monitoring - # namespace, which is where blackbox-exporter runs, and Cilium unions - # the allow rules of every CNP selecting an endpoint (the mechanism - # apps/paperless/network-policy.yaml:8-12 relies on). Nothing in the - # paperless namespace is edited by this change. + # CiliumNetworkPolicy first". That sentence is simply WRONG, not + # merely inapplicable: paperless-ns.yaml:24-26 already admits the + # whole monitoring namespace, which is where blackbox-exporter runs, + # and Cilium unions the allow rules of every CNP selecting an + # endpoint (the mechanism apps/paperless/network-policy.yaml:8-12 + # relies on). The EGRESS paragraph in the block above applies here + # identically. Nothing in the paperless namespace is edited by this + # change. + # + # The stale sentence is deliberately left in place in + # apps/gatus/configmap.yaml: it sits INSIDE the `config.yaml: |` + # block scalar, so editing it changes ConfigMap data and + # `configmap.reloader.stakater.com/reload` (gatus/deployment.yaml:13) + # rolls the gatus Deployment. The correction is written at the top of + # that file instead, outside `data:`, where it costs no restart. # # /accounts/login/ because paperless-ngx has no unauthenticated health # endpoint at all. It is the app's own established "the webserver @@ -145,6 +190,10 @@ spec: # The cluster-local Host header is accepted only because # PAPERLESS_ALLOWED_HOSTS is "*" (paperless.yaml:27). Narrow that and # Django answers 400 here and this probe goes permanently red. + # + # This is the target most likely to fail the STATUS CODE check in the + # block above — a Django login view can answer 302 — so check it + # first, and revert rather than widen if it does. - http://paperless.paperless.svc.cluster.local:80/accounts/login/ # WAN-path probes of the public edge BY PROXY IP. The name-based # home.renner.uno probe above resolves in-cluster (CoreDNS/k8s_gateway)