docs(monitoring): correct the CNP reasoning, record the egress dependency (REN-21) #979

Merged
Ghost merged 2 commits from docs/probe-cnp-egress-and-status-caveats into master 2026-09-25 20:01:41 +00:00
2 changed files with 93 additions and 15 deletions
+29
View File
@@ -30,6 +30,35 @@ metadata:
# Web Service 9 = Music Assistant
# Web Service 10 = TRMNL
# Web Service 11 = Paperless (gate-level only, see its entry)
#
# CORRECTION (REN-21, 2026-09-25) to the Web Service 11 comment further down,
# which claims that an in-cluster probe of the paperless ClusterIP "needs a
# CiliumNetworkPolicy first — the monitoring probes get away without one only
# because their targets share a namespace with the prober".
#
# That is wrong, and it was wrong when written. It is stated here rather than
# fixed in place because the sentence lives inside the `config.yaml: |` block
# scalar below: any edit to it changes ConfigMap DATA, and
# `configmap.reloader.stakater.com/reload: gatus-config`
# (gatus/deployment.yaml:13) would then roll the gatus Deployment — a
# production restart for a comment. This header is outside `data:`, so it is
# inert; the served config is byte-identical.
#
# * infrastructure/network-policies/paperless-ns.yaml:24-26 ALREADY admits
# the whole monitoring namespace on every port, and monitoring is where
# blackbox-exporter runs. Cilium unions the allow rules of every CNP
# selecting an endpoint, so no extra policy is required.
# * The in-cluster paperless probe therefore exists, added with no change
# to the paperless namespace:
# monitoring-rules/probes-blackbox.yaml, target
# http://paperless.paperless.svc.cluster.local:80/accounts/login/
# * The claim does NOT generalise to gatus itself. Gatus runs in `apps`,
# which is a different source namespace; a gatus-sourced in-cluster check
# would need its own allow, exactly as
# cnp-cross-namespace-allow-apps-gatus-to-monitored-services describes.
#
# The Web Service 11 entry below stays a reachability-only check on purpose.
# The app-health half of the pair is the blackbox probe named above.
data:
config.yaml: |
storage:
@@ -100,21 +100,58 @@ spec:
# a live gate is dead. That external signal is still wanted, and REN-2
# adds it from UptimeRobot; these are the other half of the pair.
#
# NO CiliumNetworkPolicy change is needed, unlike what a reading of
# cnp-cross-namespace-allow-apps-gatus-to-monitored-services suggests.
# That gotcha is about `apps`, where GATUS runs; the influxdb
# allow-list it describes already contained `monitoring`. The
# navidrome baseline CNP likewise already admits the whole monitoring
# namespace on every port
# NO CiliumNetworkPolicy change is needed — but NOT because
# cnp-cross-namespace-allow-apps-gatus-to-monitored-services fails to
# apply here. It applies: its Generalization section names
# blackbox-exporter by hand. The reason we are safe is that the allow
# that note demands is ALREADY PRESENT — the navidrome baseline CNP
# admits the whole monitoring namespace on every port
# (infrastructure/network-policies/navidrome-ns.yaml:25-27), which is
# where blackbox-exporter runs. Nothing in the navidrome namespace is
# edited by this change.
# where blackbox-exporter runs. Run that check for the next namespace
# you add; do not skip it on the grounds that the note "is about
# gatus". Nothing in the navidrome namespace is edited by this change.
#
# EGRESS — the half the wiki note is silent on, and the half that is
# actually load-bearing here. Cilium enforces at BOTH ends, so the
# destination ingress allow above is worthless if the prober itself
# is in egress enforcement without a matching rule. It is not: an
# endpoint only enters egress enforcement once some policy selecting
# it declares an `egress:` block, and none does for monitoring.
# Verified at 0a82d61f across the whole repo — 73 Cilium policies, 38
# with an egress block, none of them in the monitoring namespace, and
# the single CiliumClusterwideNetworkPolicy is scoped to
# `reserved.health` so it cannot select a pod. 17 other namespaces
# here already have an egress block. If a routine hardening pass ever
# adds one to monitoring, it MUST list the navidrome and paperless
# namespaces or these three cross-namespace targets go red while the
# *.monitoring.svc ones above keep working — a failure mode nobody
# will connect back to the hardening commit.
#
# /ping on both: unauthenticated, and already the pods' own kubelet
# probe path (navidrome/deployment.yaml:61-74,
# navidrome/lidarr.yaml:114-127). For lidarr that it is ungated is
# proven independently of the gate — lidarr.yaml:29 curls /ping in a
# loop BEFORE it has read the API key out of config.xml.
#
# STATUS CODE, unproven for all three of the new targets: module
# http_2xx follows redirects and passes only on a FINAL 2xx, while
# every piece of repo evidence behind these paths establishes only
# `< 400` — a kubelet httpGet accepts 200-399, and lidarr.yaml:29's
# `curl -f` without -L exits 0 on a 302. That evidence proves the
# paths are UNGATED, not that they answer 200. Check
# probe_http_status_code on the first scrapes; a target that settles
# on a 3xx gets REVERTED, not widened to accept 3xx.
#
# REVERT (recipe tested by review at 0a82d61f): delete everything
# from the "APP-HEALTH probes for the two gated navidrome-namespace
# apps" line above through the lidarr target below. Do NOT use
# `git revert -m 1 16238d39` — it conflicts, and resolving the
# conflict the natural way also drops the paperless target below,
# silently leaving no app-health signal at all. The paperless block
# below reverts on its own with `git revert -m 1 0a82d61f`, but only
# against a tree where this comment block is absent — this commit
# edits that block, so expect one conflict covering the whole
# paperless block, and resolve it by taking the deletion.
- http://navidrome.navidrome.svc.cluster.local:80/ping
- http://lidarr.navidrome.svc.cluster.local:80/ping
# APP-HEALTH probe for paperless. Same rule as the block above
@@ -127,13 +164,21 @@ spec:
# this is the other half of the pair.
#
# That gatus comment says the in-cluster probe "needs a
# CiliumNetworkPolicy first". It does not, for THIS prober — the
# sentence was written for gatus, which runs in `apps`.
# paperless-ns.yaml:24-26 already admits the whole monitoring
# namespace, which is where blackbox-exporter runs, and Cilium unions
# the allow rules of every CNP selecting an endpoint (the mechanism
# apps/paperless/network-policy.yaml:8-12 relies on). Nothing in the
# paperless namespace is edited by this change.
# CiliumNetworkPolicy first". That sentence is simply WRONG, not
# merely inapplicable: paperless-ns.yaml:24-26 already admits the
# whole monitoring namespace, which is where blackbox-exporter runs,
# and Cilium unions the allow rules of every CNP selecting an
# endpoint (the mechanism apps/paperless/network-policy.yaml:8-12
# relies on). The EGRESS paragraph in the block above applies here
# identically. Nothing in the paperless namespace is edited by this
# change.
#
# The stale sentence is deliberately left in place in
# apps/gatus/configmap.yaml: it sits INSIDE the `config.yaml: |`
# block scalar, so editing it changes ConfigMap data and
# `configmap.reloader.stakater.com/reload` (gatus/deployment.yaml:13)
# rolls the gatus Deployment. The correction is written at the top of
# that file instead, outside `data:`, where it costs no restart.
#
# /accounts/login/ because paperless-ngx has no unauthenticated health
# endpoint at all. It is the app's own established "the webserver
@@ -145,6 +190,10 @@ spec:
# The cluster-local Host header is accepted only because
# PAPERLESS_ALLOWED_HOSTS is "*" (paperless.yaml:27). Narrow that and
# Django answers 400 here and this probe goes permanently red.
#
# This is the target most likely to fail the STATUS CODE check in the
# block above — a Django login view can answer 302 — so check it
# first, and revert rather than widen if it does.
- http://paperless.paperless.svc.cluster.local:80/accounts/login/
# WAN-path probes of the public edge BY PROXY IP. The name-based
# home.renner.uno probe above resolves in-cluster (CoreDNS/k8s_gateway)