Gate Alertmanager behind Authentik #146

2026-07-10T15:28:41Z

k.renner commented

2026-07-10 15:28:41 +00:00

Why

Alertmanager (alerts.renner.uno, monitoring ns, :9093) has no auth of its own, and its v2 API is writable: an unauthenticated LAN client can POST /api/v2/silences and suppress all alerting. That's a tamper capability, not just disclosure — the highest-teeth of the deferred unauthenticated findings from the 2026-07-10 audit.

What

Same Authentik proxy-outpost pattern as paperless/vscode/shelly-manager:

Provider + application + dedicated outpost (ak-outpost-alertmanager-gate), forwarding to the ClusterIP Service kube-prometheus-stack-alertmanager.monitoring.svc:9093.
New git-owned "Monitoring Access" group (k.renner) bound to the app. Named generically so it can also gate the still-deferred Prometheus UI later.
The Helm-generated Alertmanager Ingress is disabled (alertmanager.ingress.enabled: false); a standalone gate Ingress backends the outpost ExternalName. Two Ingresses for the same host would conflict. Mirrors loki-ingress.yaml.
Scoped CNPs: monitoring ingress rule allowing only the outpost → :9093; matching egress rule in the authentik namespace.

Two judgment calls (stated, reversible)

implicit-consent, not passkey step-up. Alertmanager is a frequently-checked dashboard; requiring an authenticated Authentik session already closes the unauthenticated-LAN exposure. Passkey re-auth on every visit is disproportionate for a dashboard — paperless-gate uses implicit-consent for the same reason. Bump to passkey-stepup-authorization if you want the write API behind a step-up.

No skip_path_regex. The Alertmanager UI is a plain Elm SPA — verified 2026-07-10 that its index.html has no serviceWorker/manifest/webmanifest refs — so none of the credentials:'omit' PWA fetches that forced skip carve-outs on paperless/vscode or killed the music-assistant gate.

Why alerting doesn't break

Prometheus posts alerts to Alertmanager via the ClusterIP Service, not this ingress — verified: spec.alerting.alertmanagers targets monitoring/kube-prometheus-stack-alertmanager:http-web. Flux's notification-controller path (also :9093 ClusterIP) is likewise untouched; the monitoring CNP still allows it. Gating the ingress only affects the human/browser path.

Prometheus's own ingress is untouched (still enabled) — its unauth exposure is a separate deferred finding, and it's read-only.

Verified so far

kubectl kustomize builds for authentik, monitoring, network-policies; the embedded outpost survived the blueprint insert; rendered Ingress → ak-outpost-alertmanager-gate:9000; egress scoped to alertmanager:9093; alertmanager.ingress.enabled=false while prometheus.ingress.enabled=true.

⚠️ Possible brief blip during rollout

The Helm ingress (removed by helm-controller) and the standalone gate ingress (applied by kustomize-controller) are reconciled by different controllers, so there may be a few seconds where both or neither exist for alerts.renner.uno. Self-resolving. I'll confirm only the gate ingress remains before testing.

Post-merge checks

Only the gate Ingress exists for alerts.renner.uno (Helm one gone).
Unauthenticated GET / → 302 to auth.renner.uno.
Unauthenticated POST /api/v2/silences → refused (not 200).
Outpost forwards to :9093 after login.
Prometheus still delivering alerts to Alertmanager (ClusterIP path).

## Why Alertmanager (`alerts.renner.uno`, monitoring ns, `:9093`) has no auth of its own, and its v2 API is **writable**: an unauthenticated LAN client can `POST /api/v2/silences` and **suppress all alerting**. That's a tamper capability, not just disclosure — the highest-teeth of the deferred unauthenticated findings from the 2026-07-10 audit. ## What Same Authentik proxy-outpost pattern as paperless/vscode/shelly-manager: - Provider + application + dedicated outpost (`ak-outpost-alertmanager-gate`), forwarding to the ClusterIP Service `kube-prometheus-stack-alertmanager.monitoring.svc:9093`. - New git-owned **"Monitoring Access"** group (`k.renner`) bound to the app. Named generically so it can also gate the still-deferred Prometheus UI later. - The **Helm-generated Alertmanager Ingress is disabled** (`alertmanager.ingress.enabled: false`); a standalone gate Ingress backends the outpost ExternalName. Two Ingresses for the same host would conflict. Mirrors `loki-ingress.yaml`. - Scoped CNPs: monitoring ingress rule allowing only the outpost → `:9093`; matching egress rule in the authentik namespace. ## Two judgment calls (stated, reversible) **implicit-consent, not passkey step-up.** Alertmanager is a frequently-checked dashboard; requiring an authenticated Authentik session already closes the unauthenticated-LAN exposure. Passkey re-auth on every visit is disproportionate for a dashboard — `paperless-gate` uses implicit-consent for the same reason. Bump to `passkey-stepup-authorization` if you want the write API behind a step-up. **No `skip_path_regex`.** The Alertmanager UI is a plain Elm SPA — verified 2026-07-10 that its `index.html` has no `serviceWorker`/`manifest`/`webmanifest` refs — so none of the `credentials:'omit'` PWA fetches that forced skip carve-outs on paperless/vscode or killed the music-assistant gate. ## Why alerting doesn't break Prometheus posts alerts to Alertmanager via the **ClusterIP Service**, not this ingress — verified: `spec.alerting.alertmanagers` targets `monitoring/kube-prometheus-stack-alertmanager:http-web`. Flux's notification-controller path (also `:9093` ClusterIP) is likewise untouched; the monitoring CNP still allows it. Gating the ingress only affects the human/browser path. Prometheus's own ingress is **untouched** (still enabled) — its unauth exposure is a separate deferred finding, and it's read-only. ## Verified so far `kubectl kustomize` builds for authentik, monitoring, network-policies; the embedded outpost survived the blueprint insert; rendered Ingress → `ak-outpost-alertmanager-gate:9000`; egress scoped to `alertmanager:9093`; `alertmanager.ingress.enabled=false` while `prometheus.ingress.enabled=true`. ## ⚠️ Possible brief blip during rollout The Helm ingress (removed by helm-controller) and the standalone gate ingress (applied by kustomize-controller) are reconciled by different controllers, so there may be a few seconds where both or neither exist for `alerts.renner.uno`. Self-resolving. I'll confirm **only** the gate ingress remains before testing. ## Post-merge checks 1. Only the gate Ingress exists for `alerts.renner.uno` (Helm one gone). 2. Unauthenticated `GET /` → `302` to `auth.renner.uno`. 3. Unauthenticated `POST /api/v2/silences` → refused (not 200). 4. Outpost forwards to `:9093` after login. 5. Prometheus still delivering alerts to Alertmanager (ClusterIP path).

k.renner added 1 commit 2026-07-10 15:28:41 +00:00

Gate Alertmanager behind Authentik 6ec3725371

Alertmanager (alerts.renner.uno, monitoring ns, :9093) had no auth of its
own, and its v2 API is writable: an unauthenticated LAN client could POST
/api/v2/silences and suppress ALL alerting — a tamper capability, not just
disclosure (2026-07-10 audit, highest-teeth of the deferred unauth
findings).

Gate it with the same Authentik proxy-outpost pattern as
paperless/vscode/shelly-manager:

- New proxy provider + application + dedicated outpost
  (ak-outpost-alertmanager-gate), mode proxy, forwarding to the ClusterIP
  Service (kube-prometheus-stack-alertmanager.monitoring.svc:9093).
- New git-owned "Monitoring Access" group (k.renner) bound to the app.
  Named generically so it can also gate the still-deferred Prometheus UI.
- The Helm-chart-generated Alertmanager Ingress is disabled
  (alertmanager.ingress.enabled=false); a standalone gate Ingress
  (alertmanager-gate-ingress.yaml) backends the outpost ExternalName —
  two Ingresses for the same host would conflict. Mirrors loki-ingress.
- Scoped CNPs: monitoring ingress rule allowing only the outpost -> :9093;
  matching egress rule in the authentik namespace.

implicit-consent flow, NOT passkey step-up: Alertmanager is a
frequently-checked monitoring UI, and requiring an authenticated Authentik
session already closes the unauthenticated-LAN exposure. Passkey re-auth on
every visit is disproportionate friction for a dashboard (paperless-gate
uses implicit-consent for the same reason). Bump to
passkey-stepup-authorization if the write API warrants it.

No skip_path_regex: the Alertmanager UI is a plain Elm SPA with no service
worker and no manifest (verified 2026-07-10 — index.html has no
serviceWorker/manifest/webmanifest), so none of the credentials:'omit' PWA
fetches that forced skip carve-outs elsewhere.

Prometheus's alerting path is UNAFFECTED: it posts to the ClusterIP Service
(verified: spec.alerting.alertmanagers targets
monitoring/kube-prometheus-stack-alertmanager:http-web), not this ingress.
Prometheus's own ingress is untouched (still enabled) — its unauth exposure
is a separate deferred finding, and it is read-only.

Verified: kubectl kustomize builds for authentik, monitoring and
network-policies; the embedded outpost survived the blueprint insert; the
rendered ingress backends ak-outpost-alertmanager-gate:9000, egress is
scoped to alertmanager:9093, and alertmanager.ingress.enabled=false while
prometheus.ingress.enabled=true.

Post-merge checks to run: only the gate Ingress exists for alerts.renner.uno
(the Helm one is gone); unauthenticated GET -> 302 to auth.renner.uno;
POST /api/v2/silences unauthenticated is refused; the outpost forwards to
:9093 after login; Prometheus is still delivering alerts to Alertmanager.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

k.renner merged commit 3734e48be2 into master

2026-07-10 15:31:56 +00:00

k.renner referenced this issue from a commit

2026-07-10 15:31:57 +00:00

Merge pull request 'Gate Alertmanager behind Authentik' (#146) from fix/alertmanager-authentik-gate into master

k.renner referenced this issue from a commit

2026-07-10 16:41:55 +00:00

Gate Prometheus behind Authentik

k.renner referenced this pull request

2026-07-10 16:43:19 +00:00

Gate Prometheus behind Authentik #147

k.renner referenced this issue from a commit