fix(wordpress): route the outpost OAuth path so the gate is a gate, not a dead end

Verified against the deployed cluster: GET /wp-admin/ correctly 302s to
https://www.renner.uno/outpost.goauthentik.io/start?rd=... — and that URL
returned WordPress's 404 page, because /outpost.goauthentik.io fell through to
the public '/' prefix router.

The effect was that /wp-admin was unreachable rather than protected. An
attacker gains nothing, but neither does the editor the gate exists for.

paperless needs the same route and solves it with a separate Ingress on
auth.renner.uno (apps/authentik/paperless-gate-ingress.yaml). Here the whole
flow stays on www.renner.uno, so the path belongs in this rule set.

These endpoints are the login flow itself and are served by the outpost, so
routing them to it is not an authentication hole.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014giKN9N6e5J2DF8QZmVh98

.gitignore

@@ -20,5 +20,11 @@ kubernetes/access-staging/
 trivy-iac.json
 trivy-image-*.json
 
+# Alert-monitor pipeline workspace (generated in CI, never commit)
+.ci-alert-monitor/
+
+# CI diagnostic kubeconfig + kubectl for the agent steps (generated, never commit)
+.ci-diag/
+
 # Plaintext secret templates awaiting SOPS encryption (never commit)
 *.plaintext.yaml

.woodpecker/alert-monitor.yml

@@ -0,0 +1,268 @@
+# Alert monitor — every 15 minutes, read the active alerts from Alertmanager.
+# When a NEW alert is firing, an AI agent triages it against the repo and the
+# cluster context gathered here; alerts it confirms as real problems become
+# Gitea issues, which the issue-fix pipeline (issue-fix.yml) later turns into
+# PRs. Alerts it classifies as noise are recorded in a persistent "Alert
+# monitor: noise log" issue so they are never re-triaged (delete a line from
+# that issue to un-suppress an alert).
+#
+# Alertmanager access: the woodpecker-ci namespace cannot reach monitoring
+# ClusterIPs directly (Cilium default-deny-ingress on monitoring, see
+# kubernetes/infrastructure/network-policies/monitoring-ns.yaml), and the
+# public hostname alerts.renner.uno sits behind the Authentik gate. The
+# apiserver *service proxy* path works: apiserver→pod traffic arrives with
+# host/remote-node identity, which that policy explicitly allows. Verified
+# 2026-07-16 with `kubectl get --raw .../services/kube-prometheus-stack-
+# alertmanager:9093/proxy/api/v2/alerts`.
+#
+# Step split: the ci-k8s image has kubectl/flux but NO jq; the ci-claude image
+# has jq but no kubectl. So the fetch step only talks to the cluster and dumps
+# raw files into the workspace, and every JSON decision (heartbeat filter,
+# dedup, gating) lives in the agent step.
+#
+# Dedup contract (what makes 15-minute re-runs cheap): an alert is dropped
+# before the agent ever sees it when EITHER
+#   - an open issue titled exactly "[alert] <alertname> (<namespace|job|cluster>)"
+#     exists (namespace first, job as fallback for aggregate alerts), OR
+#   - its Alertmanager fingerprint appears in the noise-log issue body.
+# Runs where nothing new is firing stop before the agent — no tokens spent.
+#
+# Triggering:
+#   - cron   : a cron NAMED exactly "alert-monitor" (name filter, same
+#              mechanism as update-audit.yml):
+#                woodpecker repo cron add --repo k.renner/homelab \
+#                  --name alert-monitor --branch master --schedule '*/15 * * * *'
+#   - manual : woodpecker pipeline create --branch master \
+#                --var PIPELINE=alert-monitor k.renner/homelab
+when:
+  - event: cron
+    cron: alert-monitor
+  - event: manual
+    evaluate: 'PIPELINE == "alert-monitor"'
+
+steps:
+  - name: fetch-alerts
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    environment:
+      SOPS_AGE_KEY:
+        from_secret: sops_age_key
+      S3_ACCESS_KEY_ID:
+        from_secret: hetzner_s3_access_key
+      S3_SECRET_ACCESS_KEY:
+        from_secret: hetzner_s3_secret_key
+    commands:
+      - bash scripts/ci/fetch-kubeconfig.sh
+      - |
+        # Active, unsilenced, uninhibited alerts — raw dump; filtering happens
+        # in the agent step (no jq in this image).
+        set -eu
+        mkdir -p .ci-alert-monitor
+        AM="/api/v1/namespaces/monitoring/services/kube-prometheus-stack-alertmanager:9093/proxy"
+        kubectl get --raw "$$AM/api/v2/alerts?active=true&silenced=false&inhibited=false" \
+          > .ci-alert-monitor/alerts-raw.json
+        head -c 400 .ci-alert-monitor/alerts-raw.json; echo
+      - |
+        # Blip lookback: an alert that fires and resolves BETWEEN ticks is
+        # invisible to the live snapshot (a 17-minute KubeHpaMaxedOut escaped
+        # exactly this way on 2026-07-16). Scan Prometheus's ALERTS history
+        # over ~2 tick intervals + jitter; the overlap between runs is
+        # deduped downstream like everything else.
+        set -eu
+        PROM="/api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy"
+        kubectl get --raw "$$PROM/api/v1/query?query=max%20by%20(alertname,namespace,job,severity)%20(max_over_time(ALERTS%7Balertstate%3D%22firing%22,alertname!~%22Watchdog%7CInfoInhibitor%22%7D%5B35m%5D))" \
+          > .ci-alert-monitor/blips-raw.json \
+          || printf '{"data":{"result":[]}}' > .ci-alert-monitor/blips-raw.json
+        # (no jq in this image — counting happens in the agent step)
+        head -c 300 .ci-alert-monitor/blips-raw.json; echo
+      - |
+        # Cluster context for the triage agent — gathered here because the
+        # agent itself gets NO cluster access (no Bash, no kubeconfig). Runs
+        # unconditionally (a handful of read-only calls every 15 min is
+        # cheaper than teaching this jq-less step to gate on the JSON).
+        set -eu
+        {
+          echo "=== nodes ==="
+          kubectl get nodes -o wide
+          echo "=== pods not Running/Succeeded ==="
+          kubectl get pods -A --no-headers | grep -Ev 'Running|Completed|Succeeded' || echo "(none)"
+          echo "=== warning events (most recent last, tail 100) ==="
+          kubectl get events -A --field-selector type=Warning --sort-by=.lastTimestamp | tail -100
+          echo "=== flux kustomizations not ready ==="
+          flux get kustomizations -A --status-selector ready=false 2>&1 || true
+          echo "=== flux helmreleases not ready ==="
+          flux get helmreleases -A --status-selector ready=false 2>&1 || true
+        } > .ci-alert-monitor/context.txt
+        wc -l .ci-alert-monitor/context.txt
+      - |
+        # Least-privilege live-diagnostics kubeconfig (+ kubectl binary — the
+        # ci-claude image ships none) so the triage agent can VERIFY instead
+        # of hypothesize. Degrades to snapshot-only if the ServiceAccount
+        # isn't reconciled yet.
+        bash scripts/ci/make-diag-kubeconfig.sh
+
+  - name: alert-triage-agent
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
+    environment:
+      CLAUDE_CODE_OAUTH_TOKEN:
+        from_secret: claude_code_oauth_token
+      # Dedicated identity: alert-monitor-bot, PAT scoped to write:issue ONLY.
+      # This step runs an LLM agent over untrusted input (alert annotations,
+      # cluster event text) and its output is published verbatim as a public
+      # issue body, so the credential it holds must not be able to do anything
+      # but write issues. The account is a write collaborator, but the PAT
+      # scope is the real cap: verified 403 on /user/repos and /admin/users.
+      # Replaces the admin-owned gitea_token that used to sit here (#330).
+      GITEA_TOKEN:
+        from_secret: alert_monitor_bot_token
+    commands:
+      - |
+        # Filter + dedup + gate. Drops the always-firing heartbeats
+        # (Watchdog, InfoInhibitor), alerts routed ONLY to the `null`
+        # receiver (the owner already declared those unactionable in the
+        # Alertmanager route config — triaging them re-litigates a settled
+        # decision; found via #252/KubeCPUOvercommit), alerts that already
+        # have an open "[alert] ..." issue, and fingerprints recorded in the
+        # noise log. The repo is public, so these reads need no token. Exits
+        # the step — agent never starts — when nothing new is firing.
+        set -eu
+        jq '[.[]
+             | select(.labels.alertname != "Watchdog" and .labels.alertname != "InfoInhibitor")
+             | select([.receivers[]?.name] | (length == 0) or any(. != "null"))]' \
+          .ci-alert-monitor/alerts-raw.json > .ci-alert-monitor/alerts-active.json
+        echo "Active alerts (after heartbeat filter): $$(jq length .ci-alert-monitor/alerts-active.json)"
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
+        curl -sfL "$$API/issues?state=open&type=issues&limit=100" > .ci-alert-monitor/open-issues.json
+        TITLES=$$(jq -c '[.[].title]' .ci-alert-monitor/open-issues.json)
+        NOISE_BODY=$$(jq -r 'map(select(.title == "Alert monitor: noise log")) | .[0].body // ""' \
+          .ci-alert-monitor/open-issues.json)
+        FPS=$$(printf '%s' "$$NOISE_BODY" | grep -oE '\b[0-9a-f]{16}\b' \
+          | jq -R -s -c 'split("\n") | map(select(length > 0))')
+        # Synthesize pseudo-alerts for BLIPS: lookback series whose label set
+        # is not currently firing at all (checked against the PRE-filter raw
+        # snapshot, so live-but-null-routed alerts don't reappear as blips).
+        # Pseudo-fingerprint = sha256(alertname|scope)[:16] — STABLE across
+        # recurrences, so one noise verdict suppresses a flappy alert
+        # permanently via the noise log.
+        : > .ci-alert-monitor/blips-new.jsonl
+        jq -c '.data.result[]?.metric // empty' .ci-alert-monitor/blips-raw.json | while IFS= read -r M; do
+          NAME=$$(printf '%s' "$$M" | jq -r '.alertname')
+          SCOPE=$$(printf '%s' "$$M" | jq -r '.namespace // .job // "cluster"')
+          if jq -e --arg n "$$NAME" --arg s "$$SCOPE" \
+               '[.[] | select(.labels.alertname == $$n and ((.labels.namespace // .labels.job // "cluster") == $$s))] | length > 0' \
+               .ci-alert-monitor/alerts-raw.json > /dev/null; then
+            continue
+          fi
+          FP=$$(printf '%s|%s' "$$NAME" "$$SCOPE" | sha256sum | cut -c1-16)
+          jq -n --arg fp "$$FP" --argjson m "$$M" \
+            '{labels: $$m, fingerprint: $$fp, blip: true,
+              annotations: {summary: "BLIP: fired within the lookback window (~35 min) and already resolved — caught by the ALERTS history scan, not currently active."}}' \
+            >> .ci-alert-monitor/blips-new.jsonl
+        done
+        jq -s '.' .ci-alert-monitor/blips-new.jsonl > .ci-alert-monitor/blips-candidates.json
+        # Title key: namespace, else job, else "cluster". The job fallback
+        # exists because aggregate alerts like TargetDown carry no namespace
+        # — with only "(cluster)" two simultaneous TargetDowns for different
+        # jobs (pve and dns-servers, 2026-07-16) collapsed into ONE issue
+        # and the second root cause was never triaged.
+        jq -s 'add' .ci-alert-monitor/alerts-active.json .ci-alert-monitor/blips-candidates.json \
+          > .ci-alert-monitor/alerts-combined.json
+        jq --argjson titles "$$TITLES" --argjson fps "$${FPS:-[]}" '
+          [ .[]
+            | ("[alert] " + .labels.alertname + " (" + (.labels.namespace // .labels.job // "cluster") + ")") as $$t
+            | select( ($$t | IN($$titles[]) | not) and (.fingerprint | IN($$fps[]) | not) )
+            | . + {issue_title: $$t} ]' \
+          .ci-alert-monitor/alerts-combined.json > .ci-alert-monitor/alerts-new.json
+        NEW=$$(jq length .ci-alert-monitor/alerts-new.json)
+        jq -r '.[] | "\(.fingerprint)\t\(.issue_title)"' .ci-alert-monitor/alerts-new.json
+        if [ "$$NEW" -eq 0 ]; then
+          echo "No new alerts — agent not started."
+          exit 0
+        fi
+      - |
+        # Headless triage run. Same auth model as security-scan.yml
+        # (subscription OAuth, prompt piped via stdin — see the comments
+        # there). Tool allowlist is read-only on the repo plus Write for the
+        # /tmp verdict file, plus EXACTLY ONE live command: the RBAC-scoped
+        # kubectl from make-diag-kubeconfig.sh (read-only ClusterRole, no
+        # Secrets). The prefix match means no pipes/redirects and no other
+        # binaries — the agent verifies cluster state, it cannot mutate it,
+        # fetch the web, or read secrets.
+        export KUBECONFIG="$$PWD/.ci-diag/kubeconfig"
+        # Model chosen live from the catalog (not hardcoded, no family assumed) —
+        # see the header of scripts/ci/pick-model.sh. Empty MODEL -> CLI default.
+        MODEL=$$(sh scripts/ci/pick-model.sh "triage firing Prometheus/Alertmanager alerts against read-only live cluster state and classify each as a real problem or noise; careful diagnostic reasoning, no changes")
+        if [ -n "$$MODEL" ]; then echo "alert-monitor: selected model $$MODEL"; else echo "alert-monitor: no live catalog — using the CLI default model"; fi
+        cat ci/agents/alert-monitor/prompt.md | claude $${MODEL:+--model $$MODEL} \
+          --permission-mode acceptEdits \
+          --allowedTools "Read,Glob,Grep,Write,Bash(./.ci-diag/kubectl:*)" \
+          --settings ci/agents/alert-monitor/settings.json \
+          -p
+      - |
+        # Post the verdicts: one Gitea issue per confirmed problem, and one
+        # appended line in the persistent noise log per noise verdict. Every
+        # write checks its HTTP status explicitly (see update-audit.yml for
+        # why piping into `curl -sf | jq` is not enough).
+        set -eu
+        if [ ! -s /tmp/alert-verdicts.json ]; then
+          echo "Agent wrote no verdicts (/tmp/alert-verdicts.json missing) — nothing to post."
+          exit 0
+        fi
+        jq . /tmp/alert-verdicts.json > /dev/null   # fail loudly on malformed output
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
+        STAMP=$$(date -u +%Y-%m-%d)
+
+        gitea_write() {
+          code=$$(curl -s -o /tmp/gitea-resp.json -w '%{http_code}' \
+            -X "$$1" -H "Authorization: token $$GITEA_TOKEN" \
+            -H "Content-Type: application/json" --data @"$$3" "$$2")
+          echo "$$1 $$2 -> HTTP $$code"
+          if [ "$$code" -lt 200 ] || [ "$$code" -ge 300 ]; then
+            echo "Gitea API error body:"; cat /tmp/gitea-resp.json; echo
+            return 1
+          fi
+        }
+
+        N=$$(jq length /tmp/alert-verdicts.json)
+        i=0
+        : > /tmp/noise-lines.txt
+        while [ "$$i" -lt "$$N" ]; do
+          VERDICT=$$(jq -r ".[$$i].verdict" /tmp/alert-verdicts.json)
+          TITLE=$$(jq -r ".[$$i].title" /tmp/alert-verdicts.json)
+          FP=$$(jq -r ".[$$i].fingerprint" /tmp/alert-verdicts.json)
+          NAME=$$(jq -r ".[$$i].alertname" /tmp/alert-verdicts.json)
+          REASON=$$(jq -r ".[$$i].reason" /tmp/alert-verdicts.json)
+          if [ "$$VERDICT" = "problem" ]; then
+            # Safety re-check: never double-file if a parallel run won the race.
+            EXISTS=$$(curl -sfL "$$API/issues?state=open&type=issues&limit=100" \
+              | jq -r --arg t "$$TITLE" 'map(select(.title == $$t)) | length')
+            if [ "$$EXISTS" -gt 0 ]; then
+              echo "Issue already open for '$$TITLE' — skipping."
+            else
+              jq --arg footer "$$(printf '\n\n---\n<!-- alert-monitor:fingerprint=%s -->\n_Filed automatically by the alert-monitor pipeline (%s). The issue-fix pipeline may pick this up and open a PR._' "$$FP" "$$STAMP")" \
+                 --argjson idx "$$i" \
+                 '.[$$idx] | {title: .title, body: (.body + $$footer)}' \
+                 /tmp/alert-verdicts.json > /tmp/gitea-create.json
+              gitea_write POST "$$API/issues" /tmp/gitea-create.json
+              jq '{number, html_url, title}' /tmp/gitea-resp.json
+            fi
+          else
+            printf -- '- %s %s — %s: %s\n' "$$FP" "$$NAME" "$$STAMP" "$$REASON" >> /tmp/noise-lines.txt
+          fi
+          i=$$((i+1))
+        done
+
+        if [ -s /tmp/noise-lines.txt ]; then
+          NOISE_NUM=$$(curl -sfL "$$API/issues?state=open&type=issues&limit=100" \
+            | jq -r 'map(select(.title == "Alert monitor: noise log")) | .[0].number // empty')
+          if [ -n "$$NOISE_NUM" ]; then
+            curl -sfL "$$API/issues/$$NOISE_NUM" | jq -r '.body // ""' > /tmp/noise-old.txt
+            jq -n --rawfile old /tmp/noise-old.txt --rawfile add /tmp/noise-lines.txt \
+              '{body: ($$old + "\n" + $$add)}' > /tmp/gitea-patch.json
+            gitea_write PATCH "$$API/issues/$$NOISE_NUM" /tmp/gitea-patch.json
+          else
+            printf '%s\n\n' "Fingerprints the alert-monitor triage agent classified as noise. The dedup step skips any alert whose fingerprint appears below — delete a line to make that alert eligible for triage again. Keep this issue open; it is the suppression state." > /tmp/noise-header.txt
+            jq -n --rawfile hdr /tmp/noise-header.txt --rawfile add /tmp/noise-lines.txt \
+              '{title: "Alert monitor: noise log", body: ($$hdr + $$add)}' > /tmp/gitea-create.json
+            gitea_write POST "$$API/issues" /tmp/gitea-create.json
+          fi
+        fi

.woodpecker/build-ci-images.yml

@@ -1,7 +1,24 @@
+# Builds the CI tooling images (and the secret-rotator runtime image) with
+# kaniko and pushes them to the Gitea container registry.
+#
+# Credential: ci_images_token — a PAT scoped to `package: read+write` and
+# NOTHING else (verified: 403 on /user/repos, /admin/users and the issues
+# API; 200 on the /v2/ registry endpoints kaniko uses). It replaces the
+# admin-owned gitea_token that used to sit here, which could do anything
+# the admin account could (#330).
+#
+# The token is still OWNED by gitea_admin, because Gitea scopes packages to
+# a user or org namespace and these images live under `gitea_admin/*` — a
+# different account cannot publish there. Moving them to a dedicated org is
+# the remaining cleanup; it is a 38-reference migration across pipelines and
+# Flux-deployed CronJobs, so it is tracked separately in #330 rather than
+# bundled here. Note this pipeline runs no LLM agent, so the exfil path that
+# motivated the other cutovers does not apply to it.
 when:
   - event: push
     path:
       - "ci/images/**"
+      - "secret-rotator/**"
       - ".woodpecker/build-ci-images.yml"
   - event: manual
     evaluate: 'PIPELINE == "build-ci-images"'
@@ -11,10 +28,18 @@ steps:
     image: gcr.io/kaniko-project/executor:v1.23.2-debug
     directory: ci/images/ansible
     depends_on: []
+    # Only rebuild when THIS image's source (or the pipeline file) changed, or
+    # on a manual run. Without this every step ran on any matching push, so a
+    # secret-rotator commit needlessly rebuilt all four ~1.3G ci-* images and
+    # orphaned ~5G of registry layers per run — a main driver of #336.
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/ansible/**", ".woodpecker/build-ci-images.yml"]
     environment:
       GITEA_USER: gitea_admin
       GITEA_TOKEN:
-        from_secret: gitea_token
+        from_secret: ci_images_token
     commands:
       - mkdir -p /kaniko/.docker
       - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
@@ -31,10 +56,14 @@ steps:
     image: gcr.io/kaniko-project/executor:v1.23.2-debug
     directory: ci/images/k8s
     depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/k8s/**", ".woodpecker/build-ci-images.yml"]
     environment:
       GITEA_USER: gitea_admin
       GITEA_TOKEN:
-        from_secret: gitea_token
+        from_secret: ci_images_token
     commands:
       - mkdir -p /kaniko/.docker
       - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
@@ -51,10 +80,14 @@ steps:
     image: gcr.io/kaniko-project/executor:v1.23.2-debug
     directory: ci/images/tofu
     depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/tofu/**", ".woodpecker/build-ci-images.yml"]
     environment:
       GITEA_USER: gitea_admin
       GITEA_TOKEN:
-        from_secret: gitea_token
+        from_secret: ci_images_token
     commands:
       - mkdir -p /kaniko/.docker
       - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
@@ -67,14 +100,25 @@ steps:
           --destination gitea.renner.uno/gitea_admin/ci-tofu:$TAG \
           --destination gitea.renner.uno/gitea_admin/ci-tofu:latest
 
-  - name: build-claude
+  # NOTE: ci-claude is deliberately NOT built here. It tracks upstream Claude
+  # Code releases on its own cadence and is built + version-tagged by
+  # ci-claude-refresh.yml, which also opens the PR that rolls a new version out
+  # to the consumer pipelines. Building it here too would fight over tags.
+
+  # Runtime image, not CI tooling: a static musl binary in a FROM-scratch image,
+  # used as the base for secret-rotation CronJobs. Replaces per-job inline Python.
+  - name: build-secret-rotator
     image: gcr.io/kaniko-project/executor:v1.23.2-debug
-    directory: ci/images/claude
+    directory: secret-rotator
     depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["secret-rotator/**", ".woodpecker/build-ci-images.yml"]
     environment:
       GITEA_USER: gitea_admin
       GITEA_TOKEN:
-        from_secret: gitea_token
+        from_secret: ci_images_token
     commands:
       - mkdir -p /kaniko/.docker
       - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
@@ -84,5 +128,121 @@ steps:
         /kaniko/executor \
           --context . \
           --dockerfile Containerfile \
-          --destination gitea.renner.uno/gitea_admin/ci-claude:$TAG \
-          --destination gitea.renner.uno/gitea_admin/ci-claude:latest
+          --destination gitea.renner.uno/gitea_admin/secret-rotator:$TAG \
+          --destination gitea.renner.uno/gitea_admin/secret-rotator:latest
+
+  # Runtime image, not CI tooling: a general-purpose alpine + jq base for
+  # workloads that need to read JSON from a shell script. Named for what it IS,
+  # not for who uses it — its first consumer is the zigbee2mqtt seed-data init
+  # container, but nothing about it is zigbee-specific. Exists so tools come from
+  # the image rather than an `apk add` on every container start.
+  - name: build-alpine-jq
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/alpine-jq
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/alpine-jq/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/alpine-jq:$TAG \
+          --destination gitea.renner.uno/gitea_admin/alpine-jq:latest
+
+  # Runtime image, not CI tooling: upstream containernetworking/plugins binaries
+  # baked in, consumed by the multus cni-plugins-installer DaemonSet. Replaces a
+  # node-side fetch from GitHub that had NO checksum verification and sat on the
+  # critical path of every node rebuild. The download now happens here, once,
+  # where the sha256 is checked — see the Containerfile for the full rationale.
+  - name: build-cni-plugins
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/cni-plugins
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/cni-plugins/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/cni-plugins:$TAG \
+          --destination gitea.renner.uno/gitea_admin/cni-plugins:latest
+
+  # Runtime image, not CI tooling: alpine + the openssl CLI, for shell scripts
+  # needing s_client/x509. Named for what it is; first consumer is the paperless
+  # fetch-bridge-cert init container. Replaces an `apk add openssl` that ran on
+  # every paperless pod start. The openssl CLI is genuinely absent from the alpine
+  # base, so this one needs an image rather than a busybox substitute.
+  - name: build-alpine-openssl
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/alpine-openssl
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/alpine-openssl/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/alpine-openssl:$TAG \
+          --destination gitea.renner.uno/gitea_admin/alpine-openssl:latest
+
+  # Heating CAN readers: open3e (Vitocal/VX3) + E3onCAN (E380 meter), one image
+  # run as two services on edge-1. Replaces pip-into-venv on the host, where an
+  # unpinned VCS requirement re-cloned and restarted the heating bridge on every
+  # Ansible converge. Not a ci-* image -- it is a runtime artifact -- but it
+  # builds the same way and lands in the same registry so the audit sees it.
+  - name: build-heating-can
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/heating-can
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/heating-can/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
+          --destination gitea.renner.uno/gitea_admin/heating-can:latest

.woodpecker/ci-claude-refresh.yml

@@ -0,0 +1,214 @@
+# ci-claude refresh — keep the CI agent image on the LATEST Claude Code and
+# roll that version out to every pipeline that runs it, automatically.
+#
+# Why this exists (#351, follow-up from #336): the ci-claude image used to pin
+# a fixed Claude Code version in its Containerfile, and every consumer pipeline
+# pinned a stale dated image tag. Old Claude Code releases eventually stop
+# working against the API, so both pins silently rot the agent pipelines. This
+# pipeline owns the whole lifecycle instead:
+#
+#   1. decide   — resolve the latest published Claude Code version and compare
+#                 it to the tag the consumer pipelines currently reference.
+#   2. build    — (only when a rebuild is warranted) kaniko-build the image
+#                 pinned to that exact version and push  ci-claude:<version>
+#                 and  ci-claude:latest.  The Containerfile makes this layer
+#                 byte-reproducible, so rebuilding an unchanged version dedups
+#                 in the registry instead of orphaning a fresh ~260 MB copy.
+#   3. bump-pr  — (only when the version actually changed) rewrite the 5
+#                 consumer pins to  ci-claude:<version>  and open a PR. A human
+#                 reviews and merges it; nothing auto-merges.
+#
+# TAG SCHEME — deliberately version tags (ci-claude:2.1.218), NOT the dated
+# tags the other ci-* images use. The tag encodes the thing that changes here
+# (the Claude version), which makes "did it change?" a trivial, race-free
+# string compare and keeps the weekly cron from opening no-op PRs. The other
+# images change only when their Containerfile changes, so dated tags fit them;
+# this one tracks an upstream release cadence.
+#
+# This pipeline OWNS building ci-claude: the image's build step was removed
+# from build-ci-images.yml so the two never fight over tags.
+#
+# TOKENS — two least-privilege, single-purpose secrets, per the #330 rule that
+# a pipeline never gets a broader credential than it needs and never the admin
+# gitea_token:
+#   - ci_images_token          : package read+write ONLY (the kaniko push).
+#                                Shared with build-ci-images.yml — same job.
+#   - ci_claude_bump_bot_token : a claude-bot PAT scoped write:repository ONLY
+#                                (verified: 200 on the repo, 403 on
+#                                /admin/users). It pushes the bump branch and
+#                                opens the PR; it does NOT comment on issues, so
+#                                it carries no write:issue. A dedicated Gitea
+#                                *user* would need admin to create, which
+#                                claude-bot lacks; a scoped-down claude-bot PAT
+#                                gives the same least-privilege at the token
+#                                level. claude-bot is already a write
+#                                collaborator here. Secret scoped to the
+#                                push/cron/manual events this pipeline fires on.
+#
+# OWNER SETUP — the token + secret above are already provisioned. The one
+# remaining step, to be run AFTER this merges to master (a cron references a
+# pipeline that must exist on the target branch), is to register the schedule:
+#   woodpecker repo cron add --repo k.renner/homelab \
+#     --name ci-claude-refresh --branch master --schedule '@weekly'
+# Weekly is plenty; the pipelines tolerate being a few days behind a release.
+#
+# Triggering:
+#   - cron   : the cron NAMED exactly "ci-claude-refresh" (see above).
+#   - manual : woodpecker pipeline create --branch master \
+#                --var PIPELINE=ci-claude-refresh k.renner/homelab
+#   - push   : a change to the Containerfile (or this file) ON master forces a
+#              rebuild of the current latest version so Containerfile fixes ship
+#              promptly. Restricted to master ON PURPOSE — a feature branch must
+#              never build/push an image or open a bump PR (only master pushes
+#              images), and without this filter every PR that edits these paths
+#              would do exactly that from its own branch.
+when:
+  - event: cron
+    cron: ci-claude-refresh
+  - event: manual
+    evaluate: 'PIPELINE == "ci-claude-refresh"'
+  - event: push
+    branch: master
+    path:
+      - "ci/images/claude/**"
+      - ".woodpecker/ci-claude-refresh.yml"
+
+steps:
+  # 1. Resolve latest vs. currently-pinned, and decide what to do. Writes
+  #    sentinels to $CI_WORKSPACE/.ci-refresh/ (the workspace volume is the
+  #    only path shared across steps; /tmp is per-container). Uses the ci-claude
+  #    image purely as a toolbox (it ships npm, jq, git, curl).
+  - name: decide
+    image: gitea.renner.uno/gitea_admin/ci-claude:latest
+    commands:
+      - |
+        set -eu
+        mkdir -p "$$CI_WORKSPACE/.ci-refresh"
+        # Files that pin the image. Keep this list in sync with the consumers.
+        CONSUMERS="
+          .woodpecker/security-scan.yml
+          .woodpecker/alert-monitor.yml
+          .woodpecker/update-audit.yml
+          .woodpecker/pr-review.yml
+          .woodpecker/issue-fix.yml
+        "
+        NEW=$$(npm view @anthropic-ai/claude-code version)
+        case "$$NEW" in
+          [0-9]*.[0-9]*.[0-9]*) : ;;
+          *) echo "npm returned an implausible version '$$NEW' — aborting."; exit 1 ;;
+        esac
+        # The tag every consumer currently references. They are kept identical,
+        # so this must resolve to EXACTLY ONE non-empty tag; anything else
+        # (no match, or a partial/disagreeing set) aborts rather than risk a
+        # corrupting sed later. Counting with `grep -c .` (non-empty lines)
+        # instead of `wc -l` is deliberate: an empty result yields 0 here but
+        # 1 under wc -l, which would slip past the guard.
+        CUR=$$(grep -hoE 'ci-claude:[0-9A-Za-z._-]+' $$CONSUMERS | sed 's/^ci-claude://' | sort -u)
+        NTAGS=$$(printf '%s' "$$CUR" | grep -c . || true)
+        if [ -z "$$CUR" ] || [ "$$NTAGS" -ne 1 ]; then
+          echo "Expected exactly one ci-claude tag across the consumers, got [$$CUR]."
+          echo "Refusing to guess — fix the pins by hand."; exit 1
+        fi
+        echo "latest=$$NEW  current=$$CUR  event=$$CI_PIPELINE_EVENT"
+        printf '%s' "$$NEW" > "$$CI_WORKSPACE/.ci-refresh/version"
+        printf '%s' "$$CUR" > "$$CI_WORKSPACE/.ci-refresh/current"
+        # Build when the version changed, or on a push (Containerfile edit must
+        # ship even if the version is unchanged). Bump the pins only when the
+        # version the consumers reference is not the latest.
+        if [ "$$CUR" != "$$NEW" ] || [ "$$CI_PIPELINE_EVENT" = "push" ]; then
+          touch "$$CI_WORKSPACE/.ci-refresh/do-build"; echo "will build $$NEW"
+        else
+          echo "already on $$NEW — nothing to build"
+        fi
+        if [ "$$CUR" != "$$NEW" ]; then
+          touch "$$CI_WORKSPACE/.ci-refresh/do-bump"; echo "will open bump PR $$CUR -> $$NEW"
+        else
+          echo "consumers already pin $$NEW — no PR"
+        fi
+
+  # 2. Build the exact resolved version and push version + latest tags. Pinning
+  #    --build-arg to the resolved version (not "latest") guarantees the tag and
+  #    the image content agree even if a new release lands mid-run.
+  - name: build
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/claude
+    depends_on: [decide]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - |
+        [ -f "$$CI_WORKSPACE/.ci-refresh/do-build" ] || { echo "no build needed"; exit 0; }
+        set -eu
+        VERSION=$$(cat "$$CI_WORKSPACE/.ci-refresh/version")
+        mkdir -p /kaniko/.docker
+        AUTH=$$(printf '%s:%s' "$$GITEA_USER" "$$GITEA_TOKEN" | base64 | tr -d '\n')
+        printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$$AUTH" > /kaniko/.docker/config.json
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --build-arg CLAUDE_CODE_VERSION="$$VERSION" \
+          --destination gitea.renner.uno/gitea_admin/ci-claude:"$$VERSION" \
+          --destination gitea.renner.uno/gitea_admin/ci-claude:latest
+
+  # 3. Rewrite the consumer pins and open a PR. Never auto-merges. Skips itself
+  #    when the version was unchanged, and when a bump branch already exists (a
+  #    prior run's PR is still open / being merged).
+  - name: bump-pr
+    image: gitea.renner.uno/gitea_admin/ci-claude:latest
+    depends_on: [build]
+    environment:
+      PUSH_TOKEN:
+        from_secret: ci_claude_bump_bot_token
+    commands:
+      - |
+        [ -f "$$CI_WORKSPACE/.ci-refresh/do-bump" ] || { echo "no version change — no PR"; exit 0; }
+        set -eu
+        NEW=$$(cat "$$CI_WORKSPACE/.ci-refresh/version")
+        CUR=$$(cat "$$CI_WORKSPACE/.ci-refresh/current")
+        # Both were validated non-empty by the decide step; re-assert before a
+        # sed that would corrupt every pin if either were blank.
+        [ -n "$$NEW" ] && [ -n "$$CUR" ] || { echo "empty version/current sentinel — aborting"; exit 1; }
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
+        BRANCH="ci-image/claude-$$NEW"
+        if git ls-remote --heads "https://gitea.renner.uno/k.renner/homelab.git" "$$BRANCH" | grep -q .; then
+          echo "Branch $$BRANCH already exists — a PR for $$NEW is already open. Nothing to do."
+          exit 0
+        fi
+        CONSUMERS="
+          .woodpecker/security-scan.yml
+          .woodpecker/alert-monitor.yml
+          .woodpecker/update-audit.yml
+          .woodpecker/pr-review.yml
+          .woodpecker/issue-fix.yml
+        "
+        # Exact-string swap of the current tag for the new one, consumers only.
+        for f in $$CONSUMERS; do
+          sed -i "s#ci-claude:$$CUR#ci-claude:$$NEW#g" "$$f"
+        done
+        # Author is claude-bot (the PAT's account); the commit subject makes
+        # the automated-bump nature clear.
+        git config user.email "claude-bot@renner.uno"
+        git config user.name "Claude Bot"
+        git checkout -b "$$BRANCH"
+        git add $$CONSUMERS
+        git commit -m "chore(ci): bump ci-claude image to $$NEW"
+        git push "https://$$PUSH_TOKEN@gitea.renner.uno/k.renner/homelab.git" HEAD:"$$BRANCH"
+        {
+          printf 'Automated bump of the ci-claude agent image from `%s` to `%s`.\n\n' "$$CUR" "$$NEW"
+          printf 'The refresh pipeline built and pushed `ci-claude:%s` (and `:latest`) from the\n' "$$NEW"
+          printf 'latest published `@anthropic-ai/claude-code`. This rewires the 5 pipelines that\n'
+          printf 'run the image so CI stops running a stale Claude Code.\n\n'
+          printf 'Merging is a human decision — review, then merge to roll it out.\n'
+        } > /tmp/pr-body.md
+        jq -n --arg title "chore(ci): bump ci-claude image to $$NEW" \
+              --rawfile body /tmp/pr-body.md \
+              --arg head "$$BRANCH" \
+          '{title: $$title, body: $$body, head: $$head, base: "master"}' > /tmp/pr.json
+        code=$$(curl -s -o /tmp/resp.json -w '%{http_code}' \
+          -X POST -H "Authorization: token $$PUSH_TOKEN" \
+          -H "Content-Type: application/json" --data @/tmp/pr.json "$$API/pulls")
+        echo "POST pull -> HTTP $$code"
+        [ "$$code" -ge 200 ] && [ "$$code" -lt 300 ] || { cat /tmp/resp.json; exit 1; }
+        jq '{number, html_url, title}' /tmp/resp.json

.woodpecker/ci-manifests-test.yml

@@ -0,0 +1,254 @@
+# Validate the GitOps tree and the pipeline definitions on PRs that touch them.
+#
+# Nothing covered kubernetes/** before this. deploy-infrastructure.yml only
+# gates opentofu/**, and ci-scripts-test.yml only runs scripts/ci/ unit tests —
+# so a broken manifest reached the cluster and was caught by Flux failing to
+# reconcile, or by a human noticing. Three separate defects on 2026-07-30 would
+# have been caught here instead:
+#
+#   * an unquoted YAML scalar containing ": " in .woodpecker/build-ci-images.yml,
+#     which made EVERY pipeline in the repo fail to compile (#443). `yq` parsed
+#     it happily — it is valid YAML, just a map instead of a string — so
+#     "it parses" was mistaken for "it is valid". Only woodpecker lint catches it.
+#   * a rotator pinned to an image digest that no longer existed (#458).
+#   * manifests whose kustomize wiring was only ever checked by hand.
+#
+# Deliberately CHEAP and offline: no cluster, no registry, no secrets. Everything
+# here is a static check that either passes or fails on the checkout alone, so it
+# can gate PRs without becoming a bottleneck or a flake source.
+when:
+  # Path-filtered on purpose, unlike ci-scripts-test.yml. That pipeline is one
+  # step on one already-cached image, so running it everywhere is free; this one
+  # pulls three images and builds every kustomization in the tree, which is real
+  # cost to put on a docs-only PR. The filter covers everything the steps actually read:
+  # kubernetes/** for the manifests and scripts, .woodpecker/** for the lint (and
+  # so an edit to this file re-runs its own checks).
+  - event: pull_request
+    path:
+      - "kubernetes/**"
+      - ".woodpecker/**"
+
+# ADDING A THIRD DISCOVERY CHECK: kustomize-build and shellcheck both hand-roll
+# the same shape — `find`, a floor guard against empty discovery, then a
+# file-fed `while read`. Two copies is deliberate: the two traps being commented
+# around (BusyBox find lacking `-printf`, and a piped loop running in a subshell
+# where `fail` cannot propagate) are different, and both are worth spelling out
+# where they bite. A third instance is the point to stop copying and factor it
+# into a shared script under `scripts/ci/` — which is already covered by
+# ci-scripts-test.yml, so it would gain unit tests in the process.
+#
+# Every step carries `depends_on: []` so all four run in PARALLEL and every
+# failure shows in one run. Woodpecker's default is sequential, which would stop
+# at the first failure and make a contributor fix-and-repush to discover the
+# next — and these four are independent, cheap, offline checks with no ordering
+# between them. Keep it on every step: `depends_on` is per-step, so dropping it
+# from one silently re-serialises that step behind the others.
+steps:
+  # Every kustomization must build. This is the check that catches a resource
+  # entry pointing at a file that does not exist, a bad patch target, or YAML
+  # that is fine in isolation but broken once overlaid — the class of error that
+  # otherwise surfaces as a Flux reconcile failure after merge.
+  #
+  # Verified green across every kustomization in the tree when this was added,
+  # so this gates new breakage rather than pre-existing debt.
+  #
+  # ci-k8s TAG SPLIT: this file pins a newer dated tag than e2e.yml, e2e-test.yml,
+  # alert-monitor.yml and issue-fix.yml, which still sit on an older one. That is
+  # staleness in those four, not a deliberate split here — everything should be on
+  # the newest dated tag. They are not bumped here because none of them are
+  # triggered by this PR, so it would ship untested changes under cover of a green
+  # run that never touched them. Tracked in #461.
+  - name: kustomize-build
+    depends_on: [] # see note above `steps:`
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30
+    commands:
+      - |
+        set -eu
+        # NOT `find -printf '%h\n'`: that is a GNU findutils extension and this
+        # image ships BusyBox find (v1.37.0), which rejects it outright with
+        # "find: unrecognized: -printf". Because the result feeds a command
+        # substitution used as a word list, the failure is SILENT — empty output,
+        # zero loop iterations, fail=0, step exits green having validated
+        # nothing. That is exactly what this pipeline shipped in its first
+        # revision, and CI reported it as passing.
+        dirs=$(find kubernetes -name kustomization.yaml | sed 's|/kustomization.yaml$||' | sort)
+
+        # Assert the discovery actually found something. Without this, any future
+        # breakage of the find/sed above degrades back into a green no-op rather
+        # than a failure — the specific trap that made the first revision of this
+        # step useless while looking healthy.
+        count=$(printf '%s\n' "$dirs" | grep -c . || true)
+        if [ "$count" -lt 50 ]; then
+          echo "FAIL: found only $count kustomizations; floor is 50."
+          echo "      If the tree legitimately shrank, lower this floor in"
+          echo "      .woodpecker/ci-manifests-test.yml — do not delete it."
+          echo "      Otherwise discovery is broken — refusing to report success."
+          exit 1
+        fi
+        echo "discovered $count kustomizations"
+
+        # `while read` fed from a FILE, not a pipe: a pipe would run the loop in
+        # a subshell where `fail` cannot propagate, so a failing build would exit
+        # 0 — the same silent-pass shape this step already shipped once. Reading
+        # a file keeps the loop in the current shell, and read -r handles paths
+        # containing spaces without IFS games.
+        printf '%s\n' "$dirs" > /tmp/dirs
+        fail=0
+        while IFS= read -r d; do
+          [ -n "$d" ] || continue
+          if kubectl kustomize "$d" >/dev/null 2>/tmp/err; then
+            echo "ok    $d"
+          else
+            echo "FAIL  $d"
+            sed 's/^/        /' /tmp/err
+            fail=1
+          fi
+        done < /tmp/dirs
+        exit $fail
+
+  # Lint the pipeline definitions themselves. This is the one that would have
+  # prevented the repo-wide CI outage: `woodpecker lint` exits 1 on a config that
+  # fails to compile, and exits 0 on mere warnings — so it fails on real breakage
+  # without blocking on pre-existing noise. Both behaviours were verified against
+  # this repo (rc=1 reproducing #443, rc=0 on the tree as it stands).
+  #
+  # An earlier revision of this comment also asserted that several deploy-*.yml
+  # files emit "Additional property timeout is not allowed" warnings. That was
+  # observed under the v3.9.0 CLI this step used to pin, and it is NOT known to
+  # still hold on v3.16.0 — review raised the likelihood that the warning was
+  # schema drift in a 7-minor-old CLI rather than real debt in those files, which
+  # is one of the reasons this pin moved to latest stable. It cannot be confirmed
+  # from a green run either, because the loop below sends lint output to /tmp/out
+  # and prints it only on failure, so warnings are invisible when the step passes.
+  # The claim is dropped rather than restated: what this step guarantees is the
+  # exit-code contract above, which is what the gate actually rests on.
+  # Latest stable, per the standing rule that any software uses the latest
+  # stable version. Matching the deployed woodpecker-server (v3.16.0) is a
+  # CONSEQUENCE of that, not a separate requirement — the server is bumped by
+  # image-automation within the 3.x range, so both land on the same release.
+  #
+  # It matters here because the CLI compiles the config against its own schema:
+  # one behind the server can pass a config the server then rejects, or reject
+  # one it accepts, and the gate stops predicting what the server will do —
+  # which is its only purpose.
+  #
+  # `-alpine` is required, not cosmetic: the plain vX.Y.Z CLI image ships no
+  # shell, and the loop below needs `for`, `sed` and `echo` around the binary.
+  # A step using the plain tag fails to start rather than failing a check.
+  #
+  # DRIFT RISK — accepted, and NOT fixable here. The server auto-bumps via
+  # image-automation; this pin cannot, so the two will drift apart.
+  # Image-automation can only patch YAML that is a Kubernetes manifest —
+  # fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the
+  # YAML manifest must have `apiVersion` and `kind` fields." A Woodpecker
+  # pipeline file has neither, so an `$imagepolicy` marker here is silently
+  # ignored. That is a property of the FILE, not of `update.path`: widening the
+  # automation's path to `./` would buy nothing, and would additionally drag the
+  # whole ansible/ and esphome/ trees into the walk. Do not try it — it looks
+  # like the fix and is not.
+  #
+  # So this bump is manual, and drift is caught by the update-audit agent
+  # (ci/agents/update-audit/prompt.md, item 5) which is told to flag this image
+  # whenever it trails the deployed server. If you bump the server, bump here.
+  - name: woodpecker-lint
+    depends_on: [] # see note above `steps:`
+    image: woodpeckerci/woodpecker-cli:v3.16.0-alpine
+    commands:
+      - |
+        set -eu
+        fail=0
+        for f in .woodpecker/*.yml; do
+          if /bin/woodpecker-cli lint "$f" >/tmp/out 2>&1; then
+            echo "ok    $f"
+          else
+            echo "FAIL  $f"
+            sed 's/^/        /' /tmp/out
+            fail=1
+          fi
+        done
+        exit $fail
+
+  # Standalone shell scripts under kubernetes/ (sidecar and bootstrap scripts
+  # mounted via configMapGenerator). Severity is pinned to `error` on purpose:
+  # style and info findings would fail the build on existing scripts for no
+  # safety gain. All scripts in the tree are clean at this level today.
+  #
+  # NOTE this does NOT cover shell embedded in manifests (init containers, probe
+  # exec blocks) — shellcheck cannot see inside YAML. That embedded shell is
+  # where several real bugs lived this year, so it remains a gap worth closing
+  # separately rather than one this step quietly implies is covered.
+  - name: shellcheck
+    depends_on: [] # see note above `steps:`
+    image: koalaman/shellcheck-alpine:v0.11.0
+    commands:
+      - |
+        set -eu
+        files=$(find kubernetes -name '*.sh' | sort)
+        # Same no-op guard as kustomize-build: if discovery ever returns nothing,
+        # fail loudly instead of passing having checked no files.
+        #
+        # This floor is 1 while kustomize-build's is 50, and the asymmetry is
+        # deliberate rather than an oversight. That floor is loose relative to the
+        # tree it guards, so it catches both total failure and gross partial loss
+        # while tolerating normal churn. The shell-script set is small enough that
+        # any floor tight enough to catch PARTIAL loss would sit inside normal
+        # churn — deleting one script would fail CI spuriously, and the fix a
+        # hurried person reaches for is lowering the floor, which is how a guard
+        # quietly becomes decoration. A floor that has to be edited whenever the
+        # tree changes is not a guard, it is a snapshot with an exit code.
+        #
+        # So this catches the failure that actually happened here (discovery
+        # silently returning nothing, step green having checked no files) and not
+        # partial loss. Worth revisiting only if kubernetes/ grows a lot more
+        # shell, at which point a loose proportionate floor becomes possible.
+        count=$(printf '%s\n' "$files" | grep -c . || true)
+        if [ "$count" -lt 1 ]; then
+          echo "FAIL: no shell scripts found under kubernetes/ — discovery is broken."
+          exit 1
+        fi
+        echo "discovered $count shell scripts"
+
+        # Same file-fed while-read reasoning as kustomize-build above.
+        printf '%s\n' "$files" > /tmp/files
+        fail=0
+        while IFS= read -r f; do
+          [ -n "$f" ] || continue
+          if shellcheck -S error "$f"; then
+            echo "ok    $f"
+          else
+            echo "FAIL  $f"
+            fail=1
+          fi
+        done < /tmp/files
+        exit $fail
+
+  # Guard the convention that cost a day to diagnose: registry-local images are
+  # pinned by DATED TAG, never by digest. Digests here are garbage-collected once
+  # :latest and the dated tags move past them — and that happens on edits to
+  # .woodpecker/build-ci-images.yml, a shared file, so an unrelated image addition
+  # silently orphans the manifest a rotator was pinned to. 11 rotators were dead
+  # this way and would have failed on their first scheduled run months later.
+  #
+  # Scoped to the rotation directory, which is where the convention is documented
+  # and where the failure actually bit.
+  #
+  # The `grep -v kustomization.yaml` is load-bearing, not tidying: that file
+  # documents this very rule, so its comment contains the literal string
+  # `image:.*@sha256` and therefore MATCHES the pattern. Anchoring on `image:`
+  # does not prevent that — verified by running the bare grep, which returns
+  # kustomization.yaml. Only the filename filter excludes it. Drop the filter and
+  # this step fails permanently on its own documentation.
+  - name: no-digest-pins
+    depends_on: [] # see note above `steps:`
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30
+    commands:
+      - |
+        set -eu
+        hits=$(grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml 2>/dev/null \
+                 | grep -v kustomization.yaml || true)
+        if [ -n "$hits" ]; then
+          echo "FAIL: digest-pinned images found; use a dated tag (see rotation/kustomization.yaml):"
+          echo "$hits" | sed 's/^/        /'
+          exit 1
+        fi
+        echo "ok    no digest pins under openbao/rotation/"

.woodpecker/ci-scripts-test.yml

@@ -0,0 +1,22 @@
+# Gate the CI helper-script unit tests on every homelab pull_request.
+#
+# scripts/ci/test-model-selection.sh is the regression test for the model-
+# selection injection gate (valid-model.sh) and the fetch-models.sh response
+# mapping. It is ALSO run on the daily/weekly crons via e2e.yml, but that would
+# only catch a weakening of the gate up to a week after it merged — by which
+# time pick-model.sh / pr-review.yml may have already run the weakened gate
+# against adversarial PR input. Running it here fails the offending PR's checks
+# before it can merge.
+#
+# Only in homelab (scripts/ci/ lives here); the file is not deployed to the
+# mirrored repos, so it does not run on their PRs. The test is offline (no token,
+# no network, no cluster) — the ci-claude image just provides sh + jq, and it is
+# already pulled by pr-review.yml on the same event, so the runner has it cached.
+when:
+  - event: pull_request
+
+steps:
+  - name: model-selection-tests
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
+    commands:
+      - sh scripts/ci/test-model-selection.sh

.woodpecker/deploy-claude-workstation.yml

@@ -0,0 +1,33 @@
+when:
+  # Deploy only on master (merge) + manual — never auto-deploy from a branch.
+  - event: push
+    branch: master
+    path:
+      - "ansible/roles/claude-workstation/**"
+      - "ansible/plays/install-claude-workstation.yml"
+      - ".woodpecker/deploy-claude-workstation.yml"
+  - event: manual
+    evaluate: 'PIPELINE == "deploy-claude-workstation"'
+
+timeout: 30
+
+steps:
+  - name: deploy-claude-workstation
+    image: gitea.renner.uno/gitea_admin/ci-ansible:2026-04-29
+    environment:
+      ANSIBLE_FORCE_COLOR: "true"
+      ANSIBLE_HOST_KEY_CHECKING: "false"
+      ANSIBLE_ROLES_PATH: "../roles:./roles"
+      SSH_PRIVATE_KEY:
+        from_secret: ssh_private_key
+      VAULT_PASS:
+        from_secret: vault_pass
+    commands:
+      - mkdir -p ~/.ssh
+      - echo "$SSH_PRIVATE_KEY" | base64 -d > ~/.ssh/id_ed25519
+      - chmod 600 ~/.ssh/id_ed25519
+      - printf "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n" > ~/.ssh/config
+      - cd ansible
+      - echo "$VAULT_PASS" > .vault_pass && chmod 600 .vault_pass
+      - export ANSIBLE_VAULT_PASSWORD_FILE=.vault_pass
+      - ansible-playbook -i inventory/ plays/install-claude-workstation.yml -v

.woodpecker/deploy-infrastructure.yml

@@ -1,5 +1,10 @@
 when:
+  # Plan only on master (merge) + manual — a push trigger without this gate
+  # would run `tofu plan` (which evaluates provider/data-source code) with
+  # all proxmox/hetzner/unifi secrets mounted for any pushed branch,
+  # including malicious feature-branch .tf.
   - event: push
+    branch: master
     path:
       - "opentofu/**"
       - ".woodpecker/deploy-infrastructure.yml"
@@ -126,9 +131,9 @@ steps:
       - |
         # Ensure Talos image exists on all Proxmox nodes via API
         TALOS_VERSION="v1.12.6"
-        TALOS_SCHEMATIC="ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515"
+        TALOS_SCHEMATIC="88e3752db032a08082972bd0adf16a80849e3870c3f95cf5b669936c5699e5ad"
         TALOS_URL="https://factory.talos.dev/image/$TALOS_SCHEMATIC/$TALOS_VERSION/nocloud-amd64.raw"
-        TALOS_FILENAME="talos-$TALOS_VERSION-nocloud-amd64.raw"
+        TALOS_FILENAME="talos-$TALOS_VERSION-$(echo "$TALOS_SCHEMATIC" | cut -c1-12)-nocloud-amd64.raw"
         API_TOKEN="$TF_VAR_proxmox_user!$TF_VAR_proxmox_token_id=$TF_VAR_proxmox_token_secret"
         for node in jupiter saturn neptune; do
           EXISTS=$(curl -sk "$TF_VAR_proxmox_api_url/api2/json/nodes/$node/storage/local/content" \

.woodpecker/deploy-netbird-client.yml

@@ -0,0 +1,35 @@
+when:
+  # Deploy only on master (merge) + manual — never auto-deploy from a branch.
+  - event: push
+    branch: master
+    path:
+      - "ansible/roles/netbird-client/**"
+      - "ansible/plays/install-netbird-client.yml"
+      - "ansible/inventory/group_vars/claude_workstation/**"
+      - ".woodpecker/deploy-netbird-client.yml"
+  - event: manual
+    evaluate: 'PIPELINE == "deploy-netbird-client"'
+
+timeout: 30
+
+steps:
+  - name: deploy-netbird-client
+    image: gitea.renner.uno/gitea_admin/ci-ansible:2026-04-29
+    environment:
+      ANSIBLE_FORCE_COLOR: "true"
+      ANSIBLE_HOST_KEY_CHECKING: "false"
+      ANSIBLE_ROLES_PATH: "../roles:./roles"
+      SSH_PRIVATE_KEY:
+        from_secret: ssh_private_key
+      VAULT_PASS:
+        from_secret: vault_pass
+    commands:
+      - mkdir -p ~/.ssh
+      - echo "$SSH_PRIVATE_KEY" | base64 -d > ~/.ssh/id_ed25519
+      - chmod 600 ~/.ssh/id_ed25519
+      - printf "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n" > ~/.ssh/config
+      - cd ansible
+      - echo "$VAULT_PASS" > .vault_pass && chmod 600 .vault_pass
+      - export ANSIBLE_VAULT_PASSWORD_FILE=.vault_pass
+      # Enrol only the off-site Claude workstation into the Netbird mesh.
+      - ansible-playbook -i inventory/ plays/install-netbird-client.yml --limit claude_workstation -v

.woodpecker/deploy-netbird-server.yml

@@ -1,5 +1,7 @@
 when:
+  # Deploy only on master (merge) + manual — never auto-deploy from a branch.
   - event: push
+    branch: master
     path:
       - "ansible/roles/netbird-server/**"
       # install-netbird-server.yml also applies the netbird-proxy-watchdog role,

.woodpecker/deploy-woodpecker-agent.yml

@@ -1,43 +0,0 @@
-when:
-  # Deploy only on master (merge) + manual — never from a feature branch.
-  - event: push
-    branch: master
-    path:
-      - "ansible/roles/woodpecker-agent/**"
-      - "ansible/plays/install-woodpecker-agent.yml"
-      - ".woodpecker/deploy-woodpecker-agent.yml"
-  - event: manual
-    evaluate: 'PIPELINE == "deploy-woodpecker-agent"'
-
-# NOTE: this deploys the agents *from* an agent, so applying a change that
-# restarts Docker (e.g. the daemon.json IPv6 config) will kill this job's own
-# container mid-run — the pipeline reports failure, but the change still
-# applies on both agents (the restart commands are dispatched in parallel
-# before the executing agent's daemon goes down). Re-run / re-trigger
-# deploy-netbird-proxy afterwards.
-timeout: 30
-
-steps:
-  - name: deploy-woodpecker-agent
-    image: gitea.renner.uno/gitea_admin/ci-ansible:2026-04-29
-    environment:
-      ANSIBLE_FORCE_COLOR: "true"
-      ANSIBLE_HOST_KEY_CHECKING: "false"
-      ANSIBLE_ROLES_PATH: "../roles:./roles"
-      SSH_PRIVATE_KEY:
-        from_secret: ssh_private_key
-      VAULT_PASS:
-        from_secret: vault_pass
-      PROXMOX_TOKEN_ID:
-        from_secret: proxmox_token_id
-      PROXMOX_TOKEN_SECRET:
-        from_secret: proxmox_token_secret
-    commands:
-      - mkdir -p ~/.ssh
-      - echo "$SSH_PRIVATE_KEY" | base64 -d > ~/.ssh/id_ed25519
-      - chmod 600 ~/.ssh/id_ed25519
-      - printf "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n" > ~/.ssh/config
-      - cd ansible
-      - echo "$VAULT_PASS" > .vault_pass && chmod 600 .vault_pass
-      - export ANSIBLE_VAULT_PASSWORD_FILE=.vault_pass
-      - ansible-playbook -i inventory/ plays/install-woodpecker-agent.yml -v

.woodpecker/e2e.yml

@@ -2,16 +2,33 @@
 # observability gap, OIDC config drift, RGW outage) on a schedule.
 #
 # Triggers:
-#   - cron     : configure schedule via Woodpecker UI / API (Cron tab on this
-#                repo). Suggested: daily at 06:00 UTC (`0 6 * * *`).
-#   - manual   : `woodpecker pipeline create --branch master --var PIPELINE=e2e gitea_admin/homelab`
+#   - cron     : scoped to the named daily/weekly crons below. This used to be
+#                a bare `event: cron` (run on EVERY cron of this repo), which
+#                was harmless at daily/weekly cadence — but alert-monitor.yml
+#                fires every 15 minutes, so piggybacking must be opt-in now.
+#                A new cron that SHOULD also run e2e gets its name added here.
+#   - manual   : `woodpecker pipeline create --branch master --var PIPELINE=e2e k.renner/homelab`
 
 when:
   - event: cron
+    cron: daily-security-scan
+  - event: cron
+    cron: update-audit
+  - event: cron
+    cron: weekly-update
   - event: manual
     evaluate: 'PIPELINE == "e2e"'
 
 steps:
+  - name: model-selection-tests
+    # Offline unit tests for the CI model-selection gate (valid-model.sh) and the
+    # fetch-models.sh response mapping. No cluster, no secrets, no network — the
+    # ci-claude image just provides sh + jq. Keeps the flag-injection regression
+    # test from rotting (it exercises the exact injection shapes).
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
+    commands:
+      - sh scripts/ci/test-model-selection.sh
+
   - name: e2e-test
     image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
     environment:

.woodpecker/install-woodpecker-server.yml

@@ -1,28 +0,0 @@
-when:
-  - event: manual
-    evaluate: 'PIPELINE == "install-woodpecker-server"'
-
-steps:
-  - name: run-playbook
-    image: gitea.renner.uno/gitea_admin/ci-ansible:2026-04-29
-    environment:
-      ANSIBLE_FORCE_COLOR: "true"
-      ANSIBLE_HOST_KEY_CHECKING: "false"
-      ANSIBLE_ROLES_PATH: "../roles:./roles"
-      SSH_PRIVATE_KEY:
-        from_secret: ssh_private_key
-      VAULT_PASS:
-        from_secret: vault_pass
-      PROXMOX_TOKEN_ID:
-        from_secret: proxmox_token_id
-      PROXMOX_TOKEN_SECRET:
-        from_secret: proxmox_token_secret
-    commands:
-      - mkdir -p ~/.ssh
-      - echo "$SSH_PRIVATE_KEY" | base64 -d > ~/.ssh/id_ed25519
-      - chmod 600 ~/.ssh/id_ed25519
-      - 'printf "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n" > ~/.ssh/config'
-      - cd ansible
-      - echo "$$VAULT_PASS" > .vault_pass && chmod 600 .vault_pass
-      - export ANSIBLE_VAULT_PASSWORD_FILE=.vault_pass
-      - ansible-playbook -i inventory/ plays/install-woodpecker-server.yml -v

.woodpecker/issue-fix.yml

@@ -0,0 +1,217 @@
+# Issue fixer — hourly, pick the oldest open "[alert] ..." Gitea issue (filed
+# by the alert-monitor pipeline, alert-monitor.yml) that nobody has worked on
+# yet, hand it to an AI agent, and open a PR with the fix. One issue per run
+# keeps runs bounded and PRs reviewable.
+#
+# "Not worked on yet" means BOTH:
+#   - no alert-fix/issue-<n> branch exists on origin (a branch means a PR was
+#     already opened — or is being opened by a concurrent run), and
+#   - no issue comment contains "<!-- alert-fix:assessed -->" (the marker this
+#     pipeline leaves when the agent concluded the problem is NOT fixable in
+#     the repo, so we don't burn an agent run on it every hour) — UNLESS a
+#     TRUSTED_AUTHORS comment is newer than the latest marker: an owner reply
+#     re-opens the issue for the agent, with the reply passed to it as
+#     pipeline-verified guidance (see below).
+#
+# Trusted owner guidance: comments by TRUSTED_AUTHORS are appended to the
+# agent's issue.md under "Verified owner guidance". The trust anchor is the
+# hardcoded author list in THIS file (reviewed via PR; agents cannot edit
+# .woodpecker/). Deliberately NOT the Gitea permission API: this pipeline's
+# own comments post as issue-fix-bot, which is itself a write collaborator,
+# so "has write access" cannot distinguish the human from the pipeline — and
+# an injected instruction inside an agent-authored comment must never be able
+# to masquerade as owner guidance (the #259 lesson, in both directions: the
+# agent rightly refused unauthenticated body text, but then had no channel
+# for the owner's authentic reply).
+#
+# Tokens: a single dedicated identity, issue-fix-bot, for both the issue
+# comments and the push/PR. Its PAT is scoped to write:issue +
+# write:repository and nothing else. It deliberately does NOT reuse
+# security_bot_token (that identity belongs to the security-review pipeline)
+# nor the admin-owned gitea_token — see #330.
+#
+# Triggering:
+#   - cron   : a cron NAMED exactly "issue-fix":
+#                woodpecker repo cron add --repo k.renner/homelab \
+#                  --name issue-fix --branch master --schedule '41 * * * *'
+#   - manual : woodpecker pipeline create --branch master \
+#                --var PIPELINE=issue-fix k.renner/homelab
+when:
+  - event: cron
+    cron: issue-fix
+  - event: manual
+    evaluate: 'PIPELINE == "issue-fix"'
+
+steps:
+  - name: fetch-diag
+    # Least-privilege live-diagnostics kubeconfig (+ kubectl binary) for the
+    # fix agent, same mechanism as alert-monitor.yml. Runs unconditionally
+    # (the issue-pick logic needs jq, which only the ci-claude image has);
+    # minting a 1h read-only token per run is cheap. Degrades to
+    # snapshot-only if the ci-diagnostics ServiceAccount isn't reconciled.
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    environment:
+      SOPS_AGE_KEY:
+        from_secret: sops_age_key
+      S3_ACCESS_KEY_ID:
+        from_secret: hetzner_s3_access_key
+      S3_SECRET_ACCESS_KEY:
+        from_secret: hetzner_s3_secret_key
+    commands:
+      - bash scripts/ci/fetch-kubeconfig.sh
+      - bash scripts/ci/make-diag-kubeconfig.sh
+
+  - name: issue-fix-agent
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
+    environment:
+      CLAUDE_CODE_OAUTH_TOKEN:
+        from_secret: claude_code_oauth_token
+      # Dedicated identity: issue-fix-bot, PAT scoped to write:issue +
+      # write:repository (it comments, pushes a branch, and opens a PR).
+      # One identity for both roles — the previous split borrowed
+      # security_bot_token, which belongs to the security-review pipeline
+      # and has no business in this trust domain (#330).
+      GITEA_TOKEN:
+        from_secret: issue_fix_bot_token
+      PUSH_TOKEN:
+        from_secret: issue_fix_bot_token
+    commands:
+      - |
+        # Pick the oldest unhandled [alert] issue. The repo is public, so the
+        # candidate reads are unauthenticated; only writes need tokens.
+        set -eu
+        TRUSTED_AUTHORS="k.renner"   # space-separated; the trust anchor (see header)
+        mkdir -p /tmp/alert-fix
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
+        curl -sfL "$$API/issues?state=open&type=issues&limit=100" \
+          | jq '[.[] | select(.title | startswith("[alert] "))] | sort_by(.number)' \
+          > /tmp/alert-fix/candidates.json
+        COUNT=$$(jq length /tmp/alert-fix/candidates.json)
+        echo "Open [alert] issues: $$COUNT"
+        TRUSTED_JSON=$$(printf '%s\n' $$TRUSTED_AUTHORS | jq -R . | jq -s -c .)
+        i=0
+        while [ "$$i" -lt "$$COUNT" ]; do
+          NUM=$$(jq -r ".[$$i].number" /tmp/alert-fix/candidates.json)
+          curl -sfL "$$API/issues/$$NUM/comments" > /tmp/alert-fix/comments.json
+          # Eligibility in jq (ISO timestamps compare lexically; shell test
+          # has no portable string-< in busybox ash): eligible unless an
+          # assessed marker exists with no NEWER trusted-author comment.
+          STATE=$$(jq -r --argjson t "$$TRUSTED_JSON" '
+            ([.[] | select(.body | contains("alert-fix:assessed")) | .created_at] | max // "") as $$mark
+            | ([.[] | select(.user.login as $$u | $$t | index($$u)) | .created_at] | max // "") as $$trust
+            | if $$mark == "" then "fresh"
+              elif $$trust > $$mark then "retry"
+              else "skip" end' /tmp/alert-fix/comments.json)
+          if git ls-remote --heads origin "alert-fix/issue-$$NUM" | grep -q .; then
+            echo "#$$NUM: branch alert-fix/issue-$$NUM already exists — skipping."
+          elif [ "$$STATE" = "skip" ]; then
+            echo "#$$NUM: assessed and no newer owner comment — skipping."
+          else
+            if [ "$$STATE" = "retry" ]; then echo "#$$NUM: owner replied after assessment — retrying with guidance."; fi
+            echo "Picking issue #$$NUM."
+            jq ".[$$i]" /tmp/alert-fix/candidates.json > /tmp/alert-fix/issue.json
+            {
+              printf '# %s\n\n' "$$(jq -r '.title' /tmp/alert-fix/issue.json)"
+              printf 'Issue number: %s\n\n' "$$NUM"
+              jq -r '.body' /tmp/alert-fix/issue.json
+              # Owner comments, authorship checked against TRUSTED_AUTHORS
+              # above. Everything in the section is authentic owner input;
+              # agent-authored comments never match (they post as
+              # issue-fix-bot) and marker comments are excluded regardless.
+              if [ "$$(jq -r --argjson t "$$TRUSTED_JSON" '[.[] | select((.user.login as $$u | $$t | index($$u)) and (.body | contains("alert-fix:") | not))] | length' /tmp/alert-fix/comments.json)" -gt 0 ]; then
+                printf '\n\n## Verified owner guidance (authorship checked by the pipeline)\n'
+                jq -r --argjson t "$$TRUSTED_JSON" '.[] | select((.user.login as $$u | $$t | index($$u)) and (.body | contains("alert-fix:") | not)) | "\n### \(.user.login), \(.created_at)\n\n\(.body)"' /tmp/alert-fix/comments.json
+              fi
+            } > /tmp/alert-fix/issue.md
+            break
+          fi
+          i=$$((i+1))
+        done
+        [ -f /tmp/alert-fix/issue.json ] || echo "Nothing to fix — all [alert] issues handled."
+      - |
+        # Branch + git identity. The bot user owns the commits so PRs are
+        # visibly machine-authored, and the identity names THIS pipeline —
+        # previously these commits and PRs were attributed to security-bot,
+        # which made alert fixes look like security-review output (#330).
+        [ -f /tmp/alert-fix/issue.json ] || exit 0
+        set -eu
+        NUM=$$(jq -r '.number' /tmp/alert-fix/issue.json)
+        BRANCH="alert-fix/issue-$$NUM"
+        git config user.email "issue-fix-bot@renner.uno"
+        git config user.name "Alert Fix Agent"
+        git checkout -b "$$BRANCH"
+        echo "$$BRANCH" > /tmp/alert-fix/branch
+      - |
+        # Headless fix run. Prompt piped via stdin, subscription OAuth, file
+        # tools plus EXACTLY ONE live command: the RBAC-scoped read-only
+        # kubectl from fetch-diag (no Secrets readable, prefix match forbids
+        # pipes and other binaries). No web (see security-scan.yml).
+        [ -f /tmp/alert-fix/issue.json ] || exit 0
+        export KUBECONFIG="$$PWD/.ci-diag/kubeconfig"
+        # Model chosen live from the catalog (not hardcoded, no family assumed) —
+        # see the header of scripts/ci/pick-model.sh. Empty MODEL -> CLI default.
+        MODEL=$$(sh scripts/ci/pick-model.sh "diagnose an [alert] issue against read-only live cluster state and fix the root cause by editing repo manifests; multi-step reasoning that changes infrastructure code and needs a capable model")
+        if [ -n "$$MODEL" ]; then echo "issue-fix: selected model $$MODEL"; else echo "issue-fix: no live catalog — using the CLI default model"; fi
+        cat ci/agents/issue-fix/prompt.md | claude $${MODEL:+--model $$MODEL} \
+          --permission-mode acceptEdits \
+          --allowedTools "Read,Edit,Write,Glob,Grep,Bash(./.ci-diag/kubectl:*)" \
+          --settings ci/agents/issue-fix/settings.json \
+          -p
+      - |
+        # No edits → the agent judged this not fixable in the repo. Post its
+        # summary (with the assessed-marker) as an issue comment so the next
+        # run skips this issue, and stop cleanly.
+        [ -f /tmp/alert-fix/issue.json ] || exit 0
+        set -eu
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
+        NUM=$$(jq -r '.number' /tmp/alert-fix/issue.json)
+        if git diff --quiet && git diff --cached --quiet \
+           && [ -z "$$(git ls-files --others --exclude-standard)" ]; then
+          echo "Agent made no edits — commenting assessment on issue #$$NUM."
+          [ -s /tmp/alert-fix/summary.md ] \
+            || echo "Agent made no edits and wrote no summary." > /tmp/alert-fix/summary.md
+          printf '\n\n<!-- alert-fix:assessed -->\n_Assessed by the issue-fix pipeline: no repo change proposed. Remove this comment to make the pipeline retry._\n' \
+            >> /tmp/alert-fix/summary.md
+          jq -n --rawfile body /tmp/alert-fix/summary.md '{body: $$body}' > /tmp/gitea-comment.json
+          code=$$(curl -s -o /tmp/gitea-resp.json -w '%{http_code}' \
+            -X POST -H "Authorization: token $$GITEA_TOKEN" \
+            -H "Content-Type: application/json" --data @/tmp/gitea-comment.json \
+            "$$API/issues/$$NUM/comments")
+          echo "POST comment -> HTTP $$code"
+          [ "$$code" -ge 200 ] && [ "$$code" -lt 300 ] || { cat /tmp/gitea-resp.json; exit 1; }
+          touch /tmp/alert-fix/no-edits
+        fi
+      - |
+        # Edits → commit, push, open the PR (Closes #<n> so the merge closes
+        # the issue), and link the PR from the issue.
+        [ -f /tmp/alert-fix/issue.json ] || exit 0
+        if [ -f /tmp/alert-fix/no-edits ]; then exit 0; fi
+        set -eu
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
+        NUM=$$(jq -r '.number' /tmp/alert-fix/issue.json)
+        BRANCH=$$(cat /tmp/alert-fix/branch)
+        TITLE="fix: $$(jq -r '.title' /tmp/alert-fix/issue.json) (#$$NUM)"
+        [ -s /tmp/alert-fix/summary.md ] \
+          || echo "Agent-proposed fix (no summary written by the agent)." > /tmp/alert-fix/summary.md
+        printf '\n\nCloses #%s\n' "$$NUM" >> /tmp/alert-fix/summary.md
+        git add -A
+        git commit -m "$$TITLE"
+        git push "https://$$PUSH_TOKEN@gitea.renner.uno/k.renner/homelab.git" HEAD:"$$BRANCH"
+        jq -n --arg title "$$TITLE" --rawfile body /tmp/alert-fix/summary.md \
+          --arg head "$$BRANCH" \
+          '{title: $$title, body: $$body, head: $$head, base: "master"}' > /tmp/gitea-pr.json
+        code=$$(curl -s -o /tmp/gitea-resp.json -w '%{http_code}' \
+          -X POST -H "Authorization: token $$PUSH_TOKEN" \
+          -H "Content-Type: application/json" --data @/tmp/gitea-pr.json \
+          "$$API/pulls")
+        echo "POST pull -> HTTP $$code"
+        [ "$$code" -ge 200 ] && [ "$$code" -lt 300 ] || { cat /tmp/gitea-resp.json; exit 1; }
+        jq '{number, html_url, title}' /tmp/gitea-resp.json
+        PR_URL=$$(jq -r '.html_url' /tmp/gitea-resp.json)
+        jq -n --arg body "🤖 Opened $$PR_URL with a proposed fix." '{body: $$body}' > /tmp/gitea-comment.json
+        code=$$(curl -s -o /tmp/gitea-resp.json -w '%{http_code}' \
+          -X POST -H "Authorization: token $$GITEA_TOKEN" \
+          -H "Content-Type: application/json" --data @/tmp/gitea-comment.json \
+          "$$API/issues/$$NUM/comments")
+        echo "POST comment -> HTTP $$code"
+        [ "$$code" -ge 200 ] && [ "$$code" -lt 300 ] || { cat /tmp/gitea-resp.json; exit 1; }

.woodpecker/pr-review.yml

@@ -0,0 +1,301 @@
+# PR review bot — event-driven. Woodpecker fires this on `pull_request` (which
+# covers BOTH "PR opened" and "new commits pushed to an open PR"), so a review
+# is produced the moment a PR appears or is updated — no polling, no cron.
+#
+# This file is deployed UNCHANGED into every repo the fleet wants reviewed
+# (k.renner/homelab, shelly-manager, authentik-samba-bridge, homeassistant).
+# Woodpecker events are per-repo, so each repo triggers a review of ITS OWN PR
+# only; the triggering PR is identified from the CI event (CI_REPO +
+# CI_COMMIT_PULL_REQUEST) instead of the old hourly cross-repo discovery loop.
+# Because it runs on the PR event, Woodpecker's built-in clone hands the step
+# THIS repo already checked out at the PR head — so, unlike the old cron model,
+# there is no manual clone of the target repo (the source of an earlier token
+# leak); the agents read the checkout in the workspace directly.
+#
+# Prompts: the two agent prompts live once, in k.renner/homelab under
+# ci/agents/pr-review/. The homelab checkout already has them; the other repos
+# don't, so the step falls back to a shallow clone of public homelab@master.
+# One source of truth for the review rubric across all repos.
+#
+# Model selection: no model id — and no model family — is baked into this file.
+# scripts/ci/fetch-models.sh pulls the LIVE catalog (Anthropic /v1/models, via
+# CLAUDE_CODE_OAUTH_TOKEN), and a triage agent reads the PR diff + stats and
+# picks whichever currently-available model best fits the review, judged on the
+# catalog's own capability metadata. A model released tomorrow is eligible the
+# day it appears. The shell validates the pick against the catalog snapshot taken
+# BEFORE the triage runs; if the fetch fails, the pick is rejected, or the review
+# is complex/security-sensitive (where the injectable pick is deliberately
+# ignored), it leaves the model unset and the review runs on the CLI's default
+# model. A wrong-way miss only over/under-spends one review.
+#
+# Posture: posted as a Gitea PR review, event COMMENT normally or
+# REQUEST_CHANGES on a serious defect. It NEVER approves — a bot must not
+# satisfy branch protection. Humans still merge.
+#
+# Dedup (guards pipeline retries/restarts, not cron re-runs anymore): the PR is
+# skipped when review-bot already left a review carrying the marker
+# "<!-- pr-review:sha=<head_sha> -->" for the PR's CURRENT head commit. A pushed
+# commit changes the head sha and fires a fresh pull_request event -> fresh
+# review.
+#
+# Tokens:
+#   CLAUDE_CODE_OAUTH_TOKEN (claude_code_oauth_token) — Claude subscription
+#     OAuth, how the `claude` CLI authenticates (same as the other agents).
+#   REVIEW_TOKEN (review_bot_token) — a dedicated `review-bot` Gitea account's
+#     PAT, scope write:repository. Reads the PR + diff and POSTs the review, all
+#     over the Gitea API (ephemeral curl Authorization headers — the token is
+#     never written to disk; the repo is NOT cloned). review-bot must be a
+#     collaborator (read) on every reviewed repo.
+#   Both secrets MUST be granted the `pull_request` event, or Woodpecker will
+#     withhold them from this pipeline. Because homelab is PUBLIC, it is also set
+#     to require approval for forked-PR pipelines, so an untrusted fork cannot
+#     rewrite this file to exfiltrate the tokens.
+#
+# Triggering: pull_request only — automatic on PR open + push-to-open-PR. To
+# re-review a PR, push a commit (the new head sha clears the dedup marker below
+# and earns a fresh review). There is deliberately no manual PIPELINE trigger:
+# Woodpecker exposes a --var only to YAML variable substitution and `when`
+# filters, NOT to a step's shell environment, so a --var PR=<n> re-review would
+# silently no-op (the step would find no PR number and exit green).
+# (Note: keep literal dollar-brace out of THIS file entirely, even in comments —
+# Woodpecker runs its variable substitution over the whole YAML text, comments
+# included, and an unparseable one errors the pipeline before any step runs.)
+when:
+  - event: pull_request
+
+steps:
+  - name: pr-review-agent
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
+    environment:
+      CLAUDE_CODE_OAUTH_TOKEN:
+        from_secret: claude_code_oauth_token
+      REVIEW_TOKEN:
+        from_secret: review_bot_token
+    commands:
+      - |
+        # ---- Resolve the single triggering PR --------------------------------
+        # Woodpecker hands us the repo (CI_REPO, canonical owner/repo) and the PR
+        # number (CI_COMMIT_PULL_REQUEST, a pull_request-event built-in). Build
+        # /tmp/pr-review/worklist.jsonl with 0 or 1 entries; an empty file makes
+        # the next block a no-op (the agent never starts).
+        set -eu
+        API="https://gitea.renner.uno/api/v1"
+        AUTH="Authorization: token $$REVIEW_TOKEN"
+        SLUG="$$CI_REPO"
+        NUM="$${CI_COMMIT_PULL_REQUEST:-}"
+        REVIEW_BOT_LOGIN="review-bot"
+        mkdir -p /tmp/pr-review
+        : > /tmp/pr-review/worklist.jsonl
+
+        if [ -z "$$NUM" ]; then
+          echo "No PR number in CI env (event=$${CI_PIPELINE_EVENT:-?}) — nothing to review."
+          exit 0
+        fi
+        echo "Triggered for $$SLUG#$$NUM (event $${CI_PIPELINE_EVENT:-?})."
+
+        if ! curl -sf -H "$$AUTH" "$$API/repos/$$SLUG/pulls/$$NUM" > /tmp/pr-review/pr.json; then
+          echo "Cannot fetch $$SLUG#$$NUM (review-bot not a collaborator / PR gone?) — skipping."
+          exit 0
+        fi
+        STATE=$$(jq -r '.state'        /tmp/pr-review/pr.json)
+        DRAFT=$$(jq -r '.draft // false' /tmp/pr-review/pr.json)
+        TITLE=$$(jq -r '.title'        /tmp/pr-review/pr.json)
+        HEAD=$$(jq -r '.head.sha'      /tmp/pr-review/pr.json)
+        SHORT=$$(printf '%s' "$$HEAD" | cut -c1-8)
+
+        # Skip closed/merged, drafts, and WIP-titled PRs — not ready for review.
+        case "$$STATE" in open) ;; *) echo "#$$NUM is $$STATE — skip."; exit 0;; esac
+        case "$$DRAFT" in true) echo "#$$NUM draft — skip."; exit 0;; esac
+        case "$$TITLE" in [Ww][Ii][Pp]:*|[Ww][Ii][Pp]\ *) echo "#$$NUM WIP — skip."; exit 0;; esac
+
+        # Dedup: already reviewed at THIS head sha? (guards re-runs / restarts)
+        REVIEWED=$$(curl -sf -H "$$AUTH" "$$API/repos/$$SLUG/pulls/$$NUM/reviews?limit=50" \
+          | jq -r --arg u "$$REVIEW_BOT_LOGIN" --arg m "pr-review:sha=$$HEAD" \
+              '[.[] | select(.user.login == $$u) | select(.body | contains($$m))] | length' \
+          || echo 0)
+        if [ "$${REVIEWED:-0}" -gt 0 ]; then
+          echo "#$$NUM already reviewed at $$SHORT — skip."
+          exit 0
+        fi
+
+        jq -c --arg slug "$$SLUG" \
+          '{slug: $$slug, number: .number, head: .head.sha, base: .base.sha,
+            title: .title, author: .user.login}' \
+          /tmp/pr-review/pr.json >> /tmp/pr-review/worklist.jsonl
+        echo "#$$NUM enqueued (head $$SHORT)."
+      - |
+        # ---- Clone, pick model, review, post ---------------------------------
+        set -eu
+        [ -s /tmp/pr-review/worklist.jsonl ] || { echo "Nothing to review."; exit 0; }
+        API="https://gitea.renner.uno/api/v1"
+        AUTH="Authorization: token $$REVIEW_TOKEN"
+        WS="$$PWD"                     # workspace = this repo, checked out at the PR head by Woodpecker
+        MAX_DIFF_LINES=20000
+
+        # Agent prompts: use the local homelab checkout if present, else clone
+        # public homelab@master for them (one source of truth for the rubric).
+        PROMPTS="$$WS/ci/agents/pr-review"
+        HOMELAB="$$WS"                 # repo root holding ci/ AND scripts/ci/
+        # NB: for a homelab PR, HOMELAB is the PR head checkout, so the
+        # fetch-models.sh run below executes branch-supplied shell. It is wrapped
+        # in `env -u REVIEW_TOKEN`, so the only credential it can see is
+        # CLAUDE_CODE_OAUTH_TOKEN. This is not a new exposure regardless: on
+        # pull_request Woodpecker already takes .woodpecker/*.yml itself from the
+        # PR branch, so a branch can run arbitrary commands in this step anyway.
+        if [ ! -f "$$PROMPTS/prompt.md" ]; then
+          rm -rf /tmp/pr-review/homelab
+          git clone --quiet --depth 1 https://gitea.renner.uno/k.renner/homelab \
+            /tmp/pr-review/homelab
+          PROMPTS=/tmp/pr-review/homelab/ci/agents/pr-review
+          HOMELAB=/tmp/pr-review/homelab
+        fi
+        echo "Using prompts from $$PROMPTS"
+
+        gitea_post() {                 # method url datafile ; echoes+checks HTTP
+          code=$$(curl -s -o /tmp/pr-review/resp.json -w '%{http_code}' \
+            -X "$$1" -H "$$AUTH" -H "Content-Type: application/json" \
+            --data @"$$3" "$$2")
+          echo "$$1 $$2 -> HTTP $$code"
+          if [ "$$code" -lt 200 ] || [ "$$code" -ge 300 ]; then
+            echo "Gitea API error body:"; cat /tmp/pr-review/resp.json; echo; return 1
+          fi
+        }
+
+        while IFS= read -r ITEM; do
+          SLUG=$$(printf '%s' "$$ITEM"  | jq -r '.slug')
+          NUM=$$(printf '%s' "$$ITEM"   | jq -r '.number')
+          HEAD=$$(printf '%s' "$$ITEM"  | jq -r '.head')
+          TITLE=$$(printf '%s' "$$ITEM" | jq -r '.title')
+          SHORT=$$(printf '%s' "$$HEAD" | cut -c1-8)
+          echo "=== Reviewing $$SLUG#$$NUM ($$SHORT) ==="
+          D="/tmp/pr-review/$$NUM"
+          rm -rf "$$D"; mkdir -p "$$D"
+
+          # PR metadata + unified diff (authenticated; private repos need it).
+          curl -sf -H "$$AUTH" "$$API/repos/$$SLUG/pulls/$$NUM" > "$$D/pr.json"
+          curl -sf -H "$$AUTH" "$$API/repos/$$SLUG/pulls/$$NUM.diff" > "$$D/diff.full.patch" \
+            || { echo "Could not fetch diff for $$SLUG#$$NUM — skipping."; continue; }
+          FILES=$$(grep -c '^diff --git ' "$$D/diff.full.patch" || true)
+          ADDED=$$(grep -c '^+[^+]' "$$D/diff.full.patch" || true)
+          DELETED=$$(grep -c '^-[^-]' "$$D/diff.full.patch" || true)
+          TOTAL=$$(wc -l < "$$D/diff.full.patch")
+          if [ "$$TOTAL" -gt "$$MAX_DIFF_LINES" ]; then
+            head -n "$$MAX_DIFF_LINES" "$$D/diff.full.patch" > "$$D/diff.patch"
+            printf '\n... [diff truncated at %s of %s lines] ...\n' \
+              "$$MAX_DIFF_LINES" "$$TOTAL" >> "$$D/diff.patch"
+          else
+            cp "$$D/diff.full.patch" "$$D/diff.patch"
+          fi
+          printf 'repo: %s\nPR: #%s\nfiles_changed: %s\nlines_added: %s\nlines_deleted: %s\ndiff_total_lines: %s\n' \
+            "$$SLUG" "$$NUM" "$$FILES" "$$ADDED" "$$DELETED" "$$TOTAL" > "$$D/stats.txt"
+
+          # Cheaper inputs for the TRIAGE agent (which now runs on a capable
+          # default model): the COMPLETE changed-file list, plus a diff capped far
+          # smaller than the review's. Triage only gauges size/shape/sensitivity,
+          # so it never needs the full (up to 20k-line) review diff — this keeps
+          # "triage is cheap" true now that its model is no longer pinned to Haiku.
+          grep '^diff --git ' "$$D/diff.full.patch" | sed 's|^diff --git a/||; s| b/.*$$||' \
+            | sort -u > "$$D/files.txt"
+          head -n 4000 "$$D/diff.patch" > "$$D/diff.triage.patch"
+
+          # No manual clone of the target repo: this runs on a pull_request
+          # event, so Woodpecker's built-in clone already checked THIS repo out
+          # at the PR head in $$WS — the agents (which cd into $$WS) read it
+          # directly for context. The old cron model had to clone each target
+          # repo by hand (its workspace was homelab@master), and that clone
+          # embedded REVIEW_TOKEN in the URL, persisting a write-scoped PAT in
+          # work/target/.git/config — readable by the review agent whose report
+          # is posted publicly. Per-repo pull_request triggering makes the manual
+          # clone redundant AND keeps the token off disk (it now lives only in
+          # the ephemeral curl Authorization headers used for the API calls).
+
+          # Inputs the agents read live at ./.pr-review (symlink into $$D).
+          rm -f "$$WS/.pr-review"; ln -s "$$D" "$$WS/.pr-review"
+
+          # --- Triage: an agent picks the model from the LIVE catalog, so no
+          #     model id and no model family is baked into this file.
+          #     fetch-models.sh asks the Anthropic /v1/models list (via the same
+          #     CLAUDE_CODE_OAUTH_TOKEN) and we hand the triage the result at
+          #     ./.pr-review/models.txt. If the fetch fails the file is empty and
+          #     MODEL stays empty below, so the review runs on the CLI default.
+          if MODELS_TABLE=$$(env -u REVIEW_TOKEN sh "$$HOMELAB/scripts/ci/fetch-models.sh" --table); then
+            printf '%s\n' "$$MODELS_TABLE" > "$$D/models.txt"
+            echo "Live model catalog ($$(wc -l < "$$D/models.txt") models) written for triage."
+          else
+            : > "$$D/models.txt"
+            echo "Model catalog fetch failed — triage/review will use the CLI default model."
+          fi
+          # env -u REVIEW_TOKEN: the agents never call Gitea (the shell does, via
+          # $$AUTH), so the write:repository PAT must not sit in the agent's env
+          # where a prompt-injected diff could read /proc/self/environ. Defence in
+          # depth alongside the settings.json Read-sandbox. Triage itself runs on
+          # the CLI default model (no --model) — we don't name a family even to
+          # bootstrap the pick.
+          cat "$$PROMPTS/triage-prompt.md" | env -u REVIEW_TOKEN claude \
+            --permission-mode acceptEdits \
+            --allowedTools "Read,Glob,Grep,Write" \
+            --settings "$$PROMPTS/settings.json" \
+            -p || echo "(triage agent exited non-zero)"
+          # Use the triage's pick only if it is a currently-available id;
+          # otherwise (hallucinated id, or empty catalog) leave MODEL empty so
+          # the review runs on the CLI default — a run never dies on selection.
+          MODEL=$$(jq -r '.model // empty' "$$D/triage.json" 2>/dev/null || true)
+          CPLX=$$(jq -r '.complexity // "unknown"' "$$D/triage.json" 2>/dev/null || echo unknown)
+          # triage.json is UNTRUSTED (the triage reads the attacker-controlled
+          # diff and its Write is not path-scoped). valid-model.sh is the single,
+          # unit-tested gate (scripts/ci/test-model-selection.sh): it echoes the id
+          # only if it is a safe [A-Za-z0-9._-] token present in the catalog, else
+          # nothing — so a value like "claude-x\n--dangerously-skip-permissions"
+          # can never reach the unquoted --model expansion below. Empty -> default.
+          # CRITICAL: validate against $$MODELS_TABLE — the catalog captured in a
+          # shell variable BEFORE the triage ran — NOT the on-disk ./.pr-review/
+          # models.txt, which the triage's unscoped Write could overwrite to smuggle
+          # an attacker-chosen token past the membership check (the triage reads the
+          # file, but only $$MODELS_TABLE is trusted). This mirrors pick-model.sh,
+          # which validates its pick against its in-memory table for the same reason.
+          # env -u REVIEW_TOKEN: like fetch-models.sh above, this runs a script from
+          # $$HOMELAB (branch-supplied for a homelab PR) — it must not see the PAT.
+          # `|| true`: keep the "a run never dies on selection" invariant even if
+          # the script is absent (e.g. a PR that moves scripts/ci/).
+          MODEL=$$(printf '%s\n' "$$MODELS_TABLE" | env -u REVIEW_TOKEN sh "$$HOMELAB/scripts/ci/valid-model.sh" "$$MODEL" || true)
+          [ -n "$$MODEL" ] || echo "Triage model rejected or not in live catalog — using the CLI default model."
+          # Defence in depth: for a complex / security-sensitive review, do NOT
+          # trust the triage's pick (it read the attacker-controlled diff and could
+          # be injected into naming the weakest catalog entry to get a shallow
+          # review). Fall back to the CLI default — the strongest configured model.
+          if [ "$$CPLX" = complex ] && [ -n "$$MODEL" ]; then
+            echo "complex review — ignoring triage pick '$$MODEL', using the CLI default (strongest) model."
+            MODEL=
+          fi
+          if [ -n "$$MODEL" ]; then echo "Triage: complexity=$$CPLX -> model=$$MODEL"; else echo "Triage: complexity=$$CPLX -> CLI default model"; fi
+
+          # --- Review with the chosen model ---
+          ( cd "$$WS" && cat "$$PROMPTS/prompt.md" | env -u REVIEW_TOKEN claude $${MODEL:+--model $$MODEL} \
+              --permission-mode acceptEdits \
+              --allowedTools "Read,Glob,Grep,Write" \
+              --settings "$$PROMPTS/settings.json" \
+              -p ) || echo "(review agent exited non-zero)"
+
+          if [ ! -s "$$D/report.md" ]; then
+            echo "Agent wrote no report for $$SLUG#$$NUM — skipping post."
+            continue
+          fi
+          EVENT=$$(jq -r '.event // "COMMENT"' "$$D/verdict.json" 2>/dev/null || echo COMMENT)
+          case "$$EVENT" in REQUEST_CHANGES|COMMENT) ;; *) EVENT=COMMENT;; esac   # never APPROVE
+
+          # Assemble the review body: header + report + marker footer.
+          {
+            printf '🤖 **Automated review** — model `%s` (complexity: %s), commit `%s`.\n\n' \
+              "$${MODEL:-CLI default}" "$$CPLX" "$${HEAD}"
+            printf '> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.\n\n---\n\n'
+            cat "$$D/report.md"
+            printf '\n\n<!-- pr-review:sha=%s -->\n' "$$HEAD"
+          } > "$$D/body.md"
+          jq -n --rawfile b "$$D/body.md" --arg e "$$EVENT" --arg c "$$HEAD" \
+            '{body: $$b, event: $$e, commit_id: $$c}' > "$$D/review.json"
+          gitea_post POST "$$API/repos/$$SLUG/pulls/$$NUM/reviews" "$$D/review.json" \
+            && echo "Posted $$EVENT review on $$SLUG#$$NUM." \
+            || echo "FAILED to post review on $$SLUG#$$NUM (continuing)."
+          rm -f "$$WS/.pr-review"
+        done < /tmp/pr-review/worklist.jsonl
+        echo "PR review run complete."

.woodpecker/security-scan.yml

@@ -42,7 +42,7 @@ steps:
     # this step is skipped — otherwise we'd fork-bomb PRs.
     when:
       - branch: master
-    image: gitea.renner.uno/gitea_admin/ci-claude:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
     environment:
       CLAUDE_CODE_OAUTH_TOKEN:
         from_secret: claude_code_oauth_token
@@ -96,7 +96,11 @@ steps:
         # latter passes a multi-KB substituted string into the command
         # line which busybox ash's tracer (set -x) misparsed under
         # woodpecker, producing a spurious "unterminated quoted string".
-        cat ci/agents/security-fix/prompt.md | claude \
+        # Model chosen live from the catalog (not hardcoded, no family assumed) —
+        # see the header of scripts/ci/pick-model.sh. Empty MODEL -> CLI default.
+        MODEL=$$(sh scripts/ci/pick-model.sh "fix CRITICAL/HIGH CVEs and IaC misconfigurations (Trivy findings) in Kubernetes/Ansible manifests by editing files; multi-step, security-sensitive work touching container versions, RBAC and configuration that needs careful, capable reasoning")
+        if [ -n "$$MODEL" ]; then echo "security-fix: selected model $$MODEL"; else echo "security-fix: no live catalog — using the CLI default model"; fi
+        cat ci/agents/security-fix/prompt.md | claude $${MODEL:+--model $$MODEL} \
           --permission-mode acceptEdits \
           --allowedTools "Read,Edit,Write,Glob,Grep" \
           -p
@@ -115,15 +119,15 @@ steps:
         TITLE="security: agent-proposed fixes ($$(date +%Y-%m-%d))"
         git add -A
         git commit -m "$$TITLE"
-        git push "https://$$GITEA_TOKEN@gitea.renner.uno/gitea_admin/homelab.git" HEAD:"$$BRANCH"
+        git push "https://$$GITEA_TOKEN@gitea.renner.uno/k.renner/homelab.git" HEAD:"$$BRANCH"
         jq -n \
           --arg title "$$TITLE" \
           --arg body  "$$BODY"  \
           --arg head  "$$BRANCH" \
           '{title: $$title, body: $$body, head: $$head, base: "master"}' \
-        | curl -sf -X POST \
+        | curl -sfL -X POST \
             -H "Authorization: token $$GITEA_TOKEN" \
             -H "Content-Type: application/json" \
             --data @- \
-            https://gitea.renner.uno/api/v1/repos/gitea_admin/homelab/pulls \
+            https://gitea.renner.uno/api/v1/repos/k.renner/homelab/pulls \
         | jq '{number, html_url, title}'

.woodpecker/update-audit.yml

@@ -14,14 +14,14 @@
 #              here (which gate on an injected `PIPELINE` var set at cron-create
 #              time in the UI), this uses Woodpecker's documented `cron:` name
 #              filter so the schedule can be created entirely from the CLI:
-#                woodpecker repo cron add --repo gitea_admin/homelab \
+#                woodpecker repo cron add --repo k.renner/homelab \
 #                  --name update-audit --branch master --schedule '@weekly'
 #   - manual : woodpecker pipeline create --branch master \
-#                --var PIPELINE=update-audit gitea_admin/homelab
+#                --var PIPELINE=update-audit k.renner/homelab
 #
-# NOTE: e2e.yml has a bare `event: cron` (no filter), so it runs on EVERY cron,
-# including this one — a weekly e2e run will piggyback on the audit. That's a
-# pre-existing property of e2e.yml, not something this pipeline introduces.
+# NOTE: e2e.yml runs on an explicit list of cron names that includes this one,
+# so a weekly e2e run piggybacks on the audit (it was a bare `event: cron`
+# until the 15-minute alert-monitor cron made that untenable).
 when:
   - event: cron
     cron: update-audit
@@ -30,19 +30,25 @@ when:
 
 steps:
   - name: update-audit-agent
-    image: gitea.renner.uno/gitea_admin/ci-claude:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-claude:2.1.218
     environment:
       CLAUDE_CODE_OAUTH_TOKEN:
         from_secret: claude_code_oauth_token
-      # security_bot_token is scoped for code/PRs only — its issue POST
-      # returned HTTP 403 "token does not have ... write:issue", so the
-      # audit ran green but never created an issue. Switch to gitea_token,
-      # the broader machine account already used by build-caddy /
-      # build-ci-images for release + registry writes, which is the most
-      # likely existing secret to also carry write:issue. Audit issues are
-      # then attributed to this token's owner rather than the security bot.
+      # Dedicated identity: update-audit-bot, PAT scoped to write:issue ONLY.
+      #
+      # This resolves the gap #197 tracked. The history: security_bot_token is
+      # scoped for code/PRs, so its issue POST returned 403 and the audit ran
+      # green while creating nothing; the fix at the time was the admin-owned
+      # gitea_token, which handed a broad credential to a WebFetch-capable
+      # agent that reads untrusted third-party release pages. That is now a
+      # narrow token that cannot do anything but write issues — verified 403
+      # on /user/repos and /admin/users. See #330.
+      #
+      # The other mitigations still apply and are still load-bearing: tool
+      # scoping (no WebSearch), the Read sandbox in settings.json, and the
+      # prompt hardening that treats fetched content as untrusted data.
       GITEA_TOKEN:
-        from_secret: gitea_token
+        from_secret: update_audit_bot_token
     commands:
       - |
         # Hand off to Claude Code in headless mode. Same auth model as the
@@ -51,9 +57,38 @@ steps:
         # pins, fetches upstream releases, and writes a report to /tmp. It
         # has NO Edit/Bash, so it cannot mutate the repo. Prompt is piped via
         # stdin (see security-scan.yml for why -p "$(cat ...)" is avoided).
-        cat ci/agents/update-audit/prompt.md | claude \
+        #
+        # Security hardening (gitea issue #197 — prompt-injection exfil via
+        # the untrusted release pages this agent fetches):
+        #   - WebSearch is dropped. It can't be domain-scoped (unlike
+        #     WebFetch) and the prompt already gives explicit source URLs or
+        #     a well-known registry (Artifact Hub / chart source repo) for
+        #     everything it needs to look up.
+        #   - --settings loads ci/agents/update-audit/settings.json, which
+        #     denies Read on /proc, /root, /home, /etc, and this repo's own
+        #     .git/ — closing off the credential-file and env-dump read
+        #     paths a prompt-injected instruction could otherwise try.
+        #     (Write path-scoping was tested and found NOT to be enforced
+        #     under --permission-mode acceptEdits in the current Claude Code
+        #     CLI, so it's intentionally not relied on here; the pipeline
+        #     only ever trusts /tmp/update-audit.md regardless of what else
+        #     the agent writes.)
+        #   - prompt.md now tells the agent fetched content is untrusted
+        #     data, not instructions, and to never echo it verbatim.
+        # A full WebFetch domain allowlist was considered and rejected: the
+        # Helm-chart-version check legitimately needs to reach whichever
+        # chart-repo domain each HelmRelease's own (already-trusted)
+        # HelmRepository/OCIRepository URL points at — a fixed allowlist
+        # across ~20 charts would need updating every time a chart is added
+        # and would silently stop auditing anything not yet listed.
+        # Model chosen live from the catalog (not hardcoded, no family assumed) —
+        # see the header of scripts/ci/pick-model.sh. Empty MODEL -> CLI default.
+        MODEL=$$(sh scripts/ci/pick-model.sh "weekly, read-only homelab infrastructure update audit: compare Talos/Kubernetes/Flux/Helm/Ansible/OpenTofu version pins against upstream releases and write a drift report; moderate-difficulty analysis, no code changes")
+        if [ -n "$$MODEL" ]; then echo "update-audit: selected model $$MODEL"; else echo "update-audit: no live catalog — using the CLI default model"; fi
+        cat ci/agents/update-audit/prompt.md | claude $${MODEL:+--model $$MODEL} \
           --permission-mode acceptEdits \
-          --allowedTools "Read,Glob,Grep,Write,WebFetch,WebSearch" \
+          --allowedTools "Read,Glob,Grep,Write,WebFetch" \
+          --settings ci/agents/update-audit/settings.json \
           -p
       - |
         # Bail cleanly if the agent produced no report.
@@ -76,7 +111,7 @@ steps:
         set -eu
         TITLE="Infrastructure update audit"
         STAMP=$$(date -u +%Y-%m-%dT%H:%M:%SZ)
-        API="https://gitea.renner.uno/api/v1/repos/gitea_admin/homelab"
+        API="https://gitea.renner.uno/api/v1/repos/k.renner/homelab"
 
         # write <method> <url> <payload-file>  -> prints HTTP code, fails non-2xx
         gitea_write() {
@@ -90,7 +125,7 @@ steps:
           fi
         }
 
-        ISSUE=$$(curl -sf -H "Authorization: token $$GITEA_TOKEN" \
+        ISSUE=$$(curl -sfL -H "Authorization: token $$GITEA_TOKEN" \
           "$$API/issues?state=open&type=issues&limit=50" \
           | jq -r --arg t "$$TITLE" 'map(select(.title == $$t)) | .[0].number // empty')
 

ansible/.vscode/settings.json

@@ -23,34 +23,6 @@
         "/^cd /home/krenner/opentofu && tofu validate$/": {
             "approve": true,
             "matchCommandLine": true
-        },
-        "/^ssh root@10\\.2\\.40\\.214 \"journalctl -u act_runner -n 50\"$/": {
-            "approve": true,
-            "matchCommandLine": true
-        },
-        "/^ssh -o StrictHostKeyChecking=no root@gitea-runner-1 \"journalctl -u act_runner -n 100 --no-pager\"$/": {
-            "approve": true,
-            "matchCommandLine": true
-        },
-        "/^ssh -o StrictHostKeyChecking=no root@gitea-runner-1 \"journalctl -u act_runner -n 50 --no-pager\"$/": {
-            "approve": true,
-            "matchCommandLine": true
-        },
-        "/^ssh -o StrictHostKeyChecking=no root@gitea-runner-1 \"journalctl -u act_runner -n 200 --no-pager\"$/": {
-            "approve": true,
-            "matchCommandLine": true
-        },
-        "/^sleep 5 && ssh -o StrictHostKeyChecking=no root@gitea-runner-1 \"journalctl -u act_runner -n 200 --no-pager\"$/": {
-            "approve": true,
-            "matchCommandLine": true
-        },
-        "/^sleep 15 && ssh -o StrictHostKeyChecking=no root@gitea-runner-1 \"journalctl -u act_runner -n 200 --no-pager\"$/": {
-            "approve": true,
-            "matchCommandLine": true
-        },
-        "/^sleep 20 && ssh -o StrictHostKeyChecking=no root@gitea-runner-1 \"journalctl -u act_runner -n 200 --no-pager\"$/": {
-            "approve": true,
-            "matchCommandLine": true
         }
     }
 }

ansible/docs/edge-host.md

@@ -33,13 +33,43 @@ Build and boot the dedicated installer image — see
    (partitions the internal disk — NVMe if present, else the eMMC; the
    installer autodetects — installs the ansible-ready base system with SSH
    keys, DHCP, docker — asks for confirmation before wiping).
-3. Add the DHCP reservation/DNS entry for `edge-1.srv.renner.uno` on the
-   UDM, confirm `ssh root@edge-1.srv.renner.uno` works.
-4. From the workstation:
+3. Confirm the box resolves and is reachable by name — `ssh root@<host>` must
+   work before anything else. The UDM registers DHCP leases in DNS, so the
+   record appears on its own; the host is identified by name everywhere
+   (inventory, plays, this file) and its address is never recorded. Do not
+   pin it to a reservation.
+4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
+   That group is the base: every edge thin client belongs to it, and it
+   assumes no peripherals.
+5. From the workstation:
    ```
-   ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host
    ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
    ```
+   One unrestricted run over the whole group. The play is self-contained --
+   it carries `certificates` and `common` itself, so there is no second
+   command and no `--limit`. Running it converges every edge host, which is
+   the point: a run that touches only the new box tells you nothing about
+   the others.
+
+## Peripherals
+
+Each attached device is its own inventory group and its own playbook. Group
+membership states which hardware a host physically has -- there are no flags
+and no `when:` on the roles, so moving a stick between boxes is a one-line
+inventory change.
+
+| device | group | playbook |
+|---|---|---|
+| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
+| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
+| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
+
+Add the host to the matching group, then run that playbook. They depend on
+the base play having run: all three need the local MQTT bridge, and two of
+them need docker.
+
+A host with nothing attached is simply absent from all three groups and needs
+none of these commands.
 
 ## Secrets (before first play run)
 
@@ -85,7 +115,7 @@ zwave-js (order matters — never two servers on one radio):
       `kubectl -n home-assistant exec deploy/zwave-js -- tar cf - -C /usr/src/app store | tar xf - -C /opt/edge-host/zwave-js/`
 - [ ] Scale down in-cluster: `kubectl -n home-assistant scale deploy/zwave-js --replicas 0`
 - [ ] Move the ZWA-2 stick to edge-1, `docker compose up -d zwave-js`
-- [ ] Point HA `zwave_js` integration at `ws://edge-1.srv.renner.uno:3000`
+- [ ] Point HA `zwave_js` integration at `ws://edge-1.iot.renner.uno:3000`
 - [ ] Nodes alive in the UI on :8091 → remove the flux kustomization
       `home-assistant-zwavejs` + PVC in a follow-up PR
 
@@ -113,8 +143,12 @@ Afterwards:
   deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
   changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
   the preloaded `.so` (see "Logging & eMMC" → zwave-js).
-- open3e updates: re-run the play; the pip task tracks the git default
-  branch (pin `edge_open3e_pkg` to a tag if that ever bites).
+- open3e / E3onCAN updates: both are pinned to commits in
+  `roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
+  up upstream changes -- that is deliberate, these drive the heat pump. The
+  weekly update-audit reports when either falls behind (see
+  `ci/agents/update-audit/prompt.md`); bump the pin and re-run.
+  Pinning to a tag is not an option for open3e: it publishes none.
 - Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
   restart afterwards.
 - can-utils (`candump`) is AUR-only on Arch — install manually via

ansible/inventory/group_vars/all/vars.yml

@@ -6,7 +6,26 @@ ansible_user: "ladmin"
 ansible_python_interpreter: /usr/bin/python3
 
 # SSH Public Keys for Cloud-Init
+# ansible-runner@ci-new removed (issue #184): dead key, no such host exists.
 common_ssh_public_keys: |
   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPzFNBP3bq2m/6JqDqod9w+FP/uxXxHMolqhq3Ojflr1 krenner@fortress
-  ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGfe3HJqFR+jIGrboweseAO7XjIVE24sje/b1HybgU9C ansible-runner@ci-new
   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICyNnX1I4W9jyDvCJ1DJauHSmbo1ZXSyBxVhSAwMWgen woodpecker-ci@homelab
+
+# Network-of-origin restriction for the woodpecker-ci@homelab CI key
+# (issue #184). Consumed by plays/configure-ssh-keys.yml as the authorized_key
+# `from=` option so the CI key only authenticates from within the cluster's
+# pod network — NOT from any host-pinned IP.
+#
+# The in-cluster Woodpecker step pods SSH to fleet hosts using their POD IP as
+# source, not a node IP: Cilium runs `ipv4-native-routing-cidr=10.0.0.0/8` +
+# `ipv6-native-routing-cidr=fd00::/8` with masquerade enabled, so pod egress to
+# any 10.x / fd.. LAN destination is NOT SNAT'd, and each node's PodCIDR is
+# advertised to the UDM over BGP (kubernetes/infrastructure/post-crds/
+# cilium-bgp.yaml) so return traffic L3-routes back to the pod. Hosts therefore
+# see the real pod source IP.
+#
+# The cluster pod subnets (kubernetes/talos/patches/cluster.yaml podSubnets)
+# are the stable boundary — they survive node DHCP-lease churn and pod
+# rescheduling across k8s-1/2/3, unlike pinning any node's current IP.
+# v6 is included because some SSH targets (edge IoT hosts) are IPv6-only.
+ci_ssh_from_cidrs: '10.244.0.0/16,fd10:244::/48'

ansible/inventory/group_vars/claude_workstation/vault.yml

@@ -0,0 +1,16 @@
+$ANSIBLE_VAULT;1.1;AES256
+35646538363164363061366433343963623166336630376233333033646637336266343337343230
+3233356135323762636439336466663430366238363338650a633762623366643137303333383634
+62623739363132303335636261626263333332623066303161633466306431623930666166323833
+6439663964353338350a656435626664646666363831363164393335386666356637613037323838
+63653461656366613331353064656161346263356131346437666632386166623037373363333534
+37323435623366643534633333303534313434613135653237303966656162633266623433363934
+36343133393834393765353338383764303664363431653865633764306138376366623832346262
+36303766623331666632613639366139333661636665363631326565653938343637323861653064
+37393465383263376337376639613162306563653365623036376662303030613264653534353332
+66363030356363373439306164386561363734353631363738656236633164633062613437663366
+62386430396262366336316166353366383233316333333234666538643962336361363630363762
+31653963303032373531386566336632336130383034343430303964396538663237346434613737
+39643765623766373639306132316635376434643232336163303066393736313130323065666232
+39393038616162666666383837326137376233613437636361636238663566336365653863643933
+353338646134383038323134323034653564

ansible/inventory/group_vars/edge_host/vars.yml

@@ -1,6 +1,6 @@
 ---
 # Secret indirection — real values live in vault.yml (ansible-vault) next
-# to this file; see roles/edge-host/README.md for how to populate it from
+# to this file; see docs/edge-host.md for how to populate it from
 # the cluster secrets.
 edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
 edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
@@ -9,3 +9,45 @@ edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"
 # loki.renner.uno ingress; the matching htpasswd line lives in the cluster
 # secret kubernetes/apps/monitoring/loki-basic-auth.sops.yaml).
 edge_loki_password: "{{ vault_edge_loki_password }}"
+
+# --- NTP (see plays/install-ntp-ha.yml) ---
+# edge-1 serves NTP from its untagged VLAN 30 address (10.2.30.108); clients on
+# other VLANs reach it via inter-VLAN routing, so no tagged interface is needed
+# for chrony itself.
+#
+# keepalived stays off until enp1s0.40 exists: VRRP needs layer-2 adjacency with
+# the PVE members on VLAN 40, and the host currently has only untagged VLAN 30
+# (the switch port now permits VLAN 40, but /etc/systemd/network/ has no matching
+# .netdev/.network pair). Adding it needs care — 20-wired.network matches
+# 'Type=ether' broadly, and this box is headless.
+ntp_keepalived_enabled: true
+ntp_keepalived_interface: "enp1s0.40"
+
+# Tagged VLAN 40 sub-interface (switch port permits VLAN 40; VLAN 30 stays
+# untagged/native). Prerequisite for ntp_keepalived_enabled.
+edge_vlan40_enabled: true
+
+# VRRP needs ip_nonlocal_bind but NOT ip_forward. The role's default enables
+# both (correct on the PVE hypervisors); enabling forwarding here would make
+# edge-1 a router between IoT VLAN 30 and service VLAN 40 once the tagged
+# interface exists, bypassing the UDM's inter-VLAN policy.
+keepalived_ntp_sysctl:
+  - {name: "net.ipv4.ip_nonlocal_bind", value: "1"}
+
+# Realtek NIC on the Wyse 3040 has no hardware timestamping.
+chrony_hwtimestamp: false
+
+# --- Local MQTT contract -------------------------------------------------
+# Shared between the broker and its clients, so it belongs to the inventory
+# rather than to either side's role. mqtt-bridge listens here; heating-can and
+# power-meter-sml publish here; and the bridge authenticates to the cluster
+# broker AS the open3e user, so it needs that identity too.
+#
+# Keeping these in one role's defaults only worked while a single play loaded
+# every role at once. Splitting the plays by capability exposed it: the bridge
+# could not render without a heating-can default, and the CAN services could
+# not render without an mqtt-bridge default.
+edge_mqtt_host: 127.0.0.1
+edge_mqtt_port: 1883
+edge_open3e_mqtt_user: open3e
+edge_open3e_mqtt_topic: open3e

ansible/inventory/group_vars/ntp_cluster.yml

@@ -0,0 +1,18 @@
+---
+# Shared configuration for every NTP server (see plays/install-ntp-ha.yml).
+
+# IPv6 service address -- what ntp.srv.renner.uno resolves to, and the path
+# clients actually use. The IPv4 VIP is legacy.
+#
+# Defined at group level rather than in the play because install-ntp-ha.yml runs
+# two plays over this group -- withdraw the static assignments fleet-wide, then
+# hand the address to keepalived -- and both need the same value. Duplicating a
+# service address across plays is exactly the sort of thing that drifts.
+ntp_vip6: "fd40::123/128"
+
+# Link-local address carried as the FIRST virtual_ipaddress of the IPv6 VRRP
+# instance. RFC 5798 gives an IPv6 virtual router a link-local primary, and
+# keepalived warns "the first IPv6 VIP address should be link local" without
+# one. fe80::123 mirrors the service address and cannot collide with a real
+# NIC's EUI-64 link-local.
+ntp_vip6_linklocal: "fe80::123/64"

ansible/inventory/group_vars/pve_cluster.yml

@@ -0,0 +1,8 @@
+---
+# --- NTP (see plays/install-ntp-ha.yml) ---
+# The PVE hosts hold the 10.2.40.4 VIP via keepalived on the VLAN 40 bridge
+# sub-interface. This role disappears when these nodes are rebuilt as bare-metal
+# Talos, which cannot run chrony or keepalived — edge-1 is being brought into
+# ntp_cluster to take over.
+ntp_keepalived_enabled: true
+ntp_keepalived_interface: "vmbr1.40"

ansible/inventory/group_vars/woodpecker_agents/vars.yml

@@ -1,2 +0,0 @@
----
-woodpecker_grpc_secret: "{{ vault_woodpecker_grpc_secret }}"

ansible/inventory/group_vars/woodpecker_server/vars.yml

@@ -1,4 +0,0 @@
----
-woodpecker_grpc_secret: "{{ vault_woodpecker_grpc_secret }}"
-woodpecker_gitea_client: "{{ vault_woodpecker_gitea_client }}"
-woodpecker_gitea_secret: "{{ vault_woodpecker_gitea_secret }}"

ansible/inventory/pve_hosts.yml

@@ -1,7 +1,19 @@
+# Reached by name, like every other host in this file. These three were the last
+# entries still addressed by IPv4 literal; authentik, netbird, proxy, nas-1,
+# grafana and edge-1 have always used their FQDN.
+#
+# An address in an inventory is allocator state standing in for identity, and it
+# pins access to the legacy family -- the names resolve IPv6-first, which is the
+# primary path on this network. DNS is the authoritative source for where a host
+# is; this file should say which host, not where it currently sits.
+#
+# The short inventory_hostname is kept deliberately: plays key on it
+# (ntp_servers[inventory_hostname] in install-ntp-ha.yml, node names in opentofu,
+# the prometheus targets), so only ansible_host changes.
 [pve_cluster]
-jupiter ansible_host=10.2.40.201 ansible_user=root
-neptune ansible_host=10.2.40.202 ansible_user=root
-saturn ansible_host=10.2.40.203 ansible_user=root
+jupiter ansible_host=jupiter.renner.uno ansible_user=root
+neptune ansible_host=neptune.renner.uno ansible_user=root
+saturn ansible_host=saturn.renner.uno ansible_user=root
 
 [authentik_db]
 authentik-db-1.srv.renner.uno ansible_user=root
@@ -15,16 +27,12 @@ authentik_app
 authentik-app-1.srv.renner.uno ansible_user=root
 authentik-app-2.srv.renner.uno ansible_user=root
 
-[proxmox_dns_ha]
-proxmox-dns-1.srv.renner.uno ansible_host=10.2.40.54 ansible_user=root
-proxmox-dns-2.srv.renner.uno ansible_host=10.2.40.55 ansible_user=root
-proxmox-dns-3.srv.renner.uno ansible_host=10.2.40.56 ansible_user=root
-
 [netbird_server]
 netbird.renner.uno ansible_user=root
 
 [netbird_proxy]
 proxy-1.renner.uno ansible_user=root
+proxy-2.renner.uno ansible_user=root
 
 
 [synology_nas]
@@ -33,12 +41,48 @@ nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
 [grafana]
 grafana.srv.renner.uno ansible_user=ladmin
 
-[woodpecker_server]
-woodpecker.srv.renner.uno ansible_user=root
-
-[woodpecker_agents]
-wp-agent-1.srv.renner.uno ansible_user=root
-wp-agent-2.srv.renner.uno ansible_user=root
-
+# Every bare-metal edge thin client. Gets the host-generic roles only:
+# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
+# sub-interface. Nothing here assumes any peripheral is attached.
 [edge_host]
 edge-1.iot.renner.uno ansible_user=root
+
+# Capability groups. Membership states which hardware a host physically has,
+# and each service playbook targets the group for its own concern. No flags and
+# no `when:` on the roles -- moving a stick to another box is a one-line change
+# here, which is what it should be.
+#
+# These are children of [edge_host], declared below: a peripheral is added to an
+# edge client, it does not replace the base. Making that a :children relation
+# rather than a comment means a host added only to a capability group still
+# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
+# cannot silently come up without edge_mqtt_host and friends.
+[zwave_gateway]
+edge-1.iot.renner.uno
+
+[heating_can]
+edge-1.iot.renner.uno
+
+[power_meter]
+edge-1.iot.renner.uno
+
+[edge_host:children]
+zwave_gateway
+heating_can
+power_meter
+
+# NTP servers. Includes edge-1 so the role survives the PVE hosts being
+# retired for bare-metal Talos, which cannot run chrony or keepalived itself.
+# See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly
+# than chrony within this group.
+[ntp_cluster:children]
+pve_cluster
+edge_host
+
+# Off-site Debian VM (Synology VMM on nas-2, 10.3.40.0/24) running Claude Code.
+# Reached over the site-to-site tunnel at its current DHCP lease. Switch
+# ansible_host to the stable Netbird mesh IP once the host is enrolled — the
+# obg.renner.uno FQDN is split-horizon and resolves to the public proxy from
+# home, so it must NOT be used as ansible_host.
+[claude_workstation]
+claude-ws ansible_host=10.3.40.78 ansible_user=root

ansible/plays/archived/cleanup-gitea-mess.yml

@@ -7,8 +7,18 @@
     gitea_binary: "/usr/local/bin/gitea"
     gitea_conf_dir: "/etc/gitea"
     gitea_service_file: "/etc/systemd/system/gitea.service"
-    
+    # Safety switch: must be set to true explicitly. This play is archived
+    # (one-off cleanup of an accidental install) and deletes the gitea
+    # service/binary/config/user on every host except gitea.srv.renner.uno
+    # if run unguarded.
+    confirm_destroy: false
+
   tasks:
+    - name: Safety Check
+      ansible.builtin.fail:
+        msg: "This archived playbook removes the gitea service/binary/config/user on all hosts except gitea.srv.renner.uno. You must run with -e 'confirm_destroy=true'."
+      when: not confirm_destroy | bool
+
     - name: Stop and disable gitea service
       systemd:
         name: gitea

ansible/plays/archived/remove_ceph_cpool.yml

@@ -2,7 +2,17 @@
 - name: Remove cpool storage and uninstall Ceph
   hosts: pve_cluster
   become: true
+  vars:
+    # Safety switch: must be set to true explicitly. This play is archived
+    # (one-off Ceph decommission) and destroys ZFS volumes fleet-wide via
+    # `zfs destroy -rf` if run unguarded.
+    confirm_destroy: false
   tasks:
+    - name: Safety Check
+      ansible.builtin.fail:
+        msg: "This archived playbook destroys Ceph OSD ZFS volumes and purges Ceph fleet-wide. You must run with -e 'confirm_destroy=true'."
+      when: not confirm_destroy | bool
+
     - name: Remove cpool configuration from Proxmox storage
       command: pvesm remove cpool
       register: pvesm_remove

ansible/plays/backup-immich-db.yml

@@ -1,59 +0,0 @@
----
-# Dump the Immich DB from the legacy LXC (225) to a local file on the
-# controller (CI runner), where a downstream pipeline step consumes it
-# to produce a Barman seed for the CNPG `bootstrap.recovery` path.
-# The library tarball is handled separately by `backup-immich.yml`.
-
-- name: Dump Immich DB from legacy LXC to CI runner
-  hosts: immich.srv.renner.uno
-  become: true
-  vars:
-    immich_db_name: immich
-    immich_db_user: immich
-  tasks:
-    - name: Find running Immich DB container
-      ansible.builtin.shell: >
-        docker ps --filter "label=com.docker.compose.service=database"
-        --format '{{ "{{" }}.Names{{ "}}" }}' | head -1
-      register: immich_db_container
-      changed_when: false
-
-    - name: Fail if DB container not found
-      ansible.builtin.fail:
-        msg: "Could not find Immich DB container (compose service=database)"
-      when: immich_db_container.stdout | trim == ""
-
-    - name: Dump DB to /tmp on legacy host
-      ansible.builtin.shell: >
-        docker exec -i {{ immich_db_container.stdout | trim }}
-        pg_dump --clean --if-exists --no-owner --no-privileges --format=custom
-        -U {{ immich_db_user }} {{ immich_db_name }}
-        > /tmp/immich-backup.dump
-      args:
-        executable: /bin/bash
-      changed_when: true
-
-    - name: Make dump readable
-      ansible.builtin.file:
-        path: /tmp/immich-backup.dump
-        mode: "0644"
-
-    - name: Report dump size
-      ansible.builtin.stat:
-        path: /tmp/immich-backup.dump
-      register: dump_stat
-
-    - name: Show dump size
-      ansible.builtin.debug:
-        msg: "Dump created: {{ (dump_stat.stat.size / 1024 / 1024) | round(1) }} MB"
-
-    - name: Fetch dump to controller
-      ansible.builtin.fetch:
-        src: /tmp/immich-backup.dump
-        dest: /tmp/immich-backup.dump
-        flat: true
-
-    - name: Remove remote dump
-      ansible.builtin.file:
-        path: /tmp/immich-backup.dump
-        state: absent

ansible/plays/check-chaos-testing.yml

@@ -1,200 +0,0 @@
----
-# Chaos Testing Playbook
-# Tests failover capabilities of HA components
-#
-# Usage:
-#   # Test all components
-#   ansible-playbook -i inventory/ plays/chaos-testing.yml
-#
-#   # Test specific component
-#   ansible-playbook -i inventory/ plays/chaos-testing.yml --tags dns
-#   ansible-playbook -i inventory/ plays/chaos-testing.yml --tags postgres
-#
-# WARNING: This will temporarily disrupt services. Run during maintenance windows.
-
-- name: Chaos Testing - Infrastructure Failover Validation
-  hosts: localhost
-  gather_facts: no
-  vars:
-    dns_vip: "10.2.40.53"
-    dns_primary: "10.2.40.54"
-    dns_backup1: "10.2.40.55"
-    dns_backup2: "10.2.40.56"
-    postgres_vip: "10.2.40.135"
-    postgres_primary: "authentik-db-1.srv.renner.uno"
-    postgres_backup: "authentik-db-2.srv.renner.uno"
-    test_results: []
-
-  tasks:
-    # =========================================================================
-    # DNS CLUSTER FAILOVER TEST
-    # =========================================================================
-    - name: "DNS Test - Record initial state"
-      tags: [dns]
-      block:
-        - name: Check which node holds DNS VIP
-          shell: |
-            for ip in {{ dns_primary }} {{ dns_backup1 }} {{ dns_backup2 }}; do
-              if ssh -o ConnectTimeout=2 root@$ip 'ip addr show | grep -q {{ dns_vip }}' 2>/dev/null; then
-                echo $ip
-                exit 0
-              fi
-            done
-            echo "NONE"
-          register: dns_vip_holder
-          changed_when: false
-
-        - name: Record DNS VIP holder
-          set_fact:
-            dns_master_before: "{{ dns_vip_holder.stdout }}"
-
-        - name: Verify DNS resolution works before test
-          shell: dig @{{ dns_vip }} gitea.srv.renner.uno +short +time=2
-          register: dns_before
-          failed_when: dns_before.rc != 0
-
-        - name: "DNS CHAOS - Stop DNS service on master ({{ dns_master_before }})"
-          shell: ssh root@{{ dns_master_before }} 'systemctl stop proxmox-dns'
-          when: dns_master_before != "NONE"
-
-        - name: Wait for VIP migration (max 10 seconds)
-          pause:
-            seconds: 10
-
-        - name: Verify DNS still resolves via VIP
-          shell: dig @{{ dns_vip }} gitea.srv.renner.uno +short +time=5
-          register: dns_after
-          failed_when: dns_after.rc != 0
-          ignore_errors: yes
-
-        - name: Check which node now holds VIP
-          shell: |
-            for ip in {{ dns_primary }} {{ dns_backup1 }} {{ dns_backup2 }}; do
-              if ssh -o ConnectTimeout=2 root@$ip 'ip addr show | grep -q {{ dns_vip }}' 2>/dev/null; then
-                echo $ip
-                exit 0
-              fi
-            done
-            echo "NONE"
-          register: dns_vip_holder_after
-          changed_when: false
-
-        - name: Record DNS failover result
-          set_fact:
-            test_results: "{{ test_results + [{'component': 'DNS Cluster', 'status': 'PASS' if dns_after.rc == 0 else 'FAIL', 'master_before': dns_master_before, 'master_after': dns_vip_holder_after.stdout, 'failover_time': '< 10 seconds'}] }}"
-
-        - name: "DNS RESTORE - Restart DNS service on original master"
-          shell: ssh root@{{ dns_master_before }} 'systemctl start proxmox-dns'
-          when: dns_master_before != "NONE"
-
-        - name: Wait for service to stabilize
-          pause:
-            seconds: 5
-
-    # =========================================================================
-    # POSTGRESQL HA FAILOVER TEST
-    # =========================================================================
-    - name: "PostgreSQL Test - Record initial state"
-      tags: [postgres]
-      block:
-        - name: Check PostgreSQL replication status on primary
-          shell: |
-            ssh root@{{ postgres_primary }} 'sudo -u postgres psql -t -c "SELECT pg_is_in_recovery();"' 2>/dev/null | tr -d ' '
-          register: pg_primary_status
-          changed_when: false
-          ignore_errors: yes
-
-        - name: Check which node holds PostgreSQL VIP
-          shell: |
-            if ssh -o ConnectTimeout=2 root@{{ postgres_primary }} 'ip addr show | grep -q {{ postgres_vip }}' 2>/dev/null; then
-              echo "{{ postgres_primary }}"
-            elif ssh -o ConnectTimeout=2 root@{{ postgres_backup }} 'ip addr show | grep -q {{ postgres_vip }}' 2>/dev/null; then
-              echo "{{ postgres_backup }}"
-            else
-              echo "NONE"
-            fi
-          register: pg_vip_holder
-          changed_when: false
-
-        - name: Record PostgreSQL VIP holder
-          set_fact:
-            pg_master_before: "{{ pg_vip_holder.stdout }}"
-
-        - name: Verify PostgreSQL connection via VIP
-          shell: |
-            ssh root@{{ postgres_primary }} 'PGPASSWORD=test psql -h {{ postgres_vip }} -U postgres -c "SELECT 1;" 2>&1' || echo "Connection test skipped"
-          register: pg_before
-          changed_when: false
-          ignore_errors: yes
-
-        - name: "PostgreSQL CHAOS - Stop keepalived on VIP holder"
-          shell: ssh root@{{ pg_master_before }} 'systemctl stop keepalived'
-          when: pg_master_before != "NONE"
-
-        - name: Wait for VIP migration (max 10 seconds)
-          pause:
-            seconds: 10
-
-        - name: Check which node now holds PostgreSQL VIP
-          shell: |
-            if ssh -o ConnectTimeout=2 root@{{ postgres_primary }} 'ip addr show | grep -q {{ postgres_vip }}' 2>/dev/null; then
-              echo "{{ postgres_primary }}"
-            elif ssh -o ConnectTimeout=2 root@{{ postgres_backup }} 'ip addr show | grep -q {{ postgres_vip }}' 2>/dev/null; then
-              echo "{{ postgres_backup }}"
-            else
-              echo "NONE"
-            fi
-          register: pg_vip_holder_after
-          changed_when: false
-
-        - name: Record PostgreSQL failover result
-          set_fact:
-            test_results: "{{ test_results + [{'component': 'PostgreSQL HA', 'status': 'PASS' if pg_vip_holder_after.stdout != 'NONE' and pg_vip_holder_after.stdout != pg_master_before else 'FAIL', 'master_before': pg_master_before, 'master_after': pg_vip_holder_after.stdout, 'failover_time': '< 10 seconds'}] }}"
-
-        - name: "PostgreSQL RESTORE - Restart keepalived on original master"
-          shell: ssh root@{{ pg_master_before }} 'systemctl start keepalived'
-          when: pg_master_before != "NONE"
-
-        - name: Wait for service to stabilize
-          pause:
-            seconds: 5
-
-    # =========================================================================
-    # TEST SUMMARY
-    # =========================================================================
-    - name: Display test results
-      tags: [always]
-      debug:
-        msg: |
-          ╔══════════════════════════════════════════════════════════════╗
-          ║           CHAOS TESTING RESULTS - {{ ansible_date_time.iso8601 | default('NOW') }}
-          ╠══════════════════════════════════════════════════════════════╣
-          {% for result in test_results %}
-          ║ {{ result.component | ljust(20) }} │ {{ result.status | ljust(6) }} │ {{ result.get('failover_time', result.get('details', 'N/A')) }}
-          {% endfor %}
-          ╚══════════════════════════════════════════════════════════════╝
-
-          {% if test_results | selectattr('status', 'equalto', 'FAIL') | list | length > 0 %}
-          ⚠️  SOME TESTS FAILED - Review results above
-          {% else %}
-          ✅ ALL TESTS PASSED - HA failover working correctly
-          {% endif %}
-
-    - name: Save test results to file
-      tags: [always]
-      copy:
-        content: |
-          # Chaos Test Results
-          # Date: {{ ansible_date_time.iso8601 | default('unknown') }}
-
-          {% for result in test_results %}
-          ## {{ result.component }}
-          - Status: {{ result.status }}
-          - Master Before: {{ result.get('master_before', 'N/A') }}
-          - Master After: {{ result.get('master_after', 'N/A') }}
-          - Failover Time: {{ result.get('failover_time', 'N/A') }}
-          - Details: {{ result.get('details', 'N/A') }}
-
-          {% endfor %}
-        dest: "/tmp/chaos-test-results-{{ ansible_date_time.date | default('latest') }}.md"
-      delegate_to: localhost

ansible/plays/configure-gitea-runner-logging.yml

@@ -1,72 +0,0 @@
----
-# Setup log collection for Gitea runners
-# Deploys Promtail to collect act_runner logs and ship to Loki
-#
-# Usage:
-#   ansible-playbook -i inventory/ plays/setup-gitea-runner-logging.yml
-
-- name: Setup Gitea Runner Log Collection
-  hosts: proxmox_gitea_runner
-  become: yes
-  vars:
-    promtail_loki_url: "http://loki.srv.renner.uno:3100/loki/api/v1/push"
-    promtail_scrape_configs:
-      # Collect act_runner service logs via journald
-      - job_name: act_runner
-        journal:
-          max_age: 12h
-          matches: _SYSTEMD_UNIT=act_runner.service
-        relabel_configs:
-          - source_labels: ['__journal__systemd_unit']
-            target_label: unit
-          - source_labels: ['__journal__hostname']
-            target_label: host
-
-      # Collect system logs
-      - job_name: syslog
-        static_configs:
-          - targets:
-              - localhost
-            labels:
-              job: syslog
-              host: "{{ inventory_hostname }}"
-              service: gitea-runner
-              __path__: /var/log/syslog
-
-      # Collect Docker daemon logs (runners use Docker)
-      - job_name: docker
-        journal:
-          max_age: 12h
-          matches: _SYSTEMD_UNIT=docker.service
-        relabel_configs:
-          - source_labels: ['__journal__systemd_unit']
-            target_label: unit
-          - source_labels: ['__journal__hostname']
-            target_label: host
-
-  roles:
-    - promtail
-
-  post_tasks:
-    - name: Verify promtail is running
-      systemd:
-        name: promtail
-        state: started
-      register: promtail_status
-
-    - name: Display promtail status
-      debug:
-        msg: "Promtail is {{ 'running' if promtail_status.status.ActiveState == 'active' else 'not running' }} on {{ inventory_hostname }}"
-
-    - name: Test Loki connectivity
-      uri:
-        url: "http://loki.srv.renner.uno:3100/ready"
-        method: GET
-        status_code: 200
-        timeout: 10
-      register: loki_health
-      ignore_errors: yes
-
-    - name: Display Loki connectivity status
-      debug:
-        msg: "Loki server is {{ 'reachable' if loki_health.status == 200 else 'NOT reachable - check network' }}"

ansible/plays/configure-grafana-observability.yml

@@ -1,34 +0,0 @@
----
-# Configure Grafana with observability data sources and dashboards
-# Usage: ansible-playbook -i inventory/ plays/configure-grafana-observability.yml
-
-- name: Configure Grafana Observability Stack
-  hosts: grafana.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Data source URLs
-    loki_url: "http://loki.srv.renner.uno:3100"
-    prometheus_url: "http://prometheus.srv.renner.uno:9090"
-
-    # Alerting configuration - update these with your values
-    alerting_email_enabled: true
-    alerting_email_to: ""  # Set in group_vars or command line
-
-    alerting_ha_webhook_enabled: true
-    alerting_ha_webhook_url: ""  # Set to your Home Assistant webhook URL
-
-  roles:
-    - grafana-observability
-
-  post_tasks:
-    - name: Display configuration status
-      debug:
-        msg: |
-          Grafana observability stack configured:
-          - Prometheus data source: {{ prometheus_url }}
-          - Loki data source: {{ loki_url }}
-          - Dashboards deployed to {{ grafana_dashboards_dir }}
-
-          Access Grafana at https://grafana.renner.uno

ansible/plays/configure-immich-db-password.yml

@@ -1,25 +0,0 @@
----
-- name: Fix Immich Database Passwords
-  hosts: immich.srv.renner.uno
-  become: true
-  tasks:
-    - name: Load Immich Role Variables
-      ansible.builtin.include_role:
-        name: immich
-        tasks_from: empty
-        public: yes
-
-    - name: Reset 'immich' user password
-      ansible.builtin.command:
-        cmd: >
-          docker exec -u postgres immich_postgres psql -c "ALTER USER {{ immich_db_user }} WITH PASSWORD '{{ immich_db_password }}';"
-
-    - name: Reset 'postgres' user password
-      ansible.builtin.command:
-        cmd: >
-          docker exec -u postgres immich_postgres psql -c "ALTER USER postgres WITH PASSWORD '{{ immich_db_password }}';"
-
-    - name: Restart Immich
-      community.docker.docker_compose_v2:
-        project_src: "{{ immich_data_dir }}"
-        state: restarted

ansible/plays/configure-pve-node-network.yml

@@ -0,0 +1,53 @@
+---
+# Configure Proxmox VE Node Host Networking
+#
+# Fixes two defects found by live inspection on 2026-07-19:
+#
+#   1. All three nodes had no IPv6 address on any external interface. FRR
+#      enables ipv6 forwarding for the OSPFv3 corosync mesh, and with the
+#      kernel default accept_ra=1 the nodes silently discarded the Router
+#      Advertisements the gateway sends on VLAN 40. Fixed with accept_ra=2.
+#
+#   2. saturn's vmbr0 sits at MTU 1500 while its own uplink and both peer
+#      nodes are at 9000, so large frames between saturn's guests and the
+#      other nodes' guests blackhole instead of fragmenting.
+#
+# The sysctl fix is non-disruptive and applies on every run. The MTU fix is
+# staged as a pending PVE network change and only applied when explicitly
+# requested, because applying it reloads networking on the node and briefly
+# interrupts the Talos k8s VM and its Ceph OSD.
+#
+# Safe (converges IPv6, stages the MTU change without applying it):
+#   ansible-playbook -i inventory/ plays/configure-pve-node-network.yml
+#
+# Applying the staged MTU change, one node at a time:
+#   ansible-playbook -i inventory/ plays/configure-pve-node-network.yml \
+#     --limit saturn -e pve_node_network_apply_pending=true
+#
+# See wiki: pve-ipv6-accept-ra-forwarding-defect,
+#           pve-saturn-vmbr0-mtu-asymmetry
+
+- name: Configure PVE node host networking
+  hosts: pve_cluster
+  become: yes
+  serial: 1
+
+  roles:
+    - role: pve-node-network
+
+  post_tasks:
+    - name: Summarise node IPv6 state
+      ansible.builtin.command: ip -6 addr show dev vmbr1.40 scope global
+      register: pve_node_network_v6_summary
+      changed_when: false
+      failed_when: false
+
+    - name: Display node IPv6 state
+      ansible.builtin.debug:
+        msg: >-
+          {{ inventory_hostname }}:
+          {{ pve_node_network_v6_summary.stdout_lines
+             | select('search', 'inet6')
+             | map('trim')
+             | list
+             | default(['no global IPv6'], true) }}

ansible/plays/configure-ssh-keys.yml

@@ -3,6 +3,22 @@
   hosts: all:!localhost
   become: true
   gather_facts: false
+  # Authorized keys deployed to root and ladmin on every managed host.
+  #
+  # - krenner@fortress (human): unrestricted.
+  # - woodpecker-ci@homelab (CI): restricted with from= to the cluster's pod
+  #   network (ci_ssh_from_cidrs in group_vars/all/vars.yml). The in-cluster
+  #   Woodpecker step pods SSH to fleet hosts presenting their POD IP as source
+  #   (Cilium native-routing-cidr covers the LAN, so no SNAT; PodCIDRs are
+  #   BGP-advertised). This is a network-of-origin restriction, NOT host-pinned,
+  #   so it survives node DHCP churn and pod rescheduling. See vars.yml for the
+  #   full derivation.
+  # - ansible-runner@ci-new: removed — dead key, no such host exists.
+  vars:
+    authorized_keys:
+      - key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPzFNBP3bq2m/6JqDqod9w+FP/uxXxHMolqhq3Ojflr1 krenner@fortress"
+      - key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICyNnX1I4W9jyDvCJ1DJauHSmbo1ZXSyBxVhSAwMWgen woodpecker-ci@homelab"
+        options: 'from="{{ ci_ssh_from_cidrs }}"'
   tasks:
     - name: Check SSH connectivity
       ansible.builtin.wait_for_connection:
@@ -17,20 +33,16 @@
     - name: Deploy authorized keys to root
       ansible.builtin.authorized_key:
         user: root
-        key: "{{ item }}"
+        key: "{{ item.key }}"
+        key_options: "{{ item.options | default(omit) }}"
         state: present
-      loop:
-        - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPzFNBP3bq2m/6JqDqod9w+FP/uxXxHMolqhq3Ojflr1 krenner@fortress"
-        - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGfe3HJqFR+jIGrboweseAO7XjIVE24sje/b1HybgU9C ansible-runner@ci-new"
-        - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICyNnX1I4W9jyDvCJ1DJauHSmbo1ZXSyBxVhSAwMWgen woodpecker-ci@homelab"
+      loop: "{{ authorized_keys }}"
 
     - name: Deploy authorized keys to ladmin
       ansible.builtin.authorized_key:
         user: ladmin
-        key: "{{ item }}"
+        key: "{{ item.key }}"
+        key_options: "{{ item.options | default(omit) }}"
         state: present
-      loop:
-        - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPzFNBP3bq2m/6JqDqod9w+FP/uxXxHMolqhq3Ojflr1 krenner@fortress"
-        - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGfe3HJqFR+jIGrboweseAO7XjIVE24sje/b1HybgU9C ansible-runner@ci-new"
-        - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICyNnX1I4W9jyDvCJ1DJauHSmbo1ZXSyBxVhSAwMWgen woodpecker-ci@homelab"
+      loop: "{{ authorized_keys }}"
       ignore_errors: true

ansible/plays/install-blackbox-exporter.yml

@@ -1,19 +0,0 @@
----
-# Deploy Blackbox Exporter to Prometheus server for endpoint probing
-# Usage: ansible-playbook -i inventory/ plays/install-blackbox-exporter.yml
-
-- name: Install Blackbox Exporter
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  roles:
-    - blackbox-exporter
-
-  post_tasks:
-    - name: Verify blackbox_exporter is running
-      uri:
-        url: "http://127.0.0.1:9115/metrics"
-        status_code: 200
-      register: exporter_health
-      retries: 3
-      delay: 5
-      until: exporter_health.status == 200

ansible/plays/install-claude-workstation.yml

@@ -0,0 +1,20 @@
+---
+# Off-site Debian VM running Claude Code — reattachable interactive sessions
+# plus an unattended notes-distill timer. Host creation (Synology VMM), the
+# Netbird dashboard route and secret material are manual prerequisites; see
+# roles/claude-workstation/README.md and docs/claude-workstation.md.
+#
+# CI reaches this off-site host over the site-to-site tunnel (home -> off-site
+# works), same as it reaches every other inventory host.
+
+- name: Deploy the Claude Code workstation
+  hosts: claude_workstation
+  become: true
+  # qemu-guest-agent + monitoring-agent (Alloy -> Loki/Prometheus) are applied
+  # to every inventory host by the umbrella plays/common.yml, same as for the
+  # netbird-proxy host — run that against this host too (see RUNBOOK Step 5).
+  roles:
+    - common
+    - role: netbird-client       # mesh enrolment (needs netbird_setup_key in vault)
+      when: netbird_setup_key is defined
+    - claude-workstation

ansible/plays/install-edge-host.yml

@@ -1,13 +1,41 @@
 ---
-# Bare-metal Arch Linux edge host (Dell Wyse 3040, internal eMMC) — replaces
-# the k8s-edge Talos VM
-# (opentofu vm_153) as the home for hardware-bound services:
-#   - zwave-js-ui  (ZWA-2 Z-Wave USB radio, docker compose)
-#   - sml2mqtt     (FTDI power-meter reader, docker compose)
-#   - open3e       (Innomaker USB2CAN -> Vitocal/VX3, native venv + systemd)
-# Base install procedure and cutover checklist: ansible/roles/edge-host/README.md
-- name: Install Arch edge host
+# Bare-metal Arch edge hosts (Dell Wyse 3040 and similar) — the home for
+# hardware-bound services, because Talos supports neither SocketCAN nor USB
+# passthrough.
+#
+# This play is the BASE only: what every edge thin client gets regardless of
+# what is plugged into it. The peripherals live in their own playbooks against
+# their own inventory groups:
+#
+#   plays/install-zwave-gateway.yml   [zwave_gateway]   ZWA-2 radio
+#   plays/install-heating-can.yml     [heating_can]     Innomaker USB2CAN
+#   plays/install-power-meter.yml     [power_meter]     FTDI meter head
+#
+# Splitting the roles without splitting the plays would have kept the coupling:
+# one play applying every role to one group is the monolith again, a layer up.
+# A host with no peripherals is simply absent from those groups -- no flags, no
+# `when:` on the roles.
+#
+# Self-contained: `certificates` and `common` are the parts of plays/common.yml
+# that apply to an edge host, carried here so bootstrapping one is a single
+# unrestricted run over the group. Not carried over: qemu-guest-agent (bare
+# metal, not VMs) and monitoring-agent (Debian-oriented; these hosts ship logs
+# via edge-log-shipper).
+#
+# Order is dependency order: edge-flash-wear before docker, so the journal is
+# volatile before containers start logging into it.
+- name: Install Arch edge host (base)
   hosts: edge_host
+  # Connection is already root (ansible_user=root), so this is a root-to-root
+  # no-op today. Kept explicit, matching plays/common.yml, so the roles below
+  # do not silently depend on the connection user staying privileged.
+  become: true
+
   roles:
+    - certificates
+    - common
+    - edge-flash-wear
     - docker-install
-    - edge-host
+    - edge-log-shipper
+    - mqtt-bridge
+    - edge-vlan

ansible/plays/install-gitea-runner.yml

@@ -1,18 +0,0 @@
----
-- name: Setup Gitea Action Runners
-  hosts: proxmox_gitea_runner
-  become: yes
-  gather_facts: yes
-  user: root
-
-  pre_tasks:
-    - name: Ensure python3 is installed
-      raw: apt-get update && apt-get install -y python3 python3-pip
-      register: output
-      changed_when: output.stdout != ""
-      when: ansible_python_interpreter is not defined
-
-  roles:
-    - common
-    - docker-install
-    - gitea-runner

ansible/plays/install-gitea.yml

@@ -1,8 +0,0 @@
----
-- name: Install and configure Gitea on LXC
-  hosts: gitea.srv.renner.uno
-  remote_user: root
-  become: true
-  roles:
-    - common
-    - gitea-lxc

ansible/plays/install-heating-can.yml

@@ -0,0 +1,18 @@
+---
+# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
+# USB2CAN adapter attached.
+#
+# Targets [heating_can]: membership means the adapter is physically present.
+# Two roles because they are two concerns -- the interface is host network
+# state, the services consume it -- but one group, because both need the same
+# hardware.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: these
+# services publish to the local MQTT bridge it sets up.
+- name: Install heating CAN services
+  hosts: heating_can
+  become: true
+
+  roles:
+    - socketcan
+    - heating-can

ansible/plays/install-immich.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure Immich
-  hosts: proxmox_immich
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: immich

ansible/plays/install-log-aggregation.yml

@@ -1,86 +0,0 @@
----
-# Setup Log Aggregation Infrastructure
-# This playbook configures the complete logging stack:
-# 1. Loki server for log storage and querying
-# 2. Alloy agents on all VMs/CTs for log collection
-# 3. Syslog receiver for remote devices (Synology NAS, network devices)
-#
-# Usage: ansible-playbook -i inventory/ plays/setup-log-aggregation.yml
-#
-# To limit to specific hosts:
-#   --limit loki.srv.renner.uno          # Only configure Loki server
-#   --limit proxmox_all_running          # Only configure log agents
-
-- name: Setup Loki Log Aggregation Server
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
-
-- name: Deploy Monitoring Agent to Loki Server (with syslog receiver)
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-    # Syslog receiver settings are defined in host_vars/loki.srv.renner.uno/vars.yml:
-    # - alloy_syslog_receiver_enabled: true
-    # - alloy_syslog_receiver_port: 1514
-    # - alloy_syslog_receiver_protocol: "udp"
-    # - alloy_syslog_receiver_format: "raw"
-    # - alloy_stability_level: "experimental"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Display syslog receiver info
-      debug:
-        msg: |
-          Syslog receiver is listening on UDP port {{ alloy_syslog_receiver_port | default(1514) }}
-          Configure remote devices to send syslog to: {{ inventory_hostname }}:{{ alloy_syslog_receiver_port | default(1514) }}
-      when: alloy_syslog_receiver_enabled | default(false)
-
-- name: Deploy Monitoring Agent to All Running VMs/CTs
-  hosts: proxmox_all_running:!proxmox_logging:!proxmox_noansible
-  become: yes
-  gather_facts: yes
-  serial: 10  # Deploy to 10 hosts at a time to avoid overwhelming the network
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Verify Alloy is running
-      systemd:
-        name: alloy
-        state: started
-      register: alloy_service
-
-    - name: Display agent status
-      debug:
-        msg: "Monitoring agent deployed successfully on {{ inventory_hostname }}"
-      when: alloy_service is defined

ansible/plays/install-loki.yml

@@ -1,26 +0,0 @@
----
-# Install Loki Log Aggregation Server
-# Usage: ansible-playbook -i inventory/ plays/install-loki.yml
-
-- name: Install Loki Log Aggregation Server
-  hosts: loki.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"

ansible/plays/install-netbird-proxy.yml

@@ -16,6 +16,10 @@
 
 - name: Deploy NetBird reverse-proxy cluster member
   hosts: netbird_proxy
+  # Roll through cluster members one at a time. proxy.renner.uno round-robins
+  # across every member, so restarting them in parallel takes the whole proxy
+  # cluster down at once and defeats the point of running more than one.
+  serial: 1
   become: true
   roles:
     - common

ansible/plays/install-ntp-ha.yml

@@ -1,13 +1,75 @@
 ---
-# Install NTP Server in High Availability Configuration on Proxmox Hosts
+# Install NTP Server in High Availability Configuration
 # Deploys chrony as NTP server + keepalived for VIP failover
+#
+# Targets the ntp_cluster group: the three PVE hosts plus edge-1.
+#
+# The two roles have different reach on purpose:
+#
+#   chrony-ntp-server  runs everywhere. It only needs to listen on udp/123 and
+#                      'allow 10.2.0.0/16' covers every VLAN via inter-VLAN
+#                      routing, so edge-1 serves time from its VLAN 30 address
+#                      (10.2.30.108) with no network reconfiguration.
+#
+#   keepalived-ntp     runs only where ntp_keepalived_enabled is true. VRRP
+#                      requires layer-2 adjacency with the other members on
+#                      VLAN 40, and edge-1 has no VLAN 40 sub-interface yet
+#                      (only untagged VLAN 30 on enp1s0). Once enp1s0.40 exists
+#                      and is verified, flip the flag below.
+#
+# Migration note: as the PVE hosts are retired for bare-metal Talos, edge-1 is
+# intended to become the sole NTP server. Its keepalived priority is set lowest
+# so it stays BACKUP while the PVE nodes exist and wins the VIP by default once
+# they are gone -- no config change required at cutover.
 
+# Phase 1. Every static assignment of the IPv6 service address must be gone
+# fleet-wide before ANY host claims it via VRRP.
+#
+# DAD is enabled on the VRRP address (deliberately -- under single-master VRRP a
+# duplicate is a real fault worth detecting). So a static copy on one host and a
+# VRRP-managed copy on another cannot coexist: the kernel marks the newcomer
+# `dadfailed deprecated`, and its master then believes it is serving while
+# carrying an unusable address. That is not hypothetical -- it is what edge-1
+# did on 2026-08-07 when it took the address while three PVE nodes still held it
+# statically.
+#
+# Deliberately NOT serial: this play must complete everywhere before phase 2
+# starts anywhere. Doing the withdrawal per host inside phase 2 cannot avoid the
+# overlap, because with serial: 1 the first host claims the address while the
+# rest are still static.
+#
+# There is a gap between the phases where no host holds the address. It is
+# bounded by one play's run time and clients simply retry.
+- name: Withdraw static assignments of the NTP service address
+  hosts: ntp_cluster
+  become: true
+
+  vars:
+    keepalived_ntp_vip6: "{{ ntp_vip6 }}"
+    keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
+
+  tasks:
+    - name: Withdraw any static assignment of the service address
+      ansible.builtin.include_role:
+        name: keepalived-ntp
+        tasks_from: withdraw-static.yml
+      when: ntp_keepalived_enabled | default(false)
+
+# Phase 2. Configure chrony and hand the address to keepalived.
 - name: Install NTP Server (HA)
-  hosts: pve_cluster
-  become: yes
+  hosts: ntp_cluster
+  become: true
+  # One node at a time. The keepalived role withdraws the statically assigned
+  # IPv6 service address and restarts keepalived so the address is re-taken
+  # under VRRP control; doing that across every NTP server at once would leave
+  # the whole fleet with nobody holding fd40::123 until the elections settle.
+  # Serialised, the remaining members keep answering throughout.
+  serial: 1
 
   vars:
     ntp_vip: "10.2.40.4"
+    # ntp_vip6 / ntp_vip6_linklocal come from group_vars/ntp_cluster.yml, so
+    # both plays in this file see one definition.
     ntp_servers:
       jupiter:
         priority: 100
@@ -18,58 +80,122 @@
       neptune:
         priority: 80
         state: "BACKUP"
+      edge-1.iot.renner.uno:
+        priority: 70
+        state: "BACKUP"
 
   roles:
     - role: chrony-ntp-server
+
     - role: keepalived-ntp
+      when: ntp_keepalived_enabled | default(false)
       vars:
         keepalived_ntp_vip: "{{ ntp_vip }}"
+        keepalived_ntp_vip6: "{{ ntp_vip6 }}"
+        keepalived_ntp_vip6_linklocal: "{{ ntp_vip6_linklocal }}"
         keepalived_ntp_state: "{{ ntp_servers[inventory_hostname].state }}"
         keepalived_ntp_priority: "{{ ntp_servers[inventory_hostname].priority }}"
+        keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
 
   post_tasks:
     - name: Wait for chrony to settle
-      pause:
+      ansible.builtin.pause:
         seconds: 5
 
     - name: Verify chrony is serving NTP
-      shell: chronyc clients | head -10
+      ansible.builtin.shell: chronyc clients | head -10
       register: chrony_clients
       changed_when: false
 
     - name: Check NTP port is listening
-      shell: ss -lun | grep ':123 '
+      ansible.builtin.shell: ss -lun | grep ':123 '
       register: ntp_port
       failed_when: false
       changed_when: false
 
     - name: Display NTP port status
-      debug:
+      ansible.builtin.debug:
         msg: "NTP port 123: {{ 'LISTENING' if ntp_port.rc == 0 else 'NOT LISTENING' }}"
 
     - name: Check if this server has the VIP
-      shell: ip addr show | grep {{ ntp_vip }}
+      ansible.builtin.shell: ip addr show | grep {{ ntp_vip }}
       register: vip_check
       failed_when: false
       changed_when: false
+      when: ntp_keepalived_enabled | default(false)
 
     - name: Display VIP status
-      debug:
+      ansible.builtin.debug:
         msg: >
           {% if vip_check.rc == 0 %}
           This server ({{ inventory_hostname }}) is MASTER and holds the VIP {{ ntp_vip }}
           {% else %}
           This server ({{ inventory_hostname }}) is BACKUP
           {% endif %}
+      when: ntp_keepalived_enabled | default(false)
+
+    # The v6 address is the path clients actually use, so it gets the same
+    # check as the v4 VIP. Without this a broken v6 rollout -- the address
+    # never claimed, or -nodad not behaving as expected on a given kernel --
+    # would not surface here at all, only later by reading
+    # /var/log/keepalived-state.log by hand.
+    #
+    # `ip -6 addr show ... to <addr>` prints the address when present and
+    # nothing when absent, so emptiness is the test.
+    #
+    # Presence cannot itself distinguish a VRRP-held address from a static
+    # leftover -- they look identical. It is only meaningful because the
+    # keepalived-ntp role asserts, on both distros, that no static assignment
+    # survives: the play fails before reaching here if one does.
+    - name: Check if this server holds the IPv6 service address
+      ansible.builtin.command:
+        argv:
+          - ip
+          - -6
+          - addr
+          - show
+          - dev
+          - "{{ ntp_keepalived_interface }}"
+          - to
+          - "{{ ntp_vip6 }}"
+      register: vip6_check
+      failed_when: false
+      changed_when: false
+      when: ntp_keepalived_enabled | default(false)
+
+    - name: Display IPv6 service address status
+      ansible.builtin.debug:
+        msg: >
+          {% if vip6_check.stdout | default('') | length > 0 %}
+          This server ({{ inventory_hostname }}) holds the IPv6 service address
+          {{ ntp_vip6 }} -- it is the v6 MASTER and is serving ntp.srv.renner.uno
+          {% else %}
+          This server ({{ inventory_hostname }}) does not hold {{ ntp_vip6 }}.
+          Expected on a v6 BACKUP. If NO host holds it, check_ntp is failing
+          everywhere -- inspect `chronyc tracking` and
+          /var/log/keepalived-ntp-health.log
+          {% endif %}
+      when: ntp_keepalived_enabled | default(false)
+
+    - name: Report hosts serving NTP without participating in the VIP
+      ansible.builtin.debug:
+        msg: >
+          {{ inventory_hostname }} serves NTP directly on its own address
+          (keepalived not enabled on this host).
+      when: not (ntp_keepalived_enabled | default(false))
 
     - name: Test NTP via VIP (only on master)
-      shell: |
+      ansible.builtin.shell: |
         chronyc -h {{ ntp_vip }} tracking 2>&1 || echo "Cannot connect to VIP (may need to wait for failover)"
       register: ntp_vip_test
-      when: vip_check.rc == 0
+      when:
+        - ntp_keepalived_enabled | default(false)
+        - vip_check.rc == 0
       changed_when: false
 
     - name: Display VIP NTP test result
-      debug:
+      ansible.builtin.debug:
         msg: "{{ ntp_vip_test.stdout_lines | default(['Not master, skipping VIP test']) }}"
-      when: vip_check.rc == 0
+      when:
+        - ntp_keepalived_enabled | default(false)
+        - vip_check.rc == 0

ansible/plays/install-power-meter.yml

@@ -0,0 +1,14 @@
+---
+# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
+# attached.
+#
+# Targets [power_meter]: membership means the meter head is physically present.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker and publishes to the local MQTT bridge.
+- name: Install power-meter reader
+  hosts: power_meter
+  become: true
+
+  roles:
+    - power-meter-sml

ansible/plays/install-prometheus.yml

@@ -1,43 +0,0 @@
----
-# Install Prometheus Metrics Server
-# Usage: ansible-playbook -i inventory/ plays/install-prometheus.yml
-
-- name: Install Prometheus Metrics Server
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Synology NAS SNMP targets
-    snmp_synology_targets:
-      - name: nas-1
-        target: "10.2.40.50"
-      - name: nas-2
-        target: "10.3.40.57"
-
-    # SNMPv3 credentials from vault
-    snmp_use_v3: true
-    snmp_v3_username: "{{ vault_snmp_v3_username }}"
-    snmp_v3_auth_password: "{{ vault_snmp_v3_auth_password }}"
-    snmp_v3_priv_password: "{{ vault_snmp_v3_priv_password }}"
-
-  roles:
-    - common
-    - prometheus-server
-    - snmp-exporter
-    - pve-exporter
-    - unifi-poller
-
-  post_tasks:
-    - name: Verify Prometheus is running
-      uri:
-        url: "http://127.0.0.1:9090/-/ready"
-        status_code: 200
-      register: prometheus_health
-      retries: 5
-      delay: 10
-      until: prometheus_health.status == 200
-
-    - name: Display Prometheus status
-      debug:
-        msg: "Prometheus is running and healthy at http://{{ inventory_hostname }}:9090"

ansible/plays/install-proxmox-dns-ha.yml

@@ -1,72 +0,0 @@
----
-# Install Proxmox DNS Server in High Availability Configuration
-# This playbook installs the DNS servers + keepalived for VIP failover
-
-- name: Install Proxmox DNS Server (HA)
-  hosts: proxmox_dns_ha
-  become: yes
-
-  vars:
-    dns_vip: "10.2.40.53"
-    dns_servers:
-      proxmox-dns-1.srv.renner.uno:
-        priority: 100
-        state: "MASTER"
-      proxmox-dns-2.srv.renner.uno:
-        priority: 90
-        state: "BACKUP"
-      proxmox-dns-3.srv.renner.uno:
-        priority: 80
-        state: "BACKUP"
-
-  roles:
-    - role: proxmox-dns
-    - role: keepalived-vip
-      vars:
-        keepalived_vip: "{{ dns_vip }}"
-        keepalived_state: "{{ dns_servers[inventory_hostname].state }}"
-        keepalived_priority: "{{ dns_servers[inventory_hostname].priority }}"
-
-  post_tasks:
-    - name: Wait for DNS server to start
-      wait_for:
-        port: 53
-        timeout: 30
-
-    - name: Wait for keepalived to start
-      wait_for:
-        timeout: 10
-
-    - name: Check if this server has the VIP
-      shell: ip addr show | grep {{ dns_vip }}
-      register: vip_check
-      failed_when: false
-      changed_when: false
-
-    - name: Display VIP status
-      debug:
-        msg: >
-          {% if vip_check.rc == 0 %}
-          This server ({{ inventory_hostname }}) is MASTER and holds the VIP {{ dns_vip }}
-          {% else %}
-          This server ({{ inventory_hostname }}) is BACKUP
-          {% endif %}
-
-    - name: Test DNS resolution locally
-      shell: |
-        dig @localhost gitea.srv.renner.uno +short
-      register: dns_test
-      failed_when: false
-      changed_when: false
-
-    - name: Display DNS test result
-      debug:
-        msg: "DNS test result: {{ dns_test.stdout if dns_test.stdout else 'No result (may be expected if host does not exist yet)' }}"
-
-    - name: Install notification script
-      copy:
-        src: "{{ playbook_dir }}/../ansible/roles/keepalived-vip/files/keepalived-notify.sh"
-        dest: /usr/local/bin/keepalived-notify.sh
-        owner: root
-        group: root
-        mode: '0755'

ansible/plays/install-proxmox-dns.yml

@@ -1,28 +0,0 @@
----
-# Install Proxmox DNS Server
-# This playbook installs the custom DNS server that resolves VM/LXC hostnames
-# by querying the Proxmox API
-
-- name: Install Proxmox DNS Server
-  hosts: proxmox_dns
-  become: yes
-
-  roles:
-    - proxmox-dns
-
-  post_tasks:
-    - name: Wait for DNS server to start
-      wait_for:
-        port: 53
-        timeout: 30
-
-    - name: Test DNS resolution
-      shell: |
-        dig @localhost gitea.srv.renner.uno +short
-      register: dns_test
-      failed_when: false
-      changed_when: false
-
-    - name: Display DNS test result
-      debug:
-        msg: "DNS test result: {{ dns_test.stdout }}"

ansible/plays/install-woodpecker-agent.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure Woodpecker CI Agents
-  hosts: woodpecker_agents
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: woodpecker-agent

ansible/plays/install-woodpecker-server.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure Woodpecker CI Server
-  hosts: woodpecker_server
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: woodpecker-server

ansible/plays/install-zwave-gateway.yml

@@ -0,0 +1,15 @@
+---
+# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
+#
+# Targets [zwave_gateway]: membership means the radio is physically present.
+# Moving the stick to another edge host is a one-line inventory change.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker, and the flash-wear handling that the libeatmydata preload exists to
+# complement.
+- name: Install Z-Wave gateway
+  hosts: zwave_gateway
+  become: true
+
+  roles:
+    - zwave-gateway

ansible/plays/update.yml

@@ -11,12 +11,11 @@
 - import_playbook: update-lxc.yml
 - import_playbook: update-authentik.yml
 # Re-render docker-compose templates before pulling updates
-- import_playbook: install-immich.yml
+# (immich decommissioned — migrated to in-cluster deployment, see kubernetes/apps/immich/)
 - import_playbook: install-n8n.yml
 - import_playbook: install-grist.yml
 - import_playbook: install-music-assistant.yml
 - import_playbook: update-docker.yml
-- import_playbook: install-log-aggregation.yml
 - import_playbook: update-pve-rolling.yml
 
 - name: Update Summary

ansible/plays/vars/proxmox_dns_vault.yml

@@ -1,8 +0,0 @@
----
-# Proxmox DNS Server Secrets
-# Encrypt this file with: ansible-vault encrypt proxmox_dns_vault.yml
-
-# Proxmox API token for DNS server
-# Create in Proxmox: Datacenter > Permissions > API Tokens
-# Format: <token-id>!<token-name> = <secret>
-proxmox_dns_api_token_value: "CHANGEME-your-proxmox-api-token-here"

ansible/plays/vars/rustfs_vault.yml

@@ -1,7 +0,0 @@
-vault_rustfs_secret_key: !vault |
-          $ANSIBLE_VAULT;1.1;AES256
-          32363232626635353039653538623036386161393539353265366433306533346666616362373036
-          3633316536626339386236333839636538643438616531650a316539373264636438363262333633
-          63653737363433643833393561343138303564623635653437636164306238303539343934646361
-          3666343466333439630a323432666430386637663165323165363536633838303938323239313762
-          62396631366438303265653566366462363137353135663839303439313537666531

ansible/roles/blackbox-exporter/defaults/main.yml

@@ -1,29 +0,0 @@
----
-blackbox_exporter_version: "0.25.0"
-blackbox_exporter_port: 9115
-blackbox_exporter_user: "blackbox_exporter"
-blackbox_exporter_group: "blackbox_exporter"
-
-# Endpoints to probe
-blackbox_http_targets:
-  - name: "gitea"
-    url: "https://gitea.renner.uno"
-  - name: "immich"
-    url: "https://immich.renner.uno"
-  - name: "authentik"
-    url: "https://auth.renner.uno"
-  - name: "home-assistant"
-    url: "https://ha.renner.uno"
-  - name: "grafana"
-    url: "https://grafana.renner.uno"
-  - name: "prometheus"
-    url: "https://prometheus.renner.uno"
-
-blackbox_dns_targets:
-  - name: "proxmox-dns"
-    target: "gitea.srv.renner.uno"
-    server: "10.2.40.53"
-
-blackbox_tcp_targets:
-  - name: "postgres-ha"
-    target: "10.2.40.135:5432"

ansible/roles/blackbox-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/blackbox-exporter/tasks/main.yml

@@ -1,80 +0,0 @@
----
-- name: Create blackbox_exporter group
-  group:
-    name: "{{ blackbox_exporter_group }}"
-    system: yes
-    state: present
-
-- name: Create blackbox_exporter user
-  user:
-    name: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    createhome: no
-    state: present
-
-- name: Create blackbox_exporter directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0755'
-  loop:
-    - /etc/blackbox_exporter
-    - /var/lib/blackbox_exporter
-
-- name: Download blackbox_exporter
-  get_url:
-    url: "https://github.com/prometheus/blackbox_exporter/releases/download/v{{ blackbox_exporter_version }}/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64.tar.gz"
-    dest: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    mode: '0644'
-
-- name: Extract blackbox_exporter
-  unarchive:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    dest: /tmp
-    remote_src: yes
-    creates: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
-
-- name: Install blackbox_exporter binary
-  copy:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64/blackbox_exporter"
-    dest: /usr/local/bin/blackbox_exporter
-    owner: root
-    group: root
-    mode: '0755'
-    remote_src: yes
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter configuration
-  template:
-    src: blackbox.yml.j2
-    dest: /etc/blackbox_exporter/blackbox.yml
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter systemd service
-  template:
-    src: blackbox_exporter.service.j2
-    dest: /etc/systemd/system/blackbox_exporter.service
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Enable and start blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"

ansible/roles/blackbox-exporter/templates/blackbox.yml.j2

@@ -1,60 +0,0 @@
-modules:
-  http_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
-      valid_status_codes: []  # Defaults to 2xx
-      method: GET
-      follow_redirects: true
-      fail_if_ssl: false
-      fail_if_not_ssl: false
-      tls_config:
-        insecure_skip_verify: false
-
-  http_post_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      method: POST
-      valid_status_codes: []
-
-  tcp_connect:
-    prober: tcp
-    timeout: 5s
-
-  # DNS module for external domain resolution (tests upstream/cache)
-  dns_external:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "google.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for internal domain resolution (tests local resolution)
-  dns_internal:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "gitea.srv.renner.uno"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for common cached domain
-  dns_cached:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "cloudflare.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  icmp:
-    prober: icmp
-    timeout: 5s
-    icmp:
-      preferred_ip_protocol: "ip4"

ansible/roles/blackbox-exporter/templates/blackbox_exporter.service.j2

@@ -1,22 +0,0 @@
-[Unit]
-Description=Prometheus Blackbox Exporter
-Documentation=https://github.com/prometheus/blackbox_exporter
-Wants=network-online.target
-After=network-online.target
-
-[Service]
-Type=simple
-User={{ blackbox_exporter_user }}
-Group={{ blackbox_exporter_group }}
-ExecStart=/usr/local/bin/blackbox_exporter \
-    --config.file=/etc/blackbox_exporter/blackbox.yml \
-    --web.listen-address=:{{ blackbox_exporter_port }}
-Restart=on-failure
-RestartSec=5s
-
-# Allow ICMP probing (requires CAP_NET_RAW)
-AmbientCapabilities=CAP_NET_RAW
-CapabilityBoundingSet=CAP_NET_RAW
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/certificates/handlers/main.yml

@@ -1,4 +0,0 @@
----
-- name: Update CA Trust
-  command: update-ca-certificates
-  listen: "Update CA Trust"

ansible/roles/certificates/tasks/main.yml

@@ -1,7 +1,21 @@
 ---
-- name: Fetch Authentik Certificate
-  get_url:
-    url: https://authentik.renner.uno
-    dest: /usr/local/share/ca-certificates/authentik.crt
-    validate_certs: false
-  notify: Update CA Trust
+# This role previously fetched https://authentik.renner.uno with
+# validate_certs: false and installed whatever certificate it returned into
+# the system-wide CA trust store (update-ca-certificates) on every
+# Ansible-managed host. That is a MITM hole: an on-path or DNS-spoofing
+# attacker during the fetch could plant an attacker-controlled CA into every
+# host's trust store (gitea_admin/homelab#185).
+#
+# It also turned out to be dead weight: authentik.renner.uno is a stale
+# hostname that isn't routed by the current Traefik/cert-manager ingress.
+# The live service (auth.renner.uno, ldap.renner.uno) gets a real
+# publicly-trusted certificate from Let's Encrypt via cert-manager
+# (kubernetes/infrastructure/post-crds/clusterissuer.yaml), which is already
+# covered by the OS default CA bundle. No custom/internal CA needs to be
+# trusted for authentik, so there is nothing for this role to install.
+#
+# Left intentionally empty: if a genuine internal/private CA ever needs to
+# be trusted cluster-wide, add a task here that installs a CA cert checked
+# into this repo (or fetched with validate_certs: true over a channel that
+# is actually verified) — never re-introduce an unverified fetch feeding
+# update-ca-certificates.

ansible/roles/chrony-ntp-server/defaults/main.yml

@@ -1,9 +1,25 @@
 ---
 # Chrony NTP Server Role Defaults
 
-# Networks allowed to use this server as NTP source
+# Networks allowed to use this server as NTP source.
+#
+# BOTH families are required. IPv6 is the primary address family here -- the
+# service address ntp.srv.renner.uno resolves to fd40::123 -- so an IPv4-only
+# allow list refuses every client that uses the documented name.
+#
+# The hosts carried `allow fd00::/8` for a long time without it ever being in
+# this file. Applying the role on 2026-08-07 rendered the template over it and
+# dropped the line on all four NTP servers at once, which took internal NTP
+# down over IPv6 until it was added here. Same class of untracked host state as
+# the anycast service address, and the same failure: config that exists only on
+# the box survives right up until something re-renders it.
 chrony_allowed_networks:
   - "10.2.0.0/16"
+  - "fd00::/8"
+  # Link-local. A client on the same segment may source its query from fe80::,
+  # and without this chrony refuses it -- the request arrives and is dropped,
+  # which looks like the server being unreachable rather than a policy refusal.
+  - "fe80::/10"
 
 # Primary NTP servers (PTB - Physikalisch-Technische Bundesanstalt, Stratum 1)
 chrony_upstream_servers:
@@ -17,3 +33,29 @@ chrony_upstream_pools:
 
 # Local stratum if disconnected from upstream
 chrony_local_stratum: 10
+
+# Step (rather than slew) the clock when the offset exceeds the threshold, but
+# only for the first N updates after chronyd starts. Without a makestep
+# directive chronyd only ever slews, so a large offset is never closed in
+# practice.
+#
+# edge-1, 2026-08-04: the CMOS battery is dead, so after a power loss the RTC
+# came up reading 2018-08-20. The kernel set the clock from it, systemd floored
+# that forward to its own built-in epoch (2026-06-26), and it ended up at
+# ~2026-07-19 -- about 16 days behind. chronyd then synced to ptbtime1, reported
+# "System time: 1376235.375 seconds slow of NTP time", and corrected none of it.
+# With host time before the cluster certificate's NotBefore, every outbound TLS
+# handshake failed x509 validation, taking out the MQTT bridge (heating
+# telemetry) and the Loki log shipper at once. The 16 days is incidental -- it
+# is just where the software flooring landed; the fault is that nothing steps.
+#
+# Limiting to the first few updates keeps steps confined to boot, so a
+# long-running server is never jumped out from under its workloads.
+chrony_makestep_threshold: 1.0
+chrony_makestep_limit: 3
+
+# Hardware timestamping. Fine on the PVE nodes' Intel NICs; the Wyse 3040
+# (edge-1) has a Realtek NIC that does not support it, and chronyd logs an
+# error per interface when asked for it. Overridden to false in
+# inventory/group_vars/edge_host.
+chrony_hwtimestamp: true

ansible/roles/chrony-ntp-server/handlers/main.yml

@@ -1,7 +1,8 @@
 ---
 # Chrony NTP Server Handlers
 
+# chrony_service_name is set by tasks/main.yml (chrony on Debian, chronyd on Arch).
 - name: restart chrony
-  systemd:
-    name: chrony
+  ansible.builtin.systemd:
+    name: "{{ chrony_service_name }}"
     state: restarted

ansible/roles/chrony-ntp-server/tasks/main.yml

@@ -1,25 +1,82 @@
 ---
 # Chrony NTP Server Role
+#
+# Supports Debian (PVE cluster nodes) and Arch (edge-1). The package is named
+# 'chrony' on both, but three things differ and are resolved from facts below:
+#   - config path:  Debian /etc/chrony/chrony.conf   Arch /etc/chrony.conf
+#   - unit name:    Debian chrony.service            Arch chronyd.service
+#   - service user: Debian _chrony                   Arch chrony
+# Arch's package also ships no /var/lib/chrony or /var/log/chrony (it defers
+# them to tmpfiles.d), so this role creates them explicitly rather than relying
+# on pacman hook ordering. Both creations are idempotent on Debian.
 
-- name: Ensure chrony is installed
-  apt:
+- name: Fail on unsupported distribution
+  ansible.builtin.fail:
+    msg: >-
+      chrony-ntp-server supports Debian and Archlinux only;
+      this host reports os_family={{ ansible_facts['os_family'] }}
+  when: ansible_facts['os_family'] not in ['Debian', 'Archlinux']
+
+- name: Resolve distro-specific chrony paths and identifiers
+  ansible.builtin.set_fact:
+    chrony_config_path: >-
+      {{ '/etc/chrony/chrony.conf'
+         if ansible_facts['os_family'] == 'Debian'
+         else '/etc/chrony.conf' }}
+    chrony_service_name: >-
+      {{ 'chrony' if ansible_facts['os_family'] == 'Debian' else 'chronyd' }}
+    chrony_system_user: >-
+      {{ '_chrony' if ansible_facts['os_family'] == 'Debian' else 'chrony' }}
+
+- name: Install chrony (Debian)
+  ansible.builtin.apt:
     name: chrony
     state: present
-    update_cache: yes
+    update_cache: true
+  when: ansible_facts['os_family'] == 'Debian'
+
+- name: Install chrony (Arch)
+  ansible.builtin.package:
+    name: chrony
+    state: present
+  when: ansible_facts['os_family'] == 'Archlinux'
+
+# systemd-timesyncd and chrony both want to steer the clock. edge-1 currently
+# runs timesyncd, so it has to be stopped before chronyd takes over.
+- name: Gather service facts
+  ansible.builtin.service_facts:
+
+- name: Disable systemd-timesyncd where present (conflicts with chrony)
+  ansible.builtin.systemd:
+    name: systemd-timesyncd
+    enabled: false
+    state: stopped
+  when: "'systemd-timesyncd.service' in ansible_facts.services"
+
+- name: Ensure chrony state and log directories exist
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: directory
+    owner: "{{ chrony_system_user }}"
+    group: "{{ chrony_system_user }}"
+    mode: '0750'
+  loop:
+    - /var/lib/chrony
+    - /var/log/chrony
 
 - name: Configure chrony as NTP server
-  template:
+  ansible.builtin.template:
     src: chrony.conf.j2
-    dest: /etc/chrony/chrony.conf
+    dest: "{{ chrony_config_path }}"
     owner: root
     group: root
     mode: '0644'
-    backup: yes
+    backup: true
   notify: restart chrony
 
 - name: Ensure chrony is enabled and running
-  systemd:
-    name: chrony
-    enabled: yes
+  ansible.builtin.systemd:
+    name: "{{ chrony_service_name }}"
+    enabled: true
     state: started
-    daemon_reload: yes
+    daemon_reload: true

ansible/roles/chrony-ntp-server/templates/chrony.conf.j2

@@ -14,6 +14,11 @@ pool {{ pool }}
 # Record the rate at which the system clock gains/losses time
 driftfile /var/lib/chrony/chrony.drift
 
+# Step the clock instead of slewing it when the offset is large, limited to the
+# first few updates after start so only boot-time corrections jump. Rationale
+# and the incident behind it: chrony_makestep_* in the role's defaults/main.yml.
+makestep {{ chrony_makestep_threshold }} {{ chrony_makestep_limit }}
+
 # Allow NTP client access from local networks
 {% for network in chrony_allowed_networks %}
 allow {{ network }}
@@ -30,7 +35,9 @@ logdir /var/log/chrony
 log measurements statistics tracking
 
 # Use hardware timestamping if available
+{% if chrony_hwtimestamp | default(true) %}
 hwtimestamp *
+{% endif %}
 
 # Allow command access only from localhost
 bindcmdaddress 127.0.0.1

ansible/roles/claude-workstation/README.md

@@ -0,0 +1,57 @@
+# claude-workstation
+
+An always-on, **off-site** headless Claude Code host (Debian 12 VM under Synology
+VMM on nas-2). It runs (a) reattachable interactive `tmux` sessions and (b) an
+unattended notes-distill timer. It deliberately holds the **least** privilege
+that still does the job.
+
+## What lives here vs. what's manual
+
+This role configures everything **inside** the VM. Three things are **manual**
+(no IaC path) and precede/accompany it — see [`RUNBOOK.md`](RUNBOOK.md) for the
+full step-by-step:
+
+1. **VM creation** in Synology VMM (`ladmin` has no sudo; the DSM API is
+   root-only), with a cloud-init drive that installs the CI SSH key.
+2. **Netbird dashboard route** advertising the k8s API to the mesh, and the
+   **site-to-site return route** on the off-site gateway (backup path).
+3. **Secret material**: `claude setup-token` (subscription OAuth), the claude-bot
+   creds, and the read-only SA kubeconfig — encrypted into the group vault.
+
+## Credential isolation
+
+Two Unix users with separate credential stores so an unattended job cannot read
+the interactive session's secrets:
+
+| User | Holds | Runs |
+|------|-------|------|
+| `claude` | OAuth token, claude-bot creds, **read-only** SA kubeconfig | interactive tmux |
+| `claude-job` | its **own** OAuth token, claude-bot creds (no kubeconfig) | notes-distill timer |
+
+**No talosconfig, no SOPS age key** ever land here (Fable review #1). The
+kubeconfig is a read-only `view` ServiceAccount (`kubernetes/apps/claude-workstation/`).
+
+## Egress lockdown (staged)
+
+`claude_ws_egress_enforce` ships **`false`** so a fresh host is never bricked.
+The deny-default nftables ruleset, the DNS-resolver, and its refresh timer are
+installed regardless. Flip to `true` (in host/group vars) only after confirming
+`claude_ws_egress_allow_fqdns` / `claude_ws_egress_allow_cidrs*` cover real
+traffic — watch the `claude-egress-dropped` counter first.
+
+## Vault keys (encrypt into `inventory/group_vars/claude_workstation/vault.yml`)
+
+```
+netbird_setup_key                  # from the Netbird dashboard
+claude_ws_oauth_token_interactive  # `claude setup-token` (interactive user)
+claude_ws_oauth_token_job          # `claude setup-token` (job user, separate)
+claude_ws_gitea_token              # claude-bot PAT
+claude_ws_gitea_git_credentials    # https://claude-bot:<pat>@gitea.renner.uno
+claude_ws_kubeconfig               # read-only SA kubeconfig (literal YAML)
+```
+
+## Kill switch
+
+Everything unattended hangs off systemd timers; `systemctl disable --now
+claude-notes-distill.timer` (and the egress refresh) stops it. Every credential
+is independently revocable (SA token, claude-bot PAT, OAuth token, Netbird peer).

ansible/roles/claude-workstation/RUNBOOK.md

@@ -0,0 +1,223 @@
+# Runbook — off-site Claude Code workstation (nas-2)
+
+Operator guide for the manual steps around the `claude-workstation` role. The
+role automates everything **inside** the VM; this covers what it can't:
+VM creation, networking, secret material, egress cutover, rotation and DR.
+
+## Architecture (scope)
+
+An always-on **Debian 13 VM under Synology VMM on the off-site nas-2**
+(DS1522+, 10.3.40.0/24, reached over the site-to-site VPN, ~55 ms). It runs:
+
+- **Reattachable interactive sessions** (`tmux` over SSH as user `claude`).
+- **Unattended notes-distill timer** (as user `claude-job`).
+- **A thin is-home-up signal** (the existing external Gatus on the Netbird VPS
+  already covers this — don't duplicate it).
+
+**Out of scope (runs in-cluster instead):** scheduled repo-automation and
+cluster-reports go in as Woodpecker cron / CronJobs with an in-cluster
+ServiceAccount — *not* on this VM, so the VM holds no cluster-write creds.
+
+Credentials on the VM are minimal: a **read-only** `view` SA kubeconfig,
+claude-bot (non-admin) gitea creds, and a Claude subscription OAuth token.
+**No talosconfig, no SOPS age key.**
+
+---
+
+## Step 1 — Create + install the VM (Synology VMM)
+
+`ladmin` has no sudo and the DSM API is root-only, so VMM is driven through the
+DSM **web UI**. Two hard facts dictate the method (all confirmed 2026-07-16, as
+built): **VMM has no cloud-init**, and its ISO-install wizard only creates blank
+disks. So the OS is installed from a **remastered Debian netinst ISO that
+auto-boots a preseed** — no console interaction, no boot-menu race.
+
+VMM gotchas that shape this (see the wiki build-record):
+- VMM refuses to change/unmount a VM's ISO while it is **running** ("Failed to
+  replace ISO files"). Do all ISO/boot-device edits with the VM **off**.
+- A **guest reboot does not re-read** VMM's boot-device/ISO config — only a full
+  VMM power-cycle applies it. So an auto-booting install ISO **loops** (reboot →
+  CD → reinstall, wiping the disk). Break it by preseeding the installer to
+  **power off** at the end: `d-i debian-installer/exit/poweroff boolean true`.
+- Attach the NIC to the **untagged `Default VM Network`** — the VLAN-40
+  `40_Server` network is isolated (no DHCP, no network). The LAN has DHCP.
+
+Procedure (build the ISO on a host with `xorriso`):
+1. Download the current Debian **netinst** (`.../debian-cd/current/amd64/iso-cd/`).
+2. Remaster it — add `/preseed.cfg` and replace `/isolinux/isolinux.cfg` with a
+   minimal one (`default preseed`, `timeout 1`, one label:
+   `kernel /install.amd/vmlinuz` /
+   `append auto=true priority=critical preseed/file=/cdrom/preseed.cfg vga=788 initrd=/install.amd/initrd.gz --- quiet`):
+   `xorriso -indev in.iso -outdev out.iso -boot_image any replay -map preseed.cfg /preseed.cfg -map isolinux.cfg /isolinux/isolinux.cfg`.
+   The preseed uses **DHCP** (`netcfg/dhcp_timeout 120` — the virtual-switch port
+   is slow to link), disk **`/dev/sda`** (VirtIO SCSI), **no user** + a locked
+   random root hash (key-only via `PermitRootLogin prohibit-password`), packages
+   `openssh-server sudo qemu-guest-agent python3`, and a `late_command` that
+   injects the SSH keys (`krenner@fortress`, `woodpecker-ci@homelab`, your
+   workstation key) into `/root/.ssh/authorized_keys`, plus the `poweroff` line.
+3. Upload to nas-2 via `ssh 'cat > /volume1/Install/<iso>' < <iso>` — **scp /
+   sftp / rsync are blocked on DSM**. Then VMM → Image → Add → From Synology NAS
+   → import (VMM **copies** it; re-import if you change the source file).
+4. Create VM: 2 vCPU, 4 GB, 50 GB on `/volume1`, **Default VM Network**, Legacy
+   BIOS, **Autostart: Yes**; attach the ISO, **Boot from DVD/CD-ROM**, power on.
+5. It installs unattended and **powers off**. Then (VM off) set **ISO =
+   Unmounted**, **Boot from = Virtual Disk**, power on → boots the installed
+   Debian. `qemu-guest-agent` reports the DHCP lease in VMM's IP column.
+6. **Pin a DSM update window** (Control Panel → Update & Restore) so an
+   auto-reboot doesn't kill tmux/jobs.
+
+> As built 2026-07-16: Debian 13 (trixie), got DHCP `10.3.40.78`, all three keys
+> injected, packages present. Freeing RAM required powering off the pre-existing
+> `pbs1` PBS VM (kept its 3.4 TB backup disk).
+
+> **At-rest encryption (Fable #6).** Off-site physical custody is weaker than
+> home. For true at-rest protection of the VM's secrets *with* unattended
+> autostart, use **network-bound decryption** (clevis + a Tang server at home):
+> a stolen disk off-site can't unlock because it can't reach Tang. Full-disk
+> LUKS with a manual passphrase would break autostart. Until Tang is stood up,
+> secrets rely on 0600 perms + VM access control — documented gap, not a fix.
+
+---
+
+## Step 2 — Networking (two paths)
+
+**Netbird mesh (done via CI):** mint a setup key in the dashboard
+(`https://netbird.renner.uno`), vault it as `netbird_setup_key` under
+`group_vars/claude_workstation/`, and run `.woodpecker/deploy-netbird-client.yml`
+(`install-netbird-client.yml --limit claude_workstation`). As built 2026-07-16
+the VM enrolled as `claude-ws-190-112.nb.renner.uno` / `100.96.190.112` (+ IPv6).
+
+> **Mesh reachability caveat.** The mesh IP (`100.96.x`) is only reachable from
+> **other Netbird peers**. The home workstation and the Woodpecker CI agent are
+> **not** peers — they reach the VM only via the site-to-site tunnel at its DHCP
+> lease. So a *stable* mesh-based `ansible_host` needs CI (or a home host) on the
+> mesh first, or a Netbird routing peer bridging tunnel↔mesh. Until then the
+> inventory targets the tunnel DHCP IP.
+
+> **DNS trap.** `claude-ws.obg.renner.uno` resolves to the VM's lease **only at
+> the off-site DNS** (`@10.3.40.1`). The home resolver catch-alls `*.renner.uno`
+> to the public proxy (`162.55.50.97`, the Netbird VPS) — so from home the FQDN
+> lands on `root@netbird`, **not** the VM. Never use it as `ansible_host`.
+
+**Optional — mesh route to the k8s API (redundant read path):** to reach the API
+over the mesh too, add a Netbird dashboard **network route** advertising the
+apiserver's ClusterIP to the `netbird-ingress` peer group, and add an egress
+rule to the `netbird-ingress-egress` CiliumNetworkPolicy
+(`kubernetes/infrastructure/network-policies/ingress-isolation.yaml`) permitting
+`app: netbird-ingress` pods to reach `kube-apiserver:443`. Deferred from this PR
+because it edits a security-critical CNP that can't be validated offline — do it
+as a follow-up when you want the redundant API path.
+
+---
+
+## Step 3 — Read-only SA kubeconfig
+
+After the `kubernetes/apps/claude-workstation/` PR merges and Flux reconciles:
+
+```sh
+NS=apps
+TOKEN=$(kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.token}' | base64 -d)
+kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/ca.crt
+# server = the cluster's stable API endpoint (kubernetes/talos/patches/cluster.yaml).
+# Use the name, not an address: it is what the API server certificate carries as a
+# SAN, so an IP here fails TLS verification even when it is reachable. It must
+# resolve from the workstation over the site-to-site route.
+kubectl config set-cluster homelab --server=https://k8s.renner.uno:6443 \
+  --certificate-authority=/tmp/ca.crt --embed-certs=true --kubeconfig=/tmp/claude-ws.kubeconfig
+kubectl config set-credentials claude-workstation --token="$TOKEN" --kubeconfig=/tmp/claude-ws.kubeconfig
+kubectl config set-context homelab --cluster=homelab --user=claude-workstation --kubeconfig=/tmp/claude-ws.kubeconfig
+kubectl config use-context homelab --kubeconfig=/tmp/claude-ws.kubeconfig
+# verify read-only:
+KUBECONFIG=/tmp/claude-ws.kubeconfig kubectl get nodes        # works
+KUBECONFIG=/tmp/claude-ws.kubeconfig kubectl get secrets -A   # Forbidden (view excludes secrets)
+```
+
+The resulting file becomes the `claude_ws_kubeconfig` vault value.
+
+---
+
+## Step 4 — Secret material → vault
+
+```sh
+# Two independent OAuth tokens so job jobs can't impersonate the interactive user:
+claude setup-token   # -> claude_ws_oauth_token_interactive
+claude setup-token   # -> claude_ws_oauth_token_job  (run again / separate session)
+```
+
+Encrypt into `ansible/inventory/group_vars/claude_workstation/vault.yml`
+(ansible-vault; same VAULT_PASS the CI uses):
+
+```yaml
+netbird_setup_key: "..."
+claude_ws_oauth_token_interactive: "..."
+claude_ws_oauth_token_job: "..."
+claude_ws_gitea_token: "..."                # claude-bot PAT
+claude_ws_gitea_git_credentials: "https://claude-bot:<PAT>@gitea.renner.uno"
+claude_ws_kubeconfig: |
+  apiVersion: v1
+  ...                                        # the file from Step 3
+```
+
+> **Never run `claude` with `--bare`** here or on the VM — bare mode silently
+> ignores `CLAUDE_CODE_OAUTH_TOKEN` and the jobs will fail auth.
+
+---
+
+## Step 5 — Deploy
+
+Merge the PR, then trigger the pipeline (Woodpecker → Run → `PIPELINE ==
+"deploy-claude-workstation"`), or push a change under the role's paths. CI SSHes
+to `10.3.40.60` over the tunnel and applies `common`, `netbird-client`, and this
+role. Also run the umbrella `plays/common.yml` against the new host once, so it
+picks up `qemu-guest-agent` and `monitoring-agent` (Alloy → Loki/Prometheus)
+like every other inventory host.
+
+---
+
+## Step 6 — Egress cutover (staged)
+
+The nftables allowlist ships **disabled**. To enforce:
+
+1. Leave it off for a few days; watch what the box actually talks to.
+2. Confirm `claude_ws_egress_allow_fqdns` + `claude_ws_egress_allow_cidrs*`
+   (home Prometheus/Loki/API) cover it.
+3. Set `claude_ws_egress_enforce: true` in host/group vars, redeploy.
+4. Watch the `claude-egress-dropped` nft counter (`nft list ruleset`) and Loki
+   for anything legitimate being dropped; widen the allowlist as needed.
+
+---
+
+## Step 7 — Notes (Syncthing pairing)
+
+The role installs Syncthing for `claude-job` and starts
+`syncthing@claude-job`. Pair it once (manual): from the VM's GUI (SSH tunnel to
+`127.0.0.1:8384`) add your existing vault's device ID and accept the shared
+`Notes` folder pointed at `{{ claude_ws_notes_dir }}`. The distill timer runs
+daily at 04:30 and only ever **creates** new atomic notes.
+
+---
+
+## Rotation
+
+- **OAuth token** (expires/can be revoked): re-run `claude setup-token` on your
+  laptop, update the two vault keys, redeploy. Headless auth failures surface in
+  Loki (the wrapper logs `result=failure`) — don't let them go quiet.
+- **claude-bot PAT**: rotate in Gitea, update `claude_ws_gitea_*`, redeploy.
+- **SA token**: `kubectl -n apps delete secret claude-workstation-token` (Flux
+  recreates it), re-assemble the kubeconfig (Step 3), update vault, redeploy.
+
+## Kill switch
+
+```sh
+systemctl disable --now claude-notes-distill.timer claude-egress-refresh.timer
+```
+
+Every credential is independently revocable: delete the SA, revoke the PAT,
+revoke the OAuth token, or remove the Netbird peer — no shared blast radius.
+
+## Disaster recovery (cattle, not pet)
+
+The VM holds no unique state — the only real state is the Notes replica, which
+re-syncs from Syncthing. To rebuild: redo **Step 1** (VM + cloud-init), then
+**Step 5** (pipeline). No VMM snapshots needed (they'd bloat the 86%-full
+`/volume1`).

ansible/roles/claude-workstation/defaults/main.yml

@@ -0,0 +1,68 @@
+---
+# claude-workstation role defaults
+#
+# An always-on headless host that runs Claude Code for (a) reattachable
+# interactive sessions and (b) unattended notes/wiki timers. It deliberately
+# holds the LEAST privilege that still lets it do its job: a read-only k8s
+# ServiceAccount kubeconfig, claude-bot (non-admin) gitea creds, and a Claude
+# subscription OAuth token — NO talosconfig and NO SOPS age key. See README.
+
+# --- Users -----------------------------------------------------------------
+# Two Unix users with separate credential stores so an unattended (timer) job
+# cannot read the interactive session's OAuth token / kube / gitea creds.
+claude_ws_interactive_user: claude          # human tmux sessions
+claude_ws_job_user: claude-job              # unattended systemd-timer jobs
+
+claude_ws_base_dir: /opt/claude-workstation
+
+# --- Toolchain -------------------------------------------------------------
+claude_ws_node_major: 22                     # matches the ci-claude image
+claude_ws_claude_code_pkg: "@anthropic-ai/claude-code"
+claude_ws_base_packages:
+  - git
+  - tmux
+  - jq
+  - curl
+  - ca-certificates
+  - ripgrep
+  - unzip
+  - nftables
+  - python3
+  - python3-venv
+
+# --- Egress allowlist ------------------------------------------------------
+# Deny-default OUTPUT is the target end-state, but it can brick a fresh host if
+# an allowed destination is missed, so it ships DISABLED and is tightened after
+# observing real egress (staged rollout). Flip claude_ws_egress_enforce to true
+# once the allowlist below is confirmed complete. See templates/nftables.conf.j2.
+claude_ws_egress_enforce: false
+# Static destinations the host must always reach. CDN-backed hosts (Anthropic,
+# npm, apt) are matched by name at apply time via the fqdn set; home services
+# are matched by IP. Tune in host_vars.
+claude_ws_egress_allow_fqdns:
+  - api.anthropic.com
+  - statsig.anthropic.com
+  - gitea.renner.uno
+  - fsn1.your-objectstorage.com          # Hetzner Object Storage (SOPS bundle)
+  - deb.debian.org
+  - security.debian.org
+  - deb.nodesource.com
+  - pkgs.netbird.io
+  - registry.npmjs.org
+claude_ws_egress_allow_cidrs: []          # home Prometheus/Loki/API (v4), per-host
+claude_ws_egress_allow_cidrs6: []         # same (v6) — IPv6-first, prefer these
+
+# --- Notes / wiki pipeline -------------------------------------------------
+# The Obsidian vault syncs onto this host via Syncthing (matches the user's
+# existing multi-device setup). Unattended pipeline jobs must CREATE new atomic
+# files rather than rewrite existing ones, to keep Syncthing conflicts rare.
+claude_ws_notes_enabled: true
+claude_ws_notes_dir: "/home/{{ claude_ws_job_user }}/Notes"
+claude_ws_syncthing_gui_bind: "127.0.0.1:8384"
+
+# --- Secrets (populated from the ansible-vault'd group_vars — see README) ---
+# claude_ws_oauth_token_interactive : CLAUDE_CODE_OAUTH_TOKEN for the human user
+# claude_ws_oauth_token_job         : CLAUDE_CODE_OAUTH_TOKEN for timer jobs
+# claude_ws_gitea_token             : claude-bot PAT
+# claude_ws_gitea_git_credentials   : https://claude-bot:<pat>@gitea.renner.uno line
+# claude_ws_kubeconfig              : read-only SA kubeconfig (base64 or literal)

ansible/roles/claude-workstation/files/claude-notes-distill-prompt.md

@@ -0,0 +1,22 @@
+You are running unattended as a scheduled job on the claude-workstation host.
+Your job is a conservative distillation pass over the Obsidian wiki vault.
+
+Scope and rules (follow exactly — you have only Read, Write, Glob, Grep):
+
+1. Look at `~/Notes/raw/` (ignore `~/Notes/raw/_processed/` and `~/Notes/_archive/`
+   entirely) for source material.
+2. For each raw source, check whether `~/Notes/wiki/` already contains a note
+   grounded in it (Grep the wiki for the source filename / its `sources:` slug).
+   If a note already exists, SKIP that source — do not rewrite or duplicate it.
+3. For raw sources with NO corresponding wiki note yet, create new atomic
+   concept notes in `~/Notes/wiki/`: one concept per file, self-contained,
+   tagged, cross-linked with `[[wikilinks]]`, every claim grounded in a
+   `sources:` entry. Never invent facts not present in the source.
+4. CREATE new files only. Never rewrite, move, delete, or truncate an existing
+   file in `~/Notes/` — appending a missing cross-link to a note you created
+   THIS run is fine; touching pre-existing notes is not.
+5. If there is nothing new to distill, do nothing and stop. Do not manufacture
+   work.
+
+Preserve disagreement: if sources conflict, surface the conflict in the note
+rather than silently picking a winner. Keep each note short and single-purpose.

ansible/roles/claude-workstation/files/tmux.conf

@@ -0,0 +1,8 @@
+# claude-workstation shared tmux config — reattachable long-running sessions.
+set -g history-limit 100000
+set -g mouse on
+set -g base-index 1
+setw -g pane-base-index 1
+set -g status-right "#H  %Y-%m-%d %H:%M"
+# Keep sessions alive across client disconnects (the whole point of this host).
+set -g destroy-unattached off

ansible/roles/claude-workstation/handlers/main.yml

@@ -0,0 +1,10 @@
+---
+- name: Reload systemd
+  ansible.builtin.systemd:
+    daemon_reload: true
+
+- name: Reload nftables
+  ansible.builtin.systemd:
+    name: nftables.service
+    state: reloaded
+  when: claude_ws_egress_enforce | bool

ansible/roles/claude-workstation/tasks/firewall.yml

@@ -0,0 +1,42 @@
+---
+# Deny-default egress allowlist. Ships DISABLED (claude_ws_egress_enforce:
+# false) so a fresh host is never bricked by a missed destination; the ruleset,
+# the FQDN-resolver, and its refresh timer are installed regardless, and the
+# operator flips the toggle to true once egress is confirmed complete against
+# real traffic (staged rollout). CDN-backed hosts are matched via nft named
+# sets that the refresh unit repopulates from DNS; static home services are
+# matched by claude_ws_egress_allow_cidrs.
+
+- name: Deploy the egress-refresh resolver
+  ansible.builtin.template:
+    src: claude-egress-refresh.sh.j2
+    dest: "{{ claude_ws_base_dir }}/claude-egress-refresh.sh"
+    mode: "0755"
+
+- name: Deploy the egress-refresh service + timer
+  ansible.builtin.template:
+    src: "{{ item }}.j2"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - claude-egress-refresh.service
+    - claude-egress-refresh.timer
+  notify: Reload systemd
+
+- name: Deploy the nftables ruleset
+  ansible.builtin.template:
+    src: nftables.conf.j2
+    dest: /etc/nftables.conf
+    mode: "0644"
+    validate: "nft -c -f %s"
+  notify: Reload nftables
+
+- name: Enforce or relax egress lockdown
+  ansible.builtin.systemd:
+    name: "{{ item }}"
+    enabled: "{{ claude_ws_egress_enforce | bool }}"
+    state: "{{ 'started' if (claude_ws_egress_enforce | bool) else 'stopped' }}"
+    daemon_reload: true
+  loop:
+    - nftables.service
+    - claude-egress-refresh.timer

ansible/roles/claude-workstation/tasks/main.yml

@@ -0,0 +1,183 @@
+---
+# claude-workstation: an always-on headless Claude Code host.
+# Debian 12 VM on the off-site nas-2 (Synology VMM). See README for the manual
+# bootstrap (VM creation, Netbird route, secret material) that precedes this.
+#
+# common.yml already applies `common`, `qemu-guest-agent` and `monitoring-agent`
+# (Alloy -> Loki/Prometheus) to every inventory host, so this role owns only the
+# Claude-specific surface: toolchain, the two credential-isolated users, the
+# read-only creds, notes sync, the unattended timers, and egress lockdown.
+
+- name: Assert we are on Debian (role is apt-only)
+  ansible.builtin.assert:
+    that:
+      - ansible_facts['os_family'] == 'Debian'
+    fail_msg: "claude-workstation targets Debian; got {{ ansible_facts['os_family'] }}."
+
+# --- Base packages ---------------------------------------------------------
+- name: Install base packages
+  ansible.builtin.apt:
+    name: "{{ claude_ws_base_packages }}"
+    state: present
+    update_cache: true
+
+# --- Node.js 22 + Claude Code (matches the ci-claude image toolchain) -------
+- name: Add NodeSource signing key
+  ansible.builtin.get_url:
+    url: https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key
+    dest: /usr/share/keyrings/nodesource.gpg
+    mode: "0644"
+    force: false
+
+- name: Add NodeSource apt repository
+  ansible.builtin.apt_repository:
+    repo: "deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_{{ claude_ws_node_major }}.x nodistro main"
+    state: present
+    filename: nodesource
+
+- name: Install Node.js
+  ansible.builtin.apt:
+    name: nodejs
+    state: present
+    update_cache: true
+
+# NodeSource installs Node under /usr, so global npm modules land in
+# /usr/lib/node_modules and `claude` is on PATH for every user/timer. `creates`
+# keeps it idempotent (re-run `npm install -g` manually to bump the version).
+- name: Install Claude Code globally
+  ansible.builtin.command:
+    cmd: "npm install -g {{ claude_ws_claude_code_pkg }}"
+    creates: /usr/lib/node_modules/@anthropic-ai/claude-code
+
+# --- Credential-isolated users --------------------------------------------
+- name: Create Claude users
+  ansible.builtin.user:
+    name: "{{ item }}"
+    shell: /bin/bash
+    create_home: true
+  loop:
+    - "{{ claude_ws_interactive_user }}"
+    - "{{ claude_ws_job_user }}"
+
+- name: Create the role base directory (scripts, prompts)
+  ansible.builtin.file:
+    path: "{{ claude_ws_base_dir }}"
+    state: directory
+    mode: "0755"
+
+# Let the interactive user's tmux server survive SSH logout / reboot.
+- name: Enable lingering for the interactive user
+  ansible.builtin.command: "loginctl enable-linger {{ claude_ws_interactive_user }}"
+  args:
+    creates: "/var/lib/systemd/linger/{{ claude_ws_interactive_user }}"
+
+- name: Deploy shared tmux config
+  ansible.builtin.copy:
+    src: tmux.conf
+    dest: "/home/{{ item }}/.tmux.conf"
+    owner: "{{ item }}"
+    group: "{{ item }}"
+    mode: "0644"
+  loop:
+    - "{{ claude_ws_interactive_user }}"
+    - "{{ claude_ws_job_user }}"
+
+# --- Secrets: each user gets only what its role needs ----------------------
+# The interactive user gets gitea + OAuth + read-only kubeconfig. The job user
+# gets its OWN OAuth token + gitea (to open PRs) but NO kubeconfig.
+- name: Create ~/.config/claude-bot for the interactive user
+  ansible.builtin.file:
+    path: "/home/{{ item }}/.config/claude-bot"
+    state: directory
+    owner: "{{ item }}"
+    group: "{{ item }}"
+    mode: "0700"
+  loop:
+    - "{{ claude_ws_interactive_user }}"
+    - "{{ claude_ws_job_user }}"
+
+- name: Deploy claude-bot gitea token
+  ansible.builtin.copy:
+    content: "{{ claude_ws_gitea_token }}\n"
+    dest: "/home/{{ item }}/.config/claude-bot/gitea-token"
+    owner: "{{ item }}"
+    group: "{{ item }}"
+    mode: "0600"
+  loop:
+    - "{{ claude_ws_interactive_user }}"
+    - "{{ claude_ws_job_user }}"
+  no_log: true
+
+- name: Deploy claude-bot git-credentials
+  ansible.builtin.copy:
+    content: "{{ claude_ws_gitea_git_credentials }}\n"
+    dest: "/home/{{ item }}/.config/claude-bot/git-credentials"
+    owner: "{{ item }}"
+    group: "{{ item }}"
+    mode: "0600"
+  loop:
+    - "{{ claude_ws_interactive_user }}"
+    - "{{ claude_ws_job_user }}"
+  no_log: true
+
+# CLAUDE_CODE_OAUTH_TOKEN is delivered as an EnvironmentFile so both interactive
+# shells and timer units can source it without it landing in shell history.
+# NEVER run claude with --bare here: bare mode silently ignores this token.
+- name: Deploy per-user Claude OAuth environment file
+  ansible.builtin.copy:
+    content: "CLAUDE_CODE_OAUTH_TOKEN={{ item.token }}\n"
+    dest: "/home/{{ item.user }}/.config/claude-code.env"
+    owner: "{{ item.user }}"
+    group: "{{ item.user }}"
+    mode: "0600"
+  loop:
+    - { user: "{{ claude_ws_interactive_user }}", token: "{{ claude_ws_oauth_token_interactive }}" }
+    - { user: "{{ claude_ws_job_user }}", token: "{{ claude_ws_oauth_token_job }}" }
+  loop_control:
+    label: "{{ item.user }}"
+  no_log: true
+
+- name: Create ~/.kube for the interactive user
+  ansible.builtin.file:
+    path: "/home/{{ claude_ws_interactive_user }}/.kube"
+    state: directory
+    owner: "{{ claude_ws_interactive_user }}"
+    group: "{{ claude_ws_interactive_user }}"
+    mode: "0700"
+
+- name: Deploy read-only SA kubeconfig (interactive user only)
+  ansible.builtin.copy:
+    content: "{{ claude_ws_kubeconfig }}"
+    dest: "/home/{{ claude_ws_interactive_user }}/.kube/config"
+    owner: "{{ claude_ws_interactive_user }}"
+    group: "{{ claude_ws_interactive_user }}"
+    mode: "0600"
+  no_log: true
+
+# This VM's only git identity IS claude-bot (unlike the shared human
+# workstation, where a global credential.helper is forbidden), so a per-user
+# ~/.gitconfig pointing at each user's own claude-bot store is appropriate.
+- name: Configure git identity + credential store for the Claude users
+  ansible.builtin.copy:
+    dest: "/home/{{ item }}/.gitconfig"
+    owner: "{{ item }}"
+    group: "{{ item }}"
+    mode: "0644"
+    content: |
+      [user]
+      	name = Claude Bot
+      	email = claude-bot@renner.uno
+      [credential]
+      	helper = store --file=/home/{{ item }}/.config/claude-bot/git-credentials
+  loop:
+    - "{{ claude_ws_interactive_user }}"
+    - "{{ claude_ws_job_user }}"
+
+# --- Notes / wiki pipeline (Syncthing + timers) ----------------------------
+- name: Notes pipeline
+  ansible.builtin.include_tasks: notes.yml
+  when: claude_ws_notes_enabled | bool
+
+# --- Egress lockdown -------------------------------------------------------
+- name: Egress firewall
+  ansible.builtin.include_tasks: firewall.yml

ansible/roles/claude-workstation/tasks/notes.yml

@@ -0,0 +1,48 @@
+---
+# Notes/wiki pipeline: the Obsidian vault syncs onto this host via Syncthing
+# (matching the user's existing multi-device setup), and a hardened, bounded
+# systemd timer runs an unattended `claude -p` distill pass over it as the
+# credential-isolated job user. Device pairing is a one-time manual step (see
+# README) — this only installs Syncthing and the timer.
+
+- name: Install Syncthing
+  ansible.builtin.apt:
+    name: syncthing
+    state: present
+    update_cache: true
+
+- name: Enable Syncthing for the job user
+  ansible.builtin.systemd:
+    name: "syncthing@{{ claude_ws_job_user }}.service"
+    enabled: true
+    state: started
+    daemon_reload: true
+
+- name: Deploy the notes-distill prompt
+  ansible.builtin.copy:
+    src: claude-notes-distill-prompt.md
+    dest: "{{ claude_ws_base_dir }}/claude-notes-distill-prompt.md"
+    mode: "0644"
+
+- name: Deploy the notes-distill wrapper
+  ansible.builtin.template:
+    src: claude-notes-distill.sh.j2
+    dest: "{{ claude_ws_base_dir }}/claude-notes-distill.sh"
+    mode: "0755"
+
+- name: Deploy the notes-distill service + timer
+  ansible.builtin.template:
+    src: "{{ item }}.j2"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - claude-notes-distill.service
+    - claude-notes-distill.timer
+  notify: Reload systemd
+
+- name: Enable the notes-distill timer
+  ansible.builtin.systemd:
+    name: claude-notes-distill.timer
+    enabled: true
+    state: started
+    daemon_reload: true

ansible/roles/claude-workstation/templates/claude-egress-refresh.service.j2

@@ -0,0 +1,10 @@
+[Unit]
+# {{ ansible_managed }}
+Description=Refresh claude-workstation egress allowlist from DNS
+After=nftables.service network-online.target
+Wants=network-online.target
+Requires=nftables.service
+
+[Service]
+Type=oneshot
+ExecStart={{ claude_ws_base_dir }}/claude-egress-refresh.sh

ansible/roles/claude-workstation/templates/claude-egress-refresh.sh.j2

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+# {{ ansible_managed }}
+# Resolve the egress FQDN allowlist to current IPs and load them into the
+# nftables @allow4/@allow6 sets. CDN IPs rotate, so this runs on a timer.
+# No-op unless the claude_egress table is loaded (i.e. enforcement is on).
+set -euo pipefail
+
+TABLE="inet claude_egress"
+FQDNS=(
+{% for fqdn in claude_ws_egress_allow_fqdns %}
+    "{{ fqdn }}"
+{% endfor %}
+)
+
+nft list table $TABLE >/dev/null 2>&1 || { echo "claude_egress table not loaded; enforcement off"; exit 0; }
+
+v4=(); v6=()
+for host in "${FQDNS[@]}"; do
+    while read -r ip; do [ -n "$ip" ] && v4+=("$ip"); done < <(getent ahostsv4 "$host" | awk '{print $1}' | sort -u)
+    while read -r ip; do [ -n "$ip" ] && v6+=("$ip"); done < <(getent ahostsv6 "$host" | awk '{print $1}' | sort -u)
+done
+
+nft flush set $TABLE allow4
+nft flush set $TABLE allow6
+[ ${#v4[@]} -gt 0 ] && nft add element $TABLE allow4 "{ $(IFS=,; echo "${v4[*]}") }"
+[ ${#v6[@]} -gt 0 ] && nft add element $TABLE allow6 "{ $(IFS=,; echo "${v6[*]}") }"
+echo "claude-egress-refresh: ${#v4[@]} v4 + ${#v6[@]} v6 addresses loaded"

ansible/roles/claude-workstation/templates/claude-egress-refresh.timer.j2

@@ -0,0 +1,10 @@
+[Unit]
+# {{ ansible_managed }}
+Description=Periodically refresh the claude-workstation egress allowlist
+
+[Timer]
+OnBootSec=30s
+OnUnitActiveSec=15min
+
+[Install]
+WantedBy=timers.target

ansible/roles/claude-workstation/templates/claude-notes-distill.service.j2

@@ -0,0 +1,32 @@
+[Unit]
+# {{ ansible_managed }}
+Description=Claude Code unattended notes-distill pass
+After=network-online.target syncthing@{{ claude_ws_job_user }}.service
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+User={{ claude_ws_job_user }}
+Group={{ claude_ws_job_user }}
+EnvironmentFile=/home/{{ claude_ws_job_user }}/.config/claude-code.env
+ExecStart={{ claude_ws_base_dir }}/claude-notes-distill.sh
+# Hard wall-clock kill — the runaway/kill switch for this unattended agent.
+TimeoutStartSec=1800
+
+# --- Hardening: an unattended LLM job on internet-influenced input ----------
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ReadWritePaths=/home/{{ claude_ws_job_user }}
+# Credential isolation: cannot read the interactive session's tokens/kubeconfig.
+InaccessiblePaths=/home/{{ claude_ws_interactive_user }}
+ProtectKernelTunables=true
+ProtectKernelModules=true
+ProtectControlGroups=true
+RestrictSUIDSGID=true
+RestrictNamespaces=true
+LockPersonality=true
+MemoryDenyWriteExecute=false
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/claude-workstation/templates/claude-notes-distill.sh.j2

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# {{ ansible_managed }}
+# Unattended notes-distill pass. Prompt via stdin (never shell-substituted),
+# tool-allowlisted, no --bare (bare silently drops CLAUDE_CODE_OAUTH_TOKEN).
+# Bounded by the unit's TimeoutStartSec, not a CLI flag (--max-turns is absent
+# in the installed Claude Code). Emits a terminal state line to the journal on
+# EVERY outcome so a Loki alert can catch silent failures (dead-man switch).
+set -euo pipefail
+
+export HOME="/home/{{ claude_ws_job_user }}"
+NOTES="{{ claude_ws_notes_dir }}"
+PROMPT="{{ claude_ws_base_dir }}/claude-notes-distill-prompt.md"
+
+log() { logger -t claude-notes-distill -- "$*"; echo "claude-notes-distill: $*"; }
+
+if [ ! -d "$NOTES/raw" ]; then
+  log "result=skipped reason=no-vault path=$NOTES"
+  exit 0
+fi
+
+log "result=start"
+if claude -p \
+     --add-dir "$NOTES" \
+     --allowedTools "Read,Write,Glob,Grep" \
+     --model sonnet \
+     < "$PROMPT"; then
+  log "result=success"
+else
+  rc=$?
+  log "result=failure rc=$rc"
+  exit "$rc"
+fi

ansible/roles/claude-workstation/templates/claude-notes-distill.timer.j2

@@ -0,0 +1,11 @@
+[Unit]
+# {{ ansible_managed }}
+Description=Run the Claude Code notes-distill pass daily
+
+[Timer]
+OnCalendar=*-*-* 04:30:00
+Persistent=true
+RandomizedDelaySec=900
+
+[Install]
+WantedBy=timers.target

ansible/roles/claude-workstation/templates/nftables.conf.j2

@@ -0,0 +1,55 @@
+#!/usr/sbin/nft -f
+# {{ ansible_managed }}
+# Deny-default egress for the claude-workstation. CDN-backed destinations are
+# matched via the @allow4/@allow6 named sets, repopulated from DNS by
+# claude-egress-refresh. Static home services are matched by the @static* sets.
+# Only loaded when claude_ws_egress_enforce is true.
+flush ruleset
+
+table inet claude_egress {
+    set allow4 { type ipv4_addr; flags interval; }
+    set allow6 { type ipv6_addr; flags interval; }
+
+    set static4 {
+        type ipv4_addr
+        flags interval
+{% if (claude_ws_egress_allow_cidrs | default([])) | length > 0 %}
+        elements = { {{ claude_ws_egress_allow_cidrs | join(', ') }} }
+{% endif %}
+    }
+    set static6 {
+        type ipv6_addr
+        flags interval
+{% if (claude_ws_egress_allow_cidrs6 | default([])) | length > 0 %}
+        elements = { {{ claude_ws_egress_allow_cidrs6 | join(', ') }} }
+{% endif %}
+    }
+
+    chain output {
+        type filter hook output priority 0; policy drop;
+
+        oif "lo" accept
+        ct state established,related accept
+
+        # DNS — required to resolve the allowlist itself.
+        udp dport 53 accept
+        tcp dport 53 accept
+
+        # Diagnostics + DHCP lease renewal on the bridged interface.
+        ip protocol icmp accept
+        ip6 nexthdr icmpv6 accept
+        udp dport { 67, 68 } accept
+
+        # NetBird / WireGuard control + data plane (initial handshakes).
+        udp dport { 3478, 51820 } accept
+
+        # Allowed destinations.
+        ip  daddr @allow4  accept
+        ip6 daddr @allow6  accept
+        ip  daddr @static4 accept
+        ip6 daddr @static6 accept
+
+        # Everything else is dropped and counted for inspection.
+        counter comment "claude-egress-dropped"
+    }
+}

ansible/roles/common/tasks/main.yml

@@ -51,15 +51,10 @@
 # -------------------------------------------------------
 # 4. SSH Configuration
 # -------------------------------------------------------
-- name: Add Ansible Runner SSH Key
-  authorized_key:
-    user: "{{ item }}"
-    state: present
-    key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGfe3HJqFR+jIGrboweseAO7XjIVE24sje/b1HybgU9C ansible-runner@ci-new"
-  loop:
-    - ladmin
-    - root
-  ignore_errors: true
+# Authorized keys (human + woodpecker-ci, the latter from=-restricted) are
+# managed by the dedicated ansible/plays/configure-ssh-keys.yml play. The old
+# "Add Ansible Runner SSH Key" task deployed ansible-runner@ci-new, a dead key
+# for a host that no longer exists — removed (issue #184).
 
 # -------------------------------------------------------
 # 5. Disable NTP on VMs/LXC (use host time)

ansible/roles/edge-flash-wear/defaults/main.yml

@@ -0,0 +1,15 @@
+---
+# Flash-wear mitigation.
+
+# --- Logging & eMMC wear mitigation ------------------------------------
+# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
+# kept off local flash: containers + native services log to a RAM-only
+# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
+# wear check round out the flash-longevity measures. See README "Logging &
+# eMMC".
+edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
+
+# Container stdout -> journal, no json log files on the flash. The journal
+# itself is volatile (see the tasks above), so container logs land in RAM and
+# are shipped off the box by edge-log-shipper.
+docker_log_driver: journald

ansible/roles/edge-flash-wear/files/edge-emmc-health

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-# edge-1 eMMC wear/health check. Managed by roles/edge-host.
+# edge-1 eMMC wear/health check. Managed by roles/edge-flash-wear.
 #
 # Reads the JEDEC eMMC 5.x life-time and pre-EOL estimators exposed via sysfs
 # and emits a warning-priority journal message when the flash leaves the

ansible/roles/edge-flash-wear/handlers/main.yml

@@ -0,0 +1,17 @@
+---
+- name: Restart journald
+  ansible.builtin.service:
+    name: systemd-journald
+    state: restarted
+
+# Handlers run in definition order, so this fires after 'Restart journald':
+# once the journal is volatile, the on-disk store is dead weight on the flash.
+- name: Purge persistent journal store
+  ansible.builtin.file:
+    path: /var/log/journal
+    state: absent
+
+- name: Restart docker
+  ansible.builtin.service:
+    name: docker
+    state: restarted

ansible/roles/edge-flash-wear/tasks/main.yml

@@ -0,0 +1,100 @@
+---
+# Flash-wear mitigation for hosts on small eMMC (Dell Wyse 3040 and similar).
+#
+# journald -> volatile (RAM) and size-capped, root mounted noatime+lazytime,
+# periodic TRIM, and a weekly wear check. Shipping the volatile journal off the
+# box is a separate concern -- see the edge-log-shipper role.
+- name: Install mmc-utils
+  ansible.builtin.package:
+    name: mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
+    state: present
+
+- name: Ensure journald drop-in directory exists
+  ansible.builtin.file:
+    path: /etc/systemd/journald.conf.d
+    state: directory
+    mode: "0755"
+
+- name: Switch the systemd journal to volatile (RAM) storage
+  ansible.builtin.template:
+    src: journald-edge.conf.j2
+    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
+    mode: "0644"
+  notify:
+    - Restart journald
+    - Purge persistent journal store
+
+- name: Resolve root filesystem facts (for noatime)
+  ansible.builtin.set_fact:
+    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
+
+- name: Refuse to touch fstab without a stable root UUID
+  ansible.builtin.assert:
+    that:
+      - edge_root_fs.uuid | default('') | length > 0
+    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
+
+# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
+# heartbeat that otherwise dirties the journal every couple seconds); noatime
+# drops access-time writes. Neither weakens data durability.
+- name: Mount root with noatime + lazytime (persisted in fstab)
+  ansible.posix.mount:
+    path: /
+    src: "UUID={{ edge_root_fs.uuid }}"
+    fstype: "{{ edge_root_fs.fstype }}"
+    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
+    dump: "0"
+    passno: "1"   # root fs — keep boot-time fsck ordering
+    backup: true
+    state: mounted
+
+- name: Enable periodic TRIM (helps eMMC wear-leveling)
+  ansible.builtin.systemd:
+    name: fstrim.timer
+    enabled: true
+    state: started
+
+- name: Install eMMC health-check script
+  ansible.builtin.copy:
+    src: edge-emmc-health
+    dest: /usr/local/bin/edge-emmc-health
+    mode: "0755"
+
+- name: Install eMMC health-check service + timer
+  ansible.builtin.copy:
+    src: "{{ item }}"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - edge-emmc-health.service
+    - edge-emmc-health.timer
+
+- name: Enable eMMC health-check timer
+  ansible.builtin.systemd:
+    name: edge-emmc-health.timer
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# Container stdout into the journal instead of json files on disk. This lives
+# here, with the rest of the flash-wear handling, rather than in docker-install:
+# that role is shared by ten other plays (authentik, n8n, owncloud, grist,
+# netbird, gatus, music-assistant, ical-proxy), none of which have a
+# daemon.json today. Adding it there would have created one on the next run of
+# any of those plays and restarted every container on those hosts.
+#
+# /etc/docker is created explicitly because this role runs before
+# docker-install on a first bootstrap, so the directory may not exist yet. The
+# handler runs at the end of the play, by which time docker is installed.
+- name: Ensure the docker config directory exists
+  ansible.builtin.file:
+    path: /etc/docker
+    state: directory
+    mode: "0755"
+
+- name: Route container logs to the journal (keeps json logs off the flash)
+  ansible.builtin.template:
+    src: docker-daemon.json.j2
+    dest: /etc/docker/daemon.json
+    mode: "0644"
+  notify: Restart docker

ansible/roles/edge-flash-wear/templates/docker-daemon.json.j2

@@ -0,0 +1,3 @@
+{
+  "log-driver": "{{ docker_log_driver }}"
+}

ansible/roles/edge-flash-wear/templates/journald-edge.conf.j2

@@ -1,4 +1,4 @@
-# Managed by roles/edge-host. edge-1 boots off a small internal eMMC, so the
+# Managed by roles/edge-flash-wear. edge-1 boots off a small internal eMMC, so the
 # journal must not persist to flash: keep it in RAM (/run/log/journal) and cap
 # its size. Grafana Alloy (edge-alloy.service) ships this journal to Loki
 # before it rotates out of RAM — nothing durable lives here.

ansible/roles/edge-host/defaults/main.yml

@@ -1,107 +0,0 @@
----
-edge_base_dir: /opt/edge-host
-edge_tz: Europe/Berlin
-
-# --- Logging & eMMC wear mitigation ------------------------------------
-# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
-# kept off local flash: containers + native services log to a RAM-only
-# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
-# wear check round out the flash-longevity measures. See README "Logging &
-# eMMC".
-edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
-edge_docker_log_driver: journald      # container stdout -> journal, no json logs on eMMC
-
-# Grafana Alloy log shipper. The srv-network Loki (loki.srv.renner.uno:3100,
-# no auth) is firewalled off the IoT VLAN; the only endpoint edge-1 can reach
-# is the authenticated cluster ingress (Traefik basic-auth).
-edge_alloy_config_file: /etc/alloy/edge.alloy
-edge_alloy_http_port: 12345
-edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
-edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
-edge_loki_username: edge-1
-edge_alloy_labels:
-  environment: production
-  datacenter: homelab
-
-# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
-# mosquitto on loopback, which bridges to the central cluster broker over TLS
-# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
-# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
-# blips. The local broker writes nothing to the eMMC (persistence off, logs to
-# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
-edge_mqtt_host: 127.0.0.1
-edge_mqtt_port: 1883
-# Upstream cluster broker the local bridge connects to (TLS; the public
-# Let's Encrypt cert is validated against the system CA bundle).
-edge_mqtt_cluster_host: mqtt.renner.uno
-edge_mqtt_cluster_tls_port: 8883
-edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
-
-# --- USB devices (stable by-id symlinks). After physically moving the
-# sticks to edge-1, verify with `ls -l /dev/serial/by-id/` — the by-id
-# names travel with the device, not the host, so these should not change.
-edge_zwave_device: /dev/serial/by-id/usb-Nabu_Casa_ZWA-2_80B54EE15720-if00
-edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
-
-# --- Images. Versions mirror the k8s manifests at migration time.
-# sml2mqtt stays hand-pinned (critical meter bridge — was deliberately
-# excluded from image-automation in k8s, keep that property here).
-edge_zwavejs_image: ghcr.io/zwave-js/zwave-js-ui:11.21.1
-edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
-# Alpine version the zwave-js-ui image is built on — the source for the musl
-# libeatmydata.so preloaded into that container (see tasks "Extract musl
-# libeatmydata.so"). Keep in sync with the base if edge_zwavejs_image bumps
-# its Alpine major (check: docker exec zwave-js cat /etc/os-release).
-edge_zwave_alpine_base: alpine:3.22
-
-# --- CAN / open3e (Viessmann E3 external bus) ---
-edge_can_interface: can0
-edge_can_bitrate: 250000
-edge_open3e_pkg: git+https://github.com/open3e/open3e.git
-edge_open3e_mqtt_user: open3e
-edge_open3e_mqtt_topic: open3e
-edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-edge_open3e_poll_seconds: 30
-# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
-# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
-# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
-# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
-# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
-# the internal PV-surplus EMS depends on). Decodes via the master O3EList
-# codec inherited through the vx3plus overlay.
-# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
-# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
-# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
-# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
-# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
-# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
-# entries when added), so the multi-frame read is cheap at a 30 s poll.
-# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
-# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
-edge_open3e_datapoints: >-
-  vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
-# ECU addresses; -dev selects codec set, tx selects the UDS address.
-# vcmu (refrigerant-circuit controller) has no dedicated codec set —
-# omitting dpList makes open3e fall back to the generic datapoint list,
-# which decodes all polled VCMU DIDs.
-edge_open3e_devices:
-  vitocal: { tx: "0x680", dpList: "vcal" }
-  # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
-  # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
-  vx3: { tx: "0x6a1", dpList: "/opt/edge-host/open3e/vx3plus.py" }
-  vcmu: { tx: "0x68c" }
-
-# --- E3onCAN: passive E380 energy-meter listener (no bus writes, so it
-# coexists with the single open3e instance). The meter broadcasts at
-# ~5 Hz; the gap throttles MQTT to one update per DID per interval.
-edge_e3oncan_repo: https://github.com/MyHomeMyData/E3onCAN.git
-edge_e3oncan_version: 461dcbdf97e8d72d2edb9929c63b39382af6c3a1  # V0.5.2
-edge_e3oncan_gap_seconds: 10
-edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-
-# --- Secrets: define these in inventory/group_vars/edge_host/vault.yml
-# (ansible-vault). See README.md for the kubectl one-liners that read the
-# current values out of the cluster during migration.
-# edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
-# edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
-# edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"

ansible/roles/edge-host/handlers/main.yml

@@ -1,57 +0,0 @@
----
-# reload alone only re-reads config files; existing links keep their old
-# state until reconfigured.
-- name: Reload networkd
-  ansible.builtin.shell:
-    cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
-  changed_when: true
-
-# Ordered before 'Restart edge compose services': the daemon must pick up the
-# new default log driver before the containers are recreated onto it.
-- name: Restart docker
-  ansible.builtin.service:
-    name: docker
-    state: restarted
-
-- name: Restart journald
-  ansible.builtin.service:
-    name: systemd-journald
-    state: restarted
-
-# Handlers run in definition order, so this fires after 'Restart journald':
-# once the journal is volatile, the on-disk store is dead weight on the eMMC.
-- name: Purge persistent journal store
-  ansible.builtin.file:
-    path: /var/log/journal
-    state: absent
-
-- name: Restart alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    state: restarted
-    daemon_reload: true
-
-# Ordered before the MQTT clients (compose/open3e/e3oncan) so they reconnect
-# to a freshly-configured local broker.
-- name: Restart mosquitto
-  ansible.builtin.service:
-    name: mosquitto
-    state: restarted
-
-- name: Restart edge compose services
-  ansible.builtin.command:
-    cmd: docker compose up -d --force-recreate
-    chdir: "{{ edge_base_dir }}"
-  changed_when: true
-
-- name: Restart open3e
-  ansible.builtin.systemd:
-    name: open3e
-    state: restarted
-    daemon_reload: true
-
-- name: Restart e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    state: restarted
-    daemon_reload: true

ansible/roles/edge-host/tasks/main.yml

@@ -1,401 +0,0 @@
----
-# Arch edge host: zwave-js-ui + sml2mqtt as docker compose services,
-# open3e natively (venv + systemd) because it needs a SocketCAN interface,
-# which is host network state, not something to hide in a container.
-# Docker itself comes from the docker-install role (Arch-aware).
-
-- name: Install base packages
-  ansible.builtin.package:
-    name:
-      - python
-      - python-packaging  # ansible's pip module imports it on the target
-      - git
-      - mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
-      - grafana-alloy  # log shipper -> Loki (see Alloy section below)
-      - mosquitto      # local MQTT bridge broker -> cluster over TLS
-    state: present
-
-# --- Local MQTT bridge broker. Edge clients publish to this over loopback;
-# it bridges to the cluster broker over TLS (open3e/e3oncan can't do MQTTS).
-# Configured with persistence off + stdout logging so it writes nothing to the
-# eMMC. Deployed before the clients so they have somewhere to connect.
-# See edge-mqtt-bridge.conf.j2 / README "MQTT bridge".
-- name: Deploy local mosquitto bridge config
-  ansible.builtin.template:
-    src: edge-mqtt-bridge.conf.j2
-    dest: /etc/mosquitto/mosquitto.conf
-    owner: root
-    group: mosquitto
-    mode: "0640"   # carries the cluster bridge credentials
-  notify: Restart mosquitto
-
-- name: Enable local mosquitto bridge
-  ansible.builtin.systemd:
-    name: mosquitto
-    enabled: true
-    state: started
-
-# ======================================================================
-# eMMC wear mitigation + log shipping. edge-1 (Dell Wyse 3040) runs on a
-# small internal eMMC, so logs must not churn local flash. Strategy:
-#   - journald -> volatile (RAM), size-capped        (journald-edge.conf)
-#   - docker   -> journald log driver, no json files (docker-daemon.json)
-#   - Grafana Alloy tails the RAM journal -> Loki, WAL on tmpfs
-# plus TRIM, noatime and a weekly wear check for flash longevity.
-# See roles/edge-host/README.md "Logging & eMMC".
-# ======================================================================
-
-- name: Ensure journald drop-in directory exists
-  ansible.builtin.file:
-    path: /etc/systemd/journald.conf.d
-    state: directory
-    mode: "0755"
-
-- name: Switch the systemd journal to volatile (RAM) storage
-  ansible.builtin.template:
-    src: journald-edge.conf.j2
-    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
-    mode: "0644"
-  notify:
-    - Restart journald
-    - Purge persistent journal store
-
-- name: Resolve root filesystem facts (for noatime)
-  ansible.builtin.set_fact:
-    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
-
-- name: Refuse to touch fstab without a stable root UUID
-  ansible.builtin.assert:
-    that:
-      - edge_root_fs.uuid | default('') | length > 0
-    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
-
-# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
-# heartbeat that otherwise dirties the journal every couple seconds); noatime
-# drops access-time writes. Neither weakens data durability.
-- name: Mount root with noatime + lazytime (persisted in fstab)
-  ansible.posix.mount:
-    path: /
-    src: "UUID={{ edge_root_fs.uuid }}"
-    fstype: "{{ edge_root_fs.fstype }}"
-    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
-    dump: "0"
-    passno: "1"   # root fs — keep boot-time fsck ordering
-    backup: true
-    state: mounted
-
-- name: Enable periodic TRIM (helps eMMC wear-leveling)
-  ansible.builtin.systemd:
-    name: fstrim.timer
-    enabled: true
-    state: started
-
-- name: Install eMMC health-check script
-  ansible.builtin.copy:
-    src: edge-emmc-health
-    dest: /usr/local/bin/edge-emmc-health
-    mode: "0755"
-
-- name: Install eMMC health-check service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-emmc-health.service
-    - edge-emmc-health.timer
-
-- name: Enable eMMC health-check timer
-  ansible.builtin.systemd:
-    name: edge-emmc-health.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- Grafana Alloy: ship the volatile journal to Loki. Container logs reach
-# the journal via docker's journald driver (configured below), native
-# services log to the journal directly, so a single journal source covers
-# everything. Storage/WAL sits on /run (tmpfs) — see edge-alloy.service.
-- name: Create alloy system group
-  ansible.builtin.group:
-    name: alloy
-    system: true
-
-- name: Create alloy system user
-  ansible.builtin.user:
-    name: alloy
-    group: alloy
-    groups: systemd-journal
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-- name: Create alloy config directory
-  ansible.builtin.file:
-    path: /etc/alloy
-    state: directory
-    owner: alloy
-    group: alloy
-    mode: "0750"
-
-- name: Deploy alloy config (journal -> Loki)
-  ansible.builtin.template:
-    src: edge-alloy.alloy.j2
-    dest: "{{ edge_alloy_config_file }}"
-    owner: alloy
-    group: alloy
-    mode: "0600"   # carries the Loki basic-auth password
-  notify: Restart alloy
-
-- name: Deploy alloy systemd unit
-  ansible.builtin.template:
-    src: edge-alloy.service.j2
-    dest: /etc/systemd/system/edge-alloy.service
-    mode: "0644"
-  notify: Restart alloy
-
-- name: Enable edge-alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- SocketCAN: systemd-networkd owns can0. gs_usb is an in-tree module
-# and autoloads when the Innomaker adapter enumerates; networkd then
-# matches the interface and applies bitrate + up. RestartSec recovers the
-# link after bus-off without manual intervention.
-- name: Configure can0 via systemd-networkd
-  ansible.builtin.template:
-    src: 80-can0.network.j2
-    dest: /etc/systemd/network/80-can0.network
-    mode: "0644"
-  notify: Reload networkd
-
-- name: Enable systemd-networkd
-  ansible.builtin.service:
-    name: systemd-networkd
-    state: started
-    enabled: true
-
-# Docker networking needs no daemon.json (both services use network_mode:
-# host — real addresses, v4+v6, no bridge/NAT). The daemon.json below exists
-# only to route container stdout into the journald log driver so json logs
-# never touch the eMMC; the journal itself is volatile (RAM) and shipped to
-# Loki by edge-alloy. Both compose services inherit this default driver.
-- name: Configure docker daemon (journald log driver, eMMC-friendly)
-  ansible.builtin.template:
-    src: docker-daemon.json.j2
-    dest: /etc/docker/daemon.json
-    mode: "0644"
-  notify:
-    - Restart docker
-    - Restart edge compose services
-
-# --- Directory layout ---
-- name: Create service directories
-  ansible.builtin.file:
-    path: "{{ item }}"
-    state: directory
-    mode: "0755"
-  loop:
-    - "{{ edge_base_dir }}"
-    - "{{ edge_base_dir }}/zwave-js/store"
-    - "{{ edge_base_dir }}/sml2mqtt"
-
-# --- zwave-js eMMC write reduction: batch fsync via libeatmydata ---------
-# zwave-js persists its value DB with an fsync on *every* value change
-# (jsonl-db, throttleFS unset by zwave-js-ui). On a chatty network that
-# fsync storm churns the eMMC (~2.6 GiB/day for ~0.1 KiB/s of real data).
-# Preloading libeatmydata makes fsync a no-op so the kernel batches the
-# writes via normal writeback — full HA data resolution and cache
-# persistence are untouched (see docker-compose.yml.j2 for the mount + env).
-# The .so must match the zwave-js-ui image's musl/Alpine base, so extract it
-# from that exact Alpine version (bridge networking is broken on this host,
-# hence --network=host). Refresh by deleting the .so if the base ever bumps
-# its Alpine major version.
-- name: Extract musl libeatmydata.so (matches the zwave-js-ui Alpine base)
-  ansible.builtin.shell:
-    cmd: |
-      set -euo pipefail
-      tmp="$(mktemp)"
-      trap 'rm -f "$tmp"' EXIT
-      docker run --rm --network=host {{ edge_zwave_alpine_base }} \
-        sh -c 'apk add --no-cache libeatmydata >/dev/null 2>&1 && cat /usr/lib/libeatmydata.so' > "$tmp"
-      test "$(stat -c%s "$tmp")" -ge 8000   # sanity: real .so is ~14 KB, guards against a failed/empty pull
-      install -m 0644 "$tmp" {{ edge_base_dir }}/zwave-js/libeatmydata.so
-    creates: "{{ edge_base_dir }}/zwave-js/libeatmydata.so"
-  notify: Restart edge compose services
-
-# Because fsync is neutralized, a hard power-loss during the (rare) settings
-# write could truncate settings.json — which holds the Z-Wave network keys.
-# Keep a validated rolling backup on the eMMC so the keys stay recoverable.
-- name: Install zwave-js settings.json backup script
-  ansible.builtin.template:
-    src: edge-zwave-settings-backup.j2
-    dest: /usr/local/bin/edge-zwave-settings-backup
-    mode: "0755"
-
-- name: Install zwave-js settings backup service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-zwave-settings-backup.service
-    - edge-zwave-settings-backup.timer
-
-- name: Enable zwave-js settings backup timer
-  ansible.builtin.systemd:
-    name: edge-zwave-settings-backup.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- Compose services: zwave-js-ui + sml2mqtt ---
-- name: Deploy docker-compose.yml
-  ansible.builtin.template:
-    src: docker-compose.yml.j2
-    dest: "{{ edge_base_dir }}/docker-compose.yml"
-    mode: "0644"
-  notify: Restart edge compose services
-
-- name: Deploy zwave-js env (session secret)
-  ansible.builtin.template:
-    src: zwavejs.env.j2
-    dest: "{{ edge_base_dir }}/zwavejs.env"
-    mode: "0600"
-  notify: Restart edge compose services
-
-# sml2mqtt reads only a config file and has no env override for the
-# password, so the rendered config carries it — 0600, root-owned, same
-# trust boundary as the k8s init-container approach it replaces.
-- name: Deploy sml2mqtt config
-  ansible.builtin.template:
-    src: sml2mqtt-config.yml.j2
-    dest: "{{ edge_base_dir }}/sml2mqtt/config.yml"
-    mode: "0600"
-  notify: Restart edge compose services
-
-- name: Pull compose images
-  ansible.builtin.command:
-    cmd: docker compose pull
-    chdir: "{{ edge_base_dir }}"
-  changed_when: false
-
-# Tolerates missing USB devices: before the physical cutover the ZWA-2 /
-# FTDI sticks are still on the old k8s-edge VM, and a container whose
-# device is absent must not block the rest of the role (open3e/CAN moves
-# first). Re-run the play — or `docker compose up -d` — after each stick
-# arrives.
-- name: Start compose services
-  ansible.builtin.command:
-    cmd: docker compose up -d
-    chdir: "{{ edge_base_dir }}"
-  register: compose_up
-  failed_when: >-
-    compose_up.rc != 0
-    and 'error gathering device information' not in compose_up.stderr
-  changed_when: >-
-    'Started' in compose_up.stderr or 'Created' in compose_up.stderr
-    or 'Started' in compose_up.stdout or 'Creating' in compose_up.stdout
-
-- name: Warn about containers waiting for their USB device
-  ansible.builtin.debug:
-    msg: >-
-      docker compose could not start every service (USB device not present
-      yet — expected before cutover): {{ compose_up.stderr_lines | select('search', 'device') | list }}
-  when: compose_up.rc != 0
-
-# --- open3e: native install. CAN_RAW sockets need no capabilities, so
-# the service runs as an unprivileged system user.
-- name: Create open3e system user
-  ansible.builtin.user:
-    name: open3e
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-# Owned by the service user: udsoncan writes its log file into the
-# working directory.
-- name: Create open3e directory
-  ansible.builtin.file:
-    path: "{{ edge_base_dir }}/open3e"
-    state: directory
-    owner: open3e
-    group: open3e
-    mode: "0755"
-
-- name: Install open3e into a venv
-  ansible.builtin.pip:
-    name: "{{ edge_open3e_pkg }}"
-    virtualenv: "{{ edge_base_dir }}/open3e/venv"
-    virtualenv_command: python -m venv
-  notify: Restart open3e
-
-- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
-  ansible.builtin.copy:
-    src: open3e-vx3plus.py
-    dest: "{{ edge_base_dir }}/open3e/vx3plus.py"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e devices.json
-  ansible.builtin.copy:
-    content: "{{ edge_open3e_devices | to_nice_json }}\n"
-    dest: "{{ edge_base_dir }}/open3e/devices.json"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e environment (broker credential)
-  ansible.builtin.template:
-    src: open3e.env.j2
-    dest: "{{ edge_base_dir }}/open3e/open3e.env"
-    owner: open3e
-    mode: "0600"
-  notify: Restart open3e
-
-- name: Deploy open3e systemd unit
-  ansible.builtin.template:
-    src: open3e.service.j2
-    dest: /etc/systemd/system/open3e.service
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Enable open3e
-  ansible.builtin.systemd:
-    name: open3e
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- E3onCAN: direct E380 meter reads (passive; per-review decision not
-# to trust the VX3's relayed meter values). Same broker user as open3e;
-# topics land under the same open3e/ prefix as e380_<did>_<name>.
-- name: Check out E3onCAN (pinned)
-  ansible.builtin.git:
-    repo: "{{ edge_e3oncan_repo }}"
-    dest: "{{ edge_base_dir }}/e3oncan/src"
-    version: "{{ edge_e3oncan_version }}"
-  notify: Restart e3oncan
-
-- name: Create E3onCAN venv with requirements
-  ansible.builtin.pip:
-    requirements: "{{ edge_base_dir }}/e3oncan/src/requirements.txt"
-    virtualenv: "{{ edge_base_dir }}/e3oncan/venv"
-    virtualenv_command: python -m venv
-  notify: Restart e3oncan
-
-- name: Deploy e3oncan systemd unit
-  ansible.builtin.template:
-    src: e3oncan.service.j2
-    dest: /etc/systemd/system/e3oncan.service
-    mode: "0644"
-  notify: Restart e3oncan
-
-- name: Enable e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    enabled: true
-    state: started
-    daemon_reload: true

ansible/roles/edge-host/templates/docker-compose.yml.j2

@@ -1,56 +0,0 @@
-# Managed by ansible (roles/edge-host). Mirrors the former k8s manifests:
-# kubernetes/apps/home-assistant-zwavejs/deployment.yaml and
-# kubernetes/apps/home-assistant/sml2mqtt-deployment.yaml.
-#
-# No `privileged` needed here: docker's --device handles the device-cgroup
-# grant that hostPath in k8s could not (that was the only reason the pods
-# ran privileged).
-# Both services use host networking: real host addresses (v4+v6), no
-# bridge, no NAT. The broker LB is IPv6-first and docker bridge NAT66 is
-# a tarpit (unmasqueraded SYNs die silently); the host stack just works.
-services:
-  zwave-js:
-    image: {{ edge_zwavejs_image }}
-    container_name: zwave-js
-    restart: unless-stopped
-    stop_signal: SIGINT
-    tty: true
-    environment:
-      - TZ={{ edge_tz }}
-      # Batch the value-DB fsync storm to spare the eMMC (see role tasks /
-      # README "Logging & eMMC"). Neutralizes fsync only for this container;
-      # the WS/MQTT value stream to HA and cache persistence are unchanged.
-      - LD_PRELOAD=/usr/lib/libeatmydata.so
-    env_file:
-      - zwavejs.env
-    devices:
-      - "{{ edge_zwave_device }}:/dev/zwave"
-    volumes:
-      # musl libeatmydata.so extracted from the image's Alpine base (ro)
-      - ./zwave-js/libeatmydata.so:/usr/lib/libeatmydata.so:ro
-      # Persistent store: settings.json, device DB, network keys. Seeded
-      # from the k8s PVC `zwavejs-store` at cutover (see README). Since
-      # this host is stateful there is no init-container merge step —
-      # settings.json is edited via the UI and persists here; zwave.port
-      # stays /dev/zwave.
-      - ./zwave-js/store:/usr/src/app/store
-    # Host networking: binds 3000 (Z-Wave JS websocket for HA) and 8091
-    # (web UI) directly on the host.
-    network_mode: host
-
-  sml2mqtt:
-    image: {{ edge_sml2mqtt_image }}
-    container_name: sml2mqtt
-    restart: unless-stopped
-    # Outbound-only (serial in, MQTT out) — host networking sidesteps the
-    # docker NAT66 gap: the broker LB is IPv6-first and bridge-network v6
-    # SYNs leave unmasqueraded and die. The host stack reaches fd99::81
-    # fine (open3e proves it continuously).
-    network_mode: host
-    # Serial exclusivity: this is the ONLY reader of the meter port —
-    # HA's edl21 integration stays disabled (see sml2mqtt-config comment
-    # history in the k8s repo).
-    devices:
-      - "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
-    volumes:
-      - ./sml2mqtt/config.yml:/sml2mqtt/config.yml:ro

ansible/roles/edge-host/templates/docker-daemon.json.j2

@@ -1,3 +0,0 @@
-{
-  "log-driver": "{{ edge_docker_log_driver }}"
-}

ansible/roles/edge-host/templates/e3oncan.service.j2

@@ -1,32 +0,0 @@
-# Managed by ansible (roles/edge-host).
-# PASSIVE listener for the E380 energy meter's CANopen broadcasts — no
-# writes on the bus, so it may run alongside the (single) open3e
-# instance. Reads the meter directly instead of trusting the VX3's
-# relayed values.
-[Unit]
-Description=E3onCAN E380 energy-meter CAN listener (read-only)
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker e3oncan publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/e3oncan
-# Journald-friendly (python block-buffers pipes; lost on restarts).
-Environment=PYTHONUNBUFFERED=1
-ExecStart={{ edge_base_dir }}/e3oncan/venv/bin/python {{ edge_base_dir }}/e3oncan/src/E3onCANcollect.py \
-  -c {{ edge_can_interface }} \
-  -dev e380 \
-  -g {{ edge_e3oncan_gap_seconds }} \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser {{ edge_open3e_mqtt_user }} \
-  -mpass ${OPEN3E_MQTT_PASSWORD} \
-  -mfstr "{{ edge_e3oncan_mqtt_format }}" \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/edge-host/templates/open3e.env.j2

@@ -1,7 +0,0 @@
-# Managed by ansible (roles/edge-host). Read by open3e.service and
-# e3oncan.service.
-# Full user:password pair as one variable so the open3e unit can pass it
-# as a single -muser argument without exposing it in the unit file;
-# e3oncan takes user and password separately (-muser/-mpass).
-OPEN3E_MUSER={{ edge_open3e_mqtt_user }}:{{ edge_open3e_mqtt_password }}
-OPEN3E_MQTT_PASSWORD={{ edge_open3e_mqtt_password }}