docs: answer the direct-routing question from source instead of a test

The first draft ended on an open question and proposed a two-VM sandbox to
answer it. That was unnecessary: the question is answerable from the Cilium
source and the agent flag reference, and the answer is negative.

installDirectRoute takes its next-hop as a parameter, and nodeUpdate supplies
GetNodeIP(false), which prefers NodeInternalIP. NodeInternalIP comes from the
Kubernetes Node object; --ipv4-node looks like an override but initFromK8s()
runs after initFromConfig() and overwrites it. --direct-routing-device selects
a device, not an address. The same value feeds the VXLAN endpoint in tunnel
mode.

So cluster-internal traffic is welded to the kubelet InternalIP, and the routing
mode does not matter — tunnel sources the VXLAN from it, native uses it as the
next-hop. That deletes the one disruptive, non-rolling step the first draft
contained. Native routing buys nothing for this problem.

Since cluster-internal cannot be moved, the separation has to come from moving
management services off the node IP, and those do not share a mechanism, so each
is checked separately: etcd moves via cluster.etcd.advertisedSubnets, the
apiserver endpoint is already a VIP whose Layer2VIPConfig carries a link field,
the Talos API turned out to answer on every node address, and kubelet cannot
move because it is the node IP.

The result is four VLANs, but the boundary does not fall where the class names
suggest — kubelet stays with cluster-internal. Stated plainly rather than filed
under management and hoped over.

Recommends leaving the node IP where it is and relabelling VLAN 40 as the
cluster VLAN, rather than moving every node's identity to a new VLAN. Same end
state, and the address the cluster is administered through never changes.

Also records that the Ceph pattern is the only mechanism that fully detaches a
workload from the node IP, and that it is too expensive to generalise to
east-west traffic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.woodpecker/alert-monitor.yml

@@ -42,7 +42,7 @@ when:
 
 steps:
   - name: fetch-alerts
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key
@@ -135,8 +135,38 @@ steps:
         TITLES=$$(jq -c '[.[].title]' .ci-alert-monitor/open-issues.json)
         NOISE_BODY=$$(jq -r 'map(select(.title == "Alert monitor: noise log")) | .[0].body // ""' \
           .ci-alert-monitor/open-issues.json)
-        FPS=$$(printf '%s' "$$NOISE_BODY" | grep -oE '\b[0-9a-f]{16}\b' \
+        # Suppression entries are exactly the structured lines the write-back
+        # step emits: "- <fp> <ALERTNAME> — <YYYY-MM-DD>: <reason>". Parse THAT
+        # shape rather than grepping the whole body for 16-hex tokens. A reason
+        # routinely cites another alert's fingerprint in prose ("see the issue
+        # filed for fingerprint <fp>"), and the body-wide grep turned every such
+        # citation into a silent, permanent suppression of an alert that was
+        # never classified as noise — verified against the live noise log, which
+        # yielded 22 fingerprints for 21 real entries, the extra one being
+        # 2ddc41edb689fc33, an alert that HAD been filed as a genuine issue.
+        #
+        # Entries also expire after NOISE_TTL_DAYS. A noise verdict says "this
+        # occurrence was benign", not "this alert can never be interesting
+        # again" — but the fingerprint is sha256(alertname|scope) and therefore
+        # stable across recurrences, so without an expiry a single blip mutes
+        # that alertname+scope pair forever. Re-triaging a recurrence weeks
+        # later is cheap; missing a real outage because a July blip muted
+        # openbao's KubePodNotReady is not.
+        #
+        # Lines that fail to parse are simply dropped, i.e. they stop
+        # suppressing — this fails toward MORE alerting, never less.
+        #
+        # date is BusyBox here, not GNU: `date -d '30 days ago'` errors out, so
+        # the cutoff goes through epoch arithmetic and `date -d @<epoch>`, which
+        # this image does support.
+        NOISE_TTL_DAYS=$${NOISE_TTL_DAYS:-30}
+        CUTOFF=$$(date -u -d @$$(( $$(date -u +%s) - NOISE_TTL_DAYS * 86400 )) +%F)
+        FPS=$$(printf '%s\n' "$$NOISE_BODY" \
+          | sed -nE 's/^- ([0-9a-f]{16}) [^ ]+ — ([0-9]{4}-[0-9]{2}-[0-9]{2}):.*/\2 \1/p' \
+          | awk -v c="$$CUTOFF" 'BEGIN{gsub(/-/,"",c)} { d=$$1; gsub(/-/,"",d); if (d+0 >= c+0) print $$2 }' \
           | jq -R -s -c 'split("\n") | map(select(length > 0))')
+        echo "Noise suppressions in effect: $$(printf '%s' "$${FPS:-[]}" | jq length)" \
+             "(TTL $${NOISE_TTL_DAYS}d, entries dated >= $$CUTOFF)"
         # Synthesize pseudo-alerts for BLIPS: lookback series whose label set
         # is not currently firing at all (checked against the PRE-filter raw
         # snapshot, so live-but-null-routed alerts don't reappear as blips).
@@ -260,7 +290,7 @@ steps:
               '{body: ($$old + "\n" + $$add)}' > /tmp/gitea-patch.json
             gitea_write PATCH "$$API/issues/$$NOISE_NUM" /tmp/gitea-patch.json
           else
-            printf '%s\n\n' "Fingerprints the alert-monitor triage agent classified as noise. The dedup step skips any alert whose fingerprint appears below — delete a line to make that alert eligible for triage again. Keep this issue open; it is the suppression state." > /tmp/noise-header.txt
+            printf '%s\n\n' "Fingerprints the alert-monitor triage agent classified as noise. The dedup step skips any alert whose fingerprint appears below — delete a line to make that alert eligible for triage again. Keep this issue open; it is the suppression state. Entries EXPIRE after NOISE_TTL_DAYS (default 30) counted from the date in the line, so a one-off blip stops suppressing its alertname+scope pair once the window passes; re-add happens automatically if it is triaged as noise again. Only lines in the exact '- <fingerprint> <ALERTNAME> — <YYYY-MM-DD>: <reason>' form suppress anything — a fingerprint mentioned in prose does not." > /tmp/noise-header.txt
             jq -n --rawfile hdr /tmp/noise-header.txt --rawfile add /tmp/noise-lines.txt \
               '{title: "Alert monitor: noise log", body: ($$hdr + $$add)}' > /tmp/gitea-create.json
             gitea_write POST "$$API/issues" /tmp/gitea-create.json

.woodpecker/build-ci-images.yml

@@ -188,3 +188,61 @@ steps:
           --dockerfile Containerfile \
           --destination gitea.renner.uno/gitea_admin/cni-plugins:$TAG \
           --destination gitea.renner.uno/gitea_admin/cni-plugins:latest
+
+  # Runtime image, not CI tooling: alpine + the openssl CLI, for shell scripts
+  # needing s_client/x509. Named for what it is; first consumer is the paperless
+  # fetch-bridge-cert init container. Replaces an `apk add openssl` that ran on
+  # every paperless pod start. The openssl CLI is genuinely absent from the alpine
+  # base, so this one needs an image rather than a busybox substitute.
+  - name: build-alpine-openssl
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/alpine-openssl
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/alpine-openssl/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/alpine-openssl:$TAG \
+          --destination gitea.renner.uno/gitea_admin/alpine-openssl:latest
+
+  # Heating CAN readers: open3e (Vitocal/VX3) + E3onCAN (E380 meter), one image
+  # run as two services on edge-1. Replaces pip-into-venv on the host, where an
+  # unpinned VCS requirement re-cloned and restarted the heating bridge on every
+  # Ansible converge. Not a ci-* image -- it is a runtime artifact -- but it
+  # builds the same way and lands in the same registry so the audit sees it.
+  - name: build-heating-can
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/heating-can
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/heating-can/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
+          --destination gitea.renner.uno/gitea_admin/heating-can:latest

.woodpecker/ci-manifests-test.yml

@@ -0,0 +1,260 @@
+# Validate the GitOps tree and the pipeline definitions on PRs that touch them.
+#
+# Nothing covered kubernetes/** before this. deploy-infrastructure.yml only
+# gates opentofu/**, and ci-scripts-test.yml only runs scripts/ci/ unit tests —
+# so a broken manifest reached the cluster and was caught by Flux failing to
+# reconcile, or by a human noticing. Three separate defects on 2026-07-30 would
+# have been caught here instead:
+#
+#   * an unquoted YAML scalar containing ": " in .woodpecker/build-ci-images.yml,
+#     which made EVERY pipeline in the repo fail to compile (#443). `yq` parsed
+#     it happily — it is valid YAML, just a map instead of a string — so
+#     "it parses" was mistaken for "it is valid". Only woodpecker lint catches it.
+#   * a rotator pinned to an image digest that no longer existed (#458).
+#   * manifests whose kustomize wiring was only ever checked by hand.
+#
+# Deliberately CHEAP and offline: no cluster, no registry, no secrets. Everything
+# here is a static check that either passes or fails on the checkout alone, so it
+# can gate PRs without becoming a bottleneck or a flake source.
+when:
+  # Path-filtered on purpose, unlike ci-scripts-test.yml. That pipeline is one
+  # step on one already-cached image, so running it everywhere is free; this one
+  # pulls three images and builds every kustomization in the tree, which is real
+  # cost to put on a docs-only PR. The filter covers everything the steps actually read:
+  # kubernetes/** for the manifests and scripts, .woodpecker/** for the lint (and
+  # so an edit to this file re-runs its own checks).
+  - event: pull_request
+    path:
+      - "kubernetes/**"
+      - ".woodpecker/**"
+
+# ADDING A THIRD DISCOVERY CHECK: kustomize-build and shellcheck both hand-roll
+# the same shape — `find`, a floor guard against empty discovery, then a
+# file-fed `while read`. Two copies is deliberate: the two traps being commented
+# around (BusyBox find lacking `-printf`, and a piped loop running in a subshell
+# where `fail` cannot propagate) are different, and both are worth spelling out
+# where they bite. A third instance is the point to stop copying and factor it
+# into a shared script under `scripts/ci/` — which is already covered by
+# ci-scripts-test.yml, so it would gain unit tests in the process.
+#
+# Every step carries `depends_on: []` so all four run in PARALLEL and every
+# failure shows in one run. Woodpecker's default is sequential, which would stop
+# at the first failure and make a contributor fix-and-repush to discover the
+# next — and these four are independent, cheap, offline checks with no ordering
+# between them. Keep it on every step: `depends_on` is per-step, so dropping it
+# from one silently re-serialises that step behind the others.
+steps:
+  # Every kustomization must build. This is the check that catches a resource
+  # entry pointing at a file that does not exist, a bad patch target, or YAML
+  # that is fine in isolation but broken once overlaid — the class of error that
+  # otherwise surfaces as a Flux reconcile failure after merge.
+  #
+  # Verified green across every kustomization in the tree when this was added,
+  # so this gates new breakage rather than pre-existing debt.
+  #
+  # ci-k8s TAG SPLIT: this file pins a newer dated tag than e2e.yml, e2e-test.yml,
+  # alert-monitor.yml and issue-fix.yml, which still sit on an older one. That is
+  # staleness in those four, not a deliberate split here — everything should be on
+  # the newest dated tag. They are not bumped here because none of them are
+  # triggered by this PR, so it would ship untested changes under cover of a green
+  # run that never touched them. Tracked in #461.
+  - name: kustomize-build
+    depends_on: [] # see note above `steps:`
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
+    commands:
+      - |
+        set -eu
+        # NOT `find -printf '%h\n'`: that is a GNU findutils extension and this
+        # image ships BusyBox find (v1.37.0), which rejects it outright with
+        # "find: unrecognized: -printf". Because the result feeds a command
+        # substitution used as a word list, the failure is SILENT — empty output,
+        # zero loop iterations, fail=0, step exits green having validated
+        # nothing. That is exactly what this pipeline shipped in its first
+        # revision, and CI reported it as passing.
+        dirs=$(find kubernetes -name kustomization.yaml | sed 's|/kustomization.yaml$||' | sort)
+
+        # Assert the discovery actually found something. Without this, any future
+        # breakage of the find/sed above degrades back into a green no-op rather
+        # than a failure — the specific trap that made the first revision of this
+        # step useless while looking healthy.
+        count=$(printf '%s\n' "$dirs" | grep -c . || true)
+        if [ "$count" -lt 50 ]; then
+          echo "FAIL: found only $count kustomizations; floor is 50."
+          echo "      If the tree legitimately shrank, lower this floor in"
+          echo "      .woodpecker/ci-manifests-test.yml — do not delete it."
+          echo "      Otherwise discovery is broken — refusing to report success."
+          exit 1
+        fi
+        echo "discovered $count kustomizations"
+
+        # `while read` fed from a FILE, not a pipe: a pipe would run the loop in
+        # a subshell where `fail` cannot propagate, so a failing build would exit
+        # 0 — the same silent-pass shape this step already shipped once. Reading
+        # a file keeps the loop in the current shell, and read -r handles paths
+        # containing spaces without IFS games.
+        printf '%s\n' "$dirs" > /tmp/dirs
+        fail=0
+        while IFS= read -r d; do
+          [ -n "$d" ] || continue
+          if kubectl kustomize "$d" >/dev/null 2>/tmp/err; then
+            echo "ok    $d"
+          else
+            echo "FAIL  $d"
+            sed 's/^/        /' /tmp/err
+            fail=1
+          fi
+        done < /tmp/dirs
+        exit $fail
+
+  # Lint the pipeline definitions themselves. This is the one that would have
+  # prevented the repo-wide CI outage: `woodpecker lint` exits 1 on a config that
+  # fails to compile, and exits 0 on mere warnings — so it fails on real breakage
+  # without blocking on pre-existing noise. Both behaviours were verified against
+  # this repo (rc=1 reproducing #443, rc=0 on the tree as it stands).
+  #
+  # An earlier revision of this comment also asserted that several deploy-*.yml
+  # files emit "Additional property timeout is not allowed" warnings. That was
+  # observed under the v3.9.0 CLI this step used to pin, and it is NOT known to
+  # still hold on current CLIs — review raised the likelihood that the warning was
+  # schema drift in a 7-minor-old CLI rather than real debt in those files, which
+  # is one of the reasons this pin moved to latest stable. It cannot be confirmed
+  # from a green run either, because the loop below sends lint output to /tmp/out
+  # and prints it only on failure, so warnings are invisible when the step passes.
+  # The claim is dropped rather than restated: what this step guarantees is the
+  # exit-code contract above, which is what the gate actually rests on.
+  # Latest stable, per the standing rule that any software uses the latest
+  # stable version. Matching the deployed woodpecker-server is a CONSEQUENCE of
+  # that, not a separate requirement — the server is bumped by image-automation
+  # within the 3.x range, so both should land on the same release.
+  #
+  # They do not land together, though: the server is a Kubernetes manifest and
+  # is bumped automatically, this pipeline file is not and cannot be (see
+  # infrastructure/image-automation/README.md). So the server moves first and
+  # this trails until someone bumps it by hand — which is what happened at
+  # v3.16.0 -> v3.17.0, caught by the update audit (#426) rather than by CI.
+  #
+  # It matters here because the CLI compiles the config against its own schema:
+  # one behind the server can pass a config the server then rejects, or reject
+  # one it accepts, and the gate stops predicting what the server will do —
+  # which is its only purpose.
+  #
+  # `-alpine` is required, not cosmetic: the plain vX.Y.Z CLI image ships no
+  # shell, and the loop below needs `for`, `sed` and `echo` around the binary.
+  # A step using the plain tag fails to start rather than failing a check.
+  #
+  # DRIFT RISK — accepted, and NOT fixable here. The server auto-bumps via
+  # image-automation; this pin cannot, so the two will drift apart.
+  # Image-automation can only patch YAML that is a Kubernetes manifest —
+  # fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the
+  # YAML manifest must have `apiVersion` and `kind` fields." A Woodpecker
+  # pipeline file has neither, so an `$imagepolicy` marker here is silently
+  # ignored. That is a property of the FILE, not of `update.path`: widening the
+  # automation's path to `./` would buy nothing, and would additionally drag the
+  # whole ansible/ and esphome/ trees into the walk. Do not try it — it looks
+  # like the fix and is not.
+  #
+  # So this bump is manual, and drift is caught by the update-audit agent
+  # (ci/agents/update-audit/prompt.md, item 5) which is told to flag this image
+  # whenever it trails the deployed server. If you bump the server, bump here.
+  - name: woodpecker-lint
+    depends_on: [] # see note above `steps:`
+    image: woodpeckerci/woodpecker-cli:v3.17.0-alpine
+    commands:
+      - |
+        set -eu
+        fail=0
+        for f in .woodpecker/*.yml; do
+          if /bin/woodpecker-cli lint "$f" >/tmp/out 2>&1; then
+            echo "ok    $f"
+          else
+            echo "FAIL  $f"
+            sed 's/^/        /' /tmp/out
+            fail=1
+          fi
+        done
+        exit $fail
+
+  # Standalone shell scripts under kubernetes/ (sidecar and bootstrap scripts
+  # mounted via configMapGenerator). Severity is pinned to `error` on purpose:
+  # style and info findings would fail the build on existing scripts for no
+  # safety gain. All scripts in the tree are clean at this level today.
+  #
+  # NOTE this does NOT cover shell embedded in manifests (init containers, probe
+  # exec blocks) — shellcheck cannot see inside YAML. That embedded shell is
+  # where several real bugs lived this year, so it remains a gap worth closing
+  # separately rather than one this step quietly implies is covered.
+  - name: shellcheck
+    depends_on: [] # see note above `steps:`
+    image: koalaman/shellcheck-alpine:v0.11.0
+    commands:
+      - |
+        set -eu
+        files=$(find kubernetes -name '*.sh' | sort)
+        # Same no-op guard as kustomize-build: if discovery ever returns nothing,
+        # fail loudly instead of passing having checked no files.
+        #
+        # This floor is 1 while kustomize-build's is 50, and the asymmetry is
+        # deliberate rather than an oversight. That floor is loose relative to the
+        # tree it guards, so it catches both total failure and gross partial loss
+        # while tolerating normal churn. The shell-script set is small enough that
+        # any floor tight enough to catch PARTIAL loss would sit inside normal
+        # churn — deleting one script would fail CI spuriously, and the fix a
+        # hurried person reaches for is lowering the floor, which is how a guard
+        # quietly becomes decoration. A floor that has to be edited whenever the
+        # tree changes is not a guard, it is a snapshot with an exit code.
+        #
+        # So this catches the failure that actually happened here (discovery
+        # silently returning nothing, step green having checked no files) and not
+        # partial loss. Worth revisiting only if kubernetes/ grows a lot more
+        # shell, at which point a loose proportionate floor becomes possible.
+        count=$(printf '%s\n' "$files" | grep -c . || true)
+        if [ "$count" -lt 1 ]; then
+          echo "FAIL: no shell scripts found under kubernetes/ — discovery is broken."
+          exit 1
+        fi
+        echo "discovered $count shell scripts"
+
+        # Same file-fed while-read reasoning as kustomize-build above.
+        printf '%s\n' "$files" > /tmp/files
+        fail=0
+        while IFS= read -r f; do
+          [ -n "$f" ] || continue
+          if shellcheck -S error "$f"; then
+            echo "ok    $f"
+          else
+            echo "FAIL  $f"
+            fail=1
+          fi
+        done < /tmp/files
+        exit $fail
+
+  # Guard the convention that cost a day to diagnose: registry-local images are
+  # pinned by DATED TAG, never by digest. Digests here are garbage-collected once
+  # :latest and the dated tags move past them — and that happens on edits to
+  # .woodpecker/build-ci-images.yml, a shared file, so an unrelated image addition
+  # silently orphans the manifest a rotator was pinned to. 11 rotators were dead
+  # this way and would have failed on their first scheduled run months later.
+  #
+  # Scoped to the rotation directory, which is where the convention is documented
+  # and where the failure actually bit.
+  #
+  # The `grep -v kustomization.yaml` is load-bearing, not tidying: that file
+  # documents this very rule, so its comment contains the literal string
+  # `image:.*@sha256` and therefore MATCHES the pattern. Anchoring on `image:`
+  # does not prevent that — verified by running the bare grep, which returns
+  # kustomization.yaml. Only the filename filter excludes it. Drop the filter and
+  # this step fails permanently on its own documentation.
+  - name: no-digest-pins
+    depends_on: [] # see note above `steps:`
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
+    commands:
+      - |
+        set -eu
+        hits=$(grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml 2>/dev/null \
+                 | grep -v kustomization.yaml || true)
+        if [ -n "$hits" ]; then
+          echo "FAIL: digest-pinned images found; use a dated tag (see rotation/kustomization.yaml):"
+          echo "$hits" | sed 's/^/        /'
+          exit 1
+        fi
+        echo "ok    no digest pins under openbao/rotation/"

.woodpecker/deploy-infrastructure.yml

@@ -13,7 +13,7 @@ when:
 
 steps:
   - name: validate-secrets
-    image: alpine:3.21
+    image: alpine:3.24.1
     environment:
       TF_VAR_proxmox_api_url:
         from_secret: proxmox_api_url
@@ -128,35 +128,6 @@ steps:
       - printf "Host *\n\tStrictHostKeyChecking no\n" > ~/.ssh/config
       - echo "$VAULT_PASS" > /tmp/.vault_pass && chmod 600 /tmp/.vault_pass
       - export ANSIBLE_VAULT_PASSWORD_FILE=/tmp/.vault_pass
-      - |
-        # Ensure Talos image exists on all Proxmox nodes via API
-        TALOS_VERSION="v1.12.6"
-        TALOS_SCHEMATIC="88e3752db032a08082972bd0adf16a80849e3870c3f95cf5b669936c5699e5ad"
-        TALOS_URL="https://factory.talos.dev/image/$TALOS_SCHEMATIC/$TALOS_VERSION/nocloud-amd64.raw"
-        TALOS_FILENAME="talos-$TALOS_VERSION-$(echo "$TALOS_SCHEMATIC" | cut -c1-12)-nocloud-amd64.raw"
-        API_TOKEN="$TF_VAR_proxmox_user!$TF_VAR_proxmox_token_id=$TF_VAR_proxmox_token_secret"
-        for node in jupiter saturn neptune; do
-          EXISTS=$(curl -sk "$TF_VAR_proxmox_api_url/api2/json/nodes/$node/storage/local/content" \
-            -H "Authorization: PVEAPIToken=$API_TOKEN" 2>/dev/null | grep -c "$TALOS_FILENAME" || true)
-          if [ "$EXISTS" -gt 0 ]; then
-            echo "Talos image already exists on $node"
-            continue
-          fi
-          echo "Downloading Talos image to $node..."
-          curl -sk "$TF_VAR_proxmox_api_url/api2/json/nodes/$node/storage/local/download-url" \
-            -H "Authorization: PVEAPIToken=$API_TOKEN" \
-            --data-urlencode "url=$TALOS_URL" \
-            --data-urlencode "content=import" \
-            --data-urlencode "filename=$TALOS_FILENAME" \
-            --max-time 600
-          echo "Waiting for download on $node..."
-          for i in $(seq 1 60); do
-            sleep 10
-            EXISTS=$(curl -sk "$TF_VAR_proxmox_api_url/api2/json/nodes/$node/storage/local/content" \
-              -H "Authorization: PVEAPIToken=$API_TOKEN" 2>/dev/null | grep -c "$TALOS_FILENAME" || true)
-            if [ "$EXISTS" -gt 0 ]; then echo "Download complete on $node"; break; fi
-          done
-        done
       - tofu init -upgrade -backend-config="access_key=$AWS_ACCESS_KEY_ID" -backend-config="secret_key=$AWS_SECRET_ACCESS_KEY"
       - |
         if [ -n "$$TOFU_STATE_RM" ]; then

.woodpecker/e2e-test.yml

@@ -4,7 +4,7 @@ when:
 
 steps:
   - name: e2e
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key

.woodpecker/e2e.yml

@@ -30,7 +30,7 @@ steps:
       - sh scripts/ci/test-model-selection.sh
 
   - name: e2e-test
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key

.woodpecker/issue-fix.yml

@@ -49,7 +49,7 @@ steps:
     # (the issue-pick logic needs jq, which only the ci-claude image has);
     # minting a 1h read-only token per run is cheap. Degrades to
     # snapshot-only if the ci-diagnostics ServiceAccount isn't reconciled.
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key

.woodpecker/update-all.yml

@@ -1,32 +0,0 @@
-when:
-  - event: cron
-    evaluate: 'PIPELINE == "update-all"'
-  - event: manual
-    evaluate: 'PIPELINE == "update-all"'
-
-timeout: 120
-
-steps:
-  - name: update-all
-    image: gitea.renner.uno/gitea_admin/ci-ansible:2026-04-29
-    environment:
-      ANSIBLE_FORCE_COLOR: "true"
-      ANSIBLE_HOST_KEY_CHECKING: "false"
-      ANSIBLE_ROLES_PATH: "../roles:./roles"
-      SSH_PRIVATE_KEY:
-        from_secret: ssh_private_key
-      VAULT_PASS:
-        from_secret: vault_pass
-      PROXMOX_TOKEN_ID:
-        from_secret: proxmox_token_id
-      PROXMOX_TOKEN_SECRET:
-        from_secret: proxmox_token_secret
-    commands:
-      - mkdir -p ~/.ssh
-      - echo "$SSH_PRIVATE_KEY" | base64 -d > ~/.ssh/id_ed25519
-      - chmod 600 ~/.ssh/id_ed25519
-      - printf "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n" > ~/.ssh/config
-      - cd ansible
-      - echo "$VAULT_PASS" > .vault_pass && chmod 600 .vault_pass
-      - export ANSIBLE_VAULT_PASSWORD_FILE=.vault_pass
-      - ansible-playbook -i inventory/ plays/update.yml -v

ansible/docs/edge-host.md

@@ -33,13 +33,43 @@ Build and boot the dedicated installer image — see
    (partitions the internal disk — NVMe if present, else the eMMC; the
    installer autodetects — installs the ansible-ready base system with SSH
    keys, DHCP, docker — asks for confirmation before wiping).
-3. Add the DHCP reservation/DNS entry for `edge-1.srv.renner.uno` on the
-   UDM, confirm `ssh root@edge-1.srv.renner.uno` works.
-4. From the workstation:
+3. Confirm the box resolves and is reachable by name — `ssh root@<host>` must
+   work before anything else. The UDM registers DHCP leases in DNS, so the
+   record appears on its own; the host is identified by name everywhere
+   (inventory, plays, this file) and its address is never recorded. Do not
+   pin it to a reservation.
+4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
+   That group is the base: every edge thin client belongs to it, and it
+   assumes no peripherals.
+5. From the workstation:
    ```
-   ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host
    ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
    ```
+   One unrestricted run over the whole group. The play is self-contained --
+   it carries `certificates` and `common` itself, so there is no second
+   command and no `--limit`. Running it converges every edge host, which is
+   the point: a run that touches only the new box tells you nothing about
+   the others.
+
+## Peripherals
+
+Each attached device is its own inventory group and its own playbook. Group
+membership states which hardware a host physically has -- there are no flags
+and no `when:` on the roles, so moving a stick between boxes is a one-line
+inventory change.
+
+| device | group | playbook |
+|---|---|---|
+| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
+| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
+| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
+
+Add the host to the matching group, then run that playbook. They depend on
+the base play having run: all three need the local MQTT bridge, and two of
+them need docker.
+
+A host with nothing attached is simply absent from all three groups and needs
+none of these commands.
 
 ## Secrets (before first play run)
 
@@ -85,7 +115,7 @@ zwave-js (order matters — never two servers on one radio):
       `kubectl -n home-assistant exec deploy/zwave-js -- tar cf - -C /usr/src/app store | tar xf - -C /opt/edge-host/zwave-js/`
 - [ ] Scale down in-cluster: `kubectl -n home-assistant scale deploy/zwave-js --replicas 0`
 - [ ] Move the ZWA-2 stick to edge-1, `docker compose up -d zwave-js`
-- [ ] Point HA `zwave_js` integration at `ws://edge-1.srv.renner.uno:3000`
+- [ ] Point HA `zwave_js` integration at `ws://edge-1.iot.renner.uno:3000`
 - [ ] Nodes alive in the UI on :8091 → remove the flux kustomization
       `home-assistant-zwavejs` + PVC in a follow-up PR
 
@@ -113,8 +143,12 @@ Afterwards:
   deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
   changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
   the preloaded `.so` (see "Logging & eMMC" → zwave-js).
-- open3e updates: re-run the play; the pip task tracks the git default
-  branch (pin `edge_open3e_pkg` to a tag if that ever bites).
+- open3e / E3onCAN updates: both are pinned to commits in
+  `roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
+  up upstream changes -- that is deliberate, these drive the heat pump. The
+  weekly update-audit reports when either falls behind (see
+  `ci/agents/update-audit/prompt.md`); bump the pin and re-run.
+  Pinning to a tag is not an option for open3e: it publishes none.
 - Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
   restart afterwards.
 - can-utils (`candump`) is AUR-only on Arch — install manually via
@@ -136,8 +170,8 @@ role keeps logs off local flash and monitors wear:
   WAL/positions sit on `/run/alloy` (tmpfs), so the agent writes nothing to
   the eMMC either. Net effect: no log writes hit local flash — the trade-off
   is that logs are RAM-only and lost on reboot unless already shipped.
-- **Loki endpoint:** the unauthenticated srv-network Loki
-  (`loki.srv.renner.uno:3100`) is firewalled off the IoT VLAN, so Alloy pushes
+- **Loki endpoint:** the old srv-network Loki
+  (`loki.srv.renner.uno:3100`) no longer exists — Loki moved into the cluster, so Alloy pushes
   to the authenticated cluster ingress `https://loki.renner.uno/loki/api/v1/push`
   with basic-auth (`edge_loki_username` / `vault_edge_loki_password`).
   **Delivery only works once the `edge-1` htpasswd line has reconciled into

ansible/inventory/api.proxmox.yaml

@@ -1,22 +0,0 @@
-plugin: community.proxmox.proxmox
-user: root@pam
-# token_id: ansible
-# token_secret: provided via PROXMOX_TOKEN_SECRET env var
-url: 'https://jupiter.renner.uno:8006'
-validate_certs: true
-want_facts: true
-want_nodes: false
-keyed_groups:
-    # proxmox_tags_parsed is an example of a fact only returned when 'want_facts=true'
-  - key: proxmox_tags_parsed
-    separator: "_"
-    prefix: proxmox
-  - key: proxmox_status
-    prefix: status
-compose:
-  ansible_host: >-
-    (proxmox_lxc_interfaces | default([]) | selectattr('name', 'equalto', 'eth0') | map(attribute='inet') | list | first | default('') | split('/') | first)
-    if proxmox_vmtype == 'lxc' else
-    (proxmox_agent_interfaces | default([]) | selectattr('name', 'equalto', 'eth0') | map(attribute='ip-addresses') | list | first | list | first | default('') | split('/') | first)
-  ansible_user: "'root' if proxmox_vmtype == 'lxc' else 'ladmin'"
-

ansible/inventory/group_vars/all/netbird-rotatable.yml

@@ -0,0 +1,13 @@
+$ANSIBLE_VAULT;1.1;AES256
+32373134633930343261336362373335623735656634313034393731393661643361346634623439
+3561363963333533313264643036663731656465343162330a333964383462373934303766323533
+63613065313063626261383236396363656565623538666565646661353766643133306334396561
+3961376234623236620a616339326238366538333666316538396465666365373834313064653963
+38623233616633383435373836613438333763343932636239386263306238376137643531373161
+35386437313465643437343832623935353238316565366537646530323233363837656631383962
+32653332333162623966366163633732356637323461613966313263313066643966373335313162
+30306362346561633566316166313164323635356363643034663933363464663562343136646137
+66643536363734316533663032643637356136363237376365613330633961616665323430633135
+36323733633464306330666232663433616533373466633939353061663136666665323432373666
+36393930333435333238306238373931393232393861333334643139306138653336633534626362
+61323038303838396438

ansible/inventory/group_vars/all/vault.yml

@@ -1,201 +1,29 @@
 $ANSIBLE_VAULT;1.1;AES256
-32666334326535346466653862343263373365346261313136653366366664633436356536383437
-6666326436366433373163353262383031333264633530390a346539316165633939346663623264
-30653839376635626261323234326539313235623962306639396633343533363561376363363864
-3165323130633061350a346233396530303737316361653337643264383230613635383932636133
-61336561663264393831313364366665636166306437383835303332643934393062623163343861
-31643063373432643531663538666363613331626564376539336435343161333237333036343664
-31636635316631376563326637303337386337313330323838666562303463363361346531613363
-64383062396132373536663832356266383430316465643934636537396466366335636131623339
-34316166323730316463666465613734383339636264306431623535656362633739386362393134
-30343837363361333666616332646131663738663737663434303437336130313133306236313739
-36363731386163353136376230393262653334366239653032613034323333386433666262353364
-30326364343735643739626563393661616238313265633566356639623639383138303934653737
-64356139656163666263346539663733356239386333313763653434343163353436323937663034
-35653236616135383465656536323738346662396366653435383337383233643232656435356335
-63616138666662386337323865356439303961396138613965303465373766343939333933316664
-35356561353366663361306336303339633362623332386330383533336461306139613232306363
-62613731613835336338346236373636343463383433613938393865666165653935353830633966
-61393266623961343665343136323235393764313461306138343334366631313531396663666136
-34636436666238376138633461636438356461643461356139313437393736666338623038353933
-34633032653930623065623639646231346133343231383032663438326164313136613361356232
-38306565353737623532653231303963616336323563663137316139303136623234663432373661
-37333431663233656635373433323461663164303137633831306232626465303761633934336461
-65613534633665663565366634623362303964313862333936383032363137306336636563383435
-35366666313065393463656430303436346334313930616166383861613830373962353536303866
-39636165393861363161633734343838346338303036306639316231313738383831626639303833
-62316336623135336462633439323533383931666563643839383236636134363466383561633363
-64366466336362376234336463383639323165306631633437333639333263373837666565306163
-34666162336166623831353436363831666561376562613732333938353638333134323133346435
-61663261373433333635643761373465653031303236613465643931356433656161373463316331
-34326163633037633737393338313931353130306134663839623332666365373663643330353864
-36366535313032333262373038333161336230303165646133303261333839376432393435613434
-39633830316236653962663464363637333433393665353066653230383339666130653135343332
-32623165386535323832626536316665396564623064633564663437613930313665616163326637
-61363538333761633264396430326539643833313830346565643937656137376464646664313636
-38626165306661353631393031383063646463363031633435373735653838636634316634323738
-38313832376265616631623761393666346133636135666664643661646130306237383134383934
-39643561363962323261343236393630656435653261633933656262623737613130353531386438
-39643335306436636238393734646238653465626232643161316539346435353130633138393863
-63343965346361353036646561636339303063393033633162666639303166303265333233386137
-38643835616131303263366539646130643630366139623032366532653031303336323463326263
-37313566633032313931313565616131376635643030373833323838393633653531653664333032
-33623566333731636462366133653837636238383832353362653230393164343135663062373438
-61613832336633343038373565316431643432626532636435653438383536336166383436393035
-66613037663739616337383136663034643433333933643862396330356533636439666536353338
-30313761633961383337353030333862396164376337393338303565663963633662633134653362
-30346563643537323431313137636364623231376364386364633739313438376663343339333330
-64623663303535393262376437626637316532643139353262393231376135663161663862646666
-65303463393966393633613964353132303863373565353131396435623634336364313864373465
-61376335313830383534643431383966616565346466633633613933643231303732373938313434
-38383731653938656331646239353330666266336566633232353835653239393232366634663165
-31313766643539373133396563633637336133633034633236393336663064626630376632336264
-63346136613165613063316264646561633430613630396263333562366430326432323264383233
-30616631616535666631303838326330363339343463303831376531396164333539386263343334
-64313363626363316266343735313263613038383464396564313336373831393936336237303361
-65346161323961336631366263353161616562633533323036383366613637333832323466326535
-36376661346133383461626461646237373531333532656239333566323433333139633966343465
-36653130646636336534353331623062313863666637306462343139333031313234633636613232
-64383663626566616465623938346531623362383839316361366566336166386461336634303339
-63363931303866376462303634633033386332373564393932616638323764383630643839393630
-38373533636561613131343138383765376332336434303930326630303735313462316666333861
-37346336613339643466636665326466386664376438646666363537306533303036613135303230
-65303733366163363964613135306538393234636262326639613439363063326237306337666564
-32303161373661633231333662303565383134333737343530616631363337396333346261646636
-38663439653435623236366437643334393432383636316664306530303133396561396435313936
-38316637393365343936636438666461326436353636323537343363623337656536613662393333
-32346638636532333266323036623932316265303134633265366634623336386138653165643662
-35396637636136386366663762346236343565623133623336363530653039363166383534313632
-39333162396534376335646664383362366663373531646132643462396131333932653464326532
-39326633616335633030373463303665383261323763303863383566386536646136613362323034
-35313064313533616262666439663266353136616466643562666131366636623166333030346333
-62353539396131666434356238646439326337343239366234383866613065636232613863323931
-38363339623634643433393135396264363835386135303061623964343832323936313536623634
-34396531643839623932663565656535396337353934373462336231356133303335643931323131
-33383030653362323830376435393134666339333036336333376539363033666666396534636131
-64353663663633313966336439643334616332616536336238613162363435323938313636303930
-66306135613735653437366164336237656331346365646265656362316639656162386239326632
-34613431653433306633666632363639306266303361383230316539306161396566346137363133
-30333131666530636264656263303639636161366134633964393834663436326539626264383034
-34636134663833333764613237663233346363643566353166316230396235373261393739316237
-38373466623635333432326531613637303839633033636166633364383964626134386261636536
-64616432343936363664383532323936386136393332653131343533653633323639356234353363
-62613631333864663435616633396537373035646238653565646535306163623262396136356663
-63383537653231633766613231316666303830336334323738626138366434323164643465663034
-64343965363738626638363733646136316338633161383330306363643461643664303734663863
-38656135616165366438623237333161363234356538356466646266316136313832666161626566
-65306234663338323263376437363961353033396532643132336261336662656136656530343532
-61353736626665386437376230633336656663356361626437393932633765336439613665666632
-34363763656164326565633433643836383234363962613535353230356461613133666232396336
-32316264353535353162373833666365633266376634366362646264616161363634363138303832
-30656465326562653430373962346438646138336364363833646165353636346161336336613565
-37303931393865356136333431656264666331313165373037656535333832643662343434396337
-37656263363161396438626437643839633834643265313963653337363463373530333930303163
-34353163333461353763643631303763623034333665373235656531393561363432633433393137
-63333438646565633533353466373930313831316132306662363831623732336435346334323331
-32373736396239613839396535316166386637613435643637323932303239643339613737653761
-63323062333430636161316437383035326531346466336263353761636661326363656631613063
-62623863616533333036643563333463366432386232386436356632323863663862613466613931
-35636538663263336333613664353934623338356262643832356163326161373166633833333838
-36373537363934393137663462633833373364356230356136646639633737636130396231633730
-61356536373064356665646335633435646164623164333964323135343536663162656564653961
-63626432653439373938333039386661363330373136633831656230336237376436313664653633
-61623830326330663235376534313135303566636131373835383534363730616433363536346666
-38326532376338393462363430643334343137613363626132356133363534643336663063623136
-66623261343932373662663134323538666334623764376131623530373065316338653535313165
-35313337303563373737313963303134303435643563616561323963613565646466353162353637
-62623961626630323432323133666636356531316362633331313630393761303030363736643163
-63333565616232366665356535373632626366366532633036323839393266613730373331343765
-39343132643734623162386162353438353165653537303033313439326132343036333739623832
-30316132396532626236633665333162653536633134313339306661613066343937323032376235
-64386161623639636361313865613536373338333931623438366338616338326536383232363564
-30356139373865313432366337646366613361393262333336643535656438383562343763636236
-38616263613834613030363033613839613334323964343436326136626565643838323064313037
-34663964646636363038393134343634663637663030636339613864316165666265383632373637
-32373063616130343762393832663030636334646263626236663730336135376461643134646337
-63346534353264663031336164623330633737333766633662613863343431333138363337326536
-38363937313632633537363261633133636266376534346430643230623663333464393262396564
-37323632653036653336623636646537326636383865383338366138653463336638363965356536
-30373664313863396237613363373236353966303862376561653562666231353930653233383733
-38303261343436393261653436626136353237666633613166623035636466346539363436616364
-36386438666165316137303932346435313736616434643337303038383166386135326132656265
-64633866363532333465623135323761393837623934316561633637613433316334376138343764
-64646262356631373865393363353465643733623535626434666639613965383239343736356231
-31306135666539646234333934326664353137643261613131646364363739333035396437316362
-62343531656234376438376266373932376233643630383836323032633537383961383537376631
-38303062623032636264666534313266363835636132646664366162303430393161313964616633
-32613063393965653438643766373434613237393736643635653465366533363535623932386433
-38653530316364323033383330373433663865393837653466376666633434343062376432633138
-64653365373062643431353533363764666339663334393432353466623062646632383163656365
-63633365326565353730333164303237383766386135376630613234333834626335643738376339
-31626161346562613439396263646361343464636136636663363766356263636630323366326131
-37356166343031376635346433363335373137303933343539303832613363656334663964333462
-31613531383166666261336439356434323263306364616536636333316131373930646262386366
-33333032643464313165336265326130333637323461643131393437326535316164356665313636
-61613237356331393463336431316539633133326334396134663035396531633538666331393132
-64393134643832376230656332633633396333313464333066656461366335343131613038656535
-32353037316132393936373834633535646631653464616331633830326131333361316464393762
-34306136306633323766353235353663666539663036383636393335336263353134303634626131
-31316334663733666530653330333263653135643539353439306165353339623032643633393632
-37653636663164646438626638643630393135623934646166666635316532366264623165616131
-64653736306431623639303564306431616263613966656563373861616339316466383761643439
-64323836303162613931313534393436353464333330353635663034396631636563616464653530
-34616164323362653231343433323338633930323035396630326461666461346338363039383737
-38666431373633376536323739316132316231353738306632336537633938653535316530346634
-65323937336635373736346438323133353333343235623463663264623838316239366165306535
-30323136626561636537393066383539303233366262396538383563353730356537623266623038
-33316135303833323663363733353634653763383166633663313465303838646266303264363339
-30363637396161656332323333653931336661393566663232326466646563356436306332333532
-35656331383661376538633664346537613161303135643937643161383034643532653961346533
-31396237653234363832383437613433326165636262333236393334613437386638636535616634
-34313330643139386232396332376135376332613333633938366533616435303232376366623237
-37323535383932623631363431383165323338306338633066373937323361383336616265303234
-32343132303366343866303837326566383865393764343130616366373536623766316238363430
-31633734633438313065336431316161636465333166653765653962326636663065336532393731
-33386139383466626565663666373664313030663132313233356136613435663265396332656633
-33636436646565643838643930343938326231623738393661393738343062303661353362646465
-31653335343638613238333636326337396438373263663666373263616662653634613036663364
-66323639336233623538663061366462663766383539333432336363613938346263636436336430
-36646639353163316661356636393034653934666563356335396631323166313532353361653630
-61353261656431613135313965373061313038653237626231366332306432633538656462323739
-38313831663861663738336564333731363937653663663031343261633538643535336264306230
-31656365326261633730613765393861376662656137623832363832633266346135643133633331
-39623836633337323934653464333731626361316638333565393435656132626631613739366535
-37653266333432386334333461653231346630646435343230636531326338396566396466376532
-32666632383430393064353861373033393538366266333339373436353932656333366264393936
-39373938346633616534643734313934643438333965323133313037373737303964643337653135
-31623038356263643438343635643530663362656630393164663634366461623464353532353239
-38623662666235383139663162343836316331396266313062316639373034376337393563643739
-65313564303962353061616265393135326266386266316538616430383966306163386533376137
-36393565336435636535376639623931616435353532356438666663383961356638353333366139
-63313962643638323862353738316233613862346532306631386536323261636235343565353563
-64613465343662663034643138623865363862316131363461343636333864656132626166393836
-32393737386166343630323861376562353337663030613263613664663938376337333533613432
-33343064636365643831373135623466613738303164326562386430386339653831313536366532
-32613430313461663662313966373238643934633938633764363365386139303337643533313961
-61646131653638616365346236616330363566636261636362363635323566353562326239393530
-33323934623530306436623963663735353337306162646638383934326530333163396165366139
-61653339363166376266643733653536636639653939346666663834376361373533363231636433
-35363665373031393231363161333036303239313032326134363432363034623762366637643830
-32353132353961343830303562613964376234303461623263343735336238353565353238363438
-61326164323265386431353130346364393466353334386336376139303036646634376331313561
-39386331366337646634366635623066313432623537323164646263323532303465343735326364
-33383162626139383232636362613831303832633562636136393138316334303832383666313963
-37643835663065613236303338353435663830343637326563343665616532633830623035336534
-39376536663937653939326261346263613265636363333531323931646633313837386238333435
-33646339626462386635316230636135626330643630633664636364313539626332643331356334
-32313736343031633631393662363638626137303331383263656335373966383335653133333932
-34393861303665613633366461666664376466383437303065653938323166633562643834356139
-32616564373731313134316661343839333830663338636132323265376133656337393632353931
-35633531383438656461613135323134623134393062356661326435323065616331656638306432
-62313130643763626336383762306535656238393961653537336465373064653432653665373362
-63666366343866613235376535313835623261313531653265373636383030613930353530346265
-64303531316239386630626565323966616535373936393766663063623464663864363338343466
-33666237613532326661626461343166323162366532386638333835663838343865346266646533
-66343533326562396331363137353631633965623561333963613565636264653031616339613232
-39666362643739663837326432623462646233306239326463616232383236346236383132396637
-34623332346533366362613465363263373339643033663264363332336463343337333466393866
-33333363313331376634363431366636376466336131363330323530306162666434393764393563
-35633530353839386362326262343635383434333338626563383463653263333430383465626535
-63363933343365326665343166633939313739316231383938376166646331623663343161623865
-36646630336337373865
+30353730633631376539373334376531353263396339313464373538396231616166653738383161
+6438656132616365646463653464396236313437323537620a363632643061333036353537626461
+34393065613836343536383565303066326133363937386263653961663666643430663139653961
+6462363235343063350a636165376232366636363233376637636231363839343738346134623562
+63663839326435303964313730336436396365343163343366613966323938656263303166326435
+65303931393233623064343763633161353161643635396365316362333137353732346366643162
+36386132386638623333623839373161386330353635346331393735366662393935353264646439
+36386539393432373765396562613033656339646532323864366230393638313161383363636636
+32353266376563393239393734316461313566386233333164623135303132346164353165323338
+62306333363838393236346435303339393639623231353232643966613163313134336265633636
+64366539326639303362393933303465376532626663346431613563353032383037363532646637
+38376266323537343437643635343634636164393230313632663533666434306134366261383230
+66383135303565383763646331383338646339613539623835366438393635363832366134316531
+63636466653239383231616139663231623636376564376234383762383066613030313937616533
+33396135616236643034303236353230376434313365326633303961303232363766633161353434
+64333832623535343733353462313862343934633363616237353331386264386632643538333939
+34636437393934326131363236346636663665613537343134313636623439356665636663613938
+37346634356164343537396363663730306232623964636635333333653738383538343236383535
+66326462373964333234343435336630643930333433393034303439366631386666393330343238
+61336237373236313533333732343064336637326636663433396362346631343165656163313561
+30383331326231383539343639343130323731333432306630353835343338613861376631646264
+66383539616663376237663234313431313064663566313566366664386439316161353135306338
+35373962636537653535333935306564303235356232313463386237353737313736386463396164
+62666165336539316565623634326562356434313833643833353635303361353937666663356263
+35653665323736386263333437666432623332333936373938376637633463653930326466383139
+38336239646135316565356564373731386662383234386433633632393165343331353835623437
+30663138633430356334353832633434633939373337663437326263336537356633313763353337
+64373764666430643433

ansible/inventory/group_vars/authentik.yml

@@ -1,39 +0,0 @@
-$ANSIBLE_VAULT;1.1;AES256
-32656264356562366161643464343065313163663461663038646531306532383532393238396566
-3831646133656131336266346531323035313234663336380a356536313965383066363135343739
-32663032373165643931663762333036393430333437366630326562653034636464336166313730
-6666366634346664330a643063343061623465393436656461303862336431393931643638333735
-36663561626134333639326635373735623739366366396564653763656661663866623639323233
-63326130303065663265356136343266613966333230653935343836343461623535363161346439
-64653130383539363933636437643035306536376439326539666662383064636337363265356135
-39623537303165356635373666643634383731333236643061343033376466643762373636333834
-65626435353563623938363566626330356662366666666263363038646334323731393631356531
-37346230303134326135633033393162333839346664363062633736363861343435623331343762
-36643266333965323861633836336234333839306330343961653636623439656432613662303865
-33313438316236653461346264373338393539326438666337346162326163313837626364623461
-37353135323866326536303138363165396339306237393064366335386339303238373563373064
-62646430313131373964346466373164306432363361626661653531356261313233616563353465
-33353661663930333037636637353234663231633339333436393362393363653066376337376364
-61663035396563643665353162356466623032643938353933393930656162386631376132383461
-65383265623136646261393863336635323634336136376534323231383930383632306161393136
-66366563633737613931633233666336363166313930613963396636633761633638323465306332
-65333764343762643133613036353937386338303336656530366433333031323336366165346435
-62626533363961636361306664616638396538646530633730373636643131303636333036326534
-65616361353435346161383366626334386536376630326334353837396161356637356431316364
-31643636326262383362336535346237666465363165643830613932336362623635366138303336
-66376362386531313063336266623033653736616430386635656134616461643238373831333062
-62343964343032366434316365626232656337613832313263353237353336643735353464656262
-66313765393332393663303562393438376139373565623734653432626134376630343863353061
-31383530643264643330313136356638646265663831363637363464636634356639313936336437
-33393961363930616435623864303732633564303937343638373761653761666339373433373432
-39653333396138383332353465396639346535666536373232653536393163353835613239386530
-62666536633636363035613063643965623735366333386335363066326238323835663535333162
-36613366326137303564653463393665666337363664363563643361373766663331333363326533
-36353265616261613232633833616362363433313562303536326537346662383239343965313964
-36353534666535313239316166343163343132323134313266653537646136383262626233376332
-31616166366234653632373732633061373362336137303831353763373166643031313466656434
-64623362663762316465633564633462363866373762383937386362623462333461666462613336
-64393633336137663631386562366339646464363433396366303466336561653938386639393563
-33613936356336346134323830366236326231653465373438623433613634656666386162336331
-32316637383637336266656637343639613233623739303438323033333935663261373230626661
-6635353534313662646665613438336264303835633564303166

ansible/inventory/group_vars/edge_host/vars.yml

@@ -1,6 +1,6 @@
 ---
 # Secret indirection — real values live in vault.yml (ansible-vault) next
-# to this file; see roles/edge-host/README.md for how to populate it from
+# to this file; see docs/edge-host.md for how to populate it from
 # the cluster secrets.
 edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
 edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
@@ -36,3 +36,18 @@ keepalived_ntp_sysctl:
 
 # Realtek NIC on the Wyse 3040 has no hardware timestamping.
 chrony_hwtimestamp: false
+
+# --- Local MQTT contract -------------------------------------------------
+# Shared between the broker and its clients, so it belongs to the inventory
+# rather than to either side's role. mqtt-bridge listens here; heating-can and
+# power-meter-sml publish here; and the bridge authenticates to the cluster
+# broker AS the open3e user, so it needs that identity too.
+#
+# Keeping these in one role's defaults only worked while a single play loaded
+# every role at once. Splitting the plays by capability exposed it: the bridge
+# could not render without a heating-can default, and the CAN services could
+# not render without an mqtt-bridge default.
+edge_mqtt_host: 127.0.0.1
+edge_mqtt_port: 1883
+edge_open3e_mqtt_user: open3e
+edge_open3e_mqtt_topic: open3e

ansible/inventory/group_vars/n8n/vars.yml

@@ -1,6 +0,0 @@
----
-# n8n group variables
-# Maps vault secrets to role defaults
-
-n8n_encryption_key: "{{ vault_n8n_encryption_key }}"
-n8n_postgres_password: "{{ vault_n8n_postgres_password }}"

ansible/inventory/group_vars/ntp_cluster.yml

@@ -0,0 +1,18 @@
+---
+# Shared configuration for every NTP server (see plays/install-ntp-ha.yml).
+
+# IPv6 service address -- what ntp.srv.renner.uno resolves to, and the path
+# clients actually use. The IPv4 VIP is legacy.
+#
+# Defined at group level rather than in the play because install-ntp-ha.yml runs
+# two plays over this group -- withdraw the static assignments fleet-wide, then
+# hand the address to keepalived -- and both need the same value. Duplicating a
+# service address across plays is exactly the sort of thing that drifts.
+ntp_vip6: "fd40::123/128"
+
+# Link-local address carried as the FIRST virtual_ipaddress of the IPv6 VRRP
+# instance. RFC 5798 gives an IPv6 virtual router a link-local primary, and
+# keepalived warns "the first IPv6 VIP address should be link local" without
+# one. fe80::123 mirrors the service address and cannot collide with a real
+# NIC's EUI-64 link-local.
+ntp_vip6_linklocal: "fe80::123/64"

ansible/inventory/group_vars/proxmox_all_lxc.yml

@@ -1,2 +0,0 @@
----
-ansible_user: root

ansible/inventory/group_vars/proxmox_docker/vars.yml

@@ -1,3 +0,0 @@
----
-# Docker hosts - tagged with 'docker' in Proxmox
-docker_monitoring_enabled: true

ansible/inventory/group_vars/pve_cluster.yml

@@ -1,8 +0,0 @@
----
-# --- NTP (see plays/install-ntp-ha.yml) ---
-# The PVE hosts hold the 10.2.40.4 VIP via keepalived on the VLAN 40 bridge
-# sub-interface. This role disappears when these nodes are rebuilt as bare-metal
-# Talos, which cannot run chrony or keepalived — edge-1 is being brought into
-# ntp_cluster to take over.
-ntp_keepalived_enabled: true
-ntp_keepalived_interface: "vmbr1.40"

ansible/inventory/host_vars/grist.srv.renner.uno/vault.yml

@@ -1,20 +0,0 @@
-$ANSIBLE_VAULT;1.1;AES256
-66356134353363643130323632626530653863306635396432353539623731346138323635336632
-3536633563333231373333323564373763363936356535630a333135623336613832313833626531
-34646137633633353266653536363934303939626564353036343033643462373439643139353065
-6539383436343233300a396339356562363832373962306337663464363961346335656130653235
-38646563656538373530643531643964633934623465643065626266396130643966343166313937
-65663965613937323634323436386130653633656261653065306365316135343531653737663138
-65303339323137653362383766343361643837356531643339613431333238613564343563333761
-30616530613133633762306533363134383265313031343230313764633238626565623935313134
-38323435653264643534366366313862646161373434623135366164323236363733653533656233
-34336233396465626264666637363532646562306237363533616461613764306433366639623562
-30393234396238353038356636373632326130306431663439316363646530636563666364316139
-34326333363536343963393833616465303636323166643638623061353964303265393733366234
-33373435643237636536363364373830353833346235333534626561343465373236643165646664
-39396233363531393835656463666662616633643566383061323161663061323831356337333337
-65653336663735633536353263316634373639313238343365353833646666353131656530373361
-32643133353737366465633635663339666138323035633537653462303532366632626262373532
-32643132363062623033356134356135346361343234656636663936336338303230616665326630
-65353231356538633034626332633831396362373733646539313834386535666631613463656639
-373864663230353339666137653835656630

ansible/inventory/host_vars/loki.srv.renner.uno/vars.yml

@@ -1,11 +0,0 @@
----
-# Loki server - central log aggregation host
-
-# Enable syslog receiver for remote devices (Synology NAS, etc.)
-# Synology DSM's "IETF" format actually sends BSD/RFC 3164 format: <pri>MMM DD HH:MM:SS host app: msg
-# Using raw format to accept messages and parse them with regex
-alloy_syslog_receiver_enabled: true
-alloy_syslog_receiver_port: 1514
-alloy_syslog_receiver_protocol: "udp"
-alloy_syslog_receiver_format: "raw"
-alloy_stability_level: "experimental"

ansible/inventory/host_vars/paperless.srv.renner.uno.yml

@@ -1,8 +0,0 @@
----
-# Paperless server specific configuration
-
-# Extra scrape targets for Alloy
-alloy_extra_scrape_targets:
-  - name: "paperless"
-    address: "127.0.0.1:9199"
-    job: "paperless"

ansible/inventory/host_vars/wiki.srv.renner.uno/vars.yml

@@ -1,7 +0,0 @@
----
-# Wiki server configuration
-
-# Home Assistant documentation generation (disabled - see HA Wiki Documentation project)
-homeassistant_docs_enabled: false
-homeassistant_url: "http://10.2.30.11:8123"
-homeassistant_token: "{{ vault_homeassistant_token }}"

ansible/inventory/pve_hosts.yml

@@ -1,19 +1,14 @@
-[pve_cluster]
-jupiter ansible_host=10.2.40.201 ansible_user=root
-neptune ansible_host=10.2.40.202 ansible_user=root
-saturn ansible_host=10.2.40.203 ansible_user=root
-
-[authentik_db]
-authentik-db-1.srv.renner.uno ansible_user=root
-authentik-db-2.srv.renner.uno ansible_user=root
-
-[authentik:children]
-authentik_db
-authentik_app
-
-[authentik_app]
-authentik-app-1.srv.renner.uno ansible_user=root
-authentik-app-2.srv.renner.uno ansible_user=root
+# Hosts are reached by name. An address in an inventory is allocator state
+# standing in for identity, and it pins access to the legacy family -- the names
+# resolve IPv6-first, which is the primary path on this network. DNS is the
+# authoritative source for where a host is; this file says which host, not where
+# it currently sits.
+#
+# Gone from this file, and why:
+#   pve_cluster (jupiter/neptune/saturn)  run Talos on bare metal; no SSH at all
+#   authentik_db / authentik_app          authentik runs in-cluster (CNPG)
+#   grafana                               runs in-cluster
+# Their hosts no longer resolve, except the three that resolve to Talos nodes.
 
 [netbird_server]
 netbird.renner.uno ansible_user=root
@@ -26,18 +21,51 @@ proxy-2.renner.uno ansible_user=root
 [synology_nas]
 nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
 
-[grafana]
-grafana.srv.renner.uno ansible_user=ladmin
-
+# Every bare-metal edge thin client. Gets the host-generic roles only:
+# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
+# sub-interface. Nothing here assumes any peripheral is attached.
 [edge_host]
 edge-1.iot.renner.uno ansible_user=root
 
-# NTP servers. Includes edge-1 so the role survives the PVE hosts being
-# retired for bare-metal Talos, which cannot run chrony or keepalived itself.
-# See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly
-# than chrony within this group.
+# Capability groups. Membership states which hardware a host physically has,
+# and each service playbook targets the group for its own concern. No flags and
+# no `when:` on the roles -- moving a stick to another box is a one-line change
+# here, which is what it should be.
+#
+# These are children of [edge_host], declared below: a peripheral is added to an
+# edge client, it does not replace the base. Making that a :children relation
+# rather than a comment means a host added only to a capability group still
+# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
+# cannot silently come up without edge_mqtt_host and friends.
+[zwave_gateway]
+edge-1.iot.renner.uno
+
+[heating_can]
+edge-1.iot.renner.uno
+
+[power_meter]
+edge-1.iot.renner.uno
+
+[edge_host:children]
+zwave_gateway
+heating_can
+power_meter
+
+# NTP servers. edge-1 was added here so the role would survive the PVE hosts
+# being retired for bare-metal Talos, which cannot run chrony or keepalived
+# itself. That retirement has now happened, so edge-1 is the ONLY member.
+#
+# THIS IS NO LONGER HIGHLY AVAILABLE. The group was jupiter (MASTER, prio 100),
+# saturn (90), neptune (80) and edge-1 (BACKUP, 70); the first three are gone.
+# A single VRRP member takes the address regardless of its configured state, so
+# fd40::123 is still served — verified answering — but by one thin client with
+# no failover behind it. install-ntp-ha.yml still carries the four-host
+# ntp_servers priority map, which now describes hosts that do not exist.
+#
+# Restoring redundancy needs a second chrony host that is not a Talos node.
+# See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly than
+# chrony within this group.
 [ntp_cluster:children]
-pve_cluster
 edge_host
 
 # Off-site Debian VM (Synology VMM on nas-2, 10.3.40.0/24) running Claude Code.

ansible/plays/backup-authentik-db.yml

@@ -1,48 +0,0 @@
----
-# Backup Authentik PostgreSQL database for K8s migration
-# Creates a pg_dump on the primary DB host
-#
-# Usage (via CI):
-#   ansible-playbook -i inventory/ plays/backup-authentik-db.yml
-#
-# Output: /tmp/authentik-backup.sql on the primary DB host
-
-- name: Backup Authentik PostgreSQL database
-  hosts: authentik-db-1.srv.renner.uno
-  become: true
-  tasks:
-    - name: Create database backup
-      become_user: postgres
-      command: >
-        pg_dump
-        --clean --if-exists
-        --no-owner --no-privileges
-        --format=custom
-        --file=/tmp/authentik-backup.dump
-        {{ authentik_db_name | default('authentik') }}
-      changed_when: true
-
-    - name: Make backup readable
-      file:
-        path: /tmp/authentik-backup.dump
-        mode: "0644"
-
-    - name: Get backup size
-      stat:
-        path: /tmp/authentik-backup.dump
-      register: backup_stat
-
-    - name: Report backup size
-      debug:
-        msg: "Backup created: {{ (backup_stat.stat.size / 1024 / 1024) | round(1) }} MB"
-
-    - name: Fetch backup to controller
-      fetch:
-        src: /tmp/authentik-backup.dump
-        dest: /tmp/authentik-backup.dump
-        flat: true
-
-    - name: Clean up remote backup
-      file:
-        path: /tmp/authentik-backup.dump
-        state: absent

ansible/plays/backup-paperless-db.yml

@@ -1,59 +0,0 @@
----
-# Dump Paperless-ngx DB from the legacy VM to a local file on the
-# controller (CI runner), where a downstream pipeline step consumes it
-# to produce a Barman seed for the CNPG `bootstrap.recovery` path.
-# Data + media tarballs are handled separately by `backup-paperless.yml`.
-
-- name: Dump Paperless DB from legacy VM to CI runner
-  hosts: paperless.srv.renner.uno
-  become: true
-  vars:
-    paperless_db_name: paperless
-    paperless_db_user: paperless
-  tasks:
-    - name: Find running paperless DB container
-      ansible.builtin.shell: >
-        docker ps --filter "label=com.docker.compose.service=db"
-        --format '{{ "{{" }}.Names{{ "}}" }}' | head -1
-      register: paperless_db_container
-      changed_when: false
-
-    - name: Fail if DB container not found
-      ansible.builtin.fail:
-        msg: "Could not find paperless DB container (compose service=db)"
-      when: paperless_db_container.stdout | trim == ""
-
-    - name: Dump DB to /tmp on legacy host
-      ansible.builtin.shell: >
-        docker exec -i {{ paperless_db_container.stdout | trim }}
-        pg_dump --clean --if-exists --no-owner --no-privileges --format=custom
-        -U {{ paperless_db_user }} {{ paperless_db_name }}
-        > /tmp/paperless-backup.dump
-      args:
-        executable: /bin/bash
-      changed_when: true
-
-    - name: Make dump readable
-      ansible.builtin.file:
-        path: /tmp/paperless-backup.dump
-        mode: "0644"
-
-    - name: Report dump size
-      ansible.builtin.stat:
-        path: /tmp/paperless-backup.dump
-      register: dump_stat
-
-    - name: Show dump size
-      ansible.builtin.debug:
-        msg: "Dump created: {{ (dump_stat.stat.size / 1024 / 1024) | round(1) }} MB"
-
-    - name: Fetch dump to controller
-      ansible.builtin.fetch:
-        src: /tmp/paperless-backup.dump
-        dest: /tmp/paperless-backup.dump
-        flat: true
-
-    - name: Remove remote dump
-      ansible.builtin.file:
-        path: /tmp/paperless-backup.dump
-        state: absent

ansible/plays/check-dist-upgrade.yml

@@ -1,95 +0,0 @@
----
-- name: Check for Distribution Upgrades
-  hosts: proxmox_all_qemu:&status_running:!proxmox_noansible
-  become: true
-  gather_facts: true
-  ignore_unreachable: true
-  
-  tasks:
-    - name: Check Ubuntu Release Availability
-      ansible.builtin.command: do-release-upgrade -c
-      register: ubuntu_check
-      changed_when: false
-      failed_when: false
-      when: 
-        - ansible_distribution is defined
-        - ansible_distribution == 'Ubuntu'
-
-    - name: Determine Upgrade Status
-      ansible.builtin.set_fact:
-        upgrade_needed: >-
-          {%- if ansible_distribution is not defined -%}
-            unknown
-          {%- elif ansible_distribution == 'Ubuntu' -%}
-            {%- if ubuntu_check.stdout is defined and 'New release' in ubuntu_check.stdout -%}
-              true
-            {%- else -%}
-              false
-            {%- endif -%}
-          {%- elif ansible_distribution == 'Debian' -%}
-            {%- if ansible_distribution_major_version | int < 12 -%}
-              true
-            {%- else -%}
-              false
-            {%- endif -%}
-          {%- else -%}
-            false
-          {%- endif -%}
-        current_version_str: "{{ ansible_distribution }} {{ ansible_distribution_version }}"
-        target_version_str: >-
-          {%- if ansible_distribution is not defined -%}
-            Unknown
-          {%- elif ansible_distribution == 'Ubuntu' -%}
-            {%- if ubuntu_check.stdout is defined and 'New release' in ubuntu_check.stdout -%}
-              {{ (ubuntu_check.stdout | regex_findall("New release '([^']+)'") | first) | default('Unknown') }}
-            {%- else -%}
-              -
-            {%- endif -%}
-          {%- elif ansible_distribution == 'Debian' -%}
-            {%- if ansible_distribution_major_version | int < 12 -%}
-              Debian 12 (Bookworm)
-            {%- else -%}
-              -
-            {%- endif -%}
-          {%- else -%}
-            -
-          {%- endif -%}
-      when: ansible_distribution is defined
-
-    - name: Group hosts by status
-      ansible.builtin.group_by:
-        key: "upgrade_{{ upgrade_needed | trim }}"
-      when: upgrade_needed is defined
-
-- name: Generate Report
-  hosts: localhost
-  gather_facts: false
-  tasks:
-    - name: Display Upgrade Candidates
-      ansible.builtin.debug:
-        msg: "{{ report.splitlines() }}"
-      vars:
-        report: |
-          
-          DISTRIBUTION UPGRADE REPORT
-          ===========================
-
-          REQUIRES UPGRADE
-          ================
-          {{ '%-30s' | format('HOST') }}   {{ '%-20s' | format('CURRENT') }}   {{ 'TARGET' }}
-          {{ '------------------------------' }}   {{ '--------------------' }}   {{ '--------------------' }}
-          {% for host in groups['upgrade_true'] | default([]) %}
-          {{ '%-30s' | format(host) }}   {{ '%-20s' | format(hostvars[host]['current_version_str']) }}   {{ hostvars[host]['target_version_str'] }}
-          {% endfor %}
-          {% if groups['upgrade_true'] is not defined or groups['upgrade_true'] | length == 0 %}
-          (No hosts require upgrade)
-          {% endif %}
-
-          UP TO DATE
-          ==========
-          {{ '%-30s' | format('HOST') }}   {{ 'CURRENT' }}
-          {{ '------------------------------' }}   {{ '--------------------' }}
-          {% for host in groups['upgrade_false'] | default([]) %}
-          {{ '%-30s' | format(host) }}   {{ hostvars[host]['current_version_str'] }}
-          {% endfor %}
-

ansible/plays/check-template.yml

@@ -1,8 +0,0 @@
-- name: Ensure debian-12 template exists on saturn
-  hosts: saturn
-  become: true
-  tasks:
-    - name: Download debian-12 template
-      command: pveam download local debian-12-standard_12.12-1_amd64.tar.zst
-      args:
-        creates: /var/lib/vz/template/cache/debian-12-standard_12.12-1_amd64.tar.zst

ansible/plays/common.yml

@@ -1,22 +1,16 @@
 ---
 - name: Install Custom CA Certificates
-  hosts: all:!proxmox_noansible
+  hosts: all
   become: true
   roles:
     - certificates
 
 - name: Base System Hardening & Common Configuration
-  hosts: all:!pve:!pbs:!proxmox_noansible:!proxmox_nodes:!pve_cluster
+  hosts: all
   become: true
   ignore_unreachable: true
   gather_facts: true
 
-  vars:
-    # Monitoring agent endpoints
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-
   roles:
     - common
     - qemu-guest-agent
-    - monitoring-agent

ansible/plays/configure-audiobookshelf.yml

@@ -1,44 +0,0 @@
----
-# Configure Audiobookshelf permissions
-# Fixes upload issues by ensuring audiobookshelf user can write to storage directories
-
-- name: Configure Audiobookshelf
-  hosts: audiobookshelf.srv.renner.uno
-  become: true
-
-  tasks:
-    - name: Ensure audiobookshelf user exists
-      ansible.builtin.user:
-        name: audiobookshelf
-        system: true
-        create_home: false
-        state: present
-
-    - name: Set ownership on audiobooks directory
-      ansible.builtin.file:
-        path: /audiobooks
-        owner: audiobookshelf
-        group: audiobookshelf
-        mode: "0755"
-        state: directory
-        recurse: false
-
-    - name: Set ownership on audiobooks content recursively
-      ansible.builtin.command:
-        cmd: chown -R audiobookshelf:audiobookshelf /audiobooks
-      changed_when: true
-
-    - name: Set ownership on metadata directory
-      ansible.builtin.file:
-        path: /metadata
-        owner: audiobookshelf
-        group: audiobookshelf
-        mode: "0755"
-        state: directory
-        recurse: true
-
-    - name: Restart audiobookshelf service
-      ansible.builtin.systemd:
-        name: audiobookshelf
-        state: restarted
-        enabled: true

ansible/plays/configure-grafana-authentik.yml

@@ -1,40 +0,0 @@
----
-# Configure Grafana to use Authentik for authentication
-#
-# Prerequisites:
-# 1. Create OAuth2/OpenID application in Authentik:
-#    - Name: Grafana
-#    - Slug: grafana
-#    - Client type: Confidential
-#    - Redirect URIs: https://dash.renner.uno/login/generic_oauth
-#    - Signing Key: authentik Self-signed Certificate
-#
-# 2. Add to vault (ansible/inventory/group_vars/all/vault.yml):
-#    vault_grafana_authentik_client_id: "<client-id-from-authentik>"
-#    vault_grafana_authentik_client_secret: "<client-secret-from-authentik>"
-#
-# 3. (Optional) Create groups in Authentik:
-#    - "Grafana Admins" - gets Admin role
-#    - "Grafana Editors" - gets Editor role
-#    - Others get Viewer role
-
-- name: Configure Grafana Authentik Authentication
-  hosts: grafana.srv.renner.uno
-  become: yes
-  roles:
-    - grafana-authentik
-
-  post_tasks:
-    - name: Display configuration status
-      debug:
-        msg: |
-          Grafana Authentik OAuth configured!
-
-          Users can now login at: {{ grafana_root_url }}
-
-          Role mapping:
-          - Members of "Grafana Admins" -> Admin
-          - Members of "Grafana Editors" -> Editor
-          - All others -> Viewer
-
-          If auto_login is enabled, users will be redirected to Authentik automatically.

ansible/plays/configure-pve-k8s-zfs-tuning.yml

@@ -1,68 +0,0 @@
----
-# Set sync=disabled on the zvols backing the K8s control-plane VM root disks
-# (k8s-1/2/3, VMIDs 150/151/152) to eliminate ZFS fsync amplification on the
-# shared local-zfs pool. Trades a small durability window for major fsync
-# latency reduction on etcd's WAL writes:
-#
-#   - etcd's WAL fsync returns immediately; ZFS still flushes via txgs
-#     (default ~5s)
-#   - Worst case on hard power loss: up to ~5s of writes lost per CP member
-#   - With 3-member quorum and Velero backups, this is acceptable for homelab
-#   - The whole zvol is sync=disabled (Talos OS, containerd, kubelet
-#     ephemeral, etcd WAL all share it), but etcd is the dominant fsync user;
-#     other writers tolerate small loss windows
-#
-# Each PVE host owns exactly one CP zvol (jupiter→150, saturn→151, neptune→152).
-# Auto-discovery handles whatever the host has — no need to update if a CP
-# VM ever moves.
-#
-# Idempotent: skips zvols that already have sync=disabled.
-#
-# Run manually against freshly-created CP zvols so they get the property
-# before Talos starts writing (previously wired into the now-removed
-# rebuild-k8s pipeline between `recreate-vms` and `talos-bootstrap`).
-
-- name: Set sync=disabled on K8s CP VM zvols
-  hosts: pve_cluster
-  become: yes
-  gather_facts: no
-
-  tasks:
-    - name: Discover CP-VM zvols on this host
-      shell: |
-        set -o pipefail
-        zfs list -H -t volume -o name | grep -E "/vm-(150|151|152)-disk-[0-9]+$" || true
-      args:
-        executable: /bin/bash
-      register: cp_zvols
-      changed_when: false
-
-    - name: Read current sync property
-      shell: "zfs get -H -o value sync {{ item }}"
-      register: current_sync
-      changed_when: false
-      loop: "{{ cp_zvols.stdout_lines }}"
-      when: cp_zvols.stdout_lines | length > 0
-
-    - name: Set sync=disabled where not already
-      shell: "zfs set sync=disabled {{ item.item }}"
-      when:
-        - cp_zvols.stdout_lines | length > 0
-        - item.stdout | trim != "disabled"
-      loop: "{{ current_sync.results | default([]) }}"
-      loop_control:
-        label: "{{ item.item }}"
-
-    - name: Verify sync=disabled
-      shell: "zfs get -H -o value sync {{ item }}"
-      register: verify_sync
-      changed_when: false
-      failed_when: verify_sync.stdout | trim != "disabled"
-      loop: "{{ cp_zvols.stdout_lines }}"
-      when: cp_zvols.stdout_lines | length > 0
-
-    - name: Summary
-      debug:
-        msg: >-
-          {{ inventory_hostname }}: tuned {{ cp_zvols.stdout_lines | length }}
-          zvol(s) [{{ cp_zvols.stdout_lines | join(', ') }}]

ansible/plays/configure-pve-node-network.yml

@@ -1,53 +0,0 @@
----
-# Configure Proxmox VE Node Host Networking
-#
-# Fixes two defects found by live inspection on 2026-07-19:
-#
-#   1. All three nodes had no IPv6 address on any external interface. FRR
-#      enables ipv6 forwarding for the OSPFv3 corosync mesh, and with the
-#      kernel default accept_ra=1 the nodes silently discarded the Router
-#      Advertisements the gateway sends on VLAN 40. Fixed with accept_ra=2.
-#
-#   2. saturn's vmbr0 sits at MTU 1500 while its own uplink and both peer
-#      nodes are at 9000, so large frames between saturn's guests and the
-#      other nodes' guests blackhole instead of fragmenting.
-#
-# The sysctl fix is non-disruptive and applies on every run. The MTU fix is
-# staged as a pending PVE network change and only applied when explicitly
-# requested, because applying it reloads networking on the node and briefly
-# interrupts the Talos k8s VM and its Ceph OSD.
-#
-# Safe (converges IPv6, stages the MTU change without applying it):
-#   ansible-playbook -i inventory/ plays/configure-pve-node-network.yml
-#
-# Applying the staged MTU change, one node at a time:
-#   ansible-playbook -i inventory/ plays/configure-pve-node-network.yml \
-#     --limit saturn -e pve_node_network_apply_pending=true
-#
-# See wiki: pve-ipv6-accept-ra-forwarding-defect,
-#           pve-saturn-vmbr0-mtu-asymmetry
-
-- name: Configure PVE node host networking
-  hosts: pve_cluster
-  become: yes
-  serial: 1
-
-  roles:
-    - role: pve-node-network
-
-  post_tasks:
-    - name: Summarise node IPv6 state
-      ansible.builtin.command: ip -6 addr show dev vmbr1.40 scope global
-      register: pve_node_network_v6_summary
-      changed_when: false
-      failed_when: false
-
-    - name: Display node IPv6 state
-      ansible.builtin.debug:
-        msg: >-
-          {{ inventory_hostname }}:
-          {{ pve_node_network_v6_summary.stdout_lines
-             | select('search', 'inet6')
-             | map('trim')
-             | list
-             | default(['no global IPv6'], true) }}

ansible/plays/install-authentik-ha.yml

@@ -1,20 +0,0 @@
----
-- name: Setup PostgreSQL HA Cluster
-  hosts: authentik_db
-  become: true
-  vars:
-    # Set this in inventory or pass as extra var.
-    # For initial setup, we define authentik-db-1 as primary.
-    postgres_primary_host: "authentik-db-1.srv.renner.uno"
-    
-  roles:
-    - common
-    - postgres-simple-ha
-
-- name: Setup Authentik Stateless App
-  hosts: authentik_app
-  become: true
-  roles:
-    - common
-    - docker-install
-    - authentik-stateless

ansible/plays/install-blackbox-exporter.yml

@@ -1,19 +0,0 @@
----
-# Deploy Blackbox Exporter to Prometheus server for endpoint probing
-# Usage: ansible-playbook -i inventory/ plays/install-blackbox-exporter.yml
-
-- name: Install Blackbox Exporter
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  roles:
-    - blackbox-exporter
-
-  post_tasks:
-    - name: Verify blackbox_exporter is running
-      uri:
-        url: "http://127.0.0.1:9115/metrics"
-        status_code: 200
-      register: exporter_health
-      retries: 3
-      delay: 5
-      until: exporter_health.status == 200

ansible/plays/install-claude-workstation.yml

@@ -10,7 +10,7 @@
 - name: Deploy the Claude Code workstation
   hosts: claude_workstation
   become: true
-  # qemu-guest-agent + monitoring-agent (Alloy -> Loki/Prometheus) are applied
+  # qemu-guest-agent is applied
   # to every inventory host by the umbrella plays/common.yml, same as for the
   # netbird-proxy host — run that against this host too (see RUNBOOK Step 5).
   roles:

ansible/plays/install-edge-host.yml

@@ -1,13 +1,42 @@
 ---
-# Bare-metal Arch Linux edge host (Dell Wyse 3040, internal eMMC) — replaces
-# the k8s-edge Talos VM
-# (opentofu vm_153) as the home for hardware-bound services:
-#   - zwave-js-ui  (ZWA-2 Z-Wave USB radio, docker compose)
-#   - sml2mqtt     (FTDI power-meter reader, docker compose)
-#   - open3e       (Innomaker USB2CAN -> Vitocal/VX3, native venv + systemd)
-# Base install procedure and cutover checklist: ansible/roles/edge-host/README.md
-- name: Install Arch edge host
+# Bare-metal Arch edge hosts (Dell Wyse 3040 and similar) — the home for
+# hardware-bound services, because Talos supports neither SocketCAN nor USB
+# passthrough.
+#
+# This play is the BASE only: what every edge thin client gets regardless of
+# what is plugged into it. The peripherals live in their own playbooks against
+# their own inventory groups:
+#
+#   plays/install-zwave-gateway.yml   [zwave_gateway]   ZWA-2 radio
+#   plays/install-heating-can.yml     [heating_can]     Innomaker USB2CAN
+#   plays/install-power-meter.yml     [power_meter]     FTDI meter head
+#
+# Splitting the roles without splitting the plays would have kept the coupling:
+# one play applying every role to one group is the monolith again, a layer up.
+# A host with no peripherals is simply absent from those groups -- no flags, no
+# `when:` on the roles.
+#
+# Self-contained: `certificates` and `common` are the parts of plays/common.yml
+# that apply to an edge host, carried here so bootstrapping one is a single
+# unrestricted run over the group. Not carried over: qemu-guest-agent, since
+# these are bare metal rather than VMs. Log shipping is edge-log-shipper's job
+# and lives below — it pushes to the authenticated cluster ingress with a
+# per-host credential, which is the only Loki these hosts can reach.
+#
+# Order is dependency order: edge-flash-wear before docker, so the journal is
+# volatile before containers start logging into it.
+- name: Install Arch edge host (base)
   hosts: edge_host
+  # Connection is already root (ansible_user=root), so this is a root-to-root
+  # no-op today. Kept explicit, matching plays/common.yml, so the roles below
+  # do not silently depend on the connection user staying privileged.
+  become: true
+
   roles:
+    - certificates
+    - common
+    - edge-flash-wear
     - docker-install
-    - edge-host
+    - edge-log-shipper
+    - mqtt-bridge
+    - edge-vlan

ansible/plays/install-gatus.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure Gatus
-  hosts: proxmox_gatus
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: gatus

ansible/plays/install-grist.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure Grist
-  hosts: grist
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: grist

ansible/plays/install-heating-can.yml

@@ -0,0 +1,18 @@
+---
+# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
+# USB2CAN adapter attached.
+#
+# Targets [heating_can]: membership means the adapter is physically present.
+# Two roles because they are two concerns -- the interface is host network
+# state, the services consume it -- but one group, because both need the same
+# hardware.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: these
+# services publish to the local MQTT bridge it sets up.
+- name: Install heating CAN services
+  hosts: heating_can
+  become: true
+
+  roles:
+    - socketcan
+    - heating-can

ansible/plays/install-ical-proxy.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure ical-proxy
-  hosts: proxmox_ical_proxy
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: ical-proxy

ansible/plays/install-log-aggregation.yml

@@ -1,86 +0,0 @@
----
-# Setup Log Aggregation Infrastructure
-# This playbook configures the complete logging stack:
-# 1. Loki server for log storage and querying
-# 2. Alloy agents on all VMs/CTs for log collection
-# 3. Syslog receiver for remote devices (Synology NAS, network devices)
-#
-# Usage: ansible-playbook -i inventory/ plays/setup-log-aggregation.yml
-#
-# To limit to specific hosts:
-#   --limit loki.srv.renner.uno          # Only configure Loki server
-#   --limit proxmox_all_running          # Only configure log agents
-
-- name: Setup Loki Log Aggregation Server
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
-
-- name: Deploy Monitoring Agent to Loki Server (with syslog receiver)
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-    # Syslog receiver settings are defined in host_vars/loki.srv.renner.uno/vars.yml:
-    # - alloy_syslog_receiver_enabled: true
-    # - alloy_syslog_receiver_port: 1514
-    # - alloy_syslog_receiver_protocol: "udp"
-    # - alloy_syslog_receiver_format: "raw"
-    # - alloy_stability_level: "experimental"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Display syslog receiver info
-      debug:
-        msg: |
-          Syslog receiver is listening on UDP port {{ alloy_syslog_receiver_port | default(1514) }}
-          Configure remote devices to send syslog to: {{ inventory_hostname }}:{{ alloy_syslog_receiver_port | default(1514) }}
-      when: alloy_syslog_receiver_enabled | default(false)
-
-- name: Deploy Monitoring Agent to All Running VMs/CTs
-  hosts: proxmox_all_running:!proxmox_logging:!proxmox_noansible
-  become: yes
-  gather_facts: yes
-  serial: 10  # Deploy to 10 hosts at a time to avoid overwhelming the network
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Verify Alloy is running
-      systemd:
-        name: alloy
-        state: started
-      register: alloy_service
-
-    - name: Display agent status
-      debug:
-        msg: "Monitoring agent deployed successfully on {{ inventory_hostname }}"
-      when: alloy_service is defined

ansible/plays/install-loki.yml

@@ -1,26 +0,0 @@
----
-# Install Loki Log Aggregation Server
-# Usage: ansible-playbook -i inventory/ plays/install-loki.yml
-
-- name: Install Loki Log Aggregation Server
-  hosts: loki.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"

ansible/plays/install-mkdocs-wiki.yml

@@ -1,19 +0,0 @@
----
-# Deploy MkDocs wiki from Obsidian vault
-#
-# Usage:
-#   ansible-playbook -i inventory plays/install-mkdocs-wiki.yml
-#
-- name: Deploy MkDocs Wiki
-  hosts: proxmox_wiki
-  become: true
-
-  pre_tasks:
-    - name: Ensure wiki group exists
-      ansible.builtin.group:
-        name: wiki
-        system: true
-        state: present
-
-  roles:
-    - role: mkdocs-wiki

ansible/plays/install-music-assistant.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure Music Assistant
-  hosts: proxmox_music_assistant
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: music-assistant

ansible/plays/install-n8n.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure n8n
-  hosts: n8n
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: n8n

ansible/plays/install-netbird-monitoring.yml

@@ -1,38 +0,0 @@
----
-# Deploy monitoring agent to NetBird server (external datacenter)
-# Uses NetBird mesh IPs to reach Loki and Prometheus
-# Usage: ansible-playbook -i inventory/ plays/setup-netbird-monitoring.yml
-
-- name: Deploy Monitoring Agent to NetBird Server
-  hosts: netbird_server
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Use ARM64 binary for Hetzner ARM server
-    alloy_arch: "linux-arm64"
-
-    # Monitoring endpoints via NetBird mesh
-    loki_endpoint: "http://loki.nb.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.nb.renner.uno:9090/api/v1/write"
-
-    # NetBird server labels
-    alloy_extra_labels:
-      environment: "production"
-      datacenter: "hetzner"
-      type: "netbird-server"
-      service: "netbird"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Verify Alloy agent is running
-      systemd:
-        name: alloy
-        state: started
-      register: alloy_status
-
-    - name: Display agent status
-      debug:
-        msg: "Monitoring agent deployed to {{ inventory_hostname }} via NetBird mesh"

ansible/plays/install-node-exporter-pve.yml

@@ -1,41 +0,0 @@
----
-# Install Node Exporter and Chrony Exporter on Proxmox VE hosts
-# Usage: ansible-playbook -i inventory/ plays/install-node-exporter-pve.yml
-
-- name: Install Exporters on PVE Cluster
-  hosts: pve_cluster
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - node-exporter
-    - chrony-exporter
-
-  post_tasks:
-    - name: Verify node-exporter is running
-      uri:
-        url: "http://127.0.0.1:9100/metrics"
-        status_code: 200
-      register: node_exporter_health
-      retries: 3
-      delay: 5
-      until: node_exporter_health.status == 200
-
-    - name: Verify chrony-exporter is running
-      uri:
-        url: "http://127.0.0.1:9123/metrics"
-        status_code: 200
-      register: chrony_exporter_health
-      retries: 3
-      delay: 5
-      until: chrony_exporter_health.status == 200
-
-    - name: Verify chrony metrics include stratum
-      uri:
-        url: "http://127.0.0.1:9123/metrics"
-        return_content: yes
-      register: chrony_metrics
-
-    - name: Check chrony exporter output
-      debug:
-        msg: "Chrony stratum metrics available: {{ 'chrony_tracking_stratum' in chrony_metrics.content }}"

ansible/plays/install-node-exporter.yml

@@ -3,7 +3,7 @@
 # Usage: ansible-playbook -i inventory/ plays/install-node-exporter.yml
 
 - name: Install Node Exporter
-  hosts: all:!pve_cluster
+  hosts: all
   become: yes
   gather_facts: yes
 

ansible/plays/install-ntp-ha.yml

@@ -22,12 +22,54 @@
 # so it stays BACKUP while the PVE nodes exist and wins the VIP by default once
 # they are gone -- no config change required at cutover.
 
-- name: Install NTP Server (HA)
+# Phase 1. Every static assignment of the IPv6 service address must be gone
+# fleet-wide before ANY host claims it via VRRP.
+#
+# DAD is enabled on the VRRP address (deliberately -- under single-master VRRP a
+# duplicate is a real fault worth detecting). So a static copy on one host and a
+# VRRP-managed copy on another cannot coexist: the kernel marks the newcomer
+# `dadfailed deprecated`, and its master then believes it is serving while
+# carrying an unusable address. That is not hypothetical -- it is what edge-1
+# did on 2026-08-07 when it took the address while three PVE nodes still held it
+# statically.
+#
+# Deliberately NOT serial: this play must complete everywhere before phase 2
+# starts anywhere. Doing the withdrawal per host inside phase 2 cannot avoid the
+# overlap, because with serial: 1 the first host claims the address while the
+# rest are still static.
+#
+# There is a gap between the phases where no host holds the address. It is
+# bounded by one play's run time and clients simply retry.
+- name: Withdraw static assignments of the NTP service address
   hosts: ntp_cluster
   become: true
 
+  vars:
+    keepalived_ntp_vip6: "{{ ntp_vip6 }}"
+    keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
+
+  tasks:
+    - name: Withdraw any static assignment of the service address
+      ansible.builtin.include_role:
+        name: keepalived-ntp
+        tasks_from: withdraw-static.yml
+      when: ntp_keepalived_enabled | default(false)
+
+# Phase 2. Configure chrony and hand the address to keepalived.
+- name: Install NTP Server (HA)
+  hosts: ntp_cluster
+  become: true
+  # One node at a time. The keepalived role withdraws the statically assigned
+  # IPv6 service address and restarts keepalived so the address is re-taken
+  # under VRRP control; doing that across every NTP server at once would leave
+  # the whole fleet with nobody holding fd40::123 until the elections settle.
+  # Serialised, the remaining members keep answering throughout.
+  serial: 1
+
   vars:
     ntp_vip: "10.2.40.4"
+    # ntp_vip6 / ntp_vip6_linklocal come from group_vars/ntp_cluster.yml, so
+    # both plays in this file see one definition.
     ntp_servers:
       jupiter:
         priority: 100
@@ -49,6 +91,8 @@
       when: ntp_keepalived_enabled | default(false)
       vars:
         keepalived_ntp_vip: "{{ ntp_vip }}"
+        keepalived_ntp_vip6: "{{ ntp_vip6 }}"
+        keepalived_ntp_vip6_linklocal: "{{ ntp_vip6_linklocal }}"
         keepalived_ntp_state: "{{ ntp_servers[inventory_hostname].state }}"
         keepalived_ntp_priority: "{{ ntp_servers[inventory_hostname].priority }}"
         keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
@@ -90,6 +134,49 @@
           {% endif %}
       when: ntp_keepalived_enabled | default(false)
 
+    # The v6 address is the path clients actually use, so it gets the same
+    # check as the v4 VIP. Without this a broken v6 rollout -- the address
+    # never claimed, or -nodad not behaving as expected on a given kernel --
+    # would not surface here at all, only later by reading
+    # /var/log/keepalived-state.log by hand.
+    #
+    # `ip -6 addr show ... to <addr>` prints the address when present and
+    # nothing when absent, so emptiness is the test.
+    #
+    # Presence cannot itself distinguish a VRRP-held address from a static
+    # leftover -- they look identical. It is only meaningful because the
+    # keepalived-ntp role asserts, on both distros, that no static assignment
+    # survives: the play fails before reaching here if one does.
+    - name: Check if this server holds the IPv6 service address
+      ansible.builtin.command:
+        argv:
+          - ip
+          - -6
+          - addr
+          - show
+          - dev
+          - "{{ ntp_keepalived_interface }}"
+          - to
+          - "{{ ntp_vip6 }}"
+      register: vip6_check
+      failed_when: false
+      changed_when: false
+      when: ntp_keepalived_enabled | default(false)
+
+    - name: Display IPv6 service address status
+      ansible.builtin.debug:
+        msg: >
+          {% if vip6_check.stdout | default('') | length > 0 %}
+          This server ({{ inventory_hostname }}) holds the IPv6 service address
+          {{ ntp_vip6 }} -- it is the v6 MASTER and is serving ntp.srv.renner.uno
+          {% else %}
+          This server ({{ inventory_hostname }}) does not hold {{ ntp_vip6 }}.
+          Expected on a v6 BACKUP. If NO host holds it, check_ntp is failing
+          everywhere -- inspect `chronyc tracking` and
+          /var/log/keepalived-ntp-health.log
+          {% endif %}
+      when: ntp_keepalived_enabled | default(false)
+
     - name: Report hosts serving NTP without participating in the VIP
       ansible.builtin.debug:
         msg: >

ansible/plays/install-owncloud.yml

@@ -1,8 +0,0 @@
----
-- name: Install and Configure ownCloud Infinite Scale
-  hosts: proxmox_owncloud
-  become: true
-  roles:
-    - role: common
-    - role: docker-install
-    - role: owncloud

ansible/plays/install-postgres-monitoring.yml

@@ -1,62 +0,0 @@
----
-# Setup PostgreSQL monitoring user and exporter
-# Usage: ansible-playbook -i inventory/ plays/setup-postgres-monitoring.yml
-
-- name: Create PostgreSQL monitoring user on primary
-  hosts: authentik-db-1.srv.renner.uno
-  become: yes
-  vars_files:
-    - ../inventory/group_vars/all/vault.yml
-  tasks:
-    - name: Install psycopg2 for Ansible PostgreSQL modules
-      apt:
-        name: python3-psycopg2
-        state: present
-
-    - name: Create postgres_exporter user
-      become_user: postgres
-      community.postgresql.postgresql_user:
-        name: postgres_exporter
-        password: "{{ vault_postgres_exporter_password }}"
-        role_attr_flags: "NOSUPERUSER,NOCREATEDB,NOCREATEROLE"
-        state: present
-
-    - name: Grant pg_monitor role to exporter user
-      become_user: postgres
-      community.postgresql.postgresql_privs:
-        database: postgres
-        type: group
-        objs: pg_monitor
-        roles: postgres_exporter
-        grant_option: no
-
-    - name: Grant connect on authentik database
-      become_user: postgres
-      community.postgresql.postgresql_privs:
-        database: authentik
-        type: database
-        privs: CONNECT
-        roles: postgres_exporter
-
-- name: Deploy postgres-exporter to Prometheus server
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  vars_files:
-    - ../inventory/group_vars/all/vault.yml
-  vars:
-    postgres_exporter_db_host: "10.2.40.135"  # PostgreSQL HA VIP
-    postgres_exporter_db_user: "postgres_exporter"
-    postgres_exporter_db_password: "{{ vault_postgres_exporter_password }}"
-    postgres_exporter_db_name: "postgres"
-  roles:
-    - postgres-exporter
-
-  post_tasks:
-    - name: Verify postgres-exporter is running
-      uri:
-        url: "http://127.0.0.1:9187/metrics"
-        status_code: 200
-      register: exporter_health
-      retries: 3
-      delay: 5
-      until: exporter_health.status == 200

ansible/plays/install-power-meter.yml

@@ -0,0 +1,14 @@
+---
+# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
+# attached.
+#
+# Targets [power_meter]: membership means the meter head is physically present.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker and publishes to the local MQTT bridge.
+- name: Install power-meter reader
+  hosts: power_meter
+  become: true
+
+  roles:
+    - power-meter-sml

ansible/plays/install-prometheus.yml

@@ -1,43 +0,0 @@
----
-# Install Prometheus Metrics Server
-# Usage: ansible-playbook -i inventory/ plays/install-prometheus.yml
-
-- name: Install Prometheus Metrics Server
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Synology NAS SNMP targets
-    snmp_synology_targets:
-      - name: nas-1
-        target: "10.2.40.50"
-      - name: nas-2
-        target: "10.3.40.57"
-
-    # SNMPv3 credentials from vault
-    snmp_use_v3: true
-    snmp_v3_username: "{{ vault_snmp_v3_username }}"
-    snmp_v3_auth_password: "{{ vault_snmp_v3_auth_password }}"
-    snmp_v3_priv_password: "{{ vault_snmp_v3_priv_password }}"
-
-  roles:
-    - common
-    - prometheus-server
-    - snmp-exporter
-    - pve-exporter
-    - unifi-poller
-
-  post_tasks:
-    - name: Verify Prometheus is running
-      uri:
-        url: "http://127.0.0.1:9090/-/ready"
-        status_code: 200
-      register: prometheus_health
-      retries: 5
-      delay: 10
-      until: prometheus_health.status == 200
-
-    - name: Display Prometheus status
-      debug:
-        msg: "Prometheus is running and healthy at http://{{ inventory_hostname }}:9090"

ansible/plays/install-pve-monitoring.yml

@@ -1,33 +0,0 @@
----
-# Deploy monitoring agents to Proxmox VE nodes
-# Usage: ansible-playbook -i inventory/ plays/setup-pve-monitoring.yml
-
-- name: Deploy Monitoring Agents to Proxmox Nodes
-  hosts: pve_cluster
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Monitoring endpoints
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-
-    # PVE-specific labels
-    alloy_extra_labels:
-      environment: "production"
-      datacenter: "homelab"
-      type: "hypervisor"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Verify Alloy agent is running
-      systemd:
-        name: alloy
-        state: started
-      register: alloy_status
-
-    - name: Display agent status
-      debug:
-        msg: "Monitoring agent deployed to {{ inventory_hostname }}"

ansible/plays/install-vault.yml

@@ -1,7 +0,0 @@
----
-- name: Install HashiCorp Vault
-  hosts: vault.srv.renner.uno
-  become: true
-  roles:
-    - common
-    - vault

ansible/plays/install-zwave-gateway.yml

@@ -0,0 +1,15 @@
+---
+# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
+#
+# Targets [zwave_gateway]: membership means the radio is physically present.
+# Moving the stick to another edge host is a one-line inventory change.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker, and the flash-wear handling that the libeatmydata preload exists to
+# complement.
+- name: Install Z-Wave gateway
+  hosts: zwave_gateway
+  become: true
+
+  roles:
+    - zwave-gateway

ansible/plays/rotate-netbird-secrets.yml

@@ -0,0 +1,250 @@
+---
+# Rotate the NetBird shared secrets that Ansible owns BOTH ends of.
+#
+# Deliberately opt-in. Rotation is not idempotent and it interrupts the mesh
+# briefly, so this play refuses to run unless invoked explicitly:
+#
+#   ansible-playbook plays/rotate-netbird-secrets.yml -e rotate_confirm=yes
+#
+# ONE-TIME MIGRATION REQUIRED BEFORE FIRST USE. The three keys below currently
+# live in group_vars/all/vault.yml. They must be deleted from there, because
+# group_vars files merge alphabetically and vault.yml sorts after
+# netbird-rotatable.yml — leaving them would let a later ordinary deploy
+# overwrite a freshly rotated secret with the old one. The play checks for this
+# and refuses to run until it is done:
+#
+#   ansible-vault edit inventory/group_vars/all/vault.yml
+#   # delete vault_netbird_turn_secret, vault_netbird_relay_secret,
+#   #        and netbird_turn_secret
+#
+# That last one is easy to miss and breaks the rotation silently. vault.yml
+# carried BOTH vault_netbird_turn_secret and a bare netbird_turn_secret. The
+# role default is `netbird_turn_secret: "{{ vault_netbird_turn_secret }}"`, but
+# a group var outranks a role default — so the bare key wins, and rotating only
+# the vault_-prefixed one changes nothing that reaches the template. The check
+# below matches both spellings for that reason.
+#
+# The first rotation then populates netbird-rotatable.yml with fresh values, so
+# nothing needs copying across.
+#
+# WHAT IS ROTATED, and why each is safe to rotate from here:
+#
+#   vault_netbird_turn_secret   coturn's `user=netbird:<secret>` in
+#                               turnserver.conf and the management server's
+#                               Turn.Secret/Password in management.json. Both
+#                               templates, one role, one host — the two ends
+#                               cannot disagree across a run.
+#   vault_netbird_relay_secret  management.json on the server and relay.env on
+#                               the proxies. Two hosts, both managed here.
+#
+# The test for "safe to rotate here" is whether the value appears in
+# management.json.j2. If the management server does not read it from a file
+# Ansible writes, Ansible does not own both ends, whatever else it templates.
+#
+# WHAT IS NOT, and must not be added:
+#
+#   vault_netbird_proxy_token    LOOKS symmetric — it is in proxy.env on both
+#                                the server and the proxies, all Ansible
+#                                templates — but the management server does not
+#                                validate it from any of them. It is registered
+#                                in the management DATABASE. Rotating it changes
+#                                only what the proxies present, so every proxy
+#                                is locked out: "proxy auth failed: too many
+#                                failed authentication attempts", and the
+#                                proxies stop receiving their domain
+#                                configuration. Learned by doing it.
+#
+#   vault_netbird_datastore_key  encrypts NetBird's datastore at rest. Changing
+#                                it without re-encrypting the store makes the
+#                                existing data unreadable. Rotating this needs a
+#                                dump/restore, not a template rewrite.
+#   vault_netbird_auth_client_secret  the other half lives in Authentik. Ansible
+#                                only writes the consumer side, so rotating here
+#                                breaks login until Authentik is changed too.
+#
+# These are Ansible's OWN credentials, not a copy of the cluster's. OpenBao
+# carries credentials for some of the same systems (see
+# kubernetes/infrastructure/openbao/rotation/), and those are deliberately DIFFERENT
+# values: one consumer, one credential, independently revocable. Rotating here
+# must not touch OpenBao and does not invalidate anything the cluster holds —
+# that is the point of them being distinct, not drift to be reconciled.
+#
+# Corollary: if a credential ever has to be shared between the two, that is the
+# bug. Mint a second one instead.
+#
+# There IS a brief interruption. The server is reconfigured before the proxies,
+# so relay/proxy auth fails for the seconds between the two plays. Clients
+# reconnect; existing peer-to-peer tunnels are not torn down.
+
+- name: Generate and store new NetBird shared secrets
+  hosts: localhost
+  connection: local
+  gather_facts: false
+
+  vars:
+    rotatable_vault: "{{ playbook_dir }}/../inventory/group_vars/all/netbird-rotatable.yml"
+
+  pre_tasks:
+    - name: Require explicit confirmation
+      ansible.builtin.assert:
+        that:
+          - rotate_confirm | default('') == 'yes'
+        fail_msg: >-
+          Refusing to rotate without -e rotate_confirm=yes. This changes live
+          credentials and briefly interrupts the NetBird mesh.
+        success_msg: Confirmed; rotating NetBird shared secrets.
+
+    - name: Verify the vault password is available before touching anything
+      ansible.builtin.command:
+        cmd: "{{ playbook_dir }}/../vault-password.sh"
+      register: _vault_pw_check
+      changed_when: false
+      no_log: true
+
+    # Files inside group_vars/<group>/ are merged ALPHABETICALLY and the last
+    # one wins. netbird-rotatable.yml sorts before vault.yml, so if vault.yml
+    # still defines these keys it silently overrides everything this play
+    # writes. The rotation itself would still work — the plays below pass the
+    # new values explicitly, and play vars outrank group vars — but the NEXT
+    # ordinary run of install-netbird-server.yml or install-netbird-proxy.yml
+    # would read the stale value from vault.yml and push it back out, breaking
+    # server<->proxy auth. The deploy pipelines run unattended, so that would
+    # land on an unrelated change to those roles.
+    #
+    # Counting matches rather than printing them: the value never reaches
+    # stdout, only the key name does.
+    # Proving the vault is readable FIRST. Without this, a failed decrypt (wrong
+    # password, moved file) sends empty input to grep, which then reports zero
+    # matches — indistinguishable from a clean vault, so the guard below would
+    # wave the rotation through precisely when it knows least.
+    - name: Prove vault.yml is readable before trusting the key check
+      ansible.builtin.shell:
+        cmd: >-
+          set -o pipefail;
+          ansible-vault view
+          --vault-password-file {{ playbook_dir }}/../vault-password.sh
+          {{ playbook_dir }}/../inventory/group_vars/all/vault.yml
+          | wc -l
+        executable: /bin/bash
+      register: _vault_lines
+      changed_when: false
+
+    - name: Check vault.yml does not still define the rotated keys
+      ansible.builtin.shell:
+        cmd: >-
+          ansible-vault view
+          --vault-password-file {{ playbook_dir }}/../vault-password.sh
+          {{ playbook_dir }}/../inventory/group_vars/all/vault.yml
+          | grep -cE '^(vault_)?netbird_(turn_secret|relay_secret):'
+      register: _stale_keys
+      changed_when: false
+      # grep exits 1 on "no matches", which is the good case here.
+      failed_when: false
+
+    - name: Refuse to rotate while vault.yml still owns these keys
+      ansible.builtin.assert:
+        that:
+          - _vault_lines.stdout | int > 0
+          - _stale_keys.stdout | int == 0
+        fail_msg: >-
+          vault.yml still defines {{ _stale_keys.stdout }} of the keys this play
+          rotates. group_vars files merge alphabetically and vault.yml sorts
+          last, so it would override netbird-rotatable.yml on the next deploy
+          and revert the secrets. Remove vault_netbird_turn_secret,
+          netbird_turn_secret and vault_netbird_relay_secret from vault.yml
+          first (one-time migration), then re-run.
+        success_msg: vault.yml does not shadow the rotated keys.
+
+  tasks:
+    # 48 alphanumerics. coturn's shared secret and the relay secret are both
+    # opaque strings; length is the only property that matters.
+    #
+    # ansible.builtin.password, NOT community.general.random_string: the CI
+    # image (ci/images/ansible/Containerfile) installs only community.proxmox
+    # and community.docker, so anything from community.general fails at this
+    # task when the play runs through the project's own tooling — even though
+    # it resolves fine on a workstation that happens to have the collection.
+    #
+    # `/dev/null` as the path makes the lookup generate without persisting;
+    # the value exists only in this fact.
+    - name: Generate replacement secrets
+      ansible.builtin.set_fact:
+        new_turn_secret: "{{ lookup('ansible.builtin.password', '/dev/null length=48 chars=ascii_letters,digits') }}"
+        new_relay_secret: "{{ lookup('ansible.builtin.password', '/dev/null length=48 chars=ascii_letters,digits') }}"
+      no_log: true
+
+    - name: Render the new values to a temporary file
+      ansible.builtin.copy:
+        dest: "{{ rotatable_vault }}.new"
+        mode: "0600"
+        content: |
+          ---
+          # Generated by plays/rotate-netbird-secrets.yml. Do not hand-edit —
+          # the next rotation overwrites this file wholesale.
+          vault_netbird_turn_secret: "{{ new_turn_secret }}"
+          vault_netbird_relay_secret: "{{ new_relay_secret }}"
+      no_log: true
+
+    # Encrypting the whole file rather than inline !vault strings: these three
+    # values are always rewritten together, so per-value encryption buys nothing
+    # and makes the write non-atomic.
+    - name: Encrypt the new file
+      ansible.builtin.command:
+        cmd: >-
+          ansible-vault encrypt
+          --vault-password-file {{ playbook_dir }}/../vault-password.sh
+          {{ rotatable_vault }}.new
+      changed_when: true
+
+    # Move into place only after encryption succeeded, so a failure above never
+    # leaves plaintext where the inventory would load it.
+    - name: Install the encrypted file
+      ansible.builtin.command:
+        cmd: "mv {{ rotatable_vault }}.new {{ rotatable_vault }}"
+      changed_when: true
+
+    - name: Confirm the installed file is encrypted
+      ansible.builtin.command:
+        cmd: "head -1 {{ rotatable_vault }}"
+      register: _vault_header
+      changed_when: false
+      failed_when: "'$ANSIBLE_VAULT' not in _vault_header.stdout"
+
+- name: Apply the new secrets to the NetBird server
+  hosts: netbird_server
+  become: true
+
+  vars:
+    netbird_turn_secret: "{{ hostvars['localhost']['new_turn_secret'] }}"
+    netbird_relay_secret: "{{ hostvars['localhost']['new_relay_secret'] }}"
+
+  roles:
+    - netbird-server
+
+- name: Apply the new secrets to the NetBird proxies
+  hosts: netbird_proxy
+  become: true
+  # One at a time so at least one relay stays reachable while the other
+  # restarts. The server is already on the new secret at this point, so a
+  # proxy is broken only between its own template write and its restart.
+  serial: 1
+
+  vars:
+    netbird_relay_secret: "{{ hostvars['localhost']['new_relay_secret'] }}"
+
+  roles:
+    - netbird-proxy
+
+- name: Report what still needs doing by hand
+  hosts: localhost
+  connection: local
+  gather_facts: false
+
+  tasks:
+    - name: Remaining manual steps
+      ansible.builtin.debug:
+        msg:
+          - "Rotated: turn_secret and relay_secret."
+          - "Commit inventory/group_vars/all/netbird-rotatable.yml — it is encrypted."
+          - "NOT rotated: datastore_key (needs dump/restore), auth_client_secret (Authentik side)."
+          - "OpenBao is unaffected by design — its NetBird credentials are separate ones."

ansible/plays/update-authentik.yml

@@ -1,10 +0,0 @@
----
-- name: Update Authentik HA
-  hosts: authentik_app
-  become: true
-  serial: 1
-  vars:
-    authentik_path: /home/ladmin
-
-  roles:
-    - authentik-update

ansible/plays/update-docker.yml

@@ -1,13 +0,0 @@
----
-- name: Update Docker Containers
-  hosts: proxmox_docker
-  become: true
-  gather_facts: false
-
-  vars:
-    # Define the root directory where your docker-compose files live.
-    # You can override this per-host in host_vars if they differ.
-    docker_compose_dir: "/opt" 
-
-  roles:
-    - docker-update

ansible/plays/update-lxc.yml

@@ -1,32 +0,0 @@
----
-- name: Update LXC Containers
-  hosts: proxmox_all_lxc:!proxmox_noansible
-  become: true
-  gather_facts: true
-  
-  tasks:
-    - name: Update APT Cache
-      ansible.builtin.apt:
-        update_cache: true
-        cache_valid_time: 3600
-      when: ansible_pkg_mgr == 'apt'
-
-    - name: Upgrade all packages
-      ansible.builtin.apt:
-        upgrade: dist
-      when: ansible_pkg_mgr == 'apt'
-      register: apt_upgrade
-
-    - name: Check if reboot required
-      ansible.builtin.stat:
-        path: /var/run/reboot-required
-      register: reboot_required_file
-      when: ansible_pkg_mgr == 'apt'
-
-    - name: Reboot if required
-      ansible.builtin.reboot:
-        msg: "Rebooting due to system updates"
-      when: 
-        - ansible_pkg_mgr == 'apt'
-        - reboot_required_file.stat.exists
-        - apt_upgrade.changed

ansible/plays/update-pve-rolling.yml

@@ -1,9 +0,0 @@
----
-- name: Proxmox VE Rolling Update
-  hosts: proxmox_nodes  # Replace with your inventory group name
-  become: true
-  serial: 1           # CRITICAL: Only update one node at a time
-  any_errors_fatal: true
-  
-  roles:
-    - pve-update

ansible/plays/update-vm.yml

@@ -1,9 +0,0 @@
----
-- name: Update Proxmox VMs (Linux)
-  hosts: proxmox_all_qemu:&status_running:!proxmox_noansible:!proxmox_eol
-  become: true
-  serial: 1
-  gather_facts: false # We gather manually after checking connection
-
-  roles:
-    - vm-update

ansible/plays/update.yml

@@ -1,28 +0,0 @@
----
-- name: Full Infrastructure Update
-  hosts: localhost
-  gather_facts: false
-  tasks:
-    - name: Banner
-      debug:
-        msg: "Starting Full Infrastructure Update Sequence"
-
-- import_playbook: update-vm.yml
-- import_playbook: update-lxc.yml
-- import_playbook: update-authentik.yml
-# Re-render docker-compose templates before pulling updates
-# (immich decommissioned — migrated to in-cluster deployment, see kubernetes/apps/immich/)
-- import_playbook: install-n8n.yml
-- import_playbook: install-grist.yml
-- import_playbook: install-music-assistant.yml
-- import_playbook: update-docker.yml
-- import_playbook: install-log-aggregation.yml
-- import_playbook: update-pve-rolling.yml
-
-- name: Update Summary
-  hosts: localhost
-  gather_facts: false
-  tasks:
-    - name: Done
-      debug:
-        msg: "Full Infrastructure Update Completed Successfully"

ansible/roles/authentik-stateless/defaults/main.yml

@@ -1,18 +0,0 @@
----
-authentik_path: /home/ladmin
-authentik_db_host: "authentik-db.renner.uno"
-authentik_db_user: "authentik"
-authentik_db_name: "authentik"
-authentik_db_password: "changeMeInVault"
-authentik_secret_key: "changeMeInVault"
-authentik_tag: "latest" # or "2025.10.x"
-
-authentik_port_http: 9000
-authentik_port_https: 9443
-
-authentik_email_host: "smtp.example.com"
-authentik_email_port: 587
-authentik_email_user: "user"
-authentik_email_password: "password"
-authentik_email_use_tls: true
-authentik_email_from: "authentik@example.com"

ansible/roles/authentik-stateless/handlers/main.yml

@@ -1,5 +0,0 @@
----
-- name: Restart Authentik
-  community.docker.docker_compose_v2: 
-    project_src: "{{ authentik_path }}"
-    state: restarted

ansible/roles/authentik-stateless/tasks/main.yml

@@ -1,42 +0,0 @@
----
-- name: Ensure Authentik directory exists
-  ansible.builtin.file:
-    path: "{{ authentik_path }}"
-    state: directory
-    owner: "{{ ansible_user | default('root') }}"
-    group: "{{ ansible_user | default('root') }}"
-
-- name: Get latest Authentik release if no tag specified
-  ansible.builtin.uri:
-    url: https://api.github.com/repos/goauthentik/authentik/releases/latest
-    return_content: true
-  register: github_response
-  when: authentik_tag is not defined
-
-- name: Set target version fact
-  set_fact:
-    authentik_version: "{{ github_response.json.tag_name | regex_replace('^version/', '') if authentik_tag is not defined else authentik_tag }}"
-
-- name: Get Docker group ID
-  ansible.builtin.shell: "getent group docker | cut -d: -f3"
-  register: docker_gid_result
-  changed_when: false
-
-- name: Deploy .env configuration
-  ansible.builtin.template:
-    src: .env.j2
-    dest: "{{ authentik_path }}/.env"
-    mode: '0600'
-  notify: Restart Authentik
-
-- name: Deploy Docker Compose file
-  ansible.builtin.template:
-    src: docker-compose.yml.j2
-    dest: "{{ authentik_path }}/docker-compose.yml"
-  notify: Restart Authentik
-
-- name: Ensure Authentik is started
-  community.docker.docker_compose_v2:
-    project_src: "{{ authentik_path }}"
-    state: present
-    pull: always

ansible/roles/authentik-stateless/templates/.env.j2

@@ -1,38 +0,0 @@
-PG_PASS={{ authentik_db_password }}
-PG_HOST={{ authentik_db_host }}
-PG_USER={{ authentik_db_user }}
-PG_DB={{ authentik_db_name }}
-AUTHENTIK_SECRET_KEY={{ authentik_secret_key }}
-AUTHENTIK_ERROR_REPORTING__ENABLED={{ authentik_error_reporting_enabled | default('true') }}
-AUTHENTIK_EMAIL__HOST={{ authentik_email_host }}
-AUTHENTIK_EMAIL__PORT={{ authentik_email_port }}
-AUTHENTIK_EMAIL__USERNAME={{ authentik_email_username }}
-AUTHENTIK_EMAIL__PASSWORD={{ authentik_email_password }}
-AUTHENTIK_EMAIL__USE_TLS={{ authentik_email_use_tls | lower }}
-AUTHENTIK_EMAIL__USE_SSL={{ authentik_email_use_ssl | lower }}
-AUTHENTIK_EMAIL__TIMEOUT={{ authentik_email_timeout }}
-AUTHENTIK_EMAIL__FROM={{ authentik_email_from }}
-AUTHENTIK_LOG_LEVEL={{ authentik_log_level | default('info') }}
-AUTHENTIK_LISTEN__TRUSTED_PROXY_CIDRS={{ authentik_listen_trusted_proxy_cidrs | default('10.0.0.0/8,172.16.0.0/12') }}
-AUTHENTIK_TAG={{ authentik_tag | default(authentik_version) }}
-AUTHENTIK_ERROR_REPORTING__ENABLED=true
-AUTHENTIK_EMAIL__HOST={{ authentik_email_host }}
-AUTHENTIK_EMAIL__PORT={{ authentik_email_port }}
-AUTHENTIK_EMAIL__USERNAME={{ authentik_email_user }}
-AUTHENTIK_EMAIL__PASSWORD={{ authentik_email_password }}
-AUTHENTIK_EMAIL__USE_TLS={{ authentik_email_use_tls }}
-AUTHENTIK_EMAIL__FROM={{ authentik_email_from }}
-AUTHENTIK_PORT_HTTP={{ authentik_port_http }}
-AUTHENTIK_PORT_HTTPS={{ authentik_port_https }}
-AUTHENTIK_CSRF_TRUSTED_ORIGINS=https://authentik.renner.uno,https://auth.renner.uno,https://authentik-app-1.srv.renner.uno,https://authentik-app-2.srv.renner.uno
-
-# HA specific config:
-# Disable Redis (Use Postgres)
-AUTHENTIK_REDIS__HOST=""
-# Worker needs to use shared storage for media
-# Assuming nfs/glusterfs or just volume syncing?
-# Or maybe just use local volume if media uploads ar rare?
-# For true stateless, external S3 storage is recommended:
-# AUTHENTIK_STORAGE__MEDIA__BACKEND="aws"
-# AUTHENTIK_STORAGE__MEDIA__S3__BUCKET="..."
-# For now, local volume (will not sync across nodes without NFS).

ansible/roles/authentik-stateless/templates/docker-compose.yml.j2

@@ -1,41 +0,0 @@
-services:
-  server:
-    image: ghcr.io/goauthentik/server:${AUTHENTIK_TAG:-latest}
-    restart: unless-stopped
-    command: server
-    env_file:
-      - .env
-    environment:
-      AUTHENTIK_REDIS__HOST: ""
-      AUTHENTIK_POSTGRESQL__HOST: ${PG_HOST}
-      AUTHENTIK_POSTGRESQL__NAME: ${PG_DB}
-      AUTHENTIK_POSTGRESQL__USER: ${PG_USER}
-      AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
-      AUTHENTIK_LISTEN__METRICS: "0.0.0.0:9300"
-    volumes:
-      - ./media:/media
-      - ./custom-templates:/templates
-      - ./certs:/certs
-    ports:
-      - "${AUTHENTIK_PORT_HTTP:-9000}:9000"
-      - "${AUTHENTIK_PORT_HTTPS:-9443}:9443"
-      - "9300:9300"
-
-  worker:
-    image: ghcr.io/goauthentik/server:${AUTHENTIK_TAG:-latest}
-    restart: unless-stopped
-    command: worker
-    user: "1000:{{ docker_gid_result.stdout }}"
-    env_file:
-      - .env
-    environment:
-      AUTHENTIK_REDIS__HOST: ""
-      AUTHENTIK_POSTGRESQL__HOST: ${PG_HOST}
-      AUTHENTIK_POSTGRESQL__NAME: ${PG_DB}
-      AUTHENTIK_POSTGRESQL__USER: ${PG_USER}
-      AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
-    volumes:
-      - /var/run/docker.sock:/var/run/docker.sock
-      - ./media:/media
-      - ./certs:/certs
-      - ./custom-templates:/templates

ansible/roles/authentik-update/tasks/main.yml

@@ -1,47 +0,0 @@
----
-- name: Get latest Authentik release
-  ansible.builtin.uri:
-    url: https://api.github.com/repos/goauthentik/authentik/releases/latest
-    return_content: true
-  register: github_response
-
-- name: Set target version
-  ansible.builtin.set_fact:
-    target_version: "{{ github_response.json.tag_name | regex_replace('^version/', '') }}"
-
-- name: Display target version
-  ansible.builtin.debug:
-    msg: "Updating Authentik to version {{ target_version }}"
-
-- name: Update AUTHENTIK_TAG in .env
-  ansible.builtin.lineinfile:
-    path: "{{ authentik_path | default('/home/ladmin') }}/.env"
-    regexp: '^AUTHENTIK_TAG='
-    line: "AUTHENTIK_TAG={{ target_version }}"
-    create: true
-
-- name: Pull new images
-  ansible.builtin.command:
-    cmd: docker compose pull
-    chdir: "{{ authentik_path | default('/home/ladmin') }}"
-
-- name: Pull Outpost images
-  ansible.builtin.shell: |
-    docker pull ghcr.io/goauthentik/proxy:{{ target_version }}
-    docker pull ghcr.io/goauthentik/ldap:{{ target_version }}
-    docker pull ghcr.io/goauthentik/radius:{{ target_version }}
-  changed_when: true
-
-- name: Apply updates
-  ansible.builtin.command:
-    cmd: docker compose up -d --force-recreate
-    chdir: "{{ authentik_path | default('/home/ladmin') }}"
-
-- name: Restart worker to trigger outpost update
-  ansible.builtin.command: 
-    cmd: docker compose restart worker
-    chdir: "{{ authentik_path | default('/home/ladmin') }}"
-
-- name: Prune unused images
-  ansible.builtin.command:
-    cmd: docker image prune -f

ansible/roles/blackbox-exporter/defaults/main.yml

@@ -1,26 +0,0 @@
----
-blackbox_exporter_version: "0.25.0"
-blackbox_exporter_port: 9115
-blackbox_exporter_user: "blackbox_exporter"
-blackbox_exporter_group: "blackbox_exporter"
-
-# Endpoints to probe
-blackbox_http_targets:
-  - name: "gitea"
-    url: "https://gitea.renner.uno"
-  - name: "immich"
-    url: "https://immich.renner.uno"
-  - name: "authentik"
-    url: "https://auth.renner.uno"
-  - name: "home-assistant"
-    url: "https://ha.renner.uno"
-  - name: "grafana"
-    url: "https://grafana.renner.uno"
-  - name: "prometheus"
-    url: "https://prometheus.renner.uno"
-
-blackbox_dns_targets: []
-
-blackbox_tcp_targets:
-  - name: "postgres-ha"
-    target: "10.2.40.135:5432"

ansible/roles/blackbox-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/blackbox-exporter/tasks/main.yml

@@ -1,80 +0,0 @@
----
-- name: Create blackbox_exporter group
-  group:
-    name: "{{ blackbox_exporter_group }}"
-    system: yes
-    state: present
-
-- name: Create blackbox_exporter user
-  user:
-    name: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    createhome: no
-    state: present
-
-- name: Create blackbox_exporter directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0755'
-  loop:
-    - /etc/blackbox_exporter
-    - /var/lib/blackbox_exporter
-
-- name: Download blackbox_exporter
-  get_url:
-    url: "https://github.com/prometheus/blackbox_exporter/releases/download/v{{ blackbox_exporter_version }}/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64.tar.gz"
-    dest: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    mode: '0644'
-
-- name: Extract blackbox_exporter
-  unarchive:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    dest: /tmp
-    remote_src: yes
-    creates: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
-
-- name: Install blackbox_exporter binary
-  copy:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64/blackbox_exporter"
-    dest: /usr/local/bin/blackbox_exporter
-    owner: root
-    group: root
-    mode: '0755'
-    remote_src: yes
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter configuration
-  template:
-    src: blackbox.yml.j2
-    dest: /etc/blackbox_exporter/blackbox.yml
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter systemd service
-  template:
-    src: blackbox_exporter.service.j2
-    dest: /etc/systemd/system/blackbox_exporter.service
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Enable and start blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"

ansible/roles/blackbox-exporter/templates/blackbox.yml.j2

@@ -1,60 +0,0 @@
-modules:
-  http_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
-      valid_status_codes: []  # Defaults to 2xx
-      method: GET
-      follow_redirects: true
-      fail_if_ssl: false
-      fail_if_not_ssl: false
-      tls_config:
-        insecure_skip_verify: false
-
-  http_post_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      method: POST
-      valid_status_codes: []
-
-  tcp_connect:
-    prober: tcp
-    timeout: 5s
-
-  # DNS module for external domain resolution (tests upstream/cache)
-  dns_external:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "google.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for internal domain resolution (tests local resolution)
-  dns_internal:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "k8s-1.srv.renner.uno"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for common cached domain
-  dns_cached:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "cloudflare.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  icmp:
-    prober: icmp
-    timeout: 5s
-    icmp:
-      preferred_ip_protocol: "ip4"

ansible/roles/blackbox-exporter/templates/blackbox_exporter.service.j2

@@ -1,22 +0,0 @@
-[Unit]
-Description=Prometheus Blackbox Exporter
-Documentation=https://github.com/prometheus/blackbox_exporter
-Wants=network-online.target
-After=network-online.target
-
-[Service]
-Type=simple
-User={{ blackbox_exporter_user }}
-Group={{ blackbox_exporter_group }}
-ExecStart=/usr/local/bin/blackbox_exporter \
-    --config.file=/etc/blackbox_exporter/blackbox.yml \
-    --web.listen-address=:{{ blackbox_exporter_port }}
-Restart=on-failure
-RestartSec=5s
-
-# Allow ICMP probing (requires CAP_NET_RAW)
-AmbientCapabilities=CAP_NET_RAW
-CapabilityBoundingSet=CAP_NET_RAW
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/chrony-ntp-server/defaults/main.yml

@@ -1,9 +1,25 @@
 ---
 # Chrony NTP Server Role Defaults
 
-# Networks allowed to use this server as NTP source
+# Networks allowed to use this server as NTP source.
+#
+# BOTH families are required. IPv6 is the primary address family here -- the
+# service address ntp.srv.renner.uno resolves to fd40::123 -- so an IPv4-only
+# allow list refuses every client that uses the documented name.
+#
+# The hosts carried `allow fd00::/8` for a long time without it ever being in
+# this file. Applying the role on 2026-08-07 rendered the template over it and
+# dropped the line on all four NTP servers at once, which took internal NTP
+# down over IPv6 until it was added here. Same class of untracked host state as
+# the anycast service address, and the same failure: config that exists only on
+# the box survives right up until something re-renders it.
 chrony_allowed_networks:
   - "10.2.0.0/16"
+  - "fd00::/8"
+  # Link-local. A client on the same segment may source its query from fe80::,
+  # and without this chrony refuses it -- the request arrives and is dropped,
+  # which looks like the server being unreachable rather than a policy refusal.
+  - "fe80::/10"
 
 # Primary NTP servers (PTB - Physikalisch-Technische Bundesanstalt, Stratum 1)
 chrony_upstream_servers:
@@ -18,6 +34,26 @@ chrony_upstream_pools:
 # Local stratum if disconnected from upstream
 chrony_local_stratum: 10
 
+# Step (rather than slew) the clock when the offset exceeds the threshold, but
+# only for the first N updates after chronyd starts. Without a makestep
+# directive chronyd only ever slews, so a large offset is never closed in
+# practice.
+#
+# edge-1, 2026-08-04: the CMOS battery is dead, so after a power loss the RTC
+# came up reading 2018-08-20. The kernel set the clock from it, systemd floored
+# that forward to its own built-in epoch (2026-06-26), and it ended up at
+# ~2026-07-19 -- about 16 days behind. chronyd then synced to ptbtime1, reported
+# "System time: 1376235.375 seconds slow of NTP time", and corrected none of it.
+# With host time before the cluster certificate's NotBefore, every outbound TLS
+# handshake failed x509 validation, taking out the MQTT bridge (heating
+# telemetry) and the Loki log shipper at once. The 16 days is incidental -- it
+# is just where the software flooring landed; the fault is that nothing steps.
+#
+# Limiting to the first few updates keeps steps confined to boot, so a
+# long-running server is never jumped out from under its workloads.
+chrony_makestep_threshold: 1.0
+chrony_makestep_limit: 3
+
 # Hardware timestamping. Fine on the PVE nodes' Intel NICs; the Wyse 3040
 # (edge-1) has a Realtek NIC that does not support it, and chronyd logs an
 # error per interface when asked for it. Overridden to false in

ansible/roles/chrony-ntp-server/templates/chrony.conf.j2

@@ -14,6 +14,11 @@ pool {{ pool }}
 # Record the rate at which the system clock gains/losses time
 driftfile /var/lib/chrony/chrony.drift
 
+# Step the clock instead of slewing it when the offset is large, limited to the
+# first few updates after start so only boot-time corrections jump. Rationale
+# and the incident behind it: chrony_makestep_* in the role's defaults/main.yml.
+makestep {{ chrony_makestep_threshold }} {{ chrony_makestep_limit }}
+
 # Allow NTP client access from local networks
 {% for network in chrony_allowed_networks %}
 allow {{ network }}

ansible/roles/claude-workstation/RUNBOOK.md

@@ -118,8 +118,11 @@ After the `kubernetes/apps/claude-workstation/` PR merges and Flux reconciles:
 NS=apps
 TOKEN=$(kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.token}' | base64 -d)
 kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/ca.crt
-# server = the API endpoint reachable over the site-to-site route (a CP node or VIP):
-kubectl config set-cluster homelab --server=https://10.2.40.201:6443 \
+# server = the cluster's stable API endpoint (kubernetes/talos/patches/cluster.yaml).
+# Use the name, not an address: it is what the API server certificate carries as a
+# SAN, so an IP here fails TLS verification even when it is reachable. It must
+# resolve from the workstation over the site-to-site route.
+kubectl config set-cluster homelab --server=https://k8s.renner.uno:6443 \
   --certificate-authority=/tmp/ca.crt --embed-certs=true --kubeconfig=/tmp/claude-ws.kubeconfig
 kubectl config set-credentials claude-workstation --token="$TOKEN" --kubeconfig=/tmp/claude-ws.kubeconfig
 kubectl config set-context homelab --cluster=homelab --user=claude-workstation --kubeconfig=/tmp/claude-ws.kubeconfig
@@ -166,8 +169,7 @@ Merge the PR, then trigger the pipeline (Woodpecker → Run → `PIPELINE ==
 "deploy-claude-workstation"`), or push a change under the role's paths. CI SSHes
 to `10.3.40.60` over the tunnel and applies `common`, `netbird-client`, and this
 role. Also run the umbrella `plays/common.yml` against the new host once, so it
-picks up `qemu-guest-agent` and `monitoring-agent` (Alloy → Loki/Prometheus)
-like every other inventory host.
+picks up `qemu-guest-agent` like every other inventory host.
 
 ---
 

ansible/roles/claude-workstation/tasks/main.yml

@@ -3,8 +3,8 @@
 # Debian 12 VM on the off-site nas-2 (Synology VMM). See README for the manual
 # bootstrap (VM creation, Netbird route, secret material) that precedes this.
 #
-# common.yml already applies `common`, `qemu-guest-agent` and `monitoring-agent`
-# (Alloy -> Loki/Prometheus) to every inventory host, so this role owns only the
+# common.yml already applies `common` and `qemu-guest-agent`
+# to every inventory host, so this role owns only the
 # Claude-specific surface: toolchain, the two credential-isolated users, the
 # read-only creds, notes sync, the unattended timers, and egress lockdown.
 

ansible/roles/edge-flash-wear/defaults/main.yml

@@ -0,0 +1,15 @@
+---
+# Flash-wear mitigation.
+
+# --- Logging & eMMC wear mitigation ------------------------------------
+# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
+# kept off local flash: containers + native services log to a RAM-only
+# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
+# wear check round out the flash-longevity measures. See README "Logging &
+# eMMC".
+edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
+
+# Container stdout -> journal, no json log files on the flash. The journal
+# itself is volatile (see the tasks above), so container logs land in RAM and
+# are shipped off the box by edge-log-shipper.
+docker_log_driver: journald

ansible/roles/edge-flash-wear/files/edge-emmc-health

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-# edge-1 eMMC wear/health check. Managed by roles/edge-host.
+# edge-1 eMMC wear/health check. Managed by roles/edge-flash-wear.
 #
 # Reads the JEDEC eMMC 5.x life-time and pre-EOL estimators exposed via sysfs
 # and emits a warning-priority journal message when the flash leaves the

ansible/roles/edge-flash-wear/handlers/main.yml

@@ -0,0 +1,17 @@
+---
+- name: Restart journald
+  ansible.builtin.service:
+    name: systemd-journald
+    state: restarted
+
+# Handlers run in definition order, so this fires after 'Restart journald':
+# once the journal is volatile, the on-disk store is dead weight on the flash.
+- name: Purge persistent journal store
+  ansible.builtin.file:
+    path: /var/log/journal
+    state: absent
+
+- name: Restart docker
+  ansible.builtin.service:
+    name: docker
+    state: restarted

ansible/roles/edge-flash-wear/tasks/main.yml

@@ -0,0 +1,100 @@
+---
+# Flash-wear mitigation for hosts on small eMMC (Dell Wyse 3040 and similar).
+#
+# journald -> volatile (RAM) and size-capped, root mounted noatime+lazytime,
+# periodic TRIM, and a weekly wear check. Shipping the volatile journal off the
+# box is a separate concern -- see the edge-log-shipper role.
+- name: Install mmc-utils
+  ansible.builtin.package:
+    name: mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
+    state: present
+
+- name: Ensure journald drop-in directory exists
+  ansible.builtin.file:
+    path: /etc/systemd/journald.conf.d
+    state: directory
+    mode: "0755"
+
+- name: Switch the systemd journal to volatile (RAM) storage
+  ansible.builtin.template:
+    src: journald-edge.conf.j2
+    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
+    mode: "0644"
+  notify:
+    - Restart journald
+    - Purge persistent journal store
+
+- name: Resolve root filesystem facts (for noatime)
+  ansible.builtin.set_fact:
+    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
+
+- name: Refuse to touch fstab without a stable root UUID
+  ansible.builtin.assert:
+    that:
+      - edge_root_fs.uuid | default('') | length > 0
+    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
+
+# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
+# heartbeat that otherwise dirties the journal every couple seconds); noatime
+# drops access-time writes. Neither weakens data durability.
+- name: Mount root with noatime + lazytime (persisted in fstab)
+  ansible.posix.mount:
+    path: /
+    src: "UUID={{ edge_root_fs.uuid }}"
+    fstype: "{{ edge_root_fs.fstype }}"
+    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
+    dump: "0"
+    passno: "1"   # root fs — keep boot-time fsck ordering
+    backup: true
+    state: mounted
+
+- name: Enable periodic TRIM (helps eMMC wear-leveling)
+  ansible.builtin.systemd:
+    name: fstrim.timer
+    enabled: true
+    state: started
+
+- name: Install eMMC health-check script
+  ansible.builtin.copy:
+    src: edge-emmc-health
+    dest: /usr/local/bin/edge-emmc-health
+    mode: "0755"
+
+- name: Install eMMC health-check service + timer
+  ansible.builtin.copy:
+    src: "{{ item }}"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - edge-emmc-health.service
+    - edge-emmc-health.timer
+
+- name: Enable eMMC health-check timer
+  ansible.builtin.systemd:
+    name: edge-emmc-health.timer
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# Container stdout into the journal instead of json files on disk. This lives
+# here, with the rest of the flash-wear handling, rather than in docker-install:
+# that role is shared by ten other plays (authentik, n8n, owncloud, grist,
+# netbird, gatus, music-assistant, ical-proxy), none of which have a
+# daemon.json today. Adding it there would have created one on the next run of
+# any of those plays and restarted every container on those hosts.
+#
+# /etc/docker is created explicitly because this role runs before
+# docker-install on a first bootstrap, so the directory may not exist yet. The
+# handler runs at the end of the play, by which time docker is installed.
+- name: Ensure the docker config directory exists
+  ansible.builtin.file:
+    path: /etc/docker
+    state: directory
+    mode: "0755"
+
+- name: Route container logs to the journal (keeps json logs off the flash)
+  ansible.builtin.template:
+    src: docker-daemon.json.j2
+    dest: /etc/docker/daemon.json
+    mode: "0644"
+  notify: Restart docker

ansible/roles/edge-flash-wear/templates/docker-daemon.json.j2

@@ -0,0 +1,3 @@
+{
+  "log-driver": "{{ docker_log_driver }}"
+}

ansible/roles/edge-flash-wear/templates/journald-edge.conf.j2

@@ -1,4 +1,4 @@
-# Managed by roles/edge-host. edge-1 boots off a small internal eMMC, so the
+# Managed by roles/edge-flash-wear. edge-1 boots off a small internal eMMC, so the
 # journal must not persist to flash: keep it in RAM (/run/log/journal) and cap
 # its size. Grafana Alloy (edge-alloy.service) ships this journal to Loki
 # before it rotates out of RAM — nothing durable lives here.

ansible/roles/edge-host/defaults/main.yml

@@ -1,118 +0,0 @@
----
-edge_base_dir: /opt/edge-host
-edge_tz: Europe/Berlin
-
-# --- Logging & eMMC wear mitigation ------------------------------------
-# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
-# kept off local flash: containers + native services log to a RAM-only
-# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
-# wear check round out the flash-longevity measures. See README "Logging &
-# eMMC".
-edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
-edge_docker_log_driver: journald      # container stdout -> journal, no json logs on eMMC
-
-# Grafana Alloy log shipper. The srv-network Loki (loki.srv.renner.uno:3100,
-# no auth) is firewalled off the IoT VLAN; the only endpoint edge-1 can reach
-# is the authenticated cluster ingress (Traefik basic-auth).
-edge_alloy_config_file: /etc/alloy/edge.alloy
-edge_alloy_http_port: 12345
-edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
-edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
-edge_loki_username: edge-1
-edge_alloy_labels:
-  environment: production
-  datacenter: homelab
-
-# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
-# mosquitto on loopback, which bridges to the central cluster broker over TLS
-# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
-# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
-# blips. The local broker writes nothing to the eMMC (persistence off, logs to
-# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
-edge_mqtt_host: 127.0.0.1
-edge_mqtt_port: 1883
-# Upstream cluster broker the local bridge connects to (TLS; the public
-# Let's Encrypt cert is validated against the system CA bundle).
-edge_mqtt_cluster_host: mqtt.renner.uno
-edge_mqtt_cluster_tls_port: 8883
-edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
-
-# --- USB devices (stable by-id symlinks). After physically moving the
-# sticks to edge-1, verify with `ls -l /dev/serial/by-id/` — the by-id
-# names travel with the device, not the host, so these should not change.
-edge_zwave_device: /dev/serial/by-id/usb-Nabu_Casa_ZWA-2_80B54EE15720-if00
-edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
-
-# --- Images. Versions mirror the k8s manifests at migration time.
-# sml2mqtt stays hand-pinned (critical meter bridge — was deliberately
-# excluded from image-automation in k8s, keep that property here).
-edge_zwavejs_image: ghcr.io/zwave-js/zwave-js-ui:11.21.1
-edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
-# Alpine version the zwave-js-ui image is built on — the source for the musl
-# libeatmydata.so preloaded into that container (see tasks "Extract musl
-# libeatmydata.so"). Keep in sync with the base if edge_zwavejs_image bumps
-# its Alpine major (check: docker exec zwave-js cat /etc/os-release).
-edge_zwave_alpine_base: alpine:3.22
-
-# --- CAN / open3e (Viessmann E3 external bus) ---
-edge_can_interface: can0
-edge_can_bitrate: 250000
-edge_open3e_pkg: git+https://github.com/open3e/open3e.git
-edge_open3e_mqtt_user: open3e
-edge_open3e_mqtt_topic: open3e
-edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-edge_open3e_poll_seconds: 30
-# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
-# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
-# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
-# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
-# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
-# the internal PV-surplus EMS depends on). Decodes via the master O3EList
-# codec inherited through the vx3plus overlay.
-# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
-# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
-# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
-# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
-# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
-# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
-# entries when added), so the multi-frame read is cheap at a 30 s poll.
-# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
-# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
-edge_open3e_datapoints: >-
-  vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
-# ECU addresses; -dev selects codec set, tx selects the UDS address.
-# vcmu (refrigerant-circuit controller) has no dedicated codec set —
-# omitting dpList makes open3e fall back to the generic datapoint list,
-# which decodes all polled VCMU DIDs.
-edge_open3e_devices:
-  vitocal: { tx: "0x680", dpList: "vcal" }
-  # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
-  # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
-  vx3: { tx: "0x6a1", dpList: "/opt/edge-host/open3e/vx3plus.py" }
-  vcmu: { tx: "0x68c" }
-
-# --- E3onCAN: passive E380 energy-meter listener (no bus writes, so it
-# coexists with the single open3e instance). The meter broadcasts at
-# ~5 Hz; the gap throttles MQTT to one update per DID per interval.
-edge_e3oncan_repo: https://github.com/MyHomeMyData/E3onCAN.git
-edge_e3oncan_version: 461dcbdf97e8d72d2edb9929c63b39382af6c3a1  # V0.5.2
-edge_e3oncan_gap_seconds: 10
-edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-
-# --- Secrets: define these in inventory/group_vars/edge_host/vault.yml
-# (ansible-vault). See README.md for the kubectl one-liners that read the
-# current values out of the cluster during migration.
-# edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
-# edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
-# edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"
-
-# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
-# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
-# VRRP group, which needs layer-2 adjacency with the other members on the
-# service VLAN. The untagged VLAN 30 lease stays this host's only default
-# route -- see templates/10-vlan40.network.j2.
-edge_vlan40_enabled: false
-edge_vlan40_parent: enp1s0
-edge_vlan40_id: 40
-edge_vlan40_name: enp1s0.40
-edge_vlan40_parent_network: 20-wired.network

ansible/roles/edge-host/handlers/main.yml

@@ -1,66 +0,0 @@
----
-# reload alone only re-reads config files; existing links keep their old
-# state until reconfigured.
-- name: Reload networkd
-  ansible.builtin.shell:
-    cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
-  changed_when: true
-
-# Reconfiguring the parent is what actually materialises the VLAN child -- a
-# bare reload leaves the existing link as-is (see above). This re-runs DHCP on
-# enp1s0, which is this headless host's only path to the network, so it is the
-# one genuinely risky step in the VLAN change.
-- name: Reload networkd for VLAN
-  ansible.builtin.shell:
-    cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
-  changed_when: true
-
-# Ordered before 'Restart edge compose services': the daemon must pick up the
-# new default log driver before the containers are recreated onto it.
-- name: Restart docker
-  ansible.builtin.service:
-    name: docker
-    state: restarted
-
-- name: Restart journald
-  ansible.builtin.service:
-    name: systemd-journald
-    state: restarted
-
-# Handlers run in definition order, so this fires after 'Restart journald':
-# once the journal is volatile, the on-disk store is dead weight on the eMMC.
-- name: Purge persistent journal store
-  ansible.builtin.file:
-    path: /var/log/journal
-    state: absent
-
-- name: Restart alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    state: restarted
-    daemon_reload: true
-
-# Ordered before the MQTT clients (compose/open3e/e3oncan) so they reconnect
-# to a freshly-configured local broker.
-- name: Restart mosquitto
-  ansible.builtin.service:
-    name: mosquitto
-    state: restarted
-
-- name: Restart edge compose services
-  ansible.builtin.command:
-    cmd: docker compose up -d --force-recreate
-    chdir: "{{ edge_base_dir }}"
-  changed_when: true
-
-- name: Restart open3e
-  ansible.builtin.systemd:
-    name: open3e
-    state: restarted
-    daemon_reload: true
-
-- name: Restart e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    state: restarted
-    daemon_reload: true

ansible/roles/edge-host/tasks/main.yml

@@ -1,440 +0,0 @@
----
-# Arch edge host: zwave-js-ui + sml2mqtt as docker compose services,
-# open3e natively (venv + systemd) because it needs a SocketCAN interface,
-# which is host network state, not something to hide in a container.
-# Docker itself comes from the docker-install role (Arch-aware).
-
-- name: Install base packages
-  ansible.builtin.package:
-    name:
-      - python
-      - python-packaging  # ansible's pip module imports it on the target
-      - git
-      - mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
-      - grafana-alloy  # log shipper -> Loki (see Alloy section below)
-      - mosquitto      # local MQTT bridge broker -> cluster over TLS
-    state: present
-
-# --- Local MQTT bridge broker. Edge clients publish to this over loopback;
-# it bridges to the cluster broker over TLS (open3e/e3oncan can't do MQTTS).
-# Configured with persistence off + stdout logging so it writes nothing to the
-# eMMC. Deployed before the clients so they have somewhere to connect.
-# See edge-mqtt-bridge.conf.j2 / README "MQTT bridge".
-- name: Deploy local mosquitto bridge config
-  ansible.builtin.template:
-    src: edge-mqtt-bridge.conf.j2
-    dest: /etc/mosquitto/mosquitto.conf
-    owner: root
-    group: mosquitto
-    mode: "0640"   # carries the cluster bridge credentials
-  notify: Restart mosquitto
-
-- name: Enable local mosquitto bridge
-  ansible.builtin.systemd:
-    name: mosquitto
-    enabled: true
-    state: started
-
-# ======================================================================
-# eMMC wear mitigation + log shipping. edge-1 (Dell Wyse 3040) runs on a
-# small internal eMMC, so logs must not churn local flash. Strategy:
-#   - journald -> volatile (RAM), size-capped        (journald-edge.conf)
-#   - docker   -> journald log driver, no json files (docker-daemon.json)
-#   - Grafana Alloy tails the RAM journal -> Loki, WAL on tmpfs
-# plus TRIM, noatime and a weekly wear check for flash longevity.
-# See roles/edge-host/README.md "Logging & eMMC".
-# ======================================================================
-
-- name: Ensure journald drop-in directory exists
-  ansible.builtin.file:
-    path: /etc/systemd/journald.conf.d
-    state: directory
-    mode: "0755"
-
-- name: Switch the systemd journal to volatile (RAM) storage
-  ansible.builtin.template:
-    src: journald-edge.conf.j2
-    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
-    mode: "0644"
-  notify:
-    - Restart journald
-    - Purge persistent journal store
-
-- name: Resolve root filesystem facts (for noatime)
-  ansible.builtin.set_fact:
-    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
-
-- name: Refuse to touch fstab without a stable root UUID
-  ansible.builtin.assert:
-    that:
-      - edge_root_fs.uuid | default('') | length > 0
-    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
-
-# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
-# heartbeat that otherwise dirties the journal every couple seconds); noatime
-# drops access-time writes. Neither weakens data durability.
-- name: Mount root with noatime + lazytime (persisted in fstab)
-  ansible.posix.mount:
-    path: /
-    src: "UUID={{ edge_root_fs.uuid }}"
-    fstype: "{{ edge_root_fs.fstype }}"
-    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
-    dump: "0"
-    passno: "1"   # root fs — keep boot-time fsck ordering
-    backup: true
-    state: mounted
-
-- name: Enable periodic TRIM (helps eMMC wear-leveling)
-  ansible.builtin.systemd:
-    name: fstrim.timer
-    enabled: true
-    state: started
-
-- name: Install eMMC health-check script
-  ansible.builtin.copy:
-    src: edge-emmc-health
-    dest: /usr/local/bin/edge-emmc-health
-    mode: "0755"
-
-- name: Install eMMC health-check service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-emmc-health.service
-    - edge-emmc-health.timer
-
-- name: Enable eMMC health-check timer
-  ansible.builtin.systemd:
-    name: edge-emmc-health.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- Grafana Alloy: ship the volatile journal to Loki. Container logs reach
-# the journal via docker's journald driver (configured below), native
-# services log to the journal directly, so a single journal source covers
-# everything. Storage/WAL sits on /run (tmpfs) — see edge-alloy.service.
-- name: Create alloy system group
-  ansible.builtin.group:
-    name: alloy
-    system: true
-
-- name: Create alloy system user
-  ansible.builtin.user:
-    name: alloy
-    group: alloy
-    groups: systemd-journal
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-- name: Create alloy config directory
-  ansible.builtin.file:
-    path: /etc/alloy
-    state: directory
-    owner: alloy
-    group: alloy
-    mode: "0750"
-
-- name: Deploy alloy config (journal -> Loki)
-  ansible.builtin.template:
-    src: edge-alloy.alloy.j2
-    dest: "{{ edge_alloy_config_file }}"
-    owner: alloy
-    group: alloy
-    mode: "0600"   # carries the Loki basic-auth password
-  notify: Restart alloy
-
-- name: Deploy alloy systemd unit
-  ansible.builtin.template:
-    src: edge-alloy.service.j2
-    dest: /etc/systemd/system/edge-alloy.service
-    mode: "0644"
-  notify: Restart alloy
-
-- name: Enable edge-alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- SocketCAN: systemd-networkd owns can0. gs_usb is an in-tree module
-# and autoloads when the Innomaker adapter enumerates; networkd then
-# matches the interface and applies bitrate + up. RestartSec recovers the
-# link after bus-off without manual intervention.
-- name: Configure can0 via systemd-networkd
-  ansible.builtin.template:
-    src: 80-can0.network.j2
-    dest: /etc/systemd/network/80-can0.network
-    mode: "0644"
-  notify: Reload networkd
-
-# --- Tagged VLAN 40. Additive by construction: the parent link keeps its own
-# .network file untouched and gains the VLAN via a drop-in, so reverting is a
-# matter of deleting these three paths. This host is headless and enp1s0 is its
-# only route to the network, hence the caution.
-- name: Create VLAN sub-interface netdev
-  ansible.builtin.template:
-    src: 10-vlan40.netdev.j2
-    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
-- name: Configure VLAN sub-interface addressing
-  ansible.builtin.template:
-    src: 10-vlan40.network.j2
-    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
-- name: Ensure parent network drop-in directory exists
-  ansible.builtin.file:
-    path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
-    state: directory
-    mode: "0755"
-  when: edge_vlan40_enabled
-  tags: [vlan40]
-
-- name: Attach VLAN to parent link via drop-in
-  ansible.builtin.template:
-    src: vlan40-parent-dropin.conf.j2
-    dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
-- name: Enable systemd-networkd
-  ansible.builtin.service:
-    name: systemd-networkd
-    state: started
-    enabled: true
-
-# Docker networking needs no daemon.json (both services use network_mode:
-# host — real addresses, v4+v6, no bridge/NAT). The daemon.json below exists
-# only to route container stdout into the journald log driver so json logs
-# never touch the eMMC; the journal itself is volatile (RAM) and shipped to
-# Loki by edge-alloy. Both compose services inherit this default driver.
-- name: Configure docker daemon (journald log driver, eMMC-friendly)
-  ansible.builtin.template:
-    src: docker-daemon.json.j2
-    dest: /etc/docker/daemon.json
-    mode: "0644"
-  notify:
-    - Restart docker
-    - Restart edge compose services
-
-# --- Directory layout ---
-- name: Create service directories
-  ansible.builtin.file:
-    path: "{{ item }}"
-    state: directory
-    mode: "0755"
-  loop:
-    - "{{ edge_base_dir }}"
-    - "{{ edge_base_dir }}/zwave-js/store"
-    - "{{ edge_base_dir }}/sml2mqtt"
-
-# --- zwave-js eMMC write reduction: batch fsync via libeatmydata ---------
-# zwave-js persists its value DB with an fsync on *every* value change
-# (jsonl-db, throttleFS unset by zwave-js-ui). On a chatty network that
-# fsync storm churns the eMMC (~2.6 GiB/day for ~0.1 KiB/s of real data).
-# Preloading libeatmydata makes fsync a no-op so the kernel batches the
-# writes via normal writeback — full HA data resolution and cache
-# persistence are untouched (see docker-compose.yml.j2 for the mount + env).
-# The .so must match the zwave-js-ui image's musl/Alpine base, so extract it
-# from that exact Alpine version (bridge networking is broken on this host,
-# hence --network=host). Refresh by deleting the .so if the base ever bumps
-# its Alpine major version.
-- name: Extract musl libeatmydata.so (matches the zwave-js-ui Alpine base)
-  ansible.builtin.shell:
-    cmd: |
-      set -euo pipefail
-      tmp="$(mktemp)"
-      trap 'rm -f "$tmp"' EXIT
-      docker run --rm --network=host {{ edge_zwave_alpine_base }} \
-        sh -c 'apk add --no-cache libeatmydata >/dev/null 2>&1 && cat /usr/lib/libeatmydata.so' > "$tmp"
-      test "$(stat -c%s "$tmp")" -ge 8000   # sanity: real .so is ~14 KB, guards against a failed/empty pull
-      install -m 0644 "$tmp" {{ edge_base_dir }}/zwave-js/libeatmydata.so
-    creates: "{{ edge_base_dir }}/zwave-js/libeatmydata.so"
-  notify: Restart edge compose services
-
-# Because fsync is neutralized, a hard power-loss during the (rare) settings
-# write could truncate settings.json — which holds the Z-Wave network keys.
-# Keep a validated rolling backup on the eMMC so the keys stay recoverable.
-- name: Install zwave-js settings.json backup script
-  ansible.builtin.template:
-    src: edge-zwave-settings-backup.j2
-    dest: /usr/local/bin/edge-zwave-settings-backup
-    mode: "0755"
-
-- name: Install zwave-js settings backup service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-zwave-settings-backup.service
-    - edge-zwave-settings-backup.timer
-
-- name: Enable zwave-js settings backup timer
-  ansible.builtin.systemd:
-    name: edge-zwave-settings-backup.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- Compose services: zwave-js-ui + sml2mqtt ---
-- name: Deploy docker-compose.yml
-  ansible.builtin.template:
-    src: docker-compose.yml.j2
-    dest: "{{ edge_base_dir }}/docker-compose.yml"
-    mode: "0644"
-  notify: Restart edge compose services
-
-- name: Deploy zwave-js env (session secret)
-  ansible.builtin.template:
-    src: zwavejs.env.j2
-    dest: "{{ edge_base_dir }}/zwavejs.env"
-    mode: "0600"
-  notify: Restart edge compose services
-
-# sml2mqtt reads only a config file and has no env override for the
-# password, so the rendered config carries it — 0600, root-owned, same
-# trust boundary as the k8s init-container approach it replaces.
-- name: Deploy sml2mqtt config
-  ansible.builtin.template:
-    src: sml2mqtt-config.yml.j2
-    dest: "{{ edge_base_dir }}/sml2mqtt/config.yml"
-    mode: "0600"
-  notify: Restart edge compose services
-
-- name: Pull compose images
-  ansible.builtin.command:
-    cmd: docker compose pull
-    chdir: "{{ edge_base_dir }}"
-  changed_when: false
-
-# Tolerates missing USB devices: before the physical cutover the ZWA-2 /
-# FTDI sticks are still on the old k8s-edge VM, and a container whose
-# device is absent must not block the rest of the role (open3e/CAN moves
-# first). Re-run the play — or `docker compose up -d` — after each stick
-# arrives.
-- name: Start compose services
-  ansible.builtin.command:
-    cmd: docker compose up -d
-    chdir: "{{ edge_base_dir }}"
-  register: compose_up
-  failed_when: >-
-    compose_up.rc != 0
-    and 'error gathering device information' not in compose_up.stderr
-  changed_when: >-
-    'Started' in compose_up.stderr or 'Created' in compose_up.stderr
-    or 'Started' in compose_up.stdout or 'Creating' in compose_up.stdout
-
-- name: Warn about containers waiting for their USB device
-  ansible.builtin.debug:
-    msg: >-
-      docker compose could not start every service (USB device not present
-      yet — expected before cutover): {{ compose_up.stderr_lines | select('search', 'device') | list }}
-  when: compose_up.rc != 0
-
-# --- open3e: native install. CAN_RAW sockets need no capabilities, so
-# the service runs as an unprivileged system user.
-- name: Create open3e system user
-  ansible.builtin.user:
-    name: open3e
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-# Owned by the service user: udsoncan writes its log file into the
-# working directory.
-- name: Create open3e directory
-  ansible.builtin.file:
-    path: "{{ edge_base_dir }}/open3e"
-    state: directory
-    owner: open3e
-    group: open3e
-    mode: "0755"
-
-- name: Install open3e into a venv
-  ansible.builtin.pip:
-    name: "{{ edge_open3e_pkg }}"
-    virtualenv: "{{ edge_base_dir }}/open3e/venv"
-    virtualenv_command: python -m venv
-  notify: Restart open3e
-
-- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
-  ansible.builtin.copy:
-    src: open3e-vx3plus.py
-    dest: "{{ edge_base_dir }}/open3e/vx3plus.py"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e devices.json
-  ansible.builtin.copy:
-    content: "{{ edge_open3e_devices | to_nice_json }}\n"
-    dest: "{{ edge_base_dir }}/open3e/devices.json"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e environment (broker credential)
-  ansible.builtin.template:
-    src: open3e.env.j2
-    dest: "{{ edge_base_dir }}/open3e/open3e.env"
-    owner: open3e
-    mode: "0600"
-  notify: Restart open3e
-
-- name: Deploy open3e systemd unit
-  ansible.builtin.template:
-    src: open3e.service.j2
-    dest: /etc/systemd/system/open3e.service
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Enable open3e
-  ansible.builtin.systemd:
-    name: open3e
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- E3onCAN: direct E380 meter reads (passive; per-review decision not
-# to trust the VX3's relayed meter values). Same broker user as open3e;
-# topics land under the same open3e/ prefix as e380_<did>_<name>.
-- name: Check out E3onCAN (pinned)
-  ansible.builtin.git:
-    repo: "{{ edge_e3oncan_repo }}"
-    dest: "{{ edge_base_dir }}/e3oncan/src"
-    version: "{{ edge_e3oncan_version }}"
-  notify: Restart e3oncan
-
-- name: Create E3onCAN venv with requirements
-  ansible.builtin.pip:
-    requirements: "{{ edge_base_dir }}/e3oncan/src/requirements.txt"
-    virtualenv: "{{ edge_base_dir }}/e3oncan/venv"
-    virtualenv_command: python -m venv
-  notify: Restart e3oncan
-
-- name: Deploy e3oncan systemd unit
-  ansible.builtin.template:
-    src: e3oncan.service.j2
-    dest: /etc/systemd/system/e3oncan.service
-    mode: "0644"
-  notify: Restart e3oncan
-
-- name: Enable e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    enabled: true
-    state: started
-    daemon_reload: true

ansible/roles/edge-host/templates/docker-compose.yml.j2

@@ -1,56 +0,0 @@
-# Managed by ansible (roles/edge-host). Mirrors the former k8s manifests:
-# kubernetes/apps/home-assistant-zwavejs/deployment.yaml and
-# kubernetes/apps/home-assistant/sml2mqtt-deployment.yaml.
-#
-# No `privileged` needed here: docker's --device handles the device-cgroup
-# grant that hostPath in k8s could not (that was the only reason the pods
-# ran privileged).
-# Both services use host networking: real host addresses (v4+v6), no
-# bridge, no NAT. The broker LB is IPv6-first and docker bridge NAT66 is
-# a tarpit (unmasqueraded SYNs die silently); the host stack just works.
-services:
-  zwave-js:
-    image: {{ edge_zwavejs_image }}
-    container_name: zwave-js
-    restart: unless-stopped
-    stop_signal: SIGINT
-    tty: true
-    environment:
-      - TZ={{ edge_tz }}
-      # Batch the value-DB fsync storm to spare the eMMC (see role tasks /
-      # README "Logging & eMMC"). Neutralizes fsync only for this container;
-      # the WS/MQTT value stream to HA and cache persistence are unchanged.
-      - LD_PRELOAD=/usr/lib/libeatmydata.so
-    env_file:
-      - zwavejs.env
-    devices:
-      - "{{ edge_zwave_device }}:/dev/zwave"
-    volumes:
-      # musl libeatmydata.so extracted from the image's Alpine base (ro)
-      - ./zwave-js/libeatmydata.so:/usr/lib/libeatmydata.so:ro
-      # Persistent store: settings.json, device DB, network keys. Seeded
-      # from the k8s PVC `zwavejs-store` at cutover (see README). Since
-      # this host is stateful there is no init-container merge step —
-      # settings.json is edited via the UI and persists here; zwave.port
-      # stays /dev/zwave.
-      - ./zwave-js/store:/usr/src/app/store
-    # Host networking: binds 3000 (Z-Wave JS websocket for HA) and 8091
-    # (web UI) directly on the host.
-    network_mode: host
-
-  sml2mqtt:
-    image: {{ edge_sml2mqtt_image }}
-    container_name: sml2mqtt
-    restart: unless-stopped
-    # Outbound-only (serial in, MQTT out) — host networking sidesteps the
-    # docker NAT66 gap: the broker LB is IPv6-first and bridge-network v6
-    # SYNs leave unmasqueraded and die. The host stack reaches fd99::81
-    # fine (open3e proves it continuously).
-    network_mode: host
-    # Serial exclusivity: this is the ONLY reader of the meter port —
-    # HA's edl21 integration stays disabled (see sml2mqtt-config comment
-    # history in the k8s repo).
-    devices:
-      - "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
-    volumes:
-      - ./sml2mqtt/config.yml:/sml2mqtt/config.yml:ro

ansible/roles/edge-host/templates/docker-daemon.json.j2

@@ -1,3 +0,0 @@
-{
-  "log-driver": "{{ edge_docker_log_driver }}"
-}

ansible/roles/edge-host/templates/e3oncan.service.j2

@@ -1,32 +0,0 @@
-# Managed by ansible (roles/edge-host).
-# PASSIVE listener for the E380 energy meter's CANopen broadcasts — no
-# writes on the bus, so it may run alongside the (single) open3e
-# instance. Reads the meter directly instead of trusting the VX3's
-# relayed values.
-[Unit]
-Description=E3onCAN E380 energy-meter CAN listener (read-only)
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker e3oncan publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/e3oncan
-# Journald-friendly (python block-buffers pipes; lost on restarts).
-Environment=PYTHONUNBUFFERED=1
-ExecStart={{ edge_base_dir }}/e3oncan/venv/bin/python {{ edge_base_dir }}/e3oncan/src/E3onCANcollect.py \
-  -c {{ edge_can_interface }} \
-  -dev e380 \
-  -g {{ edge_e3oncan_gap_seconds }} \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser {{ edge_open3e_mqtt_user }} \
-  -mpass ${OPEN3E_MQTT_PASSWORD} \
-  -mfstr "{{ edge_e3oncan_mqtt_format }}" \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/edge-host/templates/open3e.env.j2

@@ -1,7 +0,0 @@
-# Managed by ansible (roles/edge-host). Read by open3e.service and
-# e3oncan.service.
-# Full user:password pair as one variable so the open3e unit can pass it
-# as a single -muser argument without exposing it in the unit file;
-# e3oncan takes user and password separately (-muser/-mpass).
-OPEN3E_MUSER={{ edge_open3e_mqtt_user }}:{{ edge_open3e_mqtt_password }}
-OPEN3E_MQTT_PASSWORD={{ edge_open3e_mqtt_password }}

ansible/roles/edge-host/templates/open3e.service.j2

@@ -1,39 +0,0 @@
-# Managed by ansible (roles/edge-host).
-# Read-mostly listener — no -w flag, no service 0x77. The ONE sanctioned
-# write path is the MQTT command topic (-l): HA publishes the DHW
-# one-time-charge command there (scope decision 2026-07-02, see vican
-# CLAUDE.md). Anything beyond that single write gets its own review.
-# -j is load-bearing: without it open3e flattens JSON payloads into
-# per-key subtopics and the HA package (homeassistant repo,
-# packages/open3e.yaml) receives nothing.
-# Exactly ONE open3e instance may be on the bus at a time (CAN collisions
-# stop open3e) — stop this service before running ad-hoc scans.
-[Unit]
-Description=open3e Viessmann E3 CAN-to-MQTT bridge (read-only)
-# Tie the service to the CAN device: stops on unplug, starts on plug via
-# the device unit rather than polling.
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker open3e publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/open3e
-ExecStart={{ edge_base_dir }}/open3e/venv/bin/open3e \
-  -c {{ edge_can_interface }} \
-  -cnfg {{ edge_base_dir }}/open3e/devices.json \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser ${OPEN3E_MUSER} \
-  -mcid open3e-edge \
-  -mfstr "{{ edge_open3e_mqtt_format }}" \
-  -r {{ edge_open3e_datapoints }} \
-  -t {{ edge_open3e_poll_seconds }} \
-  -l {{ edge_open3e_mqtt_topic }}/cmnd \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/edge-log-shipper/defaults/main.yml

@@ -0,0 +1,15 @@
+---
+# Journal -> Loki shipping.
+#
+# The srv-network Loki (loki.srv.renner.uno:3100) no longer exists — it moved
+# into the cluster. The only endpoint these hosts can reach is the authenticated
+# cluster ingress. edge_loki_password comes from the vault.
+
+edge_alloy_config_file: /etc/alloy/edge.alloy
+edge_alloy_http_port: 12345
+edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
+edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
+edge_loki_username: edge-1
+edge_alloy_labels:
+  environment: production
+  datacenter: homelab

ansible/roles/edge-log-shipper/handlers/main.yml

@@ -0,0 +1,6 @@
+---
+- name: Restart alloy
+  ansible.builtin.systemd:
+    name: edge-alloy
+    state: restarted
+    daemon_reload: true

ansible/roles/edge-log-shipper/tasks/main.yml

@@ -0,0 +1,57 @@
+---
+# Ship the systemd journal to Loki with Grafana Alloy.
+#
+# Pairs with edge-flash-wear, which makes the journal volatile: container logs
+# reach the journal via docker s journald driver and native services log to it
+# directly, so one journal source covers everything. Alloy s storage/WAL sits
+# on /run (tmpfs) so it does not write to flash either.
+
+- name: Install grafana-alloy
+  ansible.builtin.package:
+    name: grafana-alloy
+    state: present
+
+- name: Create alloy system group
+  ansible.builtin.group:
+    name: alloy
+    system: true
+
+- name: Create alloy system user
+  ansible.builtin.user:
+    name: alloy
+    group: alloy
+    groups: systemd-journal
+    system: true
+    create_home: false
+    shell: /usr/bin/nologin
+
+- name: Create alloy config directory
+  ansible.builtin.file:
+    path: /etc/alloy
+    state: directory
+    owner: alloy
+    group: alloy
+    mode: "0750"
+
+- name: Deploy alloy config (journal -> Loki)
+  ansible.builtin.template:
+    src: edge-alloy.alloy.j2
+    dest: "{{ edge_alloy_config_file }}"
+    owner: alloy
+    group: alloy
+    mode: "0600"   # carries the Loki basic-auth password
+  notify: Restart alloy
+
+- name: Deploy alloy systemd unit
+  ansible.builtin.template:
+    src: edge-alloy.service.j2
+    dest: /etc/systemd/system/edge-alloy.service
+    mode: "0644"
+  notify: Restart alloy
+
+- name: Enable edge-alloy
+  ansible.builtin.systemd:
+    name: edge-alloy
+    enabled: true
+    state: started
+    daemon_reload: true

ansible/roles/edge-log-shipper/templates/edge-alloy.alloy.j2

@@ -1,4 +1,4 @@
-// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-host.
+// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-log-shipper.
 //
 // edge-1 runs on internal eMMC, so logs are kept out of local flash: the
 // systemd journal is volatile (RAM, see journald-edge.conf) and container
@@ -7,8 +7,8 @@
 // Loki. Its own WAL/positions live on /run (tmpfs, see edge-alloy.service),
 // so this agent writes nothing to the eMMC either.
 //
-// The srv-network Loki (loki.srv.renner.uno:3100, unauthenticated) is not
-// reachable from the IoT VLAN; the only reachable endpoint is the cluster
+// The srv-network Loki (loki.srv.renner.uno:3100) no longer exists — it moved
+// into the cluster. The only reachable endpoint is the cluster
 // ingress, which enforces Traefik basic-auth — hence the credential below.
 
 loki.relabel "journal" {