perf(grafana-db): raise archive_timeout to 30min on gf-pg

A WAL segment is a fixed 16 MiB file. archive_timeout forces a segment
switch on a timer so the archive never trails reality by more than that
interval, and the switched-out segment is written and archived at its full
16 MiB no matter how few bytes of real WAL it holds. CNPG defaults this to
5min, which permits up to 288 segments/day = 4.6 GB/day per cluster.

Measured on gf-pg: the database is 21 MB and generates 4.5 MB/day of actual
WAL, while its RBD volume takes 211 MB/hour -- against 192 MB/hour predicted
from 12 switches of 16 MiB. Essentially every byte written to this volume is
padding, roughly 1100x amplification.

This resolves an anomaly that has been open since the start of the endurance
work: gf-pg appeared to write gigabytes per day for a 21 MB database, and
neither WAL volume nor data changes could account for it. Segment padding
does, exactly.

Applied to gf-pg alone as a staged rollout. It is the lowest-risk cluster
and gives the cleanest signal -- its real WAL is small enough that any change
in volume is unambiguously attributable to this parameter. The same default
affects all seven clusters and is worth an estimated ~24 GB/day of logical
writes fleet-wide, but that only goes ahead once this one is verified.

The cost is RPO. On total loss of the cluster, up to 30min of Grafana state
becomes unrecoverable instead of 5min. That state is dashboards, annotations
and preferences, changing at ~4.5 MB/day, so the window is nearly always
empty. Clusters holding credentials or job state get evaluated separately
rather than inheriting this value by default.

archive_timeout is in CNPG's GlobalDefaultSettings, not its
FixedConfigurationParameters, so it is user-overridable -- verified against
pkg/postgres/configuration.go rather than assumed.

Context: drive-endurance work on three QLC NVMe OSDs at 81-87% wear.

.woodpecker/alert-monitor.yml

@@ -42,7 +42,7 @@ when:
 
 steps:
   - name: fetch-alerts
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key
@@ -135,8 +135,38 @@ steps:
         TITLES=$$(jq -c '[.[].title]' .ci-alert-monitor/open-issues.json)
         NOISE_BODY=$$(jq -r 'map(select(.title == "Alert monitor: noise log")) | .[0].body // ""' \
           .ci-alert-monitor/open-issues.json)
-        FPS=$$(printf '%s' "$$NOISE_BODY" | grep -oE '\b[0-9a-f]{16}\b' \
+        # Suppression entries are exactly the structured lines the write-back
+        # step emits: "- <fp> <ALERTNAME> — <YYYY-MM-DD>: <reason>". Parse THAT
+        # shape rather than grepping the whole body for 16-hex tokens. A reason
+        # routinely cites another alert's fingerprint in prose ("see the issue
+        # filed for fingerprint <fp>"), and the body-wide grep turned every such
+        # citation into a silent, permanent suppression of an alert that was
+        # never classified as noise — verified against the live noise log, which
+        # yielded 22 fingerprints for 21 real entries, the extra one being
+        # 2ddc41edb689fc33, an alert that HAD been filed as a genuine issue.
+        #
+        # Entries also expire after NOISE_TTL_DAYS. A noise verdict says "this
+        # occurrence was benign", not "this alert can never be interesting
+        # again" — but the fingerprint is sha256(alertname|scope) and therefore
+        # stable across recurrences, so without an expiry a single blip mutes
+        # that alertname+scope pair forever. Re-triaging a recurrence weeks
+        # later is cheap; missing a real outage because a July blip muted
+        # openbao's KubePodNotReady is not.
+        #
+        # Lines that fail to parse are simply dropped, i.e. they stop
+        # suppressing — this fails toward MORE alerting, never less.
+        #
+        # date is BusyBox here, not GNU: `date -d '30 days ago'` errors out, so
+        # the cutoff goes through epoch arithmetic and `date -d @<epoch>`, which
+        # this image does support.
+        NOISE_TTL_DAYS=$${NOISE_TTL_DAYS:-30}
+        CUTOFF=$$(date -u -d @$$(( $$(date -u +%s) - NOISE_TTL_DAYS * 86400 )) +%F)
+        FPS=$$(printf '%s\n' "$$NOISE_BODY" \
+          | sed -nE 's/^- ([0-9a-f]{16}) [^ ]+ — ([0-9]{4}-[0-9]{2}-[0-9]{2}):.*/\2 \1/p' \
+          | awk -v c="$$CUTOFF" 'BEGIN{gsub(/-/,"",c)} { d=$$1; gsub(/-/,"",d); if (d+0 >= c+0) print $$2 }' \
           | jq -R -s -c 'split("\n") | map(select(length > 0))')
+        echo "Noise suppressions in effect: $$(printf '%s' "$${FPS:-[]}" | jq length)" \
+             "(TTL $${NOISE_TTL_DAYS}d, entries dated >= $$CUTOFF)"
         # Synthesize pseudo-alerts for BLIPS: lookback series whose label set
         # is not currently firing at all (checked against the PRE-filter raw
         # snapshot, so live-but-null-routed alerts don't reappear as blips).
@@ -260,7 +290,7 @@ steps:
               '{body: ($$old + "\n" + $$add)}' > /tmp/gitea-patch.json
             gitea_write PATCH "$$API/issues/$$NOISE_NUM" /tmp/gitea-patch.json
           else
-            printf '%s\n\n' "Fingerprints the alert-monitor triage agent classified as noise. The dedup step skips any alert whose fingerprint appears below — delete a line to make that alert eligible for triage again. Keep this issue open; it is the suppression state." > /tmp/noise-header.txt
+            printf '%s\n\n' "Fingerprints the alert-monitor triage agent classified as noise. The dedup step skips any alert whose fingerprint appears below — delete a line to make that alert eligible for triage again. Keep this issue open; it is the suppression state. Entries EXPIRE after NOISE_TTL_DAYS (default 30) counted from the date in the line, so a one-off blip stops suppressing its alertname+scope pair once the window passes; re-add happens automatically if it is triaged as noise again. Only lines in the exact '- <fingerprint> <ALERTNAME> — <YYYY-MM-DD>: <reason>' form suppress anything — a fingerprint mentioned in prose does not." > /tmp/noise-header.txt
             jq -n --rawfile hdr /tmp/noise-header.txt --rawfile add /tmp/noise-lines.txt \
               '{title: "Alert monitor: noise log", body: ($$hdr + $$add)}' > /tmp/gitea-create.json
             gitea_write POST "$$API/issues" /tmp/gitea-create.json

.woodpecker/build-ci-images.yml

@@ -217,3 +217,32 @@ steps:
           --dockerfile Containerfile \
           --destination gitea.renner.uno/gitea_admin/alpine-openssl:$TAG \
           --destination gitea.renner.uno/gitea_admin/alpine-openssl:latest
+
+  # Heating CAN readers: open3e (Vitocal/VX3) + E3onCAN (E380 meter), one image
+  # run as two services on edge-1. Replaces pip-into-venv on the host, where an
+  # unpinned VCS requirement re-cloned and restarted the heating bridge on every
+  # Ansible converge. Not a ci-* image -- it is a runtime artifact -- but it
+  # builds the same way and lands in the same registry so the audit sees it.
+  - name: build-heating-can
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/heating-can
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/heating-can/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
+          --destination gitea.renner.uno/gitea_admin/heating-can:latest

.woodpecker/ci-manifests-test.yml

@@ -0,0 +1,260 @@
+# Validate the GitOps tree and the pipeline definitions on PRs that touch them.
+#
+# Nothing covered kubernetes/** before this. deploy-infrastructure.yml only
+# gates opentofu/**, and ci-scripts-test.yml only runs scripts/ci/ unit tests —
+# so a broken manifest reached the cluster and was caught by Flux failing to
+# reconcile, or by a human noticing. Three separate defects on 2026-07-30 would
+# have been caught here instead:
+#
+#   * an unquoted YAML scalar containing ": " in .woodpecker/build-ci-images.yml,
+#     which made EVERY pipeline in the repo fail to compile (#443). `yq` parsed
+#     it happily — it is valid YAML, just a map instead of a string — so
+#     "it parses" was mistaken for "it is valid". Only woodpecker lint catches it.
+#   * a rotator pinned to an image digest that no longer existed (#458).
+#   * manifests whose kustomize wiring was only ever checked by hand.
+#
+# Deliberately CHEAP and offline: no cluster, no registry, no secrets. Everything
+# here is a static check that either passes or fails on the checkout alone, so it
+# can gate PRs without becoming a bottleneck or a flake source.
+when:
+  # Path-filtered on purpose, unlike ci-scripts-test.yml. That pipeline is one
+  # step on one already-cached image, so running it everywhere is free; this one
+  # pulls three images and builds every kustomization in the tree, which is real
+  # cost to put on a docs-only PR. The filter covers everything the steps actually read:
+  # kubernetes/** for the manifests and scripts, .woodpecker/** for the lint (and
+  # so an edit to this file re-runs its own checks).
+  - event: pull_request
+    path:
+      - "kubernetes/**"
+      - ".woodpecker/**"
+
+# ADDING A THIRD DISCOVERY CHECK: kustomize-build and shellcheck both hand-roll
+# the same shape — `find`, a floor guard against empty discovery, then a
+# file-fed `while read`. Two copies is deliberate: the two traps being commented
+# around (BusyBox find lacking `-printf`, and a piped loop running in a subshell
+# where `fail` cannot propagate) are different, and both are worth spelling out
+# where they bite. A third instance is the point to stop copying and factor it
+# into a shared script under `scripts/ci/` — which is already covered by
+# ci-scripts-test.yml, so it would gain unit tests in the process.
+#
+# Every step carries `depends_on: []` so all four run in PARALLEL and every
+# failure shows in one run. Woodpecker's default is sequential, which would stop
+# at the first failure and make a contributor fix-and-repush to discover the
+# next — and these four are independent, cheap, offline checks with no ordering
+# between them. Keep it on every step: `depends_on` is per-step, so dropping it
+# from one silently re-serialises that step behind the others.
+steps:
+  # Every kustomization must build. This is the check that catches a resource
+  # entry pointing at a file that does not exist, a bad patch target, or YAML
+  # that is fine in isolation but broken once overlaid — the class of error that
+  # otherwise surfaces as a Flux reconcile failure after merge.
+  #
+  # Verified green across every kustomization in the tree when this was added,
+  # so this gates new breakage rather than pre-existing debt.
+  #
+  # ci-k8s TAG SPLIT: this file pins a newer dated tag than e2e.yml, e2e-test.yml,
+  # alert-monitor.yml and issue-fix.yml, which still sit on an older one. That is
+  # staleness in those four, not a deliberate split here — everything should be on
+  # the newest dated tag. They are not bumped here because none of them are
+  # triggered by this PR, so it would ship untested changes under cover of a green
+  # run that never touched them. Tracked in #461.
+  - name: kustomize-build
+    depends_on: [] # see note above `steps:`
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
+    commands:
+      - |
+        set -eu
+        # NOT `find -printf '%h\n'`: that is a GNU findutils extension and this
+        # image ships BusyBox find (v1.37.0), which rejects it outright with
+        # "find: unrecognized: -printf". Because the result feeds a command
+        # substitution used as a word list, the failure is SILENT — empty output,
+        # zero loop iterations, fail=0, step exits green having validated
+        # nothing. That is exactly what this pipeline shipped in its first
+        # revision, and CI reported it as passing.
+        dirs=$(find kubernetes -name kustomization.yaml | sed 's|/kustomization.yaml$||' | sort)
+
+        # Assert the discovery actually found something. Without this, any future
+        # breakage of the find/sed above degrades back into a green no-op rather
+        # than a failure — the specific trap that made the first revision of this
+        # step useless while looking healthy.
+        count=$(printf '%s\n' "$dirs" | grep -c . || true)
+        if [ "$count" -lt 50 ]; then
+          echo "FAIL: found only $count kustomizations; floor is 50."
+          echo "      If the tree legitimately shrank, lower this floor in"
+          echo "      .woodpecker/ci-manifests-test.yml — do not delete it."
+          echo "      Otherwise discovery is broken — refusing to report success."
+          exit 1
+        fi
+        echo "discovered $count kustomizations"
+
+        # `while read` fed from a FILE, not a pipe: a pipe would run the loop in
+        # a subshell where `fail` cannot propagate, so a failing build would exit
+        # 0 — the same silent-pass shape this step already shipped once. Reading
+        # a file keeps the loop in the current shell, and read -r handles paths
+        # containing spaces without IFS games.
+        printf '%s\n' "$dirs" > /tmp/dirs
+        fail=0
+        while IFS= read -r d; do
+          [ -n "$d" ] || continue
+          if kubectl kustomize "$d" >/dev/null 2>/tmp/err; then
+            echo "ok    $d"
+          else
+            echo "FAIL  $d"
+            sed 's/^/        /' /tmp/err
+            fail=1
+          fi
+        done < /tmp/dirs
+        exit $fail
+
+  # Lint the pipeline definitions themselves. This is the one that would have
+  # prevented the repo-wide CI outage: `woodpecker lint` exits 1 on a config that
+  # fails to compile, and exits 0 on mere warnings — so it fails on real breakage
+  # without blocking on pre-existing noise. Both behaviours were verified against
+  # this repo (rc=1 reproducing #443, rc=0 on the tree as it stands).
+  #
+  # An earlier revision of this comment also asserted that several deploy-*.yml
+  # files emit "Additional property timeout is not allowed" warnings. That was
+  # observed under the v3.9.0 CLI this step used to pin, and it is NOT known to
+  # still hold on current CLIs — review raised the likelihood that the warning was
+  # schema drift in a 7-minor-old CLI rather than real debt in those files, which
+  # is one of the reasons this pin moved to latest stable. It cannot be confirmed
+  # from a green run either, because the loop below sends lint output to /tmp/out
+  # and prints it only on failure, so warnings are invisible when the step passes.
+  # The claim is dropped rather than restated: what this step guarantees is the
+  # exit-code contract above, which is what the gate actually rests on.
+  # Latest stable, per the standing rule that any software uses the latest
+  # stable version. Matching the deployed woodpecker-server is a CONSEQUENCE of
+  # that, not a separate requirement — the server is bumped by image-automation
+  # within the 3.x range, so both should land on the same release.
+  #
+  # They do not land together, though: the server is a Kubernetes manifest and
+  # is bumped automatically, this pipeline file is not and cannot be (see
+  # infrastructure/image-automation/README.md). So the server moves first and
+  # this trails until someone bumps it by hand — which is what happened at
+  # v3.16.0 -> v3.17.0, caught by the update audit (#426) rather than by CI.
+  #
+  # It matters here because the CLI compiles the config against its own schema:
+  # one behind the server can pass a config the server then rejects, or reject
+  # one it accepts, and the gate stops predicting what the server will do —
+  # which is its only purpose.
+  #
+  # `-alpine` is required, not cosmetic: the plain vX.Y.Z CLI image ships no
+  # shell, and the loop below needs `for`, `sed` and `echo` around the binary.
+  # A step using the plain tag fails to start rather than failing a check.
+  #
+  # DRIFT RISK — accepted, and NOT fixable here. The server auto-bumps via
+  # image-automation; this pin cannot, so the two will drift apart.
+  # Image-automation can only patch YAML that is a Kubernetes manifest —
+  # fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the
+  # YAML manifest must have `apiVersion` and `kind` fields." A Woodpecker
+  # pipeline file has neither, so an `$imagepolicy` marker here is silently
+  # ignored. That is a property of the FILE, not of `update.path`: widening the
+  # automation's path to `./` would buy nothing, and would additionally drag the
+  # whole ansible/ and esphome/ trees into the walk. Do not try it — it looks
+  # like the fix and is not.
+  #
+  # So this bump is manual, and drift is caught by the update-audit agent
+  # (ci/agents/update-audit/prompt.md, item 5) which is told to flag this image
+  # whenever it trails the deployed server. If you bump the server, bump here.
+  - name: woodpecker-lint
+    depends_on: [] # see note above `steps:`
+    image: woodpeckerci/woodpecker-cli:v3.17.0-alpine
+    commands:
+      - |
+        set -eu
+        fail=0
+        for f in .woodpecker/*.yml; do
+          if /bin/woodpecker-cli lint "$f" >/tmp/out 2>&1; then
+            echo "ok    $f"
+          else
+            echo "FAIL  $f"
+            sed 's/^/        /' /tmp/out
+            fail=1
+          fi
+        done
+        exit $fail
+
+  # Standalone shell scripts under kubernetes/ (sidecar and bootstrap scripts
+  # mounted via configMapGenerator). Severity is pinned to `error` on purpose:
+  # style and info findings would fail the build on existing scripts for no
+  # safety gain. All scripts in the tree are clean at this level today.
+  #
+  # NOTE this does NOT cover shell embedded in manifests (init containers, probe
+  # exec blocks) — shellcheck cannot see inside YAML. That embedded shell is
+  # where several real bugs lived this year, so it remains a gap worth closing
+  # separately rather than one this step quietly implies is covered.
+  - name: shellcheck
+    depends_on: [] # see note above `steps:`
+    image: koalaman/shellcheck-alpine:v0.11.0
+    commands:
+      - |
+        set -eu
+        files=$(find kubernetes -name '*.sh' | sort)
+        # Same no-op guard as kustomize-build: if discovery ever returns nothing,
+        # fail loudly instead of passing having checked no files.
+        #
+        # This floor is 1 while kustomize-build's is 50, and the asymmetry is
+        # deliberate rather than an oversight. That floor is loose relative to the
+        # tree it guards, so it catches both total failure and gross partial loss
+        # while tolerating normal churn. The shell-script set is small enough that
+        # any floor tight enough to catch PARTIAL loss would sit inside normal
+        # churn — deleting one script would fail CI spuriously, and the fix a
+        # hurried person reaches for is lowering the floor, which is how a guard
+        # quietly becomes decoration. A floor that has to be edited whenever the
+        # tree changes is not a guard, it is a snapshot with an exit code.
+        #
+        # So this catches the failure that actually happened here (discovery
+        # silently returning nothing, step green having checked no files) and not
+        # partial loss. Worth revisiting only if kubernetes/ grows a lot more
+        # shell, at which point a loose proportionate floor becomes possible.
+        count=$(printf '%s\n' "$files" | grep -c . || true)
+        if [ "$count" -lt 1 ]; then
+          echo "FAIL: no shell scripts found under kubernetes/ — discovery is broken."
+          exit 1
+        fi
+        echo "discovered $count shell scripts"
+
+        # Same file-fed while-read reasoning as kustomize-build above.
+        printf '%s\n' "$files" > /tmp/files
+        fail=0
+        while IFS= read -r f; do
+          [ -n "$f" ] || continue
+          if shellcheck -S error "$f"; then
+            echo "ok    $f"
+          else
+            echo "FAIL  $f"
+            fail=1
+          fi
+        done < /tmp/files
+        exit $fail
+
+  # Guard the convention that cost a day to diagnose: registry-local images are
+  # pinned by DATED TAG, never by digest. Digests here are garbage-collected once
+  # :latest and the dated tags move past them — and that happens on edits to
+  # .woodpecker/build-ci-images.yml, a shared file, so an unrelated image addition
+  # silently orphans the manifest a rotator was pinned to. 11 rotators were dead
+  # this way and would have failed on their first scheduled run months later.
+  #
+  # Scoped to the rotation directory, which is where the convention is documented
+  # and where the failure actually bit.
+  #
+  # The `grep -v kustomization.yaml` is load-bearing, not tidying: that file
+  # documents this very rule, so its comment contains the literal string
+  # `image:.*@sha256` and therefore MATCHES the pattern. Anchoring on `image:`
+  # does not prevent that — verified by running the bare grep, which returns
+  # kustomization.yaml. Only the filename filter excludes it. Drop the filter and
+  # this step fails permanently on its own documentation.
+  - name: no-digest-pins
+    depends_on: [] # see note above `steps:`
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
+    commands:
+      - |
+        set -eu
+        hits=$(grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml 2>/dev/null \
+                 | grep -v kustomization.yaml || true)
+        if [ -n "$hits" ]; then
+          echo "FAIL: digest-pinned images found; use a dated tag (see rotation/kustomization.yaml):"
+          echo "$hits" | sed 's/^/        /'
+          exit 1
+        fi
+        echo "ok    no digest pins under openbao/rotation/"

.woodpecker/e2e-test.yml

@@ -4,7 +4,7 @@ when:
 
 steps:
   - name: e2e
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key

.woodpecker/e2e.yml

@@ -30,7 +30,7 @@ steps:
       - sh scripts/ci/test-model-selection.sh
 
   - name: e2e-test
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key

.woodpecker/issue-fix.yml

@@ -49,7 +49,7 @@ steps:
     # (the issue-pick logic needs jq, which only the ci-claude image has);
     # minting a 1h read-only token per run is cheap. Degrades to
     # snapshot-only if the ci-diagnostics ServiceAccount isn't reconciled.
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
+    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
     environment:
       SOPS_AGE_KEY:
         from_secret: sops_age_key

ansible/docs/edge-host.md

@@ -33,13 +33,43 @@ Build and boot the dedicated installer image — see
    (partitions the internal disk — NVMe if present, else the eMMC; the
    installer autodetects — installs the ansible-ready base system with SSH
    keys, DHCP, docker — asks for confirmation before wiping).
-3. Add the DHCP reservation/DNS entry for `edge-1.srv.renner.uno` on the
-   UDM, confirm `ssh root@edge-1.srv.renner.uno` works.
-4. From the workstation:
+3. Confirm the box resolves and is reachable by name — `ssh root@<host>` must
+   work before anything else. The UDM registers DHCP leases in DNS, so the
+   record appears on its own; the host is identified by name everywhere
+   (inventory, plays, this file) and its address is never recorded. Do not
+   pin it to a reservation.
+4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
+   That group is the base: every edge thin client belongs to it, and it
+   assumes no peripherals.
+5. From the workstation:
    ```
-   ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host
    ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
    ```
+   One unrestricted run over the whole group. The play is self-contained --
+   it carries `certificates` and `common` itself, so there is no second
+   command and no `--limit`. Running it converges every edge host, which is
+   the point: a run that touches only the new box tells you nothing about
+   the others.
+
+## Peripherals
+
+Each attached device is its own inventory group and its own playbook. Group
+membership states which hardware a host physically has -- there are no flags
+and no `when:` on the roles, so moving a stick between boxes is a one-line
+inventory change.
+
+| device | group | playbook |
+|---|---|---|
+| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
+| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
+| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
+
+Add the host to the matching group, then run that playbook. They depend on
+the base play having run: all three need the local MQTT bridge, and two of
+them need docker.
+
+A host with nothing attached is simply absent from all three groups and needs
+none of these commands.
 
 ## Secrets (before first play run)
 
@@ -85,7 +115,7 @@ zwave-js (order matters — never two servers on one radio):
       `kubectl -n home-assistant exec deploy/zwave-js -- tar cf - -C /usr/src/app store | tar xf - -C /opt/edge-host/zwave-js/`
 - [ ] Scale down in-cluster: `kubectl -n home-assistant scale deploy/zwave-js --replicas 0`
 - [ ] Move the ZWA-2 stick to edge-1, `docker compose up -d zwave-js`
-- [ ] Point HA `zwave_js` integration at `ws://edge-1.srv.renner.uno:3000`
+- [ ] Point HA `zwave_js` integration at `ws://edge-1.iot.renner.uno:3000`
 - [ ] Nodes alive in the UI on :8091 → remove the flux kustomization
       `home-assistant-zwavejs` + PVC in a follow-up PR
 
@@ -113,8 +143,12 @@ Afterwards:
   deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
   changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
   the preloaded `.so` (see "Logging & eMMC" → zwave-js).
-- open3e updates: re-run the play; the pip task tracks the git default
-  branch (pin `edge_open3e_pkg` to a tag if that ever bites).
+- open3e / E3onCAN updates: both are pinned to commits in
+  `roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
+  up upstream changes -- that is deliberate, these drive the heat pump. The
+  weekly update-audit reports when either falls behind (see
+  `ci/agents/update-audit/prompt.md`); bump the pin and re-run.
+  Pinning to a tag is not an option for open3e: it publishes none.
 - Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
   restart afterwards.
 - can-utils (`candump`) is AUR-only on Arch — install manually via

ansible/inventory/group_vars/edge_host/vars.yml

@@ -1,6 +1,6 @@
 ---
 # Secret indirection — real values live in vault.yml (ansible-vault) next
-# to this file; see roles/edge-host/README.md for how to populate it from
+# to this file; see docs/edge-host.md for how to populate it from
 # the cluster secrets.
 edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
 edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
@@ -36,3 +36,18 @@ keepalived_ntp_sysctl:
 
 # Realtek NIC on the Wyse 3040 has no hardware timestamping.
 chrony_hwtimestamp: false
+
+# --- Local MQTT contract -------------------------------------------------
+# Shared between the broker and its clients, so it belongs to the inventory
+# rather than to either side's role. mqtt-bridge listens here; heating-can and
+# power-meter-sml publish here; and the bridge authenticates to the cluster
+# broker AS the open3e user, so it needs that identity too.
+#
+# Keeping these in one role's defaults only worked while a single play loaded
+# every role at once. Splitting the plays by capability exposed it: the bridge
+# could not render without a heating-can default, and the CAN services could
+# not render without an mqtt-bridge default.
+edge_mqtt_host: 127.0.0.1
+edge_mqtt_port: 1883
+edge_open3e_mqtt_user: open3e
+edge_open3e_mqtt_topic: open3e

ansible/inventory/group_vars/ntp_cluster.yml

@@ -0,0 +1,18 @@
+---
+# Shared configuration for every NTP server (see plays/install-ntp-ha.yml).
+
+# IPv6 service address -- what ntp.srv.renner.uno resolves to, and the path
+# clients actually use. The IPv4 VIP is legacy.
+#
+# Defined at group level rather than in the play because install-ntp-ha.yml runs
+# two plays over this group -- withdraw the static assignments fleet-wide, then
+# hand the address to keepalived -- and both need the same value. Duplicating a
+# service address across plays is exactly the sort of thing that drifts.
+ntp_vip6: "fd40::123/128"
+
+# Link-local address carried as the FIRST virtual_ipaddress of the IPv6 VRRP
+# instance. RFC 5798 gives an IPv6 virtual router a link-local primary, and
+# keepalived warns "the first IPv6 VIP address should be link local" without
+# one. fe80::123 mirrors the service address and cannot collide with a real
+# NIC's EUI-64 link-local.
+ntp_vip6_linklocal: "fe80::123/64"

ansible/inventory/pve_hosts.yml

@@ -1,7 +1,19 @@
+# Reached by name, like every other host in this file. These three were the last
+# entries still addressed by IPv4 literal; authentik, netbird, proxy, nas-1,
+# grafana and edge-1 have always used their FQDN.
+#
+# An address in an inventory is allocator state standing in for identity, and it
+# pins access to the legacy family -- the names resolve IPv6-first, which is the
+# primary path on this network. DNS is the authoritative source for where a host
+# is; this file should say which host, not where it currently sits.
+#
+# The short inventory_hostname is kept deliberately: plays key on it
+# (ntp_servers[inventory_hostname] in install-ntp-ha.yml, node names in opentofu,
+# the prometheus targets), so only ansible_host changes.
 [pve_cluster]
-jupiter ansible_host=10.2.40.201 ansible_user=root
-neptune ansible_host=10.2.40.202 ansible_user=root
-saturn ansible_host=10.2.40.203 ansible_user=root
+jupiter ansible_host=jupiter.renner.uno ansible_user=root
+neptune ansible_host=neptune.renner.uno ansible_user=root
+saturn ansible_host=saturn.renner.uno ansible_user=root
 
 [authentik_db]
 authentik-db-1.srv.renner.uno ansible_user=root
@@ -29,9 +41,36 @@ nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
 [grafana]
 grafana.srv.renner.uno ansible_user=ladmin
 
+# Every bare-metal edge thin client. Gets the host-generic roles only:
+# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
+# sub-interface. Nothing here assumes any peripheral is attached.
 [edge_host]
 edge-1.iot.renner.uno ansible_user=root
 
+# Capability groups. Membership states which hardware a host physically has,
+# and each service playbook targets the group for its own concern. No flags and
+# no `when:` on the roles -- moving a stick to another box is a one-line change
+# here, which is what it should be.
+#
+# These are children of [edge_host], declared below: a peripheral is added to an
+# edge client, it does not replace the base. Making that a :children relation
+# rather than a comment means a host added only to a capability group still
+# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
+# cannot silently come up without edge_mqtt_host and friends.
+[zwave_gateway]
+edge-1.iot.renner.uno
+
+[heating_can]
+edge-1.iot.renner.uno
+
+[power_meter]
+edge-1.iot.renner.uno
+
+[edge_host:children]
+zwave_gateway
+heating_can
+power_meter
+
 # NTP servers. Includes edge-1 so the role survives the PVE hosts being
 # retired for bare-metal Talos, which cannot run chrony or keepalived itself.
 # See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly

ansible/plays/install-blackbox-exporter.yml

@@ -1,19 +0,0 @@
----
-# Deploy Blackbox Exporter to Prometheus server for endpoint probing
-# Usage: ansible-playbook -i inventory/ plays/install-blackbox-exporter.yml
-
-- name: Install Blackbox Exporter
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  roles:
-    - blackbox-exporter
-
-  post_tasks:
-    - name: Verify blackbox_exporter is running
-      uri:
-        url: "http://127.0.0.1:9115/metrics"
-        status_code: 200
-      register: exporter_health
-      retries: 3
-      delay: 5
-      until: exporter_health.status == 200

ansible/plays/install-edge-host.yml

@@ -1,13 +1,41 @@
 ---
-# Bare-metal Arch Linux edge host (Dell Wyse 3040, internal eMMC) — replaces
-# the k8s-edge Talos VM
-# (opentofu vm_153) as the home for hardware-bound services:
-#   - zwave-js-ui  (ZWA-2 Z-Wave USB radio, docker compose)
-#   - sml2mqtt     (FTDI power-meter reader, docker compose)
-#   - open3e       (Innomaker USB2CAN -> Vitocal/VX3, native venv + systemd)
-# Base install procedure and cutover checklist: ansible/roles/edge-host/README.md
-- name: Install Arch edge host
+# Bare-metal Arch edge hosts (Dell Wyse 3040 and similar) — the home for
+# hardware-bound services, because Talos supports neither SocketCAN nor USB
+# passthrough.
+#
+# This play is the BASE only: what every edge thin client gets regardless of
+# what is plugged into it. The peripherals live in their own playbooks against
+# their own inventory groups:
+#
+#   plays/install-zwave-gateway.yml   [zwave_gateway]   ZWA-2 radio
+#   plays/install-heating-can.yml     [heating_can]     Innomaker USB2CAN
+#   plays/install-power-meter.yml     [power_meter]     FTDI meter head
+#
+# Splitting the roles without splitting the plays would have kept the coupling:
+# one play applying every role to one group is the monolith again, a layer up.
+# A host with no peripherals is simply absent from those groups -- no flags, no
+# `when:` on the roles.
+#
+# Self-contained: `certificates` and `common` are the parts of plays/common.yml
+# that apply to an edge host, carried here so bootstrapping one is a single
+# unrestricted run over the group. Not carried over: qemu-guest-agent (bare
+# metal, not VMs) and monitoring-agent (Debian-oriented; these hosts ship logs
+# via edge-log-shipper).
+#
+# Order is dependency order: edge-flash-wear before docker, so the journal is
+# volatile before containers start logging into it.
+- name: Install Arch edge host (base)
   hosts: edge_host
+  # Connection is already root (ansible_user=root), so this is a root-to-root
+  # no-op today. Kept explicit, matching plays/common.yml, so the roles below
+  # do not silently depend on the connection user staying privileged.
+  become: true
+
   roles:
+    - certificates
+    - common
+    - edge-flash-wear
     - docker-install
-    - edge-host
+    - edge-log-shipper
+    - mqtt-bridge
+    - edge-vlan

ansible/plays/install-heating-can.yml

@@ -0,0 +1,18 @@
+---
+# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
+# USB2CAN adapter attached.
+#
+# Targets [heating_can]: membership means the adapter is physically present.
+# Two roles because they are two concerns -- the interface is host network
+# state, the services consume it -- but one group, because both need the same
+# hardware.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: these
+# services publish to the local MQTT bridge it sets up.
+- name: Install heating CAN services
+  hosts: heating_can
+  become: true
+
+  roles:
+    - socketcan
+    - heating-can

ansible/plays/install-log-aggregation.yml

@@ -1,86 +0,0 @@
----
-# Setup Log Aggregation Infrastructure
-# This playbook configures the complete logging stack:
-# 1. Loki server for log storage and querying
-# 2. Alloy agents on all VMs/CTs for log collection
-# 3. Syslog receiver for remote devices (Synology NAS, network devices)
-#
-# Usage: ansible-playbook -i inventory/ plays/setup-log-aggregation.yml
-#
-# To limit to specific hosts:
-#   --limit loki.srv.renner.uno          # Only configure Loki server
-#   --limit proxmox_all_running          # Only configure log agents
-
-- name: Setup Loki Log Aggregation Server
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
-
-- name: Deploy Monitoring Agent to Loki Server (with syslog receiver)
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-    # Syslog receiver settings are defined in host_vars/loki.srv.renner.uno/vars.yml:
-    # - alloy_syslog_receiver_enabled: true
-    # - alloy_syslog_receiver_port: 1514
-    # - alloy_syslog_receiver_protocol: "udp"
-    # - alloy_syslog_receiver_format: "raw"
-    # - alloy_stability_level: "experimental"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Display syslog receiver info
-      debug:
-        msg: |
-          Syslog receiver is listening on UDP port {{ alloy_syslog_receiver_port | default(1514) }}
-          Configure remote devices to send syslog to: {{ inventory_hostname }}:{{ alloy_syslog_receiver_port | default(1514) }}
-      when: alloy_syslog_receiver_enabled | default(false)
-
-- name: Deploy Monitoring Agent to All Running VMs/CTs
-  hosts: proxmox_all_running:!proxmox_logging:!proxmox_noansible
-  become: yes
-  gather_facts: yes
-  serial: 10  # Deploy to 10 hosts at a time to avoid overwhelming the network
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Verify Alloy is running
-      systemd:
-        name: alloy
-        state: started
-      register: alloy_service
-
-    - name: Display agent status
-      debug:
-        msg: "Monitoring agent deployed successfully on {{ inventory_hostname }}"
-      when: alloy_service is defined

ansible/plays/install-loki.yml

@@ -1,26 +0,0 @@
----
-# Install Loki Log Aggregation Server
-# Usage: ansible-playbook -i inventory/ plays/install-loki.yml
-
-- name: Install Loki Log Aggregation Server
-  hosts: loki.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"

ansible/plays/install-ntp-ha.yml

@@ -22,12 +22,54 @@
 # so it stays BACKUP while the PVE nodes exist and wins the VIP by default once
 # they are gone -- no config change required at cutover.
 
-- name: Install NTP Server (HA)
+# Phase 1. Every static assignment of the IPv6 service address must be gone
+# fleet-wide before ANY host claims it via VRRP.
+#
+# DAD is enabled on the VRRP address (deliberately -- under single-master VRRP a
+# duplicate is a real fault worth detecting). So a static copy on one host and a
+# VRRP-managed copy on another cannot coexist: the kernel marks the newcomer
+# `dadfailed deprecated`, and its master then believes it is serving while
+# carrying an unusable address. That is not hypothetical -- it is what edge-1
+# did on 2026-08-07 when it took the address while three PVE nodes still held it
+# statically.
+#
+# Deliberately NOT serial: this play must complete everywhere before phase 2
+# starts anywhere. Doing the withdrawal per host inside phase 2 cannot avoid the
+# overlap, because with serial: 1 the first host claims the address while the
+# rest are still static.
+#
+# There is a gap between the phases where no host holds the address. It is
+# bounded by one play's run time and clients simply retry.
+- name: Withdraw static assignments of the NTP service address
   hosts: ntp_cluster
   become: true
 
+  vars:
+    keepalived_ntp_vip6: "{{ ntp_vip6 }}"
+    keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
+
+  tasks:
+    - name: Withdraw any static assignment of the service address
+      ansible.builtin.include_role:
+        name: keepalived-ntp
+        tasks_from: withdraw-static.yml
+      when: ntp_keepalived_enabled | default(false)
+
+# Phase 2. Configure chrony and hand the address to keepalived.
+- name: Install NTP Server (HA)
+  hosts: ntp_cluster
+  become: true
+  # One node at a time. The keepalived role withdraws the statically assigned
+  # IPv6 service address and restarts keepalived so the address is re-taken
+  # under VRRP control; doing that across every NTP server at once would leave
+  # the whole fleet with nobody holding fd40::123 until the elections settle.
+  # Serialised, the remaining members keep answering throughout.
+  serial: 1
+
   vars:
     ntp_vip: "10.2.40.4"
+    # ntp_vip6 / ntp_vip6_linklocal come from group_vars/ntp_cluster.yml, so
+    # both plays in this file see one definition.
     ntp_servers:
       jupiter:
         priority: 100
@@ -49,6 +91,8 @@
       when: ntp_keepalived_enabled | default(false)
       vars:
         keepalived_ntp_vip: "{{ ntp_vip }}"
+        keepalived_ntp_vip6: "{{ ntp_vip6 }}"
+        keepalived_ntp_vip6_linklocal: "{{ ntp_vip6_linklocal }}"
         keepalived_ntp_state: "{{ ntp_servers[inventory_hostname].state }}"
         keepalived_ntp_priority: "{{ ntp_servers[inventory_hostname].priority }}"
         keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
@@ -90,6 +134,49 @@
           {% endif %}
       when: ntp_keepalived_enabled | default(false)
 
+    # The v6 address is the path clients actually use, so it gets the same
+    # check as the v4 VIP. Without this a broken v6 rollout -- the address
+    # never claimed, or -nodad not behaving as expected on a given kernel --
+    # would not surface here at all, only later by reading
+    # /var/log/keepalived-state.log by hand.
+    #
+    # `ip -6 addr show ... to <addr>` prints the address when present and
+    # nothing when absent, so emptiness is the test.
+    #
+    # Presence cannot itself distinguish a VRRP-held address from a static
+    # leftover -- they look identical. It is only meaningful because the
+    # keepalived-ntp role asserts, on both distros, that no static assignment
+    # survives: the play fails before reaching here if one does.
+    - name: Check if this server holds the IPv6 service address
+      ansible.builtin.command:
+        argv:
+          - ip
+          - -6
+          - addr
+          - show
+          - dev
+          - "{{ ntp_keepalived_interface }}"
+          - to
+          - "{{ ntp_vip6 }}"
+      register: vip6_check
+      failed_when: false
+      changed_when: false
+      when: ntp_keepalived_enabled | default(false)
+
+    - name: Display IPv6 service address status
+      ansible.builtin.debug:
+        msg: >
+          {% if vip6_check.stdout | default('') | length > 0 %}
+          This server ({{ inventory_hostname }}) holds the IPv6 service address
+          {{ ntp_vip6 }} -- it is the v6 MASTER and is serving ntp.srv.renner.uno
+          {% else %}
+          This server ({{ inventory_hostname }}) does not hold {{ ntp_vip6 }}.
+          Expected on a v6 BACKUP. If NO host holds it, check_ntp is failing
+          everywhere -- inspect `chronyc tracking` and
+          /var/log/keepalived-ntp-health.log
+          {% endif %}
+      when: ntp_keepalived_enabled | default(false)
+
     - name: Report hosts serving NTP without participating in the VIP
       ansible.builtin.debug:
         msg: >

ansible/plays/install-power-meter.yml

@@ -0,0 +1,14 @@
+---
+# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
+# attached.
+#
+# Targets [power_meter]: membership means the meter head is physically present.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker and publishes to the local MQTT bridge.
+- name: Install power-meter reader
+  hosts: power_meter
+  become: true
+
+  roles:
+    - power-meter-sml

ansible/plays/install-prometheus.yml

@@ -1,43 +0,0 @@
----
-# Install Prometheus Metrics Server
-# Usage: ansible-playbook -i inventory/ plays/install-prometheus.yml
-
-- name: Install Prometheus Metrics Server
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Synology NAS SNMP targets
-    snmp_synology_targets:
-      - name: nas-1
-        target: "10.2.40.50"
-      - name: nas-2
-        target: "10.3.40.57"
-
-    # SNMPv3 credentials from vault
-    snmp_use_v3: true
-    snmp_v3_username: "{{ vault_snmp_v3_username }}"
-    snmp_v3_auth_password: "{{ vault_snmp_v3_auth_password }}"
-    snmp_v3_priv_password: "{{ vault_snmp_v3_priv_password }}"
-
-  roles:
-    - common
-    - prometheus-server
-    - snmp-exporter
-    - pve-exporter
-    - unifi-poller
-
-  post_tasks:
-    - name: Verify Prometheus is running
-      uri:
-        url: "http://127.0.0.1:9090/-/ready"
-        status_code: 200
-      register: prometheus_health
-      retries: 5
-      delay: 10
-      until: prometheus_health.status == 200
-
-    - name: Display Prometheus status
-      debug:
-        msg: "Prometheus is running and healthy at http://{{ inventory_hostname }}:9090"

ansible/plays/install-zwave-gateway.yml

@@ -0,0 +1,15 @@
+---
+# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
+#
+# Targets [zwave_gateway]: membership means the radio is physically present.
+# Moving the stick to another edge host is a one-line inventory change.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker, and the flash-wear handling that the libeatmydata preload exists to
+# complement.
+- name: Install Z-Wave gateway
+  hosts: zwave_gateway
+  become: true
+
+  roles:
+    - zwave-gateway

ansible/plays/update.yml

@@ -16,7 +16,6 @@
 - import_playbook: install-grist.yml
 - import_playbook: install-music-assistant.yml
 - import_playbook: update-docker.yml
-- import_playbook: install-log-aggregation.yml
 - import_playbook: update-pve-rolling.yml
 
 - name: Update Summary

ansible/roles/blackbox-exporter/defaults/main.yml

@@ -1,26 +0,0 @@
----
-blackbox_exporter_version: "0.25.0"
-blackbox_exporter_port: 9115
-blackbox_exporter_user: "blackbox_exporter"
-blackbox_exporter_group: "blackbox_exporter"
-
-# Endpoints to probe
-blackbox_http_targets:
-  - name: "gitea"
-    url: "https://gitea.renner.uno"
-  - name: "immich"
-    url: "https://immich.renner.uno"
-  - name: "authentik"
-    url: "https://auth.renner.uno"
-  - name: "home-assistant"
-    url: "https://ha.renner.uno"
-  - name: "grafana"
-    url: "https://grafana.renner.uno"
-  - name: "prometheus"
-    url: "https://prometheus.renner.uno"
-
-blackbox_dns_targets: []
-
-blackbox_tcp_targets:
-  - name: "postgres-ha"
-    target: "10.2.40.135:5432"

ansible/roles/blackbox-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/blackbox-exporter/tasks/main.yml

@@ -1,80 +0,0 @@
----
-- name: Create blackbox_exporter group
-  group:
-    name: "{{ blackbox_exporter_group }}"
-    system: yes
-    state: present
-
-- name: Create blackbox_exporter user
-  user:
-    name: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    createhome: no
-    state: present
-
-- name: Create blackbox_exporter directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0755'
-  loop:
-    - /etc/blackbox_exporter
-    - /var/lib/blackbox_exporter
-
-- name: Download blackbox_exporter
-  get_url:
-    url: "https://github.com/prometheus/blackbox_exporter/releases/download/v{{ blackbox_exporter_version }}/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64.tar.gz"
-    dest: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    mode: '0644'
-
-- name: Extract blackbox_exporter
-  unarchive:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    dest: /tmp
-    remote_src: yes
-    creates: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
-
-- name: Install blackbox_exporter binary
-  copy:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64/blackbox_exporter"
-    dest: /usr/local/bin/blackbox_exporter
-    owner: root
-    group: root
-    mode: '0755'
-    remote_src: yes
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter configuration
-  template:
-    src: blackbox.yml.j2
-    dest: /etc/blackbox_exporter/blackbox.yml
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter systemd service
-  template:
-    src: blackbox_exporter.service.j2
-    dest: /etc/systemd/system/blackbox_exporter.service
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Enable and start blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"

ansible/roles/blackbox-exporter/templates/blackbox.yml.j2

@@ -1,60 +0,0 @@
-modules:
-  http_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
-      valid_status_codes: []  # Defaults to 2xx
-      method: GET
-      follow_redirects: true
-      fail_if_ssl: false
-      fail_if_not_ssl: false
-      tls_config:
-        insecure_skip_verify: false
-
-  http_post_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      method: POST
-      valid_status_codes: []
-
-  tcp_connect:
-    prober: tcp
-    timeout: 5s
-
-  # DNS module for external domain resolution (tests upstream/cache)
-  dns_external:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "google.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for internal domain resolution (tests local resolution)
-  dns_internal:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "k8s-1.srv.renner.uno"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for common cached domain
-  dns_cached:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "cloudflare.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  icmp:
-    prober: icmp
-    timeout: 5s
-    icmp:
-      preferred_ip_protocol: "ip4"

ansible/roles/blackbox-exporter/templates/blackbox_exporter.service.j2

@@ -1,22 +0,0 @@
-[Unit]
-Description=Prometheus Blackbox Exporter
-Documentation=https://github.com/prometheus/blackbox_exporter
-Wants=network-online.target
-After=network-online.target
-
-[Service]
-Type=simple
-User={{ blackbox_exporter_user }}
-Group={{ blackbox_exporter_group }}
-ExecStart=/usr/local/bin/blackbox_exporter \
-    --config.file=/etc/blackbox_exporter/blackbox.yml \
-    --web.listen-address=:{{ blackbox_exporter_port }}
-Restart=on-failure
-RestartSec=5s
-
-# Allow ICMP probing (requires CAP_NET_RAW)
-AmbientCapabilities=CAP_NET_RAW
-CapabilityBoundingSet=CAP_NET_RAW
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/chrony-ntp-server/defaults/main.yml

@@ -1,9 +1,25 @@
 ---
 # Chrony NTP Server Role Defaults
 
-# Networks allowed to use this server as NTP source
+# Networks allowed to use this server as NTP source.
+#
+# BOTH families are required. IPv6 is the primary address family here -- the
+# service address ntp.srv.renner.uno resolves to fd40::123 -- so an IPv4-only
+# allow list refuses every client that uses the documented name.
+#
+# The hosts carried `allow fd00::/8` for a long time without it ever being in
+# this file. Applying the role on 2026-08-07 rendered the template over it and
+# dropped the line on all four NTP servers at once, which took internal NTP
+# down over IPv6 until it was added here. Same class of untracked host state as
+# the anycast service address, and the same failure: config that exists only on
+# the box survives right up until something re-renders it.
 chrony_allowed_networks:
   - "10.2.0.0/16"
+  - "fd00::/8"
+  # Link-local. A client on the same segment may source its query from fe80::,
+  # and without this chrony refuses it -- the request arrives and is dropped,
+  # which looks like the server being unreachable rather than a policy refusal.
+  - "fe80::/10"
 
 # Primary NTP servers (PTB - Physikalisch-Technische Bundesanstalt, Stratum 1)
 chrony_upstream_servers:
@@ -18,6 +34,26 @@ chrony_upstream_pools:
 # Local stratum if disconnected from upstream
 chrony_local_stratum: 10
 
+# Step (rather than slew) the clock when the offset exceeds the threshold, but
+# only for the first N updates after chronyd starts. Without a makestep
+# directive chronyd only ever slews, so a large offset is never closed in
+# practice.
+#
+# edge-1, 2026-08-04: the CMOS battery is dead, so after a power loss the RTC
+# came up reading 2018-08-20. The kernel set the clock from it, systemd floored
+# that forward to its own built-in epoch (2026-06-26), and it ended up at
+# ~2026-07-19 -- about 16 days behind. chronyd then synced to ptbtime1, reported
+# "System time: 1376235.375 seconds slow of NTP time", and corrected none of it.
+# With host time before the cluster certificate's NotBefore, every outbound TLS
+# handshake failed x509 validation, taking out the MQTT bridge (heating
+# telemetry) and the Loki log shipper at once. The 16 days is incidental -- it
+# is just where the software flooring landed; the fault is that nothing steps.
+#
+# Limiting to the first few updates keeps steps confined to boot, so a
+# long-running server is never jumped out from under its workloads.
+chrony_makestep_threshold: 1.0
+chrony_makestep_limit: 3
+
 # Hardware timestamping. Fine on the PVE nodes' Intel NICs; the Wyse 3040
 # (edge-1) has a Realtek NIC that does not support it, and chronyd logs an
 # error per interface when asked for it. Overridden to false in

ansible/roles/chrony-ntp-server/templates/chrony.conf.j2

@@ -14,6 +14,11 @@ pool {{ pool }}
 # Record the rate at which the system clock gains/losses time
 driftfile /var/lib/chrony/chrony.drift
 
+# Step the clock instead of slewing it when the offset is large, limited to the
+# first few updates after start so only boot-time corrections jump. Rationale
+# and the incident behind it: chrony_makestep_* in the role's defaults/main.yml.
+makestep {{ chrony_makestep_threshold }} {{ chrony_makestep_limit }}
+
 # Allow NTP client access from local networks
 {% for network in chrony_allowed_networks %}
 allow {{ network }}

ansible/roles/claude-workstation/RUNBOOK.md

@@ -118,8 +118,11 @@ After the `kubernetes/apps/claude-workstation/` PR merges and Flux reconciles:
 NS=apps
 TOKEN=$(kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.token}' | base64 -d)
 kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/ca.crt
-# server = the API endpoint reachable over the site-to-site route (a CP node or VIP):
-kubectl config set-cluster homelab --server=https://10.2.40.201:6443 \
+# server = the cluster's stable API endpoint (kubernetes/talos/patches/cluster.yaml).
+# Use the name, not an address: it is what the API server certificate carries as a
+# SAN, so an IP here fails TLS verification even when it is reachable. It must
+# resolve from the workstation over the site-to-site route.
+kubectl config set-cluster homelab --server=https://k8s.renner.uno:6443 \
   --certificate-authority=/tmp/ca.crt --embed-certs=true --kubeconfig=/tmp/claude-ws.kubeconfig
 kubectl config set-credentials claude-workstation --token="$TOKEN" --kubeconfig=/tmp/claude-ws.kubeconfig
 kubectl config set-context homelab --cluster=homelab --user=claude-workstation --kubeconfig=/tmp/claude-ws.kubeconfig

ansible/roles/edge-flash-wear/defaults/main.yml

@@ -0,0 +1,15 @@
+---
+# Flash-wear mitigation.
+
+# --- Logging & eMMC wear mitigation ------------------------------------
+# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
+# kept off local flash: containers + native services log to a RAM-only
+# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
+# wear check round out the flash-longevity measures. See README "Logging &
+# eMMC".
+edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
+
+# Container stdout -> journal, no json log files on the flash. The journal
+# itself is volatile (see the tasks above), so container logs land in RAM and
+# are shipped off the box by edge-log-shipper.
+docker_log_driver: journald

ansible/roles/edge-flash-wear/files/edge-emmc-health

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-# edge-1 eMMC wear/health check. Managed by roles/edge-host.
+# edge-1 eMMC wear/health check. Managed by roles/edge-flash-wear.
 #
 # Reads the JEDEC eMMC 5.x life-time and pre-EOL estimators exposed via sysfs
 # and emits a warning-priority journal message when the flash leaves the

ansible/roles/edge-flash-wear/handlers/main.yml

@@ -0,0 +1,17 @@
+---
+- name: Restart journald
+  ansible.builtin.service:
+    name: systemd-journald
+    state: restarted
+
+# Handlers run in definition order, so this fires after 'Restart journald':
+# once the journal is volatile, the on-disk store is dead weight on the flash.
+- name: Purge persistent journal store
+  ansible.builtin.file:
+    path: /var/log/journal
+    state: absent
+
+- name: Restart docker
+  ansible.builtin.service:
+    name: docker
+    state: restarted

ansible/roles/edge-flash-wear/tasks/main.yml

@@ -0,0 +1,100 @@
+---
+# Flash-wear mitigation for hosts on small eMMC (Dell Wyse 3040 and similar).
+#
+# journald -> volatile (RAM) and size-capped, root mounted noatime+lazytime,
+# periodic TRIM, and a weekly wear check. Shipping the volatile journal off the
+# box is a separate concern -- see the edge-log-shipper role.
+- name: Install mmc-utils
+  ansible.builtin.package:
+    name: mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
+    state: present
+
+- name: Ensure journald drop-in directory exists
+  ansible.builtin.file:
+    path: /etc/systemd/journald.conf.d
+    state: directory
+    mode: "0755"
+
+- name: Switch the systemd journal to volatile (RAM) storage
+  ansible.builtin.template:
+    src: journald-edge.conf.j2
+    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
+    mode: "0644"
+  notify:
+    - Restart journald
+    - Purge persistent journal store
+
+- name: Resolve root filesystem facts (for noatime)
+  ansible.builtin.set_fact:
+    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
+
+- name: Refuse to touch fstab without a stable root UUID
+  ansible.builtin.assert:
+    that:
+      - edge_root_fs.uuid | default('') | length > 0
+    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
+
+# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
+# heartbeat that otherwise dirties the journal every couple seconds); noatime
+# drops access-time writes. Neither weakens data durability.
+- name: Mount root with noatime + lazytime (persisted in fstab)
+  ansible.posix.mount:
+    path: /
+    src: "UUID={{ edge_root_fs.uuid }}"
+    fstype: "{{ edge_root_fs.fstype }}"
+    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
+    dump: "0"
+    passno: "1"   # root fs — keep boot-time fsck ordering
+    backup: true
+    state: mounted
+
+- name: Enable periodic TRIM (helps eMMC wear-leveling)
+  ansible.builtin.systemd:
+    name: fstrim.timer
+    enabled: true
+    state: started
+
+- name: Install eMMC health-check script
+  ansible.builtin.copy:
+    src: edge-emmc-health
+    dest: /usr/local/bin/edge-emmc-health
+    mode: "0755"
+
+- name: Install eMMC health-check service + timer
+  ansible.builtin.copy:
+    src: "{{ item }}"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - edge-emmc-health.service
+    - edge-emmc-health.timer
+
+- name: Enable eMMC health-check timer
+  ansible.builtin.systemd:
+    name: edge-emmc-health.timer
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# Container stdout into the journal instead of json files on disk. This lives
+# here, with the rest of the flash-wear handling, rather than in docker-install:
+# that role is shared by ten other plays (authentik, n8n, owncloud, grist,
+# netbird, gatus, music-assistant, ical-proxy), none of which have a
+# daemon.json today. Adding it there would have created one on the next run of
+# any of those plays and restarted every container on those hosts.
+#
+# /etc/docker is created explicitly because this role runs before
+# docker-install on a first bootstrap, so the directory may not exist yet. The
+# handler runs at the end of the play, by which time docker is installed.
+- name: Ensure the docker config directory exists
+  ansible.builtin.file:
+    path: /etc/docker
+    state: directory
+    mode: "0755"
+
+- name: Route container logs to the journal (keeps json logs off the flash)
+  ansible.builtin.template:
+    src: docker-daemon.json.j2
+    dest: /etc/docker/daemon.json
+    mode: "0644"
+  notify: Restart docker

ansible/roles/edge-flash-wear/templates/docker-daemon.json.j2

@@ -0,0 +1,3 @@
+{
+  "log-driver": "{{ docker_log_driver }}"
+}

ansible/roles/edge-flash-wear/templates/journald-edge.conf.j2

@@ -1,4 +1,4 @@
-# Managed by roles/edge-host. edge-1 boots off a small internal eMMC, so the
+# Managed by roles/edge-flash-wear. edge-1 boots off a small internal eMMC, so the
 # journal must not persist to flash: keep it in RAM (/run/log/journal) and cap
 # its size. Grafana Alloy (edge-alloy.service) ships this journal to Loki
 # before it rotates out of RAM — nothing durable lives here.

ansible/roles/edge-host/defaults/main.yml

@@ -1,118 +0,0 @@
----
-edge_base_dir: /opt/edge-host
-edge_tz: Europe/Berlin
-
-# --- Logging & eMMC wear mitigation ------------------------------------
-# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
-# kept off local flash: containers + native services log to a RAM-only
-# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
-# wear check round out the flash-longevity measures. See README "Logging &
-# eMMC".
-edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
-edge_docker_log_driver: journald      # container stdout -> journal, no json logs on eMMC
-
-# Grafana Alloy log shipper. The srv-network Loki (loki.srv.renner.uno:3100,
-# no auth) is firewalled off the IoT VLAN; the only endpoint edge-1 can reach
-# is the authenticated cluster ingress (Traefik basic-auth).
-edge_alloy_config_file: /etc/alloy/edge.alloy
-edge_alloy_http_port: 12345
-edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
-edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
-edge_loki_username: edge-1
-edge_alloy_labels:
-  environment: production
-  datacenter: homelab
-
-# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
-# mosquitto on loopback, which bridges to the central cluster broker over TLS
-# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
-# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
-# blips. The local broker writes nothing to the eMMC (persistence off, logs to
-# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
-edge_mqtt_host: 127.0.0.1
-edge_mqtt_port: 1883
-# Upstream cluster broker the local bridge connects to (TLS; the public
-# Let's Encrypt cert is validated against the system CA bundle).
-edge_mqtt_cluster_host: mqtt.renner.uno
-edge_mqtt_cluster_tls_port: 8883
-edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
-
-# --- USB devices (stable by-id symlinks). After physically moving the
-# sticks to edge-1, verify with `ls -l /dev/serial/by-id/` — the by-id
-# names travel with the device, not the host, so these should not change.
-edge_zwave_device: /dev/serial/by-id/usb-Nabu_Casa_ZWA-2_80B54EE15720-if00
-edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
-
-# --- Images. Versions mirror the k8s manifests at migration time.
-# sml2mqtt stays hand-pinned (critical meter bridge — was deliberately
-# excluded from image-automation in k8s, keep that property here).
-edge_zwavejs_image: ghcr.io/zwave-js/zwave-js-ui:11.21.1
-edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
-# Alpine version the zwave-js-ui image is built on — the source for the musl
-# libeatmydata.so preloaded into that container (see tasks "Extract musl
-# libeatmydata.so"). Keep in sync with the base if edge_zwavejs_image bumps
-# its Alpine major (check: docker exec zwave-js cat /etc/os-release).
-edge_zwave_alpine_base: alpine:3.22
-
-# --- CAN / open3e (Viessmann E3 external bus) ---
-edge_can_interface: can0
-edge_can_bitrate: 250000
-edge_open3e_pkg: git+https://github.com/open3e/open3e.git
-edge_open3e_mqtt_user: open3e
-edge_open3e_mqtt_topic: open3e
-edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-edge_open3e_poll_seconds: 30
-# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
-# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
-# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
-# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
-# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
-# the internal PV-surplus EMS depends on). Decodes via the master O3EList
-# codec inherited through the vx3plus overlay.
-# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
-# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
-# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
-# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
-# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
-# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
-# entries when added), so the multi-frame read is cheap at a 30 s poll.
-# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
-# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
-edge_open3e_datapoints: >-
-  vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
-# ECU addresses; -dev selects codec set, tx selects the UDS address.
-# vcmu (refrigerant-circuit controller) has no dedicated codec set —
-# omitting dpList makes open3e fall back to the generic datapoint list,
-# which decodes all polled VCMU DIDs.
-edge_open3e_devices:
-  vitocal: { tx: "0x680", dpList: "vcal" }
-  # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
-  # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
-  vx3: { tx: "0x6a1", dpList: "/opt/edge-host/open3e/vx3plus.py" }
-  vcmu: { tx: "0x68c" }
-
-# --- E3onCAN: passive E380 energy-meter listener (no bus writes, so it
-# coexists with the single open3e instance). The meter broadcasts at
-# ~5 Hz; the gap throttles MQTT to one update per DID per interval.
-edge_e3oncan_repo: https://github.com/MyHomeMyData/E3onCAN.git
-edge_e3oncan_version: 461dcbdf97e8d72d2edb9929c63b39382af6c3a1  # V0.5.2
-edge_e3oncan_gap_seconds: 10
-edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-
-# --- Secrets: define these in inventory/group_vars/edge_host/vault.yml
-# (ansible-vault). See README.md for the kubectl one-liners that read the
-# current values out of the cluster during migration.
-# edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
-# edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
-# edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"
-
-# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
-# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
-# VRRP group, which needs layer-2 adjacency with the other members on the
-# service VLAN. The untagged VLAN 30 lease stays this host's only default
-# route -- see templates/10-vlan40.network.j2.
-edge_vlan40_enabled: false
-edge_vlan40_parent: enp1s0
-edge_vlan40_id: 40
-edge_vlan40_name: enp1s0.40
-edge_vlan40_parent_network: 20-wired.network

ansible/roles/edge-host/handlers/main.yml

@@ -1,66 +0,0 @@
----
-# reload alone only re-reads config files; existing links keep their old
-# state until reconfigured.
-- name: Reload networkd
-  ansible.builtin.shell:
-    cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
-  changed_when: true
-
-# Reconfiguring the parent is what actually materialises the VLAN child -- a
-# bare reload leaves the existing link as-is (see above). This re-runs DHCP on
-# enp1s0, which is this headless host's only path to the network, so it is the
-# one genuinely risky step in the VLAN change.
-- name: Reload networkd for VLAN
-  ansible.builtin.shell:
-    cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
-  changed_when: true
-
-# Ordered before 'Restart edge compose services': the daemon must pick up the
-# new default log driver before the containers are recreated onto it.
-- name: Restart docker
-  ansible.builtin.service:
-    name: docker
-    state: restarted
-
-- name: Restart journald
-  ansible.builtin.service:
-    name: systemd-journald
-    state: restarted
-
-# Handlers run in definition order, so this fires after 'Restart journald':
-# once the journal is volatile, the on-disk store is dead weight on the eMMC.
-- name: Purge persistent journal store
-  ansible.builtin.file:
-    path: /var/log/journal
-    state: absent
-
-- name: Restart alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    state: restarted
-    daemon_reload: true
-
-# Ordered before the MQTT clients (compose/open3e/e3oncan) so they reconnect
-# to a freshly-configured local broker.
-- name: Restart mosquitto
-  ansible.builtin.service:
-    name: mosquitto
-    state: restarted
-
-- name: Restart edge compose services
-  ansible.builtin.command:
-    cmd: docker compose up -d --force-recreate
-    chdir: "{{ edge_base_dir }}"
-  changed_when: true
-
-- name: Restart open3e
-  ansible.builtin.systemd:
-    name: open3e
-    state: restarted
-    daemon_reload: true
-
-- name: Restart e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    state: restarted
-    daemon_reload: true

ansible/roles/edge-host/tasks/main.yml

@@ -1,440 +0,0 @@
----
-# Arch edge host: zwave-js-ui + sml2mqtt as docker compose services,
-# open3e natively (venv + systemd) because it needs a SocketCAN interface,
-# which is host network state, not something to hide in a container.
-# Docker itself comes from the docker-install role (Arch-aware).
-
-- name: Install base packages
-  ansible.builtin.package:
-    name:
-      - python
-      - python-packaging  # ansible's pip module imports it on the target
-      - git
-      - mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
-      - grafana-alloy  # log shipper -> Loki (see Alloy section below)
-      - mosquitto      # local MQTT bridge broker -> cluster over TLS
-    state: present
-
-# --- Local MQTT bridge broker. Edge clients publish to this over loopback;
-# it bridges to the cluster broker over TLS (open3e/e3oncan can't do MQTTS).
-# Configured with persistence off + stdout logging so it writes nothing to the
-# eMMC. Deployed before the clients so they have somewhere to connect.
-# See edge-mqtt-bridge.conf.j2 / README "MQTT bridge".
-- name: Deploy local mosquitto bridge config
-  ansible.builtin.template:
-    src: edge-mqtt-bridge.conf.j2
-    dest: /etc/mosquitto/mosquitto.conf
-    owner: root
-    group: mosquitto
-    mode: "0640"   # carries the cluster bridge credentials
-  notify: Restart mosquitto
-
-- name: Enable local mosquitto bridge
-  ansible.builtin.systemd:
-    name: mosquitto
-    enabled: true
-    state: started
-
-# ======================================================================
-# eMMC wear mitigation + log shipping. edge-1 (Dell Wyse 3040) runs on a
-# small internal eMMC, so logs must not churn local flash. Strategy:
-#   - journald -> volatile (RAM), size-capped        (journald-edge.conf)
-#   - docker   -> journald log driver, no json files (docker-daemon.json)
-#   - Grafana Alloy tails the RAM journal -> Loki, WAL on tmpfs
-# plus TRIM, noatime and a weekly wear check for flash longevity.
-# See roles/edge-host/README.md "Logging & eMMC".
-# ======================================================================
-
-- name: Ensure journald drop-in directory exists
-  ansible.builtin.file:
-    path: /etc/systemd/journald.conf.d
-    state: directory
-    mode: "0755"
-
-- name: Switch the systemd journal to volatile (RAM) storage
-  ansible.builtin.template:
-    src: journald-edge.conf.j2
-    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
-    mode: "0644"
-  notify:
-    - Restart journald
-    - Purge persistent journal store
-
-- name: Resolve root filesystem facts (for noatime)
-  ansible.builtin.set_fact:
-    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
-
-- name: Refuse to touch fstab without a stable root UUID
-  ansible.builtin.assert:
-    that:
-      - edge_root_fs.uuid | default('') | length > 0
-    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
-
-# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
-# heartbeat that otherwise dirties the journal every couple seconds); noatime
-# drops access-time writes. Neither weakens data durability.
-- name: Mount root with noatime + lazytime (persisted in fstab)
-  ansible.posix.mount:
-    path: /
-    src: "UUID={{ edge_root_fs.uuid }}"
-    fstype: "{{ edge_root_fs.fstype }}"
-    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
-    dump: "0"
-    passno: "1"   # root fs — keep boot-time fsck ordering
-    backup: true
-    state: mounted
-
-- name: Enable periodic TRIM (helps eMMC wear-leveling)
-  ansible.builtin.systemd:
-    name: fstrim.timer
-    enabled: true
-    state: started
-
-- name: Install eMMC health-check script
-  ansible.builtin.copy:
-    src: edge-emmc-health
-    dest: /usr/local/bin/edge-emmc-health
-    mode: "0755"
-
-- name: Install eMMC health-check service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-emmc-health.service
-    - edge-emmc-health.timer
-
-- name: Enable eMMC health-check timer
-  ansible.builtin.systemd:
-    name: edge-emmc-health.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- Grafana Alloy: ship the volatile journal to Loki. Container logs reach
-# the journal via docker's journald driver (configured below), native
-# services log to the journal directly, so a single journal source covers
-# everything. Storage/WAL sits on /run (tmpfs) — see edge-alloy.service.
-- name: Create alloy system group
-  ansible.builtin.group:
-    name: alloy
-    system: true
-
-- name: Create alloy system user
-  ansible.builtin.user:
-    name: alloy
-    group: alloy
-    groups: systemd-journal
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-- name: Create alloy config directory
-  ansible.builtin.file:
-    path: /etc/alloy
-    state: directory
-    owner: alloy
-    group: alloy
-    mode: "0750"
-
-- name: Deploy alloy config (journal -> Loki)
-  ansible.builtin.template:
-    src: edge-alloy.alloy.j2
-    dest: "{{ edge_alloy_config_file }}"
-    owner: alloy
-    group: alloy
-    mode: "0600"   # carries the Loki basic-auth password
-  notify: Restart alloy
-
-- name: Deploy alloy systemd unit
-  ansible.builtin.template:
-    src: edge-alloy.service.j2
-    dest: /etc/systemd/system/edge-alloy.service
-    mode: "0644"
-  notify: Restart alloy
-
-- name: Enable edge-alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- SocketCAN: systemd-networkd owns can0. gs_usb is an in-tree module
-# and autoloads when the Innomaker adapter enumerates; networkd then
-# matches the interface and applies bitrate + up. RestartSec recovers the
-# link after bus-off without manual intervention.
-- name: Configure can0 via systemd-networkd
-  ansible.builtin.template:
-    src: 80-can0.network.j2
-    dest: /etc/systemd/network/80-can0.network
-    mode: "0644"
-  notify: Reload networkd
-
-# --- Tagged VLAN 40. Additive by construction: the parent link keeps its own
-# .network file untouched and gains the VLAN via a drop-in, so reverting is a
-# matter of deleting these three paths. This host is headless and enp1s0 is its
-# only route to the network, hence the caution.
-- name: Create VLAN sub-interface netdev
-  ansible.builtin.template:
-    src: 10-vlan40.netdev.j2
-    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
-- name: Configure VLAN sub-interface addressing
-  ansible.builtin.template:
-    src: 10-vlan40.network.j2
-    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
-- name: Ensure parent network drop-in directory exists
-  ansible.builtin.file:
-    path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
-    state: directory
-    mode: "0755"
-  when: edge_vlan40_enabled
-  tags: [vlan40]
-
-- name: Attach VLAN to parent link via drop-in
-  ansible.builtin.template:
-    src: vlan40-parent-dropin.conf.j2
-    dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
-- name: Enable systemd-networkd
-  ansible.builtin.service:
-    name: systemd-networkd
-    state: started
-    enabled: true
-
-# Docker networking needs no daemon.json (both services use network_mode:
-# host — real addresses, v4+v6, no bridge/NAT). The daemon.json below exists
-# only to route container stdout into the journald log driver so json logs
-# never touch the eMMC; the journal itself is volatile (RAM) and shipped to
-# Loki by edge-alloy. Both compose services inherit this default driver.
-- name: Configure docker daemon (journald log driver, eMMC-friendly)
-  ansible.builtin.template:
-    src: docker-daemon.json.j2
-    dest: /etc/docker/daemon.json
-    mode: "0644"
-  notify:
-    - Restart docker
-    - Restart edge compose services
-
-# --- Directory layout ---
-- name: Create service directories
-  ansible.builtin.file:
-    path: "{{ item }}"
-    state: directory
-    mode: "0755"
-  loop:
-    - "{{ edge_base_dir }}"
-    - "{{ edge_base_dir }}/zwave-js/store"
-    - "{{ edge_base_dir }}/sml2mqtt"
-
-# --- zwave-js eMMC write reduction: batch fsync via libeatmydata ---------
-# zwave-js persists its value DB with an fsync on *every* value change
-# (jsonl-db, throttleFS unset by zwave-js-ui). On a chatty network that
-# fsync storm churns the eMMC (~2.6 GiB/day for ~0.1 KiB/s of real data).
-# Preloading libeatmydata makes fsync a no-op so the kernel batches the
-# writes via normal writeback — full HA data resolution and cache
-# persistence are untouched (see docker-compose.yml.j2 for the mount + env).
-# The .so must match the zwave-js-ui image's musl/Alpine base, so extract it
-# from that exact Alpine version (bridge networking is broken on this host,
-# hence --network=host). Refresh by deleting the .so if the base ever bumps
-# its Alpine major version.
-- name: Extract musl libeatmydata.so (matches the zwave-js-ui Alpine base)
-  ansible.builtin.shell:
-    cmd: |
-      set -euo pipefail
-      tmp="$(mktemp)"
-      trap 'rm -f "$tmp"' EXIT
-      docker run --rm --network=host {{ edge_zwave_alpine_base }} \
-        sh -c 'apk add --no-cache libeatmydata >/dev/null 2>&1 && cat /usr/lib/libeatmydata.so' > "$tmp"
-      test "$(stat -c%s "$tmp")" -ge 8000   # sanity: real .so is ~14 KB, guards against a failed/empty pull
-      install -m 0644 "$tmp" {{ edge_base_dir }}/zwave-js/libeatmydata.so
-    creates: "{{ edge_base_dir }}/zwave-js/libeatmydata.so"
-  notify: Restart edge compose services
-
-# Because fsync is neutralized, a hard power-loss during the (rare) settings
-# write could truncate settings.json — which holds the Z-Wave network keys.
-# Keep a validated rolling backup on the eMMC so the keys stay recoverable.
-- name: Install zwave-js settings.json backup script
-  ansible.builtin.template:
-    src: edge-zwave-settings-backup.j2
-    dest: /usr/local/bin/edge-zwave-settings-backup
-    mode: "0755"
-
-- name: Install zwave-js settings backup service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-zwave-settings-backup.service
-    - edge-zwave-settings-backup.timer
-
-- name: Enable zwave-js settings backup timer
-  ansible.builtin.systemd:
-    name: edge-zwave-settings-backup.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- Compose services: zwave-js-ui + sml2mqtt ---
-- name: Deploy docker-compose.yml
-  ansible.builtin.template:
-    src: docker-compose.yml.j2
-    dest: "{{ edge_base_dir }}/docker-compose.yml"
-    mode: "0644"
-  notify: Restart edge compose services
-
-- name: Deploy zwave-js env (session secret)
-  ansible.builtin.template:
-    src: zwavejs.env.j2
-    dest: "{{ edge_base_dir }}/zwavejs.env"
-    mode: "0600"
-  notify: Restart edge compose services
-
-# sml2mqtt reads only a config file and has no env override for the
-# password, so the rendered config carries it — 0600, root-owned, same
-# trust boundary as the k8s init-container approach it replaces.
-- name: Deploy sml2mqtt config
-  ansible.builtin.template:
-    src: sml2mqtt-config.yml.j2
-    dest: "{{ edge_base_dir }}/sml2mqtt/config.yml"
-    mode: "0600"
-  notify: Restart edge compose services
-
-- name: Pull compose images
-  ansible.builtin.command:
-    cmd: docker compose pull
-    chdir: "{{ edge_base_dir }}"
-  changed_when: false
-
-# Tolerates missing USB devices: before the physical cutover the ZWA-2 /
-# FTDI sticks are still on the old k8s-edge VM, and a container whose
-# device is absent must not block the rest of the role (open3e/CAN moves
-# first). Re-run the play — or `docker compose up -d` — after each stick
-# arrives.
-- name: Start compose services
-  ansible.builtin.command:
-    cmd: docker compose up -d
-    chdir: "{{ edge_base_dir }}"
-  register: compose_up
-  failed_when: >-
-    compose_up.rc != 0
-    and 'error gathering device information' not in compose_up.stderr
-  changed_when: >-
-    'Started' in compose_up.stderr or 'Created' in compose_up.stderr
-    or 'Started' in compose_up.stdout or 'Creating' in compose_up.stdout
-
-- name: Warn about containers waiting for their USB device
-  ansible.builtin.debug:
-    msg: >-
-      docker compose could not start every service (USB device not present
-      yet — expected before cutover): {{ compose_up.stderr_lines | select('search', 'device') | list }}
-  when: compose_up.rc != 0
-
-# --- open3e: native install. CAN_RAW sockets need no capabilities, so
-# the service runs as an unprivileged system user.
-- name: Create open3e system user
-  ansible.builtin.user:
-    name: open3e
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-# Owned by the service user: udsoncan writes its log file into the
-# working directory.
-- name: Create open3e directory
-  ansible.builtin.file:
-    path: "{{ edge_base_dir }}/open3e"
-    state: directory
-    owner: open3e
-    group: open3e
-    mode: "0755"
-
-- name: Install open3e into a venv
-  ansible.builtin.pip:
-    name: "{{ edge_open3e_pkg }}"
-    virtualenv: "{{ edge_base_dir }}/open3e/venv"
-    virtualenv_command: python -m venv
-  notify: Restart open3e
-
-- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
-  ansible.builtin.copy:
-    src: open3e-vx3plus.py
-    dest: "{{ edge_base_dir }}/open3e/vx3plus.py"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e devices.json
-  ansible.builtin.copy:
-    content: "{{ edge_open3e_devices | to_nice_json }}\n"
-    dest: "{{ edge_base_dir }}/open3e/devices.json"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e environment (broker credential)
-  ansible.builtin.template:
-    src: open3e.env.j2
-    dest: "{{ edge_base_dir }}/open3e/open3e.env"
-    owner: open3e
-    mode: "0600"
-  notify: Restart open3e
-
-- name: Deploy open3e systemd unit
-  ansible.builtin.template:
-    src: open3e.service.j2
-    dest: /etc/systemd/system/open3e.service
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Enable open3e
-  ansible.builtin.systemd:
-    name: open3e
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# --- E3onCAN: direct E380 meter reads (passive; per-review decision not
-# to trust the VX3's relayed meter values). Same broker user as open3e;
-# topics land under the same open3e/ prefix as e380_<did>_<name>.
-- name: Check out E3onCAN (pinned)
-  ansible.builtin.git:
-    repo: "{{ edge_e3oncan_repo }}"
-    dest: "{{ edge_base_dir }}/e3oncan/src"
-    version: "{{ edge_e3oncan_version }}"
-  notify: Restart e3oncan
-
-- name: Create E3onCAN venv with requirements
-  ansible.builtin.pip:
-    requirements: "{{ edge_base_dir }}/e3oncan/src/requirements.txt"
-    virtualenv: "{{ edge_base_dir }}/e3oncan/venv"
-    virtualenv_command: python -m venv
-  notify: Restart e3oncan
-
-- name: Deploy e3oncan systemd unit
-  ansible.builtin.template:
-    src: e3oncan.service.j2
-    dest: /etc/systemd/system/e3oncan.service
-    mode: "0644"
-  notify: Restart e3oncan
-
-- name: Enable e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    enabled: true
-    state: started
-    daemon_reload: true

ansible/roles/edge-host/templates/docker-compose.yml.j2

@@ -1,56 +0,0 @@
-# Managed by ansible (roles/edge-host). Mirrors the former k8s manifests:
-# kubernetes/apps/home-assistant-zwavejs/deployment.yaml and
-# kubernetes/apps/home-assistant/sml2mqtt-deployment.yaml.
-#
-# No `privileged` needed here: docker's --device handles the device-cgroup
-# grant that hostPath in k8s could not (that was the only reason the pods
-# ran privileged).
-# Both services use host networking: real host addresses (v4+v6), no
-# bridge, no NAT. The broker LB is IPv6-first and docker bridge NAT66 is
-# a tarpit (unmasqueraded SYNs die silently); the host stack just works.
-services:
-  zwave-js:
-    image: {{ edge_zwavejs_image }}
-    container_name: zwave-js
-    restart: unless-stopped
-    stop_signal: SIGINT
-    tty: true
-    environment:
-      - TZ={{ edge_tz }}
-      # Batch the value-DB fsync storm to spare the eMMC (see role tasks /
-      # README "Logging & eMMC"). Neutralizes fsync only for this container;
-      # the WS/MQTT value stream to HA and cache persistence are unchanged.
-      - LD_PRELOAD=/usr/lib/libeatmydata.so
-    env_file:
-      - zwavejs.env
-    devices:
-      - "{{ edge_zwave_device }}:/dev/zwave"
-    volumes:
-      # musl libeatmydata.so extracted from the image's Alpine base (ro)
-      - ./zwave-js/libeatmydata.so:/usr/lib/libeatmydata.so:ro
-      # Persistent store: settings.json, device DB, network keys. Seeded
-      # from the k8s PVC `zwavejs-store` at cutover (see README). Since
-      # this host is stateful there is no init-container merge step —
-      # settings.json is edited via the UI and persists here; zwave.port
-      # stays /dev/zwave.
-      - ./zwave-js/store:/usr/src/app/store
-    # Host networking: binds 3000 (Z-Wave JS websocket for HA) and 8091
-    # (web UI) directly on the host.
-    network_mode: host
-
-  sml2mqtt:
-    image: {{ edge_sml2mqtt_image }}
-    container_name: sml2mqtt
-    restart: unless-stopped
-    # Outbound-only (serial in, MQTT out) — host networking sidesteps the
-    # docker NAT66 gap: the broker LB is IPv6-first and bridge-network v6
-    # SYNs leave unmasqueraded and die. The host stack reaches fd99::81
-    # fine (open3e proves it continuously).
-    network_mode: host
-    # Serial exclusivity: this is the ONLY reader of the meter port —
-    # HA's edl21 integration stays disabled (see sml2mqtt-config comment
-    # history in the k8s repo).
-    devices:
-      - "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
-    volumes:
-      - ./sml2mqtt/config.yml:/sml2mqtt/config.yml:ro

ansible/roles/edge-host/templates/docker-daemon.json.j2

@@ -1,3 +0,0 @@
-{
-  "log-driver": "{{ edge_docker_log_driver }}"
-}

ansible/roles/edge-host/templates/e3oncan.service.j2

@@ -1,32 +0,0 @@
-# Managed by ansible (roles/edge-host).
-# PASSIVE listener for the E380 energy meter's CANopen broadcasts — no
-# writes on the bus, so it may run alongside the (single) open3e
-# instance. Reads the meter directly instead of trusting the VX3's
-# relayed values.
-[Unit]
-Description=E3onCAN E380 energy-meter CAN listener (read-only)
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker e3oncan publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/e3oncan
-# Journald-friendly (python block-buffers pipes; lost on restarts).
-Environment=PYTHONUNBUFFERED=1
-ExecStart={{ edge_base_dir }}/e3oncan/venv/bin/python {{ edge_base_dir }}/e3oncan/src/E3onCANcollect.py \
-  -c {{ edge_can_interface }} \
-  -dev e380 \
-  -g {{ edge_e3oncan_gap_seconds }} \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser {{ edge_open3e_mqtt_user }} \
-  -mpass ${OPEN3E_MQTT_PASSWORD} \
-  -mfstr "{{ edge_e3oncan_mqtt_format }}" \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/edge-host/templates/open3e.env.j2

@@ -1,7 +0,0 @@
-# Managed by ansible (roles/edge-host). Read by open3e.service and
-# e3oncan.service.
-# Full user:password pair as one variable so the open3e unit can pass it
-# as a single -muser argument without exposing it in the unit file;
-# e3oncan takes user and password separately (-muser/-mpass).
-OPEN3E_MUSER={{ edge_open3e_mqtt_user }}:{{ edge_open3e_mqtt_password }}
-OPEN3E_MQTT_PASSWORD={{ edge_open3e_mqtt_password }}

ansible/roles/edge-host/templates/open3e.service.j2

@@ -1,39 +0,0 @@
-# Managed by ansible (roles/edge-host).
-# Read-mostly listener — no -w flag, no service 0x77. The ONE sanctioned
-# write path is the MQTT command topic (-l): HA publishes the DHW
-# one-time-charge command there (scope decision 2026-07-02, see vican
-# CLAUDE.md). Anything beyond that single write gets its own review.
-# -j is load-bearing: without it open3e flattens JSON payloads into
-# per-key subtopics and the HA package (homeassistant repo,
-# packages/open3e.yaml) receives nothing.
-# Exactly ONE open3e instance may be on the bus at a time (CAN collisions
-# stop open3e) — stop this service before running ad-hoc scans.
-[Unit]
-Description=open3e Viessmann E3 CAN-to-MQTT bridge (read-only)
-# Tie the service to the CAN device: stops on unplug, starts on plug via
-# the device unit rather than polling.
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker open3e publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/open3e
-ExecStart={{ edge_base_dir }}/open3e/venv/bin/open3e \
-  -c {{ edge_can_interface }} \
-  -cnfg {{ edge_base_dir }}/open3e/devices.json \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser ${OPEN3E_MUSER} \
-  -mcid open3e-edge \
-  -mfstr "{{ edge_open3e_mqtt_format }}" \
-  -r {{ edge_open3e_datapoints }} \
-  -t {{ edge_open3e_poll_seconds }} \
-  -l {{ edge_open3e_mqtt_topic }}/cmnd \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/edge-log-shipper/defaults/main.yml

@@ -0,0 +1,15 @@
+---
+# Journal -> Loki shipping.
+#
+# The srv-network Loki (loki.srv.renner.uno:3100, no auth) is firewalled off
+# the IoT VLAN; the only endpoint these hosts can reach is the authenticated
+# cluster ingress. edge_loki_password comes from the vault.
+
+edge_alloy_config_file: /etc/alloy/edge.alloy
+edge_alloy_http_port: 12345
+edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
+edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
+edge_loki_username: edge-1
+edge_alloy_labels:
+  environment: production
+  datacenter: homelab

ansible/roles/edge-log-shipper/handlers/main.yml

@@ -0,0 +1,6 @@
+---
+- name: Restart alloy
+  ansible.builtin.systemd:
+    name: edge-alloy
+    state: restarted
+    daemon_reload: true

ansible/roles/edge-log-shipper/tasks/main.yml

@@ -0,0 +1,57 @@
+---
+# Ship the systemd journal to Loki with Grafana Alloy.
+#
+# Pairs with edge-flash-wear, which makes the journal volatile: container logs
+# reach the journal via docker s journald driver and native services log to it
+# directly, so one journal source covers everything. Alloy s storage/WAL sits
+# on /run (tmpfs) so it does not write to flash either.
+
+- name: Install grafana-alloy
+  ansible.builtin.package:
+    name: grafana-alloy
+    state: present
+
+- name: Create alloy system group
+  ansible.builtin.group:
+    name: alloy
+    system: true
+
+- name: Create alloy system user
+  ansible.builtin.user:
+    name: alloy
+    group: alloy
+    groups: systemd-journal
+    system: true
+    create_home: false
+    shell: /usr/bin/nologin
+
+- name: Create alloy config directory
+  ansible.builtin.file:
+    path: /etc/alloy
+    state: directory
+    owner: alloy
+    group: alloy
+    mode: "0750"
+
+- name: Deploy alloy config (journal -> Loki)
+  ansible.builtin.template:
+    src: edge-alloy.alloy.j2
+    dest: "{{ edge_alloy_config_file }}"
+    owner: alloy
+    group: alloy
+    mode: "0600"   # carries the Loki basic-auth password
+  notify: Restart alloy
+
+- name: Deploy alloy systemd unit
+  ansible.builtin.template:
+    src: edge-alloy.service.j2
+    dest: /etc/systemd/system/edge-alloy.service
+    mode: "0644"
+  notify: Restart alloy
+
+- name: Enable edge-alloy
+  ansible.builtin.systemd:
+    name: edge-alloy
+    enabled: true
+    state: started
+    daemon_reload: true

ansible/roles/edge-log-shipper/templates/edge-alloy.alloy.j2

@@ -1,4 +1,4 @@
-// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-host.
+// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-log-shipper.
 //
 // edge-1 runs on internal eMMC, so logs are kept out of local flash: the
 // systemd journal is volatile (RAM, see journald-edge.conf) and container

ansible/roles/edge-vlan/defaults/main.yml

@@ -0,0 +1,11 @@
+---
+# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
+# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
+# VRRP group, which needs layer-2 adjacency with the other members on the
+# service VLAN. The untagged VLAN 30 lease stays this host's only default
+# route -- see templates/10-vlan40.network.j2.
+edge_vlan40_enabled: false
+edge_vlan40_parent: enp1s0
+edge_vlan40_id: 40
+edge_vlan40_name: enp1s0.40
+edge_vlan40_parent_network: 20-wired.network

ansible/roles/edge-vlan/handlers/main.yml

@@ -0,0 +1,9 @@
+---
+# Reconfiguring the parent is what actually materialises the VLAN child -- a
+# bare reload leaves the existing link as-is. This re-runs DHCP on the parent,
+# which is a headless host's only path to the network, so it is the one
+# genuinely risky step in the VLAN change.
+- name: Reload networkd for VLAN
+  ansible.builtin.shell:
+    cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
+  changed_when: true

ansible/roles/edge-vlan/tasks/main.yml

@@ -0,0 +1,48 @@
+---
+# Tagged VLAN sub-interface via systemd-networkd.
+#
+# Additive by construction: the parent link keeps its own .network file and
+# gains the VLAN through a drop-in, so reverting is deleting three paths. These
+# hosts are headless and the parent is their only route to the network, hence
+# the caution.
+
+- name: Create VLAN sub-interface netdev
+  ansible.builtin.template:
+    src: 10-vlan40.netdev.j2
+    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
+    mode: "0644"
+  when: edge_vlan40_enabled
+  notify: Reload networkd for VLAN
+  tags: [vlan40]
+
+- name: Configure VLAN sub-interface addressing
+  ansible.builtin.template:
+    src: 10-vlan40.network.j2
+    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
+    mode: "0644"
+  when: edge_vlan40_enabled
+  notify: Reload networkd for VLAN
+  tags: [vlan40]
+
+- name: Ensure parent network drop-in directory exists
+  ansible.builtin.file:
+    path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
+    state: directory
+    mode: "0755"
+  when: edge_vlan40_enabled
+  tags: [vlan40]
+
+- name: Attach VLAN to parent link via drop-in
+  ansible.builtin.template:
+    src: vlan40-parent-dropin.conf.j2
+    dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
+    mode: "0644"
+  when: edge_vlan40_enabled
+  notify: Reload networkd for VLAN
+  tags: [vlan40]
+
+- name: Enable systemd-networkd
+  ansible.builtin.service:
+    name: systemd-networkd
+    state: started
+    enabled: true

ansible/roles/heating-can/defaults/main.yml

@@ -0,0 +1,60 @@
+---
+# Heating system over the Viessmann E3 CAN bus: open3e (heat pump) and
+# e3oncan (E380 energy meter). The can0 interface itself is the socketcan
+# role.
+
+# Service root. Kept per-role so the role is usable on its own.
+edge_base_dir: /opt/edge-host
+
+edge_open3e_mqtt_user: open3e
+edge_open3e_mqtt_topic: open3e
+edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
+edge_open3e_poll_seconds: 30
+# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
+# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
+# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
+# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
+# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
+# the internal PV-surplus EMS depends on). Decodes via the master O3EList
+# codec inherited through the vx3plus overlay.
+# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
+# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
+# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
+# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
+# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
+# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
+# entries when added), so the multi-frame read is cheap at a 30 s poll.
+# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
+# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
+edge_open3e_datapoints: >-
+  vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
+# ECU addresses; -dev selects codec set, tx selects the UDS address.
+# vcmu (refrigerant-circuit controller) has no dedicated codec set —
+# omitting dpList makes open3e fall back to the generic datapoint list,
+# which decodes all polled VCMU DIDs.
+edge_open3e_devices:
+  vitocal: { tx: "0x680", dpList: "vcal" }
+  # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
+  # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
+  vx3: { tx: "0x6a1", dpList: "/config/vx3plus.py" }
+  vcmu: { tx: "0x68c" }
+
+edge_e3oncan_gap_seconds: 10
+edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
+
+# Secrets come from inventory/group_vars/edge_host/vault.yml:
+#   edge_open3e_mqtt_password
+
+# The image carrying both readers, built and pinned in CI
+# (ci/images/heating-can/Containerfile). The source pins live in the
+# Containerfile, in the artifact -- not here -- so what is deployed is a tag,
+# and `docker compose up -d` on an unchanged tag is a genuine no-op.
+#
+# Tracked by the weekly update-audit as an ordinary image, which is why this is
+# simpler to keep current than the two git refs it replaced: one publishes no
+# tags at all, the other was pinned to an untagged commit between releases.
+heating_can_image: gitea.renner.uno/gitea_admin/heating-can:latest
+
+# Compose project directory. Separate from the old native layout under
+# {{ edge_base_dir }}/open3e, which this role removes.
+heating_can_base_dir: "{{ edge_base_dir }}/heating-can"

ansible/roles/heating-can/files/open3e-vx3plus.py

@@ -1,4 +1,4 @@
-# Managed by ansible (roles/edge-host). Datapoint overlay for the VX3:
+# Managed by ansible (roles/heating-can). Datapoint overlay for the VX3:
 # the stock vx3 list acts as a FILTER (DIDs missing from it are popped
 # from the master table, Open3Eclass.py), so the per-battery-module DIDs
 # were never pollable with dpList "vx3". This overlay re-adds them with

ansible/roles/heating-can/handlers/main.yml

@@ -0,0 +1,10 @@
+---
+- name: Reload systemd
+  ansible.builtin.systemd:
+    daemon_reload: true
+
+- name: Restart heating-can
+  ansible.builtin.command:
+    cmd: docker compose up -d --force-recreate
+    chdir: "{{ heating_can_base_dir }}"
+  changed_when: true

ansible/roles/heating-can/tasks/main.yml

@@ -0,0 +1,164 @@
+---
+# Heating system over the Viessmann E3 CAN bus.
+#
+# Two readers of the same heating installation, supervised inside one container
+# from one image (ci/images/heating-can):
+#   open3e    UDS request/response to the Vitocal heat pump and Vitocharge VX3
+#   e3oncan   passive CANopen listener for the E380 grid energy meter, which
+#             feeds the heat pump's own energy management
+#
+# They coexist because e3oncan issues no bus writes.
+#
+# Nothing is installed here. The image is built in CI with both sources pinned,
+# published to the registry, and this host only pulls and runs it. That is what
+# removes the churn: these were pip-installed into venvs on the node, and pip
+# re-cloned and rebuilt a VCS requirement on every converge -- restarting the
+# heating bridge each time -- which pinning the SHA did not fix.
+#
+# network_mode: host is required, not incidental: SocketCAN interfaces live in
+# the host network namespace, so can0 is only reachable that way. It is also how
+# the local MQTT broker on 127.0.0.1 is reached. CAN_RAW needs no capabilities,
+# so both run unprivileged.
+#
+# The can0 interface itself belongs to the socketcan role; this role consumes it.
+
+- name: Create heating-can directory
+  ansible.builtin.file:
+    path: "{{ heating_can_base_dir }}"
+    state: directory
+    mode: "0755"
+
+# --- Migration off the native install ---------------------------------------
+# Must happen before the containers start. Two open3e instances issuing UDS
+# requests on one CAN bus is the failure mode to avoid, so the old units are
+# stopped and masked rather than merely disabled -- masked so a stray
+# `systemctl start` or a leftover dependency cannot bring them back alongside
+# the containers.
+- name: Find which native CAN units still exist
+  ansible.builtin.stat:
+    path: "/etc/systemd/system/{{ item }}.service"
+  loop:
+    - open3e
+    - e3oncan
+  register: heating_can_native_units
+
+# Stop and disable BEFORE anything else, and do NOT swallow the result.
+#
+# The first version did stop + disable + mask in one call with
+# failed_when: false, then deleted the unit files. Masking a unit whose file
+# still exists conflicts, so the call failed, failed_when hid it, nothing was
+# stopped -- and the next task deleted the unit files out from under two
+# running processes. systemctl is-enabled then reported "not-found" and looked
+# clean while is-active still said "active".
+#
+# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
+# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
+# kicking each other off the broker.
+#
+# Only units that actually exist are touched, so a fresh host does not need an
+# error swallowed to get through this.
+- name: Stop and disable the native CAN services
+  ansible.builtin.systemd:
+    name: "{{ item.item }}"
+    state: stopped
+    enabled: false
+  loop: "{{ heating_can_native_units.results }}"
+  loop_control:
+    label: "{{ item.item }}"
+  when: item.stat.exists
+
+# Masking is deliberately NOT used. It writes a symlink at the same path the
+# next task removes, so the two cancel out -- which is half of how the original
+# failed. With the unit files gone there is nothing left to start.
+- name: Remove the native unit files
+  ansible.builtin.file:
+    path: "/etc/systemd/system/{{ item }}.service"
+    state: absent
+  loop:
+    - open3e
+    - e3oncan
+  notify: Reload systemd
+
+# The invariant that actually matters is "no native process is on the CAN bus",
+# and systemd reporting success is not the same claim. Two open3e instances
+# polling one bus is the failure this whole block exists to prevent, so it is
+# asserted directly rather than inferred from the calls above.
+- name: Check for surviving native CAN processes
+  ansible.builtin.command:
+    argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
+  register: heating_can_survivors
+  changed_when: false
+  failed_when: false
+
+- name: Assert no native CAN process survived
+  ansible.builtin.assert:
+    that:
+      - heating_can_survivors.stdout | default('') | length == 0
+    fail_msg: >-
+      A native open3e/e3oncan process is still running after the units were
+      stopped and removed:
+      {{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
+      Starting the container now would put two readers on the same CAN bus --
+      two open3e instances issue competing UDS requests and take over each
+      other's MQTT session. Kill the survivors, then re-run.
+    success_msg: No native CAN processes remain; safe to start the container.
+
+- name: Remove the native venvs
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "{{ edge_base_dir }}/open3e/venv"
+    - "{{ edge_base_dir }}/e3oncan/venv"
+    - "{{ edge_base_dir }}/e3oncan/src"
+    - "{{ edge_base_dir }}/open3e"
+    - "{{ edge_base_dir }}/e3oncan"
+
+# --- Configuration (bind-mounted into the containers) ------------------------
+- name: Deploy open3e devices.json
+  ansible.builtin.copy:
+    content: "{{ edge_open3e_devices | to_nice_json }}\n"
+    dest: "{{ heating_can_base_dir }}/devices.json"
+    mode: "0644"
+  notify: Restart heating-can
+
+- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
+  ansible.builtin.copy:
+    src: open3e-vx3plus.py
+    dest: "{{ heating_can_base_dir }}/vx3plus.py"
+    mode: "0644"
+  notify: Restart heating-can
+
+# --- Compose project ---------------------------------------------------------
+- name: Deploy the broker credential env file
+  ansible.builtin.template:
+    src: heating-can.env.j2
+    dest: "{{ heating_can_base_dir }}/heating-can.env"
+    mode: "0600"
+  notify: Restart heating-can
+
+- name: Deploy heating-can compose project
+  ansible.builtin.template:
+    src: docker-compose.yml.j2
+    dest: "{{ heating_can_base_dir }}/docker-compose.yml"
+    mode: "0644"
+  notify: Restart heating-can
+
+- name: Pull the heating-can image
+  ansible.builtin.command:
+    cmd: docker compose pull
+    chdir: "{{ heating_can_base_dir }}"
+  changed_when: false
+
+# Unlike the zwave and meter services there is no missing-device tolerance here:
+# can0 is created by the socketcan role from a USB adapter that is part of this
+# host's purpose. If it is absent the heating telemetry is down and the run
+# should say so rather than pass quietly.
+- name: Start heating-can services
+  ansible.builtin.command:
+    cmd: docker compose up -d
+    chdir: "{{ heating_can_base_dir }}"
+  register: heating_can_up
+  changed_when: >-
+    'Started' in heating_can_up.stderr or 'Created' in heating_can_up.stderr
+    or 'Started' in heating_can_up.stdout or 'Creating' in heating_can_up.stdout

ansible/roles/heating-can/templates/docker-compose.yml.j2

@@ -0,0 +1,35 @@
+# Managed by ansible (roles/heating-can).
+#
+# One container. Both CAN readers live in the image and are supervised inside
+# it -- this file supplies configuration and nothing else. It does not know
+# open3e's or E3onCAN's command line.
+#
+# network_mode: host is required: SocketCAN interfaces live in the host network
+# namespace, so {{ edge_can_interface }} is only reachable this way, and it is
+# also how the local MQTT broker is reached. CAN_RAW needs no capabilities, so
+# the image runs unprivileged.
+services:
+  heating-can:
+    image: {{ heating_can_image }}
+    container_name: heating-can
+    restart: unless-stopped
+    network_mode: host
+    volumes:
+      - ./devices.json:/config/devices.json:ro
+      - ./vx3plus.py:/config/vx3plus.py:ro
+    env_file:
+      # Broker credential, 0600 -- see heating-can.env.j2.
+      - heating-can.env
+    environment:
+      CAN_INTERFACE: "{{ edge_can_interface }}"
+      MQTT_HOST: "{{ edge_mqtt_host }}"
+      MQTT_PORT: "{{ edge_mqtt_port }}"
+      MQTT_TOPIC: "{{ edge_open3e_mqtt_topic }}"
+      MQTT_FORMAT: "{{ edge_open3e_mqtt_format }}"
+      E3ONCAN_MQTT_FORMAT: "{{ edge_e3oncan_mqtt_format }}"
+      OPEN3E_CONFIG: /config/devices.json
+      OPEN3E_CLIENT_ID: open3e-edge
+      OPEN3E_POLL_SECONDS: "{{ edge_open3e_poll_seconds }}"
+      OPEN3E_DATAPOINTS: "{{ edge_open3e_datapoints }}"
+      E3ONCAN_DEVICE: e380
+      E3ONCAN_GAP_SECONDS: "{{ edge_e3oncan_gap_seconds }}"

ansible/roles/heating-can/templates/heating-can.env.j2

@@ -0,0 +1,8 @@
+# Managed by ansible (roles/heating-can). 0600 -- carries the broker credential.
+#
+# Kept out of docker-compose.yml, which is 0644: the sibling roles on this host
+# do the same (power-meter-sml renders its config at 0600 while its compose file
+# stays world-readable), and the native unit this replaced used a 0600 env file
+# for exactly this value.
+MQTT_USER={{ edge_open3e_mqtt_user }}
+MQTT_PASSWORD={{ edge_open3e_mqtt_password }}

ansible/roles/keepalived-ntp/defaults/main.yml

@@ -4,6 +4,29 @@
 keepalived_ntp_router_id: 4
 keepalived_ntp_vip: "10.2.40.4"
 keepalived_ntp_interface: "vmbr1.40"
+
+# IPv6 service address for NTP (ntp.srv.renner.uno). Empty disables the v6
+# instance entirely. IPv6 is the primary address family on this network, so
+# this is the path that actually matters -- the v4 VIP above is legacy.
+#
+# Held by keepalived rather than assigned statically by networkd so that it is
+# gated on check_ntp: present only while this host is fit to serve time, and
+# withdrawn the moment it is not. A statically configured address is carried
+# unconditionally from boot, which is how edge-1 came to answer NTP queries on
+# 2026-08-04 while its clock was ~16 days in the past.
+#
+# Needs its own virtual_router_id: it is a distinct VRRP instance, and a VRID
+# collision with the IPv4 instance on the same interface would have the two
+# fighting over a single election.
+keepalived_ntp_vip6: ""
+keepalived_ntp_router_id6: 6
+
+# Link-local address for the IPv6 VRRP instance, carried as its FIRST
+# virtual_ipaddress. RFC 5798 gives an IPv6 virtual router a link-local primary,
+# and keepalived warns "the first IPv6 VIP address should be link local" without
+# one. Chosen to mirror the service address rather than derived from the virtual
+# MAC, since no real NIC generates fe80::123 so there is nothing to collide with.
+keepalived_ntp_vip6_linklocal: "fe80::123/64"
 # NOTE: plaintext VRRP auth, unvaulted. Carried over as-is from the original
 # role; VRRP PASS auth is weak regardless (sent in the clear on the segment),
 # so this is a documented weakness rather than a secret worth vaulting.
@@ -18,6 +41,17 @@ keepalived_ntp_check_script: "/usr/local/bin/check-ntp-health.sh"
 keepalived_ntp_check_interval: 2
 keepalived_ntp_check_weight: -20
 
+# Maximum |system clock offset| in seconds before the host refuses to hold the
+# VIP. chronyd reports this as "System time : <n> seconds slow/fast of NTP
+# time". Anything above this means the host knows its clock is wrong and must
+# not serve time to clients -- see the reasoning in check-ntp-health.sh.j2.
+#
+# A serving NTP host should be within milliseconds of real time, so 1s is
+# already several orders of magnitude beyond normal. It is deliberately not
+# tighter: a brief excursion at boot, before makestep lands, should not flap
+# the VIP (fall 2 at a 2s interval also requires the failure to persist).
+keepalived_ntp_max_offset: 1.0
+
 # sysctl applied for VRRP. Default preserves the PVE hosts' existing behaviour
 # exactly. ip_forward is NOT appropriate everywhere -- on a host with legs in
 # two VLANs it turns the box into a router between them, bypassing the gateway's

ansible/roles/keepalived-ntp/files/keepalived-notify.sh

@@ -1,22 +1,39 @@
 #!/bin/bash
 # Keepalived State Change Notification Script
+#
+# Two VRRP instances call this: NTP_VIP (IPv4, legacy) passes MASTER/BACKUP/
+# FAULT, and NTP_VIP6 (IPv6, the primary path) passes the same suffixed with
+# '6'. They are logged distinctly because the two can legitimately diverge --
+# a host can hold one family and not the other -- and a log line that does not
+# say which family it means is useless during an incident.
 
 TYPE=$1
 NAME=$(hostname)
 TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
 
+log() {
+    echo "$TIMESTAMP: $NAME $1" >> /var/log/keepalived-state.log
+    logger -t keepalived "$NAME $1"
+}
+
 case $TYPE in
     MASTER)
-        echo "$TIMESTAMP: $NAME transitioned to MASTER state (NTP)" >> /var/log/keepalived-state.log
-        logger -t keepalived "$NAME is now NTP MASTER"
+        log "transitioned to MASTER state (NTP, IPv4)"
         ;;
     BACKUP)
-        echo "$TIMESTAMP: $NAME transitioned to BACKUP state (NTP)" >> /var/log/keepalived-state.log
-        logger -t keepalived "$NAME is now NTP BACKUP"
+        log "transitioned to BACKUP state (NTP, IPv4)"
         ;;
     FAULT)
-        echo "$TIMESTAMP: $NAME transitioned to FAULT state (NTP)" >> /var/log/keepalived-state.log
-        logger -t keepalived "$NAME is now in NTP FAULT state"
+        log "transitioned to FAULT state (NTP, IPv4)"
+        ;;
+    MASTER6)
+        log "transitioned to MASTER state (NTP, IPv6) - now serving ntp.srv.renner.uno"
+        ;;
+    BACKUP6)
+        log "transitioned to BACKUP state (NTP, IPv6) - released ntp.srv.renner.uno"
+        ;;
+    FAULT6)
+        log "transitioned to FAULT state (NTP, IPv6) - withdrew ntp.srv.renner.uno, unfit to serve time"
         ;;
     *)
         echo "$TIMESTAMP: $NAME unknown state: $TYPE" >> /var/log/keepalived-state.log

ansible/roles/keepalived-ntp/tasks/main.yml

@@ -35,6 +35,121 @@
     state: present
   when: ansible_facts['os_family'] == 'Archlinux'
 
+# --- Verify nothing else is assigning the IPv6 service address ---------------
+#
+# The removal itself lives in withdraw-static.yml, run as a separate fleet-wide
+# play before any host reaches this role (see install-ntp-ha.yml). It has to
+# complete everywhere first: DAD is enabled on the VRRP address, so a static
+# copy on one host and a VRRP-managed copy on another cannot coexist -- the
+# kernel marks the newcomer `dadfailed` and its master serves nothing.
+#
+# What remains here are the guards. They run on every execution, not just the
+# migration, and fail the play if any static assignment survives -- which would
+# both defeat the health gating (a static copy does not disappear when
+# keepalived decides the host is unfit) and break the VRRP copy via DAD.
+#
+# The config file is what gets checked, not the running address, and that is
+# what makes it safe to run every time: keepalived never writes to these files,
+# so anything found is by definition static, whereas the live address is
+# legitimately present on whichever host is master.
+#
+# Prefix-agnostic and \b-anchored on purpose. Matching the full
+# address/prefixlen would fail open against a hand-written assignment using any
+# other mask.
+- name: Detect any remaining static assignment of the service address (ifupdown)
+  ansible.builtin.command:
+    argv:
+      - grep
+      - -rlE
+      - "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
+      - /etc/network/interfaces
+      - /etc/network/interfaces.d/
+  register: keepalived_ntp_static6_ifupdown
+  changed_when: false
+  failed_when: false
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - ansible_facts['os_family'] == 'Debian'
+
+- name: Assert ifupdown is not also assigning the service address
+  ansible.builtin.assert:
+    that:
+      - keepalived_ntp_static6_ifupdown.stdout | default('') | length == 0
+    fail_msg: >-
+      {{ keepalived_ntp_vip6 }} is still assigned statically by ifupdown on
+      this host
+      ({{ keepalived_ntp_static6_ifupdown.stdout_lines | default([]) | join(', ') }}).
+      The automatic removal only matches the exact form
+      `up ip -6 addr replace {{ keepalived_ntp_vip6 }} dev <if> nodad`, so a
+      different prefix length or layout is left in place. Handing the address
+      to keepalived while ifupdown also assigns it means it would NOT be
+      withdrawn when check_ntp fails -- the host would keep serving time on an
+      address VRRP believes it has released. Remove the assignment from the
+      file(s) above, then re-run.
+    success_msg: >-
+      No static ifupdown assignment of {{ keepalived_ntp_vip6 }}; safe for
+      keepalived to own it.
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - ansible_facts['os_family'] == 'Debian'
+
+# On Arch (edge-1) the static assignment lives in a systemd-networkd file owned
+# by the edge-vlan role, so this role must not rewrite it -- but it must not
+# quietly proceed either. install-edge-host.yml is a separate play with no
+# ordering relationship to this one, so nothing guarantees it has been re-run
+# to drop the [Address] block. If it has not, edge-1 would end up holding
+# fd40::123 statically AND running a VRRP instance for the same address:
+# keepalived would believe it had withdrawn the address while networkd kept it
+# assigned, and the host would go on answering NTP with a clock the health
+# check had already rejected.
+#
+# That is the precise failure of 2026-08-04, on the host it happened to, so it
+# fails fast rather than being left to inspection.
+# Matched on the bare address, not the full address/prefixlen, and anchored
+# with \b so it cannot also hit a longer address sharing the prefix (fd40::1234).
+#
+# Searching for the literal "fd40::123/128" would fail open: a hand-written
+# [Address] using any other mask -- /64, say -- would not match, the assert
+# below would pass, and keepalived would be handed an address networkd is
+# still assigning. Since this check is detect-only, with no delete afterwards
+# to catch the mistake, a miss here is silent. The Debian path does not share
+# the problem: `ip -6 addr show ... to <prefix>` matches by range regardless of
+# the assigned prefix length, and its delete is re-verified by assert.
+- name: Detect a static assignment of the service address (systemd-networkd)
+  ansible.builtin.command:
+    argv:
+      - grep
+      - -rlE
+      - "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
+      - /etc/systemd/network/
+  register: keepalived_ntp_static6_networkd
+  changed_when: false
+  failed_when: false
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - ansible_facts['os_family'] == 'Archlinux'
+
+- name: Assert networkd is not also assigning the service address
+  ansible.builtin.assert:
+    that:
+      - keepalived_ntp_static6_networkd.stdout | default('') | length == 0
+    fail_msg: >-
+      {{ keepalived_ntp_vip6 }} is still assigned statically by
+      systemd-networkd on this host
+      ({{ keepalived_ntp_static6_networkd.stdout_lines | default([]) | join(', ') }}).
+      Handing the address to keepalived while networkd also assigns it means
+      it would NOT be withdrawn when check_ntp fails -- the host would keep
+      serving time on an address VRRP believes it has released.
+      Re-apply the edge-vlan role first, whose template no longer carries the
+      [Address] block:
+      ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
+    success_msg: >-
+      No static networkd assignment of {{ keepalived_ntp_vip6 }}; safe for
+      keepalived to own it.
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - ansible_facts['os_family'] == 'Archlinux'
+
 - name: Create keepalived configuration
   ansible.builtin.template:
     src: keepalived.conf.j2

ansible/roles/keepalived-ntp/tasks/withdraw-static.yml

@@ -0,0 +1,134 @@
+---
+# Withdraw a static assignment of the IPv6 NTP service address.
+#
+# Included by install-ntp-ha.yml as a FIRST play across the whole ntp_cluster,
+# before any host is given the NTP_VIP6 instance. That ordering is the point of
+# splitting it out.
+#
+# DAD is enabled on the VRRP address (see keepalived.conf.j2), so a static copy
+# on any host and a VRRP-managed copy on another cannot coexist: the kernel sees
+# the duplicate and marks the newcomer `dadfailed`, leaving a master that
+# believes it is serving while carrying an unusable address. Observed on edge-1
+# on 2026-08-07, where keepalived reported MASTER while the address sat
+# `dadfailed deprecated tentative` because three PVE nodes still held it
+# statically.
+#
+# Doing the withdrawal per host inside the main role could not avoid that: with
+# serial: 1 the first host would claim the address via VRRP while the rest were
+# still static. Fleet-wide first, then claim.
+#
+# fd40::123 was assigned by hand on all four servers, in two different ways,
+# none of it in version control:
+#
+#   PVE nodes  an `up ip -6 addr replace ... nodad` hook in
+#              /etc/network/interfaces
+#   edge-1     an [Address] block in /etc/systemd/network/, owned by the
+#              edge-vlan role and removed by re-applying it -- not touched
+#              here, since this role must not rewrite another role's file
+#
+# There is a service gap between this play and the next, while no host holds
+# the address. It is bounded by the run time of one play and clients simply
+# retry; the alternative is the dadfailed state above, which does not resolve
+# on its own.
+
+- name: Remove static assignment of the NTP service address (ifupdown)
+  ansible.builtin.lineinfile:
+    path: /etc/network/interfaces
+    regexp: '^\s*up\s+ip\s+-6\s+addr\s+(replace|add)\s+{{ keepalived_ntp_vip6 | regex_escape }}\b'
+    state: absent
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - ansible_facts['os_family'] == 'Debian'
+  register: keepalived_ntp_static6_removed
+
+# Everything below is gated on the config line having just been removed, so the
+# runtime delete happens once -- on the migration -- and never again.
+#
+# The gate is load-bearing. Once this has landed, the address is legitimately
+# present on whichever host is the elected v6 master, placed there by
+# keepalived, and a delete cannot tell that apart from a static assignment (the
+# address looks identical either way). Ungated, every later run would delete the
+# master's address and restart keepalived, dropping the service address and
+# forcing a re-election each time -- the flapping this whole change exists to
+# prevent.
+- name: Check whether the service address is still assigned statically
+  ansible.builtin.command:
+    argv:
+      - ip
+      - -6
+      - addr
+      - show
+      - dev
+      - "{{ keepalived_ntp_interface }}"
+      - to
+      - "{{ keepalived_ntp_vip6 }}"
+  register: keepalived_ntp_vip6_present
+  changed_when: false
+  failed_when: false
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - keepalived_ntp_static6_removed.changed | default(false)
+
+# failed_when: false because the desired end state is "address absent". An
+# `ip addr del` against an address that has already gone is the outcome this
+# task wants, not a reason to abort the play for that host.
+- name: Withdraw the statically assigned service address
+  ansible.builtin.command:
+    argv:
+      - ip
+      - -6
+      - addr
+      - del
+      - "{{ keepalived_ntp_vip6 }}"
+      - dev
+      - "{{ keepalived_ntp_interface }}"
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - keepalived_ntp_static6_removed.changed | default(false)
+    - keepalived_ntp_vip6_present.stdout | default('') | length > 0
+  failed_when: false
+
+# The withdrawal gets exactly one attempt -- the gate that makes it idempotent
+# also means it never retries -- and the delete swallows its own errors. So the
+# result is asserted rather than assumed.
+#
+# `ip addr del` matches on address AND prefix length: if a host's static
+# assignment used a different mask or interface than ntp_vip6 /
+# keepalived_ntp_interface, the delete is a silent no-op and the address stays
+# outside VRRP control permanently, with the play reporting success.
+- name: Confirm the static service address was actually withdrawn
+  ansible.builtin.command:
+    argv:
+      - ip
+      - -6
+      - addr
+      - show
+      - dev
+      - "{{ keepalived_ntp_interface }}"
+      - to
+      - "{{ keepalived_ntp_vip6 }}"
+  register: keepalived_ntp_vip6_after
+  changed_when: false
+  failed_when: false
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - keepalived_ntp_static6_removed.changed | default(false)
+
+- name: Assert the service address is no longer assigned outside keepalived
+  ansible.builtin.assert:
+    that:
+      - keepalived_ntp_vip6_after.stdout | default('') | length == 0
+    fail_msg: >-
+      {{ keepalived_ntp_vip6 }} is still assigned on
+      {{ keepalived_ntp_interface }} after the static assignment was removed.
+      `ip -6 addr del` matches address AND prefix length, so this usually means
+      the on-host assignment differs from ntp_vip6 or uses another interface.
+      Leaving it would both keep the address outside VRRP control and make the
+      VRRP-managed copy fail DAD elsewhere. Inspect with
+      `ip -6 addr show dev {{ keepalived_ntp_interface }}` and
+      `grep -rn {{ keepalived_ntp_vip6 }} /etc/network/ /etc/systemd/network/`.
+    success_msg: >-
+      {{ keepalived_ntp_vip6 }} withdrawn; safe for keepalived to claim it.
+  when:
+    - keepalived_ntp_vip6 | length > 0
+    - keepalived_ntp_static6_removed.changed | default(false)

ansible/roles/keepalived-ntp/templates/check-ntp-health.sh.j2

@@ -6,23 +6,80 @@
 # The chrony unit is named 'chrony' on Debian and 'chronyd' on Arch. This is
 # templated rather than hardcoded: a wrong name here fails every check, which
 # parks the host in permanent FAULT so it never takes the VIP.
+#
+# This check must answer "is it safe for clients to take time from this host",
+# NOT merely "is chronyd alive". Those came apart on 2026-08-04: edge-1's dead
+# CMOS battery left its clock ~16 days in the past, chronyd was running,
+# answering queries and listening on 123, and the `local stratum` directive made
+# it advertise itself to clients as a valid reference. Every liveness check
+# passed for ~18h while the host stood ready to serve a clock from July. The
+# earlier version of this script ran `chronyc tracking` and discarded its
+# output -- the one command that knew the answer.
+#
+# Deliberate distinction, because these must not be conflated:
+#   * "provably wrong"  -> chronyd HAS upstream measurements and they say the
+#                          clock is far off. Fail: drop the VIP so a host with a
+#                          sane clock serves instead.
+#   * "no upstream yet" -> nothing reachable / not yet measured. Also fail, but
+#                          note that when the WAN is down every host fails
+#                          together, they all lose the same weight, relative
+#                          priority is unchanged and the VIP stays put. So the
+#                          local-reference fallback still works; what this
+#                          prevents is one host with a broken clock winning.
+
+LOG=/var/log/keepalived-ntp-health.log
+MAX_OFFSET={{ keepalived_ntp_max_offset }}
+
+fail() {
+    echo "$(date): $1" >> "$LOG"
+    exit 1
+}
 
 # Check 1: Is chrony service running?
 if ! systemctl is-active --quiet {{ chrony_service_name }}; then
-    echo "$(date): {{ chrony_service_name }} service not active" >> /var/log/keepalived-ntp-health.log
-    exit 1
+    fail "{{ chrony_service_name }} service not active"
 fi
 
-# Check 2: Is chrony synchronized or at least running?
-if ! chronyc tracking >/dev/null 2>&1; then
-    echo "$(date): Chrony not responding to tracking query" >> /var/log/keepalived-ntp-health.log
-    exit 1
+# Check 2: Is chrony responding, and what does it actually say?
+# Captured, not discarded -- the offset below is the whole point of this script.
+if ! TRACKING=$(chronyc tracking 2>/dev/null); then
+    fail "Chrony not responding to tracking query"
 fi
 
 # Check 3: Is NTP serving clients (port 123)?
 if ! ss -lun | grep -q ':123 '; then
-    echo "$(date): NTP port 123 not listening" >> /var/log/keepalived-ntp-health.log
-    exit 1
+    fail "NTP port 123 not listening"
+fi
+
+# Check 4: Is the clock provably wrong?
+#
+# "System time : <n> seconds (slow|fast) of NTP time" is chronyd's own estimate
+# of how far the system clock is from real time, derived from upstream
+# measurements. On edge-1 during the incident this line read
+# "1376235.375000000 seconds slow of NTP time" while every other check passed.
+OFFSET=$(echo "$TRACKING" | awk '/^System time/ {print $4}')
+if [ -n "$OFFSET" ]; then
+    if awk -v o="$OFFSET" -v m="$MAX_OFFSET" 'BEGIN {exit !(o > m)}'; then
+        fail "System clock off by ${OFFSET}s (max ${MAX_OFFSET}s) - refusing to serve time"
+    fi
+fi
+
+# Check 5: Has chronyd ever validated its clock against a real source?
+#
+# With `local stratum N` configured, chronyd advertises itself as a usable
+# reference even when it has never synchronised to anything -- the upstream
+# chrony documentation is explicit that the local reference activates
+# immediately in that case. So a host that has never reached an upstream still
+# looks authoritative to clients. Refuse the VIP until a real source has been
+# selected.
+#
+# `chronyc sources` marks the selected source with '*' (best) or '+' (combined)
+# in the second column. Neither present means nothing upstream is accepted.
+if ! SOURCES=$(chronyc sources 2>/dev/null); then
+    fail "Chrony not responding to sources query"
+fi
+if ! echo "$SOURCES" | grep -qE '^.\*|^.\+'; then
+    fail "No upstream source selected - clock not validated, refusing to serve time"
 fi
 
 # All checks passed

ansible/roles/keepalived-ntp/templates/keepalived.conf.j2

@@ -39,3 +39,72 @@ vrrp_instance NTP_VIP {
     notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP"
     notify_fault "/usr/local/bin/keepalived-notify.sh FAULT"
 }
+{% if keepalived_ntp_vip6 %}
+
+# IPv6 service address for NTP (ntp.srv.renner.uno). IPv6 is the primary
+# address family here, so this instance -- not the IPv4 one above -- is the
+# path that matters.
+#
+# A separate instance is mandatory, not stylistic: keepalived requires that
+# "all addresses in virtual_ipaddress must be of the same family", so the v6
+# address cannot be added to NTP_VIP.
+#
+# No authentication block: VRRPv3 removed authentication (RFC 5798), and
+# keepalived always uses version 3 for IPv6 instances. Configuring auth here
+# is rejected rather than ignored.
+#
+# The first address is link-local, and must be. RFC 5798 gives an IPv6 virtual
+# router a link-local address as its primary, and keepalived warns
+# "the first IPv6 VIP address should be link local" otherwise. It carries on,
+# but the instance is then sourcing VRRPv3 from something the protocol does not
+# expect.
+#
+# DAD is deliberately left ON (no -nodad). The flag was inherited from the
+# previous design, where every NTP server carried fd40::123 simultaneously and
+# duplicate detection had to be suppressed for that to work at all. Under
+# single-master VRRP the opposite is true: exactly one host should hold the
+# address, so a duplicate is a genuine fault and detecting it is the point.
+#
+# This does mean the migration cannot leave a static copy anywhere while a
+# VRRP-managed one exists -- DAD sees the two and marks the newcomer
+# `dadfailed`, leaving a master that believes it is serving while carrying an
+# unusable address. Observed on edge-1 on 2026-08-07. install-ntp-ha.yml
+# therefore withdraws every static assignment fleet-wide in a first play
+# before any host configures this instance.
+#
+# track_script is what makes this health-gated: the address is only held while
+# check_ntp passes, and is withdrawn as soon as it fails. On withdrawal
+# keepalived sends unsolicited Neighbour Advertisements so client neighbour
+# caches are updated immediately -- a statically assigned address cannot do
+# this, and clients would keep sending to the dead holder until their own
+# neighbour unreachability detection expired.
+vrrp_instance NTP_VIP6 {
+    state {{ keepalived_ntp_state }}
+    interface {{ keepalived_ntp_interface }}
+    virtual_router_id {{ keepalived_ntp_router_id6 }}
+    priority {{ keepalived_ntp_priority }}
+    advert_int 1
+
+    virtual_ipaddress {
+        {{ keepalived_ntp_vip6_linklocal }} dev {{ keepalived_ntp_interface }}
+        {{ keepalived_ntp_vip6 }} dev {{ keepalived_ntp_interface }}
+    }
+
+    # weight 0 overrides the script's own weight, and the distinction decides
+    # whether this works at all. A non-zero weight only SUBTRACTS from priority
+    # on failure -- so the last remaining host, having no one to lose the
+    # election to, stays MASTER and keeps serving time with a clock it has
+    # already reported as wrong. Weight 0 instead transitions the instance to
+    # FAULT after <fall> failures, which releases the address unconditionally.
+    #
+    # That is the required behaviour here: a host unfit to serve time must stop
+    # answering, whether or not a replacement exists.
+    track_script {
+        check_ntp weight 0
+    }
+
+    notify_master "/usr/local/bin/keepalived-notify.sh MASTER6"
+    notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP6"
+    notify_fault "/usr/local/bin/keepalived-notify.sh FAULT6"
+}
+{% endif %}

ansible/roles/loki-server/defaults/main.yml

@@ -1,44 +0,0 @@
----
-# Loki Server Configuration
-
-# Version and download
-loki_version: "3.4.2"
-loki_arch: "linux-amd64"
-loki_download_url: "https://github.com/grafana/loki/releases/download/v{{ loki_version }}/loki-{{ loki_arch }}.zip"
-loki_bin_path: "/usr/local/bin/loki"
-
-# Directories
-loki_data_dir: "/var/lib/loki"
-loki_config_dir: "/etc/loki"
-loki_config_file: "{{ loki_config_dir }}/loki-config.yaml"
-
-# Server configuration
-loki_http_listen_port: 3100
-loki_grpc_listen_port: 9096
-loki_http_listen_address: "0.0.0.0"
-
-# Retention settings (90 days)
-loki_retention_period: "2160h"  # 90 days
-loki_retention_delete_delay: "2h"
-
-# Storage configuration
-loki_storage_type: "filesystem"
-loki_chunks_directory: "{{ loki_data_dir }}/chunks"
-loki_rules_directory: "{{ loki_data_dir }}/rules"
-
-# Index configuration
-loki_index_prefix: "loki_index_"
-loki_index_period: "24h"
-
-# Limits
-loki_ingestion_rate_mb: 10
-loki_ingestion_burst_size_mb: 20
-loki_max_streams_per_user: 10000
-loki_max_entries_limit_per_query: 5000
-
-# Query settings
-loki_max_query_length: "720h"  # 30 days max query span
-
-# Systemd service
-loki_user: "loki"
-loki_group: "loki"

ansible/roles/loki-server/handlers/main.yml

@@ -1,10 +0,0 @@
----
-- name: Restart loki
-  systemd:
-    name: loki
-    state: restarted
-    daemon_reload: yes
-
-- name: Reload systemd
-  systemd:
-    daemon_reload: yes

ansible/roles/loki-server/tasks/main.yml

@@ -1,95 +0,0 @@
----
-# Loki Server Installation and Configuration
-
-- name: Install dependencies
-  apt:
-    name:
-      - unzip
-      - curl
-    state: present
-    update_cache: yes
-
-- name: Create loki user
-  user:
-    name: "{{ loki_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ loki_user }}"
-    group: "{{ loki_group }}"
-    mode: '0755'
-  loop:
-    - "{{ loki_config_dir }}"
-    - "{{ loki_data_dir }}"
-    - "{{ loki_chunks_directory }}"
-    - "{{ loki_rules_directory }}"
-    - "{{ loki_data_dir }}/index"
-    - "{{ loki_data_dir }}/index_cache"
-    - "{{ loki_data_dir }}/compactor"
-
-- name: Download Loki
-  get_url:
-    url: "{{ loki_download_url }}"
-    dest: "/tmp/loki.zip"
-    mode: '0644'
-
-- name: Unzip Loki
-  unarchive:
-    src: "/tmp/loki.zip"
-    dest: "/tmp"
-    remote_src: yes
-
-- name: Install Loki binary
-  copy:
-    src: "/tmp/loki-{{ loki_arch }}"
-    dest: "{{ loki_bin_path }}"
-    mode: '0755'
-    remote_src: yes
-  notify: Restart loki
-
-- name: Create Loki configuration
-  template:
-    src: loki-config.yaml.j2
-    dest: "{{ loki_config_file }}"
-    owner: "{{ loki_user }}"
-    group: "{{ loki_group }}"
-    mode: '0644'
-  notify: Restart loki
-
-- name: Create Loki systemd service
-  template:
-    src: loki.service.j2
-    dest: /etc/systemd/system/loki.service
-    mode: '0644'
-  notify:
-    - Reload systemd
-    - Restart loki
-
-- name: Enable and start Loki service
-  systemd:
-    name: loki
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Wait for Loki to be ready
-  uri:
-    url: "http://127.0.0.1:{{ loki_http_listen_port }}/ready"
-    status_code: 200
-  register: result
-  until: result.status == 200
-  retries: 30
-  delay: 5
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/loki.zip"
-    - "/tmp/loki-{{ loki_arch }}"

ansible/roles/loki-server/templates/loki-config.yaml.j2

@@ -1,72 +0,0 @@
-# Loki Configuration
-# Generated by Ansible
-
-auth_enabled: false
-
-server:
-  http_listen_address: {{ loki_http_listen_address }}
-  http_listen_port: {{ loki_http_listen_port }}
-  grpc_listen_port: {{ loki_grpc_listen_port }}
-  log_level: info
-
-common:
-  instance_addr: 127.0.0.1
-  path_prefix: {{ loki_data_dir }}
-  storage:
-    filesystem:
-      chunks_directory: {{ loki_chunks_directory }}
-      rules_directory: {{ loki_rules_directory }}
-  replication_factor: 1
-  ring:
-    kvstore:
-      store: inmemory
-
-query_range:
-  results_cache:
-    cache:
-      embedded_cache:
-        enabled: true
-        max_size_mb: 100
-
-schema_config:
-  configs:
-    - from: 2024-01-01
-      store: tsdb
-      object_store: filesystem
-      schema: v13
-      index:
-        prefix: {{ loki_index_prefix }}
-        period: {{ loki_index_period }}
-
-storage_config:
-  tsdb_shipper:
-    active_index_directory: {{ loki_data_dir }}/index
-    cache_location: {{ loki_data_dir }}/index_cache
-
-compactor:
-  working_directory: {{ loki_data_dir }}/compactor
-  compaction_interval: 10m
-  retention_enabled: true
-  retention_delete_delay: {{ loki_retention_delete_delay }}
-  retention_delete_worker_count: 150
-  delete_request_store: filesystem
-
-limits_config:
-  retention_period: {{ loki_retention_period }}
-  ingestion_rate_mb: {{ loki_ingestion_rate_mb }}
-  ingestion_burst_size_mb: {{ loki_ingestion_burst_size_mb }}
-  max_streams_per_user: {{ loki_max_streams_per_user }}
-  max_entries_limit_per_query: {{ loki_max_entries_limit_per_query }}
-  max_query_length: {{ loki_max_query_length }}
-  reject_old_samples: true
-  reject_old_samples_max_age: 168h  # 7 days
-
-ruler:
-  alertmanager_url: ""
-  storage:
-    type: local
-    local:
-      directory: {{ loki_rules_directory }}
-
-analytics:
-  reporting_enabled: false

ansible/roles/loki-server/templates/loki.service.j2

@@ -1,25 +0,0 @@
-[Unit]
-Description=Loki Log Aggregation System
-Documentation=https://grafana.com/docs/loki/latest/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ loki_user }}
-Group={{ loki_group }}
-ExecStart={{ loki_bin_path }} -config.file={{ loki_config_file }}
-ExecReload=/bin/kill -HUP $MAINPID
-Restart=on-failure
-RestartSec=10s
-LimitNOFILE=65536
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadWritePaths={{ loki_data_dir }}
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/mqtt-bridge/defaults/main.yml

@@ -0,0 +1,12 @@
+---
+# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
+# mosquitto on loopback, which bridges to the central cluster broker over TLS
+# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
+# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
+# blips. The local broker writes nothing to the eMMC (persistence off, logs to
+# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
+# Upstream cluster broker the local bridge connects to (TLS; the public
+# Let's Encrypt cert is validated against the system CA bundle).
+edge_mqtt_cluster_host: mqtt.renner.uno
+edge_mqtt_cluster_tls_port: 8883
+edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt

ansible/roles/mqtt-bridge/handlers/main.yml

@@ -0,0 +1,7 @@
+---
+# The MQTT clients (zwave-js, sml2mqtt, open3e, e3oncan) reconnect on their
+# own once the local broker is back.
+- name: Restart mosquitto
+  ansible.builtin.service:
+    name: mosquitto
+    state: restarted

ansible/roles/mqtt-bridge/tasks/main.yml

@@ -0,0 +1,27 @@
+---
+# Local MQTT bridge broker.
+#
+# Edge clients (open3e, e3oncan, sml2mqtt) publish here over loopback; this
+# broker bridges to the cluster broker over TLS, because those clients have no
+# native MQTTS. Persistence is off and logging goes to stdout so it writes
+# nothing to local flash.
+
+- name: Install mosquitto
+  ansible.builtin.package:
+    name: mosquitto
+    state: present
+
+- name: Deploy local mosquitto bridge config
+  ansible.builtin.template:
+    src: edge-mqtt-bridge.conf.j2
+    dest: /etc/mosquitto/mosquitto.conf
+    owner: root
+    group: mosquitto
+    mode: "0640"   # carries the cluster bridge credentials
+  notify: Restart mosquitto
+
+- name: Enable local mosquitto bridge
+  ansible.builtin.systemd:
+    name: mosquitto
+    enabled: true
+    state: started

ansible/roles/mqtt-bridge/templates/edge-mqtt-bridge.conf.j2

@@ -1,4 +1,4 @@
-# Managed by roles/edge-host. Local mosquitto that the edge clients (open3e,
+# Managed by roles/mqtt-bridge. Local mosquitto that the edge clients (open3e,
 # e3oncan, sml2mqtt) publish to over loopback; it bridges to the central
 # cluster broker over TLS. open3e/e3oncan have no native MQTTS, so this is how
 # edge-1's MQTT gets off plaintext on the untrusted IoT VLAN, and it buffers

ansible/roles/power-meter-sml/defaults/main.yml

@@ -0,0 +1,23 @@
+---
+# Power-meter reader: sml2mqtt against the FTDI serial meter head.
+
+# Service root. Kept per-role so the role is usable on its own.
+edge_base_dir: /opt/edge-host
+
+# Own compose project directory. Unchanged from the path used under the
+# previous shared project.
+sml_base_dir: "{{ edge_base_dir }}/sml2mqtt"
+
+# The compose file that used to hold both this service and zwave-js. Removed
+# once, so the fixed container_name is released for the new projects.
+edge_legacy_compose: "{{ edge_base_dir }}/docker-compose.yml"
+
+# Stable by-id symlink -- see the zwave-gateway role.
+edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
+
+# Hand-pinned on purpose: critical meter bridge, deliberately excluded from
+# image automation.
+edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
+
+# Secret from inventory/group_vars/edge_host/vault.yml:
+#   edge_sml2mqtt_mqtt_password

ansible/roles/power-meter-sml/handlers/main.yml

@@ -0,0 +1,6 @@
+---
+- name: Restart sml2mqtt
+  ansible.builtin.command:
+    cmd: docker compose up -d --force-recreate
+    chdir: "{{ sml_base_dir }}"
+  changed_when: true

ansible/roles/power-meter-sml/tasks/main.yml

@@ -0,0 +1,55 @@
+---
+# Power-meter reader: sml2mqtt against the FTDI serial meter head.
+#
+# Its own compose project. It shared a compose file with the Z-Wave gateway,
+# which meant a change to either recreated both -- two unrelated services bound
+# to different USB devices. Data paths are unchanged.
+
+- name: Create sml2mqtt directory
+  ansible.builtin.file:
+    path: "{{ sml_base_dir }}"
+    state: directory
+    mode: "0755"
+
+# sml2mqtt reads only a config file and has no env override for the password,
+# so the rendered config carries it -- 0600, root-owned, same trust boundary as
+# the k8s init-container approach it replaced.
+- name: Deploy sml2mqtt config
+  ansible.builtin.template:
+    src: sml2mqtt-config.yml.j2
+    dest: "{{ sml_base_dir }}/config.yml"
+    mode: "0600"
+  notify: Restart sml2mqtt
+
+- name: Deploy sml2mqtt compose project
+  ansible.builtin.template:
+    src: docker-compose.yml.j2
+    dest: "{{ sml_base_dir }}/docker-compose.yml"
+    mode: "0644"
+  notify: Restart sml2mqtt
+
+- name: Pull sml2mqtt image
+  ansible.builtin.command:
+    cmd: docker compose pull
+    chdir: "{{ sml_base_dir }}"
+  changed_when: false
+
+# Tolerates a missing USB device, as for the Z-Wave gateway.
+- name: Start sml2mqtt
+  ansible.builtin.command:
+    cmd: docker compose up -d
+    chdir: "{{ sml_base_dir }}"
+  register: sml_compose_up
+  failed_when: >-
+    sml_compose_up.rc != 0
+    and 'error gathering device information' not in sml_compose_up.stderr
+  changed_when: >-
+    'Started' in sml_compose_up.stderr or 'Created' in sml_compose_up.stderr
+    or 'Started' in sml_compose_up.stdout or 'Creating' in sml_compose_up.stdout
+
+- name: Warn if sml2mqtt is waiting for its USB device
+  ansible.builtin.debug:
+    msg: >-
+      sml2mqtt could not start -- the FTDI meter head is not present:
+      {{ sml_compose_up.stderr_lines | select('search', 'device') | list }}
+  when: sml_compose_up.rc != 0

ansible/roles/power-meter-sml/templates/docker-compose.yml.j2

@@ -0,0 +1,21 @@
+# Managed by ansible (roles/power-meter-sml).
+#
+# Its own compose project -- see the role tasks for why it no longer shares a
+# file with the Z-Wave gateway.
+#
+# Outbound-only (serial in, MQTT out). Host networking sidesteps the docker
+# NAT66 gap: the broker LB is IPv6-first and bridge-network v6 SYNs leave
+# unmasqueraded and die. The host stack reaches it fine (open3e proves it
+# continuously).
+services:
+  sml2mqtt:
+    image: {{ edge_sml2mqtt_image }}
+    container_name: sml2mqtt
+    restart: unless-stopped
+    network_mode: host
+    # Serial exclusivity: this is the ONLY reader of the meter port --
+    # HA's edl21 integration stays disabled.
+    devices:
+      - "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
+    volumes:
+      - ./config.yml:/sml2mqtt/config.yml:ro

ansible/roles/power-meter-sml/templates/sml2mqtt-config.yml.j2

@@ -1,4 +1,4 @@
-# Managed by ansible (roles/edge-host). Ported 1:1 from the k8s ConfigMap
+# Managed by ansible (roles/power-meter-sml). Ported 1:1 from the k8s ConfigMap
 # sml2mqtt-config; only host and password delivery differ (broker is now
 # reached via its LoadBalancer hostname, password rendered by ansible
 # instead of an init container).

ansible/roles/prometheus-server/defaults/main.yml

@@ -1,48 +0,0 @@
----
-# Prometheus Server Configuration
-
-# Version and download
-prometheus_version: "3.2.1"
-prometheus_arch: "linux-amd64"
-prometheus_download_url: "https://github.com/prometheus/prometheus/releases/download/v{{ prometheus_version }}/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}.tar.gz"
-prometheus_bin_path: "/usr/local/bin/prometheus"
-prometheus_promtool_path: "/usr/local/bin/promtool"
-
-# Directories
-prometheus_data_dir: "/var/lib/prometheus"
-prometheus_config_dir: "/etc/prometheus"
-prometheus_config_file: "{{ prometheus_config_dir }}/prometheus.yml"
-prometheus_rules_dir: "{{ prometheus_config_dir }}/rules"
-prometheus_targets_dir: "{{ prometheus_config_dir }}/targets"
-
-# Server configuration
-prometheus_listen_address: "0.0.0.0"
-prometheus_listen_port: 9090
-prometheus_web_external_url: ""
-
-# Retention settings (90 days)
-prometheus_retention_time: "90d"
-prometheus_retention_size: "30GB"
-
-# Scrape settings
-prometheus_scrape_interval: "30s"
-prometheus_evaluation_interval: "30s"
-prometheus_scrape_timeout: "10s"
-
-# Remote write (for Alloy agents)
-prometheus_remote_write_enabled: true
-
-# Systemd service
-prometheus_user: "prometheus"
-prometheus_group: "prometheus"
-
-# Alert notification endpoints
-prometheus_alertmanager_url: ""
-
-# Email alerting (via Grafana)
-alerting_email_enabled: true
-alerting_email_to: ""
-
-# Home Assistant webhook for alerts
-alerting_ha_webhook_enabled: true
-alerting_ha_webhook_url: ""

ansible/roles/prometheus-server/handlers/main.yml

@@ -1,15 +0,0 @@
----
-- name: Restart prometheus
-  systemd:
-    name: prometheus
-    state: restarted
-    daemon_reload: yes
-
-- name: Reload prometheus
-  systemd:
-    name: prometheus
-    state: reloaded
-
-- name: Reload systemd
-  systemd:
-    daemon_reload: yes

ansible/roles/prometheus-server/tasks/main.yml

@@ -1,129 +0,0 @@
----
-# Prometheus Server Installation and Configuration
-
-- name: Install dependencies
-  apt:
-    name:
-      - curl
-    state: present
-    update_cache: yes
-
-- name: Create prometheus user
-  user:
-    name: "{{ prometheus_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0755'
-  loop:
-    - "{{ prometheus_config_dir }}"
-    - "{{ prometheus_data_dir }}"
-    - "{{ prometheus_rules_dir }}"
-    - "{{ prometheus_targets_dir }}"
-
-- name: Download Prometheus
-  get_url:
-    url: "{{ prometheus_download_url }}"
-    dest: "/tmp/prometheus.tar.gz"
-    mode: '0644'
-
-- name: Extract Prometheus
-  unarchive:
-    src: "/tmp/prometheus.tar.gz"
-    dest: "/tmp"
-    remote_src: yes
-
-- name: Install Prometheus binaries
-  copy:
-    src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item.src }}"
-    dest: "{{ item.dest }}"
-    mode: '0755'
-    remote_src: yes
-  loop:
-    - { src: "prometheus", dest: "{{ prometheus_bin_path }}" }
-    - { src: "promtool", dest: "{{ prometheus_promtool_path }}" }
-  notify: Restart prometheus
-
-- name: Check for console templates
-  stat:
-    path: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/consoles"
-  register: consoles_dir
-
-- name: Copy console templates
-  copy:
-    src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item }}/"
-    dest: "{{ prometheus_config_dir }}/{{ item }}/"
-    remote_src: yes
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-  loop:
-    - consoles
-    - console_libraries
-  when: consoles_dir.stat.exists
-
-- name: Create Prometheus configuration
-  template:
-    src: prometheus.yml.j2
-    dest: "{{ prometheus_config_file }}"
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0644'
-    validate: "{{ prometheus_promtool_path }} check config %s"
-  notify: Restart prometheus
-
-- name: Create alert rules
-  template:
-    src: alert-rules.yml.j2
-    dest: "{{ prometheus_rules_dir }}/alert-rules.yml"
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0644'
-  notify: Reload prometheus
-
-- name: Create node exporter targets file
-  template:
-    src: nodes.yml.j2
-    dest: "{{ prometheus_targets_dir }}/nodes.yml"
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0644'
-
-- name: Create Prometheus systemd service
-  template:
-    src: prometheus.service.j2
-    dest: /etc/systemd/system/prometheus.service
-    mode: '0644'
-  notify:
-    - Reload systemd
-    - Restart prometheus
-
-- name: Enable and start Prometheus service
-  systemd:
-    name: prometheus
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Wait for Prometheus to be ready
-  uri:
-    url: "http://127.0.0.1:{{ prometheus_listen_port }}/-/ready"
-    status_code: 200
-  register: result
-  until: result.status == 200
-  retries: 30
-  delay: 5
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/prometheus.tar.gz"
-    - "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}"

ansible/roles/prometheus-server/templates/alert-rules.yml.j2

@@ -1,100 +0,0 @@
-# Prometheus Alert Rules
-# Generated by Ansible
-
-groups:
-  - name: infrastructure
-    rules:
-      # Disk usage alerts
-      - alert: HighDiskUsage
-        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 15
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Disk usage is above 85% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
-
-      - alert: CriticalDiskUsage
-        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 5
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Critical disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Disk usage is above 95% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
-
-      # Memory alerts
-      - alert: HighMemoryUsage
-        expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 > 90
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High memory usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Memory usage is above 90% ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
-
-      # CPU alerts
-      - alert: HighCPUUsage
-        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
-        for: 10m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High CPU usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "CPU usage is above 85% for 10 minutes ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
-
-  - name: services
-    rules:
-      # Service down alerts
-      - alert: ServiceDown
-        expr: up == 0
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Service {{ '{{' }} $labels.job {{ '}}' }} down on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Service has been unreachable for more than 2 minutes"
-
-      # DNS VIP health
-      - alert: DNSVIPDown
-        expr: absent(up{job="dns-servers"} == 1) or count(up{job="dns-servers"} == 1) < 2
-        for: 1m
-        labels:
-          severity: critical
-        annotations:
-          summary: "DNS cluster degraded"
-          description: "Less than 2 DNS servers are healthy"
-
-  - name: security
-    rules:
-      # High auth failure rate (placeholder - needs log-based metrics from Loki)
-      - alert: HighAuthFailureRate
-        expr: rate(authentik_login_failed_total[5m]) > 5
-        for: 2m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High authentication failure rate"
-          description: "More than 5 auth failures per minute detected"
-
-  - name: loki
-    rules:
-      # Loki health
-      - alert: LokiDown
-        expr: up{job="loki"} == 0
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Loki log aggregation is down"
-          description: "Loki has been unreachable for more than 2 minutes"
-
-      # Loki ingestion rate
-      - alert: LokiHighIngestionRate
-        expr: rate(loki_distributor_bytes_received_total[5m]) > 10485760
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High log ingestion rate in Loki"
-          description: "Log ingestion rate is above 10MB/s"

ansible/roles/prometheus-server/templates/nodes.yml.j2

@@ -1,26 +0,0 @@
-# Node Exporter targets - auto-generated by Ansible
-# Updates require re-running the prometheus playbook
-#
-# NOTE: Most hosts have Alloy agents that push metrics via remote_write.
-# This file only includes hosts that need direct scraping (standalone node_exporter).
-# Alloy-managed hosts are excluded to avoid duplicate "down" targets.
-
-# PVE cluster nodes - need direct scraping (no Alloy agent)
-- targets:
-    - '10.2.40.201:9100'
-  labels:
-    job: 'node'
-    group: 'pve_cluster'
-    hostname: 'jupiter'
-- targets:
-    - '10.2.40.202:9100'
-  labels:
-    job: 'node'
-    group: 'pve_cluster'
-    hostname: 'neptune'
-- targets:
-    - '10.2.40.203:9100'
-  labels:
-    job: 'node'
-    group: 'pve_cluster'
-    hostname: 'saturn'

ansible/roles/prometheus-server/templates/prometheus.service.j2

@@ -1,38 +0,0 @@
-[Unit]
-Description=Prometheus Monitoring System
-Documentation=https://prometheus.io/docs/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ prometheus_user }}
-Group={{ prometheus_group }}
-ExecStart={{ prometheus_bin_path }} \
-  --config.file={{ prometheus_config_file }} \
-  --storage.tsdb.path={{ prometheus_data_dir }} \
-  --storage.tsdb.retention.time={{ prometheus_retention_time }} \
-  --storage.tsdb.retention.size={{ prometheus_retention_size }} \
-  --web.listen-address={{ prometheus_listen_address }}:{{ prometheus_listen_port }} \
-{% if prometheus_remote_write_enabled %}
-  --web.enable-remote-write-receiver \
-{% endif %}
-{% if prometheus_web_external_url %}
-  --web.external-url={{ prometheus_web_external_url }} \
-{% endif %}
-  --web.enable-lifecycle
-
-ExecReload=/bin/kill -HUP $MAINPID
-Restart=on-failure
-RestartSec=10s
-LimitNOFILE=65536
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadWritePaths={{ prometheus_data_dir }}
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/prometheus-server/templates/prometheus.yml.j2

@@ -1,303 +0,0 @@
-# Prometheus Configuration
-# Generated by Ansible
-
-global:
-  scrape_interval: {{ prometheus_scrape_interval }}
-  evaluation_interval: {{ prometheus_evaluation_interval }}
-  scrape_timeout: {{ prometheus_scrape_timeout }}
-  external_labels:
-    cluster: homelab
-    datacenter: renner
-
-rule_files:
-  - "{{ prometheus_rules_dir }}/*.yml"
-
-{% if prometheus_alertmanager_url %}
-alerting:
-  alertmanagers:
-    - static_configs:
-        - targets:
-            - {{ prometheus_alertmanager_url }}
-{% endif %}
-
-scrape_configs:
-  # Prometheus self-monitoring
-  - job_name: 'prometheus'
-    static_configs:
-      - targets: ['localhost:{{ prometheus_listen_port }}']
-
-  # Loki metrics
-  - job_name: 'loki'
-    static_configs:
-      - targets: ['loki.srv.renner.uno:3100']
-
-  # File-based service discovery for dynamic targets
-  - job_name: 'alloy-nodes'
-    file_sd_configs:
-      - files:
-          - '{{ prometheus_targets_dir }}/*.yml'
-          - '{{ prometheus_targets_dir }}/*.json'
-        refresh_interval: 30s
-
-{% if prometheus_remote_write_enabled %}
-# Remote write receiver for Alloy agents
-# Agents push metrics to /api/v1/write
-{% endif %}
-
-# Static targets for infrastructure components
-  # Proxmox VE API metrics (via pve-exporter)
-  - job_name: 'pve'
-    scrape_interval: 60s
-    scrape_timeout: 55s
-    static_configs:
-      - targets:
-          - '10.2.40.201'  # jupiter
-          - '10.2.40.202'  # neptune
-          - '10.2.40.203'  # saturn
-    metrics_path: /pve
-    params:
-      module: [default]
-      cluster: ['1']
-      node: ['1']
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9221  # pve-exporter address
-      - target_label: group
-        replacement: proxmox
-      - target_label: type
-        replacement: hypervisor
-
-  # RustFS (S3-compatible storage)
-  # NOTE: RustFS uses OpenTelemetry push-model via --obs-endpoint, not Prometheus pull.
-  # Unlike MinIO, it doesn't expose /minio/v2/metrics/cluster endpoint.
-  # Metrics collection would require setting up RUSTFS_OBS_ENDPOINT to push to an OTEL collector.
-  # For now, node-level metrics are available via Alloy agents on the RustFS hosts.
-
-  # Home Assistant metrics
-  - job_name: 'homeassistant'
-    metrics_path: /api/prometheus
-    bearer_token: '{{ homeassistant_prometheus_token }}'
-    scheme: http
-    static_configs:
-      - targets: ['homeassistant.iot.renner.uno:8123']
-        labels:
-          group: 'smarthome'
-          type: 'homeassistant'
-
-  # UniFi Poller metrics
-  - job_name: 'unifi'
-    static_configs:
-      - targets: ['localhost:9130']
-        labels:
-          group: 'network'
-          type: 'unifi'
-
-  # Caddy reverse proxy metrics (HA cluster)
-  - job_name: 'caddy'
-    static_configs:
-      - targets:
-          - 'caddy.dmz.renner.uno:2019'
-          - 'caddy2.dmz.renner.uno:2019'
-        labels:
-          group: 'ingress'
-          type: 'reverse-proxy'
-
-  # PostgreSQL metrics (via postgres_exporter)
-  - job_name: 'postgresql'
-    static_configs:
-      - targets: ['localhost:9187']
-        labels:
-          group: 'database'
-          type: 'postgresql'
-
-  # Authentik identity provider metrics
-  - job_name: 'authentik'
-    static_configs:
-      - targets:
-          - 'authentik-app-1.srv.renner.uno:9300'
-          - 'authentik-app-2.srv.renner.uno:9300'
-        labels:
-          group: 'identity'
-          type: 'authentik'
-
-  # Node Exporter - system metrics from all hosts
-  - job_name: 'node'
-    file_sd_configs:
-      - files:
-          - '{{ prometheus_targets_dir }}/nodes.yml'
-        refresh_interval: 60s
-
-  # Chrony Exporter - NTP metrics from PVE cluster
-  - job_name: 'chrony'
-    static_configs:
-      - targets:
-          - '10.2.40.201:9123'
-          - '10.2.40.202:9123'
-          - '10.2.40.203:9123'
-        labels:
-          group: 'ntp'
-          type: 'time-server'
-    relabel_configs:
-      - source_labels: [__address__]
-        regex: '10\.2\.40\.201:.*'
-        target_label: hostname
-        replacement: 'jupiter'
-      - source_labels: [__address__]
-        regex: '10\.2\.40\.202:.*'
-        target_label: hostname
-        replacement: 'neptune'
-      - source_labels: [__address__]
-        regex: '10\.2\.40\.203:.*'
-        target_label: hostname
-        replacement: 'saturn'
-
-  # Immich photo management metrics
-  - job_name: 'immich'
-    static_configs:
-      - targets: ['immich.srv.renner.uno:8081']
-        labels:
-          group: 'media'
-          type: 'photo-management'
-
-  # Blackbox Exporter - HTTP endpoint probing
-  - job_name: 'blackbox-http'
-    metrics_path: /probe
-    params:
-      module: [http_2xx]
-    static_configs:
-      - targets:
-          - https://gitea.renner.uno
-          - https://immich.renner.uno
-          - https://auth.renner.uno
-          - https://ha.renner.uno
-          - https://grafana.renner.uno
-          - https://prometheus.renner.uno
-        labels:
-          group: 'probe'
-          type: 'http'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - TCP probing
-  - job_name: 'blackbox-tcp'
-    metrics_path: /probe
-    params:
-      module: [tcp_connect]
-    static_configs:
-      - targets:
-          - 10.2.40.135:5432  # PostgreSQL HA VIP
-        labels:
-          group: 'probe'
-          type: 'tcp'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - DNS probing (external domains like google.com)
-  - job_name: 'blackbox-dns-external'
-    metrics_path: /probe
-    params:
-      module: [dns_external]
-    static_configs:
-      - targets:
-          - 10.2.40.53  # Proxmox DNS VIP
-          - 10.2.40.1   # UDM Gateway DNS
-        labels:
-          group: 'probe'
-          type: 'dns'
-          query_type: 'external'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - DNS probing (internal domains like k8s-1.srv.renner.uno)
-  - job_name: 'blackbox-dns-internal'
-    metrics_path: /probe
-    params:
-      module: [dns_internal]
-    static_configs:
-      - targets:
-          - 10.2.40.53  # Proxmox DNS VIP
-          - 10.2.40.1   # UDM Gateway DNS
-        labels:
-          group: 'probe'
-          type: 'dns'
-          query_type: 'internal'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - DNS probing (cached domains)
-  - job_name: 'blackbox-dns-cached'
-    metrics_path: /probe
-    params:
-      module: [dns_cached]
-    static_configs:
-      - targets:
-          - 10.2.40.53  # Proxmox DNS VIP
-          - 10.2.40.1   # UDM Gateway DNS
-        labels:
-          group: 'probe'
-          type: 'dns'
-          query_type: 'cached'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-{% if snmp_synology_targets is defined and snmp_synology_targets | length > 0 %}
-  # Synology NAS SNMP metrics
-  # snmp_exporter v0.27+ requires both synology and ucd_la_table modules
-  - job_name: 'synology'
-    scrape_interval: 60s
-    scrape_timeout: 55s
-    metrics_path: /snmp
-    params:
-      module: [synology, ucd_la_table, ucd_memory, if_mib]
-{% if snmp_use_v3 | default(false) %}
-      auth: [synology_v3]
-{% else %}
-      auth: [synology_v2]
-{% endif %}
-    static_configs:
-      - targets:
-{% for target in snmp_synology_targets %}
-          - '{{ target.target }}'  # {{ target.name }}
-{% endfor %}
-        labels:
-          group: 'storage'
-          type: 'nas'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__address__]
-        regex: '([^:]+)(:\d+)?'
-        replacement: '${1}'
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9116  # snmp_exporter address
-{% endif %}

ansible/roles/promtail/defaults/main.yml

@@ -1,36 +0,0 @@
----
-# Promtail Configuration
-
-# Version and download
-promtail_version: "3.4.2"
-promtail_arch: "linux-amd64"
-promtail_download_url: "https://github.com/grafana/loki/releases/download/v{{ promtail_version }}/promtail-{{ promtail_arch }}.zip"
-promtail_bin_path: "/usr/local/bin/promtail"
-
-# Directories
-promtail_config_dir: "/etc/promtail"
-promtail_config_file: "{{ promtail_config_dir }}/promtail-config.yaml"
-promtail_positions_file: "/var/lib/promtail/positions.yaml"
-
-# Server configuration
-promtail_http_listen_port: 9080
-promtail_http_listen_address: "0.0.0.0"
-
-# Loki server
-promtail_loki_url: "http://loki.srv.renner.uno:3100/loki/api/v1/push"
-
-# Log scrape configurations
-# Override in playbook or host_vars for specific services
-promtail_scrape_configs:
-  - job_name: system
-    static_configs:
-      - targets:
-          - localhost
-        labels:
-          job: syslog
-          host: "{{ inventory_hostname }}"
-          __path__: /var/log/syslog
-
-# Systemd service
-promtail_user: "promtail"
-promtail_group: "promtail"

ansible/roles/promtail/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart promtail
-  systemd:
-    name: promtail
-    state: restarted
-    daemon_reload: yes

ansible/roles/promtail/tasks/main.yml

@@ -1,112 +0,0 @@
----
-- name: Install dependencies
-  apt:
-    name:
-      - unzip
-      - curl
-    state: present
-    update_cache: yes
-
-- name: Create promtail group
-  group:
-    name: "{{ promtail_group }}"
-    system: yes
-    state: present
-
-- name: Create promtail user
-  user:
-    name: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    home: /var/lib/promtail
-    create_home: yes
-    state: present
-
-- name: Add promtail user to systemd-journal group
-  user:
-    name: "{{ promtail_user }}"
-    groups: systemd-journal
-    append: yes
-
-- name: Create config directory
-  file:
-    path: "{{ promtail_config_dir }}"
-    state: directory
-    owner: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    mode: '0755'
-
-- name: Create positions directory
-  file:
-    path: "{{ promtail_positions_file | dirname }}"
-    state: directory
-    owner: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    mode: '0755'
-
-- name: Check if promtail binary exists
-  stat:
-    path: "{{ promtail_bin_path }}"
-  register: promtail_binary
-
-- name: Get installed promtail version
-  command: "{{ promtail_bin_path }} --version"
-  register: installed_version
-  changed_when: false
-  failed_when: false
-  when: promtail_binary.stat.exists
-
-- name: Download promtail
-  get_url:
-    url: "{{ promtail_download_url }}"
-    dest: /tmp/promtail.zip
-    mode: '0644'
-  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
-
-- name: Extract promtail binary
-  unarchive:
-    src: /tmp/promtail.zip
-    dest: /tmp/
-    remote_src: yes
-  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
-
-- name: Install promtail binary
-  copy:
-    src: "/tmp/promtail-{{ promtail_arch }}"
-    dest: "{{ promtail_bin_path }}"
-    remote_src: yes
-    mode: '0755'
-  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
-  notify: Restart promtail
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - /tmp/promtail.zip
-    - "/tmp/promtail-{{ promtail_arch }}"
-
-- name: Configure promtail
-  template:
-    src: promtail-config.yaml.j2
-    dest: "{{ promtail_config_file }}"
-    owner: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    mode: '0644'
-  notify: Restart promtail
-
-- name: Create systemd service
-  template:
-    src: promtail.service.j2
-    dest: /etc/systemd/system/promtail.service
-    mode: '0644'
-  notify: Restart promtail
-
-- name: Enable and start promtail
-  systemd:
-    name: promtail
-    state: started
-    enabled: yes
-    daemon_reload: yes

ansible/roles/promtail/templates/promtail-config.yaml.j2

@@ -1,79 +0,0 @@
-server:
-  http_listen_port: {{ promtail_http_listen_port }}
-  http_listen_address: {{ promtail_http_listen_address }}
-  grpc_listen_port: 0
-
-positions:
-  filename: {{ promtail_positions_file }}
-
-clients:
-  - url: {{ promtail_loki_url }}
-    tenant_id: default
-
-scrape_configs:
-{% for config in promtail_scrape_configs %}
-  - job_name: {{ config.job_name }}
-{% if config.journal is defined %}
-    journal:
-      max_age: {{ config.journal.max_age | default('12h') }}
-      labels:
-        job: {{ config.job_name }}
-        host: {{ inventory_hostname }}
-{% if config.journal.matches is defined %}
-      matches: {{ config.journal.matches }}
-{% endif %}
-{% if config.relabel_configs is defined %}
-    relabel_configs:
-{% for relabel in config.relabel_configs %}
-      - source_labels: {{ relabel.source_labels | to_json }}
-        target_label: {{ relabel.target_label }}
-{% if relabel.regex is defined %}
-        regex: '{{ relabel.regex }}'
-{% endif %}
-{% if relabel.replacement is defined %}
-        replacement: '{{ relabel.replacement }}'
-{% endif %}
-{% endfor %}
-{% endif %}
-{% endif %}
-{% if config.static_configs is defined %}
-    static_configs:
-{% for static in config.static_configs %}
-      - targets:
-{% for target in static.targets %}
-          - {{ target }}
-{% endfor %}
-        labels:
-{% for key, value in static.labels.items() %}
-          {{ key }}: {{ value }}
-{% endfor %}
-{% endfor %}
-{% if config.pipeline_stages is defined %}
-    pipeline_stages:
-{% for stage in config.pipeline_stages %}
-{% if stage.multiline is defined %}
-      - multiline:
-          firstline: '{{ stage.multiline.firstline }}'
-{% if stage.multiline.max_wait_time is defined %}
-          max_wait_time: {{ stage.multiline.max_wait_time }}
-{% endif %}
-{% endif %}
-{% if stage.regex is defined %}
-      - regex:
-          expression: '{{ stage.regex.expression }}'
-{% endif %}
-{% if stage.labels is defined %}
-      - labels:
-{% for label in stage.labels %}
-          {{ label }}:
-{% endfor %}
-{% endif %}
-{% if stage.timestamp is defined %}
-      - timestamp:
-          source: {{ stage.timestamp.source }}
-          format: '{{ stage.timestamp.format }}'
-{% endif %}
-{% endfor %}
-{% endif %}
-{% endif %}
-{% endfor %}

ansible/roles/promtail/templates/promtail.service.j2

@@ -1,24 +0,0 @@
-[Unit]
-Description=Promtail Log Collector
-Documentation=https://grafana.com/docs/loki/latest/clients/promtail/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ promtail_user }}
-Group={{ promtail_group }}
-ExecStart={{ promtail_bin_path }} -config.file={{ promtail_config_file }}
-Restart=always
-RestartSec=5
-TimeoutStopSec=30
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadWritePaths=/var/lib/promtail
-CapabilityBoundingSet=CAP_DAC_READ_SEARCH
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/pve-exporter/defaults/main.yml

@@ -1,19 +0,0 @@
----
-# PVE Exporter defaults
-
-# Installation
-pve_exporter_version: "3.4.5"
-pve_exporter_install_dir: /opt/pve-exporter
-pve_exporter_config_dir: /etc/pve-exporter
-pve_exporter_venv_path: "{{ pve_exporter_install_dir }}/venv"
-
-# Service
-pve_exporter_user: pve-exporter
-pve_exporter_group: pve-exporter
-pve_exporter_port: 9221
-
-# Proxmox connection (override with vault variables)
-pve_exporter_pve_user: "{{ pve_user | default('root@pam') }}"
-pve_exporter_pve_token_name: "{{ pve_token_id | default('ansible') }}"
-pve_exporter_pve_token_value: "{{ pve_token_secret }}"
-pve_exporter_pve_verify_ssl: false

ansible/roles/pve-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart pve-exporter
-  systemd:
-    name: pve-exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/pve-exporter/tasks/main.yml

@@ -1,78 +0,0 @@
----
-# PVE Exporter installation and configuration
-
-- name: Install Python dependencies
-  apt:
-    name:
-      - python3
-      - python3-pip
-      - python3-venv
-    state: present
-    update_cache: yes
-
-- name: Create pve-exporter system user
-  user:
-    name: "{{ pve_exporter_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create installation directory
-  file:
-    path: "{{ pve_exporter_install_dir }}"
-    state: directory
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    mode: '0755'
-
-- name: Create configuration directory
-  file:
-    path: "{{ pve_exporter_config_dir }}"
-    state: directory
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    mode: '0750'
-
-- name: Create Python virtual environment
-  command: python3 -m venv {{ pve_exporter_venv_path }}
-  args:
-    creates: "{{ pve_exporter_venv_path }}/bin/python"
-
-- name: Set venv ownership
-  file:
-    path: "{{ pve_exporter_venv_path }}"
-    state: directory
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    recurse: yes
-
-- name: Install prometheus-pve-exporter
-  pip:
-    name: prometheus-pve-exporter
-    virtualenv: "{{ pve_exporter_venv_path }}"
-    state: present
-
-- name: Create configuration file
-  template:
-    src: pve.yml.j2
-    dest: "{{ pve_exporter_config_dir }}/pve.yml"
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    mode: '0640'
-  notify: Restart pve-exporter
-
-- name: Create systemd service
-  template:
-    src: pve-exporter.service.j2
-    dest: /etc/systemd/system/pve-exporter.service
-    owner: root
-    group: root
-    mode: '0644'
-  notify: Restart pve-exporter
-
-- name: Enable and start pve-exporter service
-  systemd:
-    name: pve-exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes

ansible/roles/pve-exporter/templates/pve-exporter.service.j2

@@ -1,22 +0,0 @@
-[Unit]
-Description=Prometheus PVE Exporter
-Documentation=https://github.com/prometheus-pve/prometheus-pve-exporter
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ pve_exporter_user }}
-Group={{ pve_exporter_group }}
-ExecStart={{ pve_exporter_venv_path }}/bin/pve_exporter \
-  --config.file {{ pve_exporter_config_dir }}/pve.yml \
-  --web.listen-address 0.0.0.0:{{ pve_exporter_port }}
-Restart=on-failure
-RestartSec=10s
-
-# Security hardening
-NoNewPrivileges=yes
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/pve-exporter/templates/pve.yml.j2

@@ -1,8 +0,0 @@
-# PVE Exporter Configuration
-# Generated by Ansible
-
-default:
-  user: "{{ pve_exporter_pve_user }}"
-  token_name: "{{ pve_exporter_pve_token_name }}"
-  token_value: "{{ pve_exporter_pve_token_value }}"
-  verify_ssl: {{ pve_exporter_pve_verify_ssl | lower }}

ansible/roles/snmp-exporter/defaults/main.yml

@@ -1,33 +0,0 @@
----
-# SNMP Exporter defaults
-
-snmp_exporter_version: "0.30.1"
-snmp_exporter_listen_port: 9116
-snmp_exporter_user: "snmp_exporter"
-snmp_exporter_group: "snmp_exporter"
-
-snmp_exporter_install_dir: "/opt/snmp_exporter"
-snmp_exporter_config_dir: "/etc/snmp_exporter"
-
-# Synology NAS SNMP targets
-# Configure in host_vars or group_vars
-snmp_synology_targets: []
-# Example:
-# snmp_synology_targets:
-#   - name: nas-1
-#     target: 10.2.40.50
-#   - name: nas-2
-#     target: 10.2.40.51
-
-# SNMPv3 authentication (recommended for security)
-# These should be set in vault
-snmp_v3_username: "prometheus"
-snmp_v3_auth_protocol: "SHA"  # MD5 or SHA
-snmp_v3_auth_password: ""     # Set in vault
-snmp_v3_priv_protocol: "AES"  # DES or AES
-snmp_v3_priv_password: ""     # Set in vault
-snmp_v3_security_level: "authPriv"  # noAuthNoPriv, authNoPriv, authPriv
-
-# SNMPv2c fallback (less secure, but simpler)
-snmp_v2_community: "public"
-snmp_use_v3: true

ansible/roles/snmp-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart snmp_exporter
-  systemd:
-    name: snmp_exporter
-    state: restarted
-    daemon_reload: yes