fix(multus): drop runtime apk add from the CNI plugins installer

Owner directive: tools belong in the image, not installed at container start. This is the highest-blast-radius instance of the pattern left in the repo — a DaemonSet, so it runs on every node. No image was needed. `apk add --no-cache curl tar` turns out to have been unnecessary from the start: verified in alpine:3.21 that `tar` is already busybox (v1.37.0) and handles -xzf, and that busybox `wget` follows GitHub's release redirect to objects.githubusercontent.com over TLS using the base image's /etc/ssl/certs/ca-certificates.crt. Only `curl` is genuinely absent, and wget covers what it was used for. So this replaces the install with `wget -q -O` rather than introducing another layered image. Severity note for the record: the apk add sat inside the else branch, so steady-state nodes never reached it. It ran when plugins were absent — i.e. on a fresh node or after /opt/cni/bin was wiped, which is precisely the moment you least want a dependency on package-index availability. Verified by running the RENDERED init script in alpine:3.21 against the real release URL, both paths: empty dir -> Fetching ...; Installed: host-local, macvlan, static macvlan executable, /opt/cni/bin owner 0:0, exit 0 already there -> "CNI plugins already installed (CNI macvlan plugin v1.5.1)", exit 0, no fetch The second line is worth noting: that version string comes from executing the extracted binary, so the download and extraction are confirmed good, not merely present. The chown 0:0 also still applies, which the existing comment explains is required or Cilium's apply-sysctl-overwrites init container fails with "Permission denied". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 13:47:11 +02:00
198 changed files with 3074 additions and 5429 deletions
@@ -159,90 +159,3 @@ steps:
          --dockerfile Containerfile \
          --destination gitea.renner.uno/gitea_admin/alpine-jq:$TAG \
          --destination gitea.renner.uno/gitea_admin/alpine-jq:latest
-
-  # Runtime image, not CI tooling: upstream containernetworking/plugins binaries
-  # baked in, consumed by the multus cni-plugins-installer DaemonSet. Replaces a
-  # node-side fetch from GitHub that had NO checksum verification and sat on the
-  # critical path of every node rebuild. The download now happens here, once,
-  # where the sha256 is checked — see the Containerfile for the full rationale.
-  - name: build-cni-plugins
-    image: gcr.io/kaniko-project/executor:v1.23.2-debug
-    directory: ci/images/cni-plugins
-    depends_on: []
-    when:
-      - event: manual
-      - event: push
-        path: ["ci/images/cni-plugins/**", ".woodpecker/build-ci-images.yml"]
-    environment:
-      GITEA_USER: gitea_admin
-      GITEA_TOKEN:
-        from_secret: ci_images_token
-    commands:
-      - mkdir -p /kaniko/.docker
-      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
-      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
-      - TAG=$(date +%Y-%m-%d)
-      - |
-        /kaniko/executor \
-          --context . \
-          --dockerfile Containerfile \
-          --destination gitea.renner.uno/gitea_admin/cni-plugins:$TAG \
-          --destination gitea.renner.uno/gitea_admin/cni-plugins:latest
-
-  # Runtime image, not CI tooling: alpine + the openssl CLI, for shell scripts
-  # needing s_client/x509. Named for what it is; first consumer is the paperless
-  # fetch-bridge-cert init container. Replaces an `apk add openssl` that ran on
-  # every paperless pod start. The openssl CLI is genuinely absent from the alpine
-  # base, so this one needs an image rather than a busybox substitute.
-  - name: build-alpine-openssl
-    image: gcr.io/kaniko-project/executor:v1.23.2-debug
-    directory: ci/images/alpine-openssl
-    depends_on: []
-    when:
-      - event: manual
-      - event: push
-        path: ["ci/images/alpine-openssl/**", ".woodpecker/build-ci-images.yml"]
-    environment:
-      GITEA_USER: gitea_admin
-      GITEA_TOKEN:
-        from_secret: ci_images_token
-    commands:
-      - mkdir -p /kaniko/.docker
-      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
-      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
-      - TAG=$(date +%Y-%m-%d)
-      - |
-        /kaniko/executor \
-          --context . \
-          --dockerfile Containerfile \
-          --destination gitea.renner.uno/gitea_admin/alpine-openssl:$TAG \
-          --destination gitea.renner.uno/gitea_admin/alpine-openssl:latest
-
-  # Heating CAN readers: open3e (Vitocal/VX3) + E3onCAN (E380 meter), one image
-  # run as two services on edge-1. Replaces pip-into-venv on the host, where an
-  # unpinned VCS requirement re-cloned and restarted the heating bridge on every
-  # Ansible converge. Not a ci-* image -- it is a runtime artifact -- but it
-  # builds the same way and lands in the same registry so the audit sees it.
-  - name: build-heating-can
-    image: gcr.io/kaniko-project/executor:v1.23.2-debug
-    directory: ci/images/heating-can
-    depends_on: []
-    when:
-      - event: manual
-      - event: push
-        path: ["ci/images/heating-can/**", ".woodpecker/build-ci-images.yml"]
-    environment:
-      GITEA_USER: gitea_admin
-      GITEA_TOKEN:
-        from_secret: ci_images_token
-    commands:
-      - mkdir -p /kaniko/.docker
-      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
-      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
-      - TAG=$(date +%Y-%m-%d)
-      - |
-        /kaniko/executor \
-          --context . \
-          --dockerfile Containerfile \
-          --destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
-          --destination gitea.renner.uno/gitea_admin/heating-can:latest
@@ -1,254 +0,0 @@
-# Validate the GitOps tree and the pipeline definitions on PRs that touch them.
-#
-# Nothing covered kubernetes/** before this. deploy-infrastructure.yml only
-# gates opentofu/**, and ci-scripts-test.yml only runs scripts/ci/ unit tests —
-# so a broken manifest reached the cluster and was caught by Flux failing to
-# reconcile, or by a human noticing. Three separate defects on 2026-07-30 would
-# have been caught here instead:
-#
-#   * an unquoted YAML scalar containing ": " in .woodpecker/build-ci-images.yml,
-#     which made EVERY pipeline in the repo fail to compile (#443). `yq` parsed
-#     it happily — it is valid YAML, just a map instead of a string — so
-#     "it parses" was mistaken for "it is valid". Only woodpecker lint catches it.
-#   * a rotator pinned to an image digest that no longer existed (#458).
-#   * manifests whose kustomize wiring was only ever checked by hand.
-#
-# Deliberately CHEAP and offline: no cluster, no registry, no secrets. Everything
-# here is a static check that either passes or fails on the checkout alone, so it
-# can gate PRs without becoming a bottleneck or a flake source.
-when:
-  # Path-filtered on purpose, unlike ci-scripts-test.yml. That pipeline is one
-  # step on one already-cached image, so running it everywhere is free; this one
-  # pulls three images and builds every kustomization in the tree, which is real
-  # cost to put on a docs-only PR. The filter covers everything the steps actually read:
-  # kubernetes/** for the manifests and scripts, .woodpecker/** for the lint (and
-  # so an edit to this file re-runs its own checks).
-  - event: pull_request
-    path:
-      - "kubernetes/**"
-      - ".woodpecker/**"
-
-# ADDING A THIRD DISCOVERY CHECK: kustomize-build and shellcheck both hand-roll
-# the same shape — `find`, a floor guard against empty discovery, then a
-# file-fed `while read`. Two copies is deliberate: the two traps being commented
-# around (BusyBox find lacking `-printf`, and a piped loop running in a subshell
-# where `fail` cannot propagate) are different, and both are worth spelling out
-# where they bite. A third instance is the point to stop copying and factor it
-# into a shared script under `scripts/ci/` — which is already covered by
-# ci-scripts-test.yml, so it would gain unit tests in the process.
-#
-# Every step carries `depends_on: []` so all four run in PARALLEL and every
-# failure shows in one run. Woodpecker's default is sequential, which would stop
-# at the first failure and make a contributor fix-and-repush to discover the
-# next — and these four are independent, cheap, offline checks with no ordering
-# between them. Keep it on every step: `depends_on` is per-step, so dropping it
-# from one silently re-serialises that step behind the others.
-steps:
-  # Every kustomization must build. This is the check that catches a resource
-  # entry pointing at a file that does not exist, a bad patch target, or YAML
-  # that is fine in isolation but broken once overlaid — the class of error that
-  # otherwise surfaces as a Flux reconcile failure after merge.
-  #
-  # Verified green across every kustomization in the tree when this was added,
-  # so this gates new breakage rather than pre-existing debt.
-  #
-  # ci-k8s TAG SPLIT: this file pins a newer dated tag than e2e.yml, e2e-test.yml,
-  # alert-monitor.yml and issue-fix.yml, which still sit on an older one. That is
-  # staleness in those four, not a deliberate split here — everything should be on
-  # the newest dated tag. They are not bumped here because none of them are
-  # triggered by this PR, so it would ship untested changes under cover of a green
-  # run that never touched them. Tracked in #461.
-  - name: kustomize-build
-    depends_on: [] # see note above `steps:`
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30
-    commands:
-      - |
-        set -eu
-        # NOT `find -printf '%h\n'`: that is a GNU findutils extension and this
-        # image ships BusyBox find (v1.37.0), which rejects it outright with
-        # "find: unrecognized: -printf". Because the result feeds a command
-        # substitution used as a word list, the failure is SILENT — empty output,
-        # zero loop iterations, fail=0, step exits green having validated
-        # nothing. That is exactly what this pipeline shipped in its first
-        # revision, and CI reported it as passing.
-        dirs=$(find kubernetes -name kustomization.yaml | sed 's|/kustomization.yaml$||' | sort)
-
-        # Assert the discovery actually found something. Without this, any future
-        # breakage of the find/sed above degrades back into a green no-op rather
-        # than a failure — the specific trap that made the first revision of this
-        # step useless while looking healthy.
-        count=$(printf '%s\n' "$dirs" | grep -c . || true)
-        if [ "$count" -lt 50 ]; then
-          echo "FAIL: found only $count kustomizations; floor is 50."
-          echo "      If the tree legitimately shrank, lower this floor in"
-          echo "      .woodpecker/ci-manifests-test.yml — do not delete it."
-          echo "      Otherwise discovery is broken — refusing to report success."
-          exit 1
-        fi
-        echo "discovered $count kustomizations"
-
-        # `while read` fed from a FILE, not a pipe: a pipe would run the loop in
-        # a subshell where `fail` cannot propagate, so a failing build would exit
-        # 0 — the same silent-pass shape this step already shipped once. Reading
-        # a file keeps the loop in the current shell, and read -r handles paths
-        # containing spaces without IFS games.
-        printf '%s\n' "$dirs" > /tmp/dirs
-        fail=0
-        while IFS= read -r d; do
-          [ -n "$d" ] || continue
-          if kubectl kustomize "$d" >/dev/null 2>/tmp/err; then
-            echo "ok    $d"
-          else
-            echo "FAIL  $d"
-            sed 's/^/        /' /tmp/err
-            fail=1
-          fi
-        done < /tmp/dirs
-        exit $fail
-
-  # Lint the pipeline definitions themselves. This is the one that would have
-  # prevented the repo-wide CI outage: `woodpecker lint` exits 1 on a config that
-  # fails to compile, and exits 0 on mere warnings — so it fails on real breakage
-  # without blocking on pre-existing noise. Both behaviours were verified against
-  # this repo (rc=1 reproducing #443, rc=0 on the tree as it stands).
-  #
-  # An earlier revision of this comment also asserted that several deploy-*.yml
-  # files emit "Additional property timeout is not allowed" warnings. That was
-  # observed under the v3.9.0 CLI this step used to pin, and it is NOT known to
-  # still hold on v3.16.0 — review raised the likelihood that the warning was
-  # schema drift in a 7-minor-old CLI rather than real debt in those files, which
-  # is one of the reasons this pin moved to latest stable. It cannot be confirmed
-  # from a green run either, because the loop below sends lint output to /tmp/out
-  # and prints it only on failure, so warnings are invisible when the step passes.
-  # The claim is dropped rather than restated: what this step guarantees is the
-  # exit-code contract above, which is what the gate actually rests on.
-  # Latest stable, per the standing rule that any software uses the latest
-  # stable version. Matching the deployed woodpecker-server (v3.16.0) is a
-  # CONSEQUENCE of that, not a separate requirement — the server is bumped by
-  # image-automation within the 3.x range, so both land on the same release.
-  #
-  # It matters here because the CLI compiles the config against its own schema:
-  # one behind the server can pass a config the server then rejects, or reject
-  # one it accepts, and the gate stops predicting what the server will do —
-  # which is its only purpose.
-  #
-  # `-alpine` is required, not cosmetic: the plain vX.Y.Z CLI image ships no
-  # shell, and the loop below needs `for`, `sed` and `echo` around the binary.
-  # A step using the plain tag fails to start rather than failing a check.
-  #
-  # DRIFT RISK — accepted, and NOT fixable here. The server auto-bumps via
-  # image-automation; this pin cannot, so the two will drift apart.
-  # Image-automation can only patch YAML that is a Kubernetes manifest —
-  # fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the
-  # YAML manifest must have `apiVersion` and `kind` fields." A Woodpecker
-  # pipeline file has neither, so an `$imagepolicy` marker here is silently
-  # ignored. That is a property of the FILE, not of `update.path`: widening the
-  # automation's path to `./` would buy nothing, and would additionally drag the
-  # whole ansible/ and esphome/ trees into the walk. Do not try it — it looks
-  # like the fix and is not.
-  #
-  # So this bump is manual, and drift is caught by the update-audit agent
-  # (ci/agents/update-audit/prompt.md, item 5) which is told to flag this image
-  # whenever it trails the deployed server. If you bump the server, bump here.
-  - name: woodpecker-lint
-    depends_on: [] # see note above `steps:`
-    image: woodpeckerci/woodpecker-cli:v3.16.0-alpine
-    commands:
-      - |
-        set -eu
-        fail=0
-        for f in .woodpecker/*.yml; do
-          if /bin/woodpecker-cli lint "$f" >/tmp/out 2>&1; then
-            echo "ok    $f"
-          else
-            echo "FAIL  $f"
-            sed 's/^/        /' /tmp/out
-            fail=1
-          fi
-        done
-        exit $fail
-
-  # Standalone shell scripts under kubernetes/ (sidecar and bootstrap scripts
-  # mounted via configMapGenerator). Severity is pinned to `error` on purpose:
-  # style and info findings would fail the build on existing scripts for no
-  # safety gain. All scripts in the tree are clean at this level today.
-  #
-  # NOTE this does NOT cover shell embedded in manifests (init containers, probe
-  # exec blocks) — shellcheck cannot see inside YAML. That embedded shell is
-  # where several real bugs lived this year, so it remains a gap worth closing
-  # separately rather than one this step quietly implies is covered.
-  - name: shellcheck
-    depends_on: [] # see note above `steps:`
-    image: koalaman/shellcheck-alpine:v0.11.0
-    commands:
-      - |
-        set -eu
-        files=$(find kubernetes -name '*.sh' | sort)
-        # Same no-op guard as kustomize-build: if discovery ever returns nothing,
-        # fail loudly instead of passing having checked no files.
-        #
-        # This floor is 1 while kustomize-build's is 50, and the asymmetry is
-        # deliberate rather than an oversight. That floor is loose relative to the
-        # tree it guards, so it catches both total failure and gross partial loss
-        # while tolerating normal churn. The shell-script set is small enough that
-        # any floor tight enough to catch PARTIAL loss would sit inside normal
-        # churn — deleting one script would fail CI spuriously, and the fix a
-        # hurried person reaches for is lowering the floor, which is how a guard
-        # quietly becomes decoration. A floor that has to be edited whenever the
-        # tree changes is not a guard, it is a snapshot with an exit code.
-        #
-        # So this catches the failure that actually happened here (discovery
-        # silently returning nothing, step green having checked no files) and not
-        # partial loss. Worth revisiting only if kubernetes/ grows a lot more
-        # shell, at which point a loose proportionate floor becomes possible.
-        count=$(printf '%s\n' "$files" | grep -c . || true)
-        if [ "$count" -lt 1 ]; then
-          echo "FAIL: no shell scripts found under kubernetes/ — discovery is broken."
-          exit 1
-        fi
-        echo "discovered $count shell scripts"
-
-        # Same file-fed while-read reasoning as kustomize-build above.
-        printf '%s\n' "$files" > /tmp/files
-        fail=0
-        while IFS= read -r f; do
-          [ -n "$f" ] || continue
-          if shellcheck -S error "$f"; then
-            echo "ok    $f"
-          else
-            echo "FAIL  $f"
-            fail=1
-          fi
-        done < /tmp/files
-        exit $fail
-
-  # Guard the convention that cost a day to diagnose: registry-local images are
-  # pinned by DATED TAG, never by digest. Digests here are garbage-collected once
-  # :latest and the dated tags move past them — and that happens on edits to
-  # .woodpecker/build-ci-images.yml, a shared file, so an unrelated image addition
-  # silently orphans the manifest a rotator was pinned to. 11 rotators were dead
-  # this way and would have failed on their first scheduled run months later.
-  #
-  # Scoped to the rotation directory, which is where the convention is documented
-  # and where the failure actually bit.
-  #
-  # The `grep -v kustomization.yaml` is load-bearing, not tidying: that file
-  # documents this very rule, so its comment contains the literal string
-  # `image:.*@sha256` and therefore MATCHES the pattern. Anchoring on `image:`
-  # does not prevent that — verified by running the bare grep, which returns
-  # kustomization.yaml. Only the filename filter excludes it. Drop the filter and
-  # this step fails permanently on its own documentation.
-  - name: no-digest-pins
-    depends_on: [] # see note above `steps:`
-    image: gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30
-    commands:
-      - |
-        set -eu
-        hits=$(grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml 2>/dev/null \
-                 | grep -v kustomization.yaml || true)
-        if [ -n "$hits" ]; then
-          echo "FAIL: digest-pinned images found; use a dated tag (see rotation/kustomization.yaml):"
-          echo "$hits" | sed 's/^/        /'
-          exit 1
-        fi
-        echo "ok    no digest pins under openbao/rotation/"
@@ -1,6 +1,6 @@
 ---
 # Secret indirection — real values live in vault.yml (ansible-vault) next
-# to this file; see docs/edge-host.md for how to populate it from
+# to this file; see roles/edge-host/README.md for how to populate it from
 # the cluster secrets.
 edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
 edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
@@ -36,18 +36,3 @@ keepalived_ntp_sysctl:

 # Realtek NIC on the Wyse 3040 has no hardware timestamping.
 chrony_hwtimestamp: false
-
-# --- Local MQTT contract -------------------------------------------------
-# Shared between the broker and its clients, so it belongs to the inventory
-# rather than to either side's role. mqtt-bridge listens here; heating-can and
-# power-meter-sml publish here; and the bridge authenticates to the cluster
-# broker AS the open3e user, so it needs that identity too.
-#
-# Keeping these in one role's defaults only worked while a single play loaded
-# every role at once. Splitting the plays by capability exposed it: the bridge
-# could not render without a heating-can default, and the CAN services could
-# not render without an mqtt-bridge default.
-edge_mqtt_host: 127.0.0.1
-edge_mqtt_port: 1883
-edge_open3e_mqtt_user: open3e
-edge_open3e_mqtt_topic: open3e
@@ -1,18 +0,0 @@
---
-# Shared configuration for every NTP server (see plays/install-ntp-ha.yml).
-
-# IPv6 service address -- what ntp.srv.renner.uno resolves to, and the path
-# clients actually use. The IPv4 VIP is legacy.
-#
-# Defined at group level rather than in the play because install-ntp-ha.yml runs
-# two plays over this group -- withdraw the static assignments fleet-wide, then
-# hand the address to keepalived -- and both need the same value. Duplicating a
-# service address across plays is exactly the sort of thing that drifts.
-ntp_vip6: "fd40::123/128"
-
-# Link-local address carried as the FIRST virtual_ipaddress of the IPv6 VRRP
-# instance. RFC 5798 gives an IPv6 virtual router a link-local primary, and
-# keepalived warns "the first IPv6 VIP address should be link local" without
-# one. fe80::123 mirrors the service address and cannot collide with a real
-# NIC's EUI-64 link-local.
-ntp_vip6_linklocal: "fe80::123/64"
@@ -1,19 +1,7 @@
-# Reached by name, like every other host in this file. These three were the last
-# entries still addressed by IPv4 literal; authentik, netbird, proxy, nas-1,
-# grafana and edge-1 have always used their FQDN.
-#
-# An address in an inventory is allocator state standing in for identity, and it
-# pins access to the legacy family -- the names resolve IPv6-first, which is the
-# primary path on this network. DNS is the authoritative source for where a host
-# is; this file should say which host, not where it currently sits.
-#
-# The short inventory_hostname is kept deliberately: plays key on it
-# (ntp_servers[inventory_hostname] in install-ntp-ha.yml, node names in opentofu,
-# the prometheus targets), so only ansible_host changes.
 [pve_cluster]
-jupiter ansible_host=jupiter.renner.uno ansible_user=root
-neptune ansible_host=neptune.renner.uno ansible_user=root
-saturn ansible_host=saturn.renner.uno ansible_user=root
+jupiter ansible_host=10.2.40.201 ansible_user=root
+neptune ansible_host=10.2.40.202 ansible_user=root
+saturn ansible_host=10.2.40.203 ansible_user=root

 [authentik_db]
 authentik-db-1.srv.renner.uno ansible_user=root
@@ -41,36 +29,9 @@ nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
 [grafana]
 grafana.srv.renner.uno ansible_user=ladmin

-# Every bare-metal edge thin client. Gets the host-generic roles only:
-# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
-# sub-interface. Nothing here assumes any peripheral is attached.
 [edge_host]
 edge-1.iot.renner.uno ansible_user=root

-# Capability groups. Membership states which hardware a host physically has,
-# and each service playbook targets the group for its own concern. No flags and
-# no `when:` on the roles -- moving a stick to another box is a one-line change
-# here, which is what it should be.
-#
-# These are children of [edge_host], declared below: a peripheral is added to an
-# edge client, it does not replace the base. Making that a :children relation
-# rather than a comment means a host added only to a capability group still
-# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
-# cannot silently come up without edge_mqtt_host and friends.
-[zwave_gateway]
-edge-1.iot.renner.uno
-
-[heating_can]
-edge-1.iot.renner.uno
-
-[power_meter]
-edge-1.iot.renner.uno
-
-[edge_host:children]
-zwave_gateway
-heating_can
-power_meter
-
 # NTP servers. Includes edge-1 so the role survives the PVE hosts being
 # retired for bare-metal Talos, which cannot run chrony or keepalived itself.
 # See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly
@@ -0,0 +1,19 @@
+---
+# Deploy Blackbox Exporter to Prometheus server for endpoint probing
+# Usage: ansible-playbook -i inventory/ plays/install-blackbox-exporter.yml
+
+- name: Install Blackbox Exporter
+  hosts: prometheus.srv.renner.uno
+  become: yes
+  roles:
+    - blackbox-exporter
+
+  post_tasks:
+    - name: Verify blackbox_exporter is running
+      uri:
+        url: "http://127.0.0.1:9115/metrics"
+        status_code: 200
+      register: exporter_health
+      retries: 3
+      delay: 5
+      until: exporter_health.status == 200
@@ -1,41 +1,13 @@
 ---
-# Bare-metal Arch edge hosts (Dell Wyse 3040 and similar) — the home for
-# hardware-bound services, because Talos supports neither SocketCAN nor USB
-# passthrough.
-#
-# This play is the BASE only: what every edge thin client gets regardless of
-# what is plugged into it. The peripherals live in their own playbooks against
-# their own inventory groups:
-#
-#   plays/install-zwave-gateway.yml   [zwave_gateway]   ZWA-2 radio
-#   plays/install-heating-can.yml     [heating_can]     Innomaker USB2CAN
-#   plays/install-power-meter.yml     [power_meter]     FTDI meter head
-#
-# Splitting the roles without splitting the plays would have kept the coupling:
-# one play applying every role to one group is the monolith again, a layer up.
-# A host with no peripherals is simply absent from those groups -- no flags, no
-# `when:` on the roles.
-#
-# Self-contained: `certificates` and `common` are the parts of plays/common.yml
-# that apply to an edge host, carried here so bootstrapping one is a single
-# unrestricted run over the group. Not carried over: qemu-guest-agent (bare
-# metal, not VMs) and monitoring-agent (Debian-oriented; these hosts ship logs
-# via edge-log-shipper).
-#
-# Order is dependency order: edge-flash-wear before docker, so the journal is
-# volatile before containers start logging into it.
- name: Install Arch edge host (base)
+# Bare-metal Arch Linux edge host (Dell Wyse 3040, internal eMMC) — replaces
+# the k8s-edge Talos VM
+# (opentofu vm_153) as the home for hardware-bound services:
+#   - zwave-js-ui  (ZWA-2 Z-Wave USB radio, docker compose)
+#   - sml2mqtt     (FTDI power-meter reader, docker compose)
+#   - open3e       (Innomaker USB2CAN -> Vitocal/VX3, native venv + systemd)
+# Base install procedure and cutover checklist: ansible/roles/edge-host/README.md
+- name: Install Arch edge host
  hosts: edge_host
-  # Connection is already root (ansible_user=root), so this is a root-to-root
-  # no-op today. Kept explicit, matching plays/common.yml, so the roles below
-  # do not silently depend on the connection user staying privileged.
-  become: true
-
  roles:
-    - certificates
-    - common
-    - edge-flash-wear
    - docker-install
-    - edge-log-shipper
-    - mqtt-bridge
-    - edge-vlan
+    - edge-host
@@ -1,18 +0,0 @@
---
-# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
-# USB2CAN adapter attached.
-#
-# Targets [heating_can]: membership means the adapter is physically present.
-# Two roles because they are two concerns -- the interface is host network
-# state, the services consume it -- but one group, because both need the same
-# hardware.
-#
-# Requires the base play (plays/install-edge-host.yml) to have run: these
-# services publish to the local MQTT bridge it sets up.
- name: Install heating CAN services
-  hosts: heating_can
-  become: true
-
-  roles:
-    - socketcan
-    - heating-can
@@ -0,0 +1,86 @@
+---
+# Setup Log Aggregation Infrastructure
+# This playbook configures the complete logging stack:
+# 1. Loki server for log storage and querying
+# 2. Alloy agents on all VMs/CTs for log collection
+# 3. Syslog receiver for remote devices (Synology NAS, network devices)
+#
+# Usage: ansible-playbook -i inventory/ plays/setup-log-aggregation.yml
+#
+# To limit to specific hosts:
+#   --limit loki.srv.renner.uno          # Only configure Loki server
+#   --limit proxmox_all_running          # Only configure log agents
+
+- name: Setup Loki Log Aggregation Server
+  hosts: proxmox_logging
+  become: yes
+  gather_facts: yes
+
+  roles:
+    - common
+    - loki-server
+
+  post_tasks:
+    - name: Verify Loki is running
+      uri:
+        url: "http://127.0.0.1:3100/ready"
+        status_code: 200
+      register: loki_health
+      retries: 5
+      delay: 10
+      until: loki_health.status == 200
+
+    - name: Display Loki status
+      debug:
+        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
+
+- name: Deploy Monitoring Agent to Loki Server (with syslog receiver)
+  hosts: proxmox_logging
+  become: yes
+  gather_facts: yes
+
+  vars:
+    loki_endpoint: "http://loki.srv.renner.uno:3100"
+    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
+    # Syslog receiver settings are defined in host_vars/loki.srv.renner.uno/vars.yml:
+    # - alloy_syslog_receiver_enabled: true
+    # - alloy_syslog_receiver_port: 1514
+    # - alloy_syslog_receiver_protocol: "udp"
+    # - alloy_syslog_receiver_format: "raw"
+    # - alloy_stability_level: "experimental"
+
+  roles:
+    - monitoring-agent
+
+  post_tasks:
+    - name: Display syslog receiver info
+      debug:
+        msg: |
+          Syslog receiver is listening on UDP port {{ alloy_syslog_receiver_port | default(1514) }}
+          Configure remote devices to send syslog to: {{ inventory_hostname }}:{{ alloy_syslog_receiver_port | default(1514) }}
+      when: alloy_syslog_receiver_enabled | default(false)
+
+- name: Deploy Monitoring Agent to All Running VMs/CTs
+  hosts: proxmox_all_running:!proxmox_logging:!proxmox_noansible
+  become: yes
+  gather_facts: yes
+  serial: 10  # Deploy to 10 hosts at a time to avoid overwhelming the network
+
+  vars:
+    loki_endpoint: "http://loki.srv.renner.uno:3100"
+    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
+
+  roles:
+    - monitoring-agent
+
+  post_tasks:
+    - name: Verify Alloy is running
+      systemd:
+        name: alloy
+        state: started
+      register: alloy_service
+
+    - name: Display agent status
+      debug:
+        msg: "Monitoring agent deployed successfully on {{ inventory_hostname }}"
+      when: alloy_service is defined
@@ -0,0 +1,26 @@
+---
+# Install Loki Log Aggregation Server
+# Usage: ansible-playbook -i inventory/ plays/install-loki.yml
+
+- name: Install Loki Log Aggregation Server
+  hosts: loki.srv.renner.uno
+  become: yes
+  gather_facts: yes
+
+  roles:
+    - common
+    - loki-server
+
+  post_tasks:
+    - name: Verify Loki is running
+      uri:
+        url: "http://127.0.0.1:3100/ready"
+        status_code: 200
+      register: loki_health
+      retries: 5
+      delay: 10
+      until: loki_health.status == 200
+
+    - name: Display Loki status
+      debug:
+        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
@@ -22,54 +22,12 @@
 # so it stays BACKUP while the PVE nodes exist and wins the VIP by default once
 # they are gone -- no config change required at cutover.

-# Phase 1. Every static assignment of the IPv6 service address must be gone
-# fleet-wide before ANY host claims it via VRRP.
-#
-# DAD is enabled on the VRRP address (deliberately -- under single-master VRRP a
-# duplicate is a real fault worth detecting). So a static copy on one host and a
-# VRRP-managed copy on another cannot coexist: the kernel marks the newcomer
-# `dadfailed deprecated`, and its master then believes it is serving while
-# carrying an unusable address. That is not hypothetical -- it is what edge-1
-# did on 2026-08-07 when it took the address while three PVE nodes still held it
-# statically.
-#
-# Deliberately NOT serial: this play must complete everywhere before phase 2
-# starts anywhere. Doing the withdrawal per host inside phase 2 cannot avoid the
-# overlap, because with serial: 1 the first host claims the address while the
-# rest are still static.
-#
-# There is a gap between the phases where no host holds the address. It is
-# bounded by one play's run time and clients simply retry.
- name: Withdraw static assignments of the NTP service address
-  hosts: ntp_cluster
-  become: true
-
-  vars:
-    keepalived_ntp_vip6: "{{ ntp_vip6 }}"
-    keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
-
-  tasks:
-    - name: Withdraw any static assignment of the service address
-      ansible.builtin.include_role:
-        name: keepalived-ntp
-        tasks_from: withdraw-static.yml
-      when: ntp_keepalived_enabled | default(false)
-
-# Phase 2. Configure chrony and hand the address to keepalived.
 - name: Install NTP Server (HA)
  hosts: ntp_cluster
  become: true
-  # One node at a time. The keepalived role withdraws the statically assigned
-  # IPv6 service address and restarts keepalived so the address is re-taken
-  # under VRRP control; doing that across every NTP server at once would leave
-  # the whole fleet with nobody holding fd40::123 until the elections settle.
-  # Serialised, the remaining members keep answering throughout.
-  serial: 1

  vars:
    ntp_vip: "10.2.40.4"
-    # ntp_vip6 / ntp_vip6_linklocal come from group_vars/ntp_cluster.yml, so
-    # both plays in this file see one definition.
    ntp_servers:
      jupiter:
        priority: 100
@@ -91,8 +49,6 @@
      when: ntp_keepalived_enabled | default(false)
      vars:
        keepalived_ntp_vip: "{{ ntp_vip }}"
-        keepalived_ntp_vip6: "{{ ntp_vip6 }}"
-        keepalived_ntp_vip6_linklocal: "{{ ntp_vip6_linklocal }}"
        keepalived_ntp_state: "{{ ntp_servers[inventory_hostname].state }}"
        keepalived_ntp_priority: "{{ ntp_servers[inventory_hostname].priority }}"
        keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
@@ -134,49 +90,6 @@
          {% endif %}
      when: ntp_keepalived_enabled | default(false)

-    # The v6 address is the path clients actually use, so it gets the same
-    # check as the v4 VIP. Without this a broken v6 rollout -- the address
-    # never claimed, or -nodad not behaving as expected on a given kernel --
-    # would not surface here at all, only later by reading
-    # /var/log/keepalived-state.log by hand.
-    #
-    # `ip -6 addr show ... to <addr>` prints the address when present and
-    # nothing when absent, so emptiness is the test.
-    #
-    # Presence cannot itself distinguish a VRRP-held address from a static
-    # leftover -- they look identical. It is only meaningful because the
-    # keepalived-ntp role asserts, on both distros, that no static assignment
-    # survives: the play fails before reaching here if one does.
-    - name: Check if this server holds the IPv6 service address
-      ansible.builtin.command:
-        argv:
-          - ip
-          - -6
-          - addr
-          - show
-          - dev
-          - "{{ ntp_keepalived_interface }}"
-          - to
-          - "{{ ntp_vip6 }}"
-      register: vip6_check
-      failed_when: false
-      changed_when: false
-      when: ntp_keepalived_enabled | default(false)
-
-    - name: Display IPv6 service address status
-      ansible.builtin.debug:
-        msg: >
-          {% if vip6_check.stdout | default('') | length > 0 %}
-          This server ({{ inventory_hostname }}) holds the IPv6 service address
-          {{ ntp_vip6 }} -- it is the v6 MASTER and is serving ntp.srv.renner.uno
-          {% else %}
-          This server ({{ inventory_hostname }}) does not hold {{ ntp_vip6 }}.
-          Expected on a v6 BACKUP. If NO host holds it, check_ntp is failing
-          everywhere -- inspect `chronyc tracking` and
-          /var/log/keepalived-ntp-health.log
-          {% endif %}
-      when: ntp_keepalived_enabled | default(false)
-
    - name: Report hosts serving NTP without participating in the VIP
      ansible.builtin.debug:
        msg: >
@@ -1,14 +0,0 @@
---
-# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
-# attached.
-#
-# Targets [power_meter]: membership means the meter head is physically present.
-#
-# Requires the base play (plays/install-edge-host.yml) to have run: it needs
-# docker and publishes to the local MQTT bridge.
- name: Install power-meter reader
-  hosts: power_meter
-  become: true
-
-  roles:
-    - power-meter-sml
@@ -0,0 +1,43 @@
+---
+# Install Prometheus Metrics Server
+# Usage: ansible-playbook -i inventory/ plays/install-prometheus.yml
+
+- name: Install Prometheus Metrics Server
+  hosts: prometheus.srv.renner.uno
+  become: yes
+  gather_facts: yes
+
+  vars:
+    # Synology NAS SNMP targets
+    snmp_synology_targets:
+      - name: nas-1
+        target: "10.2.40.50"
+      - name: nas-2
+        target: "10.3.40.57"
+
+    # SNMPv3 credentials from vault
+    snmp_use_v3: true
+    snmp_v3_username: "{{ vault_snmp_v3_username }}"
+    snmp_v3_auth_password: "{{ vault_snmp_v3_auth_password }}"
+    snmp_v3_priv_password: "{{ vault_snmp_v3_priv_password }}"
+
+  roles:
+    - common
+    - prometheus-server
+    - snmp-exporter
+    - pve-exporter
+    - unifi-poller
+
+  post_tasks:
+    - name: Verify Prometheus is running
+      uri:
+        url: "http://127.0.0.1:9090/-/ready"
+        status_code: 200
+      register: prometheus_health
+      retries: 5
+      delay: 10
+      until: prometheus_health.status == 200
+
+    - name: Display Prometheus status
+      debug:
+        msg: "Prometheus is running and healthy at http://{{ inventory_hostname }}:9090"
@@ -1,15 +0,0 @@
---
-# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
-#
-# Targets [zwave_gateway]: membership means the radio is physically present.
-# Moving the stick to another edge host is a one-line inventory change.
-#
-# Requires the base play (plays/install-edge-host.yml) to have run: it needs
-# docker, and the flash-wear handling that the libeatmydata preload exists to
-# complement.
- name: Install Z-Wave gateway
-  hosts: zwave_gateway
-  become: true
-
-  roles:
-    - zwave-gateway
@@ -16,6 +16,7 @@
 - import_playbook: install-grist.yml
 - import_playbook: install-music-assistant.yml
 - import_playbook: update-docker.yml
+- import_playbook: install-log-aggregation.yml
 - import_playbook: update-pve-rolling.yml

 - name: Update Summary
@@ -0,0 +1,26 @@
+---
+blackbox_exporter_version: "0.25.0"
+blackbox_exporter_port: 9115
+blackbox_exporter_user: "blackbox_exporter"
+blackbox_exporter_group: "blackbox_exporter"
+
+# Endpoints to probe
+blackbox_http_targets:
+  - name: "gitea"
+    url: "https://gitea.renner.uno"
+  - name: "immich"
+    url: "https://immich.renner.uno"
+  - name: "authentik"
+    url: "https://auth.renner.uno"
+  - name: "home-assistant"
+    url: "https://ha.renner.uno"
+  - name: "grafana"
+    url: "https://grafana.renner.uno"
+  - name: "prometheus"
+    url: "https://prometheus.renner.uno"
+
+blackbox_dns_targets: []
+
+blackbox_tcp_targets:
+  - name: "postgres-ha"
+    target: "10.2.40.135:5432"
@@ -0,0 +1,6 @@
+---
+- name: Restart blackbox_exporter
+  systemd:
+    name: blackbox_exporter
+    state: restarted
+    daemon_reload: yes
@@ -0,0 +1,80 @@
+---
+- name: Create blackbox_exporter group
+  group:
+    name: "{{ blackbox_exporter_group }}"
+    system: yes
+    state: present
+
+- name: Create blackbox_exporter user
+  user:
+    name: "{{ blackbox_exporter_user }}"
+    group: "{{ blackbox_exporter_group }}"
+    system: yes
+    shell: /usr/sbin/nologin
+    createhome: no
+    state: present
+
+- name: Create blackbox_exporter directories
+  file:
+    path: "{{ item }}"
+    state: directory
+    owner: "{{ blackbox_exporter_user }}"
+    group: "{{ blackbox_exporter_group }}"
+    mode: '0755'
+  loop:
+    - /etc/blackbox_exporter
+    - /var/lib/blackbox_exporter
+
+- name: Download blackbox_exporter
+  get_url:
+    url: "https://github.com/prometheus/blackbox_exporter/releases/download/v{{ blackbox_exporter_version }}/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64.tar.gz"
+    dest: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
+    mode: '0644'
+
+- name: Extract blackbox_exporter
+  unarchive:
+    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
+    dest: /tmp
+    remote_src: yes
+    creates: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
+
+- name: Install blackbox_exporter binary
+  copy:
+    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64/blackbox_exporter"
+    dest: /usr/local/bin/blackbox_exporter
+    owner: root
+    group: root
+    mode: '0755'
+    remote_src: yes
+  notify: Restart blackbox_exporter
+
+- name: Deploy blackbox_exporter configuration
+  template:
+    src: blackbox.yml.j2
+    dest: /etc/blackbox_exporter/blackbox.yml
+    owner: "{{ blackbox_exporter_user }}"
+    group: "{{ blackbox_exporter_group }}"
+    mode: '0644'
+  notify: Restart blackbox_exporter
+
+- name: Deploy blackbox_exporter systemd service
+  template:
+    src: blackbox_exporter.service.j2
+    dest: /etc/systemd/system/blackbox_exporter.service
+    mode: '0644'
+  notify: Restart blackbox_exporter
+
+- name: Enable and start blackbox_exporter
+  systemd:
+    name: blackbox_exporter
+    state: started
+    enabled: yes
+    daemon_reload: yes
+
+- name: Clean up downloaded files
+  file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
+    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
@@ -0,0 +1,60 @@
+modules:
+  http_2xx:
+    prober: http
+    timeout: 5s
+    http:
+      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
+      valid_status_codes: []  # Defaults to 2xx
+      method: GET
+      follow_redirects: true
+      fail_if_ssl: false
+      fail_if_not_ssl: false
+      tls_config:
+        insecure_skip_verify: false
+
+  http_post_2xx:
+    prober: http
+    timeout: 5s
+    http:
+      method: POST
+      valid_status_codes: []
+
+  tcp_connect:
+    prober: tcp
+    timeout: 5s
+
+  # DNS module for external domain resolution (tests upstream/cache)
+  dns_external:
+    prober: dns
+    timeout: 5s
+    dns:
+      query_name: "google.com"
+      query_type: "A"
+      valid_rcodes:
+        - NOERROR
+
+  # DNS module for internal domain resolution (tests local resolution)
+  dns_internal:
+    prober: dns
+    timeout: 5s
+    dns:
+      query_name: "k8s-1.srv.renner.uno"
+      query_type: "A"
+      valid_rcodes:
+        - NOERROR
+
+  # DNS module for common cached domain
+  dns_cached:
+    prober: dns
+    timeout: 5s
+    dns:
+      query_name: "cloudflare.com"
+      query_type: "A"
+      valid_rcodes:
+        - NOERROR
+
+  icmp:
+    prober: icmp
+    timeout: 5s
+    icmp:
+      preferred_ip_protocol: "ip4"
@@ -0,0 +1,22 @@
+[Unit]
+Description=Prometheus Blackbox Exporter
+Documentation=https://github.com/prometheus/blackbox_exporter
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+Type=simple
+User={{ blackbox_exporter_user }}
+Group={{ blackbox_exporter_group }}
+ExecStart=/usr/local/bin/blackbox_exporter \
+    --config.file=/etc/blackbox_exporter/blackbox.yml \
+    --web.listen-address=:{{ blackbox_exporter_port }}
+Restart=on-failure
+RestartSec=5s
+
+# Allow ICMP probing (requires CAP_NET_RAW)
+AmbientCapabilities=CAP_NET_RAW
+CapabilityBoundingSet=CAP_NET_RAW
+
+[Install]
+WantedBy=multi-user.target
@@ -1,25 +1,9 @@
 ---
 # Chrony NTP Server Role Defaults

-# Networks allowed to use this server as NTP source.
-#
-# BOTH families are required. IPv6 is the primary address family here -- the
-# service address ntp.srv.renner.uno resolves to fd40::123 -- so an IPv4-only
-# allow list refuses every client that uses the documented name.
-#
-# The hosts carried `allow fd00::/8` for a long time without it ever being in
-# this file. Applying the role on 2026-08-07 rendered the template over it and
-# dropped the line on all four NTP servers at once, which took internal NTP
-# down over IPv6 until it was added here. Same class of untracked host state as
-# the anycast service address, and the same failure: config that exists only on
-# the box survives right up until something re-renders it.
+# Networks allowed to use this server as NTP source
 chrony_allowed_networks:
  - "10.2.0.0/16"
-  - "fd00::/8"
-  # Link-local. A client on the same segment may source its query from fe80::,
-  # and without this chrony refuses it -- the request arrives and is dropped,
-  # which looks like the server being unreachable rather than a policy refusal.
-  - "fe80::/10"

 # Primary NTP servers (PTB - Physikalisch-Technische Bundesanstalt, Stratum 1)
 chrony_upstream_servers:
@@ -34,26 +18,6 @@ chrony_upstream_pools:
 # Local stratum if disconnected from upstream
 chrony_local_stratum: 10

-# Step (rather than slew) the clock when the offset exceeds the threshold, but
-# only for the first N updates after chronyd starts. Without a makestep
-# directive chronyd only ever slews, so a large offset is never closed in
-# practice.
-#
-# edge-1, 2026-08-04: the CMOS battery is dead, so after a power loss the RTC
-# came up reading 2018-08-20. The kernel set the clock from it, systemd floored
-# that forward to its own built-in epoch (2026-06-26), and it ended up at
-# ~2026-07-19 -- about 16 days behind. chronyd then synced to ptbtime1, reported
-# "System time: 1376235.375 seconds slow of NTP time", and corrected none of it.
-# With host time before the cluster certificate's NotBefore, every outbound TLS
-# handshake failed x509 validation, taking out the MQTT bridge (heating
-# telemetry) and the Loki log shipper at once. The 16 days is incidental -- it
-# is just where the software flooring landed; the fault is that nothing steps.
-#
-# Limiting to the first few updates keeps steps confined to boot, so a
-# long-running server is never jumped out from under its workloads.
-chrony_makestep_threshold: 1.0
-chrony_makestep_limit: 3
-
 # Hardware timestamping. Fine on the PVE nodes' Intel NICs; the Wyse 3040
 # (edge-1) has a Realtek NIC that does not support it, and chronyd logs an
 # error per interface when asked for it. Overridden to false in
@@ -14,11 +14,6 @@ pool {{ pool }}
 # Record the rate at which the system clock gains/losses time
 driftfile /var/lib/chrony/chrony.drift

-# Step the clock instead of slewing it when the offset is large, limited to the
-# first few updates after start so only boot-time corrections jump. Rationale
-# and the incident behind it: chrony_makestep_* in the role's defaults/main.yml.
-makestep {{ chrony_makestep_threshold }} {{ chrony_makestep_limit }}
-
 # Allow NTP client access from local networks
 {% for network in chrony_allowed_networks %}
 allow {{ network }}
@@ -118,11 +118,8 @@ After the `kubernetes/apps/claude-workstation/` PR merges and Flux reconciles:
 NS=apps
 TOKEN=$(kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.token}' | base64 -d)
 kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/ca.crt
-# server = the cluster's stable API endpoint (kubernetes/talos/patches/cluster.yaml).
-# Use the name, not an address: it is what the API server certificate carries as a
-# SAN, so an IP here fails TLS verification even when it is reachable. It must
-# resolve from the workstation over the site-to-site route.
-kubectl config set-cluster homelab --server=https://k8s.renner.uno:6443 \
+# server = the API endpoint reachable over the site-to-site route (a CP node or VIP):
+kubectl config set-cluster homelab --server=https://10.2.40.201:6443 \
  --certificate-authority=/tmp/ca.crt --embed-certs=true --kubeconfig=/tmp/claude-ws.kubeconfig
 kubectl config set-credentials claude-workstation --token="$TOKEN" --kubeconfig=/tmp/claude-ws.kubeconfig
 kubectl config set-context homelab --cluster=homelab --user=claude-workstation --kubeconfig=/tmp/claude-ws.kubeconfig
@@ -1,15 +0,0 @@
---
-# Flash-wear mitigation.
-
-# --- Logging & eMMC wear mitigation ------------------------------------
-# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
-# kept off local flash: containers + native services log to a RAM-only
-# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
-# wear check round out the flash-longevity measures. See README "Logging &
-# eMMC".
-edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
-
-# Container stdout -> journal, no json log files on the flash. The journal
-# itself is volatile (see the tasks above), so container logs land in RAM and
-# are shipped off the box by edge-log-shipper.
-docker_log_driver: journald
@@ -1,17 +0,0 @@
---
- name: Restart journald
-  ansible.builtin.service:
-    name: systemd-journald
-    state: restarted
-
-# Handlers run in definition order, so this fires after 'Restart journald':
-# once the journal is volatile, the on-disk store is dead weight on the flash.
- name: Purge persistent journal store
-  ansible.builtin.file:
-    path: /var/log/journal
-    state: absent
-
- name: Restart docker
-  ansible.builtin.service:
-    name: docker
-    state: restarted
@@ -1,100 +0,0 @@
---
-# Flash-wear mitigation for hosts on small eMMC (Dell Wyse 3040 and similar).
-#
-# journald -> volatile (RAM) and size-capped, root mounted noatime+lazytime,
-# periodic TRIM, and a weekly wear check. Shipping the volatile journal off the
-# box is a separate concern -- see the edge-log-shipper role.
- name: Install mmc-utils
-  ansible.builtin.package:
-    name: mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
-    state: present
-
- name: Ensure journald drop-in directory exists
-  ansible.builtin.file:
-    path: /etc/systemd/journald.conf.d
-    state: directory
-    mode: "0755"
-
- name: Switch the systemd journal to volatile (RAM) storage
-  ansible.builtin.template:
-    src: journald-edge.conf.j2
-    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
-    mode: "0644"
-  notify:
-    - Restart journald
-    - Purge persistent journal store
-
- name: Resolve root filesystem facts (for noatime)
-  ansible.builtin.set_fact:
-    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
-
- name: Refuse to touch fstab without a stable root UUID
-  ansible.builtin.assert:
-    that:
-      - edge_root_fs.uuid | default('') | length > 0
-    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
-
-# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
-# heartbeat that otherwise dirties the journal every couple seconds); noatime
-# drops access-time writes. Neither weakens data durability.
- name: Mount root with noatime + lazytime (persisted in fstab)
-  ansible.posix.mount:
-    path: /
-    src: "UUID={{ edge_root_fs.uuid }}"
-    fstype: "{{ edge_root_fs.fstype }}"
-    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
-    dump: "0"
-    passno: "1"   # root fs — keep boot-time fsck ordering
-    backup: true
-    state: mounted
-
- name: Enable periodic TRIM (helps eMMC wear-leveling)
-  ansible.builtin.systemd:
-    name: fstrim.timer
-    enabled: true
-    state: started
-
- name: Install eMMC health-check script
-  ansible.builtin.copy:
-    src: edge-emmc-health
-    dest: /usr/local/bin/edge-emmc-health
-    mode: "0755"
-
- name: Install eMMC health-check service + timer
-  ansible.builtin.copy:
-    src: "{{ item }}"
-    dest: "/etc/systemd/system/{{ item }}"
-    mode: "0644"
-  loop:
-    - edge-emmc-health.service
-    - edge-emmc-health.timer
-
- name: Enable eMMC health-check timer
-  ansible.builtin.systemd:
-    name: edge-emmc-health.timer
-    enabled: true
-    state: started
-    daemon_reload: true
-
-# Container stdout into the journal instead of json files on disk. This lives
-# here, with the rest of the flash-wear handling, rather than in docker-install:
-# that role is shared by ten other plays (authentik, n8n, owncloud, grist,
-# netbird, gatus, music-assistant, ical-proxy), none of which have a
-# daemon.json today. Adding it there would have created one on the next run of
-# any of those plays and restarted every container on those hosts.
-#
-# /etc/docker is created explicitly because this role runs before
-# docker-install on a first bootstrap, so the directory may not exist yet. The
-# handler runs at the end of the play, by which time docker is installed.
- name: Ensure the docker config directory exists
-  ansible.builtin.file:
-    path: /etc/docker
-    state: directory
-    mode: "0755"
-
- name: Route container logs to the journal (keeps json logs off the flash)
-  ansible.builtin.template:
-    src: docker-daemon.json.j2
-    dest: /etc/docker/daemon.json
-    mode: "0644"
-  notify: Restart docker
@@ -1,3 +0,0 @@
-{
-  "log-driver": "{{ docker_log_driver }}"
-}
@@ -33,43 +33,13 @@ Build and boot the dedicated installer image — see
   (partitions the internal disk — NVMe if present, else the eMMC; the
   installer autodetects — installs the ansible-ready base system with SSH
   keys, DHCP, docker — asks for confirmation before wiping).
-3. Confirm the box resolves and is reachable by name — `ssh root@<host>` must
-   work before anything else. The UDM registers DHCP leases in DNS, so the
-   record appears on its own; the host is identified by name everywhere
-   (inventory, plays, this file) and its address is never recorded. Do not
-   pin it to a reservation.
-4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
-   That group is the base: every edge thin client belongs to it, and it
-   assumes no peripherals.
-5. From the workstation:
+3. Add the DHCP reservation/DNS entry for `edge-1.srv.renner.uno` on the
+   UDM, confirm `ssh root@edge-1.srv.renner.uno` works.
+4. From the workstation:
   ```
+   ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host
   ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
   ```
-   One unrestricted run over the whole group. The play is self-contained --
-   it carries `certificates` and `common` itself, so there is no second
-   command and no `--limit`. Running it converges every edge host, which is
-   the point: a run that touches only the new box tells you nothing about
-   the others.
-
-## Peripherals
-
-Each attached device is its own inventory group and its own playbook. Group
-membership states which hardware a host physically has -- there are no flags
-and no `when:` on the roles, so moving a stick between boxes is a one-line
-inventory change.
-
-| device | group | playbook |
-|---|---|---|
-| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
-| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
-| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
-
-Add the host to the matching group, then run that playbook. They depend on
-the base play having run: all three need the local MQTT bridge, and two of
-them need docker.
-
-A host with nothing attached is simply absent from all three groups and needs
-none of these commands.

 ## Secrets (before first play run)

@@ -115,7 +85,7 @@ zwave-js (order matters — never two servers on one radio):
      `kubectl -n home-assistant exec deploy/zwave-js -- tar cf - -C /usr/src/app store | tar xf - -C /opt/edge-host/zwave-js/`
 - [ ] Scale down in-cluster: `kubectl -n home-assistant scale deploy/zwave-js --replicas 0`
 - [ ] Move the ZWA-2 stick to edge-1, `docker compose up -d zwave-js`
- [ ] Point HA `zwave_js` integration at `ws://edge-1.iot.renner.uno:3000`
+- [ ] Point HA `zwave_js` integration at `ws://edge-1.srv.renner.uno:3000`
 - [ ] Nodes alive in the UI on :8091 → remove the flux kustomization
      `home-assistant-zwavejs` + PVC in a follow-up PR

@@ -143,12 +113,8 @@ Afterwards:
  deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
  changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
  the preloaded `.so` (see "Logging & eMMC" → zwave-js).
- open3e / E3onCAN updates: both are pinned to commits in
-  `roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
-  up upstream changes -- that is deliberate, these drive the heat pump. The
-  weekly update-audit reports when either falls behind (see
-  `ci/agents/update-audit/prompt.md`); bump the pin and re-run.
-  Pinning to a tag is not an option for open3e: it publishes none.
+- open3e updates: re-run the play; the pip task tracks the git default
+  branch (pin `edge_open3e_pkg` to a tag if that ever bites).
 - Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
  restart afterwards.
 - can-utils (`candump`) is AUR-only on Arch — install manually via
@@ -0,0 +1,118 @@
+---
+edge_base_dir: /opt/edge-host
+edge_tz: Europe/Berlin
+
+# --- Logging & eMMC wear mitigation ------------------------------------
+# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
+# kept off local flash: containers + native services log to a RAM-only
+# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
+# wear check round out the flash-longevity measures. See README "Logging &
+# eMMC".
+edge_journald_runtime_max: "128M"     # cap the volatile (RAM) journal
+edge_docker_log_driver: journald      # container stdout -> journal, no json logs on eMMC
+
+# Grafana Alloy log shipper. The srv-network Loki (loki.srv.renner.uno:3100,
+# no auth) is firewalled off the IoT VLAN; the only endpoint edge-1 can reach
+# is the authenticated cluster ingress (Traefik basic-auth).
+edge_alloy_config_file: /etc/alloy/edge.alloy
+edge_alloy_http_port: 12345
+edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
+edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
+edge_loki_username: edge-1
+edge_alloy_labels:
+  environment: production
+  datacenter: homelab
+
+# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
+# mosquitto on loopback, which bridges to the central cluster broker over TLS
+# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
+# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
+# blips. The local broker writes nothing to the eMMC (persistence off, logs to
+# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
+edge_mqtt_host: 127.0.0.1
+edge_mqtt_port: 1883
+# Upstream cluster broker the local bridge connects to (TLS; the public
+# Let's Encrypt cert is validated against the system CA bundle).
+edge_mqtt_cluster_host: mqtt.renner.uno
+edge_mqtt_cluster_tls_port: 8883
+edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
+
+# --- USB devices (stable by-id symlinks). After physically moving the
+# sticks to edge-1, verify with `ls -l /dev/serial/by-id/` — the by-id
+# names travel with the device, not the host, so these should not change.
+edge_zwave_device: /dev/serial/by-id/usb-Nabu_Casa_ZWA-2_80B54EE15720-if00
+edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
+
+# --- Images. Versions mirror the k8s manifests at migration time.
+# sml2mqtt stays hand-pinned (critical meter bridge — was deliberately
+# excluded from image-automation in k8s, keep that property here).
+edge_zwavejs_image: ghcr.io/zwave-js/zwave-js-ui:11.21.1
+edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
+# Alpine version the zwave-js-ui image is built on — the source for the musl
+# libeatmydata.so preloaded into that container (see tasks "Extract musl
+# libeatmydata.so"). Keep in sync with the base if edge_zwavejs_image bumps
+# its Alpine major (check: docker exec zwave-js cat /etc/os-release).
+edge_zwave_alpine_base: alpine:3.22
+
+# --- CAN / open3e (Viessmann E3 external bus) ---
+edge_can_interface: can0
+edge_can_bitrate: 250000
+edge_open3e_pkg: git+https://github.com/open3e/open3e.git
+edge_open3e_mqtt_user: open3e
+edge_open3e_mqtt_topic: open3e
+edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
+edge_open3e_poll_seconds: 30
+# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
+# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
+# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
+# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
+# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
+# the internal PV-surplus EMS depends on). Decodes via the master O3EList
+# codec inherited through the vx3plus overlay.
+# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
+# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
+# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
+# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
+# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
+# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
+# entries when added), so the multi-frame read is cheap at a 30 s poll.
+# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
+# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
+edge_open3e_datapoints: >-
+  vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
+# ECU addresses; -dev selects codec set, tx selects the UDS address.
+# vcmu (refrigerant-circuit controller) has no dedicated codec set —
+# omitting dpList makes open3e fall back to the generic datapoint list,
+# which decodes all polled VCMU DIDs.
+edge_open3e_devices:
+  vitocal: { tx: "0x680", dpList: "vcal" }
+  # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
+  # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
+  vx3: { tx: "0x6a1", dpList: "/opt/edge-host/open3e/vx3plus.py" }
+  vcmu: { tx: "0x68c" }
+
+# --- E3onCAN: passive E380 energy-meter listener (no bus writes, so it
+# coexists with the single open3e instance). The meter broadcasts at
+# ~5 Hz; the gap throttles MQTT to one update per DID per interval.
+edge_e3oncan_repo: https://github.com/MyHomeMyData/E3onCAN.git
+edge_e3oncan_version: 461dcbdf97e8d72d2edb9929c63b39382af6c3a1  # V0.5.2
+edge_e3oncan_gap_seconds: 10
+edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
+
+# --- Secrets: define these in inventory/group_vars/edge_host/vault.yml
+# (ansible-vault). See README.md for the kubectl one-liners that read the
+# current values out of the cluster during migration.
+# edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
+# edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
+# edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"
+
+# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
+# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
+# VRRP group, which needs layer-2 adjacency with the other members on the
+# service VLAN. The untagged VLAN 30 lease stays this host's only default
+# route -- see templates/10-vlan40.network.j2.
+edge_vlan40_enabled: false
+edge_vlan40_parent: enp1s0
+edge_vlan40_id: 40
+edge_vlan40_name: enp1s0.40
+edge_vlan40_parent_network: 20-wired.network
@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-# edge-1 eMMC wear/health check. Managed by roles/edge-flash-wear.
+# edge-1 eMMC wear/health check. Managed by roles/edge-host.
 #
 # Reads the JEDEC eMMC 5.x life-time and pre-EOL estimators exposed via sysfs
 # and emits a warning-priority journal message when the flash leaves the
@@ -1,4 +1,4 @@
-# Managed by ansible (roles/heating-can). Datapoint overlay for the VX3:
+# Managed by ansible (roles/edge-host). Datapoint overlay for the VX3:
 # the stock vx3 list acts as a FILTER (DIDs missing from it are popped
 # from the master table, Open3Eclass.py), so the per-battery-module DIDs
 # were never pollable with dpList "vx3". This overlay re-adds them with
@@ -0,0 +1,66 @@
+---
+# reload alone only re-reads config files; existing links keep their old
+# state until reconfigured.
+- name: Reload networkd
+  ansible.builtin.shell:
+    cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
+  changed_when: true
+
+# Reconfiguring the parent is what actually materialises the VLAN child -- a
+# bare reload leaves the existing link as-is (see above). This re-runs DHCP on
+# enp1s0, which is this headless host's only path to the network, so it is the
+# one genuinely risky step in the VLAN change.
+- name: Reload networkd for VLAN
+  ansible.builtin.shell:
+    cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
+  changed_when: true
+
+# Ordered before 'Restart edge compose services': the daemon must pick up the
+# new default log driver before the containers are recreated onto it.
+- name: Restart docker
+  ansible.builtin.service:
+    name: docker
+    state: restarted
+
+- name: Restart journald
+  ansible.builtin.service:
+    name: systemd-journald
+    state: restarted
+
+# Handlers run in definition order, so this fires after 'Restart journald':
+# once the journal is volatile, the on-disk store is dead weight on the eMMC.
+- name: Purge persistent journal store
+  ansible.builtin.file:
+    path: /var/log/journal
+    state: absent
+
+- name: Restart alloy
+  ansible.builtin.systemd:
+    name: edge-alloy
+    state: restarted
+    daemon_reload: true
+
+# Ordered before the MQTT clients (compose/open3e/e3oncan) so they reconnect
+# to a freshly-configured local broker.
+- name: Restart mosquitto
+  ansible.builtin.service:
+    name: mosquitto
+    state: restarted
+
+- name: Restart edge compose services
+  ansible.builtin.command:
+    cmd: docker compose up -d --force-recreate
+    chdir: "{{ edge_base_dir }}"
+  changed_when: true
+
+- name: Restart open3e
+  ansible.builtin.systemd:
+    name: open3e
+    state: restarted
+    daemon_reload: true
+
+- name: Restart e3oncan
+  ansible.builtin.systemd:
+    name: e3oncan
+    state: restarted
+    daemon_reload: true
@@ -0,0 +1,440 @@
+---
+# Arch edge host: zwave-js-ui + sml2mqtt as docker compose services,
+# open3e natively (venv + systemd) because it needs a SocketCAN interface,
+# which is host network state, not something to hide in a container.
+# Docker itself comes from the docker-install role (Arch-aware).
+
+- name: Install base packages
+  ansible.builtin.package:
+    name:
+      - python
+      - python-packaging  # ansible's pip module imports it on the target
+      - git
+      - mmc-utils      # eMMC lifetime/EOL introspection (mmc extcsd read)
+      - grafana-alloy  # log shipper -> Loki (see Alloy section below)
+      - mosquitto      # local MQTT bridge broker -> cluster over TLS
+    state: present
+
+# --- Local MQTT bridge broker. Edge clients publish to this over loopback;
+# it bridges to the cluster broker over TLS (open3e/e3oncan can't do MQTTS).
+# Configured with persistence off + stdout logging so it writes nothing to the
+# eMMC. Deployed before the clients so they have somewhere to connect.
+# See edge-mqtt-bridge.conf.j2 / README "MQTT bridge".
+- name: Deploy local mosquitto bridge config
+  ansible.builtin.template:
+    src: edge-mqtt-bridge.conf.j2
+    dest: /etc/mosquitto/mosquitto.conf
+    owner: root
+    group: mosquitto
+    mode: "0640"   # carries the cluster bridge credentials
+  notify: Restart mosquitto
+
+- name: Enable local mosquitto bridge
+  ansible.builtin.systemd:
+    name: mosquitto
+    enabled: true
+    state: started
+
+# ======================================================================
+# eMMC wear mitigation + log shipping. edge-1 (Dell Wyse 3040) runs on a
+# small internal eMMC, so logs must not churn local flash. Strategy:
+#   - journald -> volatile (RAM), size-capped        (journald-edge.conf)
+#   - docker   -> journald log driver, no json files (docker-daemon.json)
+#   - Grafana Alloy tails the RAM journal -> Loki, WAL on tmpfs
+# plus TRIM, noatime and a weekly wear check for flash longevity.
+# See roles/edge-host/README.md "Logging & eMMC".
+# ======================================================================
+
+- name: Ensure journald drop-in directory exists
+  ansible.builtin.file:
+    path: /etc/systemd/journald.conf.d
+    state: directory
+    mode: "0755"
+
+- name: Switch the systemd journal to volatile (RAM) storage
+  ansible.builtin.template:
+    src: journald-edge.conf.j2
+    dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
+    mode: "0644"
+  notify:
+    - Restart journald
+    - Purge persistent journal store
+
+- name: Resolve root filesystem facts (for noatime)
+  ansible.builtin.set_fact:
+    edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
+
+- name: Refuse to touch fstab without a stable root UUID
+  ansible.builtin.assert:
+    that:
+      - edge_root_fs.uuid | default('') | length > 0
+    fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
+
+# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
+# heartbeat that otherwise dirties the journal every couple seconds); noatime
+# drops access-time writes. Neither weakens data durability.
+- name: Mount root with noatime + lazytime (persisted in fstab)
+  ansible.posix.mount:
+    path: /
+    src: "UUID={{ edge_root_fs.uuid }}"
+    fstype: "{{ edge_root_fs.fstype }}"
+    opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
+    dump: "0"
+    passno: "1"   # root fs — keep boot-time fsck ordering
+    backup: true
+    state: mounted
+
+- name: Enable periodic TRIM (helps eMMC wear-leveling)
+  ansible.builtin.systemd:
+    name: fstrim.timer
+    enabled: true
+    state: started
+
+- name: Install eMMC health-check script
+  ansible.builtin.copy:
+    src: edge-emmc-health
+    dest: /usr/local/bin/edge-emmc-health
+    mode: "0755"
+
+- name: Install eMMC health-check service + timer
+  ansible.builtin.copy:
+    src: "{{ item }}"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - edge-emmc-health.service
+    - edge-emmc-health.timer
+
+- name: Enable eMMC health-check timer
+  ansible.builtin.systemd:
+    name: edge-emmc-health.timer
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# --- Grafana Alloy: ship the volatile journal to Loki. Container logs reach
+# the journal via docker's journald driver (configured below), native
+# services log to the journal directly, so a single journal source covers
+# everything. Storage/WAL sits on /run (tmpfs) — see edge-alloy.service.
+- name: Create alloy system group
+  ansible.builtin.group:
+    name: alloy
+    system: true
+
+- name: Create alloy system user
+  ansible.builtin.user:
+    name: alloy
+    group: alloy
+    groups: systemd-journal
+    system: true
+    create_home: false
+    shell: /usr/bin/nologin
+
+- name: Create alloy config directory
+  ansible.builtin.file:
+    path: /etc/alloy
+    state: directory
+    owner: alloy
+    group: alloy
+    mode: "0750"
+
+- name: Deploy alloy config (journal -> Loki)
+  ansible.builtin.template:
+    src: edge-alloy.alloy.j2
+    dest: "{{ edge_alloy_config_file }}"
+    owner: alloy
+    group: alloy
+    mode: "0600"   # carries the Loki basic-auth password
+  notify: Restart alloy
+
+- name: Deploy alloy systemd unit
+  ansible.builtin.template:
+    src: edge-alloy.service.j2
+    dest: /etc/systemd/system/edge-alloy.service
+    mode: "0644"
+  notify: Restart alloy
+
+- name: Enable edge-alloy
+  ansible.builtin.systemd:
+    name: edge-alloy
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# --- SocketCAN: systemd-networkd owns can0. gs_usb is an in-tree module
+# and autoloads when the Innomaker adapter enumerates; networkd then
+# matches the interface and applies bitrate + up. RestartSec recovers the
+# link after bus-off without manual intervention.
+- name: Configure can0 via systemd-networkd
+  ansible.builtin.template:
+    src: 80-can0.network.j2
+    dest: /etc/systemd/network/80-can0.network
+    mode: "0644"
+  notify: Reload networkd
+
+# --- Tagged VLAN 40. Additive by construction: the parent link keeps its own
+# .network file untouched and gains the VLAN via a drop-in, so reverting is a
+# matter of deleting these three paths. This host is headless and enp1s0 is its
+# only route to the network, hence the caution.
+- name: Create VLAN sub-interface netdev
+  ansible.builtin.template:
+    src: 10-vlan40.netdev.j2
+    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
+    mode: "0644"
+  when: edge_vlan40_enabled
+  notify: Reload networkd for VLAN
+  tags: [vlan40]
+
+- name: Configure VLAN sub-interface addressing
+  ansible.builtin.template:
+    src: 10-vlan40.network.j2
+    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
+    mode: "0644"
+  when: edge_vlan40_enabled
+  notify: Reload networkd for VLAN
+  tags: [vlan40]
+
+- name: Ensure parent network drop-in directory exists
+  ansible.builtin.file:
+    path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
+    state: directory
+    mode: "0755"
+  when: edge_vlan40_enabled
+  tags: [vlan40]
+
+- name: Attach VLAN to parent link via drop-in
+  ansible.builtin.template:
+    src: vlan40-parent-dropin.conf.j2
+    dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
+    mode: "0644"
+  when: edge_vlan40_enabled
+  notify: Reload networkd for VLAN
+  tags: [vlan40]
+
+- name: Enable systemd-networkd
+  ansible.builtin.service:
+    name: systemd-networkd
+    state: started
+    enabled: true
+
+# Docker networking needs no daemon.json (both services use network_mode:
+# host — real addresses, v4+v6, no bridge/NAT). The daemon.json below exists
+# only to route container stdout into the journald log driver so json logs
+# never touch the eMMC; the journal itself is volatile (RAM) and shipped to
+# Loki by edge-alloy. Both compose services inherit this default driver.
+- name: Configure docker daemon (journald log driver, eMMC-friendly)
+  ansible.builtin.template:
+    src: docker-daemon.json.j2
+    dest: /etc/docker/daemon.json
+    mode: "0644"
+  notify:
+    - Restart docker
+    - Restart edge compose services
+
+# --- Directory layout ---
+- name: Create service directories
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: directory
+    mode: "0755"
+  loop:
+    - "{{ edge_base_dir }}"
+    - "{{ edge_base_dir }}/zwave-js/store"
+    - "{{ edge_base_dir }}/sml2mqtt"
+
+# --- zwave-js eMMC write reduction: batch fsync via libeatmydata ---------
+# zwave-js persists its value DB with an fsync on *every* value change
+# (jsonl-db, throttleFS unset by zwave-js-ui). On a chatty network that
+# fsync storm churns the eMMC (~2.6 GiB/day for ~0.1 KiB/s of real data).
+# Preloading libeatmydata makes fsync a no-op so the kernel batches the
+# writes via normal writeback — full HA data resolution and cache
+# persistence are untouched (see docker-compose.yml.j2 for the mount + env).
+# The .so must match the zwave-js-ui image's musl/Alpine base, so extract it
+# from that exact Alpine version (bridge networking is broken on this host,
+# hence --network=host). Refresh by deleting the .so if the base ever bumps
+# its Alpine major version.
+- name: Extract musl libeatmydata.so (matches the zwave-js-ui Alpine base)
+  ansible.builtin.shell:
+    cmd: |
+      set -euo pipefail
+      tmp="$(mktemp)"
+      trap 'rm -f "$tmp"' EXIT
+      docker run --rm --network=host {{ edge_zwave_alpine_base }} \
+        sh -c 'apk add --no-cache libeatmydata >/dev/null 2>&1 && cat /usr/lib/libeatmydata.so' > "$tmp"
+      test "$(stat -c%s "$tmp")" -ge 8000   # sanity: real .so is ~14 KB, guards against a failed/empty pull
+      install -m 0644 "$tmp" {{ edge_base_dir }}/zwave-js/libeatmydata.so
+    creates: "{{ edge_base_dir }}/zwave-js/libeatmydata.so"
+  notify: Restart edge compose services
+
+# Because fsync is neutralized, a hard power-loss during the (rare) settings
+# write could truncate settings.json — which holds the Z-Wave network keys.
+# Keep a validated rolling backup on the eMMC so the keys stay recoverable.
+- name: Install zwave-js settings.json backup script
+  ansible.builtin.template:
+    src: edge-zwave-settings-backup.j2
+    dest: /usr/local/bin/edge-zwave-settings-backup
+    mode: "0755"
+
+- name: Install zwave-js settings backup service + timer
+  ansible.builtin.copy:
+    src: "{{ item }}"
+    dest: "/etc/systemd/system/{{ item }}"
+    mode: "0644"
+  loop:
+    - edge-zwave-settings-backup.service
+    - edge-zwave-settings-backup.timer
+
+- name: Enable zwave-js settings backup timer
+  ansible.builtin.systemd:
+    name: edge-zwave-settings-backup.timer
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# --- Compose services: zwave-js-ui + sml2mqtt ---
+- name: Deploy docker-compose.yml
+  ansible.builtin.template:
+    src: docker-compose.yml.j2
+    dest: "{{ edge_base_dir }}/docker-compose.yml"
+    mode: "0644"
+  notify: Restart edge compose services
+
+- name: Deploy zwave-js env (session secret)
+  ansible.builtin.template:
+    src: zwavejs.env.j2
+    dest: "{{ edge_base_dir }}/zwavejs.env"
+    mode: "0600"
+  notify: Restart edge compose services
+
+# sml2mqtt reads only a config file and has no env override for the
+# password, so the rendered config carries it — 0600, root-owned, same
+# trust boundary as the k8s init-container approach it replaces.
+- name: Deploy sml2mqtt config
+  ansible.builtin.template:
+    src: sml2mqtt-config.yml.j2
+    dest: "{{ edge_base_dir }}/sml2mqtt/config.yml"
+    mode: "0600"
+  notify: Restart edge compose services
+
+- name: Pull compose images
+  ansible.builtin.command:
+    cmd: docker compose pull
+    chdir: "{{ edge_base_dir }}"
+  changed_when: false
+
+# Tolerates missing USB devices: before the physical cutover the ZWA-2 /
+# FTDI sticks are still on the old k8s-edge VM, and a container whose
+# device is absent must not block the rest of the role (open3e/CAN moves
+# first). Re-run the play — or `docker compose up -d` — after each stick
+# arrives.
+- name: Start compose services
+  ansible.builtin.command:
+    cmd: docker compose up -d
+    chdir: "{{ edge_base_dir }}"
+  register: compose_up
+  failed_when: >-
+    compose_up.rc != 0
+    and 'error gathering device information' not in compose_up.stderr
+  changed_when: >-
+    'Started' in compose_up.stderr or 'Created' in compose_up.stderr
+    or 'Started' in compose_up.stdout or 'Creating' in compose_up.stdout
+
+- name: Warn about containers waiting for their USB device
+  ansible.builtin.debug:
+    msg: >-
+      docker compose could not start every service (USB device not present
+      yet — expected before cutover): {{ compose_up.stderr_lines | select('search', 'device') | list }}
+  when: compose_up.rc != 0
+
+# --- open3e: native install. CAN_RAW sockets need no capabilities, so
+# the service runs as an unprivileged system user.
+- name: Create open3e system user
+  ansible.builtin.user:
+    name: open3e
+    system: true
+    create_home: false
+    shell: /usr/bin/nologin
+
+# Owned by the service user: udsoncan writes its log file into the
+# working directory.
+- name: Create open3e directory
+  ansible.builtin.file:
+    path: "{{ edge_base_dir }}/open3e"
+    state: directory
+    owner: open3e
+    group: open3e
+    mode: "0755"
+
+- name: Install open3e into a venv
+  ansible.builtin.pip:
+    name: "{{ edge_open3e_pkg }}"
+    virtualenv: "{{ edge_base_dir }}/open3e/venv"
+    virtualenv_command: python -m venv
+  notify: Restart open3e
+
+- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
+  ansible.builtin.copy:
+    src: open3e-vx3plus.py
+    dest: "{{ edge_base_dir }}/open3e/vx3plus.py"
+    mode: "0644"
+  notify: Restart open3e
+
+- name: Deploy open3e devices.json
+  ansible.builtin.copy:
+    content: "{{ edge_open3e_devices | to_nice_json }}\n"
+    dest: "{{ edge_base_dir }}/open3e/devices.json"
+    mode: "0644"
+  notify: Restart open3e
+
+- name: Deploy open3e environment (broker credential)
+  ansible.builtin.template:
+    src: open3e.env.j2
+    dest: "{{ edge_base_dir }}/open3e/open3e.env"
+    owner: open3e
+    mode: "0600"
+  notify: Restart open3e
+
+- name: Deploy open3e systemd unit
+  ansible.builtin.template:
+    src: open3e.service.j2
+    dest: /etc/systemd/system/open3e.service
+    mode: "0644"
+  notify: Restart open3e
+
+- name: Enable open3e
+  ansible.builtin.systemd:
+    name: open3e
+    enabled: true
+    state: started
+    daemon_reload: true
+
+# --- E3onCAN: direct E380 meter reads (passive; per-review decision not
+# to trust the VX3's relayed meter values). Same broker user as open3e;
+# topics land under the same open3e/ prefix as e380_<did>_<name>.
+- name: Check out E3onCAN (pinned)
+  ansible.builtin.git:
+    repo: "{{ edge_e3oncan_repo }}"
+    dest: "{{ edge_base_dir }}/e3oncan/src"
+    version: "{{ edge_e3oncan_version }}"
+  notify: Restart e3oncan
+
+- name: Create E3onCAN venv with requirements
+  ansible.builtin.pip:
+    requirements: "{{ edge_base_dir }}/e3oncan/src/requirements.txt"
+    virtualenv: "{{ edge_base_dir }}/e3oncan/venv"
+    virtualenv_command: python -m venv
+  notify: Restart e3oncan
+
+- name: Deploy e3oncan systemd unit
+  ansible.builtin.template:
+    src: e3oncan.service.j2
+    dest: /etc/systemd/system/e3oncan.service
+    mode: "0644"
+  notify: Restart e3oncan
+
+- name: Enable e3oncan
+  ansible.builtin.systemd:
+    name: e3oncan
+    enabled: true
+    state: started
+    daemon_reload: true
@@ -1,4 +1,4 @@
-# Managed by ansible (roles/socketcan).
+# Managed by ansible (roles/edge-host).
 # Viessmann E3 external CAN bus via Innomaker USB2CAN (gs_usb).
 # The adapter is a stub tap on an already-terminated bus (120R jumper OFF).
 [Match]
@@ -0,0 +1,56 @@
+# Managed by ansible (roles/edge-host). Mirrors the former k8s manifests:
+# kubernetes/apps/home-assistant-zwavejs/deployment.yaml and
+# kubernetes/apps/home-assistant/sml2mqtt-deployment.yaml.
+#
+# No `privileged` needed here: docker's --device handles the device-cgroup
+# grant that hostPath in k8s could not (that was the only reason the pods
+# ran privileged).
+# Both services use host networking: real host addresses (v4+v6), no
+# bridge, no NAT. The broker LB is IPv6-first and docker bridge NAT66 is
+# a tarpit (unmasqueraded SYNs die silently); the host stack just works.
+services:
+  zwave-js:
+    image: {{ edge_zwavejs_image }}
+    container_name: zwave-js
+    restart: unless-stopped
+    stop_signal: SIGINT
+    tty: true
+    environment:
+      - TZ={{ edge_tz }}
+      # Batch the value-DB fsync storm to spare the eMMC (see role tasks /
+      # README "Logging & eMMC"). Neutralizes fsync only for this container;
+      # the WS/MQTT value stream to HA and cache persistence are unchanged.
+      - LD_PRELOAD=/usr/lib/libeatmydata.so
+    env_file:
+      - zwavejs.env
+    devices:
+      - "{{ edge_zwave_device }}:/dev/zwave"
+    volumes:
+      # musl libeatmydata.so extracted from the image's Alpine base (ro)
+      - ./zwave-js/libeatmydata.so:/usr/lib/libeatmydata.so:ro
+      # Persistent store: settings.json, device DB, network keys. Seeded
+      # from the k8s PVC `zwavejs-store` at cutover (see README). Since
+      # this host is stateful there is no init-container merge step —
+      # settings.json is edited via the UI and persists here; zwave.port
+      # stays /dev/zwave.
+      - ./zwave-js/store:/usr/src/app/store
+    # Host networking: binds 3000 (Z-Wave JS websocket for HA) and 8091
+    # (web UI) directly on the host.
+    network_mode: host
+
+  sml2mqtt:
+    image: {{ edge_sml2mqtt_image }}
+    container_name: sml2mqtt
+    restart: unless-stopped
+    # Outbound-only (serial in, MQTT out) — host networking sidesteps the
+    # docker NAT66 gap: the broker LB is IPv6-first and bridge-network v6
+    # SYNs leave unmasqueraded and die. The host stack reaches fd99::81
+    # fine (open3e proves it continuously).
+    network_mode: host
+    # Serial exclusivity: this is the ONLY reader of the meter port —
+    # HA's edl21 integration stays disabled (see sml2mqtt-config comment
+    # history in the k8s repo).
+    devices:
+      - "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
+    volumes:
+      - ./sml2mqtt/config.yml:/sml2mqtt/config.yml:ro
@@ -0,0 +1,3 @@
+{
+  "log-driver": "{{ edge_docker_log_driver }}"
+}
@@ -0,0 +1,32 @@
+# Managed by ansible (roles/edge-host).
+# PASSIVE listener for the E380 energy meter's CANopen broadcasts — no
+# writes on the bus, so it may run alongside the (single) open3e
+# instance. Reads the meter directly instead of trusting the VX3's
+# relayed values.
+[Unit]
+Description=E3onCAN E380 energy-meter CAN listener (read-only)
+BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
+# mosquitto.service = the local MQTT bridge broker e3oncan publishes to (loopback).
+After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
+Wants=network-online.target mosquitto.service
+
+[Service]
+User=open3e
+EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
+WorkingDirectory={{ edge_base_dir }}/e3oncan
+# Journald-friendly (python block-buffers pipes; lost on restarts).
+Environment=PYTHONUNBUFFERED=1
+ExecStart={{ edge_base_dir }}/e3oncan/venv/bin/python {{ edge_base_dir }}/e3oncan/src/E3onCANcollect.py \
+  -c {{ edge_can_interface }} \
+  -dev e380 \
+  -g {{ edge_e3oncan_gap_seconds }} \
+  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
+  -muser {{ edge_open3e_mqtt_user }} \
+  -mpass ${OPEN3E_MQTT_PASSWORD} \
+  -mfstr "{{ edge_e3oncan_mqtt_format }}" \
+  -j
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -1,4 +1,4 @@
-// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-log-shipper.
+// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-host.
 //
 // edge-1 runs on internal eMMC, so logs are kept out of local flash: the
 // systemd journal is volatile (RAM, see journald-edge.conf) and container
@@ -1,4 +1,4 @@
-# Managed by roles/mqtt-bridge. Local mosquitto that the edge clients (open3e,
+# Managed by roles/edge-host. Local mosquitto that the edge clients (open3e,
 # e3oncan, sml2mqtt) publish to over loopback; it bridges to the central
 # cluster broker over TLS. open3e/e3oncan have no native MQTTS, so this is how
 # edge-1's MQTT gets off plaintext on the untrusted IoT VLAN, and it buffers
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 # Rolling backup of zwave-js settings.json (holds the Z-Wave network keys).
-# Managed by roles/zwave-gateway. edge-1 runs zwave-js under libeatmydata (fsync
+# Managed by roles/edge-host. edge-1 runs zwave-js under libeatmydata (fsync
 # neutralized to spare the eMMC), so a hard power-loss during the rare
 # settings write could truncate settings.json. Keep a validated backup on the
 # eMMC so the keys are always recoverable. Runs hourly via a systemd timer.
@@ -1,4 +1,4 @@
-# Managed by roles/edge-flash-wear. edge-1 boots off a small internal eMMC, so the
+# Managed by roles/edge-host. edge-1 boots off a small internal eMMC, so the
 # journal must not persist to flash: keep it in RAM (/run/log/journal) and cap
 # its size. Grafana Alloy (edge-alloy.service) ships this journal to Loki
 # before it rotates out of RAM — nothing durable lives here.
@@ -0,0 +1,7 @@
+# Managed by ansible (roles/edge-host). Read by open3e.service and
+# e3oncan.service.
+# Full user:password pair as one variable so the open3e unit can pass it
+# as a single -muser argument without exposing it in the unit file;
+# e3oncan takes user and password separately (-muser/-mpass).
+OPEN3E_MUSER={{ edge_open3e_mqtt_user }}:{{ edge_open3e_mqtt_password }}
+OPEN3E_MQTT_PASSWORD={{ edge_open3e_mqtt_password }}
@@ -0,0 +1,39 @@
+# Managed by ansible (roles/edge-host).
+# Read-mostly listener — no -w flag, no service 0x77. The ONE sanctioned
+# write path is the MQTT command topic (-l): HA publishes the DHW
+# one-time-charge command there (scope decision 2026-07-02, see vican
+# CLAUDE.md). Anything beyond that single write gets its own review.
+# -j is load-bearing: without it open3e flattens JSON payloads into
+# per-key subtopics and the HA package (homeassistant repo,
+# packages/open3e.yaml) receives nothing.
+# Exactly ONE open3e instance may be on the bus at a time (CAN collisions
+# stop open3e) — stop this service before running ad-hoc scans.
+[Unit]
+Description=open3e Viessmann E3 CAN-to-MQTT bridge (read-only)
+# Tie the service to the CAN device: stops on unplug, starts on plug via
+# the device unit rather than polling.
+BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
+# mosquitto.service = the local MQTT bridge broker open3e publishes to (loopback).
+After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
+Wants=network-online.target mosquitto.service
+
+[Service]
+User=open3e
+EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
+WorkingDirectory={{ edge_base_dir }}/open3e
+ExecStart={{ edge_base_dir }}/open3e/venv/bin/open3e \
+  -c {{ edge_can_interface }} \
+  -cnfg {{ edge_base_dir }}/open3e/devices.json \
+  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
+  -muser ${OPEN3E_MUSER} \
+  -mcid open3e-edge \
+  -mfstr "{{ edge_open3e_mqtt_format }}" \
+  -r {{ edge_open3e_datapoints }} \
+  -t {{ edge_open3e_poll_seconds }} \
+  -l {{ edge_open3e_mqtt_topic }}/cmnd \
+  -j
+Restart=on-failure
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
@@ -1,4 +1,4 @@
-# Managed by ansible (roles/power-meter-sml). Ported 1:1 from the k8s ConfigMap
+# Managed by ansible (roles/edge-host). Ported 1:1 from the k8s ConfigMap
 # sml2mqtt-config; only host and password delivery differ (broker is now
 # reached via its LoadBalancer hostname, password rendered by ansible
 # instead of an init container).
@@ -1,15 +0,0 @@
---
-# Journal -> Loki shipping.
-#
-# The srv-network Loki (loki.srv.renner.uno:3100, no auth) is firewalled off
-# the IoT VLAN; the only endpoint these hosts can reach is the authenticated
-# cluster ingress. edge_loki_password comes from the vault.
-
-edge_alloy_config_file: /etc/alloy/edge.alloy
-edge_alloy_http_port: 12345
-edge_alloy_bin: /usr/bin/grafana-alloy   # Arch `grafana-alloy` pkg names the binary this, not `alloy`
-edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
-edge_loki_username: edge-1
-edge_alloy_labels:
-  environment: production
-  datacenter: homelab
@@ -1,6 +0,0 @@
---
- name: Restart alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    state: restarted
-    daemon_reload: true
@@ -1,57 +0,0 @@
---
-# Ship the systemd journal to Loki with Grafana Alloy.
-#
-# Pairs with edge-flash-wear, which makes the journal volatile: container logs
-# reach the journal via docker s journald driver and native services log to it
-# directly, so one journal source covers everything. Alloy s storage/WAL sits
-# on /run (tmpfs) so it does not write to flash either.
-
- name: Install grafana-alloy
-  ansible.builtin.package:
-    name: grafana-alloy
-    state: present
-
- name: Create alloy system group
-  ansible.builtin.group:
-    name: alloy
-    system: true
-
- name: Create alloy system user
-  ansible.builtin.user:
-    name: alloy
-    group: alloy
-    groups: systemd-journal
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
- name: Create alloy config directory
-  ansible.builtin.file:
-    path: /etc/alloy
-    state: directory
-    owner: alloy
-    group: alloy
-    mode: "0750"
-
- name: Deploy alloy config (journal -> Loki)
-  ansible.builtin.template:
-    src: edge-alloy.alloy.j2
-    dest: "{{ edge_alloy_config_file }}"
-    owner: alloy
-    group: alloy
-    mode: "0600"   # carries the Loki basic-auth password
-  notify: Restart alloy
-
- name: Deploy alloy systemd unit
-  ansible.builtin.template:
-    src: edge-alloy.service.j2
-    dest: /etc/systemd/system/edge-alloy.service
-    mode: "0644"
-  notify: Restart alloy
-
- name: Enable edge-alloy
-  ansible.builtin.systemd:
-    name: edge-alloy
-    enabled: true
-    state: started
-    daemon_reload: true
@@ -1,11 +0,0 @@
---
-# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
-# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
-# VRRP group, which needs layer-2 adjacency with the other members on the
-# service VLAN. The untagged VLAN 30 lease stays this host's only default
-# route -- see templates/10-vlan40.network.j2.
-edge_vlan40_enabled: false
-edge_vlan40_parent: enp1s0
-edge_vlan40_id: 40
-edge_vlan40_name: enp1s0.40
-edge_vlan40_parent_network: 20-wired.network
@@ -1,9 +0,0 @@
---
-# Reconfiguring the parent is what actually materialises the VLAN child -- a
-# bare reload leaves the existing link as-is. This re-runs DHCP on the parent,
-# which is a headless host's only path to the network, so it is the one
-# genuinely risky step in the VLAN change.
- name: Reload networkd for VLAN
-  ansible.builtin.shell:
-    cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
-  changed_when: true
@@ -1,48 +0,0 @@
---
-# Tagged VLAN sub-interface via systemd-networkd.
-#
-# Additive by construction: the parent link keeps its own .network file and
-# gains the VLAN through a drop-in, so reverting is deleting three paths. These
-# hosts are headless and the parent is their only route to the network, hence
-# the caution.
-
- name: Create VLAN sub-interface netdev
-  ansible.builtin.template:
-    src: 10-vlan40.netdev.j2
-    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
- name: Configure VLAN sub-interface addressing
-  ansible.builtin.template:
-    src: 10-vlan40.network.j2
-    dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
- name: Ensure parent network drop-in directory exists
-  ansible.builtin.file:
-    path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
-    state: directory
-    mode: "0755"
-  when: edge_vlan40_enabled
-  tags: [vlan40]
-
- name: Attach VLAN to parent link via drop-in
-  ansible.builtin.template:
-    src: vlan40-parent-dropin.conf.j2
-    dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
-    mode: "0644"
-  when: edge_vlan40_enabled
-  notify: Reload networkd for VLAN
-  tags: [vlan40]
-
- name: Enable systemd-networkd
-  ansible.builtin.service:
-    name: systemd-networkd
-    state: started
-    enabled: true
@@ -1,60 +0,0 @@
---
-# Heating system over the Viessmann E3 CAN bus: open3e (heat pump) and
-# e3oncan (E380 energy meter). The can0 interface itself is the socketcan
-# role.
-
-# Service root. Kept per-role so the role is usable on its own.
-edge_base_dir: /opt/edge-host
-
-edge_open3e_mqtt_user: open3e
-edge_open3e_mqtt_topic: open3e
-edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-edge_open3e_poll_seconds: 30
-# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
-# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
-# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
-# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
-# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
-# the internal PV-surplus EMS depends on). Decodes via the master O3EList
-# codec inherited through the vx3plus overlay.
-# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
-# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
-# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
-# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
-# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
-# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
-# entries when added), so the multi-frame read is cheap at a 30 s poll.
-# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
-# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
-edge_open3e_datapoints: >-
-  vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
-# ECU addresses; -dev selects codec set, tx selects the UDS address.
-# vcmu (refrigerant-circuit controller) has no dedicated codec set —
-# omitting dpList makes open3e fall back to the generic datapoint list,
-# which decodes all polled VCMU DIDs.
-edge_open3e_devices:
-  vitocal: { tx: "0x680", dpList: "vcal" }
-  # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
-  # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
-  vx3: { tx: "0x6a1", dpList: "/config/vx3plus.py" }
-  vcmu: { tx: "0x68c" }
-
-edge_e3oncan_gap_seconds: 10
-edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
-
-# Secrets come from inventory/group_vars/edge_host/vault.yml:
-#   edge_open3e_mqtt_password
-
-# The image carrying both readers, built and pinned in CI
-# (ci/images/heating-can/Containerfile). The source pins live in the
-# Containerfile, in the artifact -- not here -- so what is deployed is a tag,
-# and `docker compose up -d` on an unchanged tag is a genuine no-op.
-#
-# Tracked by the weekly update-audit as an ordinary image, which is why this is
-# simpler to keep current than the two git refs it replaced: one publishes no
-# tags at all, the other was pinned to an untagged commit between releases.
-heating_can_image: gitea.renner.uno/gitea_admin/heating-can:latest
-
-# Compose project directory. Separate from the old native layout under
-# {{ edge_base_dir }}/open3e, which this role removes.
-heating_can_base_dir: "{{ edge_base_dir }}/heating-can"
@@ -1,10 +0,0 @@
---
- name: Reload systemd
-  ansible.builtin.systemd:
-    daemon_reload: true
-
- name: Restart heating-can
-  ansible.builtin.command:
-    cmd: docker compose up -d --force-recreate
-    chdir: "{{ heating_can_base_dir }}"
-  changed_when: true
@@ -1,164 +0,0 @@
---
-# Heating system over the Viessmann E3 CAN bus.
-#
-# Two readers of the same heating installation, supervised inside one container
-# from one image (ci/images/heating-can):
-#   open3e    UDS request/response to the Vitocal heat pump and Vitocharge VX3
-#   e3oncan   passive CANopen listener for the E380 grid energy meter, which
-#             feeds the heat pump's own energy management
-#
-# They coexist because e3oncan issues no bus writes.
-#
-# Nothing is installed here. The image is built in CI with both sources pinned,
-# published to the registry, and this host only pulls and runs it. That is what
-# removes the churn: these were pip-installed into venvs on the node, and pip
-# re-cloned and rebuilt a VCS requirement on every converge -- restarting the
-# heating bridge each time -- which pinning the SHA did not fix.
-#
-# network_mode: host is required, not incidental: SocketCAN interfaces live in
-# the host network namespace, so can0 is only reachable that way. It is also how
-# the local MQTT broker on 127.0.0.1 is reached. CAN_RAW needs no capabilities,
-# so both run unprivileged.
-#
-# The can0 interface itself belongs to the socketcan role; this role consumes it.
-
- name: Create heating-can directory
-  ansible.builtin.file:
-    path: "{{ heating_can_base_dir }}"
-    state: directory
-    mode: "0755"
-
-# --- Migration off the native install ---------------------------------------
-# Must happen before the containers start. Two open3e instances issuing UDS
-# requests on one CAN bus is the failure mode to avoid, so the old units are
-# stopped and masked rather than merely disabled -- masked so a stray
-# `systemctl start` or a leftover dependency cannot bring them back alongside
-# the containers.
- name: Find which native CAN units still exist
-  ansible.builtin.stat:
-    path: "/etc/systemd/system/{{ item }}.service"
-  loop:
-    - open3e
-    - e3oncan
-  register: heating_can_native_units
-
-# Stop and disable BEFORE anything else, and do NOT swallow the result.
-#
-# The first version did stop + disable + mask in one call with
-# failed_when: false, then deleted the unit files. Masking a unit whose file
-# still exists conflicts, so the call failed, failed_when hid it, nothing was
-# stopped -- and the next task deleted the unit files out from under two
-# running processes. systemctl is-enabled then reported "not-found" and looked
-# clean while is-active still said "active".
-#
-# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
-# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
-# kicking each other off the broker.
-#
-# Only units that actually exist are touched, so a fresh host does not need an
-# error swallowed to get through this.
- name: Stop and disable the native CAN services
-  ansible.builtin.systemd:
-    name: "{{ item.item }}"
-    state: stopped
-    enabled: false
-  loop: "{{ heating_can_native_units.results }}"
-  loop_control:
-    label: "{{ item.item }}"
-  when: item.stat.exists
-
-# Masking is deliberately NOT used. It writes a symlink at the same path the
-# next task removes, so the two cancel out -- which is half of how the original
-# failed. With the unit files gone there is nothing left to start.
- name: Remove the native unit files
-  ansible.builtin.file:
-    path: "/etc/systemd/system/{{ item }}.service"
-    state: absent
-  loop:
-    - open3e
-    - e3oncan
-  notify: Reload systemd
-
-# The invariant that actually matters is "no native process is on the CAN bus",
-# and systemd reporting success is not the same claim. Two open3e instances
-# polling one bus is the failure this whole block exists to prevent, so it is
-# asserted directly rather than inferred from the calls above.
- name: Check for surviving native CAN processes
-  ansible.builtin.command:
-    argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
-  register: heating_can_survivors
-  changed_when: false
-  failed_when: false
-
- name: Assert no native CAN process survived
-  ansible.builtin.assert:
-    that:
-      - heating_can_survivors.stdout | default('') | length == 0
-    fail_msg: >-
-      A native open3e/e3oncan process is still running after the units were
-      stopped and removed:
-      {{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
-      Starting the container now would put two readers on the same CAN bus --
-      two open3e instances issue competing UDS requests and take over each
-      other's MQTT session. Kill the survivors, then re-run.
-    success_msg: No native CAN processes remain; safe to start the container.
-
- name: Remove the native venvs
-  ansible.builtin.file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "{{ edge_base_dir }}/open3e/venv"
-    - "{{ edge_base_dir }}/e3oncan/venv"
-    - "{{ edge_base_dir }}/e3oncan/src"
-    - "{{ edge_base_dir }}/open3e"
-    - "{{ edge_base_dir }}/e3oncan"
-
-# --- Configuration (bind-mounted into the containers) ------------------------
- name: Deploy open3e devices.json
-  ansible.builtin.copy:
-    content: "{{ edge_open3e_devices | to_nice_json }}\n"
-    dest: "{{ heating_can_base_dir }}/devices.json"
-    mode: "0644"
-  notify: Restart heating-can
-
- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
-  ansible.builtin.copy:
-    src: open3e-vx3plus.py
-    dest: "{{ heating_can_base_dir }}/vx3plus.py"
-    mode: "0644"
-  notify: Restart heating-can
-
-# --- Compose project ---------------------------------------------------------
- name: Deploy the broker credential env file
-  ansible.builtin.template:
-    src: heating-can.env.j2
-    dest: "{{ heating_can_base_dir }}/heating-can.env"
-    mode: "0600"
-  notify: Restart heating-can
-
- name: Deploy heating-can compose project
-  ansible.builtin.template:
-    src: docker-compose.yml.j2
-    dest: "{{ heating_can_base_dir }}/docker-compose.yml"
-    mode: "0644"
-  notify: Restart heating-can
-
- name: Pull the heating-can image
-  ansible.builtin.command:
-    cmd: docker compose pull
-    chdir: "{{ heating_can_base_dir }}"
-  changed_when: false
-
-# Unlike the zwave and meter services there is no missing-device tolerance here:
-# can0 is created by the socketcan role from a USB adapter that is part of this
-# host's purpose. If it is absent the heating telemetry is down and the run
-# should say so rather than pass quietly.
- name: Start heating-can services
-  ansible.builtin.command:
-    cmd: docker compose up -d
-    chdir: "{{ heating_can_base_dir }}"
-  register: heating_can_up
-  changed_when: >-
-    'Started' in heating_can_up.stderr or 'Created' in heating_can_up.stderr
-    or 'Started' in heating_can_up.stdout or 'Creating' in heating_can_up.stdout
@@ -1,35 +0,0 @@
-# Managed by ansible (roles/heating-can).
-#
-# One container. Both CAN readers live in the image and are supervised inside
-# it -- this file supplies configuration and nothing else. It does not know
-# open3e's or E3onCAN's command line.
-#
-# network_mode: host is required: SocketCAN interfaces live in the host network
-# namespace, so {{ edge_can_interface }} is only reachable this way, and it is
-# also how the local MQTT broker is reached. CAN_RAW needs no capabilities, so
-# the image runs unprivileged.
-services:
-  heating-can:
-    image: {{ heating_can_image }}
-    container_name: heating-can
-    restart: unless-stopped
-    network_mode: host
-    volumes:
-      - ./devices.json:/config/devices.json:ro
-      - ./vx3plus.py:/config/vx3plus.py:ro
-    env_file:
-      # Broker credential, 0600 -- see heating-can.env.j2.
-      - heating-can.env
-    environment:
-      CAN_INTERFACE: "{{ edge_can_interface }}"
-      MQTT_HOST: "{{ edge_mqtt_host }}"
-      MQTT_PORT: "{{ edge_mqtt_port }}"
-      MQTT_TOPIC: "{{ edge_open3e_mqtt_topic }}"
-      MQTT_FORMAT: "{{ edge_open3e_mqtt_format }}"
-      E3ONCAN_MQTT_FORMAT: "{{ edge_e3oncan_mqtt_format }}"
-      OPEN3E_CONFIG: /config/devices.json
-      OPEN3E_CLIENT_ID: open3e-edge
-      OPEN3E_POLL_SECONDS: "{{ edge_open3e_poll_seconds }}"
-      OPEN3E_DATAPOINTS: "{{ edge_open3e_datapoints }}"
-      E3ONCAN_DEVICE: e380
-      E3ONCAN_GAP_SECONDS: "{{ edge_e3oncan_gap_seconds }}"
@@ -1,8 +0,0 @@
-# Managed by ansible (roles/heating-can). 0600 -- carries the broker credential.
-#
-# Kept out of docker-compose.yml, which is 0644: the sibling roles on this host
-# do the same (power-meter-sml renders its config at 0600 while its compose file
-# stays world-readable), and the native unit this replaced used a 0600 env file
-# for exactly this value.
-MQTT_USER={{ edge_open3e_mqtt_user }}
-MQTT_PASSWORD={{ edge_open3e_mqtt_password }}
@@ -4,29 +4,6 @@
 keepalived_ntp_router_id: 4
 keepalived_ntp_vip: "10.2.40.4"
 keepalived_ntp_interface: "vmbr1.40"
-
-# IPv6 service address for NTP (ntp.srv.renner.uno). Empty disables the v6
-# instance entirely. IPv6 is the primary address family on this network, so
-# this is the path that actually matters -- the v4 VIP above is legacy.
-#
-# Held by keepalived rather than assigned statically by networkd so that it is
-# gated on check_ntp: present only while this host is fit to serve time, and
-# withdrawn the moment it is not. A statically configured address is carried
-# unconditionally from boot, which is how edge-1 came to answer NTP queries on
-# 2026-08-04 while its clock was ~16 days in the past.
-#
-# Needs its own virtual_router_id: it is a distinct VRRP instance, and a VRID
-# collision with the IPv4 instance on the same interface would have the two
-# fighting over a single election.
-keepalived_ntp_vip6: ""
-keepalived_ntp_router_id6: 6
-
-# Link-local address for the IPv6 VRRP instance, carried as its FIRST
-# virtual_ipaddress. RFC 5798 gives an IPv6 virtual router a link-local primary,
-# and keepalived warns "the first IPv6 VIP address should be link local" without
-# one. Chosen to mirror the service address rather than derived from the virtual
-# MAC, since no real NIC generates fe80::123 so there is nothing to collide with.
-keepalived_ntp_vip6_linklocal: "fe80::123/64"
 # NOTE: plaintext VRRP auth, unvaulted. Carried over as-is from the original
 # role; VRRP PASS auth is weak regardless (sent in the clear on the segment),
 # so this is a documented weakness rather than a secret worth vaulting.
@@ -41,17 +18,6 @@ keepalived_ntp_check_script: "/usr/local/bin/check-ntp-health.sh"
 keepalived_ntp_check_interval: 2
 keepalived_ntp_check_weight: -20

-# Maximum |system clock offset| in seconds before the host refuses to hold the
-# VIP. chronyd reports this as "System time : <n> seconds slow/fast of NTP
-# time". Anything above this means the host knows its clock is wrong and must
-# not serve time to clients -- see the reasoning in check-ntp-health.sh.j2.
-#
-# A serving NTP host should be within milliseconds of real time, so 1s is
-# already several orders of magnitude beyond normal. It is deliberately not
-# tighter: a brief excursion at boot, before makestep lands, should not flap
-# the VIP (fall 2 at a 2s interval also requires the failure to persist).
-keepalived_ntp_max_offset: 1.0
-
 # sysctl applied for VRRP. Default preserves the PVE hosts' existing behaviour
 # exactly. ip_forward is NOT appropriate everywhere -- on a host with legs in
 # two VLANs it turns the box into a router between them, bypassing the gateway's
@@ -1,39 +1,22 @@
 #!/bin/bash
 # Keepalived State Change Notification Script
-#
-# Two VRRP instances call this: NTP_VIP (IPv4, legacy) passes MASTER/BACKUP/
-# FAULT, and NTP_VIP6 (IPv6, the primary path) passes the same suffixed with
-# '6'. They are logged distinctly because the two can legitimately diverge --
-# a host can hold one family and not the other -- and a log line that does not
-# say which family it means is useless during an incident.

 TYPE=$1
 NAME=$(hostname)
 TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

-log() {
-    echo "$TIMESTAMP: $NAME $1" >> /var/log/keepalived-state.log
-    logger -t keepalived "$NAME $1"
-}
-
 case $TYPE in
    MASTER)
-        log "transitioned to MASTER state (NTP, IPv4)"
+        echo "$TIMESTAMP: $NAME transitioned to MASTER state (NTP)" >> /var/log/keepalived-state.log
+        logger -t keepalived "$NAME is now NTP MASTER"
        ;;
    BACKUP)
-        log "transitioned to BACKUP state (NTP, IPv4)"
+        echo "$TIMESTAMP: $NAME transitioned to BACKUP state (NTP)" >> /var/log/keepalived-state.log
+        logger -t keepalived "$NAME is now NTP BACKUP"
        ;;
    FAULT)
-        log "transitioned to FAULT state (NTP, IPv4)"
-        ;;
-    MASTER6)
-        log "transitioned to MASTER state (NTP, IPv6) - now serving ntp.srv.renner.uno"
-        ;;
-    BACKUP6)
-        log "transitioned to BACKUP state (NTP, IPv6) - released ntp.srv.renner.uno"
-        ;;
-    FAULT6)
-        log "transitioned to FAULT state (NTP, IPv6) - withdrew ntp.srv.renner.uno, unfit to serve time"
+        echo "$TIMESTAMP: $NAME transitioned to FAULT state (NTP)" >> /var/log/keepalived-state.log
+        logger -t keepalived "$NAME is now in NTP FAULT state"
        ;;
    *)
        echo "$TIMESTAMP: $NAME unknown state: $TYPE" >> /var/log/keepalived-state.log
@@ -35,121 +35,6 @@
    state: present
  when: ansible_facts['os_family'] == 'Archlinux'

-# --- Verify nothing else is assigning the IPv6 service address ---------------
-#
-# The removal itself lives in withdraw-static.yml, run as a separate fleet-wide
-# play before any host reaches this role (see install-ntp-ha.yml). It has to
-# complete everywhere first: DAD is enabled on the VRRP address, so a static
-# copy on one host and a VRRP-managed copy on another cannot coexist -- the
-# kernel marks the newcomer `dadfailed` and its master serves nothing.
-#
-# What remains here are the guards. They run on every execution, not just the
-# migration, and fail the play if any static assignment survives -- which would
-# both defeat the health gating (a static copy does not disappear when
-# keepalived decides the host is unfit) and break the VRRP copy via DAD.
-#
-# The config file is what gets checked, not the running address, and that is
-# what makes it safe to run every time: keepalived never writes to these files,
-# so anything found is by definition static, whereas the live address is
-# legitimately present on whichever host is master.
-#
-# Prefix-agnostic and \b-anchored on purpose. Matching the full
-# address/prefixlen would fail open against a hand-written assignment using any
-# other mask.
- name: Detect any remaining static assignment of the service address (ifupdown)
-  ansible.builtin.command:
-    argv:
-      - grep
-      - -rlE
-      - "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
-      - /etc/network/interfaces
-      - /etc/network/interfaces.d/
-  register: keepalived_ntp_static6_ifupdown
-  changed_when: false
-  failed_when: false
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - ansible_facts['os_family'] == 'Debian'
-
- name: Assert ifupdown is not also assigning the service address
-  ansible.builtin.assert:
-    that:
-      - keepalived_ntp_static6_ifupdown.stdout | default('') | length == 0
-    fail_msg: >-
-      {{ keepalived_ntp_vip6 }} is still assigned statically by ifupdown on
-      this host
-      ({{ keepalived_ntp_static6_ifupdown.stdout_lines | default([]) | join(', ') }}).
-      The automatic removal only matches the exact form
-      `up ip -6 addr replace {{ keepalived_ntp_vip6 }} dev <if> nodad`, so a
-      different prefix length or layout is left in place. Handing the address
-      to keepalived while ifupdown also assigns it means it would NOT be
-      withdrawn when check_ntp fails -- the host would keep serving time on an
-      address VRRP believes it has released. Remove the assignment from the
-      file(s) above, then re-run.
-    success_msg: >-
-      No static ifupdown assignment of {{ keepalived_ntp_vip6 }}; safe for
-      keepalived to own it.
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - ansible_facts['os_family'] == 'Debian'
-
-# On Arch (edge-1) the static assignment lives in a systemd-networkd file owned
-# by the edge-vlan role, so this role must not rewrite it -- but it must not
-# quietly proceed either. install-edge-host.yml is a separate play with no
-# ordering relationship to this one, so nothing guarantees it has been re-run
-# to drop the [Address] block. If it has not, edge-1 would end up holding
-# fd40::123 statically AND running a VRRP instance for the same address:
-# keepalived would believe it had withdrawn the address while networkd kept it
-# assigned, and the host would go on answering NTP with a clock the health
-# check had already rejected.
-#
-# That is the precise failure of 2026-08-04, on the host it happened to, so it
-# fails fast rather than being left to inspection.
-# Matched on the bare address, not the full address/prefixlen, and anchored
-# with \b so it cannot also hit a longer address sharing the prefix (fd40::1234).
-#
-# Searching for the literal "fd40::123/128" would fail open: a hand-written
-# [Address] using any other mask -- /64, say -- would not match, the assert
-# below would pass, and keepalived would be handed an address networkd is
-# still assigning. Since this check is detect-only, with no delete afterwards
-# to catch the mistake, a miss here is silent. The Debian path does not share
-# the problem: `ip -6 addr show ... to <prefix>` matches by range regardless of
-# the assigned prefix length, and its delete is re-verified by assert.
- name: Detect a static assignment of the service address (systemd-networkd)
-  ansible.builtin.command:
-    argv:
-      - grep
-      - -rlE
-      - "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
-      - /etc/systemd/network/
-  register: keepalived_ntp_static6_networkd
-  changed_when: false
-  failed_when: false
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - ansible_facts['os_family'] == 'Archlinux'
-
- name: Assert networkd is not also assigning the service address
-  ansible.builtin.assert:
-    that:
-      - keepalived_ntp_static6_networkd.stdout | default('') | length == 0
-    fail_msg: >-
-      {{ keepalived_ntp_vip6 }} is still assigned statically by
-      systemd-networkd on this host
-      ({{ keepalived_ntp_static6_networkd.stdout_lines | default([]) | join(', ') }}).
-      Handing the address to keepalived while networkd also assigns it means
-      it would NOT be withdrawn when check_ntp fails -- the host would keep
-      serving time on an address VRRP believes it has released.
-      Re-apply the edge-vlan role first, whose template no longer carries the
-      [Address] block:
-      ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
-    success_msg: >-
-      No static networkd assignment of {{ keepalived_ntp_vip6 }}; safe for
-      keepalived to own it.
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - ansible_facts['os_family'] == 'Archlinux'
-
 - name: Create keepalived configuration
  ansible.builtin.template:
    src: keepalived.conf.j2
@@ -1,134 +0,0 @@
---
-# Withdraw a static assignment of the IPv6 NTP service address.
-#
-# Included by install-ntp-ha.yml as a FIRST play across the whole ntp_cluster,
-# before any host is given the NTP_VIP6 instance. That ordering is the point of
-# splitting it out.
-#
-# DAD is enabled on the VRRP address (see keepalived.conf.j2), so a static copy
-# on any host and a VRRP-managed copy on another cannot coexist: the kernel sees
-# the duplicate and marks the newcomer `dadfailed`, leaving a master that
-# believes it is serving while carrying an unusable address. Observed on edge-1
-# on 2026-08-07, where keepalived reported MASTER while the address sat
-# `dadfailed deprecated tentative` because three PVE nodes still held it
-# statically.
-#
-# Doing the withdrawal per host inside the main role could not avoid that: with
-# serial: 1 the first host would claim the address via VRRP while the rest were
-# still static. Fleet-wide first, then claim.
-#
-# fd40::123 was assigned by hand on all four servers, in two different ways,
-# none of it in version control:
-#
-#   PVE nodes  an `up ip -6 addr replace ... nodad` hook in
-#              /etc/network/interfaces
-#   edge-1     an [Address] block in /etc/systemd/network/, owned by the
-#              edge-vlan role and removed by re-applying it -- not touched
-#              here, since this role must not rewrite another role's file
-#
-# There is a service gap between this play and the next, while no host holds
-# the address. It is bounded by the run time of one play and clients simply
-# retry; the alternative is the dadfailed state above, which does not resolve
-# on its own.
-
- name: Remove static assignment of the NTP service address (ifupdown)
-  ansible.builtin.lineinfile:
-    path: /etc/network/interfaces
-    regexp: '^\s*up\s+ip\s+-6\s+addr\s+(replace|add)\s+{{ keepalived_ntp_vip6 | regex_escape }}\b'
-    state: absent
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - ansible_facts['os_family'] == 'Debian'
-  register: keepalived_ntp_static6_removed
-
-# Everything below is gated on the config line having just been removed, so the
-# runtime delete happens once -- on the migration -- and never again.
-#
-# The gate is load-bearing. Once this has landed, the address is legitimately
-# present on whichever host is the elected v6 master, placed there by
-# keepalived, and a delete cannot tell that apart from a static assignment (the
-# address looks identical either way). Ungated, every later run would delete the
-# master's address and restart keepalived, dropping the service address and
-# forcing a re-election each time -- the flapping this whole change exists to
-# prevent.
- name: Check whether the service address is still assigned statically
-  ansible.builtin.command:
-    argv:
-      - ip
-      - -6
-      - addr
-      - show
-      - dev
-      - "{{ keepalived_ntp_interface }}"
-      - to
-      - "{{ keepalived_ntp_vip6 }}"
-  register: keepalived_ntp_vip6_present
-  changed_when: false
-  failed_when: false
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - keepalived_ntp_static6_removed.changed | default(false)
-
-# failed_when: false because the desired end state is "address absent". An
-# `ip addr del` against an address that has already gone is the outcome this
-# task wants, not a reason to abort the play for that host.
- name: Withdraw the statically assigned service address
-  ansible.builtin.command:
-    argv:
-      - ip
-      - -6
-      - addr
-      - del
-      - "{{ keepalived_ntp_vip6 }}"
-      - dev
-      - "{{ keepalived_ntp_interface }}"
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - keepalived_ntp_static6_removed.changed | default(false)
-    - keepalived_ntp_vip6_present.stdout | default('') | length > 0
-  failed_when: false
-
-# The withdrawal gets exactly one attempt -- the gate that makes it idempotent
-# also means it never retries -- and the delete swallows its own errors. So the
-# result is asserted rather than assumed.
-#
-# `ip addr del` matches on address AND prefix length: if a host's static
-# assignment used a different mask or interface than ntp_vip6 /
-# keepalived_ntp_interface, the delete is a silent no-op and the address stays
-# outside VRRP control permanently, with the play reporting success.
- name: Confirm the static service address was actually withdrawn
-  ansible.builtin.command:
-    argv:
-      - ip
-      - -6
-      - addr
-      - show
-      - dev
-      - "{{ keepalived_ntp_interface }}"
-      - to
-      - "{{ keepalived_ntp_vip6 }}"
-  register: keepalived_ntp_vip6_after
-  changed_when: false
-  failed_when: false
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - keepalived_ntp_static6_removed.changed | default(false)
-
- name: Assert the service address is no longer assigned outside keepalived
-  ansible.builtin.assert:
-    that:
-      - keepalived_ntp_vip6_after.stdout | default('') | length == 0
-    fail_msg: >-
-      {{ keepalived_ntp_vip6 }} is still assigned on
-      {{ keepalived_ntp_interface }} after the static assignment was removed.
-      `ip -6 addr del` matches address AND prefix length, so this usually means
-      the on-host assignment differs from ntp_vip6 or uses another interface.
-      Leaving it would both keep the address outside VRRP control and make the
-      VRRP-managed copy fail DAD elsewhere. Inspect with
-      `ip -6 addr show dev {{ keepalived_ntp_interface }}` and
-      `grep -rn {{ keepalived_ntp_vip6 }} /etc/network/ /etc/systemd/network/`.
-    success_msg: >-
-      {{ keepalived_ntp_vip6 }} withdrawn; safe for keepalived to claim it.
-  when:
-    - keepalived_ntp_vip6 | length > 0
-    - keepalived_ntp_static6_removed.changed | default(false)
@@ -6,80 +6,23 @@
 # The chrony unit is named 'chrony' on Debian and 'chronyd' on Arch. This is
 # templated rather than hardcoded: a wrong name here fails every check, which
 # parks the host in permanent FAULT so it never takes the VIP.
-#
-# This check must answer "is it safe for clients to take time from this host",
-# NOT merely "is chronyd alive". Those came apart on 2026-08-04: edge-1's dead
-# CMOS battery left its clock ~16 days in the past, chronyd was running,
-# answering queries and listening on 123, and the `local stratum` directive made
-# it advertise itself to clients as a valid reference. Every liveness check
-# passed for ~18h while the host stood ready to serve a clock from July. The
-# earlier version of this script ran `chronyc tracking` and discarded its
-# output -- the one command that knew the answer.
-#
-# Deliberate distinction, because these must not be conflated:
-#   * "provably wrong"  -> chronyd HAS upstream measurements and they say the
-#                          clock is far off. Fail: drop the VIP so a host with a
-#                          sane clock serves instead.
-#   * "no upstream yet" -> nothing reachable / not yet measured. Also fail, but
-#                          note that when the WAN is down every host fails
-#                          together, they all lose the same weight, relative
-#                          priority is unchanged and the VIP stays put. So the
-#                          local-reference fallback still works; what this
-#                          prevents is one host with a broken clock winning.
-
-LOG=/var/log/keepalived-ntp-health.log
-MAX_OFFSET={{ keepalived_ntp_max_offset }}
-
-fail() {
-    echo "$(date): $1" >> "$LOG"
-    exit 1
-}

 # Check 1: Is chrony service running?
 if ! systemctl is-active --quiet {{ chrony_service_name }}; then
-    fail "{{ chrony_service_name }} service not active"
+    echo "$(date): {{ chrony_service_name }} service not active" >> /var/log/keepalived-ntp-health.log
+    exit 1
 fi

-# Check 2: Is chrony responding, and what does it actually say?
-# Captured, not discarded -- the offset below is the whole point of this script.
-if ! TRACKING=$(chronyc tracking 2>/dev/null); then
-    fail "Chrony not responding to tracking query"
+# Check 2: Is chrony synchronized or at least running?
+if ! chronyc tracking >/dev/null 2>&1; then
+    echo "$(date): Chrony not responding to tracking query" >> /var/log/keepalived-ntp-health.log
+    exit 1
 fi

 # Check 3: Is NTP serving clients (port 123)?
 if ! ss -lun | grep -q ':123 '; then
-    fail "NTP port 123 not listening"
-fi
-
-# Check 4: Is the clock provably wrong?
-#
-# "System time : <n> seconds (slow|fast) of NTP time" is chronyd's own estimate
-# of how far the system clock is from real time, derived from upstream
-# measurements. On edge-1 during the incident this line read
-# "1376235.375000000 seconds slow of NTP time" while every other check passed.
-OFFSET=$(echo "$TRACKING" | awk '/^System time/ {print $4}')
-if [ -n "$OFFSET" ]; then
-    if awk -v o="$OFFSET" -v m="$MAX_OFFSET" 'BEGIN {exit !(o > m)}'; then
-        fail "System clock off by ${OFFSET}s (max ${MAX_OFFSET}s) - refusing to serve time"
-    fi
-fi
-
-# Check 5: Has chronyd ever validated its clock against a real source?
-#
-# With `local stratum N` configured, chronyd advertises itself as a usable
-# reference even when it has never synchronised to anything -- the upstream
-# chrony documentation is explicit that the local reference activates
-# immediately in that case. So a host that has never reached an upstream still
-# looks authoritative to clients. Refuse the VIP until a real source has been
-# selected.
-#
-# `chronyc sources` marks the selected source with '*' (best) or '+' (combined)
-# in the second column. Neither present means nothing upstream is accepted.
-if ! SOURCES=$(chronyc sources 2>/dev/null); then
-    fail "Chrony not responding to sources query"
-fi
-if ! echo "$SOURCES" | grep -qE '^.\*|^.\+'; then
-    fail "No upstream source selected - clock not validated, refusing to serve time"
+    echo "$(date): NTP port 123 not listening" >> /var/log/keepalived-ntp-health.log
+    exit 1
 fi

 # All checks passed
@@ -39,72 +39,3 @@ vrrp_instance NTP_VIP {
    notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP"
    notify_fault "/usr/local/bin/keepalived-notify.sh FAULT"
 }
-{% if keepalived_ntp_vip6 %}
-
-# IPv6 service address for NTP (ntp.srv.renner.uno). IPv6 is the primary
-# address family here, so this instance -- not the IPv4 one above -- is the
-# path that matters.
-#
-# A separate instance is mandatory, not stylistic: keepalived requires that
-# "all addresses in virtual_ipaddress must be of the same family", so the v6
-# address cannot be added to NTP_VIP.
-#
-# No authentication block: VRRPv3 removed authentication (RFC 5798), and
-# keepalived always uses version 3 for IPv6 instances. Configuring auth here
-# is rejected rather than ignored.
-#
-# The first address is link-local, and must be. RFC 5798 gives an IPv6 virtual
-# router a link-local address as its primary, and keepalived warns
-# "the first IPv6 VIP address should be link local" otherwise. It carries on,
-# but the instance is then sourcing VRRPv3 from something the protocol does not
-# expect.
-#
-# DAD is deliberately left ON (no -nodad). The flag was inherited from the
-# previous design, where every NTP server carried fd40::123 simultaneously and
-# duplicate detection had to be suppressed for that to work at all. Under
-# single-master VRRP the opposite is true: exactly one host should hold the
-# address, so a duplicate is a genuine fault and detecting it is the point.
-#
-# This does mean the migration cannot leave a static copy anywhere while a
-# VRRP-managed one exists -- DAD sees the two and marks the newcomer
-# `dadfailed`, leaving a master that believes it is serving while carrying an
-# unusable address. Observed on edge-1 on 2026-08-07. install-ntp-ha.yml
-# therefore withdraws every static assignment fleet-wide in a first play
-# before any host configures this instance.
-#
-# track_script is what makes this health-gated: the address is only held while
-# check_ntp passes, and is withdrawn as soon as it fails. On withdrawal
-# keepalived sends unsolicited Neighbour Advertisements so client neighbour
-# caches are updated immediately -- a statically assigned address cannot do
-# this, and clients would keep sending to the dead holder until their own
-# neighbour unreachability detection expired.
-vrrp_instance NTP_VIP6 {
-    state {{ keepalived_ntp_state }}
-    interface {{ keepalived_ntp_interface }}
-    virtual_router_id {{ keepalived_ntp_router_id6 }}
-    priority {{ keepalived_ntp_priority }}
-    advert_int 1
-
-    virtual_ipaddress {
-        {{ keepalived_ntp_vip6_linklocal }} dev {{ keepalived_ntp_interface }}
-        {{ keepalived_ntp_vip6 }} dev {{ keepalived_ntp_interface }}
-    }
-
-    # weight 0 overrides the script's own weight, and the distinction decides
-    # whether this works at all. A non-zero weight only SUBTRACTS from priority
-    # on failure -- so the last remaining host, having no one to lose the
-    # election to, stays MASTER and keeps serving time with a clock it has
-    # already reported as wrong. Weight 0 instead transitions the instance to
-    # FAULT after <fall> failures, which releases the address unconditionally.
-    #
-    # That is the required behaviour here: a host unfit to serve time must stop
-    # answering, whether or not a replacement exists.
-    track_script {
-        check_ntp weight 0
-    }
-
-    notify_master "/usr/local/bin/keepalived-notify.sh MASTER6"
-    notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP6"
-    notify_fault "/usr/local/bin/keepalived-notify.sh FAULT6"
-}
-{% endif %}
@@ -0,0 +1,44 @@
+---
+# Loki Server Configuration
+
+# Version and download
+loki_version: "3.4.2"
+loki_arch: "linux-amd64"
+loki_download_url: "https://github.com/grafana/loki/releases/download/v{{ loki_version }}/loki-{{ loki_arch }}.zip"
+loki_bin_path: "/usr/local/bin/loki"
+
+# Directories
+loki_data_dir: "/var/lib/loki"
+loki_config_dir: "/etc/loki"
+loki_config_file: "{{ loki_config_dir }}/loki-config.yaml"
+
+# Server configuration
+loki_http_listen_port: 3100
+loki_grpc_listen_port: 9096
+loki_http_listen_address: "0.0.0.0"
+
+# Retention settings (90 days)
+loki_retention_period: "2160h"  # 90 days
+loki_retention_delete_delay: "2h"
+
+# Storage configuration
+loki_storage_type: "filesystem"
+loki_chunks_directory: "{{ loki_data_dir }}/chunks"
+loki_rules_directory: "{{ loki_data_dir }}/rules"
+
+# Index configuration
+loki_index_prefix: "loki_index_"
+loki_index_period: "24h"
+
+# Limits
+loki_ingestion_rate_mb: 10
+loki_ingestion_burst_size_mb: 20
+loki_max_streams_per_user: 10000
+loki_max_entries_limit_per_query: 5000
+
+# Query settings
+loki_max_query_length: "720h"  # 30 days max query span
+
+# Systemd service
+loki_user: "loki"
+loki_group: "loki"
@@ -0,0 +1,10 @@
+---
+- name: Restart loki
+  systemd:
+    name: loki
+    state: restarted
+    daemon_reload: yes
+
+- name: Reload systemd
+  systemd:
+    daemon_reload: yes
@@ -0,0 +1,95 @@
+---
+# Loki Server Installation and Configuration
+
+- name: Install dependencies
+  apt:
+    name:
+      - unzip
+      - curl
+    state: present
+    update_cache: yes
+
+- name: Create loki user
+  user:
+    name: "{{ loki_user }}"
+    system: yes
+    shell: /usr/sbin/nologin
+    create_home: no
+
+- name: Create directories
+  file:
+    path: "{{ item }}"
+    state: directory
+    owner: "{{ loki_user }}"
+    group: "{{ loki_group }}"
+    mode: '0755'
+  loop:
+    - "{{ loki_config_dir }}"
+    - "{{ loki_data_dir }}"
+    - "{{ loki_chunks_directory }}"
+    - "{{ loki_rules_directory }}"
+    - "{{ loki_data_dir }}/index"
+    - "{{ loki_data_dir }}/index_cache"
+    - "{{ loki_data_dir }}/compactor"
+
+- name: Download Loki
+  get_url:
+    url: "{{ loki_download_url }}"
+    dest: "/tmp/loki.zip"
+    mode: '0644'
+
+- name: Unzip Loki
+  unarchive:
+    src: "/tmp/loki.zip"
+    dest: "/tmp"
+    remote_src: yes
+
+- name: Install Loki binary
+  copy:
+    src: "/tmp/loki-{{ loki_arch }}"
+    dest: "{{ loki_bin_path }}"
+    mode: '0755'
+    remote_src: yes
+  notify: Restart loki
+
+- name: Create Loki configuration
+  template:
+    src: loki-config.yaml.j2
+    dest: "{{ loki_config_file }}"
+    owner: "{{ loki_user }}"
+    group: "{{ loki_group }}"
+    mode: '0644'
+  notify: Restart loki
+
+- name: Create Loki systemd service
+  template:
+    src: loki.service.j2
+    dest: /etc/systemd/system/loki.service
+    mode: '0644'
+  notify:
+    - Reload systemd
+    - Restart loki
+
+- name: Enable and start Loki service
+  systemd:
+    name: loki
+    state: started
+    enabled: yes
+    daemon_reload: yes
+
+- name: Wait for Loki to be ready
+  uri:
+    url: "http://127.0.0.1:{{ loki_http_listen_port }}/ready"
+    status_code: 200
+  register: result
+  until: result.status == 200
+  retries: 30
+  delay: 5
+
+- name: Clean up downloaded files
+  file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "/tmp/loki.zip"
+    - "/tmp/loki-{{ loki_arch }}"
@@ -0,0 +1,72 @@
+# Loki Configuration
+# Generated by Ansible
+
+auth_enabled: false
+
+server:
+  http_listen_address: {{ loki_http_listen_address }}
+  http_listen_port: {{ loki_http_listen_port }}
+  grpc_listen_port: {{ loki_grpc_listen_port }}
+  log_level: info
+
+common:
+  instance_addr: 127.0.0.1
+  path_prefix: {{ loki_data_dir }}
+  storage:
+    filesystem:
+      chunks_directory: {{ loki_chunks_directory }}
+      rules_directory: {{ loki_rules_directory }}
+  replication_factor: 1
+  ring:
+    kvstore:
+      store: inmemory
+
+query_range:
+  results_cache:
+    cache:
+      embedded_cache:
+        enabled: true
+        max_size_mb: 100
+
+schema_config:
+  configs:
+    - from: 2024-01-01
+      store: tsdb
+      object_store: filesystem
+      schema: v13
+      index:
+        prefix: {{ loki_index_prefix }}
+        period: {{ loki_index_period }}
+
+storage_config:
+  tsdb_shipper:
+    active_index_directory: {{ loki_data_dir }}/index
+    cache_location: {{ loki_data_dir }}/index_cache
+
+compactor:
+  working_directory: {{ loki_data_dir }}/compactor
+  compaction_interval: 10m
+  retention_enabled: true
+  retention_delete_delay: {{ loki_retention_delete_delay }}
+  retention_delete_worker_count: 150
+  delete_request_store: filesystem
+
+limits_config:
+  retention_period: {{ loki_retention_period }}
+  ingestion_rate_mb: {{ loki_ingestion_rate_mb }}
+  ingestion_burst_size_mb: {{ loki_ingestion_burst_size_mb }}
+  max_streams_per_user: {{ loki_max_streams_per_user }}
+  max_entries_limit_per_query: {{ loki_max_entries_limit_per_query }}
+  max_query_length: {{ loki_max_query_length }}
+  reject_old_samples: true
+  reject_old_samples_max_age: 168h  # 7 days
+
+ruler:
+  alertmanager_url: ""
+  storage:
+    type: local
+    local:
+      directory: {{ loki_rules_directory }}
+
+analytics:
+  reporting_enabled: false
@@ -0,0 +1,25 @@
+[Unit]
+Description=Loki Log Aggregation System
+Documentation=https://grafana.com/docs/loki/latest/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User={{ loki_user }}
+Group={{ loki_group }}
+ExecStart={{ loki_bin_path }} -config.file={{ loki_config_file }}
+ExecReload=/bin/kill -HUP $MAINPID
+Restart=on-failure
+RestartSec=10s
+LimitNOFILE=65536
+
+# Security hardening
+NoNewPrivileges=yes
+ProtectSystem=strict
+ProtectHome=yes
+ReadWritePaths={{ loki_data_dir }}
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target
@@ -1,12 +0,0 @@
---
-# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
-# mosquitto on loopback, which bridges to the central cluster broker over TLS
-# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
-# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
-# blips. The local broker writes nothing to the eMMC (persistence off, logs to
-# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
-# Upstream cluster broker the local bridge connects to (TLS; the public
-# Let's Encrypt cert is validated against the system CA bundle).
-edge_mqtt_cluster_host: mqtt.renner.uno
-edge_mqtt_cluster_tls_port: 8883
-edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
@@ -1,7 +0,0 @@
---
-# The MQTT clients (zwave-js, sml2mqtt, open3e, e3oncan) reconnect on their
-# own once the local broker is back.
- name: Restart mosquitto
-  ansible.builtin.service:
-    name: mosquitto
-    state: restarted
@@ -1,27 +0,0 @@
---
-# Local MQTT bridge broker.
-#
-# Edge clients (open3e, e3oncan, sml2mqtt) publish here over loopback; this
-# broker bridges to the cluster broker over TLS, because those clients have no
-# native MQTTS. Persistence is off and logging goes to stdout so it writes
-# nothing to local flash.
-
- name: Install mosquitto
-  ansible.builtin.package:
-    name: mosquitto
-    state: present
-
- name: Deploy local mosquitto bridge config
-  ansible.builtin.template:
-    src: edge-mqtt-bridge.conf.j2
-    dest: /etc/mosquitto/mosquitto.conf
-    owner: root
-    group: mosquitto
-    mode: "0640"   # carries the cluster bridge credentials
-  notify: Restart mosquitto
-
- name: Enable local mosquitto bridge
-  ansible.builtin.systemd:
-    name: mosquitto
-    enabled: true
-    state: started
@@ -1,23 +0,0 @@
---
-# Power-meter reader: sml2mqtt against the FTDI serial meter head.
-
-# Service root. Kept per-role so the role is usable on its own.
-edge_base_dir: /opt/edge-host
-
-# Own compose project directory. Unchanged from the path used under the
-# previous shared project.
-sml_base_dir: "{{ edge_base_dir }}/sml2mqtt"
-
-# The compose file that used to hold both this service and zwave-js. Removed
-# once, so the fixed container_name is released for the new projects.
-edge_legacy_compose: "{{ edge_base_dir }}/docker-compose.yml"
-
-# Stable by-id symlink -- see the zwave-gateway role.
-edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
-
-# Hand-pinned on purpose: critical meter bridge, deliberately excluded from
-# image automation.
-edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
-
-# Secret from inventory/group_vars/edge_host/vault.yml:
-#   edge_sml2mqtt_mqtt_password
@@ -1,6 +0,0 @@
---
- name: Restart sml2mqtt
-  ansible.builtin.command:
-    cmd: docker compose up -d --force-recreate
-    chdir: "{{ sml_base_dir }}"
-  changed_when: true
@@ -1,55 +0,0 @@
---
-# Power-meter reader: sml2mqtt against the FTDI serial meter head.
-#
-# Its own compose project. It shared a compose file with the Z-Wave gateway,
-# which meant a change to either recreated both -- two unrelated services bound
-# to different USB devices. Data paths are unchanged.
-
- name: Create sml2mqtt directory
-  ansible.builtin.file:
-    path: "{{ sml_base_dir }}"
-    state: directory
-    mode: "0755"
-
-# sml2mqtt reads only a config file and has no env override for the password,
-# so the rendered config carries it -- 0600, root-owned, same trust boundary as
-# the k8s init-container approach it replaced.
- name: Deploy sml2mqtt config
-  ansible.builtin.template:
-    src: sml2mqtt-config.yml.j2
-    dest: "{{ sml_base_dir }}/config.yml"
-    mode: "0600"
-  notify: Restart sml2mqtt
-
- name: Deploy sml2mqtt compose project
-  ansible.builtin.template:
-    src: docker-compose.yml.j2
-    dest: "{{ sml_base_dir }}/docker-compose.yml"
-    mode: "0644"
-  notify: Restart sml2mqtt
-
- name: Pull sml2mqtt image
-  ansible.builtin.command:
-    cmd: docker compose pull
-    chdir: "{{ sml_base_dir }}"
-  changed_when: false
-
-# Tolerates a missing USB device, as for the Z-Wave gateway.
- name: Start sml2mqtt
-  ansible.builtin.command:
-    cmd: docker compose up -d
-    chdir: "{{ sml_base_dir }}"
-  register: sml_compose_up
-  failed_when: >-
-    sml_compose_up.rc != 0
-    and 'error gathering device information' not in sml_compose_up.stderr
-  changed_when: >-
-    'Started' in sml_compose_up.stderr or 'Created' in sml_compose_up.stderr
-    or 'Started' in sml_compose_up.stdout or 'Creating' in sml_compose_up.stdout
-
- name: Warn if sml2mqtt is waiting for its USB device
-  ansible.builtin.debug:
-    msg: >-
-      sml2mqtt could not start -- the FTDI meter head is not present:
-      {{ sml_compose_up.stderr_lines | select('search', 'device') | list }}
-  when: sml_compose_up.rc != 0
@@ -1,21 +0,0 @@
-# Managed by ansible (roles/power-meter-sml).
-#
-# Its own compose project -- see the role tasks for why it no longer shares a
-# file with the Z-Wave gateway.
-#
-# Outbound-only (serial in, MQTT out). Host networking sidesteps the docker
-# NAT66 gap: the broker LB is IPv6-first and bridge-network v6 SYNs leave
-# unmasqueraded and die. The host stack reaches it fine (open3e proves it
-# continuously).
-services:
-  sml2mqtt:
-    image: {{ edge_sml2mqtt_image }}
-    container_name: sml2mqtt
-    restart: unless-stopped
-    network_mode: host
-    # Serial exclusivity: this is the ONLY reader of the meter port --
-    # HA's edl21 integration stays disabled.
-    devices:
-      - "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
-    volumes:
-      - ./config.yml:/sml2mqtt/config.yml:ro
@@ -0,0 +1,48 @@
+---
+# Prometheus Server Configuration
+
+# Version and download
+prometheus_version: "3.2.1"
+prometheus_arch: "linux-amd64"
+prometheus_download_url: "https://github.com/prometheus/prometheus/releases/download/v{{ prometheus_version }}/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}.tar.gz"
+prometheus_bin_path: "/usr/local/bin/prometheus"
+prometheus_promtool_path: "/usr/local/bin/promtool"
+
+# Directories
+prometheus_data_dir: "/var/lib/prometheus"
+prometheus_config_dir: "/etc/prometheus"
+prometheus_config_file: "{{ prometheus_config_dir }}/prometheus.yml"
+prometheus_rules_dir: "{{ prometheus_config_dir }}/rules"
+prometheus_targets_dir: "{{ prometheus_config_dir }}/targets"
+
+# Server configuration
+prometheus_listen_address: "0.0.0.0"
+prometheus_listen_port: 9090
+prometheus_web_external_url: ""
+
+# Retention settings (90 days)
+prometheus_retention_time: "90d"
+prometheus_retention_size: "30GB"
+
+# Scrape settings
+prometheus_scrape_interval: "30s"
+prometheus_evaluation_interval: "30s"
+prometheus_scrape_timeout: "10s"
+
+# Remote write (for Alloy agents)
+prometheus_remote_write_enabled: true
+
+# Systemd service
+prometheus_user: "prometheus"
+prometheus_group: "prometheus"
+
+# Alert notification endpoints
+prometheus_alertmanager_url: ""
+
+# Email alerting (via Grafana)
+alerting_email_enabled: true
+alerting_email_to: ""
+
+# Home Assistant webhook for alerts
+alerting_ha_webhook_enabled: true
+alerting_ha_webhook_url: ""
@@ -0,0 +1,15 @@
+---
+- name: Restart prometheus
+  systemd:
+    name: prometheus
+    state: restarted
+    daemon_reload: yes
+
+- name: Reload prometheus
+  systemd:
+    name: prometheus
+    state: reloaded
+
+- name: Reload systemd
+  systemd:
+    daemon_reload: yes
@@ -0,0 +1,129 @@
+---
+# Prometheus Server Installation and Configuration
+
+- name: Install dependencies
+  apt:
+    name:
+      - curl
+    state: present
+    update_cache: yes
+
+- name: Create prometheus user
+  user:
+    name: "{{ prometheus_user }}"
+    system: yes
+    shell: /usr/sbin/nologin
+    create_home: no
+
+- name: Create directories
+  file:
+    path: "{{ item }}"
+    state: directory
+    owner: "{{ prometheus_user }}"
+    group: "{{ prometheus_group }}"
+    mode: '0755'
+  loop:
+    - "{{ prometheus_config_dir }}"
+    - "{{ prometheus_data_dir }}"
+    - "{{ prometheus_rules_dir }}"
+    - "{{ prometheus_targets_dir }}"
+
+- name: Download Prometheus
+  get_url:
+    url: "{{ prometheus_download_url }}"
+    dest: "/tmp/prometheus.tar.gz"
+    mode: '0644'
+
+- name: Extract Prometheus
+  unarchive:
+    src: "/tmp/prometheus.tar.gz"
+    dest: "/tmp"
+    remote_src: yes
+
+- name: Install Prometheus binaries
+  copy:
+    src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item.src }}"
+    dest: "{{ item.dest }}"
+    mode: '0755'
+    remote_src: yes
+  loop:
+    - { src: "prometheus", dest: "{{ prometheus_bin_path }}" }
+    - { src: "promtool", dest: "{{ prometheus_promtool_path }}" }
+  notify: Restart prometheus
+
+- name: Check for console templates
+  stat:
+    path: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/consoles"
+  register: consoles_dir
+
+- name: Copy console templates
+  copy:
+    src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item }}/"
+    dest: "{{ prometheus_config_dir }}/{{ item }}/"
+    remote_src: yes
+    owner: "{{ prometheus_user }}"
+    group: "{{ prometheus_group }}"
+  loop:
+    - consoles
+    - console_libraries
+  when: consoles_dir.stat.exists
+
+- name: Create Prometheus configuration
+  template:
+    src: prometheus.yml.j2
+    dest: "{{ prometheus_config_file }}"
+    owner: "{{ prometheus_user }}"
+    group: "{{ prometheus_group }}"
+    mode: '0644'
+    validate: "{{ prometheus_promtool_path }} check config %s"
+  notify: Restart prometheus
+
+- name: Create alert rules
+  template:
+    src: alert-rules.yml.j2
+    dest: "{{ prometheus_rules_dir }}/alert-rules.yml"
+    owner: "{{ prometheus_user }}"
+    group: "{{ prometheus_group }}"
+    mode: '0644'
+  notify: Reload prometheus
+
+- name: Create node exporter targets file
+  template:
+    src: nodes.yml.j2
+    dest: "{{ prometheus_targets_dir }}/nodes.yml"
+    owner: "{{ prometheus_user }}"
+    group: "{{ prometheus_group }}"
+    mode: '0644'
+
+- name: Create Prometheus systemd service
+  template:
+    src: prometheus.service.j2
+    dest: /etc/systemd/system/prometheus.service
+    mode: '0644'
+  notify:
+    - Reload systemd
+    - Restart prometheus
+
+- name: Enable and start Prometheus service
+  systemd:
+    name: prometheus
+    state: started
+    enabled: yes
+    daemon_reload: yes
+
+- name: Wait for Prometheus to be ready
+  uri:
+    url: "http://127.0.0.1:{{ prometheus_listen_port }}/-/ready"
+    status_code: 200
+  register: result
+  until: result.status == 200
+  retries: 30
+  delay: 5
+
+- name: Clean up downloaded files
+  file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "/tmp/prometheus.tar.gz"
+    - "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}"
@@ -0,0 +1,100 @@
+# Prometheus Alert Rules
+# Generated by Ansible
+
+groups:
+  - name: infrastructure
+    rules:
+      # Disk usage alerts
+      - alert: HighDiskUsage
+        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 15
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
+          description: "Disk usage is above 85% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
+
+      - alert: CriticalDiskUsage
+        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 5
+        for: 2m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Critical disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
+          description: "Disk usage is above 95% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
+
+      # Memory alerts
+      - alert: HighMemoryUsage
+        expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 > 90
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High memory usage on {{ '{{' }} $labels.instance {{ '}}' }}"
+          description: "Memory usage is above 90% ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
+
+      # CPU alerts
+      - alert: HighCPUUsage
+        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High CPU usage on {{ '{{' }} $labels.instance {{ '}}' }}"
+          description: "CPU usage is above 85% for 10 minutes ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
+
+  - name: services
+    rules:
+      # Service down alerts
+      - alert: ServiceDown
+        expr: up == 0
+        for: 2m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Service {{ '{{' }} $labels.job {{ '}}' }} down on {{ '{{' }} $labels.instance {{ '}}' }}"
+          description: "Service has been unreachable for more than 2 minutes"
+
+      # DNS VIP health
+      - alert: DNSVIPDown
+        expr: absent(up{job="dns-servers"} == 1) or count(up{job="dns-servers"} == 1) < 2
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: "DNS cluster degraded"
+          description: "Less than 2 DNS servers are healthy"
+
+  - name: security
+    rules:
+      # High auth failure rate (placeholder - needs log-based metrics from Loki)
+      - alert: HighAuthFailureRate
+        expr: rate(authentik_login_failed_total[5m]) > 5
+        for: 2m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High authentication failure rate"
+          description: "More than 5 auth failures per minute detected"
+
+  - name: loki
+    rules:
+      # Loki health
+      - alert: LokiDown
+        expr: up{job="loki"} == 0
+        for: 2m
+        labels:
+          severity: critical
+        annotations:
+          summary: "Loki log aggregation is down"
+          description: "Loki has been unreachable for more than 2 minutes"
+
+      # Loki ingestion rate
+      - alert: LokiHighIngestionRate
+        expr: rate(loki_distributor_bytes_received_total[5m]) > 10485760
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "High log ingestion rate in Loki"
+          description: "Log ingestion rate is above 10MB/s"
@@ -0,0 +1,26 @@
+# Node Exporter targets - auto-generated by Ansible
+# Updates require re-running the prometheus playbook
+#
+# NOTE: Most hosts have Alloy agents that push metrics via remote_write.
+# This file only includes hosts that need direct scraping (standalone node_exporter).
+# Alloy-managed hosts are excluded to avoid duplicate "down" targets.
+
+# PVE cluster nodes - need direct scraping (no Alloy agent)
+- targets:
+    - '10.2.40.201:9100'
+  labels:
+    job: 'node'
+    group: 'pve_cluster'
+    hostname: 'jupiter'
+- targets:
+    - '10.2.40.202:9100'
+  labels:
+    job: 'node'
+    group: 'pve_cluster'
+    hostname: 'neptune'
+- targets:
+    - '10.2.40.203:9100'
+  labels:
+    job: 'node'
+    group: 'pve_cluster'
+    hostname: 'saturn'
@@ -0,0 +1,38 @@
+[Unit]
+Description=Prometheus Monitoring System
+Documentation=https://prometheus.io/docs/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User={{ prometheus_user }}
+Group={{ prometheus_group }}
+ExecStart={{ prometheus_bin_path }} \
+  --config.file={{ prometheus_config_file }} \
+  --storage.tsdb.path={{ prometheus_data_dir }} \
+  --storage.tsdb.retention.time={{ prometheus_retention_time }} \
+  --storage.tsdb.retention.size={{ prometheus_retention_size }} \
+  --web.listen-address={{ prometheus_listen_address }}:{{ prometheus_listen_port }} \
+{% if prometheus_remote_write_enabled %}
+  --web.enable-remote-write-receiver \
+{% endif %}
+{% if prometheus_web_external_url %}
+  --web.external-url={{ prometheus_web_external_url }} \
+{% endif %}
+  --web.enable-lifecycle
+
+ExecReload=/bin/kill -HUP $MAINPID
+Restart=on-failure
+RestartSec=10s
+LimitNOFILE=65536
+
+# Security hardening
+NoNewPrivileges=yes
+ProtectSystem=strict
+ProtectHome=yes
+ReadWritePaths={{ prometheus_data_dir }}
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,303 @@
+# Prometheus Configuration
+# Generated by Ansible
+
+global:
+  scrape_interval: {{ prometheus_scrape_interval }}
+  evaluation_interval: {{ prometheus_evaluation_interval }}
+  scrape_timeout: {{ prometheus_scrape_timeout }}
+  external_labels:
+    cluster: homelab
+    datacenter: renner
+
+rule_files:
+  - "{{ prometheus_rules_dir }}/*.yml"
+
+{% if prometheus_alertmanager_url %}
+alerting:
+  alertmanagers:
+    - static_configs:
+        - targets:
+            - {{ prometheus_alertmanager_url }}
+{% endif %}
+
+scrape_configs:
+  # Prometheus self-monitoring
+  - job_name: 'prometheus'
+    static_configs:
+      - targets: ['localhost:{{ prometheus_listen_port }}']
+
+  # Loki metrics
+  - job_name: 'loki'
+    static_configs:
+      - targets: ['loki.srv.renner.uno:3100']
+
+  # File-based service discovery for dynamic targets
+  - job_name: 'alloy-nodes'
+    file_sd_configs:
+      - files:
+          - '{{ prometheus_targets_dir }}/*.yml'
+          - '{{ prometheus_targets_dir }}/*.json'
+        refresh_interval: 30s
+
+{% if prometheus_remote_write_enabled %}
+# Remote write receiver for Alloy agents
+# Agents push metrics to /api/v1/write
+{% endif %}
+
+# Static targets for infrastructure components
+  # Proxmox VE API metrics (via pve-exporter)
+  - job_name: 'pve'
+    scrape_interval: 60s
+    scrape_timeout: 55s
+    static_configs:
+      - targets:
+          - '10.2.40.201'  # jupiter
+          - '10.2.40.202'  # neptune
+          - '10.2.40.203'  # saturn
+    metrics_path: /pve
+    params:
+      module: [default]
+      cluster: ['1']
+      node: ['1']
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9221  # pve-exporter address
+      - target_label: group
+        replacement: proxmox
+      - target_label: type
+        replacement: hypervisor
+
+  # RustFS (S3-compatible storage)
+  # NOTE: RustFS uses OpenTelemetry push-model via --obs-endpoint, not Prometheus pull.
+  # Unlike MinIO, it doesn't expose /minio/v2/metrics/cluster endpoint.
+  # Metrics collection would require setting up RUSTFS_OBS_ENDPOINT to push to an OTEL collector.
+  # For now, node-level metrics are available via Alloy agents on the RustFS hosts.
+
+  # Home Assistant metrics
+  - job_name: 'homeassistant'
+    metrics_path: /api/prometheus
+    bearer_token: '{{ homeassistant_prometheus_token }}'
+    scheme: http
+    static_configs:
+      - targets: ['homeassistant.iot.renner.uno:8123']
+        labels:
+          group: 'smarthome'
+          type: 'homeassistant'
+
+  # UniFi Poller metrics
+  - job_name: 'unifi'
+    static_configs:
+      - targets: ['localhost:9130']
+        labels:
+          group: 'network'
+          type: 'unifi'
+
+  # Caddy reverse proxy metrics (HA cluster)
+  - job_name: 'caddy'
+    static_configs:
+      - targets:
+          - 'caddy.dmz.renner.uno:2019'
+          - 'caddy2.dmz.renner.uno:2019'
+        labels:
+          group: 'ingress'
+          type: 'reverse-proxy'
+
+  # PostgreSQL metrics (via postgres_exporter)
+  - job_name: 'postgresql'
+    static_configs:
+      - targets: ['localhost:9187']
+        labels:
+          group: 'database'
+          type: 'postgresql'
+
+  # Authentik identity provider metrics
+  - job_name: 'authentik'
+    static_configs:
+      - targets:
+          - 'authentik-app-1.srv.renner.uno:9300'
+          - 'authentik-app-2.srv.renner.uno:9300'
+        labels:
+          group: 'identity'
+          type: 'authentik'
+
+  # Node Exporter - system metrics from all hosts
+  - job_name: 'node'
+    file_sd_configs:
+      - files:
+          - '{{ prometheus_targets_dir }}/nodes.yml'
+        refresh_interval: 60s
+
+  # Chrony Exporter - NTP metrics from PVE cluster
+  - job_name: 'chrony'
+    static_configs:
+      - targets:
+          - '10.2.40.201:9123'
+          - '10.2.40.202:9123'
+          - '10.2.40.203:9123'
+        labels:
+          group: 'ntp'
+          type: 'time-server'
+    relabel_configs:
+      - source_labels: [__address__]
+        regex: '10\.2\.40\.201:.*'
+        target_label: hostname
+        replacement: 'jupiter'
+      - source_labels: [__address__]
+        regex: '10\.2\.40\.202:.*'
+        target_label: hostname
+        replacement: 'neptune'
+      - source_labels: [__address__]
+        regex: '10\.2\.40\.203:.*'
+        target_label: hostname
+        replacement: 'saturn'
+
+  # Immich photo management metrics
+  - job_name: 'immich'
+    static_configs:
+      - targets: ['immich.srv.renner.uno:8081']
+        labels:
+          group: 'media'
+          type: 'photo-management'
+
+  # Blackbox Exporter - HTTP endpoint probing
+  - job_name: 'blackbox-http'
+    metrics_path: /probe
+    params:
+      module: [http_2xx]
+    static_configs:
+      - targets:
+          - https://gitea.renner.uno
+          - https://immich.renner.uno
+          - https://auth.renner.uno
+          - https://ha.renner.uno
+          - https://grafana.renner.uno
+          - https://prometheus.renner.uno
+        labels:
+          group: 'probe'
+          type: 'http'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9115
+
+  # Blackbox Exporter - TCP probing
+  - job_name: 'blackbox-tcp'
+    metrics_path: /probe
+    params:
+      module: [tcp_connect]
+    static_configs:
+      - targets:
+          - 10.2.40.135:5432  # PostgreSQL HA VIP
+        labels:
+          group: 'probe'
+          type: 'tcp'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9115
+
+  # Blackbox Exporter - DNS probing (external domains like google.com)
+  - job_name: 'blackbox-dns-external'
+    metrics_path: /probe
+    params:
+      module: [dns_external]
+    static_configs:
+      - targets:
+          - 10.2.40.53  # Proxmox DNS VIP
+          - 10.2.40.1   # UDM Gateway DNS
+        labels:
+          group: 'probe'
+          type: 'dns'
+          query_type: 'external'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9115
+
+  # Blackbox Exporter - DNS probing (internal domains like k8s-1.srv.renner.uno)
+  - job_name: 'blackbox-dns-internal'
+    metrics_path: /probe
+    params:
+      module: [dns_internal]
+    static_configs:
+      - targets:
+          - 10.2.40.53  # Proxmox DNS VIP
+          - 10.2.40.1   # UDM Gateway DNS
+        labels:
+          group: 'probe'
+          type: 'dns'
+          query_type: 'internal'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9115
+
+  # Blackbox Exporter - DNS probing (cached domains)
+  - job_name: 'blackbox-dns-cached'
+    metrics_path: /probe
+    params:
+      module: [dns_cached]
+    static_configs:
+      - targets:
+          - 10.2.40.53  # Proxmox DNS VIP
+          - 10.2.40.1   # UDM Gateway DNS
+        labels:
+          group: 'probe'
+          type: 'dns'
+          query_type: 'cached'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9115
+
+{% if snmp_synology_targets is defined and snmp_synology_targets | length > 0 %}
+  # Synology NAS SNMP metrics
+  # snmp_exporter v0.27+ requires both synology and ucd_la_table modules
+  - job_name: 'synology'
+    scrape_interval: 60s
+    scrape_timeout: 55s
+    metrics_path: /snmp
+    params:
+      module: [synology, ucd_la_table, ucd_memory, if_mib]
+{% if snmp_use_v3 | default(false) %}
+      auth: [synology_v3]
+{% else %}
+      auth: [synology_v2]
+{% endif %}
+    static_configs:
+      - targets:
+{% for target in snmp_synology_targets %}
+          - '{{ target.target }}'  # {{ target.name }}
+{% endfor %}
+        labels:
+          group: 'storage'
+          type: 'nas'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__address__]
+        regex: '([^:]+)(:\d+)?'
+        replacement: '${1}'
+        target_label: instance
+      - target_label: __address__
+        replacement: localhost:9116  # snmp_exporter address
+{% endif %}
@@ -0,0 +1,36 @@
+---
+# Promtail Configuration
+
+# Version and download
+promtail_version: "3.4.2"
+promtail_arch: "linux-amd64"
+promtail_download_url: "https://github.com/grafana/loki/releases/download/v{{ promtail_version }}/promtail-{{ promtail_arch }}.zip"
+promtail_bin_path: "/usr/local/bin/promtail"
+
+# Directories
+promtail_config_dir: "/etc/promtail"
+promtail_config_file: "{{ promtail_config_dir }}/promtail-config.yaml"
+promtail_positions_file: "/var/lib/promtail/positions.yaml"
+
+# Server configuration
+promtail_http_listen_port: 9080
+promtail_http_listen_address: "0.0.0.0"
+
+# Loki server
+promtail_loki_url: "http://loki.srv.renner.uno:3100/loki/api/v1/push"
+
+# Log scrape configurations
+# Override in playbook or host_vars for specific services
+promtail_scrape_configs:
+  - job_name: system
+    static_configs:
+      - targets:
+          - localhost
+        labels:
+          job: syslog
+          host: "{{ inventory_hostname }}"
+          __path__: /var/log/syslog
+
+# Systemd service
+promtail_user: "promtail"
+promtail_group: "promtail"
@@ -0,0 +1,6 @@
+---
+- name: Restart promtail
+  systemd:
+    name: promtail
+    state: restarted
+    daemon_reload: yes
@@ -0,0 +1,112 @@
+---
+- name: Install dependencies
+  apt:
+    name:
+      - unzip
+      - curl
+    state: present
+    update_cache: yes
+
+- name: Create promtail group
+  group:
+    name: "{{ promtail_group }}"
+    system: yes
+    state: present
+
+- name: Create promtail user
+  user:
+    name: "{{ promtail_user }}"
+    group: "{{ promtail_group }}"
+    system: yes
+    shell: /usr/sbin/nologin
+    home: /var/lib/promtail
+    create_home: yes
+    state: present
+
+- name: Add promtail user to systemd-journal group
+  user:
+    name: "{{ promtail_user }}"
+    groups: systemd-journal
+    append: yes
+
+- name: Create config directory
+  file:
+    path: "{{ promtail_config_dir }}"
+    state: directory
+    owner: "{{ promtail_user }}"
+    group: "{{ promtail_group }}"
+    mode: '0755'
+
+- name: Create positions directory
+  file:
+    path: "{{ promtail_positions_file | dirname }}"
+    state: directory
+    owner: "{{ promtail_user }}"
+    group: "{{ promtail_group }}"
+    mode: '0755'
+
+- name: Check if promtail binary exists
+  stat:
+    path: "{{ promtail_bin_path }}"
+  register: promtail_binary
+
+- name: Get installed promtail version
+  command: "{{ promtail_bin_path }} --version"
+  register: installed_version
+  changed_when: false
+  failed_when: false
+  when: promtail_binary.stat.exists
+
+- name: Download promtail
+  get_url:
+    url: "{{ promtail_download_url }}"
+    dest: /tmp/promtail.zip
+    mode: '0644'
+  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
+
+- name: Extract promtail binary
+  unarchive:
+    src: /tmp/promtail.zip
+    dest: /tmp/
+    remote_src: yes
+  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
+
+- name: Install promtail binary
+  copy:
+    src: "/tmp/promtail-{{ promtail_arch }}"
+    dest: "{{ promtail_bin_path }}"
+    remote_src: yes
+    mode: '0755'
+  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
+  notify: Restart promtail
+
+- name: Clean up downloaded files
+  file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - /tmp/promtail.zip
+    - "/tmp/promtail-{{ promtail_arch }}"
+
+- name: Configure promtail
+  template:
+    src: promtail-config.yaml.j2
+    dest: "{{ promtail_config_file }}"
+    owner: "{{ promtail_user }}"
+    group: "{{ promtail_group }}"
+    mode: '0644'
+  notify: Restart promtail
+
+- name: Create systemd service
+  template:
+    src: promtail.service.j2
+    dest: /etc/systemd/system/promtail.service
+    mode: '0644'
+  notify: Restart promtail
+
+- name: Enable and start promtail
+  systemd:
+    name: promtail
+    state: started
+    enabled: yes
+    daemon_reload: yes
@@ -0,0 +1,79 @@
+server:
+  http_listen_port: {{ promtail_http_listen_port }}
+  http_listen_address: {{ promtail_http_listen_address }}
+  grpc_listen_port: 0
+
+positions:
+  filename: {{ promtail_positions_file }}
+
+clients:
+  - url: {{ promtail_loki_url }}
+    tenant_id: default
+
+scrape_configs:
+{% for config in promtail_scrape_configs %}
+  - job_name: {{ config.job_name }}
+{% if config.journal is defined %}
+    journal:
+      max_age: {{ config.journal.max_age | default('12h') }}
+      labels:
+        job: {{ config.job_name }}
+        host: {{ inventory_hostname }}
+{% if config.journal.matches is defined %}
+      matches: {{ config.journal.matches }}
+{% endif %}
+{% if config.relabel_configs is defined %}
+    relabel_configs:
+{% for relabel in config.relabel_configs %}
+      - source_labels: {{ relabel.source_labels | to_json }}
+        target_label: {{ relabel.target_label }}
+{% if relabel.regex is defined %}
+        regex: '{{ relabel.regex }}'
+{% endif %}
+{% if relabel.replacement is defined %}
+        replacement: '{{ relabel.replacement }}'
+{% endif %}
+{% endfor %}
+{% endif %}
+{% endif %}
+{% if config.static_configs is defined %}
+    static_configs:
+{% for static in config.static_configs %}
+      - targets:
+{% for target in static.targets %}
+          - {{ target }}
+{% endfor %}
+        labels:
+{% for key, value in static.labels.items() %}
+          {{ key }}: {{ value }}
+{% endfor %}
+{% endfor %}
+{% if config.pipeline_stages is defined %}
+    pipeline_stages:
+{% for stage in config.pipeline_stages %}
+{% if stage.multiline is defined %}
+      - multiline:
+          firstline: '{{ stage.multiline.firstline }}'
+{% if stage.multiline.max_wait_time is defined %}
+          max_wait_time: {{ stage.multiline.max_wait_time }}
+{% endif %}
+{% endif %}
+{% if stage.regex is defined %}
+      - regex:
+          expression: '{{ stage.regex.expression }}'
+{% endif %}
+{% if stage.labels is defined %}
+      - labels:
+{% for label in stage.labels %}
+          {{ label }}:
+{% endfor %}
+{% endif %}
+{% if stage.timestamp is defined %}
+      - timestamp:
+          source: {{ stage.timestamp.source }}
+          format: '{{ stage.timestamp.format }}'
+{% endif %}
+{% endfor %}
+{% endif %}
+{% endif %}
+{% endfor %}
@@ -0,0 +1,24 @@
+[Unit]
+Description=Promtail Log Collector
+Documentation=https://grafana.com/docs/loki/latest/clients/promtail/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User={{ promtail_user }}
+Group={{ promtail_group }}
+ExecStart={{ promtail_bin_path }} -config.file={{ promtail_config_file }}
+Restart=always
+RestartSec=5
+TimeoutStopSec=30
+
+# Security hardening
+NoNewPrivileges=yes
+ProtectSystem=strict
+ProtectHome=yes
+ReadWritePaths=/var/lib/promtail
+CapabilityBoundingSet=CAP_DAC_READ_SEARCH
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,19 @@
+---
+# PVE Exporter defaults
+
+# Installation
+pve_exporter_version: "3.4.5"
+pve_exporter_install_dir: /opt/pve-exporter
+pve_exporter_config_dir: /etc/pve-exporter
+pve_exporter_venv_path: "{{ pve_exporter_install_dir }}/venv"
+
+# Service
+pve_exporter_user: pve-exporter
+pve_exporter_group: pve-exporter
+pve_exporter_port: 9221
+
+# Proxmox connection (override with vault variables)
+pve_exporter_pve_user: "{{ pve_user | default('root@pam') }}"
+pve_exporter_pve_token_name: "{{ pve_token_id | default('ansible') }}"
+pve_exporter_pve_token_value: "{{ pve_token_secret }}"
+pve_exporter_pve_verify_ssl: false
@@ -0,0 +1,6 @@
+---
+- name: Restart pve-exporter
+  systemd:
+    name: pve-exporter
+    state: restarted
+    daemon_reload: yes
@@ -0,0 +1,78 @@
+---
+# PVE Exporter installation and configuration
+
+- name: Install Python dependencies
+  apt:
+    name:
+      - python3
+      - python3-pip
+      - python3-venv
+    state: present
+    update_cache: yes
+
+- name: Create pve-exporter system user
+  user:
+    name: "{{ pve_exporter_user }}"
+    system: yes
+    shell: /usr/sbin/nologin
+    create_home: no
+
+- name: Create installation directory
+  file:
+    path: "{{ pve_exporter_install_dir }}"
+    state: directory
+    owner: "{{ pve_exporter_user }}"
+    group: "{{ pve_exporter_group }}"
+    mode: '0755'
+
+- name: Create configuration directory
+  file:
+    path: "{{ pve_exporter_config_dir }}"
+    state: directory
+    owner: "{{ pve_exporter_user }}"
+    group: "{{ pve_exporter_group }}"
+    mode: '0750'
+
+- name: Create Python virtual environment
+  command: python3 -m venv {{ pve_exporter_venv_path }}
+  args:
+    creates: "{{ pve_exporter_venv_path }}/bin/python"
+
+- name: Set venv ownership
+  file:
+    path: "{{ pve_exporter_venv_path }}"
+    state: directory
+    owner: "{{ pve_exporter_user }}"
+    group: "{{ pve_exporter_group }}"
+    recurse: yes
+
+- name: Install prometheus-pve-exporter
+  pip:
+    name: prometheus-pve-exporter
+    virtualenv: "{{ pve_exporter_venv_path }}"
+    state: present
+
+- name: Create configuration file
+  template:
+    src: pve.yml.j2
+    dest: "{{ pve_exporter_config_dir }}/pve.yml"
+    owner: "{{ pve_exporter_user }}"
+    group: "{{ pve_exporter_group }}"
+    mode: '0640'
+  notify: Restart pve-exporter
+
+- name: Create systemd service
+  template:
+    src: pve-exporter.service.j2
+    dest: /etc/systemd/system/pve-exporter.service
+    owner: root
+    group: root
+    mode: '0644'
+  notify: Restart pve-exporter
+
+- name: Enable and start pve-exporter service
+  systemd:
+    name: pve-exporter
+    state: started
+    enabled: yes
+    daemon_reload: yes
@@ -0,0 +1,22 @@
+[Unit]
+Description=Prometheus PVE Exporter
+Documentation=https://github.com/prometheus-pve/prometheus-pve-exporter
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User={{ pve_exporter_user }}
+Group={{ pve_exporter_group }}
+ExecStart={{ pve_exporter_venv_path }}/bin/pve_exporter \
+  --config.file {{ pve_exporter_config_dir }}/pve.yml \
+  --web.listen-address 0.0.0.0:{{ pve_exporter_port }}
+Restart=on-failure
+RestartSec=10s
+
+# Security hardening
+NoNewPrivileges=yes
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,8 @@
+# PVE Exporter Configuration
+# Generated by Ansible
+
+default:
+  user: "{{ pve_exporter_pve_user }}"
+  token_name: "{{ pve_exporter_pve_token_name }}"
+  token_value: "{{ pve_exporter_pve_token_value }}"
+  verify_ssl: {{ pve_exporter_pve_verify_ssl | lower }}
@@ -0,0 +1,33 @@
+---
+# SNMP Exporter defaults
+
+snmp_exporter_version: "0.30.1"
+snmp_exporter_listen_port: 9116
+snmp_exporter_user: "snmp_exporter"
+snmp_exporter_group: "snmp_exporter"
+
+snmp_exporter_install_dir: "/opt/snmp_exporter"
+snmp_exporter_config_dir: "/etc/snmp_exporter"
+
+# Synology NAS SNMP targets
+# Configure in host_vars or group_vars
+snmp_synology_targets: []
+# Example:
+# snmp_synology_targets:
+#   - name: nas-1
+#     target: 10.2.40.50
+#   - name: nas-2
+#     target: 10.2.40.51
+
+# SNMPv3 authentication (recommended for security)
+# These should be set in vault
+snmp_v3_username: "prometheus"
+snmp_v3_auth_protocol: "SHA"  # MD5 or SHA
+snmp_v3_auth_password: ""     # Set in vault
+snmp_v3_priv_protocol: "AES"  # DES or AES
+snmp_v3_priv_password: ""     # Set in vault
+snmp_v3_security_level: "authPriv"  # noAuthNoPriv, authNoPriv, authPriv
+
+# SNMPv2c fallback (less secure, but simpler)
+snmp_v2_community: "public"
+snmp_use_v3: true
--- a/Show More
+++ b/Show More