Compare commits

..
Author SHA1 Message Date
claude-botandClaude Opus 5 2c5c416787 fix(chrony): step the clock at boot so a bad RTC self-corrects
chrony.conf.j2 had no makestep directive, so chronyd only ever slewed.
A large offset is therefore never closed in practice.

edge-1 hit this on 2026-08-04. Its CMOS battery is dead, so after a power
loss the RTC came up reading 2018-08-20. The kernel set the system clock
from it:

  PM: RTC time: 09:18:56, date: 2018-08-20
  rtc_cmos 00:01: setting system clock to 2018-08-20T09:18:58 UTC

systemd floored that forward to its own build date:

  systemd[1]: System time advanced to built-in epoch: Fri 2026-06-26 11:27:23 UTC

and it ended up at ~2026-07-19, roughly 16 days behind. chronyd then
synced to ptbtime1 at stratum 2 with a sub-millisecond RMS offset, and
still reported "System time: 1376235.375 seconds slow of NTP time" while
correcting none of it.

With host time before the cluster certificate's NotBefore, every outbound
TLS handshake failed x509 validation:

  tls: failed to verify certificate: x509: certificate has expired or is
  not yet valid: current time 2026-07-20T16:15:36+02:00 is before
  2026-07-23T06:15:01Z

That took out the local mosquitto -> cluster MQTT bridge (all Vitocal
250-A / Vitocharge VX3 / E380 heating telemetry went unavailable in Home
Assistant for ~18h) and the Alloy -> Loki log shipper simultaneously.
Because the same skew broke log shipping, the outage was invisible to
log-based alerting.

The 16 days is incidental - it is just where the software flooring landed.
The fault this commit addresses is that nothing steps. Recovery required a
manual `chronyc makestep`; the directive lets it self-correct at boot.

The limit keeps steps confined to the first few updates after chronyd
starts, so a long-running NTP server is never jumped out from under its
workloads mid-flight.

Note this does not fix edge-1's dead RTC battery, which is hardware.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 17:28:41 +02:00
84 changed files with 846 additions and 1975 deletions
-29
View File
@@ -217,32 +217,3 @@ steps:
--dockerfile Containerfile \
--destination gitea.renner.uno/gitea_admin/alpine-openssl:$TAG \
--destination gitea.renner.uno/gitea_admin/alpine-openssl:latest
# Heating CAN readers: open3e (Vitocal/VX3) + E3onCAN (E380 meter), one image
# run as two services on edge-1. Replaces pip-into-venv on the host, where an
# unpinned VCS requirement re-cloned and restarted the heating bridge on every
# Ansible converge. Not a ci-* image -- it is a runtime artifact -- but it
# builds the same way and lands in the same registry so the audit sees it.
- name: build-heating-can
image: gcr.io/kaniko-project/executor:v1.23.2-debug
directory: ci/images/heating-can
depends_on: []
when:
- event: manual
- event: push
path: ["ci/images/heating-can/**", ".woodpecker/build-ci-images.yml"]
environment:
GITEA_USER: gitea_admin
GITEA_TOKEN:
from_secret: ci_images_token
commands:
- mkdir -p /kaniko/.docker
- AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
- printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
- TAG=$(date +%Y-%m-%d)
- |
/kaniko/executor \
--context . \
--dockerfile Containerfile \
--destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
--destination gitea.renner.uno/gitea_admin/heating-can:latest
@@ -1,6 +1,6 @@
---
# Secret indirection — real values live in vault.yml (ansible-vault) next
# to this file; see docs/edge-host.md for how to populate it from
# to this file; see roles/edge-host/README.md for how to populate it from
# the cluster secrets.
edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
@@ -36,18 +36,3 @@ keepalived_ntp_sysctl:
# Realtek NIC on the Wyse 3040 has no hardware timestamping.
chrony_hwtimestamp: false
# --- Local MQTT contract -------------------------------------------------
# Shared between the broker and its clients, so it belongs to the inventory
# rather than to either side's role. mqtt-bridge listens here; heating-can and
# power-meter-sml publish here; and the bridge authenticates to the cluster
# broker AS the open3e user, so it needs that identity too.
#
# Keeping these in one role's defaults only worked while a single play loaded
# every role at once. Splitting the plays by capability exposed it: the bridge
# could not render without a heating-can default, and the CAN services could
# not render without an mqtt-bridge default.
edge_mqtt_host: 127.0.0.1
edge_mqtt_port: 1883
edge_open3e_mqtt_user: open3e
edge_open3e_mqtt_topic: open3e
@@ -1,18 +0,0 @@
---
# Shared configuration for every NTP server (see plays/install-ntp-ha.yml).
# IPv6 service address -- what ntp.srv.renner.uno resolves to, and the path
# clients actually use. The IPv4 VIP is legacy.
#
# Defined at group level rather than in the play because install-ntp-ha.yml runs
# two plays over this group -- withdraw the static assignments fleet-wide, then
# hand the address to keepalived -- and both need the same value. Duplicating a
# service address across plays is exactly the sort of thing that drifts.
ntp_vip6: "fd40::123/128"
# Link-local address carried as the FIRST virtual_ipaddress of the IPv6 VRRP
# instance. RFC 5798 gives an IPv6 virtual router a link-local primary, and
# keepalived warns "the first IPv6 VIP address should be link local" without
# one. fe80::123 mirrors the service address and cannot collide with a real
# NIC's EUI-64 link-local.
ntp_vip6_linklocal: "fe80::123/64"
+3 -42
View File
@@ -1,19 +1,7 @@
# Reached by name, like every other host in this file. These three were the last
# entries still addressed by IPv4 literal; authentik, netbird, proxy, nas-1,
# grafana and edge-1 have always used their FQDN.
#
# An address in an inventory is allocator state standing in for identity, and it
# pins access to the legacy family -- the names resolve IPv6-first, which is the
# primary path on this network. DNS is the authoritative source for where a host
# is; this file should say which host, not where it currently sits.
#
# The short inventory_hostname is kept deliberately: plays key on it
# (ntp_servers[inventory_hostname] in install-ntp-ha.yml, node names in opentofu,
# the prometheus targets), so only ansible_host changes.
[pve_cluster]
jupiter ansible_host=jupiter.renner.uno ansible_user=root
neptune ansible_host=neptune.renner.uno ansible_user=root
saturn ansible_host=saturn.renner.uno ansible_user=root
jupiter ansible_host=10.2.40.201 ansible_user=root
neptune ansible_host=10.2.40.202 ansible_user=root
saturn ansible_host=10.2.40.203 ansible_user=root
[authentik_db]
authentik-db-1.srv.renner.uno ansible_user=root
@@ -41,36 +29,9 @@ nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
[grafana]
grafana.srv.renner.uno ansible_user=ladmin
# Every bare-metal edge thin client. Gets the host-generic roles only:
# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
# sub-interface. Nothing here assumes any peripheral is attached.
[edge_host]
edge-1.iot.renner.uno ansible_user=root
# Capability groups. Membership states which hardware a host physically has,
# and each service playbook targets the group for its own concern. No flags and
# no `when:` on the roles -- moving a stick to another box is a one-line change
# here, which is what it should be.
#
# These are children of [edge_host], declared below: a peripheral is added to an
# edge client, it does not replace the base. Making that a :children relation
# rather than a comment means a host added only to a capability group still
# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
# cannot silently come up without edge_mqtt_host and friends.
[zwave_gateway]
edge-1.iot.renner.uno
[heating_can]
edge-1.iot.renner.uno
[power_meter]
edge-1.iot.renner.uno
[edge_host:children]
zwave_gateway
heating_can
power_meter
# NTP servers. Includes edge-1 so the role survives the PVE hosts being
# retired for bare-metal Talos, which cannot run chrony or keepalived itself.
# See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly
+9 -37
View File
@@ -1,41 +1,13 @@
---
# Bare-metal Arch edge hosts (Dell Wyse 3040 and similar) — the home for
# hardware-bound services, because Talos supports neither SocketCAN nor USB
# passthrough.
#
# This play is the BASE only: what every edge thin client gets regardless of
# what is plugged into it. The peripherals live in their own playbooks against
# their own inventory groups:
#
# plays/install-zwave-gateway.yml [zwave_gateway] ZWA-2 radio
# plays/install-heating-can.yml [heating_can] Innomaker USB2CAN
# plays/install-power-meter.yml [power_meter] FTDI meter head
#
# Splitting the roles without splitting the plays would have kept the coupling:
# one play applying every role to one group is the monolith again, a layer up.
# A host with no peripherals is simply absent from those groups -- no flags, no
# `when:` on the roles.
#
# Self-contained: `certificates` and `common` are the parts of plays/common.yml
# that apply to an edge host, carried here so bootstrapping one is a single
# unrestricted run over the group. Not carried over: qemu-guest-agent (bare
# metal, not VMs) and monitoring-agent (Debian-oriented; these hosts ship logs
# via edge-log-shipper).
#
# Order is dependency order: edge-flash-wear before docker, so the journal is
# volatile before containers start logging into it.
- name: Install Arch edge host (base)
# Bare-metal Arch Linux edge host (Dell Wyse 3040, internal eMMC) — replaces
# the k8s-edge Talos VM
# (opentofu vm_153) as the home for hardware-bound services:
# - zwave-js-ui (ZWA-2 Z-Wave USB radio, docker compose)
# - sml2mqtt (FTDI power-meter reader, docker compose)
# - open3e (Innomaker USB2CAN -> Vitocal/VX3, native venv + systemd)
# Base install procedure and cutover checklist: ansible/roles/edge-host/README.md
- name: Install Arch edge host
hosts: edge_host
# Connection is already root (ansible_user=root), so this is a root-to-root
# no-op today. Kept explicit, matching plays/common.yml, so the roles below
# do not silently depend on the connection user staying privileged.
become: true
roles:
- certificates
- common
- edge-flash-wear
- docker-install
- edge-log-shipper
- mqtt-bridge
- edge-vlan
- edge-host
-18
View File
@@ -1,18 +0,0 @@
---
# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
# USB2CAN adapter attached.
#
# Targets [heating_can]: membership means the adapter is physically present.
# Two roles because they are two concerns -- the interface is host network
# state, the services consume it -- but one group, because both need the same
# hardware.
#
# Requires the base play (plays/install-edge-host.yml) to have run: these
# services publish to the local MQTT bridge it sets up.
- name: Install heating CAN services
hosts: heating_can
become: true
roles:
- socketcan
- heating-can
-87
View File
@@ -22,54 +22,12 @@
# so it stays BACKUP while the PVE nodes exist and wins the VIP by default once
# they are gone -- no config change required at cutover.
# Phase 1. Every static assignment of the IPv6 service address must be gone
# fleet-wide before ANY host claims it via VRRP.
#
# DAD is enabled on the VRRP address (deliberately -- under single-master VRRP a
# duplicate is a real fault worth detecting). So a static copy on one host and a
# VRRP-managed copy on another cannot coexist: the kernel marks the newcomer
# `dadfailed deprecated`, and its master then believes it is serving while
# carrying an unusable address. That is not hypothetical -- it is what edge-1
# did on 2026-08-07 when it took the address while three PVE nodes still held it
# statically.
#
# Deliberately NOT serial: this play must complete everywhere before phase 2
# starts anywhere. Doing the withdrawal per host inside phase 2 cannot avoid the
# overlap, because with serial: 1 the first host claims the address while the
# rest are still static.
#
# There is a gap between the phases where no host holds the address. It is
# bounded by one play's run time and clients simply retry.
- name: Withdraw static assignments of the NTP service address
hosts: ntp_cluster
become: true
vars:
keepalived_ntp_vip6: "{{ ntp_vip6 }}"
keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
tasks:
- name: Withdraw any static assignment of the service address
ansible.builtin.include_role:
name: keepalived-ntp
tasks_from: withdraw-static.yml
when: ntp_keepalived_enabled | default(false)
# Phase 2. Configure chrony and hand the address to keepalived.
- name: Install NTP Server (HA)
hosts: ntp_cluster
become: true
# One node at a time. The keepalived role withdraws the statically assigned
# IPv6 service address and restarts keepalived so the address is re-taken
# under VRRP control; doing that across every NTP server at once would leave
# the whole fleet with nobody holding fd40::123 until the elections settle.
# Serialised, the remaining members keep answering throughout.
serial: 1
vars:
ntp_vip: "10.2.40.4"
# ntp_vip6 / ntp_vip6_linklocal come from group_vars/ntp_cluster.yml, so
# both plays in this file see one definition.
ntp_servers:
jupiter:
priority: 100
@@ -91,8 +49,6 @@
when: ntp_keepalived_enabled | default(false)
vars:
keepalived_ntp_vip: "{{ ntp_vip }}"
keepalived_ntp_vip6: "{{ ntp_vip6 }}"
keepalived_ntp_vip6_linklocal: "{{ ntp_vip6_linklocal }}"
keepalived_ntp_state: "{{ ntp_servers[inventory_hostname].state }}"
keepalived_ntp_priority: "{{ ntp_servers[inventory_hostname].priority }}"
keepalived_ntp_interface: "{{ ntp_keepalived_interface }}"
@@ -134,49 +90,6 @@
{% endif %}
when: ntp_keepalived_enabled | default(false)
# The v6 address is the path clients actually use, so it gets the same
# check as the v4 VIP. Without this a broken v6 rollout -- the address
# never claimed, or -nodad not behaving as expected on a given kernel --
# would not surface here at all, only later by reading
# /var/log/keepalived-state.log by hand.
#
# `ip -6 addr show ... to <addr>` prints the address when present and
# nothing when absent, so emptiness is the test.
#
# Presence cannot itself distinguish a VRRP-held address from a static
# leftover -- they look identical. It is only meaningful because the
# keepalived-ntp role asserts, on both distros, that no static assignment
# survives: the play fails before reaching here if one does.
- name: Check if this server holds the IPv6 service address
ansible.builtin.command:
argv:
- ip
- -6
- addr
- show
- dev
- "{{ ntp_keepalived_interface }}"
- to
- "{{ ntp_vip6 }}"
register: vip6_check
failed_when: false
changed_when: false
when: ntp_keepalived_enabled | default(false)
- name: Display IPv6 service address status
ansible.builtin.debug:
msg: >
{% if vip6_check.stdout | default('') | length > 0 %}
This server ({{ inventory_hostname }}) holds the IPv6 service address
{{ ntp_vip6 }} -- it is the v6 MASTER and is serving ntp.srv.renner.uno
{% else %}
This server ({{ inventory_hostname }}) does not hold {{ ntp_vip6 }}.
Expected on a v6 BACKUP. If NO host holds it, check_ntp is failing
everywhere -- inspect `chronyc tracking` and
/var/log/keepalived-ntp-health.log
{% endif %}
when: ntp_keepalived_enabled | default(false)
- name: Report hosts serving NTP without participating in the VIP
ansible.builtin.debug:
msg: >
-14
View File
@@ -1,14 +0,0 @@
---
# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
# attached.
#
# Targets [power_meter]: membership means the meter head is physically present.
#
# Requires the base play (plays/install-edge-host.yml) to have run: it needs
# docker and publishes to the local MQTT bridge.
- name: Install power-meter reader
hosts: power_meter
become: true
roles:
- power-meter-sml
-15
View File
@@ -1,15 +0,0 @@
---
# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
#
# Targets [zwave_gateway]: membership means the radio is physically present.
# Moving the stick to another edge host is a one-line inventory change.
#
# Requires the base play (plays/install-edge-host.yml) to have run: it needs
# docker, and the flash-wear handling that the libeatmydata preload exists to
# complement.
- name: Install Z-Wave gateway
hosts: zwave_gateway
become: true
roles:
- zwave-gateway
@@ -1,25 +1,9 @@
---
# Chrony NTP Server Role Defaults
# Networks allowed to use this server as NTP source.
#
# BOTH families are required. IPv6 is the primary address family here -- the
# service address ntp.srv.renner.uno resolves to fd40::123 -- so an IPv4-only
# allow list refuses every client that uses the documented name.
#
# The hosts carried `allow fd00::/8` for a long time without it ever being in
# this file. Applying the role on 2026-08-07 rendered the template over it and
# dropped the line on all four NTP servers at once, which took internal NTP
# down over IPv6 until it was added here. Same class of untracked host state as
# the anycast service address, and the same failure: config that exists only on
# the box survives right up until something re-renders it.
# Networks allowed to use this server as NTP source
chrony_allowed_networks:
- "10.2.0.0/16"
- "fd00::/8"
# Link-local. A client on the same segment may source its query from fe80::,
# and without this chrony refuses it -- the request arrives and is dropped,
# which looks like the server being unreachable rather than a policy refusal.
- "fe80::/10"
# Primary NTP servers (PTB - Physikalisch-Technische Bundesanstalt, Stratum 1)
chrony_upstream_servers:
+2 -5
View File
@@ -118,11 +118,8 @@ After the `kubernetes/apps/claude-workstation/` PR merges and Flux reconciles:
NS=apps
TOKEN=$(kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.token}' | base64 -d)
kubectl -n $NS get secret claude-workstation-token -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/ca.crt
# server = the cluster's stable API endpoint (kubernetes/talos/patches/cluster.yaml).
# Use the name, not an address: it is what the API server certificate carries as a
# SAN, so an IP here fails TLS verification even when it is reachable. It must
# resolve from the workstation over the site-to-site route.
kubectl config set-cluster homelab --server=https://k8s.renner.uno:6443 \
# server = the API endpoint reachable over the site-to-site route (a CP node or VIP):
kubectl config set-cluster homelab --server=https://10.2.40.201:6443 \
--certificate-authority=/tmp/ca.crt --embed-certs=true --kubeconfig=/tmp/claude-ws.kubeconfig
kubectl config set-credentials claude-workstation --token="$TOKEN" --kubeconfig=/tmp/claude-ws.kubeconfig
kubectl config set-context homelab --cluster=homelab --user=claude-workstation --kubeconfig=/tmp/claude-ws.kubeconfig
@@ -1,15 +0,0 @@
---
# Flash-wear mitigation.
# --- Logging & eMMC wear mitigation ------------------------------------
# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
# kept off local flash: containers + native services log to a RAM-only
# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
# wear check round out the flash-longevity measures. See README "Logging &
# eMMC".
edge_journald_runtime_max: "128M" # cap the volatile (RAM) journal
# Container stdout -> journal, no json log files on the flash. The journal
# itself is volatile (see the tasks above), so container logs land in RAM and
# are shipped off the box by edge-log-shipper.
docker_log_driver: journald
@@ -1,17 +0,0 @@
---
- name: Restart journald
ansible.builtin.service:
name: systemd-journald
state: restarted
# Handlers run in definition order, so this fires after 'Restart journald':
# once the journal is volatile, the on-disk store is dead weight on the flash.
- name: Purge persistent journal store
ansible.builtin.file:
path: /var/log/journal
state: absent
- name: Restart docker
ansible.builtin.service:
name: docker
state: restarted
@@ -1,100 +0,0 @@
---
# Flash-wear mitigation for hosts on small eMMC (Dell Wyse 3040 and similar).
#
# journald -> volatile (RAM) and size-capped, root mounted noatime+lazytime,
# periodic TRIM, and a weekly wear check. Shipping the volatile journal off the
# box is a separate concern -- see the edge-log-shipper role.
- name: Install mmc-utils
ansible.builtin.package:
name: mmc-utils # eMMC lifetime/EOL introspection (mmc extcsd read)
state: present
- name: Ensure journald drop-in directory exists
ansible.builtin.file:
path: /etc/systemd/journald.conf.d
state: directory
mode: "0755"
- name: Switch the systemd journal to volatile (RAM) storage
ansible.builtin.template:
src: journald-edge.conf.j2
dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
mode: "0644"
notify:
- Restart journald
- Purge persistent journal store
- name: Resolve root filesystem facts (for noatime)
ansible.builtin.set_fact:
edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
- name: Refuse to touch fstab without a stable root UUID
ansible.builtin.assert:
that:
- edge_root_fs.uuid | default('') | length > 0
fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
# heartbeat that otherwise dirties the journal every couple seconds); noatime
# drops access-time writes. Neither weakens data durability.
- name: Mount root with noatime + lazytime (persisted in fstab)
ansible.posix.mount:
path: /
src: "UUID={{ edge_root_fs.uuid }}"
fstype: "{{ edge_root_fs.fstype }}"
opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
dump: "0"
passno: "1" # root fs — keep boot-time fsck ordering
backup: true
state: mounted
- name: Enable periodic TRIM (helps eMMC wear-leveling)
ansible.builtin.systemd:
name: fstrim.timer
enabled: true
state: started
- name: Install eMMC health-check script
ansible.builtin.copy:
src: edge-emmc-health
dest: /usr/local/bin/edge-emmc-health
mode: "0755"
- name: Install eMMC health-check service + timer
ansible.builtin.copy:
src: "{{ item }}"
dest: "/etc/systemd/system/{{ item }}"
mode: "0644"
loop:
- edge-emmc-health.service
- edge-emmc-health.timer
- name: Enable eMMC health-check timer
ansible.builtin.systemd:
name: edge-emmc-health.timer
enabled: true
state: started
daemon_reload: true
# Container stdout into the journal instead of json files on disk. This lives
# here, with the rest of the flash-wear handling, rather than in docker-install:
# that role is shared by ten other plays (authentik, n8n, owncloud, grist,
# netbird, gatus, music-assistant, ical-proxy), none of which have a
# daemon.json today. Adding it there would have created one on the next run of
# any of those plays and restarted every container on those hosts.
#
# /etc/docker is created explicitly because this role runs before
# docker-install on a first bootstrap, so the directory may not exist yet. The
# handler runs at the end of the play, by which time docker is installed.
- name: Ensure the docker config directory exists
ansible.builtin.file:
path: /etc/docker
state: directory
mode: "0755"
- name: Route container logs to the journal (keeps json logs off the flash)
ansible.builtin.template:
src: docker-daemon.json.j2
dest: /etc/docker/daemon.json
mode: "0644"
notify: Restart docker
@@ -1,3 +0,0 @@
{
"log-driver": "{{ docker_log_driver }}"
}
@@ -33,43 +33,13 @@ Build and boot the dedicated installer image — see
(partitions the internal disk — NVMe if present, else the eMMC; the
installer autodetects — installs the ansible-ready base system with SSH
keys, DHCP, docker — asks for confirmation before wiping).
3. Confirm the box resolves and is reachable by name — `ssh root@<host>` must
work before anything else. The UDM registers DHCP leases in DNS, so the
record appears on its own; the host is identified by name everywhere
(inventory, plays, this file) and its address is never recorded. Do not
pin it to a reservation.
4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
That group is the base: every edge thin client belongs to it, and it
assumes no peripherals.
5. From the workstation:
3. Add the DHCP reservation/DNS entry for `edge-1.srv.renner.uno` on the
UDM, confirm `ssh root@edge-1.srv.renner.uno` works.
4. From the workstation:
```
ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host
ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
```
One unrestricted run over the whole group. The play is self-contained --
it carries `certificates` and `common` itself, so there is no second
command and no `--limit`. Running it converges every edge host, which is
the point: a run that touches only the new box tells you nothing about
the others.
## Peripherals
Each attached device is its own inventory group and its own playbook. Group
membership states which hardware a host physically has -- there are no flags
and no `when:` on the roles, so moving a stick between boxes is a one-line
inventory change.
| device | group | playbook |
|---|---|---|
| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
Add the host to the matching group, then run that playbook. They depend on
the base play having run: all three need the local MQTT bridge, and two of
them need docker.
A host with nothing attached is simply absent from all three groups and needs
none of these commands.
## Secrets (before first play run)
@@ -115,7 +85,7 @@ zwave-js (order matters — never two servers on one radio):
`kubectl -n home-assistant exec deploy/zwave-js -- tar cf - -C /usr/src/app store | tar xf - -C /opt/edge-host/zwave-js/`
- [ ] Scale down in-cluster: `kubectl -n home-assistant scale deploy/zwave-js --replicas 0`
- [ ] Move the ZWA-2 stick to edge-1, `docker compose up -d zwave-js`
- [ ] Point HA `zwave_js` integration at `ws://edge-1.iot.renner.uno:3000`
- [ ] Point HA `zwave_js` integration at `ws://edge-1.srv.renner.uno:3000`
- [ ] Nodes alive in the UI on :8091 → remove the flux kustomization
`home-assistant-zwavejs` + PVC in a follow-up PR
@@ -143,12 +113,8 @@ Afterwards:
deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
the preloaded `.so` (see "Logging & eMMC" → zwave-js).
- open3e / E3onCAN updates: both are pinned to commits in
`roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
up upstream changes -- that is deliberate, these drive the heat pump. The
weekly update-audit reports when either falls behind (see
`ci/agents/update-audit/prompt.md`); bump the pin and re-run.
Pinning to a tag is not an option for open3e: it publishes none.
- open3e updates: re-run the play; the pip task tracks the git default
branch (pin `edge_open3e_pkg` to a tag if that ever bites).
- Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
restart afterwards.
- can-utils (`candump`) is AUR-only on Arch — install manually via
+118
View File
@@ -0,0 +1,118 @@
---
edge_base_dir: /opt/edge-host
edge_tz: Europe/Berlin
# --- Logging & eMMC wear mitigation ------------------------------------
# edge-1 runs on a small internal eMMC (Dell Wyse 3040, 7.3 GB), so logs are
# kept off local flash: containers + native services log to a RAM-only
# journal and Grafana Alloy ships it to Loki. TRIM, noatime and a weekly
# wear check round out the flash-longevity measures. See README "Logging &
# eMMC".
edge_journald_runtime_max: "128M" # cap the volatile (RAM) journal
edge_docker_log_driver: journald # container stdout -> journal, no json logs on eMMC
# Grafana Alloy log shipper. The srv-network Loki (loki.srv.renner.uno:3100,
# no auth) is firewalled off the IoT VLAN; the only endpoint edge-1 can reach
# is the authenticated cluster ingress (Traefik basic-auth).
edge_alloy_config_file: /etc/alloy/edge.alloy
edge_alloy_http_port: 12345
edge_alloy_bin: /usr/bin/grafana-alloy # Arch `grafana-alloy` pkg names the binary this, not `alloy`
edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
edge_loki_username: edge-1
edge_alloy_labels:
environment: production
datacenter: homelab
# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
# mosquitto on loopback, which bridges to the central cluster broker over TLS
# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
# blips. The local broker writes nothing to the eMMC (persistence off, logs to
# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
edge_mqtt_host: 127.0.0.1
edge_mqtt_port: 1883
# Upstream cluster broker the local bridge connects to (TLS; the public
# Let's Encrypt cert is validated against the system CA bundle).
edge_mqtt_cluster_host: mqtt.renner.uno
edge_mqtt_cluster_tls_port: 8883
edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
# --- USB devices (stable by-id symlinks). After physically moving the
# sticks to edge-1, verify with `ls -l /dev/serial/by-id/` — the by-id
# names travel with the device, not the host, so these should not change.
edge_zwave_device: /dev/serial/by-id/usb-Nabu_Casa_ZWA-2_80B54EE15720-if00
edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
# --- Images. Versions mirror the k8s manifests at migration time.
# sml2mqtt stays hand-pinned (critical meter bridge — was deliberately
# excluded from image-automation in k8s, keep that property here).
edge_zwavejs_image: ghcr.io/zwave-js/zwave-js-ui:11.21.1
edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
# Alpine version the zwave-js-ui image is built on — the source for the musl
# libeatmydata.so preloaded into that container (see tasks "Extract musl
# libeatmydata.so"). Keep in sync with the base if edge_zwavejs_image bumps
# its Alpine major (check: docker exec zwave-js cat /etc/os-release).
edge_zwave_alpine_base: alpine:3.22
# --- CAN / open3e (Viessmann E3 external bus) ---
edge_can_interface: can0
edge_can_bitrate: 250000
edge_open3e_pkg: git+https://github.com/open3e/open3e.git
edge_open3e_mqtt_user: open3e
edge_open3e_mqtt_topic: open3e
edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
edge_open3e_poll_seconds: 30
# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
# the internal PV-surplus EMS depends on). Decodes via the master O3EList
# codec inherited through the vx3plus overlay.
# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
# entries when added), so the multi-frame read is cheap at a 30 s poll.
# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
edge_open3e_datapoints: >-
vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
# ECU addresses; -dev selects codec set, tx selects the UDS address.
# vcmu (refrigerant-circuit controller) has no dedicated codec set —
# omitting dpList makes open3e fall back to the generic datapoint list,
# which decodes all polled VCMU DIDs.
edge_open3e_devices:
vitocal: { tx: "0x680", dpList: "vcal" }
# dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
# files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
vx3: { tx: "0x6a1", dpList: "/opt/edge-host/open3e/vx3plus.py" }
vcmu: { tx: "0x68c" }
# --- E3onCAN: passive E380 energy-meter listener (no bus writes, so it
# coexists with the single open3e instance). The meter broadcasts at
# ~5 Hz; the gap throttles MQTT to one update per DID per interval.
edge_e3oncan_repo: https://github.com/MyHomeMyData/E3onCAN.git
edge_e3oncan_version: 461dcbdf97e8d72d2edb9929c63b39382af6c3a1 # V0.5.2
edge_e3oncan_gap_seconds: 10
edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
# --- Secrets: define these in inventory/group_vars/edge_host/vault.yml
# (ansible-vault). See README.md for the kubectl one-liners that read the
# current values out of the cluster during migration.
# edge_sml2mqtt_mqtt_password: "{{ vault_edge_sml2mqtt_mqtt_password }}"
# edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
# edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"
# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
# VRRP group, which needs layer-2 adjacency with the other members on the
# service VLAN. The untagged VLAN 30 lease stays this host's only default
# route -- see templates/10-vlan40.network.j2.
edge_vlan40_enabled: false
edge_vlan40_parent: enp1s0
edge_vlan40_id: 40
edge_vlan40_name: enp1s0.40
edge_vlan40_parent_network: 20-wired.network
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# edge-1 eMMC wear/health check. Managed by roles/edge-flash-wear.
# edge-1 eMMC wear/health check. Managed by roles/edge-host.
#
# Reads the JEDEC eMMC 5.x life-time and pre-EOL estimators exposed via sysfs
# and emits a warning-priority journal message when the flash leaves the
@@ -1,4 +1,4 @@
# Managed by ansible (roles/heating-can). Datapoint overlay for the VX3:
# Managed by ansible (roles/edge-host). Datapoint overlay for the VX3:
# the stock vx3 list acts as a FILTER (DIDs missing from it are popped
# from the master table, Open3Eclass.py), so the per-battery-module DIDs
# were never pollable with dpList "vx3". This overlay re-adds them with
+66
View File
@@ -0,0 +1,66 @@
---
# reload alone only re-reads config files; existing links keep their old
# state until reconfigured.
- name: Reload networkd
ansible.builtin.shell:
cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
changed_when: true
# Reconfiguring the parent is what actually materialises the VLAN child -- a
# bare reload leaves the existing link as-is (see above). This re-runs DHCP on
# enp1s0, which is this headless host's only path to the network, so it is the
# one genuinely risky step in the VLAN change.
- name: Reload networkd for VLAN
ansible.builtin.shell:
cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
changed_when: true
# Ordered before 'Restart edge compose services': the daemon must pick up the
# new default log driver before the containers are recreated onto it.
- name: Restart docker
ansible.builtin.service:
name: docker
state: restarted
- name: Restart journald
ansible.builtin.service:
name: systemd-journald
state: restarted
# Handlers run in definition order, so this fires after 'Restart journald':
# once the journal is volatile, the on-disk store is dead weight on the eMMC.
- name: Purge persistent journal store
ansible.builtin.file:
path: /var/log/journal
state: absent
- name: Restart alloy
ansible.builtin.systemd:
name: edge-alloy
state: restarted
daemon_reload: true
# Ordered before the MQTT clients (compose/open3e/e3oncan) so they reconnect
# to a freshly-configured local broker.
- name: Restart mosquitto
ansible.builtin.service:
name: mosquitto
state: restarted
- name: Restart edge compose services
ansible.builtin.command:
cmd: docker compose up -d --force-recreate
chdir: "{{ edge_base_dir }}"
changed_when: true
- name: Restart open3e
ansible.builtin.systemd:
name: open3e
state: restarted
daemon_reload: true
- name: Restart e3oncan
ansible.builtin.systemd:
name: e3oncan
state: restarted
daemon_reload: true
+440
View File
@@ -0,0 +1,440 @@
---
# Arch edge host: zwave-js-ui + sml2mqtt as docker compose services,
# open3e natively (venv + systemd) because it needs a SocketCAN interface,
# which is host network state, not something to hide in a container.
# Docker itself comes from the docker-install role (Arch-aware).
- name: Install base packages
ansible.builtin.package:
name:
- python
- python-packaging # ansible's pip module imports it on the target
- git
- mmc-utils # eMMC lifetime/EOL introspection (mmc extcsd read)
- grafana-alloy # log shipper -> Loki (see Alloy section below)
- mosquitto # local MQTT bridge broker -> cluster over TLS
state: present
# --- Local MQTT bridge broker. Edge clients publish to this over loopback;
# it bridges to the cluster broker over TLS (open3e/e3oncan can't do MQTTS).
# Configured with persistence off + stdout logging so it writes nothing to the
# eMMC. Deployed before the clients so they have somewhere to connect.
# See edge-mqtt-bridge.conf.j2 / README "MQTT bridge".
- name: Deploy local mosquitto bridge config
ansible.builtin.template:
src: edge-mqtt-bridge.conf.j2
dest: /etc/mosquitto/mosquitto.conf
owner: root
group: mosquitto
mode: "0640" # carries the cluster bridge credentials
notify: Restart mosquitto
- name: Enable local mosquitto bridge
ansible.builtin.systemd:
name: mosquitto
enabled: true
state: started
# ======================================================================
# eMMC wear mitigation + log shipping. edge-1 (Dell Wyse 3040) runs on a
# small internal eMMC, so logs must not churn local flash. Strategy:
# - journald -> volatile (RAM), size-capped (journald-edge.conf)
# - docker -> journald log driver, no json files (docker-daemon.json)
# - Grafana Alloy tails the RAM journal -> Loki, WAL on tmpfs
# plus TRIM, noatime and a weekly wear check for flash longevity.
# See roles/edge-host/README.md "Logging & eMMC".
# ======================================================================
- name: Ensure journald drop-in directory exists
ansible.builtin.file:
path: /etc/systemd/journald.conf.d
state: directory
mode: "0755"
- name: Switch the systemd journal to volatile (RAM) storage
ansible.builtin.template:
src: journald-edge.conf.j2
dest: /etc/systemd/journald.conf.d/00-edge-emmc.conf
mode: "0644"
notify:
- Restart journald
- Purge persistent journal store
- name: Resolve root filesystem facts (for noatime)
ansible.builtin.set_fact:
edge_root_fs: "{{ ansible_facts.mounts | selectattr('mount', 'equalto', '/') | first }}"
- name: Refuse to touch fstab without a stable root UUID
ansible.builtin.assert:
that:
- edge_root_fs.uuid | default('') | length > 0
fail_msg: "Could not determine root UUID from facts; not rewriting fstab."
# lazytime batches inode-timestamp writeback (e.g. the zwave-js lockfile
# heartbeat that otherwise dirties the journal every couple seconds); noatime
# drops access-time writes. Neither weakens data durability.
- name: Mount root with noatime + lazytime (persisted in fstab)
ansible.posix.mount:
path: /
src: "UUID={{ edge_root_fs.uuid }}"
fstype: "{{ edge_root_fs.fstype }}"
opts: "{{ ((edge_root_fs.options.split(',') | reject('in', ['atime', 'noatime', 'relatime', 'diratime', 'lazytime', 'nolazytime']) | list) + ['noatime', 'lazytime']) | unique | join(',') }}"
dump: "0"
passno: "1" # root fs — keep boot-time fsck ordering
backup: true
state: mounted
- name: Enable periodic TRIM (helps eMMC wear-leveling)
ansible.builtin.systemd:
name: fstrim.timer
enabled: true
state: started
- name: Install eMMC health-check script
ansible.builtin.copy:
src: edge-emmc-health
dest: /usr/local/bin/edge-emmc-health
mode: "0755"
- name: Install eMMC health-check service + timer
ansible.builtin.copy:
src: "{{ item }}"
dest: "/etc/systemd/system/{{ item }}"
mode: "0644"
loop:
- edge-emmc-health.service
- edge-emmc-health.timer
- name: Enable eMMC health-check timer
ansible.builtin.systemd:
name: edge-emmc-health.timer
enabled: true
state: started
daemon_reload: true
# --- Grafana Alloy: ship the volatile journal to Loki. Container logs reach
# the journal via docker's journald driver (configured below), native
# services log to the journal directly, so a single journal source covers
# everything. Storage/WAL sits on /run (tmpfs) — see edge-alloy.service.
- name: Create alloy system group
ansible.builtin.group:
name: alloy
system: true
- name: Create alloy system user
ansible.builtin.user:
name: alloy
group: alloy
groups: systemd-journal
system: true
create_home: false
shell: /usr/bin/nologin
- name: Create alloy config directory
ansible.builtin.file:
path: /etc/alloy
state: directory
owner: alloy
group: alloy
mode: "0750"
- name: Deploy alloy config (journal -> Loki)
ansible.builtin.template:
src: edge-alloy.alloy.j2
dest: "{{ edge_alloy_config_file }}"
owner: alloy
group: alloy
mode: "0600" # carries the Loki basic-auth password
notify: Restart alloy
- name: Deploy alloy systemd unit
ansible.builtin.template:
src: edge-alloy.service.j2
dest: /etc/systemd/system/edge-alloy.service
mode: "0644"
notify: Restart alloy
- name: Enable edge-alloy
ansible.builtin.systemd:
name: edge-alloy
enabled: true
state: started
daemon_reload: true
# --- SocketCAN: systemd-networkd owns can0. gs_usb is an in-tree module
# and autoloads when the Innomaker adapter enumerates; networkd then
# matches the interface and applies bitrate + up. RestartSec recovers the
# link after bus-off without manual intervention.
- name: Configure can0 via systemd-networkd
ansible.builtin.template:
src: 80-can0.network.j2
dest: /etc/systemd/network/80-can0.network
mode: "0644"
notify: Reload networkd
# --- Tagged VLAN 40. Additive by construction: the parent link keeps its own
# .network file untouched and gains the VLAN via a drop-in, so reverting is a
# matter of deleting these three paths. This host is headless and enp1s0 is its
# only route to the network, hence the caution.
- name: Create VLAN sub-interface netdev
ansible.builtin.template:
src: 10-vlan40.netdev.j2
dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
mode: "0644"
when: edge_vlan40_enabled
notify: Reload networkd for VLAN
tags: [vlan40]
- name: Configure VLAN sub-interface addressing
ansible.builtin.template:
src: 10-vlan40.network.j2
dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
mode: "0644"
when: edge_vlan40_enabled
notify: Reload networkd for VLAN
tags: [vlan40]
- name: Ensure parent network drop-in directory exists
ansible.builtin.file:
path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
state: directory
mode: "0755"
when: edge_vlan40_enabled
tags: [vlan40]
- name: Attach VLAN to parent link via drop-in
ansible.builtin.template:
src: vlan40-parent-dropin.conf.j2
dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
mode: "0644"
when: edge_vlan40_enabled
notify: Reload networkd for VLAN
tags: [vlan40]
- name: Enable systemd-networkd
ansible.builtin.service:
name: systemd-networkd
state: started
enabled: true
# Docker networking needs no daemon.json (both services use network_mode:
# host — real addresses, v4+v6, no bridge/NAT). The daemon.json below exists
# only to route container stdout into the journald log driver so json logs
# never touch the eMMC; the journal itself is volatile (RAM) and shipped to
# Loki by edge-alloy. Both compose services inherit this default driver.
- name: Configure docker daemon (journald log driver, eMMC-friendly)
ansible.builtin.template:
src: docker-daemon.json.j2
dest: /etc/docker/daemon.json
mode: "0644"
notify:
- Restart docker
- Restart edge compose services
# --- Directory layout ---
- name: Create service directories
ansible.builtin.file:
path: "{{ item }}"
state: directory
mode: "0755"
loop:
- "{{ edge_base_dir }}"
- "{{ edge_base_dir }}/zwave-js/store"
- "{{ edge_base_dir }}/sml2mqtt"
# --- zwave-js eMMC write reduction: batch fsync via libeatmydata ---------
# zwave-js persists its value DB with an fsync on *every* value change
# (jsonl-db, throttleFS unset by zwave-js-ui). On a chatty network that
# fsync storm churns the eMMC (~2.6 GiB/day for ~0.1 KiB/s of real data).
# Preloading libeatmydata makes fsync a no-op so the kernel batches the
# writes via normal writeback — full HA data resolution and cache
# persistence are untouched (see docker-compose.yml.j2 for the mount + env).
# The .so must match the zwave-js-ui image's musl/Alpine base, so extract it
# from that exact Alpine version (bridge networking is broken on this host,
# hence --network=host). Refresh by deleting the .so if the base ever bumps
# its Alpine major version.
- name: Extract musl libeatmydata.so (matches the zwave-js-ui Alpine base)
ansible.builtin.shell:
cmd: |
set -euo pipefail
tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
docker run --rm --network=host {{ edge_zwave_alpine_base }} \
sh -c 'apk add --no-cache libeatmydata >/dev/null 2>&1 && cat /usr/lib/libeatmydata.so' > "$tmp"
test "$(stat -c%s "$tmp")" -ge 8000 # sanity: real .so is ~14 KB, guards against a failed/empty pull
install -m 0644 "$tmp" {{ edge_base_dir }}/zwave-js/libeatmydata.so
creates: "{{ edge_base_dir }}/zwave-js/libeatmydata.so"
notify: Restart edge compose services
# Because fsync is neutralized, a hard power-loss during the (rare) settings
# write could truncate settings.json — which holds the Z-Wave network keys.
# Keep a validated rolling backup on the eMMC so the keys stay recoverable.
- name: Install zwave-js settings.json backup script
ansible.builtin.template:
src: edge-zwave-settings-backup.j2
dest: /usr/local/bin/edge-zwave-settings-backup
mode: "0755"
- name: Install zwave-js settings backup service + timer
ansible.builtin.copy:
src: "{{ item }}"
dest: "/etc/systemd/system/{{ item }}"
mode: "0644"
loop:
- edge-zwave-settings-backup.service
- edge-zwave-settings-backup.timer
- name: Enable zwave-js settings backup timer
ansible.builtin.systemd:
name: edge-zwave-settings-backup.timer
enabled: true
state: started
daemon_reload: true
# --- Compose services: zwave-js-ui + sml2mqtt ---
- name: Deploy docker-compose.yml
ansible.builtin.template:
src: docker-compose.yml.j2
dest: "{{ edge_base_dir }}/docker-compose.yml"
mode: "0644"
notify: Restart edge compose services
- name: Deploy zwave-js env (session secret)
ansible.builtin.template:
src: zwavejs.env.j2
dest: "{{ edge_base_dir }}/zwavejs.env"
mode: "0600"
notify: Restart edge compose services
# sml2mqtt reads only a config file and has no env override for the
# password, so the rendered config carries it — 0600, root-owned, same
# trust boundary as the k8s init-container approach it replaces.
- name: Deploy sml2mqtt config
ansible.builtin.template:
src: sml2mqtt-config.yml.j2
dest: "{{ edge_base_dir }}/sml2mqtt/config.yml"
mode: "0600"
notify: Restart edge compose services
- name: Pull compose images
ansible.builtin.command:
cmd: docker compose pull
chdir: "{{ edge_base_dir }}"
changed_when: false
# Tolerates missing USB devices: before the physical cutover the ZWA-2 /
# FTDI sticks are still on the old k8s-edge VM, and a container whose
# device is absent must not block the rest of the role (open3e/CAN moves
# first). Re-run the play — or `docker compose up -d` — after each stick
# arrives.
- name: Start compose services
ansible.builtin.command:
cmd: docker compose up -d
chdir: "{{ edge_base_dir }}"
register: compose_up
failed_when: >-
compose_up.rc != 0
and 'error gathering device information' not in compose_up.stderr
changed_when: >-
'Started' in compose_up.stderr or 'Created' in compose_up.stderr
or 'Started' in compose_up.stdout or 'Creating' in compose_up.stdout
- name: Warn about containers waiting for their USB device
ansible.builtin.debug:
msg: >-
docker compose could not start every service (USB device not present
yet — expected before cutover): {{ compose_up.stderr_lines | select('search', 'device') | list }}
when: compose_up.rc != 0
# --- open3e: native install. CAN_RAW sockets need no capabilities, so
# the service runs as an unprivileged system user.
- name: Create open3e system user
ansible.builtin.user:
name: open3e
system: true
create_home: false
shell: /usr/bin/nologin
# Owned by the service user: udsoncan writes its log file into the
# working directory.
- name: Create open3e directory
ansible.builtin.file:
path: "{{ edge_base_dir }}/open3e"
state: directory
owner: open3e
group: open3e
mode: "0755"
- name: Install open3e into a venv
ansible.builtin.pip:
name: "{{ edge_open3e_pkg }}"
virtualenv: "{{ edge_base_dir }}/open3e/venv"
virtualenv_command: python -m venv
notify: Restart open3e
- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
ansible.builtin.copy:
src: open3e-vx3plus.py
dest: "{{ edge_base_dir }}/open3e/vx3plus.py"
mode: "0644"
notify: Restart open3e
- name: Deploy open3e devices.json
ansible.builtin.copy:
content: "{{ edge_open3e_devices | to_nice_json }}\n"
dest: "{{ edge_base_dir }}/open3e/devices.json"
mode: "0644"
notify: Restart open3e
- name: Deploy open3e environment (broker credential)
ansible.builtin.template:
src: open3e.env.j2
dest: "{{ edge_base_dir }}/open3e/open3e.env"
owner: open3e
mode: "0600"
notify: Restart open3e
- name: Deploy open3e systemd unit
ansible.builtin.template:
src: open3e.service.j2
dest: /etc/systemd/system/open3e.service
mode: "0644"
notify: Restart open3e
- name: Enable open3e
ansible.builtin.systemd:
name: open3e
enabled: true
state: started
daemon_reload: true
# --- E3onCAN: direct E380 meter reads (passive; per-review decision not
# to trust the VX3's relayed meter values). Same broker user as open3e;
# topics land under the same open3e/ prefix as e380_<did>_<name>.
- name: Check out E3onCAN (pinned)
ansible.builtin.git:
repo: "{{ edge_e3oncan_repo }}"
dest: "{{ edge_base_dir }}/e3oncan/src"
version: "{{ edge_e3oncan_version }}"
notify: Restart e3oncan
- name: Create E3onCAN venv with requirements
ansible.builtin.pip:
requirements: "{{ edge_base_dir }}/e3oncan/src/requirements.txt"
virtualenv: "{{ edge_base_dir }}/e3oncan/venv"
virtualenv_command: python -m venv
notify: Restart e3oncan
- name: Deploy e3oncan systemd unit
ansible.builtin.template:
src: e3oncan.service.j2
dest: /etc/systemd/system/e3oncan.service
mode: "0644"
notify: Restart e3oncan
- name: Enable e3oncan
ansible.builtin.systemd:
name: e3oncan
enabled: true
state: started
daemon_reload: true
@@ -1,4 +1,4 @@
# Managed by ansible (roles/socketcan).
# Managed by ansible (roles/edge-host).
# Viessmann E3 external CAN bus via Innomaker USB2CAN (gs_usb).
# The adapter is a stub tap on an already-terminated bus (120R jumper OFF).
[Match]
@@ -0,0 +1,56 @@
# Managed by ansible (roles/edge-host). Mirrors the former k8s manifests:
# kubernetes/apps/home-assistant-zwavejs/deployment.yaml and
# kubernetes/apps/home-assistant/sml2mqtt-deployment.yaml.
#
# No `privileged` needed here: docker's --device handles the device-cgroup
# grant that hostPath in k8s could not (that was the only reason the pods
# ran privileged).
# Both services use host networking: real host addresses (v4+v6), no
# bridge, no NAT. The broker LB is IPv6-first and docker bridge NAT66 is
# a tarpit (unmasqueraded SYNs die silently); the host stack just works.
services:
zwave-js:
image: {{ edge_zwavejs_image }}
container_name: zwave-js
restart: unless-stopped
stop_signal: SIGINT
tty: true
environment:
- TZ={{ edge_tz }}
# Batch the value-DB fsync storm to spare the eMMC (see role tasks /
# README "Logging & eMMC"). Neutralizes fsync only for this container;
# the WS/MQTT value stream to HA and cache persistence are unchanged.
- LD_PRELOAD=/usr/lib/libeatmydata.so
env_file:
- zwavejs.env
devices:
- "{{ edge_zwave_device }}:/dev/zwave"
volumes:
# musl libeatmydata.so extracted from the image's Alpine base (ro)
- ./zwave-js/libeatmydata.so:/usr/lib/libeatmydata.so:ro
# Persistent store: settings.json, device DB, network keys. Seeded
# from the k8s PVC `zwavejs-store` at cutover (see README). Since
# this host is stateful there is no init-container merge step —
# settings.json is edited via the UI and persists here; zwave.port
# stays /dev/zwave.
- ./zwave-js/store:/usr/src/app/store
# Host networking: binds 3000 (Z-Wave JS websocket for HA) and 8091
# (web UI) directly on the host.
network_mode: host
sml2mqtt:
image: {{ edge_sml2mqtt_image }}
container_name: sml2mqtt
restart: unless-stopped
# Outbound-only (serial in, MQTT out) — host networking sidesteps the
# docker NAT66 gap: the broker LB is IPv6-first and bridge-network v6
# SYNs leave unmasqueraded and die. The host stack reaches fd99::81
# fine (open3e proves it continuously).
network_mode: host
# Serial exclusivity: this is the ONLY reader of the meter port —
# HA's edl21 integration stays disabled (see sml2mqtt-config comment
# history in the k8s repo).
devices:
- "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
volumes:
- ./sml2mqtt/config.yml:/sml2mqtt/config.yml:ro
@@ -0,0 +1,3 @@
{
"log-driver": "{{ edge_docker_log_driver }}"
}
@@ -0,0 +1,32 @@
# Managed by ansible (roles/edge-host).
# PASSIVE listener for the E380 energy meter's CANopen broadcasts — no
# writes on the bus, so it may run alongside the (single) open3e
# instance. Reads the meter directly instead of trusting the VX3's
# relayed values.
[Unit]
Description=E3onCAN E380 energy-meter CAN listener (read-only)
BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
# mosquitto.service = the local MQTT bridge broker e3oncan publishes to (loopback).
After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
Wants=network-online.target mosquitto.service
[Service]
User=open3e
EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
WorkingDirectory={{ edge_base_dir }}/e3oncan
# Journald-friendly (python block-buffers pipes; lost on restarts).
Environment=PYTHONUNBUFFERED=1
ExecStart={{ edge_base_dir }}/e3oncan/venv/bin/python {{ edge_base_dir }}/e3oncan/src/E3onCANcollect.py \
-c {{ edge_can_interface }} \
-dev e380 \
-g {{ edge_e3oncan_gap_seconds }} \
-m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-muser {{ edge_open3e_mqtt_user }} \
-mpass ${OPEN3E_MQTT_PASSWORD} \
-mfstr "{{ edge_e3oncan_mqtt_format }}" \
-j
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
@@ -1,4 +1,4 @@
// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-log-shipper.
// Grafana Alloy — edge-1 log shipper. Managed by roles/edge-host.
//
// edge-1 runs on internal eMMC, so logs are kept out of local flash: the
// systemd journal is volatile (RAM, see journald-edge.conf) and container
@@ -1,4 +1,4 @@
# Managed by roles/mqtt-bridge. Local mosquitto that the edge clients (open3e,
# Managed by roles/edge-host. Local mosquitto that the edge clients (open3e,
# e3oncan, sml2mqtt) publish to over loopback; it bridges to the central
# cluster broker over TLS. open3e/e3oncan have no native MQTTS, so this is how
# edge-1's MQTT gets off plaintext on the untrusted IoT VLAN, and it buffers
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Rolling backup of zwave-js settings.json (holds the Z-Wave network keys).
# Managed by roles/zwave-gateway. edge-1 runs zwave-js under libeatmydata (fsync
# Managed by roles/edge-host. edge-1 runs zwave-js under libeatmydata (fsync
# neutralized to spare the eMMC), so a hard power-loss during the rare
# settings write could truncate settings.json. Keep a validated backup on the
# eMMC so the keys are always recoverable. Runs hourly via a systemd timer.
@@ -1,4 +1,4 @@
# Managed by roles/edge-flash-wear. edge-1 boots off a small internal eMMC, so the
# Managed by roles/edge-host. edge-1 boots off a small internal eMMC, so the
# journal must not persist to flash: keep it in RAM (/run/log/journal) and cap
# its size. Grafana Alloy (edge-alloy.service) ships this journal to Loki
# before it rotates out of RAM — nothing durable lives here.
@@ -0,0 +1,7 @@
# Managed by ansible (roles/edge-host). Read by open3e.service and
# e3oncan.service.
# Full user:password pair as one variable so the open3e unit can pass it
# as a single -muser argument without exposing it in the unit file;
# e3oncan takes user and password separately (-muser/-mpass).
OPEN3E_MUSER={{ edge_open3e_mqtt_user }}:{{ edge_open3e_mqtt_password }}
OPEN3E_MQTT_PASSWORD={{ edge_open3e_mqtt_password }}
@@ -0,0 +1,39 @@
# Managed by ansible (roles/edge-host).
# Read-mostly listener — no -w flag, no service 0x77. The ONE sanctioned
# write path is the MQTT command topic (-l): HA publishes the DHW
# one-time-charge command there (scope decision 2026-07-02, see vican
# CLAUDE.md). Anything beyond that single write gets its own review.
# -j is load-bearing: without it open3e flattens JSON payloads into
# per-key subtopics and the HA package (homeassistant repo,
# packages/open3e.yaml) receives nothing.
# Exactly ONE open3e instance may be on the bus at a time (CAN collisions
# stop open3e) — stop this service before running ad-hoc scans.
[Unit]
Description=open3e Viessmann E3 CAN-to-MQTT bridge (read-only)
# Tie the service to the CAN device: stops on unplug, starts on plug via
# the device unit rather than polling.
BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
# mosquitto.service = the local MQTT bridge broker open3e publishes to (loopback).
After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
Wants=network-online.target mosquitto.service
[Service]
User=open3e
EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
WorkingDirectory={{ edge_base_dir }}/open3e
ExecStart={{ edge_base_dir }}/open3e/venv/bin/open3e \
-c {{ edge_can_interface }} \
-cnfg {{ edge_base_dir }}/open3e/devices.json \
-m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-muser ${OPEN3E_MUSER} \
-mcid open3e-edge \
-mfstr "{{ edge_open3e_mqtt_format }}" \
-r {{ edge_open3e_datapoints }} \
-t {{ edge_open3e_poll_seconds }} \
-l {{ edge_open3e_mqtt_topic }}/cmnd \
-j
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
@@ -1,4 +1,4 @@
# Managed by ansible (roles/power-meter-sml). Ported 1:1 from the k8s ConfigMap
# Managed by ansible (roles/edge-host). Ported 1:1 from the k8s ConfigMap
# sml2mqtt-config; only host and password delivery differ (broker is now
# reached via its LoadBalancer hostname, password rendered by ansible
# instead of an init container).
@@ -1,15 +0,0 @@
---
# Journal -> Loki shipping.
#
# The srv-network Loki (loki.srv.renner.uno:3100, no auth) is firewalled off
# the IoT VLAN; the only endpoint these hosts can reach is the authenticated
# cluster ingress. edge_loki_password comes from the vault.
edge_alloy_config_file: /etc/alloy/edge.alloy
edge_alloy_http_port: 12345
edge_alloy_bin: /usr/bin/grafana-alloy # Arch `grafana-alloy` pkg names the binary this, not `alloy`
edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
edge_loki_username: edge-1
edge_alloy_labels:
environment: production
datacenter: homelab
@@ -1,6 +0,0 @@
---
- name: Restart alloy
ansible.builtin.systemd:
name: edge-alloy
state: restarted
daemon_reload: true
@@ -1,57 +0,0 @@
---
# Ship the systemd journal to Loki with Grafana Alloy.
#
# Pairs with edge-flash-wear, which makes the journal volatile: container logs
# reach the journal via docker s journald driver and native services log to it
# directly, so one journal source covers everything. Alloy s storage/WAL sits
# on /run (tmpfs) so it does not write to flash either.
- name: Install grafana-alloy
ansible.builtin.package:
name: grafana-alloy
state: present
- name: Create alloy system group
ansible.builtin.group:
name: alloy
system: true
- name: Create alloy system user
ansible.builtin.user:
name: alloy
group: alloy
groups: systemd-journal
system: true
create_home: false
shell: /usr/bin/nologin
- name: Create alloy config directory
ansible.builtin.file:
path: /etc/alloy
state: directory
owner: alloy
group: alloy
mode: "0750"
- name: Deploy alloy config (journal -> Loki)
ansible.builtin.template:
src: edge-alloy.alloy.j2
dest: "{{ edge_alloy_config_file }}"
owner: alloy
group: alloy
mode: "0600" # carries the Loki basic-auth password
notify: Restart alloy
- name: Deploy alloy systemd unit
ansible.builtin.template:
src: edge-alloy.service.j2
dest: /etc/systemd/system/edge-alloy.service
mode: "0644"
notify: Restart alloy
- name: Enable edge-alloy
ansible.builtin.systemd:
name: edge-alloy
enabled: true
state: started
daemon_reload: true
-11
View File
@@ -1,11 +0,0 @@
---
# --- Tagged VLAN 40 sub-interface. Off by default; enabled in
# inventory/group_vars/edge_host. Exists so edge-1 can join the NTP keepalived
# VRRP group, which needs layer-2 adjacency with the other members on the
# service VLAN. The untagged VLAN 30 lease stays this host's only default
# route -- see templates/10-vlan40.network.j2.
edge_vlan40_enabled: false
edge_vlan40_parent: enp1s0
edge_vlan40_id: 40
edge_vlan40_name: enp1s0.40
edge_vlan40_parent_network: 20-wired.network
@@ -1,9 +0,0 @@
---
# Reconfiguring the parent is what actually materialises the VLAN child -- a
# bare reload leaves the existing link as-is. This re-runs DHCP on the parent,
# which is a headless host's only path to the network, so it is the one
# genuinely risky step in the VLAN change.
- name: Reload networkd for VLAN
ansible.builtin.shell:
cmd: networkctl reload && networkctl reconfigure {{ edge_vlan40_parent }}
changed_when: true
-48
View File
@@ -1,48 +0,0 @@
---
# Tagged VLAN sub-interface via systemd-networkd.
#
# Additive by construction: the parent link keeps its own .network file and
# gains the VLAN through a drop-in, so reverting is deleting three paths. These
# hosts are headless and the parent is their only route to the network, hence
# the caution.
- name: Create VLAN sub-interface netdev
ansible.builtin.template:
src: 10-vlan40.netdev.j2
dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.netdev"
mode: "0644"
when: edge_vlan40_enabled
notify: Reload networkd for VLAN
tags: [vlan40]
- name: Configure VLAN sub-interface addressing
ansible.builtin.template:
src: 10-vlan40.network.j2
dest: "/etc/systemd/network/10-{{ edge_vlan40_name }}.network"
mode: "0644"
when: edge_vlan40_enabled
notify: Reload networkd for VLAN
tags: [vlan40]
- name: Ensure parent network drop-in directory exists
ansible.builtin.file:
path: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d"
state: directory
mode: "0755"
when: edge_vlan40_enabled
tags: [vlan40]
- name: Attach VLAN to parent link via drop-in
ansible.builtin.template:
src: vlan40-parent-dropin.conf.j2
dest: "/etc/systemd/network/{{ edge_vlan40_parent_network }}.d/10-vlan40.conf"
mode: "0644"
when: edge_vlan40_enabled
notify: Reload networkd for VLAN
tags: [vlan40]
- name: Enable systemd-networkd
ansible.builtin.service:
name: systemd-networkd
state: started
enabled: true
@@ -1,60 +0,0 @@
---
# Heating system over the Viessmann E3 CAN bus: open3e (heat pump) and
# e3oncan (E380 energy meter). The can0 interface itself is the socketcan
# role.
# Service root. Kept per-role so the role is usable on its own.
edge_base_dir: /opt/edge-host
edge_open3e_mqtt_user: open3e
edge_open3e_mqtt_topic: open3e
edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
edge_open3e_poll_seconds: 30
# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
# the internal PV-surplus EMS depends on). Decodes via the master O3EList
# codec inherited through the vx3plus overlay.
# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
# entries when added), so the multi-frame read is cheap at a 30 s poll.
# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
edge_open3e_datapoints: >-
vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791
# ECU addresses; -dev selects codec set, tx selects the UDS address.
# vcmu (refrigerant-circuit controller) has no dedicated codec set —
# omitting dpList makes open3e fall back to the generic datapoint list,
# which decodes all polled VCMU DIDs.
edge_open3e_devices:
vitocal: { tx: "0x680", dpList: "vcal" }
# dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
# files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
vx3: { tx: "0x6a1", dpList: "/config/vx3plus.py" }
vcmu: { tx: "0x68c" }
edge_e3oncan_gap_seconds: 10
edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
# Secrets come from inventory/group_vars/edge_host/vault.yml:
# edge_open3e_mqtt_password
# The image carrying both readers, built and pinned in CI
# (ci/images/heating-can/Containerfile). The source pins live in the
# Containerfile, in the artifact -- not here -- so what is deployed is a tag,
# and `docker compose up -d` on an unchanged tag is a genuine no-op.
#
# Tracked by the weekly update-audit as an ordinary image, which is why this is
# simpler to keep current than the two git refs it replaced: one publishes no
# tags at all, the other was pinned to an untagged commit between releases.
heating_can_image: gitea.renner.uno/gitea_admin/heating-can:latest
# Compose project directory. Separate from the old native layout under
# {{ edge_base_dir }}/open3e, which this role removes.
heating_can_base_dir: "{{ edge_base_dir }}/heating-can"
@@ -1,10 +0,0 @@
---
- name: Reload systemd
ansible.builtin.systemd:
daemon_reload: true
- name: Restart heating-can
ansible.builtin.command:
cmd: docker compose up -d --force-recreate
chdir: "{{ heating_can_base_dir }}"
changed_when: true
-164
View File
@@ -1,164 +0,0 @@
---
# Heating system over the Viessmann E3 CAN bus.
#
# Two readers of the same heating installation, supervised inside one container
# from one image (ci/images/heating-can):
# open3e UDS request/response to the Vitocal heat pump and Vitocharge VX3
# e3oncan passive CANopen listener for the E380 grid energy meter, which
# feeds the heat pump's own energy management
#
# They coexist because e3oncan issues no bus writes.
#
# Nothing is installed here. The image is built in CI with both sources pinned,
# published to the registry, and this host only pulls and runs it. That is what
# removes the churn: these were pip-installed into venvs on the node, and pip
# re-cloned and rebuilt a VCS requirement on every converge -- restarting the
# heating bridge each time -- which pinning the SHA did not fix.
#
# network_mode: host is required, not incidental: SocketCAN interfaces live in
# the host network namespace, so can0 is only reachable that way. It is also how
# the local MQTT broker on 127.0.0.1 is reached. CAN_RAW needs no capabilities,
# so both run unprivileged.
#
# The can0 interface itself belongs to the socketcan role; this role consumes it.
- name: Create heating-can directory
ansible.builtin.file:
path: "{{ heating_can_base_dir }}"
state: directory
mode: "0755"
# --- Migration off the native install ---------------------------------------
# Must happen before the containers start. Two open3e instances issuing UDS
# requests on one CAN bus is the failure mode to avoid, so the old units are
# stopped and masked rather than merely disabled -- masked so a stray
# `systemctl start` or a leftover dependency cannot bring them back alongside
# the containers.
- name: Find which native CAN units still exist
ansible.builtin.stat:
path: "/etc/systemd/system/{{ item }}.service"
loop:
- open3e
- e3oncan
register: heating_can_native_units
# Stop and disable BEFORE anything else, and do NOT swallow the result.
#
# The first version did stop + disable + mask in one call with
# failed_when: false, then deleted the unit files. Masking a unit whose file
# still exists conflicts, so the call failed, failed_when hid it, nothing was
# stopped -- and the next task deleted the unit files out from under two
# running processes. systemctl is-enabled then reported "not-found" and looked
# clean while is-active still said "active".
#
# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
# kicking each other off the broker.
#
# Only units that actually exist are touched, so a fresh host does not need an
# error swallowed to get through this.
- name: Stop and disable the native CAN services
ansible.builtin.systemd:
name: "{{ item.item }}"
state: stopped
enabled: false
loop: "{{ heating_can_native_units.results }}"
loop_control:
label: "{{ item.item }}"
when: item.stat.exists
# Masking is deliberately NOT used. It writes a symlink at the same path the
# next task removes, so the two cancel out -- which is half of how the original
# failed. With the unit files gone there is nothing left to start.
- name: Remove the native unit files
ansible.builtin.file:
path: "/etc/systemd/system/{{ item }}.service"
state: absent
loop:
- open3e
- e3oncan
notify: Reload systemd
# The invariant that actually matters is "no native process is on the CAN bus",
# and systemd reporting success is not the same claim. Two open3e instances
# polling one bus is the failure this whole block exists to prevent, so it is
# asserted directly rather than inferred from the calls above.
- name: Check for surviving native CAN processes
ansible.builtin.command:
argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
register: heating_can_survivors
changed_when: false
failed_when: false
- name: Assert no native CAN process survived
ansible.builtin.assert:
that:
- heating_can_survivors.stdout | default('') | length == 0
fail_msg: >-
A native open3e/e3oncan process is still running after the units were
stopped and removed:
{{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
Starting the container now would put two readers on the same CAN bus --
two open3e instances issue competing UDS requests and take over each
other's MQTT session. Kill the survivors, then re-run.
success_msg: No native CAN processes remain; safe to start the container.
- name: Remove the native venvs
ansible.builtin.file:
path: "{{ item }}"
state: absent
loop:
- "{{ edge_base_dir }}/open3e/venv"
- "{{ edge_base_dir }}/e3oncan/venv"
- "{{ edge_base_dir }}/e3oncan/src"
- "{{ edge_base_dir }}/open3e"
- "{{ edge_base_dir }}/e3oncan"
# --- Configuration (bind-mounted into the containers) ------------------------
- name: Deploy open3e devices.json
ansible.builtin.copy:
content: "{{ edge_open3e_devices | to_nice_json }}\n"
dest: "{{ heating_can_base_dir }}/devices.json"
mode: "0644"
notify: Restart heating-can
- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
ansible.builtin.copy:
src: open3e-vx3plus.py
dest: "{{ heating_can_base_dir }}/vx3plus.py"
mode: "0644"
notify: Restart heating-can
# --- Compose project ---------------------------------------------------------
- name: Deploy the broker credential env file
ansible.builtin.template:
src: heating-can.env.j2
dest: "{{ heating_can_base_dir }}/heating-can.env"
mode: "0600"
notify: Restart heating-can
- name: Deploy heating-can compose project
ansible.builtin.template:
src: docker-compose.yml.j2
dest: "{{ heating_can_base_dir }}/docker-compose.yml"
mode: "0644"
notify: Restart heating-can
- name: Pull the heating-can image
ansible.builtin.command:
cmd: docker compose pull
chdir: "{{ heating_can_base_dir }}"
changed_when: false
# Unlike the zwave and meter services there is no missing-device tolerance here:
# can0 is created by the socketcan role from a USB adapter that is part of this
# host's purpose. If it is absent the heating telemetry is down and the run
# should say so rather than pass quietly.
- name: Start heating-can services
ansible.builtin.command:
cmd: docker compose up -d
chdir: "{{ heating_can_base_dir }}"
register: heating_can_up
changed_when: >-
'Started' in heating_can_up.stderr or 'Created' in heating_can_up.stderr
or 'Started' in heating_can_up.stdout or 'Creating' in heating_can_up.stdout
@@ -1,35 +0,0 @@
# Managed by ansible (roles/heating-can).
#
# One container. Both CAN readers live in the image and are supervised inside
# it -- this file supplies configuration and nothing else. It does not know
# open3e's or E3onCAN's command line.
#
# network_mode: host is required: SocketCAN interfaces live in the host network
# namespace, so {{ edge_can_interface }} is only reachable this way, and it is
# also how the local MQTT broker is reached. CAN_RAW needs no capabilities, so
# the image runs unprivileged.
services:
heating-can:
image: {{ heating_can_image }}
container_name: heating-can
restart: unless-stopped
network_mode: host
volumes:
- ./devices.json:/config/devices.json:ro
- ./vx3plus.py:/config/vx3plus.py:ro
env_file:
# Broker credential, 0600 -- see heating-can.env.j2.
- heating-can.env
environment:
CAN_INTERFACE: "{{ edge_can_interface }}"
MQTT_HOST: "{{ edge_mqtt_host }}"
MQTT_PORT: "{{ edge_mqtt_port }}"
MQTT_TOPIC: "{{ edge_open3e_mqtt_topic }}"
MQTT_FORMAT: "{{ edge_open3e_mqtt_format }}"
E3ONCAN_MQTT_FORMAT: "{{ edge_e3oncan_mqtt_format }}"
OPEN3E_CONFIG: /config/devices.json
OPEN3E_CLIENT_ID: open3e-edge
OPEN3E_POLL_SECONDS: "{{ edge_open3e_poll_seconds }}"
OPEN3E_DATAPOINTS: "{{ edge_open3e_datapoints }}"
E3ONCAN_DEVICE: e380
E3ONCAN_GAP_SECONDS: "{{ edge_e3oncan_gap_seconds }}"
@@ -1,8 +0,0 @@
# Managed by ansible (roles/heating-can). 0600 -- carries the broker credential.
#
# Kept out of docker-compose.yml, which is 0644: the sibling roles on this host
# do the same (power-meter-sml renders its config at 0600 while its compose file
# stays world-readable), and the native unit this replaced used a 0600 env file
# for exactly this value.
MQTT_USER={{ edge_open3e_mqtt_user }}
MQTT_PASSWORD={{ edge_open3e_mqtt_password }}
@@ -4,29 +4,6 @@
keepalived_ntp_router_id: 4
keepalived_ntp_vip: "10.2.40.4"
keepalived_ntp_interface: "vmbr1.40"
# IPv6 service address for NTP (ntp.srv.renner.uno). Empty disables the v6
# instance entirely. IPv6 is the primary address family on this network, so
# this is the path that actually matters -- the v4 VIP above is legacy.
#
# Held by keepalived rather than assigned statically by networkd so that it is
# gated on check_ntp: present only while this host is fit to serve time, and
# withdrawn the moment it is not. A statically configured address is carried
# unconditionally from boot, which is how edge-1 came to answer NTP queries on
# 2026-08-04 while its clock was ~16 days in the past.
#
# Needs its own virtual_router_id: it is a distinct VRRP instance, and a VRID
# collision with the IPv4 instance on the same interface would have the two
# fighting over a single election.
keepalived_ntp_vip6: ""
keepalived_ntp_router_id6: 6
# Link-local address for the IPv6 VRRP instance, carried as its FIRST
# virtual_ipaddress. RFC 5798 gives an IPv6 virtual router a link-local primary,
# and keepalived warns "the first IPv6 VIP address should be link local" without
# one. Chosen to mirror the service address rather than derived from the virtual
# MAC, since no real NIC generates fe80::123 so there is nothing to collide with.
keepalived_ntp_vip6_linklocal: "fe80::123/64"
# NOTE: plaintext VRRP auth, unvaulted. Carried over as-is from the original
# role; VRRP PASS auth is weak regardless (sent in the clear on the segment),
# so this is a documented weakness rather than a secret worth vaulting.
@@ -41,17 +18,6 @@ keepalived_ntp_check_script: "/usr/local/bin/check-ntp-health.sh"
keepalived_ntp_check_interval: 2
keepalived_ntp_check_weight: -20
# Maximum |system clock offset| in seconds before the host refuses to hold the
# VIP. chronyd reports this as "System time : <n> seconds slow/fast of NTP
# time". Anything above this means the host knows its clock is wrong and must
# not serve time to clients -- see the reasoning in check-ntp-health.sh.j2.
#
# A serving NTP host should be within milliseconds of real time, so 1s is
# already several orders of magnitude beyond normal. It is deliberately not
# tighter: a brief excursion at boot, before makestep lands, should not flap
# the VIP (fall 2 at a 2s interval also requires the failure to persist).
keepalived_ntp_max_offset: 1.0
# sysctl applied for VRRP. Default preserves the PVE hosts' existing behaviour
# exactly. ip_forward is NOT appropriate everywhere -- on a host with legs in
# two VLANs it turns the box into a router between them, bypassing the gateway's
@@ -1,39 +1,22 @@
#!/bin/bash
# Keepalived State Change Notification Script
#
# Two VRRP instances call this: NTP_VIP (IPv4, legacy) passes MASTER/BACKUP/
# FAULT, and NTP_VIP6 (IPv6, the primary path) passes the same suffixed with
# '6'. They are logged distinctly because the two can legitimately diverge --
# a host can hold one family and not the other -- and a log line that does not
# say which family it means is useless during an incident.
TYPE=$1
NAME=$(hostname)
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
log() {
echo "$TIMESTAMP: $NAME $1" >> /var/log/keepalived-state.log
logger -t keepalived "$NAME $1"
}
case $TYPE in
MASTER)
log "transitioned to MASTER state (NTP, IPv4)"
echo "$TIMESTAMP: $NAME transitioned to MASTER state (NTP)" >> /var/log/keepalived-state.log
logger -t keepalived "$NAME is now NTP MASTER"
;;
BACKUP)
log "transitioned to BACKUP state (NTP, IPv4)"
echo "$TIMESTAMP: $NAME transitioned to BACKUP state (NTP)" >> /var/log/keepalived-state.log
logger -t keepalived "$NAME is now NTP BACKUP"
;;
FAULT)
log "transitioned to FAULT state (NTP, IPv4)"
;;
MASTER6)
log "transitioned to MASTER state (NTP, IPv6) - now serving ntp.srv.renner.uno"
;;
BACKUP6)
log "transitioned to BACKUP state (NTP, IPv6) - released ntp.srv.renner.uno"
;;
FAULT6)
log "transitioned to FAULT state (NTP, IPv6) - withdrew ntp.srv.renner.uno, unfit to serve time"
echo "$TIMESTAMP: $NAME transitioned to FAULT state (NTP)" >> /var/log/keepalived-state.log
logger -t keepalived "$NAME is now in NTP FAULT state"
;;
*)
echo "$TIMESTAMP: $NAME unknown state: $TYPE" >> /var/log/keepalived-state.log
-115
View File
@@ -35,121 +35,6 @@
state: present
when: ansible_facts['os_family'] == 'Archlinux'
# --- Verify nothing else is assigning the IPv6 service address ---------------
#
# The removal itself lives in withdraw-static.yml, run as a separate fleet-wide
# play before any host reaches this role (see install-ntp-ha.yml). It has to
# complete everywhere first: DAD is enabled on the VRRP address, so a static
# copy on one host and a VRRP-managed copy on another cannot coexist -- the
# kernel marks the newcomer `dadfailed` and its master serves nothing.
#
# What remains here are the guards. They run on every execution, not just the
# migration, and fail the play if any static assignment survives -- which would
# both defeat the health gating (a static copy does not disappear when
# keepalived decides the host is unfit) and break the VRRP copy via DAD.
#
# The config file is what gets checked, not the running address, and that is
# what makes it safe to run every time: keepalived never writes to these files,
# so anything found is by definition static, whereas the live address is
# legitimately present on whichever host is master.
#
# Prefix-agnostic and \b-anchored on purpose. Matching the full
# address/prefixlen would fail open against a hand-written assignment using any
# other mask.
- name: Detect any remaining static assignment of the service address (ifupdown)
ansible.builtin.command:
argv:
- grep
- -rlE
- "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
- /etc/network/interfaces
- /etc/network/interfaces.d/
register: keepalived_ntp_static6_ifupdown
changed_when: false
failed_when: false
when:
- keepalived_ntp_vip6 | length > 0
- ansible_facts['os_family'] == 'Debian'
- name: Assert ifupdown is not also assigning the service address
ansible.builtin.assert:
that:
- keepalived_ntp_static6_ifupdown.stdout | default('') | length == 0
fail_msg: >-
{{ keepalived_ntp_vip6 }} is still assigned statically by ifupdown on
this host
({{ keepalived_ntp_static6_ifupdown.stdout_lines | default([]) | join(', ') }}).
The automatic removal only matches the exact form
`up ip -6 addr replace {{ keepalived_ntp_vip6 }} dev <if> nodad`, so a
different prefix length or layout is left in place. Handing the address
to keepalived while ifupdown also assigns it means it would NOT be
withdrawn when check_ntp fails -- the host would keep serving time on an
address VRRP believes it has released. Remove the assignment from the
file(s) above, then re-run.
success_msg: >-
No static ifupdown assignment of {{ keepalived_ntp_vip6 }}; safe for
keepalived to own it.
when:
- keepalived_ntp_vip6 | length > 0
- ansible_facts['os_family'] == 'Debian'
# On Arch (edge-1) the static assignment lives in a systemd-networkd file owned
# by the edge-vlan role, so this role must not rewrite it -- but it must not
# quietly proceed either. install-edge-host.yml is a separate play with no
# ordering relationship to this one, so nothing guarantees it has been re-run
# to drop the [Address] block. If it has not, edge-1 would end up holding
# fd40::123 statically AND running a VRRP instance for the same address:
# keepalived would believe it had withdrawn the address while networkd kept it
# assigned, and the host would go on answering NTP with a clock the health
# check had already rejected.
#
# That is the precise failure of 2026-08-04, on the host it happened to, so it
# fails fast rather than being left to inspection.
# Matched on the bare address, not the full address/prefixlen, and anchored
# with \b so it cannot also hit a longer address sharing the prefix (fd40::1234).
#
# Searching for the literal "fd40::123/128" would fail open: a hand-written
# [Address] using any other mask -- /64, say -- would not match, the assert
# below would pass, and keepalived would be handed an address networkd is
# still assigning. Since this check is detect-only, with no delete afterwards
# to catch the mistake, a miss here is silent. The Debian path does not share
# the problem: `ip -6 addr show ... to <prefix>` matches by range regardless of
# the assigned prefix length, and its delete is re-verified by assert.
- name: Detect a static assignment of the service address (systemd-networkd)
ansible.builtin.command:
argv:
- grep
- -rlE
- "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
- /etc/systemd/network/
register: keepalived_ntp_static6_networkd
changed_when: false
failed_when: false
when:
- keepalived_ntp_vip6 | length > 0
- ansible_facts['os_family'] == 'Archlinux'
- name: Assert networkd is not also assigning the service address
ansible.builtin.assert:
that:
- keepalived_ntp_static6_networkd.stdout | default('') | length == 0
fail_msg: >-
{{ keepalived_ntp_vip6 }} is still assigned statically by
systemd-networkd on this host
({{ keepalived_ntp_static6_networkd.stdout_lines | default([]) | join(', ') }}).
Handing the address to keepalived while networkd also assigns it means
it would NOT be withdrawn when check_ntp fails -- the host would keep
serving time on an address VRRP believes it has released.
Re-apply the edge-vlan role first, whose template no longer carries the
[Address] block:
ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
success_msg: >-
No static networkd assignment of {{ keepalived_ntp_vip6 }}; safe for
keepalived to own it.
when:
- keepalived_ntp_vip6 | length > 0
- ansible_facts['os_family'] == 'Archlinux'
- name: Create keepalived configuration
ansible.builtin.template:
src: keepalived.conf.j2
@@ -1,134 +0,0 @@
---
# Withdraw a static assignment of the IPv6 NTP service address.
#
# Included by install-ntp-ha.yml as a FIRST play across the whole ntp_cluster,
# before any host is given the NTP_VIP6 instance. That ordering is the point of
# splitting it out.
#
# DAD is enabled on the VRRP address (see keepalived.conf.j2), so a static copy
# on any host and a VRRP-managed copy on another cannot coexist: the kernel sees
# the duplicate and marks the newcomer `dadfailed`, leaving a master that
# believes it is serving while carrying an unusable address. Observed on edge-1
# on 2026-08-07, where keepalived reported MASTER while the address sat
# `dadfailed deprecated tentative` because three PVE nodes still held it
# statically.
#
# Doing the withdrawal per host inside the main role could not avoid that: with
# serial: 1 the first host would claim the address via VRRP while the rest were
# still static. Fleet-wide first, then claim.
#
# fd40::123 was assigned by hand on all four servers, in two different ways,
# none of it in version control:
#
# PVE nodes an `up ip -6 addr replace ... nodad` hook in
# /etc/network/interfaces
# edge-1 an [Address] block in /etc/systemd/network/, owned by the
# edge-vlan role and removed by re-applying it -- not touched
# here, since this role must not rewrite another role's file
#
# There is a service gap between this play and the next, while no host holds
# the address. It is bounded by the run time of one play and clients simply
# retry; the alternative is the dadfailed state above, which does not resolve
# on its own.
- name: Remove static assignment of the NTP service address (ifupdown)
ansible.builtin.lineinfile:
path: /etc/network/interfaces
regexp: '^\s*up\s+ip\s+-6\s+addr\s+(replace|add)\s+{{ keepalived_ntp_vip6 | regex_escape }}\b'
state: absent
when:
- keepalived_ntp_vip6 | length > 0
- ansible_facts['os_family'] == 'Debian'
register: keepalived_ntp_static6_removed
# Everything below is gated on the config line having just been removed, so the
# runtime delete happens once -- on the migration -- and never again.
#
# The gate is load-bearing. Once this has landed, the address is legitimately
# present on whichever host is the elected v6 master, placed there by
# keepalived, and a delete cannot tell that apart from a static assignment (the
# address looks identical either way). Ungated, every later run would delete the
# master's address and restart keepalived, dropping the service address and
# forcing a re-election each time -- the flapping this whole change exists to
# prevent.
- name: Check whether the service address is still assigned statically
ansible.builtin.command:
argv:
- ip
- -6
- addr
- show
- dev
- "{{ keepalived_ntp_interface }}"
- to
- "{{ keepalived_ntp_vip6 }}"
register: keepalived_ntp_vip6_present
changed_when: false
failed_when: false
when:
- keepalived_ntp_vip6 | length > 0
- keepalived_ntp_static6_removed.changed | default(false)
# failed_when: false because the desired end state is "address absent". An
# `ip addr del` against an address that has already gone is the outcome this
# task wants, not a reason to abort the play for that host.
- name: Withdraw the statically assigned service address
ansible.builtin.command:
argv:
- ip
- -6
- addr
- del
- "{{ keepalived_ntp_vip6 }}"
- dev
- "{{ keepalived_ntp_interface }}"
when:
- keepalived_ntp_vip6 | length > 0
- keepalived_ntp_static6_removed.changed | default(false)
- keepalived_ntp_vip6_present.stdout | default('') | length > 0
failed_when: false
# The withdrawal gets exactly one attempt -- the gate that makes it idempotent
# also means it never retries -- and the delete swallows its own errors. So the
# result is asserted rather than assumed.
#
# `ip addr del` matches on address AND prefix length: if a host's static
# assignment used a different mask or interface than ntp_vip6 /
# keepalived_ntp_interface, the delete is a silent no-op and the address stays
# outside VRRP control permanently, with the play reporting success.
- name: Confirm the static service address was actually withdrawn
ansible.builtin.command:
argv:
- ip
- -6
- addr
- show
- dev
- "{{ keepalived_ntp_interface }}"
- to
- "{{ keepalived_ntp_vip6 }}"
register: keepalived_ntp_vip6_after
changed_when: false
failed_when: false
when:
- keepalived_ntp_vip6 | length > 0
- keepalived_ntp_static6_removed.changed | default(false)
- name: Assert the service address is no longer assigned outside keepalived
ansible.builtin.assert:
that:
- keepalived_ntp_vip6_after.stdout | default('') | length == 0
fail_msg: >-
{{ keepalived_ntp_vip6 }} is still assigned on
{{ keepalived_ntp_interface }} after the static assignment was removed.
`ip -6 addr del` matches address AND prefix length, so this usually means
the on-host assignment differs from ntp_vip6 or uses another interface.
Leaving it would both keep the address outside VRRP control and make the
VRRP-managed copy fail DAD elsewhere. Inspect with
`ip -6 addr show dev {{ keepalived_ntp_interface }}` and
`grep -rn {{ keepalived_ntp_vip6 }} /etc/network/ /etc/systemd/network/`.
success_msg: >-
{{ keepalived_ntp_vip6 }} withdrawn; safe for keepalived to claim it.
when:
- keepalived_ntp_vip6 | length > 0
- keepalived_ntp_static6_removed.changed | default(false)
@@ -6,80 +6,23 @@
# The chrony unit is named 'chrony' on Debian and 'chronyd' on Arch. This is
# templated rather than hardcoded: a wrong name here fails every check, which
# parks the host in permanent FAULT so it never takes the VIP.
#
# This check must answer "is it safe for clients to take time from this host",
# NOT merely "is chronyd alive". Those came apart on 2026-08-04: edge-1's dead
# CMOS battery left its clock ~16 days in the past, chronyd was running,
# answering queries and listening on 123, and the `local stratum` directive made
# it advertise itself to clients as a valid reference. Every liveness check
# passed for ~18h while the host stood ready to serve a clock from July. The
# earlier version of this script ran `chronyc tracking` and discarded its
# output -- the one command that knew the answer.
#
# Deliberate distinction, because these must not be conflated:
# * "provably wrong" -> chronyd HAS upstream measurements and they say the
# clock is far off. Fail: drop the VIP so a host with a
# sane clock serves instead.
# * "no upstream yet" -> nothing reachable / not yet measured. Also fail, but
# note that when the WAN is down every host fails
# together, they all lose the same weight, relative
# priority is unchanged and the VIP stays put. So the
# local-reference fallback still works; what this
# prevents is one host with a broken clock winning.
LOG=/var/log/keepalived-ntp-health.log
MAX_OFFSET={{ keepalived_ntp_max_offset }}
fail() {
echo "$(date): $1" >> "$LOG"
exit 1
}
# Check 1: Is chrony service running?
if ! systemctl is-active --quiet {{ chrony_service_name }}; then
fail "{{ chrony_service_name }} service not active"
echo "$(date): {{ chrony_service_name }} service not active" >> /var/log/keepalived-ntp-health.log
exit 1
fi
# Check 2: Is chrony responding, and what does it actually say?
# Captured, not discarded -- the offset below is the whole point of this script.
if ! TRACKING=$(chronyc tracking 2>/dev/null); then
fail "Chrony not responding to tracking query"
# Check 2: Is chrony synchronized or at least running?
if ! chronyc tracking >/dev/null 2>&1; then
echo "$(date): Chrony not responding to tracking query" >> /var/log/keepalived-ntp-health.log
exit 1
fi
# Check 3: Is NTP serving clients (port 123)?
if ! ss -lun | grep -q ':123 '; then
fail "NTP port 123 not listening"
fi
# Check 4: Is the clock provably wrong?
#
# "System time : <n> seconds (slow|fast) of NTP time" is chronyd's own estimate
# of how far the system clock is from real time, derived from upstream
# measurements. On edge-1 during the incident this line read
# "1376235.375000000 seconds slow of NTP time" while every other check passed.
OFFSET=$(echo "$TRACKING" | awk '/^System time/ {print $4}')
if [ -n "$OFFSET" ]; then
if awk -v o="$OFFSET" -v m="$MAX_OFFSET" 'BEGIN {exit !(o > m)}'; then
fail "System clock off by ${OFFSET}s (max ${MAX_OFFSET}s) - refusing to serve time"
fi
fi
# Check 5: Has chronyd ever validated its clock against a real source?
#
# With `local stratum N` configured, chronyd advertises itself as a usable
# reference even when it has never synchronised to anything -- the upstream
# chrony documentation is explicit that the local reference activates
# immediately in that case. So a host that has never reached an upstream still
# looks authoritative to clients. Refuse the VIP until a real source has been
# selected.
#
# `chronyc sources` marks the selected source with '*' (best) or '+' (combined)
# in the second column. Neither present means nothing upstream is accepted.
if ! SOURCES=$(chronyc sources 2>/dev/null); then
fail "Chrony not responding to sources query"
fi
if ! echo "$SOURCES" | grep -qE '^.\*|^.\+'; then
fail "No upstream source selected - clock not validated, refusing to serve time"
echo "$(date): NTP port 123 not listening" >> /var/log/keepalived-ntp-health.log
exit 1
fi
# All checks passed
@@ -39,72 +39,3 @@ vrrp_instance NTP_VIP {
notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP"
notify_fault "/usr/local/bin/keepalived-notify.sh FAULT"
}
{% if keepalived_ntp_vip6 %}
# IPv6 service address for NTP (ntp.srv.renner.uno). IPv6 is the primary
# address family here, so this instance -- not the IPv4 one above -- is the
# path that matters.
#
# A separate instance is mandatory, not stylistic: keepalived requires that
# "all addresses in virtual_ipaddress must be of the same family", so the v6
# address cannot be added to NTP_VIP.
#
# No authentication block: VRRPv3 removed authentication (RFC 5798), and
# keepalived always uses version 3 for IPv6 instances. Configuring auth here
# is rejected rather than ignored.
#
# The first address is link-local, and must be. RFC 5798 gives an IPv6 virtual
# router a link-local address as its primary, and keepalived warns
# "the first IPv6 VIP address should be link local" otherwise. It carries on,
# but the instance is then sourcing VRRPv3 from something the protocol does not
# expect.
#
# DAD is deliberately left ON (no -nodad). The flag was inherited from the
# previous design, where every NTP server carried fd40::123 simultaneously and
# duplicate detection had to be suppressed for that to work at all. Under
# single-master VRRP the opposite is true: exactly one host should hold the
# address, so a duplicate is a genuine fault and detecting it is the point.
#
# This does mean the migration cannot leave a static copy anywhere while a
# VRRP-managed one exists -- DAD sees the two and marks the newcomer
# `dadfailed`, leaving a master that believes it is serving while carrying an
# unusable address. Observed on edge-1 on 2026-08-07. install-ntp-ha.yml
# therefore withdraws every static assignment fleet-wide in a first play
# before any host configures this instance.
#
# track_script is what makes this health-gated: the address is only held while
# check_ntp passes, and is withdrawn as soon as it fails. On withdrawal
# keepalived sends unsolicited Neighbour Advertisements so client neighbour
# caches are updated immediately -- a statically assigned address cannot do
# this, and clients would keep sending to the dead holder until their own
# neighbour unreachability detection expired.
vrrp_instance NTP_VIP6 {
state {{ keepalived_ntp_state }}
interface {{ keepalived_ntp_interface }}
virtual_router_id {{ keepalived_ntp_router_id6 }}
priority {{ keepalived_ntp_priority }}
advert_int 1
virtual_ipaddress {
{{ keepalived_ntp_vip6_linklocal }} dev {{ keepalived_ntp_interface }}
{{ keepalived_ntp_vip6 }} dev {{ keepalived_ntp_interface }}
}
# weight 0 overrides the script's own weight, and the distinction decides
# whether this works at all. A non-zero weight only SUBTRACTS from priority
# on failure -- so the last remaining host, having no one to lose the
# election to, stays MASTER and keeps serving time with a clock it has
# already reported as wrong. Weight 0 instead transitions the instance to
# FAULT after <fall> failures, which releases the address unconditionally.
#
# That is the required behaviour here: a host unfit to serve time must stop
# answering, whether or not a replacement exists.
track_script {
check_ntp weight 0
}
notify_master "/usr/local/bin/keepalived-notify.sh MASTER6"
notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP6"
notify_fault "/usr/local/bin/keepalived-notify.sh FAULT6"
}
{% endif %}
@@ -1,12 +0,0 @@
---
# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
# mosquitto on loopback, which bridges to the central cluster broker over TLS
# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
# blips. The local broker writes nothing to the eMMC (persistence off, logs to
# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
# Upstream cluster broker the local bridge connects to (TLS; the public
# Let's Encrypt cert is validated against the system CA bundle).
edge_mqtt_cluster_host: mqtt.renner.uno
edge_mqtt_cluster_tls_port: 8883
edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
@@ -1,7 +0,0 @@
---
# The MQTT clients (zwave-js, sml2mqtt, open3e, e3oncan) reconnect on their
# own once the local broker is back.
- name: Restart mosquitto
ansible.builtin.service:
name: mosquitto
state: restarted
-27
View File
@@ -1,27 +0,0 @@
---
# Local MQTT bridge broker.
#
# Edge clients (open3e, e3oncan, sml2mqtt) publish here over loopback; this
# broker bridges to the cluster broker over TLS, because those clients have no
# native MQTTS. Persistence is off and logging goes to stdout so it writes
# nothing to local flash.
- name: Install mosquitto
ansible.builtin.package:
name: mosquitto
state: present
- name: Deploy local mosquitto bridge config
ansible.builtin.template:
src: edge-mqtt-bridge.conf.j2
dest: /etc/mosquitto/mosquitto.conf
owner: root
group: mosquitto
mode: "0640" # carries the cluster bridge credentials
notify: Restart mosquitto
- name: Enable local mosquitto bridge
ansible.builtin.systemd:
name: mosquitto
enabled: true
state: started
@@ -1,23 +0,0 @@
---
# Power-meter reader: sml2mqtt against the FTDI serial meter head.
# Service root. Kept per-role so the role is usable on its own.
edge_base_dir: /opt/edge-host
# Own compose project directory. Unchanged from the path used under the
# previous shared project.
sml_base_dir: "{{ edge_base_dir }}/sml2mqtt"
# The compose file that used to hold both this service and zwave-js. Removed
# once, so the fixed container_name is released for the new projects.
edge_legacy_compose: "{{ edge_base_dir }}/docker-compose.yml"
# Stable by-id symlink -- see the zwave-gateway role.
edge_stromzaehler_device: /dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_DK0E1GP5-if00-port0
# Hand-pinned on purpose: critical meter bridge, deliberately excluded from
# image automation.
edge_sml2mqtt_image: spacemanspiff2007/sml2mqtt:3.6.0
# Secret from inventory/group_vars/edge_host/vault.yml:
# edge_sml2mqtt_mqtt_password
@@ -1,6 +0,0 @@
---
- name: Restart sml2mqtt
ansible.builtin.command:
cmd: docker compose up -d --force-recreate
chdir: "{{ sml_base_dir }}"
changed_when: true
@@ -1,55 +0,0 @@
---
# Power-meter reader: sml2mqtt against the FTDI serial meter head.
#
# Its own compose project. It shared a compose file with the Z-Wave gateway,
# which meant a change to either recreated both -- two unrelated services bound
# to different USB devices. Data paths are unchanged.
- name: Create sml2mqtt directory
ansible.builtin.file:
path: "{{ sml_base_dir }}"
state: directory
mode: "0755"
# sml2mqtt reads only a config file and has no env override for the password,
# so the rendered config carries it -- 0600, root-owned, same trust boundary as
# the k8s init-container approach it replaced.
- name: Deploy sml2mqtt config
ansible.builtin.template:
src: sml2mqtt-config.yml.j2
dest: "{{ sml_base_dir }}/config.yml"
mode: "0600"
notify: Restart sml2mqtt
- name: Deploy sml2mqtt compose project
ansible.builtin.template:
src: docker-compose.yml.j2
dest: "{{ sml_base_dir }}/docker-compose.yml"
mode: "0644"
notify: Restart sml2mqtt
- name: Pull sml2mqtt image
ansible.builtin.command:
cmd: docker compose pull
chdir: "{{ sml_base_dir }}"
changed_when: false
# Tolerates a missing USB device, as for the Z-Wave gateway.
- name: Start sml2mqtt
ansible.builtin.command:
cmd: docker compose up -d
chdir: "{{ sml_base_dir }}"
register: sml_compose_up
failed_when: >-
sml_compose_up.rc != 0
and 'error gathering device information' not in sml_compose_up.stderr
changed_when: >-
'Started' in sml_compose_up.stderr or 'Created' in sml_compose_up.stderr
or 'Started' in sml_compose_up.stdout or 'Creating' in sml_compose_up.stdout
- name: Warn if sml2mqtt is waiting for its USB device
ansible.builtin.debug:
msg: >-
sml2mqtt could not start -- the FTDI meter head is not present:
{{ sml_compose_up.stderr_lines | select('search', 'device') | list }}
when: sml_compose_up.rc != 0
@@ -1,21 +0,0 @@
# Managed by ansible (roles/power-meter-sml).
#
# Its own compose project -- see the role tasks for why it no longer shares a
# file with the Z-Wave gateway.
#
# Outbound-only (serial in, MQTT out). Host networking sidesteps the docker
# NAT66 gap: the broker LB is IPv6-first and bridge-network v6 SYNs leave
# unmasqueraded and die. The host stack reaches it fine (open3e proves it
# continuously).
services:
sml2mqtt:
image: {{ edge_sml2mqtt_image }}
container_name: sml2mqtt
restart: unless-stopped
network_mode: host
# Serial exclusivity: this is the ONLY reader of the meter port --
# HA's edl21 integration stays disabled.
devices:
- "{{ edge_stromzaehler_device }}:/dev/stromzaehler"
volumes:
- ./config.yml:/sml2mqtt/config.yml:ro
@@ -7,19 +7,19 @@
# PVE cluster nodes - need direct scraping (no Alloy agent)
- targets:
- 'jupiter.renner.uno:9100'
- '10.2.40.201:9100'
labels:
job: 'node'
group: 'pve_cluster'
hostname: 'jupiter'
- targets:
- 'neptune.renner.uno:9100'
- '10.2.40.202:9100'
labels:
job: 'node'
group: 'pve_cluster'
hostname: 'neptune'
- targets:
- 'saturn.renner.uno:9100'
- '10.2.40.203:9100'
labels:
job: 'node'
group: 'pve_cluster'
@@ -51,9 +51,9 @@ scrape_configs:
scrape_timeout: 55s
static_configs:
- targets:
- 'jupiter.renner.uno'
- 'neptune.renner.uno'
- 'saturn.renner.uno'
- '10.2.40.201' # jupiter
- '10.2.40.202' # neptune
- '10.2.40.203' # saturn
metrics_path: /pve
params:
module: [default]
@@ -132,29 +132,28 @@ scrape_configs:
refresh_interval: 60s
# Chrony Exporter - NTP metrics from PVE cluster
# One static_config per host so each carries its own hostname label. The
# previous form listed all three targets together and recovered the hostname
# in relabel_configs by regex-matching __address__ against the node's IP --
# which silently coupled the labels to the addressing. Changing a target
# would have left the regex unmatched and the hostname label simply absent,
# with the scrape still succeeding.
- job_name: 'chrony'
static_configs:
- targets: ['jupiter.renner.uno:9123']
- targets:
- '10.2.40.201:9123'
- '10.2.40.202:9123'
- '10.2.40.203:9123'
labels:
group: 'ntp'
type: 'time-server'
hostname: 'jupiter'
- targets: ['neptune.renner.uno:9123']
labels:
group: 'ntp'
type: 'time-server'
hostname: 'neptune'
- targets: ['saturn.renner.uno:9123']
labels:
group: 'ntp'
type: 'time-server'
hostname: 'saturn'
relabel_configs:
- source_labels: [__address__]
regex: '10\.2\.40\.201:.*'
target_label: hostname
replacement: 'jupiter'
- source_labels: [__address__]
regex: '10\.2\.40\.202:.*'
target_label: hostname
replacement: 'neptune'
- source_labels: [__address__]
regex: '10\.2\.40\.203:.*'
target_label: hostname
replacement: 'saturn'
# Immich photo management metrics
- job_name: 'immich'
@@ -1,5 +0,0 @@
---
# SocketCAN interface. Consumed by heating-can.
# --- CAN / open3e (Viessmann E3 external bus) ---
edge_can_interface: can0
edge_can_bitrate: 250000
@@ -1,7 +0,0 @@
---
# reload alone only re-reads config files; existing links keep their old state
# until reconfigured.
- name: Reload networkd
ansible.builtin.shell:
cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
changed_when: true
-20
View File
@@ -1,20 +0,0 @@
---
# SocketCAN interface, owned by systemd-networkd.
#
# gs_usb is an in-tree module and autoloads when the adapter enumerates;
# networkd then matches the interface and applies bitrate + up. Separate from
# the services that consume the bus (heating-can, e380-can-meter) because the
# interface is host network state shared by both.
- name: Configure can0 via systemd-networkd
ansible.builtin.template:
src: 80-can0.network.j2
dest: /etc/systemd/network/80-can0.network
mode: "0644"
notify: Reload networkd
- name: Enable systemd-networkd
ansible.builtin.service:
name: systemd-networkd
state: started
enabled: true
@@ -1,24 +0,0 @@
---
# Z-Wave gateway: zwave-js-ui against the ZWA-2 USB radio.
# Service root. Kept per-role so the role is usable on its own.
edge_base_dir: /opt/edge-host
edge_tz: Europe/Berlin
# Own compose project directory. Unchanged from the path used under the
# previous shared project, so the store is not migrated.
zwave_base_dir: "{{ edge_base_dir }}/zwave-js"
# Stable by-id symlink: the name travels with the device, not the host, so it
# survives moving the stick between machines.
edge_zwave_device: /dev/serial/by-id/usb-Nabu_Casa_ZWA-2_80B54EE15720-if00
edge_zwavejs_image: ghcr.io/zwave-js/zwave-js-ui:11.21.1
# Alpine version the zwave-js-ui image is built on -- the source for the musl
# libeatmydata.so preloaded into that container. Keep in sync if the image
# bumps its Alpine major (check: docker exec zwave-js cat /etc/os-release).
edge_zwave_alpine_base: alpine:3.22
# Secret from inventory/group_vars/edge_host/vault.yml:
# edge_zwavejs_session_secret
@@ -1,6 +0,0 @@
---
- name: Restart zwave-js
ansible.builtin.command:
cmd: docker compose up -d --force-recreate
chdir: "{{ zwave_base_dir }}"
changed_when: true
@@ -1,95 +0,0 @@
---
# Z-Wave gateway: zwave-js-ui against the ZWA-2 USB radio.
#
# Its own compose project, in its own directory, so the Z-Wave gateway and the
# power-meter reader are not coupled through a shared compose file. Data paths
# are unchanged -- the store stays at {{ zwave_base_dir }}/store -- so this is a
# project boundary change, not a data migration.
- name: Create zwave-js directories
ansible.builtin.file:
path: "{{ item }}"
state: directory
mode: "0755"
loop:
- "{{ zwave_base_dir }}"
- "{{ zwave_base_dir }}/store"
- name: Extract musl libeatmydata.so (matches the zwave-js-ui Alpine base)
ansible.builtin.shell:
cmd: |
set -euo pipefail
tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
docker run --rm --network=host {{ edge_zwave_alpine_base }} \
sh -c 'apk add --no-cache libeatmydata >/dev/null 2>&1 && cat /usr/lib/libeatmydata.so' > "$tmp"
test "$(stat -c%s "$tmp")" -ge 8000 # sanity: real .so is ~14 KB, guards against a failed/empty pull
install -m 0644 "$tmp" {{ zwave_base_dir }}/libeatmydata.so
creates: "{{ zwave_base_dir }}/libeatmydata.so"
notify: Restart zwave-js
# Because fsync is neutralized, a hard power-loss during the (rare) settings
# write could truncate settings.json — which holds the Z-Wave network keys.
# Keep a validated rolling backup on the eMMC so the keys stay recoverable.
- name: Install zwave-js settings.json backup script
ansible.builtin.template:
src: edge-zwave-settings-backup.j2
dest: /usr/local/bin/edge-zwave-settings-backup
mode: "0755"
- name: Install zwave-js settings backup service + timer
ansible.builtin.copy:
src: "{{ item }}"
dest: "/etc/systemd/system/{{ item }}"
mode: "0644"
loop:
- edge-zwave-settings-backup.service
- edge-zwave-settings-backup.timer
- name: Enable zwave-js settings backup timer
ansible.builtin.systemd:
name: edge-zwave-settings-backup.timer
enabled: true
state: started
daemon_reload: true
- name: Deploy zwave-js env (session secret)
ansible.builtin.template:
src: zwavejs.env.j2
dest: "{{ zwave_base_dir }}/zwavejs.env"
mode: "0600"
notify: Restart zwave-js
- name: Deploy zwave-js compose project
ansible.builtin.template:
src: docker-compose.yml.j2
dest: "{{ zwave_base_dir }}/docker-compose.yml"
mode: "0644"
notify: Restart zwave-js
- name: Pull zwave-js image
ansible.builtin.command:
cmd: docker compose pull
chdir: "{{ zwave_base_dir }}"
changed_when: false
# Tolerates a missing USB device: if the ZWA-2 is absent the container cannot
# start, and that must not fail the run for everything else on the host.
- name: Start zwave-js
ansible.builtin.command:
cmd: docker compose up -d
chdir: "{{ zwave_base_dir }}"
register: zwave_compose_up
failed_when: >-
zwave_compose_up.rc != 0
and 'error gathering device information' not in zwave_compose_up.stderr
changed_when: >-
'Started' in zwave_compose_up.stderr or 'Created' in zwave_compose_up.stderr
or 'Started' in zwave_compose_up.stdout or 'Creating' in zwave_compose_up.stdout
- name: Warn if zwave-js is waiting for its USB device
ansible.builtin.debug:
msg: >-
zwave-js could not start -- the ZWA-2 radio is not present:
{{ zwave_compose_up.stderr_lines | select('search', 'device') | list }}
when: zwave_compose_up.rc != 0
@@ -1,39 +0,0 @@
# Managed by ansible (roles/zwave-gateway).
#
# Its own compose project. zwave-js and the power-meter reader used to share
# one file under {{ edge_base_dir | default('/opt/edge-host') }}; they are
# unrelated services bound to different USB devices, so a change to one had no
# business recreating the other.
#
# No `privileged` needed: docker's --device handles the device-cgroup grant
# that hostPath in k8s could not (that was the only reason the pod ran
# privileged).
#
# Host networking: real host addresses (v4+v6), no bridge, no NAT. The broker
# LB is IPv6-first and docker bridge NAT66 is a tarpit (unmasqueraded SYNs die
# silently); the host stack just works.
services:
zwave-js:
image: {{ edge_zwavejs_image }}
container_name: zwave-js
restart: unless-stopped
stop_signal: SIGINT
tty: true
environment:
- TZ={{ edge_tz }}
# Batch the value-DB fsync storm to spare the eMMC (see the edge-flash-wear
# role). Neutralizes fsync only for this container; the WS/MQTT value
# stream to HA and cache persistence are unchanged.
- LD_PRELOAD=/usr/lib/libeatmydata.so
env_file:
- zwavejs.env
devices:
- "{{ edge_zwave_device }}:/dev/zwave"
volumes:
# musl libeatmydata.so extracted from the image's Alpine base (ro)
- ./libeatmydata.so:/usr/lib/libeatmydata.so:ro
# Persistent store: settings.json, device DB, network keys. Path is
# unchanged from the previous shared project, so nothing is migrated.
- ./store:/usr/src/app/store
# Binds 3000 (Z-Wave JS websocket for HA) and 8091 (web UI) on the host.
network_mode: host
-23
View File
@@ -78,29 +78,6 @@ For each, find the **current pin in the repo** (Read/Glob/Grep), then look up th
6. **Ansible roles & OpenTofu providers** — `ansible/requirements.yml` (Galaxy
roles/collections) and `opentofu/providers.tf` + `opentofu/.terraform.lock.hcl`.
Report any with a newer published version.
7. **Edge heating CAN services** — open3e (Vitocal heat pump, Vitocharge VX3)
and E3onCAN (E380 grid energy meter), shipped as one image built from
`ci/images/heating-can/Containerfile` and deployed by
`ansible/roles/heating-can`. Nothing else tracks them. They read and (for two
datapoints) write the heat pump over its CAN bus, so silent drift here
matters more than most.
Both upstream pins are `ARG`s in the Containerfile -- `OPEN3E_REF` and
`E3ONCAN_REF` -- not Ansible variables. Read them there, and note the
awkward bits so normal states are not reported as problems:
- open3e publishes **no tags at all** (its tags API returns an empty array),
so compare `OPEN3E_REF` against the default branch:
`https://api.github.com/repos/open3e/open3e/commits?per_page=1`. Report how
many commits behind and the dates. "No release found" is normal here.
- E3onCAN tags, but sparsely, and `E3ONCAN_REF` is an untagged commit later
than the newest tag. Compare against both
`https://api.github.com/repos/MyHomeMyData/E3onCAN/tags` and the default
branch, and say which the pin sits between.
The deployed image tag is `heating_can_image` in
`ansible/roles/heating-can/defaults/main.yml`. It is `:latest` today, so the
node follows whatever CI last built -- the pins above are what actually fix
the versions, and bumping them is what requires a rebuild.
## Method & rigor
-122
View File
@@ -1,122 +0,0 @@
# Heating system over the Viessmann E3 CAN bus: open3e (Vitocal heat pump and
# Vitocharge VX3) and E3onCAN (E380 grid energy meter).
#
# One image, two services. They read the same physical installation over the
# same bus and are versioned together, so they are one artifact -- but they run
# as two containers from this image so either can crash and restart on its own
# without a supervisor as PID 1.
#
# Why an image at all: both were pip-installed into venvs on the host, open3e
# straight from a git URL. pip cannot compare an installed VCS dist against a
# requested one without fetching it, so `pip install` re-cloned and rebuilt on
# every Ansible converge -- reporting changed, restarting the service, and
# putting a gap in heating telemetry each time. Pinning the SHA did not fix
# that (verified: pip still clones and rebuilds a pinned VCS requirement). A
# tagged image does: `docker compose up -d` on an unchanged tag is a genuine
# no-op.
#
# It also gives the weekly update-audit a normal artifact to track instead of
# two awkward special cases -- open3e publishes no tags at all, and E3onCAN is
# pinned to an untagged commit between releases.
#
# Runtime needs host networking: SocketCAN interfaces live in the host network
# namespace, so can0 is only reachable with network_mode: host. That is also
# how the local MQTT broker on 127.0.0.1:1883 is reached. CAN_RAW sockets need
# no capabilities, so both run unprivileged.
FROM python:3.13-slim AS build
# Pins live here, in the artifact, rather than in Ansible variables.
#
# open3e: a commit, because the project publishes NO tags (its tags API returns
# an empty array). Self-reported package version at this commit: 0.7.5.
ARG OPEN3E_REF=6553c10453a6bae1210908f38a5af2481a46a3e5
# E3onCAN: an untagged commit on the default branch (2026-06-21, "Fixed: MQTT
# client ID collision caused stale retained values"). Later than the repo's
# only tag, v0.5.1.
ARG E3ONCAN_REF=461dcbdf97e8d72d2edb9929c63b39382af6c3a1
# git is needed to FETCH the sources and exists only in this stage. The runtime
# stage below copies the built venv and the E3onCAN source out, so neither git
# nor the apt cache ends up in the published image.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
RUN pip install --no-cache-dir supervisor \
&& pip install --no-cache-dir "git+https://github.com/open3e/open3e.git@${OPEN3E_REF}"
# E3onCAN is not packaged -- it is run as a script out of its checkout, so the
# source has to travel into the runtime image alongside its dependencies.
RUN git clone --quiet https://github.com/MyHomeMyData/E3onCAN.git /opt/e3oncan \
&& git -C /opt/e3oncan checkout --quiet "${E3ONCAN_REF}" \
&& pip install --no-cache-dir -r /opt/e3oncan/requirements.txt \
&& rm -rf /opt/e3oncan/.git
FROM python:3.13-slim
ARG OPEN3E_REF=6553c10453a6bae1210908f38a5af2481a46a3e5
ARG E3ONCAN_REF=461dcbdf97e8d72d2edb9929c63b39382af6c3a1
LABEL org.opencontainers.image.title="heating-can" \
org.opencontainers.image.description="open3e + E3onCAN, Viessmann E3 CAN bus readers" \
org.opencontainers.image.source="https://gitea.renner.uno/k.renner/homelab" \
uno.renner.open3e.ref="${OPEN3E_REF}" \
uno.renner.e3oncan.ref="${E3ONCAN_REF}"
# Everything is installed at build time. The runtime image carries the finished
# venv and source only -- no git, no compiler, no package index.
COPY --from=build /opt/venv /opt/venv
COPY --from=build /opt/e3oncan /opt/e3oncan
ENV PATH="/opt/venv/bin:$PATH"
# Both services log to stdout, which docker's journald driver forwards to the
# journal and edge-log-shipper ships to Loki. open3e's systemd unit did NOT set
# this, so its output was block-buffered and effectively never reached the
# journal -- a long-standing blind spot that this fixes for both.
ENV PYTHONUNBUFFERED=1
# One container, two services, so PID 1 is a supervisor. It restarts either
# reader on its own if it dies, reaps children and forwards shutdown -- which a
# bare process as PID 1 cannot do for a sibling.
#
# Configuration arrives as environment variables and supervisord expands them,
# so nothing outside this image constructs open3e's or E3onCAN's argv. Those
# CLIs are an implementation detail here; a compose file spelling them out
# would break whenever either upstream renamed a flag.
COPY supervisord.conf /etc/supervisord.conf
# Build-time self-test. Fails the build if either service cannot actually run,
# which is exactly what shipped before: supervisord pointed at
# /opt/e3oncan/src/E3onCANcollect.py, a path that does not exist -- the file is
# at the repo root. The image built green and would have run open3e while
# e3oncan crash-looped, the E380 meter silently absent.
#
# Done here rather than as a CI step because this needs no registry
# credentials and no runner able to pull a private image -- and because a
# broken image should never be published in the first place.
RUN open3e --help > /dev/null \
&& python /opt/e3oncan/E3onCANcollect.py --help > /dev/null \
&& supervisord --help > /dev/null \
&& grep -q '^\[program:open3e\]' /etc/supervisord.conf \
&& grep -q '^\[program:e3oncan\]' /etc/supervisord.conf \
&& grep -q '^\[supervisorctl\]' /etc/supervisord.conf \
&& supervisorctl --help > /dev/null \
&& echo "self-test OK: both services execute; supervisord has both programs"
# CAN_RAW needs no capabilities, so nothing here runs as root.
RUN useradd --system --no-create-home --shell /usr/sbin/nologin open3e
# A writable working directory, owned by the service user. udsoncan writes its
# log file into the CWD, and the native install this replaces gave open3e's
# directory `owner: open3e` for exactly that reason. /opt is root-owned from the
# COPY layers, so running there would fail that write.
RUN install -d -o open3e -g open3e -m 0755 /var/lib/open3e
USER open3e
WORKDIR /var/lib/open3e
ENTRYPOINT ["supervisord", "-c", "/etc/supervisord.conf"]
-80
View File
@@ -1,80 +0,0 @@
# Both CAN readers, supervised in one container.
#
# They read the same heating installation over the same bus and are versioned
# and deployed together, so they are one container. A container runs one PID 1;
# with two processes that PID 1 is a supervisor, which is what this is --
# supervisord restarts either service on its own if it dies, reaps children,
# and forwards shutdown.
#
# Configuration comes from the environment (see the compose file). supervisord
# expands %(ENV_NAME)s at start, so neither the compose file nor a wrapper
# script constructs open3e's or E3onCAN's argv -- those CLIs stay an
# implementation detail of this image.
# Control socket. Without these two sections supervisorctl cannot connect, so
# there is no way to query or restart an individual service inside the
# container -- which is the entire reason for running a supervisor rather than
# one process per container. The first revision shipped without them.
#
# Unix socket only, inside the container: nothing outside needs to reach it,
# and the container runs with host networking, so a TCP listener would be
# exposed on the host.
[unix_http_server]
file=/tmp/supervisor.sock
chmod=0700
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock
[supervisord]
nodaemon=true
user=open3e
logfile=/dev/null
logfile_maxbytes=0
pidfile=/tmp/supervisord.pid
# open3e: UDS request/response to the Vitocal heat pump and Vitocharge VX3.
[program:open3e]
command=open3e
-c %(ENV_CAN_INTERFACE)s
-cnfg %(ENV_OPEN3E_CONFIG)s
-m %(ENV_MQTT_HOST)s:%(ENV_MQTT_PORT)s:%(ENV_MQTT_TOPIC)s
-muser %(ENV_MQTT_USER)s:%(ENV_MQTT_PASSWORD)s
-mcid %(ENV_OPEN3E_CLIENT_ID)s
-mfstr %(ENV_MQTT_FORMAT)s
-r %(ENV_OPEN3E_DATAPOINTS)s
-t %(ENV_OPEN3E_POLL_SECONDS)s
-l %(ENV_MQTT_TOPIC)s/cmnd
-j
autostart=true
autorestart=true
startsecs=10
stopsignal=INT
# Straight to the container's stdout/stderr, so docker's journald driver picks
# them up and edge-log-shipper ships them to Loki.
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true
# e3oncan: passive CANopen listener for the E380 grid energy meter, which feeds
# the heat pump's own energy management. Issues no bus writes, which is why the
# two can share the bus.
[program:e3oncan]
command=python /opt/e3oncan/E3onCANcollect.py
-c %(ENV_CAN_INTERFACE)s
-dev %(ENV_E3ONCAN_DEVICE)s
-g %(ENV_E3ONCAN_GAP_SECONDS)s
-m %(ENV_MQTT_HOST)s:%(ENV_MQTT_PORT)s:%(ENV_MQTT_TOPIC)s
-muser %(ENV_MQTT_USER)s
-mpass %(ENV_MQTT_PASSWORD)s
-mfstr %(ENV_E3ONCAN_MQTT_FORMAT)s
-j
autostart=true
autorestart=true
startsecs=10
stopsignal=INT
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true
+6 -17
View File
@@ -1,12 +1,7 @@
# edge-host installer ISO
Bootable Arch installer image for the bare-metal edge hosts — the machines
that replaced the k8s-edge VM, because Talos cannot do CAN or USB
passthrough. See `ansible/docs/edge-host.md`.
Not tied to one box or one model: the installer autodetects the target disk
(NVMe, else eMMC, else the first non-USB disk), which covers both the Wyse
3040 thin clients and larger hardware.
Bootable Arch installer image for `edge-1.srv.renner.uno` (the bare-metal
N100 that replaces the k8s-edge VM — see `ansible/roles/edge-host`).
The image is the official archiso **releng** profile plus:
- `edge-install` in the live environment — partitions the NVMe
@@ -17,7 +12,7 @@ The image is the official archiso **releng** profile plus:
Division of labour: the ISO produces an **ansible-ready base system**;
all service configuration (zwave-js-ui, sml2mqtt, open3e/CAN) stays in
`ansible/docs/edge-host.md` as the single source of truth. Rebuilding the
`ansible/roles/edge-host` as the single source of truth. Rebuilding the
box is: boot ISO → `edge-install` → run the play.
## Build
@@ -43,16 +38,10 @@ archiso updates flow in automatically.
(NVMe, else eMMC — covers the Wyse 3040 — else first non-USB disk),
shows the target and requires typing the hostname before wiping
anything. Override with `edge-install /dev/sdY`.
4. Reboot. Confirm the box is reachable by name — `ssh root@<host>` — before
going further; it is identified by name everywhere and its address is
never recorded. Then add it to `[edge_host]` in
`ansible/inventory/pve_hosts.yml` and, from the workstation:
4. Reboot, then from the workstation:
```
cd ansible
ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host
ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
```
One run over the whole group, no `--limit`: the play carries
`certificates` and `common` itself, so there is no separate `common.yml`
step. Running it converges every edge host, which is the point — a run
that touches only the new box tells you nothing about the others.
5. Continue with the cutover checklist in `ansible/docs/edge-host.md`.
5. Continue with the cutover checklist in `ansible/roles/edge-host/README.md`.
@@ -50,7 +50,7 @@ mount "$(part 2)" /mnt
mount --mkdir "$(part 1)" /mnt/boot
# Everything the edge-host ansible role expects to find; the role itself
# is applied over SSH after first boot (see ansible/docs/edge-host.md).
# is applied over SSH after first boot (see ansible/roles/edge-host).
pacstrap -K /mnt base linux linux-firmware intel-ucode openssh python git \
docker docker-compose
@@ -99,13 +99,7 @@ EOF
umount -R /mnt
echo
echo "${HOST} installed. Remove the USB stick and reboot."
echo
echo "Then, from the workstation:"
echo " 1. confirm 'ssh root@${HOST}' works (it must be reachable by name)"
echo " 2. add ${HOST} to [edge_host] in ansible/inventory/pve_hosts.yml"
echo " 3. cd ansible && ansible-playbook -i inventory/pve_hosts.yml \\"
echo " plays/install-edge-host.yml"
echo
echo "One run, whole group, no --limit: the play carries certificates and"
echo "common itself, so there is no separate common.yml step."
echo "${HOST} installed. Remove the USB stick, reboot, then from the"
echo "workstation: cd ansible &&"
echo " ansible-playbook -i inventory/pve_hosts.yml plays/common.yml -l edge_host"
echo " ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml"
@@ -38,7 +38,7 @@ spec:
# gets the heavy HA pod (8Gi) off the soon-to-be-tiny edge box.
containers:
- name: home-assistant
image: ghcr.io/home-assistant/home-assistant:2026.8.1 # {"$imagepolicy": "flux-system:home-assistant"}
image: ghcr.io/home-assistant/home-assistant:2026.7.4 # {"$imagepolicy": "flux-system:home-assistant"}
env:
- name: TZ
value: Europe/Berlin
@@ -27,7 +27,7 @@ spec:
type: RuntimeDefault
containers:
- name: pve-exporter
image: prompve/prometheus-pve-exporter:3.10.0 # {"$imagepolicy": "flux-system:pve-exporter"}
image: prompve/prometheus-pve-exporter:3.9.0 # {"$imagepolicy": "flux-system:pve-exporter"}
# Image ENTRYPOINT is already `pve_exporter`; pass flags only.
args:
- --config.file=/etc/pve-exporter/pve.yml
@@ -52,7 +52,7 @@ spec:
# stdlib only; Flux keeps the tag in lockstep via the shared image
# policy) — no extra dependency.
- name: reconcile-publish-ip
image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
image: ghcr.io/music-assistant/server:2.9.10 # {"$imagepolicy": "flux-system:music-assistant"}
command: ["/app/venv/bin/python", "/scripts/reconcile.py"]
volumeMounts:
- name: data
@@ -62,7 +62,7 @@ spec:
readOnly: true
containers:
- name: music-assistant
image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
image: ghcr.io/music-assistant/server:2.9.10 # {"$imagepolicy": "flux-system:music-assistant"}
env:
- name: LOG_LEVEL
value: info
+1 -1
View File
@@ -22,7 +22,7 @@ spec:
type: RuntimeDefault
containers:
- name: gotenberg
image: gotenberg/gotenberg:8.35.0 # {"$imagepolicy": "flux-system:gotenberg"}
image: gotenberg/gotenberg:8.34.0 # {"$imagepolicy": "flux-system:gotenberg"}
args:
- "gotenberg"
- "--chromium-disable-javascript=true"
+1 -1
View File
@@ -25,7 +25,7 @@ ACCESS_STAGING="$REPO_ROOT/kubernetes/access-staging"
NODES=(k8s-1 k8s-2 k8s-3)
# No dedicated workers since the k8s-edge VM was retired (2026-07-02) —
# hardware-bound services live on the bare-metal edge-1 Arch host
# (ansible/docs/edge-host.md).
# (ansible/roles/edge-host).
WORKERS=()
ALL_NODES=("${NODES[@]}" "${WORKERS[@]}")
DOMAIN="srv.renner.uno"