Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5 33d92e133c fix(audit): track the heating-can image pins, not the deleted Ansible vars
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
#476 merged first, which removed edge_open3e_pkg, edge_e3oncan_repo and
edge_e3oncan_version in favour of the image. The audit entry added here still
told the agent to read those, so it would have gone looking for variables
that no longer exist and reported nothing -- the same silent non-auditing the
entry was written to prevent.

Repointed at where the pins actually live now: OPEN3E_REF and E3ONCAN_REF as
ARGs in ci/images/heating-can/Containerfile. The awkward specifics carry over
unchanged, since they are properties of the upstreams rather than of how we
package them: open3e publishes no tags so the comparison must be against
commits, and E3onCAN's pin is an untagged commit later than its newest tag.

Added a note the previous version could not have made: heating_can_image is
:latest, so the node follows whatever CI last built. The Containerfile ARGs
are what actually fix the versions, and bumping one is what requires a
rebuild -- otherwise a reader could reasonably conclude the tag is the pin.

The pin commit itself is now a no-op against master and survives only as the
audit registration; its conflict on heating-can/defaults resolved in favour of
the image.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:17:42 +02:00
claude-botandClaude Opus 5 150a83ee38 fix(edge): make the capability-group invariant structural, and unstale a note
Both from review on #475.

The rule "a host in a capability group must also be in [edge_host]" was
enforced by a comment, in a file that already uses :children a few lines
below for ntp_cluster. The three capability groups are now children of
edge_host, so the relation is structural: a host added only to [heating_can]
still resolves group_vars/edge_host, where the local MQTT contract lives, and
cannot silently come up without edge_mqtt_host and friends.

That matters more than it looks. The MQTT contract only moved to group_vars
in this PR because splitting the plays exposed that the roles could not
render without each other's defaults -- so an edge host outside [edge_host]
would hit exactly the failure that change was made to fix.

Verified all four groups still resolve to edge-1, and the redundant
ansible_user on the capability entries is dropped: it is inherited from the
[edge_host] declaration rather than repeated three times.

Also unstaled a Day-2 note that still read "the pip task tracks the git
default branch (pin edge_open3e_pkg to a tag if that ever bites)". Both
services are pinned to commits now, deliberately, and pinning to a tag is not
available for open3e -- it publishes none. Replaced with the actual workflow:
the weekly update-audit reports drift, bump the pin and re-run.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:16:37 +02:00
claude-botandClaude Opus 5 dd6553ab07 fix(heating-can): pin open3e, and register both CAN services with the audit
open3e was installed from an unpinned git URL, so the heating bridge
silently tracked upstream main: whatever HEAD happened to be on the day a
converge ran was what ended up driving the heat pump. Now pinned to
6553c104, which was simultaneously upstream HEAD and the copy already
installed on edge-1, so the pin froze the running state and changed nothing.

A commit rather than a tag because open3e publishes none -- its tags API
returns an empty array. The package self-reports 0.7.5 at that commit, which
is useful context but is not a tag.

Honest about what this does NOT fix: pip still re-clones and rebuilds a VCS
requirement even when the SHA is pinned, because it cannot compare an
installed VCS dist against a requested URL without fetching it first.
Verified directly on the host:

  pip install --dry-run "git+...@6553c104"
    -> Cloning ... Resolved ... to commit 6553c104
       Installing build dependencies: started

So the task still reports changed and still restarts open3e on every
converge. Pinning buys deliberate versioning, not idempotence.

Pinning without an update path only converts churn into silent rot, so both
services are now registered with the weekly update-audit -- which had been
auditing neither, despite E3onCAN having been pinned since July. The audit
prompt warns that anything not listed is silently not audited, and these two
were exactly that.

The new scope entry records the awkward specifics so the agent does not
report normal states as problems: open3e has no tags at all, so it must
compare commits against the default branch; E3onCAN does tag, but sparsely,
and its pin sits between the newest tag and HEAD.

Also corrected while there: edge_e3oncan_version was commented "# V0.5.2",
which does not exist upstream. The repo's only tag is v0.5.1 (6f43b3fb), and
461dcbdf is a later untagged commit from 2026-06-21, "Fixed: MQTT client ID
collision caused stale retained values; added publish error logging".

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:16:37 +02:00
claude-botandClaude Opus 5 0b845e7c76 refactor(edge): split the plays by capability, not just the roles
Splitting edge-host into single-concern roles left the coupling one layer
up: install-edge-host.yml was still one play applying all eleven roles to
one group. A host in [edge_host] got the Z-Wave gateway and the CAN services
whether or not it had the hardware -- the monolith again, in the play.

Inventory groups now state which hardware a host physically has, and each
playbook targets the group for its own concern:

  [edge_host]       install-edge-host.yml       base: flash-wear, log
                                                shipper, MQTT bridge, VLAN,
                                                docker
  [heating_can]     install-heating-can.yml     Innomaker USB2CAN
  [zwave_gateway]   install-zwave-gateway.yml   Nabu Casa ZWA-2
  [power_meter]     install-power-meter.yml     FTDI meter head

No flags and no `when:` on the roles. Moving a stick to another box is a
one-line inventory change, which is what it should be. A host with nothing
attached is simply absent from the capability groups -- which is what lets a
second thin client join as an NTP peer without being handed peripherals it
does not have.

The split exposed a circular dependency the single play had been hiding: the
local MQTT contract was scattered across two roles' defaults, in both
directions. mqtt-bridge's config needs edge_open3e_mqtt_user, because the
bridge authenticates to the cluster broker AS that user; heating-can and
power-meter-sml need edge_mqtt_host, because that is where they publish.
Neither role could render without the other's defaults, and it only ever
worked because one play loaded every role at once.

Those four values are the interface between the broker and its clients, not
the property of either side, so they move to group_vars/edge_host -- where
the matching secrets (edge_open3e_mqtt_password, edge_sml2mqtt_mqtt_password)
already lived. Removed from the role defaults so there is one source of
truth. Each role keeps what is genuinely its own: mqtt-bridge the cluster
endpoint and CA path, heating-can the datapoints and poll interval.

Also dropped: the legacy shared-compose teardown pre_tasks. That migration
completed on the 2026-08-08 apply (verified: /opt/edge-host/docker-compose.yml
is gone), and carrying migration cruft in a base play would be the same
mixing this commit is undoing.

Verified by --check --diff against the live host, all four plays:

  install-edge-host      ok=33 changed=0 failed=0
  install-zwave-gateway  ok=8  changed=0 failed=0
  install-power-meter    ok=4  changed=0 failed=0
  install-heating-can    ok=18 changed=4 failed=0

Three of four are fully idempotent. heating-can's four are the known open3e
pip churn plus the E3onCAN venv, which reports changed only under check mode
(it was ok in the real apply) -- and their two restart handlers.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:16:17 +02:00
k.renner 685905b1df Merge pull request 'build(heating-can): image for the CAN readers, replacing pip-into-venv' (#476) from heating-can-image into master
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#476
2026-08-10 19:04:49 +00:00
security-bot 2e62216bc7 chore(image-automation): bump prompve/prometheus-pve-exporter:3.10.0
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-10 06:35:03 +00:00
10 changed files with 156 additions and 68 deletions
+28 -2
View File
@@ -39,6 +39,8 @@ Build and boot the dedicated installer image — see
(inventory, plays, this file) and its address is never recorded. Do not
pin it to a reservation.
4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
That group is the base: every edge thin client belongs to it, and it
assumes no peripherals.
5. From the workstation:
```
ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
@@ -49,6 +51,26 @@ Build and boot the dedicated installer image — see
the point: a run that touches only the new box tells you nothing about
the others.
## Peripherals
Each attached device is its own inventory group and its own playbook. Group
membership states which hardware a host physically has -- there are no flags
and no `when:` on the roles, so moving a stick between boxes is a one-line
inventory change.
| device | group | playbook |
|---|---|---|
| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
Add the host to the matching group, then run that playbook. They depend on
the base play having run: all three need the local MQTT bridge, and two of
them need docker.
A host with nothing attached is simply absent from all three groups and needs
none of these commands.
## Secrets (before first play run)
Create `inventory/group_vars/edge_host/vault.yml` with ansible-vault; the
@@ -121,8 +143,12 @@ Afterwards:
deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
the preloaded `.so` (see "Logging & eMMC" → zwave-js).
- open3e updates: re-run the play; the pip task tracks the git default
branch (pin `edge_open3e_pkg` to a tag if that ever bites).
- open3e / E3onCAN updates: both are pinned to commits in
`roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
up upstream changes -- that is deliberate, these drive the heat pump. The
weekly update-audit reports when either falls behind (see
`ci/agents/update-audit/prompt.md`); bump the pin and re-run.
Pinning to a tag is not an option for open3e: it publishes none.
- Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
restart afterwards.
- can-utils (`candump`) is AUR-only on Arch — install manually via
@@ -36,3 +36,18 @@ keepalived_ntp_sysctl:
# Realtek NIC on the Wyse 3040 has no hardware timestamping.
chrony_hwtimestamp: false
# --- Local MQTT contract -------------------------------------------------
# Shared between the broker and its clients, so it belongs to the inventory
# rather than to either side's role. mqtt-bridge listens here; heating-can and
# power-meter-sml publish here; and the bridge authenticates to the cluster
# broker AS the open3e user, so it needs that identity too.
#
# Keeping these in one role's defaults only worked while a single play loaded
# every role at once. Splitting the plays by capability exposed it: the bridge
# could not render without a heating-can default, and the CAN services could
# not render without an mqtt-bridge default.
edge_mqtt_host: 127.0.0.1
edge_mqtt_port: 1883
edge_open3e_mqtt_user: open3e
edge_open3e_mqtt_topic: open3e
+27
View File
@@ -41,9 +41,36 @@ nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
[grafana]
grafana.srv.renner.uno ansible_user=ladmin
# Every bare-metal edge thin client. Gets the host-generic roles only:
# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
# sub-interface. Nothing here assumes any peripheral is attached.
[edge_host]
edge-1.iot.renner.uno ansible_user=root
# Capability groups. Membership states which hardware a host physically has,
# and each service playbook targets the group for its own concern. No flags and
# no `when:` on the roles -- moving a stick to another box is a one-line change
# here, which is what it should be.
#
# These are children of [edge_host], declared below: a peripheral is added to an
# edge client, it does not replace the base. Making that a :children relation
# rather than a comment means a host added only to a capability group still
# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
# cannot silently come up without edge_mqtt_host and friends.
[zwave_gateway]
edge-1.iot.renner.uno
[heating_can]
edge-1.iot.renner.uno
[power_meter]
edge-1.iot.renner.uno
[edge_host:children]
zwave_gateway
heating_can
power_meter
# NTP servers. Includes edge-1 so the role survives the PVE hosts being
# retired for bare-metal Talos, which cannot run chrony or keepalived itself.
# See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly
+15 -63
View File
@@ -3,10 +3,18 @@
# hardware-bound services, because Talos supports neither SocketCAN nor USB
# passthrough.
#
# One role per concern. Previously all of this lived in a single `edge-host`
# role, which meant a host could not take the flash-wear handling or the log
# shipper without also being handed the Z-Wave radio and the CAN bus. That is
# what blocked a second thin client from joining as an NTP peer.
# This play is the BASE only: what every edge thin client gets regardless of
# what is plugged into it. The peripherals live in their own playbooks against
# their own inventory groups:
#
# plays/install-zwave-gateway.yml [zwave_gateway] ZWA-2 radio
# plays/install-heating-can.yml [heating_can] Innomaker USB2CAN
# plays/install-power-meter.yml [power_meter] FTDI meter head
#
# Splitting the roles without splitting the plays would have kept the coupling:
# one play applying every role to one group is the monolith again, a layer up.
# A host with no peripherals is simply absent from those groups -- no flags, no
# `when:` on the roles.
#
# Self-contained: `certificates` and `common` are the parts of plays/common.yml
# that apply to an edge host, carried here so bootstrapping one is a single
@@ -14,76 +22,20 @@
# metal, not VMs) and monitoring-agent (Debian-oriented; these hosts ship logs
# via edge-log-shipper).
#
# Order is dependency order, not alphabetical:
# mqtt-bridge before its clients, so they have somewhere to publish
# edge-flash-wear before docker, so the journal is volatile before
# containers start logging into it
# socketcan before heating-can, which consumes can0
- name: Install Arch edge host
# Order is dependency order: edge-flash-wear before docker, so the journal is
# volatile before containers start logging into it.
- name: Install Arch edge host (base)
hosts: edge_host
# Connection is already root (ansible_user=root), so this is a root-to-root
# no-op today. Kept explicit, matching plays/common.yml, so the roles below
# do not silently depend on the connection user staying privileged.
become: true
# Defined on the play, not taken from a role's defaults: these are used by
# pre_tasks, which must not depend on role default resolution.
#
# heating-can, zwave-gateway and power-meter-sml each declare edge_base_dir
# in their own defaults too, so they stay usable standalone. Play vars win,
# so these are the effective values -- keep them in step if the path moves.
vars:
edge_base_dir: /opt/edge-host
edge_legacy_compose: "{{ edge_base_dir }}/docker-compose.yml"
# One-time migration off the shared compose project, BEFORE any role tries
# to start a container.
#
# zwave-js and sml2mqtt used to live in one project at edge_legacy_compose,
# both with a fixed container_name. Docker enforces container names globally,
# not per project, so while the old project still owns those names the new
# per-service projects cannot create them -- `docker compose up -d` fails with
# "Conflict: container name already in use", which is not the missing-USB-device
# error the roles tolerate, so the play would abort.
#
# It belongs here rather than in either role: it is a migration step for both
# of them, and putting it in one meant whichever role ran first hit the
# conflict. Guarded on the legacy file, so it runs once and then never again.
#
# check mode cannot verify this: command tasks are skipped, not simulated, so
# a --check run neither hits the conflict nor proves the teardown works.
pre_tasks:
- name: Retire the legacy shared compose project
ansible.builtin.command:
cmd: docker compose -f "{{ edge_legacy_compose }}" down
args:
removes: "{{ edge_legacy_compose }}"
failed_when: false
- name: Remove the legacy shared compose file
ansible.builtin.file:
path: "{{ edge_legacy_compose }}"
state: absent
- name: Remove the legacy zwave-js env from the shared project root
ansible.builtin.file:
path: "{{ edge_base_dir }}/zwavejs.env"
state: absent
roles:
- certificates
- common
# Host-generic. An edge host with no attached peripherals takes these and
# nothing below them.
- edge-flash-wear
- docker-install
- edge-log-shipper
- mqtt-bridge
- edge-vlan
# Hardware-bound. Each needs its own device present.
- socketcan
- heating-can
- zwave-gateway
- power-meter-sml
+18
View File
@@ -0,0 +1,18 @@
---
# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
# USB2CAN adapter attached.
#
# Targets [heating_can]: membership means the adapter is physically present.
# Two roles because they are two concerns -- the interface is host network
# state, the services consume it -- but one group, because both need the same
# hardware.
#
# Requires the base play (plays/install-edge-host.yml) to have run: these
# services publish to the local MQTT bridge it sets up.
- name: Install heating CAN services
hosts: heating_can
become: true
roles:
- socketcan
- heating-can
+14
View File
@@ -0,0 +1,14 @@
---
# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
# attached.
#
# Targets [power_meter]: membership means the meter head is physically present.
#
# Requires the base play (plays/install-edge-host.yml) to have run: it needs
# docker and publishes to the local MQTT bridge.
- name: Install power-meter reader
hosts: power_meter
become: true
roles:
- power-meter-sml
+15
View File
@@ -0,0 +1,15 @@
---
# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
#
# Targets [zwave_gateway]: membership means the radio is physically present.
# Moving the stick to another edge host is a one-line inventory change.
#
# Requires the base play (plays/install-edge-host.yml) to have run: it needs
# docker, and the flash-wear handling that the libeatmydata preload exists to
# complement.
- name: Install Z-Wave gateway
hosts: zwave_gateway
become: true
roles:
- zwave-gateway
@@ -5,8 +5,6 @@
# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
# blips. The local broker writes nothing to the eMMC (persistence off, logs to
# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
edge_mqtt_host: 127.0.0.1
edge_mqtt_port: 1883
# Upstream cluster broker the local bridge connects to (TLS; the public
# Let's Encrypt cert is validated against the system CA bundle).
edge_mqtt_cluster_host: mqtt.renner.uno
+23
View File
@@ -78,6 +78,29 @@ For each, find the **current pin in the repo** (Read/Glob/Grep), then look up th
6. **Ansible roles & OpenTofu providers** — `ansible/requirements.yml` (Galaxy
roles/collections) and `opentofu/providers.tf` + `opentofu/.terraform.lock.hcl`.
Report any with a newer published version.
7. **Edge heating CAN services** — open3e (Vitocal heat pump, Vitocharge VX3)
and E3onCAN (E380 grid energy meter), shipped as one image built from
`ci/images/heating-can/Containerfile` and deployed by
`ansible/roles/heating-can`. Nothing else tracks them. They read and (for two
datapoints) write the heat pump over its CAN bus, so silent drift here
matters more than most.
Both upstream pins are `ARG`s in the Containerfile -- `OPEN3E_REF` and
`E3ONCAN_REF` -- not Ansible variables. Read them there, and note the
awkward bits so normal states are not reported as problems:
- open3e publishes **no tags at all** (its tags API returns an empty array),
so compare `OPEN3E_REF` against the default branch:
`https://api.github.com/repos/open3e/open3e/commits?per_page=1`. Report how
many commits behind and the dates. "No release found" is normal here.
- E3onCAN tags, but sparsely, and `E3ONCAN_REF` is an untagged commit later
than the newest tag. Compare against both
`https://api.github.com/repos/MyHomeMyData/E3onCAN/tags` and the default
branch, and say which the pin sits between.
The deployed image tag is `heating_can_image` in
`ansible/roles/heating-can/defaults/main.yml`. It is `:latest` today, so the
node follows whatever CI last built -- the pins above are what actually fix
the versions, and bumping them is what requires a rebuild.
## Method & rigor
@@ -27,7 +27,7 @@ spec:
type: RuntimeDefault
containers:
- name: pve-exporter
image: prompve/prometheus-pve-exporter:3.9.0 # {"$imagepolicy": "flux-system:pve-exporter"}
image: prompve/prometheus-pve-exporter:3.10.0 # {"$imagepolicy": "flux-system:pve-exporter"}
# Image ENTRYPOINT is already `pve_exporter`; pass flags only.
args:
- --config.file=/etc/pve-exporter/pve.yml