Compare commits

..
Author SHA1 Message Date
claude-botandClaude Opus 5.5 5c068f735d fix(edge-host-iso): include sudo in the base system
The ssh-ca role installs the claude-bot sudo rules into /etc/sudoers.d,
which a base system without sudo does not have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 11:19:29 +01:00
59 changed files with 1127 additions and 672 deletions
+81 -13
View File
@@ -1,17 +1,13 @@
# edge-host — bare-metal Arch thin client (replaces the k8s-edge Talos VM)
Runs the hardware-bound services that used to be pinned to the `k8s-edge`
Talos node (opentofu `vm_153_k8s_edge.tf`):
Talos node (opentofu `vm_153_k8s_edge.tf`), plus the new CAN bridge:
| Service | How it runs | Hardware | Talks to |
|---|---|---|---|
| zwave-js-ui | docker compose | ZWA-2 Z-Wave USB radio | HA `zwave_js` via ws://:3000, UI :8091 |
| sml2mqtt | docker compose | FTDI FT230X meter reader | Mosquitto (`sml2mqtt` user) |
The Viessmann E3 CAN bus is not read by an edge host: the ESP32 gateway
`vitocal-can` (open3e-esp32) at the Vitocal publishes `open3e/#` directly to
the cluster broker. `plays/install-edge-host.yml` removes the former
heating-can stack and the can0 configuration from edge hosts.
| open3e | venv + systemd | Innomaker USB2CAN (gs_usb) | Mosquitto (`open3e` user) |
Everything else that once lived on k8s-edge (HA core, matter-server, OTBR,
music-assistant) already moved to the general node pool and stays in the
@@ -23,6 +19,10 @@ The deployed unit is a **Dell Wyse 3040** (Atom x5-Z8350, 2 GB RAM) on a
flash-wear handling that hardware implies. The installer also supports an
N100/NVMe box; it autodetects the target disk, so either works.
Rationale for leaving Kubernetes for the CAN part: the stock Talos kernel
ships with `CONFIG_CAN` unset (siderolabs/talos#7686), so SocketCAN cannot
work in any pod without a custom kernel image per Talos release.
## Base install (once, from the edge-host installer ISO)
Build and boot the dedicated installer image — see
@@ -58,18 +58,24 @@ inventory change.
| device | group | playbook |
|---|---|---|
| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
Add the host to the matching group, then run that playbook. It depends on
the base play having run: it needs docker.
Add the host to the matching group, then run that playbook. They depend on
the base play having run: all three need the local MQTT bridge, and two of
them need docker.
A host with nothing attached is simply absent from these groups and needs
A host with nothing attached is simply absent from all three groups and needs
none of these commands.
## Secrets (before first play run)
Create `inventory/group_vars/edge_host/vault.yml` with ansible-vault. The
zwave-js value comes out of the cluster:
cluster broker has no `open3e` user: edge-1 logs in only as the register
account `edge-1` (see "MQTT bridge" below), so
`vault_edge_open3e_mqtt_password` reaches only the loopback broker, which
allows anonymous clients; any random value works (`openssl rand -hex 24`).
The zwave-js value comes out of the cluster:
```
# zwave-js session secret
@@ -78,6 +84,7 @@ kubectl -n home-assistant get secret zwave-js-session -o jsonpath='{.data.sessio
```yaml
# vault.yml
vault_edge_open3e_mqtt_password: "..."
vault_edge_zwavejs_session_secret: "..."
vault_edge_loki_password: "..." # dedicated edge-1 Loki push cred (see "Logging & eMMC")
```
@@ -95,6 +102,9 @@ The Z-Wave network keys do NOT need separate handling — they live in
Preparation:
- [ ] `vault.yml` populated (above), play runs green against edge-1
- [ ] CAN: Innomaker adapter connected to the VX3 external CAN port,
`ip -details link show can0` shows `state ERROR-ACTIVE bitrate 250000`,
`candump can0 -n 5` shows frames
zwave-js (order matters — never two servers on one radio):
- [ ] Copy the store out of the PVC while the pod still runs:
@@ -112,6 +122,11 @@ sml2mqtt:
port, Wh stays Wh) → remove sml2mqtt from the home-assistant
kustomization in the follow-up PR
open3e:
- [ ] Stop any workstation listener (ONE instance per bus, hard rule)
- [ ] Move the USB2CAN adapter, `systemctl start open3e`
- [ ] Topics under `open3e/#` update on the broker
Afterwards:
- [ ] Retire VM 153: remove `vm_153_k8s_edge.tf`, `talos/patches/k8s-edge.yaml`,
node from the cluster (`kubectl delete node k8s-edge`, tofu apply)
@@ -126,6 +141,17 @@ Afterwards:
mesh-connected workstation. The play reads the image's Alpine release
itself and rebuilds the preloaded `.so` when it changes (see
"Logging & eMMC" → zwave-js).
- open3e / E3onCAN updates: both are pinned to commits in
`roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
up upstream changes -- that is deliberate, these drive the heat pump. The
weekly update-audit reports when either falls behind (see
`ci/agents/update-audit/prompt.md`); bump the pin and re-run.
Pinning to a tag is not an option for open3e: it publishes none.
- Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
restart afterwards.
- can-utils (`candump`) is AUR-only on Arch — install manually via
`pamac build can-utils` if needed for debugging; the role deliberately
doesn't manage AUR packages.
## Logging & eMMC
@@ -138,9 +164,7 @@ role keeps logs off local flash and monitors wear:
- **Docker logs to journald** (`docker-daemon.json`, `log-driver: journald`)
instead of unbounded json files under `/var/lib/docker/containers`. Both
compose services inherit it; `docker logs` still works.
- **Grafana Alloy ships the journal to Loki** (`edge-alloy.service`, which
runs the upstream `grafana/alloy` image with `/run/log/journal` mounted
read-only; Arch's `grafana-alloy` package has no `loki.source.journal`). Its
- **Grafana Alloy ships the journal to Loki** (`edge-alloy.service`). Its
WAL/positions sit on `/run/alloy` (tmpfs), so the agent writes nothing to
the eMMC either. Net effect: no log writes hit local flash — the trade-off
is that logs are RAM-only and lost on reboot unless already shipped.
@@ -189,3 +213,47 @@ journal commit).
- Verify the drop with the eMMC-write sampling described above; roll back by
removing the `LD_PRELOAD` env + `.so` mount from the compose file.
## MQTT bridge (TLS to the cluster broker)
open3e, e3oncan and sml2mqtt do **not** publish to the cluster broker directly.
They connect to a **local mosquitto on loopback** (`127.0.0.1:1883`,
`edge_mqtt_host`), which **bridges to the cluster broker over TLS**
(`mqtt.renner.uno:8883`). Two reasons:
- **Security:** open3e and e3oncan have no native MQTTS support (their MQTT
code does a plain `connect()`), so they can't use the broker's 8883 TLS
listener. Terminating TLS in the local broker keeps their plaintext MQTT on
loopback only — off the untrusted IoT VLAN — while the actual network hop is
encrypted. The public Let's Encrypt cert is validated against the system CA
bundle (`edge_mqtt_bridge_cafile`); no cert distribution needed.
- **Availability:** the local broker buffers telemetry (in RAM) if the cluster
broker is briefly unreachable, and flushes on reconnect — edge-1 no longer
hard-depends on cluster/network uptime for every publish.
The local broker is `allow_anonymous` on loopback — the trust boundary is the
host; nothing off-host can reach it.
**Bridge credential (REN-1307).** One bridge `connection` logs in as the
register account `edge-1` (`kubernetes/apps/home-assistant/mqtt-accounts.yaml`)
and carries `open3e/#` out and `open3e/cmnd` back. The cluster broker does not
accept the vault-password bridges in `bridge.d/legacy.conf`, which exists only
until the first push. Its password is pushed
by mqtt-account-controller to `edge-mqtt-cred` (`:8443`, the controller's
esp32 API over HTTPS). That endpoint accepts only a client certificate from
the MQTT CA with CN `mqtt-account-controller`; edge-1 holds the public CA and
its own server key, nothing else. On a push it writes
`/etc/mosquitto/bridge.d/edge-1.conf`, deletes `bridge.d/legacy.conf` and
restarts the local mosquitto. The controller sets a new password whenever the
bridge reports "not connected" (also after a reinstall), so no play run is
needed for a rotation. Diagnosis: `journalctl -u
edge-mqtt-cred`, and `mosquitto_sub -t '$SYS/broker/connection/edge-1/state'`
(retained `1` = connected).
**eMMC:** the local broker writes **nothing** to flash — `persistence false`
(retained/queued messages in RAM) and `log_dest stdout` (to the volatile
journal → Loki). Do not add persistence or log-file settings to
`edge-mqtt-bridge.conf.j2`.
Rollback: point `edge_mqtt_host` back to `mqtt.renner.uno` and re-run (clients
reconnect directly on 1883). Config: `edge-mqtt-bridge.conf.j2` →
`/etc/mosquitto/mosquitto.conf`.
@@ -2,6 +2,7 @@
# Secret indirection — real values live in vault.yml (ansible-vault) next
# to this file; see docs/edge-host.md for how to populate it from
# the cluster secrets.
edge_open3e_mqtt_password: "{{ vault_edge_open3e_mqtt_password }}"
edge_zwavejs_session_secret: "{{ vault_edge_zwavejs_session_secret }}"
# Loki push credential (dedicated 'edge-1' user, Traefik basic-auth on the
# loki.renner.uno ingress; the matching htpasswd line lives in the cluster
@@ -35,8 +36,25 @@ keepalived_ntp_sysctl:
# Realtek NIC on the Wyse 3040 has no hardware timestamping.
chrony_hwtimestamp: false
# --- Local MQTT contract -------------------------------------------------
# Shared between the broker and its clients, so it belongs to the inventory
# rather than to either side's role. mqtt-bridge listens here; heating-can
# publishes here; and the bridge authenticates to the cluster
# broker AS the open3e user, so it needs that identity too.
#
# Keeping these in one role's defaults only worked while a single play loaded
# every role at once. Splitting the plays by capability exposed it: the bridge
# could not render without a heating-can default, and the CAN services could
# not render without an mqtt-bridge default.
edge_mqtt_host: 127.0.0.1
edge_mqtt_port: 1883
edge_open3e_mqtt_user: open3e
edge_open3e_mqtt_topic: open3e
# Units claude-bot may restart (ssh-ca role); the containers go through docker.
ssh_ca_claude_sudo_units:
- mosquitto
- edge-mqtt-cred
- edge-alloy
- keepalived
- chronyd
+22 -10
View File
@@ -18,11 +18,10 @@ proxy-1.renner.uno ansible_host=proxy-1.nb.renner.uno ansible_user=root
proxy-2.renner.uno ansible_host=proxy-2.nb.renner.uno ansible_user=root
# Every bare-metal edge thin client. Gets the host-generic roles only:
# flash-wear handling, the log shipper, the VLAN sub-interface. Nothing here
# assumes any peripheral is attached.
# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
# sub-interface. Nothing here assumes any peripheral is attached.
[edge_host]
edge-1.iot.renner.uno ansible_user=root
edge-2.iot.renner.uno ansible_user=root
# Synology NAS (DSM): nas-1 RS1221+ on site, nas-2 DS1522+ off site.
[synology_nas]
@@ -37,18 +36,32 @@ nas-2.renner.uno ansible_user=root
# These are children of [edge_host], declared below: a peripheral is added to an
# edge client, it does not replace the base. Making that a :children relation
# rather than a comment means a host added only to a capability group still
# resolves group_vars/edge_host.
# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
# cannot silently come up without edge_mqtt_host and friends.
[zwave_gateway]
edge-1.iot.renner.uno
[heating_can]
edge-1.iot.renner.uno
[edge_host:children]
zwave_gateway
heating_can
# NTP servers: every edge host (edge-1 BACKUP prio 70, edge-2 BACKUP prio 60;
# see the ntp_servers map in plays/install-ntp-ha.yml). Talos nodes cannot run
# chrony or keepalived. The jupiter/saturn/neptune entries in that map
# describe retired PVE hosts. See plays/install-ntp-ha.yml for why keepalived
# is scoped more narrowly than chrony within this group.
# NTP servers. edge-1 was added here so the role would survive the PVE hosts
# being retired for bare-metal Talos, which cannot run chrony or keepalived
# itself. That retirement has now happened, so edge-1 is the ONLY member.
#
# THIS IS NO LONGER HIGHLY AVAILABLE. The group was jupiter (MASTER, prio 100),
# saturn (90), neptune (80) and edge-1 (BACKUP, 70); the first three are gone.
# A single VRRP member takes the address regardless of its configured state, so
# the service address is still served, but by one thin client with
# no failover behind it. install-ntp-ha.yml still carries the four-host
# ntp_servers priority map, which now describes hosts that do not exist.
#
# Restoring redundancy needs a second chrony host that is not a Talos node.
# See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly than
# chrony within this group.
[ntp_cluster:children]
edge_host
@@ -60,7 +73,6 @@ proxy-1.renner.uno
proxy-2.renner.uno
netbird.renner.uno
edge-1.iot.renner.uno
edge-2.iot.renner.uno
nas-1.renner.uno
nas-2.renner.uno
+70
View File
@@ -0,0 +1,70 @@
---
# Configure VMs to use host time sync via kvm-clock
# Disables systemd-timesyncd to prevent NTP conflicts
#
# VMs get time from Proxmox host via kvm-clock paravirtualized clock
# This is more accurate and reliable than network NTP for VMs
- name: Configure VM time synchronization from host
hosts: all
become: yes
gather_facts: yes
tasks:
- name: Check if running as VM (not LXC)
stat:
path: /sys/devices/system/clocksource/clocksource0/current_clocksource
register: clocksource_file
- name: Get current clock source
slurp:
src: /sys/devices/system/clocksource/clocksource0/current_clocksource
register: clocksource
when: clocksource_file.stat.exists
- name: Display clock source
debug:
msg: "Clock source: {{ (clocksource.content | b64decode | trim) if clocksource_file.stat.exists else 'N/A (LXC)' }}"
- name: Check if systemd-timesyncd is installed
command: systemctl list-unit-files systemd-timesyncd.service
register: timesyncd_installed
check_mode: false
failed_when: false
changed_when: false
- name: Stop and disable systemd-timesyncd on VMs using kvm-clock
systemd:
name: systemd-timesyncd
state: stopped
enabled: no
when:
- clocksource_file.stat.exists
- clocksource.content is defined
- "'kvm-clock' in (clocksource.content | b64decode)"
- "'systemd-timesyncd.service' in timesyncd_installed.stdout"
- name: Mask systemd-timesyncd to prevent re-enabling
systemd:
name: systemd-timesyncd
masked: yes
when:
- clocksource_file.stat.exists
- clocksource.content is defined
- "'kvm-clock' in (clocksource.content | b64decode)"
- "'systemd-timesyncd.service' in timesyncd_installed.stdout"
- name: Verify time sync status
command: timedatectl show --property=NTPSynchronized
register: timesync_status
check_mode: false
changed_when: false
failed_when: false
- name: Display final status
debug:
msg: |
Host: {{ inventory_hostname }}
Type: {{ 'VM (kvm-clock)' if (clocksource_file.stat.exists and 'kvm-clock' in (clocksource.content | b64decode | default(''))) else 'LXC/Other' }}
Time sync: {{ timesync_status.stdout | default('unknown') }}
Note: VMs sync time from Proxmox host, LXCs inherit host time automatically
+2 -86
View File
@@ -8,6 +8,7 @@
# their own inventory groups:
#
# plays/install-zwave-gateway.yml [zwave_gateway] ZWA-2 radio
# plays/install-heating-can.yml [heating_can] Innomaker USB2CAN
#
# Splitting the roles without splitting the plays would have kept the coupling:
# one play applying every role to one group is the monolith again, a layer up.
@@ -30,96 +31,11 @@
# do not silently depend on the connection user staying privileged.
become: true
# The Viessmann E3 CAN bus is read by vitocal-can (open3e-esp32 at the
# Vitocal), which publishes to the cluster broker itself, not by an edge
# host. These remove the former heating-can stack, the can0 configuration
# and the open3e MQTT bridge (loopback mosquitto + edge-mqtt-cred); each is
# a no-op where they are absent.
pre_tasks:
- name: Read the service list
ansible.builtin.service_facts:
- name: Stop and disable the retired MQTT bridge services
ansible.builtin.systemd:
name: "{{ item }}"
state: stopped
enabled: false
loop:
- edge-mqtt-cred
- mosquitto
when: (item ~ '.service') in ansible_facts.services
- name: Remove the retired MQTT bridge files
ansible.builtin.file:
path: "{{ item }}"
state: absent
loop:
- /etc/systemd/system/edge-mqtt-cred.service
- /usr/local/sbin/edge-mqtt-cred
- /etc/edge-mqtt-cred
- /etc/mosquitto
register: edge_mqtt_bridge_files
- name: Reload systemd without the edge-mqtt-cred unit
ansible.builtin.systemd:
daemon_reload: true
when: edge_mqtt_bridge_files.changed
- name: Uninstall mosquitto
ansible.builtin.package:
name: mosquitto
state: absent
- name: Check for the retired heating-can compose project
ansible.builtin.stat:
path: /opt/edge-host/heating-can/docker-compose.yml
register: edge_heating_can_compose
- name: Stop and remove the retired heating-can containers and image
ansible.builtin.command:
cmd: docker compose down --remove-orphans --rmi all
chdir: /opt/edge-host/heating-can
when: edge_heating_can_compose.stat.exists
changed_when: true
- name: Remove the retired heating-can directory
ansible.builtin.file:
path: /opt/edge-host/heating-can
state: absent
- name: Remove the can0 networkd configuration
ansible.builtin.file:
path: /etc/systemd/network/80-can0.network
state: absent
register: edge_can0_network
- name: Reload networkd without the can0 configuration
ansible.builtin.command:
cmd: networkctl reload
when: edge_can0_network.changed
changed_when: true
- name: Read the can0 link flags
ansible.builtin.command:
cmd: ip -o link show can0
register: edge_can0_link
check_mode: false
changed_when: false
failed_when: false
- name: Take can0 down
ansible.builtin.command:
cmd: ip link set can0 down
when:
- edge_can0_link.rc == 0
- edge_can0_link.stdout is search('[<,]UP[,>]')
changed_when: true
roles:
- certificates
- common
- pacman-cache
- edge-flash-wear
- docker-install
- edge-log-shipper
- mqtt-bridge
- edge-vlan
+18
View File
@@ -0,0 +1,18 @@
---
# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
# USB2CAN adapter attached.
#
# Targets [heating_can]: membership means the adapter is physically present.
# Two roles because they are two concerns -- the interface is host network
# state, the services consume it -- but one group, because both need the same
# hardware.
#
# Requires the base play (plays/install-edge-host.yml) to have run: these
# services publish to the local MQTT bridge it sets up.
- name: Install heating CAN services
hosts: heating_can
become: true
roles:
- socketcan
- heating-can
-1
View File
@@ -22,7 +22,6 @@
become: true
roles:
- common
- timesyncd
- docker-install
- netbird-proxy
- netbird-proxy-watchdog
-1
View File
@@ -16,6 +16,5 @@
become: true
roles:
- common
- timesyncd
- docker-install
- netbird-server
-3
View File
@@ -83,9 +83,6 @@
edge-1.iot.renner.uno:
priority: 70
state: "BACKUP"
edge-2.iot.renner.uno:
priority: 60
state: "BACKUP"
roles:
- role: chrony-ntp-server
+11 -2
View File
@@ -54,9 +54,18 @@
# via plays/configure-ssh-ca.yml.
# -------------------------------------------------------
# 5. Time sync
# 5. Disable NTP on VMs/LXC (use host time)
# -------------------------------------------------------
# Moved to separate roles: timesyncd (internet VPS), chrony-ntp-server (edge).
- name: Stop and disable systemd-timesyncd on VMs/LXC
systemd:
name: systemd-timesyncd
state: stopped
enabled: no
masked: yes
when:
- ansible_facts['virtualization_role'] == 'guest'
- ansible_facts['virtualization_type'] in ['kvm', 'lxc']
ignore_errors: yes
# -------------------------------------------------------
# 6. Certificates
@@ -7,9 +7,7 @@
edge_alloy_config_file: /etc/alloy/edge.alloy
edge_alloy_http_port: 12345
# Upstream image: Arch's grafana-alloy 1.21 is built without
# loki.source.journal. Renovate-tracked (customManagers in renovate.json).
edge_alloy_image: grafana/alloy:v1.21.3@sha256:8ab871206bd2d8f89277f53abffee5f452a293980788d52f85fa183a87a505a4
edge_alloy_bin: /usr/bin/grafana-alloy # Arch `grafana-alloy` pkg names the binary this, not `alloy`
edge_loki_push_url: "https://loki.renner.uno/loki/api/v1/push"
edge_loki_username: edge-1
edge_alloy_labels:
+2 -24
View File
@@ -6,18 +6,15 @@
# directly, so one journal source covers everything. Alloy s storage/WAL sits
# on /run (tmpfs) so it does not write to flash either.
# Alloy runs from the upstream image (see edge-alloy.service); the Arch package
# lacks journal support.
- name: Remove the grafana-alloy package
- name: Install grafana-alloy
ansible.builtin.package:
name: grafana-alloy
state: absent
state: present
- name: Create alloy system group
ansible.builtin.group:
name: alloy
system: true
register: edge_alloy_group
- name: Create alloy system user
ansible.builtin.user:
@@ -27,25 +24,6 @@
system: true
create_home: false
shell: /usr/bin/nologin
register: edge_alloy_user
- name: Look up the systemd-journal group
ansible.builtin.getent:
database: group
key: systemd-journal
- name: Set the container identity
ansible.builtin.set_fact:
edge_alloy_uid: "{{ edge_alloy_user.uid }}"
edge_alloy_gid: "{{ edge_alloy_group.gid }}"
edge_alloy_journal_gid: "{{ ansible_facts.getent_group['systemd-journal'][1] }}"
- name: Pull the alloy image
ansible.builtin.command:
argv: [docker, pull, "{{ edge_alloy_image }}"]
register: edge_alloy_pull
changed_when: "'Downloaded newer image' in edge_alloy_pull.stdout"
notify: Restart alloy
- name: Create alloy config directory
ansible.builtin.file:
@@ -1,35 +1,32 @@
[Unit]
Description=Grafana Alloy (journal -> Loki)
Description=Grafana Alloy (edge-1 log shipper -> Loki)
Documentation=https://grafana.com/docs/alloy/
After=docker.service network-online.target
Requires=docker.service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
# WAL + positions on tmpfs so the agent never writes to the eMMC.
ExecStartPre=/usr/bin/install -d -o alloy -g alloy -m 0750 /run/alloy
ExecStartPre=-/usr/bin/docker rm -f edge-alloy
# Attached run with --log-driver none: the container's output reaches the
# journal once, through this unit.
ExecStart=/usr/bin/docker run --rm --name edge-alloy \
--log-driver none \
--network host \
--user {{ edge_alloy_uid }}:{{ edge_alloy_gid }} \
--group-add {{ edge_alloy_journal_gid }} \
--read-only --tmpfs /tmp \
--cap-drop ALL --security-opt no-new-privileges \
-v /run/log/journal:/run/log/journal:ro \
-v /etc/machine-id:/etc/machine-id:ro \
-v {{ edge_alloy_config_file }}:/etc/alloy/config.alloy:ro \
-v /run/alloy:/var/lib/alloy \
{{ edge_alloy_image }} run \
User=alloy
Group=alloy
# Read the (volatile) systemd journal.
SupplementaryGroups=systemd-journal
# WAL + positions on tmpfs so the agent never writes to the eMMC. %t is the
# runtime dir root (/run); RuntimeDirectory creates /run/alloy owned by us.
RuntimeDirectory=alloy
ExecStart={{ edge_alloy_bin }} run \
--server.http.listen-addr=127.0.0.1:{{ edge_alloy_http_port }} \
--storage.path=/var/lib/alloy \
/etc/alloy/config.alloy
ExecStop=/usr/bin/docker stop edge-alloy
Restart=always
--storage.path=%t/alloy \
{{ edge_alloy_config_file }}
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=10s
LimitNOFILE=65536
# Hardening (bare metal, no LXC namespace limits here).
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,71 @@
---
# Heating system over the Viessmann E3 CAN bus: open3e (heat pump) and
# e3oncan (E380 energy meter). The can0 interface itself is the socketcan
# role.
# Service root. Kept per-role so the role is usable on its own.
edge_base_dir: /opt/edge-host
edge_open3e_mqtt_user: open3e
edge_open3e_mqtt_topic: open3e
edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
edge_open3e_poll_seconds: 30
# Read-only datapoint set (Phase 1 extended set, established 2026-07-02 on
# the commissioning workstation; see vican repo CLAUDE.md for the rationale).
# NO -w / write DIDs here — live heating + battery, writes are Phase 3.
# vx3.266 = ErrorDtcHistory (O3EList) — the VX3 fault log; polled to catch
# recurrences of FailureCommunicationEnergyMeter (the E380 energy-meter link
# the internal PV-surplus EMS depends on). Decodes via the master O3EList
# codec inherited through the vx3plus overlay.
# vitocal.543/2350/2543/2560: SmartGrid-/EMS-Status-Beobachtung (2026-07-06)
# — klärt, ob die Eigenstromoptimierung (EMS) bei PV-Überschuss von selbst
# den WW-Boost auslöst (2543.2 steht laut Scan bereits auf +10 K), bevor
# SG-Ready-Verdrahtung oder andere Trigger gebaut werden. Alle vier ro.
# vitocal.259/260 = InfoDtcList/InfoDtcHistory (O3EList) — the HPMU event log,
# the Vitocal-side counterpart to vx3.266. Both are capped lists (2 and 9
# entries when added), so the multi-frame read is cheap at a 30 s poll.
# Records faults/info events only (NormalOperation, LagDeviceDtcReported,
# LegionellaProtectionTemperatureReached) — NOT setpoint or program changes.
# onebase.*: the 29 paired ZigBee slots on the One Base gateway (ECU 0x6c3) --
# 8 radiator valves, 9 climate sensors, 9 underfloor actuators, 3 floor
# thermostats. Per-device room temperature, active setpoint, humidity, valve
# opening/demand, battery and RSSI; the local replacement for the ViCare cloud
# room sensors. Needs the onebase overlay (files/open3e-onebase.py) -- the stock
# codec declares 57 bytes and a paired slot answers with 68.
# Only CurrentValues are polled. The room and device registries (1884.., 2084..)
# are static configuration and are read on demand, not every 30 s.
edge_open3e_datapoints: >-
vitocal.259,vitocal.260,vitocal.268,vitocal.269,vitocal.271,vitocal.274,vitocal.286,vitocal.288,vitocal.318,vitocal.321,vitocal.324,vitocal.335,vitocal.336,vitocal.396,vitocal.402,vitocal.403,vitocal.476,vitocal.477,vitocal.543,vitocal.548,vitocal.565,vitocal.1007,vitocal.1043,vitocal.1190,vitocal.1211,vitocal.1391,vitocal.1416,vitocal.1417,vitocal.2320,vitocal.2346,vitocal.2350,vitocal.2351,vitocal.2352,vitocal.2369,vitocal.2370,vitocal.2486,vitocal.2487,vitocal.2488,vitocal.2543,vitocal.2560,vitocal.2622,vitocal.2624,vitocal.2625,vitocal.2735,vitocal.3016,vx3.266,vx3.378,vx3.379,vx3.380,vx3.535,vx3.1603,vx3.1664,vx3.1690,vx3.1801,vx3.1802,vx3.1810,vx3.1828,vx3.1830,vx3.1577,vx3.1578,vx3.1579,vx3.1580,vx3.1581,vx3.1582,vx3.1834,vx3.1836,vx3.2214,vx3.2240,vx3.3085,vx3.3086,vx3.3087,vx3.3088,vx3.3089,vx3.3090,vcmu.320,vcmu.322,vcmu.325,vcmu.355,vcmu.389,vcmu.391,vcmu.1772,vcmu.1773,vcmu.1775,vcmu.1776,vcmu.2256,vcmu.2259,vcmu.2333,vcmu.2334,vcmu.2569,vcmu.2571,vitocal.2791,onebase.2086,onebase.2089,onebase.2092,onebase.2095,onebase.2098,onebase.2101,onebase.2104,onebase.2107,onebase.2110,onebase.2113,onebase.2116,onebase.2119,onebase.2122,onebase.2125,onebase.2128,onebase.2131,onebase.2134,onebase.2137,onebase.2140,onebase.2143,onebase.2262,onebase.2265,onebase.2268,onebase.2271,onebase.2274,onebase.2277,onebase.2280,onebase.2283,onebase.2286
# ECU addresses; -dev selects codec set, tx selects the UDS address.
# vcmu (refrigerant-circuit controller) has no dedicated codec set —
# omitting dpList makes open3e fall back to the generic datapoint list,
# which decodes all polled VCMU DIDs.
edge_open3e_devices:
vitocal: { tx: "0x680", dpList: "vcal" }
# dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
# files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
vx3: { tx: "0x6a1", dpList: "/config/vx3plus.py" }
vcmu: { tx: "0x68c" }
# One Base ZigBee gateway. Overlay re-declares the paired ZigBee slots at
# their real 68-byte length (see files/open3e-onebase.py).
onebase: { tx: "0x6c3", dpList: "/config/onebase.py" }
edge_e3oncan_gap_seconds: 10
edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
# Secrets come from inventory/group_vars/edge_host/vault.yml:
# edge_open3e_mqtt_password
# The image carrying both readers, built and pinned in CI
# (ci/images/heating-can/Containerfile). The source pins live in the
# Containerfile, in the artifact -- not here -- so what is deployed is a tag,
# and `docker compose up -d` on an unchanged tag is a genuine no-op.
#
# Tracked by the weekly update-audit as an ordinary image, which is why this is
# simpler to keep current than the two git refs it replaced: one publishes no
# tags at all, the other was pinned to an untagged commit between releases.
heating_can_image: gitea.renner.uno/gitea_admin/heating-can:latest
# Compose project directory. Separate from the old native layout under
# {{ edge_base_dir }}/open3e, which this role removes.
heating_can_base_dir: "{{ edge_base_dir }}/heating-can"
@@ -0,0 +1,54 @@
# Managed by ansible (roles/heating-can). Datapoint overlay for the One Base
# ZigBee gateway on ECU 0x6c3 (BACKENDGATEWAY).
#
# Upstream declares ZigBee<N>DeviceCurrentValues as 57 bytes. A PAIRED device
# slot on this gateway answers with 68: the 57 known bytes plus the parent
# device serial and three trailing bytes. udsoncan reads only the declared
# length and then treats the next two bytes -- the first half of that serial --
# as a foreign data identifier, so every paired slot fails to decode with
# "Server returned values for data identifier 0x.... that was not requested".
# The bogus identifier differs per device because the serial does.
#
# This overlay re-declares the 29 paired slots at 68 bytes, reusing upstream's
# sub-type list so the per-device-type O3ESwitch (climate sensor / TRV / floor
# thermostat / actuator, incl. DeviceTemperatureSetpoint) is inherited rather
# than copied.
#
# EMPTY slots answer with 57 and decode against the stock codec, so they are
# left alone and are not polled. A slot that is later unpaired therefore starts
# reporting a decode error on its topic instead of a value.
#
# Verified on ECU 0x6c3, 2026-09-17: 29 paired slots, all 68 bytes.
from open3e.Open3Ecodecs import O3EComplexType, RawCodec
from open3e.Open3Edatapoints import dataIdentifiers as _master
# ZigBee<N>DeviceCurrentValues of the paired slots: devices 1-20 run 2086..2143
# in steps of 3, devices 21-29 continue at 2262..2286.
_PAIRED_CURRENT_VALUES = [2086 + 3 * n for n in range(20)] + [
2262 + 3 * n for n in range(9)
]
def _with_parent_serial(did):
stock = _master["dids"][did]
return O3EComplexType(
68,
stock.id,
list(stock.subTypes)
+ [
RawCodec(8, "ParentDeviceSerialnumber"),
RawCodec(2, "Unknown_65_66"),
RawCodec(1, "Unknown_67"),
],
desc=stock.desc,
info=stock.info,
acc=stock.acc,
)
dataIdentifiers = {
"dids": {
**_master["dids"],
**{did: _with_parent_serial(did) for did in _PAIRED_CURRENT_VALUES},
}
}
@@ -0,0 +1,20 @@
# Managed by ansible (roles/heating-can). Datapoint overlay for the VX3:
# the stock vx3 list acts as a FILTER (DIDs missing from it are popped
# from the master table, Open3Eclass.py), so the per-battery-module DIDs
# were never pollable with dpList "vx3". This overlay re-adds them with
# codec None (= keep the master/general codec, RawCodec hex strings).
#
# Module DIDs (verified answering on ECU 0x6a1, 2026-07-03):
# 1577-1582 ElectricalEnergyStorageModule{One..Six}OperatingData
# (139 B: 8x temperature 0.1 °C + 15x cell voltage mV)
# 3085-3090 ElectricalEnergyStorageModule{One..Six}Information
# (18 B: module voltage 0.01 V, SoH?, counters)
from open3e.Open3EdatapointsVx3 import dataIdentifiers as _vx3
dataIdentifiers = {
"dids": {
**_vx3["dids"],
1577: None, 1578: None, 1579: None, 1580: None, 1581: None, 1582: None,
3085: None, 3086: None, 3087: None, 3088: None, 3089: None, 3090: None,
}
}
@@ -0,0 +1,10 @@
---
- name: Reload systemd
ansible.builtin.systemd:
daemon_reload: true
- name: Restart heating-can
ansible.builtin.command:
cmd: docker compose up -d --force-recreate
chdir: "{{ heating_can_base_dir }}"
changed_when: true
+172
View File
@@ -0,0 +1,172 @@
---
# Heating system over the Viessmann E3 CAN bus.
#
# Two readers of the same heating installation, supervised inside one container
# from one image (ci/images/heating-can):
# open3e UDS request/response to the Vitocal heat pump and Vitocharge VX3
# e3oncan passive CANopen listener for the E380 grid energy meter, which
# feeds the heat pump's own energy management
#
# They coexist because e3oncan issues no bus writes.
#
# Nothing is installed here. The image is built in CI with both sources pinned,
# published to the registry, and this host only pulls and runs it. That is what
# removes the churn: these were pip-installed into venvs on the node, and pip
# re-cloned and rebuilt a VCS requirement on every converge -- restarting the
# heating bridge each time -- which pinning the SHA did not fix.
#
# network_mode: host is required, not incidental: SocketCAN interfaces live in
# the host network namespace, so can0 is only reachable that way. It is also how
# the local MQTT broker on 127.0.0.1 is reached. CAN_RAW needs no capabilities,
# so both run unprivileged.
#
# The can0 interface itself belongs to the socketcan role; this role consumes it.
- name: Create heating-can directory
ansible.builtin.file:
path: "{{ heating_can_base_dir }}"
state: directory
mode: "0755"
# --- Migration off the native install ---------------------------------------
# Must happen before the containers start. Two open3e instances issuing UDS
# requests on one CAN bus is the failure mode to avoid, so the old units are
# stopped and masked rather than merely disabled -- masked so a stray
# `systemctl start` or a leftover dependency cannot bring them back alongside
# the containers.
- name: Find which native CAN units still exist
ansible.builtin.stat:
path: "/etc/systemd/system/{{ item }}.service"
loop:
- open3e
- e3oncan
register: heating_can_native_units
# Stop and disable BEFORE anything else, and do NOT swallow the result.
#
# The first version did stop + disable + mask in one call with
# failed_when: false, then deleted the unit files. Masking a unit whose file
# still exists conflicts, so the call failed, failed_when hid it, nothing was
# stopped -- and the next task deleted the unit files out from under two
# running processes. systemctl is-enabled then reported "not-found" and looked
# clean while is-active still said "active".
#
# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
# kicking each other off the broker.
#
# Only units that actually exist are touched, so a fresh host does not need an
# error swallowed to get through this.
- name: Stop and disable the native CAN services
ansible.builtin.systemd:
name: "{{ item.item }}"
state: stopped
enabled: false
loop: "{{ heating_can_native_units.results }}"
loop_control:
label: "{{ item.item }}"
when: item.stat.exists
# Masking is deliberately NOT used. It writes a symlink at the same path the
# next task removes, so the two cancel out -- which is half of how the original
# failed. With the unit files gone there is nothing left to start.
- name: Remove the native unit files
ansible.builtin.file:
path: "/etc/systemd/system/{{ item }}.service"
state: absent
loop:
- open3e
- e3oncan
notify: Reload systemd
# The invariant that actually matters is "no native process is on the CAN bus",
# and systemd reporting success is not the same claim. Two open3e instances
# polling one bus is the failure this whole block exists to prevent, so it is
# asserted directly rather than inferred from the calls above.
- name: Check for surviving native CAN processes
ansible.builtin.command:
argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
register: heating_can_survivors
check_mode: false
changed_when: false
failed_when: false
- name: Assert no native CAN process survived
ansible.builtin.assert:
that:
- heating_can_survivors.stdout | default('') | length == 0
fail_msg: >-
A native open3e/e3oncan process is still running after the units were
stopped and removed:
{{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
Starting the container now would put two readers on the same CAN bus --
two open3e instances issue competing UDS requests and take over each
other's MQTT session. Kill the survivors, then re-run.
success_msg: No native CAN processes remain; safe to start the container.
- name: Remove the native venvs
ansible.builtin.file:
path: "{{ item }}"
state: absent
loop:
- "{{ edge_base_dir }}/open3e/venv"
- "{{ edge_base_dir }}/e3oncan/venv"
- "{{ edge_base_dir }}/e3oncan/src"
- "{{ edge_base_dir }}/open3e"
- "{{ edge_base_dir }}/e3oncan"
# --- Configuration (bind-mounted into the containers) ------------------------
- name: Deploy open3e devices.json
ansible.builtin.copy:
content: "{{ edge_open3e_devices | to_nice_json }}\n"
dest: "{{ heating_can_base_dir }}/devices.json"
mode: "0644"
notify: Restart heating-can
- name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
ansible.builtin.copy:
src: open3e-vx3plus.py
dest: "{{ heating_can_base_dir }}/vx3plus.py"
mode: "0644"
notify: Restart heating-can
- name: Deploy open3e One Base datapoint overlay (68-byte ZigBee slots)
ansible.builtin.copy:
src: open3e-onebase.py
dest: "{{ heating_can_base_dir }}/onebase.py"
mode: "0644"
notify: Restart heating-can
# --- Compose project ---------------------------------------------------------
- name: Deploy the broker credential env file
ansible.builtin.template:
src: heating-can.env.j2
dest: "{{ heating_can_base_dir }}/heating-can.env"
mode: "0600"
notify: Restart heating-can
- name: Deploy heating-can compose project
ansible.builtin.template:
src: docker-compose.yml.j2
dest: "{{ heating_can_base_dir }}/docker-compose.yml"
mode: "0644"
notify: Restart heating-can
- name: Pull the heating-can image
ansible.builtin.command:
cmd: docker compose pull
chdir: "{{ heating_can_base_dir }}"
changed_when: false
# Unlike the zwave and meter services there is no missing-device tolerance here:
# can0 is created by the socketcan role from a USB adapter that is part of this
# host's purpose. If it is absent the heating telemetry is down and the run
# should say so rather than pass quietly.
- name: Start heating-can services
ansible.builtin.command:
cmd: docker compose up -d
chdir: "{{ heating_can_base_dir }}"
register: heating_can_up
changed_when: >-
'Started' in heating_can_up.stderr or 'Created' in heating_can_up.stderr
or 'Started' in heating_can_up.stdout or 'Creating' in heating_can_up.stdout
@@ -0,0 +1,36 @@
# Managed by ansible (roles/heating-can).
#
# One container. Both CAN readers live in the image and are supervised inside
# it -- this file supplies configuration and nothing else. It does not know
# open3e's or E3onCAN's command line.
#
# network_mode: host is required: SocketCAN interfaces live in the host network
# namespace, so {{ edge_can_interface }} is only reachable this way, and it is
# also how the local MQTT broker is reached. CAN_RAW needs no capabilities, so
# the image runs unprivileged.
services:
heating-can:
image: {{ heating_can_image }}
container_name: heating-can
restart: unless-stopped
network_mode: host
volumes:
- ./devices.json:/config/devices.json:ro
- ./vx3plus.py:/config/vx3plus.py:ro
- ./onebase.py:/config/onebase.py:ro
env_file:
# Broker credential, 0600 -- see heating-can.env.j2.
- heating-can.env
environment:
CAN_INTERFACE: "{{ edge_can_interface }}"
MQTT_HOST: "{{ edge_mqtt_host }}"
MQTT_PORT: "{{ edge_mqtt_port }}"
MQTT_TOPIC: "{{ edge_open3e_mqtt_topic }}"
MQTT_FORMAT: "{{ edge_open3e_mqtt_format }}"
E3ONCAN_MQTT_FORMAT: "{{ edge_e3oncan_mqtt_format }}"
OPEN3E_CONFIG: /config/devices.json
OPEN3E_CLIENT_ID: open3e-edge
OPEN3E_POLL_SECONDS: "{{ edge_open3e_poll_seconds }}"
OPEN3E_DATAPOINTS: "{{ edge_open3e_datapoints }}"
E3ONCAN_DEVICE: e380
E3ONCAN_GAP_SECONDS: "{{ edge_e3oncan_gap_seconds }}"
@@ -0,0 +1,8 @@
# Managed by ansible (roles/heating-can). 0600 -- carries the broker credential.
#
# Kept out of docker-compose.yml, which is 0644: the sibling roles on this host
# do the same (power-meter-sml renders its config at 0600 while its compose file
# stays world-readable), and the native unit this replaced used a 0600 env file
# for exactly this value.
MQTT_USER={{ edge_open3e_mqtt_user }}
MQTT_PASSWORD={{ edge_open3e_mqtt_password }}
@@ -0,0 +1,23 @@
---
# --- MQTT: edge clients (open3e, e3oncan, sml2mqtt) publish to a LOCAL
# mosquitto on loopback, which bridges to the central cluster broker over TLS
# (open3e/e3oncan have no native MQTTS). This keeps plaintext on loopback only
# — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
# blips. The local broker writes nothing to the eMMC (persistence off, logs to
# the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
# Upstream cluster broker the local bridge connects to (TLS; the public
# Let's Encrypt cert is validated against the system CA bundle).
edge_mqtt_cluster_host: mqtt.renner.uno
edge_mqtt_cluster_tls_port: 8883
edge_mqtt_bridge_cafile: /etc/ssl/certs/ca-certificates.crt
# --- Bridge credential from mqtt-account-controller (REN-1307) ------------
# The cluster account the bridge logs in as (register entry in
# kubernetes/apps/home-assistant/mqtt-accounts.yaml). The controller pushes
# its password to edge-mqtt-cred and rotates it when the bridge reports
# "not connected"; there is no copy in ansible-vault.
edge_mqtt_bridge_account: edge-1
# HTTPS port of edge-mqtt-cred (esp32 device API, mTLS). Only a client
# certificate from the MQTT CA with this CN is accepted.
edge_mqtt_cred_port: 8443
edge_mqtt_cred_allowed_cn: mqtt-account-controller
@@ -0,0 +1,141 @@
#!/usr/bin/env python3
# Managed by ansible (roles/mqtt-bridge). Receives the cluster bridge password
# from mqtt-account-controller (REN-1307) so no plaintext copy lives in
# ansible-vault and no play run is needed to rotate it.
#
# Speaks the controller's esp32 device API over HTTPS:
# GET /api/status -> {"mqtt": {"connected": bool}}
# PUT /api/settings <- {"mqtt": {"user": "...", "pass": "..."}}
# A client certificate from the MQTT CA with CN mqtt-account-controller is
# required; nothing else may call it. The server key is generated on this
# host and never leaves it (the controller pins it on first contact).
#
# PUT writes the bridge file (template + credentials), removes the legacy
# vault-password bridges and restarts the local mosquitto. The password is
# kept only in that 0640 root:mosquitto file.
import grp
import json
import os
import socket
import ssl
import subprocess
import sys
import tempfile
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
CONF = json.load(open(sys.argv[1]))
WRITE = threading.Lock()
def connected():
try:
out = subprocess.run(
["mosquitto_sub", "-h", "127.0.0.1", "-p", str(CONF["local_port"]),
"-t", CONF["state_topic"], "-C", "1", "-W", "3"],
capture_output=True, text=True, timeout=10).stdout
except (OSError, subprocess.TimeoutExpired):
return False
return out.strip() == "1"
def write_bridge(user, password):
with open(CONF["template"]) as f:
body = f.read()
body += f"remote_username {user}\nremote_password {password}\n"
d = os.path.dirname(CONF["bridge_file"])
fd, tmp = tempfile.mkstemp(dir=d, prefix=".edge-mqtt-cred.")
with os.fdopen(fd, "w") as f:
f.write(body)
os.chown(tmp, 0, grp.getgrnam("mosquitto").gr_gid)
os.chmod(tmp, 0o640)
os.replace(tmp, CONF["bridge_file"])
for legacy in CONF["legacy_files"]:
if os.path.exists(legacy):
os.remove(legacy)
subprocess.run(["systemctl", "restart", "mosquitto"], check=True, timeout=60)
def valid(s):
# Controller passwords are random URL-safe tokens; anything with
# whitespace or a control character would break the config line.
return isinstance(s, str) and 0 < len(s) <= 256 and s.isprintable() and not any(c.isspace() for c in s)
class Handler(BaseHTTPRequestHandler):
def authorized(self):
cert = self.connection.getpeercert() or {}
cns = [v for rdn in cert.get("subject", ()) for k, v in rdn if k == "commonName"]
if cns != [CONF["allowed_cn"]]:
self.send_error(403)
return False
return True
def reply(self, code, obj=None):
data = b"" if obj is None else json.dumps(obj).encode()
self.send_response(code)
if data:
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self):
if not self.authorized():
return
if self.path != "/api/status":
return self.send_error(404)
self.reply(200, {"mqtt": {"connected": connected()}})
def do_PUT(self):
if not self.authorized():
return
if self.path != "/api/settings":
return self.send_error(404)
try:
n = int(self.headers.get("Content-Length", "0"))
if not 0 < n <= 4096:
raise ValueError("body size")
m = json.loads(self.rfile.read(n))["mqtt"]
user, password = m["user"], m["pass"]
except (ValueError, KeyError, TypeError):
return self.send_error(400)
if user != CONF["user"] or not valid(password):
return self.send_error(400)
with WRITE:
write_bridge(user, password)
self.log_message("bridge credentials for %s replaced, mosquitto restarted", user)
self.reply(204)
def log_message(self, fmt, *args):
sys.stderr.write("%s %s\n" % (self.address_string(), fmt % args))
class Server(ThreadingHTTPServer):
address_family = socket.AF_INET6
def server_bind(self):
self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
super().server_bind()
def get_request(self):
# The TLS handshake runs in the request thread, with a timeout, so a
# stalled peer cannot block the accept loop.
sock, addr = self.socket.accept()
sock.settimeout(10)
return self.ctx.wrap_socket(sock, server_side=True, do_handshake_on_connect=False), addr
def main():
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.minimum_version = ssl.TLSVersion.TLSv1_2
ctx.load_cert_chain(CONF["cert"], CONF["key"])
ctx.load_verify_locations(CONF["ca"])
ctx.verify_mode = ssl.CERT_REQUIRED
srv = Server(("::", CONF["port"]), Handler)
srv.ctx = ctx
srv.serve_forever()
if __name__ == "__main__":
main()
@@ -0,0 +1,10 @@
-----BEGIN CERTIFICATE-----
MIIBdTCCARygAwIBAgIUN8ne4k4Wif9O83GDyKiBP0Ma7EIwCgYIKoZIzj0EAwIw
GTEXMBUGA1UEAxMOUmVubmVyIE1RVFQgQ0EwHhcNMjYxMDA2MTUzMDE1WhcNMzYx
MDAzMTUzMDE1WjAZMRcwFQYDVQQDEw5SZW5uZXIgTVFUVCBDQTBZMBMGByqGSM49
AgEGCCqGSM49AwEHA0IABOZhZF2JDoyvVg2oIXSbJAuAWDcoG22RxPA4oMKRMA/b
XzIY566D2k6cVnWqbil9s/q4O2mj+Epo7plBw8NwuOOjQjBAMA4GA1UdDwEB/wQE
AwICpDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBQcqSDDiLZzhrvqq9f16Y08
CO7QfzAKBggqhkjOPQQDAgNHADBEAiBFkxd9pXnqIcjReEcOTvCBZ3HxKu2jNQGL
/le7gunTGgIgZapWAcu4awa+wmZcR/z2luOjD2GhIT15um14Z+sstJk=
-----END CERTIFICATE-----
@@ -0,0 +1,13 @@
---
# The MQTT clients (zwave-js, sml2mqtt, open3e, e3oncan) reconnect on their
# own once the local broker is back.
- name: Restart mosquitto
ansible.builtin.service:
name: mosquitto
state: restarted
- name: Restart edge-mqtt-cred
ansible.builtin.systemd:
name: edge-mqtt-cred
state: restarted
daemon_reload: true
+109
View File
@@ -0,0 +1,109 @@
---
# Local MQTT bridge broker.
#
# Edge clients (open3e, e3oncan, sml2mqtt) publish here over loopback; this
# broker bridges to the cluster broker over TLS, because those clients have no
# native MQTTS. Persistence is off and logging goes to stdout so it writes
# nothing to local flash.
- name: Install mosquitto
ansible.builtin.package:
name: mosquitto
state: present
- name: Create the bridge directory
ansible.builtin.file:
path: /etc/mosquitto/bridge.d
state: directory
owner: root
group: mosquitto
mode: "0750"
- name: Check for the controller-managed bridge file
ansible.builtin.stat:
path: "/etc/mosquitto/bridge.d/{{ edge_mqtt_bridge_account }}.conf"
register: edge_mqtt_bridge_managed
# Until mqtt-account-controller has pushed the first password, keep the
# vault-password bridges so the cluster never loses edge-1's data. Once the
# managed file exists, never render them again (edge-mqtt-cred removed them).
- name: Deploy legacy vault-password bridges
ansible.builtin.template:
src: edge-mqtt-bridge-legacy.conf.j2
dest: /etc/mosquitto/bridge.d/legacy.conf
owner: root
group: mosquitto
mode: "0640" # carries the legacy cluster bridge credentials
when: not edge_mqtt_bridge_managed.stat.exists
notify: Restart mosquitto
- name: Deploy local mosquitto config
ansible.builtin.template:
src: edge-mqtt-bridge.conf.j2
dest: /etc/mosquitto/mosquitto.conf
owner: root
group: mosquitto
mode: "0640"
notify: Restart mosquitto
- name: Enable local mosquitto bridge
ansible.builtin.systemd:
name: mosquitto
enabled: true
state: started
# --- edge-mqtt-cred (REN-1307): receives the bridge password from
# mqtt-account-controller. edge-1 holds only the public MQTT CA and its own
# server key, generated here and never copied off the host.
- name: Create edge-mqtt-cred config directory
ansible.builtin.file:
path: /etc/edge-mqtt-cred
state: directory
owner: root
group: root
mode: "0700"
- name: Generate edge-mqtt-cred server key and certificate
ansible.builtin.command:
cmd: >-
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1
-nodes -days 36500 -subj "/CN={{ inventory_hostname }}"
-keyout /etc/edge-mqtt-cred/tls.key -out /etc/edge-mqtt-cred/tls.crt
creates: /etc/edge-mqtt-cred/tls.key
notify: Restart edge-mqtt-cred
- name: Deploy edge-mqtt-cred files
ansible.builtin.copy:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: root
group: root
mode: "{{ item.mode }}"
loop:
- {src: edge-mqtt-cred.py, dest: /usr/local/sbin/edge-mqtt-cred, mode: "0755"}
# Public CA of the cluster's MQTT client certificates
# (kubernetes/infrastructure/post-crds/mqtt-ca.yaml), valid until 2036-10-03.
- {src: mqtt-ca.crt, dest: /etc/edge-mqtt-cred/mqtt-ca.crt, mode: "0644"}
notify: Restart edge-mqtt-cred
- name: Deploy edge-mqtt-cred config and bridge template
ansible.builtin.template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: root
group: root
mode: "{{ item.mode }}"
loop:
- {src: edge-mqtt-cred.json.j2, dest: /etc/edge-mqtt-cred/config.json, mode: "0600"}
# Read only on the controller's next push: a change here reaches the
# running bridge when the password is next set.
- {src: edge-1-bridge.conf.in.j2, dest: /etc/edge-mqtt-cred/bridge.conf.in, mode: "0600"}
- {src: edge-mqtt-cred.service.j2, dest: /etc/systemd/system/edge-mqtt-cred.service, mode: "0644"}
notify: Restart edge-mqtt-cred
- name: Enable edge-mqtt-cred
ansible.builtin.systemd:
name: edge-mqtt-cred
enabled: true
state: started
daemon_reload: true
@@ -0,0 +1,21 @@
# Managed by roles/mqtt-bridge (REN-1307). edge-mqtt-cred appends
# remote_username/remote_password to this and writes the result to
# /etc/mosquitto/bridge.d/{{ edge_mqtt_bridge_account }}.conf. One bridge for
# open3e, as the register account `{{ edge_mqtt_bridge_account }}`
# (ACL readwrite open3e/#). sml2mqtt is retired (SML reader, 2026-09-19).
connection {{ edge_mqtt_bridge_account }}
address {{ edge_mqtt_cluster_host }}:{{ edge_mqtt_cluster_tls_port }}
bridge_cafile {{ edge_mqtt_bridge_cafile }}
bridge_insecure false
remote_clientid edge1-bridge
cleansession false
try_private true
restart_timeout 5 30
queue_qos0_messages true
# The bridge state goes to the LOCAL broker only (retained 1/0); edge-mqtt-cred
# reads it to answer the controller's "connected?".
notifications true
notifications_local_only true
notification_topic $SYS/broker/connection/{{ edge_mqtt_bridge_account }}/state
topic open3e/# out 0
topic open3e/cmnd in 0
@@ -0,0 +1,21 @@
# Managed by roles/mqtt-bridge. Legacy vault-password bridges, rendered only
# while /etc/mosquitto/bridge.d/{{ edge_mqtt_bridge_account }}.conf does not
# exist; edge-mqtt-cred deletes this file on the controller's first push
# (REN-1307). Remove with the vault keys once edge-1 is on the register account.
# --- Bridge 1: open3e datapoints + command topic, as the cluster `open3e`
# user (its ACL is `readwrite open3e/#`, which covers publishing datapoints
# and subscribing to open3e/cmnd). e3oncan also publishes under open3e/#.
connection edge1-open3e
address {{ edge_mqtt_cluster_host }}:{{ edge_mqtt_cluster_tls_port }}
bridge_cafile {{ edge_mqtt_bridge_cafile }}
bridge_insecure false
remote_username {{ edge_open3e_mqtt_user }}
remote_password {{ edge_open3e_mqtt_password }}
remote_clientid edge1-bridge-open3e
cleansession false
try_private true
notifications false
restart_timeout 5 30
queue_qos0_messages true
topic open3e/# out 0
topic open3e/cmnd in 0
@@ -0,0 +1,30 @@
# Managed by roles/mqtt-bridge. Local mosquitto that the edge clients (open3e,
# e3oncan, sml2mqtt) publish to over loopback; it bridges to the central
# cluster broker over TLS. open3e/e3oncan have no native MQTTS, so this is how
# edge-1's MQTT gets off plaintext on the untrusted IoT VLAN, and it buffers
# telemetry in RAM if the cluster broker is briefly unreachable.
#
# eMMC: this host runs on a small internal eMMC, so the local broker writes
# NOTHING to flash — `persistence false` keeps retained/queued messages in RAM
# only, and `log_dest stdout` goes to the (volatile, RAM) journal that Alloy
# ships to Loki. Do not add persistence/log-file settings here.
persistence false
log_dest stdout
log_type error
log_type warning
log_type notice
# Loopback only — the trust boundary is the host itself. Any process on edge-1
# may publish here (same trust as the bridged services); nothing off-host can
# reach it. Real authentication to the cluster happens on the bridge below.
# Single v4-loopback listener: all edge clients dial 127.0.0.1 (edge_mqtt_host),
# and a second same-port listener can make some builds refuse to start.
listener 1883 127.0.0.1
allow_anonymous true
# --- Bridges to the cluster broker (REN-1307). bridge.d holds either
# {{ edge_mqtt_bridge_account }}.conf, written by edge-mqtt-cred with the password
# mqtt-account-controller pushed, or, until that first push, legacy.conf with
# the two vault-password bridges. edge-mqtt-cred deletes legacy.conf when it
# writes the new file.
include_dir /etc/mosquitto/bridge.d
@@ -0,0 +1,13 @@
{{ {
"port": edge_mqtt_cred_port,
"allowed_cn": edge_mqtt_cred_allowed_cn,
"user": edge_mqtt_bridge_account,
"cert": "/etc/edge-mqtt-cred/tls.crt",
"key": "/etc/edge-mqtt-cred/tls.key",
"ca": "/etc/edge-mqtt-cred/mqtt-ca.crt",
"template": "/etc/edge-mqtt-cred/bridge.conf.in",
"bridge_file": "/etc/mosquitto/bridge.d/" ~ edge_mqtt_bridge_account ~ ".conf",
"legacy_files": ["/etc/mosquitto/bridge.d/legacy.conf"],
"local_port": edge_mqtt_port,
"state_topic": "$SYS/broker/connection/" ~ edge_mqtt_bridge_account ~ "/state",
} | to_nice_json }}
@@ -0,0 +1,19 @@
# Managed by roles/mqtt-bridge (REN-1307). See /usr/local/sbin/edge-mqtt-cred.
[Unit]
Description=Receives the MQTT bridge credential from mqtt-account-controller
After=network-online.target mosquitto.service
Wants=network-online.target
[Service]
ExecStart=/usr/bin/python3 /usr/local/sbin/edge-mqtt-cred /etc/edge-mqtt-cred/config.json
Restart=always
RestartSec=5
# Root only to write the bridge file and restart mosquitto.
ProtectSystem=strict
ReadWritePaths=/etc/mosquitto/bridge.d
ProtectHome=true
PrivateTmp=true
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
@@ -1,14 +0,0 @@
# Managed by ansible (roles/pacman-cache).
# Downloaded packages are not kept; older versions come from the Arch Linux
# Archive.
[Trigger]
Operation = Install
Operation = Upgrade
Operation = Remove
Type = Package
Target = *
[Action]
Description = Removing downloaded packages from the cache...
When = PostTransaction
Exec = /usr/bin/find /var/cache/pacman/pkg -mindepth 1 -delete
-21
View File
@@ -1,21 +0,0 @@
---
# Keep the pacman package cache empty. pacman keeps every downloaded package;
# on a 7 GB eMMC root the cache grows to gigabytes. Arch hosts only.
- name: Ensure the pacman hook directory exists
ansible.builtin.file:
path: /etc/pacman.d/hooks
state: directory
mode: "0755"
- name: Empty the package cache after every transaction
ansible.builtin.copy:
src: pacman-cache.hook
dest: /etc/pacman.d/hooks/pacman-cache.hook
mode: "0644"
- name: Empty the package cache
ansible.builtin.command:
argv: [find, /var/cache/pacman/pkg, -mindepth, "1", -delete, -print]
register: pacman_cache_clean
changed_when: pacman_cache_clean.stdout | length > 0
@@ -0,0 +1,5 @@
---
# SocketCAN interface. Consumed by heating-can.
# --- CAN / open3e (Viessmann E3 external bus) ---
edge_can_interface: can0
edge_can_bitrate: 250000
@@ -0,0 +1,7 @@
---
# reload alone only re-reads config files; existing links keep their old state
# until reconfigured.
- name: Reload networkd
ansible.builtin.shell:
cmd: networkctl reload && networkctl reconfigure {{ edge_can_interface }}
changed_when: true
+20
View File
@@ -0,0 +1,20 @@
---
# SocketCAN interface, owned by systemd-networkd.
#
# gs_usb is an in-tree module and autoloads when the adapter enumerates;
# networkd then matches the interface and applies bitrate + up. Separate from
# the services that consume the bus (heating-can, e380-can-meter) because the
# interface is host network state shared by both.
- name: Configure can0 via systemd-networkd
ansible.builtin.template:
src: 80-can0.network.j2
dest: /etc/systemd/network/80-can0.network
mode: "0644"
notify: Reload networkd
- name: Enable systemd-networkd
ansible.builtin.service:
name: systemd-networkd
state: started
enabled: true
@@ -0,0 +1,17 @@
# Managed by ansible (roles/socketcan).
# Viessmann E3 external CAN bus via Innomaker USB2CAN (gs_usb).
# The adapter is a stub tap on an already-terminated bus (120R jumper OFF).
[Match]
Name={{ edge_can_interface }}
[Link]
# networkd applies [CAN] settings but does not raise a CAN link on its
# own (observed on edge-1); force it up, and back up after any down.
ActivationPolicy=always-up
[CAN]
BitRate={{ edge_can_bitrate }}
# NO RestartSec here: gs_usb does not support hardware bus-off restart,
# and networkd fails the ENTIRE link configuration on the unsupported
# option ("off (failed)", link never raised — found by reboot test).
# Bus-off recovery falls to the open3e service restart loop.
-6
View File
@@ -143,12 +143,6 @@
group: root
mode: "0644"
- name: Install sudo
ansible.builtin.package:
name: sudo
state: present
when: not ssh_ca_synouser.stat.exists
- name: Install the Claude sudo rules
ansible.builtin.template:
src: sudoers-claude.j2
-26
View File
@@ -1,26 +0,0 @@
---
# Keep the system clock on NTP with systemd-timesyncd, on hosts that run no
# chrony. A KVM guest's clocksource does not steer wall-clock time; without an
# NTP client the clock drifts until JWT iat/exp checks fail.
- name: Install systemd-timesyncd
ansible.builtin.apt:
name: systemd-timesyncd
state: present
when: ansible_facts['os_family'] == 'Debian'
- name: Run systemd-timesyncd
ansible.builtin.systemd_service:
name: systemd-timesyncd
masked: false
enabled: true
state: started
- name: Wait for the clock to synchronize
ansible.builtin.command:
argv: [timedatectl, show, --property=NTPSynchronized, --value]
register: timesyncd_synchronized
changed_when: false
until: timesyncd_synchronized.stdout == "yes"
retries: 12
delay: 5
+4
View File
@@ -12,6 +12,8 @@
ansible.builtin.import_playbook: plays/configure-synology-python.yml
- name: Base system
ansible.builtin.import_playbook: plays/common.yml
- name: VM time sync
ansible.builtin.import_playbook: plays/configure-vm-time-sync.yml
- name: SSH user CA
ansible.builtin.import_playbook: plays/configure-ssh-ca.yml
@@ -31,6 +33,8 @@
ansible.builtin.import_playbook: plays/install-edge-host.yml
- name: Z-Wave gateway
ansible.builtin.import_playbook: plays/install-zwave-gateway.yml
- name: Heating CAN
ansible.builtin.import_playbook: plays/install-heating-can.yml
- name: NTP service
ansible.builtin.import_playbook: plays/install-ntp-ha.yml
+1 -1
View File
@@ -19,7 +19,7 @@ The image is the official archiso **releng** profile plus:
- the extra disk tools the installer needs
Division of labour: the ISO produces an **ansible-ready base system**;
all service configuration (zwave-js-ui, sml2mqtt) stays in
all service configuration (zwave-js-ui, sml2mqtt, open3e/CAN) stays in
`ansible/docs/edge-host.md` as the single source of truth. Rebuilding the
box is: boot ISO → `edge-install` → run the play.
+1 -3
View File
@@ -113,10 +113,8 @@ spec:
value: "10.244.0.0/16,fd10:244::/48,127.0.0.0/8,::1/128"
authentik:
log_level: info
# Off: reports would go to authentik.error-reporting.a7k.io (Sentry),
# which egress policy denies. The outposts take this from the server.
error_reporting:
enabled: false
enabled: true
postgresql:
host: "ak-pg-rw.authentik.svc.cluster.local"
name: "authentik"
@@ -34,10 +34,3 @@ spec:
protocol: TCP
- port: "6636"
protocol: TCP
# The UDM's LDAPS admin-auth sync.
- fromCIDRSet:
- cidrGroupRef: udm-k8s-ingress
toPorts:
- ports:
- port: "6636"
protocol: TCP
@@ -44,7 +44,7 @@ spec:
annotations:
# Bump to restart the pod and re-attach net1 (DHCP via multus).
# 2026-10-06: first attach on containernetworking/plugins v1.9.1.
homelab.renner.uno/restarted-for: "2026-11-07 net1 without RA (S28-36)"
homelab.renner.uno/restarted-for: "2026-10-06 cni-plugins v1.9.1 attach check"
k8s.v1.cni.cncf.io/networks: mdns-relay-iot
spec:
# Runs where the node patch declares the IoT VLAN (ens19).
@@ -31,11 +31,6 @@ spec:
# dropped. The relay's forward path to Home Assistant runs over eth0, so
# that route has to stay intact.
#
# sbr covers IPv4 only. With Router Advertisements net1 gets a global IPv6
# address and default route in the main table, and the IPv6 forward to the
# Service leaves via net1 with an IoT source, which HA's CNP drops. tuning
# turns RA processing off; the link-local address ff02::fb needs stays.
#
# Requires the cni-dhcp-daemon DaemonSet in infrastructure/multus.
config: |
{
@@ -50,13 +45,6 @@ spec:
"type": "dhcp"
}
},
{
"type": "tuning",
"sysctl": {
"net.ipv6.conf.IFNAME.accept_ra": "0",
"net.ipv6.conf.IFNAME.autoconf": "0"
}
},
{
"type": "sbr"
}
@@ -64,7 +64,7 @@ spec:
type: RuntimeDefault
containers:
- name: shelly-manager
image: gitea.renner.uno/konairius/shelly-manager:2026-11-07-2052 # {"$imagepolicy": "flux-system:shelly-manager"}
image: gitea.renner.uno/konairius/shelly-manager:2026-11-05-0732 # {"$imagepolicy": "flux-system:shelly-manager"}
imagePullPolicy: IfNotPresent
ports:
- name: http
@@ -49,12 +49,17 @@
auth: cert
deliver: {namespace: home-assistant}
acl: ["readwrite #"]
# vitocal-can: open3e-esp32 CAN gateway at the Vitocal 250-A (S28-28).
# The firmware registers its compiled-in DHCP name `open3e`, not its
# configured hostname.
- name: vitocal-can
# edge-1's local mosquitto bridge (REN-1307): one bridge for open3e/e3oncan.
# The controller pushes
# the password to edge-mqtt-cred on edge-1 (ansible role mqtt-bridge) over
# HTTPS with its own client certificate, and pins edge-1's self-generated
# server certificate on first contact (key edge-1.pin in mqtt-account-hashes;
# delete it after a reinstall of edge-1).
- name: edge-1
auth: password
deliver: {esp32: "http://open3e.iot.renner.uno"}
deliver: {esp32: "https://edge-1.iot.renner.uno:8443"}
# No sml2mqtt/#: the SML meter reader is retired for good (owner,
# 2026-09-19; grid data comes from the E380 on the CAN bus).
acl: ["readwrite open3e/#"]
# Shellies (REN-1017): one password account per device, pushed by the
# controller through shelly-manager's per-device API. Name shelly-<id> =
@@ -101,10 +106,6 @@
auth: password
deliver: {shelly-manager: 25} # Plug S Gen1 1c1a8f
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-41
auth: password
deliver: {shelly-manager: 41} # Plug Gen1 f4cfa26d9cb2
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-15
auth: password
deliver: {shelly-manager: 15} # Plug S G3 8cbfea910578
@@ -187,34 +188,6 @@
auth: password
deliver: {shelly-manager: 30} # Plus 2PM b48a0a226c28 (reachable again, REN-1108)
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-7
auth: password
deliver: {shelly-manager: 7} # Plus 1PM Mini 348518df3500
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-23
auth: password
deliver: {shelly-manager: 23} # Plus 1PM Mini 348518e05b88
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-11
auth: password
deliver: {shelly-manager: 11} # Plus 2PM ccdba7d0b804
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-28
auth: password
deliver: {shelly-manager: 28} # Plus 2PM 0cdc7efa8dc0
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-17
auth: password
deliver: {shelly-manager: 17} # Plus Plug S d4d4daed2054
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-18
auth: password
deliver: {shelly-manager: 18} # Plus Plug S d4d4daecc47c
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-20
auth: password
deliver: {shelly-manager: 20} # Plus Plug S d4d4daecb690
acl: ["readwrite shellies/#", "write homeassistant/#"]
# Sleeping H&Ts (REN-1107): awake only 8-13 s every 1-2 h, so the
# controller pushes when the device publishes `online` on its wake topic
# (shelly-manager topic prefix `shellies/<hostname>`), not in the 2-min loop.
@@ -238,8 +211,3 @@
deliver: {shelly-manager: 38} # H&T G3 ecda3bc0b6a0
wake: shellies/shellyhtg3-ecda3bc0b6a0/online
acl: ["readwrite shellies/#", "write homeassistant/#"]
- name: shelly-40
auth: password
deliver: {shelly-manager: 40} # Flood G4 7c2c67732e08
wake: shellies/shellyfloodg4-7c2c67732e08/online
acl: ["readwrite shellies/#", "write homeassistant/#"]
@@ -100,9 +100,6 @@ spec:
# Keep tokens unhashed; enable hashing later as its own change.
- name: INFLUXD_USE_HASHED_TOKENS
value: "false"
# No usage telemetry to telemetry.influxdata.com (egress is denied).
- name: INFLUXD_REPORTING_DISABLED
value: "true"
# The chart defaults to liveness/readiness with initialDelaySeconds=0,
# so after a crash restart the kubelet kills the pod before shards
# finish opening. Give it 10 minutes to come up cleanly.
@@ -54,9 +54,6 @@ spec:
# non-RFC-framed payload). Scoped to this single-purpose receiver
# only — the pod-log Alloy DaemonSet stays at default stability.
stabilityLevel: experimental
# No usage reporting to stats.grafana.org (egress is denied anyway).
extraArgs:
- --disable-reporting
resources:
requests:
cpu: 20m
@@ -116,9 +116,6 @@ spec:
retries: 3
values:
alloy:
# No usage reporting to stats.grafana.org (egress is denied anyway).
extraArgs:
- --disable-reporting
configMap:
content: |
// Discover and collect K8s pod logs
@@ -156,39 +156,13 @@ spec:
k8sServicePort: 7445
# Hubble itself is on in every agent (chart default); the relay aggregates
# the three nodes' flow buffers behind one gRPC endpoint so `hubble observe`
# works without exec into kube-system (REN-487). The relay has no client
# auth; who may reach it is network-policies/kube-system-hubble-relay.yaml.
# works without exec into kube-system (REN-487). Renders no change to
# cilium-config or the agent DaemonSet, so the reloader annotation above
# does not roll the CNI. The relay has no client auth; who may reach it is
# network-policies/kube-system-hubble-relay.yaml.
hubble:
relay:
enabled: true
# Every policy drop goes to the agent's stdout and from there to Loki
# ({namespace="kube-system", container="cilium-agent"} |= "POLICY_DENIED"),
# so denied traffic can be reviewed after the fact. Changes cilium-config,
# so the reloader rolls the agents.
export:
static:
enabled: true
filePath: stdout
allowList:
- '{"verdict":["DROPPED"],"drop_reason_desc":["POLICY_DENIED"],"event_type":[{"type":1}]}'
fieldMask:
- time
- node_name
- verdict
- drop_reason_desc
- traffic_direction
- IP
- l4
- source.namespace
- source.pod_name
- source.identity
- source.workloads
- source_names
- destination.namespace
- destination.pod_name
- destination.identity
- destination.workloads
- destination_names
l2announcements:
enabled: true
externalIPs:
@@ -1,14 +1,11 @@
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
# 6h, not 1h: both repositories carry tens to hundreds of thousands of PR
# build tags, and every scan pages the full list anonymously, which runs
# into ghcr.io's rate limit.
metadata:
name: immich-server
namespace: flux-system
spec:
image: ghcr.io/immich-app/immich-server
interval: 6h
interval: 1h
---
# v-prefix tags only, semver 3.x. The 4.x cutover would need manual
# review — Immich major bumps historically require migration.
@@ -34,7 +31,7 @@ metadata:
namespace: flux-system
spec:
image: ghcr.io/immich-app/immich-machine-learning
interval: 6h
interval: 1h
---
# Same policy as immich-server; machine-learning image must stay in
# version-sync with the server image (Immich docs require it).
@@ -1,161 +0,0 @@
# Client networks as Cilium CIDR groups, referenced from CNPs via
# fromCIDRSet.cidrGroupRef. ULA + IPv4 only: the ISP-delegated GUA prefixes
# change and are not listed.
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-default
spec:
externalCIDRs:
- fd00::/64
- 192.168.8.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-management
spec:
externalCIDRs:
- fd02:10::/64
- 10.2.10.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-radius
spec:
externalCIDRs:
- fd02:11::/64
- 10.2.11.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-video
spec:
externalCIDRs:
- 10.2.20.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-iot
spec:
externalCIDRs:
- fd02:30::/64
- 10.2.30.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-server
spec:
externalCIDRs:
- fd02:40::/64
- fd40::/64
- 10.2.40.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-k8s-node
spec:
externalCIDRs:
- fd02:46::/64
- 10.2.46.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-k8s-ingress
spec:
externalCIDRs:
- fd02:47::/64
- 10.2.47.0/24
---
# The UDM's own address on k8s-ingress, which it sources from towards the
# cluster's LoadBalancer IPs.
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: udm-k8s-ingress
spec:
externalCIDRs:
- fd02:47::1/128
- 10.2.47.1/32
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-k8s-pod
spec:
externalCIDRs:
- fd02:48::/64
- 10.2.48.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-privat
spec:
externalCIDRs:
- fd02:50::/64
- 10.2.50.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-phone
spec:
externalCIDRs:
- 10.2.60.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-access
spec:
externalCIDRs:
- 10.2.70.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-vpn
spec:
externalCIDRs:
- 10.2.80.0/24
- 192.168.32.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-guest
spec:
externalCIDRs:
- 10.2.90.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-multi-transfer
spec:
externalCIDRs:
- 10.2.122.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-multi-lan
spec:
externalCIDRs:
- 10.2.123.0/24
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
name: net-mutti
spec:
externalCIDRs:
- 10.2.200.0/24
@@ -186,21 +186,14 @@ spec:
matchLabels:
app.kubernetes.io/name: mosquitto
ingress:
- fromCIDRSet:
- cidrGroupRef: net-iot
- fromEntities:
- world
toPorts:
- ports:
- port: "1883"
protocol: TCP
- port: "8883"
protocol: TCP
# MQTT clients on the private LAN, TLS only.
- fromCIDRSet:
- cidrGroupRef: net-privat
toPorts:
- ports:
- port: "8883"
protocol: TCP
---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
@@ -212,11 +205,8 @@ spec:
matchLabels:
app.kubernetes.io/name: home-assistant
ingress:
- fromCIDRSet:
- cidrGroupRef: net-privat
- cidrGroupRef: net-vpn
- cidrGroupRef: net-iot
- cidrGroupRef: net-server
- fromEntities:
- world
toPorts:
- ports:
- port: "8123"
@@ -235,8 +225,8 @@ spec:
matchLabels:
app.kubernetes.io/name: shelly-manager
ingress:
- fromCIDRSet:
- cidrGroupRef: net-iot
- fromEntities:
- world
toPorts:
- ports:
- port: "8082"
@@ -467,8 +457,7 @@ spec:
- port: "443"
protocol: TCP
---
# zigbee2mqtt: Zigbee coordinator (SLZB) TCP socket, and the OTA index and
# firmware images (Koenkk/zigbee-OTA) on raw.githubusercontent.com.
# zigbee2mqtt: Zigbee coordinator (SLZB) TCP socket.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
@@ -499,49 +488,8 @@ spec:
- ports:
- port: "6638"
protocol: TCP
- toFQDNs:
- matchName: "raw.githubusercontent.com"
toPorts:
- ports:
- port: "443"
protocol: TCP
---
# matter-server: the CSA DCL (PAA root certificates, vendor names, OTA
# metadata) and the development PAA certificates of project-chip on GitHub.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: matter-server-egress
namespace: home-assistant
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: matter-server
egress:
# rules.dns lets Cilium learn the toFQDNs mapping
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
rules:
dns:
- matchPattern: "*"
- toFQDNs:
- matchName: "on.dcl.csa-iot.org"
- matchName: "api.github.com"
- matchName: "raw.githubusercontent.com"
toPorts:
- ports:
- port: "443"
protocol: TCP
---
# mqtt-account-controller: credential delivery to deliver.esp32 targets in mqtt-accounts.yaml.
# mqtt-account-controller: credential delivery to edge-1 (deliver.esp32 in mqtt-accounts.yaml).
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
@@ -566,12 +514,11 @@ spec:
rules:
dns:
- matchPattern: "*"
# vitocal-can (open3e-esp32), plain HTTP API.
- toFQDNs:
- matchName: "open3e.iot.renner.uno"
- matchName: "edge-1.iot.renner.uno"
toPorts:
- ports:
- port: "80"
- port: "8443"
protocol: TCP
---
# whisper and speech-to-phrase: model downloads over HTTPS.
@@ -1,40 +0,0 @@
# CoreDNS is reachable from the LAN through the kube-dns-lb LoadBalancer
# (ETP=Local, client address reaches the pod). Selecting the pods makes
# them default-deny for ingress, so the whole cluster (pods, nodes, kubelet
# probes, Prometheus) is allowed on every port; LAN clients only on 53.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: coredns-ingress
namespace: kube-system
spec:
endpointSelector:
matchLabels:
k8s-app: kube-dns
ingress:
- fromEntities:
- cluster
- fromCIDRSet:
- cidrGroupRef: net-default
- cidrGroupRef: net-management
- cidrGroupRef: net-radius
- cidrGroupRef: net-video
- cidrGroupRef: net-iot
- cidrGroupRef: net-server
- cidrGroupRef: net-k8s-node
- cidrGroupRef: net-k8s-ingress
- cidrGroupRef: net-k8s-pod
- cidrGroupRef: net-privat
- cidrGroupRef: net-phone
- cidrGroupRef: net-access
- cidrGroupRef: net-vpn
- cidrGroupRef: net-guest
- cidrGroupRef: net-multi-transfer
- cidrGroupRef: net-multi-lan
- cidrGroupRef: net-mutti
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
@@ -1,7 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- cidr-groups.yaml
- default-deny.yaml
- allow-ingress.yaml
- ingress-isolation.yaml
@@ -41,4 +40,3 @@ resources:
- allow-cilium-health.yaml
- cnpg-barman-s3.yaml
- kube-system-hubble-relay.yaml
- kube-system-coredns.yaml
@@ -1,6 +1,13 @@
# Syslog senders reaching the alloy-syslog LB (ETP=Local, client address
# reaches the pod): network gear on Management, the SLZB coordinators on
# IoT, and the UDM itself, which sources from its k8s-ingress address.
# Targeted carve-out: alloy-syslog needs world ingress on the syslog
# ports specifically. The Service is ETP=Cluster, so external syslog
# senders (Proxmox, LXC hosts, network gear) hit the LB IP, get DNAT'd
# to an alloy pod, and get SNAT'd to the receiving node's cilium_host
# address. Cilium then sees the source as `world-ipv4` identity (the
# cilium_host IP isn't a known endpoint), NOT `host` — so the broad
# monitoring CNP's `fromEntities: host` rule doesn't catch it.
#
# Scoped via endpointSelector so this only widens ingress on the
# alloy-syslog pod, not the whole namespace.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
@@ -12,10 +19,8 @@ spec:
app.kubernetes.io/instance: alloy-syslog
app.kubernetes.io/name: alloy
ingress:
- fromCIDRSet:
- cidrGroupRef: net-management
- cidrGroupRef: net-iot
- cidrGroupRef: net-k8s-ingress
- fromEntities:
- world
toPorts:
- ports:
- port: "514"
@@ -112,7 +112,7 @@ spec:
- { type: 136, family: IPv6 }
- { type: 143, family: IPv6 }
---
# FTP from the scanners (IoT and Privat) to the ftp-scanner pod.
# Allow FTP access (from scanners on the LAN) to the ftp-scanner pod.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
@@ -123,24 +123,8 @@ spec:
matchLabels:
app.kubernetes.io/name: ftp-scanner
ingress:
- fromCIDRSet:
- cidrGroupRef: net-iot
- cidrGroupRef: net-privat
toPorts:
- ports:
- port: "21"
protocol: TCP
- port: "21100"
protocol: TCP
- port: "21101"
protocol: TCP
- port: "21102"
protocol: TCP
- port: "21103"
protocol: TCP
- port: "21104"
protocol: TCP
- fromEntities:
- world
- host
- remote-node
toPorts:
@@ -158,7 +142,7 @@ spec:
- port: "21104"
protocol: TCP
---
# Allow Syncthing peer-to-peer sync from Privat and VPN.
# Allow Syncthing peer-to-peer sync from the LAN/world.
# - 22000 TCP+QUIC: actual sync traffic between peers.
# - 21027 UDP: local-discovery broadcast packets from peers on the LAN.
# UI (8384) is intentionally NOT listed — authentik-gate-sync generates the
@@ -173,18 +157,8 @@ spec:
matchLabels:
app.kubernetes.io/name: syncthing
ingress:
- fromCIDRSet:
- cidrGroupRef: net-privat
- cidrGroupRef: net-vpn
toPorts:
- ports:
- port: "22000"
protocol: TCP
- port: "22000"
protocol: UDP
- port: "21027"
protocol: UDP
- fromEntities:
- world
- host
- remote-node
toPorts:
@@ -62,10 +62,9 @@ spec:
# CA, so this bypasses Traefik entirely and the namespace default-deny has
# to be opened for it — same shape as paperless's allow-ftp-scanner.
#
# Scoped to the terminus pods, port 2345 and the IoT network (the panel's
# network; the LB is ETP=Local, so the client address reaches the pod).
# Requests still have to carry a known device MAC in the `ID` header plus
# its api_key; this rule opens the port, not the application.
# Scoped to the terminus pods and port 2345 only. Requests still have to
# carry a known device MAC in the `ID` header plus its api_key; this rule
# opens the port, not the application.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
@@ -76,13 +75,8 @@ spec:
matchLabels:
app.kubernetes.io/name: terminus
ingress:
- fromCIDRSet:
- cidrGroupRef: net-iot
toPorts:
- ports:
- port: "2345"
protocol: TCP
- fromEntities:
- world
- host
- remote-node
toPorts:
@@ -160,12 +154,6 @@ spec:
---
# In-cluster + DNS for every pod except terminus (terminus-egress is its
# allowlist; selecting it here would widen it to all of `cluster`).
#
# trmnl-renderer's Chromium calls clients2.google.com:80,
# update.googleapis.com:443 and accounts.google.com:443; these drops are
# expected and harmless. The launch flags are fixed in upstream
# trmnl-ha/ha-trmnl/screenshot.ts (PUPPETEER_ARGS, no override) and lack
# --disable-background-networking.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
+7 -8
View File
@@ -68,14 +68,13 @@ locals {
# idx is a static per-network slot for rule_index derivation — never
# reuse a freed idx for a new network.
#
# Deliberately absent: (40) Server idx=4, (49) k8s Ceph idx=6,
# (46) k8s Node, (47) k8s Ingress and (48) k8s Pod. The cluster nodes
# have interfaces on those VLANs (mgmt0, node0, ingress0, pod0), so a
# DNATed flow's reply would go directly over the shared L2 and bypass
# the UDM's conntrack (asymmetric, connection drops) — and clients there
# are the nodes themselves, whose DNS chain (CoreDNS -> k8s_gateway)
# already resolves the public names in-cluster. idx=5 stays reserved
# (its network, (42) Ingress, no longer exists).
# Deliberately absent: (40) Server idx=4, (42) Ingress idx=5, and
# (49) k8s Ceph idx=6. The cluster nodes have interfaces on those VLANs,
# so a DNATed flow's reply would go directly over the shared L2 and
# bypass the UDM's conntrack (asymmetric, connection drops) — and
# clients there are the nodes themselves, whose DNS chain (CoreDNS ->
# k8s_gateway) already resolves the public names in-cluster. Their idx
# slots stay reserved.
hairpin_networks = {
"Default" = { id = "5d4fba77ce63ee614841d53d", idx = 0 }
"(10) Management" = { id = "600334abf9ee0c059da6be8f", idx = 1 }