feat(iot): v30iot workloads run only where a node declares the IoT VLAN; HA no longer waits on them

Five workloads attach a macvlan on ens19 (v30iot) with DHCP IPAM:
shelly-manager and the mDNS relay (inside the home-assistant Kustomization),
matter-server, music-assistant and otbr. On a node without that VLAN the CNI
add fails and the pod never starts.

- k8s-1..3 carry net.renner.uno/iot-vlan: "true" beside the GPU label; each
  of them declares ens19 in its VLANConfig. The five Deployments select on
  it. The comments they replace described the retired k8s-edge taint.
- shelly-manager and the mDNS relay move to apps/home-assistant-iot, with a
  Kustomization of their own (depends on restore-home-assistant, gitea,
  home-assistant and the infrastructure Kustomizations). home-assistant's
  wait: true no longer covers them, so HA and everything waiting on it
  (evcc, trmnl, zone-configurator, speech-to-phrase) are Ready without the
  VLAN, and a broken IoT bridge no longer holds them back in production
  either.

On a site without the label the IoT Kustomizations fail at their timeout,
naming the Pending pods; nothing else depends on them.

Verified: home-assistant, home-assistant-iot, matter-server,
music-assistant, otbr, authentik and monitoring-rules build; the
home-assistant render has no IoT attachment left; every IoT workload carries
the selector; k8s-1 with site-homelab passes talosctl validate --strict.

Production rollout:
1. Label the nodes first (talosctl patch the nodeLabels), or the five pods go
   Pending.
2. The move between Kustomizations is two-phase (add under
   home-assistant-iot, then drop from home-assistant). shelly-manager's PVC
   already carries prune: disabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-16 21:36:01 +02:00
co-authored by Claude Opus 5.5
parent cffb7e027a
commit 420bd35e7f
24 changed files with 87 additions and 48 deletions
@@ -131,7 +131,7 @@ data:
state: absent
# shelly-manager.home.renner.uno is gated by authentik-gate-sync
# (kubernetes/apps/home-assistant/shelly-manager-ingress.yaml), which
# (kubernetes/apps/home-assistant-iot/shelly-manager-ingress.yaml), which
# admits this group. Members are managed in Authentik.
- model: authentik_core.group
identifiers:
@@ -0,0 +1,34 @@
# The home-assistant workloads attached to the IoT VLAN (v30iot). Separate from
# ../home-assistant so that HA and what depends on it do not wait on a VLAN
# the node may not have; the pods run only where it does
# (net.renner.uno/iot-vlan).
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- shelly-manager-pvc.yaml
- shelly-manager-network-attachment.yaml
- shelly-manager-firmware-rbac.yaml
- shelly-ble-proxy-key-externalsecret.yaml
- shelly-manager-auth-externalsecret.yaml
- shelly-manager-llat-externalsecret.yaml
- shelly-manager-deployment.yaml
- shelly-manager-service.yaml
- shelly-manager-ingress.yaml
# mDNS relay — carries IoT-VLAN announcements into HA's zeroconf listener as
# unicast, which is the only direction the CNI will move them. Without it HA
# never learns that a device's DHCP lease moved. See the header comment in
# mdns-relay-deployment.yaml.
- mdns-relay-network-attachment.yaml
- mdns-relay-service.yaml
- mdns-relay-deployment.yaml
configMapGenerator:
# shelly-manager api-proxy sidecar config (REN-935). Hash suffix disabled
# (name stays stable), so editing shelly-manager-api-proxy.conf does not
# auto-roll the pod — restart manually with
# `kubectl rollout restart deploy/shelly-manager -n home-assistant`.
- name: shelly-manager-api-proxy
namespace: home-assistant
files:
- api.conf.template=shelly-manager-api-proxy.conf
generatorOptions:
disableNameSuffixHash: true
@@ -47,6 +47,9 @@ spec:
homelab.renner.uno/restarted-for: "2026-10-06 cni-plugins v1.9.1 attach check"
k8s.v1.cni.cncf.io/networks: mdns-relay-iot
spec:
# Runs where the node patch declares the IoT VLAN (ens19).
nodeSelector:
net.renner.uno/iot-vlan: "true"
enableServiceLinks: false
securityContext:
runAsNonRoot: true
@@ -47,10 +47,10 @@ spec:
spec:
# eth0 (Cilium-managed pod network) is still the primary — it
# carries the ingress, mosquitto LB return path, etc. The macvlan
# above is additive. No explicit edge exclusion is needed: the
# `workload-class=pinned:NoSchedule` taint on k8s-edge already
# keeps this pod off there (we never tolerate it), and the NAD's
# `master: ens19` (stable predictable name for the v30iot NIC) is present on every k8s-{1,2,3} worker.
# above is additive.
# Runs where the node patch declares the IoT VLAN (ens19).
nodeSelector:
net.renner.uno/iot-vlan: "true"
enableServiceLinks: false
# Grants shelly-manager the in-cluster identity used by the firmware
# pipeline to create esphome flash Jobs and stream their logs. RBAC is
@@ -21,26 +21,11 @@ resources:
- mosquitto-iot-service.yaml
- mosquitto-deployment.yaml
- mosquitto-evcc-credentials-externalsecret.yaml
- shelly-manager-pvc.yaml
- shelly-manager-network-attachment.yaml
- shelly-manager-firmware-rbac.yaml
- shelly-ble-proxy-key-externalsecret.yaml
- shelly-manager-auth-externalsecret.yaml
- shelly-manager-llat-externalsecret.yaml
- shelly-manager-deployment.yaml
- shelly-manager-service.yaml
- shelly-manager-ingress.yaml
# shelly-manager and the mDNS relay sit on the IoT VLAN: ../home-assistant-iot.
- esphome-pvc.yaml
- esphome-deployment.yaml
- esphome-service.yaml
- esphome-ingress.yaml
# mDNS relay — carries IoT-VLAN announcements into HA's zeroconf listener as
# unicast, which is the only direction the CNI will move them. Without it HA
# never learns that a device's DHCP lease moved. See the header comment in
# mdns-relay-deployment.yaml.
- mdns-relay-network-attachment.yaml
- mdns-relay-service.yaml
- mdns-relay-deployment.yaml
# Zigbee2MQTT — ZHA replacement bridge. Ships scaled to 0; the cutover
# runbook (zigbee2mqtt-migration-runbook.md) scales it up. The MQTT
# `zigbee2mqtt` broker user is provisioned by mosquitto-bootstrap.yaml
@@ -115,13 +100,5 @@ configMapGenerator:
namespace: home-assistant
files:
- mqtt-accounts.yaml
# shelly-manager api-proxy sidecar config (REN-935). Hash suffix disabled
# (name stays stable), so editing shelly-manager-api-proxy.conf does not
# auto-roll the pod — restart manually with
# `kubectl rollout restart deploy/shelly-manager -n home-assistant`.
- name: shelly-manager-api-proxy
namespace: home-assistant
files:
- api.conf.template=shelly-manager-api-proxy.conf
generatorOptions:
disableNameSuffixHash: true
@@ -32,11 +32,9 @@ spec:
# OTBR via OTBR's ClusterIP Service). The macvlan attachment
# above is additive, not a replacement.
#
# No nodeSelector/toleration: this pod only needs v30iot (present
# on all nodes), not the edge USB devices. Dropping the
# workload-class=pinned toleration lets the edge taint keep it off
# the (tiny) edge box, reserving that node for the USB-bound
# home-assistant + zwave-js pods.
# Runs where the node patch declares the IoT VLAN (ens19).
nodeSelector:
net.renner.uno/iot-vlan: "true"
containers:
- name: matter-server
# Use the HAOS-published image from `ghcr.io/matter-js/...`,
@@ -269,9 +269,9 @@ spec:
# per the app's own test (shelly-manager repo,
# AuthenticationTests.Health_IsUnauthenticated); it is also the pod's
# kubelet liveness/readiness path
# (../home-assistant/shelly-manager-deployment.yaml). AllowedHosts is
# (../home-assistant-iot/shelly-manager-deployment.yaml). AllowedHosts is
# "*", so the cluster-local Host header is accepted. Service :80 ->
# container :8080 (../home-assistant/shelly-manager-service.yaml).
# container :8080 (../home-assistant-iot/shelly-manager-service.yaml).
#
# NO CiliumNetworkPolicy change: the home-assistant default-deny
# admits the whole monitoring namespace on every port
@@ -33,12 +33,9 @@ spec:
k8s.v1.cni.cncf.io/networks: music-assistant-iot
spec:
enableServiceLinks: false
# No nodeSelector/toleration: MA only needs a node on v30iot (all
# of them) for audio-sync L2, not the edge USB devices. It's the
# heaviest of the formerly edge-pinned pods (2Gi limit), so
# dropping the workload-class=pinned toleration — which lets the
# edge taint keep it off the tiny edge box — is the main capacity
# win for this step.
# Runs where the node patch declares the IoT VLAN (ens19).
nodeSelector:
net.renner.uno/iot-vlan: "true"
initContainers:
# MA advertises a stored publish_ip to players (Sendspin / Snapcast /
# SlimProto) and binds its stream + Sendspin servers to a stored
+4 -5
View File
@@ -30,11 +30,10 @@ spec:
# the IoT VLAN for mDNS/SRP advertisements and RA listening.
k8s.v1.cni.cncf.io/networks: otbr-iot
spec:
# No nodeSelector/toleration: OTBR only needs v30iot L2 presence
# (macvlan master: ens19, present on every node), not the edge USB
# devices, and its Thread RCP is reached over the network
# (slzb-thread.renner.uno). Dropping the workload-class=pinned
# toleration lets the edge taint keep it off the tiny edge box.
# Runs where the node patch declares the IoT VLAN (ens19). Its Thread
# RCP is reached over the network (slzb-thread.renner.uno).
nodeSelector:
net.renner.uno/iot-vlan: "true"
containers:
# The Thread RCP is a network-attached radio (Serial-over-IP), so the
# spinel transport needs a TCP-to-PTY bridge. Two obvious approaches
@@ -28,7 +28,7 @@ spec:
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
- name: cnpg-object-stores # the ObjectStores its Cluster names
- name: gitea # shelly-manager, mqtt-account-controller images are in Gitea's registry
- name: gitea # mqtt-account-controller's image is in Gitea's registry
- name: influxdb # HA's InfluxDB integration writes there
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
@@ -52,6 +52,30 @@ spec:
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
---
# shelly-manager and the mDNS relay: home-assistant's IoT-VLAN workloads.
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: home-assistant-iot
namespace: flux-system
spec:
interval: 10m
retryInterval: 1m
timeout: 10m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/home-assistant-iot
prune: true
wait: true
dependsOn:
- name: restore-home-assistant # shelly-manager's PVC
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
- name: gitea # shelly-manager's image is in Gitea's registry
- name: home-assistant # both serve HA: its API, its zeroconf listener
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
+3
View File
@@ -57,6 +57,9 @@ machine:
# This machine has the Intel iGPU. The GPU plugin and everything that needs
# gpu.intel.com run only on nodes with this label.
hardware.renner.uno/intel-gpu: "true"
# This machine is on the IoT VLAN: ens19, declared below in its VLANConfig.
# The macvlan workloads on v30iot run only on nodes with this label.
net.renner.uno/iot-vlan: "true"
# Publishes this node's pod0 addresses, declared below in its VLANConfig, so
# the bird DaemonSet reads its peers from the node list rather than from a
# copy. Annotations, not labels: a label value cannot contain a colon.
+2
View File
@@ -36,6 +36,8 @@ machine:
storage.renner.uno/ceph-osd: "true"
# Has the Intel iGPU — see k8s-1.yaml.
hardware.renner.uno/intel-gpu: "true"
# See k8s-1.yaml.
net.renner.uno/iot-vlan: "true"
# pod0 addresses for the bird peer list — see k8s-1.yaml.
nodeAnnotations:
net.renner.uno/pod0-v6: fd02:48::12
+2
View File
@@ -32,6 +32,8 @@ machine:
storage.renner.uno/ceph-osd: "true"
# Has the Intel iGPU — see k8s-1.yaml.
hardware.renner.uno/intel-gpu: "true"
# See k8s-1.yaml.
net.renner.uno/iot-vlan: "true"
# pod0 addresses for the bird peer list — see k8s-1.yaml.
nodeAnnotations:
net.renner.uno/pod0-v6: fd02:48::13