feat(iot): v30iot workloads run only where a node declares the IoT VLAN; HA no longer waits on them
Five workloads attach a macvlan on ens19 (v30iot) with DHCP IPAM: shelly-manager and the mDNS relay (inside the home-assistant Kustomization), matter-server, music-assistant and otbr. On a node without that VLAN the CNI add fails and the pod never starts. - k8s-1..3 carry net.renner.uno/iot-vlan: "true"; each of them declares ens19 in its VLANConfig. The five Deployments select on it. The comments they replace described the retired k8s-edge taint. - shelly-manager and the mDNS relay move to apps/home-assistant-iot, with a Kustomization of their own (depends on gitea, home-assistant and the infrastructure Kustomizations). home-assistant's wait: true no longer covers them, so HA and everything waiting on it (evcc, trmnl, zone-configurator, speech-to-phrase) are Ready without the VLAN, and a broken IoT bridge no longer holds them back in production either. - shelly-manager-auth stays in apps/home-assistant: mqtt-account-controller mounts it too, and that Kustomization comes first. Moved with shelly-manager, mqtt-account-controller would wait for a Secret that only the later Kustomization creates. - The probes-blackbox and authentik blueprint comments name the new paths. On a site without the label the IoT Kustomizations fail at their timeout, naming the Pending pods; nothing else depends on them. Production rollout: 1. Label the nodes first (talosctl patch the nodeLabels), or the five pods go Pending. 2. The move between Kustomizations is two-phase (add under home-assistant-iot, then drop from home-assistant). shelly-manager's PVC already carries prune: disabled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -131,7 +131,7 @@ data:
|
||||
state: absent
|
||||
|
||||
# shelly-manager.home.renner.uno is gated by authentik-gate-sync
|
||||
# (kubernetes/apps/home-assistant/shelly-manager-ingress.yaml), which
|
||||
# (kubernetes/apps/home-assistant-iot/shelly-manager-ingress.yaml), which
|
||||
# admits this group. Members are managed in Authentik.
|
||||
- model: authentik_core.group
|
||||
identifiers:
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
# The home-assistant workloads attached to the IoT VLAN (v30iot). Separate from
|
||||
# ../home-assistant so that HA and what depends on it do not wait on a VLAN
|
||||
# the node may not have; the pods run only where it does
|
||||
# (net.renner.uno/iot-vlan).
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- shelly-manager-pvc.yaml
|
||||
- shelly-manager-network-attachment.yaml
|
||||
- shelly-manager-firmware-rbac.yaml
|
||||
- shelly-ble-proxy-key-externalsecret.yaml
|
||||
# shelly-manager-auth comes from ../home-assistant: mqtt-account-controller
|
||||
# mounts it too, and that Kustomization comes first.
|
||||
- shelly-manager-llat-externalsecret.yaml
|
||||
- shelly-manager-deployment.yaml
|
||||
- shelly-manager-service.yaml
|
||||
- shelly-manager-ingress.yaml
|
||||
# mDNS relay — carries IoT-VLAN announcements into HA's zeroconf listener as
|
||||
# unicast, which is the only direction the CNI will move them. Without it HA
|
||||
# never learns that a device's DHCP lease moved. See the header comment in
|
||||
# mdns-relay-deployment.yaml.
|
||||
- mdns-relay-network-attachment.yaml
|
||||
- mdns-relay-service.yaml
|
||||
- mdns-relay-deployment.yaml
|
||||
configMapGenerator:
|
||||
# shelly-manager api-proxy sidecar config (REN-935). Hash suffix disabled
|
||||
# (name stays stable), so editing shelly-manager-api-proxy.conf does not
|
||||
# auto-roll the pod — restart manually with
|
||||
# `kubectl rollout restart deploy/shelly-manager -n home-assistant`.
|
||||
- name: shelly-manager-api-proxy
|
||||
namespace: home-assistant
|
||||
files:
|
||||
- api.conf.template=shelly-manager-api-proxy.conf
|
||||
generatorOptions:
|
||||
disableNameSuffixHash: true
|
||||
+3
@@ -47,6 +47,9 @@ spec:
|
||||
homelab.renner.uno/restarted-for: "2026-10-06 cni-plugins v1.9.1 attach check"
|
||||
k8s.v1.cni.cncf.io/networks: mdns-relay-iot
|
||||
spec:
|
||||
# Runs where the node patch declares the IoT VLAN (ens19).
|
||||
nodeSelector:
|
||||
net.renner.uno/iot-vlan: "true"
|
||||
enableServiceLinks: false
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
+4
-4
@@ -47,10 +47,10 @@ spec:
|
||||
spec:
|
||||
# eth0 (Cilium-managed pod network) is still the primary — it
|
||||
# carries the ingress, mosquitto LB return path, etc. The macvlan
|
||||
# above is additive. No explicit edge exclusion is needed: the
|
||||
# `workload-class=pinned:NoSchedule` taint on k8s-edge already
|
||||
# keeps this pod off there (we never tolerate it), and the NAD's
|
||||
# `master: ens19` (stable predictable name for the v30iot NIC) is present on every k8s-{1,2,3} worker.
|
||||
# above is additive.
|
||||
# Runs where the node patch declares the IoT VLAN (ens19).
|
||||
nodeSelector:
|
||||
net.renner.uno/iot-vlan: "true"
|
||||
enableServiceLinks: false
|
||||
# Grants shelly-manager the in-cluster identity used by the firmware
|
||||
# pipeline to create esphome flash Jobs and stream their logs. RBAC is
|
||||
@@ -21,26 +21,11 @@ resources:
|
||||
- mosquitto-iot-service.yaml
|
||||
- mosquitto-deployment.yaml
|
||||
- mosquitto-evcc-credentials-externalsecret.yaml
|
||||
- shelly-manager-pvc.yaml
|
||||
- shelly-manager-network-attachment.yaml
|
||||
- shelly-manager-firmware-rbac.yaml
|
||||
- shelly-ble-proxy-key-externalsecret.yaml
|
||||
- shelly-manager-auth-externalsecret.yaml
|
||||
- shelly-manager-llat-externalsecret.yaml
|
||||
- shelly-manager-deployment.yaml
|
||||
- shelly-manager-service.yaml
|
||||
- shelly-manager-ingress.yaml
|
||||
# shelly-manager and the mDNS relay sit on the IoT VLAN: ../home-assistant-iot.
|
||||
- esphome-pvc.yaml
|
||||
- esphome-deployment.yaml
|
||||
- esphome-service.yaml
|
||||
- esphome-ingress.yaml
|
||||
# mDNS relay — carries IoT-VLAN announcements into HA's zeroconf listener as
|
||||
# unicast, which is the only direction the CNI will move them. Without it HA
|
||||
# never learns that a device's DHCP lease moved. See the header comment in
|
||||
# mdns-relay-deployment.yaml.
|
||||
- mdns-relay-network-attachment.yaml
|
||||
- mdns-relay-service.yaml
|
||||
- mdns-relay-deployment.yaml
|
||||
# Zigbee2MQTT — ZHA replacement bridge. Ships scaled to 0; the cutover
|
||||
# runbook (zigbee2mqtt-migration-runbook.md) scales it up. The MQTT
|
||||
# `zigbee2mqtt` broker user is provisioned by mosquitto-bootstrap.yaml
|
||||
@@ -66,6 +51,8 @@ resources:
|
||||
# Register: mqtt-accounts.yaml (configMapGenerator).
|
||||
- mosquitto-dynsec-test.yaml
|
||||
- mqtt-account-controller.yaml
|
||||
# Also mounted by shelly-manager (../home-assistant-iot), which runs after this.
|
||||
- shelly-manager-auth-externalsecret.yaml
|
||||
configMapGenerator:
|
||||
# nginx config for the reverse-proxy sidecar. Hash suffix disabled so
|
||||
# the cm name stays stable; editing proxy.conf does not auto-roll the
|
||||
@@ -116,13 +103,5 @@ configMapGenerator:
|
||||
namespace: home-assistant
|
||||
files:
|
||||
- mqtt-accounts.yaml
|
||||
# shelly-manager api-proxy sidecar config (REN-935). Hash suffix disabled
|
||||
# (name stays stable), so editing shelly-manager-api-proxy.conf does not
|
||||
# auto-roll the pod — restart manually with
|
||||
# `kubectl rollout restart deploy/shelly-manager -n home-assistant`.
|
||||
- name: shelly-manager-api-proxy
|
||||
namespace: home-assistant
|
||||
files:
|
||||
- api.conf.template=shelly-manager-api-proxy.conf
|
||||
generatorOptions:
|
||||
disableNameSuffixHash: true
|
||||
|
||||
@@ -32,11 +32,9 @@ spec:
|
||||
# OTBR via OTBR's ClusterIP Service). The macvlan attachment
|
||||
# above is additive, not a replacement.
|
||||
#
|
||||
# No nodeSelector/toleration: this pod only needs v30iot (present
|
||||
# on all nodes), not the edge USB devices. Dropping the
|
||||
# workload-class=pinned toleration lets the edge taint keep it off
|
||||
# the (tiny) edge box, reserving that node for the USB-bound
|
||||
# home-assistant + zwave-js pods.
|
||||
# Runs where the node patch declares the IoT VLAN (ens19).
|
||||
nodeSelector:
|
||||
net.renner.uno/iot-vlan: "true"
|
||||
containers:
|
||||
- name: matter-server
|
||||
# Use the HAOS-published image from `ghcr.io/matter-js/...`,
|
||||
|
||||
@@ -269,9 +269,9 @@ spec:
|
||||
# per the app's own test (shelly-manager repo,
|
||||
# AuthenticationTests.Health_IsUnauthenticated); it is also the pod's
|
||||
# kubelet liveness/readiness path
|
||||
# (../home-assistant/shelly-manager-deployment.yaml). AllowedHosts is
|
||||
# (../home-assistant-iot/shelly-manager-deployment.yaml). AllowedHosts is
|
||||
# "*", so the cluster-local Host header is accepted. Service :80 ->
|
||||
# container :8080 (../home-assistant/shelly-manager-service.yaml).
|
||||
# container :8080 (../home-assistant-iot/shelly-manager-service.yaml).
|
||||
#
|
||||
# NO CiliumNetworkPolicy change: the home-assistant default-deny
|
||||
# admits the whole monitoring namespace on every port
|
||||
|
||||
@@ -33,12 +33,9 @@ spec:
|
||||
k8s.v1.cni.cncf.io/networks: music-assistant-iot
|
||||
spec:
|
||||
enableServiceLinks: false
|
||||
# No nodeSelector/toleration: MA only needs a node on v30iot (all
|
||||
# of them) for audio-sync L2, not the edge USB devices. It's the
|
||||
# heaviest of the formerly edge-pinned pods (2Gi limit), so
|
||||
# dropping the workload-class=pinned toleration — which lets the
|
||||
# edge taint keep it off the tiny edge box — is the main capacity
|
||||
# win for this step.
|
||||
# Runs where the node patch declares the IoT VLAN (ens19).
|
||||
nodeSelector:
|
||||
net.renner.uno/iot-vlan: "true"
|
||||
initContainers:
|
||||
# MA advertises a stored publish_ip to players (Sendspin / Snapcast /
|
||||
# SlimProto) and binds its stream + Sendspin servers to a stored
|
||||
|
||||
@@ -30,11 +30,10 @@ spec:
|
||||
# the IoT VLAN for mDNS/SRP advertisements and RA listening.
|
||||
k8s.v1.cni.cncf.io/networks: otbr-iot
|
||||
spec:
|
||||
# No nodeSelector/toleration: OTBR only needs v30iot L2 presence
|
||||
# (macvlan master: ens19, present on every node), not the edge USB
|
||||
# devices, and its Thread RCP is reached over the network
|
||||
# (slzb-thread.renner.uno). Dropping the workload-class=pinned
|
||||
# toleration lets the edge taint keep it off the tiny edge box.
|
||||
# Runs where the node patch declares the IoT VLAN (ens19). Its Thread
|
||||
# RCP is reached over the network (slzb-thread.renner.uno).
|
||||
nodeSelector:
|
||||
net.renner.uno/iot-vlan: "true"
|
||||
containers:
|
||||
# The Thread RCP is a network-attached radio (Serial-over-IP), so the
|
||||
# spinel transport needs a TCP-to-PTY bridge. Two obvious approaches
|
||||
|
||||
@@ -35,7 +35,7 @@ spec:
|
||||
# it took speech-to-phrase, zone-configurator and matter-server down with
|
||||
# it, since all three wait on home-assistant.
|
||||
- name: infrastructure-cnpg-barman
|
||||
- name: gitea # shelly-manager, mqtt-account-controller images are in Gitea's registry
|
||||
- name: gitea # mqtt-account-controller's image is in Gitea's registry
|
||||
- name: influxdb # HA's InfluxDB integration writes there
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
@@ -58,6 +58,29 @@ spec:
|
||||
- name: infrastructure-crds
|
||||
- name: infrastructure-ceph-cluster
|
||||
---
|
||||
# shelly-manager and the mDNS relay: home-assistant's IoT-VLAN workloads.
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: home-assistant-iot
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 10m
|
||||
retryInterval: 1m
|
||||
timeout: 10m
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
path: ./kubernetes/apps/home-assistant-iot
|
||||
prune: true
|
||||
wait: true
|
||||
dependsOn:
|
||||
- name: infrastructure
|
||||
- name: infrastructure-crds
|
||||
- name: infrastructure-ceph-cluster
|
||||
- name: gitea # shelly-manager's image is in Gitea's registry
|
||||
- name: home-assistant # both serve HA: its API, its zeroconf listener
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
|
||||
Reference in New Issue
Block a user