Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c90406cca7 | ||
|
|
14d62f3aa1 | ||
|
|
85b8f0eba9 |
@@ -1,80 +1,22 @@
|
||||
---
|
||||
# mDNS relay: gives Home Assistant sight of the IoT VLAN's mDNS announcements
|
||||
# without putting a macvlan on the HA pod itself.
|
||||
# mDNS relay: holds the multicast membership on the IoT VLAN through its own
|
||||
# macvlan and forwards each datagram to Home Assistant as ordinary UNICAST UDP.
|
||||
#
|
||||
# THE PROBLEM
|
||||
# It must stay unicast. An avahi-style reflector cannot work here — Cilium runs
|
||||
# in tunnel mode without enable-multicast, so re-emitted multicast is dropped by
|
||||
# the CNI. Unicast is carried, and python-zeroconf accepts it: its datagram
|
||||
# handler checks size and duplicates but never the destination or source
|
||||
# address. RFC 6762's QU bit makes this protocol-legal rather than a trick.
|
||||
#
|
||||
# HA's zeroconf listener joins both mDNS groups on eth0 — 224.0.0.251 and
|
||||
# ff02::fb — and never hears anything on either.
|
||||
# Multicast is link-local, so IoT-VLAN announcements do not reach the pod
|
||||
# CIDR; and Cilium runs in tunnel mode (VXLAN) without `enable-multicast`, so
|
||||
# it does not replicate multicast between pods either. Both halves of the path
|
||||
# are dead, which is why an avahi-style reflector — receive on one interface,
|
||||
# re-emit as multicast on the other — cannot work here: the re-emitted packets
|
||||
# would be dropped by the CNI.
|
||||
# This restores passive reception only — announcements, including the
|
||||
# re-announce a device sends when its address changes. HA-initiated browsing
|
||||
# still dies on eth0; that would need a querier on the VLAN side, which this
|
||||
# deliberately is not.
|
||||
#
|
||||
# Consequence for HA: the ESPHome integration dials devices at the address
|
||||
# cached in its config entry. When a lease rotates, every attempt fails with
|
||||
# `[Errno 113] Connect call failed` and the device stays unreachable, because
|
||||
# the mDNS announcement that would carry the new address never arrives. New
|
||||
# devices are never discovered at all.
|
||||
#
|
||||
# THE MECHANISM
|
||||
#
|
||||
# This relay holds the multicast membership on the IoT VLAN through its own
|
||||
# macvlan (mdns-relay-network-attachment.yaml) and forwards each datagram it
|
||||
# receives to HA as ordinary UNICAST UDP. That is the one direction of travel
|
||||
# the CNI will carry.
|
||||
#
|
||||
# It works because python-zeroconf does not filter on how a packet arrived:
|
||||
# its datagram handler validates size, deduplicates, and parses, but performs
|
||||
# no check on destination address, source address, or source port. A unicast
|
||||
# datagram delivered to :5353 is processed exactly like a multicast one. (mDNS
|
||||
# unicast responses are part of RFC 6762 — the QU bit — so this is within the
|
||||
# protocol, not a trick.)
|
||||
#
|
||||
# WHAT THIS DOES AND DOES NOT RESTORE
|
||||
#
|
||||
# Restored: passive reception. RFC 6762 requires a responder to re-announce
|
||||
# when its address changes, so a device that moves to a new lease announces
|
||||
# the move, HA sees it, and the ESPHome integration updates the config entry.
|
||||
# That is the failure this exists to fix. New devices announcing themselves
|
||||
# are likewise seen.
|
||||
#
|
||||
# NOT restored: HA-initiated browsing. Queries HA sends leave on eth0 as
|
||||
# multicast and die there, so HA cannot actively solicit a response. Anything
|
||||
# depending on a query rather than an announcement stays broken. Making that
|
||||
# work needs a querier on the VLAN side, which this relay deliberately is not.
|
||||
#
|
||||
# EXPOSURE
|
||||
#
|
||||
# The receive sockets carry so-bindtodevice=net1, so only datagrams that
|
||||
# actually arrived on the IoT VLAN interface are delivered. Binding to net1's
|
||||
# address instead is not possible — it is DHCP-leased and unknown at exec time
|
||||
# — but binding to the device achieves the same confinement without needing to
|
||||
# know the address, and needs no capability: SO_BINDTODEVICE succeeds as an
|
||||
# unprivileged uid here.
|
||||
#
|
||||
# Without it socat would bind the wildcard, and since the namespace CNP grants
|
||||
# every pod in home-assistant unrestricted access to every other pod on every
|
||||
# port, any sibling pod could reach :5353 over the cluster network and have
|
||||
# records forwarded into HA's zeroconf cache.
|
||||
#
|
||||
# What remains, and cannot be closed here: any host on the IoT VLAN can send
|
||||
# this port a datagram. That is the same trust already extended to genuine
|
||||
# mDNS from that segment, which any device there can equally forge — the relay
|
||||
# widens who can reach HA's listener, not what can be claimed. macvlan traffic
|
||||
# bypasses Cilium policy entirely, so the namespace CNP does not constrain
|
||||
# this interface either way.
|
||||
#
|
||||
# Note that the relay is not what makes HA's zeroconf reachable from inside
|
||||
# the cluster: HA binds :5353 on the pod address (v4) and the wildcard (v6),
|
||||
# and the namespace CNP already permits any sibling pod to reach it directly.
|
||||
# Confining the relay removes a second route to that listener; it does not
|
||||
# remove the first.
|
||||
#
|
||||
# Nothing else is exposed: the containers run socat and only socat, with no
|
||||
# second listener to reach.
|
||||
# so-bindtodevice=net1 on both receive sockets is load-bearing: without it
|
||||
# socat binds the wildcard, and the namespace CNP lets any sibling pod reach
|
||||
# :5353 and inject records into HA's zeroconf cache. Binding net1's address
|
||||
# instead is impossible — it is DHCP-leased and unknown at exec time.
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
|
||||
@@ -6,32 +6,23 @@ metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: mosquitto
|
||||
annotations:
|
||||
# REQUIRED for credential rotation to take effect at all. The startup shim
|
||||
# rebuilds /mosquitto/auth/passwd from these Secrets on every pod start, so
|
||||
# a rotated Secret is inert until the pod restarts — without this the new
|
||||
# password would sit unused until some unrelated restart, while the old one
|
||||
# stayed valid indefinitely.
|
||||
# REQUIRED for credential rotation to take effect. The startup shim rebuilds
|
||||
# the password file only on pod start, so without this a rotated Secret sits
|
||||
# inert while the old password stays valid indefinitely.
|
||||
#
|
||||
# Listed per-Secret rather than using the blanket `reloader.stakater.com/
|
||||
# auto: "true"`, so that enrolling each remaining broker user in rotation is
|
||||
# a visible, reviewable edit to this line rather than something that starts
|
||||
# happening implicitly.
|
||||
# Listed per-Secret rather than with the blanket auto annotation, so that
|
||||
# enrolling another broker user is a visible edit to this line.
|
||||
secret.reloader.stakater.com/reload: mosquitto-health-credentials
|
||||
spec:
|
||||
replicas: 1
|
||||
strategy:
|
||||
# No PVC anymore, so nothing to fence: the new broker can come up and pass
|
||||
# its readiness probe before the old one is torn down. maxUnavailable:0 keeps
|
||||
# a broker serving throughout an image bump / reschedule (clients on the old
|
||||
# pod just reconnect to the new one); maxSurge:1 permits the brief two-pod
|
||||
# overlap — two independent stateless brokers behind the LB, fine since there
|
||||
# is no shared state.
|
||||
# No PVC, so nothing to fence — the new broker can serve before the old one
|
||||
# is torn down. Two stateless brokers overlapping behind the LB is fine.
|
||||
#
|
||||
# NOTE: connectivity survives the roll, but RETAINED STATE DOES NOT — the
|
||||
# new pod starts with an empty retained store, so a roll is equivalent to a
|
||||
# broker restart for every client that depends on retention. Zigbee2MQTT
|
||||
# does not recover from that on its own; see mosquitto-config.yaml and
|
||||
# gitea issue #381.
|
||||
# Connectivity survives a roll but RETAINED STATE DOES NOT: the new pod
|
||||
# starts with an empty retained store, so every roll is a broker restart
|
||||
# for clients that depend on retention. Zigbee2MQTT does not recover from
|
||||
# that on its own (#381).
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxUnavailable: 0
|
||||
@@ -44,14 +35,10 @@ spec:
|
||||
labels:
|
||||
app.kubernetes.io/name: mosquitto
|
||||
spec:
|
||||
# Runs on any node (no nodeSelector); the mosquitto-iot LB forwards
|
||||
# IoT-VLAN traffic to the broker wherever it lands.
|
||||
# Availability: home-critical priority (never preempted, reschedules
|
||||
# first) + short fault tolerations so a node failure reschedules the
|
||||
# broker in ~30s instead of the default ~5min. With persistence off there
|
||||
# is no PVC to fence/detach/reattach, so the broker can (re)start anywhere
|
||||
# instantly, no longer depends on Ceph being healthy, and rolls without a
|
||||
# tear-down-first gap (see the RollingUpdate strategy above).
|
||||
# Runs anywhere; the mosquitto-iot LB finds it. Short fault tolerations
|
||||
# reschedule the broker in ~30s rather than the default ~5min, which is
|
||||
# only possible because there is no PVC to fence — so it also does not
|
||||
# depend on Ceph being healthy.
|
||||
priorityClassName: home-critical
|
||||
terminationGracePeriodSeconds: 10
|
||||
tolerations:
|
||||
|
||||
@@ -6,12 +6,9 @@ metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: zigbee2mqtt
|
||||
spec:
|
||||
# CUTOVER DONE (2026-07-23): ZHA disabled, the coordinator-backup Secret is
|
||||
# populated in OpenBao, and Z2M is now the live Zigbee stack — so replicas is
|
||||
# 1. It shipped at 0 as a safety interlock (Z2M contends with ZHA for the
|
||||
# coordinator's single TCP session, and starting without the restored backup
|
||||
# would form a NEW network and drop every device); that interlock is spent
|
||||
# once the migration is committed. To roll back, set 0 and re-enable ZHA.
|
||||
# Z2M is the live Zigbee stack. Never run it alongside ZHA: they contend for
|
||||
# the coordinator's single TCP session, and a Z2M start without the restored
|
||||
# coordinator backup forms a NEW network and drops every paired device.
|
||||
replicas: 1
|
||||
# RWO PVC → single writer; never run two pods against the same /app/data.
|
||||
strategy:
|
||||
@@ -39,14 +36,10 @@ spec:
|
||||
# network_key, the latter extracted from the OpenBao coordinator backup
|
||||
# as a natural-byte-order int array.
|
||||
- name: seed-data
|
||||
# alpine + jq, built by .woodpecker/build-ci-images.yml from
|
||||
# ci/images/alpine-jq/. Replaces `apk add --no-cache jq` on every pod
|
||||
# start: tools belong in the image, not installed at runtime.
|
||||
# Referenced by tag rather than digest — these are our own images.
|
||||
# DATED tag, not :latest, matching the ci-ansible:2026-04-29 precedent
|
||||
# in kubernetes/infrastructure/netbird-backup/cronjob.yaml. With
|
||||
# :latest, rebuilding this base for some unrelated consumer would
|
||||
# silently change what this init container runs on its next pod
|
||||
# Our own alpine + jq image, built by build-ci-images.yml. DATED tag,
|
||||
# never :latest — rebuilding this base for an unrelated consumer
|
||||
# would otherwise silently change what this init container runs on
|
||||
# its next pod
|
||||
# restart, with no diff here to review — and this is the container that
|
||||
# renders the Zigbee network key. Bumping the tag is a deliberate,
|
||||
# reviewable act; the cost is remembering to bump it.
|
||||
|
||||
@@ -5,9 +5,8 @@ metadata:
|
||||
namespace: monitoring
|
||||
labels:
|
||||
app.kubernetes.io/name: smartctl-exporter
|
||||
# kube-prometheus-stack's Prometheus resource selects ServiceMonitors
|
||||
# by this label; without it the resource exists but is never
|
||||
# discovered.
|
||||
# Not required for discovery — serviceMonitorSelector is empty — but kept
|
||||
# for consistency with the other monitors.
|
||||
release: kube-prometheus-stack
|
||||
spec:
|
||||
selector:
|
||||
|
||||
@@ -5,25 +5,15 @@ metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: otbr
|
||||
spec:
|
||||
# Disabled 2026-07-11 (replicas: 0) after otbr broke Matter.
|
||||
# DISABLED (replicas: 0). Running otbr broke Matter: its radio is RF-isolated
|
||||
# enough from the Nest mesh that it forms a rival partition of the same
|
||||
# network, and the two bridge only over Ethernet — fast enough for a ping,
|
||||
# too slow for the fragmented CASE handshake, so devices stay discoverable
|
||||
# while every commissioning attempt times out.
|
||||
#
|
||||
# otbr is seeded with the Nest network's dataset so it should join the
|
||||
# existing partition as an additive co-BR. In practice it does not merge
|
||||
# cleanly: its SLZB-06 radio is RF-isolated enough from the Nest mesh that
|
||||
# otbr elects itself leader of a rival partition (or the network re-addresses
|
||||
# under a new OMR prefix, observed fd5f -> fdd5). The two same-network
|
||||
# partitions then bridge only over the Ethernet backbone, which carries a
|
||||
# ping but is too slow for the large fragmented CASE handshake -- every
|
||||
# Matter device stays mDNS-discoverable while CASE times out. Removing otbr
|
||||
# re-merges the mesh onto the Nest hubs and Matter recovers.
|
||||
#
|
||||
# This is NOT a diagnosis of a specific config bug: the DHCP/sbr routing
|
||||
# changes (19a4482, df61407) were investigated and matter-server's IPv6 path
|
||||
# to the mesh works (reaches devices, ping ok); sbr's v6 table 100 is an
|
||||
# inert orphan and otbr's eth0 default is outranked by net1's pref-high RA
|
||||
# default. The instability is at the Thread partition layer. Re-enable only
|
||||
# after the co-BR integration is verified to MERGE (one partition, one OMR,
|
||||
# otbr's router table shows the Nest routers) instead of split.
|
||||
# The instability is at the Thread partition layer, not in the config below.
|
||||
# Re-enable only once co-BR integration is verified to MERGE: one partition,
|
||||
# one OMR prefix, and the Nest routers visible in otbr's router table.
|
||||
replicas: 0
|
||||
strategy:
|
||||
type: Recreate
|
||||
@@ -46,25 +36,17 @@ spec:
|
||||
# (slzb-thread.renner.uno). Dropping the workload-class=pinned
|
||||
# toleration lets the edge taint keep it off the tiny edge box.
|
||||
containers:
|
||||
# The Thread RCP lives on a SLZB-06 over Ethernet at
|
||||
# slzb-thread.renner.uno:6638 (Serial-over-IP). otbr-agent's
|
||||
# spinel transport supports `spinel+hdlc+forkpty://<exec>?<args>`
|
||||
# which forks an exec whose stdio becomes the spinel-HDLC
|
||||
# transport, but the upstream openthread/otbr image does NOT
|
||||
# ship socat (or any TCP-stream tool), so that URL form ends in
|
||||
# silent ENOENT and otbr-agent dies right after logging the
|
||||
# radio URL.
|
||||
# The Thread RCP is a network-attached radio (Serial-over-IP), so the
|
||||
# spinel transport needs a TCP-to-PTY bridge. Two obvious approaches
|
||||
# do not work and should not be retried:
|
||||
#
|
||||
# A sidecar socat won't work either: each container has its own
|
||||
# devpts namespace, so a PTY (e.g. /dev/pts/0) created in a
|
||||
# sibling container is invisible from this container — a shared
|
||||
# emptyDir symlink to /dev/pts/0 is dangling.
|
||||
# - `spinel+hdlc+forkpty://` needs socat, which the upstream image
|
||||
# does not ship; the URL fails with a silent ENOENT and otbr-agent
|
||||
# dies right after logging it.
|
||||
# - A sidecar cannot provide it either: devpts is per-container, so
|
||||
# a PTY created next door is invisible here and the symlink dangles.
|
||||
#
|
||||
# The bridge has to live in the otbr container's own namespace.
|
||||
# Install socat at startup via apt (Ubuntu 18.04 base, no egress
|
||||
# restrictions on this namespace), run it in the background as
|
||||
# the spinel-HDLC TCP↔PTY bridge, then exec the upstream
|
||||
# entrypoint with RADIO_URL pointing at the PTY symlink.
|
||||
# Hence socat is installed at startup and run inside this container.
|
||||
- name: otbr
|
||||
image: openthread/otbr:latest
|
||||
command:
|
||||
|
||||
@@ -1,20 +1,15 @@
|
||||
# Headless renderer: screenshots the Home Assistant e-ink dashboard,
|
||||
# dithers it to 16 greys and hands it to Terminus.
|
||||
#
|
||||
# Upstream ships this as a Home Assistant add-on. Our HA is a plain
|
||||
# container with no Supervisor, so it runs standalone here instead —
|
||||
# viable because the app prefers HOME_ASSISTANT_URL/ACCESS_TOKEN env
|
||||
# vars over /data/options.json and authenticates with an ordinary
|
||||
# long-lived token. SUPERVISOR_TOKEN is deliberately NOT set: it only
|
||||
# flips an internal isAddOn flag, and setting it would switch the app
|
||||
# to add-on behaviour (fixed port, /data assumptions).
|
||||
# Upstream ships this as a Home Assistant add-on; it runs standalone here
|
||||
# because the app prefers the env vars over /data/options.json. Do NOT set
|
||||
# SUPERVISOR_TOKEN — it flips an internal isAddOn flag and switches the app to
|
||||
# add-on behaviour (fixed port, /data assumptions).
|
||||
#
|
||||
# DIRECTION OF TRAVEL, which is the non-obvious part: with the
|
||||
# byos-hanami webhook in `uri` delivery mode the renderer does NOT
|
||||
# push image bytes. It POSTs a URL pointing at its own /output/<file>
|
||||
# endpoint, and TERMINUS FETCHES THE IMAGE BACK from it. That is why
|
||||
# both live in this namespace — the pull is intra-namespace and is
|
||||
# already permitted by the default-deny-ingress rule.
|
||||
# DIRECTION OF TRAVEL, the non-obvious part: in `uri` delivery mode the
|
||||
# renderer does not push image bytes. It POSTs a URL to its own /output/<file>
|
||||
# and Terminus fetches the image back. Both therefore live in this namespace,
|
||||
# where that pull is already permitted.
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
|
||||
Reference in New Issue
Block a user