PR #135 moved the three v30iot macvlan attachments from static IPAM to
DHCP. That was right about ownership and wrong about routing: a leased
address arrives with a default gateway (DHCP option 3, 10.2.30.1), and
the dhcp IPAM plugin installs it as a *second* default route on net1 at
the same metric as eth0's Cilium-managed one.
Cluster-internal replies then egress the IoT VLAN and are dropped. The
kubelet readiness probe comes from the node address and keeps passing for
a few seconds, which makes the pod look healthy; anything from the pod
CIDR -- Traefik -- times out. shelly-manager went NotReady and its
ingress returned 504.
The old static IPAM sidestepped this by simply never setting a gateway,
as its comment said: "No gateway is set on this attachment: default route
stays on the pod's primary (Cilium-managed) interface, so non-IoT egress
doesn't leak through eth1." A leased address comes with a gateway, so it
has to be confined rather than omitted.
Chain the `sbr` plugin (source-based routing, containernetworking/plugins
v1.5.1, already on every node from cni-plugins-installer) after macvlan.
It moves net1's routes into a routing table selected by net1's source
address, leaving eth0's default alone in the main table.
matter-server's chain becomes macvlan -> tuning -> sbr; otbr and
shelly-manager convert from a single-plugin config to a plugins array.
Only shelly-manager currently runs a post-#135 sandbox, so it is the only
pod actually broken right now. matter-server and otbr still hold their
pre-#135 static addresses and single eth0 default route -- but their NADs
already say dhcp, so they would break the same way on any restart. This
change is a strict improvement for them either way.
UNVERIFIED, called out in the matter-server NAD: sbr's effect on the IPv6
OMR route. Matter CASE sessions to Thread devices need the fd5f::/64 RIO
that OTBR publishes on v30iot. Source-based routing should keep it
reachable for traffic sourced from net1's address, but this has not been
demonstrated. Check `ip -6 route` inside matter-server after its first
restart under this config.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
k.rennerandClaude Opus 4.8