fix(bird): take the pod prefixes out of bird.conf too

The first revision removed the pod0 address list but still carried 10.244.0.0/16 and fd10:244::/48 in the filters — the cluster podSubnets from talos/patches/cluster.yaml, copied a second time. Same defect, different addresses.

The Node objects already carry .spec.podCIDRs, so the render script emits defines.conf with this node's own pod CIDRs alongside its pod0 addresses, and the filters match OWN_POD4/OWN_POD6 exactly instead of an aggregate. That is also stricter: the aggregate would have matched the peers' CIDRs as well, leaving the ifname check as the only discriminator.

The BGP channels drop their prefix filters entirely. The only route this node offers is its own CIDR, already selected by own_pod4/6, and the only peers are the other cluster nodes — so there was nothing left for them to decide. source address moves back into the templates, since defines.conf is included above them.

bird.conf now contains no address at all; grep for one finds nothing.

Verified: shellcheck clean; the podCIDR selectors return fd10:244::/64 and 10.244.0.0/24 against the live k8s-1 object; bird -p parses bird.conf with a generated defines.conf and peers.conf inside the 2.17.2 image, exit 0.

Refs #576

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

ci/images/bird/Containerfile

@@ -9,14 +9,14 @@ FROM alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc
 # Config is read from /etc/bird/bird.conf, not the package default
 # /etc/bird.conf, so the mount cannot shadow what the package ships.
 #
-# kubectl renders the init container's per-node config: go-template-file for
-# the templating, a field selector for excluding the node itself, in-cluster
-# config for the auth. From the distro, so there is no version literal to drift
-# and no download to trust. It is only ever used to read core/v1 Nodes.
-RUN apk add --no-cache bird2 kubectl
+# curl and jq are for the init container, which renders the per-node peer list
+# from the Kubernetes API. busybox wget cannot verify against the API server's
+# CA bundle, and the query must not run unverified.
+RUN apk add --no-cache bird2 curl jq
 
 # Fails the build rather than the first pod if a binary or its path is wrong.
 RUN /usr/sbin/bird --version \
- && kubectl version --client > /dev/null
+ && curl --version > /dev/null \
+ && jq --version > /dev/null
 
 ENTRYPOINT ["/usr/sbin/bird", "-f", "-c", "/etc/bird/bird.conf"]

kubernetes/infrastructure/bird/bird.conf

@@ -1,107 +1,91 @@
-# Identical on every node and free of addresses. defines.conf and peers.conf
-# are rendered per node by the init container; the first is included above the
-# templates that reference it, the second below them.
+# Identical on every node, and free of addresses: everything node-specific is
+# rendered by render-peers.sh from the Kubernetes API.
+#
+# defines.conf first, because the templates below reference it; peers.conf last,
+# because it instantiates them.
 include "/etc/bird/defines.conf";
 
 router id from "pod0";
 log stderr all;
 
+# Mirrors of the kernel main table, filtered to this node's own pod CIDR.
+# master4/master6 are implicit and hold what BGP exchanges.
+ipv4 table kmain4;
+ipv6 table kmain6;
+
 protocol device { scan time 10; }
 
-# Everything a pod may need to reach, in one table, resolved by longest prefix:
-# the node's attached networks, the peers' pod CIDRs, and a default for the
-# rest. No rule has to carve out exceptions, because the specific routes win on
-# their own.
-#
-# The table is BIRD's alone. If this daemon stops, it empties, the rule finds
-# nothing and falls through to main — pod traffic returns to VLAN 40,
-# unseparated rather than misrouted. That is why the default route lives here
-# and not in the Talos config: a default that outlived the specific routes
-# would send pod traffic to the UDM.
-
-# The node's attached networks, taken from the interfaces rather than written
-# down. The usual advice is not to export these to the kernel because it
-# maintains them itself — true of main, not of a table the kernel never fills.
-protocol direct attached {
-  interface "mgmt0", "ingress0", "pod0";
-  ipv4 { table master4; };
-  ipv6 { table master6; };
+# Two conditions, both needed. The prefix alone would also match Cilium's
+# routes for the OTHER nodes, which are protocol kernel as well; the interface
+# alone would also match cilium_host's own host address. ifname is set for the
+# gatewayless IPv6 device route too, which is why this works in both families.
+filter own_pod4 {
+  if net != OWN_POD4 then reject;
+  if ifname !~ "cilium_host" then reject;
+  accept;
 }
-
-# Pod egress. GW4/GW6 are the UDM on the ingress segment, published as node
-# annotations beside the address that defines them.
-protocol static egress4 {
-  ipv4 { table master4; };
-  route 0.0.0.0/0 via GW4;
-}
-protocol static egress6 {
-  ipv6 { table master6; };
-  route ::/0 via GW6;
-}
-
-# This node's own pod CIDR, originated rather than read back out of the kernel.
-# Reading it back made BIRD's view depend on the shape of Cilium's route, and
-# the IPv4 one is rejected outright: its next hop sits inside the prefix it
-# describes, which KRT refuses with "strange next-hop".
-#
-# blackhole because it is only ever advertised — it is filtered out below.
-protocol static own4 {
-  ipv4 { table master4; };
-  route OWN_POD4 blackhole;
-}
-protocol static own6 {
-  ipv6 { table master6; };
-  route OWN_POD6 blackhole;
-}
-
-# Everything except this node's own CIDR, which cilium_host already serves in
-# main and which is a blackhole here.
-filter to_pod_table {
-  if source = RTS_STATIC && dest = RTD_BLACKHOLE then reject;
+filter own_pod6 {
+  if net != OWN_POD6 then reject;
+  if ifname !~ "cilium_host" then reject;
   accept;
 }
 
-protocol kernel k4 {
-  kernel table 47;
-  metric 0;
-  ipv4 { table master4; import none; export filter to_pod_table; };
+# learn all, not learn: plain learn skips routes marked proto kernel, which is
+# what Cilium's are.
+protocol kernel klearn4 {
+  learn all;
+  scan time 10;
+  ipv4 { table kmain4; import filter own_pod4; export none; };
 }
-protocol kernel k6 {
-  kernel table 47;
+protocol kernel klearn6 {
+  learn all;
+  scan time 10;
+  ipv6 { table kmain6; import filter own_pod6; export none; };
+}
+
+protocol pipe own4 { table kmain4; peer table master4; export all; import none; }
+protocol pipe own6 { table kmain6; peer table master6; export all; import none; }
+
+# Only BGP-learned routes reach table 48. The local CIDR stays in main, where
+# cilium_host already serves it.
+filter to_t48 {
+  if source != RTS_BGP then reject;
+  accept;
+}
+
+protocol kernel k48v4 {
+  kernel table 48;
   metric 0;
-  ipv6 { table master6; import none; export filter to_pod_table; };
+  ipv4 { table master4; import none; export filter to_t48; };
+}
+protocol kernel k48v6 {
+  kernel table 48;
+  metric 0;
+  ipv6 { table master6; import none; export filter to_t48; };
 }
 
 # iBGP over pod0. 64514 keeps clear of Cilium's 64513, which peers with the UDM
 # from ingress0 and is unrelated.
 #
+# No prefix filters: the only route this node offers is its own pod CIDR, which
+# own_pod4/6 already selected, and the only peers are the other cluster nodes.
+#
 # direct is explicit because iBGP defaults to multihop, which makes the gateway
 # recursive — BIRD would try to resolve the peer in the pod RIB. strict bind
 # keeps the listener off mgmt0 and ingress0.
-#
-# Only the node's own CIDR is offered: the export filter drops everything the
-# static protocols and the direct protocol produce.
 template bgp pod4 {
   local as 64514;
   source address MY4;
   direct;
   strict bind;
-  ipv4 {
-    next hop self;
-    import all;
-    export filter { if net = OWN_POD4 then accept; reject; };
-  };
+  ipv4 { next hop self; import all; export all; };
 }
 template bgp pod6 {
   local as 64514;
   source address MY6;
   direct;
   strict bind;
-  ipv6 {
-    next hop self;
-    import all;
-    export filter { if net = OWN_POD6 then accept; reject; };
-  };
+  ipv6 { next hop self; import all; export all; };
 }
 
 include "/etc/bird/peers.conf";

kubernetes/infrastructure/bird/daemonset.yaml

@@ -9,7 +9,7 @@ kind: ClusterRole
 metadata:
   name: bird
 rules:
-  # The peer list is read from the nodes' pod0 annotations, so no address is
+  # The peer list comes from the nodes' pod0 annotations, so no address is
   # written into this repo twice.
   - apiGroups: [""]
     resources: ["nodes"]
@@ -59,26 +59,12 @@ spec:
         - operator: Exists
           effect: NoSchedule
       initContainers:
-        - name: render-config
-          # By digest, not by the dated tag: the tag is rebuilt within the day
-          # it names, and the cluster's image path keeps resolving it to the
-          # first build of that day — imagePullPolicy: Always does not help,
-          # because the tag itself resolves stale. Update this when the image
-          # is rebuilt.
-          image: gitea.renner.uno/gitea_admin/bird@sha256:3bc8500613eeb45b2d8858b5445198bae3990ecf64f6d891aa94797f5a09dc1d
-          command: ["/bin/sh", "-c"]
-          args:
-            - |
-              set -eu
-              cp /config/bird.conf /etc/bird/bird.conf
-              kubectl get node "$NODE_NAME" \
-                -o go-template-file=/config/defines.tmpl > /etc/bird/defines.conf
-              kubectl get nodes --field-selector "metadata.name!=$NODE_NAME" \
-                -o go-template-file=/config/peers.tmpl > /etc/bird/peers.conf
-              # BIRD's own parser is the check: a node whose annotation is
-              # missing renders as <no value> and fails here rather than
-              # crashlooping the daemon or forming a mesh with a hole.
-              bird -p -c /etc/bird/bird.conf
+        - name: render-peers
+          image: gitea.renner.uno/gitea_admin/bird:2026-09-16
+          # The dated tag was rebuilt today to add curl and jq, so a node may
+          # hold the earlier build of the same tag.
+          imagePullPolicy: Always
+          command: ["/bin/sh", "/config/render-peers.sh"]
           env:
             - name: NODE_NAME
               valueFrom:
@@ -91,7 +77,8 @@ spec:
               mountPath: /etc/bird
       containers:
         - name: bird
-          image: gitea.renner.uno/gitea_admin/bird@sha256:3bc8500613eeb45b2d8858b5445198bae3990ecf64f6d891aa94797f5a09dc1d
+          image: gitea.renner.uno/gitea_admin/bird:2026-09-16
+          imagePullPolicy: Always
           resources:
             requests:
               cpu: "50m"

kubernetes/infrastructure/bird/defines.tmpl

@@ -1,9 +0,0 @@
-define MY6 = {{ index .metadata.annotations "net.renner.uno/pod0-v6" }};
-define MY4 = {{ index .metadata.annotations "net.renner.uno/pod0-v4" }};
-define GW6 = {{ index .metadata.annotations "net.renner.uno/egress-gw-v6" }};
-define GW4 = {{ index .metadata.annotations "net.renner.uno/egress-gw-v4" }};
-{{/* podCIDRs follows the cluster's podSubnets order, v4 first. A flip puts an
-     IPv6 prefix into an ipv4 protocol, which fails the parse in the init
-     container rather than passing silently. */}}
-define OWN_POD4 = {{ index .spec.podCIDRs 0 }};
-define OWN_POD6 = {{ index .spec.podCIDRs 1 }};

kubernetes/infrastructure/bird/kustomization.yaml

@@ -3,10 +3,9 @@ kind: Kustomization
 resources:
   - daemonset.yaml
 configMapGenerator:
-  # Hashed name, so editing any of these rolls the DaemonSet.
+  # Hashed name, so editing either file rolls the DaemonSet.
   - name: bird-config
     namespace: kube-system
     files:
       - bird.conf
-      - defines.tmpl
-      - peers.tmpl
+      - render-peers.sh

kubernetes/infrastructure/bird/peers.tmpl

@@ -1,7 +0,0 @@
-{{- range $i, $n := .items -}}
-{{- $v6 := index $n.metadata.annotations "net.renner.uno/pod0-v6" -}}
-{{- $v4 := index $n.metadata.annotations "net.renner.uno/pod0-v4" -}}
-# {{ $n.metadata.name }}
-protocol bgp peer{{ $i }}_v6 from pod6 { neighbor {{ $v6 }} as 64514; }
-protocol bgp peer{{ $i }}_v4 from pod4 { neighbor {{ $v4 }} as 64514; }
-{{ end -}}

kubernetes/infrastructure/bird/render-peers.sh

@@ -0,0 +1,50 @@
+#!/bin/sh
+# Renders everything node-specific from the Kubernetes API, so no address is
+# written into this repo twice. pod0 addresses come from annotations set beside
+# the addresses themselves in kubernetes/talos/patches/k8s-*.yaml; pod CIDRs
+# come from the Node objects, where Kubernetes assigns them.
+set -eu
+
+SA=/var/run/secrets/kubernetes.io/serviceaccount
+A6=net.renner.uno/pod0-v6
+A4=net.renner.uno/pod0-v4
+
+nodes=$(curl -sS --fail \
+  --cacert "$SA/ca.crt" \
+  -H "Authorization: Bearer $(cat "$SA/token")" \
+  https://kubernetes.default.svc/api/v1/nodes)
+
+# -e throughout, so a node missing an annotation or a pod CIDR fails here,
+# loudly, rather than forming a mesh with a hole in it.
+self=$(printf '%s' "$nodes" | jq -er --arg n "$NODE_NAME" \
+  '.items[] | select(.metadata.name == $n)')
+
+my6=$(printf '%s' "$self" | jq -er --arg a "$A6" '.metadata.annotations[$a]')
+my4=$(printf '%s' "$self" | jq -er --arg a "$A4" '.metadata.annotations[$a]')
+
+# podCIDRs is ordered by the cluster's family list, so pick by content rather
+# than by index.
+pod6=$(printf '%s' "$self" | jq -er '.spec.podCIDRs[] | select(contains(":"))')
+pod4=$(printf '%s' "$self" | jq -er '.spec.podCIDRs[] | select(contains(":") | not)')
+
+cp /config/bird.conf /etc/bird/bird.conf
+
+cat > /etc/bird/defines.conf <<EOF
+define MY6 = $my6;
+define MY4 = $my4;
+define OWN_POD6 = $pod6;
+define OWN_POD4 = $pod4;
+EOF
+
+# Protocol names are BIRD symbols, so the node name is sanitised.
+printf '%s' "$nodes" | jq -r \
+  --arg n "$NODE_NAME" --arg a6 "$A6" --arg a4 "$A4" '
+  .items[]
+  | select(.metadata.name != $n)
+  | select(.metadata.annotations[$a6] != null and .metadata.annotations[$a4] != null)
+  | (.metadata.name | gsub("[^a-zA-Z0-9]"; "_")) as $p
+  | "protocol bgp p_\($p)_v6 from pod6 { neighbor \(.metadata.annotations[$a6]) as 64514; }",
+    "protocol bgp p_\($p)_v4 from pod4 { neighbor \(.metadata.annotations[$a4]) as 64514; }"
+' > /etc/bird/peers.conf
+
+echo "$NODE_NAME: $(grep -c . /etc/bird/peers.conf) peer protocols, own pod CIDRs $pod6 $pod4"

kubernetes/talos/patches/k8s-1.yaml

@@ -53,8 +53,8 @@ machine:
     # stated once, here, by the machine that knows.
     storage.renner.uno/ceph-osd: "true"
   # Publishes this node's pod0 addresses, declared below in its VLANConfig, so
-  # the bird DaemonSet reads its peers from the node list rather than from a
-  # copy. Annotations, not labels: a label value cannot contain a colon.
+  # the bird DaemonSet builds its peer list from the node list rather than from
+  # a copy. Annotations, not labels: a label value cannot contain a colon.
   nodeAnnotations:
     net.renner.uno/pod0-v6: fd02:48::11
     net.renner.uno/pod0-v4: 10.2.48.11
@@ -188,13 +188,14 @@ addresses:
 # network's clients selectively, so the client goes rather than the record.
 # DHCP is off on VLAN 47 for the same reason.
 #
-# No route here. Table 47 belongs to bird, which fills it with this node's
-# attached networks, the peers' pod CIDRs and the egress default, and empties it
-# if it stops — leaving the rule to fall through to main. A default declared
-# here would outlive the specific routes and send pod traffic to the UDM.
+# Both defaults live in table 47, not in main, so only the pod-source rules in
+# site-homelab.yaml reach them. mgmt0 keeps main's default and with it every
+# reply the node sends from its own addresses.
 #
-# accept_ra_defrtr stays 0 on this link so the RA default does not land in main,
-# where mgmt0 owns it.
+# The v6 gateway is the UDM's configured ULA on this segment, the same address
+# BGP peers with. The RA-learned link-local would be the alternative and is not
+# a value to write down. accept_ra_defrtr stays 0 for that reason: the RA
+# default would land in main, where it does not belong.
 apiVersion: v1alpha1
 kind: VLANConfig
 name: ingress0
@@ -203,6 +204,11 @@ parent: uplink
 up: true
 addresses:
   - address: 10.2.47.11/24
+routes:
+  - gateway: fd02:47::1
+    table: "47"
+  - gateway: 10.2.47.1
+    table: "47"
 ---
 # Dedicated Ceph cluster network on X710 port 1, VLAN 49, untagged on the
 # wire. OSD-to-OSD replication and nothing else. The udev rule in the machine

kubernetes/talos/patches/k8s-2.yaml

@@ -132,6 +132,11 @@ parent: uplink
 up: true
 addresses:
   - address: 10.2.47.12/24
+routes:
+  - gateway: fd02:47::1
+    table: "47"
+  - gateway: 10.2.47.1
+    table: "47"
 ---
 # Dedicated Ceph cluster network on X710 port 1, VLAN 49, untagged on the
 # wire. OSD-to-OSD replication only, not routed off its own segment.

kubernetes/talos/patches/k8s-3.yaml

@@ -119,6 +119,11 @@ parent: uplink
 up: true
 addresses:
   - address: 10.2.47.13/24
+routes:
+  - gateway: fd02:47::1
+    table: "47"
+  - gateway: 10.2.47.1
+    table: "47"
 ---
 # Dedicated Ceph cluster network — see k8s-1.yaml. The node holds NO address
 # here on purpose; the link is handed whole to the OSD pod (host-device).

kubernetes/talos/patches/site-homelab.yaml

@@ -20,32 +20,41 @@ machine:
       validSubnets:
         - 10.2.40.0/24
         - fd40::/64
-  # The UDM on the ingress segment. Published here rather than as a route
-  # because bird owns table 47 and originates the default from it; this is the
-  # one place the address is stated.
-  nodeAnnotations:
-    net.renner.uno/egress-gw-v6: fd02:47::1
-    net.renner.uno/egress-gw-v4: 10.2.47.1
 ---
-# All pod traffic goes to table 47, and the table decides — longest prefix
-# match, no exceptions to carve out. bird fills it with the node's attached
-# networks, the peers' pod CIDRs and the egress default, so a packet to a node
-# on the mgmt segment takes the connected route, a packet to a pod on another
-# node takes pod0, and anything else takes the default over ingress0.
+# Pod egress leaves over ingress0. Matching on the pod source prefix is what
+# keeps the node's own traffic out of it: the node IP, the API VIP and apid all
+# answer on mgmt0, and a reply that left over ingress0 would carry an mgmt0
+# source onto VLAN 47.
 #
-# Matching the source is what keeps the node's own traffic out: it answers on
-# mgmt0, and a reply leaving over ingress0 would carry an mgmt0 source onto
-# VLAN 47. The routing decision runs before Cilium's masquerade hook, so the
-# rule sees the pod address and the masquerade then rewrites it.
+# The routing decision runs before Cilium's masquerade hook, so the rule sees
+# the pod address and the masquerade then rewrites it to ingress0's.
 #
-# If bird stops, table 47 empties, the lookup misses and the rule falls through
-# to main — pod traffic returns to VLAN 40, unseparated rather than misrouted.
+# Table 47 holds nothing but the two default routes, so the destination rules
+# below have to run first: without them a pod's packet to a service or to a pod
+# on another node also matches the source rule, finds only the default, and
+# leaves for the UDM. Measured — in-cluster DNS stopped resolving.
+#
+# They send everything inside the cluster and the local networks back to main.
+# The prefixes are Cilium's ipv4/ipv6NativeRoutingCIDR, which already mark what
+# is not foreign traffic.
 #
 # Priorities are below main (32766) and clear of the rule Cilium installs at 9.
 # The two families are separate rule tables, so the priorities need not match;
 # the document name is the priority and must be unique.
 apiVersion: v1alpha1
 kind: RoutingRuleConfig
+name: "990"
+dst: fd00::/8
+table: main
+---
+apiVersion: v1alpha1
+kind: RoutingRuleConfig
+name: "991"
+dst: 10.0.0.0/8
+table: main
+---
+apiVersion: v1alpha1
+kind: RoutingRuleConfig
 name: "1000"
 src: fd10:244::/48
 table: "47"