Compare commits

..
Author SHA1 Message Date
claude-botandClaude Opus 5 5b729a3068 fix(bird): take the pod prefixes out of bird.conf too
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The first revision removed the pod0 address list but still carried 10.244.0.0/16 and fd10:244::/48 in the filters — the cluster podSubnets from talos/patches/cluster.yaml, copied a second time. Same defect, different addresses.

The Node objects already carry .spec.podCIDRs, so the render script emits defines.conf with this node's own pod CIDRs alongside its pod0 addresses, and the filters match OWN_POD4/OWN_POD6 exactly instead of an aggregate. That is also stricter: the aggregate would have matched the peers' CIDRs as well, leaving the ifname check as the only discriminator.

The BGP channels drop their prefix filters entirely. The only route this node offers is its own CIDR, already selected by own_pod4/6, and the only peers are the other cluster nodes — so there was nothing left for them to decide. source address moves back into the templates, since defines.conf is included above them.

bird.conf now contains no address at all; grep for one finds nothing.

Verified: shellcheck clean; the podCIDR selectors return fd10:244::/64 and 10.244.0.0/24 against the live k8s-1 object; bird -p parses bird.conf with a generated defines.conf and peers.conf inside the 2.17.2 image, exit 0.

Refs #576

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:19:18 +02:00
claude-botandClaude Opus 5 0b12305600 feat(bird): iBGP mesh over pod0, routes into table 48 (#576)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was canceled
Replaces the withdrawn #848, which shipped a ConfigMap listing every node's pod0 addresses — a second copy of what the Talos patches already assign. Each node now publishes its own addresses as annotations, beside the VLANConfig that assigns them, and the init container builds the peer list from the node list. No address is written down twice, and a fourth node needs no entry anywhere else.

Nothing reads table 48 yet. The mesh fills it so the routing rules land on a table that is already correct, and so a mistake here shows as an empty table rather than as broken pod traffic.

The own-CIDR filter keys on ifname, not the prefix: Cilium's routes for the other nodes are protocol kernel as well, and ifname is set for the gatewayless IPv6 device route too. learn all rather than learn, since plain learn skips proto kernel. direct is explicit because iBGP defaults to multihop, which would make the gateway recursive. strict bind keeps the listener off mgmt0 and ingress0.

Verified: kubectl kustomize builds; shellcheck clean at default and error severity; the nodeAnnotations applied to k8s-1 with --mode=try reached the Node object without a reboot; the jq rendering run against the live node list produced the expected protocol lines, excluding self and sanitising the name; and bird -p parsed bird.conf plus a full peers.conf inside the 2.17.2 image with exit 0.

Refs #576

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:15:01 +02:00
8 changed files with 80 additions and 51 deletions
+6 -6
View File
@@ -9,14 +9,14 @@ FROM alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc
# Config is read from /etc/bird/bird.conf, not the package default
# /etc/bird.conf, so the mount cannot shadow what the package ships.
#
# kubectl renders the init container's per-node config: go-template-file for
# the templating, a field selector for excluding the node itself, in-cluster
# config for the auth. From the distro, so there is no version literal to drift
# and no download to trust. It is only ever used to read core/v1 Nodes.
RUN apk add --no-cache bird2 kubectl
# curl and jq are for the init container, which renders the per-node peer list
# from the Kubernetes API. busybox wget cannot verify against the API server's
# CA bundle, and the query must not run unverified.
RUN apk add --no-cache bird2 curl jq
# Fails the build rather than the first pod if a binary or its path is wrong.
RUN /usr/sbin/bird --version \
&& kubectl version --client > /dev/null
&& curl --version > /dev/null \
&& jq --version > /dev/null
ENTRYPOINT ["/usr/sbin/bird", "-f", "-c", "/etc/bird/bird.conf"]
+11 -9
View File
@@ -1,6 +1,8 @@
# Identical on every node and free of addresses. defines.conf and peers.conf
# are rendered per node by the init container; the first is included above the
# templates that reference it, the second below them.
# Identical on every node, and free of addresses: everything node-specific is
# rendered by render-peers.sh from the Kubernetes API.
#
# defines.conf first, because the templates below reference it; peers.conf last,
# because it instantiates them.
include "/etc/bird/defines.conf";
router id from "pod0";
@@ -13,18 +15,18 @@ ipv6 table kmain6;
protocol device { scan time 10; }
# Two conditions. cilium_host also carries its own host address, which the
# length check drops; and Cilium's routes for the OTHER nodes are protocol
# kernel too, which the interface check drops. ifname is set for the
# gatewayless IPv6 device route as well, so this works in both families.
# Two conditions, both needed. The prefix alone would also match Cilium's
# routes for the OTHER nodes, which are protocol kernel as well; the interface
# alone would also match cilium_host's own host address. ifname is set for the
# gatewayless IPv6 device route too, which is why this works in both families.
filter own_pod4 {
if net != OWN_POD4 then reject;
if ifname !~ "cilium_host" then reject;
if net.len = 32 then reject;
accept;
}
filter own_pod6 {
if net != OWN_POD6 then reject;
if ifname !~ "cilium_host" then reject;
if net.len = 128 then reject;
accept;
}
+9 -22
View File
@@ -9,7 +9,7 @@ kind: ClusterRole
metadata:
name: bird
rules:
# The peer list is read from the nodes' pod0 annotations, so no address is
# The peer list comes from the nodes' pod0 annotations, so no address is
# written into this repo twice.
- apiGroups: [""]
resources: ["nodes"]
@@ -59,26 +59,12 @@ spec:
- operator: Exists
effect: NoSchedule
initContainers:
- name: render-config
# By digest, not by the dated tag: the tag is rebuilt within the day
# it names, and the cluster's image path keeps resolving it to the
# first build of that day — imagePullPolicy: Always does not help,
# because the tag itself resolves stale. Update this when the image
# is rebuilt.
image: gitea.renner.uno/gitea_admin/bird@sha256:3bc8500613eeb45b2d8858b5445198bae3990ecf64f6d891aa94797f5a09dc1d
command: ["/bin/sh", "-c"]
args:
- |
set -eu
cp /config/bird.conf /etc/bird/bird.conf
kubectl get node "$NODE_NAME" \
-o go-template-file=/config/defines.tmpl > /etc/bird/defines.conf
kubectl get nodes --field-selector "metadata.name!=$NODE_NAME" \
-o go-template-file=/config/peers.tmpl > /etc/bird/peers.conf
# BIRD's own parser is the check: a node whose annotation is
# missing renders as <no value> and fails here rather than
# crashlooping the daemon or forming a mesh with a hole.
bird -p -c /etc/bird/bird.conf
- name: render-peers
image: gitea.renner.uno/gitea_admin/bird:2026-09-16
# The dated tag was rebuilt today to add curl and jq, so a node may
# hold the earlier build of the same tag.
imagePullPolicy: Always
command: ["/bin/sh", "/config/render-peers.sh"]
env:
- name: NODE_NAME
valueFrom:
@@ -91,7 +77,8 @@ spec:
mountPath: /etc/bird
containers:
- name: bird
image: gitea.renner.uno/gitea_admin/bird@sha256:3bc8500613eeb45b2d8858b5445198bae3990ecf64f6d891aa94797f5a09dc1d
image: gitea.renner.uno/gitea_admin/bird:2026-09-16
imagePullPolicy: Always
resources:
requests:
cpu: "50m"
@@ -1,2 +0,0 @@
define MY6 = {{ index .metadata.annotations "net.renner.uno/pod0-v6" }};
define MY4 = {{ index .metadata.annotations "net.renner.uno/pod0-v4" }};
@@ -3,10 +3,9 @@ kind: Kustomization
resources:
- daemonset.yaml
configMapGenerator:
# Hashed name, so editing any of these rolls the DaemonSet.
# Hashed name, so editing either file rolls the DaemonSet.
- name: bird-config
namespace: kube-system
files:
- bird.conf
- defines.tmpl
- peers.tmpl
- render-peers.sh
@@ -1,7 +0,0 @@
{{- range $i, $n := .items -}}
{{- $v6 := index $n.metadata.annotations "net.renner.uno/pod0-v6" -}}
{{- $v4 := index $n.metadata.annotations "net.renner.uno/pod0-v4" -}}
# {{ $n.metadata.name }}
protocol bgp peer{{ $i }}_v6 from pod6 { neighbor {{ $v6 }} as 64514; }
protocol bgp peer{{ $i }}_v4 from pod4 { neighbor {{ $v4 }} as 64514; }
{{ end -}}
@@ -0,0 +1,50 @@
#!/bin/sh
# Renders everything node-specific from the Kubernetes API, so no address is
# written into this repo twice. pod0 addresses come from annotations set beside
# the addresses themselves in kubernetes/talos/patches/k8s-*.yaml; pod CIDRs
# come from the Node objects, where Kubernetes assigns them.
set -eu
SA=/var/run/secrets/kubernetes.io/serviceaccount
A6=net.renner.uno/pod0-v6
A4=net.renner.uno/pod0-v4
nodes=$(curl -sS --fail \
--cacert "$SA/ca.crt" \
-H "Authorization: Bearer $(cat "$SA/token")" \
https://kubernetes.default.svc/api/v1/nodes)
# -e throughout, so a node missing an annotation or a pod CIDR fails here,
# loudly, rather than forming a mesh with a hole in it.
self=$(printf '%s' "$nodes" | jq -er --arg n "$NODE_NAME" \
'.items[] | select(.metadata.name == $n)')
my6=$(printf '%s' "$self" | jq -er --arg a "$A6" '.metadata.annotations[$a]')
my4=$(printf '%s' "$self" | jq -er --arg a "$A4" '.metadata.annotations[$a]')
# podCIDRs is ordered by the cluster's family list, so pick by content rather
# than by index.
pod6=$(printf '%s' "$self" | jq -er '.spec.podCIDRs[] | select(contains(":"))')
pod4=$(printf '%s' "$self" | jq -er '.spec.podCIDRs[] | select(contains(":") | not)')
cp /config/bird.conf /etc/bird/bird.conf
cat > /etc/bird/defines.conf <<EOF
define MY6 = $my6;
define MY4 = $my4;
define OWN_POD6 = $pod6;
define OWN_POD4 = $pod4;
EOF
# Protocol names are BIRD symbols, so the node name is sanitised.
printf '%s' "$nodes" | jq -r \
--arg n "$NODE_NAME" --arg a6 "$A6" --arg a4 "$A4" '
.items[]
| select(.metadata.name != $n)
| select(.metadata.annotations[$a6] != null and .metadata.annotations[$a4] != null)
| (.metadata.name | gsub("[^a-zA-Z0-9]"; "_")) as $p
| "protocol bgp p_\($p)_v6 from pod6 { neighbor \(.metadata.annotations[$a6]) as 64514; }",
"protocol bgp p_\($p)_v4 from pod4 { neighbor \(.metadata.annotations[$a4]) as 64514; }"
' > /etc/bird/peers.conf
echo "$NODE_NAME: $(grep -c . /etc/bird/peers.conf) peer protocols, own pod CIDRs $pod6 $pod4"
+2 -2
View File
@@ -53,8 +53,8 @@ machine:
# stated once, here, by the machine that knows.
storage.renner.uno/ceph-osd: "true"
# Publishes this node's pod0 addresses, declared below in its VLANConfig, so
# the bird DaemonSet reads its peers from the node list rather than from a
# copy. Annotations, not labels: a label value cannot contain a colon.
# the bird DaemonSet builds its peer list from the node list rather than from
# a copy. Annotations, not labels: a label value cannot contain a colon.
nodeAnnotations:
net.renner.uno/pod0-v6: fd02:48::11
net.renner.uno/pod0-v4: 10.2.48.11