feat(bird): iBGP mesh over pod0, routes into table 48 (2/3) (#576) #848

2026-09-16T07:56:48Z

claude-bot commented

2026-09-16 07:56:48 +00:00

Second of three. The mesh runs and fills kernel table 48; nothing reads that table yet. The routing rules come in 3/3, so a mistake here shows as an empty table rather than as broken pod traffic.

What each node does

Advertises only its own pod CIDR, learns the other two, installs what it learns into table 48 with the peer's pod0 address as next-hop. Cilium's own routes in the main table are untouched.

The two non-obvious bits

The own-CIDR filter keys on ifname, not the prefix. Cilium's routes for the other nodes' pod CIDRs are protocol kernel as well, so 10.244.0.0/16 alone does not identify the local one. The outgoing interface does — local is cilium_host, learned is mgmt0 — and ifname is populated for the gatewayless IPv6 device route too. That is exactly what FRR could not express: its route-maps have no match interface, and match ipv6 next-hop has nothing to match on a device route.

learn all, not learn. Documented: "routes created by kernel (marked as 'proto kernel') are not imported. Use learn all option to import even these routes." Cilium's are exactly that.

Two more that would bite silently:

direct; is explicit because iBGP defaults to multihop, which makes gateway recursive — BIRD would try to resolve the peer inside the pod RIB.
strict bind keeps the listener on pod0 only. FRR has no equivalent: its --listenon "implies the --no_kernel option".

Only BGP-learned routes are exported to table 48; the local CIDR stays in main, where cilium_host already serves it.

Per-node config

One ConfigMap key per node, copied into place by an init container from spec.nodeName. neighbor range is passive-only per the docs, so an all-listen mesh never forms, and whether BIRD tolerates a neighbour equal to a local address is undocumented — so the peer list is rendered rather than uniform.

The addresses duplicate the pod0 statics in kubernetes/talos/patches/k8s-{1,2,3}.yaml. Reading them off the interface instead would need iproute2 in the image — Alpine's busybox does not provide ip, it comes from iproute2-minimal — so this keeps the image as it is and puts the mapping somewhere reviewable.

ASN

64514, clear of Cilium's 64513. That session peers with the UDM from ingress0 and is unrelated; verified that nothing listens on TCP/179 on the nodes today, so bird can bind it.

Verified

kubectl kustomize kubernetes/infrastructure/bird builds.
bird -p parses the rendered config inside the 2.17.2 image, exit 0. A deliberately broken copy exits non-zero with syntax error, unexpected CF_SYM_UNDEFINED — so the check distinguishes pass from fail rather than reporting a silent zero.

Not verified here, because it is runtime behaviour: session establishment, route exchange, and whether the table-48 writes actually appear. That is what the rollout checks.

Rollout

Merge, let Flux apply, then per node confirm birdc show protocols has both peers up and talosctl get routes shows the two peer CIDRs in table 48 via pod0. Pod traffic still rides mgmt0 throughout — nothing consults table 48 until 3/3.

Open, and stated rather than assumed

The BIRD docs do not say whether the kernel protocol removes routes it does not know about from its table. Table 48 has no other writer, so it does not arise here, but it is an assumption rather than a guarantee.

Refs #576

🤖 Generated with Claude Code

Second of three. The mesh runs and fills kernel table 48; **nothing reads that table yet**. The routing rules come in 3/3, so a mistake here shows as an empty table rather than as broken pod traffic. ## What each node does Advertises only its own pod CIDR, learns the other two, installs what it learns into table 48 with the peer's `pod0` address as next-hop. Cilium's own routes in the main table are untouched. ## The two non-obvious bits **The own-CIDR filter keys on `ifname`, not the prefix.** Cilium's routes for the *other* nodes' pod CIDRs are `protocol kernel` as well, so `10.244.0.0/16` alone does not identify the local one. The outgoing interface does — local is `cilium_host`, learned is `mgmt0` — and `ifname` is populated for the gatewayless IPv6 device route too. That is exactly what FRR could not express: its route-maps have no `match interface`, and `match ipv6 next-hop` has nothing to match on a device route. **`learn all`, not `learn`.** Documented: *"routes created by kernel (marked as 'proto kernel') are not imported. Use `learn all` option to import even these routes."* Cilium's are exactly that. Two more that would bite silently: - `direct;` is explicit because iBGP defaults to `multihop`, which makes `gateway` recursive — BIRD would try to resolve the peer inside the pod RIB. - `strict bind` keeps the listener on `pod0` only. FRR has no equivalent: its `--listenon` *"implies the --no_kernel option"*. Only BGP-learned routes are exported to table 48; the local CIDR stays in main, where `cilium_host` already serves it. ## Per-node config One ConfigMap key per node, copied into place by an init container from `spec.nodeName`. `neighbor range` is passive-only per the docs, so an all-listen mesh never forms, and whether BIRD tolerates a neighbour equal to a local address is undocumented — so the peer list is rendered rather than uniform. The addresses duplicate the `pod0` statics in `kubernetes/talos/patches/k8s-{1,2,3}.yaml`. Reading them off the interface instead would need `iproute2` in the image — Alpine's busybox does not provide `ip`, it comes from `iproute2-minimal` — so this keeps the image as it is and puts the mapping somewhere reviewable. ## ASN 64514, clear of Cilium's 64513. That session peers with the UDM from `ingress0` and is unrelated; verified that nothing listens on TCP/179 on the nodes today, so `bird` can bind it. ## Verified - `kubectl kustomize kubernetes/infrastructure/bird` builds. - `bird -p` parses the rendered config **inside the 2.17.2 image**, exit 0. A deliberately broken copy exits non-zero with `syntax error, unexpected CF_SYM_UNDEFINED` — so the check distinguishes pass from fail rather than reporting a silent zero. Not verified here, because it is runtime behaviour: session establishment, route exchange, and whether the table-48 writes actually appear. That is what the rollout checks. ## Rollout Merge, let Flux apply, then per node confirm `birdc show protocols` has both peers up and `talosctl get routes` shows the two peer CIDRs in table 48 via `pod0`. Pod traffic still rides `mgmt0` throughout — nothing consults table 48 until 3/3. ## Open, and stated rather than assumed The BIRD docs do not say whether the kernel protocol removes routes it does not know about from its table. Table 48 has no other writer, so it does not arise here, but it is an assumption rather than a guarantee. Refs #576 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-09-16 07:56:48 +00:00

feat(bird): iBGP mesh over pod0, routes into table 48 (2/3) (#576 )

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

ci/woodpecker/pr/ci-manifests-test Pipeline was successful

Details

ci/woodpecker/pr/pr-review Pipeline was successful

Details

4d948a50f4

Nothing reads table 48 yet. The DaemonSet fills it so the routing rules in 3/3 land on a table that is already correct, and so a mistake here shows up as an empty table rather than as broken pod traffic.

Each node advertises only its own pod CIDR and learns the other two. The own-CIDR filter keys on ifname rather than the prefix, because Cilium's routes for the OTHER nodes are protocol kernel as well — local is cilium_host, learned is mgmt0 — and ifname is set for the gatewayless IPv6 device route too, which is what FRR's route-maps could not match. learn all rather than learn, since plain learn skips routes marked proto kernel.

Only BGP-learned routes are exported to table 48; the local CIDR stays in main, where cilium_host already serves it. direct is explicit because iBGP defaults to multihop and would make the gateway recursive. strict bind keeps the listener off mgmt0 and ingress0 — the option FRR lacks, since its --listenon implies --no_kernel.

Per-node config is one ConfigMap key per node, copied by an init container from spec.nodeName. The addresses duplicate the pod0 statics in kubernetes/talos/patches/k8s-{1,2,3}.yaml; reading them off the interface instead would need iproute2 in the image, which Alpine does not put in busybox.

Verified: kubectl kustomize builds, and bird -p parses the rendered config inside the 2.17.2 image with exit 0. A deliberately broken copy exits non-zero with a syntax error, so the check distinguishes the two. Peering, route exchange and the table-48 writes are runtime behaviour and are proven by the rollout, not by this.

Refs #576

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

review-bot reviewed

2026-09-16 08:02:42 +00:00

review-bot left a comment

🤖 Automated review — model CLI default (complexity: complex), commit 4d948a50f4eeac4b93a6e9eb1657eef534602971.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

Summary

Adds a BIRD2 DaemonSet that forms an iBGP full mesh between the three control-plane
nodes over pod0, filters Cilium's kernel-installed pod-CIDR routes into a private
BIRD table via an ifname-based heuristic, and re-exports only BGP-learned routes
into kernel table 48. Nothing reads table 48 yet (that's PR 3/3), so the blast
radius of a mistake here is contained to an empty/wrong table rather than broken
pod traffic — as the description states. Cross-checked against the rest of the
repo: the pod CIDRs (10.244.0.0/16 / fd10:244::/48), the pod0 static
addresses in kubernetes/talos/patches/k8s-{1,2,3}.yaml, the ASN (64514, clear
of Cilium's 64513), and the table-number-matches-VLAN-ID convention (table 47
for ingress0/VLAN 47, so table 48 for pod0/VLAN 48) are all internally
consistent. No blocking issues found.

🟡 Suggestions

kubernetes/infrastructure/bird/daemonset.yaml:167,185 — the image tag
gitea.renner.uno/gitea_admin/bird:2026-09-16 is a fixed build date pushed by
the separate build-bird step in .woodpecker/build-ci-images.yml, which only
triggers on: push, path: ci/images/bird/**. This PR's diff doesn't touch that
path, so merging it does not itself rebuild the image — it only works if the
commit that added ci/images/bird/Containerfile (1/3) already ran that job
successfully on 2026-09-16 and pushed the tag. I couldn't query the registry
from this sandbox to confirm the tag exists; worth a quick check
(crane ls/UI) before merge to avoid an ImagePullBackOff across all three
nodes at once (no working peer to fall back to, since the mesh is symmetric).
Config changes won't roll out automatically. bird-config/bird-nodes
(daemonset.yaml:1-133) are plain ConfigMaps referenced by name, not run
through configMapGenerator (which this repo does use elsewhere, e.g.
kubernetes/apps/home-assistant/kustomization.yaml). Also, bird.conf/
node.conf are copied into the pod by the init container rather than
mounted directly, so a future edit to either ConfigMap updates neither the pod
spec hash nor the live file inside a running container — it needs a manual
kubectl rollout restart daemonset/bird -n kube-system to take effect. Fine
for this initial rollout; worth a one-line comment for future editors so a
ConfigMap-only change doesn't silently no-op.
The single biggest unverified assumption is the ifname heuristic
(daemonset.yaml:21-34): distinguishing the local pod CIDR from peers' by
ifname !~ "cilium_host" rather than by prefix. The PR body already flags
this as unverified runtime behavior ("Not verified here..."), and correctly
notes a wrong assumption here only produces an empty/wrong table 48, not
broken pod traffic, since nothing consumes it until 3/3. Given that's the
crux of the whole design, I'd promote "check ip route show table 48 and
birdc show route table kmain4/kmain6 on all three nodes" to the first
rollout step, ahead of the peer/session checks already listed — a wrong
own_pod4/own_pod6 filter would still show BGP sessions up.

🟢 Nits

daemonset.yaml:170 — the init container's cp ... && cp ... has no
set -e/explicit exit check, but sh -c 'a && b' already short-circuits on
the first failure, so this is fine as written; noting it only because a
reader might otherwise wonder.
The init container runs with no explicit securityContext (default runtime
capabilities), while the main container explicitly drops ALL and adds back
only what it needs. The init container doesn't need any of the defaults
(NET_RAW, SYS_CHROOT, etc.) for a cp, so capabilities: { drop: ["ALL"] }
there would match the main container's posture — cosmetic, low value given
it's a short-lived container copying files from ConfigMaps it already trusts.

Tests

No test suite exists for this (declarative infra, consistent with the rest of
the repo). The PR description documents two manual checks already run
(kubectl kustomize builds; bird -p parses inside the real 2.17.2 image, with
a deliberately-broken copy shown to fail the same check) — that's an adequate
substitute for a syntax-level test here, and I have no way to re-run bird -p
in this sandbox to independently reproduce it. Session establishment and the
table-48 population itself are explicitly deferred to the rollout checks in the
PR body, which is reasonable given the change is inert until 3/3 lands.

Recommendation

Looks reasonable — no blocking issues; comments above are optional (though I'd
confirm the bird:2026-09-16 image tag actually exists in the registry before
merging, since a miss would fail closed on all three nodes simultaneously).

🤖 **Automated review** — model `CLI default` (complexity: complex), commit `4d948a50f4eeac4b93a6e9eb1657eef534602971`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary Adds a BIRD2 DaemonSet that forms an iBGP full mesh between the three control-plane nodes over `pod0`, filters Cilium's kernel-installed pod-CIDR routes into a private BIRD table via an `ifname`-based heuristic, and re-exports only BGP-learned routes into kernel table 48. Nothing reads table 48 yet (that's PR 3/3), so the blast radius of a mistake here is contained to an empty/wrong table rather than broken pod traffic — as the description states. Cross-checked against the rest of the repo: the pod CIDRs (`10.244.0.0/16` / `fd10:244::/48`), the `pod0` static addresses in `kubernetes/talos/patches/k8s-{1,2,3}.yaml`, the ASN (`64514`, clear of Cilium's `64513`), and the table-number-matches-VLAN-ID convention (table `47` for `ingress0`/VLAN 47, so table `48` for `pod0`/VLAN 48) are all internally consistent. No blocking issues found. ## 🟡 Suggestions - **`kubernetes/infrastructure/bird/daemonset.yaml:167,185`** — the image tag `gitea.renner.uno/gitea_admin/bird:2026-09-16` is a fixed build date pushed by the separate `build-bird` step in `.woodpecker/build-ci-images.yml`, which only triggers `on: push, path: ci/images/bird/**`. This PR's diff doesn't touch that path, so merging it does not itself rebuild the image — it only works if the commit that added `ci/images/bird/Containerfile` (1/3) already ran that job successfully *on 2026-09-16* and pushed the tag. I couldn't query the registry from this sandbox to confirm the tag exists; worth a quick check (`crane ls`/UI) before merge to avoid an `ImagePullBackOff` across all three nodes at once (no working peer to fall back to, since the mesh is symmetric). - **Config changes won't roll out automatically.** `bird-config`/`bird-nodes` (`daemonset.yaml:1-133`) are plain `ConfigMap`s referenced by name, not run through `configMapGenerator` (which this repo does use elsewhere, e.g. `kubernetes/apps/home-assistant/kustomization.yaml`). Also, `bird.conf`/ `node.conf` are copied into the pod by the **init container** rather than mounted directly, so a future edit to either ConfigMap updates neither the pod spec hash nor the live file inside a running container — it needs a manual `kubectl rollout restart daemonset/bird -n kube-system` to take effect. Fine for this initial rollout; worth a one-line comment for future editors so a ConfigMap-only change doesn't silently no-op. - **The single biggest unverified assumption is the `ifname` heuristic** (`daemonset.yaml:21-34`): distinguishing the local pod CIDR from peers' by `ifname !~ "cilium_host"` rather than by prefix. The PR body already flags this as unverified runtime behavior ("Not verified here..."), and correctly notes a wrong assumption here only produces an empty/wrong table 48, not broken pod traffic, since nothing consumes it until 3/3. Given that's the crux of the whole design, I'd promote "check `ip route show table 48` and `birdc show route table kmain4`/`kmain6` on all three nodes" to the first rollout step, ahead of the peer/session checks already listed — a wrong `own_pod4`/`own_pod6` filter would still show BGP sessions up. ## 🟢 Nits - `daemonset.yaml:170` — the init container's `cp ... && cp ...` has no `set -e`/explicit exit check, but `sh -c 'a && b'` already short-circuits on the first failure, so this is fine as written; noting it only because a reader might otherwise wonder. - The init container runs with no explicit `securityContext` (default runtime capabilities), while the main container explicitly drops `ALL` and adds back only what it needs. The init container doesn't need any of the defaults (`NET_RAW`, `SYS_CHROOT`, etc.) for a `cp`, so `capabilities: { drop: ["ALL"] }` there would match the main container's posture — cosmetic, low value given it's a short-lived container copying files from ConfigMaps it already trusts. ## Tests No test suite exists for this (declarative infra, consistent with the rest of the repo). The PR description documents two manual checks already run (`kubectl kustomize` builds; `bird -p` parses inside the real 2.17.2 image, with a deliberately-broken copy shown to fail the same check) — that's an adequate substitute for a syntax-level test here, and I have no way to re-run `bird -p` in this sandbox to independently reproduce it. Session establishment and the table-48 population itself are explicitly deferred to the rollout checks in the PR body, which is reasonable given the change is inert until 3/3 lands. ## Recommendation Looks reasonable — no blocking issues; comments above are optional (though I'd confirm the `bird:2026-09-16` image tag actually exists in the registry before merging, since a miss would fail closed on all three nodes simultaneously).

claude-bot closed this pull request

2026-09-16 08:04:18 +00:00

claude-bot referenced this pull request

2026-09-16 08:10:25 +00:00

feat(bird): add curl and jq for the peer-list init container #849

claude-bot referenced this issue from a commit

2026-09-16 08:15:08 +00:00

feat(bird): iBGP mesh over pod0, routes into table 48 (#576)

claude-bot referenced this pull request

2026-09-16 08:15:28 +00:00

feat(bird): iBGP mesh over pod0, routes into table 48 (#576) #850