docs(#576): record the rejected options in the traffic-class proposal

The proposal still recommended option B (node IP stays on VLAN 40, etcd and the API VIP move to a new management VLAN), which the owner has rejected repeatedly, and it described the cluster as running in tunnel mode. Rewritten to the measured state of 2026-09-15: world-pod is on VLAN 47 since #835/#836, the cluster runs native routing since #837, pod-pod still follows the kubelet InternalIP on mgmt0. Options A (#838) and B are marked rejected, C (#839, Antrea) shelved. The owner's 2026-09-08 target and the no-UDM constraint are kept as the reference.

Refs #576

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.woodpecker/build-ci-images.yml

@@ -376,29 +376,3 @@ steps:
           --dockerfile Containerfile \
           --destination gitea.renner.uno/gitea_admin/netbird-proxy-services:$TAG \
           --destination gitea.renner.uno/gitea_admin/netbird-proxy-services:latest
-
-  # bird: BGP daemon for the pod-network mesh over pod0 (#576). Runtime
-  # artifact like heating-can; same registry so the audit tracks it.
-  - name: build-bird
-    image: registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug
-    directory: ci/images/bird
-    depends_on: []
-    when:
-      - event: deployment
-      - event: push
-        path: ["ci/images/bird/**", ".woodpecker/build-ci-images.yml"]
-    environment:
-      GITEA_USER: gitea_admin
-      GITEA_TOKEN:
-        from_secret: ci_images_token
-    commands:
-      - mkdir -p /kaniko/.docker
-      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
-      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
-      - TAG=$(date +%Y-%m-%d)
-      - |
-        /kaniko/executor \
-          --context . \
-          --dockerfile Containerfile \
-          --destination gitea.renner.uno/gitea_admin/bird:$TAG \
-          --destination gitea.renner.uno/gitea_admin/bird:latest

ci/images/bird/Containerfile

@@ -1,22 +0,0 @@
-FROM alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce
-
-# bird: BGP daemon for the pod-network mesh over pod0. Distro package, no build
-# step — the image is the binary plus its runtime.
-#
-# 3.22 rather than the 3.21 the other images here pin: bird2 first appears in
-# 3.22/community.
-#
-# Config is read from /etc/bird/bird.conf, not the package default
-# /etc/bird.conf, so the mount cannot shadow what the package ships.
-#
-# kubectl renders the init container's per-node config: go-template-file for
-# the templating, a field selector for excluding the node itself, in-cluster
-# config for the auth. From the distro, so there is no version literal to drift
-# and no download to trust. It is only ever used to read core/v1 Nodes.
-RUN apk add --no-cache bird2 kubectl
-
-# Fails the build rather than the first pod if a binary or its path is wrong.
-RUN /usr/sbin/bird --version \
- && kubectl version --client > /dev/null
-
-ENTRYPOINT ["/usr/sbin/bird", "-f", "-c", "/etc/bird/bird.conf"]

kubernetes/NETWORK-traffic-classes.md

@@ -1,117 +0,0 @@
-# Traffic classes on the k8s nodes
-
-Issue #576. Three of the four classes are separated; management is not.
-
-## Where each class runs
-
-| class | link | mechanism |
-|---|---|---|
-| Ceph replication | `ceph-san`, VLAN 49 | the whole link is handed to the OSD pod (Multus `host-device`) |
-| world→pod | `ingress0`, VLAN 47 | BGP session sourced there, so the advertised next hop is that address |
-| pod→world | `ingress0`, VLAN 47 | routing rule on the pod source prefix → table 47 |
-| pod↔pod | `pod0`, VLAN 48 | routing rule on pod source **and** destination → table 48 |
-| management | `mgmt0`, VLAN 40 | not started |
-
-Hard constraint, set by the owner: inter-node and pod traffic never traverses
-the UDM. Everything above stays on the 10G switch.
-
-## Why policy routing rather than moving the node IP
-
-Cilium reaches a peer's pods at that peer's Kubernetes InternalIP, in native
-and tunnel mode alike. `--ipv4-node` is overwritten from the Node object, and
-`nodePort.directRoutingDevice` selects the BPF NodePort device, not a next hop.
-Both were established by test (#837, #838), not by reading.
-
-So the InternalIP decides where pod traffic goes — unless the lookup happens in
-a different table. The rules do that, and nothing about the node's identity
-moves. The routing decision runs before Cilium's masquerade hook, so a rule
-matching the pod source sees the pod address, and the masquerade then rewrites
-it to the egress link's.
-
-## Rule order
-
-Lower priority runs first.
-
-| prio | match | table |
-|---|---|---|
-| 980 | src pod, dst pod | 48 |
-| 981 | src pod, dst pod (v4) | 48 |
-| 990 | dst `fd00::/8` | main |
-| 991 | dst `10.0.0.0/8` | main |
-| 1000 | src pod | 47 |
-| 1001 | src pod (v4) | 47 |
-
-980/981 match source **and** destination. Source alone would also catch the
-node's own traffic to a pod, which would then leave with a `pod0` source — an
-address `CiliumNode.spec.addresses` does not list, so Cilium classifies it as
-`world` rather than `host`.
-
-990/991 are Cilium's `ipv4/ipv6NativeRoutingCIDR`, and they are what keeps
-cluster-internal traffic out of table 47. Without them a pod's packet to a
-service matches the source rule, finds only a default route, and leaves for the
-UDM — measured, in-cluster DNS stopped resolving.
-
-Traffic to a pod on the same node falls through on its own: table 48 holds only
-the peers' CIDRs, so the lookup misses and 990/991 send it to main.
-
-## Where each table comes from
-
-- **47** — two default routes declared per node in `talos/patches/k8s-*.yaml`.
-  The IPv6 gateway is the UDM's configured address on the segment, not the
-  RA-learned link-local; `accept_ra_defrtr` stays `0` on `ingress0` so the RA
-  default does not land in main.
-- **48** — filled by the `bird` DaemonSet: an iBGP mesh over `pod0`, one route
-  per peer pod CIDR with the peer's `pod0` address as next hop.
-
-## The bird mesh
-
-`kubernetes/infrastructure/bird/`. Each node advertises only its own pod CIDR
-and installs what it learns into table 48. Cilium's routes in main are
-untouched, which is why the two coexist.
-
-No address is written into this repo twice. Each node publishes its `pod0`
-addresses as `nodeAnnotations`, beside the `VLANConfig` that assigns them; the
-pod CIDRs come from the Node objects. An init container renders both from the
-API with `kubectl -o go-template-file`, excludes the node itself with a field
-selector, and ends with `bird -p` so BIRD's own parser gates startup.
-
-The node's own CIDR is **originated**, not read back from the kernel. Reading
-it back made BIRD depend on the shape Cilium installs, and the IPv4 route is
-rejected outright: its next hop sits inside the prefix it describes, which KRT
-refuses with `strange next-hop`. The IPv6 route is a device route and passed,
-so the failure was silent on one family only.
-
-## Failure behaviour
-
-Benign at every layer. An empty or absent table 48 makes 980/981 fall through
-to main, and pod traffic returns to VLAN 40 — unseparated, not broken. The same
-holds if the DaemonSet cannot start, which is the case during a cold rebuild
-until Gitea serves its image.
-
-## Management, the remaining class
-
-Not started. The levers differ per service and none of them is the node IP:
-
-| service | can leave the node IP? | lever |
-|---|---|---|
-| etcd | yes | `cluster.etcd.advertisedSubnets`; needs a reboot per node |
-| kube-apiserver endpoint | yes | it is a VIP — `Layer2VIPConfig.link` |
-| Talos API | n/a | `apid` answers on every node address; a firewall question |
-| kubelet | no | it *is* the node IP |
-
-Two approaches were rejected by the owner and are not to be proposed again:
-moving the node IP and etcd to `pod0` (#838), and relabelling VLAN 40 as the
-cluster VLAN while moving etcd and the API VIP to a new management VLAN.
-Replacing the CNI (#839, Antrea) was shelved.
-
-Note that the reasoning which produced that list also concluded pod↔pod was
-blocked, which policy routing disproved. Treat it as a record of decisions, not
-as an exhaustive analysis.
-
-## Sources
-
-Cilium v1.20.1 `pkg/datapath/linux/node.go`, `pkg/node/types/node.go`,
-`pkg/node/sync/local_node_sync.go`; BIRD 2.17 user manual (kernel protocol,
-`learn all`, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig` and
-`machine.nodeAnnotations`. PRs #831, #833, #835, #836, #837, #842, #852, #854,
-#855; closed unmerged: #830, #838, #839.

kubernetes/PROPOSAL-traffic-class-separation.md

@@ -0,0 +1,103 @@
+# Proposal — separate the four traffic classes onto their own VLANs
+
+Status: open, issue #576. Two of the four classes are separated. Pod east-west
+cannot leave the node IP under Cilium. Both ways of moving it within Cilium
+(options A and B below) are rejected by the owner, and replacing the CNI
+(option C) is shelved.
+
+## Target
+
+Set by the owner on 2026-09-08:
+
+| class | link |
+|---|---|
+| Ceph replication | X710 p1, VLAN 49 |
+| pod-pod | X710 trunk, a VLAN of its own |
+| world-pod | X710 trunk, a VLAN of its own |
+| node management | 1 Gb onboard NIC |
+
+Hard constraint: inter-node and pod traffic never traverses the UDM. It stays
+on the 10G switch.
+
+## Where the classes are
+
+Measured 2026-09-15.
+
+| class | today | state |
+|---|---|---|
+| Ceph replication | `ceph-san`, VLAN 49, handed whole to the OSD pod (Multus `host-device`) | done |
+| world→pod | BGP from each node's `ingress0` (VLAN 47) to `fd02:47::1` / `10.2.47.1`; LB VIPs and PodCIDRs are advertised with that next-hop | done (#835, #836) |
+| pod→world | default routes via `mgmt0`: v4 metric 1024 against `ingress0`'s 2048, v6 from SLAAC on `mgmt0`; `devices` unset | open |
+| pod-pod | native routing, `<PodCIDR> via <peer InternalIP> dev mgmt0`, VLAN 40 | blocked, see below |
+| management | `mgmt0`, VLAN 40 | not started |
+
+`ingress0` (VLAN 47) and `pod0` (VLAN 48) exist on all three nodes (#831,
+#833). `pod0` holds addresses and carries no traffic.
+
+## Why pod-pod is blocked
+
+Cilium's route to a peer's pods uses the peer's Kubernetes InternalIP as the
+next-hop, in native mode, and as the VXLAN endpoint in tunnel mode:
+
+- `pkg/datapath/linux/node.go`: `installDirectRoute(…, nodeIP net.IP, …)` is
+  fed from `newNode.GetNodeIP(false)`.
+- `pkg/node/types/node.go`: `GetNodeIP` prefers `NodeInternalIP`.
+- `pkg/node/sync/local_node_sync.go`: `initFromK8s()` runs after
+  `initFromConfig()` and overwrites `--ipv4-node` / `--ipv6-node` from the
+  Node object.
+- `--direct-routing-device` selects a device for the BPF NodePort path, not a
+  next-hop.
+
+Tested on the cluster, none of these moved the next-hop:
+
+| attempt | result |
+|---|---|
+| `ipv4-node` / `ipv6-node` via `CiliumNodeConfig` | agent logged both flags, still published its `mgmt0` address |
+| `nodePort.directRoutingDevice: pod0` (#837) | agent logged `direct-routing-device=pod0`, still installed routes via `mgmt0` |
+| tunnel and native mode, v4 and v6 | endpoints and routes keyed to the peer's InternalIP |
+
+The InternalIP comes from `machine.kubelet.nodeIP.validSubnets` in
+`talos/patches/site-homelab.yaml`, currently `10.2.40.0/24` and `fd40::/64`.
+Pod-pod traffic therefore sits on whichever VLAN holds the InternalIP. The
+cluster has been in native routing mode since #837. `directRoutingDevice: pod0`
+has no effect on pod-pod routing.
+
+## Options for pod-pod
+
+| | option | status |
+|---|---|---|
+| A | move the node IP and etcd to `pod0` | **rejected** by the owner; #838 closed 2026-09-08 |
+| B | keep the node IP on VLAN 40, relabel VLAN 40 as the cluster VLAN, move etcd and the API VIP to a new management VLAN | **rejected** by the owner, repeatedly. Not to be proposed again |
+| C | replace Cilium with a CNI that picks the transport interface independently of the node IP (Antrea `transportInterface`) | **shelved** 2026-09-10; #839 closed. Needs a cold rebuild, rewriting the CiliumNetworkPolicy objects, and Antrea's BGP is alpha |
+
+A per-workload Multus attachment detaches a single workload from the node IP,
+the way Ceph replication is detached. It does not cover east-west traffic in
+general.
+
+## Levers per management service
+
+These hold regardless of which option is chosen:
+
+| service | can leave the node IP? | lever |
+|---|---|---|
+| etcd | yes | `cluster.etcd.advertisedSubnets` (+ `listenSubnets`, which defaults to it); needs a reboot per node |
+| kube-apiserver endpoint | yes | VIP, `Layer2VIPConfig` with a `link:` field |
+| Talos API | n/a | `apid` answers on every node address; a client and firewall choice |
+| kubelet | no | it is the node IP |
+
+## Remaining, independent of pod-pod
+
+- **pod→world egress**: default route and masquerade source onto `ingress0`,
+  via route metrics, `devices` and `enable-masquerade-to-route-source`. This
+  moves the nodes' own egress (image pulls, NTP) to VLAN 47 too.
+- **BGP rollback path**: the UDM still accepts sessions from VLAN 40; to be
+  removed once ingress on VLAN 47 is settled.
+
+## Sources
+
+- Cilium v1.20.1 source files listed above; `cilium-agent` flag reference for
+  `--direct-routing-device`.
+- Talos multihoming guide: `cluster.etcd.advertisedSubnets`, `listenSubnets`.
+- PRs #831, #833, #835, #836, #837, #838, #839 and the comments on #576.
+- Live state 2026-09-15: `kubectl get nodes -o wide`, `cilium-config`,
+  `CiliumBGPNodeConfig` status, node routing tables on k8s-1 and k8s-2.

kubernetes/apps/immich/machine-learning.yaml

@@ -32,7 +32,7 @@ spec:
           type: RuntimeDefault
       containers:
         - name: immich-machine-learning
-          image: ghcr.io/immich-app/immich-machine-learning:v3.2.2 # {"$imagepolicy": "flux-system:immich-machine-learning"}
+          image: ghcr.io/immich-app/immich-machine-learning:v3.2.1 # {"$imagepolicy": "flux-system:immich-machine-learning"}
           ports:
             - name: http
               containerPort: 3003

kubernetes/apps/immich/server.yaml

@@ -87,7 +87,7 @@ spec:
               mountPath: /ca
       containers:
         - name: immich-server
-          image: ghcr.io/immich-app/immich-server:v3.2.2 # {"$imagepolicy": "flux-system:immich-server"}
+          image: ghcr.io/immich-app/immich-server:v3.2.1 # {"$imagepolicy": "flux-system:immich-server"}
           ports:
             - name: http
               containerPort: 3001

kubernetes/apps/libation/cronjob.yaml

@@ -85,9 +85,7 @@ spec:
                   type: RuntimeDefault
           containers:
             - name: libation
-              # Pinned by digest: Spegel caches mutable tag resolutions, so
-              # :latest stayed on 13.7.11 despite imagePullPolicy Always.
-              image: docker.io/rmcrackan/libation:14.2.1@sha256:6f9d8ba19dc3803dd6ba295ab5a7abfb6ff3bc06286eebf4ecd8ccbe43245888
+              image: docker.io/rmcrackan/libation:latest
               env:
                 - name: LIBATION_CONFIG_DIR
                   value: /libation-state

kubernetes/apps/paperless/ftp-scanner.yaml

@@ -70,19 +70,24 @@ spec:
             - name: consume
               mountPath: /home/scanner
               subPath: scanner
-          # Asserts the control socket is listening without opening an FTP
-          # connection: vsftpd 3.0.5 segfaults its listener on a fraction of
-          # the connections it accepts. The passive range is negotiated per
-          # transfer and is not listening at rest, so it is not probed.
+          # FTP has no health verb, so the control port accepting a connection
+          # is what can be asserted. It is enough for the failure that matters:
+          # if vsftpd dies the scanner cannot deposit anything and nothing else
+          # in the chain notices, because paperless only ever sees an empty
+          # consume directory — which is indistinguishable from nobody having
+          # scanned.
+          #
+          # Only the control port is probed. The passive range is negotiated
+          # per transfer and is not listening at rest.
           livenessProbe:
-            exec:
-              command: ["sh", "-c", "netstat -lnt | grep -q ':21 '"]
+            tcpSocket:
+              port: ftp-ctrl
             initialDelaySeconds: 15
             periodSeconds: 30
             timeoutSeconds: 5
           readinessProbe:
-            exec:
-              command: ["sh", "-c", "netstat -lnt | grep -q ':21 '"]
+            tcpSocket:
+              port: ftp-ctrl
             initialDelaySeconds: 5
             periodSeconds: 10
             timeoutSeconds: 5

kubernetes/apps/paperless/kustomization.yaml

@@ -25,15 +25,18 @@ resources:
 configMapGenerator:
   # vsftpd config for the ftp-scanner deployment. The content-hash name
   # suffix is intentionally LEFT ENABLED (unlike home-assistant's nginx
-  # config): vsftpd reads these into vsftpd.conf at container start, so a
-  # changed value must roll the pod, which the renamed ConfigMap does.
-  #
-  # No PASV_ADDRESS: the LB address is declared once, on the Service
-  # (io.cilium/lb-ipam-ips). The image default 0.0.0.0 tells PASV clients
-  # to reuse the control-connection address.
+  # config): vsftpd templates pasv_address from PASV_ADDRESS at container
+  # start, so a changed value must roll the pod. With the hash suffix,
+  # editing any value here renames the generated ConfigMap, which mutates
+  # the Deployment's pod template and triggers an automatic rolling
+  # restart. Without it, an IP change (e.g. an LB re-allocation) silently
+  # leaves the running pod advertising the stale PASV address, which
+  # breaks classic-PASV clients (older Brother scanners) while EPSV
+  # clients keep working.
   - name: ftp-scanner-config
     namespace: paperless
     literals:
+      - PASV_ADDRESS=10.2.99.82
       - PASV_MIN_PORT=21100
       - PASV_MAX_PORT=21104
       - FTP_USER=scanner

kubernetes/apps/zone-configurator/deployment.yaml

@@ -21,7 +21,7 @@ spec:
       # Service; nothing on this path needs the host's network namespace.
       containers:
         - name: zone-configurator
-          image: everythingsmarthome/everything-presence-mmwave-configurator:2.2.5 # {"$imagepolicy": "flux-system:mmwave-configurator"}
+          image: everythingsmarthome/everything-presence-mmwave-configurator:2.2.4 # {"$imagepolicy": "flux-system:mmwave-configurator"}
           env:
             - name: HA_BASE_URL
               value: "http://home-assistant:80"

kubernetes/bootstrap.sh

@@ -610,8 +610,8 @@ if [[ "$PHASE" == "hcloudtest" ]]; then
     # Constraints on the choice: cpx11/cpx21 are US-only and cax* are ARM, which
     # would not match the amd64 schematic this cluster installs. That leaves the
     # cx*/cpx*2 families in the German locations.
-    # Two segments only, unlike the homelab's split in
-    # NETWORK-traffic-classes.md: here the node address carries management,
+    # Today's traffic-class split, not the target one in
+    # PROPOSAL-traffic-class-separation.md: the node address carries management,
     # cluster-internal and external alike, and Ceph replication has its own.
     #
     # The order is load-bearing: it decides which kernel interface each segment

kubernetes/infrastructure/bird/bird.conf

@@ -1,103 +0,0 @@
-# Identical on every node and free of addresses. defines.conf and peers.conf
-# are rendered per node by the init container; the first is included above the
-# templates that reference it, the second below them.
-include "/etc/bird/defines.conf";
-
-router id from "pod0";
-log stderr all;
-
-protocol device { scan time 10; }
-
-# Everything a pod may need to reach, in one table, resolved by longest prefix:
-# the node's attached networks, the peers' pod CIDRs, and a default for the
-# rest. No rule has to carve out exceptions, because the specific routes win on
-# their own.
-#
-# The table is BIRD's alone. If this daemon stops, it empties, the rule finds
-# nothing and falls through to main — pod traffic returns to VLAN 40,
-# unseparated rather than misrouted. That is why the default route lives here
-# and not in the Talos config: a default that outlived the specific routes
-# would send pod traffic to the UDM.
-
-# The node's attached networks, taken from the interfaces rather than written
-# down. The usual advice is not to export these to the kernel because it
-# maintains them itself — true of main, not of a table the kernel never fills.
-protocol direct attached {
-  interface "mgmt0", "ingress0", "pod0";
-  ipv4 { table master4; };
-  ipv6 { table master6; };
-}
-
-# Pod egress. GW4/GW6 are the UDM on the ingress segment, published as node
-# annotations beside the address that defines them.
-protocol static egress4 {
-  ipv4 { table master4; };
-  route 0.0.0.0/0 via GW4;
-}
-protocol static egress6 {
-  ipv6 { table master6; };
-  route ::/0 via GW6;
-}
-
-# This node's own pod CIDR, via the interface Cilium serves it on. Originated
-# rather than read back out of the kernel: reading it back made BIRD's view
-# depend on the shape of Cilium's route, and the IPv4 one is rejected outright,
-# its next hop sitting inside the prefix it describes.
-#
-# It belongs in the table, not just in the BGP announcement. A reply from a pod
-# on another node is forwarded traffic with a pod source, so the rule matches it
-# too; without this route it would find only the default and leave for the UDM.
-# Measured — cross-node pod traffic, and with it cluster DNS, stopped.
-protocol static own4 {
-  ipv4 { table master4; };
-  route OWN_POD4 via "cilium_host";
-}
-protocol static own6 {
-  ipv6 { table master6; };
-  route OWN_POD6 via "cilium_host";
-}
-
-protocol kernel k4 {
-  kernel table 47;
-  metric 0;
-  ipv4 { table master4; import none; export all; };
-}
-protocol kernel k6 {
-  kernel table 47;
-  metric 0;
-  ipv6 { table master6; import none; export all; };
-}
-
-# iBGP over pod0. 64514 keeps clear of Cilium's 64513, which peers with the UDM
-# from ingress0 and is unrelated.
-#
-# direct is explicit because iBGP defaults to multihop, which makes the gateway
-# recursive — BIRD would try to resolve the peer in the pod RIB. strict bind
-# keeps the listener off mgmt0 and ingress0.
-#
-# Only the node's own CIDR is offered: the export filter drops everything the
-# static protocols and the direct protocol produce.
-template bgp pod4 {
-  local as 64514;
-  source address MY4;
-  direct;
-  strict bind;
-  ipv4 {
-    next hop self;
-    import all;
-    export filter { if net = OWN_POD4 then accept; reject; };
-  };
-}
-template bgp pod6 {
-  local as 64514;
-  source address MY6;
-  direct;
-  strict bind;
-  ipv6 {
-    next hop self;
-    import all;
-    export filter { if net = OWN_POD6 then accept; reject; };
-  };
-}
-
-include "/etc/bird/peers.conf";

kubernetes/infrastructure/bird/daemonset.yaml

@@ -1,115 +0,0 @@
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: bird
-  namespace: kube-system
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRole
-metadata:
-  name: bird
-rules:
-  # The peer list is read from the nodes' pod0 annotations, so no address is
-  # written into this repo twice.
-  - apiGroups: [""]
-    resources: ["nodes"]
-    verbs: ["get", "list"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRoleBinding
-metadata:
-  name: bird
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: ClusterRole
-  name: bird
-subjects:
-  - kind: ServiceAccount
-    name: bird
-    namespace: kube-system
----
-apiVersion: apps/v1
-kind: DaemonSet
-metadata:
-  name: bird
-  namespace: kube-system
-  labels:
-    tier: node
-    app: bird
-spec:
-  selector:
-    matchLabels:
-      app: bird
-  updateStrategy:
-    type: RollingUpdate
-  template:
-    metadata:
-      labels:
-        tier: node
-        app: bird
-    spec:
-      # Carries pod east-west once the routing rules land. Without it table 48
-      # stays empty and the rules fall through to main, i.e. back to VLAN 40.
-      priorityClassName: system-node-critical
-      # Writes host routes and peers on host addresses.
-      hostNetwork: true
-      serviceAccountName: bird
-      # Every node here is a control plane node, so this is not optional.
-      tolerations:
-        - operator: Exists
-          effect: NoSchedule
-      initContainers:
-        - name: render-config
-          # By digest, not by the dated tag: the tag is rebuilt within the day
-          # it names, and the cluster's image path keeps resolving it to the
-          # first build of that day — imagePullPolicy: Always does not help,
-          # because the tag itself resolves stale. Update this when the image
-          # is rebuilt.
-          image: gitea.renner.uno/gitea_admin/bird@sha256:3bc8500613eeb45b2d8858b5445198bae3990ecf64f6d891aa94797f5a09dc1d
-          command: ["/bin/sh", "-c"]
-          args:
-            - |
-              set -eu
-              cp /config/bird.conf /etc/bird/bird.conf
-              kubectl get node "$NODE_NAME" \
-                -o go-template-file=/config/defines.tmpl > /etc/bird/defines.conf
-              kubectl get nodes --field-selector "metadata.name!=$NODE_NAME" \
-                -o go-template-file=/config/peers.tmpl > /etc/bird/peers.conf
-              # BIRD's own parser is the check: a node whose annotation is
-              # missing renders as <no value> and fails here rather than
-              # crashlooping the daemon or forming a mesh with a hole.
-              bird -p -c /etc/bird/bird.conf
-          env:
-            - name: NODE_NAME
-              valueFrom:
-                fieldRef:
-                  fieldPath: spec.nodeName
-          volumeMounts:
-            - name: config
-              mountPath: /config
-            - name: etc-bird
-              mountPath: /etc/bird
-      containers:
-        - name: bird
-          image: gitea.renner.uno/gitea_admin/bird@sha256:3bc8500613eeb45b2d8858b5445198bae3990ecf64f6d891aa94797f5a09dc1d
-          resources:
-            requests:
-              cpu: "50m"
-              memory: "32Mi"
-            limits:
-              cpu: "200m"
-              memory: "128Mi"
-          securityContext:
-            capabilities:
-              drop: ["ALL"]
-              # NET_ADMIN writes the routes; NET_BIND_SERVICE binds 179.
-              add: ["NET_ADMIN", "NET_BIND_SERVICE"]
-          volumeMounts:
-            - name: etc-bird
-              mountPath: /etc/bird
-      volumes:
-        - name: config
-          configMap:
-            name: bird-config
-        - name: etc-bird
-          emptyDir: {}

kubernetes/infrastructure/bird/defines.tmpl

@@ -1,9 +0,0 @@
-define MY6 = {{ index .metadata.annotations "net.renner.uno/pod0-v6" }};
-define MY4 = {{ index .metadata.annotations "net.renner.uno/pod0-v4" }};
-define GW6 = {{ index .metadata.annotations "net.renner.uno/egress-gw-v6" }};
-define GW4 = {{ index .metadata.annotations "net.renner.uno/egress-gw-v4" }};
-{{/* podCIDRs follows the cluster's podSubnets order, v4 first. A flip puts an
-     IPv6 prefix into an ipv4 protocol, which fails the parse in the init
-     container rather than passing silently. */}}
-define OWN_POD4 = {{ index .spec.podCIDRs 0 }};
-define OWN_POD6 = {{ index .spec.podCIDRs 1 }};

kubernetes/infrastructure/bird/kustomization.yaml

@@ -1,12 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - daemonset.yaml
-configMapGenerator:
-  # Hashed name, so editing any of these rolls the DaemonSet.
-  - name: bird-config
-    namespace: kube-system
-    files:
-      - bird.conf
-      - defines.tmpl
-      - peers.tmpl

kubernetes/infrastructure/bird/peers.tmpl

@@ -1,7 +0,0 @@
-{{- range $i, $n := .items -}}
-{{- $v6 := index $n.metadata.annotations "net.renner.uno/pod0-v6" -}}
-{{- $v4 := index $n.metadata.annotations "net.renner.uno/pod0-v4" -}}
-# {{ $n.metadata.name }}
-protocol bgp peer{{ $i }}_v6 from pod6 { neighbor {{ $v6 }} as 64514; }
-protocol bgp peer{{ $i }}_v4 from pod4 { neighbor {{ $v4 }} as 64514; }
-{{ end -}}

kubernetes/infrastructure/kustomization.yaml

@@ -7,7 +7,6 @@ resources:
   - cluster-settings/
   - cilium/
   - multus/
-  - bird/
   - cert-manager/
   - snapshot-controller/
   - metrics-server/

kubernetes/infrastructure/rook-ceph-cluster/cephcluster.yaml

@@ -126,16 +126,6 @@ spec:
     addressRanges:
       cluster:
         - 10.99.0.0/29
-  # Reads go to the OSD in the client's own CRUSH location. crushLocationLabels
-  # is unset, so Rook's default list applies; of those the nodes carry
-  # topology.kubernetes.io/zone, one zone per node, matching the zone buckets
-  # the OSDs are created under.
-  #
-  # Passed as a krbd/mount option at map time, so a volume keeps its current
-  # behaviour until it is remounted.
-  csi:
-    readAffinity:
-      enabled: true
   # Nodes listed explicitly, not useAllNodes: under useAllNodes the operator
   # re-provisions an OSD on any node whose device still carries a BlueStore
   # signature, so purging an OSD before pulling its drive is impossible — it

kubernetes/talos/patches/k8s-1.yaml

@@ -52,12 +52,6 @@ machine:
     # this label instead of listing hostnames, so which boxes carry storage is
     # stated once, here, by the machine that knows.
     storage.renner.uno/ceph-osd: "true"
-  # Publishes this node's pod0 addresses, declared below in its VLANConfig, so
-  # the bird DaemonSet reads its peers from the node list rather than from a
-  # copy. Annotations, not labels: a label value cannot contain a colon.
-  nodeAnnotations:
-    net.renner.uno/pod0-v6: fd02:48::11
-    net.renner.uno/pod0-v4: 10.2.48.11
   udev:
     # Renames every port this node uses to its role at boot, matched by
     # burned-in MAC — kernel names derive from PCI topology and shift when
@@ -188,13 +182,8 @@ addresses:
 # network's clients selectively, so the client goes rather than the record.
 # DHCP is off on VLAN 47 for the same reason.
 #
-# No route here. Table 47 belongs to bird, which fills it with this node's
-# attached networks, the peers' pod CIDRs and the egress default, and empties it
-# if it stops — leaving the rule to fall through to main. A default declared
-# here would outlive the specific routes and send pod traffic to the UDM.
-#
-# accept_ra_defrtr stays 0 on this link so the RA default does not land in main,
-# where mgmt0 owns it.
+# The metric loses to mgmt0's default 1024 deliberately: this pass only adds
+# the segment. The default route moves here in the change that flips it.
 apiVersion: v1alpha1
 kind: VLANConfig
 name: ingress0
@@ -203,6 +192,9 @@ parent: uplink
 up: true
 addresses:
   - address: 10.2.47.11/24
+routes:
+  - gateway: 10.2.47.1
+    metric: 2048
 ---
 # Dedicated Ceph cluster network on X710 port 1, VLAN 49, untagged on the
 # wire. OSD-to-OSD replication and nothing else. The udev rule in the machine

kubernetes/talos/patches/k8s-2.yaml

@@ -33,10 +33,6 @@ machine:
     topology.kubernetes.io/zone: saturn
     # Holds a Ceph OSD drive — see k8s-1.yaml.
     storage.renner.uno/ceph-osd: "true"
-  # pod0 addresses for the bird peer list — see k8s-1.yaml.
-  nodeAnnotations:
-    net.renner.uno/pod0-v6: fd02:48::12
-    net.renner.uno/pod0-v4: 10.2.48.12
   udev:
     # Renames every port to its role at boot — see k8s-1.yaml.
     rules:
@@ -132,6 +128,9 @@ parent: uplink
 up: true
 addresses:
   - address: 10.2.47.12/24
+routes:
+  - gateway: 10.2.47.1
+    metric: 2048
 ---
 # Dedicated Ceph cluster network on X710 port 1, VLAN 49, untagged on the
 # wire. OSD-to-OSD replication only, not routed off its own segment.

kubernetes/talos/patches/k8s-3.yaml

@@ -29,10 +29,6 @@ machine:
     topology.kubernetes.io/zone: neptune
     # Holds a Ceph OSD drive — see k8s-1.yaml.
     storage.renner.uno/ceph-osd: "true"
-  # pod0 addresses for the bird peer list — see k8s-1.yaml.
-  nodeAnnotations:
-    net.renner.uno/pod0-v6: fd02:48::13
-    net.renner.uno/pod0-v4: 10.2.48.13
   udev:
     # Renames every port to its role at boot — see k8s-1.yaml.
     rules:
@@ -119,6 +115,9 @@ parent: uplink
 up: true
 addresses:
   - address: 10.2.47.13/24
+routes:
+  - gateway: 10.2.47.1
+    metric: 2048
 ---
 # Dedicated Ceph cluster network — see k8s-1.yaml. The node holds NO address
 # here on purpose; the link is handed whole to the OSD pod (host-device).

kubernetes/talos/patches/site-homelab.yaml

@@ -20,38 +20,3 @@ machine:
       validSubnets:
         - 10.2.40.0/24
         - fd40::/64
-  # The UDM on the ingress segment. Published here rather than as a route
-  # because bird owns table 47 and originates the default from it; this is the
-  # one place the address is stated.
-  nodeAnnotations:
-    net.renner.uno/egress-gw-v6: fd02:47::1
-    net.renner.uno/egress-gw-v4: 10.2.47.1
----
-# All pod traffic goes to table 47, and the table decides — longest prefix
-# match, no exceptions to carve out. bird fills it with the node's attached
-# networks, the peers' pod CIDRs and the egress default, so a packet to a node
-# on the mgmt segment takes the connected route, a packet to a pod on another
-# node takes pod0, and anything else takes the default over ingress0.
-#
-# Matching the source is what keeps the node's own traffic out: it answers on
-# mgmt0, and a reply leaving over ingress0 would carry an mgmt0 source onto
-# VLAN 47. The routing decision runs before Cilium's masquerade hook, so the
-# rule sees the pod address and the masquerade then rewrites it.
-#
-# If bird stops, table 47 empties, the lookup misses and the rule falls through
-# to main — pod traffic returns to VLAN 40, unseparated rather than misrouted.
-#
-# Priorities are below main (32766) and clear of the rule Cilium installs at 9.
-# The two families are separate rule tables, so the priorities need not match;
-# the document name is the priority and must be unique.
-apiVersion: v1alpha1
-kind: RoutingRuleConfig
-name: "1000"
-src: fd10:244::/48
-table: "47"
----
-apiVersion: v1alpha1
-kind: RoutingRuleConfig
-name: "1001"
-src: 10.244.0.0/16
-table: "47"