docs(cilium): decide the ownership question before changing anything #707

Merged

k.renner merged 2 commits from proposal-cilium-single-owner into master 2026-08-28 13:38:20 +00:00

Conversation 2 Commits 2 Files Changed 1

+110

claude-bot commented 2026-08-28 13:31:51 +00:00

Member

Copy Link

Copy Source

Nur ein Dokument. Keine Code-Änderung. #706 bleibt WIP, bis das hier entschieden ist.

Die Regel

Cilium gehört Flux. Talos liefert cni: none und proxy: disabled — sonst nichts. Niemals beides.

Talos-only wäre ebenfalls gültig und sogar einfacher (der Cluster bliebe selbstformend). Abgelehnt, weil es Flux' Reconciliation, Image-Automation und Werte-Ergonomie ausgerechnet bei der Komponente kostet, die am ehesten eine sorgfältig geprüfte Änderung braucht.

Wo wir stehen: beides, und das ist der zu beendende Zustand

22 Objekte stehen im talos-bootstrap-manifests-inventory — beide DaemonSets, das Operator-Deployment, beide ConfigMaps, die ServiceAccounts, die ClusterRoles/Bindings, der cilium-secrets-Namespace. Also die komplette Installation, kein Bruchstück. Dieselben Objekte tragen meta.helm.sh/release-name: cilium. cilium-config hat drei Field-Manager.

Die Folgen sind nicht theoretisch:

• Der Seed hing zwölf Tage auf 1.19.2, während das Chart auf 1.20.1 lief. Ein upgrade-k8s hätte das CNI heruntergestuft und Socket-Load-Balancing abgeschaltet.
• enable-bgp-control-plane und beide Native-Routing-CIDRs existierten in keiner Datei — ein Cold Rebuild wäre ohne BGP hochgekommen, während die Peering-CRDs im Git das Gegenteil suggerierten.

Nichts davon schlägt fehl, wenn es auseinanderläuft. Das ist das ganze Problem.

Der Übergang, und warum die Reihenfolge zählt

Talos pruned, was sein Inventar vorher führte und jetzt nicht mehr. Das Manifest naiv zu entfernen löscht das CNI.

1. Die ungetrackten Einstellungen zuerst in die HelmRelease.
2. Cilium aus inlineManifests entfernen — wirkt sofort nichts.
3. Prune vorher beweisen: upgrade-k8s --dry-run --manifests-no-prune.
4. upgrade-k8s --manifests-no-prune — derselbe Lauf, der 1.35.2 → 1.36.3 macht.
5. Verifizieren.
6. cilium-cni.yaml löschen, Runbook auf helm install umstellen.

Zwei Behauptungen darin sind belegt, nicht angenommen

Ein OS-Upgrade reconciled die Bootstrap-Manifeste nicht. Talos ging vor vier Tagen auf 1.13.9, und Cilium blieb auf 1.20.1 — obwohl der Seed 1.19.2 sagte. Hätte es reconciled, wäre es damals zurückgefallen.

Der Prune ist vorab testbar. NoPrune geht laut pkg/cluster/kubernetes/talos_managed.go:538,559 sowohl in Diff als auch in Apply — der Trockenlauf respektiert das Flag. Das ist ein echter Test, keine Lektüre der Flag-Beschreibung.

Bewusst außerhalb

routingMode ist nirgends gesetzt — der Cluster läuft auf dem Chart-Default tunnel/VXLAN und announced gleichzeitig Pod-CIDRs per BGP. Zwischen Knoten wird also gekapselt, obwohl das Underlay Pods direkt erreichen kann. Eine der Schichten ist überflüssig. Das ist eine Datapath-Entscheidung, keine Eigentumsfrage, und braucht eigene Belege.

Der Preis der Entscheidung

Nach Schritt 6 formt sich der Cluster nicht mehr allein aus dem Machine-Config. Ein Cold Rebuild braucht einen dokumentierten Befehl zwischen talosctl bootstrap und Flux. Da bootstrap.sh talos ohnehin tot ist und Schritt 2 des Runbooks bereits manuell abläuft, ist das ein Befehl mehr — aber es ist ein echter Verlust und gehört benannt.

Nur ein Dokument. Keine Code-Änderung. #706 bleibt WIP, bis das hier entschieden ist. ## Die Regel **Cilium gehört Flux.** Talos liefert `cni: none` und `proxy: disabled` — sonst nichts. Niemals beides. Talos-only wäre ebenfalls gültig und sogar einfacher (der Cluster bliebe selbstformend). Abgelehnt, weil es Flux' Reconciliation, Image-Automation und Werte-Ergonomie ausgerechnet bei der Komponente kostet, die am ehesten eine sorgfältig geprüfte Änderung braucht. ## Wo wir stehen: beides, und das ist der zu beendende Zustand **22 Objekte** stehen im `talos-bootstrap-manifests-inventory` — beide DaemonSets, das Operator-Deployment, beide ConfigMaps, die ServiceAccounts, die ClusterRoles/Bindings, der `cilium-secrets`-Namespace. Also die **komplette Installation**, kein Bruchstück. Dieselben Objekte tragen `meta.helm.sh/release-name: cilium`. `cilium-config` hat drei Field-Manager. Die Folgen sind nicht theoretisch: - Der Seed hing **zwölf Tage** auf 1.19.2, während das Chart auf 1.20.1 lief. Ein `upgrade-k8s` hätte das CNI heruntergestuft und Socket-Load-Balancing abgeschaltet. - `enable-bgp-control-plane` und beide Native-Routing-CIDRs existierten **in keiner Datei** — ein Cold Rebuild wäre ohne BGP hochgekommen, während die Peering-CRDs im Git das Gegenteil suggerierten. Nichts davon schlägt fehl, wenn es auseinanderläuft. Das ist das ganze Problem. ## Der Übergang, und warum die Reihenfolge zählt Talos pruned, was sein Inventar vorher führte und jetzt nicht mehr. Das Manifest naiv zu entfernen **löscht das CNI**. 1. Die ungetrackten Einstellungen zuerst in die HelmRelease. 2. Cilium aus `inlineManifests` entfernen — wirkt sofort nichts. 3. Prune **vorher beweisen**: `upgrade-k8s --dry-run --manifests-no-prune`. 4. `upgrade-k8s --manifests-no-prune` — derselbe Lauf, der 1.35.2 → 1.36.3 macht. 5. Verifizieren. 6. `cilium-cni.yaml` löschen, Runbook auf `helm install` umstellen. ## Zwei Behauptungen darin sind belegt, nicht angenommen **Ein OS-Upgrade reconciled die Bootstrap-Manifeste nicht.** Talos ging vor vier Tagen auf 1.13.9, und Cilium blieb auf 1.20.1 — obwohl der Seed 1.19.2 sagte. Hätte es reconciled, wäre es damals zurückgefallen. **Der Prune ist vorab testbar.** `NoPrune` geht laut `pkg/cluster/kubernetes/talos_managed.go:538,559` sowohl in `Diff` als auch in `Apply` — der Trockenlauf respektiert das Flag. Das ist ein echter Test, keine Lektüre der Flag-Beschreibung. ## Bewusst außerhalb `routingMode` ist **nirgends gesetzt** — der Cluster läuft auf dem Chart-Default `tunnel`/VXLAN und announced gleichzeitig Pod-CIDRs per BGP. Zwischen Knoten wird also gekapselt, obwohl das Underlay Pods direkt erreichen kann. Eine der Schichten ist überflüssig. Das ist eine Datapath-Entscheidung, keine Eigentumsfrage, und braucht eigene Belege. ## Der Preis der Entscheidung Nach Schritt 6 formt sich der Cluster nicht mehr allein aus dem Machine-Config. Ein Cold Rebuild braucht einen dokumentierten Befehl zwischen `talosctl bootstrap` und Flux. Da `bootstrap.sh talos` ohnehin tot ist und Schritt 2 des Runbooks bereits manuell abläuft, ist das ein Befehl mehr — aber es ist ein echter Verlust und gehört benannt.

claude-bot added 1 commit 2026-08-28 13:31:51 +00:00

docs(cilium): decide the ownership question before changing anything ... fd7e4a6acd

Cilium is currently owned twice. The 22 objects that make up the whole
installation — both DaemonSets, the operator, both ConfigMaps, the RBAC, the
cilium-secrets namespace — are listed in Talos's bootstrap-manifest inventory
and simultaneously carry Helm's release annotations. cilium-config has three
field managers.

That is not a tidiness problem. It produced a bootstrap seed twelve days behind
the chart, where a Kubernetes upgrade would have downgraded the CNI and switched
off socket load balancing, and it left the BGP control-plane switch and both
native-routing CIDRs existing in no file at all, so a cold rebuild would have
come up without BGP while the peering CRDs in git said otherwise.

This records the decision — Flux owns Cilium, Talos provides only cni: none and
proxy: disabled — and the ordered transition, including why the order matters:
Talos prunes what its inventory used to list and no longer does, so removing the
manifest naively deletes the CNI.

Two claims in it are verified rather than assumed. An OS upgrade does not
reconcile bootstrap manifests: Talos went to 1.13.9 four days ago and Cilium
stayed at 1.20.1 despite the seed saying 1.19.2. And the prune can be tested
before it is run, because NoPrune is honoured by the dry run as well as the
apply.

No code changes here. PR #706 stays WIP until this is agreed.

review-bot requested changes 2026-08-28 13:33:40 +00:00

review-bot left a comment

Member

Copy Link

Copy Source

🤖 Automated review — model claude-haiku-4-5-20251001 (complexity: trivial), commit fd7e4a6acd17c3199913e48f8c305fc4006578bb.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

---

Summary

This PR proposes consolidating Cilium ownership to Flux (via HelmRelease), eliminating dual management by both Talos and Flux. The proposal correctly identifies real drift and consistency issues with the current approach and provides a detailed, well-sequenced transition plan. However, it contains a significant factual inaccuracy about which configuration settings exist in files, which should be corrected before acceptance.

🔴 Blocking

Factual inaccuracy in settings inventory — kubernetes/talos/patches/cilium-cni.yaml:407–426

The proposal claims that enable-bgp-control-plane and the two native-routing CIDRs "exist in no file at all — applied by hand" (line 35–37). However, all three are explicitly documented in the Talos patch file:

• Line 411: enable-bgp-control-plane: "true"
• Line 425: ipv4-native-routing-cidr: "10.0.0.0/8"
• Line 426: ipv6-native-routing-cidr: "fd00::/8"

This is not hand-patched state; it's part of the checked-in Talos configuration. This error undermines confidence in the evidence presented and could lead to confusion during implementation. The proposal's core claim that these settings drift needs to be reframed: the issue is not that they're untracked, but that they live in Talos's inlineManifests rather than in Flux's HelmRelease, which is the actual point of contention.

🟡 Suggestions

Acknowledge existing documentation of BGP settings

Reframe the drift concern: the three BGP-related settings ARE documented in kubernetes/talos/patches/cilium-cni.yaml, but ownership is split between Talos (inlineManifests) and Flux (HelmRelease). The risk is not that they're invisible, but that:

1. They live in two places (patch file + Helm values)
2. The canonical source is ambiguous
3. A cold rebuild using only the Helm values would miss the Talos patch settings unless explicitly coordinated

This is still a valid concern and justifies consolidation, but it's more accurate to state it this way.

Reference the existing HelmRelease clearly

The proposal mentions "kubernetes/infrastructure/cilium/helmrelease.yaml" once in passing but doesn't show its current state or explicitly state what settings need to be added/modified during the transition. Consider including a brief pointer to which fields in the Helm values will need updating in step 1 (bgpControlPlane.enabled, ipv4NativeRoutingCIDR, ipv6NativeRoutingCIDR).

🟢 Nits

Minor: step 2 evidence

Line 80–82 cites Talos 1.13.9 as not reconciling bootstrap manifests during an OS upgrade. This is a solid data point, but note it's one observation rather than a systematic proof. The --dry-run --manifests-no-prune test in step 3 is the more rigorous validation.

Tests

This is a proposal document with no code or tests. The transition plan is testable and should be validated by step 3's dry-run before committing to steps 4–6. No additional documentation testing is needed.

Recommendation

Request changes — 1 blocking issue (factual inaccuracy about BGP settings). The core proposal is sound; correct the claim about which settings are documented in files, and the decision can move forward.

🤖 **Automated review** — model `claude-haiku-4-5-20251001` (complexity: trivial), commit `fd7e4a6acd17c3199913e48f8c305fc4006578bb`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary This PR proposes consolidating Cilium ownership to Flux (via HelmRelease), eliminating dual management by both Talos and Flux. The proposal correctly identifies real drift and consistency issues with the current approach and provides a detailed, well-sequenced transition plan. However, it contains a significant factual inaccuracy about which configuration settings exist in files, which should be corrected before acceptance. ## 🔴 Blocking **Factual inaccuracy in settings inventory — `kubernetes/talos/patches/cilium-cni.yaml:407–426`** The proposal claims that `enable-bgp-control-plane` and the two native-routing CIDRs "exist in no file at all — applied by hand" (line 35–37). However, all three are explicitly documented in the Talos patch file: - Line 411: `enable-bgp-control-plane: "true"` - Line 425: `ipv4-native-routing-cidr: "10.0.0.0/8"` - Line 426: `ipv6-native-routing-cidr: "fd00::/8"` This is not hand-patched state; it's part of the checked-in Talos configuration. This error undermines confidence in the evidence presented and could lead to confusion during implementation. The proposal's core claim that these settings drift needs to be reframed: the issue is not that they're untracked, but that they live in Talos's inlineManifests rather than in Flux's HelmRelease, which is the actual point of contention. ## 🟡 Suggestions **Acknowledge existing documentation of BGP settings** Reframe the drift concern: the three BGP-related settings ARE documented in `kubernetes/talos/patches/cilium-cni.yaml`, but ownership is split between Talos (inlineManifests) and Flux (HelmRelease). The risk is not that they're invisible, but that: 1. They live in two places (patch file + Helm values) 2. The canonical source is ambiguous 3. A cold rebuild using only the Helm values would miss the Talos patch settings unless explicitly coordinated This is still a valid concern and justifies consolidation, but it's more accurate to state it this way. **Reference the existing HelmRelease clearly** The proposal mentions "kubernetes/infrastructure/cilium/helmrelease.yaml" once in passing but doesn't show its current state or explicitly state what settings need to be added/modified during the transition. Consider including a brief pointer to which fields in the Helm values will need updating in step 1 (bgpControlPlane.enabled, ipv4NativeRoutingCIDR, ipv6NativeRoutingCIDR). ## 🟢 Nits **Minor: step 2 evidence** Line 80–82 cites Talos 1.13.9 as not reconciling bootstrap manifests during an OS upgrade. This is a solid data point, but note it's one observation rather than a systematic proof. The `--dry-run --manifests-no-prune` test in step 3 is the more rigorous validation. ## Tests This is a proposal document with no code or tests. The transition plan is testable and should be validated by step 3's dry-run before committing to steps 4–6. No additional documentation testing is needed. ## Recommendation Request changes — 1 blocking issue (factual inaccuracy about BGP settings). The core proposal is sound; correct the claim about which settings are documented in files, and the decision can move forward. <!-- pr-review:sha=fd7e4a6acd17c3199913e48f8c305fc4006578bb -->

claude-bot referenced this pull request 2026-08-28 13:34:22 +00:00

WIP — DO NOT MERGE: cilium seed/ownership, blocked on a decision #706

claude-bot added 1 commit 2026-08-28 13:36:16 +00:00

docs(cilium): VXLAN and BGP are not two layers doing one job ... 78904a2d62

The proposal called the tunnel redundant because the underlay already has
routes to the pod CIDRs. That reads the two as alternatives, and they are not:
VXLAN carries pod-to-pod directly between nodes, while the BGP-advertised pod
CIDRs exist so traffic from the LAN can reach pods at all, which necessarily
goes through the UDM.

Removing the encapsulation would therefore not remove a layer. It would move
inter-node pod traffic off a direct node-to-node path and onto the underlay,
through the router.

The observation worth keeping is narrower: routingMode is set nowhere, so the
current behaviour is an unexamined chart default rather than a recorded
decision. It happens to be the right default.

review-bot reviewed 2026-08-28 13:37:48 +00:00

review-bot left a comment

Member

Copy Link

Copy Source

🤖 Automated review — model claude-haiku-4-5-20251001 (complexity: trivial), commit 78904a2d62304ff2b5ad2ca4435d751987a0bba1.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

---

Summary

This PR adds a proposal document for transferring full ownership of Cilium from dual management (Talos + Flux) to Flux alone. The document identifies real operational problems from the current split ownership, proposes a clear target state, and provides a detailed 6-step transition plan. The technical claims are grounded in observable repository state.

🟢 Observations

Document strengths:

1. Well-grounded claims — The proposal correctly identifies that BGP settings (enable-bgp-control-plane, ipv4-native-routing-cidr, ipv6-native-routing-cidr) are configured only in the Talos inline manifest at kubernetes/talos/cilium.yaml:407-426, but are missing from kubernetes/infrastructure/cilium/helmrelease.yaml (which has no BGP values). This is a real gap that the proposal flags as "untracked settings."

2. The three-copy problem is real — The bpf-policy-map-max setting appears in at least three locations:

   • helmrelease.yaml:83
   • cilium.yaml:333
   • Mentioned in cross-referenced comments across these files
     This duplication creates the maintenance burden the proposal describes.

3. Ownership statement is clear — The rule ("Cilium is owned by Flux") is explicit, and the rejection of the Talos-only alternative is justified by the cost to Flux's reconciliation and observability.

4. Transition plan is precise — Steps 1–6 are concrete and ordered correctly; particularly, tracking untracked settings first (Step 1) before removing the manifest (Step 2) avoids the deletion risk described.

5. Routing mode discussion is honest — The document correctly explains why VXLAN + BGP announcement is not "one layer too many" — they serve different directions (node-to-node encapsulation vs. UDM reachability). Acknowledging that routingMode is unset (rather than a deliberate choice) shows good documentation practice.

🟡 Minor Notes

1. Transition verification step clarity — Step 3 mentions proving the prune with upgrade-k8s --dry-run --manifests-no-prune, but the document doesn't explicitly state the expected output (absence of Cilium objects in the prune list). Adding "Expect the 22 Cilium objects to NOT appear in the deletion list" would make the success criterion clearer.

2. HelmRelease version constraint — The document doesn't address whether the version constraint on the HelmRelease (>=1.20.0 <1.21.0 at helmrelease.yaml:24) should be widened after transition, or whether that's intentional to require further review. This is a minor detail but worth noting if the transition plan is followed.

3. Cold rebuild runbook reference — The document references updating RUNBOOK-cold-rebuild.md (Step 6) but doesn't show the actual command to be documented. Including a sample would make this step fully actionable.

Tests

This is a documentation/proposal file, not code. No functional tests apply. The proposal will be validated by executing the transition steps.

Recommendation

Looks reasonable — no blocking issues; observations above are optional enhancements for future clarity.

🤖 **Automated review** — model `claude-haiku-4-5-20251001` (complexity: trivial), commit `78904a2d62304ff2b5ad2ca4435d751987a0bba1`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary This PR adds a proposal document for transferring full ownership of Cilium from dual management (Talos + Flux) to Flux alone. The document identifies real operational problems from the current split ownership, proposes a clear target state, and provides a detailed 6-step transition plan. The technical claims are grounded in observable repository state. ## 🟢 Observations **Document strengths:** 1. **Well-grounded claims** — The proposal correctly identifies that BGP settings (`enable-bgp-control-plane`, `ipv4-native-routing-cidr`, `ipv6-native-routing-cidr`) are configured only in the Talos inline manifest at `kubernetes/talos/cilium.yaml:407-426`, but are missing from `kubernetes/infrastructure/cilium/helmrelease.yaml` (which has no BGP values). This is a real gap that the proposal flags as "untracked settings." 2. **The three-copy problem is real** — The `bpf-policy-map-max` setting appears in at least three locations: - `helmrelease.yaml:83` - `cilium.yaml:333` - Mentioned in cross-referenced comments across these files This duplication creates the maintenance burden the proposal describes. 3. **Ownership statement is clear** — The rule ("Cilium is owned by Flux") is explicit, and the rejection of the Talos-only alternative is justified by the cost to Flux's reconciliation and observability. 4. **Transition plan is precise** — Steps 1–6 are concrete and ordered correctly; particularly, tracking untracked settings first (Step 1) before removing the manifest (Step 2) avoids the deletion risk described. 5. **Routing mode discussion is honest** — The document correctly explains why VXLAN + BGP announcement is not "one layer too many" — they serve different directions (node-to-node encapsulation vs. UDM reachability). Acknowledging that `routingMode` is unset (rather than a deliberate choice) shows good documentation practice. ## 🟡 Minor Notes 1. **Transition verification step clarity** — Step 3 mentions proving the prune with `upgrade-k8s --dry-run --manifests-no-prune`, but the document doesn't explicitly state the expected output (absence of Cilium objects in the prune list). Adding "Expect the 22 Cilium objects to NOT appear in the deletion list" would make the success criterion clearer. 2. **HelmRelease version constraint** — The document doesn't address whether the version constraint on the HelmRelease (`>=1.20.0 <1.21.0` at helmrelease.yaml:24) should be widened after transition, or whether that's intentional to require further review. This is a minor detail but worth noting if the transition plan is followed. 3. **Cold rebuild runbook reference** — The document references updating `RUNBOOK-cold-rebuild.md` (Step 6) but doesn't show the actual command to be documented. Including a sample would make this step fully actionable. ## Tests This is a documentation/proposal file, not code. No functional tests apply. The proposal will be validated by executing the transition steps. ## Recommendation Looks reasonable — no blocking issues; observations above are optional enhancements for future clarity. <!-- pr-review:sha=78904a2d62304ff2b5ad2ca4435d751987a0bba1 -->

k.renner merged commit c0ed09754d into master 2026-08-28 13:38:20 +00:00

k.renner referenced this issue from a commit 2026-08-28 13:38:21 +00:00

Merge pull request 'docs(cilium): decide the ownership question before changing anything' (#707) from proposal-cilium-single-owner into master

claude-bot referenced this pull request 2026-08-28 13:39:50 +00:00

feat(cilium): track the BGP switch and native-routing CIDRs #708

claude-bot referenced this pull request 2026-08-28 14:13:50 +00:00

refactor(cilium): Flux is the only owner #709

Sign in to join this conversation.

Reviewers

No Reviewers

review-bot

Labels

Clear labels

apollo

Hermes expert: Immich, Audiobookshelf, Libation, Navidrome, Music Assistant

atlas

Hermes expert: Talos, Flux, network, Authentik, Matrix, monitoring

clio

Hermes expert: Paperless, wiki, ical-proxy, TRMNL, websites

daedalus

Hermes expert: Gitea, Woodpecker, Renovate, review-bot, Hermes

hestia

Hermes expert: Home Assistant, IoT, MQTT, ESPHome, voice

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

alert-monitor-bot claude-bot (Claude Bot) flux hermes-bot issue-fix-bot k.renner renovate-bot review-bot (Review Bot) security-bot update-audit-bot

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#707