Compare commits

...
Author SHA1 Message Date
claude-botandClaude Fable 5.1 de1530789e ci(wiki-maintenance): run the audit as the vault's script, commit only vault content
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The audit agent had no executable path in CI (Bash limited to the two
retrieval tools), wrote itself scripts it could not run and reported one
of seven checks (runs 10262, 10362). The checks are mechanical, so
notes/bin/wiki_audit.py does them without a model. push-notes adds only
wiki/, raw/ and output/, so scratch files in the checkout root are no
longer committed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-23 18:05:41 +02:00
k.renner c92c1f4ab5 Merge pull request 'fix(k8s-gateway): pull the chart from the Codeberg OCI registry' (#924) from fix/k8s-gateway-oci-chart into master
Reviewed-on: k.renner/homelab#924
2026-09-23 16:01:24 +00:00
k.renner 6bb8ef4776 Merge pull request 'fix: [alert] OpenBaoQuorumLost (openbao) (#920)' (#921) from alert-fix/issue-920 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#921
2026-09-23 15:59:33 +00:00
claude-botandClaude Opus 5.5 33a0079a20 fix(k8s-gateway): pull the chart from the Codeberg OCI registry
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The project moved to Codeberg; GitHub is a read-only mirror and the old
chart index at k8s-gateway.github.io returns 404, so the HelmRelease has
had no artifact. The chart is now published as an OCI artifact in the
project's own Codeberg registry (oci://codeberg.org/k8s-gateway/charts/
k8s-gateway), alongside the image.

OCIRepository + chartRef as for spegel and mariadb-operator; semver
range unchanged. Chart 3.7.2 -> 3.7.3, image moves to
codeberg.org/k8s-gateway/k8s_gateway:1.8.3. The rendered Corefile is
identical to the live one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 17:57:20 +02:00
k.renner 80816d7433 Merge pull request 'docs(network): traffic classes as built, node-to-node on VLAN 46 (#576)' (#923) from docs/network-traffic-classes-as-built into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#923
2026-09-23 12:49:20 +00:00
claude-botandClaude Opus 5.5 32a995a6bf docs(network): traffic classes as built, node-to-node on VLAN 46 (#576)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
NETWORK-traffic-classes.md described table 48 and rules 980/981/990/991,
which #857 replaced with one table (47) decided by longest prefix, and
listed management as not started. Rewritten to the live state: five
classes, table 47 contents owned by bird, node IP and etcd on node0, the
etcd leave-and-rejoin procedure, and the mixed-state hazard hit during
the node IP move. The bird DaemonSet comment still named table 48.

Closes #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 14:48:12 +02:00
k.renner 8a0ed135f8 Merge pull request 'feat(talos): node IP and etcd on VLAN 46 (#576 step 3)' (#922) from feat/node-ip-vlan-46 into master
Reviewed-on: k.renner/homelab#922
2026-09-23 12:47:06 +00:00
claude-botandClaude Opus 5.5 be643533f6 feat(talos): node IP and etcd on VLAN 46 (#576 step 3)
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Records the live state after moving all three nodes:
- site-homelab.yaml: kubelet nodeIP.validSubnets fd02:46::/64 and
  10.2.46.0/24 (was 10.2.40.0/24 and the legacy fd40::/64).
- cluster.yaml: etcd advertisedSubnets 10.2.46.0/24.
- mgmt0 comments: VLAN 40 is world-to-node only now.

Applied per node (k8s-3, k8s-2, k8s-1): drain, stage the config (delete
the old lists, set the new ones, one staged patch call), etcd
forfeit-leadership and leave, reboot. Each node rejoined etcd with its
VLAN 46 peer URL; EPHEMERAL was not wiped.

Refs #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 14:43:17 +02:00
issue-fix-bot 87083c46e7 fix: [alert] OpenBaoQuorumLost (openbao) (#920)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
2026-09-23 12:42:58 +00:00
k.renner 311b4639a6 Merge pull request 'feat(bird): put node0 in the pod routing table (#576 step 2)' (#919) from feat/bird-attached-node0 into master
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#919
2026-09-23 11:31:32 +00:00
k.renner a95efdef33 Merge pull request 'feat(talos): node network on VLAN 46, etcd pinned to VLAN 40 (#576 step 1)' (#918) from feat/node-vlan-46-link into master
Reviewed-on: k.renner/homelab#918
2026-09-23 11:31:23 +00:00
claude-botandClaude Opus 5.5 07dd6097ec feat(talos): node network on VLAN 46, etcd pinned to VLAN 40 (#576 step 1)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
- node0: VLANConfig on the uplink trunk, VLAN 46, static fd02:46::1x/64
  and 10.2.46.1x/24, accept_ra 0. Closed like pod0. Unused until the node
  IP moves there (step 3).
- cluster.etcd.advertisedSubnets pinned to 10.2.40.0/24 so a reboot
  cannot pick a node0 address: Talos never updates an existing member's
  peer URLs.

Applied live with talosctl patch --mode=no-reboot, k8s-3 first. Node IPs
unchanged, etcd 3/3 healthy, node0 reachable node to node over v6 and v4.

Refs #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 13:25:12 +02:00
12 changed files with 161 additions and 159 deletions
+12 -33
View File
@@ -1,12 +1,11 @@
# Nightly wiki maintenance for the Notes vault (k.renner/notes), replacing the
# workstation's wiki-maintenance.timer: distill the raw/ backlog into wiki/,
# then audit wiki/. Each agent is its own `claude` run, not a subagent of an
# orchestrator, so nothing depends on background tasks in print mode (the
# workstation job died on their 600 s ceiling). The pipeline commits and pushes
# as Claude Bot; the push webhook then refreshes the wiki-index service.
# workstation's wiki-maintenance.timer: distill the raw/ backlog into wiki/
# (a `claude` run of the wiki-distill agent), then audit wiki/ with the vault's
# own script (structural checks, no model). The pipeline commits and pushes as
# Claude Bot; the push webhook then refreshes the wiki-index service.
#
# The agents and their retrieval scripts address the vault as ~/Notes, so each
# agent step links $HOME/Notes to the checkout instead of rewriting them.
# The agent and its retrieval scripts address the vault as ~/Notes, so the
# distill step links $HOME/Notes to the checkout instead of rewriting them.
when:
- event: cron
cron: wiki-maintenance
@@ -64,31 +63,9 @@ steps:
- name: audit
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
pull: true
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
TYPESAFE_API_KEY:
from_secret: typesafe_api_key
WIKI_INDEX_BASIC_AUTH:
from_secret: wiki_index_basic_auth
commands:
- |
set -eu
ln -sfn "$$PWD/notes" "$$HOME/Notes"
mkdir -p "$$HOME/.config/wiki-index"
printf '%s' "$$WIKI_INDEX_BASIC_AUTH" | tr -d '\r\n' > "$$HOME/.config/wiki-index/basic-auth"
chmod 600 "$$HOME/.config/wiki-index/basic-auth"
- |
set -eu
MODEL=$$(sh scripts/ci/pick-model.sh "structural audit of ~1700 markdown notes: frontmatter completeness, broken links, orphans; mechanical checks, little reasoning")
if [ -n "$$MODEL" ]; then echo "audit: selected model $$MODEL"; else echo "audit: using the CLI default model"; fi
cd notes
cat ../ci/agents/wiki-maintenance/audit.md | claude $${MODEL:+--model $$MODEL} \
--agent wiki-audit \
--permission-mode acceptEdits \
--allowedTools "Read,Write,Glob,Grep,Bash(/root/Notes/bin/wiki_rank.py:*),Bash(/root/Notes/bin/wiki_ask.py:*)" \
--settings ../ci/agents/wiki-maintenance/settings.json \
-p
# Findings are report lines in output/, not a pipeline failure.
- python3 -B notes/bin/wiki_audit.py --vault "$$PWD/notes"
- name: push-notes
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
@@ -102,7 +79,9 @@ steps:
cd notes
git config user.email "claude-bot@renner.uno"
git config user.name "Claude Bot"
if git diff --quiet && git diff --cached --quiet && [ -z "$$(git ls-files --others --exclude-standard)" ]; then
# Only the vault content is committed; anything an agent leaves elsewhere
# in the checkout (run 10362: scratch scripts in the root) stays behind.
if [ -z "$$(git status --porcelain -- wiki raw output)" ]; then
echo "wiki-maintenance: no changes"
exit 0
fi
@@ -112,7 +91,7 @@ steps:
# leaves the PR open for a human and fails the run.
BRANCH="wiki-maintenance/$$(date -u +%F)-$$CI_PIPELINE_NUMBER"
git checkout -q -b "$$BRANCH"
git add -A
git add -A -- wiki raw output
git commit -q -m "wiki maintenance $$(date -u +%F): distill + audit (CI run $$CI_PIPELINE_NUMBER)"
TOKEN=$$(printf '%s' "$$WIKI_BOT_TOKEN" | tr -d '\r\n')
git -c http.extraheader="AUTHORIZATION: token $$TOKEN" push origin "HEAD:$$BRANCH"
-5
View File
@@ -1,5 +0,0 @@
You are running as the scheduled wiki-maintenance job in CI. The Notes vault is checked out at ~/Notes (the repository k.renner/notes).
Run the standard audit pass over ~/Notes/wiki/ and write the dated report to ~/Notes/output/. Do not modify any wiki note. Do not commit or push; the pipeline does that.
End with two lines: notes scanned, and issues found by category.
+1 -1
View File
@@ -1,5 +1,5 @@
{
"//": "Permission scope for the wiki-maintenance agents (wiki-distill, wiki-audit) in CI.",
"//": "Permission scope for the wiki-maintenance agent (wiki-distill) in CI.",
"//2": "The step's environment holds a Gitea push token, a TypeSafe key and the wiki-index",
"//3": "client credential, so the Read tool is kept away from /proc (env dumps), /root outside",
"//4": "the vault symlink target, /etc and the checkout's .git/. The vault itself is the work.",
+73 -91
View File
@@ -1,117 +1,99 @@
# Traffic classes on the k8s nodes
Issue #576. Three of the four classes are separated; management is not.
Issue #576. Five classes, each on its own VLAN.
## Where each class runs
| class | link | mechanism |
|---|---|---|
| Ceph replication | `ceph-san`, VLAN 49 | the whole link is handed to the OSD pod (Multus `host-device`) |
| world→pod | `ingress0`, VLAN 47 | BGP session sourced there, so the advertised next hop is that address |
| pod→world | `ingress0`, VLAN 47 | routing rule on the pod source prefix → table 47 |
| pod↔pod | `pod0`, VLAN 48 | routing rule on pod source **and** destination → table 48 |
| management | `mgmt0`, VLAN 40 | not started |
| pod↔pod | `pod0`, VLAN 48 | pod-sourced traffic is routed from table 47; bird puts the peers' pod CIDRs there via `pod0` |
| world↔pod | `ingress0`, VLAN 47 | BGP session sourced there (next hop of every LoadBalancer VIP); pod egress takes table 47's default |
| node↔node | `node0`, VLAN 46 | the node InternalIP and etcd live there; so do apiserver↔kubelet, cilium-health and host→remote-pod traffic |
| world↔node | `mgmt0`, VLAN 40 | API VIP for kubectl, talosctl from outside, node egress (DNS, NTP, image pulls) |
Hard constraint, set by the owner: inter-node and pod traffic never traverses
the UDM. Everything above stays on the 10G switch.
the UDM. Everything but world↔node stays on the 10G switch. VLAN 46 and 48 are
statically addressed, without a gateway; VLAN 46 is isolated in UniFi.
## Why policy routing rather than moving the node IP
## Pod traffic: one table
Cilium reaches a peer's pods at that peer's Kubernetes InternalIP, in native
and tunnel mode alike. `--ipv4-node` is overwritten from the Node object, and
`nodePort.directRoutingDevice` selects the BPF NodePort device, not a next hop.
Both were established by test (#837, #838), not by reading.
Two rules, one per family, send every packet with a pod source to table 47
(`talos/patches/site-homelab.yaml`, priorities 1000/1001, below main). The
table decides by longest prefix; no rule carves out exceptions.
So the InternalIP decides where pod traffic goes — unless the lookup happens in
a different table. The rules do that, and nothing about the node's identity
moves. The routing decision runs before Cilium's masquerade hook, so a rule
matching the pod source sees the pod address, and the masquerade then rewrites
it to the egress link's.
bird (`kubernetes/infrastructure/bird/`) owns table 47 and fills it with:
## Rule order
- the node's attached networks — `mgmt0`, `ingress0`, `pod0`, `node0` —
taken from the interfaces;
- the peers' pod CIDRs, learned over an iBGP mesh on `pod0`, next hop the
peer's `pod0` address;
- the node's own pod CIDR via `cilium_host`, originated rather than read back
from the kernel (reading it back made BIRD depend on the shape of Cilium's
route, and KRT rejects the IPv4 one);
- the egress default via the UDM on `ingress0`, from the node annotations
`net.renner.uno/egress-gw-v4/v6`.
Lower priority runs first.
`node0` has to be in the table: a pod reaching a node address — the
`kubernetes` ClusterIP, which socket LB rewrites to the apiserver's advertise
address, or a hostNetwork pod — would otherwise take the default to the UDM.
| prio | match | table |
|---|---|---|
| 980 | src pod, dst pod | 48 |
| 981 | src pod, dst pod (v4) | 48 |
| 990 | dst `fd00::/8` | main |
| 991 | dst `10.0.0.0/8` | main |
| 1000 | src pod | 47 |
| 1001 | src pod (v4) | 47 |
The routing decision runs before Cilium's masquerade hook, so the rule sees the
pod address and the masquerade then rewrites it to the egress link's.
980/981 match source **and** destination. Source alone would also catch the
node's own traffic to a pod, which would then leave with a `pod0` source — an
address `CiliumNode.spec.addresses` does not list, so Cilium classifies it as
`world` rather than `host`.
990/991 are Cilium's `ipv4/ipv6NativeRoutingCIDR`, and they are what keeps
cluster-internal traffic out of table 47. Without them a pod's packet to a
service matches the source rule, finds only a default route, and leaves for the
UDM — measured, in-cluster DNS stopped resolving.
Traffic to a pod on the same node falls through on its own: table 48 holds only
the peers' CIDRs, so the lookup misses and 990/991 send it to main.
## Where each table comes from
- **47** — two default routes declared per node in `talos/patches/k8s-*.yaml`.
The IPv6 gateway is the UDM's configured address on the segment, not the
RA-learned link-local; `accept_ra_defrtr` stays `0` on `ingress0` so the RA
default does not land in main.
- **48** — filled by the `bird` DaemonSet: an iBGP mesh over `pod0`, one route
per peer pod CIDR with the peer's `pod0` address as next hop.
## The bird mesh
`kubernetes/infrastructure/bird/`. Each node advertises only its own pod CIDR
and installs what it learns into table 48. Cilium's routes in main are
untouched, which is why the two coexist.
Node-originated traffic does not match the rules and uses main, where Cilium
routes each peer's pod CIDR via the peer's InternalIP — on `node0`.
No address is written into this repo twice. Each node publishes its `pod0`
addresses as `nodeAnnotations`, beside the `VLANConfig` that assigns them; the
pod CIDRs come from the Node objects. An init container renders both from the
API with `kubectl -o go-template-file`, excludes the node itself with a field
selector, and ends with `bird -p` so BIRD's own parser gates startup.
addresses as `nodeAnnotations` beside the `VLANConfig` that assigns them; the
pod CIDRs come from the Node objects. bird's init container renders both from
the API and ends with `bird -p`, so BIRD's own parser gates startup.
The node's own CIDR is **originated**, not read back from the kernel. Reading
it back made BIRD depend on the shape Cilium installs, and the IPv4 route is
rejected outright: its next hop sits inside the prefix it describes, which KRT
refuses with `strange next-hop`. The IPv6 route is a device route and passed,
so the failure was silent on one family only.
## Node↔node
`kubelet.nodeIP.validSubnets` and `cluster.etcd.advertisedSubnets` point at
VLAN 46. Talos picks one InternalIP per family, IPv4 first — the service CIDR
is IPv4, and the apiserver's advertise address must match it.
The API VIP stays on `mgmt0` for kubectl. No in-cluster client uses it:
kubelet, scheduler, controller-manager and Cilium go through KubePrism
(`localhost:7445`), which balances over every address of every control-plane
member; pods use the ClusterIP.
Talos never updates an existing etcd member's peer URLs
(siderolabs/talos#12720). Changing `advertisedSubnets` means, per member:
stage the config, `talosctl etcd forfeit-leadership`, `talosctl etcd leave`,
reboot. `leave` removes only `/var/lib/etcd`; the node rejoins as a learner
and is promoted. EPHEMERAL does not need a wipe.
## Moving node IPs: the mixed state
While some nodes have their InternalIP on the new network and some on the old,
host→remote-pod traffic leaves with a node address that is not an InternalIP:
the kernel picks the source from the route to the peer's InternalIP. Cilium
classifies that as `world`, and every ingress policy that admits only
`host`/`remote-node` (rook-ceph, monitoring, …) drops it. Kernel Ceph clients
lose the OSDs on the other side; RBD mounts hang.
Bridge it for the duration with a temporary `CiliumClusterwideNetworkPolicy`
admitting the node subnets of both networks, both families, with
`enableDefaultDeny: false` so it restricts nothing else, and delete it once
every node has moved. Keep all OSD nodes schedulable while one is drained:
draining two takes Ceph below its data availability.
## Failure behaviour
Benign at every layer. An empty or absent table 48 makes 980/981 fall through
to main, and pod traffic returns to VLAN 40 — unseparated, not broken. The same
holds if the DaemonSet cannot start, which is the case during a cold rebuild
until Gitea serves its image.
## Management, the remaining class
Not started. The levers differ per service and none of them is the node IP:
| service | can leave the node IP? | lever |
|---|---|---|
| etcd | yes | `cluster.etcd.advertisedSubnets`; needs a reboot per node |
| kube-apiserver endpoint | yes | it is a VIP — `Layer2VIPConfig.link` |
| Talos API | n/a | `apid` answers on every node address; a firewall question |
| kubelet | no | it *is* the node IP |
Two approaches were rejected by the owner and are not to be proposed again:
moving the node IP and etcd to `pod0` (#838), and relabelling VLAN 40 as the
cluster VLAN while moving etcd and the API VIP to a new management VLAN.
Replacing the CNI (#839, Antrea) was shelved.
Note that the reasoning which produced that list also concluded pod↔pod was
blocked, which policy routing disproved. Treat it as a record of decisions, not
as an exhaustive analysis.
If bird stops, table 47 empties, the rules find nothing and fall through to
main: pod traffic returns to the node networks, unseparated rather than
misrouted. The same holds during a cold rebuild until Gitea serves bird's
image. The default route lives in table 47, not in the Talos config, so a
default cannot outlive the specific routes and send pod traffic to the UDM.
## Sources
Cilium v1.20.1 `pkg/datapath/linux/node.go`, `pkg/node/types/node.go`,
`pkg/node/sync/local_node_sync.go`; BIRD 2.17 user manual (kernel protocol,
`learn all`, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig` and
`machine.nodeAnnotations`. PRs #831, #833, #835, #836, #837, #842, #852, #854,
#855; closed unmerged: #830, #838, #839.
Cilium v1.20 `pkg/datapath/linux/node.go`; BIRD 2.17 user manual (kernel
protocol, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig`,
`machine.nodeAnnotations`, configuration patching (lists are appended,
`$patch: delete`), etcd `LeaveCluster`; KubePrism docs;
siderolabs/talos#12720. PRs #831, #833, #835, #836, #837, #842, #852, #854,
#855, #857, #858, #918, #919, #922; closed unmerged: #830, #838, #839.
@@ -57,10 +57,14 @@ spec:
# the ClusterSecretStore goes InvalidProviderConfig and no
# ExternalSecret can refresh.
#
# No window: below quorum the secret store is down and every
# ExternalSecret stalls with it, so waiting only delays the page.
# Short window: readiness is only a proxy for quorum. A one-second
# node NotReady flap makes the node lifecycle controller mark that
# node's pods not-ready for ~30s while raft keeps its leader and
# serves (issue #920, during a rolling reboot with one replica
# already evicted). A really sealed or crashed majority persists,
# and existing Secrets keep their last values, so 2m costs little.
expr: max by (namespace, statefulset) (kube_statefulset_status_replicas_ready{namespace="openbao", statefulset="openbao"}) < 2
for: 0m
for: 2m
labels:
severity: critical
annotations:
@@ -48,8 +48,8 @@ spec:
tier: node
app: bird
spec:
# Carries pod east-west once the routing rules land. Without it table 48
# stays empty and the rules fall through to main, i.e. back to VLAN 40.
# Owns table 47, where all pod traffic is routed. Without it the table
# stays empty and the rules fall through to main, unseparated.
priorityClassName: system-node-critical
# Writes host routes and peers on host addresses.
hostNetwork: true
@@ -1,11 +1,16 @@
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
kind: OCIRepository
metadata:
name: k8s-gateway
namespace: kube-system
spec:
interval: 24h
url: https://k8s-gateway.github.io/k8s_gateway/
url: oci://codeberg.org/k8s-gateway/charts/k8s-gateway
ref:
semver: ">=2.4.0"
layerSelector:
mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
operation: copy
---
# In-cluster authoritative resolver for renner.uno.
#
@@ -28,14 +33,10 @@ metadata:
namespace: kube-system
spec:
interval: 30m
chart:
spec:
chart: k8s-gateway
version: ">=2.4.0"
sourceRef:
kind: HelmRepository
name: k8s-gateway
namespace: kube-system
chartRef:
kind: OCIRepository
name: k8s-gateway
namespace: kube-system
install:
remediation:
retries: 3
+6
View File
@@ -58,6 +58,12 @@ machine:
imageGCHighThresholdPercent: 70
imageGCLowThresholdPercent: 60
cluster:
# etcd peers on node0, VLAN 46 (#576). Pinned: Talos never updates an
# existing member's peer URLs, so changing this subnet means leave + rejoin
# per member (talosctl etcd leave, reboot).
etcd:
advertisedSubnets:
- 10.2.46.0/24
# controlPlane.endpoint is NOT here: `talosctl gen config` takes it as its
# second argument, and bootstrap.sh already passes https://k8s.renner.uno:6443
# there. Setting it again in this patch restated the same value in a second
+17 -2
View File
@@ -17,6 +17,8 @@ machine:
# pod0 is a closed L2 fabric between the three nodes: statically
# addressed, no gateway, nothing to learn from an RA.
net.ipv6.conf.pod0.accept_ra: "0"
# node0 is closed the same way.
net.ipv6.conf.node0.accept_ra: "0"
# ingress0 keeps RAs, at the default accept_ra of 2: an IPv6 default
# router is a link-local address learned only from the RA, and with
# net.ipv6.conf.all.forwarding=1 — which Cilium sets — Linux ignores RAs
@@ -122,8 +124,8 @@ kind: LinkConfig
name: uplink
up: true
---
# Management, VLAN 40. Carries the node IP, etcd, the API and Ceph's public
# network.
# Management, VLAN 40: world↔node. The API VIP, admin access to the nodes, and
# node egress (DNS, NTP, image pulls). Node↔node traffic is on node0 (#576).
apiVersion: v1alpha1
kind: VLANConfig
name: mgmt0
@@ -177,6 +179,19 @@ addresses:
- address: 10.2.48.11/24
- address: fd02:48::11/64
---
# Node network, VLAN 46: traffic among the three nodes themselves (#576) — the
# node IP, etcd, apiserver↔kubelet. Closed like pod0: statically addressed, no
# gateway, never routed via the UDM.
apiVersion: v1alpha1
kind: VLANConfig
name: node0
vlanID: 46
parent: uplink
up: true
addresses:
- address: fd02:46::11/64
- address: 10.2.46.11/24
---
# Ingress network, VLAN 47. World-to-pod: the BGP session, the next hop for
# every advertised LoadBalancer VIP, and the masquerade source for pod egress.
# Publicly routed, which is what separates it from pod0.
+13 -1
View File
@@ -16,6 +16,7 @@ machine:
# ingress0 keeps RAs: it learns its IPv6 router and its address from them,
# and only the default route is held back — see k8s-1.yaml.
net.ipv6.conf.pod0.accept_ra: "0"
net.ipv6.conf.node0.accept_ra: "0"
net.ipv6.conf.ingress0.accept_ra_defrtr: "0"
# Answer ARP only on the link that owns the address — see k8s-1.yaml (#793).
net.ipv4.conf.all.arp_ignore: "1"
@@ -82,7 +83,7 @@ kind: LinkConfig
name: uplink
up: true
---
# Management, VLAN 40. Node IP, etcd, the API and Ceph's public network.
# Management, VLAN 40: world↔node — see k8s-1.yaml.
apiVersion: v1alpha1
kind: VLANConfig
name: mgmt0
@@ -122,6 +123,17 @@ addresses:
- address: 10.2.48.12/24
- address: fd02:48::12/64
---
# Node network, VLAN 46 — see k8s-1.yaml.
apiVersion: v1alpha1
kind: VLANConfig
name: node0
vlanID: 46
parent: uplink
up: true
addresses:
- address: fd02:46::12/64
- address: 10.2.46.12/24
---
# Ingress network, VLAN 47. World-to-pod, publicly routed, statically
# addressed — see k8s-1.yaml for why it is not leased.
apiVersion: v1alpha1
+12
View File
@@ -12,6 +12,7 @@ machine:
# ingress0 keeps RAs: it learns its IPv6 router and its address from them,
# and only the default route is held back — see k8s-1.yaml.
net.ipv6.conf.pod0.accept_ra: "0"
net.ipv6.conf.node0.accept_ra: "0"
net.ipv6.conf.ingress0.accept_ra_defrtr: "0"
# Answer ARP only on the link that owns the address — see k8s-1.yaml (#793).
net.ipv4.conf.all.arp_ignore: "1"
@@ -109,6 +110,17 @@ addresses:
- address: 10.2.48.13/24
- address: fd02:48::13/64
---
# Node network, VLAN 46 — see k8s-1.yaml.
apiVersion: v1alpha1
kind: VLANConfig
name: node0
vlanID: 46
parent: uplink
up: true
addresses:
- address: fd02:46::13/64
- address: 10.2.46.13/24
---
# Ingress network, VLAN 47. World-to-pod, publicly routed, statically
# addressed — see k8s-1.yaml for why it is not leased.
apiVersion: v1alpha1
+7 -11
View File
@@ -7,19 +7,15 @@
# part of the config that could not be true anywhere but here.
machine:
kubelet:
# Constrain kubelet's auto-picked node IP to the mgmt LAN. Without this,
# kubelet picks the first non-loopback IP, which could land on the v30iot
# NIC (ens19, 10.2.30.0/24). Control-plane traffic and hostNetwork pod IPs
# must stay on mgmt LAN.
# fd40::/64 is the mgmt LAN ULA, and it is listed so the Node object carries
# an IPv6 InternalIP at all. Without one, Cilium falls back to detecting the
# node's IPv6 from mgmt0 and picks the SLAAC address from the dynamic
# provider prefix, which it then probes until the agent restarts (#789).
# The GUA is not a control-plane address: BGP peers on fd40::/64 too.
# The node IP lives on node0, VLAN 46: the closed node↔node network (#576).
# Both families are listed so the Node object carries an IPv6 InternalIP at
# all; without one Cilium detects the node's IPv6 itself and picks a SLAAC
# address from the dynamic provider prefix (#789). Talos picks one address
# per family, IPv4 first, whatever the order here.
nodeIP:
validSubnets:
- 10.2.40.0/24
- fd40::/64
- fd02:46::/64
- 10.2.46.0/24
# The UDM on the ingress segment. Published here rather than as a route
# because bird owns table 47 and originates the default from it; this is the
# one place the address is stated.