Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
de1530789e | ||
|
|
c92c1f4ab5 | ||
|
|
6bb8ef4776 | ||
|
|
33a0079a20 | ||
|
|
80816d7433 | ||
|
|
32a995a6bf | ||
|
|
8a0ed135f8 | ||
|
|
be643533f6 | ||
|
|
87083c46e7 | ||
|
|
311b4639a6 | ||
|
|
a95efdef33 | ||
|
|
07dd6097ec |
@@ -1,12 +1,11 @@
|
||||
# Nightly wiki maintenance for the Notes vault (k.renner/notes), replacing the
|
||||
# workstation's wiki-maintenance.timer: distill the raw/ backlog into wiki/,
|
||||
# then audit wiki/. Each agent is its own `claude` run, not a subagent of an
|
||||
# orchestrator, so nothing depends on background tasks in print mode (the
|
||||
# workstation job died on their 600 s ceiling). The pipeline commits and pushes
|
||||
# as Claude Bot; the push webhook then refreshes the wiki-index service.
|
||||
# workstation's wiki-maintenance.timer: distill the raw/ backlog into wiki/
|
||||
# (a `claude` run of the wiki-distill agent), then audit wiki/ with the vault's
|
||||
# own script (structural checks, no model). The pipeline commits and pushes as
|
||||
# Claude Bot; the push webhook then refreshes the wiki-index service.
|
||||
#
|
||||
# The agents and their retrieval scripts address the vault as ~/Notes, so each
|
||||
# agent step links $HOME/Notes to the checkout instead of rewriting them.
|
||||
# The agent and its retrieval scripts address the vault as ~/Notes, so the
|
||||
# distill step links $HOME/Notes to the checkout instead of rewriting them.
|
||||
when:
|
||||
- event: cron
|
||||
cron: wiki-maintenance
|
||||
@@ -64,31 +63,9 @@ steps:
|
||||
- name: audit
|
||||
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
|
||||
pull: true
|
||||
environment:
|
||||
CLAUDE_CODE_OAUTH_TOKEN:
|
||||
from_secret: claude_code_oauth_token
|
||||
TYPESAFE_API_KEY:
|
||||
from_secret: typesafe_api_key
|
||||
WIKI_INDEX_BASIC_AUTH:
|
||||
from_secret: wiki_index_basic_auth
|
||||
commands:
|
||||
- |
|
||||
set -eu
|
||||
ln -sfn "$$PWD/notes" "$$HOME/Notes"
|
||||
mkdir -p "$$HOME/.config/wiki-index"
|
||||
printf '%s' "$$WIKI_INDEX_BASIC_AUTH" | tr -d '\r\n' > "$$HOME/.config/wiki-index/basic-auth"
|
||||
chmod 600 "$$HOME/.config/wiki-index/basic-auth"
|
||||
- |
|
||||
set -eu
|
||||
MODEL=$$(sh scripts/ci/pick-model.sh "structural audit of ~1700 markdown notes: frontmatter completeness, broken links, orphans; mechanical checks, little reasoning")
|
||||
if [ -n "$$MODEL" ]; then echo "audit: selected model $$MODEL"; else echo "audit: using the CLI default model"; fi
|
||||
cd notes
|
||||
cat ../ci/agents/wiki-maintenance/audit.md | claude $${MODEL:+--model $$MODEL} \
|
||||
--agent wiki-audit \
|
||||
--permission-mode acceptEdits \
|
||||
--allowedTools "Read,Write,Glob,Grep,Bash(/root/Notes/bin/wiki_rank.py:*),Bash(/root/Notes/bin/wiki_ask.py:*)" \
|
||||
--settings ../ci/agents/wiki-maintenance/settings.json \
|
||||
-p
|
||||
# Findings are report lines in output/, not a pipeline failure.
|
||||
- python3 -B notes/bin/wiki_audit.py --vault "$$PWD/notes"
|
||||
|
||||
- name: push-notes
|
||||
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
|
||||
@@ -102,7 +79,9 @@ steps:
|
||||
cd notes
|
||||
git config user.email "claude-bot@renner.uno"
|
||||
git config user.name "Claude Bot"
|
||||
if git diff --quiet && git diff --cached --quiet && [ -z "$$(git ls-files --others --exclude-standard)" ]; then
|
||||
# Only the vault content is committed; anything an agent leaves elsewhere
|
||||
# in the checkout (run 10362: scratch scripts in the root) stays behind.
|
||||
if [ -z "$$(git status --porcelain -- wiki raw output)" ]; then
|
||||
echo "wiki-maintenance: no changes"
|
||||
exit 0
|
||||
fi
|
||||
@@ -112,7 +91,7 @@ steps:
|
||||
# leaves the PR open for a human and fails the run.
|
||||
BRANCH="wiki-maintenance/$$(date -u +%F)-$$CI_PIPELINE_NUMBER"
|
||||
git checkout -q -b "$$BRANCH"
|
||||
git add -A
|
||||
git add -A -- wiki raw output
|
||||
git commit -q -m "wiki maintenance $$(date -u +%F): distill + audit (CI run $$CI_PIPELINE_NUMBER)"
|
||||
TOKEN=$$(printf '%s' "$$WIKI_BOT_TOKEN" | tr -d '\r\n')
|
||||
git -c http.extraheader="AUTHORIZATION: token $$TOKEN" push origin "HEAD:$$BRANCH"
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
You are running as the scheduled wiki-maintenance job in CI. The Notes vault is checked out at ~/Notes (the repository k.renner/notes).
|
||||
|
||||
Run the standard audit pass over ~/Notes/wiki/ and write the dated report to ~/Notes/output/. Do not modify any wiki note. Do not commit or push; the pipeline does that.
|
||||
|
||||
End with two lines: notes scanned, and issues found by category.
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"//": "Permission scope for the wiki-maintenance agents (wiki-distill, wiki-audit) in CI.",
|
||||
"//": "Permission scope for the wiki-maintenance agent (wiki-distill) in CI.",
|
||||
"//2": "The step's environment holds a Gitea push token, a TypeSafe key and the wiki-index",
|
||||
"//3": "client credential, so the Read tool is kept away from /proc (env dumps), /root outside",
|
||||
"//4": "the vault symlink target, /etc and the checkout's .git/. The vault itself is the work.",
|
||||
|
||||
@@ -1,117 +1,99 @@
|
||||
# Traffic classes on the k8s nodes
|
||||
|
||||
Issue #576. Three of the four classes are separated; management is not.
|
||||
Issue #576. Five classes, each on its own VLAN.
|
||||
|
||||
## Where each class runs
|
||||
|
||||
| class | link | mechanism |
|
||||
|---|---|---|
|
||||
| Ceph replication | `ceph-san`, VLAN 49 | the whole link is handed to the OSD pod (Multus `host-device`) |
|
||||
| world→pod | `ingress0`, VLAN 47 | BGP session sourced there, so the advertised next hop is that address |
|
||||
| pod→world | `ingress0`, VLAN 47 | routing rule on the pod source prefix → table 47 |
|
||||
| pod↔pod | `pod0`, VLAN 48 | routing rule on pod source **and** destination → table 48 |
|
||||
| management | `mgmt0`, VLAN 40 | not started |
|
||||
| pod↔pod | `pod0`, VLAN 48 | pod-sourced traffic is routed from table 47; bird puts the peers' pod CIDRs there via `pod0` |
|
||||
| world↔pod | `ingress0`, VLAN 47 | BGP session sourced there (next hop of every LoadBalancer VIP); pod egress takes table 47's default |
|
||||
| node↔node | `node0`, VLAN 46 | the node InternalIP and etcd live there; so do apiserver↔kubelet, cilium-health and host→remote-pod traffic |
|
||||
| world↔node | `mgmt0`, VLAN 40 | API VIP for kubectl, talosctl from outside, node egress (DNS, NTP, image pulls) |
|
||||
|
||||
Hard constraint, set by the owner: inter-node and pod traffic never traverses
|
||||
the UDM. Everything above stays on the 10G switch.
|
||||
the UDM. Everything but world↔node stays on the 10G switch. VLAN 46 and 48 are
|
||||
statically addressed, without a gateway; VLAN 46 is isolated in UniFi.
|
||||
|
||||
## Why policy routing rather than moving the node IP
|
||||
## Pod traffic: one table
|
||||
|
||||
Cilium reaches a peer's pods at that peer's Kubernetes InternalIP, in native
|
||||
and tunnel mode alike. `--ipv4-node` is overwritten from the Node object, and
|
||||
`nodePort.directRoutingDevice` selects the BPF NodePort device, not a next hop.
|
||||
Both were established by test (#837, #838), not by reading.
|
||||
Two rules, one per family, send every packet with a pod source to table 47
|
||||
(`talos/patches/site-homelab.yaml`, priorities 1000/1001, below main). The
|
||||
table decides by longest prefix; no rule carves out exceptions.
|
||||
|
||||
So the InternalIP decides where pod traffic goes — unless the lookup happens in
|
||||
a different table. The rules do that, and nothing about the node's identity
|
||||
moves. The routing decision runs before Cilium's masquerade hook, so a rule
|
||||
matching the pod source sees the pod address, and the masquerade then rewrites
|
||||
it to the egress link's.
|
||||
bird (`kubernetes/infrastructure/bird/`) owns table 47 and fills it with:
|
||||
|
||||
## Rule order
|
||||
- the node's attached networks — `mgmt0`, `ingress0`, `pod0`, `node0` —
|
||||
taken from the interfaces;
|
||||
- the peers' pod CIDRs, learned over an iBGP mesh on `pod0`, next hop the
|
||||
peer's `pod0` address;
|
||||
- the node's own pod CIDR via `cilium_host`, originated rather than read back
|
||||
from the kernel (reading it back made BIRD depend on the shape of Cilium's
|
||||
route, and KRT rejects the IPv4 one);
|
||||
- the egress default via the UDM on `ingress0`, from the node annotations
|
||||
`net.renner.uno/egress-gw-v4/v6`.
|
||||
|
||||
Lower priority runs first.
|
||||
`node0` has to be in the table: a pod reaching a node address — the
|
||||
`kubernetes` ClusterIP, which socket LB rewrites to the apiserver's advertise
|
||||
address, or a hostNetwork pod — would otherwise take the default to the UDM.
|
||||
|
||||
| prio | match | table |
|
||||
|---|---|---|
|
||||
| 980 | src pod, dst pod | 48 |
|
||||
| 981 | src pod, dst pod (v4) | 48 |
|
||||
| 990 | dst `fd00::/8` | main |
|
||||
| 991 | dst `10.0.0.0/8` | main |
|
||||
| 1000 | src pod | 47 |
|
||||
| 1001 | src pod (v4) | 47 |
|
||||
The routing decision runs before Cilium's masquerade hook, so the rule sees the
|
||||
pod address and the masquerade then rewrites it to the egress link's.
|
||||
|
||||
980/981 match source **and** destination. Source alone would also catch the
|
||||
node's own traffic to a pod, which would then leave with a `pod0` source — an
|
||||
address `CiliumNode.spec.addresses` does not list, so Cilium classifies it as
|
||||
`world` rather than `host`.
|
||||
|
||||
990/991 are Cilium's `ipv4/ipv6NativeRoutingCIDR`, and they are what keeps
|
||||
cluster-internal traffic out of table 47. Without them a pod's packet to a
|
||||
service matches the source rule, finds only a default route, and leaves for the
|
||||
UDM — measured, in-cluster DNS stopped resolving.
|
||||
|
||||
Traffic to a pod on the same node falls through on its own: table 48 holds only
|
||||
the peers' CIDRs, so the lookup misses and 990/991 send it to main.
|
||||
|
||||
## Where each table comes from
|
||||
|
||||
- **47** — two default routes declared per node in `talos/patches/k8s-*.yaml`.
|
||||
The IPv6 gateway is the UDM's configured address on the segment, not the
|
||||
RA-learned link-local; `accept_ra_defrtr` stays `0` on `ingress0` so the RA
|
||||
default does not land in main.
|
||||
- **48** — filled by the `bird` DaemonSet: an iBGP mesh over `pod0`, one route
|
||||
per peer pod CIDR with the peer's `pod0` address as next hop.
|
||||
|
||||
## The bird mesh
|
||||
|
||||
`kubernetes/infrastructure/bird/`. Each node advertises only its own pod CIDR
|
||||
and installs what it learns into table 48. Cilium's routes in main are
|
||||
untouched, which is why the two coexist.
|
||||
Node-originated traffic does not match the rules and uses main, where Cilium
|
||||
routes each peer's pod CIDR via the peer's InternalIP — on `node0`.
|
||||
|
||||
No address is written into this repo twice. Each node publishes its `pod0`
|
||||
addresses as `nodeAnnotations`, beside the `VLANConfig` that assigns them; the
|
||||
pod CIDRs come from the Node objects. An init container renders both from the
|
||||
API with `kubectl -o go-template-file`, excludes the node itself with a field
|
||||
selector, and ends with `bird -p` so BIRD's own parser gates startup.
|
||||
addresses as `nodeAnnotations` beside the `VLANConfig` that assigns them; the
|
||||
pod CIDRs come from the Node objects. bird's init container renders both from
|
||||
the API and ends with `bird -p`, so BIRD's own parser gates startup.
|
||||
|
||||
The node's own CIDR is **originated**, not read back from the kernel. Reading
|
||||
it back made BIRD depend on the shape Cilium installs, and the IPv4 route is
|
||||
rejected outright: its next hop sits inside the prefix it describes, which KRT
|
||||
refuses with `strange next-hop`. The IPv6 route is a device route and passed,
|
||||
so the failure was silent on one family only.
|
||||
## Node↔node
|
||||
|
||||
`kubelet.nodeIP.validSubnets` and `cluster.etcd.advertisedSubnets` point at
|
||||
VLAN 46. Talos picks one InternalIP per family, IPv4 first — the service CIDR
|
||||
is IPv4, and the apiserver's advertise address must match it.
|
||||
|
||||
The API VIP stays on `mgmt0` for kubectl. No in-cluster client uses it:
|
||||
kubelet, scheduler, controller-manager and Cilium go through KubePrism
|
||||
(`localhost:7445`), which balances over every address of every control-plane
|
||||
member; pods use the ClusterIP.
|
||||
|
||||
Talos never updates an existing etcd member's peer URLs
|
||||
(siderolabs/talos#12720). Changing `advertisedSubnets` means, per member:
|
||||
stage the config, `talosctl etcd forfeit-leadership`, `talosctl etcd leave`,
|
||||
reboot. `leave` removes only `/var/lib/etcd`; the node rejoins as a learner
|
||||
and is promoted. EPHEMERAL does not need a wipe.
|
||||
|
||||
## Moving node IPs: the mixed state
|
||||
|
||||
While some nodes have their InternalIP on the new network and some on the old,
|
||||
host→remote-pod traffic leaves with a node address that is not an InternalIP:
|
||||
the kernel picks the source from the route to the peer's InternalIP. Cilium
|
||||
classifies that as `world`, and every ingress policy that admits only
|
||||
`host`/`remote-node` (rook-ceph, monitoring, …) drops it. Kernel Ceph clients
|
||||
lose the OSDs on the other side; RBD mounts hang.
|
||||
|
||||
Bridge it for the duration with a temporary `CiliumClusterwideNetworkPolicy`
|
||||
admitting the node subnets of both networks, both families, with
|
||||
`enableDefaultDeny: false` so it restricts nothing else, and delete it once
|
||||
every node has moved. Keep all OSD nodes schedulable while one is drained:
|
||||
draining two takes Ceph below its data availability.
|
||||
|
||||
## Failure behaviour
|
||||
|
||||
Benign at every layer. An empty or absent table 48 makes 980/981 fall through
|
||||
to main, and pod traffic returns to VLAN 40 — unseparated, not broken. The same
|
||||
holds if the DaemonSet cannot start, which is the case during a cold rebuild
|
||||
until Gitea serves its image.
|
||||
|
||||
## Management, the remaining class
|
||||
|
||||
Not started. The levers differ per service and none of them is the node IP:
|
||||
|
||||
| service | can leave the node IP? | lever |
|
||||
|---|---|---|
|
||||
| etcd | yes | `cluster.etcd.advertisedSubnets`; needs a reboot per node |
|
||||
| kube-apiserver endpoint | yes | it is a VIP — `Layer2VIPConfig.link` |
|
||||
| Talos API | n/a | `apid` answers on every node address; a firewall question |
|
||||
| kubelet | no | it *is* the node IP |
|
||||
|
||||
Two approaches were rejected by the owner and are not to be proposed again:
|
||||
moving the node IP and etcd to `pod0` (#838), and relabelling VLAN 40 as the
|
||||
cluster VLAN while moving etcd and the API VIP to a new management VLAN.
|
||||
Replacing the CNI (#839, Antrea) was shelved.
|
||||
|
||||
Note that the reasoning which produced that list also concluded pod↔pod was
|
||||
blocked, which policy routing disproved. Treat it as a record of decisions, not
|
||||
as an exhaustive analysis.
|
||||
If bird stops, table 47 empties, the rules find nothing and fall through to
|
||||
main: pod traffic returns to the node networks, unseparated rather than
|
||||
misrouted. The same holds during a cold rebuild until Gitea serves bird's
|
||||
image. The default route lives in table 47, not in the Talos config, so a
|
||||
default cannot outlive the specific routes and send pod traffic to the UDM.
|
||||
|
||||
## Sources
|
||||
|
||||
Cilium v1.20.1 `pkg/datapath/linux/node.go`, `pkg/node/types/node.go`,
|
||||
`pkg/node/sync/local_node_sync.go`; BIRD 2.17 user manual (kernel protocol,
|
||||
`learn all`, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig` and
|
||||
`machine.nodeAnnotations`. PRs #831, #833, #835, #836, #837, #842, #852, #854,
|
||||
#855; closed unmerged: #830, #838, #839.
|
||||
Cilium v1.20 `pkg/datapath/linux/node.go`; BIRD 2.17 user manual (kernel
|
||||
protocol, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig`,
|
||||
`machine.nodeAnnotations`, configuration patching (lists are appended,
|
||||
`$patch: delete`), etcd `LeaveCluster`; KubePrism docs;
|
||||
siderolabs/talos#12720. PRs #831, #833, #835, #836, #837, #842, #852, #854,
|
||||
#855, #857, #858, #918, #919, #922; closed unmerged: #830, #838, #839.
|
||||
|
||||
@@ -57,10 +57,14 @@ spec:
|
||||
# the ClusterSecretStore goes InvalidProviderConfig and no
|
||||
# ExternalSecret can refresh.
|
||||
#
|
||||
# No window: below quorum the secret store is down and every
|
||||
# ExternalSecret stalls with it, so waiting only delays the page.
|
||||
# Short window: readiness is only a proxy for quorum. A one-second
|
||||
# node NotReady flap makes the node lifecycle controller mark that
|
||||
# node's pods not-ready for ~30s while raft keeps its leader and
|
||||
# serves (issue #920, during a rolling reboot with one replica
|
||||
# already evicted). A really sealed or crashed majority persists,
|
||||
# and existing Secrets keep their last values, so 2m costs little.
|
||||
expr: max by (namespace, statefulset) (kube_statefulset_status_replicas_ready{namespace="openbao", statefulset="openbao"}) < 2
|
||||
for: 0m
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
|
||||
@@ -48,8 +48,8 @@ spec:
|
||||
tier: node
|
||||
app: bird
|
||||
spec:
|
||||
# Carries pod east-west once the routing rules land. Without it table 48
|
||||
# stays empty and the rules fall through to main, i.e. back to VLAN 40.
|
||||
# Owns table 47, where all pod traffic is routed. Without it the table
|
||||
# stays empty and the rules fall through to main, unseparated.
|
||||
priorityClassName: system-node-critical
|
||||
# Writes host routes and peers on host addresses.
|
||||
hostNetwork: true
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
apiVersion: source.toolkit.fluxcd.io/v1
|
||||
kind: HelmRepository
|
||||
kind: OCIRepository
|
||||
metadata:
|
||||
name: k8s-gateway
|
||||
namespace: kube-system
|
||||
spec:
|
||||
interval: 24h
|
||||
url: https://k8s-gateway.github.io/k8s_gateway/
|
||||
url: oci://codeberg.org/k8s-gateway/charts/k8s-gateway
|
||||
ref:
|
||||
semver: ">=2.4.0"
|
||||
layerSelector:
|
||||
mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
|
||||
operation: copy
|
||||
---
|
||||
# In-cluster authoritative resolver for renner.uno.
|
||||
#
|
||||
@@ -28,14 +33,10 @@ metadata:
|
||||
namespace: kube-system
|
||||
spec:
|
||||
interval: 30m
|
||||
chart:
|
||||
spec:
|
||||
chart: k8s-gateway
|
||||
version: ">=2.4.0"
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: k8s-gateway
|
||||
namespace: kube-system
|
||||
chartRef:
|
||||
kind: OCIRepository
|
||||
name: k8s-gateway
|
||||
namespace: kube-system
|
||||
install:
|
||||
remediation:
|
||||
retries: 3
|
||||
|
||||
@@ -58,6 +58,12 @@ machine:
|
||||
imageGCHighThresholdPercent: 70
|
||||
imageGCLowThresholdPercent: 60
|
||||
cluster:
|
||||
# etcd peers on node0, VLAN 46 (#576). Pinned: Talos never updates an
|
||||
# existing member's peer URLs, so changing this subnet means leave + rejoin
|
||||
# per member (talosctl etcd leave, reboot).
|
||||
etcd:
|
||||
advertisedSubnets:
|
||||
- 10.2.46.0/24
|
||||
# controlPlane.endpoint is NOT here: `talosctl gen config` takes it as its
|
||||
# second argument, and bootstrap.sh already passes https://k8s.renner.uno:6443
|
||||
# there. Setting it again in this patch restated the same value in a second
|
||||
|
||||
@@ -17,6 +17,8 @@ machine:
|
||||
# pod0 is a closed L2 fabric between the three nodes: statically
|
||||
# addressed, no gateway, nothing to learn from an RA.
|
||||
net.ipv6.conf.pod0.accept_ra: "0"
|
||||
# node0 is closed the same way.
|
||||
net.ipv6.conf.node0.accept_ra: "0"
|
||||
# ingress0 keeps RAs, at the default accept_ra of 2: an IPv6 default
|
||||
# router is a link-local address learned only from the RA, and with
|
||||
# net.ipv6.conf.all.forwarding=1 — which Cilium sets — Linux ignores RAs
|
||||
@@ -122,8 +124,8 @@ kind: LinkConfig
|
||||
name: uplink
|
||||
up: true
|
||||
---
|
||||
# Management, VLAN 40. Carries the node IP, etcd, the API and Ceph's public
|
||||
# network.
|
||||
# Management, VLAN 40: world↔node. The API VIP, admin access to the nodes, and
|
||||
# node egress (DNS, NTP, image pulls). Node↔node traffic is on node0 (#576).
|
||||
apiVersion: v1alpha1
|
||||
kind: VLANConfig
|
||||
name: mgmt0
|
||||
@@ -177,6 +179,19 @@ addresses:
|
||||
- address: 10.2.48.11/24
|
||||
- address: fd02:48::11/64
|
||||
---
|
||||
# Node network, VLAN 46: traffic among the three nodes themselves (#576) — the
|
||||
# node IP, etcd, apiserver↔kubelet. Closed like pod0: statically addressed, no
|
||||
# gateway, never routed via the UDM.
|
||||
apiVersion: v1alpha1
|
||||
kind: VLANConfig
|
||||
name: node0
|
||||
vlanID: 46
|
||||
parent: uplink
|
||||
up: true
|
||||
addresses:
|
||||
- address: fd02:46::11/64
|
||||
- address: 10.2.46.11/24
|
||||
---
|
||||
# Ingress network, VLAN 47. World-to-pod: the BGP session, the next hop for
|
||||
# every advertised LoadBalancer VIP, and the masquerade source for pod egress.
|
||||
# Publicly routed, which is what separates it from pod0.
|
||||
|
||||
@@ -16,6 +16,7 @@ machine:
|
||||
# ingress0 keeps RAs: it learns its IPv6 router and its address from them,
|
||||
# and only the default route is held back — see k8s-1.yaml.
|
||||
net.ipv6.conf.pod0.accept_ra: "0"
|
||||
net.ipv6.conf.node0.accept_ra: "0"
|
||||
net.ipv6.conf.ingress0.accept_ra_defrtr: "0"
|
||||
# Answer ARP only on the link that owns the address — see k8s-1.yaml (#793).
|
||||
net.ipv4.conf.all.arp_ignore: "1"
|
||||
@@ -82,7 +83,7 @@ kind: LinkConfig
|
||||
name: uplink
|
||||
up: true
|
||||
---
|
||||
# Management, VLAN 40. Node IP, etcd, the API and Ceph's public network.
|
||||
# Management, VLAN 40: world↔node — see k8s-1.yaml.
|
||||
apiVersion: v1alpha1
|
||||
kind: VLANConfig
|
||||
name: mgmt0
|
||||
@@ -122,6 +123,17 @@ addresses:
|
||||
- address: 10.2.48.12/24
|
||||
- address: fd02:48::12/64
|
||||
---
|
||||
# Node network, VLAN 46 — see k8s-1.yaml.
|
||||
apiVersion: v1alpha1
|
||||
kind: VLANConfig
|
||||
name: node0
|
||||
vlanID: 46
|
||||
parent: uplink
|
||||
up: true
|
||||
addresses:
|
||||
- address: fd02:46::12/64
|
||||
- address: 10.2.46.12/24
|
||||
---
|
||||
# Ingress network, VLAN 47. World-to-pod, publicly routed, statically
|
||||
# addressed — see k8s-1.yaml for why it is not leased.
|
||||
apiVersion: v1alpha1
|
||||
|
||||
@@ -12,6 +12,7 @@ machine:
|
||||
# ingress0 keeps RAs: it learns its IPv6 router and its address from them,
|
||||
# and only the default route is held back — see k8s-1.yaml.
|
||||
net.ipv6.conf.pod0.accept_ra: "0"
|
||||
net.ipv6.conf.node0.accept_ra: "0"
|
||||
net.ipv6.conf.ingress0.accept_ra_defrtr: "0"
|
||||
# Answer ARP only on the link that owns the address — see k8s-1.yaml (#793).
|
||||
net.ipv4.conf.all.arp_ignore: "1"
|
||||
@@ -109,6 +110,17 @@ addresses:
|
||||
- address: 10.2.48.13/24
|
||||
- address: fd02:48::13/64
|
||||
---
|
||||
# Node network, VLAN 46 — see k8s-1.yaml.
|
||||
apiVersion: v1alpha1
|
||||
kind: VLANConfig
|
||||
name: node0
|
||||
vlanID: 46
|
||||
parent: uplink
|
||||
up: true
|
||||
addresses:
|
||||
- address: fd02:46::13/64
|
||||
- address: 10.2.46.13/24
|
||||
---
|
||||
# Ingress network, VLAN 47. World-to-pod, publicly routed, statically
|
||||
# addressed — see k8s-1.yaml for why it is not leased.
|
||||
apiVersion: v1alpha1
|
||||
|
||||
@@ -7,19 +7,15 @@
|
||||
# part of the config that could not be true anywhere but here.
|
||||
machine:
|
||||
kubelet:
|
||||
# Constrain kubelet's auto-picked node IP to the mgmt LAN. Without this,
|
||||
# kubelet picks the first non-loopback IP, which could land on the v30iot
|
||||
# NIC (ens19, 10.2.30.0/24). Control-plane traffic and hostNetwork pod IPs
|
||||
# must stay on mgmt LAN.
|
||||
# fd40::/64 is the mgmt LAN ULA, and it is listed so the Node object carries
|
||||
# an IPv6 InternalIP at all. Without one, Cilium falls back to detecting the
|
||||
# node's IPv6 from mgmt0 and picks the SLAAC address from the dynamic
|
||||
# provider prefix, which it then probes until the agent restarts (#789).
|
||||
# The GUA is not a control-plane address: BGP peers on fd40::/64 too.
|
||||
# The node IP lives on node0, VLAN 46: the closed node↔node network (#576).
|
||||
# Both families are listed so the Node object carries an IPv6 InternalIP at
|
||||
# all; without one Cilium detects the node's IPv6 itself and picks a SLAAC
|
||||
# address from the dynamic provider prefix (#789). Talos picks one address
|
||||
# per family, IPv4 first, whatever the order here.
|
||||
nodeIP:
|
||||
validSubnets:
|
||||
- 10.2.40.0/24
|
||||
- fd40::/64
|
||||
- fd02:46::/64
|
||||
- 10.2.46.0/24
|
||||
# The UDM on the ingress segment. Published here rather than as a route
|
||||
# because bird owns table 47 and originates the default from it; this is the
|
||||
# one place the address is stated.
|
||||
|
||||
Reference in New Issue
Block a user