Compare commits

..
Author SHA1 Message Date
claude-bot 57acf5d8f2 fix(ceph): set mon_data_avail_warn to 20%, matching the shared /var
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses #434.

Ceph's 30% default assumes the mon owns its filesystem, where 30% free is
a real reserve. Our mons sit on dataDirHostPath /var/lib/rook — the node's
/var, shared with ~29 GB of containerd storage — and the mon DB is ~106 MB.
The default therefore measures container churn rather than Ceph state, and
k8s-2 trips MON_DISK_LOW whenever its image cache fills.

Tuning image GC does not help: measured on k8s-2, dropping the GC high
threshold to 60 deleted 28 images and moved free space by 0.02 percentage
points, because most of /var/lib/containerd is overlayfs snapshots held by
running containers, which image GC cannot reclaim.

20% of a 60 GB /var still leaves ~12 GB behind a 106 MB database.
mon_data_avail_crit is untouched at 5%, and CephMonDiskLow watches /var
directly for the real risk — the node filling, not the mon growing.

Set through cephConfig so it is reconciled from git rather than applied as
a live `ceph config set`.

The comment records that this override should be revisited, and dropped,
when mons get their own volume at bare-metal migration. That is not
possible on the current cluster: every StorageClass is Ceph-backed, so a
mon PVC would be circular, and Rook applies mon.volumeClaimTemplate only
to newly created mons.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-18 21:41:02 +02:00
k.renner 2337aa16e7 Merge pull request 'feat(ceph): add the toolbox, move image GC off Ceph's warn line, alert on health' (#500) from fix-ceph-toolbox-and-gc-thresholds into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
Reviewed-on: k.renner/homelab#500
2026-08-18 19:27:30 +00:00
claude-bot 16360aacf9 feat(ceph): add the toolbox, move image GC off Ceph's warn line, alert
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses #434.

Gap 2 — no admin path. There is still no rook-ceph-tools deployment, so
ceph status, ceph health detail, ceph osd df tree, ceph osd perf and ceph
device ls are all unreachable during an incident: mons, OSDs and mgr
mount no admin keyring and the operator's /etc/ceph is empty. Adds the
upstream toolbox from rook v1.20.4 (matching the running operator image)
with the image pinned to the cluster's own quay.io/ceph/ceph:v19.2.5 and
carrying the same imagepolicy marker as cephcluster.yaml, so the client
cannot drift ahead of the mons. Prerequisites verified present: the
rook-ceph-default ServiceAccount, the rook-ceph-mon-endpoints ConfigMap,
and ceph-secret/ceph-username in the rook-ceph-mon secret.

Image GC. The thresholds were already lowered once from the kubelet
defaults, to 70/60, explicitly to stay under Ceph's mon_data_avail_warn.
But that line is <=30% free and a high of 70 fires at exactly 30% free —
on the threshold, not below it. So the cache fills to precisely the point
where the mon warns, and MON_DISK_LOW trips before kubelet reclaims
anything. Observed on k8s-2: 29.81% free and HEALTH_WARN MON_DISK_LOW,
clearing to 31.64% once GC ran. 60/50 fires at 40% free and reclaims to
50%, leaving 10 points of margin.

There is plenty to reclaim, so the lower threshold will bite: containerd
holds ~29 GB of k8s-2's 60 GB /var and caches 179 images against 60 in
use, some dating to May. (The ~309 GB talosctl attributes to
/var/lib/kubelet is mounted Ceph PVCs counted through mount points, not
local disk.)

Alerting, the follow-up #434 parked until ceph_* series existed — they do
now, via #432. CephHealthWarn deliberately waits 6h: Ceph warns for
self-clearing conditions and paging on those teaches people to ignore it,
so what is worth waking for is a warning that does not clear.
CephHealthError fires at 5m. CephMonDiskLow watches /var free directly
rather than going through Ceph, because this fires for a full image cache
rather than for Ceph growth, and naming that keeps the next person from
debugging the wrong subsystem.

NOTE: cluster.yaml is a bootstrap patch. Merging changes nothing on
running nodes — the live kubelet value needs talosctl patch machineconfig.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-18 21:06:43 +02:00
k.renner bb136e30b3 Merge pull request 'chore(dns): drop the stale UDM records pointing at the retired Hetzner VM' (#497) from drop-stale-hetzner-dns into master
ci/woodpecker/push/deploy-infrastructure Pipeline was successful
ci/woodpecker/cron/update-audit Pipeline was successful
ci/woodpecker/cron/e2e Pipeline was successful
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#497
2026-08-16 21:15:06 +00:00
k.renner 9a1f6cce0c Merge pull request 'fix(cilium): stop the endpoint policy map filling — trim ICMP set, limit identity labels, size + alert' (#495) from fix-cilium-policy-map-exhaustion into master
Reviewed-on: k.renner/homelab#495
2026-08-16 21:14:08 +00:00
claude-bot 25c8c806b4 fix(cilium): cut policy map demand at the source, not just the ceiling
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The earlier commits in this branch raised policyMapMax and added an alert.
Neither addresses why the map fills. This does.

Measured on k8s-1 with cilium-dbg: 8 endpoint policy maps sit at exactly
16384 entries — full — and the rest cluster at ~8,500. The split tracks
policy direction, so one direction costs ~8,500 entries. Dumping velero's
endpoint 2328 (8,499 entries, ingress only) shows 217 identities x 39
port/proto values, of which 38 are ICMP types. The map is ~99% ICMP, and
3,744 of its entries (44%) belong to woodpecker-ci-step identities.

Two multiplying causes, both fixed here.

The ICMP rule in every default-deny-ingress allowed 39 enumerated types
from `namespace: Exists` — every identity in the cluster. Cilium expands
that to one entry per (identity, type), per endpoint, per direction, in
19 namespaces. The rule's own comment records it was widened "from the
error class to the full standard type set"; that widening is the cost.
Trimmed to 16: the error types that motivated the rule (v4 3/11/12, v6
1/2/3/4 — PMTUD is the documented need), echo, and the IPv6 ND/RA/MLDv2
types observed in flight via Hubble (133, 135, 143). Dropped types are
deprecated (v4 4 Source Quench, RFC 6633), router-only (v4 5, v6 137),
obsolete (v4 13-18, 30), superseded (v6 130-132 by 143), or never
deployed (v6 138, 141/142, 148/149, 151-153). openbao-ns.yaml already
runs an error-only set of 7 in production, so 16 is the conservative side
of existing practice.

Cilium had no `labels` filter, so identity came from all pod labels.
Woodpecker stamps each step pod with woodpecker-ci.org/task-uuid and
step=<name>, making every build step its own security identity — 96 of
them minted in the last hour, against zero pods in the namespace. The GC
is healthy and simply outpaced. Excluding those two label patterns
collapses 78 current identities to one. No CiliumNetworkPolicy selects on
them, so they bought churn and nothing else.

Together these take a direction from ~8,500 entries to roughly 140
identities x 17 values ~= 2,400 — about 4,800 for both directions, well
inside even the old 16384 ceiling. policyMapMax stays raised as headroom,
not as the fix.

Per Cilium's docs, existing identities are not recomputed by a `labels`
change; new ones use the filter and the old ones age out through normal
GC, so the map drains over roughly an hour rather than instantly.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-16 23:08:26 +02:00
claude-botandClaude Opus 5 0211269b91 chore(dns): drop the stale UDM records pointing at the retired Hetzner VM
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
renner.uno and www.renner.uno were pinned to 159.69.42.237, the VM that hosted
WordPress before it moved into the cluster. Neither record is live on the UDM
any more (verified: a LAN query for www.renner.uno returns the public answer,
proxy.renner.uno), so the file was drift waiting to be re-applied — a tofu
apply would have pointed the house at a host that is being decommissioned,
while the rest of the world already reaches the cluster.

Their absence is the correct end state, not an omission: www.renner.uno
carries dns.renner.uno/public, so external-dns-unifi writes no UDM record by
design and LAN clients take the public path and hairpin back in. The apex
stays with Cloudflare, which now redirects renner.uno/* to www.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014giKN9N6e5J2DF8QZmVh98
2026-08-16 22:43:42 +02:00
claude-bot 9371f188b6 docs(cilium): state the alert's map scope and complete the lockstep refs
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Two optional points from the second review on #495.

The PrometheusRule alerts on cilium_bpf_map_pressure without filtering on
map_name, so it covers conntrack, NAT and LB maps as well as the policy
map. That is intended — running out of any of them degrades the datapath
— but the PR is framed around the policy map, so a reader could take the
narrow reading. Say so in the file.

The cross-references between the three copies of bpf-policy-map-max were
one-directional: helmrelease.yaml and patches/cilium-cni.yaml each named
only one other site, while talos/cilium.yaml named both. Each now names
the other two.

Not taken: the suggestion to quote policyMapMax as "65536". The chart
declares it `type: [null, integer]` in its values schema
(helm show values cilium --version 1.20.0), so an integer is correct and
a quoted string would be the schema violation.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-16 22:41:42 +02:00
claude-bot 621e4e25bf fix(cilium): carry policyMapMax into the bootstrap inline manifest
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the blocking review finding on #495.

Raising bpf.policyMapMax only in the HelmRelease left the bootstrap path
on the old value. kubernetes/bootstrap.sh:126 passes
kubernetes/talos/patches/cilium-cni.yaml to talosctl apply-config for
every node, and that manifest hardcoded bpf-policy-map-max: "16384" — so
a cold rebuild would install Cilium with the map size this PR exists to
fix, and stay there until Flux reconciled the HelmRelease.

The sibling bpf-lb-map-max is "65536" in both files, matching the live
agent config, so keeping these in sync is the repo's existing practice.

kubernetes/talos/cilium.yaml carries the same key and is updated too.
Nothing in the repo references that file — only patches/cilium-cni.yaml
is used by bootstrap.sh — so it looks like an orphaned rendered snapshot.
Syncing it rather than deleting it: whether it should exist at all is a
separate question from this fix.

Each of the three sites now names the others, so the next change to this
value has the lockstep written down where someone will hit it.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-16 22:36:35 +02:00
k.renner fe75b912f5 Merge pull request 'chore(ci): bump all ci-k8s pipeline pins to 2026-08-10' (#496) from bump-ci-k8s-pins into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#496
2026-08-16 20:34:16 +00:00
claude-bot 8f45e2901b fix(cilium): size the endpoint policy map and alert before it fills
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Closes #451. Closes #454.

Both issues are the same fault: a pod's dedicated cilium_policy_v3_<id>
BPF map ran out of space, endpoint regeneration failed with "no space
left on device", the CNI ADD call never returned, and the pod sat in
sandbox creation. For paperless — replicas: 1 with strategy Recreate —
that was a 41-minute outage on 2026-07-30.

It is not over. Prometheus (metrics landed via #452) puts
cilium_bpf_map_pressure{map_name="cilium_policy_v3_*"} at 0.867 on all
three nodes right now, and max_over_time(...[30d]) at 1.03 — the map has
been driven past capacity. Loki has the overflow recurring as recently
as 2026-08-12 17:46Z, hitting authentik and esphome endpoints.

Three changes:

- bpf.policyMapMax 16384 (chart default) -> 65536, matching the cluster's
  existing bpf-lb-map-max. Takes steady-state pressure from ~87% to ~22%.
  The map is sized at creation, so this only takes effect as agents
  restart.
- The chart range ">=1.19.0" is what let Flux take 1.19.6 -> 1.20.0
  unattended 45 minutes before the first outage. Narrowed to
  ">=1.20.0 <1.21.0" so a CNI minor is a reviewed diff, not a surprise.
- A PrometheusRule on cilium_bpf_map_pressure: warning over 80% for 15m,
  critical at 100%. The metric was already scraped but nothing alerted on
  it, so the first signal was a downstream replica mismatch.

Not addressed: why the busiest endpoint needs ~14k entries. Only 178
CiliumIdentities exist and no CiliumNetworkPolicy declares more than 9
ports, so the entry count is not obviously explained by policy shape.
Establishing that needs `cilium-dbg bpf policy get` on an agent.

MERGING THIS ROLLS THE CNI. The DaemonSet uses
updateStrategy.rollingUpdate.maxUnavailable: 2, so two of three agents
restart at once.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-16 22:28:20 +02:00
k.renner d9b4c47cf2 Merge pull request 'fix(wordpress): route the outpost OAuth path so the gate is a gate, not a dead end' (#494) from fix-outpost-callback-path into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#494
2026-08-16 20:24:09 +00:00
claude-bot 632e092363 chore(ci): bump all ci-k8s pipeline pins to 2026-08-10
ci/woodpecker/manual/e2e-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Closes #461.

Five pipeline steps pinned ci-k8s at two stale tags: e2e-test.yml,
e2e.yml, alert-monitor.yml and issue-fix.yml at 2026-05-01, and
ci-manifests-test.yml at 2026-07-30. 2026-08-10 is the newest dated tag
in gitea_admin/ci-k8s and shares a digest with :latest
(sha256:0019f264...), so all five now run the same image the build
pipeline last produced.

A stale pin eventually points at a garbage-collected tag, and the split
had no deliberate reason behind it.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
2026-08-16 22:21:57 +02:00
33 changed files with 379 additions and 603 deletions
+1 -1
View File
@@ -42,7 +42,7 @@ when:
steps:
- name: fetch-alerts
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
environment:
SOPS_AGE_KEY:
from_secret: sops_age_key
+2 -2
View File
@@ -60,7 +60,7 @@ steps:
# run that never touched them. Tracked in #461.
- name: kustomize-build
depends_on: [] # see note above `steps:`
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
commands:
- |
set -eu
@@ -240,7 +240,7 @@ steps:
# this step fails permanently on its own documentation.
- name: no-digest-pins
depends_on: [] # see note above `steps:`
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
commands:
- |
set -eu
+1 -1
View File
@@ -4,7 +4,7 @@ when:
steps:
- name: e2e
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
environment:
SOPS_AGE_KEY:
from_secret: sops_age_key
+1 -1
View File
@@ -30,7 +30,7 @@ steps:
- sh scripts/ci/test-model-selection.sh
- name: e2e-test
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
environment:
SOPS_AGE_KEY:
from_secret: sops_age_key
+1 -1
View File
@@ -49,7 +49,7 @@ steps:
# (the issue-pick logic needs jq, which only the ci-claude image has);
# minting a 1h read-only token per run is cheap. Degrades to
# snapshot-only if the ci-diagnostics ServiceAccount isn't reconciled.
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-05-01
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-08-10
environment:
SOPS_AGE_KEY:
from_secret: sops_age_key
@@ -4,6 +4,8 @@ kind: Kustomization
# come from the kube-prometheus-stack chart. Flux applies this Kustomization
# only after `monitoring` is Ready (chart installed → CRDs present).
resources:
- prometheusrule-ceph.yaml
- prometheusrule-cilium.yaml
- prometheusrule-hetzner-backup.yaml
- prometheusrule-loki.yaml
- prometheusrule-netbird-availability.yaml
@@ -0,0 +1,53 @@
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: ceph
namespace: monitoring
labels:
# kube-prometheus-stack's Prometheus resource selects on this label;
# without it the rule is created but never loaded into Prometheus.
release: kube-prometheus-stack
spec:
groups:
- name: ceph.rules
rules:
- alert: CephHealthWarn
# ceph_health_status: 0 = HEALTH_OK, 1 = HEALTH_WARN, 2 = HEALTH_ERR.
# 6h is deliberately long. Ceph warns for self-clearing conditions —
# a slow BlueStore op during the Sunday restore sweep, a mon briefly
# below its disk threshold — and paging on those trains people to
# ignore the alert. What matters is a warning that does not clear.
expr: ceph_health_status == 1
for: 6h
labels:
severity: warning
annotations:
summary: "Ceph has been HEALTH_WARN for 6h"
description: "The Ceph cluster has held HEALTH_WARN for six hours, so it is not a self-clearing blip. Get the detail with `kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph health detail`."
- alert: CephHealthError
# HEALTH_ERR means data availability or redundancy is at risk.
expr: ceph_health_status == 2
for: 5m
labels:
severity: critical
annotations:
summary: "Ceph is HEALTH_ERR"
description: "The Ceph cluster is in HEALTH_ERR — availability or redundancy is at risk. Inspect immediately with `kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph health detail` and `ceph osd df tree`."
- alert: CephMonDiskLow
# Named explicitly rather than left to CephHealthWarn, because the
# mons sit on the node's /var alongside containerd's image cache:
# this fires for a full image cache, not for Ceph growth. Cleared by
# kubelet image GC (imageGCHighThresholdPercent in
# kubernetes/talos/patches/cluster.yaml), which must stay below
# Ceph's mon_data_avail_warn of 30% free.
expr: |
100 * node_filesystem_avail_bytes{mountpoint="/var"}
/ node_filesystem_size_bytes{mountpoint="/var"} < 30
for: 30m
labels:
severity: warning
annotations:
summary: "{{ $labels.instance }} /var below 30% free — Ceph mon will warn"
description: "/var on {{ $labels.instance }} is under 30% free, which is Ceph's mon_data_avail_warn threshold, and the mon on this node reports MON_DISK_LOW below it. Usually containerd's image cache; check whether kubelet image GC is reclaiming with `talosctl -n <node> usage /var/lib --depth 1`."
@@ -0,0 +1,44 @@
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: cilium
namespace: monitoring
labels:
# kube-prometheus-stack's Prometheus resource selects on this label;
# without it the rule is created but never loaded into Prometheus.
release: kube-prometheus-stack
spec:
groups:
- name: cilium.rules
# Both alerts are map-agnostic on purpose. cilium_bpf_map_pressure is
# emitted per map type — policy, conntrack, NAT, LB — distinguished by
# the map_name label, and running out of any of them degrades the
# datapath. The policy map is merely the one that has bitten us.
# Cilium only exports the metric for maps above
# policyMapPressureMetricsThreshold (chart default 0.1), so quiet maps
# produce no series rather than a flood of near-zero ones.
rules:
- alert: CiliumBPFMapPressureHigh
# cilium_bpf_map_pressure is used/capacity for the fullest map of
# that name on the agent. A full map is not a slow path — it is a
# hard failure: the agent cannot program the datapath, so endpoint
# regeneration fails and every new pod on the node hangs in sandbox
# creation until something frees space.
expr: cilium_bpf_map_pressure > 0.8
for: 15m
labels:
severity: warning
annotations:
summary: "Cilium BPF map {{ $labels.map_name }} is {{ $value | humanizePercentage }} full on {{ $labels.node }}"
description: "The BPF map {{ $labels.map_name }} on node {{ $labels.node }} has been over 80% full for 15 minutes. At 100% the Cilium agent can no longer program the datapath and new pods on this node fail sandbox creation. Raise the corresponding size in kubernetes/infrastructure/cilium/helmrelease.yaml (values.bpf) — the map is sized at creation, so the change needs an agent restart."
- alert: CiliumBPFMapFull
# Pressure reaches (and can exceed) 1.0 when the agent wants more
# entries than the map holds. Pods are already failing to start.
expr: cilium_bpf_map_pressure >= 1
for: 0m
labels:
severity: critical
annotations:
summary: "Cilium BPF map {{ $labels.map_name }} is full on {{ $labels.node }}"
description: "The BPF map {{ $labels.map_name }} on node {{ $labels.node }} is at or over capacity. Endpoint regeneration is failing with 'no space left on device' and pods scheduled to this node will hang in ContainerCreating. Check `{namespace=\"kube-system\",pod=~\"cilium-.+\"} |= \"no space left on device\"` in Loki for the affected endpoints."
@@ -17,7 +17,11 @@ spec:
chart:
spec:
chart: cilium
version: ">=1.19.0"
# Patch-only range: Flux reconciles every 30m and would otherwise take a
# CNI minor or major unattended. Widening this is a deliberate, reviewed
# act — a minor bump changes datapath behaviour and rolls 2 of 3 agents
# at once.
version: ">=1.20.0 <1.21.0"
sourceRef:
kind: HelmRepository
name: cilium
@@ -29,6 +33,16 @@ spec:
remediation:
retries: 3
values:
# Which pod labels feed security-identity computation. Patterns are
# start-anchored regexes; a leading "!" excludes while keeping the default
# set. Woodpecker stamps every step pod with woodpecker-ci.org/task-uuid
# and step=<name>, so without this each build step is its own identity —
# and every identity costs one entry per allowed (direction, proto, port)
# in every endpoint's policy map, cluster-wide. Nothing selects on these
# labels in policy, so they buy identity churn and nothing else.
# Keep in lockstep with kubernetes/talos/patches/cilium-cni.yaml and
# kubernetes/talos/cilium.yaml.
labels: '!woodpecker-ci\.org/.* !step$'
ipv6:
enabled: true
ipam:
@@ -46,6 +60,19 @@ spec:
serviceMonitor:
enabled: true
bpf:
# Size of each endpoint's own cilium_policy_v3_<id> map, holding one
# entry per (identity, direction, protocol, port) the endpoint is
# allowed. When it fills, endpoint regeneration fails with "no space
# left on device", the CNI ADD call never returns, and the pod is stuck
# in sandbox creation — so this must stay comfortably above the busiest
# endpoint's entry count, tracked by cilium_bpf_map_pressure.
# Takes effect on agent restart; the map is sized at creation.
# Bootstrap installs Cilium from the inline manifest in
# kubernetes/talos/patches/cilium-cni.yaml, so bpf-policy-map-max there
# must move with this value or a cold rebuild starts with the old size.
# kubernetes/talos/cilium.yaml carries a third copy of the same key.
# Nothing enforces any of this; grep bpf-policy-map-max before changing it.
policyMapMax: 65536
# Let pods reach the cluster's own LoadBalancer external IPs; Cilium
# redirects the socket directly to the backend pod instead of sending
# traffic out to the VIP and back (which fails with L2 announcements).
@@ -31,43 +31,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
@@ -103,40 +80,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -38,43 +38,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
@@ -103,40 +80,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -36,40 +36,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -32,40 +32,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -27,43 +27,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
@@ -102,40 +79,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -28,43 +28,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
@@ -103,40 +80,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -157,44 +157,21 @@ spec:
icmps:
- fields:
- { type: 0, family: IPv4 } # Echo Reply
- { type: 3, family: IPv4 } # Destination Unreachable
- { type: 4, family: IPv4 } # Source Quench
- { type: 5, family: IPv4 } # Redirect
- { type: 3, family: IPv4 } # Destination Unreachable — PMTUD
- { type: 8, family: IPv4 } # Echo Request
- { type: 9, family: IPv4 } # Router Advertisement
- { type: 10, family: IPv4 } # Router Solicitation
- { type: 11, family: IPv4 } # Time Exceeded
- { type: 12, family: IPv4 } # Parameter Problem
- { type: 13, family: IPv4 } # Timestamp
- { type: 14, family: IPv4 } # Timestamp Reply
- { type: 15, family: IPv4 } # Information Request
- { type: 16, family: IPv4 } # Information Reply
- { type: 17, family: IPv4 } # Address Mask Request
- { type: 18, family: IPv4 } # Address Mask Reply
- { type: 30, family: IPv4 } # Traceroute
- { type: 1, family: IPv6 } # Destination Unreachable
- { type: 2, family: IPv6 } # Packet Too Big
- { type: 2, family: IPv6 } # Packet Too Big — PMTUD, mandatory
- { type: 3, family: IPv6 } # Time Exceeded
- { type: 4, family: IPv6 } # Parameter Problem
- { type: 128, family: IPv6 } # Echo Request
- { type: 129, family: IPv6 } # Echo Reply
- { type: 130, family: IPv6 } # Multicast Listener Query
- { type: 131, family: IPv6 } # Multicast Listener Report
- { type: 132, family: IPv6 } # Multicast Listener Done
- { type: 133, family: IPv6 } # Router Solicitation
- { type: 134, family: IPv6 } # Router Advertisement
- { type: 135, family: IPv6 } # Neighbor Solicitation
- { type: 136, family: IPv6 } # Neighbor Advertisement
- { type: 137, family: IPv6 } # Redirect
- { type: 138, family: IPv6 } # Router Renumbering
- { type: 141, family: IPv6 } # Inverse ND Solicitation
- { type: 142, family: IPv6 } # Inverse ND Advertisement
- { type: 143, family: IPv6 } # MLDv2 Listener Report
- { type: 148, family: IPv6 } # Certification Path Solicitation
- { type: 149, family: IPv6 } # Certification Path Advertisement
- { type: 151, family: IPv6 } # Multicast Router Advertisement
- { type: 152, family: IPv6 } # Multicast Router Solicitation
- { type: 153, family: IPv6 } # Multicast Router Termination
egress:
# Cluster-internal callers HA initiates connections to: mosquitto,
# whisper, matter-server, etc.
@@ -253,43 +230,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
---
# Scoped world ingress — replaces the former namespace-wide `fromEntities:
# world` allow (see the note in the default-deny-ingress policy above). IoT-VLAN
@@ -42,40 +42,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -42,40 +42,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -67,40 +67,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -37,43 +37,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
@@ -109,43 +86,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
---
# Allow FTP access (from scanners on the LAN) to the ftp-scanner pod.
apiVersion: cilium.io/v2
@@ -41,40 +41,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -26,43 +26,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
# Peer P2P
- toEndpoints:
@@ -92,40 +69,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -42,43 +42,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
---
# The TRMNL panel polls /api/display over cleartext HTTP on the
# terminus-device LoadBalancer. The firmware cannot be handed the internal
@@ -28,40 +28,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -45,40 +45,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -53,43 +53,20 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
@@ -128,40 +105,17 @@ spec:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -24,6 +24,27 @@ spec:
# gives headroom for parallel restores during cluster rebuild too
# (paperless + audiobookshelf + immich + ... all clone at once).
cephConfig:
mon:
# Ceph's 30% default assumes the mon has a filesystem to itself, where
# 30% free is a meaningful reserve. Here the mons live on
# dataDirHostPath /var/lib/rook — the node's /var, shared with ~29 GB
# of containerd image and snapshot storage — and the mon DB itself is
# ~106 MB. So the default measures container churn, not Ceph: k8s-2
# trips MON_DISK_LOW whenever the image cache fills, and image GC does
# not clear it (most of /var/lib/containerd is overlayfs snapshots held
# by running containers, which GC cannot reclaim — see #434).
# 20% of a 60 GB /var still leaves ~12 GB against a 106 MB database.
# mon_data_avail_crit stays at the 5% default, and CephMonDiskLow in
# monitoring-rules watches /var directly for the real risk, which is the
# node filling rather than the mon growing.
#
# BARE-METAL MIGRATION: revisit this. Give the mons their own volume and
# the default 30% becomes meaningful again, at which point this override
# should be dropped rather than carried over. It cannot be done on the
# current cluster — every StorageClass is Ceph-backed, so a mon PVC
# would be circular, and Rook only applies mon.volumeClaimTemplate to
# newly created mons, never migrating existing ones.
mon_data_avail_warn: "20"
mgr:
mgr/volumes/max_concurrent_clones: "16"
# QLC tune for the Crucial P2 NVMe per host. Baseline 2026-05-05:
@@ -2,6 +2,7 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- cephcluster.yaml
- toolbox.yaml
- storageclass.yaml
- volumesnapshotclass.yaml
- objectstore.yaml
@@ -0,0 +1,172 @@
# Upstream Rook toolbox, taken verbatim from
# https://raw.githubusercontent.com/rook/rook/v1.20.4/deploy/examples/toolbox.yaml
# (matching the operator image docker.io/rook/ceph:v1.20.4), with three
# deliberate changes:
# - the image is pinned to the cluster's own Ceph version and carries the
# same imagepolicy marker as cephcluster.yaml, so the client never drifts
# ahead of the mons it talks to
# - resource requests/limits, since this is a long-lived idle pod
# - no other divergence; keep it that way so upstream diffs stay readable
#
# This is the only admin path to Ceph internals. Without it `ceph status`,
# `ceph health detail`, `ceph osd df tree`, `ceph osd perf` and `ceph device
# ls` are unreachable during an incident: the mons, OSDs and mgr mount no
# admin keyring, and the operator's /etc/ceph is empty.
#
# It mounts the client.admin keyring from the rook-ceph-mon secret, so it is
# a full-privilege credential holder, not a read-only one — Rook offers no
# lesser-privileged toolbox. Ingress to it is gated by the namespace
# default-deny-ingress CNP (network-policies/rook-ceph-ns.yaml).
apiVersion: apps/v1
kind: Deployment
metadata:
name: rook-ceph-tools
namespace: rook-ceph
labels:
app: rook-ceph-tools
spec:
replicas: 1
selector:
matchLabels:
app: rook-ceph-tools
template:
metadata:
labels:
app: rook-ceph-tools
spec:
dnsPolicy: ClusterFirstWithHostNet
serviceAccountName: rook-ceph-default
containers:
- name: rook-ceph-tools
image: quay.io/ceph/ceph:v19.2.5 # {"$imagepolicy": "flux-system:ceph"}
command:
- /bin/bash
- -c
- |
# Replicate the script from toolbox.sh inline so the ceph image
# can be run directly, instead of requiring the rook toolbox
CEPH_CONFIG="/etc/ceph/ceph.conf"
MON_CONFIG="/etc/rook/mon-endpoints"
KEYRING_FILE="/etc/ceph/keyring"
CONFIG_OVERRIDE="/etc/rook-config-override/config"
# create a ceph config file in its default location so ceph/rados tools can be used
# without specifying any arguments
write_endpoints() {
endpoints=$(cat ${MON_CONFIG})
# filter out the mon names
# external cluster can have numbers or hyphens in mon names, handling them in regex
# shellcheck disable=SC2001
mon_endpoints=$(echo "${endpoints}"| sed 's/[a-z0-9_-]\+=//g')
DATE=$(date)
echo "$DATE writing mon endpoints to ${CEPH_CONFIG}: ${endpoints}"
cat <<EOF > ${CEPH_CONFIG}
[global]
mon_host = ${mon_endpoints}
[client.admin]
keyring = ${KEYRING_FILE}
EOF
# Merge the config override if it exists and is not empty
if [ -f "${CONFIG_OVERRIDE}" ] && [ -s "${CONFIG_OVERRIDE}" ]; then
echo "$DATE merging config override from ${CONFIG_OVERRIDE}"
echo "" >> ${CEPH_CONFIG}
cat ${CONFIG_OVERRIDE} >> ${CEPH_CONFIG}
echo "" >> ${CEPH_CONFIG}
fi
}
# watch the endpoints config file and update if the mon endpoints ever change
watch_endpoints() {
# get the timestamp for the target of the soft link
real_path=$(realpath ${MON_CONFIG})
initial_time=$(stat -c %Z "${real_path}")
while true; do
real_path=$(realpath ${MON_CONFIG})
latest_time=$(stat -c %Z "${real_path}")
if [[ "${latest_time}" != "${initial_time}" ]]; then
write_endpoints
initial_time=${latest_time}
fi
sleep 10
done
}
# read the secret from an env var (for backward compatibility), or from the secret file
ceph_secret=${ROOK_CEPH_SECRET}
if [[ "$ceph_secret" == "" ]]; then
ceph_secret=$(cat /var/lib/rook-ceph-mon/secret.keyring)
fi
# create the keyring file
cat <<EOF > ${KEYRING_FILE}
[${ROOK_CEPH_USERNAME}]
key = ${ceph_secret}
EOF
# write the initial config file
write_endpoints
# continuously update the mon endpoints if they fail over
watch_endpoints
imagePullPolicy: IfNotPresent
tty: true
securityContext:
runAsNonRoot: true
runAsUser: 2016
runAsGroup: 2016
capabilities:
drop: ["ALL"]
env:
- name: ROOK_CEPH_USERNAME
valueFrom:
secretKeyRef:
name: rook-ceph-mon
key: ceph-username
resources:
requests:
cpu: 10m
memory: 128Mi
limits:
memory: 512Mi
volumeMounts:
- mountPath: /etc/ceph
name: ceph-config
- name: mon-endpoint-volume
mountPath: /etc/rook
- name: ceph-admin-secret
mountPath: /var/lib/rook-ceph-mon
readOnly: true
- name: rook-config-override
mountPath: /etc/rook-config-override
readOnly: true
volumes:
- name: ceph-admin-secret
secret:
secretName: rook-ceph-mon
optional: false
items:
- key: ceph-secret
path: secret.keyring
- name: mon-endpoint-volume
configMap:
name: rook-ceph-mon-endpoints
items:
- key: data
path: mon-endpoints
- name: rook-config-override
configMap:
name: rook-config-override
optional: true
- name: ceph-config
emptyDir: {}
tolerations:
- key: "node.kubernetes.io/unreachable"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 5
+8 -2
View File
@@ -249,6 +249,11 @@ data:
identity-heartbeat-timeout: "30m0s"
identity-gc-interval: "15m0s"
# Exclude Woodpecker's per-build labels from identity computation, so CI
# steps share one identity instead of minting one each. Keep in lockstep
# with patches/cilium-cni.yaml and values.labels in
# kubernetes/infrastructure/cilium/helmrelease.yaml.
labels: '!woodpecker-ci\.org/.* !step$'
cilium-endpoint-gc-interval: "5m0s"
nodes-gc-interval: "5m0s"
@@ -302,8 +307,9 @@ data:
# sizing of the TCP CT, non-TCP CT, NAT and policy BPF maps.
bpf-map-dynamic-size-ratio: "0.0025"
# bpf-policy-map-max specifies the maximum number of entries in endpoint
# policy map (per endpoint)
bpf-policy-map-max: "16384"
# policy map (per endpoint). Keep in lockstep with patches/cilium-cni.yaml and
# values.bpf.policyMapMax in kubernetes/infrastructure/cilium/helmrelease.yaml.
bpf-policy-map-max: "65536"
# bpf-policy-stats-map-max specifies the maximum number of entries in global
# policy stats map
bpf-policy-stats-map-max: "65536"
+11 -2
View File
@@ -267,6 +267,11 @@ cluster:
identity-heartbeat-timeout: "30m0s"
identity-gc-interval: "15m0s"
# Exclude Woodpecker's per-build labels from identity computation, so
# CI steps share one identity instead of minting one each. Keep in
# lockstep with values.labels in
# kubernetes/infrastructure/cilium/helmrelease.yaml.
labels: '!woodpecker-ci\.org/.* !step$'
cilium-endpoint-gc-interval: "5m0s"
nodes-gc-interval: "5m0s"
@@ -320,8 +325,12 @@ cluster:
# sizing of the TCP CT, non-TCP CT, NAT and policy BPF maps.
bpf-map-dynamic-size-ratio: "0.0025"
# bpf-policy-map-max specifies the maximum number of entries in endpoint
# policy map (per endpoint)
bpf-policy-map-max: "16384"
# policy map (per endpoint). Keep in lockstep with values.bpf.policyMapMax
# in kubernetes/infrastructure/cilium/helmrelease.yaml, and with the third
# copy in kubernetes/talos/cilium.yaml: bootstrap installs Cilium from this
# inline manifest, so a lower value here means a cold rebuild comes up with
# a map that overflows until Flux reconciles.
bpf-policy-map-max: "65536"
# bpf-policy-stats-map-max specifies the maximum number of entries in global
# policy stats map
bpf-policy-stats-map-max: "65536"
+16 -12
View File
@@ -26,19 +26,23 @@ machine:
nodeIP:
validSubnets:
- 10.2.40.0/24
# Lower the image GC thresholds from the kubelet defaults (85/80)
# so containerd's image cache is pruned before /var crosses Ceph's
# mon_data_avail_warn line (mon DB warns when its hostpath has
# <30% free). k8s-2 and k8s-3 sit at 77-79% used because of pod
# density (75 pods on k8s-2 vs 42 on k8s-1) — unused image layers
# accumulate untouched until the 85% high watermark, which the
# node never hits, so GC never fires. 70/60 keeps /var safely
# under 30%-free without raising the VM disk again. Spegel P2P
# pull-through softens the re-pull cost when GC'd images are
# needed back.
# Lower the image GC thresholds from the kubelet defaults (85/80) so
# containerd's image cache is pruned before /var crosses Ceph's
# mon_data_avail_warn line — the mon DB warns when its hostpath has
# <=30% free, and mons live on /var/lib/rook (hostPath, no PVC).
# High must stay strictly below that line, not on it: a high of 70
# means GC first fires at exactly 30% free, which is the warn point
# itself, so every cache fill-up trips MON_DISK_LOW before kubelet
# reclaims anything. 60/50 fires at 40% free and reclaims to 50%,
# keeping 10 points of margin.
# containerd is the dominant local consumer of /var (~29 GB of 60 on
# k8s-2, which carries 75 pods against 42 on k8s-1) and most of it is
# reclaimable — 179 images cached against 60 in use. Spegel P2P
# pull-through softens the re-pull cost when GC'd images are needed
# back.
extraConfig:
imageGCHighThresholdPercent: 70
imageGCLowThresholdPercent: 60
imageGCHighThresholdPercent: 60
imageGCLowThresholdPercent: 50
cluster:
controlPlane:
endpoint: https://k8s.renner.uno:6443
+15 -3
View File
@@ -1,8 +1,20 @@
locals {
renner_uno_records = {
# A Records
"@" = { type = "A", value = "159.69.42.237" }
"www" = { type = "A", value = "159.69.42.237" }
# No "@" / "www" entries. Both used to pin renner.uno and www.renner.uno
# to 159.69.42.237, the Hetzner VM that hosted the WordPress site until it
# moved into the cluster (kubernetes/apps/wordpress). Neither record is
# actually live on the UDM any more — verified 2026-08-16: a LAN query for
# www.renner.uno returns the public answer (proxy.renner.uno) — so this
# block was drift waiting to be re-applied.
#
# The correct end state is exactly this absence: www.renner.uno is labelled
# dns.renner.uno/public, so external-dns-unifi deliberately writes no UDM
# record and LAN clients follow the public path (wildcard -> proxy) and
# hairpin back in. Re-adding an A record here would point the house at a
# server that is being decommissioned.
#
# The apex stays with Cloudflare, where a redirect rule sends
# renner.uno/* to https://www.renner.uno/*.
# CNAME Records (External Targets)
"nut" = { type = "CNAME", value = "nut.srv.renner.uno" }