Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5.5 32a995a6bf docs(network): traffic classes as built, node-to-node on VLAN 46 (#576)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
NETWORK-traffic-classes.md described table 48 and rules 980/981/990/991,
which #857 replaced with one table (47) decided by longest prefix, and
listed management as not started. Rewritten to the live state: five
classes, table 47 contents owned by bird, node IP and etcd on node0, the
etcd leave-and-rejoin procedure, and the mixed-state hazard hit during
the node IP move. The bird DaemonSet comment still named table 48.

Closes #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 14:48:12 +02:00
k.renner 8a0ed135f8 Merge pull request 'feat(talos): node IP and etcd on VLAN 46 (#576 step 3)' (#922) from feat/node-ip-vlan-46 into master
Reviewed-on: k.renner/homelab#922
2026-09-23 12:47:06 +00:00
claude-botandClaude Opus 5.5 be643533f6 feat(talos): node IP and etcd on VLAN 46 (#576 step 3)
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Records the live state after moving all three nodes:
- site-homelab.yaml: kubelet nodeIP.validSubnets fd02:46::/64 and
  10.2.46.0/24 (was 10.2.40.0/24 and the legacy fd40::/64).
- cluster.yaml: etcd advertisedSubnets 10.2.46.0/24.
- mgmt0 comments: VLAN 40 is world-to-node only now.

Applied per node (k8s-3, k8s-2, k8s-1): drain, stage the config (delete
the old lists, set the new ones, one staged patch call), etcd
forfeit-leadership and leave, reboot. Each node rejoined etcd with its
VLAN 46 peer URL; EPHEMERAL was not wiped.

Refs #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 14:43:17 +02:00
k.renner 311b4639a6 Merge pull request 'feat(bird): put node0 in the pod routing table (#576 step 2)' (#919) from feat/bird-attached-node0 into master
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#919
2026-09-23 11:31:32 +00:00
k.renner a95efdef33 Merge pull request 'feat(talos): node network on VLAN 46, etcd pinned to VLAN 40 (#576 step 1)' (#918) from feat/node-vlan-46-link into master
Reviewed-on: k.renner/homelab#918
2026-09-23 11:31:23 +00:00
claude-botandClaude Opus 5.5 aea4be734c feat(bird): put node0 in the pod routing table (#576 step 2)
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Pod-sourced traffic is routed from table 47. Once the node IP moves to
VLAN 46 (step 3), pods reach node addresses there: the kubernetes
ClusterIP (socket LB rewrites it to the apiserver's advertise address),
hostNetwork pod IPs, and replies to kubelet probes. Without the attached
node0 route in table 47 those flows take the default to the UDM and every
in-cluster API client breaks. Taking it from the interface, like the
other attached networks, keeps the table in one owner.

Refs #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 13:28:57 +02:00
claude-botandClaude Opus 5.5 07dd6097ec feat(talos): node network on VLAN 46, etcd pinned to VLAN 40 (#576 step 1)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
- node0: VLANConfig on the uplink trunk, VLAN 46, static fd02:46::1x/64
  and 10.2.46.1x/24, accept_ra 0. Closed like pod0. Unused until the node
  IP moves there (step 3).
- cluster.etcd.advertisedSubnets pinned to 10.2.40.0/24 so a reboot
  cannot pick a node0 address: Talos never updates an existing member's
  peer URLs.

Applied live with talosctl patch --mode=no-reboot, k8s-3 first. Node IPs
unchanged, etcd 3/3 healthy, node0 reachable node to node over v6 and v4.

Refs #576

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 13:25:12 +02:00
k.renner 60361d4c8a Merge pull request 'chore(talos): drop the #832 stage labels' (#917) from chore/cni-drop-stage-labels into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#917
2026-09-23 10:46:35 +00:00
claude-botandClaude Opus 5.5 a8cef5de1b chore(talos): drop the #832 stage labels
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Nothing selects cni.renner.uno/cilium-uninstall or
cni.renner.uno/multus-sole-writer since stage B (#915, #916). Removed
live from all three nodes with a $patch: delete strategic merge patch
(--mode=no-reboot); Talos dropped them from the Kubernetes nodes and
left the other labels in place.

Closes #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 12:40:10 +02:00
k.renner 505802dad5 Merge pull request 'feat(cilium): write the conflist where only Multus reads it (stage B2 of #832)' (#916) from feat/cni-sole-writer-converge-b2 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#916
2026-09-23 10:38:00 +00:00
claude-botandClaude Opus 5.5 ba66b604d4 feat(cilium): write the conflist where only Multus reads it (stage B2 of #832)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
cni.confPath: /etc/cni/net.d/multus mounts that host directory as the
agent's /host/etc/cni/net.d, so write-cni-conf-when-ready (unchanged,
.../05-cilium.conflist) lands in /etc/cni/net.d/multus/05-cilium.conflist,
the file the per-node CiliumNodeConfigs already produced on all three
nodes. The agent finds it unchanged and does not rewrite it.

updateStrategy maxUnavailable: 1 (chart default 2) so the roll takes one
node's datapath at a time.

bootstrap.sh renders the chart from these values, so a cold rebuild
lands directly in this layout.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 11:55:12 +02:00
k.renner 4fbc49fc8c Merge pull request 'feat(cni): converge Multus as sole CNI writer, part 1 (stage B1 of #832)' (#915) from feat/cni-sole-writer-converge-b1 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#915
2026-09-23 09:53:24 +00:00
claude-botandClaude Opus 5.5 f493c9fe52 feat(cni): converge Multus as sole CNI writer, part 1 (stage B1 of #832)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
All three nodes run the sole-writer layout. Converge the Multus side and
drop the per-node Cilium overrides:
- kube-multus-ds-subdir becomes the only Multus DaemonSet, no node
  affinity; kube-multus-ds (0 pods) and its ConfigMap are pruned.
- /etc/cni/net.d/multus is mounted DirectoryOrCreate so Multus waits for
  the Cilium conflist instead of exiting on a missing directory.
- CiliumNodeConfigs removed. They only act when a cilium pod is created;
  stage B2 moves the same path into the HelmRelease (cni.confPath) and
  rolls the agents.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 11:43:48 +02:00
k.renner 8a3c66eb2e Merge pull request 'chore(talos): record #832 stage labels on k8s-1' (#914) from chore/cni-stage-k8s-1 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#914
2026-09-23 09:31:07 +00:00
claude-botandClaude Opus 5.5 9928999da4 chore(talos): record #832 stage labels on k8s-1
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
k8s-1 carries cni.renner.uno/cilium-uninstall and
cni.renner.uno/multus-sole-writer, applied live with talosctl patch.
Verified on the node: /etc/cni/net.d holds only 00-multus.conf at the
top level, and after a reboot with immediate uncordon the OSD came up
with net1 (10.99.0.1) and no restarts. All three nodes are converted.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 11:15:45 +02:00
k.renner b3def28f7c Merge pull request 'fix(whereabouts): let the reconciler respect cordons' (#913) from fix/whereabouts-reconciler-cordon into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#913
2026-09-23 09:04:29 +00:00
claude-botandClaude Opus 5.5 93dbcdb2fb fix(whereabouts): let the reconciler respect cordons
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The Deployment tolerated every NoSchedule taint (operator: Exists, no
key), including node.kubernetes.io/unschedulable. During the #832 node
reboots its ReplicaSet kept placing replacement pods on the cordoned,
shutting-down node; the kubelet rejected each with NodeShutdown, about
200 failed pods per reboot. Tolerate only the control-plane taint the
comment names.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 10:56:50 +02:00
k.renner 47bf16cfac Merge pull request 'chore(talos): record #832 stage labels on k8s-2' (#912) from chore/cni-stage-k8s-2 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#912
2026-09-23 08:45:30 +00:00
claude-botandClaude Opus 5.5 4664de8f97 chore(talos): record #832 stage labels on k8s-2
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
k8s-2 carries cni.renner.uno/cilium-uninstall and
cni.renner.uno/multus-sole-writer, applied live with talosctl patch.
Verified on the node: /etc/cni/net.d holds only 00-multus.conf at the
top level, and after a reboot with immediate uncordon the OSD came up
with net1 (10.99.0.2) and no restarts.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 10:44:17 +02:00
k.renner fb52e4be88 Merge pull request 'chore(talos): record #832 stage labels on k8s-3' (#911) from chore/cni-stage-k8s-3 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#911
2026-09-23 08:28:40 +00:00
claude-botandClaude Opus 5.5 f11a71c33f chore(talos): record #832 stage labels on k8s-3
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
k8s-3 carries cni.renner.uno/cilium-uninstall and
cni.renner.uno/multus-sole-writer, applied live with talosctl patch.
Verified on the node: /etc/cni/net.d holds only 00-multus.conf at the
top level, Cilium writes multus/05-cilium.conflist, and after a reboot
with immediate uncordon the OSD came up with net1 (10.99.0.3) and no
restarts.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 10:25:53 +02:00
k.renner 29326eead3 Merge pull request 'feat(cni): per-node switch for Multus as sole CNI writer (stage A of #832)' (#910) from feat/cni-staged-sole-writer into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#910
2026-09-23 07:52:22 +00:00
claude-botandClaude Opus 5.5 eb09b1b2a3 feat(cni): per-node switch for Multus as sole CNI writer (stage A of #832)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Nothing changes until a node is labelled:
- post-crds/cilium-nodeconfig.yaml: CiliumNodeConfig cni-uninstall
  (label cni.renner.uno/cilium-uninstall) arms the agent's preStop
  conflist cleanup; multus-sole-writer (label
  cni.renner.uno/multus-sole-writer) moves write-cni-conf-when-ready to
  /host/etc/cni/net.d/multus/05-cilium.conflist, a subdirectory libcni
  does not read.
- multus/multus-subdir.yaml: kube-multus-ds-subdir with
  multusAutoconfigDir /host/etc/cni/net.d/multus, scheduled only on
  labelled nodes; kube-multus-ds gets the inverse affinity.

The kube-multus-ds template change rolls Multus node by node once on
merge.

Refs #832

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 09:47:16 +02:00
k.renner dd3a1f1e90 Merge pull request 'feat(netbird-service-sync): run the controller in its own namespace (stage 2 of #876)' (#909) from feat/netbird-service-sync-phase2 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#909
2026-09-23 06:55:25 +00:00
claude-botandClaude Opus 5.5 98a0462137 feat(netbird-service-sync): run the controller in its own namespace (stage 2 of #876)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
- kubernetes/infrastructure/netbird-proxy-services -> netbird-service-sync:
  SA, ClusterRole(Binding) and Deployment renamed, in namespace
  netbird-service-sync, image gitea_admin/netbird-service-sync pinned to
  the master build digest, BAO_ROLE=netbird-service-sync.
- Old policy doc netbird-proxy-services.hcl removed; Traefik comment
  follows the name.

The infrastructure KS owns both old and new objects, so prune and apply
happen in one pass. OpenBao policy and role netbird-service-sync are
already written.

Closes #876

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 08:41:20 +02:00
k.renner 9aef2e0669 Merge pull request 'feat(netbird-service-sync): prepare rename and own namespace (stage 1 of #876)' (#908) from feat/netbird-service-sync-phase1 into master
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#908
2026-09-23 06:35:14 +00:00
claude-botandClaude Opus 5.5 fd7597ddc2 chore(netbird-service-sync): drop stray __pycache__
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 08:30:28 +02:00
claude-botandClaude Opus 5.5 10b0c1b853 feat(netbird-service-sync): prepare rename and own namespace (stage 1 of #876)
ci/woodpecker/push/build-ci-images Pipeline was canceled
Additive; the running controller is untouched:
- ci/images/netbird-proxy-services -> ci/images/netbird-service-sync,
  build step and image name renamed, so the merge builds
  gitea_admin/netbird-service-sync.
- Namespace netbird-service-sync with a baseline policy: probes in;
  out only to kube-dns, kube-apiserver, OpenBao :8200 and world :443
  (NetBird management API).
- openbao-ns admits netbird-service-sync on :8200, like wiki-index.
- policies/netbird-service-sync.hcl documents the new policy and role
  (applied by hand, as before).

Refs #876

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 08:30:15 +02:00
k.renner 674169cb08 Merge pull request 'chore(gitea): drop the apps-namespace grants for Gitea (stage 3 of #878)' (#907) from chore/gitea-namespace-phase3 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#907
2026-09-23 06:20:17 +00:00
claude-botandClaude Opus 5.5 0c6174b7a6 chore(gitea): drop the apps-namespace grants for Gitea (stage 3 of #878)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Gitea runs in the gitea namespace since #906. Remove what only served it
in apps:
- openbao rotator egress (flux-git, review-bot): apps/gitea target.
- ai-ns wiki-index ingress: apps/gitea source.
- apps default-deny egress to wiki-index :8080 (gitea's webhook; now in
  gitea-ns.yaml).
- Velero schedule comments.

Refs #878

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 08:09:30 +02:00
k.renner 76888c3771 Merge pull request 'feat(gitea): move Gitea into the gitea namespace (stage 2 of #878)' (#906) from feat/gitea-namespace-phase2 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#906
2026-09-23 06:05:33 +00:00
claude-botandClaude Opus 5.5 eb23d3ac6e feat(gitea): move Gitea into the gitea namespace (stage 2 of #878)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
- kubernetes/apps/gitea: every object in namespace gitea; applied by a
  new gitea Flux Kustomization (clusters/homelab/gitea.yaml) instead of
  apps. gitea-allow-metrics-scrape dropped: the gitea-ns baseline
  already admits monitoring.
- Rotators (flux-git, review-bot) and the Prometheus gitea job use
  gitea.gitea.svc.
- bootstrap.sh waits for the gitea KS.

Data: data-gitea-0 is not copied. The PV (reclaimPolicy Retain) is
rebound to a PVC gitea/data-gitea-0 pre-created with volumeName, which
the new StatefulSet adopts. gitea-tls is pre-issued in the gitea
namespace so HTTPS has no certificate gap.

Refs #878

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 07:44:26 +02:00
k.renner c1e3f3fb98 Merge pull request 'feat(gitea): prepare the gitea namespace (stage 1 of #878)' (#905) from feat/gitea-namespace-phase1 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#905
2026-09-23 05:35:18 +00:00
claude-botandClaude Opus 5.5 036399095b feat(gitea): prepare the gitea namespace (stage 1 of #878)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Additive only, nothing moves yet:
- Namespace gitea, declared in infrastructure like woodpecker.
- network-policies/gitea-ns.yaml: baseline (same-ns, monitoring, host,
  ICMP; egress to traefik, kube-dns, wiki-index, world) and
  gitea-allow-traefik for :3000 and :2222.
- openbao rotator egress (flux-git, review-bot) and wiki-index ingress
  also admit gitea in the gitea namespace; the apps entries stay until
  stage 3.
- Velero schedule gitea-daily (07:45) so data-gitea-0 stays backed up
  after the move; apps-daily comment updated (Woodpecker left apps).

Refs #878

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 07:30:36 +02:00
k.renner 0bac0051b2 Merge pull request 'chore(ci): bump ci-claude image to 2.1.280' (#904) from ci-image/claude-2.1.280 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#904
2026-09-23 05:16:03 +00:00
claude-bot 41a65b65a3 chore(ci): bump ci-claude image to 2.1.280
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
2026-09-23 05:13:16 +00:00
k.renner a530f7c8e6 Merge pull request 'fix(ci-claude-refresh): bump divergent pins instead of aborting' (#903) from fix/ci-claude-refresh-mixed-pins into master
ci/woodpecker/push/ci-claude-refresh Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#903
2026-09-23 05:12:40 +00:00
claude-botandClaude Opus 5.5 8fff4b4012 fix(ci-claude-refresh): bump divergent pins instead of aborting
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
decide required every consumer to pin the same ci-claude tag and aborted
otherwise (pipeline 10497: [2.1.250 2.1.278]), which needed a hand fix
before the latest version could roll out. Record all distinct tags, bump
whenever any is off latest, and rewrite every version-pinned tag to the
new one with a validated x.y.z version.

Merging this is itself a push touching the pipeline file, so the refresh
runs, builds the latest image, and opens the bump PR.

Refs #889

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 07:06:56 +02:00
k.renner 0ada3df260 Merge pull request 'fix(ci-claude-refresh): strip CR/LF from the bump token before use' (#901) from fix/ci-claude-bump-token-newline into master
ci/woodpecker/push/ci-claude-refresh Pipeline failed
Reviewed-on: k.renner/homelab#901
2026-09-23 05:03:19 +00:00
claude-botandClaude Opus 5.5 2144554cfc fix(ci-claude-refresh): strip CR/LF from the bump token before use
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Pipeline 10186 failed in bump-pr with 'url contains a newline in its
username component': ci_claude_bump_bot_token carries a trailing newline,
so the push URL (and the PR API auth header) broke. Strip CR/LF once at
the top of the step and fail fast if the token ends up empty. This makes
the step independent of how the secret value was written.

Closes #889

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 06:59:15 +02:00
k.renner dba810b87b Merge pull request 'fix(paperless): raise the memory limit to 4Gi' (#900) from worktree-fix-paperless-memory-limit into master
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/wiki-maintenance Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#900
2026-09-21 19:29:08 +00:00
claude-botandClaude Fable 5.1 f8aa65362b fix(paperless): cap OCR at 5 threads per worker
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Unset, paperless derives cpu_count / TASK_WORKERS, i.e. 20 parallel
ocrmypdf workers on k8s-1. Memory during OCR scales with that count, so
the limit alone is not enough headroom for large scans.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 21:26:35 +02:00
claude-botandClaude Fable 5.1 0ce0391bb1 fix(paperless): raise the memory limit to 4Gi
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
An 18 MB scan from the ADS-1700W has kept the paperless container in an
OOM crash loop since 2026-09-21 10:21: ocrmypdf runs with 20 workers
(paperless default, cpu_count / TASK_WORKERS on a 20-core node) and blows
the 2Gi limit ~40s into OCR. The consume task is re-queued on every
restart, so the same file kills every attempt. 117 restarts so far, web
UI down as a side effect since worker and webserver share the container.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 21:24:36 +02:00
security-bot 4611ea3578 chore(image-automation): bump redis:8.10.2-alpine
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-09-21 18:52:58 +00:00
k.renner 63701fbebb Merge pull request 'ci(wiki-maintenance): verify the PR merged instead of trusting the merge status' (#895) from fix/wiki-maintenance-merge-verify into master
ci/woodpecker/manual/wiki-maintenance Pipeline was successful
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#895
2026-09-21 08:49:10 +00:00
k.renner e8a047eb14 Merge pull request 'ci(pick-model): newest among comparable candidates, policy in code' (#894) from feat/pick-model-newest-among-comparable into master
Reviewed-on: k.renner/homelab#894
2026-09-21 08:48:58 +00:00
claude-botandClaude Fable 5.1 0279210595 ci(wiki-maintenance): verify the PR merged instead of trusting the merge status
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Run 10254: POST .../merge with delete_branch_after_merge returned
HTTP 500 although PR #2 on k.renner/notes was merged, so the step
failed and left the branch behind. The merge call no longer asks Gitea
to delete the branch; the step reads the PR back and requires
merged=true, then deletes the branch as a separate best-effort call.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 10:06:23 +02:00
claude-botandClaude Fable 5.1 25f09e9b52 ci(pick-model): newest among comparable candidates, policy in code
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
First live pick after #893 chose claude-sonnet-4-5 at 0.23 over
claude-sonnet-4-6: same-family models split the probability for a
mid-weight task, and "prefer newer" is a policy, not a judgment. New
scripts/ci/choose-model.sh applies it to the distribution: among
candidates within CHOOSE_MODEL_MARGIN (0.7) of the best probability the
most recently released wins; a clear best stays; NONE yields nothing.
The picker logs the top probabilities. Four policy tests added.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 09:59:40 +02:00
k.renner 82f2825d3f Merge pull request 'ci(pick-model): choose the model with one TypeSafe Choice, claude as fallback' (#893) from feat/pick-model-typesafe into master
ci/woodpecker/manual/wiki-maintenance Pipeline failed
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#893
2026-09-21 07:56:11 +00:00
claude-botandClaude Fable 5.1 0fec080b4d ci(pick-model): choose the model with one TypeSafe Choice, claude as fallback
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The picker ran a full claude -p in plan mode per selection and parsed
JSON out of prose. With TYPESAFE_API_KEY set it now sends the task and
the catalog rows as state and asks one Choice over the model ids, each
with its catalog row as criterion plus NONE; the id is validated against
the pristine catalog as before. No key or a failed request falls through
to the claude picker unchanged.

Measured on the two-model fixture (Opus 5 / Haiku 4.5): issue-fix and
security-fix -> Opus 0.99, alert triage -> Opus 0.62, wiki audit ->
Haiku 0.94, update audit -> Haiku 0.46, distill -> Haiku 0.63; about a
second per pick. Dry-run test added to test-model-selection.sh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 09:42:39 +02:00
k.renner 7424c6aefd Merge pull request 'ci(wiki-maintenance): rebase onto main before pushing, retry three times' (#892) from fix/wiki-maintenance-rebase-push into master
ci/woodpecker/manual/wiki-maintenance Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#892
2026-09-21 06:41:40 +00:00
claude-botandClaude Fable 5.1 4e3fb88309 ci(wiki-maintenance): push a branch and merge it through Gitea
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Instead of rebasing onto a moving main: the run's changes go to
wiki-maintenance/<date>-<run>, a PR is opened and merged via the API
(Do: merge, branch deleted afterwards). A merge conflict leaves the PR
open for a human and fails the run. Fields checked against the
instance's swagger.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 08:40:47 +02:00
claude-botandClaude Fable 5.1 c8c5ce9015 ci(wiki-maintenance): rebase onto main before pushing, retry three times
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Run 10241: distill and audit succeeded (5 sources, 8 new + 6 updated
notes), the push was rejected non-fast-forward because the workstation
pushed the vault while the agents ran. The push step now fetches,
rebases and retries; a real conflict still fails the run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 08:36:58 +02:00
k.renner 293f51b914 Merge pull request 'ci(wiki-maintenance): strip CR/LF from secrets before use' (#891) from fix/wiki-maintenance-strip-secrets into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
ci/woodpecker/manual/wiki-maintenance Pipeline failed
Reviewed-on: k.renner/homelab#891
2026-09-21 06:22:44 +00:00
claude-botandClaude Fable 5.1 70287db150 ci(wiki-maintenance): allow hand-started runs via the manual event
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
A deployment-event run (10239) errored before its first step: the
typesafe_api_key secret is scoped to cron and manual only. Manual
pipelines (woodpecker pipeline create) are the way to start this by
hand; the deployment trigger is dropped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 08:20:36 +02:00
claude-botandClaude Fable 5.1 efedba1ff0 ci(wiki-maintenance): strip CR/LF from secrets before use
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was canceled
First cron run (10220) failed in checkout-notes with "could not read
Username": the wiki_bot_token secret had been set from a file with a
trailing newline, which git carried into the extraheader. The secret
is rewritten without it; the pipeline now strips CR/LF from the token
and the index credential regardless of how they were entered.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 08:19:12 +02:00
security-bot f35a2d5985 chore(image-automation): bump ghcr.io/paperless-ngx/paperless-ngx:3.2.1
ci/woodpecker/cron/wiki-maintenance Pipeline failed
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/update-audit Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-09-20 21:51:31 +00:00
k.renner 19b641a2b6 Merge pull request 'ci(wiki-maintenance): run on ci-claude 2.1.278' (#890) from fix/wiki-maintenance-image-2.1.278 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#890
2026-09-20 20:46:19 +00:00
claude-botandClaude Fable 5.1 e422e6a13d ci(wiki-maintenance): run on ci-claude 2.1.278
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The refresh triggered by #888 built 2.1.278 (the first ci-claude with
python3) rather than rebuilding 2.1.250, and its bump-pr step failed on
a newline in the push token secret (#889), so the consumer bump never
came. This pipeline needs python for wiki_ask.py, so it moves ahead of
the general bump.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 22:45:02 +02:00
k.renner f4d590c40d Merge pull request 'feat(ci): nightly wiki maintenance for the Notes vault as a Woodpecker cron' (#888) from feat/wiki-maintenance-ci into master
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/push/ci-claude-refresh Pipeline failed
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#888
2026-09-20 20:41:58 +00:00
claude-botandClaude Fable 5.1 791cb6e524 feat(ci): nightly wiki maintenance for the Notes vault as a Woodpecker cron
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Replaces the workstation's wiki-maintenance.timer. The pipeline clones
k.renner/notes, runs wiki-distill over raw/ and wiki-audit over wiki/
as two separate `claude --agent` runs (no background tasks in print
mode, which killed the workstation job at the 600 s ceiling), then
commits and pushes as Claude Bot; the push webhook refreshes wiki-index.

The agent definitions now live in the vault (.claude/agents/), so CI
and workstation share them. The agents and their retrieval scripts
address the vault as ~/Notes; each agent step links $HOME/Notes to the
checkout. ci-claude gains python3 and py3-yaml for wiki_ask.py.

Secrets on the repo: claude_code_oauth_token (existing), typesafe_api_key,
wiki_bot_token (claude-bot, write:repository), wiki_index_basic_auth.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 22:37:59 +02:00
k.renner 3749df798e Merge pull request 'ci(wiki-index): per-commit image tags, deployment pinned to one' (#886) from fix/wiki-index-commit-tags into master
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
Reviewed-on: k.renner/homelab#886
2026-09-20 09:00:25 +00:00
claude-botandClaude Fable 5.1 50b4cbeb2b fix(wiki-index): pin the deployment to the per-commit tag
ci/woodpecker/pr/ci-manifests-test Pipeline failed
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
wiki-index:2026-09-20-4b529cd1 is the build of the previous commit,
i.e. master's code with the paging fix and /ready. Manifest-only
commit, so it does not trigger another build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:55:57 +02:00
claude-botandClaude Fable 5.1 4b529cd15c ci(wiki-index): also tag the image per commit
ci/woodpecker/push/build-ci-images Pipeline was successful
Three builds on 2026-09-20 (two branch pushes, the merge) rewrote
wiki-index:2026-09-20 in an order the digest pin could not follow: the
pinned digest ran the paging-only build without /ready and crash-looped
on its startup probe. A <date>-<sha8> tag is immutable per commit and
lets the deployment pin by tag without a registry lookup.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:55:38 +02:00
k.renner 53a9348254 Merge pull request 'fix(wiki-index): pin the image to the #882 build' (#885) from fix/wiki-index-pin-digest-2 into master
Reviewed-on: k.renner/homelab#885
2026-09-20 08:49:45 +00:00
claude-botandClaude Fable 5.1 ce06d4e7da fix(wiki-index): pin the image to the #882 build
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Same dated tag again; the pod holds the #879 build. Pinning the new
digest (paging fix, /ready readiness) rolls it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:49:13 +02:00
k.renner e716632c5c Merge pull request 'fix(wiki-index): stop tree paging on the short last page' (#882) from fix/wiki-index-tree-paging into master
ci/woodpecker/push/build-ci-images Pipeline was successful
Reviewed-on: k.renner/homelab#882
2026-09-20 08:46:35 +00:00
claude-botandClaude Fable 5.1 9736656fed fix(wiki-index): liveness on /health, readiness on /ready
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
/health answered 503 until an index existed and fed startup and
liveness, so a live process without an index counted as dead and was
restarted every 15 min, and a later failed sync would have pulled a pod
with a good index out of the Service. /health is now always 200
(process serves); /ready is 200 once an index exists and stays so
across failed syncs, which keep the last good index. Probes follow:
startup 60 s on /health, readiness on /ready, liveness on /health.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:45:50 +02:00
claude-botandClaude Fable 5.1 d4e5029ebd fix(wiki-index): stop tree paging on the short last page
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was canceled
The notes repo tree has 3192 entries. Gitea reports truncated=true on
every page including the last one (page 4, 192 entries) and answers
"tree": null one page further, which the sync hit:
"sync failed: 'NoneType' object is not iterable". Paging now stops on a
short page or when total_count is reached, and a null tree is empty.
Unit test with the observed page shapes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:44:10 +02:00
k.renner 0fb5ff3fb6 Merge pull request 'fix(wiki-index): pin the image to the rebuilt digest' (#881) from fix/wiki-index-pin-digest into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#881
2026-09-20 08:40:41 +00:00
claude-botandClaude Fable 5.1 58e4c25729 fix(wiki-index): pin the image to the rebuilt digest
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
#879 rebuilt wiki-index under the same dated tag 2026-09-20; the node
still holds the first build (sha256:085438d8) and IfNotPresent would
not re-pull. Pinning tag@digest rolls the exact new image.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:38:45 +02:00
k.renner a3ca6c6065 Merge pull request 'fix(wiki-index): strip OpenBao values, retry a failed sync after 60 s' (#879) from fix/wiki-index-strip-credentials into master
ci/woodpecker/push/build-ci-images Pipeline was successful
Reviewed-on: k.renner/homelab#879
2026-09-20 08:36:03 +00:00
claude-botandClaude Fable 5.1 816f161591 fix(wiki-index): strip OpenBao values, retry a failed sync after 60 s
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
`bao kv put key=@file` stores the file's trailing newline; the first
sync after #875 failed with "Invalid header value b'token ...\n'" and a
webhook signed with the newline-free secret would never have verified.
Credentials are stripped on read. A failed sync now retries after 60 s
instead of waiting the full RESYNC_SECONDS.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:29:43 +02:00
k.renner 5f86bdde64 Merge pull request 'fix(netpol): let gitea deliver the notes push webhook to wiki-index' (#877) from fix/gitea-webhook-egress-wiki-index into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#877
2026-09-20 08:24:40 +00:00
claude-botandClaude Fable 5.1 814c5b88fa fix(netpol): let gitea deliver the notes push webhook to wiki-index
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The apps namespace is egress default-deny; gitea's webhook to
wiki-index.ai.svc.cluster.local:8080 was dropped (a test delivery
produced no request at the service). Namespace-wide egress rule scoped
to the wiki-index pod on 8080, the mosquitto pattern; wiki-index's own
ingress policy already admits only the gitea pod from apps.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:21:18 +02:00
k.renner 0420be0a21 Merge pull request 'fix(openbao): admit wiki-index to OpenBao on 8200' (#875) from fix/wiki-index-openbao-ingress into master
Reviewed-on: k.renner/homelab#875
2026-09-20 08:18:02 +00:00
claude-botandClaude Fable 5.1 7da72b34cf fix(openbao): admit wiki-index to OpenBao on 8200
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The openbao namespace policy admits only its own pods, Traefik and
External Secrets. wiki-index (ns ai) logs in through the Kubernetes auth
role on every sync and timed out on the first attempt:
"POST .../v1/auth/kubernetes/login: timed out". Pod-scoped ingress rule
for it, mirroring the external-secrets entry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:14:47 +02:00
k.renner c36c290b13 Merge pull request 'feat(ci): wiki-index image — embedding index over the Notes repo' (#873) from feat/wiki-index-image into master
ci/woodpecker/push/build-ci-images Pipeline failed
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#873
2026-09-20 08:11:32 +00:00
k.renner 8fb7430fb7 Merge pull request 'feat(ai): deploy wiki-index, the embedding index service for the Notes repo' (#874) from feat/wiki-index-deploy into master
Reviewed-on: k.renner/homelab#874
2026-09-20 08:11:21 +00:00
claude-botandClaude Fable 5.1 d82b96a751 fix(wiki-index): notes repository branch is main
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:09:43 +02:00
claude-botandClaude Fable 5.1 463cfa7bde feat(ai): deploy wiki-index, the embedding index service for the Notes repo
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Deployment + Service + PVC + ServiceAccount in namespace ai, LAN-only
Ingress wiki-index.renner.uno behind Traefik BasicAuth (ExternalSecret
from OpenBao, same shape as wiki-embed), Flux Kustomization depending on
llama-server (wiki-embed is the embedding backend), pod-scoped
CiliumNetworkPolicy admitting Traefik and Gitea (push webhook) on 8080
and allowing egress to OpenBao 8200, and the OpenBao policy for the
service's Kubernetes auth role.

Image: gitea.renner.uno/gitea_admin/wiki-index:2026-09-20, built by the
step added in the previous PR; the tag is the build date.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 08:47:26 +02:00
claude-botandClaude Fable 5.1 48ba19064a feat(ci): wiki-index image — embedding index over the Notes repo
ci/woodpecker/push/build-ci-images Pipeline failed
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Runtime image for a wiki-index service in namespace ai (manifests follow
once the image exists): standard-library Python that mirrors the notes
repository's wiki/ from Gitea into a bge-m3 index and serves POST /rank
to ~/Notes/bin/wiki_rank.py.

Sync fetches the branch head and tree via the Gitea API, embeds only notes
whose blob sha changed (through the in-cluster wiki-embed), drops notes
that left the tree, and persists the index on a PVC. It runs at start, on
every push webhook (HMAC-SHA256 verified) and every RESYNC_SECONDS. Gitea
token and webhook secret come from OpenBao through the Kubernetes auth
role on every use. Five unit tests cover parsing, incremental sync,
persistence, ranking and error handling.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 08:42:03 +02:00
k.renner 761bf9ba1e Merge pull request 'fix(eso): allow namespace ai on the openbao ClusterSecretStore' (#872) from fix/eso-allow-ai into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#872
2026-09-20 05:48:21 +00:00
claude-botandClaude Fable 5.1 7b446194b3 fix(eso): allow namespace ai on the openbao ClusterSecretStore
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The wiki-embed-basic-auth ExternalSecret from #871 never synced:
"using cluster store openbao is not allowed from namespace ai: denied by
spec.condition". The store's namespace allowlist predates the ai
namespace. Adding it, sorted, as the manifest's own maintenance note
prescribes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 07:47:43 +02:00
k.renner 9eaafeb92f Merge pull request 'feat(ai): dedicated wiki-embed endpoint for the Notes ranker' (#871) from feat/wiki-embed into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#871
2026-09-20 05:38:26 +00:00
66 changed files with 1707 additions and 271 deletions
+1 -1
View File
@@ -101,7 +101,7 @@ steps:
bash scripts/ci/make-diag-kubeconfig.sh
- name: alert-triage-agent
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
+37 -6
View File
@@ -350,17 +350,17 @@ steps:
--destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
--destination gitea.renner.uno/gitea_admin/heating-can:latest
# netbird-proxy-services: the controller that reconciles NetBird reverse-proxy
# netbird-service-sync: the controller that reconciles NetBird reverse-proxy
# services (public Ingress hosts -> TLS passthrough -> Traefik ClusterIP).
# Runtime artifact like heating-can; same registry so the audit tracks it.
- name: build-netbird-proxy-services
- name: build-netbird-service-sync
image: registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug
directory: ci/images/netbird-proxy-services
directory: ci/images/netbird-service-sync
depends_on: []
when:
- event: deployment
- event: push
path: ["ci/images/netbird-proxy-services/**", ".woodpecker/build-ci-images.yml"]
path: ["ci/images/netbird-service-sync/**", ".woodpecker/build-ci-images.yml"]
environment:
GITEA_USER: gitea_admin
GITEA_TOKEN:
@@ -374,8 +374,39 @@ steps:
/kaniko/executor \
--context . \
--dockerfile Containerfile \
--destination gitea.renner.uno/gitea_admin/netbird-proxy-services:$TAG \
--destination gitea.renner.uno/gitea_admin/netbird-proxy-services:latest
--destination gitea.renner.uno/gitea_admin/netbird-service-sync:$TAG \
--destination gitea.renner.uno/gitea_admin/netbird-service-sync:latest
# wiki-index: embedding index over the Notes repo's wiki/, fed from Gitea and
# queried by ~/Notes/bin/wiki_rank.py. Runtime artifact like netbird-service-sync.
- name: build-wiki-index
image: registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug
directory: ci/images/wiki-index
depends_on: []
when:
- event: deployment
- event: push
path: ["ci/images/wiki-index/**", ".woodpecker/build-ci-images.yml"]
environment:
GITEA_USER: gitea_admin
GITEA_TOKEN:
from_secret: ci_images_token
commands:
- mkdir -p /kaniko/.docker
- AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
- printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
- TAG=$(date +%Y-%m-%d)
# Also a per-commit tag: several builds a day (branch pushes and the
# merge) all rewrite the dated tag, and the deployment pin raced them
# on 2026-09-20. The commit tag is immutable and needs no digest lookup.
- SHORT=$(printf '%s' "$CI_COMMIT_SHA" | cut -c1-8)
- |
/kaniko/executor \
--context . \
--dockerfile Containerfile \
--destination gitea.renner.uno/gitea_admin/wiki-index:$TAG \
--destination gitea.renner.uno/gitea_admin/wiki-index:$TAG-$SHORT \
--destination gitea.renner.uno/gitea_admin/wiki-index:latest
# bird: BGP daemon for the pod-network mesh over pod0 (#576). Runtime
# artifact like heating-can; same registry so the audit tracks it.
+17 -21
View File
@@ -14,8 +14,8 @@
# and ci-claude:latest. The Containerfile makes this layer
# byte-reproducible, so rebuilding an unchanged version dedups
# in the registry instead of orphaning a fresh ~260 MB copy.
# 3. bump-pr — (only when the version actually changed) rewrite the 5
# consumer pins to ci-claude:<version> and open a PR. A human
# 3. bump-pr — (only when any pin is off the latest version) rewrite every
# consumer pin to ci-claude:<version> and open a PR. A human
# reviews and merges it; nothing auto-merges.
#
# TAG SCHEME — deliberately version tags (ci-claude:<claude-version>), NOT the dated
@@ -107,31 +107,22 @@ steps:
[0-9]*.[0-9]*.[0-9]*) : ;;
*) echo "npm returned an implausible version '$$NEW' — aborting."; exit 1 ;;
esac
# The tag every consumer currently references. They are kept identical,
# so this must resolve to EXACTLY ONE non-empty tag; anything else
# (no match, or a partial/disagreeing set) aborts rather than risk a
# corrupting sed later. Counting with `grep -c .` (non-empty lines)
# instead of `wc -l` is deliberate: an empty result yields 0 here but
# 1 under wc -l, which would slip past the guard.
CUR=$$(grep -hoE 'ci-claude:[0-9A-Za-z._-]+' $$CONSUMERS | sed 's/^ci-claude://' | sort -u)
NTAGS=$$(printf '%s' "$$CUR" | grep -c . || true)
if [ -z "$$CUR" ] || [ "$$NTAGS" -ne 1 ]; then
echo "Expected exactly one ci-claude tag across the consumers, got [$$CUR]."
echo "Refusing to guess — fix the pins by hand."; exit 1
fi
echo "latest=$$NEW current=$$CUR event=$$CI_PIPELINE_EVENT"
# Distinct tags the consumers currently reference. Divergent pins are
# normal (a pipeline bumped on its own); the bump rewrites all of them.
CUR=$$(grep -hoE 'ci-claude:[0-9][0-9A-Za-z._-]*' $$CONSUMERS | sed 's/^ci-claude://' | sort -u | tr '\n' ' ' | sed 's/ $$//')
[ -n "$$CUR" ] || { echo "No ci-claude pin found in the consumers — aborting."; exit 1; }
echo "latest=$$NEW current=[$$CUR] event=$$CI_PIPELINE_EVENT"
printf '%s' "$$NEW" > "$$CI_WORKSPACE/.ci-refresh/version"
printf '%s' "$$CUR" > "$$CI_WORKSPACE/.ci-refresh/current"
# Build when the version changed, or on a push (Containerfile edit must
# ship even if the version is unchanged). Bump the pins only when the
# version the consumers reference is not the latest.
# Build when any pin is off latest, or on a push (Containerfile edit must
# ship even if the version is unchanged). Bump when any pin is off latest.
if [ "$$CUR" != "$$NEW" ] || [ "$$CI_PIPELINE_EVENT" = "push" ]; then
touch "$$CI_WORKSPACE/.ci-refresh/do-build"; echo "will build $$NEW"
else
echo "already on $$NEW — nothing to build"
fi
if [ "$$CUR" != "$$NEW" ]; then
touch "$$CI_WORKSPACE/.ci-refresh/do-bump"; echo "will open bump PR $$CUR -> $$NEW"
touch "$$CI_WORKSPACE/.ci-refresh/do-bump"; echo "will open bump PR [$$CUR] -> $$NEW"
else
echo "consumers already pin $$NEW — no PR"
fi
@@ -175,6 +166,9 @@ steps:
- |
[ -f "$$CI_WORKSPACE/.ci-refresh/do-bump" ] || { echo "no version change — no PR"; exit 0; }
set -eu
# Secret values can carry a trailing newline; it breaks the push URL and the auth header.
PUSH_TOKEN=$$(printf '%s' "$$PUSH_TOKEN" | tr -d '\r\n')
[ -n "$$PUSH_TOKEN" ] || { echo "PUSH_TOKEN is empty — aborting"; exit 1; }
NEW=$$(cat "$$CI_WORKSPACE/.ci-refresh/version")
CUR=$$(cat "$$CI_WORKSPACE/.ci-refresh/current")
# Both were validated non-empty by the decide step; re-assert before a
@@ -190,9 +184,11 @@ steps:
# is not a maintained list.
CONSUMERS=$$(grep -rlE 'ci-claude:[0-9]' .woodpecker)
[ -n "$$CONSUMERS" ] || { echo "No ci-claude consumers found — refusing to continue."; exit 1; }
# Exact-string swap of the current tag for the new one, consumers only.
# NEW goes into a sed replacement: allow only a plain x.y.z version.
printf '%s' "$$NEW" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$$' || { echo "refusing non-x.y.z version '$$NEW'"; exit 1; }
# Every version-pinned tag becomes NEW, whatever it was; consumers only.
for f in $$CONSUMERS; do
sed -i "s#ci-claude:$$CUR#ci-claude:$$NEW#g" "$$f"
sed -i "s#ci-claude:[0-9][0-9A-Za-z._-]*#ci-claude:$$NEW#g" "$$f"
done
# Author is claude-bot (the PAT's account); the commit subject makes
# the automated-bump nature clear.
+1 -1
View File
@@ -17,6 +17,6 @@ when:
steps:
- name: model-selection-tests
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
commands:
- sh scripts/ci/test-model-selection.sh
+1 -1
View File
@@ -26,7 +26,7 @@ steps:
# fetch-models.sh response mapping. No cluster, no secrets, no network — the
# ci-claude image just provides sh + jq. Keeps the flag-injection regression
# test from rotting (it exercises the exact injection shapes).
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
commands:
- sh scripts/ci/test-model-selection.sh
+1 -1
View File
@@ -62,7 +62,7 @@ steps:
- bash scripts/ci/make-diag-kubeconfig.sh
- name: issue-fix-agent
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
+1 -1
View File
@@ -65,7 +65,7 @@ when:
steps:
- name: pr-review-agent
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
+1 -1
View File
@@ -70,7 +70,7 @@ steps:
# this step is skipped — otherwise we'd fork-bomb PRs.
when:
- branch: master
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
+1 -1
View File
@@ -30,7 +30,7 @@ when:
steps:
- name: update-audit-agent
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.250
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
+140
View File
@@ -0,0 +1,140 @@
# Nightly wiki maintenance for the Notes vault (k.renner/notes), replacing the
# workstation's wiki-maintenance.timer: distill the raw/ backlog into wiki/,
# then audit wiki/. Each agent is its own `claude` run, not a subagent of an
# orchestrator, so nothing depends on background tasks in print mode (the
# workstation job died on their 600 s ceiling). The pipeline commits and pushes
# as Claude Bot; the push webhook then refreshes the wiki-index service.
#
# The agents and their retrieval scripts address the vault as ~/Notes, so each
# agent step links $HOME/Notes to the checkout instead of rewriting them.
when:
- event: cron
cron: wiki-maintenance
# Hand-started runs: `woodpecker pipeline create k.renner/homelab --branch master`.
# The typesafe_api_key secret is scoped to cron and manual, not deployment.
- event: manual
steps:
- name: checkout-notes
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
pull: true
environment:
WIKI_BOT_TOKEN:
from_secret: wiki_bot_token
commands:
- |
set -eu
# Token only on the wire, never in .git/config. Secrets set from files carry
# a trailing newline (run 10220: "could not read Username"); strip CR/LF.
TOKEN=$$(printf '%s' "$$WIKI_BOT_TOKEN" | tr -d '\r\n')
git -c http.extraheader="AUTHORIZATION: token $$TOKEN" \
clone --depth 50 --branch main https://gitea.renner.uno/k.renner/notes.git notes
echo "raw backlog: $$(ls notes/raw/*.md | grep -v _index.md | wc -l) file(s)"
- name: distill
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
pull: true
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
TYPESAFE_API_KEY:
from_secret: typesafe_api_key
WIKI_INDEX_BASIC_AUTH:
from_secret: wiki_index_basic_auth
commands:
- |
set -eu
ln -sfn "$$PWD/notes" "$$HOME/Notes"
mkdir -p "$$HOME/.config/wiki-index"
printf '%s' "$$WIKI_INDEX_BASIC_AUTH" | tr -d '\r\n' > "$$HOME/.config/wiki-index/basic-auth"
chmod 600 "$$HOME/.config/wiki-index/basic-auth"
"$$HOME/Notes/bin/wiki_rank.py" --probe-index
- |
set -eu
MODEL=$$(sh scripts/ci/pick-model.sh "distill raw source notes into atomic wiki notes: read sources, check the wiki for existing coverage, write or merge notes; careful writing, moderate reasoning")
if [ -n "$$MODEL" ]; then echo "distill: selected model $$MODEL"; else echo "distill: using the CLI default model"; fi
cd notes
cat ../ci/agents/wiki-maintenance/distill.md | claude $${MODEL:+--model $$MODEL} \
--agent wiki-distill \
--permission-mode acceptEdits \
--allowedTools "Read,Edit,Write,Glob,Grep,Bash(/root/Notes/bin/wiki_rank.py:*),Bash(/root/Notes/bin/wiki_ask.py:*),Bash(mv /root/Notes/raw/:*),Bash(mkdir -p /root/Notes/:*)" \
--settings ../ci/agents/wiki-maintenance/settings.json \
-p
- name: audit
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
pull: true
environment:
CLAUDE_CODE_OAUTH_TOKEN:
from_secret: claude_code_oauth_token
TYPESAFE_API_KEY:
from_secret: typesafe_api_key
WIKI_INDEX_BASIC_AUTH:
from_secret: wiki_index_basic_auth
commands:
- |
set -eu
ln -sfn "$$PWD/notes" "$$HOME/Notes"
mkdir -p "$$HOME/.config/wiki-index"
printf '%s' "$$WIKI_INDEX_BASIC_AUTH" | tr -d '\r\n' > "$$HOME/.config/wiki-index/basic-auth"
chmod 600 "$$HOME/.config/wiki-index/basic-auth"
- |
set -eu
MODEL=$$(sh scripts/ci/pick-model.sh "structural audit of ~1700 markdown notes: frontmatter completeness, broken links, orphans; mechanical checks, little reasoning")
if [ -n "$$MODEL" ]; then echo "audit: selected model $$MODEL"; else echo "audit: using the CLI default model"; fi
cd notes
cat ../ci/agents/wiki-maintenance/audit.md | claude $${MODEL:+--model $$MODEL} \
--agent wiki-audit \
--permission-mode acceptEdits \
--allowedTools "Read,Write,Glob,Grep,Bash(/root/Notes/bin/wiki_rank.py:*),Bash(/root/Notes/bin/wiki_ask.py:*)" \
--settings ../ci/agents/wiki-maintenance/settings.json \
-p
- name: push-notes
image: gitea.renner.uno/gitea_admin/ci-claude:2.1.280
pull: true
environment:
WIKI_BOT_TOKEN:
from_secret: wiki_bot_token
commands:
- |
set -eu
cd notes
git config user.email "claude-bot@renner.uno"
git config user.name "Claude Bot"
if git diff --quiet && git diff --cached --quiet && [ -z "$$(git ls-files --others --exclude-standard)" ]; then
echo "wiki-maintenance: no changes"
exit 0
fi
# The workstation pushes main on its own (notes-push.path), so main moves
# while the agents run (run 10241: non-fast-forward). The run's work goes
# to a branch and a PR that this step merges through Gitea; a conflict
# leaves the PR open for a human and fails the run.
BRANCH="wiki-maintenance/$$(date -u +%F)-$$CI_PIPELINE_NUMBER"
git checkout -q -b "$$BRANCH"
git add -A
git commit -q -m "wiki maintenance $$(date -u +%F): distill + audit (CI run $$CI_PIPELINE_NUMBER)"
TOKEN=$$(printf '%s' "$$WIKI_BOT_TOKEN" | tr -d '\r\n')
git -c http.extraheader="AUTHORIZATION: token $$TOKEN" push origin "HEAD:$$BRANCH"
API="https://gitea.renner.uno/api/v1/repos/k.renner/notes"
jq -n --arg title "wiki maintenance $$(date -u +%F) (CI run $$CI_PIPELINE_NUMBER)" \
--arg body "Nightly distill + audit. Merged by the pipeline; open only if the merge failed." \
--arg head "$$BRANCH" '{title: $$title, body: $$body, head: $$head, base: "main"}' > /tmp/pr.json
code=$$(curl -s -o /tmp/pr-resp.json -w '%{http_code}' -X POST \
-H "Authorization: token $$TOKEN" -H "Content-Type: application/json" --data @/tmp/pr.json "$$API/pulls")
echo "POST pull -> HTTP $$code"
[ "$$code" -ge 200 ] && [ "$$code" -lt 300 ] || { cat /tmp/pr-resp.json; exit 1; }
NUM=$$(jq -r '.number' /tmp/pr-resp.json)
# Merge without delete_branch_after_merge: run 10254 got HTTP 500 from that
# call although the merge itself had landed. Verify the PR state instead of
# trusting the status code, then drop the branch as a separate, best-effort step.
code=$$(curl -s -o /tmp/merge-resp.json -w '%{http_code}' -X POST \
-H "Authorization: token $$TOKEN" -H "Content-Type: application/json" \
--data '{"Do":"merge"}' "$$API/pulls/$$NUM/merge")
echo "POST merge #$$NUM -> HTTP $$code"
MERGED=$$(curl -s -H "Authorization: token $$TOKEN" "$$API/pulls/$$NUM" | jq -r '.merged')
echo "PR #$$NUM merged: $$MERGED"
[ "$$MERGED" = "true" ] || { cat /tmp/merge-resp.json; echo "PR #$$NUM left open"; exit 1; }
code=$$(curl -s -o /tmp/del-resp.json -w '%{http_code}' -X DELETE -H "Authorization: token $$TOKEN" \
"$$API/branches/$$(printf '%s' "$$BRANCH" | sed 's|/|%2F|g')")
echo "DELETE branch $$BRANCH -> HTTP $$code (best effort)"
+5
View File
@@ -0,0 +1,5 @@
You are running as the scheduled wiki-maintenance job in CI. The Notes vault is checked out at ~/Notes (the repository k.renner/notes).
Run the standard audit pass over ~/Notes/wiki/ and write the dated report to ~/Notes/output/. Do not modify any wiki note. Do not commit or push; the pipeline does that.
End with two lines: notes scanned, and issues found by category.
+7
View File
@@ -0,0 +1,7 @@
You are running as the scheduled wiki-maintenance job in CI. The Notes vault is checked out at ~/Notes (the repository k.renner/notes). Do exactly this, nothing else:
Glob ~/Notes/raw/*.md (top level only, never ~/Notes/raw/_processed/). Distill every file you find into ~/Notes/wiki/ following your process, including the ranker check for every concept. Move each fully processed source to ~/Notes/raw/_processed/. If raw/ has no top-level files besides _index.md, say so and stop.
Do not commit or push; the pipeline does that after you finish. Do not touch anything outside ~/Notes/wiki/ and ~/Notes/raw/.
End with one line per file: which wiki notes it produced or merged into, or "nothing distillable".
+37
View File
@@ -0,0 +1,37 @@
{
"//": "Permission scope for the wiki-maintenance agents (wiki-distill, wiki-audit) in CI.",
"//2": "The step's environment holds a Gitea push token, a TypeSafe key and the wiki-index",
"//3": "client credential, so the Read tool is kept away from /proc (env dumps), /root outside",
"//4": "the vault symlink target, /etc and the checkout's .git/. The vault itself is the work.",
"//5": "Mirrors ci/agents/issue-fix/settings.json; absolute globs because the vault is reached",
"//6": "through the ~/Notes symlink the pipeline creates.",
"permissions": {
"allow": [
"Read(//root/Notes/**)",
"Edit(//root/Notes/wiki/**)",
"Edit(//root/Notes/output/**)",
"Edit(//root/Notes/raw/**)",
"Bash(/root/Notes/bin/wiki_rank.py:*)",
"Bash(/root/Notes/bin/wiki_ask.py:*)",
"Bash(mv /root/Notes/raw/:*)",
"Bash(mkdir -p /root/Notes/:*)",
"Bash(ls:*)",
"Bash(date:*)",
"Bash(wc:*)",
"Bash(head:*)",
"Bash(tail:*)"
],
"deny": [
"Read(//proc/**)",
"Read(//etc/**)",
"Read(//root/.config/**)",
"Read(//root/Notes/.git/**)",
"Read(//root/Notes/_archive/**)",
"Edit(//root/Notes/_archive/**)",
"Bash(rm:*)",
"Bash(git:*)",
"Bash(curl:*)",
"Bash(wget:*)"
]
}
}
+3 -1
View File
@@ -13,7 +13,9 @@ RUN apk add --no-cache \
git \
jq \
openssh-client \
openssl
openssl \
python3 \
py3-yaml
# Install Claude Code, then make the resulting layer byte-stable so that a
# rebuild of the SAME version DEDUPS in the Gitea registry instead of orphaning
@@ -1,4 +1,4 @@
# netbird-proxy-services: reconciles NetBird reverse-proxy services (public
# netbird-service-sync: reconciles NetBird reverse-proxy services (public
# domains -> TLS passthrough -> Traefik ClusterIP) from Ingresses carrying the
# public label. Standard library only, so the image is the interpreter plus one
# file; runs unprivileged as its own user.
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""netbird-proxy-services: reconcile NetBird reverse-proxy services from Ingresses.
"""netbird-service-sync: reconcile NetBird reverse-proxy services from Ingresses.
Source of truth: every Ingress carrying the public label (dns.renner.uno/public).
For each host on such an Ingress the NetBird management gets one reverse-proxy
@@ -30,11 +30,11 @@ import urllib.error
import urllib.parse
import urllib.request
LOG = logging.getLogger("netbird-proxy-services")
LOG = logging.getLogger("netbird-service-sync")
NB = os.environ.get("NB_MANAGEMENT_URL", "https://netbird.renner.uno").rstrip("/")
BAO_ADDR = os.environ.get("BAO_ADDR", "http://openbao.openbao.svc.cluster.local:8200").rstrip("/")
BAO_ROLE = os.environ.get("BAO_ROLE", "netbird-proxy-services")
BAO_ROLE = os.environ.get("BAO_ROLE", "netbird-service-sync")
BAO_PAT_PATH = os.environ.get("BAO_PAT_PATH", "homelab/data/infrastructure/secret-rotators-bootstrap/netbird")
BAO_PAT_KEY = os.environ.get("BAO_PAT_KEY", "pat")
SERVICE_CIDR = os.environ.get("SERVICE_CIDR", "10.96.0.0/12")
@@ -156,7 +156,7 @@ def ensure_network(nb):
if n["name"] == NETWORK_NAME:
return n["id"]
n = nb.post("networks", {"name": NETWORK_NAME,
"description": "managed by netbird-proxy-services; public ingress via TLS passthrough"})
"description": "managed by netbird-service-sync; public ingress via TLS passthrough"})
LOG.info("created network %s (%s)", NETWORK_NAME, n["id"])
return n["id"]
@@ -166,7 +166,7 @@ def ensure_resource(nb, net):
if r.get("address") == SERVICE_CIDR:
return r["id"]
r = nb.post(f"networks/{net}/resources", {"name": "cluster service CIDR", "address": SERVICE_CIDR,
"description": "managed by netbird-proxy-services",
"description": "managed by netbird-service-sync",
"enabled": True, "groups": []})
LOG.info("created subnet resource %s (%s)", SERVICE_CIDR, r["id"])
return r["id"]
+10
View File
@@ -0,0 +1,10 @@
# wiki-index: embedding index over the Notes repo's wiki/, fed from Gitea,
# queried by ~/Notes/bin/wiki_rank.py. Standard library only; runs unprivileged.
FROM python:3.13-slim
RUN useradd --system --no-create-home --shell /usr/sbin/nologin wikiindex
COPY wiki_index.py /opt/wiki_index.py
ENV PYTHONUNBUFFERED=1
USER wikiindex
ENTRYPOINT ["python3", "/opt/wiki_index.py"]
+178
View File
@@ -0,0 +1,178 @@
#!/usr/bin/env python3
"""Unit tests for wiki_index. Run: python3 -m unittest test_wiki_index"""
import hashlib
import hmac
import json
import tempfile
import unittest
from pathlib import Path
import wiki_index as wi
NOTE = """---
title: 'UDM intercepts DNS'
type: wiki
tags: [dns, unifi]
aliases: [udm dns interception, dig 1.1.1.1 wrong]
---
# UDM intercepts DNS
The UDM answers every UDP/53 query itself.
## Details
More.
"""
class ParseTests(unittest.TestCase):
def test_note_text(self):
title, text = wi.note_text("udm-dns", NOTE)
self.assertEqual(title, "UDM intercepts DNS")
self.assertIn("udm dns UDM intercepts DNS udm dns interception", text)
self.assertIn("dns unifi", text)
self.assertTrue(text.endswith("The UDM answers every UDP/53 query itself."))
self.assertLessEqual(len(text), wi.EMBED_CHARS)
def test_credentials_are_stripped(self):
original = wi.http
wi.http = lambda method, url, body=None, headers=None, timeout=60.0: (
{"auth": {"client_token": "t"}} if url.endswith("/login")
else {"data": {"data": {"gitea_token": "abc\n", "webhook_secret": " s3cret\n"}}})
original_sa = wi.SA_TOKEN
try:
with tempfile.TemporaryDirectory() as d:
wi.SA_TOKEN = Path(d) / "token"
wi.SA_TOKEN.write_text("jwt\n")
self.assertEqual(wi.Credentials().fetch(), {"gitea_token": "abc", "webhook_secret": "s3cret"})
finally:
wi.http, wi.SA_TOKEN = original, original_sa
def test_tree_paging_stops_on_short_last_page(self):
# Gitea: truncated=true on every page including the last, "tree": null past it.
pages = {
1: {"tree": [{"path": f"wiki/n{i}.md", "type": "blob", "sha": f"s{i}"} for i in range(1000)], "truncated": True, "total_count": 1192},
2: {"tree": [{"path": f"wiki/n{i}.md", "type": "blob", "sha": f"s{i}"} for i in range(1000, 1192)] + [{"path": "raw/x.md", "type": "blob", "sha": "r"}, {"path": "wiki/sub/y.md", "type": "blob", "sha": "y"}], "truncated": True, "total_count": 1192},
3: {"tree": None, "truncated": False, "total_count": 1192},
}
calls = []
original = wi.http
def fake_http(method, url, body=None, headers=None, timeout=60.0):
page = int(url.rsplit("page=", 1)[1])
calls.append(page)
return pages[page]
wi.http = fake_http
try:
tree = wi.Gitea("t").tree("abc")
finally:
wi.http = original
self.assertEqual(calls, [1, 2])
self.assertEqual(len(tree), 1192)
self.assertNotIn("raw/x.md", tree)
self.assertNotIn("wiki/sub/y.md", tree)
self.assertEqual(tree["wiki/n1191.md"], "s1191")
def test_signature(self):
body = b'{"ref":"refs/heads/master"}'
sig = hmac.new(b"s3cret", body, hashlib.sha256).hexdigest()
self.assertTrue(wi.verify_signature("s3cret", body, sig))
self.assertFalse(wi.verify_signature("other", body, sig))
self.assertFalse(wi.verify_signature("s3cret", body, ""))
class FakeGitea:
def __init__(self, head, tree, files):
self._head, self._tree, self.files = head, tree, files
self.raw_calls = []
def head(self):
return self._head
def tree(self, commit):
return self._tree
def raw(self, path, ref):
self.raw_calls.append(path)
return self.files[path]
class IndexTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.path = Path(self.tmp.name) / "index.json"
self.original_embed = wi.embed
self.embed_calls = []
def fake_embed(texts, batch=2):
self.embed_calls.append(list(texts))
return [wi.unit([1.0, float(len(t) % 7)]) for t in texts]
wi.embed = fake_embed
def tearDown(self):
wi.embed = self.original_embed
self.tmp.cleanup()
def test_sync_is_incremental_and_persists(self):
files = {"wiki/udm-dns.md": NOTE, "wiki/ceph.md": "---\ntitle: Ceph\n---\n# Ceph\n\nOSD.\n", "wiki/_index.md": "---\ntitle: Wiki\n---\n"}
tree = {"wiki/udm-dns.md": "aaa", "wiki/ceph.md": "bbb", "wiki/_index.md": "ccc"}
g = FakeGitea("c1", tree, files)
idx = wi.Index(self.path)
s = idx.sync(g)
self.assertEqual((s["changed"], s["removed"], s["notes"]), (2, 0, 2))
self.assertEqual(set(idx.notes), {"udm-dns", "ceph"})
self.assertEqual(idx.notes["udm-dns"]["title"], "UDM intercepts DNS")
self.assertTrue(self.path.exists())
# head unchanged: nothing fetched
s = idx.sync(g)
self.assertEqual(s.get("skipped"), "head unchanged")
self.assertEqual(len(g.raw_calls), 2)
# one note changed, one removed, new head
g2 = FakeGitea("c2", {"wiki/udm-dns.md": "aaa2"}, {"wiki/udm-dns.md": NOTE.replace("itself", "itself!")})
s = idx.sync(g2)
self.assertEqual((s["changed"], s["removed"], s["notes"]), (1, 1, 1))
self.assertEqual(g2.raw_calls, ["wiki/udm-dns.md"])
# restart loads the persisted index
idx2 = wi.Index(self.path)
self.assertEqual(idx2.commit, "c2")
self.assertEqual(set(idx2.notes), {"udm-dns"})
def test_rank_orders_by_cosine(self):
idx = wi.Index(self.path)
idx.notes = {"a": {"sha": "1", "title": "A", "vector": wi.unit([1.0, 0.0])},
"b": {"sha": "2", "title": "B", "vector": wi.unit([0.0, 1.0])}}
wi.embed = lambda texts, batch=2: [wi.unit([0.2, 1.0])]
rows = idx.rank("q", k=5)
self.assertEqual([r["slug"] for r in rows], ["b", "a"])
self.assertGreater(rows[0]["score"], rows[1]["score"])
def test_status_reports_age_and_errors(self):
idx = wi.Index(self.path)
self.assertFalse(idx.status()["ready"])
g = FakeGitea("c1", {"wiki/a.md": "x"}, {"wiki/a.md": "---\ntitle: A\n---\n# A\n\nt\n"})
idx.sync(g)
st = idx.status()
self.assertTrue(st["ready"])
self.assertEqual(st["notes"], 1)
self.assertEqual(st["commit"], "c1")
class Broken(FakeGitea):
def head(self):
raise wi.ApiError("gitea down")
with self.assertRaises(wi.ApiError):
idx.sync(Broken("c1", {}, {}))
self.assertIn("gitea down", idx.status()["last_error"])
self.assertEqual(idx.status()["notes"], 1) # last good index kept
self.assertTrue(idx.status()["ready"]) # and still served
if __name__ == "__main__":
unittest.main()
+408
View File
@@ -0,0 +1,408 @@
#!/usr/bin/env python3
"""wiki-index: embedding index over the Notes repo's wiki/, fed from Gitea.
Source of truth is the notes repository on Gitea. The index holds, per wiki
note, the git blob sha, title and a bge-m3 vector of slug/title/aliases/tags/
first paragraph. A sync fetches the repo tree, embeds every note whose blob sha
changed (via wiki-embed, in-cluster) and drops notes that left the tree; it
runs at start, on every push webhook from Gitea, and every RESYNC_SECONDS.
HTTP:
POST /rank {"query": str, "k": int} -> {"rows": [{"slug","title","score"}], "index": {...}}
GET /health -> 200 always (liveness) with {"ready", "notes", "commit", "synced_at", "age_s", "syncing", "last_error"}
GET /ready -> 200 when an index exists, else 503 (readiness)
POST /webhook Gitea push event, HMAC-SHA256 in X-Gitea-Signature -> {"queued": true}
Credentials (Gitea token, webhook secret) are read from OpenBao through the
Kubernetes auth role on every use; they never land in a Secret or a log line.
Standard library only.
"""
from __future__ import annotations
import hashlib
import hmac
import json
import logging
import os
import re
import sys
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
LOG = logging.getLogger("wiki-index")
GITEA_URL = os.environ.get("GITEA_URL", "https://gitea.renner.uno").rstrip("/")
NOTES_REPO = os.environ.get("NOTES_REPO", "k.renner/notes")
NOTES_BRANCH = os.environ.get("NOTES_BRANCH", "master")
WIKI_DIR = os.environ.get("WIKI_DIR", "wiki").strip("/")
EMBED_URL = os.environ.get("EMBED_URL", "http://wiki-embed.ai.svc.cluster.local:8080").rstrip("/")
EMBED_MODEL = os.environ.get("EMBED_MODEL", "bge-m3")
EMBED_CHARS = int(os.environ.get("EMBED_CHARS", "1500"))
INDEX_PATH = Path(os.environ.get("INDEX_PATH", "/data/index.json"))
RESYNC_SECONDS = int(os.environ.get("RESYNC_SECONDS", "1800"))
LISTEN_PORT = int(os.environ.get("LISTEN_PORT", "8080"))
BAO_ADDR = os.environ.get("BAO_ADDR", "http://openbao.openbao.svc.cluster.local:8200").rstrip("/")
BAO_ROLE = os.environ.get("BAO_ROLE", "wiki-index")
BAO_SECRET_PATH = os.environ.get("BAO_SECRET_PATH", "homelab/data/apps/ai/wiki-index")
SA_TOKEN = Path(os.environ.get("SA_TOKEN", "/var/run/secrets/kubernetes.io/serviceaccount/token"))
class ApiError(Exception):
pass
# ---------------------------------------------------------------- http helpers
def http(method: str, url: str, body=None, headers=None, timeout: float = 60.0):
data = None
hdrs = dict(headers or {})
if body is not None:
data = json.dumps(body).encode("utf-8")
hdrs.setdefault("Content-Type", "application/json")
req = urllib.request.Request(url, data=data, method=method, headers=hdrs)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read()
ctype = resp.headers.get("Content-Type", "")
except urllib.error.HTTPError as e:
raise ApiError(f"{method} {url}: HTTP {e.code}: {e.read().decode('utf-8', 'replace')[:300]}") from e
except urllib.error.URLError as e:
raise ApiError(f"{method} {url}: {e.reason}") from e
if "json" in ctype:
return json.loads(raw.decode("utf-8"))
return raw
# ---------------------------------------------------------------- credentials
class Credentials:
"""Gitea token and webhook secret from OpenBao, fetched per use."""
def __init__(self, static: dict | None = None):
self.static = static # tests / local runs
def fetch(self) -> dict:
if self.static is not None:
return self.static
jwt = SA_TOKEN.read_text(encoding="utf-8").strip()
login = http("POST", f"{BAO_ADDR}/v1/auth/kubernetes/login", {"role": BAO_ROLE, "jwt": jwt})
token = login["auth"]["client_token"]
kv = http("GET", f"{BAO_ADDR}/v1/{BAO_SECRET_PATH}", headers={"X-Vault-Token": token})
data = {k: str(v).strip() for k, v in kv["data"]["data"].items()} # `bao kv put key=@file` keeps the file's newline
for key in ("gitea_token", "webhook_secret"):
if not data.get(key):
raise ApiError(f"OpenBao {BAO_SECRET_PATH} has no key {key}")
return data
# ---------------------------------------------------------------- gitea
class Gitea:
def __init__(self, token: str, base: str = GITEA_URL, repo: str = NOTES_REPO, branch: str = NOTES_BRANCH):
self.base, self.repo, self.branch = base, repo, branch
self.headers = {"Authorization": f"token {token}"}
def api(self, path: str, **params):
q = ("?" + urllib.parse.urlencode(params)) if params else ""
return http("GET", f"{self.base}/api/v1/repos/{self.repo}{path}{q}", headers=self.headers)
def head(self) -> str:
return self.api(f"/branches/{urllib.parse.quote(self.branch, safe='')}")["commit"]["id"]
def tree(self, commit: str) -> dict[str, str]:
"""path -> blob sha for every wiki note at that commit."""
out: dict[str, str] = {}
page, per_page, seen = 1, 1000, 0
while True:
t = self.api(f"/git/trees/{commit}", recursive="true", per_page=per_page, page=page)
entries = t.get("tree") or [] # Gitea answers "tree": null past the last page
for e in entries:
p = e.get("path", "")
if e.get("type") == "blob" and p.startswith(WIKI_DIR + "/") and p.endswith(".md") and "/" not in p[len(WIKI_DIR) + 1:]:
out[p] = e["sha"]
seen += len(entries)
total = int(t.get("total_count") or 0)
if not entries or len(entries) < per_page or (total and seen >= total):
break
page += 1
if page > 50:
raise ApiError("tree paging runaway")
return out
def raw(self, path: str, ref: str) -> str:
data = http("GET", f"{self.base}/api/v1/repos/{self.repo}/raw/{urllib.parse.quote(path)}?ref={urllib.parse.quote(ref, safe='')}",
headers=self.headers)
return data.decode("utf-8") if isinstance(data, bytes) else json.dumps(data)
# ---------------------------------------------------------------- notes
def parse_frontmatter(text: str) -> tuple[dict, str]:
m = re.match(r"^---[ \t]*\n(.*?)\n---[ \t]*(?:\n|$)", text, re.DOTALL)
if not m:
return {}, text
fm: dict = {}
for line in m.group(1).splitlines():
mm = re.match(r"^([A-Za-z_][\w-]*):\s*(.*)$", line)
if mm:
fm[mm.group(1)] = mm.group(2).strip()
return fm, text[m.end():]
def as_list(value: str) -> list[str]:
s = (value or "").strip()
if s.startswith("[") and s.endswith("]"):
s = s[1:-1]
return [p.strip().strip("'\"") for p in s.split(",") if p.strip()]
def first_paragraph(body: str) -> str:
para: list[str] = []
seen_h1 = False
for line in body.splitlines():
if line.startswith("# "):
seen_h1 = True
continue
if not seen_h1:
continue
if line.strip() == "" or line.startswith("#"):
if para:
break
continue
para.append(line.strip())
return " ".join(para)
def note_text(slug: str, text: str) -> tuple[str, str]:
"""(title, embedding text) for one note file."""
fm, body = parse_frontmatter(text)
title = (fm.get("title") or slug).strip().strip("'\"")
parts = [slug.replace("-", " "), title, " ".join(as_list(fm.get("aliases", ""))),
" ".join(as_list(fm.get("tags", ""))), first_paragraph(body)]
return title, " ".join(parts)[:EMBED_CHARS]
# ---------------------------------------------------------------- embeddings
def embed(texts: list[str], batch: int = 2) -> list[list[float]]:
out: list[list[float]] = []
for i in range(0, len(texts), batch):
resp = http("POST", f"{EMBED_URL}/v1/embeddings", {"model": EMBED_MODEL, "input": texts[i:i + batch]}, timeout=120)
rows = sorted(resp["data"], key=lambda d: d["index"])
out.extend(unit(d["embedding"]) for d in rows)
return out
def unit(v: list[float]) -> list[float]:
n = sum(x * x for x in v) ** 0.5 or 1.0
return [round(x / n, 5) for x in v]
# ---------------------------------------------------------------- index
class Index:
def __init__(self, path: Path):
self.path = path
self.lock = threading.Lock()
self.notes: dict[str, dict] = {} # slug -> {sha, title, vector}
self.commit = ""
self.synced_at = 0.0
self.syncing = False
self.last_error = ""
if path.exists():
try:
d = json.loads(path.read_text(encoding="utf-8"))
self.notes, self.commit, self.synced_at = d.get("notes", {}), d.get("commit", ""), d.get("synced_at", 0.0)
LOG.info("loaded index: %d notes at %s", len(self.notes), self.commit[:8])
except (OSError, ValueError) as e:
LOG.warning("index unreadable (%s); starting empty", e)
def save(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
tmp = self.path.with_suffix(".tmp")
tmp.write_text(json.dumps({"notes": self.notes, "commit": self.commit, "synced_at": self.synced_at}), encoding="utf-8")
tmp.replace(self.path)
def sync(self, gitea: Gitea) -> dict:
"""Bring the index to the branch head; returns a summary."""
with self.lock:
self.syncing = True
try:
head = gitea.head()
if head == self.commit and self.notes:
summary = {"commit": head, "changed": 0, "removed": 0, "notes": len(self.notes), "skipped": "head unchanged"}
self.synced_at = time.time()
self.save()
return summary
tree = gitea.tree(head)
slugs = {p[len(WIKI_DIR) + 1:-3]: (p, sha) for p, sha in tree.items() if not p.endswith("/_index.md")}
todo = [s for s, (p, sha) in slugs.items() if self.notes.get(s, {}).get("sha") != sha]
removed = [s for s in list(self.notes) if s not in slugs]
LOG.info("sync %s: %d notes, %d to embed, %d removed", head[:8], len(slugs), len(todo), len(removed))
fresh: dict[str, dict] = {}
for i in range(0, len(todo), 20):
chunk = todo[i:i + 20]
texts, titles = [], {}
for s in chunk:
title, text = note_text(s, gitea.raw(slugs[s][0], head))
titles[s] = title
texts.append(text)
for s, vec in zip(chunk, embed(texts)):
fresh[s] = {"sha": slugs[s][1], "title": titles[s], "vector": vec}
with self.lock:
for s in removed:
self.notes.pop(s, None)
self.notes.update(fresh)
self.commit = head
self.synced_at = time.time()
self.last_error = ""
self.save()
return {"commit": head, "changed": len(todo), "removed": len(removed), "notes": len(self.notes)}
except Exception as e:
with self.lock:
self.last_error = str(e)[:300]
raise
finally:
with self.lock:
self.syncing = False
def rank(self, query: str, k: int = 60) -> list[dict]:
q = embed([query])[0]
with self.lock:
rows = [(s, n["title"], sum(a * b for a, b in zip(q, n["vector"]))) for s, n in self.notes.items()]
rows.sort(key=lambda r: -r[2])
return [{"slug": s, "title": t, "score": round(sc, 4)} for s, t, sc in rows[:k]]
def status(self) -> dict:
"""`ready` = an index exists and can answer /rank; a failed sync keeps the last good index."""
with self.lock:
return {"ready": bool(self.notes), "notes": len(self.notes), "commit": self.commit,
"synced_at": self.synced_at, "age_s": round(time.time() - self.synced_at) if self.synced_at else None,
"syncing": self.syncing, "last_error": self.last_error or None}
# ---------------------------------------------------------------- syncer
class Syncer(threading.Thread):
"""One worker: syncs on demand (webhook) and every RESYNC_SECONDS."""
def __init__(self, index: Index, creds: Credentials, interval: int):
super().__init__(daemon=True, name="syncer")
self.index, self.creds, self.interval = index, creds, interval
self.wake = threading.Event()
def trigger(self) -> None:
self.wake.set()
RETRY_SECONDS = 60
def run(self) -> None:
while True:
try:
data = self.creds.fetch()
summary = self.index.sync(Gitea(data["gitea_token"]))
LOG.info("sync done: %s", summary)
wait = self.interval
except Exception as e: # keep serving the last good index; retry soon, not next interval
LOG.error("sync failed: %s", e)
wait = self.RETRY_SECONDS
self.wake.wait(wait)
self.wake.clear()
def verify_signature(secret: str, body: bytes, signature: str) -> bool:
digest = hmac.new(secret.encode("utf-8"), body, hashlib.sha256).hexdigest()
return hmac.compare_digest(digest, (signature or "").strip())
# ---------------------------------------------------------------- server
def make_handler(index: Index, syncer: Syncer, creds: Credentials):
class Handler(BaseHTTPRequestHandler):
def log_message(self, fmt, *args):
LOG.info("%s %s", self.address_string(), fmt % args)
def send_json(self, code: int, obj) -> None:
data = json.dumps(obj).encode("utf-8")
self.send_response(code)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def body(self) -> bytes:
n = int(self.headers.get("Content-Length") or 0)
return self.rfile.read(n) if n else b""
def do_GET(self):
if self.path == "/health": # liveness: the process serves
return self.send_json(200, index.status())
if self.path == "/ready": # readiness: an index exists
st = index.status()
return self.send_json(200 if st["ready"] else 503, st)
self.send_json(404, {"error": "not found"})
def do_POST(self):
if self.path == "/rank":
try:
req = json.loads(self.body() or b"{}")
query = str(req.get("query", "")).strip()
k = max(1, min(int(req.get("k", 60)), 500))
if not query:
return self.send_json(400, {"error": "query required"})
rows = index.rank(query, k)
return self.send_json(200, {"rows": rows, "index": index.status()})
except ApiError as e:
return self.send_json(502, {"error": str(e)})
except (ValueError, TypeError) as e:
return self.send_json(400, {"error": str(e)})
if self.path == "/webhook":
raw = self.body()
try:
secret = creds.fetch()["webhook_secret"]
except ApiError as e:
return self.send_json(503, {"error": str(e)})
if not verify_signature(secret, raw, self.headers.get("X-Gitea-Signature", "")):
return self.send_json(403, {"error": "bad signature"})
try:
event = json.loads(raw or b"{}")
except ValueError:
return self.send_json(400, {"error": "bad json"})
ref = event.get("ref", "")
if ref and ref != f"refs/heads/{NOTES_BRANCH}":
return self.send_json(200, {"queued": False, "ref": ref})
syncer.trigger()
return self.send_json(200, {"queued": True})
self.send_json(404, {"error": "not found"})
return Handler
def main() -> int:
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s", stream=sys.stdout)
index = Index(INDEX_PATH)
creds = Credentials()
syncer = Syncer(index, creds, RESYNC_SECONDS)
syncer.start()
server = ThreadingHTTPServer(("", LISTEN_PORT), make_handler(index, syncer, creds))
LOG.info("listening on :%d, repo %s@%s, embed %s", LISTEN_PORT, NOTES_REPO, NOTES_BRANCH, EMBED_URL)
server.serve_forever()
return 0
if __name__ == "__main__":
sys.exit(main())
+73 -91
View File
@@ -1,117 +1,99 @@
# Traffic classes on the k8s nodes
Issue #576. Three of the four classes are separated; management is not.
Issue #576. Five classes, each on its own VLAN.
## Where each class runs
| class | link | mechanism |
|---|---|---|
| Ceph replication | `ceph-san`, VLAN 49 | the whole link is handed to the OSD pod (Multus `host-device`) |
| world→pod | `ingress0`, VLAN 47 | BGP session sourced there, so the advertised next hop is that address |
| pod→world | `ingress0`, VLAN 47 | routing rule on the pod source prefix → table 47 |
| pod↔pod | `pod0`, VLAN 48 | routing rule on pod source **and** destination → table 48 |
| management | `mgmt0`, VLAN 40 | not started |
| pod↔pod | `pod0`, VLAN 48 | pod-sourced traffic is routed from table 47; bird puts the peers' pod CIDRs there via `pod0` |
| world↔pod | `ingress0`, VLAN 47 | BGP session sourced there (next hop of every LoadBalancer VIP); pod egress takes table 47's default |
| node↔node | `node0`, VLAN 46 | the node InternalIP and etcd live there; so do apiserver↔kubelet, cilium-health and host→remote-pod traffic |
| world↔node | `mgmt0`, VLAN 40 | API VIP for kubectl, talosctl from outside, node egress (DNS, NTP, image pulls) |
Hard constraint, set by the owner: inter-node and pod traffic never traverses
the UDM. Everything above stays on the 10G switch.
the UDM. Everything but world↔node stays on the 10G switch. VLAN 46 and 48 are
statically addressed, without a gateway; VLAN 46 is isolated in UniFi.
## Why policy routing rather than moving the node IP
## Pod traffic: one table
Cilium reaches a peer's pods at that peer's Kubernetes InternalIP, in native
and tunnel mode alike. `--ipv4-node` is overwritten from the Node object, and
`nodePort.directRoutingDevice` selects the BPF NodePort device, not a next hop.
Both were established by test (#837, #838), not by reading.
Two rules, one per family, send every packet with a pod source to table 47
(`talos/patches/site-homelab.yaml`, priorities 1000/1001, below main). The
table decides by longest prefix; no rule carves out exceptions.
So the InternalIP decides where pod traffic goes — unless the lookup happens in
a different table. The rules do that, and nothing about the node's identity
moves. The routing decision runs before Cilium's masquerade hook, so a rule
matching the pod source sees the pod address, and the masquerade then rewrites
it to the egress link's.
bird (`kubernetes/infrastructure/bird/`) owns table 47 and fills it with:
## Rule order
- the node's attached networks — `mgmt0`, `ingress0`, `pod0`, `node0` —
taken from the interfaces;
- the peers' pod CIDRs, learned over an iBGP mesh on `pod0`, next hop the
peer's `pod0` address;
- the node's own pod CIDR via `cilium_host`, originated rather than read back
from the kernel (reading it back made BIRD depend on the shape of Cilium's
route, and KRT rejects the IPv4 one);
- the egress default via the UDM on `ingress0`, from the node annotations
`net.renner.uno/egress-gw-v4/v6`.
Lower priority runs first.
`node0` has to be in the table: a pod reaching a node address — the
`kubernetes` ClusterIP, which socket LB rewrites to the apiserver's advertise
address, or a hostNetwork pod — would otherwise take the default to the UDM.
| prio | match | table |
|---|---|---|
| 980 | src pod, dst pod | 48 |
| 981 | src pod, dst pod (v4) | 48 |
| 990 | dst `fd00::/8` | main |
| 991 | dst `10.0.0.0/8` | main |
| 1000 | src pod | 47 |
| 1001 | src pod (v4) | 47 |
The routing decision runs before Cilium's masquerade hook, so the rule sees the
pod address and the masquerade then rewrites it to the egress link's.
980/981 match source **and** destination. Source alone would also catch the
node's own traffic to a pod, which would then leave with a `pod0` source — an
address `CiliumNode.spec.addresses` does not list, so Cilium classifies it as
`world` rather than `host`.
990/991 are Cilium's `ipv4/ipv6NativeRoutingCIDR`, and they are what keeps
cluster-internal traffic out of table 47. Without them a pod's packet to a
service matches the source rule, finds only a default route, and leaves for the
UDM — measured, in-cluster DNS stopped resolving.
Traffic to a pod on the same node falls through on its own: table 48 holds only
the peers' CIDRs, so the lookup misses and 990/991 send it to main.
## Where each table comes from
- **47** — two default routes declared per node in `talos/patches/k8s-*.yaml`.
The IPv6 gateway is the UDM's configured address on the segment, not the
RA-learned link-local; `accept_ra_defrtr` stays `0` on `ingress0` so the RA
default does not land in main.
- **48** — filled by the `bird` DaemonSet: an iBGP mesh over `pod0`, one route
per peer pod CIDR with the peer's `pod0` address as next hop.
## The bird mesh
`kubernetes/infrastructure/bird/`. Each node advertises only its own pod CIDR
and installs what it learns into table 48. Cilium's routes in main are
untouched, which is why the two coexist.
Node-originated traffic does not match the rules and uses main, where Cilium
routes each peer's pod CIDR via the peer's InternalIP — on `node0`.
No address is written into this repo twice. Each node publishes its `pod0`
addresses as `nodeAnnotations`, beside the `VLANConfig` that assigns them; the
pod CIDRs come from the Node objects. An init container renders both from the
API with `kubectl -o go-template-file`, excludes the node itself with a field
selector, and ends with `bird -p` so BIRD's own parser gates startup.
addresses as `nodeAnnotations` beside the `VLANConfig` that assigns them; the
pod CIDRs come from the Node objects. bird's init container renders both from
the API and ends with `bird -p`, so BIRD's own parser gates startup.
The node's own CIDR is **originated**, not read back from the kernel. Reading
it back made BIRD depend on the shape Cilium installs, and the IPv4 route is
rejected outright: its next hop sits inside the prefix it describes, which KRT
refuses with `strange next-hop`. The IPv6 route is a device route and passed,
so the failure was silent on one family only.
## Node↔node
`kubelet.nodeIP.validSubnets` and `cluster.etcd.advertisedSubnets` point at
VLAN 46. Talos picks one InternalIP per family, IPv4 first — the service CIDR
is IPv4, and the apiserver's advertise address must match it.
The API VIP stays on `mgmt0` for kubectl. No in-cluster client uses it:
kubelet, scheduler, controller-manager and Cilium go through KubePrism
(`localhost:7445`), which balances over every address of every control-plane
member; pods use the ClusterIP.
Talos never updates an existing etcd member's peer URLs
(siderolabs/talos#12720). Changing `advertisedSubnets` means, per member:
stage the config, `talosctl etcd forfeit-leadership`, `talosctl etcd leave`,
reboot. `leave` removes only `/var/lib/etcd`; the node rejoins as a learner
and is promoted. EPHEMERAL does not need a wipe.
## Moving node IPs: the mixed state
While some nodes have their InternalIP on the new network and some on the old,
host→remote-pod traffic leaves with a node address that is not an InternalIP:
the kernel picks the source from the route to the peer's InternalIP. Cilium
classifies that as `world`, and every ingress policy that admits only
`host`/`remote-node` (rook-ceph, monitoring, …) drops it. Kernel Ceph clients
lose the OSDs on the other side; RBD mounts hang.
Bridge it for the duration with a temporary `CiliumClusterwideNetworkPolicy`
admitting the node subnets of both networks, both families, with
`enableDefaultDeny: false` so it restricts nothing else, and delete it once
every node has moved. Keep all OSD nodes schedulable while one is drained:
draining two takes Ceph below its data availability.
## Failure behaviour
Benign at every layer. An empty or absent table 48 makes 980/981 fall through
to main, and pod traffic returns to VLAN 40 — unseparated, not broken. The same
holds if the DaemonSet cannot start, which is the case during a cold rebuild
until Gitea serves its image.
## Management, the remaining class
Not started. The levers differ per service and none of them is the node IP:
| service | can leave the node IP? | lever |
|---|---|---|
| etcd | yes | `cluster.etcd.advertisedSubnets`; needs a reboot per node |
| kube-apiserver endpoint | yes | it is a VIP — `Layer2VIPConfig.link` |
| Talos API | n/a | `apid` answers on every node address; a firewall question |
| kubelet | no | it *is* the node IP |
Two approaches were rejected by the owner and are not to be proposed again:
moving the node IP and etcd to `pod0` (#838), and relabelling VLAN 40 as the
cluster VLAN while moving etcd and the API VIP to a new management VLAN.
Replacing the CNI (#839, Antrea) was shelved.
Note that the reasoning which produced that list also concluded pod↔pod was
blocked, which policy routing disproved. Treat it as a record of decisions, not
as an exhaustive analysis.
If bird stops, table 47 empties, the rules find nothing and fall through to
main: pod traffic returns to the node networks, unseparated rather than
misrouted. The same holds during a cold rebuild until Gitea serves bird's
image. The default route lives in table 47, not in the Talos config, so a
default cannot outlive the specific routes and send pod traffic to the UDM.
## Sources
Cilium v1.20.1 `pkg/datapath/linux/node.go`, `pkg/node/types/node.go`,
`pkg/node/sync/local_node_sync.go`; BIRD 2.17 user manual (kernel protocol,
`learn all`, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig` and
`machine.nodeAnnotations`. PRs #831, #833, #835, #836, #837, #842, #852, #854,
#855; closed unmerged: #830, #838, #839.
Cilium v1.20 `pkg/datapath/linux/node.go`; BIRD 2.17 user manual (kernel
protocol, `strict bind`, `direct`); Talos v1.13 `RoutingRuleConfig`,
`machine.nodeAnnotations`, configuration patching (lists are appended,
`$patch: delete`), etcd `LeaveCluster`; KubePrism docs;
siderolabs/talos#12720. PRs #831, #833, #835, #836, #837, #842, #852, #854,
#855, #857, #858, #918, #919, #922; closed unmerged: #830, #838, #839.
+1 -1
View File
@@ -2,7 +2,7 @@ apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gitea
namespace: apps
namespace: gitea
labels:
app.kubernetes.io/name: gitea
annotations:
+1 -1
View File
@@ -2,7 +2,7 @@ apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
name: gitea-ssh
namespace: apps
namespace: gitea
labels:
app.kubernetes.io/name: gitea
spec:
+6 -26
View File
@@ -1,28 +1,8 @@
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: gitea-allow-metrics-scrape
namespace: apps
spec:
# Prometheus (monitoring ns) scrapes gitea's /metrics on :3000. The apps
# default-deny-ingress only allows same-ns / home-assistant / host, so the
# scrape is dropped (context deadline exceeded). This became necessary only
# after the LXC->in-cluster cutover — the old static target was external, so
# no in-cluster policy applied. Scoped to the gitea pod + :3000.
endpointSelector:
matchLabels:
app.kubernetes.io/name: gitea
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": monitoring
toPorts:
- ports:
- port: "3000"
protocol: TCP
---
# Prometheus scrape and Traefik are admitted by the namespace baseline in
# infrastructure/network-policies/gitea-ns.yaml.
#
# Secret rotation: the flux-git rotator (openbao ns) reaches gitea to mint a new
# flux PAT. It uses the in-cluster Service (gitea.apps.svc:3000) — the public
# flux PAT. It uses the in-cluster Service (gitea.gitea.svc:3000) — the public
# gitea.renner.uno name hairpins off the ingress LB from a pod (same failure as
# pve.renner.uno). Scoped to the gitea pod :3000 and the rotator label; the
# egress half is rotator-gitea-flux-egress in network-policies/openbao-ns.yaml.
@@ -30,7 +10,7 @@ apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: gitea-allow-flux-rotator
namespace: apps
namespace: gitea
spec:
endpointSelector:
matchLabels:
@@ -54,7 +34,7 @@ apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: gitea-allow-review-bot-rotator
namespace: apps
namespace: gitea
spec:
endpointSelector:
matchLabels:
+1 -1
View File
@@ -13,7 +13,7 @@ apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: gitea
namespace: apps
namespace: gitea
labels:
app.kubernetes.io/name: gitea
spec:
+1 -1
View File
@@ -2,7 +2,7 @@ apiVersion: v1
kind: Service
metadata:
name: gitea
namespace: apps
namespace: gitea
labels:
app.kubernetes.io/name: gitea
spec:
+2 -2
View File
@@ -2,7 +2,7 @@ apiVersion: apps/v1
kind: StatefulSet
metadata:
name: gitea
namespace: apps
namespace: gitea
labels:
app.kubernetes.io/name: gitea
spec:
@@ -152,7 +152,7 @@ spec:
# NOTE: volumeClaimTemplates is immutable on an existing StatefulSet,
# so bumping this value cannot be `kubectl apply`-ed in place — Flux
# will reject the whole object. Landing this size change requires a
# one-time `kubectl delete statefulset gitea -n apps --cascade=orphan`
# one-time `kubectl delete statefulset gitea -n gitea --cascade=orphan`
# (pod + PVC keep running) so Flux re-creates the STS from this
# template and re-adopts them. After the cleanup_packages GC reclaims
# the ~10G of orphaned blobs, real usage is ~9G, so 40Gi is headroom.
+1 -1
View File
@@ -3,8 +3,8 @@ kind: Kustomization
resources:
- gatus/
- external-dns/
- gitea/
- ical-proxy/
# gitea/ has its own `gitea` Kustomization (clusters/homelab/gitea.yaml, #878).
# woodpecker/ moved to its own dedicated `woodpecker` Kustomization
# (clusters/homelab/woodpecker.yaml) so its wp-pg CNPG cluster's barman
# dependency + Cluster health-gating don't contaminate this shared apps KS.
@@ -91,12 +91,11 @@ spec:
# Additional scrape targets not covered by ServiceMonitors.
additionalScrapeConfigs:
# Gitea exposes /metrics anonymously on its main HTTP port. The
# `gitea_*` metrics power the gitea-overview dashboard. Gitea now runs
# in-cluster (StatefulSet in apps ns; LXC 114 retired), so scrape the
# ClusterIP Service directly instead of the dead LXC hostname.
# `gitea_*` metrics power the gitea-overview dashboard. Scrapes the
# ClusterIP Service of the StatefulSet in the gitea namespace.
- job_name: gitea
static_configs:
- targets: ["gitea.apps.svc.cluster.local:3000"]
- targets: ["gitea.gitea.svc.cluster.local:3000"]
metrics_path: /metrics
# Grafana
+4 -2
View File
@@ -27,6 +27,8 @@ data:
PAPERLESS_ALLOWED_HOSTS: "*"
PAPERLESS_TIME_ZONE: "Europe/Berlin"
PAPERLESS_OCR_LANGUAGE: "deu+eng"
# Default is cpu_count / TASK_WORKERS, 20 on k8s-1; OCR memory scales with it.
PAPERLESS_THREADS_PER_WORKER: "5"
PAPERLESS_CONSUMER_RECURSIVE: "true"
PAPERLESS_CONSUMER_DELETE_DUPLICATES: "true"
PAPERLESS_CONSUMER_ENABLE_BARCODES: "true"
@@ -183,7 +185,7 @@ spec:
drop: ["ALL"]
containers:
- name: paperless
image: ghcr.io/paperless-ngx/paperless-ngx:3.2.0 # {"$imagepolicy": "flux-system:paperless-ngx"}
image: ghcr.io/paperless-ngx/paperless-ngx:3.2.1 # {"$imagepolicy": "flux-system:paperless-ngx"}
ports:
- name: http
containerPort: 8000
@@ -273,7 +275,7 @@ spec:
cpu: 200m
memory: 512Mi
limits:
memory: 2Gi
memory: 4Gi
securityContext:
runAsNonRoot: false
allowPrivilegeEscalation: true
+1 -1
View File
@@ -25,7 +25,7 @@ spec:
type: RuntimeDefault
containers:
- name: redis
image: redis:8.10.1-alpine # {"$imagepolicy": "flux-system:redis-paperless"}
image: redis:8.10.2-alpine # {"$imagepolicy": "flux-system:redis-paperless"}
args:
- redis-server
- --save
+137
View File
@@ -0,0 +1,137 @@
---
# ServiceAccount is the OpenBao identity (auth/kubernetes/role/wiki-index); no
# Kubernetes API access needed.
apiVersion: v1
kind: ServiceAccount
metadata:
name: wiki-index
namespace: ai
labels:
app.kubernetes.io/name: wiki-index
---
# Index (~17 MB JSON for 1700 notes). Survives restarts so a pod start does not
# re-embed the wiki; a lost volume only costs one full sync (~5 min).
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wiki-index-data
namespace: ai
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 1Gi
---
# Embedding index over the Notes repo's wiki/ (ci/images/wiki-index). Syncs from
# Gitea on push webhook and every RESYNC_SECONDS, embeds via wiki-embed, serves
# POST /rank to ~/Notes/bin/wiki_rank.py through wiki-index.renner.uno.
apiVersion: apps/v1
kind: Deployment
metadata:
name: wiki-index
namespace: ai
labels:
app.kubernetes.io/name: wiki-index
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: wiki-index
template:
metadata:
labels:
app.kubernetes.io/name: wiki-index
spec:
serviceAccountName: wiki-index
securityContext:
runAsNonRoot: true
runAsUser: 65534
runAsGroup: 65534
fsGroup: 65534
seccompProfile:
type: RuntimeDefault
containers:
- name: wiki-index
# Per-commit tag (build-ci-images.yml): immutable, no same-day overwrite.
image: gitea.renner.uno/gitea_admin/wiki-index:2026-09-20-4b529cd1
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
env:
- name: GITEA_URL
value: https://gitea.renner.uno
- name: NOTES_REPO
value: k.renner/notes
- name: NOTES_BRANCH
value: main
- name: EMBED_URL
value: http://wiki-embed.ai.svc.cluster.local:8080
- name: BAO_ADDR
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: wiki-index
- name: BAO_SECRET_PATH
value: homelab/data/apps/ai/wiki-index
- name: INDEX_PATH
value: /data/index.json
- name: RESYNC_SECONDS
value: "1800"
ports:
- name: http
containerPort: 8080
# Pure-Python cosine over 1700 x 1024 floats per query; the index itself is ~60 MB in memory.
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
# /health: the process serves (liveness). /ready: an index exists; a cold
# start on an empty PVC embeds every note (~5 min) before traffic arrives.
startupProbe:
httpGet:
path: /health
port: http
periodSeconds: 5
failureThreshold: 12
readinessProbe:
httpGet:
path: /ready
port: http
periodSeconds: 10
livenessProbe:
httpGet:
path: /health
port: http
periodSeconds: 60
timeoutSeconds: 10
failureThreshold: 3
volumeMounts:
- name: data
mountPath: /data
volumes:
- name: data
persistentVolumeClaim:
claimName: wiki-index-data
---
apiVersion: v1
kind: Service
metadata:
name: wiki-index
namespace: ai
labels:
app.kubernetes.io/name: wiki-index
spec:
selector:
app.kubernetes.io/name: wiki-index
ports:
- name: http
port: 8080
targetPort: http
+56
View File
@@ -0,0 +1,56 @@
---
# wiki-index.renner.uno — LAN-only (no dns.renner.uno/public label), machine
# client (bin/wiki_rank.py), so Traefik BasicAuth like wiki-embed.renner.uno.
# Gitea's webhook does not pass here; it targets the ClusterIP Service directly.
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: wiki-index-basic-auth
namespace: ai
spec:
basicAuth:
secret: wiki-index-basic-auth
realm: wiki-index
---
# OpenBao key apps/ai/wiki-index-basic-auth, field `users` = htpasswd line.
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: wiki-index-basic-auth
namespace: ai
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: wiki-index-basic-auth
dataFrom:
- extract:
key: apps/ai/wiki-index-basic-auth
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: wiki-index
namespace: ai
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
traefik.ingress.kubernetes.io/router.middlewares: ai-wiki-index-basic-auth@kubernetescrd
spec:
ingressClassName: traefik
tls:
- hosts:
- wiki-index.renner.uno
secretName: wiki-index-tls
rules:
- host: wiki-index.renner.uno
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: wiki-index
port:
number: 8080
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- ingress.yaml
+1
View File
@@ -1946,6 +1946,7 @@ if phase_runs wait-apps; then
# apps waits on infrastructure-ceph-cluster (Ceph mon/mgr/OSD bring-up
# is genuinely 5–10 min on a fresh OSD-wiped cluster).
wait_kustomization apps "apps" 90
wait_kustomization gitea "Gitea (incl. data-gitea-0 PVC)" 90
wait_kustomization influxdb "InfluxDB (HR install + token Jobs)" 600
wait_kustomization woodpecker "Woodpecker CI (incl. wp-pg CNPG backend)" 600
wait_kustomization authentik "Authentik (incl. ak-pg CNPG backend)" 600
+20
View File
@@ -0,0 +1,20 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: gitea
namespace: flux-system
spec:
# Gitea in its own `gitea` namespace (#878), formerly under the shared `apps` KS.
interval: 10m
retryInterval: 2m
timeout: 15m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/gitea
prune: true
wait: true
dependsOn:
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
@@ -0,0 +1,22 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: wiki-index
namespace: flux-system
spec:
interval: 10m
retryInterval: 1m
# First start on an empty PVC embeds every note (~5 min) before /health is 200.
timeout: 20m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/wiki-index
prune: true
wait: true
dependsOn:
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
# wiki-embed (the embedding backend) lives in the llama-server app.
- name: llama-server
+1 -1
View File
@@ -23,7 +23,7 @@ protocol device { scan time 10; }
# down. The usual advice is not to export these to the kernel because it
# maintains them itself — true of main, not of a table the kernel never fills.
protocol direct attached {
interface "mgmt0", "ingress0", "pod0";
interface "mgmt0", "ingress0", "pod0", "node0";
ipv4 { table master4; };
ipv6 { table master6; };
}
@@ -48,8 +48,8 @@ spec:
tier: node
app: bird
spec:
# Carries pod east-west once the routing rules land. Without it table 48
# stays empty and the rules fall through to main, i.e. back to VLAN 40.
# Owns table 47, where all pod traffic is routed. Without it the table
# stays empty and the rules fall through to main, unseparated.
priorityClassName: system-node-critical
# Writes host routes and peers on host addresses.
hostNetwork: true
@@ -118,14 +118,21 @@ spec:
# traffic out to the VIP and back (which fails with L2 announcements).
lbExternalClusterIP: true
cni:
# Multus (kubernetes/infrastructure/multus) runs alongside Cilium and
# writes 00-multus.conf to /etc/cni/net.d so kubelet picks multus
# first; multus then delegates to Cilium's 05-cilium.conflist via
# clusterNetwork. The default `exclusive: true` makes cilium-cni's
# install step rename anything alphabetically before its own config
# to *.cilium_bak on every reconcile, which kept nuking multus.
# Setting it false lets multus's config coexist.
# Multus is the only writer of /etc/cni/net.d (#832): Cilium's conflist
# goes to /etc/cni/net.d/multus/, which the container runtime does not
# read, and Multus delegates to it via clusterNetwork. Without this a
# pod created after a reboot, before Multus rewrote 00-multus.conf, came
# up with Cilium alone. See kubernetes/infrastructure/multus.
confPath: /etc/cni/net.d/multus
# The default `exclusive: true` renames other configs in the conf dir to
# *.cilium_bak on every reconcile, which kept nuking multus.
exclusive: false
# One agent at a time: the chart default of 2 restarts two of three
# nodes' datapath at once.
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
# Talos-specific settings
securityContext:
capabilities:
+1 -1
View File
@@ -19,7 +19,7 @@ resources:
- cloudnative-pg/
- mariadb-operator/
- netbird-ingress/
- netbird-proxy-services/
- netbird-service-sync/
- coredns-config/
- k8s-gateway/
- external-dns-unifi/
@@ -111,10 +111,18 @@ metadata:
name: multus
namespace: kube-system
---
# Multus is the only writer of /etc/cni/net.d (#832). Cilium writes its conflist
# to /etc/cni/net.d/multus/, a subdirectory the container runtime does not read
# (libcni skips directories); Multus picks it up there via multusAutoconfigDir
# and writes 00-multus.conf. Until that file exists the runtime has no network
# config at all, so no pod can be created with Cilium alone after a reboot.
#
# The "-subdir" names date from the node-by-node rollout; renaming would roll
# Multus again for nothing.
kind: ConfigMap
apiVersion: v1
metadata:
name: multus-daemon-config
name: multus-daemon-config-subdir
namespace: kube-system
labels:
tier: node
@@ -127,7 +135,7 @@ data:
"logLevel": "verbose",
"logToStderr": true,
"cniConfigDir": "/host/etc/cni/net.d",
"multusAutoconfigDir": "/host/etc/cni/net.d",
"multusAutoconfigDir": "/host/etc/cni/net.d/multus",
"multusConfigFile": "auto",
"socketDir": "/host/run/multus/"
}
@@ -135,16 +143,16 @@ data:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-multus-ds
name: kube-multus-ds-subdir
namespace: kube-system
labels:
tier: node
app: multus
name: multus
name: multus-subdir
spec:
selector:
matchLabels:
name: multus
name: multus-subdir
updateStrategy:
type: RollingUpdate
template:
@@ -152,7 +160,7 @@ spec:
labels:
tier: node
app: multus
name: multus
name: multus-subdir
spec:
# Upstream ships this DaemonSet with no priority class, so it runs at 0
# and can preempt nothing. Any event that makes the pod need a fresh
@@ -224,6 +232,10 @@ spec:
volumeMounts:
- name: cni
mountPath: /host/etc/cni/net.d
# Exists before Multus starts, so it waits for Cilium's conflist
# instead of exiting on a missing directory.
- name: cni-cilium-conf
mountPath: /host/etc/cni/net.d/multus
# multus-daemon expects that cnibin path must be identical between pod and container host.
# e.g. if the cni bin is in '/opt/cni/bin' on the container host side, then it should be mount to '/opt/cni/bin' in multus-daemon,
# not to any other directory, like '/opt/bin' or '/usr/bin'.
@@ -279,6 +291,10 @@ spec:
- name: cni
hostPath:
path: /etc/cni/net.d
- name: cni-cilium-conf
hostPath:
path: /etc/cni/net.d/multus
type: DirectoryOrCreate
- name: cnibin
hostPath:
path: /opt/cni/bin
@@ -287,7 +303,7 @@ spec:
path: /
- name: multus-daemon-config
configMap:
name: multus-daemon-config
name: multus-daemon-config-subdir
items:
- key: daemon-config.json
path: daemon-config.json
@@ -39,9 +39,11 @@ spec:
name: whereabouts-reconciler
spec:
serviceAccountName: whereabouts
# Every node here is a control plane node.
# Every node here is a control plane node. Keyed: a keyless Exists would
# also tolerate node.kubernetes.io/unschedulable and ignore cordons.
tolerations:
- operator: Exists
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
containers:
- name: whereabouts-reconciler
@@ -179,6 +179,22 @@ kind: Namespace
metadata:
name: woodpecker
---
# Gitea, moving out of the shared `apps` namespace (issue #878). Declared here
# like `woodpecker` above, so it exists before infrastructure-crds applies its
# CiliumNetworkPolicies. No pod-security labels: the cluster default applies.
apiVersion: v1
kind: Namespace
metadata:
name: gitea
---
# netbird-service-sync, moving out of `openbao` (issue #876). Declared here like
# `woodpecker` above, so it exists before infrastructure-crds applies its
# CiliumNetworkPolicies. No pod-security labels: the cluster default applies.
apiVersion: v1
kind: Namespace
metadata:
name: netbird-service-sync
---
# Local LLM inference (kubernetes/apps/llama-server). Declared here, like
# `woodpecker` above, so the `infrastructure` KS creates it before
# `infrastructure-crds` applies the namespace's CiliumNetworkPolicies.
@@ -1,28 +1,28 @@
# Reconciles NetBird reverse-proxy services from Ingresses that carry
# dns.renner.uno/public: one TLS-passthrough service per host on :443,
# targeting Traefik's ClusterIP through a subnet resource over the service
# CIDR (ci/images/netbird-proxy-services/controller.py).
# CIDR (ci/images/netbird-service-sync/controller.py).
#
# openbao namespace, like the rotators: its default-deny already allows egress
# to OpenBao, DNS, the apiserver and `world` (netbird.renner.uno), and the
# NetBird PAT is read from OpenBao on every pass through the
# netbird-proxy-services role — policies/netbird-proxy-services.hcl, applied
# once by hand like the rotator roles. Until that role exists the pod logs an
# auth error each minute and does nothing.
# Own namespace; its egress (DNS, apiserver, OpenBao, NetBird API) is in
# network-policies/netbird-service-sync-ns.yaml. The NetBird PAT is read from
# OpenBao on every pass through the netbird-service-sync role —
# policies/netbird-service-sync.hcl, applied once by hand like the rotator
# roles. Until that role exists the pod logs an auth error each minute and does
# nothing.
apiVersion: v1
kind: ServiceAccount
metadata:
name: netbird-proxy-services
namespace: openbao
name: netbird-service-sync
namespace: netbird-service-sync
labels:
app.kubernetes.io/name: netbird-proxy-services
app.kubernetes.io/name: netbird-service-sync
---
# Read-only, cluster-wide: public Ingresses live in many namespaces, the
# Traefik Service in ingress/.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: netbird-proxy-services-read
name: netbird-service-sync-read
rules:
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
@@ -34,23 +34,23 @@ rules:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: netbird-proxy-services-read
name: netbird-service-sync-read
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: netbird-proxy-services-read
name: netbird-service-sync-read
subjects:
- kind: ServiceAccount
name: netbird-proxy-services
namespace: openbao
name: netbird-service-sync
namespace: netbird-service-sync
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: netbird-proxy-services
namespace: openbao
name: netbird-service-sync
namespace: netbird-service-sync
labels:
app.kubernetes.io/name: netbird-proxy-services
app.kubernetes.io/name: netbird-service-sync
spec:
replicas: 1
# One writer against the management API at a time.
@@ -58,13 +58,13 @@ spec:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: netbird-proxy-services
app.kubernetes.io/name: netbird-service-sync
template:
metadata:
labels:
app.kubernetes.io/name: netbird-proxy-services
app.kubernetes.io/name: netbird-service-sync
spec:
serviceAccountName: netbird-proxy-services
serviceAccountName: netbird-service-sync
securityContext:
runAsNonRoot: true
# Numeric, or the kubelet refuses the image's named user with
@@ -80,7 +80,7 @@ spec:
# image directory (branch and master alike) and the registry keeps
# only the newest manifest per tag, so the digest must come from
# the MASTER build of the code change.
image: gitea.renner.uno/gitea_admin/netbird-proxy-services:2026-09-07@sha256:caffa308d802c6fcc6aea919ac94657c08b993651baa6b8d18de395bcc40523e
image: gitea.renner.uno/gitea_admin/netbird-service-sync:2026-09-23@sha256:64f6057c508640e1dbc50457dc26e9b2de058a111fd46ca096021d7444b5bbff
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
@@ -92,7 +92,7 @@ spec:
- name: BAO_ADDR
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: netbird-proxy-services
value: netbird-service-sync
# Must match cluster.network.serviceSubnets in talos/patches/cluster.yaml.
- name: SERVICE_CIDR
value: "10.96.0.0/12"
@@ -148,3 +148,37 @@ spec:
- ports:
- port: "8080"
protocol: TCP
---
# wiki-index: Traefik (LAN clients behind BasicAuth) and Gitea (push webhook)
# may reach it; it reaches OpenBao for its credentials. Gitea itself is reached
# through Traefik, which the namespace egress above already allows.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: wiki-index
namespace: ai
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: wiki-index
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
- matchLabels:
"k8s:io.kubernetes.pod.namespace": gitea
app.kubernetes.io/name: gitea
toPorts:
- ports:
- port: "8080"
protocol: TCP
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": openbao
app.kubernetes.io/name: openbao
toPorts:
- ports:
- port: "8200"
protocol: TCP
@@ -0,0 +1,104 @@
---
# Baseline policy for the `gitea` namespace (issue #878).
#
# endpointSelector: {} — every rule applies to every pod in the namespace.
# Workload-scoped grants go in their own policy with their own selector.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: default-deny
namespace: gitea
spec:
endpointSelector: {}
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": gitea
# Prometheus scrape of /metrics.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": monitoring
# Kubelet probes and hostNetwork siblings.
- fromEntities:
- host
- remote-node
# ICMP from any cluster pod; required once an L4 policy selects the pod.
# See woodpecker-ns.yaml for the rationale. INGRESS ONLY.
- fromEndpoints:
- matchExpressions:
- key: io.kubernetes.pod.namespace
operator: Exists
icmps:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 143, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": gitea
# Webhooks to hosts served by Traefik (ci.renner.uno).
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
# Push webhook of the notes repo to wiki-index.
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ai
app.kubernetes.io/name: wiki-index
toPorts:
- ports:
- port: "8080"
protocol: TCP
# Push mirror to GitHub and other external targets.
- toEntities:
- host
- remote-node
- world
---
# Traefik serves git.renner.uno / gitea.renner.uno (:3000) and the gitea-ssh
# entrypoint (:2222). Lives beside the baseline so both land in one reconcile.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: gitea-allow-traefik
namespace: gitea
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: gitea
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
toPorts:
- ports:
- port: "3000"
protocol: TCP
- port: "2222"
protocol: TCP
@@ -22,6 +22,8 @@ resources:
- external-secrets-ns.yaml
- woodpecker-ci-ns.yaml
- woodpecker-ns.yaml
- gitea-ns.yaml
- netbird-service-sync-ns.yaml
- trmnl-ns.yaml
- wordpress-ns.yaml
- ai-ns.yaml
@@ -0,0 +1,43 @@
---
# Baseline policy for the `netbird-service-sync` namespace (issue #876).
# The controller serves nothing; it only calls out: the Kubernetes API
# (Ingress/Service reads), OpenBao (NetBird PAT) and the NetBird management API.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: default-deny
namespace: netbird-service-sync
spec:
endpointSelector: {}
ingress:
- fromEntities:
- host
- remote-node
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
- toEntities:
- kube-apiserver
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": openbao
app.kubernetes.io/name: openbao
toPorts:
- ports:
- port: "8200"
protocol: TCP
# NetBird management API (netbird.renner.uno).
- toEntities:
- world
toPorts:
- ports:
- port: "443"
protocol: TCP
@@ -31,6 +31,26 @@ spec:
- ports:
- port: "8200"
protocol: TCP
# wiki-index (ns ai) logs in via auth/kubernetes/role/wiki-index and reads
# its Gitea token and webhook secret on every sync.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ai
app.kubernetes.io/name: wiki-index
toPorts:
- ports:
- port: "8200"
protocol: TCP
# netbird-service-sync logs in via auth/kubernetes/role/netbird-service-sync
# and reads the NetBird PAT on every pass.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": netbird-service-sync
app.kubernetes.io/name: netbird-service-sync
toPorts:
- ports:
- port: "8200"
protocol: TCP
# (Secret-rotation CronJobs run in THIS namespace since 2026-07-18, so their
# OpenBao access is covered by the intra-namespace ingress rule above — no
# per-app rule needed.)
@@ -117,7 +137,7 @@ spec:
# Egress complement to gitea-allow-flux-rotator (apps/gitea/networkpolicy.yaml).
# The flux-git rotator mints a PAT against the in-cluster gitea Service; the
# openbao default-deny only permits TCP egress to openbao/kube-dns/apiserver/
# world, and gitea (apps ns) is in-cluster, so it needs an explicit allow.
# world, and gitea (gitea ns) is in-cluster, so it needs an explicit allow.
# Scoped to the flux-git rotator label and gitea :3000.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
@@ -132,7 +152,7 @@ spec:
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": apps
"k8s:io.kubernetes.pod.namespace": gitea
app.kubernetes.io/name: gitea
toPorts:
- ports:
@@ -140,10 +160,10 @@ spec:
protocol: TCP
---
# Egress for the review-bot rotator. Its CronJob has two steps: the init step
# mints review-bot's PAT against gitea (apps :3000), the main step upserts the
# mints review-bot's PAT against gitea (gitea :3000), the main step upserts the
# Woodpecker org secret against the woodpecker server (woodpecker :8000). Both run
# under the same pod labels, so one egress policy covers both hops. Complements
# gitea-allow-review-bot-rotator (apps) + woodpecker-allow-rotators (woodpecker).
# gitea-allow-review-bot-rotator (gitea) + woodpecker-allow-rotators (woodpecker).
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
@@ -157,7 +177,7 @@ spec:
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": apps
"k8s:io.kubernetes.pod.namespace": gitea
app.kubernetes.io/name: gitea
toPorts:
- ports:
@@ -1,23 +0,0 @@
# OpenBao policy: netbird-proxy-services
#
# The reverse-proxy-services controller (Deployment netbird-proxy-services in
# the openbao namespace) reads the NetBird PAT on every reconcile pass and
# calls the management API with it. Read-only on that one path; it shares the
# bootstrap credential with rotator-netbird but not that role's PATCH rights.
#
# NOT applied by Flux. Apply ONCE with an admin token:
#
# bao policy write netbird-proxy-services kubernetes/infrastructure/openbao/policies/netbird-proxy-services.hcl
# bao write auth/kubernetes/role/netbird-proxy-services \
# bound_service_account_names=netbird-proxy-services \
# bound_service_account_namespaces=openbao \
# policies=netbird-proxy-services ttl=15m
#
# The PAT itself is the one rotator-netbird already uses
# (homelab/infrastructure/secret-rotators-bootstrap/netbird, key `pat`).
#
# Do NOT set token_no_default_policy=true (kv-v2 preflight needs the default policy).
path "homelab/data/infrastructure/secret-rotators-bootstrap/netbird" {
capabilities = ["read"]
}
@@ -0,0 +1,24 @@
# OpenBao policy: netbird-service-sync
#
# The controller that syncs public Ingresses into NetBird reverse-proxy services
# (Deployment netbird-service-sync in the netbird-service-sync namespace) reads
# the NetBird PAT on every reconcile pass and calls the management API with it.
# Read-only on that one path; it shares the bootstrap credential with
# rotator-netbird but not that role's PATCH rights.
#
# NOT applied by Flux. Apply ONCE with an admin token:
#
# bao policy write netbird-service-sync kubernetes/infrastructure/openbao/policies/netbird-service-sync.hcl
# bao write auth/kubernetes/role/netbird-service-sync \
# bound_service_account_names=netbird-service-sync \
# bound_service_account_namespaces=netbird-service-sync \
# policies=netbird-service-sync ttl=15m
#
# The PAT itself is the one rotator-netbird already uses
# (homelab/infrastructure/secret-rotators-bootstrap/netbird, key `pat`).
#
# Do NOT set token_no_default_policy=true (kv-v2 preflight needs the default policy).
path "homelab/data/infrastructure/secret-rotators-bootstrap/netbird" {
capabilities = ["read"]
}
@@ -0,0 +1,21 @@
# OpenBao policy: wiki-index
#
# The wiki-index Deployment (namespace ai) reads its Gitea token and the
# webhook HMAC secret on every use. Read-only on that one path.
#
# NOT applied by Flux. Apply ONCE with an admin token:
#
# bao policy write wiki-index kubernetes/infrastructure/openbao/policies/wiki-index.hcl
# bao write auth/kubernetes/role/wiki-index \
# bound_service_account_names=wiki-index \
# bound_service_account_namespaces=ai \
# policies=wiki-index ttl=15m
#
# Secret: homelab/apps/ai/wiki-index, keys `gitea_token` (read:repository on
# the notes repo) and `webhook_secret` (the Gitea webhook's secret).
#
# Do NOT set token_no_default_policy=true (kv-v2 preflight needs the default policy).
path "homelab/data/apps/ai/wiki-index" {
capabilities = ["read"]
}
@@ -6,7 +6,7 @@
# newest. A token that fails verification is deleted and the run aborts without
# storing, so Flux is never stranded.
#
# gitea is reached at the in-cluster Service gitea.apps.svc:3000 — the public
# gitea is reached at the in-cluster Service gitea.gitea.svc:3000 — the public
# name hairpins off the ingress LB from a pod. Needs BOTH netpol halves
# (gitea-allow-flux-rotator + rotator-gitea-flux-egress).
#
@@ -76,7 +76,7 @@ spec:
- name: BAO_PATH
value: homelab/infrastructure/flux-git
- name: GITEA_URL
value: http://gitea.apps.svc.cluster.local:3000
value: http://gitea.gitea.svc.cluster.local:3000
- name: GITEA_VERIFY_REPO
value: k.renner/homelab
- name: GITEA_SCOPES
@@ -24,7 +24,7 @@
# synced. Quarterly cadence + backoffLimit:1 makes a two-cycle failure unlikely,
# but a failed sync should be fixed before the next run rather than left.
#
# gitea is reached in-cluster at gitea.apps.svc:3000, woodpecker at
# gitea is reached in-cluster at gitea.gitea.svc:3000, woodpecker at
# woodpecker.woodpecker.svc:8000 (it moved out of `apps` in the namespace
# migration — see kubernetes/apps/woodpecker/MIGRATION-namespace.md). Needs
# the netpol halves: gitea-allow-review-bot-rotator + woodpecker-allow-rotators
@@ -99,7 +99,7 @@ spec:
- name: BAO_PATH
value: homelab/infrastructure/review-bot
- name: GITEA_URL
value: http://gitea.apps.svc.cluster.local:3000
value: http://gitea.gitea.svc.cluster.local:3000
# Private repo review-bot collaborates on: the pull check is a
# real access verification (git-upload-pack on a PRIVATE repo
# needs a token with access), not the "any valid token" weakness
@@ -14,6 +14,7 @@ spec:
# ClusterSecretStore" and the secret never syncs. Keep this list sorted.
conditions:
- namespaces:
- ai # wiki-embed-basic-auth (apps/llama-server/wiki-embed-ingress.yaml)
- apps
- audiobookshelf
- authentik
@@ -225,15 +225,8 @@ metadata:
labels:
app.kubernetes.io/part-of: homelab-backup
spec:
# Daily at 07:15 local (next free stagger slot after home-assistant-daily
# at 06:45). Covers every stateful workload in the shared `apps`
# namespace — currently Gitea (data-gitea-0, the Flux GitOps source of
# truth) and Woodpecker (data-woodpecker-server-0), plus gatus-data.
# Added 2026-07-14 (issue #186): the apps namespace had no Velero
# schedule at all, so the in-cluster Gitea/Woodpecker PVCs migrated from
# the legacy LXCs (see MIGRATION-gitea-woodpecker.md) were unprotected.
# No CNPG cluster runs in this namespace (both Gitea and Woodpecker use
# SQLite), so no labelSelector exclusion is needed.
# Daily at 07:15 local. Every PVC in the shared `apps` namespace (gatus-data).
# No CNPG cluster runs here, so no labelSelector exclusion is needed.
schedule: "15 7 * * *"
useOwnerReferencesInBackup: false
template:
@@ -247,6 +240,26 @@ spec:
---
apiVersion: velero.io/v1
kind: Schedule
metadata:
name: gitea-daily
namespace: velero
labels:
app.kubernetes.io/part-of: homelab-backup
spec:
# Daily at 07:45 local. data-gitea-0 (SQLite, repos, LFS, packages).
schedule: "45 7 * * *"
useOwnerReferencesInBackup: false
template:
includedNamespaces:
- gitea
includedResources:
- persistentvolumeclaims
storageLocation: default
ttl: 720h0m0s
snapshotMoveData: true
---
apiVersion: velero.io/v1
kind: Schedule
metadata:
name: trmnl-daily
namespace: velero
@@ -96,7 +96,7 @@ spec:
# that traffic is always a pod IP — collapsing all external clients
# to one source. The NetBird reverse-proxy services are TLS
# passthrough and carry the WAN client address in a PROXY protocol
# v2 header (options.proxy_protocol, set by netbird-proxy-services);
# v2 header (options.proxy_protocol, set by netbird-service-sync);
# X-Forwarded-For stays trusted for HTTP-mode services. Both are
# accepted from the pod CIDRs only; connections without a header
# (direct LAN/mesh) keep their RemoteAddr.
+6
View File
@@ -58,6 +58,12 @@ machine:
imageGCHighThresholdPercent: 70
imageGCLowThresholdPercent: 60
cluster:
# etcd peers on node0, VLAN 46 (#576). Pinned: Talos never updates an
# existing member's peer URLs, so changing this subnet means leave + rejoin
# per member (talosctl etcd leave, reboot).
etcd:
advertisedSubnets:
- 10.2.46.0/24
# controlPlane.endpoint is NOT here: `talosctl gen config` takes it as its
# second argument, and bootstrap.sh already passes https://k8s.renner.uno:6443
# there. Setting it again in this patch restated the same value in a second
+17 -2
View File
@@ -17,6 +17,8 @@ machine:
# pod0 is a closed L2 fabric between the three nodes: statically
# addressed, no gateway, nothing to learn from an RA.
net.ipv6.conf.pod0.accept_ra: "0"
# node0 is closed the same way.
net.ipv6.conf.node0.accept_ra: "0"
# ingress0 keeps RAs, at the default accept_ra of 2: an IPv6 default
# router is a link-local address learned only from the RA, and with
# net.ipv6.conf.all.forwarding=1 — which Cilium sets — Linux ignores RAs
@@ -122,8 +124,8 @@ kind: LinkConfig
name: uplink
up: true
---
# Management, VLAN 40. Carries the node IP, etcd, the API and Ceph's public
# network.
# Management, VLAN 40: world↔node. The API VIP, admin access to the nodes, and
# node egress (DNS, NTP, image pulls). Node↔node traffic is on node0 (#576).
apiVersion: v1alpha1
kind: VLANConfig
name: mgmt0
@@ -177,6 +179,19 @@ addresses:
- address: 10.2.48.11/24
- address: fd02:48::11/64
---
# Node network, VLAN 46: traffic among the three nodes themselves (#576) — the
# node IP, etcd, apiserver↔kubelet. Closed like pod0: statically addressed, no
# gateway, never routed via the UDM.
apiVersion: v1alpha1
kind: VLANConfig
name: node0
vlanID: 46
parent: uplink
up: true
addresses:
- address: fd02:46::11/64
- address: 10.2.46.11/24
---
# Ingress network, VLAN 47. World-to-pod: the BGP session, the next hop for
# every advertised LoadBalancer VIP, and the masquerade source for pod egress.
# Publicly routed, which is what separates it from pod0.
+13 -1
View File
@@ -16,6 +16,7 @@ machine:
# ingress0 keeps RAs: it learns its IPv6 router and its address from them,
# and only the default route is held back — see k8s-1.yaml.
net.ipv6.conf.pod0.accept_ra: "0"
net.ipv6.conf.node0.accept_ra: "0"
net.ipv6.conf.ingress0.accept_ra_defrtr: "0"
# Answer ARP only on the link that owns the address — see k8s-1.yaml (#793).
net.ipv4.conf.all.arp_ignore: "1"
@@ -82,7 +83,7 @@ kind: LinkConfig
name: uplink
up: true
---
# Management, VLAN 40. Node IP, etcd, the API and Ceph's public network.
# Management, VLAN 40: world↔node — see k8s-1.yaml.
apiVersion: v1alpha1
kind: VLANConfig
name: mgmt0
@@ -122,6 +123,17 @@ addresses:
- address: 10.2.48.12/24
- address: fd02:48::12/64
---
# Node network, VLAN 46 — see k8s-1.yaml.
apiVersion: v1alpha1
kind: VLANConfig
name: node0
vlanID: 46
parent: uplink
up: true
addresses:
- address: fd02:46::12/64
- address: 10.2.46.12/24
---
# Ingress network, VLAN 47. World-to-pod, publicly routed, statically
# addressed — see k8s-1.yaml for why it is not leased.
apiVersion: v1alpha1
+12
View File
@@ -12,6 +12,7 @@ machine:
# ingress0 keeps RAs: it learns its IPv6 router and its address from them,
# and only the default route is held back — see k8s-1.yaml.
net.ipv6.conf.pod0.accept_ra: "0"
net.ipv6.conf.node0.accept_ra: "0"
net.ipv6.conf.ingress0.accept_ra_defrtr: "0"
# Answer ARP only on the link that owns the address — see k8s-1.yaml (#793).
net.ipv4.conf.all.arp_ignore: "1"
@@ -109,6 +110,17 @@ addresses:
- address: 10.2.48.13/24
- address: fd02:48::13/64
---
# Node network, VLAN 46 — see k8s-1.yaml.
apiVersion: v1alpha1
kind: VLANConfig
name: node0
vlanID: 46
parent: uplink
up: true
addresses:
- address: fd02:46::13/64
- address: 10.2.46.13/24
---
# Ingress network, VLAN 47. World-to-pod, publicly routed, statically
# addressed — see k8s-1.yaml for why it is not leased.
apiVersion: v1alpha1
+7 -11
View File
@@ -7,19 +7,15 @@
# part of the config that could not be true anywhere but here.
machine:
kubelet:
# Constrain kubelet's auto-picked node IP to the mgmt LAN. Without this,
# kubelet picks the first non-loopback IP, which could land on the v30iot
# NIC (ens19, 10.2.30.0/24). Control-plane traffic and hostNetwork pod IPs
# must stay on mgmt LAN.
# fd40::/64 is the mgmt LAN ULA, and it is listed so the Node object carries
# an IPv6 InternalIP at all. Without one, Cilium falls back to detecting the
# node's IPv6 from mgmt0 and picks the SLAAC address from the dynamic
# provider prefix, which it then probes until the agent restarts (#789).
# The GUA is not a control-plane address: BGP peers on fd40::/64 too.
# The node IP lives on node0, VLAN 46: the closed node↔node network (#576).
# Both families are listed so the Node object carries an IPv6 InternalIP at
# all; without one Cilium detects the node's IPv6 itself and picks a SLAAC
# address from the dynamic provider prefix (#789). Talos picks one address
# per family, IPv4 first, whatever the order here.
nodeIP:
validSubnets:
- 10.2.40.0/24
- fd40::/64
- fd02:46::/64
- 10.2.46.0/24
# The UDM on the ingress segment. Published here rather than as a route
# because bird owns table 47 and originates the default from it; this is the
# one place the address is stated.
+30
View File
@@ -0,0 +1,30 @@
#!/bin/sh
# Turn the TypeSafe picker's probability distribution into one model id.
#
# The judgment (how well each catalog model fits the task) comes from the
# model; the policy lives here, in code: among the candidates whose probability
# is within MARGIN of the best one, take the most recently released. Several
# same-family models (Sonnet 4.5 / 4.6 / 5) split the probability for a
# mid-weight task, and the newest of a comparable set is the one to run.
#
# Usage: choose-model.sh <catalog-table-file> < <systemone-response.json>
# The table is fetch-models.sh --table output: id \t name \t released=YYYY-MM-DD ...
# Prints the chosen id, or nothing when the answer is NONE / unusable.
# Env: CHOOSE_MODEL_MARGIN (default 0.7): candidate if p >= MARGIN * p_best.
set -eu
TABLE_FILE="${1:?usage: choose-model.sh <catalog-table-file> < response.json}"
MARGIN="${CHOOSE_MODEL_MARGIN:-0.7}"
jq -r --rawfile table "$TABLE_FILE" --argjson margin "$MARGIN" '
($table | split("\n") | map(select(length > 0) | split("\t")
| {key: .[0], value: ((.[2:] | map(select(startswith("released="))) | .[0] // "released=") | ltrimstr("released="))})
| from_entries) as $released
| (.answers.model.probabilities // {}) | to_entries
| map(select(.key != "NONE")) | sort_by(-.value)
| if length == 0 then empty
else (.[0].value) as $best
| map(select(.value >= $margin * $best))
| sort_by([($released[.key] // ""), .value])
| .[-1].key
end' 2>&1
+36
View File
@@ -33,6 +33,42 @@ none() { log "$1"; exit 0; } # emit nothing -> caller uses CLI defaul
TABLE="$(sh "$HERE/fetch-models.sh" --table || true)"
[ -n "$TABLE" ] || none "no catalog available — caller will use the CLI default model"
# First choice: one TypeSafe (System One) Choice over the catalog. The task and
# the catalog rows are state, every model id is an option with its catalog row
# as the criterion, and the answer is typed — no prompt, no JSON parsing, and a
# request that takes a fraction of a second instead of a claude run. The id is
# still validated against the pristine catalog. Falls through to the claude
# picker when the key is absent or the request fails.
if [ -n "${TYPESAFE_API_KEY:-}" ]; then
REQ="$(printf '%s\n' "$TABLE" | jq -R -s --arg task "$GUIDANCE" '
split("\n") | map(select(length > 0) | split("\t") | {id: .[0], name: .[1], info: (.[2:] | join(", "))})
| {model: "jev-latest",
state: {task: $task, models: .},
questions: {model: {
type: "choice",
instructions: "Which model in `models` should run `task`? Match the model to the task: not the largest or newest flagship for trivial or mechanical work, not a minimal model for hard, multi-step or security-sensitive work; prefer a newer model when otherwise comparable. Consider every listed model, including unfamiliar ones. Pick NONE only if no listed model fits.",
criteria: ((map({key: .id, value: "\(.name); \(.info)"}) | from_entries) + {NONE: "No listed model fits the task"})}}}')"
if [ -n "${PICK_MODEL_TYPESAFE_DRYRUN:-}" ]; then printf '%s\n' "$REQ"; exit 0; fi
RESP="$(printf '%s' "$REQ" | curl -sS --max-time 30 -X POST https://api.typesafe.ai/v1/systemone \
-H "Authorization: Bearer $TYPESAFE_API_KEY" -H 'Content-Type: application/json' --data-binary @- || true)"
# Policy in code (choose-model.sh): among candidates within a margin of the
# best probability, the most recently released wins. Same-family models split
# the probability for mid-weight tasks (live catalog: Sonnet 4.5 chosen at
# 0.23 over Sonnet 4.6).
TABLE_FILE="$(mktemp)"
printf '%s\n' "$TABLE" > "$TABLE_FILE"
CHOICE="$(printf '%s' "$RESP" | sh "$HERE/choose-model.sh" "$TABLE_FILE" | head -1)"
rm -f "$TABLE_FILE"
TOP="$(printf '%s' "$RESP" | jq -r '(.answers.model.probabilities // {}) | to_entries | sort_by(-.value) | .[0:4] | map("\(.key)=\(.value)") | join(" ")' 2>&1 | head -1)"
VALID="$(printf '%s\n' "$TABLE" | sh "$HERE/valid-model.sh" "$CHOICE")"
if [ -n "$VALID" ]; then
log "typesafe chose '$VALID' (top: ${TOP:-?}) for: $GUIDANCE"
printf '%s\n' "$VALID"
exit 0
fi
log "typesafe picker returned '${CHOICE:-<empty>}' (rejected, NONE, or request failed) — falling back to the claude picker"
fi
PROMPT="$(cat <<EOF
You are a model-selection agent for an automated CI job. From the models
currently available (listed below with their capabilities), choose the single
+31
View File
@@ -84,9 +84,40 @@ else
printf 'FAIL recorded fixture: opus=[%s] haiku=[%s]\n' "$opusline" "$haikuline"; fails=$((fails + 1))
fi
# TypeSafe path of pick-model.sh: the request offers exactly the catalog ids plus
# NONE as options, carries the task as state, and never sends anything else.
printf '%s' '{"data":[{"type":"model","id":"claude-opus-5","display_name":"Claude Opus 5","created_at":"2026-07-24T00:00:00Z","max_input_tokens":1000000,"max_tokens":128000},{"type":"model","id":"claude-haiku-4-5-20251001","display_name":"Claude Haiku 4.5","created_at":"2025-10-15T00:00:00Z","max_input_tokens":200000,"max_tokens":64000}],"has_more":false}' > "$FX"
req="$(FETCH_MODELS_BODY_FILE="$FX" TYPESAFE_API_KEY=dry PICK_MODEL_TYPESAFE_DRYRUN=1 sh "$HERE/pick-model.sh" "a task; with \"quotes\" and $ signs" 2>/dev/null || true)"
keys="$(printf '%s' "$req" | jq -r '.questions.model.criteria | keys | join(",")' 2>/dev/null || true)"
task="$(printf '%s' "$req" | jq -r '.state.task' 2>/dev/null || true)"
if [ "$keys" = "NONE,claude-haiku-4-5-20251001,claude-opus-5" ] && [ "$task" = 'a task; with "quotes" and $ signs' ]; then
printf 'ok typesafe request offers catalog ids + NONE and carries the task verbatim\n'
else
printf 'FAIL typesafe request: keys [%s] task [%s]\n' "$keys" "$task"; fails=$((fails + 1))
fi
# choose-model.sh: newest among comparable candidates, best otherwise, NONE -> nothing.
TB="$(mktemp)"
printf '%s\n' 'claude-sonnet-4-5 Claude Sonnet 4.5 released=2025-09-29 context=200000' \
'claude-sonnet-4-6 Claude Sonnet 4.6 released=2026-02-17 context=200000' \
'claude-opus-5 Claude Opus 5 released=2026-07-24 context=1000000' \
'claude-haiku-4-5 Claude Haiku 4.5 released=2025-10-15 context=200000' > "$TB"
choose() { printf '%s' "$1" | sh "$HERE/choose-model.sh" "$TB" || true; }
got="$(choose '{"answers":{"model":{"probabilities":{"claude-sonnet-4-5":0.30,"claude-sonnet-4-6":0.25,"claude-opus-5":0.10,"NONE":0.05}}}}')"
if [ "$got" = "claude-sonnet-4-6" ]; then printf 'ok newer sibling within margin wins\n'; else printf 'FAIL comparable: got [%s]\n' "$got"; fails=$((fails + 1)); fi
got="$(choose '{"answers":{"model":{"probabilities":{"claude-opus-5":0.90,"claude-haiku-4-5":0.05,"claude-sonnet-4-6":0.04,"NONE":0.01}}}}')"
if [ "$got" = "claude-opus-5" ]; then printf 'ok clear best is kept\n'; else printf 'FAIL clear best: got [%s]\n' "$got"; fails=$((fails + 1)); fi
got="$(choose '{"answers":{"model":{"probabilities":{"claude-haiku-4-5":0.60,"claude-sonnet-4-5":0.35,"NONE":0.05}}}}')"
if [ "$got" = "claude-haiku-4-5" ]; then printf 'ok older best outside margin is kept\n'; else printf 'FAIL margin: got [%s]\n' "$got"; fails=$((fails + 1)); fi
got="$(choose '{"answers":{"model":{"probabilities":{"NONE":1.0}}}}')"
if [ -z "$got" ]; then printf 'ok NONE yields nothing\n'; else printf 'FAIL NONE: got [%s]\n' "$got"; fails=$((fails + 1)); fi
rm -f "$TB"
if [ "$fails" -eq 0 ]; then
echo "All model-selection tests passed."
else
echo "$fails test(s) FAILED."
exit 1
fi