Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5 ac16e53098 fix(flux): four cold-rebuild ordering deadlocks
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Found by rebuild rehearsals; each stopped a cold start outright and none
changes behaviour on a running cluster.

infrastructure-ceph-cluster no longer dependsOn infrastructure. That
edge closed a cycle no new cluster could escape: infrastructure is Ready
only once its openbao healthCheck passes, openbao-0 runs only once its
PVC binds, and the PVC wants the ceph-block StorageClass defined in this
very Kustomization. infrastructure runs wait: false, so everything this
path needs is applied long before readiness; on an empty API server it
self-heals on retryInterval like post-crds already does.

home-assistant gains dependsOn infrastructure-cnpg-barman: its
ObjectStore kind arrives with the Barman plugin, and one unknown kind
fails the server-side dry-run for the whole apply set — taking
speech-to-phrase, zone-configurator and matter-server down with it.
Every other ObjectStore user already declares the edge.

woodpecker-ci namespace moves to infrastructure/namespaces: its
CiliumNetworkPolicies ship with infrastructure-crds, which runs before
every app KS, so declaring the namespace in the workload KS inverted the
order and held infrastructure-crds — and the twenty app Kustomizations
behind it — on "namespaces woodpecker-ci not found". Same reasoning as
its sibling namespace, previously not carried across.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-06 20:23:05 +02:00
4 changed files with 55 additions and 17 deletions
-15
View File
@@ -1,18 +1,3 @@
# Dedicated namespace where the kubernetes backend spawns pipeline step pods,
# isolated from the apps namespace.
apiVersion: v1
kind: Namespace
metadata:
name: woodpecker-ci
labels:
app.kubernetes.io/name: woodpecker
# Image-build steps run a privileged docker/buildx plugin pod (see the
# server's WOODPECKER_PLUGINS_PRIVILEGED). Talos enforces Pod Security;
# without this the privileged step pods are rejected at admission. The
# namespace is dedicated to ephemeral CI step pods, so scoping the
# exemption here keeps it off apps/.
pod-security.kubernetes.io/enforce: privileged
---
apiVersion: v1
kind: ServiceAccount
metadata:
@@ -28,6 +28,19 @@ spec:
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
# ha-pg-objectstore.yaml declares an ObjectStore, whose CRD comes from the
# CNPG Barman plugin in infrastructure-cnpg-barman. Without this edge the
# kind is unknown when this path is first applied, and a server-side
# dry-run rejects the ENTIRE set over the one object it cannot recognise:
#
# ObjectStore/home-assistant/ha-pg-store dry-run failed:
# no matches for kind "ObjectStore" in version ...
#
# Every other ObjectStore user — authentik, immich, paperless, trmnl,
# woodpecker — already declares it. This was the only one that did not, and
# it took speech-to-phrase, zone-configurator and matter-server down with
# it, since all three wait on home-assistant.
- name: infrastructure-cnpg-barman
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
@@ -122,8 +122,20 @@ spec:
path: ./kubernetes/infrastructure/rook-ceph-cluster
prune: true
wait: true
dependsOn:
- name: infrastructure
# Deliberately NOT dependsOn: infrastructure. That edge closed a cycle no
# existing cluster could ever hit and no new one could ever escape:
# infrastructure is Ready only once its openbao healthCheck passes, openbao-0
# only runs once its PVC binds, and that PVC wants the ceph-block StorageClass
# defined in this very Kustomization. A cold rebuild stopped here for good,
# with openbao-0 Pending on "unbound immediate PersistentVolumeClaims".
#
# The edge was never needed. infrastructure runs wait: false, so it applies
# the rook-ceph namespace and the operator immediately and only its readiness
# waits on the healthChecks — everything this path needs exists long before
# infrastructure reports Ready. On an empty API server this Kustomization
# retries on retryInterval until the operator installs its CRDs, the same
# self-healing shape post-crds already relies on; on a live cluster the CRDs
# are always present and nothing changes.
# kstatus doesn't know how to read Rook CR readiness, so without these
# this Kustomization reports Ready as soon as the manifests apply — long
# before OSDs/mons/MDS are actually serving I/O. Apps depending on this
@@ -190,3 +190,31 @@ apiVersion: v1
kind: Namespace
metadata:
name: ai
---
# Where the Woodpecker kubernetes backend spawns pipeline step pods, isolated
# from `apps`. Distinct from `woodpecker`, which holds the service itself.
#
# Here rather than in apps/woodpecker for the reason spelled out above that
# namespace: post-crds ships this namespace's CiliumNetworkPolicies via
# ../network-policies/, and infrastructure-crds runs before every app KS. With
# the namespace declared in the workload KS the order inverted, and a rebuild
# stopped with
#
# CiliumNetworkPolicy/woodpecker-ci/default-deny-ingress not found:
# namespaces "woodpecker-ci" not found
#
# which held infrastructure-crds, and with it all twenty app Kustomizations
# that depend on it. The same reasoning was already written down for
# `woodpecker` and simply not carried across to its sibling.
#
# Image-build steps run a privileged docker/buildx plugin pod (see the server's
# WOODPECKER_PLUGINS_PRIVILEGED). Talos enforces Pod Security; without the
# label the privileged step pods are rejected at admission. The namespace holds
# nothing but ephemeral CI step pods, so the exemption stays off apps/.
apiVersion: v1
kind: Namespace
metadata:
name: woodpecker-ci
labels:
app.kubernetes.io/name: woodpecker
pod-security.kubernetes.io/enforce: privileged