fix(eso): allow the woodpecker namespace to use the openbao store #505

2026-08-18T21:00:51Z

claude-bot commented

2026-08-18 21:00:51 +00:00

Stage 1 of #433 is currently failing in the cluster. One line fixes it.

What is broken

The wp-pg restore job errors and retries:

can't get credentials for cluster wp-pg: while getting secret
hetzner-s3-creds: secrets "hetzner-s3-creds" not found

Because the ExternalSecret behind it never synced:

$ kubectl get externalsecret -n woodpecker
NAME               STORE     STATUS               READY
hetzner-s3-creds   openbao   SecretSyncedError    False

$ kubectl get events -n woodpecker --field-selector involvedObject.name=hetzner-s3-creds
error processing spec.dataFrom[0].extract, err: using cluster store "openbao"
is not allowed from namespace "woodpecker": denied by spec.condition

The ClusterSecretStore carries an explicit namespace allowlist as a blast-radius control — without it, any namespace could read the whole homelab/* KV tree. woodpecker wasn't on it.

Why #504 didn't catch it

The file's own header states the requirement plainly:

MAINTENANCE: adding an ExternalSecret in a NEW namespace requires adding that namespace here first, or ESO reports "not allowed to reference ClusterSecretStore" and the secret never syncs.

So this was documented and I missed it. The reference sweep for #504 asked "what references woodpecker or resolves *.apps.svc?" — and answered that thoroughly. It never asked the other question a namespace move raises: what allowlists must the new namespace join? That's a different shape of dependency, and this store is one; there may be others of the same kind worth checking before stage 2.

Nothing static could have caught it either: every manifest is valid, kustomize build passes, and --dry-run=server accepts all of it. The denial happens inside ESO when it evaluates a cross-namespace reference at sync time.

Blast radius

Adding one namespace to the allowlist, in sorted position per the file's stated convention. It grants the woodpecker namespace the same access the apps namespace already has — which is where wp-pg lives today, so this is not new reach, just reach that follows the workload.

Live state is unaffected either way: the old apps/wp-pg is untouched and serving, and the only thing currently broken is the parallel restore, which has no consumers.

Verification

kubectl kustomize kubernetes/infrastructure/post-crds builds; the allowlist goes from 17 to 18 entries.
kubectl apply --dry-run=server accepts the ClusterSecretStore.
Post-merge: kubectl get externalsecret -n woodpecker should reach SecretSynced/Ready=True, the hetzner-s3-creds Secret should appear, and CNPG's recovery job should then get past the credentials step. I'll confirm and continue the stage-1 runbook from there.

**Stage 1 of #433 is currently failing in the cluster.** One line fixes it. ## What is broken The `wp-pg` restore job errors and retries: ``` can't get credentials for cluster wp-pg: while getting secret hetzner-s3-creds: secrets "hetzner-s3-creds" not found ``` Because the ExternalSecret behind it never synced: ``` $ kubectl get externalsecret -n woodpecker NAME STORE STATUS READY hetzner-s3-creds openbao SecretSyncedError False $ kubectl get events -n woodpecker --field-selector involvedObject.name=hetzner-s3-creds error processing spec.dataFrom[0].extract, err: using cluster store "openbao" is not allowed from namespace "woodpecker": denied by spec.condition ``` The `ClusterSecretStore` carries an explicit namespace allowlist as a blast-radius control — without it, any namespace could read the whole `homelab/*` KV tree. `woodpecker` wasn't on it. ## Why #504 didn't catch it The file's own header states the requirement plainly: > **MAINTENANCE:** adding an ExternalSecret in a NEW namespace requires adding that namespace here first, or ESO reports "not allowed to reference ClusterSecretStore" and the secret never syncs. So this was documented and I missed it. The reference sweep for #504 asked *"what references woodpecker or resolves `*.apps.svc`?"* — and answered that thoroughly. It never asked the other question a namespace move raises: **what allowlists must the new namespace join?** That's a different shape of dependency, and this store is one; there may be others of the same kind worth checking before stage 2. Nothing static could have caught it either: every manifest is valid, `kustomize build` passes, and `--dry-run=server` accepts all of it. The denial happens inside ESO when it evaluates a cross-namespace reference at sync time. ## Blast radius Adding one namespace to the allowlist, in sorted position per the file's stated convention. It grants the `woodpecker` namespace the same access the `apps` namespace already has — which is where wp-pg lives today, so this is not new reach, just reach that follows the workload. Live state is unaffected either way: the old `apps/wp-pg` is untouched and serving, and the only thing currently broken is the parallel restore, which has no consumers. ## Verification - `kubectl kustomize kubernetes/infrastructure/post-crds` builds; the allowlist goes from 17 to 18 entries. - `kubectl apply --dry-run=server` accepts the ClusterSecretStore. - Post-merge: `kubectl get externalsecret -n woodpecker` should reach `SecretSynced`/`Ready=True`, the `hetzner-s3-creds` Secret should appear, and CNPG's recovery job should then get past the credentials step. I'll confirm and continue the stage-1 runbook from there.

claude-bot added 1 commit 2026-08-18 21:00:52 +00:00

fix(eso): allow the woodpecker namespace to use the openbao store

ci/woodpecker/pr/ci-manifests-test Pipeline was successful

Details

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

ci/woodpecker/pr/pr-review Pipeline was successful

Details

8156d919e1

Stage 1 of #433 is failing in the cluster. The wp-pg restore job errors
with:

  can't get credentials for cluster wp-pg: while getting secret
  hetzner-s3-creds: secrets "hetzner-s3-creds" not found

and the ExternalSecret behind it reports SecretSyncedError:

  using cluster store "openbao" is not allowed from namespace
  "woodpecker": denied by spec.condition

The ClusterSecretStore carries an explicit namespace allowlist, and the new
namespace was not on it. Its own header says so — "adding an ExternalSecret
in a NEW namespace requires adding that namespace here first, or ESO
reports 'not allowed to reference ClusterSecretStore' and the secret never
syncs" — so the requirement was documented and simply missed.

The reference sweep for #504 looked for everything that referenced
woodpecker or resolved *.apps.svc. It never asked the other question a
namespace move raises: what allowlists must the NEW namespace join? This
allowlist is exactly that shape, and it is the kind of thing kustomize
build and dry-run cannot surface, because the manifests are all valid — the
denial happens at ESO's admission of a cross-namespace reference.

Added in sorted position, keeping the list's stated ordering.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>

review-bot reviewed

2026-08-18 21:02:29 +00:00

review-bot left a comment

🤖 Automated review — model CLI default (complexity: complex), commit 8156d919e1a4e5fe56a64e326d7475a94edec206.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

Summary

One-line fix: adds woodpecker to the namespace allowlist on the shared openbao ClusterSecretStore (kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml:33). Verified against the checkout: kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml defines an ExternalSecret named hetzner-s3-creds in namespace woodpecker with secretStoreRef: {kind: ClusterSecretStore, name: openbao}, and it's wired into kubernetes/apps/woodpecker/kustomization.yaml:43. Without the namespace on the allowlist, ESO's spec.condition check on the store denies the cross-namespace reference, exactly matching the error in the PR description. No blocking issues.

🟡 Suggestions

None — this is a minimal, correctly scoped fix for a config-only allowlist. The addition is alphabetically sorted between velero and wordpress as the file's header comment requires (kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml:14).

Tests

No automated test guards this allowlist (it's a static YAML condition evaluated by ESO at sync time, as the PR body notes — kustomize build/--dry-run=server can't catch this class of error). That's consistent with the rest of the file; no existing test infrastructure covers it either. Nothing to add here beyond what the PR author already committed to doing post-merge (checking kubectl get externalsecret -n woodpecker reaches SecretSynced/Ready=True).

Recommendation

Looks reasonable — no blocking issues; comments above are optional.

🤖 **Automated review** — model `CLI default` (complexity: complex), commit `8156d919e1a4e5fe56a64e326d7475a94edec206`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary One-line fix: adds `woodpecker` to the namespace allowlist on the shared `openbao` `ClusterSecretStore` (`kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml:33`). Verified against the checkout: `kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml` defines an `ExternalSecret` named `hetzner-s3-creds` in namespace `woodpecker` with `secretStoreRef: {kind: ClusterSecretStore, name: openbao}`, and it's wired into `kubernetes/apps/woodpecker/kustomization.yaml:43`. Without the namespace on the allowlist, ESO's `spec.condition` check on the store denies the cross-namespace reference, exactly matching the error in the PR description. No blocking issues. ## 🟡 Suggestions - None — this is a minimal, correctly scoped fix for a config-only allowlist. The addition is alphabetically sorted between `velero` and `wordpress` as the file's header comment requires (`kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml:14`). ## Tests No automated test guards this allowlist (it's a static YAML condition evaluated by ESO at sync time, as the PR body notes — `kustomize build`/`--dry-run=server` can't catch this class of error). That's consistent with the rest of the file; no existing test infrastructure covers it either. Nothing to add here beyond what the PR author already committed to doing post-merge (checking `kubectl get externalsecret -n woodpecker` reaches `SecretSynced`/`Ready=True`). ## Recommendation Looks reasonable — no blocking issues; comments above are optional.

k.renner merged commit 597aa957ea into master

2026-08-18 21:04:52 +00:00

k.renner referenced this issue from a commit

2026-08-18 21:04:54 +00:00

Merge pull request 'fix(eso): allow the woodpecker namespace to use the openbao store' (#505) from fix-eso-woodpecker-namespace into master

claude-bot referenced this issue from a commit

2026-08-18 21:33:26 +00:00

feat(woodpecker): stage 3 — retire the apps cluster, promote the survivor

claude-bot referenced this pull request

2026-08-18 21:33:58 +00:00

feat(woodpecker): stage 3 — retire the apps cluster, promote the survivor #508

Sign in to join this conversation.