Closes the gap reviewers flagged on three separate PRs today: nothing validates kubernetes/**. deploy-infrastructure.yml gates only opentofu/**, and ci-scripts-test.yml runs only the scripts/ci/ unit tests — so a broken manifest reached the cluster and was caught by Flux failing to reconcile, or by someone noticing.
Four checks, all offline
No cluster, no registry, no secrets — every one passes or fails on the checkout alone.
step
what it catches
kustomize-build
resource entries pointing at missing files, bad patch targets, overlays that only break once combined
woodpecker-lint
pipeline configs that fail to compile
shellcheck
standalone .sh under kubernetes/, severity error
no-digest-pins
rotators reintroducing @sha256 pins
Each was verified green against the tree as it stands before being added, so this gates new breakage rather than failing on pre-existing debt: all 60 kustomizations build, all pipeline configs lint, all 6 shell scripts are clean at error severity, and no digest pins remain after #459.
The lint step is the one that matters
It exits 1 on the exact defect that took every pipeline in this repo down earlier today (#443) — an unquoted YAML scalar containing ": ", which yq accepts because it is valid YAML, just a map instead of a string. That's why "it parses" was mistaken for "it's valid".
Verified both directions in the image the step actually uses:
valid config -> rc=0
the #443 bug -> rc=1
and rc=0 on the pre-existing Additional property timeout is not allowed warnings in four deploy-*.yml files — so it fails on real breakage without blocking on legacy noise.
Image choices — all version-pinned, no floating tags
ci-k8s:2026-07-30 — kubectl for kustomize, and the shell for the grep guard. (I first wrote :latest here; caught and fixed, per the never-latest rule.)
woodpecker-cli:v3.16.0-alpine — the non-alpine image is scratch-based with no /bin/sh (19 MB, 2 layers), so Woodpecker commands: cannot run in it at all. The binary is /bin/woodpecker-cli, not on PATH as woodpecker. Both found by probing the image rather than assuming.
shellcheck-alpine:v0.11.0 — pinned rather than :stable, which floats exactly like :latest.
Correction — these two tags were bumped after the description was first written. Review flagged that the body still quoted woodpecker-cli:v3.9.0-alpine and shellcheck-alpine:v0.10.0; the probing described above was originally done against those. Both have since moved to latest stable (per the standing rule that any software runs the latest stable version — CLI tools included), and the properties re-verified against what actually ships: pipeline 3178 on 8a580e34 linted all 18 .woodpecker/*.yml files green under v3.16.0-alpine, which is only possible if that image has a working /bin/sh and /bin/woodpecker-cli. v3.16.0 also matches the deployed woodpecker-server, which matters because the CLI compiles configs against its own schema.
Deliberately not included
kubeconform. It needs CRD schemas for ExternalSecret, Certificate, HelmRelease and friends — so it either needs a schema-fetching step (network access, and a new flake source) or -ignore-missing-schemas, which would skip precisely the CRDs most of this tree is made of. Worth doing properly as its own change rather than adding a check that looks like coverage and isn't.
Known gap, stated rather than implied away
shellcheckcannot see shell embedded in manifests — init containers, probe exec blocks. That's where several real bugs lived this year (the grep --line-buffered no-op, the mosquitto_pub -W flag that doesn't exist). Called out in the file so nobody reads this pipeline as covering it.
Closes the gap reviewers flagged on **three separate PRs today**: nothing validates `kubernetes/**`. `deploy-infrastructure.yml` gates only `opentofu/**`, and `ci-scripts-test.yml` runs only the `scripts/ci/` unit tests — so a broken manifest reached the cluster and was caught by Flux failing to reconcile, or by someone noticing.
## Four checks, all offline
No cluster, no registry, no secrets — every one passes or fails on the checkout alone.
| step | what it catches |
|---|---|
| `kustomize-build` | resource entries pointing at missing files, bad patch targets, overlays that only break once combined |
| `woodpecker-lint` | pipeline configs that fail to compile |
| `shellcheck` | standalone `.sh` under `kubernetes/`, severity `error` |
| `no-digest-pins` | rotators reintroducing `@sha256` pins |
**Each was verified green against the tree as it stands before being added**, so this gates *new* breakage rather than failing on pre-existing debt: all **60** kustomizations build, all pipeline configs lint, all 6 shell scripts are clean at error severity, and no digest pins remain after #459.
## The lint step is the one that matters
It exits **1** on the exact defect that took every pipeline in this repo down earlier today (#443) — an unquoted YAML scalar containing `": "`, which `yq` accepts because it *is* valid YAML, just a map instead of a string. That's why "it parses" was mistaken for "it's valid".
Verified both directions **in the image the step actually uses**:
```
valid config -> rc=0
the #443 bug -> rc=1
```
and `rc=0` on the pre-existing `Additional property timeout is not allowed` warnings in four `deploy-*.yml` files — so it fails on real breakage without blocking on legacy noise.
## Image choices — all version-pinned, no floating tags
- **`ci-k8s:2026-07-30`** — `kubectl` for kustomize, and the shell for the grep guard. (I first wrote `:latest` here; caught and fixed, per the never-`latest` rule.)
- **`woodpecker-cli:v3.16.0-alpine`** — the non-alpine image is **scratch-based with no `/bin/sh`** (19 MB, 2 layers), so Woodpecker `commands:` cannot run in it at all. The binary is `/bin/woodpecker-cli`, not on `PATH` as `woodpecker`. Both found by probing the image rather than assuming.
- **`shellcheck-alpine:v0.11.0`** — pinned rather than `:stable`, which floats exactly like `:latest`.
> **Correction — these two tags were bumped after the description was first written.** Review flagged that the body still quoted `woodpecker-cli:v3.9.0-alpine` and `shellcheck-alpine:v0.10.0`; the probing described above was originally done against those. Both have since moved to latest stable (per the standing rule that *any* software runs the latest stable version — CLI tools included), and the properties re-verified against what actually ships: pipeline 3178 on `8a580e34` linted all 18 `.woodpecker/*.yml` files green under `v3.16.0-alpine`, which is only possible if that image has a working `/bin/sh` and `/bin/woodpecker-cli`. `v3.16.0` also matches the deployed `woodpecker-server`, which matters because the CLI compiles configs against its own schema.
## Deliberately not included
**`kubeconform`.** It needs CRD schemas for `ExternalSecret`, `Certificate`, `HelmRelease` and friends — so it either needs a schema-fetching step (network access, and a new flake source) or `-ignore-missing-schemas`, which would skip precisely the CRDs most of this tree is made of. Worth doing properly as its own change rather than adding a check that looks like coverage and isn't.
## Known gap, stated rather than implied away
`shellcheck` **cannot see shell embedded in manifests** — init containers, probe `exec` blocks. That's where several real bugs lived this year (the `grep --line-buffered` no-op, the `mosquitto_pub -W` flag that doesn't exist). Called out in the file so nobody reads this pipeline as covering it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Nothing covered kubernetes/** before this. deploy-infrastructure.yml gates
only opentofu/**, and ci-scripts-test.yml runs only the scripts/ci/ unit
tests — so a broken manifest reached the cluster and was caught by Flux
failing to reconcile, or by someone noticing. Reviewers flagged the gap on
three separate PRs today.
Four static checks, all offline (no cluster, no registry, no secrets):
kustomize-build every kustomization under kubernetes/ must build
woodpecker-lint every .woodpecker/*.yml must compile
shellcheck standalone .sh under kubernetes/, severity=error
no-digest-pins rotators must not reintroduce @sha256 image pins
Each was verified GREEN against the tree as it stands before being added,
so this gates new breakage rather than failing on pre-existing debt: all 60
kustomizations build, all pipeline configs lint, all 6 shell scripts are
clean at error severity, and no digest pins remain after #459.
The woodpecker-lint step is the one that matters most: it exits 1 on the
exact defect that took every pipeline in this repo down earlier today
(#443) — an unquoted YAML scalar containing ": ", which `yq` accepts
because it IS valid YAML, just a map rather than a string. Verified both
directions in the image the step actually uses: rc=1 on that construct,
rc=0 on the pre-existing "Additional property timeout is not allowed"
warnings in four deploy-*.yml files, so it fails on real breakage without
blocking on legacy noise.
Image choices, all version-pinned with no floating tags:
ci-k8s:2026-07-30 kubectl for kustomize; also the
shell for the grep-based guard
woodpecker-cli:v3.9.0-alpine the non-alpine image is distroless
and has no /bin/sh, so Woodpecker
`commands:` cannot run in it; the
binary is /bin/woodpecker-cli, not
on PATH as `woodpecker`
shellcheck-alpine:v0.10.0 pinned rather than :stable, which
floats like :latest
Deliberately NOT included: kubeconform. It needs CRD schemas for
ExternalSecret, Certificate, HelmRelease and friends, so it either needs a
schema-fetching step (network, and a new failure mode) or
-ignore-missing-schemas, which would skip precisely the CRDs most of this
tree is made of. Worth doing properly as its own change.
Known gap, called out in the file rather than implied away: shellcheck
cannot see shell embedded in manifests (init containers, probe exec
blocks), which is where several real bugs lived this year.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit cd12bc16c9b2bf78b02b95ba97870a1184ab371a.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new offline pull_request pipeline with four steps (kustomize-build, woodpecker-lint, shellcheck, no-digest-pins) closing a real gap: nothing currently validates kubernetes/**. The rationale is well-documented, the claimed facts about the current tree (60 kustomizations, 6 shell scripts, 4 deploy-*.yml files with the pre-existing timeout warning, no digest pins remaining under openbao/rotation/) all check out against the checkout. One likely portability bug in the flagship kustomize-build step needs to be settled before merge — see below.
🔴 Blocking
.woodpecker/ci-manifests-test.yml:36 — find kubernetes -name kustomization.yaml -printf '%h\n' relies on -printf, a GNU findutils extension. The kustomize-build step runs in gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30, whose Containerfile (ci/images/k8s/Containerfile) is FROM alpine:3.21 and only apk adds aws-cli bash bind-tools ca-certificates curl git openssh-client openssl sops — it never installs findutils or coreutils. That means find resolves to BusyBox's find applet, which historically does not implement -printf (its documented action set is -print/-print0/-prune/-delete/-exec/… but no -printf).
If that holds, the failure mode is not a loud error — it's silent: for d in $(find … -printf …) sits inside a command substitution used as a word list, so under set -eu a failing/erroring find just yields empty stdout, the for loop runs zero times, fail stays 0, and the step exits 0. The step would report green on every PR without ever running kubectl kustomize on anything, which is the opposite of what this PR is for.
I could not execute the actual container image from this environment to confirm BusyBox's exact feature set in this build, so I'm not stating this as certain — but it's a specific, checkable claim. The one check that settles it:docker run --rm gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30 sh -c "find --help 2>&1 | grep printf" (or just run the step's loop by hand once). If -printf isn't supported, a portable fix is -exec dirname {} \; in place of -printf '%h\n', which BusyBox's find does document.
🟡 Suggestions
None beyond the item above — the rest of the pipeline (woodpecker-lint, shellcheck, no-digest-pins) uses only basic find -name/glob/grep constructs that are portable across BusyBox and GNU implementations, and the claims in the PR body about the current state of the tree (60 kustomizations, 6 .sh files under kubernetes/, no @sha256 pins under openbao/rotation/, timeout: present in exactly the four named deploy-*.yml files) all verified against the checkout.
🟢 Nits
None.
Tests
This is the test/validation infrastructure — there's no meta-test for a CI pipeline definition beyond what the PR description already reports (each step run against the current tree, both directions verified for woodpecker-lint). That's a reasonable bar for this kind of change. The one gap is the -printf portability question above, which by nature wouldn't be caught by "run it against a healthy tree and see green" — a silently-no-op step and a correctly-working step produce the same observable result (exit 0) when every kustomization already builds cleanly, which is exactly the situation described in the PR body ("all 60 kustomizations build"). Worth having the author paste the actual kustomize-build step log (with per-directory ok/FAIL lines, not just the final exit code) to prove the loop iterated 60 times rather than zero.
Recommendation
Request changes — 1 blocking issue (unverified but concrete: confirm find -printf works in the ci-k8s:2026-07-30 image, or switch to -exec dirname {} \;).
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `cd12bc16c9b2bf78b02b95ba97870a1184ab371a`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new offline pull_request pipeline with four steps (`kustomize-build`, `woodpecker-lint`, `shellcheck`, `no-digest-pins`) closing a real gap: nothing currently validates `kubernetes/**`. The rationale is well-documented, the claimed facts about the current tree (60 kustomizations, 6 shell scripts, 4 `deploy-*.yml` files with the pre-existing `timeout` warning, no digest pins remaining under `openbao/rotation/`) all check out against the checkout. One likely portability bug in the flagship `kustomize-build` step needs to be settled before merge — see below.
## 🔴 Blocking
- **`.woodpecker/ci-manifests-test.yml:36`** — `find kubernetes -name kustomization.yaml -printf '%h\n'` relies on `-printf`, a GNU findutils extension. The `kustomize-build` step runs in `gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30`, whose `Containerfile` (`ci/images/k8s/Containerfile`) is `FROM alpine:3.21` and only `apk add`s `aws-cli bash bind-tools ca-certificates curl git openssh-client openssl sops` — it never installs `findutils` or `coreutils`. That means `find` resolves to BusyBox's `find` applet, which historically does not implement `-printf` (its documented action set is `-print`/`-print0`/`-prune`/`-delete`/`-exec`/… but no `-printf`).
If that holds, the failure mode is not a loud error — it's silent: `for d in $(find … -printf …)` sits inside a command substitution used as a word list, so under `set -eu` a failing/erroring `find` just yields empty stdout, the `for` loop runs zero times, `fail` stays `0`, and the step exits `0`. The step would report green on every PR without ever running `kubectl kustomize` on anything, which is the opposite of what this PR is for.
I could not execute the actual container image from this environment to confirm BusyBox's exact feature set in this build, so I'm not stating this as certain — but it's a specific, checkable claim. **The one check that settles it:** `docker run --rm gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30 sh -c "find --help 2>&1 | grep printf"` (or just run the step's loop by hand once). If `-printf` isn't supported, a portable fix is `-exec dirname {} \;` in place of `-printf '%h\n'`, which BusyBox's `find` does document.
## 🟡 Suggestions
- None beyond the item above — the rest of the pipeline (`woodpecker-lint`, `shellcheck`, `no-digest-pins`) uses only basic `find -name`/glob/grep constructs that are portable across BusyBox and GNU implementations, and the claims in the PR body about the current state of the tree (60 kustomizations, 6 `.sh` files under `kubernetes/`, no `@sha256` pins under `openbao/rotation/`, `timeout:` present in exactly the four named `deploy-*.yml` files) all verified against the checkout.
## 🟢 Nits
None.
## Tests
This *is* the test/validation infrastructure — there's no meta-test for a CI pipeline definition beyond what the PR description already reports (each step run against the current tree, both directions verified for `woodpecker-lint`). That's a reasonable bar for this kind of change. The one gap is the `-printf` portability question above, which by nature wouldn't be caught by "run it against a healthy tree and see green" — a silently-no-op step and a correctly-working step produce the same observable result (exit 0) when every kustomization already builds cleanly, which is exactly the situation described in the PR body ("all 60 kustomizations build"). Worth having the author paste the actual `kustomize-build` step log (with per-directory `ok`/`FAIL` lines, not just the final exit code) to prove the loop iterated 60 times rather than zero.
## Recommendation
Request changes — 1 blocking issue (unverified but concrete: confirm `find -printf` works in the `ci-k8s:2026-07-30` image, or switch to `-exec dirname {} \;`).
<!-- pr-review:sha=cd12bc16c9b2bf78b02b95ba97870a1184ab371a -->
Blocking review point on #460, and it was exactly right.
`find -printf` is a GNU findutils extension. ci-k8s is alpine-based and
never installs findutils, so find is BusyBox — confirmed by running it in
the image:
find: unrecognized: -printf
BusyBox v1.37.0
Because the result fed a command substitution used as a word list, the
failure was SILENT: empty output, zero loop iterations, fail stayed 0, and
the step exited green having run kubectl kustomize on nothing at all.
The proof is in this PR's own pipeline run (3158). The step is recorded as
"success", and its log contains ZERO `ok` lines — sixty were expected. So
the gate I added to catch silent no-ops was itself a silent no-op, and CI
reported it as passing.
Worth stating why neither of my checks caught it: I dry-ran the loop on my
workstation, which has GNU find, and the CI run cannot distinguish a
zero-iteration loop from a working one when every kustomization already
builds — both exit 0. The review's suggested test (paste the per-directory
log lines, not the exit code) is the only one that separates them.
Fixed by replacing -printf with a portable pipe through sed. Verified in
the actual image, with a synthetic tree, that the loop now iterates.
Also added a discovery assertion to both find-based steps: fail if the
count is implausible (kustomize-build expects >=50 of ~60; shellcheck
expects >=1) rather than proceeding with an empty list. That is the general
fix — without it, ANY future breakage of discovery degrades back into a
green no-op instead of a failure, which is the trap this whole commit is
about rather than the -printf detail specifically.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 790fd1933a4c45ecd49cdf31a91c8e4da2072a4b.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new pull_request-triggered pipeline with four offline, static checks over kubernetes/** and .woodpecker/**: kustomize-build, woodpecker-lint, shellcheck, and no-digest-pins. I re-derived every quantitative claim in the PR body against the checkout (kustomization count, shell-script count, current digest-pin state, the self-match/exclusion behavior of the no-digest-pins grep) and all of them check out. No blocking issues.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:140-141 — the comment "image: anchors the match so the kustomization comment documenting this rule does not flag itself" is inaccurate about the mechanism. I ran the actual grep (grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml) against the checkout: kustomization.yamldoes match — its own documentation comment at kubernetes/infrastructure/openbao/rotation/kustomization.yaml:33-34 literally contains the substring image:.*@sha256, which satisfies the regex. What actually excludes it is the subsequent grep -v kustomization.yaml filename filter, not "image: anchoring." The behavior is correct (confirmed: the pipe correctly drops it, no-digest-pins would pass today), and kubernetes/infrastructure/openbao/rotation/kustomization.yaml:36-39 already documents this self-match/filter interaction accurately — only the new file's comment mischaracterizes it. Worth a one-line fix since this codebase clearly leans on comments being trustworthy documentation of trap behavior.
.woodpecker/ci-manifests-test.yml:37,143 — ci-k8s:2026-07-30 and (line 81) woodpecker-cli:v3.9.0-alpine / (line 107) shellcheck-alpine:v0.10.0 are the first references to these tags/images anywhere in the repo (existing pipelines pin ci-k8s:2026-05-01). I can't pull images from this sandbox to confirm ci-k8s:2026-07-30 has actually been pushed to the registry — if it hasn't, both kustomize-build and no-digest-pins fail at image-pull on every PR rather than at the check itself. Circumstantial evidence is good (other rotator manifests already reference secret-rotator:2026-07-30, built by the same date-tagging convention in build-ci-images.yml, and today's date is 2026-07-30), but this is worth a final confirmation that the tag resolves before merge, since a pull failure here would block every PR, not just ones touching kubernetes/**.
Tests
Not applicable in the traditional sense — this PR is the test harness for kubernetes/** and .woodpecker/**. The right validation is running each check against the real checkout/images, which the PR body describes doing and which I independently reproduced for the parts checkable from disk: find kubernetes -name kustomization.yaml → 60 (guard threshold 50, comfortable margin), find kubernetes -name '*.sh' → 6 scripts, all with proper shebangs (bash or sh) so shellcheck can auto-detect dialect without a -s flag, and no @sha256-pinned images currently under openbao/rotation/. The woodpecker-lint and image-pull correctness claims (verified against the live images per the PR body) aren't independently checkable from this sandboxed, network-isolated checkout.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `790fd1933a4c45ecd49cdf31a91c8e4da2072a4b`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new pull_request-triggered pipeline with four offline, static checks over `kubernetes/**` and `.woodpecker/**`: `kustomize-build`, `woodpecker-lint`, `shellcheck`, and `no-digest-pins`. I re-derived every quantitative claim in the PR body against the checkout (kustomization count, shell-script count, current digest-pin state, the self-match/exclusion behavior of the `no-digest-pins` grep) and all of them check out. No blocking issues.
## 🟡 Suggestions
- `.woodpecker/ci-manifests-test.yml:140-141` — the comment "`image:` anchors the match so the kustomization comment documenting this rule does not flag itself" is inaccurate about the mechanism. I ran the actual grep (`grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml`) against the checkout: `kustomization.yaml` **does** match — its own documentation comment at `kubernetes/infrastructure/openbao/rotation/kustomization.yaml:33-34` literally contains the substring `image:.*@sha256`, which satisfies the regex. What actually excludes it is the subsequent `grep -v kustomization.yaml` filename filter, not "`image:` anchoring." The behavior is correct (confirmed: the pipe correctly drops it, `no-digest-pins` would pass today), and `kubernetes/infrastructure/openbao/rotation/kustomization.yaml:36-39` already documents this self-match/filter interaction accurately — only the new file's comment mischaracterizes it. Worth a one-line fix since this codebase clearly leans on comments being trustworthy documentation of trap behavior.
- `.woodpecker/ci-manifests-test.yml:37,143` — `ci-k8s:2026-07-30` and (line 81) `woodpecker-cli:v3.9.0-alpine` / (line 107) `shellcheck-alpine:v0.10.0` are the first references to these tags/images anywhere in the repo (existing pipelines pin `ci-k8s:2026-05-01`). I can't pull images from this sandbox to confirm `ci-k8s:2026-07-30` has actually been pushed to the registry — if it hasn't, both `kustomize-build` and `no-digest-pins` fail at image-pull on every PR rather than at the check itself. Circumstantial evidence is good (other rotator manifests already reference `secret-rotator:2026-07-30`, built by the same date-tagging convention in `build-ci-images.yml`, and today's date is 2026-07-30), but this is worth a final confirmation that the tag resolves before merge, since a pull failure here would block *every* PR, not just ones touching `kubernetes/**`.
## Tests
Not applicable in the traditional sense — this PR *is* the test harness for `kubernetes/**` and `.woodpecker/**`. The right validation is running each check against the real checkout/images, which the PR body describes doing and which I independently reproduced for the parts checkable from disk: `find kubernetes -name kustomization.yaml` → 60 (guard threshold 50, comfortable margin), `find kubernetes -name '*.sh'` → 6 scripts, all with proper shebangs (`bash` or `sh`) so `shellcheck` can auto-detect dialect without a `-s` flag, and no `@sha256`-pinned images currently under `openbao/rotation/`. The `woodpecker-lint` and image-pull correctness claims (verified against the live images per the PR body) aren't independently checkable from this sandboxed, network-isolated checkout.
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=790fd1933a4c45ecd49cdf31a91c8e4da2072a4b -->
Blocking issue confirmed and fixed in 790fd193. You were right, and the diagnosis was exact.
The gate was a silent no-op
Confirmed by running it in the image rather than reasoning about BusyBox feature sets:
find: unrecognized: -printf
BusyBox v1.37.0
And the proof it mattered is in this PR's own pipeline run 3158: the kustomize-build step is recorded as success, and its log contains zerook lines where sixty were expected. So the gate I wrote to catch silent no-ops was itself one, and CI reported it green.
Why my verification missed it, twice
Worth recording, because both failures were confident:
I dry-ran the loop on my workstation, which has GNU find, where -printf works.
I cited "all four steps passed in real CI" as evidence — which is worthless here, because a zero-iteration loop and a working loop both exit 0 when every kustomization already builds.
Your suggested check (inspect the per-directory log lines, not the exit code) is the only one that separates those cases. I had the log available and did not look at it.
Fixed, and verified the way you asked
Portable discovery via find … | sed 's|/kustomization.yaml$||'. Pipeline run 3161:
discovered 60 kustomizations
ok-lines: 60
Same success status as before, sixty times the actual work.
The general fix, which matters more than the -printf detail
Both find-based steps now assert their discovery found something before proceeding — kustomize-build requires >=50 of the ~60, shellcheck requires >=1 — and fail loudly otherwise.
Without that, any future breakage of discovery degrades back into a green no-op. -printf was the symptom; a check that cannot tell it checked nothing is the disease, and it is the same shape as the two bugs this pipeline was written to catch (grep --line-buffered silently matching nothing, mosquitto_pub -W silently failing every publish).
Blocking issue confirmed and fixed in `790fd193`. You were right, and the diagnosis was exact.
### The gate was a silent no-op
Confirmed by running it in the image rather than reasoning about BusyBox feature sets:
```
find: unrecognized: -printf
BusyBox v1.37.0
```
And the proof it mattered is in this PR's own pipeline run **3158**: the `kustomize-build` step is recorded as **success**, and its log contains **zero** `ok` lines where sixty were expected. So the gate I wrote to catch silent no-ops was itself one, and CI reported it green.
### Why my verification missed it, twice
Worth recording, because both failures were confident:
1. I dry-ran the loop **on my workstation**, which has GNU find, where `-printf` works.
2. I cited "all four steps passed in real CI" as evidence — which is **worthless here**, because a zero-iteration loop and a working loop both exit 0 when every kustomization already builds.
Your suggested check (inspect the per-directory log lines, not the exit code) is the only one that separates those cases. I had the log available and did not look at it.
### Fixed, and verified the way you asked
Portable discovery via `find … | sed 's|/kustomization.yaml$||'`. Pipeline run **3161**:
```
discovered 60 kustomizations
ok-lines: 60
```
Same `success` status as before, sixty times the actual work.
### The general fix, which matters more than the -printf detail
Both find-based steps now **assert their discovery found something** before proceeding — `kustomize-build` requires >=50 of the ~60, `shellcheck` requires >=1 — and fail loudly otherwise.
Without that, any future breakage of discovery degrades back into a green no-op. `-printf` was the symptom; a check that cannot tell it checked nothing is the disease, and it is the same shape as the two bugs this pipeline was written to catch (`grep --line-buffered` silently matching nothing, `mosquitto_pub -W` silently failing every publish).
Review point on #460. The comment claimed "`image:` anchors the match so
the kustomization comment documenting this rule does not flag itself".
That is wrong about the mechanism, and the reviewer proved it by running
the bare grep — kustomization.yaml DOES match, because the comment
documenting the rule contains the literal string `image:.*@sha256`.
What actually excludes it is the `grep -v kustomization.yaml` filter. The
step's behaviour was always correct; only the explanation was wrong.
Worth fixing rather than shrugging at: this is the third time today the
same self-matching trap has caught me. Twice while writing the guard in
rotation/kustomization.yaml (where the final version documents it
accurately), and now once more in the comment describing it here. A
comment that misidentifies which line is load-bearing invites someone to
delete the filter as redundant tidying, at which point the step fails
permanently on its own documentation.
The comment now says the filter is load-bearing and why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🟡 The comment misidentified the mechanism — you are right
Verified by running the bare grep exactly as you did:
$ grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml
kubernetes/infrastructure/openbao/rotation/kustomization.yaml <- it DOES match
$ ... | grep -v kustomization.yaml
(none) <- the filter is what excludes it
So image: anchoring does nothing for self-matching; the filename filter is load-bearing. Fixed, and the comment now says so explicitly — because a comment that misidentifies the load-bearing line invites someone to delete the filter as redundant tidying, at which point the step fails permanently on its own documentation.
Worth admitting this is the third time today the same self-matching trap caught me: twice while writing the guard in rotation/kustomization.yaml, and once more in the comment describing it here. You are also right that this codebase leans on comments being trustworthy — a wrong explanation next to correct code is its own defect.
🟡 Confirming the image tags resolve
Good instinct, and the consequence you name is right: this pipeline has no path filter, so it runs on every PR — an unpullable image would block all of them, not just kubernetes/** ones.
Stronger evidence than a manual pull: pipeline 3161 ran this pipeline against a real checkout and every step succeeded, which means all three images pulled in the runner:
And discovered 60 kustomizations / 60 ok lines in that same run, so it pulled and did the work.
On ci-k8s:2026-05-01 being what other pipelines pin: that is worth a separate look. ci-k8s is rebuilt whenever .woodpecker/build-ci-images.yml changes, so newer dated tags exist and the older pin is simply stale rather than deliberate — the same drift that left the rotators pinned to dead digests, just less severe because a tag does not get garbage-collected.
Both addressed.
### 🟡 The comment misidentified the mechanism — you are right
Verified by running the bare grep exactly as you did:
```
$ grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml
kubernetes/infrastructure/openbao/rotation/kustomization.yaml <- it DOES match
$ ... | grep -v kustomization.yaml
(none) <- the filter is what excludes it
```
So `image:` anchoring does nothing for self-matching; the filename filter is load-bearing. Fixed, and the comment now says so explicitly — because a comment that misidentifies the load-bearing line invites someone to delete the filter as redundant tidying, at which point the step fails permanently on its own documentation.
Worth admitting this is the **third** time today the same self-matching trap caught me: twice while writing the guard in `rotation/kustomization.yaml`, and once more in the comment describing it here. You are also right that this codebase leans on comments being trustworthy — a wrong explanation next to correct code is its own defect.
### 🟡 Confirming the image tags resolve
Good instinct, and the consequence you name is right: this pipeline has no path filter, so it runs on **every** PR — an unpullable image would block all of them, not just `kubernetes/**` ones.
Stronger evidence than a manual pull: pipeline **3161** ran this pipeline against a real checkout and every step succeeded, which means all three images pulled **in the runner**:
```
clone success
kustomize-build success <- ci-k8s:2026-07-30
woodpecker-lint success <- woodpecker-cli:v3.9.0-alpine
shellcheck success <- shellcheck-alpine:v0.10.0
no-digest-pins success <- ci-k8s:2026-07-30
```
And `discovered 60 kustomizations` / 60 `ok` lines in that same run, so it pulled *and* did the work.
On `ci-k8s:2026-05-01` being what other pipelines pin: that is worth a separate look. `ci-k8s` is rebuilt whenever `.woodpecker/build-ci-images.yml` changes, so newer dated tags exist and the older pin is simply stale rather than deliberate — the same drift that left the rotators pinned to dead digests, just less severe because a tag does not get garbage-collected.
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit ddf5f5007bad978c797f1e3cf75fc0468daa0142.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new offline pull_request pipeline with four steps (kustomize-build, woodpecker-lint, shellcheck, no-digest-pins) that closes a real, previously-demonstrated gap: nothing validated kubernetes/** or the pipeline configs themselves before this. I checked the script logic line by line against the actual checkout and it holds up — counts match (60 kustomizations, 6 shell scripts under kubernetes/), the no-digest-pins grep-filter self-match claim is verified against kubernetes/infrastructure/openbao/rotation/kustomization.yaml, and the errexit/if-exemption reasoning in the comments is correct POSIX sh behavior. No secrets, no cluster/network access, no blocking defects found.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:31,143 — new image tag gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30 cannot be verified from this checkout (no network/registry access here). This repo has already been bitten by exactly this failure mode once: kubernetes/infrastructure/openbao/rotation/kustomization.yaml documents 11 rotators that silently died because they were pinned to a manifest that no longer resolved (#458), and its own "BUMPING the tag" guidance says to verify a new tag exists before committing, since an unbuilt tag fails "just as quietly." Every other pipeline in .woodpecker/ that uses this image references ci-k8s:2026-05-01 (e.g. e2e.yml, alert-monitor.yml, issue-fix.yml) — this PR is the first reference to the 2026-07-30 tag anywhere in the tree, and the diff doesn't touch ci/images/k8s/**, so nothing in this PR itself triggers a rebuild that would publish it. If that tag wasn't pushed by a separate build-ci-images run today, both steps that use it will fail to pull the image on every PR. Low risk of silent failure (a bad pull fails loudly), but worth confirming the tag resolves — the first CI run of this very pipeline will prove it either way.
No path: filter on the when: block (ci-manifests-test.yml:25-26), so all four steps run on every PR regardless of whether kubernetes/** or .woodpecker/** changed (e.g. a docs-only PR still pulls three extra images and builds 60 kustomizations). This matches the existing convention in ci-scripts-test.yml (also unfiltered), so it's consistent with the repo's style rather than a new problem — just flagging in case unfiltered-by-design wasn't the intent here too.
Word-splitting on unquoted $dirs/$files in the for loops (kustomize-build, shellcheck steps) would break on a directory or filename containing whitespace. Not an issue for the current tree (checked — no such names under kubernetes/), and consistent with the informal quoting style elsewhere in .woodpecker/*.yml, so this is a latent assumption rather than a live bug.
Tests
This is itself the test: a CI-validation pipeline whose correctness is demonstrated by running against the real tree, which happens automatically on this PR. The anti-no-op guards (count -lt 50 / count -lt 1 thresholds in kustomize-build and shellcheck) are a genuinely good touch — they specifically prevent the failure mode the PR body describes happening once already (discovery silently returning nothing, step reporting green). No separate unit-test file is expected or missing here, matching how ci-scripts-test.yml/deploy-infrastructure.yml aren't unit-tested either.
Recommendation
Looks reasonable — no blocking issues; comments above (mainly: confirm the ci-k8s:2026-07-30 tag resolves) are optional and self-resolving on the first CI run.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `ddf5f5007bad978c797f1e3cf75fc0468daa0142`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new offline `pull_request` pipeline with four steps (`kustomize-build`, `woodpecker-lint`, `shellcheck`, `no-digest-pins`) that closes a real, previously-demonstrated gap: nothing validated `kubernetes/**` or the pipeline configs themselves before this. I checked the script logic line by line against the actual checkout and it holds up — counts match (60 kustomizations, 6 shell scripts under `kubernetes/`), the `no-digest-pins` grep-filter self-match claim is verified against `kubernetes/infrastructure/openbao/rotation/kustomization.yaml`, and the `errexit`/`if`-exemption reasoning in the comments is correct POSIX sh behavior. No secrets, no cluster/network access, no blocking defects found.
## 🟡 Suggestions
- **`.woodpecker/ci-manifests-test.yml:31,143` — new image tag `gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30` cannot be verified from this checkout (no network/registry access here).** This repo has already been bitten by exactly this failure mode once: `kubernetes/infrastructure/openbao/rotation/kustomization.yaml` documents 11 rotators that silently died because they were pinned to a manifest that no longer resolved (#458), and its own "BUMPING the tag" guidance says to verify a new tag exists *before* committing, since an unbuilt tag fails "just as quietly." Every other pipeline in `.woodpecker/` that uses this image references `ci-k8s:2026-05-01` (e.g. `e2e.yml`, `alert-monitor.yml`, `issue-fix.yml`) — this PR is the first reference to the `2026-07-30` tag anywhere in the tree, and the diff doesn't touch `ci/images/k8s/**`, so nothing in this PR itself triggers a rebuild that would publish it. If that tag wasn't pushed by a separate `build-ci-images` run today, both steps that use it will fail to pull the image on every PR. Low risk of silent failure (a bad pull fails loudly), but worth confirming the tag resolves — the first CI run of this very pipeline will prove it either way.
- **No `path:` filter on the `when:` block** (`ci-manifests-test.yml:25-26`), so all four steps run on every PR regardless of whether `kubernetes/**` or `.woodpecker/**` changed (e.g. a docs-only PR still pulls three extra images and builds 60 kustomizations). This matches the existing convention in `ci-scripts-test.yml` (also unfiltered), so it's consistent with the repo's style rather than a new problem — just flagging in case unfiltered-by-design wasn't the intent here too.
- Word-splitting on unquoted `$dirs`/`$files` in the `for` loops (`kustomize-build`, `shellcheck` steps) would break on a directory or filename containing whitespace. Not an issue for the current tree (checked — no such names under `kubernetes/`), and consistent with the informal quoting style elsewhere in `.woodpecker/*.yml`, so this is a latent assumption rather than a live bug.
## Tests
This is itself the test: a CI-validation pipeline whose correctness is demonstrated by running against the real tree, which happens automatically on this PR. The anti-no-op guards (`count -lt 50` / `count -lt 1` thresholds in `kustomize-build` and `shellcheck`) are a genuinely good touch — they specifically prevent the failure mode the PR body describes happening once already (discovery silently returning nothing, step reporting green). No separate unit-test file is expected or missing here, matching how `ci-scripts-test.yml`/`deploy-infrastructure.yml` aren't unit-tested either.
## Recommendation
Looks reasonable — no blocking issues; comments above (mainly: confirm the `ci-k8s:2026-07-30` tag resolves) are optional and self-resolving on the first CI run.
<!-- pr-review:sha=ddf5f5007bad978c797f1e3cf75fc0468daa0142 -->
Review points on #460.
Path-filter the pipeline to kubernetes/** and .woodpecker/**. It was
unfiltered, matching ci-scripts-test.yml, but that comparison does not
hold: ci-scripts-test is one step on one already-cached image, whereas this
pulls three images and builds 60 kustomizations — real cost to impose on a
docs-only PR. The filter still covers everything the steps read, and
including .woodpecker/** means an edit to this file re-runs its own checks.
Pin IFS to newline in both find-driven loops so a path containing spaces is
passed as one argument rather than split into fragments. Verified in the
actual image: a directory named "has space" now iterates once, intact.
Deliberately NOT rewritten as `printf | while read`, which is the usual
advice: that puts the loop in a subshell, so `fail` would not propagate and
a failing kustomize build would exit 0. That is the same silent-pass shape
this step already shipped once with -printf, and fixing a latent
whitespace bug by reintroducing a live correctness bug would be a poor
trade.
Also corrected the file header, which still claimed the gate runs on every
pull_request.
On the ci-k8s:2026-07-30 tag the review asked about again: confirmed by
execution rather than inspection. Pipeline 3161 ran this pipeline against a
real checkout with all five steps green, which requires all three images to
have pulled in the runner, and its kustomize-build step logged "discovered
60 kustomizations" with 60 ok lines — so the image both resolved and did
the work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit set IFS to a literal newline by writing
IFS="
"
inside a `- |` block scalar. The closing quote sits at column 0, which
terminates the block — so the config no longer parsed:
go-yaml load error in scanner (while scanning a simple key) at
L70.C1-L89.C3: could not find expected ':'
Same class as #443, in the very pipeline written to catch #443, and I
pushed it: `woodpecker lint` printed the error in the same command as the
push, and I read the push output rather than the lint output.
Replaced with `while IFS= read -r` fed from a FILE rather than a pipe.
That handles paths with spaces without needing a literal newline anywhere,
and the file redirection keeps the loop in the current shell so `fail`
still propagates — a pipe would have run it in a subshell and made a
failing kustomize build exit 0, which is the silent-pass shape this step
already shipped once with -printf.
Verified both properties in the actual image rather than reasoning about
them: a directory named "has space" iterates once intact, and a `fail=1`
set inside the loop is still 1 after it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 9d3dc152c1b03afe6d810673664b64ea5886b764.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new pull-request-gated pipeline with four offline checks: kubectl kustomize build for every kustomization, woodpecker-cli lint on all pipeline configs, shellcheck on standalone scripts under kubernetes/, and a grep guard against digest-pinned images in the secret-rotator manifests. I cross-checked the concrete claims in the PR body and inline comments against the actual tree rather than taking them on faith: find kubernetes -name kustomization.yaml | wc -l returns exactly 60, find kubernetes -name '*.sh' returns exactly the 6 scripts implied, and the "the grep -v kustomization.yaml filter is load-bearing" claim checks out — kubernetes/infrastructure/openbao/rotation/kustomization.yaml really does contain the literal string image:.*@sha256 in its own explanatory comment, so without the filter this step would permanently fail on its own documentation. The shell in every step is careful about the exact class of bug it's guarding against (piped find/sed swallowing failures under set -e, while read from a pipe losing fail in a subshell, silent-empty-discovery reporting green) — I traced through the set -eu interaction with if/pipeline exit-status semantics for each step and didn't find a case where a real failure gets swallowed. No blocking defects found in the logic itself.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:40 and :162 — both kustomize-build and no-digest-pins pin to gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30, today's date. That tag only exists if build-ci-images.yml's build-k8s step has already run successfully today — and that step only fires on changes to ci/images/k8s/** or .woodpecker/build-ci-images.yml (or a manual trigger), none of which this PR touches. Contrast with deploy-infrastructure.yml, which pins ci-tofu:2026-04-29 — a tag from three months ago, long since confirmed to exist. I don't have registry access from this review sandbox to confirm ci-k8s:2026-07-30 was actually pushed; if it wasn't, both steps fail immediately on an image pull error, which would be a somewhat ironic first impression for a PR about catching CI breakage before merge. Worth confirming the tag is live (check today's build-ci-images run history) before relying on this passing, or falling back to the last known-good ci-k8s tag if not.
.woodpecker/ci-manifests-test.yml:39-173 — none of the four steps set failure: ignore, and Woodpecker runs steps sequentially by default (no depends_on here). If kustomize-build fails, woodpecker-lint, shellcheck, and no-digest-pins never execute, so a contributor sees only the first failing check per push and has to fix-and-repush to discover the next one. Since all four are independent, cheap, offline checks, running them to completion (via failure: ignore on each, or splitting into a depends_on: [] fan-out like build-ci-images.yml does) would surface every failure in one run.
🟢 Nits
.woodpecker/ci-manifests-test.yml:58 — the count -lt 50 floor for "expected ~60 kustomizations" is a reasonable trip-wire today, but it's a hardcoded number with no pointer to update it; if a future legitimate consolidation drops the real count into the 50s this fails for the wrong reason. Not urgent given the current margin.
Tests
This PR is a test/validation pipeline, so there's no separate test suite to ask for — the equivalent here is the manual verification the author describes (both branches of the lint-catches-#443 bug exercised, false-positive/false-negative behavior on existing warnings checked) plus the sanity checks I independently reran above (kustomization count, script count, the load-bearing grep filter). Those all check out against the current tree.
Recommendation
Looks reasonable — no blocking issues; the image-tag freshness point above is worth confirming before merge but isn't a confirmed defect.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `9d3dc152c1b03afe6d810673664b64ea5886b764`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new pull-request-gated pipeline with four offline checks: `kubectl kustomize` build for every kustomization, `woodpecker-cli lint` on all pipeline configs, `shellcheck` on standalone scripts under `kubernetes/`, and a grep guard against digest-pinned images in the secret-rotator manifests. I cross-checked the concrete claims in the PR body and inline comments against the actual tree rather than taking them on faith: `find kubernetes -name kustomization.yaml | wc -l` returns exactly 60, `find kubernetes -name '*.sh'` returns exactly the 6 scripts implied, and the "the `grep -v kustomization.yaml` filter is load-bearing" claim checks out — `kubernetes/infrastructure/openbao/rotation/kustomization.yaml` really does contain the literal string `image:.*@sha256` in its own explanatory comment, so without the filter this step would permanently fail on its own documentation. The shell in every step is careful about the exact class of bug it's guarding against (piped `find`/`sed` swallowing failures under `set -e`, `while read` from a pipe losing `fail` in a subshell, silent-empty-discovery reporting green) — I traced through the `set -eu` interaction with `if`/pipeline exit-status semantics for each step and didn't find a case where a real failure gets swallowed. No blocking defects found in the logic itself.
## 🟡 Suggestions
- **`.woodpecker/ci-manifests-test.yml:40` and `:162`** — both `kustomize-build` and `no-digest-pins` pin to `gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30`, today's date. That tag only exists if `build-ci-images.yml`'s `build-k8s` step has already run successfully today — and that step only fires on changes to `ci/images/k8s/**` or `.woodpecker/build-ci-images.yml` (or a manual trigger), none of which this PR touches. Contrast with `deploy-infrastructure.yml`, which pins `ci-tofu:2026-04-29` — a tag from three months ago, long since confirmed to exist. I don't have registry access from this review sandbox to confirm `ci-k8s:2026-07-30` was actually pushed; if it wasn't, both steps fail immediately on an image pull error, which would be a somewhat ironic first impression for a PR about catching CI breakage before merge. Worth confirming the tag is live (check today's `build-ci-images` run history) before relying on this passing, or falling back to the last known-good `ci-k8s` tag if not.
- **`.woodpecker/ci-manifests-test.yml:39-173`** — none of the four steps set `failure: ignore`, and Woodpecker runs steps sequentially by default (no `depends_on` here). If `kustomize-build` fails, `woodpecker-lint`, `shellcheck`, and `no-digest-pins` never execute, so a contributor sees only the first failing check per push and has to fix-and-repush to discover the next one. Since all four are independent, cheap, offline checks, running them to completion (via `failure: ignore` on each, or splitting into a `depends_on: []` fan-out like `build-ci-images.yml` does) would surface every failure in one run.
## 🟢 Nits
- **`.woodpecker/ci-manifests-test.yml:58`** — the `count -lt 50` floor for "expected ~60 kustomizations" is a reasonable trip-wire today, but it's a hardcoded number with no pointer to update it; if a future legitimate consolidation drops the real count into the 50s this fails for the wrong reason. Not urgent given the current margin.
## Tests
This PR *is* a test/validation pipeline, so there's no separate test suite to ask for — the equivalent here is the manual verification the author describes (both branches of the lint-catches-#443 bug exercised, false-positive/false-negative behavior on existing warnings checked) plus the sanity checks I independently reran above (kustomization count, script count, the load-bearing grep filter). Those all check out against the current tree.
## Recommendation
Looks reasonable — no blocking issues; the image-tag freshness point above is worth confirming before merge but isn't a confirmed defect.
<!-- pr-review:sha=9d3dc152c1b03afe6d810673664b64ea5886b764 -->
Review points on #460.
Woodpecker runs steps sequentially by default, so a failing kustomize-build
hid woodpecker-lint, shellcheck and no-digest-pins entirely — a contributor
would fix one thing, repush, and only then discover the next. All four are
independent, cheap and offline, with no ordering between them, so
depends_on: [] fans them out and one run reports everything.
Verified the structure rather than the diff: all four steps now render with
depends_on=[].
Also made the discovery-floor failure actionable. The message said only
"Discovery is broken", which is wrong advice if the tree legitimately
shrank below 50 kustomizations — it now tells the reader to lower the floor
in that case, and explicitly not to delete it, since removing the guard
restores the silent-green no-op it exists to prevent.
On the ci-k8s:2026-07-30 tag, raised for the third time: confirmed by
execution, not inspection. It resolved in pipeline 3161, whose
kustomize-build and no-digest-pins steps both ran that image and logged
"discovered 60 kustomizations" with 60 ok lines. An image that failed to
pull could not have produced that output. The review is right that nothing
in THIS PR rebuilds the tag — it was published earlier today by
build-ci-images runs triggered by edits to .woodpecker/build-ci-images.yml,
which is also exactly the drift that left other pipelines pinned to
ci-k8s:2026-05-01.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 888aa60fe048bf8eb5f7a3cc6c1b099296906a54.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new PR-triggered pipeline with four parallel, offline checks over kubernetes/** and .woodpecker/**: kustomize-build (all kustomizations build), woodpecker-lint (pipeline configs compile), shellcheck (standalone scripts under kubernetes/), and no-digest-pins (guards the dated-tag convention in the openbao rotation dir). This is a pure CI-config addition (single new file, no other code touched). I checked every factual claim in the PR body and every non-trivial shell construct against the actual checkout — the counts (60 kustomizations, 6 shell scripts), the self-matching grep pattern in rotation/kustomization.yaml, the absence of remote bases/helmCharts (confirming the kustomize step really is offline), and the busybox-find-lacks--printf claim (confirmed: the ci-k8s Containerfile's apk add list has no findutils) all check out. I found no blocking defects.
🟡 Suggestions
woodpecker-lint step uses a CLI version well behind the deployed server (.woodpecker/ci-manifests-test.yml:103 uses woodpeckerci/woodpecker-cli:v3.9.0-alpine, but kubernetes/apps/woodpecker/server.yaml:43 and agent.yaml:37 run v3.16.0). The PR body itself notes woodpecker-cli lint currently emits Additional property timeout is not allowed warnings on several deploy-*.yml files and treats those as harmless pre-existing noise — but that symptom is also consistent with schema drift between a 7-minor-version-old lint tool and the actual v3.16.0 schema those files were written against, rather than genuine debt. A large version gap risks the reverse case too: syntax the real v3.16.0 server accepts could someday be flagged as a compile error by the old CLI (false-fail), or syntax it should reject could pass (false-pass). Consider pinning the lint image to (or near) v3.16.0 for higher-fidelity linting.
kustomize-build uses kubectl kustomize, not flux build, even though the same ci-k8s image already ships flux (ci/images/k8s/Containerfile) — so it doesn't exercise Flux's postBuild.substitute step used by kubernetes/clusters/homelab/infrastructure.yaml. A manifest can pass this new check (kustomize builds cleanly, since ${VAR}-style substitution placeholders are just inert text to kustomize) yet still fail at Flux reconcile time if the substitution step itself breaks. This doesn't need to block the PR, but given the file is explicit elsewhere about stating known gaps rather than implying coverage it doesn't have (the shellcheck-can't-see-embedded-shell note at .woodpecker/ci-manifests-test.yml:124-127), the same treatment here would be consistent — either a one-line callout, or switching to flux build kustomization for closer fidelity to production reconciliation.
Tests
N/A — this PR is the test/validation infrastructure; there's nothing further to test it with. The author's verification methodology (running both steps against real repo content and a known-bad historical config, described in the PR body) is a reasonable substitute for automated tests on a CI-config file, and the specific claims I could check independently (kustomization/script counts, the self-matching grep, the offline-only property) all held up.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `888aa60fe048bf8eb5f7a3cc6c1b099296906a54`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new PR-triggered pipeline with four parallel, offline checks over `kubernetes/**` and `.woodpecker/**`: `kustomize-build` (all kustomizations build), `woodpecker-lint` (pipeline configs compile), `shellcheck` (standalone scripts under `kubernetes/`), and `no-digest-pins` (guards the dated-tag convention in the openbao rotation dir). This is a pure CI-config addition (single new file, no other code touched). I checked every factual claim in the PR body and every non-trivial shell construct against the actual checkout — the counts (60 kustomizations, 6 shell scripts), the self-matching `grep` pattern in `rotation/kustomization.yaml`, the absence of remote bases/`helmCharts` (confirming the kustomize step really is offline), and the busybox-`find`-lacks-`-printf` claim (confirmed: the `ci-k8s` Containerfile's `apk add` list has no `findutils`) all check out. I found no blocking defects.
## 🟡 Suggestions
- **`woodpecker-lint` step uses a CLI version well behind the deployed server** (`.woodpecker/ci-manifests-test.yml:103` uses `woodpeckerci/woodpecker-cli:v3.9.0-alpine`, but `kubernetes/apps/woodpecker/server.yaml:43` and `agent.yaml:37` run `v3.16.0`). The PR body itself notes `woodpecker-cli lint` currently emits `Additional property timeout is not allowed` warnings on several `deploy-*.yml` files and treats those as harmless pre-existing noise — but that symptom is also consistent with schema drift between a 7-minor-version-old lint tool and the actual v3.16.0 schema those files were written against, rather than genuine debt. A large version gap risks the reverse case too: syntax the real v3.16.0 server accepts could someday be flagged as a compile error by the old CLI (false-fail), or syntax it should reject could pass (false-pass). Consider pinning the lint image to (or near) `v3.16.0` for higher-fidelity linting.
- **`kustomize-build` uses `kubectl kustomize`, not `flux build`, even though the same `ci-k8s` image already ships `flux` (`ci/images/k8s/Containerfile`)** — so it doesn't exercise Flux's `postBuild.substitute` step used by `kubernetes/clusters/homelab/infrastructure.yaml`. A manifest can pass this new check (kustomize builds cleanly, since `${VAR}`-style substitution placeholders are just inert text to kustomize) yet still fail at Flux reconcile time if the substitution step itself breaks. This doesn't need to block the PR, but given the file is explicit elsewhere about stating known gaps rather than implying coverage it doesn't have (the shellcheck-can't-see-embedded-shell note at `.woodpecker/ci-manifests-test.yml:124-127`), the same treatment here would be consistent — either a one-line callout, or switching to `flux build kustomization` for closer fidelity to production reconciliation.
## Tests
N/A — this PR *is* the test/validation infrastructure; there's nothing further to test it with. The author's verification methodology (running both steps against real repo content and a known-bad historical config, described in the PR body) is a reasonable substitute for automated tests on a CI-config file, and the specific claims I could check independently (kustomization/script counts, the self-matching grep, the offline-only property) all held up.
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=888aa60fe048bf8eb5f7a3cc6c1b099296906a54 -->
Violated the standing rule: "use the latest stable version. Old versions
must not be hardcoded without a specific, documented reason."
woodpecker-cli was pinned at v3.9.0 while the deployed server and agent run
v3.16.0 — seven minor versions behind. That version was never chosen; I
picked it while probing for any tag that had a shell and then never
revisited it.
For this step the skew is not cosmetic. The CLI compiles the config against
its own schema, so one behind the server can pass a config the server later
rejects, or reject one the server accepts. Either way the gate stops
predicting what the server will do, which is the only reason it exists —
and the failure it was added to catch (#443) is precisely a
server-side compile error.
Now v3.16.0-alpine, matched to the running deployment (checked against the
live Deployment, not assumed), with a comment saying it must be bumped
alongside the server. Verified it still behaves correctly: rc=0 on a valid
config, rc=1 on the #443 construct.
shellcheck was pinned at v0.10.0 for the same non-reason — it was simply
the newer of the two tags I happened to probe. v0.11.0 exists, so bumped.
Verified first that the existing scripts still pass at -S error under the
newer version, since a shellcheck bump can add checks and I did not want to
ship a gate that fails on the current tree.
ci-k8s stays on the dated-tag convention for registry-local images.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 7b5f244b7f44e7d1749d7a8168007fbc3e57c5b9.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new pull-request pipeline with four parallel, offline checks over kubernetes/** and .woodpecker/**: kubectl kustomize build of every kustomization, woodpecker-cli lint of every pipeline file, shellcheck -S error on standalone shell scripts, and a grep guard against digest-pinned images in the secret-rotator directory. I independently re-verified the PR's factual claims against the checkout (60 kustomizations, 6 shell scripts, zero digest pins outside the self-documenting comment, and the woodpecker-cli version matching the deployed server) and they all hold. The shell logic in each step is careful and correct — no blocking issues found.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:62 (kustomize-build) and :150 (shellcheck): neither step sets set -o pipefail (and BusyBox/ash/dash don't support it anyway), so dirs=$(find kubernetes -name kustomization.yaml | sed ... | sort) takes its exit status from sort, not find. A find failure on a subtree (e.g. a permission error) that still lets sed/sort exit 0 would silently drop entries from the list rather than failing the step. The count -lt 50 floor catches total discovery breakage (the documented "empty output" bug) but not a partial loss of a few directories. Given the floor already exists and this is a low-likelihood failure mode in a repo you fully control, this is a nit-level robustness gap, not a blocker.
The woodpecker-lint step (:109-129) is the one the PR body calls "the one that matters," and its correctness hinges on /bin/woodpecker-cli being the right binary path and the -alpine image actually containing /bin/sh for the commands: block to run at all. I don't have registry/image-pull access in this sandbox to independently confirm that against woodpeckerci/woodpecker-cli:v3.16.0-alpine, so I'm relying on the PR body's claim of having probed the image directly. Worth a final sanity check that this step actually goes green in CI on this PR (rather than failing to even start the container) before merging, since a wrong binary path fails closed in a way that's easy to misattribute to a bug in the pipeline being linted.
🟢 Nits
The PR body's version table is now stale relative to the diff: it quotes woodpecker-cli:v3.9.0-alpine and shellcheck-alpine:v0.10.0, while the actual file (after the follow-up "use latest stable versions" commit) pins v3.16.0-alpine and v0.11.0. Not a code issue, just worth updating the description before merge so reviewers reading the body aren't checking against the wrong tags.
Tests
This is a CI-config-only change, so there's no unit test to add in the usual sense — the pipeline's own woodpecker-lint step and the manual dry-runs described in the PR body (rc=0 on valid config, rc=1 reproducing the #443 bug) are the applicable verification, and that's consistent with how other pipeline-definition files in this repo (e.g. build-ci-images.yml) are validated. I re-ran the factual checks that the PR body claims (kustomization count, shell script count, digest-pin grep, server version) directly against the checkout and all matched.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `7b5f244b7f44e7d1749d7a8168007fbc3e57c5b9`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new pull-request pipeline with four parallel, offline checks over `kubernetes/**` and `.woodpecker/**`: `kubectl kustomize` build of every kustomization, `woodpecker-cli lint` of every pipeline file, `shellcheck -S error` on standalone shell scripts, and a grep guard against digest-pinned images in the secret-rotator directory. I independently re-verified the PR's factual claims against the checkout (60 kustomizations, 6 shell scripts, zero digest pins outside the self-documenting comment, and the woodpecker-cli version matching the deployed server) and they all hold. The shell logic in each step is careful and correct — no blocking issues found.
## 🟡 Suggestions
- `.woodpecker/ci-manifests-test.yml:62` (`kustomize-build`) and `:150` (`shellcheck`): neither step sets `set -o pipefail` (and BusyBox/ash/dash don't support it anyway), so `dirs=$(find kubernetes -name kustomization.yaml | sed ... | sort)` takes its exit status from `sort`, not `find`. A `find` failure on a subtree (e.g. a permission error) that still lets `sed`/`sort` exit 0 would silently drop entries from the list rather than failing the step. The `count -lt 50` floor catches total discovery breakage (the documented "empty output" bug) but not a partial loss of a few directories. Given the floor already exists and this is a low-likelihood failure mode in a repo you fully control, this is a nit-level robustness gap, not a blocker.
- The `woodpecker-lint` step (`:109-129`) is the one the PR body calls "the one that matters," and its correctness hinges on `/bin/woodpecker-cli` being the right binary path and the `-alpine` image actually containing `/bin/sh` for the `commands:` block to run at all. I don't have registry/image-pull access in this sandbox to independently confirm that against `woodpeckerci/woodpecker-cli:v3.16.0-alpine`, so I'm relying on the PR body's claim of having probed the image directly. Worth a final sanity check that this step actually goes green in CI on this PR (rather than failing to even start the container) before merging, since a wrong binary path fails closed in a way that's easy to misattribute to a bug in the pipeline being linted.
## 🟢 Nits
- The PR body's version table is now stale relative to the diff: it quotes `woodpecker-cli:v3.9.0-alpine` and `shellcheck-alpine:v0.10.0`, while the actual file (after the follow-up "use latest stable versions" commit) pins `v3.16.0-alpine` and `v0.11.0`. Not a code issue, just worth updating the description before merge so reviewers reading the body aren't checking against the wrong tags.
## Tests
This is a CI-config-only change, so there's no unit test to add in the usual sense — the pipeline's own `woodpecker-lint` step and the manual dry-runs described in the PR body (rc=0 on valid config, rc=1 reproducing the #443 bug) are the applicable verification, and that's consistent with how other pipeline-definition files in this repo (e.g. `build-ci-images.yml`) are validated. I re-ran the factual checks that the PR body claims (kustomization count, shell script count, digest-pin grep, server version) directly against the checkout and all matched.
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=7b5f244b7f44e7d1749d7a8168007fbc3e57c5b9 -->
I had written that the CLI "MUST track the deployed woodpecker-server",
which invents a rule that does not exist and misstates the one that does.
The standing rule already covers it: any software uses the latest stable
version. CLI tools are software. If the CLI and the server both follow it,
they match by construction — matching is a CONSEQUENCE, not a separate
requirement. I had read "any software" as meaning installs and images but
not CLIs, and was about to propose adding a redundant rule to the wiki off
the back of that misreading.
The comment now says that, and keeps the reason the skew mattered here: the
CLI compiles against its own schema, so one behind the server can pass a
config the server rejects, and the gate stops predicting server behaviour.
Also recorded a real limitation found while checking this. The server and
agent are bumped automatically by image-automation within the 3.x range,
and their policy comment notes they must stay version-matched. This pin is
manual, because ImageUpdateAutomation is scoped to
`update.path: ./kubernetes` and cannot maintain a tag under .woodpecker/.
So the CLI will lag a server upgrade until someone bumps it. Widening that
path would let a policy own this line the same way it owns the server and
agent — noted rather than done, since it changes automation scope repo-wide.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit fbfaf90923251aeed6b2b2ae400daac6210472da.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new pull-request pipeline with four independent, offline, secret-free checks: kustomize-build (all kustomizations under kubernetes/ build), woodpecker-lint (all .woodpecker/*.yml compile), shellcheck (standalone scripts under kubernetes/, severity error), and no-digest-pins (guards against digest-pinned rotator images). I checked every concrete claim in the PR body against the checkout — file counts, the busybox find limitation, the image:.*@sha256 self-match in rotation/kustomization.yaml, the deployed woodpecker-server version, and the absence of remote kustomize bases — and all of them hold. No blocking issues found.
🟡 Suggestions
PR description is stale on the woodpecker-cli tag. The description says woodpecker-cli:v3.9.0-alpine, but .woodpecker/ci-manifests-test.yml:118 pins woodpeckerci/woodpecker-cli:v3.16.0-alpine (which does correctly match the deployed woodpecker-server:v3.16.0 in kubernetes/apps/woodpecker/server.yaml:43, per the comment's stated rationale). The code is right; the PR body just wasn't updated after a later revision — worth a quick edit so future readers of the PR history aren't misled about which tag was actually shipped.
Registry-side existence of new tags is unverifiable from this checkout.ci-k8s:2026-07-30 (line 51/205), woodpecker-cli:v3.16.0-alpine (line 124), and shellcheck-alpine:v0.11.0 (line 155) all need to actually resolve in their registries for this pipeline to run at all; I have no registry/network access here to confirm. The PR body says these were probed directly, which I have no reason to doubt given how well the rest of the PR's claims checked out against the tree — just flagging that this is the one category of claim I couldn't independently verify.
🟢 Nits
The depends_on: [] rationale comment (lines 46-49, 119-122, 150-153, 200-203) is duplicated verbatim across all four steps. Not wrong, just repetitive — could live once at the top of steps: if it's ever edited.
Tests
This is CI infrastructure, not application code, so there's no unit-test layer to add — the "test" is the pipeline itself running green against the current tree, which the PR body says was done for each step. The design already builds in its own regression guard: both kustomize-build and shellcheck assert a minimum discovered-file count (kubernetes/ci-manifests-test.yml:69, :163) specifically so a broken find/sed pipeline fails loudly instead of silently passing with zero iterations — a real failure mode the PR body says bit the first draft of this file. That's a solid substitute for tests in a context where "tests" would just be "run the thing."
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `fbfaf90923251aeed6b2b2ae400daac6210472da`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new pull-request pipeline with four independent, offline, secret-free checks: `kustomize-build` (all kustomizations under `kubernetes/` build), `woodpecker-lint` (all `.woodpecker/*.yml` compile), `shellcheck` (standalone scripts under `kubernetes/`, severity `error`), and `no-digest-pins` (guards against digest-pinned rotator images). I checked every concrete claim in the PR body against the checkout — file counts, the `busybox find` limitation, the `image:.*@sha256` self-match in `rotation/kustomization.yaml`, the deployed `woodpecker-server` version, and the absence of remote kustomize bases — and all of them hold. No blocking issues found.
## 🟡 Suggestions
- **PR description is stale on the woodpecker-cli tag.** The description says `woodpecker-cli:v3.9.0-alpine`, but `.woodpecker/ci-manifests-test.yml:118` pins `woodpeckerci/woodpecker-cli:v3.16.0-alpine` (which does correctly match the deployed `woodpecker-server:v3.16.0` in `kubernetes/apps/woodpecker/server.yaml:43`, per the comment's stated rationale). The code is right; the PR body just wasn't updated after a later revision — worth a quick edit so future readers of the PR history aren't misled about which tag was actually shipped.
- **Registry-side existence of new tags is unverifiable from this checkout.** `ci-k8s:2026-07-30` (line 51/205), `woodpecker-cli:v3.16.0-alpine` (line 124), and `shellcheck-alpine:v0.11.0` (line 155) all need to actually resolve in their registries for this pipeline to run at all; I have no registry/network access here to confirm. The PR body says these were probed directly, which I have no reason to doubt given how well the rest of the PR's claims checked out against the tree — just flagging that this is the one category of claim I couldn't independently verify.
## 🟢 Nits
- The `depends_on: []` rationale comment (lines 46-49, 119-122, 150-153, 200-203) is duplicated verbatim across all four steps. Not wrong, just repetitive — could live once at the top of `steps:` if it's ever edited.
## Tests
This is CI infrastructure, not application code, so there's no unit-test layer to add — the "test" is the pipeline itself running green against the current tree, which the PR body says was done for each step. The design already builds in its own regression guard: both `kustomize-build` and `shellcheck` assert a minimum discovered-file count (`kubernetes/ci-manifests-test.yml:69`, `:163`) specifically so a broken `find`/`sed` pipeline fails loudly instead of silently passing with zero iterations — a real failure mode the PR body says bit the first draft of this file. That's a solid substitute for tests in a context where "tests" would just be "run the thing."
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=fbfaf90923251aeed6b2b2ae400daac6210472da -->
I proposed widening ImageUpdateAutomation's `update.path` to `./` and adding
a woodpecker-cli ImagePolicy so automation would own the lint pin the way it
owns the server and agent. Checking that before building it killed the plan,
and corrected two things I had asserted without evidence.
WRONG #1 — my reason for hesitating. I claimed the 7 ansible-vault encrypted
`.yml` files would break the walk and take all 33 automated images down with
them. That was a guess about kyaml I never verified. Objects that are not
Kubernetes manifests are SKIPPED, not fatal.
WRONG #2 — and this is the one that matters: the plan cannot work at all.
Image-automation only patches YAML that is a Kubernetes manifest. Maintainer,
fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the
YAML manifest must have `apiVersion` and `kind` fields." The docs agree —
markers go "in any Kubernetes custom resource stored in Git". A Woodpecker
pipeline file is plain YAML with no apiVersion/kind, so a marker in it is
silently ignored.
That is a property of the FILE, not of `update.path`. Widening the path buys
exactly nothing and would drag 197 unrelated ansible/esphome YAMLs into the
walk. So no policy, no second automation, no path change — all three would
have been motion that looked like a fix and changed nothing. The failure mode
would have been the quiet kind this repo keeps getting bitten by: a marker
sitting in the file looking maintained, never once updated.
What actually closes the gap, given the pin must stay manual:
* README gains a "Not enrolled (structurally impossible)" section holding
the quote and the explicit "do not try widening the path" warning, so the
next person does not re-derive this.
* update-audit item 5 now covers `.woodpecker/*.yml` step images and is told
to flag woodpecker-cli whenever it trails the DEPLOYED SERVER, not merely
the newest CLI release — the server auto-bumps and the CLI does not, so
that skew is the thing worth reporting. Also told to compare only the
vX.Y.Z part (the tag carries `-alpine`; the plain tag is scratch with no
shell and would break the step) and to ignore the `next-*`/`pull_*` dev
tags, which are the newest by date on that repo and are not releases.
* the step comment states the real constraint instead of my earlier wrong
one, which told the reader that widening the path was the fix.
Verified: v3.16.0-alpine is the newest 3.x alpine release tag, and matches the
deployed server v3.16.0. Config still parses; step image unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 8a580e341d61b194858584a484792cd5a5cc02af.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new pull_request pipeline with four offline, parallel steps (kustomize-build, woodpecker-lint, shellcheck, no-digest-pins) gating kubernetes/** and .woodpecker/**, plus doc updates recording that woodpeckerci/woodpecker-cli can't be enrolled in image-automation. The shell logic is careful (file-fed while read, discovery-count floors, quoting) and most of the PR body's factual claims check out against the checkout (60 kustomizations, 6 shell scripts, zero digest pins under openbao/rotation/, v3.16.0 matching the deployed server, exactly 4 deploy-*.yml files with the timeout property). No blocking issues confirmed, but one risk below is significant enough to verify before merge.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:51,218 — unverified image tag, and the evidence available points the wrong way. Both kustomize-build and no-digest-pins pin gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30. Every other pipeline file in the repo (alert-monitor.yml:45, e2e.yml:33, e2e-test.yml:7, issue-fix.yml:52) still pins ci-k8s:2026-05-01. Per .woodpecker/build-ci-images.yml:55-62, the build-k8s step only runs (and only mints a new dated tag) when ci/images/k8s/** or build-ci-images.yml itself changes on a push — and this PR's diff touches neither. The one piece of same-day evidence, secret-rotator:2026-07-30 (visible throughout kubernetes/infrastructure/openbao/rotation/*.yaml), doesn't transfer: that image is built by an independent step (build-secret-rotator, gated on secret-rotator/**), so its rebuild today (for #458) says nothing about whether build-k8s also ran today. If ci-k8s:2026-07-30 was never pushed, kustomize-build and no-digest-pins fail immediately on image pull for every future PR touching kubernetes/** or .woodpecker/**.
I can't confirm this from the checkout (no registry/network access here), so I'm not marking it blocking — but there's a check that settles it directly: this PR's own diff touches .woodpecker/ci-manifests-test.yml and kubernetes/infrastructure/image-automation/README.md, both matching the new pipeline's own path filter, so its own CI run already attempted these two steps. Check that build's logs/status before merging.
PR description drifted from the shipped versions. The description states verification was done against woodpecker-cli:v3.9.0-alpine and shellcheck-alpine:v0.10.0, but the diff ships v3.16.0-alpine (ci-manifests-test.yml:131) and v0.11.0 (ci-manifests-test.yml:168). The specific empirical claims in the body ("verified both directions... in the image the step actually uses", the no-/bin/sh probing) were run against different tags than what's actually in the pipeline. Probably fine (both are point-bumps within the same alpine variant), but worth a quick re-check that v3.16.0-alpine still ships /bin/sh and that lint behavior is unchanged, since the description's evidence no longer matches the code as diffed.
🟢 Nits
.woodpecker/ci-manifests-test.yml:137 — woodpecker-cli lint "$f" lints one file at a time in a loop; this matches the documented CLI usage as far as I can tell, but I couldn't execute it in this sandbox (no container runtime/network available) to confirm the single-file invocation and exit-code behavior first-hand.
Tests
N/A in the traditional sense — this PR is the test/gate. Its own effectiveness is empirically argued in the description (verified against the #443 regression and the current tree state), and the numeric claims I could check against the checkout (kustomization count, script count, absence of digest pins, server/CLI version match) all held up. The main untested-in-this-review path is the actual CI run of the new pipeline itself (see the image-tag point above).
Recommendation
Looks reasonable — no confirmed blocking issues; the ci-k8s:2026-07-30 tag risk above is worth confirming against this PR's own CI run before merging.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `8a580e341d61b194858584a484792cd5a5cc02af`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new `pull_request` pipeline with four offline, parallel steps (`kustomize-build`, `woodpecker-lint`, `shellcheck`, `no-digest-pins`) gating `kubernetes/**` and `.woodpecker/**`, plus doc updates recording that `woodpeckerci/woodpecker-cli` can't be enrolled in image-automation. The shell logic is careful (file-fed `while read`, discovery-count floors, quoting) and most of the PR body's factual claims check out against the checkout (60 kustomizations, 6 shell scripts, zero digest pins under `openbao/rotation/`, `v3.16.0` matching the deployed server, exactly 4 `deploy-*.yml` files with the `timeout` property). No blocking issues confirmed, but one risk below is significant enough to verify before merge.
## 🟡 Suggestions
- **`.woodpecker/ci-manifests-test.yml:51,218` — unverified image tag, and the evidence available points the wrong way.** Both `kustomize-build` and `no-digest-pins` pin `gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30`. Every *other* pipeline file in the repo (`alert-monitor.yml:45`, `e2e.yml:33`, `e2e-test.yml:7`, `issue-fix.yml:52`) still pins `ci-k8s:2026-05-01`. Per `.woodpecker/build-ci-images.yml:55-62`, the `build-k8s` step only runs (and only mints a new dated tag) when `ci/images/k8s/**` or `build-ci-images.yml` itself changes on a push — and this PR's diff touches neither. The one piece of same-day evidence, `secret-rotator:2026-07-30` (visible throughout `kubernetes/infrastructure/openbao/rotation/*.yaml`), doesn't transfer: that image is built by an independent step (`build-secret-rotator`, gated on `secret-rotator/**`), so its rebuild today (for #458) says nothing about whether `build-k8s` also ran today. If `ci-k8s:2026-07-30` was never pushed, `kustomize-build` and `no-digest-pins` fail immediately on image pull for every future PR touching `kubernetes/**` or `.woodpecker/**`.
I can't confirm this from the checkout (no registry/network access here), so I'm not marking it blocking — but there's a check that settles it directly: this PR's own diff touches `.woodpecker/ci-manifests-test.yml` and `kubernetes/infrastructure/image-automation/README.md`, both matching the new pipeline's own path filter, so its own CI run already attempted these two steps. Check that build's logs/status before merging.
- **PR description drifted from the shipped versions.** The description states verification was done against `woodpecker-cli:v3.9.0-alpine` and `shellcheck-alpine:v0.10.0`, but the diff ships `v3.16.0-alpine` (`ci-manifests-test.yml:131`) and `v0.11.0` (`ci-manifests-test.yml:168`). The specific empirical claims in the body ("verified both directions... in the image the step actually uses", the no-`/bin/sh` probing) were run against different tags than what's actually in the pipeline. Probably fine (both are point-bumps within the same alpine variant), but worth a quick re-check that `v3.16.0-alpine` still ships `/bin/sh` and that `lint` behavior is unchanged, since the description's evidence no longer matches the code as diffed.
## 🟢 Nits
- `.woodpecker/ci-manifests-test.yml:137` — `woodpecker-cli lint "$f"` lints one file at a time in a loop; this matches the documented CLI usage as far as I can tell, but I couldn't execute it in this sandbox (no container runtime/network available) to confirm the single-file invocation and exit-code behavior first-hand.
## Tests
N/A in the traditional sense — this PR *is* the test/gate. Its own effectiveness is empirically argued in the description (verified against the #443 regression and the current tree state), and the numeric claims I could check against the checkout (kustomization count, script count, absence of digest pins, server/CLI version match) all held up. The main untested-in-this-review path is the actual CI run of the new pipeline itself (see the image-tag point above).
## Recommendation
Looks reasonable — no confirmed blocking issues; the `ci-k8s:2026-07-30` tag risk above is worth confirming against this PR's own CI run before merging.
<!-- pr-review:sha=8a580e341d61b194858584a484792cd5a5cc02af -->
Review raised two 🟡 points and one nit. Both 🟡 points turned out to be
already answered by this PR's own CI run, but one of them exposed a real
accuracy problem in a comment, which is fixed here.
1. ci-k8s:2026-07-30 might never have been built. Good reasoning — build-k8s
only mints a dated tag when ci/images/k8s/** or build-ci-images.yml changes
on a push, and this PR touches neither; secret-rotator:2026-07-30 does not
transfer, being a separately-gated step. Checked rather than assumed: the
registry API is closed to this token (no read:package), so the evidence is
pipeline 3178 on 8a580e34, where kustomize-build and no-digest-pins both ran
green on that exact tag, with 60 real `ok` lines — a pull failure could not
produce that. Tag exists. No change needed.
2. v3.16.0-alpine unverified for /bin/sh and the lint invocation. Same run:
woodpecker-lint emitted 18 `ok` lines, one per .woodpecker/*.yml, which is
only reachable if the image has a working shell and /bin/woodpecker-cli.
But checking this found a claim I could NOT verify, which the review had
independently guessed at: the comment asserted that several deploy-*.yml
files emit "Additional property timeout is not allowed" warnings. That was
observed under the v3.9.0 CLI this step used to pin. Whether v3.16.0 still
emits it is unknown — and a green run cannot settle it, because the loop
redirects lint output to /tmp/out and prints it only on failure, so warnings
are invisible exactly when the step passes. Review's hypothesis (schema
drift in a 7-minor-old CLI, not real debt in those files) is plausible and
untested. Dropped the claim instead of restating it; the comment now
documents only the exit-code contract the gate actually rests on.
3. Nit: the depends_on rationale was duplicated verbatim across all four
steps. Hoisted to one note above `steps:`, with a warning that the key
itself must stay on every step — it is per-step, so removing it from one
silently re-serialises that step.
Also updated the PR description, which both reviews flagged as stale: it still
quoted v3.9.0-alpine / v0.10.0 as shipped, so the probing evidence in the body
referred to different tags than the diff. Corrected, with a note recording the
bump rather than quietly rewriting history.
Verified: all four steps still parse with depends_on: [] and unchanged images.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both 🟡 points and the nit are addressed in 51153788.
1. ci-k8s:2026-07-30 may never have been built — good catch on the mechanism: build-k8s only mints a dated tag when ci/images/k8s/** or build-ci-images.yml changes on a push, this PR touches neither, and you were right that secret-rotator:2026-07-30 doesn't transfer (separately-gated step). I checked rather than assumed. The registry API is closed to this token (read:package not granted), so the evidence is the run you pointed at: pipeline 3178 on 8a580e34 — kustomize-build and no-digest-pins both green on that exact tag, with 60 real ok lines. An unpullable image cannot produce that. Tag exists; no change needed.
2. v3.16.0-alpine unverified — same run: woodpecker-lint emitted 18 ok lines, one per .woodpecker/*.yml, which is only reachable if the image has a working /bin/sh and /bin/woodpecker-cli. So the single-file invocation and exit-code behaviour are confirmed against the shipped tag.
But this one exposed something I could not verify, and your earlier hypothesis was the reason I looked. The comment asserted that several deploy-*.yml files emit Additional property timeout is not allowed. That was observed under v3.9.0, the tag this step used to pin. Whether v3.16.0 still emits it is unknown — and a green run cannot settle it, because the loop sends lint output to /tmp/out and prints it only on failure, so warnings are invisible precisely when the step passes. Your read (schema drift in a 7-minor-old CLI rather than real debt in those files) is plausible and untested. I dropped the claim rather than restating it; the comment now documents only the exit-code contract the gate actually rests on.
3. Nit — duplicated depends_on rationale — hoisted to a single note above steps:, with a warning that the key itself must stay on every step (it's per-step; removing it from one silently re-serialises that step).
PR description — updated. You were right that the body still quoted v3.9.0-alpine / v0.10.0 as shipped, so the probing evidence referred to different tags than the diff. Corrected in place with an explicit note recording the bump rather than quietly rewriting it.
Also worth recording, since it's the substantive change in 8a580e34 and it went the opposite way from what I proposed: I had intended to enroll woodpecker-cli in image-automation so the pin would stop being manual, by widening update.path to ./. That cannot work. Image-automation only patches YAML that is a Kubernetes manifest — fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the YAML manifest must have apiVersion and kind fields." A Woodpecker pipeline file has neither, so a marker here is silently ignored. That's a property of the file, not of update.path, so widening buys nothing and would drag 197 unrelated ansible/esphome YAMLs into the walk. The pin stays manual by necessity; drift is now covered by the update-audit agent, which is told to flag this image whenever it trails the deployed server.
Both 🟡 points and the nit are addressed in `51153788`.
**1. `ci-k8s:2026-07-30` may never have been built** — good catch on the mechanism: `build-k8s` only mints a dated tag when `ci/images/k8s/**` or `build-ci-images.yml` changes on a push, this PR touches neither, and you were right that `secret-rotator:2026-07-30` doesn't transfer (separately-gated step). I checked rather than assumed. The registry API is closed to this token (`read:package` not granted), so the evidence is the run you pointed at: **pipeline 3178 on `8a580e34`** — `kustomize-build` and `no-digest-pins` both green on that exact tag, with **60 real `ok` lines**. An unpullable image cannot produce that. Tag exists; no change needed.
**2. `v3.16.0-alpine` unverified** — same run: `woodpecker-lint` emitted **18 `ok` lines**, one per `.woodpecker/*.yml`, which is only reachable if the image has a working `/bin/sh` and `/bin/woodpecker-cli`. So the single-file invocation and exit-code behaviour are confirmed against the shipped tag.
But this one exposed something I could **not** verify, and your earlier hypothesis was the reason I looked. The comment asserted that several `deploy-*.yml` files emit `Additional property timeout is not allowed`. That was observed under **v3.9.0**, the tag this step used to pin. Whether v3.16.0 still emits it is unknown — and a green run **cannot** settle it, because the loop sends lint output to `/tmp/out` and prints it only on failure, so warnings are invisible precisely when the step passes. Your read (schema drift in a 7-minor-old CLI rather than real debt in those files) is plausible and untested. I dropped the claim rather than restating it; the comment now documents only the exit-code contract the gate actually rests on.
**3. Nit — duplicated `depends_on` rationale** — hoisted to a single note above `steps:`, with a warning that the key itself must stay on every step (it's per-step; removing it from one silently re-serialises that step).
**PR description** — updated. You were right that the body still quoted `v3.9.0-alpine` / `v0.10.0` as shipped, so the probing evidence referred to different tags than the diff. Corrected in place with an explicit note recording the bump rather than quietly rewriting it.
---
Also worth recording, since it's the substantive change in `8a580e34` and it went the opposite way from what I proposed: I had intended to enroll `woodpecker-cli` in image-automation so the pin would stop being manual, by widening `update.path` to `./`. That cannot work. Image-automation only patches YAML that is a Kubernetes manifest — [fluxcd/flux2#2826](https://github.com/fluxcd/flux2/issues/2826): *"Flux can only patch Kubernetes manifests. This means the YAML manifest must have `apiVersion` and `kind` fields."* A Woodpecker pipeline file has neither, so a marker here is silently ignored. That's a property of the file, not of `update.path`, so widening buys nothing and would drag 197 unrelated ansible/esphome YAMLs into the walk. The pin stays manual by necessity; drift is now covered by the update-audit agent, which is told to flag this image whenever it trails the deployed server.
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 51153788358ed862125c75521709622a61b90660.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new PR-gated pipeline with four offline, parallel checks (kustomize-build, woodpecker-lint, shellcheck, no-digest-pins), plus documentation updates in ci/agents/update-audit/prompt.md and kubernetes/infrastructure/image-automation/README.md explaining why the woodpecker-cli image pin can't be enrolled in image-automation. This is round 2 (commit 5115378 already addressed a prior review's two 🟡 points and one nit). I verified the concrete claims against the checkout — kustomization count (60), shell script count (6), the no-digest-pins grep behavior against the actual rotation directory, the kubectl/find/sed toolchain in the ci-k8s image's Containerfile, the .woodpecker file count (18), and that none of the 60 kustomizations reference remote bases (so the "offline" claim holds). Everything checks out; no blocking issues.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:52-97 (kustomize-build) and :170-198 (shellcheck) each hand-roll the same "discover with find, guard against empty discovery, feed a file into while read" pattern. Not asking for an extraction — two instances doesn't justify one, and the file-vs-pipe subshell reasoning is genuinely two different traps worth spelling out locally — but if a third check with the same shape gets added later, that's the point to factor it into a shared step or script under scripts/ci/.
.woodpecker/ci-manifests-test.yml:136 claims widening update.path would "drag 197 unrelated ansible/esphome YAMLs into the walk." I counted 180 .yml/.yaml files directly under ansible/ + esphome/ in this checkout — in the right ballpark and the underlying argument (widening the path pulls in a lot of unrelated files) holds regardless, but the specific figure doesn't reproduce exactly from what's countable here. Worth a quick re-check of however "197" was originally derived (e.g. did it include other top-level YAML too) before treating it as load-bearing precision elsewhere.
Tests
This PR is the test infrastructure — there's no meaningful separate test to add. The pipeline's own correctness was validated by an actual CI run referenced in the round-2 commit message (pipeline 3178 on 8a580e34: kustomize-build and no-digest-pins green against ci-k8s:2026-07-30, woodpecker-lint emitting 18 ok lines under v3.16.0-alpine), which is reasonable evidence for image/tag availability that isn't independently checkable from this static checkout alone. The failure-mode self-checks built into two of the four steps (the count -lt 50 / count -lt 1 guards against silent no-op discovery) are themselves a good defense against the exact regression the PR's own commit history shows this pipeline suffered once already.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `51153788358ed862125c75521709622a61b90660`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new PR-gated pipeline with four offline, parallel checks (`kustomize-build`, `woodpecker-lint`, `shellcheck`, `no-digest-pins`), plus documentation updates in `ci/agents/update-audit/prompt.md` and `kubernetes/infrastructure/image-automation/README.md` explaining why the `woodpecker-cli` image pin can't be enrolled in image-automation. This is round 2 (commit `5115378` already addressed a prior review's two 🟡 points and one nit). I verified the concrete claims against the checkout — kustomization count (60), shell script count (6), the `no-digest-pins` grep behavior against the actual rotation directory, the `kubectl`/find/sed toolchain in the `ci-k8s` image's Containerfile, the `.woodpecker` file count (18), and that none of the 60 kustomizations reference remote bases (so the "offline" claim holds). Everything checks out; no blocking issues.
## 🟡 Suggestions
- `.woodpecker/ci-manifests-test.yml:52-97` (`kustomize-build`) and `:170-198` (`shellcheck`) each hand-roll the same "discover with `find`, guard against empty discovery, feed a file into `while read`" pattern. Not asking for an extraction — two instances doesn't justify one, and the file-vs-pipe subshell reasoning is genuinely two different traps worth spelling out locally — but if a third check with the same shape gets added later, that's the point to factor it into a shared step or script under `scripts/ci/`.
- `.woodpecker/ci-manifests-test.yml:136` claims widening `update.path` would "drag 197 unrelated ansible/esphome YAMLs into the walk." I counted 180 `.yml`/`.yaml` files directly under `ansible/` + `esphome/` in this checkout — in the right ballpark and the underlying argument (widening the path pulls in a lot of unrelated files) holds regardless, but the specific figure doesn't reproduce exactly from what's countable here. Worth a quick re-check of however "197" was originally derived (e.g. did it include other top-level YAML too) before treating it as load-bearing precision elsewhere.
## Tests
This PR *is* the test infrastructure — there's no meaningful separate test to add. The pipeline's own correctness was validated by an actual CI run referenced in the round-2 commit message (pipeline 3178 on `8a580e34`: `kustomize-build` and `no-digest-pins` green against `ci-k8s:2026-07-30`, `woodpecker-lint` emitting 18 `ok` lines under `v3.16.0-alpine`), which is reasonable evidence for image/tag availability that isn't independently checkable from this static checkout alone. The failure-mode self-checks built into two of the four steps (the `count -lt 50` / `count -lt 1` guards against silent no-op discovery) are themselves a good defense against the exact regression the PR's own commit history shows this pipeline suffered once already.
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=51153788358ed862125c75521709622a61b90660 -->
Review could not reproduce "197 unrelated ansible/esphome YAML files" and
counted 180. The reviewer is right, and the number was wrong twice over.
Recounted in this checkout: ansible/ has 179, esphome/ has 1 -> 180, which
matches the review exactly. My 197 came from also folding in .woodpecker/,
which is wrong on both counts: those files are neither ansible nor esphome,
and they are precisely the directory you WOULD want in the walk, so counting
them as "unrelated" inverts the argument the sentence is making. The composite
is also 198 rather than 197 today — the original count was taken against
master, where .woodpecker/ still had 17 files because ci-manifests-test.yml
did not exist yet.
The conclusion is unchanged (widening update.path is pointless because Flux
cannot patch non-Kubernetes YAML at all, and it would pull in a large number
of unrelated files), but the supporting figure now reproduces, and both places
that cite it are fixed with the derivation shown so it can be re-checked.
Also recorded the review's rule-of-three note where it will actually be read:
kustomize-build and shellcheck share a discovery shape, two copies is
deliberate because the two traps commented around are different, and a third
instance is the signal to factor it into scripts/ci/ — which would pick up
unit tests via ci-scripts-test.yml on the way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The "197" figure — you're right, and it was wrong twice over. Recounted here: ansible/ has 179, esphome/ has 1 → 180, matching your count exactly.
Where 197 came from: I folded .woodpecker/ into the total. That's wrong on both counts — those files are neither ansible nor esphome, and they're precisely the directory you'd want in the walk, so counting them as "unrelated" inverts the argument the sentence is making. The composite is also 198, not 197, today: the original count ran against master, where .woodpecker/ still had 17 files because ci-manifests-test.yml didn't exist yet.
Fixed in both places that cite it (ci-manifests-test.yml and the image-automation README), now with the derivation shown (179 + 1) so it's re-checkable rather than a bare number. The conclusion is unaffected — widening update.path is pointless because Flux can't patch non-Kubernetes YAML at all, and the file count was only ever the secondary reason — but you were right to flag load-bearing precision that doesn't reproduce.
Duplicated discovery pattern — agreed, and no extraction now. I recorded your rule-of-three where the next person will hit it: a note above steps: saying two copies is deliberate (the two traps being commented around — BusyBox find lacking -printf, and a piped loop running in a subshell where fail can't propagate — are genuinely different and worth spelling out locally), and that a third instance is the signal to factor it into scripts/ci/. Worth adding: anything moved there picks up unit tests via ci-scripts-test.yml, which a shell block inlined in a pipeline file can never have — so the third-instance threshold buys more than deduplication.
Both points addressed in `13798134`.
**The "197" figure — you're right, and it was wrong twice over.** Recounted here: `ansible/` has 179, `esphome/` has 1 → **180**, matching your count exactly.
Where 197 came from: I folded `.woodpecker/` into the total. That's wrong on both counts — those files are neither ansible nor esphome, and they're precisely the directory you'd *want* in the walk, so counting them as "unrelated" inverts the argument the sentence is making. The composite is also 198, not 197, today: the original count ran against `master`, where `.woodpecker/` still had 17 files because `ci-manifests-test.yml` didn't exist yet.
Fixed in both places that cite it (`ci-manifests-test.yml` and the image-automation README), now with the derivation shown (`179 + 1`) so it's re-checkable rather than a bare number. The conclusion is unaffected — widening `update.path` is pointless because Flux can't patch non-Kubernetes YAML *at all*, and the file count was only ever the secondary reason — but you were right to flag load-bearing precision that doesn't reproduce.
**Duplicated discovery pattern** — agreed, and no extraction now. I recorded your rule-of-three where the next person will hit it: a note above `steps:` saying two copies is deliberate (the two traps being commented around — BusyBox `find` lacking `-printf`, and a piped loop running in a subshell where `fail` can't propagate — are genuinely different and worth spelling out locally), and that a **third** instance is the signal to factor it into `scripts/ci/`. Worth adding: anything moved there picks up unit tests via `ci-scripts-test.yml`, which a shell block inlined in a pipeline file can never have — so the third-instance threshold buys more than deduplication.
The file count was meaningless and would be wrong tomorrow. Correcting it from
197 to 180 last commit was the wrong fix: a number that has to be re-derived
every time the tree changes does not belong in a comment at all. It cost two
review rounds — one to flag that 197 did not reproduce, one to correct it —
and neither round was about anything that mattered, because the count was
never the argument. Widening update.path is pointless because Flux cannot
patch non-Kubernetes YAML at all; the size of the walk was only ever a
secondary aside.
Swept every snapshot figure I had introduced, not just the one flagged:
* "180 unrelated ansible/esphome YAMLs (179 + 1)" -> "the whole ansible/ and
esphome/ trees", in both the pipeline comment and the README.
* "builds 60 kustomizations" -> "builds every kustomization in the tree".
* "verified green across all 60 kustomizations" -> "across every
kustomization", keeping the as-of-writing hedge that made it a historical
claim rather than a live one.
* "expected ~60" in the discovery-floor failure message -> "floor is 50",
which cites the actual guard on the line above instead of a tree size that
drifts away from it. The message already prints the real $count.
* "scratch-based (19 MB, 2 layers, no shell)" -> "ships no shell", plus what
the reader actually needs: a step on the plain tag fails to START rather
than failing a check, which is the confusing symptom. The byte size would
have gone stale on the next upstream release.
The floor of 50 itself stays — it is a deliberately loose guard against
discovery breaking, not a description of the tree, and it is asserted in code
where a wrong value fails loudly instead of merely misinforming.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit 13798134596471a34b314f538cc8c1f8bb6a74ea.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new offline, path-filtered pipeline that runs four independent checks (kustomize-build, woodpecker-lint, shellcheck, no-digest-pins) on PRs touching kubernetes/** or .woodpecker/**, plus matching updates to the update-audit agent prompt and the image-automation README documenting why the woodpecker-cli lint image can't be Flux-managed. This is an unusually well self-verified change — I independently re-checked most of the PR's factual claims against the checkout and they all held up (see below). No blocking issues found.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:56,221 — the new ci-k8s:2026-07-30 tag is newer than the ci-k8s:2026-05-01 pin used everywhere else in the repo (e2e-test.yml, e2e.yml, alert-monitor.yml, issue-fix.yml). That's not a bug — each pipeline pins independently and the PR body documents this image was specifically verified — but it does mean the repo now carries two different dated pins for the same image with no stated reason for the split. Worth a one-line note (or a follow-up) on whether the older pins should be bumped too, so this doesn't read as an oversight to the next reader.
Tests
This is CI configuration, not application code, so "tests" here means the checks are self-validating and the PR body documents manual verification (green run on the current tree, and the #443 defect reproduced as rc=1 against the same image). I independently re-verified several of the underlying factual claims against the checkout rather than just trusting the prose:
find kubernetes -name kustomization.yaml → exactly 60 results, consistent with the count -lt 50 floor in kustomize-build.
Exactly 6*.sh files under kubernetes/, consistent with the count -lt 1 guard in shellcheck.
No kustomization in the tree references a remote base (https:///git@) or helmCharts:, so kustomize-build really is offline as claimed.
kubernetes/infrastructure/openbao/rotation/*.yaml: only kustomization.yaml matches image:.*@sha256 (it's a comment documenting the rule), and the grep -v kustomization.yaml filter in no-digest-pins correctly excludes it — confirmed no false positive today.
kubernetes/apps/woodpecker/server.yaml:43 deploys woodpecker-server:v3.16.0, matching the woodpecker-cli:v3.16.0-alpine pin — the "CLI matches deployed server" claim in both the pipeline comment and the prompt.md addition is accurate as of this commit.
kubernetes/infrastructure/image-automation/update-automation.yaml:31 — update.path: ./kubernetes today, confirming the README's claim that widening it to ./ would pull in unrelated trees.
ansible/**/*.{yml,yaml} = 179 files, esphome/**/*.{yml,yaml} = 1 file → 180 total, matching the corrected count in both the README and prompt.md additions (and matching the most recent commit on this branch, which fixed this exact number from 197).
I did not execute kubectl kustomize, woodpecker-cli lint, or shellcheck myself (not available in this environment), so the runtime behavior of the three container-image-dependent steps rests on the PR author's stated verification rather than something I reproduced.
Recommendation
Looks reasonable — no blocking issues; the suggestion above is optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `13798134596471a34b314f538cc8c1f8bb6a74ea`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new offline, path-filtered pipeline that runs four independent checks (`kustomize-build`, `woodpecker-lint`, `shellcheck`, `no-digest-pins`) on PRs touching `kubernetes/**` or `.woodpecker/**`, plus matching updates to the update-audit agent prompt and the image-automation README documenting why the `woodpecker-cli` lint image can't be Flux-managed. This is an unusually well self-verified change — I independently re-checked most of the PR's factual claims against the checkout and they all held up (see below). No blocking issues found.
## 🟡 Suggestions
- **`.woodpecker/ci-manifests-test.yml:56,221`** — the new `ci-k8s:2026-07-30` tag is newer than the `ci-k8s:2026-05-01` pin used everywhere else in the repo (`e2e-test.yml`, `e2e.yml`, `alert-monitor.yml`, `issue-fix.yml`). That's not a bug — each pipeline pins independently and the PR body documents this image was specifically verified — but it does mean the repo now carries two different dated pins for the same image with no stated reason for the split. Worth a one-line note (or a follow-up) on whether the older pins should be bumped too, so this doesn't read as an oversight to the next reader.
## Tests
This is CI configuration, not application code, so "tests" here means the checks are self-validating and the PR body documents manual verification (green run on the current tree, and the #443 defect reproduced as rc=1 against the same image). I independently re-verified several of the underlying factual claims against the checkout rather than just trusting the prose:
- `find kubernetes -name kustomization.yaml` → exactly **60** results, consistent with the `count -lt 50` floor in `kustomize-build`.
- Exactly **6** `*.sh` files under `kubernetes/`, consistent with the `count -lt 1` guard in `shellcheck`.
- No kustomization in the tree references a remote base (`https://`/`git@`) or `helmCharts:`, so `kustomize-build` really is offline as claimed.
- `kubernetes/infrastructure/openbao/rotation/*.yaml`: only `kustomization.yaml` matches `image:.*@sha256` (it's a comment documenting the rule), and the `grep -v kustomization.yaml` filter in `no-digest-pins` correctly excludes it — confirmed no false positive today.
- `kubernetes/apps/woodpecker/server.yaml:43` deploys `woodpecker-server:v3.16.0`, matching the `woodpecker-cli:v3.16.0-alpine` pin — the "CLI matches deployed server" claim in both the pipeline comment and the prompt.md addition is accurate as of this commit.
- `kubernetes/infrastructure/image-automation/update-automation.yaml:31` — `update.path: ./kubernetes` today, confirming the README's claim that widening it to `./` would pull in unrelated trees.
- `ansible/**/*.{yml,yaml}` = 179 files, `esphome/**/*.{yml,yaml}` = 1 file → 180 total, matching the corrected count in both the README and prompt.md additions (and matching the most recent commit on this branch, which fixed this exact number from 197).
I did not execute `kubectl kustomize`, `woodpecker-cli lint`, or `shellcheck` myself (not available in this environment), so the runtime behavior of the three container-image-dependent steps rests on the PR author's stated verification rather than something I reproduced.
## Recommendation
Looks reasonable — no blocking issues; the suggestion above is optional.
<!-- pr-review:sha=13798134596471a34b314f538cc8c1f8bb6a74ea -->
Correcting 197 → 180 in the previous commit was the wrong fix. ba052332 deletes the number instead.
A count that has to be re-derived every time the tree changes doesn't belong in a comment at all — it's stale the moment anything is added, and it was never the argument here. Widening update.path is pointless because Flux can't patch non-Kubernetes YAML at all; the size of the walk was a secondary aside. That aside consumed two review rounds (one to flag that 197 didn't reproduce, one to correct it), neither of which touched anything load-bearing.
Swept every snapshot figure I'd introduced, not only the flagged one:
was
now
180 unrelated ansible/esphome YAMLs (179 + 1)
the whole ansible/ and esphome/ trees (comment + README)
builds 60 kustomizations
builds every kustomization in the tree
verified green across all 60 kustomizations
across every kustomization, keeping the as-of-writing hedge
expected ~60 in the floor failure message
floor is 50 — cites the actual guard on the line above, not a tree size that drifts away from it
scratch-based (19 MB, 2 layers, no shell)
ships no shell, plus the symptom that actually helps: a step on the plain tag fails to start rather than failing a check
The -lt 50 floor stays. That one is a deliberately loose guard asserted in code, where a wrong value fails the build loudly — as opposed to a number in a comment, which just misinforms quietly. That distinction is the general rule I should have applied from the start: if a count matters, assert it where drift breaks something; if it doesn't, don't write it down.
Correcting 197 → 180 in the previous commit was the wrong fix. `ba052332` deletes the number instead.
A count that has to be re-derived every time the tree changes doesn't belong in a comment at all — it's stale the moment anything is added, and it was never the argument here. Widening `update.path` is pointless because Flux can't patch non-Kubernetes YAML *at all*; the size of the walk was a secondary aside. That aside consumed two review rounds (one to flag that 197 didn't reproduce, one to correct it), neither of which touched anything load-bearing.
Swept every snapshot figure I'd introduced, not only the flagged one:
| was | now |
|---|---|
| `180 unrelated ansible/esphome YAMLs (179 + 1)` | `the whole ansible/ and esphome/ trees` (comment + README) |
| `builds 60 kustomizations` | `builds every kustomization in the tree` |
| `verified green across all 60 kustomizations` | `across every kustomization`, keeping the as-of-writing hedge |
| `expected ~60` in the floor failure message | `floor is 50` — cites the actual guard on the line above, not a tree size that drifts away from it |
| `scratch-based (19 MB, 2 layers, no shell)` | `ships no shell`, plus the symptom that actually helps: a step on the plain tag **fails to start** rather than failing a check |
The `-lt 50` floor stays. That one is a deliberately loose guard asserted **in code**, where a wrong value fails the build loudly — as opposed to a number in a comment, which just misinforms quietly. That distinction is the general rule I should have applied from the start: if a count matters, assert it where drift breaks something; if it doesn't, don't write it down.
Review noted this file pins ci-k8s at a newer dated tag than e2e.yml,
e2e-test.yml, alert-monitor.yml and issue-fix.yml, and asked for either a
stated reason or a follow-up so it does not read as an oversight.
There is no reason — the other four are stale. Per the latest-stable rule all
five should sit on the newest dated tag.
Not bumped here on purpose. Those four pipelines exercise different tooling in
the image and three are cron- or event-triggered, so none of them run on this
PR. Bumping them in this branch would ship four untested changes under cover of
a green run that never touched them — the same shape as the digest decay in
#458, where breakage sat unnoticed precisely because the affected jobs did not
run for months. Each wants its own verification, so it is tracked as #461
rather than smuggled in here.
Comment added above kustomize-build so the split is explained where a reader
meets it, pointing at the issue rather than restating the whole argument.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Picking up the review for 13798134 (superseded by two commits since, but its 🟡 point was still open).
ci-k8s tag split — you're right that it reads as an oversight, because it partly is: the other four pins are simply stale. e2e-test.yml:7, e2e.yml:33, alert-monitor.yml:45 and issue-fix.yml:52 all sit on the older tag, and per the latest-stable rule all five should be on the newest.
I did not bump them here, deliberately. None of those four are triggered by this PR — they're cron- or event-driven and exercise different tooling inside the image — so bumping them in this branch would ship four untested changes under cover of a green run that never touched them. That's the exact shape of the digest decay in #458, where the breakage sat unnoticed for months precisely because the affected jobs didn't run. Each wants its own verification.
Tracked as #461, with a comment above kustomize-build explaining the split where a reader meets it (b1f5d6dc).
Two notes on your verification section, since some of it no longer applies:
The 180 figure you re-derived and confirmed has since been deleted rather than corrected (ba052332). It was a snapshot that would be wrong as soon as anything is added to those trees, and it was never the argument — Flux can't patch non-Kubernetes YAML at all, so the size of the walk was a secondary aside. Same sweep removed 60 kustomizations, 19 MB, 2 layers, and expected ~60 (now floor is 50, citing the actual guard). Worth knowing that re-deriving those numbers is wasted effort on this file going forward — if a count matters it should be asserted in code where drift fails the build, as the -lt 50 floor is.
Your 60 / 6 / no-remote-bases / self-matching-grep checks all still hold and were the useful half — those are properties, not snapshots.
Picking up the review for `13798134` (superseded by two commits since, but its 🟡 point was still open).
**`ci-k8s` tag split** — you're right that it reads as an oversight, because it partly is: the other four pins are simply stale. `e2e-test.yml:7`, `e2e.yml:33`, `alert-monitor.yml:45` and `issue-fix.yml:52` all sit on the older tag, and per the latest-stable rule all five should be on the newest.
I did **not** bump them here, deliberately. None of those four are triggered by this PR — they're cron- or event-driven and exercise different tooling inside the image — so bumping them in this branch would ship four untested changes under cover of a green run that never touched them. That's the exact shape of the digest decay in #458, where the breakage sat unnoticed for months precisely because the affected jobs didn't run. Each wants its own verification.
Tracked as **#461**, with a comment above `kustomize-build` explaining the split where a reader meets it (`b1f5d6dc`).
---
Two notes on your verification section, since some of it no longer applies:
- The `180` figure you re-derived and confirmed has since been **deleted** rather than corrected (`ba052332`). It was a snapshot that would be wrong as soon as anything is added to those trees, and it was never the argument — Flux can't patch non-Kubernetes YAML *at all*, so the size of the walk was a secondary aside. Same sweep removed `60 kustomizations`, `19 MB, 2 layers`, and `expected ~60` (now `floor is 50`, citing the actual guard). Worth knowing that re-deriving those numbers is wasted effort on this file going forward — if a count matters it should be asserted in code where drift fails the build, as the `-lt 50` floor is.
- Your `60` / `6` / no-remote-bases / self-matching-grep checks all still hold and were the useful half — those are properties, not snapshots.
🤖Automated review — model claude-sonnet-5 (complexity: normal), commit b1f5d6dcbc5fa42e33457904edcbf87ee18b91ea.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new PR-gated pipeline with four parallel, offline checks over kubernetes/** and .woodpecker/** (kustomize build, woodpecker-cli lint, shellcheck at error severity, and a digest-pin guard for the openbao rotators), plus supporting doc updates in ci/agents/update-audit/prompt.md and kubernetes/infrastructure/image-automation/README.md. I checked the concrete, checkable claims against the repo checkout — kustomization count (60, floor is 50 ✓), shell-script count (6, matches "all 6 are clean" ✓), the BusyBox find in ci-k8s's base image (alpine:3.21 with no findutils installed, so -printf really would fail ✓), the rotation/kustomization.yaml self-match for the @sha256 grep (confirmed, the filter is load-bearing ✓), the depends_on: [] parallel-step convention (matches existing usage in build-ci-images.yml ✓), and that no kustomization in the tree pulls remote bases or Helm charts (so kubectl kustomize really is offline for all 60 ✓). Everything checks out; I found no blocking issues.
🟡 Suggestions
ci-manifests-test.yml:196 — the shellcheck step's discovery floor is count -lt 1 (i.e., "at least one script found"), far weaker than kustomize-build's floor of 50. That's proportionate today (only 6 scripts exist), but unlike the kustomize floor it wouldn't catch a partial-discovery regression (e.g. a future find filter that only matches half the tree) — it would only catch total discovery failure. Not worth blocking on given the small, slow-changing script count, but worth keeping in mind if the kubernetes/ tree grows a lot more shell.
ci-manifests-test.yml:69,161,188,234 — the three third-party image pins (ci-k8s:2026-07-30 freshly built, woodpecker-cli:v3.16.0-alpine, shellcheck-alpine:v0.11.0) and the /bin/woodpecker-cli binary path / working /bin/sh claim are asserted in the comments as empirically probed, but I have no way to pull images or re-run the pipeline from this checkout to confirm them independently. If CI is green on this PR's own run, that's the real confirmation; if it isn't, start there.
Tests
This is CI configuration for a homelab GitOps repo — there's no unit-test harness for Woodpecker pipeline YAML itself, and the PR description documents that each step was manually verified both for the pass case (current tree) and the specific failure case it targets (the #443 YAML-scalar bug for lint, -printf silent-no-op and subshell-swallowed failures for the discovery/loop pattern, the rotation-dir self-match for the digest guard). That's the appropriate level of verification for this kind of change; I independently re-confirmed the parts of it that are checkable from the static checkout (counts, image base packages, self-match), and they hold up. Nothing further to ask for here.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `b1f5d6dcbc5fa42e33457904edcbf87ee18b91ea`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new PR-gated pipeline with four parallel, offline checks over `kubernetes/**` and `.woodpecker/**` (kustomize build, woodpecker-cli lint, shellcheck at `error` severity, and a digest-pin guard for the openbao rotators), plus supporting doc updates in `ci/agents/update-audit/prompt.md` and `kubernetes/infrastructure/image-automation/README.md`. I checked the concrete, checkable claims against the repo checkout — kustomization count (60, floor is 50 ✓), shell-script count (6, matches "all 6 are clean" ✓), the BusyBox `find` in `ci-k8s`'s base image (alpine:3.21 with no `findutils` installed, so `-printf` really would fail ✓), the `rotation/kustomization.yaml` self-match for the `@sha256` grep (confirmed, the filter is load-bearing ✓), the `depends_on: []` parallel-step convention (matches existing usage in `build-ci-images.yml` ✓), and that no kustomization in the tree pulls remote bases or Helm charts (so `kubectl kustomize` really is offline for all 60 ✓). Everything checks out; I found no blocking issues.
## 🟡 Suggestions
- `ci-manifests-test.yml:196` — the `shellcheck` step's discovery floor is `count -lt 1` (i.e., "at least one script found"), far weaker than `kustomize-build`'s floor of 50. That's proportionate today (only 6 scripts exist), but unlike the kustomize floor it wouldn't catch a partial-discovery regression (e.g. a future `find` filter that only matches half the tree) — it would only catch total discovery failure. Not worth blocking on given the small, slow-changing script count, but worth keeping in mind if the `kubernetes/` tree grows a lot more shell.
- `ci-manifests-test.yml:69,161,188,234` — the three third-party image pins (`ci-k8s:2026-07-30` freshly built, `woodpecker-cli:v3.16.0-alpine`, `shellcheck-alpine:v0.11.0`) and the `/bin/woodpecker-cli` binary path / working `/bin/sh` claim are asserted in the comments as empirically probed, but I have no way to pull images or re-run the pipeline from this checkout to confirm them independently. If CI is green on this PR's own run, that's the real confirmation; if it isn't, start there.
## Tests
This is CI configuration for a homelab GitOps repo — there's no unit-test harness for Woodpecker pipeline YAML itself, and the PR description documents that each step was manually verified both for the pass case (current tree) and the specific failure case it targets (the `#443` YAML-scalar bug for lint, `-printf` silent-no-op and subshell-swallowed failures for the discovery/loop pattern, the rotation-dir self-match for the digest guard). That's the appropriate level of verification for this kind of change; I independently re-confirmed the parts of it that are checkable from the static checkout (counts, image base packages, self-match), and they hold up. Nothing further to ask for here.
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=b1f5d6dcbc5fa42e33457904edcbf87ee18b91ea -->
Review flagged that shellcheck's floor (1) is far weaker than kustomize-build's
(50), so it catches total discovery failure but not partial loss.
Correct, and kept as-is with the reasoning written down. The kustomize floor is
loose relative to the tree it guards, which is what lets it catch gross partial
loss without tripping on churn. The shell-script set is small enough that any
floor tight enough to catch partial loss would sit inside normal churn:
deleting one script would fail CI spuriously, and the fix a hurried person
reaches for is lowering the floor — which is how a guard becomes decoration. A
floor that must be edited whenever the tree changes is not a guard, it is a
snapshot with an exit code, and this file just had all its snapshots removed
for the same reason.
So it catches the failure that actually occurred (discovery silently returning
nothing, step green having checked nothing) and not partial loss. Noted as
worth revisiting if kubernetes/ grows a lot more shell, when a loose
proportionate floor becomes possible.
The review's other point — that the image pins and the /bin/sh and
/bin/woodpecker-cli claims are not checkable from a static checkout, and that a
green run is the real confirmation — is now settled by pipeline 3185 on
b1f5d6dc: all four steps green, with 60 / 18 / 6 ok lines from kustomize-build,
woodpecker-lint and shellcheck respectively. Those counts are cited here as
run-time evidence, not written into the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Image pins / /bin/sh / /bin/woodpecker-cli unverifiable from a static checkout — you called the right test: CI green on this PR's own run. It is. Pipeline 3185 on b1f5d6dc, all four steps green, and each did real work rather than passing vacuously:
kustomize-build → 60 ok lines (so ci-k8s:2026-07-30 pulls)
woodpecker-lint → 18 ok lines, one per .woodpecker/*.yml (so v3.16.0-alpine has a working /bin/shand/bin/woodpecker-cli — the loop couldn't produce those lines otherwise)
shellcheck → 6 ok lines (so shellcheck-alpine:v0.11.0 pulls)
shellcheck floor of 1 vs kustomize's 50 — real asymmetry, kept deliberately, now with the reasoning in the file rather than left to look like an oversight.
Your floor of 50 works because it's loose relative to the tree it guards: that slack is what lets it catch gross partial loss without tripping on churn. The shell-script set is small enough that any floor tight enough to catch partial loss would sit inside normal churn — delete one script and CI fails spuriously, and the fix a hurried person reaches for is lowering the floor. That's how a guard becomes decoration.
Put differently: a floor that has to be edited whenever the tree changes isn't a guard, it's a snapshot with an exit code — and this file just had all its snapshot numbers stripped out for exactly that reason. So the floor stays at 1, catching the failure that actually happened here (discovery silently returning nothing, step green having checked no files), with a note to revisit if kubernetes/ grows a lot more shell.
Consistent with that, the 60/18/6 counts above are cited as run-time evidence in the commit message — not written into the file.
Both points addressed in `39dc23a9`.
**Image pins / `/bin/sh` / `/bin/woodpecker-cli` unverifiable from a static checkout** — you called the right test: CI green on this PR's own run. It is. **Pipeline 3185 on `b1f5d6dc`**, all four steps green, and each did real work rather than passing vacuously:
- `kustomize-build` → 60 `ok` lines (so `ci-k8s:2026-07-30` pulls)
- `woodpecker-lint` → 18 `ok` lines, one per `.woodpecker/*.yml` (so `v3.16.0-alpine` has a working `/bin/sh` **and** `/bin/woodpecker-cli` — the loop couldn't produce those lines otherwise)
- `shellcheck` → 6 `ok` lines (so `shellcheck-alpine:v0.11.0` pulls)
**shellcheck floor of 1 vs kustomize's 50** — real asymmetry, kept deliberately, now with the reasoning in the file rather than left to look like an oversight.
Your floor of 50 works *because* it's loose relative to the tree it guards: that slack is what lets it catch gross partial loss without tripping on churn. The shell-script set is small enough that any floor tight enough to catch partial loss would sit *inside* normal churn — delete one script and CI fails spuriously, and the fix a hurried person reaches for is lowering the floor. That's how a guard becomes decoration.
Put differently: a floor that has to be edited whenever the tree changes isn't a guard, it's a snapshot with an exit code — and this file just had all its snapshot numbers stripped out for exactly that reason. So the floor stays at 1, catching the failure that actually happened here (discovery silently returning nothing, step green having checked no files), with a note to revisit if `kubernetes/` grows a lot more shell.
Consistent with that, the 60/18/6 counts above are cited as run-time evidence in the commit message — not written into the file.
🤖Automated review — model CLI default (complexity: complex), commit 39dc23a9993cf97de799c4bc13a7ba85b0011e7a.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Adds .woodpecker/ci-manifests-test.yml, a new pull-request pipeline that runs four offline, parallel checks (kustomize-build, woodpecker-lint, shellcheck, no-digest-pins) against kubernetes/** and .woodpecker/**, plus matching updates to the update-audit agent prompt and the image-automation README documenting that the new woodpecker-cli pin can't be enrolled in Flux image-automation. I checked every factual claim in the diff against the actual checkout (kustomization/script counts, the kustomization.yaml self-match trap for the digest grep, the ci-k8s tag split across other pipelines, the deployed woodpecker-server version and its 3.x auto-bump range, absence of remote bases/helmCharts that would break offline kubectl kustomize, and the depends_on: [] parallel-step convention already used in build-ci-images.yml) and all of them check out. No blocking issues.
🟡 Suggestions
.woodpecker/ci-manifests-test.yml:80 and :192 — dirs=$(find kubernetes -name kustomization.yaml | sed '...' | sort) and files=$(find kubernetes -name '*.sh' | sort) run under set -eu without pipefail (ash doesn't support it anyway). If find itself ever errored (permission issue, path typo), the command substitution's exit status is sort's, not find's, so the error would be swallowed rather than tripping set -e — the discovery-floor check would still likely catch a resulting empty/short list, but a partial silent failure (e.g. find erroring on one subtree but still emitting output for others) wouldn't be distinguished from a genuinely smaller-but-valid result. Given the step's own stated philosophy is "refusing to report success" on broken discovery, this is a small gap in that guarantee — not worth blocking on since the floor checks already catch the realistic failure modes, but worth a one-line comment if you want the reasoning as airtight as the rest of the file.
The event: pull_request + path: combination (ci-manifests-test.yml:26-29) has no other precedent in this repo's .woodpecker/*.yml (existing path filters are all on push). The PR's own last commit message cites pipeline 3185 as having fired and passed on this exact combination, which is good evidence it works — just flagging that it's a new pattern here in case that run isn't easily re-checkable by other reviewers.
Tests
This PR is the test infrastructure — there's no separate test suite to add. The author states pipeline 3185 ran all four steps green against the real counts (60 kustomizations, 18 pipeline configs, 6 shell scripts), which I independently confirmed match the current tree (find kubernetes -name kustomization.yaml → 60, find kubernetes -name '*.sh' → 6). That's reasonable verification for a CI-definition change. The ci/agents/update-audit/prompt.md and README.md changes are documentation/prompt text with no automated check, but I read them for internal consistency (item numbering, image list, drift-tracking cross-references) and found no contradictions with the rest of those files.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 **Automated review** — model `CLI default` (complexity: complex), commit `39dc23a9993cf97de799c4bc13a7ba85b0011e7a`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Adds `.woodpecker/ci-manifests-test.yml`, a new pull-request pipeline that runs four offline, parallel checks (`kustomize-build`, `woodpecker-lint`, `shellcheck`, `no-digest-pins`) against `kubernetes/**` and `.woodpecker/**`, plus matching updates to the update-audit agent prompt and the image-automation README documenting that the new `woodpecker-cli` pin can't be enrolled in Flux image-automation. I checked every factual claim in the diff against the actual checkout (kustomization/script counts, the `kustomization.yaml` self-match trap for the digest grep, the `ci-k8s` tag split across other pipelines, the deployed `woodpecker-server` version and its `3.x` auto-bump range, absence of remote bases/`helmCharts` that would break offline `kubectl kustomize`, and the `depends_on: []` parallel-step convention already used in `build-ci-images.yml`) and all of them check out. No blocking issues.
## 🟡 Suggestions
- `.woodpecker/ci-manifests-test.yml:80` and `:192` — `dirs=$(find kubernetes -name kustomization.yaml | sed '...' | sort)` and `files=$(find kubernetes -name '*.sh' | sort)` run under `set -eu` without `pipefail` (ash doesn't support it anyway). If `find` itself ever errored (permission issue, path typo), the command substitution's exit status is `sort`'s, not `find`'s, so the error would be swallowed rather than tripping `set -e` — the discovery-floor check would still likely catch a resulting empty/short list, but a *partial* silent failure (e.g. `find` erroring on one subtree but still emitting output for others) wouldn't be distinguished from a genuinely smaller-but-valid result. Given the step's own stated philosophy is "refusing to report success" on broken discovery, this is a small gap in that guarantee — not worth blocking on since the floor checks already catch the realistic failure modes, but worth a one-line comment if you want the reasoning as airtight as the rest of the file.
- The `event: pull_request` + `path:` combination (`ci-manifests-test.yml:26-29`) has no other precedent in this repo's `.woodpecker/*.yml` (existing `path` filters are all on `push`). The PR's own last commit message cites pipeline 3185 as having fired and passed on this exact combination, which is good evidence it works — just flagging that it's a new pattern here in case that run isn't easily re-checkable by other reviewers.
## Tests
This PR *is* the test infrastructure — there's no separate test suite to add. The author states pipeline 3185 ran all four steps green against the real counts (60 kustomizations, 18 pipeline configs, 6 shell scripts), which I independently confirmed match the current tree (`find kubernetes -name kustomization.yaml` → 60, `find kubernetes -name '*.sh'` → 6). That's reasonable verification for a CI-definition change. The `ci/agents/update-audit/prompt.md` and `README.md` changes are documentation/prompt text with no automated check, but I read them for internal consistency (item numbering, image list, drift-tracking cross-references) and found no contradictions with the rest of those files.
## Recommendation
Looks reasonable — no blocking issues; comments above are optional.
<!-- pr-review:sha=39dc23a9993cf97de799c4bc13a7ba85b0011e7a -->
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes the gap reviewers flagged on three separate PRs today: nothing validates
kubernetes/**.deploy-infrastructure.ymlgates onlyopentofu/**, andci-scripts-test.ymlruns only thescripts/ci/unit tests — so a broken manifest reached the cluster and was caught by Flux failing to reconcile, or by someone noticing.Four checks, all offline
No cluster, no registry, no secrets — every one passes or fails on the checkout alone.
kustomize-buildwoodpecker-lintshellcheck.shunderkubernetes/, severityerrorno-digest-pins@sha256pinsEach was verified green against the tree as it stands before being added, so this gates new breakage rather than failing on pre-existing debt: all 60 kustomizations build, all pipeline configs lint, all 6 shell scripts are clean at error severity, and no digest pins remain after #459.
The lint step is the one that matters
It exits 1 on the exact defect that took every pipeline in this repo down earlier today (#443) — an unquoted YAML scalar containing
": ", whichyqaccepts because it is valid YAML, just a map instead of a string. That's why "it parses" was mistaken for "it's valid".Verified both directions in the image the step actually uses:
and
rc=0on the pre-existingAdditional property timeout is not allowedwarnings in fourdeploy-*.ymlfiles — so it fails on real breakage without blocking on legacy noise.Image choices — all version-pinned, no floating tags
ci-k8s:2026-07-30—kubectlfor kustomize, and the shell for the grep guard. (I first wrote:latesthere; caught and fixed, per the never-latestrule.)woodpecker-cli:v3.16.0-alpine— the non-alpine image is scratch-based with no/bin/sh(19 MB, 2 layers), so Woodpeckercommands:cannot run in it at all. The binary is/bin/woodpecker-cli, not onPATHaswoodpecker. Both found by probing the image rather than assuming.shellcheck-alpine:v0.11.0— pinned rather than:stable, which floats exactly like:latest.Deliberately not included
kubeconform. It needs CRD schemas forExternalSecret,Certificate,HelmReleaseand friends — so it either needs a schema-fetching step (network access, and a new flake source) or-ignore-missing-schemas, which would skip precisely the CRDs most of this tree is made of. Worth doing properly as its own change rather than adding a check that looks like coverage and isn't.Known gap, stated rather than implied away
shellcheckcannot see shell embedded in manifests — init containers, probeexecblocks. That's where several real bugs lived this year (thegrep --line-bufferedno-op, themosquitto_pub -Wflag that doesn't exist). Called out in the file so nobody reads this pipeline as covering it.🤖 Generated with Claude Code
🤖 Automated review — model
claude-sonnet-5(complexity: normal), commitcd12bc16c9b2bf78b02b95ba97870a1184ab371a.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new offline pull_request pipeline with four steps (kustomize-build,woodpecker-lint,shellcheck,no-digest-pins) closing a real gap: nothing currently validateskubernetes/**. The rationale is well-documented, the claimed facts about the current tree (60 kustomizations, 6 shell scripts, 4deploy-*.ymlfiles with the pre-existingtimeoutwarning, no digest pins remaining underopenbao/rotation/) all check out against the checkout. One likely portability bug in the flagshipkustomize-buildstep needs to be settled before merge — see below.🔴 Blocking
.woodpecker/ci-manifests-test.yml:36—find kubernetes -name kustomization.yaml -printf '%h\n'relies on-printf, a GNU findutils extension. Thekustomize-buildstep runs ingitea.renner.uno/gitea_admin/ci-k8s:2026-07-30, whoseContainerfile(ci/images/k8s/Containerfile) isFROM alpine:3.21and onlyapk addsaws-cli bash bind-tools ca-certificates curl git openssh-client openssl sops— it never installsfindutilsorcoreutils. That meansfindresolves to BusyBox'sfindapplet, which historically does not implement-printf(its documented action set is-print/-print0/-prune/-delete/-exec/… but no-printf).If that holds, the failure mode is not a loud error — it's silent:
for d in $(find … -printf …)sits inside a command substitution used as a word list, so underset -eua failing/erroringfindjust yields empty stdout, theforloop runs zero times,failstays0, and the step exits0. The step would report green on every PR without ever runningkubectl kustomizeon anything, which is the opposite of what this PR is for.I could not execute the actual container image from this environment to confirm BusyBox's exact feature set in this build, so I'm not stating this as certain — but it's a specific, checkable claim. The one check that settles it:
docker run --rm gitea.renner.uno/gitea_admin/ci-k8s:2026-07-30 sh -c "find --help 2>&1 | grep printf"(or just run the step's loop by hand once). If-printfisn't supported, a portable fix is-exec dirname {} \;in place of-printf '%h\n', which BusyBox'sfinddoes document.🟡 Suggestions
woodpecker-lint,shellcheck,no-digest-pins) uses only basicfind -name/glob/grep constructs that are portable across BusyBox and GNU implementations, and the claims in the PR body about the current state of the tree (60 kustomizations, 6.shfiles underkubernetes/, no@sha256pins underopenbao/rotation/,timeout:present in exactly the four nameddeploy-*.ymlfiles) all verified against the checkout.🟢 Nits
None.
Tests
This is the test/validation infrastructure — there's no meta-test for a CI pipeline definition beyond what the PR description already reports (each step run against the current tree, both directions verified for
woodpecker-lint). That's a reasonable bar for this kind of change. The one gap is the-printfportability question above, which by nature wouldn't be caught by "run it against a healthy tree and see green" — a silently-no-op step and a correctly-working step produce the same observable result (exit 0) when every kustomization already builds cleanly, which is exactly the situation described in the PR body ("all 60 kustomizations build"). Worth having the author paste the actualkustomize-buildstep log (with per-directoryok/FAILlines, not just the final exit code) to prove the loop iterated 60 times rather than zero.Recommendation
Request changes — 1 blocking issue (unverified but concrete: confirm
find -printfworks in theci-k8s:2026-07-30image, or switch to-exec dirname {} \;).🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit790fd1933a4c45ecd49cdf31a91c8e4da2072a4b.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new pull_request-triggered pipeline with four offline, static checks overkubernetes/**and.woodpecker/**:kustomize-build,woodpecker-lint,shellcheck, andno-digest-pins. I re-derived every quantitative claim in the PR body against the checkout (kustomization count, shell-script count, current digest-pin state, the self-match/exclusion behavior of theno-digest-pinsgrep) and all of them check out. No blocking issues.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:140-141— the comment "image:anchors the match so the kustomization comment documenting this rule does not flag itself" is inaccurate about the mechanism. I ran the actual grep (grep -l 'image:.*@sha256' kubernetes/infrastructure/openbao/rotation/*.yaml) against the checkout:kustomization.yamldoes match — its own documentation comment atkubernetes/infrastructure/openbao/rotation/kustomization.yaml:33-34literally contains the substringimage:.*@sha256, which satisfies the regex. What actually excludes it is the subsequentgrep -v kustomization.yamlfilename filter, not "image:anchoring." The behavior is correct (confirmed: the pipe correctly drops it,no-digest-pinswould pass today), andkubernetes/infrastructure/openbao/rotation/kustomization.yaml:36-39already documents this self-match/filter interaction accurately — only the new file's comment mischaracterizes it. Worth a one-line fix since this codebase clearly leans on comments being trustworthy documentation of trap behavior..woodpecker/ci-manifests-test.yml:37,143—ci-k8s:2026-07-30and (line 81)woodpecker-cli:v3.9.0-alpine/ (line 107)shellcheck-alpine:v0.10.0are the first references to these tags/images anywhere in the repo (existing pipelines pinci-k8s:2026-05-01). I can't pull images from this sandbox to confirmci-k8s:2026-07-30has actually been pushed to the registry — if it hasn't, bothkustomize-buildandno-digest-pinsfail at image-pull on every PR rather than at the check itself. Circumstantial evidence is good (other rotator manifests already referencesecret-rotator:2026-07-30, built by the same date-tagging convention inbuild-ci-images.yml, and today's date is 2026-07-30), but this is worth a final confirmation that the tag resolves before merge, since a pull failure here would block every PR, not just ones touchingkubernetes/**.Tests
Not applicable in the traditional sense — this PR is the test harness for
kubernetes/**and.woodpecker/**. The right validation is running each check against the real checkout/images, which the PR body describes doing and which I independently reproduced for the parts checkable from disk:find kubernetes -name kustomization.yaml→ 60 (guard threshold 50, comfortable margin),find kubernetes -name '*.sh'→ 6 scripts, all with proper shebangs (bashorsh) soshellcheckcan auto-detect dialect without a-sflag, and no@sha256-pinned images currently underopenbao/rotation/. Thewoodpecker-lintand image-pull correctness claims (verified against the live images per the PR body) aren't independently checkable from this sandboxed, network-isolated checkout.Recommendation
Looks reasonable — no blocking issues; comments above are optional.
Blocking issue confirmed and fixed in
790fd193. You were right, and the diagnosis was exact.The gate was a silent no-op
Confirmed by running it in the image rather than reasoning about BusyBox feature sets:
And the proof it mattered is in this PR's own pipeline run 3158: the
kustomize-buildstep is recorded as success, and its log contains zerooklines where sixty were expected. So the gate I wrote to catch silent no-ops was itself one, and CI reported it green.Why my verification missed it, twice
Worth recording, because both failures were confident:
-printfworks.Your suggested check (inspect the per-directory log lines, not the exit code) is the only one that separates those cases. I had the log available and did not look at it.
Fixed, and verified the way you asked
Portable discovery via
find … | sed 's|/kustomization.yaml$||'. Pipeline run 3161:Same
successstatus as before, sixty times the actual work.The general fix, which matters more than the -printf detail
Both find-based steps now assert their discovery found something before proceeding —
kustomize-buildrequires >=50 of the ~60,shellcheckrequires >=1 — and fail loudly otherwise.Without that, any future breakage of discovery degrades back into a green no-op.
-printfwas the symptom; a check that cannot tell it checked nothing is the disease, and it is the same shape as the two bugs this pipeline was written to catch (grep --line-bufferedsilently matching nothing,mosquitto_pub -Wsilently failing every publish).Both addressed.
🟡 The comment misidentified the mechanism — you are right
Verified by running the bare grep exactly as you did:
So
image:anchoring does nothing for self-matching; the filename filter is load-bearing. Fixed, and the comment now says so explicitly — because a comment that misidentifies the load-bearing line invites someone to delete the filter as redundant tidying, at which point the step fails permanently on its own documentation.Worth admitting this is the third time today the same self-matching trap caught me: twice while writing the guard in
rotation/kustomization.yaml, and once more in the comment describing it here. You are also right that this codebase leans on comments being trustworthy — a wrong explanation next to correct code is its own defect.🟡 Confirming the image tags resolve
Good instinct, and the consequence you name is right: this pipeline has no path filter, so it runs on every PR — an unpullable image would block all of them, not just
kubernetes/**ones.Stronger evidence than a manual pull: pipeline 3161 ran this pipeline against a real checkout and every step succeeded, which means all three images pulled in the runner:
And
discovered 60 kustomizations/ 60oklines in that same run, so it pulled and did the work.On
ci-k8s:2026-05-01being what other pipelines pin: that is worth a separate look.ci-k8sis rebuilt whenever.woodpecker/build-ci-images.ymlchanges, so newer dated tags exist and the older pin is simply stale rather than deliberate — the same drift that left the rotators pinned to dead digests, just less severe because a tag does not get garbage-collected.🤖 Automated review — model
claude-sonnet-5(complexity: normal), commitddf5f5007bad978c797f1e3cf75fc0468daa0142.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new offlinepull_requestpipeline with four steps (kustomize-build,woodpecker-lint,shellcheck,no-digest-pins) that closes a real, previously-demonstrated gap: nothing validatedkubernetes/**or the pipeline configs themselves before this. I checked the script logic line by line against the actual checkout and it holds up — counts match (60 kustomizations, 6 shell scripts underkubernetes/), theno-digest-pinsgrep-filter self-match claim is verified againstkubernetes/infrastructure/openbao/rotation/kustomization.yaml, and theerrexit/if-exemption reasoning in the comments is correct POSIX sh behavior. No secrets, no cluster/network access, no blocking defects found.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:31,143— new image taggitea.renner.uno/gitea_admin/ci-k8s:2026-07-30cannot be verified from this checkout (no network/registry access here). This repo has already been bitten by exactly this failure mode once:kubernetes/infrastructure/openbao/rotation/kustomization.yamldocuments 11 rotators that silently died because they were pinned to a manifest that no longer resolved (#458), and its own "BUMPING the tag" guidance says to verify a new tag exists before committing, since an unbuilt tag fails "just as quietly." Every other pipeline in.woodpecker/that uses this image referencesci-k8s:2026-05-01(e.g.e2e.yml,alert-monitor.yml,issue-fix.yml) — this PR is the first reference to the2026-07-30tag anywhere in the tree, and the diff doesn't touchci/images/k8s/**, so nothing in this PR itself triggers a rebuild that would publish it. If that tag wasn't pushed by a separatebuild-ci-imagesrun today, both steps that use it will fail to pull the image on every PR. Low risk of silent failure (a bad pull fails loudly), but worth confirming the tag resolves — the first CI run of this very pipeline will prove it either way.path:filter on thewhen:block (ci-manifests-test.yml:25-26), so all four steps run on every PR regardless of whetherkubernetes/**or.woodpecker/**changed (e.g. a docs-only PR still pulls three extra images and builds 60 kustomizations). This matches the existing convention inci-scripts-test.yml(also unfiltered), so it's consistent with the repo's style rather than a new problem — just flagging in case unfiltered-by-design wasn't the intent here too.$dirs/$filesin theforloops (kustomize-build,shellchecksteps) would break on a directory or filename containing whitespace. Not an issue for the current tree (checked — no such names underkubernetes/), and consistent with the informal quoting style elsewhere in.woodpecker/*.yml, so this is a latent assumption rather than a live bug.Tests
This is itself the test: a CI-validation pipeline whose correctness is demonstrated by running against the real tree, which happens automatically on this PR. The anti-no-op guards (
count -lt 50/count -lt 1thresholds inkustomize-buildandshellcheck) are a genuinely good touch — they specifically prevent the failure mode the PR body describes happening once already (discovery silently returning nothing, step reporting green). No separate unit-test file is expected or missing here, matching howci-scripts-test.yml/deploy-infrastructure.ymlaren't unit-tested either.Recommendation
Looks reasonable — no blocking issues; comments above (mainly: confirm the
ci-k8s:2026-07-30tag resolves) are optional and self-resolving on the first CI run.The previous commit set IFS to a literal newline by writing IFS=" " inside a `- |` block scalar. The closing quote sits at column 0, which terminates the block — so the config no longer parsed: go-yaml load error in scanner (while scanning a simple key) at L70.C1-L89.C3: could not find expected ':' Same class as #443, in the very pipeline written to catch #443, and I pushed it: `woodpecker lint` printed the error in the same command as the push, and I read the push output rather than the lint output. Replaced with `while IFS= read -r` fed from a FILE rather than a pipe. That handles paths with spaces without needing a literal newline anywhere, and the file redirection keeps the loop in the current shell so `fail` still propagates — a pipe would have run it in a subshell and made a failing kustomize build exit 0, which is the silent-pass shape this step already shipped once with -printf. Verified both properties in the actual image rather than reasoning about them: a directory named "has space" iterates once intact, and a `fail=1` set inside the loop is still 1 after it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit9d3dc152c1b03afe6d810673664b64ea5886b764.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new pull-request-gated pipeline with four offline checks:kubectl kustomizebuild for every kustomization,woodpecker-cli linton all pipeline configs,shellcheckon standalone scripts underkubernetes/, and a grep guard against digest-pinned images in the secret-rotator manifests. I cross-checked the concrete claims in the PR body and inline comments against the actual tree rather than taking them on faith:find kubernetes -name kustomization.yaml | wc -lreturns exactly 60,find kubernetes -name '*.sh'returns exactly the 6 scripts implied, and the "thegrep -v kustomization.yamlfilter is load-bearing" claim checks out —kubernetes/infrastructure/openbao/rotation/kustomization.yamlreally does contain the literal stringimage:.*@sha256in its own explanatory comment, so without the filter this step would permanently fail on its own documentation. The shell in every step is careful about the exact class of bug it's guarding against (pipedfind/sedswallowing failures underset -e,while readfrom a pipe losingfailin a subshell, silent-empty-discovery reporting green) — I traced through theset -euinteraction withif/pipeline exit-status semantics for each step and didn't find a case where a real failure gets swallowed. No blocking defects found in the logic itself.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:40and:162— bothkustomize-buildandno-digest-pinspin togitea.renner.uno/gitea_admin/ci-k8s:2026-07-30, today's date. That tag only exists ifbuild-ci-images.yml'sbuild-k8sstep has already run successfully today — and that step only fires on changes toci/images/k8s/**or.woodpecker/build-ci-images.yml(or a manual trigger), none of which this PR touches. Contrast withdeploy-infrastructure.yml, which pinsci-tofu:2026-04-29— a tag from three months ago, long since confirmed to exist. I don't have registry access from this review sandbox to confirmci-k8s:2026-07-30was actually pushed; if it wasn't, both steps fail immediately on an image pull error, which would be a somewhat ironic first impression for a PR about catching CI breakage before merge. Worth confirming the tag is live (check today'sbuild-ci-imagesrun history) before relying on this passing, or falling back to the last known-goodci-k8stag if not..woodpecker/ci-manifests-test.yml:39-173— none of the four steps setfailure: ignore, and Woodpecker runs steps sequentially by default (nodepends_onhere). Ifkustomize-buildfails,woodpecker-lint,shellcheck, andno-digest-pinsnever execute, so a contributor sees only the first failing check per push and has to fix-and-repush to discover the next one. Since all four are independent, cheap, offline checks, running them to completion (viafailure: ignoreon each, or splitting into adepends_on: []fan-out likebuild-ci-images.ymldoes) would surface every failure in one run.🟢 Nits
.woodpecker/ci-manifests-test.yml:58— thecount -lt 50floor for "expected ~60 kustomizations" is a reasonable trip-wire today, but it's a hardcoded number with no pointer to update it; if a future legitimate consolidation drops the real count into the 50s this fails for the wrong reason. Not urgent given the current margin.Tests
This PR is a test/validation pipeline, so there's no separate test suite to ask for — the equivalent here is the manual verification the author describes (both branches of the lint-catches-#443 bug exercised, false-positive/false-negative behavior on existing warnings checked) plus the sanity checks I independently reran above (kustomization count, script count, the load-bearing grep filter). Those all check out against the current tree.
Recommendation
Looks reasonable — no blocking issues; the image-tag freshness point above is worth confirming before merge but isn't a confirmed defect.
🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit888aa60fe048bf8eb5f7a3cc6c1b099296906a54.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new PR-triggered pipeline with four parallel, offline checks overkubernetes/**and.woodpecker/**:kustomize-build(all kustomizations build),woodpecker-lint(pipeline configs compile),shellcheck(standalone scripts underkubernetes/), andno-digest-pins(guards the dated-tag convention in the openbao rotation dir). This is a pure CI-config addition (single new file, no other code touched). I checked every factual claim in the PR body and every non-trivial shell construct against the actual checkout — the counts (60 kustomizations, 6 shell scripts), the self-matchinggreppattern inrotation/kustomization.yaml, the absence of remote bases/helmCharts(confirming the kustomize step really is offline), and the busybox-find-lacks--printfclaim (confirmed: theci-k8sContainerfile'sapk addlist has nofindutils) all check out. I found no blocking defects.🟡 Suggestions
woodpecker-lintstep uses a CLI version well behind the deployed server (.woodpecker/ci-manifests-test.yml:103useswoodpeckerci/woodpecker-cli:v3.9.0-alpine, butkubernetes/apps/woodpecker/server.yaml:43andagent.yaml:37runv3.16.0). The PR body itself noteswoodpecker-cli lintcurrently emitsAdditional property timeout is not allowedwarnings on severaldeploy-*.ymlfiles and treats those as harmless pre-existing noise — but that symptom is also consistent with schema drift between a 7-minor-version-old lint tool and the actual v3.16.0 schema those files were written against, rather than genuine debt. A large version gap risks the reverse case too: syntax the real v3.16.0 server accepts could someday be flagged as a compile error by the old CLI (false-fail), or syntax it should reject could pass (false-pass). Consider pinning the lint image to (or near)v3.16.0for higher-fidelity linting.kustomize-builduseskubectl kustomize, notflux build, even though the sameci-k8simage already shipsflux(ci/images/k8s/Containerfile) — so it doesn't exercise Flux'spostBuild.substitutestep used bykubernetes/clusters/homelab/infrastructure.yaml. A manifest can pass this new check (kustomize builds cleanly, since${VAR}-style substitution placeholders are just inert text to kustomize) yet still fail at Flux reconcile time if the substitution step itself breaks. This doesn't need to block the PR, but given the file is explicit elsewhere about stating known gaps rather than implying coverage it doesn't have (the shellcheck-can't-see-embedded-shell note at.woodpecker/ci-manifests-test.yml:124-127), the same treatment here would be consistent — either a one-line callout, or switching toflux build kustomizationfor closer fidelity to production reconciliation.Tests
N/A — this PR is the test/validation infrastructure; there's nothing further to test it with. The author's verification methodology (running both steps against real repo content and a known-bad historical config, described in the PR body) is a reasonable substitute for automated tests on a CI-config file, and the specific claims I could check independently (kustomization/script counts, the self-matching grep, the offline-only property) all held up.
Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit7b5f244b7f44e7d1749d7a8168007fbc3e57c5b9.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new pull-request pipeline with four parallel, offline checks overkubernetes/**and.woodpecker/**:kubectl kustomizebuild of every kustomization,woodpecker-cli lintof every pipeline file,shellcheck -S erroron standalone shell scripts, and a grep guard against digest-pinned images in the secret-rotator directory. I independently re-verified the PR's factual claims against the checkout (60 kustomizations, 6 shell scripts, zero digest pins outside the self-documenting comment, and the woodpecker-cli version matching the deployed server) and they all hold. The shell logic in each step is careful and correct — no blocking issues found.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:62(kustomize-build) and:150(shellcheck): neither step setsset -o pipefail(and BusyBox/ash/dash don't support it anyway), sodirs=$(find kubernetes -name kustomization.yaml | sed ... | sort)takes its exit status fromsort, notfind. Afindfailure on a subtree (e.g. a permission error) that still letssed/sortexit 0 would silently drop entries from the list rather than failing the step. Thecount -lt 50floor catches total discovery breakage (the documented "empty output" bug) but not a partial loss of a few directories. Given the floor already exists and this is a low-likelihood failure mode in a repo you fully control, this is a nit-level robustness gap, not a blocker.woodpecker-lintstep (:109-129) is the one the PR body calls "the one that matters," and its correctness hinges on/bin/woodpecker-clibeing the right binary path and the-alpineimage actually containing/bin/shfor thecommands:block to run at all. I don't have registry/image-pull access in this sandbox to independently confirm that againstwoodpeckerci/woodpecker-cli:v3.16.0-alpine, so I'm relying on the PR body's claim of having probed the image directly. Worth a final sanity check that this step actually goes green in CI on this PR (rather than failing to even start the container) before merging, since a wrong binary path fails closed in a way that's easy to misattribute to a bug in the pipeline being linted.🟢 Nits
woodpecker-cli:v3.9.0-alpineandshellcheck-alpine:v0.10.0, while the actual file (after the follow-up "use latest stable versions" commit) pinsv3.16.0-alpineandv0.11.0. Not a code issue, just worth updating the description before merge so reviewers reading the body aren't checking against the wrong tags.Tests
This is a CI-config-only change, so there's no unit test to add in the usual sense — the pipeline's own
woodpecker-lintstep and the manual dry-runs described in the PR body (rc=0 on valid config, rc=1 reproducing the #443 bug) are the applicable verification, and that's consistent with how other pipeline-definition files in this repo (e.g.build-ci-images.yml) are validated. I re-ran the factual checks that the PR body claims (kustomization count, shell script count, digest-pin grep, server version) directly against the checkout and all matched.Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 Automated review — model
claude-sonnet-5(complexity: normal), commitfbfaf90923251aeed6b2b2ae400daac6210472da.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new pull-request pipeline with four independent, offline, secret-free checks:kustomize-build(all kustomizations underkubernetes/build),woodpecker-lint(all.woodpecker/*.ymlcompile),shellcheck(standalone scripts underkubernetes/, severityerror), andno-digest-pins(guards against digest-pinned rotator images). I checked every concrete claim in the PR body against the checkout — file counts, thebusybox findlimitation, theimage:.*@sha256self-match inrotation/kustomization.yaml, the deployedwoodpecker-serverversion, and the absence of remote kustomize bases — and all of them hold. No blocking issues found.🟡 Suggestions
woodpecker-cli:v3.9.0-alpine, but.woodpecker/ci-manifests-test.yml:118pinswoodpeckerci/woodpecker-cli:v3.16.0-alpine(which does correctly match the deployedwoodpecker-server:v3.16.0inkubernetes/apps/woodpecker/server.yaml:43, per the comment's stated rationale). The code is right; the PR body just wasn't updated after a later revision — worth a quick edit so future readers of the PR history aren't misled about which tag was actually shipped.ci-k8s:2026-07-30(line 51/205),woodpecker-cli:v3.16.0-alpine(line 124), andshellcheck-alpine:v0.11.0(line 155) all need to actually resolve in their registries for this pipeline to run at all; I have no registry/network access here to confirm. The PR body says these were probed directly, which I have no reason to doubt given how well the rest of the PR's claims checked out against the tree — just flagging that this is the one category of claim I couldn't independently verify.🟢 Nits
depends_on: []rationale comment (lines 46-49, 119-122, 150-153, 200-203) is duplicated verbatim across all four steps. Not wrong, just repetitive — could live once at the top ofsteps:if it's ever edited.Tests
This is CI infrastructure, not application code, so there's no unit-test layer to add — the "test" is the pipeline itself running green against the current tree, which the PR body says was done for each step. The design already builds in its own regression guard: both
kustomize-buildandshellcheckassert a minimum discovered-file count (kubernetes/ci-manifests-test.yml:69,:163) specifically so a brokenfind/sedpipeline fails loudly instead of silently passing with zero iterations — a real failure mode the PR body says bit the first draft of this file. That's a solid substitute for tests in a context where "tests" would just be "run the thing."Recommendation
Looks reasonable — no blocking issues; comments above are optional.
🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit8a580e341d61b194858584a484792cd5a5cc02af.Summary
Adds
.woodpecker/ci-manifests-test.yml, a newpull_requestpipeline with four offline, parallel steps (kustomize-build,woodpecker-lint,shellcheck,no-digest-pins) gatingkubernetes/**and.woodpecker/**, plus doc updates recording thatwoodpeckerci/woodpecker-clican't be enrolled in image-automation. The shell logic is careful (file-fedwhile read, discovery-count floors, quoting) and most of the PR body's factual claims check out against the checkout (60 kustomizations, 6 shell scripts, zero digest pins underopenbao/rotation/,v3.16.0matching the deployed server, exactly 4deploy-*.ymlfiles with thetimeoutproperty). No blocking issues confirmed, but one risk below is significant enough to verify before merge.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:51,218— unverified image tag, and the evidence available points the wrong way. Bothkustomize-buildandno-digest-pinspingitea.renner.uno/gitea_admin/ci-k8s:2026-07-30. Every other pipeline file in the repo (alert-monitor.yml:45,e2e.yml:33,e2e-test.yml:7,issue-fix.yml:52) still pinsci-k8s:2026-05-01. Per.woodpecker/build-ci-images.yml:55-62, thebuild-k8sstep only runs (and only mints a new dated tag) whenci/images/k8s/**orbuild-ci-images.ymlitself changes on a push — and this PR's diff touches neither. The one piece of same-day evidence,secret-rotator:2026-07-30(visible throughoutkubernetes/infrastructure/openbao/rotation/*.yaml), doesn't transfer: that image is built by an independent step (build-secret-rotator, gated onsecret-rotator/**), so its rebuild today (for #458) says nothing about whetherbuild-k8salso ran today. Ifci-k8s:2026-07-30was never pushed,kustomize-buildandno-digest-pinsfail immediately on image pull for every future PR touchingkubernetes/**or.woodpecker/**.I can't confirm this from the checkout (no registry/network access here), so I'm not marking it blocking — but there's a check that settles it directly: this PR's own diff touches
.woodpecker/ci-manifests-test.ymlandkubernetes/infrastructure/image-automation/README.md, both matching the new pipeline's own path filter, so its own CI run already attempted these two steps. Check that build's logs/status before merging.PR description drifted from the shipped versions. The description states verification was done against
woodpecker-cli:v3.9.0-alpineandshellcheck-alpine:v0.10.0, but the diff shipsv3.16.0-alpine(ci-manifests-test.yml:131) andv0.11.0(ci-manifests-test.yml:168). The specific empirical claims in the body ("verified both directions... in the image the step actually uses", the no-/bin/shprobing) were run against different tags than what's actually in the pipeline. Probably fine (both are point-bumps within the same alpine variant), but worth a quick re-check thatv3.16.0-alpinestill ships/bin/shand thatlintbehavior is unchanged, since the description's evidence no longer matches the code as diffed.🟢 Nits
.woodpecker/ci-manifests-test.yml:137—woodpecker-cli lint "$f"lints one file at a time in a loop; this matches the documented CLI usage as far as I can tell, but I couldn't execute it in this sandbox (no container runtime/network available) to confirm the single-file invocation and exit-code behavior first-hand.Tests
N/A in the traditional sense — this PR is the test/gate. Its own effectiveness is empirically argued in the description (verified against the #443 regression and the current tree state), and the numeric claims I could check against the checkout (kustomization count, script count, absence of digest pins, server/CLI version match) all held up. The main untested-in-this-review path is the actual CI run of the new pipeline itself (see the image-tag point above).
Recommendation
Looks reasonable — no confirmed blocking issues; the
ci-k8s:2026-07-30tag risk above is worth confirming against this PR's own CI run before merging.Both 🟡 points and the nit are addressed in
51153788.1.
ci-k8s:2026-07-30may never have been built — good catch on the mechanism:build-k8sonly mints a dated tag whenci/images/k8s/**orbuild-ci-images.ymlchanges on a push, this PR touches neither, and you were right thatsecret-rotator:2026-07-30doesn't transfer (separately-gated step). I checked rather than assumed. The registry API is closed to this token (read:packagenot granted), so the evidence is the run you pointed at: pipeline 3178 on8a580e34—kustomize-buildandno-digest-pinsboth green on that exact tag, with 60 realoklines. An unpullable image cannot produce that. Tag exists; no change needed.2.
v3.16.0-alpineunverified — same run:woodpecker-lintemitted 18oklines, one per.woodpecker/*.yml, which is only reachable if the image has a working/bin/shand/bin/woodpecker-cli. So the single-file invocation and exit-code behaviour are confirmed against the shipped tag.But this one exposed something I could not verify, and your earlier hypothesis was the reason I looked. The comment asserted that several
deploy-*.ymlfiles emitAdditional property timeout is not allowed. That was observed under v3.9.0, the tag this step used to pin. Whether v3.16.0 still emits it is unknown — and a green run cannot settle it, because the loop sends lint output to/tmp/outand prints it only on failure, so warnings are invisible precisely when the step passes. Your read (schema drift in a 7-minor-old CLI rather than real debt in those files) is plausible and untested. I dropped the claim rather than restating it; the comment now documents only the exit-code contract the gate actually rests on.3. Nit — duplicated
depends_onrationale — hoisted to a single note abovesteps:, with a warning that the key itself must stay on every step (it's per-step; removing it from one silently re-serialises that step).PR description — updated. You were right that the body still quoted
v3.9.0-alpine/v0.10.0as shipped, so the probing evidence referred to different tags than the diff. Corrected in place with an explicit note recording the bump rather than quietly rewriting it.Also worth recording, since it's the substantive change in
8a580e34and it went the opposite way from what I proposed: I had intended to enrollwoodpecker-cliin image-automation so the pin would stop being manual, by wideningupdate.pathto./. That cannot work. Image-automation only patches YAML that is a Kubernetes manifest — fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the YAML manifest must haveapiVersionandkindfields." A Woodpecker pipeline file has neither, so a marker here is silently ignored. That's a property of the file, not ofupdate.path, so widening buys nothing and would drag 197 unrelated ansible/esphome YAMLs into the walk. The pin stays manual by necessity; drift is now covered by the update-audit agent, which is told to flag this image whenever it trails the deployed server.🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit51153788358ed862125c75521709622a61b90660.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new PR-gated pipeline with four offline, parallel checks (kustomize-build,woodpecker-lint,shellcheck,no-digest-pins), plus documentation updates inci/agents/update-audit/prompt.mdandkubernetes/infrastructure/image-automation/README.mdexplaining why thewoodpecker-cliimage pin can't be enrolled in image-automation. This is round 2 (commit5115378already addressed a prior review's two 🟡 points and one nit). I verified the concrete claims against the checkout — kustomization count (60), shell script count (6), theno-digest-pinsgrep behavior against the actual rotation directory, thekubectl/find/sed toolchain in theci-k8simage's Containerfile, the.woodpeckerfile count (18), and that none of the 60 kustomizations reference remote bases (so the "offline" claim holds). Everything checks out; no blocking issues.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:52-97(kustomize-build) and:170-198(shellcheck) each hand-roll the same "discover withfind, guard against empty discovery, feed a file intowhile read" pattern. Not asking for an extraction — two instances doesn't justify one, and the file-vs-pipe subshell reasoning is genuinely two different traps worth spelling out locally — but if a third check with the same shape gets added later, that's the point to factor it into a shared step or script underscripts/ci/..woodpecker/ci-manifests-test.yml:136claims wideningupdate.pathwould "drag 197 unrelated ansible/esphome YAMLs into the walk." I counted 180.yml/.yamlfiles directly underansible/+esphome/in this checkout — in the right ballpark and the underlying argument (widening the path pulls in a lot of unrelated files) holds regardless, but the specific figure doesn't reproduce exactly from what's countable here. Worth a quick re-check of however "197" was originally derived (e.g. did it include other top-level YAML too) before treating it as load-bearing precision elsewhere.Tests
This PR is the test infrastructure — there's no meaningful separate test to add. The pipeline's own correctness was validated by an actual CI run referenced in the round-2 commit message (pipeline 3178 on
8a580e34:kustomize-buildandno-digest-pinsgreen againstci-k8s:2026-07-30,woodpecker-lintemitting 18oklines underv3.16.0-alpine), which is reasonable evidence for image/tag availability that isn't independently checkable from this static checkout alone. The failure-mode self-checks built into two of the four steps (thecount -lt 50/count -lt 1guards against silent no-op discovery) are themselves a good defense against the exact regression the PR's own commit history shows this pipeline suffered once already.Recommendation
Looks reasonable — no blocking issues; comments above are optional.
Both points addressed in
13798134.The "197" figure — you're right, and it was wrong twice over. Recounted here:
ansible/has 179,esphome/has 1 → 180, matching your count exactly.Where 197 came from: I folded
.woodpecker/into the total. That's wrong on both counts — those files are neither ansible nor esphome, and they're precisely the directory you'd want in the walk, so counting them as "unrelated" inverts the argument the sentence is making. The composite is also 198, not 197, today: the original count ran againstmaster, where.woodpecker/still had 17 files becauseci-manifests-test.ymldidn't exist yet.Fixed in both places that cite it (
ci-manifests-test.ymland the image-automation README), now with the derivation shown (179 + 1) so it's re-checkable rather than a bare number. The conclusion is unaffected — wideningupdate.pathis pointless because Flux can't patch non-Kubernetes YAML at all, and the file count was only ever the secondary reason — but you were right to flag load-bearing precision that doesn't reproduce.Duplicated discovery pattern — agreed, and no extraction now. I recorded your rule-of-three where the next person will hit it: a note above
steps:saying two copies is deliberate (the two traps being commented around — BusyBoxfindlacking-printf, and a piped loop running in a subshell wherefailcan't propagate — are genuinely different and worth spelling out locally), and that a third instance is the signal to factor it intoscripts/ci/. Worth adding: anything moved there picks up unit tests viaci-scripts-test.yml, which a shell block inlined in a pipeline file can never have — so the third-instance threshold buys more than deduplication.The file count was meaningless and would be wrong tomorrow. Correcting it from 197 to 180 last commit was the wrong fix: a number that has to be re-derived every time the tree changes does not belong in a comment at all. It cost two review rounds — one to flag that 197 did not reproduce, one to correct it — and neither round was about anything that mattered, because the count was never the argument. Widening update.path is pointless because Flux cannot patch non-Kubernetes YAML at all; the size of the walk was only ever a secondary aside. Swept every snapshot figure I had introduced, not just the one flagged: * "180 unrelated ansible/esphome YAMLs (179 + 1)" -> "the whole ansible/ and esphome/ trees", in both the pipeline comment and the README. * "builds 60 kustomizations" -> "builds every kustomization in the tree". * "verified green across all 60 kustomizations" -> "across every kustomization", keeping the as-of-writing hedge that made it a historical claim rather than a live one. * "expected ~60" in the discovery-floor failure message -> "floor is 50", which cites the actual guard on the line above instead of a tree size that drifts away from it. The message already prints the real $count. * "scratch-based (19 MB, 2 layers, no shell)" -> "ships no shell", plus what the reader actually needs: a step on the plain tag fails to START rather than failing a check, which is the confusing symptom. The byte size would have gone stale on the next upstream release. The floor of 50 itself stays — it is a deliberately loose guard against discovery breaking, not a description of the tree, and it is asserted in code where a wrong value fails loudly instead of merely misinforming. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>🤖 Automated review — model
claude-sonnet-5(complexity: normal), commit13798134596471a34b314f538cc8c1f8bb6a74ea.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new offline, path-filtered pipeline that runs four independent checks (kustomize-build,woodpecker-lint,shellcheck,no-digest-pins) on PRs touchingkubernetes/**or.woodpecker/**, plus matching updates to the update-audit agent prompt and the image-automation README documenting why thewoodpecker-clilint image can't be Flux-managed. This is an unusually well self-verified change — I independently re-checked most of the PR's factual claims against the checkout and they all held up (see below). No blocking issues found.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:56,221— the newci-k8s:2026-07-30tag is newer than theci-k8s:2026-05-01pin used everywhere else in the repo (e2e-test.yml,e2e.yml,alert-monitor.yml,issue-fix.yml). That's not a bug — each pipeline pins independently and the PR body documents this image was specifically verified — but it does mean the repo now carries two different dated pins for the same image with no stated reason for the split. Worth a one-line note (or a follow-up) on whether the older pins should be bumped too, so this doesn't read as an oversight to the next reader.Tests
This is CI configuration, not application code, so "tests" here means the checks are self-validating and the PR body documents manual verification (green run on the current tree, and the #443 defect reproduced as rc=1 against the same image). I independently re-verified several of the underlying factual claims against the checkout rather than just trusting the prose:
find kubernetes -name kustomization.yaml→ exactly 60 results, consistent with thecount -lt 50floor inkustomize-build.*.shfiles underkubernetes/, consistent with thecount -lt 1guard inshellcheck.https:///git@) orhelmCharts:, sokustomize-buildreally is offline as claimed.kubernetes/infrastructure/openbao/rotation/*.yaml: onlykustomization.yamlmatchesimage:.*@sha256(it's a comment documenting the rule), and thegrep -v kustomization.yamlfilter inno-digest-pinscorrectly excludes it — confirmed no false positive today.kubernetes/apps/woodpecker/server.yaml:43deployswoodpecker-server:v3.16.0, matching thewoodpecker-cli:v3.16.0-alpinepin — the "CLI matches deployed server" claim in both the pipeline comment and the prompt.md addition is accurate as of this commit.kubernetes/infrastructure/image-automation/update-automation.yaml:31—update.path: ./kubernetestoday, confirming the README's claim that widening it to./would pull in unrelated trees.ansible/**/*.{yml,yaml}= 179 files,esphome/**/*.{yml,yaml}= 1 file → 180 total, matching the corrected count in both the README and prompt.md additions (and matching the most recent commit on this branch, which fixed this exact number from 197).I did not execute
kubectl kustomize,woodpecker-cli lint, orshellcheckmyself (not available in this environment), so the runtime behavior of the three container-image-dependent steps rests on the PR author's stated verification rather than something I reproduced.Recommendation
Looks reasonable — no blocking issues; the suggestion above is optional.
Correcting 197 → 180 in the previous commit was the wrong fix.
ba052332deletes the number instead.A count that has to be re-derived every time the tree changes doesn't belong in a comment at all — it's stale the moment anything is added, and it was never the argument here. Widening
update.pathis pointless because Flux can't patch non-Kubernetes YAML at all; the size of the walk was a secondary aside. That aside consumed two review rounds (one to flag that 197 didn't reproduce, one to correct it), neither of which touched anything load-bearing.Swept every snapshot figure I'd introduced, not only the flagged one:
180 unrelated ansible/esphome YAMLs (179 + 1)the whole ansible/ and esphome/ trees(comment + README)builds 60 kustomizationsbuilds every kustomization in the treeverified green across all 60 kustomizationsacross every kustomization, keeping the as-of-writing hedgeexpected ~60in the floor failure messagefloor is 50— cites the actual guard on the line above, not a tree size that drifts away from itscratch-based (19 MB, 2 layers, no shell)ships no shell, plus the symptom that actually helps: a step on the plain tag fails to start rather than failing a checkThe
-lt 50floor stays. That one is a deliberately loose guard asserted in code, where a wrong value fails the build loudly — as opposed to a number in a comment, which just misinforms quietly. That distinction is the general rule I should have applied from the start: if a count matters, assert it where drift breaks something; if it doesn't, don't write it down.Picking up the review for
13798134(superseded by two commits since, but its 🟡 point was still open).ci-k8stag split — you're right that it reads as an oversight, because it partly is: the other four pins are simply stale.e2e-test.yml:7,e2e.yml:33,alert-monitor.yml:45andissue-fix.yml:52all sit on the older tag, and per the latest-stable rule all five should be on the newest.I did not bump them here, deliberately. None of those four are triggered by this PR — they're cron- or event-driven and exercise different tooling inside the image — so bumping them in this branch would ship four untested changes under cover of a green run that never touched them. That's the exact shape of the digest decay in #458, where the breakage sat unnoticed for months precisely because the affected jobs didn't run. Each wants its own verification.
Tracked as #461, with a comment above
kustomize-buildexplaining the split where a reader meets it (b1f5d6dc).Two notes on your verification section, since some of it no longer applies:
180figure you re-derived and confirmed has since been deleted rather than corrected (ba052332). It was a snapshot that would be wrong as soon as anything is added to those trees, and it was never the argument — Flux can't patch non-Kubernetes YAML at all, so the size of the walk was a secondary aside. Same sweep removed60 kustomizations,19 MB, 2 layers, andexpected ~60(nowfloor is 50, citing the actual guard). Worth knowing that re-deriving those numbers is wasted effort on this file going forward — if a count matters it should be asserted in code where drift fails the build, as the-lt 50floor is.60/6/ no-remote-bases / self-matching-grep checks all still hold and were the useful half — those are properties, not snapshots.🤖 Automated review — model
claude-sonnet-5(complexity: normal), commitb1f5d6dcbc5fa42e33457904edcbf87ee18b91ea.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new PR-gated pipeline with four parallel, offline checks overkubernetes/**and.woodpecker/**(kustomize build, woodpecker-cli lint, shellcheck aterrorseverity, and a digest-pin guard for the openbao rotators), plus supporting doc updates inci/agents/update-audit/prompt.mdandkubernetes/infrastructure/image-automation/README.md. I checked the concrete, checkable claims against the repo checkout — kustomization count (60, floor is 50 ✓), shell-script count (6, matches "all 6 are clean" ✓), the BusyBoxfindinci-k8s's base image (alpine:3.21 with nofindutilsinstalled, so-printfreally would fail ✓), therotation/kustomization.yamlself-match for the@sha256grep (confirmed, the filter is load-bearing ✓), thedepends_on: []parallel-step convention (matches existing usage inbuild-ci-images.yml✓), and that no kustomization in the tree pulls remote bases or Helm charts (sokubectl kustomizereally is offline for all 60 ✓). Everything checks out; I found no blocking issues.🟡 Suggestions
ci-manifests-test.yml:196— theshellcheckstep's discovery floor iscount -lt 1(i.e., "at least one script found"), far weaker thankustomize-build's floor of 50. That's proportionate today (only 6 scripts exist), but unlike the kustomize floor it wouldn't catch a partial-discovery regression (e.g. a futurefindfilter that only matches half the tree) — it would only catch total discovery failure. Not worth blocking on given the small, slow-changing script count, but worth keeping in mind if thekubernetes/tree grows a lot more shell.ci-manifests-test.yml:69,161,188,234— the three third-party image pins (ci-k8s:2026-07-30freshly built,woodpecker-cli:v3.16.0-alpine,shellcheck-alpine:v0.11.0) and the/bin/woodpecker-clibinary path / working/bin/shclaim are asserted in the comments as empirically probed, but I have no way to pull images or re-run the pipeline from this checkout to confirm them independently. If CI is green on this PR's own run, that's the real confirmation; if it isn't, start there.Tests
This is CI configuration for a homelab GitOps repo — there's no unit-test harness for Woodpecker pipeline YAML itself, and the PR description documents that each step was manually verified both for the pass case (current tree) and the specific failure case it targets (the
#443YAML-scalar bug for lint,-printfsilent-no-op and subshell-swallowed failures for the discovery/loop pattern, the rotation-dir self-match for the digest guard). That's the appropriate level of verification for this kind of change; I independently re-confirmed the parts of it that are checkable from the static checkout (counts, image base packages, self-match), and they hold up. Nothing further to ask for here.Recommendation
Looks reasonable — no blocking issues; comments above are optional.
Both points addressed in
39dc23a9.Image pins /
/bin/sh//bin/woodpecker-cliunverifiable from a static checkout — you called the right test: CI green on this PR's own run. It is. Pipeline 3185 onb1f5d6dc, all four steps green, and each did real work rather than passing vacuously:kustomize-build→ 60oklines (soci-k8s:2026-07-30pulls)woodpecker-lint→ 18oklines, one per.woodpecker/*.yml(sov3.16.0-alpinehas a working/bin/shand/bin/woodpecker-cli— the loop couldn't produce those lines otherwise)shellcheck→ 6oklines (soshellcheck-alpine:v0.11.0pulls)shellcheck floor of 1 vs kustomize's 50 — real asymmetry, kept deliberately, now with the reasoning in the file rather than left to look like an oversight.
Your floor of 50 works because it's loose relative to the tree it guards: that slack is what lets it catch gross partial loss without tripping on churn. The shell-script set is small enough that any floor tight enough to catch partial loss would sit inside normal churn — delete one script and CI fails spuriously, and the fix a hurried person reaches for is lowering the floor. That's how a guard becomes decoration.
Put differently: a floor that has to be edited whenever the tree changes isn't a guard, it's a snapshot with an exit code — and this file just had all its snapshot numbers stripped out for exactly that reason. So the floor stays at 1, catching the failure that actually happened here (discovery silently returning nothing, step green having checked no files), with a note to revisit if
kubernetes/grows a lot more shell.Consistent with that, the 60/18/6 counts above are cited as run-time evidence in the commit message — not written into the file.
🤖 Automated review — model
CLI default(complexity: complex), commit39dc23a9993cf97de799c4bc13a7ba85b0011e7a.Summary
Adds
.woodpecker/ci-manifests-test.yml, a new pull-request pipeline that runs four offline, parallel checks (kustomize-build,woodpecker-lint,shellcheck,no-digest-pins) againstkubernetes/**and.woodpecker/**, plus matching updates to the update-audit agent prompt and the image-automation README documenting that the newwoodpecker-clipin can't be enrolled in Flux image-automation. I checked every factual claim in the diff against the actual checkout (kustomization/script counts, thekustomization.yamlself-match trap for the digest grep, theci-k8stag split across other pipelines, the deployedwoodpecker-serverversion and its3.xauto-bump range, absence of remote bases/helmChartsthat would break offlinekubectl kustomize, and thedepends_on: []parallel-step convention already used inbuild-ci-images.yml) and all of them check out. No blocking issues.🟡 Suggestions
.woodpecker/ci-manifests-test.yml:80and:192—dirs=$(find kubernetes -name kustomization.yaml | sed '...' | sort)andfiles=$(find kubernetes -name '*.sh' | sort)run underset -euwithoutpipefail(ash doesn't support it anyway). Iffinditself ever errored (permission issue, path typo), the command substitution's exit status issort's, notfind's, so the error would be swallowed rather than trippingset -e— the discovery-floor check would still likely catch a resulting empty/short list, but a partial silent failure (e.g.finderroring on one subtree but still emitting output for others) wouldn't be distinguished from a genuinely smaller-but-valid result. Given the step's own stated philosophy is "refusing to report success" on broken discovery, this is a small gap in that guarantee — not worth blocking on since the floor checks already catch the realistic failure modes, but worth a one-line comment if you want the reasoning as airtight as the rest of the file.event: pull_request+path:combination (ci-manifests-test.yml:26-29) has no other precedent in this repo's.woodpecker/*.yml(existingpathfilters are all onpush). The PR's own last commit message cites pipeline 3185 as having fired and passed on this exact combination, which is good evidence it works — just flagging that it's a new pattern here in case that run isn't easily re-checkable by other reviewers.Tests
This PR is the test infrastructure — there's no separate test suite to add. The author states pipeline 3185 ran all four steps green against the real counts (60 kustomizations, 18 pipeline configs, 6 shell scripts), which I independently confirmed match the current tree (
find kubernetes -name kustomization.yaml→ 60,find kubernetes -name '*.sh'→ 6). That's reasonable verification for a CI-definition change. Theci/agents/update-audit/prompt.mdandREADME.mdchanges are documentation/prompt text with no automated check, but I read them for internal consistency (item numbering, image list, drift-tracking cross-references) and found no contradictions with the rest of those files.Recommendation
Looks reasonable — no blocking issues; comments above are optional.