Addresses the 14aa6415 review.
- pick-model.sh: the picker relied on `--allowedTools ""` to be tool-less, but an
empirical check showed an empty allowlist does NOT block tools (the Write tool
still ran) -- it is treated as "unset" and the default toolset stays live. Add
`--permission-mode plan`, which IS verified to block Write while still producing
the selection JSON. --allowedTools "" + the Read deny-list remain as additional
layers. Corrected the comments in pick-model.sh and the picker settings.json
that claimed the empty allowlist made it tool-less.
- .woodpecker/ci-scripts-test.yml: new pull_request-triggered pipeline (homelab
only) that runs scripts/ci/test-model-selection.sh, so a PR weakening the
injection gate fails checks immediately instead of only on the next scheduled
e2e cron (closes the coverage-timing gap the review flagged).
- Pin a recorded real /v1/models response (scripts/ci/testdata/v1-models-sample.json)
and assert fetch-models.sh maps it correctly (opus effort=true, haiku
effort=false), documenting the verified response shape.
Deferred with rationale: hard-failing a pipeline when the degraded-schema warning
fires -- the warning already reaches the job log (curl/jq stderr is no longer
suppressed), and hard-failing on a benign API schema addition would break all
model selection; graceful degradation to name+date is the deliberate choice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the a7890e0a review.
- pr-review.yml: strip REVIEW_TOKEN from the valid-model.sh call too (it runs a
script from $HOMELAB, branch-supplied for a homelab PR), and guard it with
`|| true` so a missing script can't kill the step -- restoring the stated "a
run never dies on selection" invariant. For a complex/security-sensitive
review, ignore the (injectable) triage pick and run on the CLI default
(strongest configured) model.
- valid-model.sh: reject a leading dash at the shape check (making the header
accurate) so an option-like value is refused even without a catalog.
- fetch-models.sh: --retry 2 --retry-connrefused so a transient blip doesn't
drop the run to the default; add a FETCH_MODELS_BODY_FILE test seam.
- pick-model.sh: pass --settings (new ci/agents/model-picker/settings.json,
Read deny-list) as belt-and-suspenders behind --allowedTools "".
- test-model-selection.sh: add fetch-models.sh mapping tests (well-formed body
-> non-null capability columns; degraded body -> loud warning) via the seam,
and wire the whole suite into .woodpecker/e2e.yml so it runs in CI.
- Condense the duplicated model-selection comment in the four fixed jobs to a
one-line pointer to pick-model.sh.
- PR description: state the cost / no-fixed-ceiling property change.
Deferred with rationale (in the PR thread): selected-model in the four crons'
posted artifacts (already logged per run), pagination past limit=1000 (API max;
has_more warns), and unit-testing pick-model.sh's JSON extraction (its output
always passes through the tested valid-model.sh gate, and the picker prompt is
developer-controlled, not attacker input).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the remaining pr-review items.
- Extract the untrusted-model validation out of the pipeline YAML into
scripts/ci/valid-model.sh (single safe-token + catalog-membership gate) and
reuse it from both pick-model.sh and pr-review.yml, so the security-critical
logic lives in one tested place. Add scripts/ci/test-model-selection.sh, an
offline fixture test (no network / claude / Woodpecker) that pins the
behaviour: valid ids pass; hallucinated, empty, space-, newline-, leading-
dash-, and metachar-injected values are rejected; empty catalog rejects all.
This is the regression test for the flag-injection fix.
- pr-review triage cost: hand the triage the COMPLETE changed-file list
(files.txt) plus a diff capped at 4000 lines (diff.triage.patch) instead of
the full up-to-20k review diff, so classifying a PR on the default model stays
cheap. stats.txt still carries the true totals; triage-prompt.md updated.
- Doc nits: note the new helpers are POSIX `sh` (not `bash` like the older
scripts/ci/*.sh), and that fetch-models.sh's json mode is a debug affordance.
Deferred with rationale (also in the PR thread):
- Model id in the four cron jobs' posted artifacts: selection (including the
fallback-to-default) is already logged per run in the CI output, the
operational surface for this; threading it through four different posted
formats is a separate observability follow-up.
- --settings sandbox on the picker: it is granted no tools (--allowedTools ""),
so there is nothing for a Read-deny sandbox to constrain -- the other agents'
sandboxes exist to scope tools they DO have.
- Pagination past limit=1000: that is the API's max page size; >1000 models is
not a real scenario, and the has_more warning turns even that into a visible
log line rather than silent truncation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pr-review triage agent reads the attacker-controlled PR diff and writes
triage.json; its Write tool is not path-scoped, so a prompt-injected diff can
make it emit any `model` string. The catalog validation used
`grep -qxF -- "$MODEL"`, and grep -F treats a MULTI-LINE operand as
newline-separated patterns: a value like
"claude-sonnet-5\n--dangerously-skip-permissions" matches on its first line,
passes validation, and -- via the unquoted `${MODEL:+--model $MODEL}` expansion
-- word-splits into an extra argv entry, injecting a CLI flag into the review
agent's own `claude` invocation (defeating its --allowedTools allowlist and Read
sandbox, with CLAUDE_CODE_OAUTH_TOKEN in that process's env).
Removing map_model() (which returned one of three literal ids) is what opened
this; restore the invariant with a single-token guard before the expansion:
`case "$MODEL" in ""|*[!A-Za-z0-9._-]*) MODEL= ;; esac`. Verified it turns the
newline/space/hallucinated payloads into a plain `claude -p` (CLI default) while
still passing real ids. Mirrored the guard in pick-model.sh as defence in depth
(its grep -oE already keeps CHOICE single-line, so it was not exploitable).
Also from the review:
- pr-review.yml: correct the HOMELAB comment -- the fetch-models.sh call is
env -u REVIEW_TOKEN'd, so CLAUDE_CODE_OAUTH_TOKEN (not REVIEW_TOKEN) is what
branch-supplied shell there can see.
- pick-model.sh / triage-prompt.md: stop asking the picker to weigh "cost" the
catalog does not expose; reason from the visible capability signals (context /
max_output / effort|thinking|code_exec) as the scale proxy instead.
- fetch-models.sh: drop 2>/dev/null on curl so its -S error reaches the log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fetch-models.sh: request ?limit=1000 so older/cheaper models are not dropped
by the API's 20-item default page; warn on has_more. Add a --table mode so
the catalog format lives in ONE place (pick-model.sh and pr-review.yml now
consume it instead of each re-deriving the jq table). Clean up the HTTP-000
double-print and surface jq parse errors instead of hiding them.
- pick-model.sh: run the picker with GITEA_TOKEN stripped from its env and
without acceptEdits (it is granted no tools); consume fetch-models.sh --table.
- pr-review.yml: print model `CLI default` instead of an empty backtick pair on
the fallback path; document that the fetch-models.sh call runs branch-supplied
shell (not a new exposure -- Woodpecker already runs the pipeline YAML from
the PR branch).
- triage-prompt.md: read stats + the changed-file list first and pull the full
diff only when needed, so the triage (now on a capable default model) does not
read a 20k-line diff just to classify a trivial PR.
Verified against the live catalog: all 11 currently-listed ids run under
`claude --model`, and the capability fields (context/effort/thinking/...) are
present in the OAuth /v1/models response -- so the reviewer's blocking concern
(a listed-but-CLI-unusable id) has no live trigger, and the fallback-to-default
path still covers it if that ever changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pr-review, update-audit, security-scan, alert-monitor and issue-fix
pipelines each pinned a dated model id (--model claude-sonnet-5, plus
pr-review's haiku/sonnet/opus map_model allowlist). Those go stale every
time a new model ships and bake in assumptions about which model families
exist, so a newly released model could never be chosen without a code edit.
Replace all of it with selection driven only by what is available right now:
- scripts/ci/fetch-models.sh queries the catalog the CLI itself uses
(Anthropic GET /v1/models, via the existing CLAUDE_CODE_OAUTH_TOKEN and the
required anthropic-beta: oauth-2025-04-20 header) and emits each model with
its objective capability signals -- context, max_output, effort, thinking,
code_exec, release date -- with no id patterns and no family names.
- scripts/ci/pick-model.sh lets a triage agent choose the best-fit id from
that catalog for a one-line task description, weighing every listed model
on its merits. pr-review's existing per-PR triage agent now does the same,
reading the live catalog instead of emitting a fixed haiku/sonnet/opus tier.
No model id -- and no model family -- is baked into any pipeline. A model
released tomorrow, of any family, becomes eligible the day the API lists it.
The bootstrap selection call runs on the CLI default model, and if the catalog
is ever unreachable --model is simply omitted (CLI default) rather than falling
back to a named model, so a run never dies over model selection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ci-claude image installed a fixed, pinned Claude Code version and every
consumer pipeline pinned a stale dated image tag. Two problems:
- The npm install layer changed on every build (install-time mtimes + the
npm cache), so it never deduped in the Gitea registry — the single largest
orphan pattern behind #336 (~260 MB orphaned per no-op rebuild).
- A frozen version silently rots: old Claude Code releases eventually stop
working against the API.
Containerfile: default to @latest (npm auto-selects the musl native binary on
this alpine base) and make the install layer byte-stable — clean the npm cache
and normalize install-time mtimes on the installed tree and the directories the
RUN touches. Two independent --no-cache builds now produce a byte-identical npm
layer, so a rebuild of the same version dedups instead of orphaning a copy.
New ci-claude-refresh.yml owns the image lifecycle: weekly (and on-demand) it
resolves the latest published version, and only when it differs from what the
consumers pin does it build ci-claude:<version> (+:latest) and open a PR
rewiring the 5 consumer pipelines. Version tags make "did it change?" a
race-free string compare and keep the cron from opening no-op PRs. Nothing
auto-merges. build-ci-images.yml no longer builds ci-claude so the two never
fight over tags.
Requires owner setup before it does anything (documented in the pipeline
header): a dedicated ci-claude-bump-bot identity (write:repository only) as the
ci_claude_bump_bot_token secret, and a cron named "ci-claude-refresh".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The triage + review agents run with REVIEW_TOKEN and CLAUDE_CODE_OAUTH_TOKEN in
their environment, process attacker-controlled PR title/body/diff, and their
output is posted verbatim as a public PR comment. Without a Read sandbox a
prompt-injected diff could read /proc/self/environ (the tokens) or .git/config
(Woodpecker's clone credential) and get it published. Every other agent
pipeline already denies this via --settings; pr-review had none.
Add ci/agents/pr-review/settings.json (mirrors issue-fix/update-audit, gitea
#197) denying Read on /proc,/root,/home,/etc,.git, and pass --settings
$PROMPTS/settings.json to both claude invocations (resolves alongside the
prompts, so it works for the repos that fetch prompts from the homelab clone).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a pull_request event Woodpecker's built-in clone already checks the repo out
at the PR head in the workspace, so the agents read that directly for context.
Restores full-context review (the prior commit over-corrected to diff-only)
while keeping the manual, token-bearing clone gone. Prompts point at the
working-directory checkout instead of ./work/target.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The clone existed only to hand review_bot_token to git, which persisted it in
work/target/.git/config — a file the review agent (Read/Glob/Grep, output
posted publicly) could read. Nothing needed the checkout that the API-fetched
diff + metadata don't already provide, so drop the clone entirely: no token on
disk, no work/target, no cleanup. The PR + diff are still read over the API
with ephemeral curl Authorization headers (never written to disk).
Prompts updated to match: the agents review the diff alone (no ./work/target),
and are told to flag, not guess, when a judgement needs code they can't see.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both agent steps hold Gitea write credentials in their environment
(issue-fix holds a push token too), but neither passed --settings, so
their Read tool was unrestricted — unlike update-audit, which got a
Read sandbox for exactly this reason (gitea issue #197).
That left a complete exfil path: Read /proc/self/environ yields the
token, Write puts it in the agent's output file, and the pipeline posts
that output verbatim as a Gitea issue body / PR body on a PUBLIC repo.
Add the same deny-list used by update-audit (/proc, /root, /home, /etc,
.git) to both agents and wire --settings into each invocation. The repo
checkout, .ci-alert-monitor/ and /tmp/alert-fix/ stay readable, which is
all either agent needs.
This is defense-in-depth only. The root cause — CI pipelines holding an
admin-owned token at all — is tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New Woodpecker cron pipeline that mirrors the alert-monitor / issue-fix
fleet pattern (cron -> headless `claude` -> Gitea API), but reviews open
pull requests instead of triaging alerts:
- .woodpecker/pr-review.yml: hourly `pr-review` cron. Discovers open PRs
across k.renner/{homelab,shelly-manager,authentik-samba-bridge,
homeassistant}, skips drafts/WIP and PRs already reviewed at their
current head sha (dedup marker), caps at 5/run. For each: clones the PR
head, has a cheap Haiku triage agent pick the model by complexity
(haiku/sonnet/opus; security-sensitive paths force opus), runs the
review with that model, and posts a detailed report as a Gitea PR
review (COMMENT, or REQUEST_CHANGES on a real defect — never APPROVE).
- ci/agents/pr-review/triage-prompt.md: model-selection rubric.
- ci/agents/pr-review/prompt.md: the reviewer's task + report format.
Runs under a dedicated `review-bot` Gitea account (secret
review_bot_token) so reviews are visibly bot-authored and independently
revocable. Static review only — no cluster access; agent tools are
Read/Glob/Grep/Write with no Bash.
Not yet active: needs the review-bot account + token, the review_bot_token
Woodpecker secret, and the `pr-review` cron registered (see file header).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The live 15-min snapshot structurally misses anything that fires and
resolves between ticks: on 2026-07-16 a 17-minute KubeHpaMaxedOut
(authentik) and last evening's four-alert home-assistant incident were
invisible to the monitor (owner noticed the blips; monitor did not).
The fetch step now also queries Prometheus's ALERTS history over a
35-minute window (~2 ticks + jitter). The dedup step synthesizes
pseudo-alerts for series not currently firing at all — checked against
the PRE-filter snapshot so live-but-null-routed alerts don't reappear
as blips — with a stable sha256-derived fingerprint so one noise
verdict suppresses a flappy alert permanently. Blips then flow through
the same title/noise dedup and triage as live alerts; the prompt
explains their semantics (self-resolved; judge the underlying cause;
null-routed per the repo route config = noise).
Verified against live data: currently-firing KubeCPUOvercommit
excluded, injected KubeHpaMaxedOut blip synthesized correctly;
sha256sum confirmed present in ci-claude; lookback query tested via
the service proxy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
The #259 lesson, both directions: the agent rightly refused
unauthenticated instructions embedded in the issue body — and then had
no channel for the owner's authentic reply 18 minutes later, so a
human ended up doing the sweep the agent should have owned.
- Pick step: an issue with an alert-fix:assessed marker becomes
eligible again when a TRUSTED_AUTHORS comment is NEWER than the
latest marker (timestamps compared in jq — busybox ash test has no
string-<). Owner replies on the issue now drive the loop.
- Trusted comments are appended to the agent's issue.md as 'Verified
owner guidance'. Trust anchor is the hardcoded author list in this
file — deliberately not the Gitea permission API, because the
pipeline itself comments as gitea_admin and agent-authored text must
never be able to masquerade as owner guidance.
- Prompt: verified guidance is authoritative for intent and scope (may
lift the surgical ceiling when it calls for a sweep), does not lift
the edit-path allowlist, and factual mismatches with live state are
surfaced, not silently resolved in either direction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
Both wrong assessments (#249 openbao 'sealed', #250 three untested
hypotheses while one 'kubectl logs' showed the 401) share one root
cause: the agents diagnose blind, so they narrate plausible fiction.
Labeling unverified claims (previous commit) treats the symptom; this
gives them the means to verify:
- kubernetes/apps/monitoring/ci-diagnostics-rbac.yaml: ServiceAccount
ci-diagnostics with a read-only ClusterRole (workloads, events,
pods/log, Flux/ESO/cert-manager/monitoring/ceph CRDs — NO secrets)
plus a namespaced services/proxy Role for the Prometheus and
Alertmanager HTTP APIs.
- scripts/ci/make-diag-kubeconfig.sh: mints a 1h token per run from
the admin kubeconfig (no new long-lived secret), builds
.ci-diag/kubeconfig, copies the static kubectl binary for the
jq-only ci-claude image, smoke-tests with the restricted identity,
and degrades to snapshot-only mode when the SA isn't reconciled.
- alert-monitor.yml / issue-fix.yml: agents get exactly one extra tool,
Bash(./.ci-diag/kubectl:*) — prefix match forbids pipes and any other
binary; RBAC forbids mutation and secret reads.
- prompts: verification is now mandatory, not aspirational — 'a
hypothesis you could have settled with one read-only command but
didn't is a defect in your output'; fix agent must test the issue's
claims before building on them; minimal-quote rule against dumping
logs into public issues.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
The first issue-fix run on #249 asserted the openbao pod was '0/1 and
sealed' — it was Ready and unsealed the whole time; the alert fired
because the chart's updateStrategy: OnDelete holds the 2.5.5→2.6.0
image bump until an operator deletes the pod. The agent has no cluster
access and invented runtime state from the alert's generic annotation
text.
- issue-fix prompt: runtime claims must be attributed or framed as
hypothesis, with read-only verification commands in the summary.
- alert-monitor prompt: when context.txt does not confirm the alert's
symptom, say so explicitly — the mismatch is often the real clue.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
Two new cron pipelines closing the loop from firing alert to fix PR:
- alert-monitor (*/15 min): reads active alerts from Alertmanager via the
apiserver service proxy (woodpecker-ci is netpol-blocked from monitoring
ClusterIPs; the Authentik gate blocks the public hostname), dedups
against open issues and a persistent noise-log issue, then a headless
Claude agent triages the new ones against the repo plus a cluster
snapshot. Confirmed problems become Gitea issues; noise is recorded in
the noise log so it is never re-triaged.
- issue-fix (hourly): picks the oldest open '[alert] ...' issue with no
fix branch and no assessed-marker comment, hands it to a fix agent
(same tool sandbox as security-fix), pushes alert-fix/issue-<n> and
opens a PR with 'Closes #<n>'. Not-repo-fixable verdicts are posted as
an issue comment instead.
- e2e.yml: scope the bare 'event: cron' to the three existing cron names
— otherwise the 15-minute cron would piggyback a full e2e run every 15
minutes.
The ci-k8s image has no jq, so the alert fetch step only talks to the
cluster; all JSON filtering/dedup/gating lives in the ci-claude step.
Cron registration (after this lands on master):
woodpecker repo cron add --repo gitea_admin/homelab \
--name alert-monitor --branch master --schedule '*/15 * * * *'
woodpecker repo cron add --repo gitea_admin/homelab \
--name issue-fix --branch master --schedule '41 * * * *'
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
Closes gitea issue #197. The update-audit agent fetches untrusted
third-party release pages via WebFetch/WebSearch and its Write output
is posted verbatim to a public Gitea issue, so injected content in a
fetched page could try to redirect the agent's behavior. Harden the
tool surface (verified empirically against the pinned Claude Code CLI):
- Drop WebSearch: it cannot be domain-scoped like WebFetch, and the
prompt already gives explicit source URLs or a well-known registry
for everything it needs to look up.
- Add ci/agents/update-audit/settings.json (loaded via --settings) that
denies Read on /proc, /root, /home, /etc, and the repo's own .git/ —
closing the credential-file and env-var-dump read paths a
prompt-injected instruction could otherwise try. Verified this deny
mechanism actually blocks reads, unlike Write path-scoping, which
was tested and found NOT enforced under --permission-mode acceptEdits
(documented in the pipeline comment; a full WebFetch domain allowlist
was also considered and rejected, since Helm chart lookups legitimately
hit whichever domain each HelmRelease's own trusted source URL points
at).
- prompt.md now tells the agent fetched content is untrusted data, not
instructions, and to never echo it verbatim into the report.
Not fixed here: GITEA_TOKEN is still the broad registry/release-writing
token (no write:issue-only token exists yet) — that requires an
operator to mint one; tracked via a comment on #197 and a code comment
at the secret binding.
Addresses #209. All four ci/images/*/Containerfile FROM lines were
tag-pinned only (alpine:3.21, node:22-alpine), letting the base silently
change between builds. Pin each to its current manifest-list digest.
ci/images/k8s/Containerfile downloaded talosctl, kubectl, and velero via
curl with no integrity check, and installed flux via `curl | bash`
(itself an unpinned, unverified fetch-and-execute). Replace the flux
install with a pinned-version tarball download verified against its
published sha256 checksum, same as talosctl/kubectl/velero now are.
kubectl's version was also resolved dynamically from dl.k8s.io's
stable.txt at build time instead of being pinned; add KUBECTL_VERSION.
ci/images/tofu/Containerfile's tofu.zip download gets the same
sha256 verification.
ci/images/claude/Containerfile's CLAUDE_CODE_VERSION defaulted to
`latest` and was never overridden by build-ci-images.yml, so the image
holding repo/issue-write tokens floated to whatever npm resolved at
build time. Pin it to the current release (2.1.209).
Checksums and digests verified against the respective registries/release
APIs at fix time (2026-07-14).
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
rebuild-k8s.yml was the one-shot PVE->K8s migration/rebuild pipeline; it
is no longer needed and still pointed at the retired Garage S3 on the
now-off nas-1. Remove it (DR restore is via Velero + flux bootstrap).
The live cluster-access path is kept and migrated off Garage onto Hetzner
Object Storage (renner-uno-k8s-secrets, fsn1.your-objectstorage.com,
region fsn1, hetzner_s3_* keypair):
- scripts/ci/fetch-kubeconfig.sh: Garage -> Hetzner; env vars renamed
GARAGE_SECRETS_* -> S3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY.
- .woodpecker/e2e.yml, e2e-test.yml: pass hetzner_s3_access_key/secret_key
(the live cron e2e pipeline needs this — it was broken against dead nas-1).
- scripts/import-cluster-access.sh: Garage -> Hetzner endpoint/bucket/region.
NOTE: the workstation aws-cli "secrets" profile must now hold the Hetzner
Object Storage keypair.
Also: delete the orphaned test-garage-smoke play + pipeline, and fix the
two dangling rebuild-k8s.yml references (zfs-tuning play comment;
update-audit prompt Talos-pin file list).
Assumes cluster-access objects (kubeconfig.sops/talosconfig.sops) were
migrated into renner-uno-k8s-secrets; with rebuild-k8s gone, nothing
auto-refreshes them, which is fine for a stable cluster.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The netbird-ingress DaemonSet, netbird reverse-proxy, and Traefik have
fully superseded the old ingress tiers, which have been stopped (not
deleted) since the 2026-05-09 PVE cutover. Remove the dead code:
- Caddy HA: LXCs 232/233/234 (opentofu/ingress.tf), caddy-ha role,
install-caddy-ha + install-crowdsec-caddy plays, caddy cred vars,
proxmox_caddy_ha group_vars, build-caddy/deploy-caddy pipelines,
ci-caddy-builder image.
- cloudflared: LXC 117, cloudflared-lxc role, install-cloudflared play,
cloudflare_tunnel/variables_cloudflare disabled tofu stubs.
- netbird-gateway: LXC 260 (replaced by the in-cluster netbird-ingress
DaemonSet), netbird-gateway role + play.
- backup_jobs.tf: drop VMIDs 117/232/233/234/260 from the nightly vzdump
include list (those LXCs are being destroyed).
- netbird-server: point the setup-key hint at netbird-ingress/netbird-proxy.
Kept deliberately:
- crowdsec role (reusable; basis for the pending Traefik bouncer).
- netbird-client role + play (loki/prometheus LXCs are tagged
netbird_client, so it remains live architecture).
Note: applying the opentofu change will plan a DESTROY of stopped LXCs
117/232/233/234/260 — review the plan before apply.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a Woodpecker cron pipeline that runs a read-only Claude Code agent to
audit the components NOT covered by existing automation, and posts the report
to a single persistent Gitea issue.
Scope (complements, does not duplicate, existing automation):
- Flux image-automation already bumps enrolled container tags.
- The daily security-scan agent already owns CVEs.
- This agent audits manually-pinned currency instead: Talos / Kubernetes /
Flux versions, Helm chart minor/major bumps held back by ~MAJOR.MINOR.0
constraints, images excluded from automation, and Ansible/OpenTofu drift.
The agent is read-only (allowedTools: Read,Glob,Grep,Write,WebFetch,WebSearch;
no Edit/Bash) so it can never mutate the repo. It writes a report to
/tmp/update-audit.md; the pipeline find-or-creates the "Infrastructure update
audit" Gitea issue and refreshes its body.
Cron uses Woodpecker's documented `cron:` name filter (rather than the injected
PIPELINE var the other pipelines use) so the schedule is CLI-creatable:
woodpecker repo cron add --repo gitea_admin/homelab \
--name update-audit --branch master --schedule '@weekly'
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Container image tag updates are Flux's job — the homelab cluster
already runs image-reflector-controller and image-automation-
controller (in flux-system/gotk-components.yaml). Setting up
ImageRepository / ImagePolicy / ImageUpdateAutomation resources
is the right home for "bump postgres:15.17 → 15.18 when a new
image is published"; doing it from a CVE-driven scan agent
duplicates that pipeline and competes with it.
Reverting:
- Agent prompt: drop the `## Image tag bumps` section; image
findings now go straight to the Skipped list with a "Flux
image-automation owns this" reason.
- Woodpecker yml: drop Bash(crane *) from the agent's tool
allowlist (no longer used) and the `pull: true` workaround
that existed only to pick up crane's mid-day rebuild.
- ci-claude Containerfile: remove the crane install. The image
is back to Node + Claude Code + git/jq/curl basics.
The agent is now scoped exclusively to IaC misconfigurations
(Trivy fs scan results). Image vulns surface in the log + summary
for visibility but no PR action.
Note: there are currently zero ImageRepository / ImagePolicy /
ImageUpdateAutomation resources in kubernetes/ — the controllers
are running idle. Wiring them up for the actual images is a
separate workstream.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The seed KS owns influxdb-seed-ssh Secret in the influxdb namespace, but
the namespace was previously created by apps/influxdb/namespace.yaml —
which dependsOn seed → seed never reconciles → circular.
Hoist the namespace into infrastructure/namespaces/namespaces.yaml
(consistent with all the other app namespaces) and drop it from the
apps/influxdb Kustomization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pipeline 858's agent run found 277 image vulns (CRITICAL/HIGH) and
correctly skipped all of them — but the right action for many is a
patch-level image-tag bump (postgres:15.17 → 15.18,
distribution:3.0.0 → 3.0.1, etc.). The agent had no way to verify
a candidate tag exists in the registry, so it conservatively
skipped.
crane (google/go-containerregistry) handles:
crane ls <repository> — list available tags
crane manifest <image>:<tag> — verify tag resolves
Single static binary, works against Docker Hub, GHCR, gitea, quay,
any OCI registry. Pinned to v0.21.5.
The agent prompt update (only-patch bumps, lockstep cross-file edits,
compatibility-caveat surfacing) and the woodpecker allowlist update
(Bash(crane ls *), Bash(crane manifest *)) ride in a follow-up
commit so this image change can build first.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pipeline 856's agent run found the Trivy JSON files via the
pipeline's own jq summary (IaC=0, Image=645 findings) but the
agent itself reported "No Trivy scan files (/tmp/trivy-iac.json
...) were present" — it hallucinated the /tmp prefix and didn't
look in the workspace root.
Prompt now states explicitly that the JSONs are in the current
working directory (project root, not /tmp), and instructs the
agent's first action to be `Glob trivy-*.json` to enumerate what
exists. Adds a stop-condition for the genuinely-empty case so
the agent can't get into a "I'll assume they're somewhere else"
loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cron security-scan now ends in an autonomous fix loop:
iac-scan → trivy-iac.json
image-scans → trivy-image-N.json (per scanned image)
security-fix-agent (master only):
1. Gates on >=1 CRITICAL/HIGH finding across all JSON outputs.
2. Creates branch security-fix/<date>-<short-sha>.
3. Hands the workspace to Claude Code in headless --bare mode
(subscription-backed via CLAUDE_CODE_OAUTH_TOKEN, no API key)
with a narrow tool allowlist (Read/Edit/Write/Glob/Grep —
no Bash). The agent reads the JSON, applies surgical fixes
under ansible/, opentofu/, kubernetes/, ci/images/, and writes
a Markdown summary to /tmp/security-fix-summary.md.
4. If the agent produced a diff, the step commits as
"Security Fix Agent <security-bot@renner.uno>" and pushes to
Gitea, then opens a PR via the pulls API using a dedicated
security_bot_token (NOT gitea_admin) so the PR is visibly
machine-authored.
The step is gated to branch=master so it doesn't fire on its own
PR branches — those still trigger the scan workflow as a re-validation
gate, but skip the agent (no fork-bombing).
Trivy now writes JSON output (--format json -o), and findings appear
both in the JSON and as one-line summaries in the step log via jq.
The legacy --exit-code 1 was kept off so the workflow always reaches
the agent step on master; on PR branches the scans are informational
and a human reviews the agent's PR.
Required Woodpecker secrets (added by user, not by this commit):
- claude_code_oauth_token (claude setup-token output)
- security_bot_token (Gitea PAT for the security-bot user)
Required Gitea setup (also out of band):
- User "security-bot" with email security-bot@renner.uno
- Write collaborator on gitea_admin/homelab
Agent constraints live in ci/agents/security-fix/prompt.md so they're
diffable + reviewable separately from the pipeline plumbing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fifth image in the ci-* family. Bakes Node.js 22 + the Claude Code
CLI (@anthropic-ai/claude-code) plus git, openssh-client, curl, jq.
Used by the upcoming security-fix-agent step in security-scan.yml,
which runs Claude headlessly inside CI to propose fixes for Trivy
findings and open a PR.
Auth pattern (deferred to the consumer commit): subscription-backed
CLAUDE_CODE_OAUTH_TOKEN secret, injected as an env var, no API key.
build-ci-images.yml gains a build-claude step that mirrors the
existing four (kaniko, parallel, registry-mirror cache, date + latest
tags pushed to gitea.renner.uno/gitea_admin/).
Pinning: Containerfile uses CLAUDE_CODE_VERSION=latest at image-build
time. The resulting date-tagged image freezes whatever was current
when build-ci-images ran. Bumping = trigger build-ci-images, get a
new date tag, bump the consumer pin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fourth image in the ci-* family (alongside ci-ansible, ci-k8s,
ci-tofu). Bakes the Go toolchain (golang:1.26 base) plus xcaddy
pinned to v0.4.5 so the build-caddy pipeline doesn't have to
`go install ...@latest` on every run.
build-ci-images.yml gains a build-caddy-builder step that mirrors
the existing three (kaniko, parallel, pull-through-cached registry,
date + latest tags pushed to gitea.renner.uno/gitea_admin/).
The build-caddy.yml consumer swap is a separate commit so it can
land after the first build-ci-images run produces the image.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes after pipeline 829's wipe-backups step blew up:
- aws-cli is not in ci-k8s:2026-04-29 — wipe ran "aws s3 rm ..." and
got "/bin/sh: aws: not found", then exited 0 because set -eu doesn't
catch failures inside pipes / command substitutions.
ci/images/k8s/Containerfile gains aws-cli; pushing this triggers the
build-ci-images pipeline, which builds gitea.renner.uno/gitea_admin/
ci-k8s:2026-05-01 (date tag).
The wipe-backups step is bumped to that tag and switches to
set -euo pipefail with an explicit "remaining objects must be 0"
check after each rm — so a failed wipe halts the rebuild before the
new cluster's seed Jobs run against stale state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ghcr.io/opentofu/opentofu:1.11 has an ONBUILD that prints a deprecation
warning and runs `exit 1` — the maintainers explicitly disabled using
that image as a base in 1.10+ (opentofu/opentofu#1931). Build pipeline
810 hit it with `error building stage: ... exit status 1`.
Switch the base to alpine:3.21 and install the tofu binary directly
from the GitHub release zip (TOFU_VERSION=1.11.0 ARG, matches what the
pipeline previously pinned).
deploy-infrastructure was the one pipeline still installing
ansible/proxmoxer/galaxy collections at job time, because tofu apply
invokes ansible playbooks via module ansible_playbook provisioners —
both tools have to be in the same image.
ci-tofu inherits FROM ghcr.io/opentofu/opentofu:1.11 and bakes ansible
+ community.{proxmox,docker} collections + python deps + utility CLIs
on top. ENTRYPOINT/CMD reset so Woodpecker can run shell commands.
Both plan and apply steps swap to ci-tofu:2026-04-29. Drops apk add
git from plan and apk/pip/galaxy from apply.
Drops the per-job apk/pip/galaxy/curl-kubectl/curl-velero/curl-flux
preamble across 14 pipelines. Setup phase eliminated; only the
credential-staging steps remain. Pilot on update-all (commit b990749)
proved the swap mechanics; this completes the rollout.
ci-k8s gains velero (v1.16.0 ARG) so the migrate-* seed-pvcs steps
don't have to download it on every run.
ci-ansible swaps:
- update-all (already landed in b990749 — listed for completeness)
- install-{garage-synology,woodpecker-server,registry-mirror}
- deploy-caddy
- test-garage-smoke
- migrate-{immich,paperless,authentik-db}.backup-legacy-db
- rebuild-k8s.tune-cp-zvols
ci-k8s swaps:
- migrate-{influxdb,immich,paperless,music-assistant,audiobookshelf}.seed-pvcs
- migrate-authentik-db.wipe-and-restore
- e2e, e2e-test
- rebuild-k8s: talos-bootstrap, wipe-osd-disks, export-access,
flux-bootstrap, wait-apps, e2e-test
Left alone:
- deploy-infrastructure (special tofu+ansible mix on tofu image)
- migrate-tfstate (alpine + aws-cli/jq is fine)
- build-caddy (golang + alpine for release)
- security-scan (trivy)
- migrate-*.seed-barman steps (postgres-version-pinned images)
Image refs pin :2026-04-29; pipelines also use shell `base64 -d` in
place of the prior `python3 -c "import base64..."` (matches the
deploy-infrastructure.yml pattern).
apk add + pip install + ansible-galaxy ran on every job across ~12
pipelines and dominated wall time. Bake the toolsets into two images
pushed to Gitea's container registry; consuming pipelines pull instead.
ci-ansible: ansible + community.{proxmox,docker} + proxmoxer/requests
+ utility CLIs (sops, aws-cli, jq, sshpass, ...).
ci-k8s: talosctl (pinned to v1.12.6), kubectl (stable at build
time), flux + sops/openssl/bind-tools.
Tags YYYY-MM-DD plus :latest. Build pipeline triggers on changes under
ci/images/** or manual PIPELINE=build-ci-images.
claude-botandClaude Opus 4.8
k.renner