Commit Graph
6 Commits
Author SHA1 Message Date
claude-botandClaude Opus 4.8 44a429685c fix(ci): round-3 pr-review hardening — REVIEW_TOKEN, no-die selection, tests
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the a7890e0a review.

- pr-review.yml: strip REVIEW_TOKEN from the valid-model.sh call too (it runs a
  script from $HOMELAB, branch-supplied for a homelab PR), and guard it with
  `|| true` so a missing script can't kill the step -- restoring the stated "a
  run never dies on selection" invariant. For a complex/security-sensitive
  review, ignore the (injectable) triage pick and run on the CLI default
  (strongest configured) model.
- valid-model.sh: reject a leading dash at the shape check (making the header
  accurate) so an option-like value is refused even without a catalog.
- fetch-models.sh: --retry 2 --retry-connrefused so a transient blip doesn't
  drop the run to the default; add a FETCH_MODELS_BODY_FILE test seam.
- pick-model.sh: pass --settings (new ci/agents/model-picker/settings.json,
  Read deny-list) as belt-and-suspenders behind --allowedTools "".
- test-model-selection.sh: add fetch-models.sh mapping tests (well-formed body
  -> non-null capability columns; degraded body -> loud warning) via the seam,
  and wire the whole suite into .woodpecker/e2e.yml so it runs in CI.
- Condense the duplicated model-selection comment in the four fixed jobs to a
  one-line pointer to pick-model.sh.
- PR description: state the cost / no-fixed-ceiling property change.

Deferred with rationale (in the PR thread): selected-model in the four crons'
posted artifacts (already logged per run), pagination past limit=1000 (API max;
has_more warns), and unit-testing pick-model.sh's JSON extraction (its output
always passes through the tested valid-model.sh gate, and the picker prompt is
developer-controlled, not attacker input).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 07:43:08 +02:00
claude-botandClaude Opus 4.8 a7890e0aa2 test(ci): extract + unit-test model validation; cut triage cost; doc nits
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the remaining pr-review items.

- Extract the untrusted-model validation out of the pipeline YAML into
  scripts/ci/valid-model.sh (single safe-token + catalog-membership gate) and
  reuse it from both pick-model.sh and pr-review.yml, so the security-critical
  logic lives in one tested place. Add scripts/ci/test-model-selection.sh, an
  offline fixture test (no network / claude / Woodpecker) that pins the
  behaviour: valid ids pass; hallucinated, empty, space-, newline-, leading-
  dash-, and metachar-injected values are rejected; empty catalog rejects all.
  This is the regression test for the flag-injection fix.
- pr-review triage cost: hand the triage the COMPLETE changed-file list
  (files.txt) plus a diff capped at 4000 lines (diff.triage.patch) instead of
  the full up-to-20k review diff, so classifying a PR on the default model stays
  cheap. stats.txt still carries the true totals; triage-prompt.md updated.
- Doc nits: note the new helpers are POSIX `sh` (not `bash` like the older
  scripts/ci/*.sh), and that fetch-models.sh's json mode is a debug affordance.

Deferred with rationale (also in the PR thread):
- Model id in the four cron jobs' posted artifacts: selection (including the
  fallback-to-default) is already logged per run in the CI output, the
  operational surface for this; threading it through four different posted
  formats is a separate observability follow-up.
- --settings sandbox on the picker: it is granted no tools (--allowedTools ""),
  so there is nothing for a Read-deny sandbox to constrain -- the other agents'
  sandboxes exist to scope tools they DO have.
- Pagination past limit=1000: that is the API's max page size; >1000 models is
  not a real scenario, and the has_more warning turns even that into a visible
  log line rather than silent truncation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 07:27:02 +02:00
claude-botandClaude Opus 4.8 f333dfd606 fix(ci): block flag injection via agent-supplied model id (pr-review)
ci/woodpecker/pr/pr-review Pipeline is running
The pr-review triage agent reads the attacker-controlled PR diff and writes
triage.json; its Write tool is not path-scoped, so a prompt-injected diff can
make it emit any `model` string. The catalog validation used
`grep -qxF -- "$MODEL"`, and grep -F treats a MULTI-LINE operand as
newline-separated patterns: a value like
"claude-sonnet-5\n--dangerously-skip-permissions" matches on its first line,
passes validation, and -- via the unquoted `${MODEL:+--model $MODEL}` expansion
-- word-splits into an extra argv entry, injecting a CLI flag into the review
agent's own `claude` invocation (defeating its --allowedTools allowlist and Read
sandbox, with CLAUDE_CODE_OAUTH_TOKEN in that process's env).

Removing map_model() (which returned one of three literal ids) is what opened
this; restore the invariant with a single-token guard before the expansion:
`case "$MODEL" in ""|*[!A-Za-z0-9._-]*) MODEL= ;; esac`. Verified it turns the
newline/space/hallucinated payloads into a plain `claude -p` (CLI default) while
still passing real ids. Mirrored the guard in pick-model.sh as defence in depth
(its grep -oE already keeps CHOICE single-line, so it was not exploitable).

Also from the review:
- pr-review.yml: correct the HOMELAB comment -- the fetch-models.sh call is
  env -u REVIEW_TOKEN'd, so CLAUDE_CODE_OAUTH_TOKEN (not REVIEW_TOKEN) is what
  branch-supplied shell there can see.
- pick-model.sh / triage-prompt.md: stop asking the picker to weigh "cost" the
  catalog does not expose; reason from the visible capability signals (context /
  max_output / effort|thinking|code_exec) as the scale proxy instead.
- fetch-models.sh: drop 2>/dev/null on curl so its -S error reaches the log.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 07:12:06 +02:00
claude-botandClaude Opus 4.8 9d36087857 fix(ci): second-round pr-review hardening for model selection
ci/woodpecker/pr/pr-review Pipeline was successful
- pick-model.sh: stop swallowing fetch-models.sh and claude stderr so the real
  cause of a fallback (no token / HTTP 401 / parse error / auth) reaches the job
  log instead of just "no catalog available"; time-bound the selection call with
  a self-probing `timeout` (the wrapper is skipped unless a probe confirms the
  image's timeout takes coreutils "SECS CMD" syntax, so it can never silently
  route every run to the default); strip PUSH_TOKEN and REVIEW_TOKEN as well --
  issue-fix binds its bot PAT a second time as PUSH_TOKEN, which survived the
  original `env -u GITEA_TOKEN`; `grep -qxF --`.
- fetch-models.sh: warn loudly when every model returns a null context window,
  so a /v1/models schema change surfaces as a message rather than silently
  reducing both selectors to name+date guessing.
- pr-review.yml: `env -u REVIEW_TOKEN` on the fetch-models.sh call (it needs only
  CLAUDE_CODE_OAUTH_TOKEN), matching the agent calls below it; `grep -qxF --`.

Deferred with rationale: hard-bounding the triage's diff input (kept as prompt
guidance -- a hard cap risks its security-sensitivity judgment) and an offline
fallback-path test harness (tracked as a follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 07:01:27 +02:00
claude-botandClaude Opus 4.8 b329c432c5 fix(ci): address pr-review feedback on dynamic model selection
ci/woodpecker/pr/pr-review Pipeline was successful
- fetch-models.sh: request ?limit=1000 so older/cheaper models are not dropped
  by the API's 20-item default page; warn on has_more. Add a --table mode so
  the catalog format lives in ONE place (pick-model.sh and pr-review.yml now
  consume it instead of each re-deriving the jq table). Clean up the HTTP-000
  double-print and surface jq parse errors instead of hiding them.
- pick-model.sh: run the picker with GITEA_TOKEN stripped from its env and
  without acceptEdits (it is granted no tools); consume fetch-models.sh --table.
- pr-review.yml: print model `CLI default` instead of an empty backtick pair on
  the fallback path; document that the fetch-models.sh call runs branch-supplied
  shell (not a new exposure -- Woodpecker already runs the pipeline YAML from
  the PR branch).
- triage-prompt.md: read stats + the changed-file list first and pull the full
  diff only when needed, so the triage (now on a capable default model) does not
  read a 20k-line diff just to classify a trivial PR.

Verified against the live catalog: all 11 currently-listed ids run under
`claude --model`, and the capability fields (context/effort/thinking/...) are
present in the OAuth /v1/models response -- so the reviewer's blocking concern
(a listed-but-CLI-unusable id) has no live trigger, and the fallback-to-default
path still covers it if that ever changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 06:34:38 +02:00
claude-botandClaude Opus 4.8 ffe3be4b54 feat(ci): select agent model from the live catalog, not hardcoded ids
ci/woodpecker/pr/pr-review Pipeline was successful
The pr-review, update-audit, security-scan, alert-monitor and issue-fix
pipelines each pinned a dated model id (--model claude-sonnet-5, plus
pr-review's haiku/sonnet/opus map_model allowlist). Those go stale every
time a new model ships and bake in assumptions about which model families
exist, so a newly released model could never be chosen without a code edit.

Replace all of it with selection driven only by what is available right now:

- scripts/ci/fetch-models.sh queries the catalog the CLI itself uses
  (Anthropic GET /v1/models, via the existing CLAUDE_CODE_OAUTH_TOKEN and the
  required anthropic-beta: oauth-2025-04-20 header) and emits each model with
  its objective capability signals -- context, max_output, effort, thinking,
  code_exec, release date -- with no id patterns and no family names.
- scripts/ci/pick-model.sh lets a triage agent choose the best-fit id from
  that catalog for a one-line task description, weighing every listed model
  on its merits. pr-review's existing per-PR triage agent now does the same,
  reading the live catalog instead of emitting a fixed haiku/sonnet/opus tier.

No model id -- and no model family -- is baked into any pipeline. A model
released tomorrow, of any family, becomes eligible the day the API lists it.
The bootstrap selection call runs on the CLI default model, and if the catalog
is ever unreachable --model is simply omitted (CLI default) rather than falling
back to a named model, so a run never dies over model selection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 06:14:45 +02:00