Commit Graph
2 Commits
Author SHA1 Message Date
claude-botandClaude Opus 4.8 1992fabb12 fix(ci): make model-picker actually tool-less; gate tests on PRs
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the 14aa6415 review.

- pick-model.sh: the picker relied on `--allowedTools ""` to be tool-less, but an
  empirical check showed an empty allowlist does NOT block tools (the Write tool
  still ran) -- it is treated as "unset" and the default toolset stays live. Add
  `--permission-mode plan`, which IS verified to block Write while still producing
  the selection JSON. --allowedTools "" + the Read deny-list remain as additional
  layers. Corrected the comments in pick-model.sh and the picker settings.json
  that claimed the empty allowlist made it tool-less.
- .woodpecker/ci-scripts-test.yml: new pull_request-triggered pipeline (homelab
  only) that runs scripts/ci/test-model-selection.sh, so a PR weakening the
  injection gate fails checks immediately instead of only on the next scheduled
  e2e cron (closes the coverage-timing gap the review flagged).
- Pin a recorded real /v1/models response (scripts/ci/testdata/v1-models-sample.json)
  and assert fetch-models.sh maps it correctly (opus effort=true, haiku
  effort=false), documenting the verified response shape.

Deferred with rationale: hard-failing a pipeline when the degraded-schema warning
fires -- the warning already reaches the job log (curl/jq stderr is no longer
suppressed), and hard-failing on a benign API schema addition would break all
model selection; graceful degradation to name+date is the deliberate choice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 09:08:39 +02:00
claude-botandClaude Opus 4.8 44a429685c fix(ci): round-3 pr-review hardening — REVIEW_TOKEN, no-die selection, tests
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the a7890e0a review.

- pr-review.yml: strip REVIEW_TOKEN from the valid-model.sh call too (it runs a
  script from $HOMELAB, branch-supplied for a homelab PR), and guard it with
  `|| true` so a missing script can't kill the step -- restoring the stated "a
  run never dies on selection" invariant. For a complex/security-sensitive
  review, ignore the (injectable) triage pick and run on the CLI default
  (strongest configured) model.
- valid-model.sh: reject a leading dash at the shape check (making the header
  accurate) so an option-like value is refused even without a catalog.
- fetch-models.sh: --retry 2 --retry-connrefused so a transient blip doesn't
  drop the run to the default; add a FETCH_MODELS_BODY_FILE test seam.
- pick-model.sh: pass --settings (new ci/agents/model-picker/settings.json,
  Read deny-list) as belt-and-suspenders behind --allowedTools "".
- test-model-selection.sh: add fetch-models.sh mapping tests (well-formed body
  -> non-null capability columns; degraded body -> loud warning) via the seam,
  and wire the whole suite into .woodpecker/e2e.yml so it runs in CI.
- Condense the duplicated model-selection comment in the four fixed jobs to a
  one-line pointer to pick-model.sh.
- PR description: state the cost / no-fixed-ceiling property change.

Deferred with rationale (in the PR thread): selected-model in the four crons'
posted artifacts (already logged per run), pagination past limit=1000 (API max;
has_more warns), and unit-testing pick-model.sh's JSON extraction (its output
always passes through the tested valid-model.sh gate, and the picker prompt is
developer-controlled, not attacker input).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 07:43:08 +02:00