Addresses the 14aa6415 review.
- pick-model.sh: the picker relied on `--allowedTools ""` to be tool-less, but an
empirical check showed an empty allowlist does NOT block tools (the Write tool
still ran) -- it is treated as "unset" and the default toolset stays live. Add
`--permission-mode plan`, which IS verified to block Write while still producing
the selection JSON. --allowedTools "" + the Read deny-list remain as additional
layers. Corrected the comments in pick-model.sh and the picker settings.json
that claimed the empty allowlist made it tool-less.
- .woodpecker/ci-scripts-test.yml: new pull_request-triggered pipeline (homelab
only) that runs scripts/ci/test-model-selection.sh, so a PR weakening the
injection gate fails checks immediately instead of only on the next scheduled
e2e cron (closes the coverage-timing gap the review flagged).
- Pin a recorded real /v1/models response (scripts/ci/testdata/v1-models-sample.json)
and assert fetch-models.sh maps it correctly (opus effort=true, haiku
effort=false), documenting the verified response shape.
Deferred with rationale: hard-failing a pipeline when the degraded-schema warning
fires -- the warning already reaches the job log (curl/jq stderr is no longer
suppressed), and hard-failing on a benign API schema addition would break all
model selection; graceful degradation to name+date is the deliberate choice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the a7890e0a review.
- pr-review.yml: strip REVIEW_TOKEN from the valid-model.sh call too (it runs a
script from $HOMELAB, branch-supplied for a homelab PR), and guard it with
`|| true` so a missing script can't kill the step -- restoring the stated "a
run never dies on selection" invariant. For a complex/security-sensitive
review, ignore the (injectable) triage pick and run on the CLI default
(strongest configured) model.
- valid-model.sh: reject a leading dash at the shape check (making the header
accurate) so an option-like value is refused even without a catalog.
- fetch-models.sh: --retry 2 --retry-connrefused so a transient blip doesn't
drop the run to the default; add a FETCH_MODELS_BODY_FILE test seam.
- pick-model.sh: pass --settings (new ci/agents/model-picker/settings.json,
Read deny-list) as belt-and-suspenders behind --allowedTools "".
- test-model-selection.sh: add fetch-models.sh mapping tests (well-formed body
-> non-null capability columns; degraded body -> loud warning) via the seam,
and wire the whole suite into .woodpecker/e2e.yml so it runs in CI.
- Condense the duplicated model-selection comment in the four fixed jobs to a
one-line pointer to pick-model.sh.
- PR description: state the cost / no-fixed-ceiling property change.
Deferred with rationale (in the PR thread): selected-model in the four crons'
posted artifacts (already logged per run), pagination past limit=1000 (API max;
has_more warns), and unit-testing pick-model.sh's JSON extraction (its output
always passes through the tested valid-model.sh gate, and the picker prompt is
developer-controlled, not attacker input).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>