Addresses the a7890e0a review.
- pr-review.yml: strip REVIEW_TOKEN from the valid-model.sh call too (it runs a
script from $HOMELAB, branch-supplied for a homelab PR), and guard it with
`|| true` so a missing script can't kill the step -- restoring the stated "a
run never dies on selection" invariant. For a complex/security-sensitive
review, ignore the (injectable) triage pick and run on the CLI default
(strongest configured) model.
- valid-model.sh: reject a leading dash at the shape check (making the header
accurate) so an option-like value is refused even without a catalog.
- fetch-models.sh: --retry 2 --retry-connrefused so a transient blip doesn't
drop the run to the default; add a FETCH_MODELS_BODY_FILE test seam.
- pick-model.sh: pass --settings (new ci/agents/model-picker/settings.json,
Read deny-list) as belt-and-suspenders behind --allowedTools "".
- test-model-selection.sh: add fetch-models.sh mapping tests (well-formed body
-> non-null capability columns; degraded body -> loud warning) via the seam,
and wire the whole suite into .woodpecker/e2e.yml so it runs in CI.
- Condense the duplicated model-selection comment in the four fixed jobs to a
one-line pointer to pick-model.sh.
- PR description: state the cost / no-fixed-ceiling property change.
Deferred with rationale (in the PR thread): selected-model in the four crons'
posted artifacts (already logged per run), pagination past limit=1000 (API max;
has_more warns), and unit-testing pick-model.sh's JSON extraction (its output
always passes through the tested valid-model.sh gate, and the picker prompt is
developer-controlled, not attacker input).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the remaining pr-review items.
- Extract the untrusted-model validation out of the pipeline YAML into
scripts/ci/valid-model.sh (single safe-token + catalog-membership gate) and
reuse it from both pick-model.sh and pr-review.yml, so the security-critical
logic lives in one tested place. Add scripts/ci/test-model-selection.sh, an
offline fixture test (no network / claude / Woodpecker) that pins the
behaviour: valid ids pass; hallucinated, empty, space-, newline-, leading-
dash-, and metachar-injected values are rejected; empty catalog rejects all.
This is the regression test for the flag-injection fix.
- pr-review triage cost: hand the triage the COMPLETE changed-file list
(files.txt) plus a diff capped at 4000 lines (diff.triage.patch) instead of
the full up-to-20k review diff, so classifying a PR on the default model stays
cheap. stats.txt still carries the true totals; triage-prompt.md updated.
- Doc nits: note the new helpers are POSIX `sh` (not `bash` like the older
scripts/ci/*.sh), and that fetch-models.sh's json mode is a debug affordance.
Deferred with rationale (also in the PR thread):
- Model id in the four cron jobs' posted artifacts: selection (including the
fallback-to-default) is already logged per run in the CI output, the
operational surface for this; threading it through four different posted
formats is a separate observability follow-up.
- --settings sandbox on the picker: it is granted no tools (--allowedTools ""),
so there is nothing for a Read-deny sandbox to constrain -- the other agents'
sandboxes exist to scope tools they DO have.
- Pagination past limit=1000: that is the API's max page size; >1000 models is
not a real scenario, and the has_more warning turns even that into a visible
log line rather than silent truncation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>