fix(ci): validate pr-review model pick against pre-triage catalog snapshot
ci/woodpecker/pr/pr-review Pipeline was successful

The pr-review triage agent runs with an unscoped Write tool and its cwd contains
./.pr-review (-> $D), so a prompt-injected diff could make it overwrite
./.pr-review/models.txt and then name a smuggled token as its `model`. Validation
re-read models.txt FROM DISK after the triage ran, so the catalog-membership
check was defeated by the agent's own writeback -- letting an attacker force a
weaker real model, or a syntactically-valid fake id that makes the review's
`claude --model <fake>` fail and silently suppress the review entirely.

Validate against $MODELS_TABLE -- the catalog captured in a shell variable BEFORE
the triage runs -- instead of re-reading the file the triage can overwrite. This
mirrors pick-model.sh, which already validates against its in-memory table before
its (tool-less) picker runs. The triage still reads models.txt for its own pick;
tampering only misleads that pick, which is then rejected against the pristine
snapshot -> CLI default (so the review still runs, it can't be suppressed).

Verified: a token present only in a tampered on-disk file but absent from the
pre-triage snapshot is now rejected (it was accepted before). Added a regression
test to scripts/ci/test-model-selection.sh, and fixed the header comment to note
the complex-review override as another path to the CLI default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-25 08:29:18 +02:00
co-authored by Claude Opus 4.8
parent 44a429685c
commit 14aa641560
2 changed files with 24 additions and 6 deletions
+14 -6
View File
@@ -22,9 +22,11 @@
# CLAUDE_CODE_OAUTH_TOKEN), and a triage agent reads the PR diff + stats and
# picks whichever currently-available model best fits the review, judged on the
# catalog's own capability metadata. A model released tomorrow is eligible the
# day it appears. The shell validates the pick against the catalog; if the fetch
# or the pick fails it leaves the model unset and the review runs on the CLI's
# default model. A wrong-way miss only over/under-spends one review.
# day it appears. The shell validates the pick against the catalog snapshot taken
# BEFORE the triage runs; if the fetch fails, the pick is rejected, or the review
# is complex/security-sensitive (where the injectable pick is deliberately
# ignored), it leaves the model unset and the review runs on the CLI's default
# model. A wrong-way miss only over/under-spends one review.
#
# Posture: posted as a Gitea PR review, event COMMENT normally or
# REQUEST_CHANGES on a serious defect. It NEVER approves — a bot must not
@@ -242,14 +244,20 @@ steps:
# triage.json is UNTRUSTED (the triage reads the attacker-controlled
# diff and its Write is not path-scoped). valid-model.sh is the single,
# unit-tested gate (scripts/ci/test-model-selection.sh): it echoes the id
# only if it is a safe [A-Za-z0-9._-] token present in the live catalog,
# else nothing — so a value like "claude-x\n--dangerously-skip-permissions"
# only if it is a safe [A-Za-z0-9._-] token present in the catalog, else
# nothing — so a value like "claude-x\n--dangerously-skip-permissions"
# can never reach the unquoted --model expansion below. Empty -> default.
# CRITICAL: validate against $$MODELS_TABLE — the catalog captured in a
# shell variable BEFORE the triage ran — NOT the on-disk ./.pr-review/
# models.txt, which the triage's unscoped Write could overwrite to smuggle
# an attacker-chosen token past the membership check (the triage reads the
# file, but only $$MODELS_TABLE is trusted). This mirrors pick-model.sh,
# which validates its pick against its in-memory table for the same reason.
# env -u REVIEW_TOKEN: like fetch-models.sh above, this runs a script from
# $$HOMELAB (branch-supplied for a homelab PR) — it must not see the PAT.
# `|| true`: keep the "a run never dies on selection" invariant even if
# the script is absent (e.g. a PR that moves scripts/ci/).
MODEL=$$(env -u REVIEW_TOKEN sh "$$HOMELAB/scripts/ci/valid-model.sh" "$$MODEL" < "$$D/models.txt" || true)
MODEL=$$(printf '%s\n' "$$MODELS_TABLE" | env -u REVIEW_TOKEN sh "$$HOMELAB/scripts/ci/valid-model.sh" "$$MODEL" || true)
[ -n "$$MODEL" ] || echo "Triage model rejected or not in live catalog — using the CLI default model."
# Defence in depth: for a complex / security-sensitive review, do NOT
# trust the triage's pick (it read the attacker-controlled diff and could
+10
View File
@@ -37,6 +37,16 @@ check "path/quote chars rejected" 'claude-opus-5;rm -rf /'
got="$(printf '' | sh "$HERE/valid-model.sh" "claude-opus-5" || true)"
if [ -z "$got" ]; then printf 'ok empty catalog rejects all\n'; else printf 'FAIL empty catalog: got [%s]\n' "$got"; fails=$((fails + 1)); fi
# Regression for the models.txt-writeback attack: pr-review must validate the
# triage's pick against the catalog snapshot captured BEFORE the (untrusted,
# unscoped-Write) triage agent runs — never a file the agent could overwrite.
# valid-model.sh takes the catalog on stdin precisely so the caller controls it;
# a token the attacker slipped into an on-disk file but that is NOT in the
# pristine snapshot passed here must be rejected.
pristine="$(printf '%s\n' 'claude-opus-5 Claude Opus 5' 'claude-sonnet-5 Claude Sonnet 5')"
got="$(printf '%s\n' "$pristine" | sh "$HERE/valid-model.sh" "claude-smuggled-weak" || true)"
if [ -z "$got" ]; then printf 'ok tampered id rejected by pristine-snapshot validation\n'; else printf 'FAIL writeback attack: got [%s]\n' "$got"; fails=$((fails + 1)); fi
# fetch-models.sh jq mapping — exercised over recorded /v1/models bodies via the
# FETCH_MODELS_BODY_FILE seam (no token, no network). Pins the response shape the
# selectors depend on, so an API change surfaces here instead of silently