claude-botandClaude Opus 5
c47c2e6bc1
A Job is immutable and a measurement is finished when it finishes — neither is something Flux should converge on. Re-running a configuration means deleting and recreating the Job, and a managed one comes straight back. Worse, eval.py was a kustomize-generated ConfigMap, so editing the script between runs and applying it by hand would have been reverted on the next reconcile: the iteration loop the harness exists for was closed off by where it was put. The namespace stays managed. It holds a database clone, PVCs and a network policy — state that should converge and should vanish when the Kustomization is removed. Only the harness moves to scripts/, alongside the other tools nobody deploys, with a README covering how to run one, how to add a candidate, and how to read the output. Two failure modes are written down there rather than left to be rediscovered: prompts past ~14k tokens make this stack emit garbage silently and poison the slot until restart, and ground truth is a filing convention that has drifted over ten years, so the comparison between engines is the durable result rather than the absolute numbers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>