2 Commits

Include renames

Author	SHA1	Message	Date
claude-botandClaude Opus 5	b64f971373	fix(paperless-eval): two concurrent runs were revoking each other's token ... Every parallel run so far was quietly broken, and it took a 401 sitting in the middle of three genuine model failures to notice. api_token() took the first superuser and deleted its existing token before inserting a new one, because Django's authtoken holds one token per user. Both candidates share this database, so the second job to start silently revoked the first one's credentials. In the log it appears as 50173: HTTP 401: b'{"detail":"Invalid token."}' among HTTP 400s from the model, where it reads like one more broken document. It was counted toward ABORT_AFTER, so a harness fault helped end a run and would have been recorded as a model verdict. Two changes. Each run now creates its own eval-<label> superuser, so the token belongs to the run and concurrency is safe — only the user can differ, the database cannot. And a 401 is now fatal with an explicit message rather than another row in the error column: it is never a fact about the model, and every number after it is worthless. The qwen35 result predates this and ran alone, so it is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-03 21:42:20 +02:00
claude-botandClaude Opus 5	22d92d158d	feat(paperless-eval): probe tool calling in two minutes, not ninety ... "These are all invocation errors" — correct, and I had written two of them off as properties of the models. probe-toolcall.sh starts a candidate, sends paperless's real 14-field DocumentClassifierSchema, and passes only on finish_reason "tool_calls". The payload is the actual schema rather than a stand-in, which is the entire point: a three-field probe got a clean call out of Ministral-3 while the real schema made it degenerate, and that false pass cost an hour of chasing prompt length. What it turned up immediately: Ministral-3 8B and 3B pass with bartowski's conversion and fail with mistralai's own, at identical quant and flags. llama-server defaults to -sp off, so [THINK] and [TOOL_CALLS] should never have appeared as text at all — they are not marked special in mistralai's file. A conversion defect, not the model. Both candidates now point at bartowski and are running. Granite fails with bartowski too, finish_reason "stop" and no tool_calls, so its cause is different and unchanged: llama.cpp b10731 cannot read Granite's tool-call format back whoever converted it. The remaining lever there is --chat-template-file with a template the server can parse, or a newer build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-03 21:36:59 +02:00