2 Commits

Include renames

Author	SHA1	Message	Date
claude-botandClaude Opus 5	9ff85f7d1f	fix(paperless-eval): probe at production prompt length too ... The probe used the real 14-field schema but a 1191-token prompt, while a real paperless request carries ~5300 tokens once the document and its RAG neighbours are in. Ministral-3 8B passed the probe and then ran to the --n-predict cap on the first document of the real run. Whether that document is pathological or the probe is systematically too easy is not yet settled — it has failed under every engine tried so far, including the one that works. But a probe is only worth having if it fails where production fails, and it costs nothing to make it realistic in both dimensions rather than one. ~5000 tokens of German filler now precedes the document text. This is the same mistake twice: the first version was too easy on schema width, this one was too easy on prompt length, and both times a pass sent a ninety-minute run after a model that could not do the job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-03 21:39:24 +02:00
claude-botandClaude Opus 5	22d92d158d	feat(paperless-eval): probe tool calling in two minutes, not ninety ... "These are all invocation errors" — correct, and I had written two of them off as properties of the models. probe-toolcall.sh starts a candidate, sends paperless's real 14-field DocumentClassifierSchema, and passes only on finish_reason "tool_calls". The payload is the actual schema rather than a stand-in, which is the entire point: a three-field probe got a clean call out of Ministral-3 while the real schema made it degenerate, and that false pass cost an hour of chasing prompt length. What it turned up immediately: Ministral-3 8B and 3B pass with bartowski's conversion and fail with mistralai's own, at identical quant and flags. llama-server defaults to -sp off, so [THINK] and [TOOL_CALLS] should never have appeared as text at all — they are not marked special in mistralai's file. A conversion defect, not the model. Both candidates now point at bartowski and are running. Granite fails with bartowski too, finish_reason "stop" and no tool_calls, so its cause is different and unchanged: llama.cpp b10731 cannot read Granite's tool-call format back whoever converted it. The remaining lever there is --chat-template-file with a template the server can parse, or a newer build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-03 21:36:59 +02:00