claude-botandClaude Opus 5
71a4c1b3a7
Everything in this comparison is Q4_K_M, including the baseline, and the quantisation damage has never been measured. That is an assumption sitting underneath every number in the README. ggml-org ships Q4_K_M, Q8_0 and BF16 of Qwen3.6-35B-A3B in one repo, so the control is exact: same model, same 50 documents, same flags, precision the only variable. There is a specific suspicion behind this, not just diligence. Ministral-3 leaking [THINK] and [TOOL_CALLS] into a tool call as literal text is the kind of fault 4-bit quantisation can produce, and Q8 would separate "the model cannot do this" from "Q4 broke it". Three candidates have been rejected on behaviour that no one has ruled out as a quantisation artefact. Q8 before BF16 deliberately: ~34 GB and ~6 tok/s against Q4's ~21 GB and 11 tok/s, so roughly double the wall clock, where BF16 would be ~65 GB at ~3.5 tok/s and near five hours. If Q8 matches Q4 within the noise floor, BF16 almost certainly does too and the expensive run is unnecessary. If it does not, BF16 becomes worth its cost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>