docs(llama-server): cite the flag semantics instead of inferring them

`llama-server --help` documents both halves of what #765 got wrong, and it ships in the image: -np, --parallel N number of server slots (default: -1, -1 = auto) -kvu, --kv-unified use single unified KV buffer shared across all sequences (default: enabled if number of slots is auto) So leaving --parallel unset did not merely pick a slot count, it enabled a shared KV buffer that every slot could draw the full --ctx-size from. That is why this server reported n_ctx_slot 16384 across four slots, and the startup log said so outright: kv_unified = 'true'. Naming a slot count switches that off and splits the context. The value stays 65536. Only the reasoning in the comment changes, from an inference drawn off two /props readings to the documented behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fix(llama-server): --ctx-size is the total, so ask for 65536
2026-09-02 16:16:16 +02:00 · 2026-09-02 16:13:35 +02:00
1 changed files with 11 additions and 7 deletions
@@ -81,19 +81,23 @@ spec:
            # every exposed name, area and floor. At 16384 the request was
            # refused outright.
            #
-            # --ctx-size is per slot, not the total: /props reports
-            # n_ctx_slot equal to this value while running four slots, where a
-            # total would have divided down to 4096. Doubling it alone would
-            # therefore double the KV cache, so the slot count halves in step
-            # and the allocation stays where it is today — 24979 MiB of the
-            # 32Gi limit, most of it the 19.5 GiB of weights.
+            # --ctx-size is the total and llama.cpp divides it across the
+            # slots, so 65536 over two slots is 32768 each.
+            #
+            # It does not read that way without --parallel, and the reason is
+            # in `llama-server --help`: --kv-unified defaults to enabled only
+            # "if number of slots is auto". Leaving --parallel unset therefore
+            # bought a single KV buffer shared across sequences, where every
+            # slot could draw the full --ctx-size — which is why this server
+            # once reported n_ctx_slot 16384 across four slots. Naming a slot
+            # count turns that off and splits the context instead.
            #
            # Two slots is what is actually in use: one holds Home Assistant's
            # prompt, one holds Paperless's. A third caller would queue rather
            # than evict either, which is the better failure mode given a cold
            # prefill of this prompt costs around two minutes.
            - --ctx-size
-            - "32768"
+            - "65536"
            - --parallel
            - "2"
            # A micro-batch of 2048 or more makes this Vulkan/Mesa stack fail