fix(llama-embed): raise the memory limit to match its own context and batch

#739 raised the context from 2048 to 4096 and the micro-batch from an effective 512 to 1024, both to match how Paperless chunks documents. It left memory at 2Gi/4Gi, which was sized for the smaller values. The pod ran fine interactively and was OOMKilled (exit 137) partway through the first sustained run, around 300 documents into an embedding pass. Nothing in the server log marked it — the process was killed outright, so the last line is a half-finished task and the reason is only visible in the container's lastState.terminated. This would have surfaced when Paperless first indexed the archive, i.e. on the first real use rather than under test. The weights are only ~600 MiB. What sets the footprint is four slots at 4096 context with a 1024-token micro-batch, plus Vulkan allocating the iGPU working set from system RAM where it counts against this cgroup — the same effect that put llama-server at 32Gi for 19 GiB of weights. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 08:14:19 +02:00
1 changed files with 12 additions and 2 deletions
@@ -91,14 +91,24 @@ spec:
            capabilities:
              drop:
                - ALL
+          # The weights are only ~600 MiB, but that is not what sets the
+          # footprint: four slots at 4096 context with a 1024-token micro-batch,
+          # and Vulkan allocating the iGPU's working set from system RAM where
+          # it counts against this cgroup.
+          #
+          # 4Gi was sized for the earlier 2048 context and 512 micro-batch. Both
+          # were raised to match Paperless's chunking without raising this, and
+          # the pod then survived interactive use and was OOMKilled (137) partway
+          # through the first sustained run — roughly 300 documents in. Indexing
+          # the archive would have hit it on day one.
          resources:
            requests:
              cpu: 250m
-              memory: 2Gi
+              memory: 4Gi
              gpu.intel.com/i915: "1"
            limits:
              cpu: "2"
-              memory: 4Gi
+              memory: 10Gi
              gpu.intel.com/i915: "1"
          # ~600 MiB rather than ~5 GiB, so a much shorter budget than the chat
          # model needs, but still sized for a cold download.