claude-botandClaude Opus 5
e99e502da7
The embedding model was picked from a recommendation and never compared. MTEB's German retrieval results put the deployed Qwen3-Embedding-0.6B well down the field, so a dozen candidates are being measured against the real archive with scripts/embed-eval-real.sh — and they will not fit in 40Gi alongside the two deployed models. Growing the shared cache rather than using a scratch volume is deliberate: on a slow connection the download is the long pole, and a shared cache lets the large candidates pull in the background while the small ones are already being measured. It also means nothing is fetched twice across runs, and a candidate that wins is already local when it is promoted. 120Gi against 309 GiB currently free in cephfs-data0. CephFS has allowVolumeExpansion, so this grows online without disturbing the running pods. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
32 lines
1.2 KiB
YAML
32 lines
1.2 KiB
YAML
---
|
|
# Holds the GGUF weights, downloaded once by llama-server itself via --hf-repo.
|
|
# Prune-disabled so retiring this Kustomization does not throw away tens of GiB
|
|
# of model files that would have to be pulled from Hugging Face again.
|
|
#
|
|
# ReadWriteMany so a second replica, a one-off benchmark Job, or a candidate
|
|
# model being evaluated can mount the same weights; the models are read-only at
|
|
# inference time.
|
|
#
|
|
# Sized well beyond the two deployed models. The cache is shared by every
|
|
# candidate under evaluation (scripts/embed-eval-real.sh), and on a slow
|
|
# connection the download is the long pole — a shared cache lets the large
|
|
# candidates pull in the background while the small ones are already being
|
|
# measured, and means a model is never fetched twice across runs.
|
|
#
|
|
# CephFS supports online expansion (allowVolumeExpansion on the storage class),
|
|
# so growing this does not disturb the running pods.
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: llama-server-models
|
|
namespace: ai
|
|
annotations:
|
|
kustomize.toolkit.fluxcd.io/prune: disabled
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteMany
|
|
storageClassName: ceph-filesystem
|
|
resources:
|
|
requests:
|
|
storage: 120Gi
|