The embedding model was picked from a recommendation and never compared. MTEB's
German retrieval results put the deployed Qwen3-Embedding-0.6B well down the
field, so a dozen candidates are being measured against the real archive with
scripts/embed-eval-real.sh — and they will not fit in 40Gi alongside the two
deployed models.
Growing the shared cache rather than using a scratch volume is deliberate: on a
slow connection the download is the long pole, and a shared cache lets the large
candidates pull in the background while the small ones are already being
measured. It also means nothing is fetched twice across runs, and a candidate
that wins is already local when it is promoted.
120Gi against 309 GiB currently free in cephfs-data0. CephFS has
allowVolumeExpansion, so this grows online without disturbing the running pods.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Immich's in-app migration to VectorChord is complete: both vector indexes,
face_index and clip_index, run on the vchordrq access method, and nothing uses
pgvecto.rs. The manifest still described a migration window.
`DROP EXTENSION vectors` was run against the immich database before this. It
was safe to do without CASCADE — nothing outside the extension depended on it,
which a rolled-back `BEGIN; DROP EXTENSION vectors; ROLLBACK;` confirmed
first. The embedding columns are `public.vector(512)`, which belongs to
pgvector, not to pgvecto.rs's type in the `vectors` schema. Afterwards both
vchordrq indexes are intact at 96 MB and 79 MB and the cluster stays healthy.
With the extension gone, vectors.so has nothing to serve and preloading it
only recreates pg_vectors in PGDATA. pgvector needs no preloading, so vchord.so
is the whole list.
instances goes back to 2. The blocker was never the extension as such — it was
that pg_basebackup's tar format cannot represent the long
pg_vectors/indexes/<uuid> paths pgvecto.rs creates, so a replica could not
join. That directory is now empty, 20 KiB in total, so the constraint is gone.
The image keeps its `:16-migration` tag. That is the upstream build's own name
for a multi-extension image, not a statement about this cluster, and it still
provides the two extensions in use. Moving to a different image is a separate
change with its own risk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both documents described work as outstanding that the cluster shows finished.
A record that says "REMAINING" long after the fact is worse than no record:
it is read as current state, and the next person plans around it.
## Home Assistant recorder
Phase 2c is done, and done correctly. Against the live database:
statistics 11,092,586 2024-03-13 20:00 -> current
statistics_short_term 1,883,709 rolling ~7 days
statistics_meta 1,480
The 2024-03-13 origin is the part that matters. It is unchanged from the
10,793,712-row count recorded on 2026-08-19, so the history was copied across
engines rather than started from zero — the exact failure the document was
written to prevent. `states` carries current rows, so db_url points at
Postgres.
The three "INERT"/"PHASE 1" comments went with it. All three described a
cluster that nothing consumed; it is now the recorder's live database, and the
render-db-url init container is load-bearing rather than inert — HA will not
start without the secret it renders.
## Bare-metal conversion
`talosctl get systeminformation` reports Minisforum MS-01 hardware with
distinct chassis serials for all three nodes and no hypervisor, so saturn and
neptune are not outstanding. Node creationTimestamp is not evidence either way:
k8s-3 still carries its 2026-05-09 object and is on metal regardless.
The runbook stays as the procedure for a rebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 08:37:19 +02:00
9 changed files with 73 additions and 32 deletions
# Holds the GGUF weights, downloaded once by the Deployment's initContainer.
# Prune-disabled so retiring this Kustomization does not throw away several
# GiB of model files that would have to be pulled from Hugging Face again.
# Holds the GGUF weights, downloaded once by llama-server itself via --hf-repo.
# Prune-disabled so retiring this Kustomization does not throw away tens of GiB
# of model files that would have to be pulled from Hugging Face again.
#
# ReadWriteMany so a second replica or a one-off benchmark Job can mount the
# same weights; the models are read-only at inference time.
# ReadWriteMany so a second replica, a one-off benchmark Job, or a candidate
# model being evaluated can mount the same weights; the models are read-only at
# inference time.
#
# Sized well beyond the two deployed models. The cache is shared by every
# candidate under evaluation (scripts/embed-eval-real.sh), and on a slow
# connection the download is the long pole — a shared cache lets the large
# candidates pull in the background while the small ones are already being
# measured, and means a model is never fetched twice across runs.
#
# CephFS supports online expansion (allowVolumeExpansion on the storage class),
# so growing this does not disturb the running pods.
apiVersion:v1
kind:PersistentVolumeClaim
metadata:
@@ -18,4 +28,4 @@ spec:
storageClassName:ceph-filesystem
resources:
requests:
storage:40Gi
storage:120Gi
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
issue-fix-bot
k.renner
claude-botandClaude Opus 5