chore(rook-ceph): drop k8s-2 from the storage fleet for saturn's conversion

Removing the node from storage.nodes before purging its OSD is what makes the
purge stick. Under useAllNodes the operator rebuilt a purged OSD within one
reconcile and began backfilling onto the drive being replaced; listing nodes
explicitly exists so a node can be taken out first.

Re-add after the conversion naming the device by /dev/disk/by-id, as k8s-1
does. A bare device name is not safe on bare metal: Talos enumerates the three
NVMe drives in a different order than Proxmox, and nvme0n1 is the system disk.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

kubernetes/apps/immich/machine-learning.yaml

@@ -6,17 +6,14 @@ metadata:
   labels:
     app.kubernetes.io/name: immich-machine-learning
 spec:
-  # TEMPORARY — RESTORE TO 1 ONCE k8s-1 HAS REJOINED.
-  #
-  # Down for the jupiter bare-metal conversion. With one of three nodes out,
-  # the remaining pair have no room: cluster CPU requests exceed two nodes'
-  # capacity, and k8s-3 runs ~5.8 GiB above the kubelet's eviction threshold,
-  # so every pod that does not need to be running is one less thing competing
-  # for both. This one is the best candidate: 500m reserved against a measured
-  # 2m idle draw, and inference bursts hard enough that lowering the request
-  # instead would be the wrong trade. Photo processing pauses; nothing else
-  # depends on it.
-  replicas: 0
+  # Scale this and immich-server to 0 for the duration of any single-node
+  # maintenance — cluster CPU requests exceed what two nodes can hold, and this
+  # pair frees ~1000m against a shortfall of roughly 200m. Stopping them is the
+  # right lever rather than lowering their requests: 500m reserved against a
+  # 2m idle draw looks like waste, but inference bursts hard and the request is
+  # honest about what it needs when someone uploads photos.
+  # See RUNBOOK-baremetal-conversion.md.
+  replicas: 1
   strategy:
     type: RollingUpdate
   selector:

kubernetes/apps/immich/server.yaml

@@ -36,11 +36,10 @@ metadata:
   labels:
     app.kubernetes.io/name: immich-server
 spec:
-  # TEMPORARY — RESTORE TO 1 ONCE k8s-1 HAS REJOINED. See the note on
-  # immich-machine-learning; immich is offline for the jupiter conversion.
-  # Its Postgres cluster is deliberately left running, so nothing needs
-  # restoring beyond these two replica counts.
-  replicas: 0
+  # Scaled to 0 alongside immich-machine-learning during single-node
+  # maintenance; see the note there. The CNPG Postgres cluster stays running,
+  # so recovery is just these two replica counts.
+  replicas: 1
   strategy:
     type: Recreate
   selector:

kubernetes/infrastructure/rook-ceph-cluster/cephcluster.yaml

@@ -185,9 +185,10 @@ spec:
     useAllNodes: false
     useAllDevices: false
     nodes:
-      - name: k8s-2
-        devices:
-          - name: nvme0n1
+      # k8s-2 is absent while saturn's OSD drive is swapped and the node is
+      # converted to bare metal. RE-ADD IT once the PM983 is in and the node has
+      # rejoined, naming the device by /dev/disk/by-id as k8s-1 does — bare
+      # metal sees three NVMe drives and the bare name is not the OSD.
       - name: k8s-3
         devices:
           - name: nvme0n1

kubernetes/infrastructure/rook-ceph-cluster/storageclass.yaml

@@ -87,7 +87,10 @@ metadata:
   name: cephfs
   namespace: rook-ceph
 spec:
-  # TEMPORARY — REVERT TO size 3 / no min_size OVERRIDE ONCE k8s-1 IS BACK.
+  # TEMPORARY — RESTORE TO size 3 / no min_size OVERRIDE ONCE SATURN AND
+  # NEPTUNE HAVE BOTH BEEN CONVERTED TO BARE METAL. It stays at 2/1 across all
+  # three conversions rather than being toggled per node: each change
+  # re-replicates the pool, and the window reopens with the next node anyway.
   #
   # The standing trade for this pool is size 3: it holds ~253 MiB against the
   # data pool's 280 GiB, so it contributes nothing measurable to drive wear —
@@ -96,13 +99,14 @@ spec:
   # quarter of a gigabyte to keep that intact is the right trade in steady
   # state.
   #
-  # It does not hold while one of three hosts is out for the jupiter bare-metal
-  # conversion. size 3 on two remaining hosts leaves every PG undersized at two
-  # copies with min_size 2, so a single further OSD loss drops the pool below
-  # min_size and blocks IO on every CephFS mount cluster-wide. At 2/1 the same
-  # two copies are active+clean instead, and a further loss degrades rather
-  # than blocks — availability bought at the cost of accepting writes onto a
-  # single copy while degraded.
+  # It does not hold while one of three hosts is out. size 3 on two remaining
+  # hosts leaves every PG undersized at two copies with min_size 2, so a single
+  # further OSD loss drops the pool below min_size and blocks IO on every
+  # CephFS mount cluster-wide. At 2/1 the same two copies are active+clean
+  # instead, and a further loss degrades rather than blocks — availability
+  # bought at the cost of accepting writes onto a single copy while degraded.
+  #
+  # See RUNBOOK-baremetal-conversion.md.
   metadataPool:
     replicated:
       size: 2