feat(rook-ceph): restore cephfs-metadata to size 3 #609

Merged
k.renner merged 1 commits from restore-cephfs-metadata-size into master 2026-08-26 00:10:38 +00:00
@@ -87,31 +87,21 @@ metadata:
name: cephfs
namespace: rook-ceph
spec:
# TEMPORARY — RESTORE TO size 3 / no min_size OVERRIDE ONCE SATURN AND
# NEPTUNE HAVE BOTH BEEN CONVERTED TO BARE METAL. It stays at 2/1 across all
# three conversions rather than being toggled per node: each change
# re-replicates the pool, and the window reopens with the next node anyway.
# size 3 with no min_size override. This pool holds ~253 MiB against the data
# pool's 280 GiB, so it contributes nothing measurable to drive wear — the
# entire reason data0 sits at 2 — while losing it would destroy the whole
# filesystem, not just the objects on the lost OSD. Paying 3x on a quarter of
# a gigabyte to keep that intact is the right trade.
#
# The standing trade for this pool is size 3: it holds ~253 MiB against the
# data pool's 280 GiB, so it contributes nothing measurable to drive wear —
# the entire reason data0 dropped to 2 — while losing it would destroy the
# whole filesystem, not just the objects on the lost OSD. Paying 3x on a
# quarter of a gigabyte to keep that intact is the right trade in steady
# state.
#
# It does not hold while one of three hosts is out. size 3 on two remaining
# hosts leaves every PG undersized at two copies with min_size 2, so a single
# further OSD loss drops the pool below min_size and blocks IO on every
# CephFS mount cluster-wide. At 2/1 the same two copies are active+clean
# instead, and a further loss degrades rather than blocks — availability
# bought at the cost of accepting writes onto a single copy while degraded.
#
# See RUNBOOK-baremetal-conversion.md.
# It drops to 2/1 whenever one of the three hosts is out for longer than a
# reboot. size 3 on two remaining hosts leaves every PG undersized at two
# copies with min_size 2, so a single further OSD loss falls below min_size
# and blocks IO on every CephFS mount cluster-wide. At 2/1 the same two
# copies are active+clean instead, and a further loss degrades rather than
# blocks. See RUNBOOK-baremetal-conversion.md.
metadataPool:
replicated:
size: 2
parameters:
min_size: "1"
size: 3
dataPools:
- name: data0
# size 2 / min_size 1 for the same reasons as replicapool above — see