kubernetes/apps/monitoring/smartctl-exporter.yaml

@@ -16,17 +16,19 @@
 # healthy drives and not the worn ones was judged not worth a new service; the
 # owner chose fleet-wide coverage with the privilege exception instead.
 #
-# The exception is kept as small as the kernel allows: no `privileged: true`,
-# no host namespaces, no hostPID. Only CAP_SYS_ADMIN and CAP_SYS_RAWIO are
-# added, everything else is dropped, and the only host mount is /dev — which
-# is required because the exporter must open the NVMe character devices and
-# Kubernetes has no way to pass individual character devices to a pod without
-# a device plugin.
+# The exception is `privileged: true`, and narrower settings were tried and
+# measured to fail — see the securityContext below for what was ruled out and
+# how. It is still bounded: no host namespaces, no hostPID, no hostNetwork,
+# read-only root filesystem, and /dev as the only host mount.
 #
-# If this ever needs revisiting: the thing to check is whether the kernel has
-# grown a sysfs interface for NVMe health, not whether some other exporter
-# needs fewer privileges. The requirement comes from the ioctl gate, not from
-# the tool.
+# If this ever needs revisiting, there are exactly two things worth checking,
+# and neither is "does another exporter ask for less":
+#   - whether the kernel has grown a sysfs interface for NVMe health, which
+#     would remove the need for device access altogether
+#   - whether enough workloads need raw device access to justify a device
+#     plugin, which is the only way Kubernetes grants individual devices
+# The requirement comes from the device cgroup and the ioctl gate, not from
+# the choice of tool.
 #
 # Upstream had no release between 2025-04-22 (v0.14.0) and this deployment.
 # The repository is not archived; the pin is simply its newest tag.
@@ -66,38 +68,52 @@ spec:
               containerPort: 9633
               protocol: TCP
           securityContext:
-            # Root plus two capabilities, NOT privileged. See the header.
+            # PRIVILEGED, and there is no narrower setting that works. This
+            # was established by measurement after two narrower attempts
+            # failed, so the reasoning is recorded here rather than rediscovered:
+            #
+            #   Attempt 1 — CAP_SYS_ADMIN + CAP_SYS_RAWIO, everything else
+            #   dropped. The exporter found all three drives per node and
+            #   could open none: "Smartctl open device: /dev/nvme0 failed:
+            #   Operation not permitted".
+            #
+            #   Attempt 2 — the same plus seccompProfile: Unconfined, on the
+            #   theory that EPERM is the default seccomp action for an
+            #   unlisted syscall and the kubelet runs seccompDefault: true.
+            #   /proc/1/status showed CapEff 0x220000 and Seccomp 2, so both
+            #   capabilities were effective and a filter was loaded. Removing
+            #   the filter changed nothing.
+            #
+            #   What actually settled it: `head -c 512 /dev/nvme0n1` inside
+            #   the container fails with EPERM too. A plain read of a block
+            #   device involves no ioctl, no capability and no syscall a
+            #   seccomp profile would filter. The block is the device cgroup —
+            #   containerd denies non-privileged containers access to host
+            #   device nodes whether or not /dev is mounted.
+            #
+            # Kubernetes exposes no per-device allowlist in a Pod spec; there
+            # is no equivalent of `docker --device`. The alternatives are
+            # `privileged: true` or a device plugin advertising the NVMe
+            # drives as a resource. A device plugin is the genuinely narrower
+            # answer and is worth building if more workloads ever need raw
+            # device access; for a single read-only metrics exporter it is a
+            # component to write, deploy and maintain in exchange for the same
+            # access.
             runAsUser: 0
             runAsNonRoot: false
-            privileged: false
-            allowPrivilegeEscalation: false
+            privileged: true
+            # Stated rather than omitted: the API rejects `privileged: true`
+            # together with `allowPrivilegeEscalation: false`, so leaving it
+            # out would only hide that privileged already implies it.
+            allowPrivilegeEscalation: true
             readOnlyRootFilesystem: true
-            capabilities:
-              drop: ["ALL"]
-              add: ["SYS_ADMIN", "SYS_RAWIO"]
-            # The capabilities alone are not enough, and this was measured
-            # rather than assumed. With the default profile the exporter found
-            # all three drives and could open none of them:
-            #   "Smartctl open device: /dev/nvme0 failed: Operation not
-            #   permitted"
-            # EPERM is the default seccomp action for a syscall the profile
-            # does not list. Reading /proc/1/status in the running container
-            # separated the two candidates: CapEff was 0x220000, so SYS_ADMIN
-            # and SYS_RAWIO were both effective — while Seccomp was 2 with one
-            # filter loaded. The kubelet sets seccompDefault: true, so every
-            # pod that does not say otherwise gets RuntimeDefault.
-            #
-            # This is still narrower than `privileged: true`, which would also
-            # grant every capability, all devices and write access to /sys.
-            # Narrowing further would mean shipping a custom seccomp profile
-            # that allows exactly the ioctl smartctl needs — worth doing if
-            # this list of exceptions ever grows, not worth it for one pod.
-            seccompProfile:
-              type: Unconfined
           volumeMounts:
+            # Not read-only. smartmontools opens NVMe devices read-write on
+            # some paths, and once the container is privileged a read-only
+            # flag on this mount is not a boundary anyone should rely on —
+            # keeping it would suggest a protection that is not there.
             - name: dev
               mountPath: /dev
-              readOnly: true
             # smartctl is not known to need scratch space, but the root
             # filesystem is read-only and a tool that turns out to want /tmp
             # would crashloop instead of degrading. Cheaper to provide it than