claude-botandClaude Fable 5.1
6f9ff9d624
A DaemonSet running clambin/intel-gpu-exporter (intel_gpu_top behind a Prometheus endpoint) with a ServiceMonitor, giving gpumon_engine_usage, gpumon_frequency, gpumon_power, gpumon_imc_* and gpumon_clients_count per node. Privilege model, validated with a one-shot pod before this commit: /dev/dri comes from the device plugin's gpu.intel.com/monitoring resource (every GPU on the node, no hostPath); the i915 PMU that intel_gpu_top reads is a system-wide event source and the nodes run perf_event_paranoid=3, so CAP_PERFMON is the one capability added. No privileged, no hostPID, no hostNetwork, read-only root filesystem. Root rather than a fixed uid because the capability has to be effective without file capabilities on the binary. End-to-end check on k8s-2 during a 40 s chat completion on llama-server: Render/3D busy 98 %, actual frequency 1449 MHz, GPU power 18.8 W, while the pod's CPU stayed near 100m — the "is it really on the GPU" question this exists to answer. XPU Manager was ruled out (Arc Pro / data-centre parts only); the archived onedr0p exporter needs privileged + hostPID. Fixes #752 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
28 lines
1.1 KiB
YAML
28 lines
1.1 KiB
YAML
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
- grafana-oauth-externalsecret.yaml
|
|
- alertmanager-smtp-externalsecret.yaml
|
|
- helmrelease-prometheus.yaml
|
|
- helmrelease-loki.yaml
|
|
- helmrelease-alloy-syslog.yaml
|
|
- helmrelease-blackbox-exporter.yaml
|
|
- loki-basic-auth-externalsecret.yaml
|
|
- loki-basic-auth-middleware.yaml
|
|
- loki-ingress.yaml
|
|
- alertmanager-gate-service.yaml
|
|
- alertmanager-gate-ingress.yaml
|
|
- prometheus-gate-service.yaml
|
|
- prometheus-gate-ingress.yaml
|
|
- ci-diagnostics-rbac.yaml
|
|
- smartctl-exporter.yaml
|
|
- intel-gpu-exporter.yaml
|
|
- dashboards/
|
|
# prometheusrule-garage.yaml lives in ./monitoring-rules/ and is applied by a
|
|
# separate Flux Kustomization that dependsOn `monitoring` — otherwise the
|
|
# server-side dry-run of the PrometheusRule fails on a fresh cluster
|
|
# ("no matches for kind PrometheusRule") because the CRD is provided by
|
|
# kube-prometheus-stack's chart, which we're applying *in this same
|
|
# Kustomization*. The split lets the chart install (CRDs and all) before
|
|
# Flux tries to admit the rule.
|