The single pve scrape job hit all three node IPs with cluster=1, and
pve-exporter returns the full cluster-wide resource set regardless of
which node answers - every node/guest series landed in the TSDB three
times (same id, differing instance). Dashboards papered over it with
max by (id); the cardinality cost stayed.
Split per upstream prometheus-pve-exporter guidance: cluster collectors
scrape one node only, node-local collectors keep scraping all three
under a new pve-node job. No dashboard queries filter on job=, and all
already aggregate by id, so this is query-compatible.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every panel was dead: all 11 datasource refs pointed at the
prometheus-legacy datasource (retired 2026-05-09). Repoint to the
active prometheus datasource.
The alloy job also carries no 'host' label, so all legends and the
Agent Versions table rendered empty labels. Join via kube_pod_info
(pod -> node) so DaemonSet panels are labelled k8s-1/2/3, collapse
the eval-time histogram per pod before quantile, and tidy the
versions table to node/pod/version columns.
All queries verified against live Prometheus (3 agents, one clean
series per node).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All 10 panels were dead: the dashboard queried datasource uid
prometheus-legacy (prometheus.srv.renner.uno:9090), which no longer
resolves (Grafana proxy: 502 dial tcp lookup i/o timeout). Gitea now
runs in-cluster and is scraped by the in-cluster Prometheus
(job="gitea", instance gitea.apps.svc.cluster.local:3000), so switch
every panel to uid prometheus.
Also wrap all targets in max(): the pre-migration instance
gitea.srv.renner.uno:3000 stopped 2026-07-12, and windows spanning
that date would otherwise render two identically-named series per
query. Stat panels additionally get instant:true so a dead exporter
shows No data instead of a stale last value.
- Historical Cost Comparison: the chained joins renamed _value columns to
_value_j2/_value (not _value_gi/_value_sp as the map expected), so the
solar ratio was always null->0 and the whole cost line was priced at full
grid price (EUR 4.67K instead of ~3.4K). Streams now pre-rename columns
to h/d/gi/ge/sp so the join is unambiguous.
- Blended price now uses counter deltas since heat-pump install (Oct 2024)
instead of device-lifetime counters that include 1.4 years of PV-only
history from May 2023 (ratio 0.369 vs 0.322 -> EUR 0.236/kWh vs 0.248).
Applied consistently to all cost/savings/ROI stats and the cost chart
(chart subtracts install-date counter baselines).
- Annual savings / payback: replace the calendar-YTD extrapolation
(12 / (month + day/30) - which overcounts elapsed time by a month and is
seasonally biased) with the actual trailing-365-day consumption spread.
- Fix stale legend 'Last' values (10734/1230/EUR 9.47K): the counters'
friendly_name tag changed 2026-03-04, splitting each series into two
tables; group()+sort() merges them before aggregateWindow.
- New oil_baseline dashboard variable (6650) replaces six hardcoded copies.
- Running Cost: neutral color (was green/yellow/red on a forever-growing
cumulative), retitled '(Since Install)'; added descriptions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- App Servers Up: job="authentik" matches nothing (real job label is
authentik-server-metrics) -> panel showed 'No data'. Also 'or vector(0)'
+ red threshold so a dead scrape shows a red 0 instead of No data.
- Task/Admin Workers, Flows/Policies Cached: raw gauges return 4 series
(2 pods x 2 pids) -> stats rendered 8 repeated numbers. Wrapped in max().
- Outposts Connected: raw per-outpost/per-pid series rendered an unreadable
grid of ~30 0/1 values (0s were per-process views, not real disconnects).
Now counts outposts with >=1 active WebSocket connection (10/10).
- Outpost Connections: legendFormat used non-existent labels {{name}}/{{uid}}
(real: outpost_name/uuid) -> every legend row was '-'. Replaced with
server-side authentik_outposts_connected as a state-timeline, one row per
outpost, red mapping for 'getrennt'; covers all 10 outposts instead of the
4 with scraped sidecar metrics.
- All stat queries switched to instant so dead sensors render honestly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Nodes Online/Down used job="node" but the scrape job is "node-exporter":
both stats showed No data. Nodes Down additionally gets OR on() vector(0)
so it renders 0 instead of No data when all nodes are up, and green-at-0/
red-at-1 thresholds (was permanently red).
- Avg Disk Usage + Disk Usage by Node filtered mountpoint="/" which does not
exist on Talos (only /var is a real xfs filesystem): both panels were empty.
- Disk I/O summed all block devices including the ~12 Rook-Ceph rbd* mappings
per node (~438 kB/s vs 8-129 kB/s physical): filter to nvme.*|vd.*.
- Network I/O filter was written for a Docker host (veth/docker/br-) and let
~40 Cilium lxc* per-pod interfaces plus kernel pseudo-devices through,
double-counting pod traffic: filter to eth.* (bond0 is down, 0 slaves).
- Legends showed instance IP:port; join node_uname_info via
on(instance) group_left(nodename) so panels label k8s-1/2/3. Both join
sides are wrapped in max by (...) so historical duplicate series from
pod restarts / the pre-2026-07-16 duplicate-scrape residue cannot break
the join or rainbow-duplicate long-window views.
- Stat panels reduce lastNotNull instead of mean: show current value, not
a time-average over the selected window.
All queries verified against live Prometheus before applying.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dashboard was 100% dead: every panel still referenced the
prometheus-legacy datasource retired on 2026-05-09, so all panels
rendered 'No data' with datasource errors.
- Repoint all panels prometheus-legacy -> prometheus
- Remove TCP Endpoint Status panel: the blackbox-tcp Probe was
deliberately deleted 2026-05-20 (see probes-blackbox.yaml), the job
no longer exists
- Remove DNS Status panel: no blackbox-dns job has ever existed in
this Prometheus (the dns-servers job is Alloy on the DNS VMs, not a
probe)
- Replace the freed row with HTTP Status Codes (probe_http_status_code,
instant, 2xx green / 3xx orange / 4xx+ and probe-fail red)
- Convert SSL Certificate Expiry from a seconds-countdown timeseries to
a days-remaining bar gauge (instant, red <7d, orange <30d, green)
- Make status stat queries instant so dead probes render No data
instead of a stale last-known value
- Pin Uptime y-axis to 0-100% and drop the fill (10 overlapping series
at 100% rendered as a muddy block)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pve-exporter is scraped via all 3 PVE nodes and each scrape returns the
full cluster view, so every series exists in triplicate. All stat counts were
3x too high (Online Nodes showed 39 instead of 3) and every bargauge/table
row appeared three times. Wrap all queries in max by (id) / count by (id).
- Online Nodes: count unique node/* ids with pve_up==1 (was 39, now 3);
threshold green at 3 so a lost node turns red
- Total VMs: count unique qemu guests excluding templates (was 12, now 3)
- Total LXC: count unique ids (was 9, now 3)
- Running Guests: pve_guest_info has no status label in this exporter
version (panel showed No data); derive from pve_up==1 per guest instead
- Node CPU/Memory bargauges + timeseries: filter to id=~"node/.*" — they
showed every guest (mostly lxc rows), not nodes
- Guest Status table: dedupe rows, add Status column from pve_up (color
mapped 1=Running/0=Stopped), drop useless Info column, migrate
custom.displayMode -> custom.cellOptions (schemaVersion 38 skips the
Grafana 10 migration, so gradient gauges/color background were silently
ignored)
- Node Disk I/O -> Guest Disk I/O: pve_disk_read/write metrics only exist
for qemu/lxc guests, never node/* (panel was permanently No data); use
the *_total counters with rate()
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dashboard was entirely dead: every panel queried postgres_exporter
metrics (pg_up, pg_stat_activity_count, ...) that have never existed in
this Prometheus. Postgres runs as four CloudNativePG clusters (ak-pg,
im-pg, gf-pg, pprl-pg) whose built-in exporter on :9187 was not scraped
at all (enablePodMonitor is false and no PodMonitor existed).
- add PodMonitors for the four CNPG clusters in monitoring-rules,
labelled release=kube-prometheus-stack so the Prometheus
podMonitorSelector actually picks them up (CNPG's own
enablePodMonitor would generate unlabelled PodMonitors that this
Prometheus ignores)
- rewrite all dashboard queries to the cnpg_* metrics the exporter
actually exposes (verified against the pods' /metrics endpoints)
- dedupe multi-instance clusters (max by namespace/datname for sizes,
primary-only filter via cnpg_pg_replication_in_recovery == 0 for
per-database counters)
- replace the Active Locks stat (no lock metric in the CNPG default
queries) with Max Replication Lag
- instant queries on all stat panels so dead instances show No data
instead of a stale last value
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Healthy Hosts: job 'integrations/unix' does not exist here -> node-exporter;
thresholds now red/yellow/green(3) so a single down node shows yellow
- Unhealthy Services: 'count(up == 0)' returns empty vector when healthy ->
'or vector(0)' renders an honest 0; both stats now instant queries
- CPU: 'avg by(host)' collapsed all 3 nodes into one series (no host label) ->
avg by(instance) joined with node_uname_info for k8s-1/2/3 legend names
- Memory: legend showed raw label dumps ({{host}} does not exist) -> nodename join
- Disk: Talos has no '/' mountpoint in node_filesystem -> use /var (the
persistent data disk); retitle panel accordingly
- Cumulative Value: Self-Use Savings and Total Value series rendered nothing.
The union'd solar/export/price tables had mismatched group keys (pivot ran
per-table) AND the sparse strompreis series almost never landed in the same
aggregation window. Fix: group() before pivot + fill(usePrevious) per column.
Verified via /api/ds/query: 471 points, final value EUR 4363 == stat panel.
- Solar Power stat: was aggregateWindow(mean) over the dashboard range (~17 h
windows at 3 y), showing 3.04 kW at midnight. Now last() -> honest 0 W at night.
- Solar Installed: HA input_number stores 8140 (Wp, mislabeled kWp); with unit
kwatt the panel would read 8.14 MW. Divide by 1000 -> 8.14 kW, documented.
- Timeseries gap honesty: e3_* series have a real data hole (Nov 2025 - Feb 2026,
verified 0 points/month). createEmpty:false let Grafana draw fake decay ramps
across it. Now createEmpty:true so gaps render as gaps (panels 20/21/22).
- Units: 'kwh' is not a valid Grafana unit id (rendered lowercase 'kwh');
replaced with 'kwatth' everywhere -> proper kWh/MWh scaling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Current by Phase: regex /smart_meter.*current/ matched nothing (smart
meter exposes no current entities) -> use verified
e380_energy_meter_e380_current_l[123]; spanNulls for sparse updates
- Energy Consumption: measurement "kWh" does not exist in the bucket
-> plot verified Wh daily counters (grid import/export, PV production,
battery charge/discharge) with readable display names
- Active Lights: light-domain data is never recorded under measurement
"%" -> replace dead table with Lights/Standby/Switches power
timeseries from verified powercalc aggregates (all_*_power)
- Top 20 Devices: add group() so sort/limit work globally (was
alphabetical, unsorted, per-table); exclude grid meters, PV/battery,
forecasts, powercalc room/floor aggregates and _device_power twins;
fresh 30-min window instead of stale last() over the full range;
rowsToFields for clean labels (was "_value xyz_power")
- Grid semantics: retitle Total Power -> Grid Power (+Import/-Export),
blue threshold for export; unstack mixed-sign phase graph; short
L1/L2/L3 legend names; taller voltage/current panels so the legend
table is not cut off
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Alarm stats: '== 1' dropped the series when false, so Armed Home/Away and
Triggered could only ever show 'No data' and never their 0-state; Disarmed
rendered 5 overlapping series. Use min()/max() aggregation over the one-hot
alarm_control_panel_state series so every tile always shows a real 0/1.
- Windows/Doors tables: replace friendly-name regexes (matched battery-low
sensors, HmIP 'Window/door is CLOSED' entities with inverted semantics,
intrusion/tamper diagnostics, _2 twins, pond-pump/fridge sensors) with a
curated entity list: one canonical is-open entity per physical sensor.
Collapse scrape labels via max by so the tables show Sensor/Name/Status.
- Motion panels: old regex required 'Bewegungs.*Occupancy' and missed the
Eingang sensor (typo 'Bewegunsmelder'), all motion-class sensors and the
mmWave Praesenzmelder. Curated list of physical detectors.
- Covers: name-based exclusion (.*Nord.*|.*Ost.* etc.) dropped 8 physical
covers incl. the garage door while letting the eg_schlafzimmer/eg_kuche
group entities through. Exclude the 5 group covers by entity instead;
label bars by entity because friendly names repeat ('Rollo' x3).
- Batteries: filter battery_notes '+' twin entities, drop PV storage SoC,
remove '> 0' (which would hide a 0% dead battery), sort ascending so the
most critical batteries are on top, instant query so stale values drop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The template variable queried label_values(node_uname_info, host), but no
node-exporter metric in this Prometheus carries a 'host' label (that label
only exists on blackbox-probe metrics). The variable resolved to empty and
all five panels rendered blank.
- Replace $host with a visible $node variable (nodename: k8s-1/2/3) plus a
hidden chained $instance variable (nodename -> instance via node_uname_info)
- Filter Cilium pod interfaces (lxc*, cilium_*) and always-zero kernel
pseudo-devices (ip6tnl0, sit0, teql0, tunl0) out of Network I/O
- Filter Ceph RBD-mapped volumes (rbd0..11) out of Disk I/O so the panel
shows physical disks (nvme0n1, vda) only
- Exclude the rootfs bind mount (/etc/extensions.yaml) from Filesystem
Usage, leaving the meaningful /var xfs mount
All queries verified against live Prometheus before commit.
- All temperature/humidity panels: replace vendor regexes (shelly.*temp etc.)
that dragged in device-internal temperatures (pond pump 34C, power meter 60C,
relay 76C) with explicit allow-lists of verified room sensors
- Add missing sensors: shelly_blu_h_t_7203 (KG Schlafzimmer),
treppenhaus_eg_bewegungsmelder (EG Treppenhaus), vento humidity sensors,
Shelly Wall Display (DG Buero)
- OG & DG panel: regex had no og_ alternative and showed only device internals;
now honest DG-only series + description (OG deliberately has no sensor)
- Heating System / Hot Water panels were empty (domain climate/water_heater
does not exist in the C measurement) -> use vitocal_250_a_* sensors
- Current-value panels: fixed -6h window so dead sensors (EG Shelly H&T trio,
DG Meter Pro) disappear instead of showing stale values as live
- Legends: map entity_ids to room names in Flux + displayName, removes the
'_value xyz' prefix
- Align CO2 thresholds between gauge and history (800/1000/1500)
- Server node: sensor.keller_server_power is dead (UPS retired) -> Wave PM
Mini server outlet (kg_vorratskeller_wave_pm_mini_server_steckdose)
- Garden node: sensor.eg_garten_teichpumpe_power_2 is dead ->
sensor.shelly_teichpumpe_power_0 (live, peaks ~77 W)
- Home node/series: floor sums only cover submetered circuits (702 W shown
vs ~1.1 kW real); Home is now the meter balance PV + Grid + Battery, with
a new grey 'Rest' node for the non-submetered remainder
- Flow lines: Heat Pump is metered inside Keller (was drawn from EG);
Garden is on floor Aussenbereich, in no floor sum (was drawn from OG)
- Drop redundant batteryIn target (derived from the signed battery sensor);
update stale comments from the retired *_absolute sensors
- History: max by (entity) guards against duplicate-scrape residue that
doubled the summed Home series; sign hints in Battery/Grid legends;
grid query pinned to entity id instead of friendly_name
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Status panel: job label is home-assistant (hyphen), was homeassistant -> No data
- Doors & Windows / Motion: exporter has no device_class label; select curated
entity lists instead (one canonical is-open entity per physical contact,
physical motion/presence sensors without zone/magic_areas duplicates)
- Temperature: was all 61 temp series incl. device-internal electronics temps
(fridge 5C, PoE switch 75C, Shelly 58-75C); curated to room/ambient sensors
+ outdoor temp, incl. currently-dead EG H&T sensors so they reappear
- Power Consumption: exclude non-consumption series (Solcast forecasts,
battery max charge/discharge capability, derived night-load stats)
- Energy: cumulative lifetime counters (up to 80 MWh) told nothing; now
bargauge of top 12 increase() over 24h, forecast counters excluded,
max by () dedup against scrape residue
- Switches: 248 unreadable micro-tiles -> single 'Switches On' count stat
- Battery: bottomk(12) instant, battery_notes *_battery_plus twins excluded
- Entity counts: max by (entity) dedup, instant queries on all stats
- Unavailable Entities: neutral blue; ~1139 is the structural baseline
(verified against HA API), old yellow>=1/red>=5 thresholds were meaningless
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All queries selected raw series by entity label only; over a 90d window HA pod
restarts plus the (since fixed) duplicate proxy scrape produce 30 distinct
series per entity, rendering the Außentemperatur & Puffer panel as an
unreadable 30-entry legend jumble. Wrap every query in max() so exactly one
series per entity survives — this also keeps the xychart joinByField transform
correct once heating data arrives, since the join needs unique field names.
The stat panel now says 'keine Heizdaten im Zeitraum' instead of a giant
'No data': the logging sensor is intentionally unavailable outside active
heating loading, so an empty summer dashboard is by design.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All four panels queried raw HA metrics that still carry ~12 duplicate
series per entity in the 90d window (two scrape jobs x six pod
lifetimes; the duplicate-scrape bug was fixed 2026-07-16 but the
residue stays in range queries until it ages out). This broke the
scatter join (no points rendered), made the stat panel show 8
conflicting averages, and drew rainbow duplicates in both time series.
Wrapping every target in max by (entity) (...) collapses the residue
to one series per entity; values between the duplicate jobs were
near-identical, so max is lossless.
- Windows/Doors/Batteries tiles: literal ${__value.raw} never interpolates in
value mappings; replaced range mappings with suffix units. Counts now use
curated entity lists (one canonical 'is open' entity per physical sensor)
instead of friendly-name regexes that matched inverted *_is_closed sensors,
'Battery low' sensors and duplicate _2 entities.
- Alarm tile: old min(...==1) query could never show TRIGGERED; now
max(...{state="triggered"}) with 0=OK/1=TRIGGERED mapping.
- Power Usage: old regex summed phases A+B+C AND total_active_power AND
device_power -> ~2x real value; now reads total_active_power only.
- Floor temperature panels: select verified room sensors by entity id instead
of vendor regexes (which pulled 65-75C device-internal temps and missed the
I-9PSL sensors that live in the EG). Instant queries so dead sensors do not
render stale values. OG: no room sensor exists (owner-confirmed) -> honest
noValue; DG SwitchBot offline -> noValue 'Sensor offline'.
- Tables: dedup via max by (entity, friendly_name), drop k8s scrape label
columns, colour only the value column; battery list excludes battery_notes
*_battery_plus twin entities and no longer hides 0% batteries.
- Power Consumption: L1/L2/L3 legends, Total from total_active_power.
- Indoor Temperatures: room sensors only (y-axis no longer squashed to 80C).
- Heating row: Hot Water/Heating pointed at post-open3e-migration vitocal
sensors (old queries returned nothing resp. only the fridge); humidity/CO2
deduplicated; row renamed to Heating & Air Quality.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The union-based timeseries series were named via set(_field, ...), which Grafana's InfluxDB frontend rendered as "_value {_start=...}" instead of the intended labels (and broke the byName color overrides). Data was correct; only the legend/colors were affected.
Switch to the keep(columns:[_time,_value]) |> rename(_value -> "Name") pattern (as used in heatpump-roi), which names each series cleanly. Verified in Grafana Explore. No data/logic change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiLCinB7NLp7YdCjGYzvfh
The 2026-07 ViCare->open3e/MQTT migration renamed the Vitocharge PV/battery entities (e3_vitocharge_03_* -> vitocharge_vx3_vx3_*) and changed some units. Both dashboards still queried the retired entities and showed No data.
power-flow (Prometheus, real-time): solar_power->pv_power, battery SoC->battery_soc; the two absolute battery-power sensors are replaced by a single signed battery_power (open3e: positive=discharge, negative=charge), so charge/discharge derive from its sign.
pv-system-roi (InfluxDB): repoint every query; fix units (PV/battery energy now Wh not kWh -> /1000; grid energy now kWh not Wh); grid power sign unchanged (negative=export). Timeseries panels UNION pre-migration e3_* with post-migration series (unit-normalized) so history back to 2023 stays unbroken. Reserved Energy recomputed as capacity x backup_discharge_limit. strompreis + the two input_number config entities survived and are kept.
Validated live via Grafana Explore: signed-battery charge/discharge union, unit normalization, pivot/value patterns (reused from merged heatpump-roi fix#271).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiLCinB7NLp7YdCjGYzvfh
Both destroy applies (runs 1180/1181) died instantly with 'context
deadline exceeded' on the pre-destroy status GET while the PVE API
answers in <1s: the bpg provider reads timeout_delete from STATE at
destroy time, and the stage-1 import stored 0 for the timeout-less
shims — a zero-second context. Read paths carry their own timeout,
which is why import and plan were clean.
v2 declares all timeouts explicitly (one in-place, no-API update
writes them into state) and mirrors the real container attributes
(unprivileged=true is ForceNew — mirroring prevents a replace).
No ignore_changes this time: the timeout update must actually apply.
Flow: merge -> plan (must be in-place only) -> apply -> follow-up PR
deletes the file -> destroy apply.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
The ViCare cloud integration was replaced by open3e/MQTT (2026-07), renaming all heat-pump/PV entities (e3_vitocal_*/e3_vitocharge_03_* -> vitocal_250_a_vitocal_*/vitocharge_vx3_vx3_*). The dashboard still queried the retired IDs, and a duplicate Prometheus scrape broke the surviving-entity panels with many-to-many errors.
Rebuild every panel on InfluxDB (bucket suedstr28, authoritative + live) using current entity IDs. Where an entity was renamed, UNION the pre-migration e3_* series with the post-migration series (unit-normalized: PV Wh->kWh) so history stays unbroken. Drop all Prometheus targets. Add a grid_price variable (0.3271) replacing the retired live strompreis read.
Also rename the home-assistant-bgp Service port http->ha so the home-assistant ServiceMonitor no longer scrapes HA a second time via the LB (:8123), which duplicated homeassistant_sensor_* series cluster-wide.
Queries validated live via Grafana Explore: blended price 0.248 EUR/kWh, total consumption 14284 kWh, projected annual savings 4698 EUR, oil proration 11912 EUR, old-union-new grid-import continuous.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiLCinB7NLp7YdCjGYzvfh
Stage 1 (#268) adopted the never-tofu-managed rustfs containers into
state (import run 1173: all three imported, plan 'No changes').
Removing the shims makes the next apply destroy the containers —
guest lifecycle fully through tofu, per owner rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
The live 15-min snapshot structurally misses anything that fires and
resolves between ticks: on 2026-07-16 a 17-minute KubeHpaMaxedOut
(authentik) and last evening's four-alert home-assistant incident were
invisible to the monitor (owner noticed the blips; monitor did not).
The fetch step now also queries Prometheus's ALERTS history over a
35-minute window (~2 ticks + jitter). The dedup step synthesizes
pseudo-alerts for series not currently firing at all — checked against
the PRE-filter snapshot so live-but-null-routed alerts don't reappear
as blips — with a stable sha256-derived fingerprint so one noise
verdict suppresses a flappy alert permanently. Blips then flow through
the same title/noise dedup and triage as live alerts; the prompt
explains their semantics (self-resolved; judge the underlying cause;
null-routed per the repo route config = noise).
Verified against live data: currently-firing KubeCPUOvercommit
excluded, injected KubeHpaMaxedOut blip synthesized correctly;
sha256sum confirmed present in ci-claude; lookback query tested via
the service proxy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
Owner rule: ALL guest lifecycle goes through tofu; no manual pct/qm
destroy, by anyone. The rustfs trio (800/801/802) was never
tofu-managed, so the #265 sweep could not reach it. Stage 1 adopts the
stopped containers into state (pipeline TOFU_IMPORT hook, commands in
the file header); stage 2 deletes this file and the apply destroys
them.
lifecycle ignore_changes=all + started=false keep the shims inert:
tofu must never reconcile or start these, only own their destruction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
k.renner
claude-botandClaude Fable 5
security-bot