influxdb(seed): drop chunk size to per-day

806e979 (per-month) still EOF'd at 2024-04 — first dense month for HA
telemetry, pivot of one month at that density overwhelms legacy's
memory. Drop to per-day chunks (~3300 iterations across 9 years,
mostly empty days no-op'ing in <1 s; dense days under 1 MB pivoted).
This commit is contained in:
2026-05-05 01:29:00 +02:00
parent 806e9794ad
commit 106d5516b6
+11 -10
View File
@@ -105,13 +105,15 @@ spec:
# columns; `drop()` strips the _start/_stop range columns
# that `influx write` rejects.
#
# Why chunked, and why monthly: pivoting in-memory in
# Why chunked, and why daily: pivoting in-memory in
# legacy means the whole result set has to fit before it
# flushes. The previous per-year shape EOF'd on the
# current year (full year of HA telemetry won't pivot in
# legacy's RAM). Per-month windows keep each pivot to
# tens of MB regardless of year, so legacy never OOMs.
# Empty months are no-ops (~1 s each).
# flushes. Per-year EOF'd on the current year; per-month
# also EOF'd on dense months (HA-telemetry from 2024+ is
# too big at one-month granularity). Per-day windows keep
# each pivot tiny regardless of write rate, so legacy
# never OOMs. Empty days are no-ops (~1 s each); the
# outer loop is ~3300 iterations across 9 years but most
# complete in <1 s.
#
# No script-side idempotency: InfluxDB writes are upserts
# on (timestamp,measurement,tagset,field), so a retry
@@ -123,12 +125,11 @@ spec:
# an `influx query` EOF inside the pipe propagates a non-
# zero exit, the loop aborts, the Job retries.
START="2018-01-01T00:00:00Z"
# Stop at the start of the month AFTER current → covers
# the in-progress month inclusively.
END=$(date -u -d "$(date -u +%Y-%m-01) +1 month" +%Y-%m-%dT00:00:00Z)
# Stop at tomorrow → covers today inclusively.
END=$(date -u -d "tomorrow" +%Y-%m-%dT00:00:00Z)
CUR="$START"
while [[ "$CUR" < "$END" ]]; do
NEXT=$(date -u -d "$CUR +1 month" +%Y-%m-%dT00:00:00Z)
NEXT=$(date -u -d "$CUR +1 day" +%Y-%m-%dT00:00:00Z)
log "Streaming $CUR → $NEXT ..."
influx query --raw \
--host "$LEGACY_HOST" \