Splitting the home/docs gating change into two commits because Flux
dry-run rejects the new *-public Ingress as long as the old Ingress
still defines the same host (validating webhook checks against live
state, not the post-apply state).
Step 1: remove home.renner.uno / docs.renner.uno from the existing
Ingresses. Step 2 (next commit): add home-assistant-public /
paperless-public Ingresses with forward-auth annotations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
home.renner.uno and docs.renner.uno now traverse the cloudflared
tunnel and are forced through an Authentik nginx proxy outpost
before reaching the HA / paperless pods.
* apps/authentik/blueprints-configmap.yaml: new "nginx" proxy
outpost. Worker provisions Deployment + Service named
ak-outpost-nginx in the authentik namespace.
* apps/home-assistant/ingress.yaml: split. Existing ingress keeps
home.k.renner.uno (LAN un-gated). New `home-assistant-public`
Ingress for home.renner.uno carries the auth-url / auth-signin /
auth-response-headers / auth-snippet annotations pointing at
ak-outpost-nginx.authentik.svc.cluster.local:9000.
* apps/paperless/paperless.yaml: same split. Existing ingress keeps
paperless.k.renner.uno + docs.k.renner.uno + paperless.renner.uno
(LAN un-gated). New `paperless-public` Ingress for docs.renner.uno
with the same forward-auth annotations.
* infrastructure/cloudflared/configmap.yaml: hostnames home.renner.uno
and docs.renner.uno added; both terminate at ingress-nginx.
Per-app Authentik Application + Proxy Provider (mode =
forward_auth_single, hostname = home.renner.uno / docs.renner.uno)
needs to be created in the Authentik UI; not in the blueprint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Outpost pods can't hairpin to the public ingress-nginx LB IP
(`auth.renner.uno` → `10.2.40.80`); Cilium times out the pod →
LB → ingress flow. Switch each outpost's `authentik_host` to
`http://authentik-server.authentik.svc.cluster.local` and carry the
public URL in `authentik_host_browser` for browser-facing redirects.
Apply the same shape to the LDAP outpost while we're touching it
(its config previously left `authentik_host` at the public URL too).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caddy and Local Radius outposts came over from the Proxmox deployment
with `kubernetes_namespace: default` in their config. The chart-rendered
Role only grants permissions in `authentik`, so the K8s controller's
first GET against `secrets/ak-outpost-{name}` in `default` returned
403. LDAP Outpost worked because its config never had the field set
(falls back to the running pod's namespace).
Pin both to `authentik` via the existing blueprint so the controller
spawns them next to authentik-server.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Authentik DB carried over a `Local Docker connection`
ServiceConnection from the Proxmox deployment, and all four outposts
(Caddy proxy, LDAP, Local Radius, embedded) still pointed at it. The
worker's `outpost_service_connection_monitor` task crashed every cycle
in-pod with `DockerException: ('Connection aborted.',
FileNotFoundError(...))` because the pod has no /var/run/docker.sock.
Ship a Blueprint via `blueprints.configMaps` that reassigns every
Outpost to the auto-created `Local Kubernetes Cluster` connection and
deletes the Docker connection. Authentik's chart-rendered Role already
grants the deploy/svc/ingress/secret/configmap perms needed to spawn
outposts via the K8s API.
While here:
- Grafana OIDC token_url/api_url were still hitting
`http://authentik-server.authentik.svc.cluster.local`, violating the
public-hostname-everywhere rule (split iss claim). Point them at
https://auth.renner.uno/ alongside auth_url.
- seed-authentik comment still described the dropped pg_dump→pg_restore
Job; replace with the current initdb + Garage CNPG recovery model.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cutover is complete; seed Jobs would re-overwrite live cluster state
on every rebuild (e.g. CNPG pg_dump|pg_restore from legacy, rsync
data import, InfluxDB backup-restore from legacy). Remove them along
with the SSH keys and legacy tokens that only existed to feed them.
Kept per app:
audiobookshelf, music-assistant: pvcs only (library lives there).
authentik, immich, paperless: CNPG cluster + objectstore +
scheduled-backup + barman creds
+ (where present) pvcs.
influxdb: chart helmrelease, bootstrap admin
secret, ha-token Job (must run
every cluster lifetime to mint
the HA-readable bucket-scoped
token).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Earlier ra-rio-sysctl init container failed with read-only /proc/sys —
NET_ADMIN alone doesn't grant write, and privileged is off the table by
standing rule. Move the sysctl into the macvlan NAD as a chained
`tuning` plugin: applied during interface creation, before any pod
container sees net1. Drops the failed init from the Deployment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default kernel `accept_ra_rt_info_max_plen=0` drops every RA route-info
entry. OTBR advertises the Thread OMR prefix (fd5f:…/64) to v30iot as
an RIO, so matter-server's net1 silently rejected it and had no route
to Thread devices — CASE sessions hit UDP retransmission timeouts
despite mDNS discovery succeeding. Add an initContainer with NET_ADMIN
that bumps the limit to 64 on net1/all/default before the main
container starts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default --vendorid=0xFFF1 / --fabricid=1 caused matter-server to allocate
a fresh fabric on first start and load nodes from
16583007271343510012.json (empty), reporting "Loaded 0 nodes" even though
chip.json's caList plus the 425 KB 10489438502500755905.json fabric file
on disk hold the original HAOS device bonds. Pin to the imported fabric
identity so the per-fabric file matches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pod was listening only on 8091 (UI), not 3000 (Z-Wave-JS server) — so
HA's "ws://zwave-js…:3000" submit failed with "Failed to connect".
zwave-js-ui's WS server is gated by zwave.serverEnabled — added it
(plus serverPort: 3000 explicit) into the managed Secret.
Init container now does jq deep-merge instead of skip-if-present:
managed keys (port + serverEnabled/serverPort + security keys) get
overlaid on every start, but UI-side fields (gateway/mqtt/ui) survive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pod's PVC is empty after a rebuild and zwave-js-ui needs port +
security keys to start the driver. Add an init container that copies
settings.json from a SOPS-encrypted Secret into the store PVC if
absent — first-start bootstrap, idempotent on re-runs, and won't
clobber UI edits once the user touches the file.
Settings.json carries the ZWA-2 port (/dev/zwave) plus the four S2
keys and the two long-range S2 keys, taken from HA's existing stored
zwave_js entry so the new driver resumes the existing Z-Wave network.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The per-integration reconfigure-flow driving was a bad fit. Removing
all of it: Spec/specs(), drive_form_chain, apply_spec/apply_recreate,
DisableRule, the WS client, the .storage drift-detection read, and
the MQTT_PASSWORD / DEV_MODE / INFLUX_TOKEN env plumbing.
What remains: trusted_proxies sync from ingress-nginx Endpoints into
configuration.yaml + reload_core_config REST call. Every integration
that had a Spec is now configured manually via the HA UI and persists
in HA's stored state across restarts/backups.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reconfigure path was failing with HA's generic invalid_config and
the recreate path doesn't help when the underlying issue is on the
InfluxDB side. Manual UI setup is the simpler path: paste the token
once and HA's stored entry survives subsequent reconciles. Drops
INFLUX_TOKEN env from the reconciler container too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Legacy HA VM is off, ZWA-2 + stromzaehler USB are now wired into
k8s-edge, zwave-js scaled to 1. Drop the DEV_MODE guard so the
reconciler stops force-disabling esphome, edl21, roomba, philips,
siku, zwave_js.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deployment referenced Secret zwave-js-session/session-secret but the
manifest was never authored — under replicas:0 the gap was invisible,
under replicas:1 the pod CreateContainerConfigError'd. SOPS-encrypted
random 32-byte hex secret added to the kustomization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pinned-edge cutover step. Both USB devices are now physically attached
to k8s-edge (HA legacy VM stopped, USB mappings released).
- zwave-js: replicas 0 → 1, hostPath the ZWA-2 by-id symlink at
/dev/zwave inside the container.
- home-assistant: hostPath the stromzaehler FTDI by-id symlink at
/dev/stromzaehler. edl21 integration stays disabled until DEV_MODE
flips (separate commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cutover step. CLUSTER_IN_DEVELOPMENT=false and ACME_SERVER points at
the LE production endpoint; cert-manager re-issues all 16 Certificates
with prod-CA-signed leaves on the next renewal trigger.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cutover prerequisite: external-dns now pushes A records for
*.renner.uno to UniFi DNS, so LAN clients resolve the canonical
hostnames directly to the cluster ingress LB. k.renner.uno stays
for the migration-temp alternate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Viessmann's ViCare cloud API rate-limits per account. With both legacy
HA and k8s HA polling against the same account the budget is exhausted
quickly. Disable on the k8s side until cutover, then flip on once the
legacy HA stops polling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three fixes after the live test revealed real bugs:
1. Drift detection always fired. Both REST list (?domain=…) and WS
config_entries/get_single use ConfigEntry.as_json_fragment, which
intentionally omits .data to keep secrets off the API surface.
Without seeing .data we couldn't compare against `desired`, so
has_drift({}, …) was True every iteration and every spec triggered
a full recreate every 30 s. Switch drift-side reads to a direct
read of /config/.storage/core.config_entries (atomic via HA's
storage.py tempfile+rename, debounced ~1 s; iteration interval is
30 s so reads are always consistent). All WRITE paths stay on the
documented APIs.
2. drive_form_chain didn't recognize external_step / show_progress /
menu — it would treat them as "unexpected" but only after the
delete had already happened. Add explicit early-detect: log "manual
UI setup required" and abort the flow. Won't help on the recreate
path (the entry is already gone), but prevents future damage if a
reconfigure flow ever returns one of these.
3. music_assistant intentionally removed from specs. Its user_flow's
first step is a single-field form {url} that LOOKS reconcilable —
but submission triggers external_step (OAuth: /login?return_url=
my.home-assistant.io/redirect/oauth?state=…). external_step is
browser-only per HA dev docs (developers.home-assistant.io/docs/
data_entry_flow_index). Manual UI setup is the only path. The
long-lived token MA mints during OAuth survives backup-restore so
this is one-time per cluster lifetime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Direct probe of the in-cluster InfluxDB confirmed `POST /api/v2/write
?org=renner&bucket=suedstr28` returns 403 "insufficient permissions
for write" with the existing token. InfluxDB resolves the bucket name
to an id at request time, and that resolution requires read scope on
the bucket — a write-only auth fails it.
That meant HA's reconfigure-flow validation (which performs a
test-write-of-empty-bytes against the live server) returned errors=
{base: invalid_config}, AND runtime telemetry writes were also
failing silently.
Add --read-bucket alongside --write-bucket. Bump the description
string so the Job's idempotency check doesn't reuse the prior
write-only auth (existing one is left orphaned in the auth db; admin
can clean up later — harmless until then).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the missing .k.renner.uno or .renner.uno host to loki, gatus,
and home-assistant Ingresses so every app is reachable on both the
internal LAN-side name and the public name. Drop the authentik test
Ingress; auth stays on auth.renner.uno only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Live test of the recreate path exposed three real bugs:
1. drive_form_chain re-submitted on validation errors. mqtt
reconfigure returned form/step=broker/errors={"base":"cannot_connect"}
and influxdb returned form/step=reconfigure_v2/errors={"base":
"invalid_config"}; the loop kept re-POSTing the same payload until
the 8-step cap. A non-empty `errors` field is a hard validation
failure — log the payload and abort instead of resubmitting.
2. mqtt reconfigure_flow MERGES user_input into entry.data, so the
restored backup's TLS leftovers (certificate="auto", tls_insecure,
transport residue from mqtt.renner.uno:8883) survived the merge and
HA validated against TLS-on-:1883 — cannot_connect. The reconfigure
schema doesn't let us null those keys cleanly. Switch mqtt to
via_recreate: delete the entry first (apply_recreate handles it),
then user_flow lands on a clean broker step.
3. After a successful recreate, wyoming entries pick up the wyoming
server's announced lowercase name (faster-whisper, openwakeword,
speech-to-phrase) rather than the HAOS-supervisor capitalized title
(Whisper, openWakeWord, Speech-to-Phrase) the backup carried.
Previously match() keyed on the capitalized title only, so the next
iteration saw "no matching entry". Match either form going forward.
4. music_assistant has three config_entries on the restored backup
(one real + two setup-flow-retry orphans). The REST list endpoint
sanitizes out .data, so the prior url-based match returned false for
all three and the recreate never fired. Match on state="loaded"
instead — orphans stay "not_loaded".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Probing the deployed HA 2026.5.0 confirmed wyoming, matter,
music_assistant, remote_calendar, and otbr all return "Handler X
doesn't support step reconfigure" on POST /api/config/config_entries/
flow with entry_id. Their entry data is just a URL/host pointer (no
integration-managed state to preserve), so reconcile via:
1. DELETE /api/config/config_entries/entry/<entry_id>
2. POST /api/config/config_entries/flow without entry_id (= SOURCE_USER)
3. Walk the form chain, submitting fields from spec.desired that
match each step's data_schema. Probed user-step schemas:
wyoming {host, port}
matter {url} (step is "manual")
music_assistant {url}
remote_calendar {calendar_name, url, verify_ssl}
otbr {url} (optional)
The form-chain driver caps at 8 steps, refuses to drive menu nodes,
and aborts the flow id on any HARestError so we don't leak open flows.
zwave_js stays on the reconfigure path (multi-step menu, not yet
driven). Delete-and-recreate would be unsafe — entry.data carries the
s2_*/s0 keys that own the Z-Wave network. The entry is DEV_MODE-
disabled today and its URL is already correct from the restored
backup, so no reconcile fires.
Removed the reconcile_otbr_json standalone path; otbr now goes through
the spec/recreate path. The only remaining file edit is configuration
.yaml http.trusted_proxies, which has no API surface.
ha_rest now wraps urllib HTTPError in a HARestError that carries the
response body, so 400s log HA's actual complaint instead of a bare
status code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
configuration.yaml uses HA-specific tags (!include, !include_dir_*,
!secret, …) that yaml.SafeLoader rejects with ConstructorError. Register
a generic multi-constructor for the "!" prefix that wraps the value in
a small _Tagged dataclass, plus a matching SafeDumper representer that
emits the original tag back. Net effect: yaml.safe_load and safe_dump
become round-trip-safe for HA's configuration.yaml shape.
Without this the trusted_proxies reconcile crashed on the first read,
which short-circuited the iteration before the token pre-flight could
even run.
Verified by round-tripping a sample with !include / !include_dir_merge_
list / !secret tags: all three survive parse + dump unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two failure modes the first deploy hit:
1. Authenticated readiness probe (GET /api/) carried the bad token,
each call counted as a login attempt, and homeassistant.components
.http.ban hit its threshold after 5 polls — banning ::1 and
blocking even unauthenticated traffic. Move ha_ready() to
/manifest.json with no auth header so token problems can't burn
the ban budget.
2. The pasted long-lived access token came from a backup that
pre-dates the token's creation, so its iss claim isn't in this
HA's /config/.storage/auth. Pre-flight token_registered() now
decodes the JWT, looks the iss up against the auth db, and on
mismatch logs a loud actionable message and falls back to a
degraded mode that does only the file-side ops (trusted_proxies
write, otbr JSON patch). The reload-core-config and WS-driven
reconfigures are skipped until the user mints a token from this
HA's UI and updates the Secret.
Defensive AuthFatal still raises if WS auth somehow fails despite the
pre-check (jwt_key tampering, off-by-one in the iss match) — main loop
sys.exits, kubelet applies restart backoff.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Flux's first apply of the API-rewrite commit rejected the generated
ConfigMap because no namespace was attached. Setting the kustomization
root namespace would have re-namespaced the cross-namespace Role +
RoleBinding in rbac.yaml (which intentionally live in the `ingress`
namespace), so scope the namespace to the generator entry instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrites the config-reconciler sidecar from ~340 lines of inline shell
that edited /config/.storage/core.config_entries via jq to a Python
script (mounted via configMapGenerator) that drives HA's documented
APIs:
REST /api/services/homeassistant/reload_core_config
/api/config/config_entries/entry?domain=<domain>
/api/config/config_entries/flow (init reconfigure)
/api/config/config_entries/flow/<flow_id> (configure step)
WS config_entries/get_single (drift detection)
config_entries/disable, config_entries/update
Behaviour changes:
- Drift fixes apply via reconfigure_flow + reload_core_config; the
pod no longer self-deletes to roll over. Convergence in seconds
instead of ~3 minutes.
- YAML influxdb: stanza dropped (deprecated upstream); influxdb is
now reconfigured via the v2 reconfigure flow, which validates
against the live server before persisting — a stronger guarantee
than the prior YAML-import path.
- Authenticated via a long-lived access token (home-assistant-api-
token Secret, manually minted in the UI; survives backup-restore
via .storage/auth).
- otbr stays as the lone .storage edit — upstream does not expose a
reconfigure flow for it. Documented inline.
RBAC: home-assistant-self-delete Role + RoleBinding removed; the SA
keeps only ingress-nginx Endpoints read access for the trusted_proxies
reconcile.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a reconcile_entry in the config-reconciler sidecar that rewrites
the music_assistant config_entry's data.url from https://music.renner.uno
(hairpin: Cloudflare Tunnel → ingress-nginx → cluster) to the cluster-
internal Service music-assistant.music-assistant.svc.cluster.local. The
token in .data is preserved by the existing shallow-merge — same MA
instance issued it, no re-auth needed.
The .data.url filter pins reconciliation to the real entry; two empty-
data orphan entries (leftovers from an HA setup-flow retry) live
alongside it on the live HA and would otherwise be over-written by the
shallow merge. CNP-wise, MA's hostNetwork=true means the Service backs
to the node IP, identified as host/remote-node by Cilium — both already
allowed in HA's CNP egress, so no CNP change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both ImagePolicy ranges were pinned to "2026.x", silently freezing on
the previous year's top tag every January until manually widened. Switch
to ">=2026.0.0" — the filterTags regex already gates candidates to
bare-numeric calver triplets, so removing the upper bound stays safe
while eliminating the annual maintenance step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The umbrella `seed` Kustomization has been `suspend: true` since
bfc8857 split it into per-app `seed-<app>` KSes; the prior assertion
of `kustomization/seed Ready=True` therefore fired a phantom failure
on every rebuild post-split.
Iterating the six `seed-<app>` KSes (audiobookshelf, authentik,
immich, influxdb, music-assistant, paperless) also means the test
now actually validates that the split-out KSes converge — which the
previous loop did not exercise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each `<app>` KS now dependsOn its own `seed-<app>` instead of the
shared `seed` KS, so a slow sub-tree (immich-library-seed at ~26m)
no longer blocks unrelated apps. Audiobookshelf / paperless /
music-assistant / Authentik unblock at their own seed-Job
completion (T+~13–17 vs T+~38–39 in pipeline #884).
Old `seed` KS is suspended + prune:false during the migration
window so SSA-driven ownership transfer to the new KSes doesn't
risk pruning Cluster CRs (which are not prune:disabled-annotated).
A follow-up commit deletes seed.yaml + seed/kustomization.yaml.
bootstrap.sh wait-apps drops the monolithic `wait_kustomization
seed`; per-app waits absorb the seed runtime (immich 1800s,
audiobookshelf 900s, paperless/authentik 600s, music-assistant
300s) and a `seed-influxdb` wait (600s) covers the InfluxDB HR
+ token Jobs since InfluxDB has no separate top-level KS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
target_node flips from the `saturn` placeholder to `neptune`: VM 103
(legacy HAOS) currently holds the ZWA-2 + stromzaehler USBs on neptune,
so co-locating k8s-edge VM 153 on the same host means at cutover the
USBs rebind from VM 103 → VM 153 in-place with no physical move.
USB block declared on VM 153 ahead of the cutover apply. Until VM 103
is destroyed, the USB devices are still claimed there; this declaration
is therefore not safe to apply in isolation — it lands at the final-
migration `tofu apply` after VM 103 destroy releases the devices.
k8s-edge also stays a Proxmox VM (no bare-metal pivot) — decision
recorded inline + in wiki note k8s-edge-target-node-neptune-decision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a global cluster-settings ConfigMap (CLUSTER_IN_DEVELOPMENT,
ACME_SERVER) consumed via Flux postBuild substitution on the
infrastructure-crds Kustomization. A new letsencrypt-cluster
ClusterIssuer reads ACME_SERVER from the ConfigMap at reconcile time;
every Certificate / Ingress in the repo now references letsencrypt-
cluster instead of letsencrypt-{prod,staging} directly.
While CLUSTER_IN_DEVELOPMENT=true, ACME_SERVER points at acme-staging-
v02, so the only certs the cluster can issue are LE staging — sidesteps
the recurring 5-cert/168h prod rate-limit hit during back-to-back
rebuilds (latest occurrence: 2026-05-05 alloy-syslog + loki-ingress).
Flip both keys together when leaving development.
Per-app KSes intentionally do NOT enable postBuild — Grafana dashboards
and HA's deployment shell scripts contain unrelated \${VAR} markers
that would be clobbered. Substitution stays scoped to post-crds/ which
has no other variable expansions.
The standalone letsencrypt-prod / letsencrypt-staging ClusterIssuers
remain defined as escape hatches but are no longer referenced.
e2e-test.sh now also asserts letsencrypt-cluster Ready.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
matter-server's seed initContainer mounts garage-migration-creds, owned
by the home-assistant Kustomization. Without an explicit dependsOn the
init container CreateContainerConfigError-loops on a fresh rebuild while
HA is still gated upstream — observed during the 2026-05-05 cold start.
home-assistant's ServiceMonitor consumes a CRD installed by the
kube-prometheus-stack chart (monitoring KS), so applying it from the HA
KS races the CRD on first reconcile. Move it under monitoring-rules,
which already exists for monitoring.coreos.com/* resources after the
chart lands its CRDs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HACS `s3_compatible` uploader on legacy HA writes nightly backups
to keys like `home-assistantAutomatic_backup_2026.4.4_2026-05-05_…tar`
— prefix + filename concatenated *without* a `/` separator. The first
upload (April 24) used the with-slash form `home-assistant/Automatic_…`
but every nightly since has been the no-slash form. The seed's
`--prefix "$S3_PREFIX/"` filter only matched the with-slash form, so
selection always came back as the April 24 backup; nightlies from
April 25 → May 5 were invisible to the seed.
Drop the trailing slash on the prefix. Both forms now match. The
loosened filter doesn't introduce false matches — no other objects
in this bucket start with `home-assistant`.
This is forward-fixing only (next time the seed runs on a fresh
PVC, it'll pick the latest). The currently-staged April 24 backup
+ sentinel stay in place; refreshing requires deleting both and
restarting HA.
`reconcile_entry` shallow-merges into the entry's `.data` (url, token,
bucket, etc.) but doesn't touch `.title`. HA constructs the title as
`{bucket} ({url})` at first-setup time and never refreshes it when
.data.url is later updated, so the UI's "Integration entries" list
keeps showing the stale legacy URL even when writes are correctly
going to the cluster.
Add a separate jq patch to set the title to
`suedstr28 (http://influxdb-influxdb2.influxdb.svc.cluster.local:8086)`
when the entry's current title doesn't match. Triggers drift=1 → pod
self-delete on rollover. Cosmetic only — the underlying connection
was already pointed at the cluster via the .data patch, but the UI
should reflect that rather than confuse the operator.
The HA core influxdb integration is YAML-config-only. Its
`core.config_entries` entry has `source: "import"` and HA only loads
the integration on boot if a `influxdb:` block exists in
configuration.yaml (HA then imports YAML → entry → setup). HAOS
restore brought the entry but no YAML block, so the entry was
orphaned and the integration never started — confirmed by zero
"influx*" log lines from HA.
Re-add the yq-on-configuration.yaml patch step that 0efe616 removed
(the assumption that the integration was UI-config / HACS was wrong;
domain "influxdb" with source "import" = HA core's YAML-import path).
Keep the core.config_entries patch in place so the entry stays
consistent with the YAML — otherwise HA's import on next boot would
overwrite the entry with whatever the YAML says, fighting the
reconciler's earlier patch.
Drift detection is split into two: yaml block + entry. Either drifting
triggers `drift=1` and the pod self-deletes for a clean rollover.
home-assistant's CNP egress allowlist named "default-deny-ingress"
(misleadingly — it's bidirectional) didn't include the influxdb
namespace, so HA's HACS InfluxDB integration's writes to the in-
cluster InfluxDB silently timed out at TCP connect (Cilium drops
without RST). The legacy http://influxdb.srv.renner.uno path was
reachable via the existing `toEntities: world` rule, but cluster-
internal traffic needs its own toEndpoints rule.
Add influxdb namespace as a permitted egress target. Pairs with
68482f2 (the ha-influxdb-token Job creating the write-bucket auth)
and 0efe616 (the config-reconciler patching HA's core.config_entries
URL/token to the cluster).
HA writes to InfluxDB via the HACS community integration, which is a
UI-config integration stored in `.storage/core.config_entries` with
`domain: "influxdb"` — not the built-in yaml `influxdb:` block I was
patching in configuration.yaml. The `core.config_entries` entry has
`url`/`token`/`bucket`/`organization`/`api_version`/`verify_ssl`/
`ssl_ca_cert` (and was pointed at the legacy
`http://influxdb.srv.renner.uno:8086` with the legacy
"Write buckets home_assistant" token).
Replace the configuration.yaml yq-patch with a reconcile_entry call
matching the existing pattern used for matter / otbr / zwave_js /
wyoming. Shallow-merge preserves ssl_ca_cert and any other HA-managed
fields. api_version is a string here ("2"), matching the shape HA
serializes in core.config_entries.
Token Job hit `secrets is forbidden: ... cannot create resource` even
though the Role listed `create` in verbs. Kubernetes RBAC rejects
`create` verbs combined with `resourceNames` — at creation time the
resource doesn't yet exist by name, so the rule never matches.
Standard pattern: split into two rules — blanket `create` on Secrets
in the namespace (creating any other Secret is the only collateral
damage, and there's no other actor with this SA), plus resourceNames-
restricted `get/update/patch` on the single named `ha-influxdb-token`
Secret. Job now creates on first run and updates in place on later
runs.
Add a Job that creates an HA-write-scoped auth on the in-cluster
InfluxDB on every fresh deploy and stashes the token in a Secret in
the home-assistant namespace; HA's config-reconciler reads it and
patches configuration.yaml's `influxdb:` block to point at
`influxdb-influxdb2.influxdb.svc.cluster.local:8086`, bucket
`suedstr28`, org `renner`, with the new token.
Two pieces:
1. kubernetes/seed/influxdb/ha-token-job.yaml — Job in influxdb ns
that, after `wait-for-influxd`:
- resolves suedstr28's bucket id,
- reuses an existing matching auth (description "home-assistant
write") if one's there, else creates a fresh write-bucket auth,
- kubectl-applies a generic Secret named `ha-influxdb-token` into
the home-assistant namespace with the resulting token.
ServiceAccount lives in influxdb; Role+RoleBinding in
home-assistant grant just get/create/update/patch on the single
named Secret. Tokens don't survive a fresh InfluxDB bolt store, so
a static SOPS Secret won't do — must be re-created per deploy.
2. kubernetes/apps/home-assistant/deployment.yaml — config-reconciler
sidecar gets `INFLUX_TOKEN` env from `ha-influxdb-token` (with
`optional: true` so HA boots even before the Job finishes), plus a
new step in the reconcile loop that drift-detects and patches the
`influxdb:` block. Only connection+auth keys are touched; HA-
managed include/exclude/tags filters stay intact.
Lives in seed/ for now (matches CNPG and the chart's own location);
moves to apps/influxdb/ at cutover when the seed phase is removed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The migrate-{audiobookshelf,authentik-db,immich,influxdb,music-assistant,
paperless} pipelines are not used anymore. Per-app seed Jobs are
applied by the seed Flux KS at rebuild time (kubernetes/seed/<app>/
seed-job.yaml etc.), so the manual trigger-and-wait shape these
Woodpecker pipelines provided is no longer needed.
migrate-influxdb.yml in particular pointed at a deleted path
(kubernetes/migrations/influxdb/seed-job.yaml from a refactor) and
assumed the old scale-STS-to-0+rsync-into-PVC shape, which is
incompatible with the current backup→restore-via-API design — keeping
it around would have meant either fixing it or leaving it broken.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Enable async TRIM (bdev_enable_discard, bdev_async_discard) so the
QLC FTL keeps a clean-block reserve as data is deleted; route ≤64 KiB
writes through the RocksDB WAL (bluestore_prefer_deferred_size_ssd
65536, batch ops 64) so random small writes coalesce into sequential
WAL bursts before flushing to QLC extents.
Baseline pre-tune from the 2026-05-05 rebuild: 71.6 MB/s avg / 0.89 s
avg latency on a 5-min replicated 4 MiB write bench. Discard requires
a rolling OSD restart to take effect (init-time); the BlueStore
deferred params are runtime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous legacy-token (re-encrypted from monitoring/grafana-influxdb-token)
appeared to have system-wide permissions but had `orgID:
1b264ff9ba729136` on the auth itself. InfluxDB's `/api/v2/backup/*`
middleware gates on `auth.OrgID == ""` (true operator identity), not
on permission strings — so the seed Job's `influx backup` always hit
401 with the misleading "read:authorizations is unauthorized" error.
Generated a real operator token on legacy via:
influxd recovery auth create-operator \
--bolt-path /var/lib/influxdb/influxd.bolt \
--username "local admin" \
--org renner
That writes a system-scoped auth (`OrgID == ""`) into the bolt store,
described as "local admin's Recovery Token". Verified: GET
/api/v2/backup/kv with this token returns HTTP 200 +
application/octet-stream.
Re-encrypted the new value into legacy-token.sops.yaml. Existing seed
Job spec (8a12b10) is unchanged — it just needed an actually-operator
token to reach /backup/kv.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
k.rennerandClaude Opus 4.7
security-bot