feat(home-assistant): automated rotation of the three HA long-lived tokens #322

2026-07-21T10:23:14Z

claude-bot commented

2026-07-21 10:23:14 +00:00

Completes the Home Assistant rotation work. Rotates all three HA long-lived access tokens via the secret-rotator binary's ha-token mode. All three tested end-to-end against live HA before shipping; two were rotated for real.

The three tokens

Token	Consumer	Reload on rotation
`prometheus-token` ("ServiceMonitor")	Prometheus ServiceMonitor	config-reloader, no restart
`shelly-manager-llat` ("Shelly Manager")	shelly-manager deploy (startup env)	added to its reload annotation
`config-sync-token` ("Config Sync")	config-sync sidecar in the HA pod (startup env)	HA Deployment reload annotation

The binary identifies the outgoing token by its JWT iss, not client_name — the stored names are misleading (prometheus-token is "ServiceMonitor"; config-sync-token was "laptop local" until fixed earlier) — mints under a unique name (HA rejects a duplicate), and revokes the old token over a fresh new-token session.

Network — the first in-cluster rotation target

HA needs both netpol halves. The openbao default-deny only permits TCP egress to openbao/kube-dns/apiserver/world; the external-API rotators (Cloudflare, NetBird, Proxmox) ride world, but HA is in-cluster:

home-assistant-rotator-ingress (home-assistant ns): rotator → HA :8123
rotator-home-assistant-egress (openbao ns): the egress complement

Reaches HA on :8123 (HA itself, not the :8124 XFF-anchor proxy — granting :8124 would let it spoof X-Forwarded-For) via the home-assistant-bgp ClusterIP; the public name hairpins off the LB. Both are dedicated policies scoped by the rotator label, not rules on the namespace-wide default-deny (which would leak the grant to every pod on that port — the trap that once let shelly-manager ride the code-server allow).

config-sync restarts HA quarterly — deliberate

config-sync reads its token at startup and lives in the HA pod, so picking up a rotation means restarting the HA pod (~1-2 min, 04:05 off-hours, quarterly). Chosen over leaving a full-admin token un-rotated. prometheus and shelly-manager reload without touching HA.

Test evidence (live)

prometheus-token rotated end-to-end: OpenBao 6af3e8e6→991bd421, old token gone, new authenticates → HA /api/prometheus 200, 22170 metric lines. Job log: minted 'ServiceMonitor [77adb8]', revoked old 'ServiceMonitor' (592eeaad...) — clean, no false warning.
shelly-manager-llat rotated end-to-end: OpenBao updated, shelly-manager restarted onto it, 0 HA-auth errors, state: RUNNING.

Both were rotated as real tests; the committed CronJobs reconcile to the same state. Image pinned by digest sha256:159e5888… (the binary with all three correctness fixes from #318/#320/#321). OpenBao policy/role bootstrapped out-of-band, committed for provenance.

🤖 Generated with Claude Code

Completes the Home Assistant rotation work. Rotates all three HA long-lived access tokens via the `secret-rotator` binary's `ha-token` mode. **All three tested end-to-end against live HA** before shipping; two were rotated for real. ## The three tokens | Token | Consumer | Reload on rotation | |---|---|---| | `prometheus-token` ("ServiceMonitor") | Prometheus ServiceMonitor | config-reloader, no restart | | `shelly-manager-llat` ("Shelly Manager") | shelly-manager deploy (startup env) | **added** to its reload annotation | | `config-sync-token` ("Config Sync") | config-sync sidecar in the HA pod (startup env) | **HA Deployment** reload annotation | The binary identifies the outgoing token by its JWT `iss`, not `client_name` — the stored names are misleading (`prometheus-token` is "ServiceMonitor"; `config-sync-token` was "laptop local" until fixed earlier) — mints under a unique name (HA rejects a duplicate), and revokes the old token over a fresh new-token session. ## Network — the first in-cluster rotation target HA needs **both** netpol halves. The openbao default-deny only permits TCP egress to openbao/kube-dns/apiserver/`world`; the external-API rotators (Cloudflare, NetBird, Proxmox) ride `world`, but HA is in-cluster: - `home-assistant-rotator-ingress` (home-assistant ns): rotator → HA `:8123` - `rotator-home-assistant-egress` (openbao ns): the egress complement Reaches HA on **`:8123`** (HA itself, not the `:8124` XFF-anchor proxy — granting `:8124` would let it spoof X-Forwarded-For) via the `home-assistant-bgp` ClusterIP; the public name hairpins off the LB. Both are dedicated policies scoped by the rotator label, not rules on the namespace-wide default-deny (which would leak the grant to every pod on that port — the trap that once let shelly-manager ride the code-server allow). ## config-sync restarts HA quarterly — deliberate config-sync reads its token at startup and lives in the HA pod, so picking up a rotation means restarting the HA pod (~1-2 min, 04:05 off-hours, quarterly). Chosen over leaving a full-admin token un-rotated. prometheus and shelly-manager reload without touching HA. ## Test evidence (live) - **prometheus-token** rotated end-to-end: OpenBao `6af3e8e6`→`991bd421`, old token gone, new authenticates → HA `/api/prometheus` **200, 22170 metric lines**. Job log: `minted 'ServiceMonitor [77adb8]', revoked old 'ServiceMonitor' (592eeaad...)` — clean, no false warning. - **shelly-manager-llat** rotated end-to-end: OpenBao updated, shelly-manager restarted onto it, **0** HA-auth errors, `state: RUNNING`. Both were rotated as real tests; the committed CronJobs reconcile to the same state. Image pinned by digest `sha256:159e5888…` (the binary with all three correctness fixes from #318/#320/#321). OpenBao policy/role bootstrapped out-of-band, committed for provenance. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-07-21 10:23:15 +00:00

feat(home-assistant): automated rotation of the three HA long-lived tokens 6807da02be

Rotates prometheus-token, config-sync-token and shelly-manager-llat via the
secret-rotator binary's ha-token mode (HA mints/revokes LLATs only over its
WebSocket API). All three tested end-to-end against live HA before shipping.

Network: HA is the first IN-CLUSTER rotation target, so it needs BOTH netpol
halves — the openbao default-deny only permits TCP egress to
openbao/kube-dns/apiserver/world (the external-API rotators ride `world`):
  - home-assistant-rotator-ingress (home-assistant ns): rotator -> HA :8123
  - rotator-home-assistant-egress (openbao ns): the egress complement
Reaches HA on :8123 (HA itself, not the :8124 XFF-anchor proxy) via the
home-assistant-bgp Service ClusterIP; the public name hairpins off the LB.
Both dedicated policies, scoped by the rotator label, not rules on the
namespace-wide default-deny (which would leak to every pod on that port).

Reloader wiring — each token's consumer picks up the new value differently:
  - prometheus-token: Prometheus config-reloader auto-reloads; no restart.
  - shelly-manager-llat: added to shelly-manager's reload annotation (it reads
    the token as a startup env var).
  - config-sync-token: the config-sync sidecar reads it at startup and lives in
    the HA pod, so the HA Deployment gets a reload annotation. This restarts HA
    itself quarterly (04:05, off-hours) — accepted so the full-admin token is
    lifetime-bounded like the others.

The binary identifies the outgoing token by its JWT `iss`, not client_name
(the stored names are misleading — prometheus-token is "ServiceMonitor",
config-sync-token was "laptop local"), mints under a unique name (HA rejects a
duplicate), and revokes the old token over a fresh new-token session.

OpenBao policy/role bootstrapped out-of-band (rotator-home-assistant.hcl,
committed for provenance; not Flux-managed).

Tested: prometheus-token and shelly-manager-llat rotated live end-to-end —
OpenBao updated, ESO synced, consumer restarted/reloaded, old token revoked and
gone, new token authenticates (Prometheus scrape 200 with 22k metrics;
shelly-manager 0 HA-auth errors).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

k.renner merged commit 36accd4359 into master

2026-07-21 14:32:19 +00:00

k.renner referenced this issue from a commit

2026-07-21 14:32:21 +00:00

Merge pull request 'feat(home-assistant): automated rotation of the three HA long-lived tokens' (#322) from feat/ha-token-rotation into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#322