Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b34450b22e | ||
|
|
3faddcadcd | ||
|
|
6ae734eef4 | ||
|
|
39885b9400 | ||
|
|
694389b569 | ||
|
|
57bb39e7ee | ||
|
|
17f913ba2f | ||
|
|
53172dff74 |
@@ -121,7 +121,9 @@ spec:
|
||||
persistentVolumeClaim:
|
||||
claimName: audiobookshelf-books
|
||||
- name: le-ca
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
|
||||
@@ -66,7 +66,52 @@ Confirm the backup actually landed in the bucket rather than trusting the CR
|
||||
status — the WordPress backup failed nightly for three days against a bucket
|
||||
that was declared but never created (see `opentofu/object_storage.tf`).
|
||||
|
||||
## Phase 2 — migrate the data (NOT in this PR)
|
||||
## Phase 2b — bulk statistics copy: DONE 2026-08-19
|
||||
|
||||
Completed with **HA running** — the SQLite file was read through a read-only
|
||||
mount, and SQLite's WAL mode gives readers a consistent snapshot, so no downtime
|
||||
was needed for the bulk copy.
|
||||
|
||||
```
|
||||
statistics_meta: sqlite=1449 pg=1449 OK
|
||||
statistics: sqlite=10799904 pg=10799904 OK (153.9s)
|
||||
statistics_short_term: sqlite=1969804 pg=1969804 OK ( 27.0s)
|
||||
|
||||
statistics span in Postgres: 2024-03-13 -> 2026-08-19
|
||||
entities with long-term history: 1449
|
||||
rows carrying sum (energy dashboard): 2627859
|
||||
database size: 2252 MB (PVC is 20Gi)
|
||||
```
|
||||
|
||||
`ANALYZE` was run on all three tables. That is not housekeeping: after a bulk
|
||||
`COPY` the planner has no statistics, and HA queries these tables constantly for
|
||||
history and energy views, so without it every view seq-scans 10.8M rows.
|
||||
|
||||
Two schema details that cost a failed first attempt, recorded so the top-up does
|
||||
not repeat them:
|
||||
|
||||
- `has_mean` / `has_sum` are INTEGER in SQLite and BOOLEAN in Postgres. Postgres
|
||||
will not coerce; convert explicitly and preserve NULL.
|
||||
- The legacy `created` / `start` / `last_reset` DATETIME columns are 100% NULL in
|
||||
both bulk tables (superseded by the `_ts` variants), which is what makes a
|
||||
tab-separated `COPY` safe — nothing in those tables needs escaping.
|
||||
|
||||
The schema itself was created with HA's own models
|
||||
(`Base.metadata.create_all`), **not** hand-written DDL, and `schema_changes` was
|
||||
left EMPTY on purpose. HA's `_initialize_database` detects a new database by
|
||||
looking for an index on `events.time_fired_ts`, then writes `SCHEMA_VERSION`
|
||||
itself. Pre-stamping that row would skip HA's own initialisation path.
|
||||
|
||||
### High-water marks for the cutover top-up
|
||||
|
||||
Rows written after the bulk copy are not yet in Postgres. Resume from:
|
||||
|
||||
```
|
||||
statistics max_id=13323904 max_start_ts=1787162400.0
|
||||
statistics_short_term max_id=159205762 max_start_ts=1787166000.0
|
||||
```
|
||||
|
||||
## Phase 2c — cutover (REMAINING)
|
||||
|
||||
Do not start until phase 1 is verified.
|
||||
|
||||
@@ -90,7 +135,18 @@ HA outage, and buys a table that is discarded within 7 days anyway.
|
||||
SELECT setval('statistics_short_term_id_seq', (SELECT MAX(id) FROM statistics_short_term));
|
||||
```
|
||||
5. **Repoint** `recorder.db_url` in the `homeassistant` repo — that change lives
|
||||
there, not here:
|
||||
there, not here. The `recorder_db_url` secret it references is already
|
||||
rendered into `secrets.yaml` by the `render-db-url` init container in
|
||||
`deployment.yaml` (phase 2a), so this order is mandatory: **the init
|
||||
container must be deployed first**, or HA refuses to start on a missing
|
||||
secret.
|
||||
|
||||
The init container percent-encodes the credentials and validates the
|
||||
resulting YAML rather than counting lines. That is not incidental — the CNPG
|
||||
password is random, and a naive `"postgresql://user:pass@host/db"` string
|
||||
makes `secrets.yaml` unparseable the moment the password contains a quote,
|
||||
which takes out *every* secret HA loads, not just this one. Verified against
|
||||
a deliberately hostile password before merge.
|
||||
```yaml
|
||||
recorder:
|
||||
db_url: !secret recorder_db_url # postgresql://homeassistant:<pw>@ha-pg-rw.home-assistant.svc.cluster.local/homeassistant
|
||||
|
||||
@@ -36,6 +36,64 @@ spec:
|
||||
# (sml2mqtt bridge owns the USB device on edge), so it no longer needs
|
||||
# the edge USB device and schedules on the general cluster pool. This
|
||||
# gets the heavy HA pod (8Gi) off the soon-to-be-tiny edge box.
|
||||
initContainers:
|
||||
# Puts CNPG's connection URI into /config/secrets.yaml as
|
||||
# `recorder_db_url`, for the recorder's move off SQLite
|
||||
# (MIGRATION-recorder-postgres.md).
|
||||
#
|
||||
# HA has no way to read a db_url from the environment — `!env_var` was
|
||||
# removed from its YAML loader — so `!secret` is the only mechanism, and
|
||||
# secrets.yaml is PVC state that config-sync deliberately does not
|
||||
# manage. Hence a small init container rather than a env: entry.
|
||||
#
|
||||
# The URI comes STRAIGHT from the ha-pg-app Secret. CNPG builds and
|
||||
# percent-encodes it, so there is nothing to escape here: the value is
|
||||
# restricted to [A-Za-z0-9:/@._%-], verified against the live secret.
|
||||
# An earlier version of this assembled the URI from user/password/dbname
|
||||
# and needed percent-encoding plus YAML round-trip validation to be
|
||||
# safe. That was solving a problem CNPG had already solved.
|
||||
#
|
||||
# Re-rendered on every pod start, so a CNPG password rotation takes
|
||||
# effect rather than silently leaving a stale URI behind.
|
||||
#
|
||||
# INERT until the recorder is repointed: nothing reads this key until
|
||||
# `db_url: !secret recorder_db_url` lands in configuration.yaml in the
|
||||
# homeassistant repo. That ordering is required — HA refuses to start
|
||||
# when a referenced secret is missing.
|
||||
- name: render-db-url
|
||||
image: busybox:1.36
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -eu
|
||||
S=/config/secrets.yaml
|
||||
T=/config/.secrets.render.tmp
|
||||
[ -f "$S" ] || : > "$S"
|
||||
# Filter-then-append, never rewrite in place: this file holds
|
||||
# every other HA secret (influxdb_token and friends).
|
||||
grep -v '^recorder_db_url:' "$S" > "$T" || true
|
||||
printf 'recorder_db_url: "%s"\n' "$DB_URI" >> "$T"
|
||||
# Refuse to shrink the file. Cheap guard against the one failure
|
||||
# that would actually hurt: losing the other keys.
|
||||
[ "$(wc -l < "$T")" -ge "$(wc -l < "$S")" ] || {
|
||||
echo "FATAL: rewrite would shrink secrets.yaml; aborting" >&2; exit 1; }
|
||||
chmod 600 "$T"
|
||||
mv "$T" "$S"
|
||||
echo "recorder_db_url rendered"
|
||||
env:
|
||||
- name: DB_URI
|
||||
valueFrom:
|
||||
secretKeyRef: { name: ha-pg-app, key: fqdn-uri }
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /config
|
||||
resources:
|
||||
requests:
|
||||
cpu: 10m
|
||||
memory: 16Mi
|
||||
limits:
|
||||
memory: 32Mi
|
||||
containers:
|
||||
- name: home-assistant
|
||||
image: ghcr.io/home-assistant/home-assistant:2026.8.2 # {"$imagepolicy": "flux-system:home-assistant"}
|
||||
@@ -221,9 +279,13 @@ spec:
|
||||
# Writable emptyDir for the nginx entrypoint's envsubst output
|
||||
# (/etc/nginx/conf.d/default.conf rendered from the .template).
|
||||
- name: proxy-conf-d
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
# code-server's settings/extensions cache. Not on the config PVC
|
||||
# so installed extensions don't pollute HA's /config listing.
|
||||
# Trade-off: extensions reset across pod restarts.
|
||||
- name: vscode-state
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
|
||||
@@ -313,7 +313,9 @@ spec:
|
||||
configMap:
|
||||
name: mosquitto-config
|
||||
- name: auth
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
- name: tls
|
||||
secret:
|
||||
secretName: mosquitto-tls
|
||||
|
||||
@@ -344,9 +344,13 @@ spec:
|
||||
configMap:
|
||||
name: zigbee2mqtt-interview-watch-script
|
||||
- name: interview-watch-state
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
- name: discovery-guard-script
|
||||
configMap:
|
||||
name: zigbee2mqtt-discovery-guard-script
|
||||
- name: discovery-guard-state
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
|
||||
@@ -147,7 +147,9 @@ spec:
|
||||
- key: immich-config.json
|
||||
path: immich-config.json
|
||||
- name: le-ca
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
|
||||
@@ -64,7 +64,9 @@ spec:
|
||||
secret:
|
||||
secretName: pve-exporter-config
|
||||
- name: tmp
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
|
||||
@@ -218,7 +218,9 @@ spec:
|
||||
persistentVolumeClaim:
|
||||
claimName: paperless-consume
|
||||
- name: bridge-cert
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
|
||||
@@ -156,4 +156,6 @@ spec:
|
||||
# each pod enrolls its own peer and a restart re-enrolls cleanly
|
||||
# (the old peer is reaped).
|
||||
- name: state
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
|
||||
@@ -5,7 +5,7 @@ metadata:
|
||||
namespace: rook-ceph
|
||||
spec:
|
||||
cephVersion:
|
||||
image: quay.io/ceph/ceph:v19.2.5 # {"$imagepolicy": "flux-system:ceph"}
|
||||
image: quay.io/ceph/ceph:v19.2.6 # {"$imagepolicy": "flux-system:ceph"}
|
||||
allowUnsupported: false
|
||||
dataDirHostPath: /var/lib/rook
|
||||
mon:
|
||||
|
||||
@@ -38,7 +38,7 @@ spec:
|
||||
serviceAccountName: rook-ceph-default
|
||||
containers:
|
||||
- name: rook-ceph-tools
|
||||
image: quay.io/ceph/ceph:v19.2.5 # {"$imagepolicy": "flux-system:ceph"}
|
||||
image: quay.io/ceph/ceph:v19.2.6 # {"$imagepolicy": "flux-system:ceph"}
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
@@ -164,7 +164,9 @@ spec:
|
||||
name: rook-config-override
|
||||
optional: true
|
||||
- name: ceph-config
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
tolerations:
|
||||
- key: "node.kubernetes.io/unreachable"
|
||||
operator: "Exists"
|
||||
|
||||
@@ -85,4 +85,6 @@ spec:
|
||||
name: restore-test-script
|
||||
defaultMode: 0755
|
||||
- name: tmp
|
||||
emptyDir: {}
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 16Mi
|
||||
|
||||
Reference in New Issue
Block a user