community.proxmox.proxmox inventory plugin needs python requests
(\"This module requires Python Requests 1.1.0 or higher\"). Matches
the apk + pip3 pattern used by update-all.yml, deploy-caddy.yml, and
the migrate-* pipelines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The proxmox_lxc module's local-exec provisioner runs the install
playbook against the dynamic Proxmox inventory, which needs the
community.proxmox plugin. deploy-infrastructure was the only pipeline
running ansible without installing it. Add community.docker too — most
post-create roles use docker_container.
Surfaced when LXC 250 (registry-mirror) was created cleanly but the
post-create provision failed with "unknown plugin
'community.proxmox.proxmox'".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Space-separated list expanded into -target=<resource> flags, mirroring
the existing TOFU_REPLACE pattern. Lets a manual run scope plan/apply
to a subset (e.g. TOFU_TARGET="module.registry_mirror") so unrelated
state drift doesn't get rolled in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mitigates docker.io anonymous rate limits (100/6h per IP) that cause
ImagePullBackOff during Talos+FluxCD cluster rebuilds. Local-disk-backed
distribution/distribution:3.0.0 in an external LXC so the cache survives
full cluster wipes; Talos containerd is pointed at it via
machine.registries.mirrors and falls back to docker.io if unreachable.
Only docker.io is mirrored; ghcr.io / quay.io / registry.k8s.io aren't
rate-limited.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cnpg-barman-plugin admission webhook auto-injects an `enabled:
true, isWALArchiver: false` entry into every Cluster's spec.plugins.
That makes the operator expect a plugin sidecar in the pod but never
add one; the in-pod instance-manager then loops on "Unknown plugin:
barman-cloud.cloudnative-pg.io" inside its Cluster reconcile, never
provisions /controller/certificates, never starts postgres. Status
port serves TLS without a cert -> probes fail with "tls: no
certificates configured", kubelet restarts the pod every hour.
Other clusters work because they explicitly declare the plugin with
`isWALArchiver: true` + an ObjectStore, which makes the operator add
a real sidecar that the instance-manager can connect to.
Grafana's data is reproducible from git (dashboards, datasources via
helm values), so backup isn't worth the SOPS+ObjectStore overhead.
Explicit `enabled: false` on the plugin entry overrides the webhook
injection and opts this Cluster out cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For IoT devices that can't do DNS or traverse the K8s ingress (legacy
Shelly Gen1 with hardcoded server IPs, dumb sensors, etc.) — HA now
also has a fixed v30iot address without going back to hostNetwork.
infrastructure/post-crds/cilium-l2.yaml:
- new CiliumLoadBalancerIPPool `iot-pool` (10.2.30.224-239), bound
only to Services labelled `iot-vlan: "true"` so it can't grab
arbitrary LoadBalancer Services.
- new CiliumL2AnnouncementPolicy `iot` that announces those IPs
only on k8s-edge (the only node with a v30iot NIC; its eth0 is
already in 10.2.30.0/24 — InternalIP 10.2.30.212 today).
apps/home-assistant/iot-service.yaml:
- new Service `home-assistant-iot`, type LoadBalancer with
`lbipam.cilium.io/ips: "10.2.30.224"` for a stable IP, label
`iot-vlan: "true"` so the pool/policy match it,
externalTrafficPolicy: Local to preserve client source IP.
- existing ClusterIP `home-assistant` Service is unchanged; cluster
traffic keeps using it.
Cluster rebuild required for the new CiliumLoadBalancerIPPool /
L2AnnouncementPolicy CRs to take effect (Flux can apply them but
existing Cilium agent state needs the rebuild per repo policy).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reuses the existing trusted-proxies sidecar as a general post-restore
config patcher. New env DEV_DISABLE_ESPHOME=true makes the loop also
walk /config/.storage/core.config_entries and set disabled_by="user"
on every esphome entry that's still enabled, then self-deletes the
pod so HA picks up the disabled state on its next boot. Stops the
aioesphomeapi timeout-error spam from devices the K8s pod network
can't reach.
Atomic mv + immediate self-delete keeps the race against HA's own
.storage save tiny; acceptable for dev. For production-grade we'd
swap the file edit for HA's WebSocket config_entries/disable using
the LLAT we already have in the prometheus-token Secret.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous v4 Job had no nodeSelector and the scheduler kept landing it
on a non-edge node, hitting Multi-Attach on HA's RWO PVC (held by HA
on edge) and stalling Flux's home-assistant-seed Kustomization
indefinitely — which in turn blocked home-assistant from rolling new
deployments.
Pinning to edge + tolerating the workload-class=pinned taint lets the
seed share the attachment within the node. Bumped to -v5 so Flux
deletes the stuck v4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Original choice was cargo-culted from music-assistant's hostNetwork comment
("for mDNS discovery"). MA actually uses hostNetwork for audio-streaming
latency, not discovery. For HA, hostNetwork was wrong from the start:
- Pod IP became the node IP (k8s-edge primary), so cluster pods talking
to HA appeared to come from sender-node IPs after Cilium SNAT for the
cross-VLAN hop. That's how speech-to-phrase's auth retries banned
k8s-2 wholesale in HA's ip_bans.yaml — every pod on that node then
hit 403 from HA.
- The trusted_proxies sidecar had to chase node IPs instead of pod IPs.
Without hostNetwork:
- Cluster pods → HA: source IP = pod IP (10.244.x.x), bypasses every
stale 10.2.40.x ban entry; STP/OTBR/etc. work without scrubbing.
- Ingress-nginx → HA: source IP = ingress-nginx pod IP, no SNAT.
- Sidecar reverts to its original "trust ingress-nginx pod IPs from
Endpoints" logic; Node ClusterRole/Binding dropped (no longer needed).
- Outbound to IoT VLAN devices routes via the gateway. Slightly higher
latency for the device API calls but functionally fine; HA's installed
devices have IPs cached from initial discovery.
Trade-off accepted: mDNS-based discovery of NEW devices via the HA UI
won't work without an L2 attachment to v30iot. If/when needed, add
multus + a v30iot NetworkAttachmentDefinition. If/when an IoT device
that can't do DNS or proxy-ingress needs to reach HA directly, add a
LoadBalancer Service + an IoT-range CiliumLoadBalancerIPPool restricted
to k8s-edge. Both are bolt-ons; not adding either preemptively.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SQLite on Ceph RBD does per-row fsync, each round-tripping to OSD,
making cold-bootstrap schema migrations exceed the 30s liveness probe
budget and flooding Grafana into CrashLoopBackOff. Postgres batches WAL
fsyncs server-side; migrations finish in seconds.
- New `grafana-db` Kustomization (apps/grafana-db) with a CNPG Cluster
(gf-pg, single instance, 5Gi). No Barman backup yet — Grafana data
is mostly reproducible from git (dashboards, datasources). Pattern
matches authentik-db / paperless-db / immich-db.
- New top-level `monitoring` Kustomization with `dependsOn: grafana-db`
and a Cluster healthCheckExpr that gates on `readyInstances >= 1`,
so the HelmRelease only fires after gf-pg-app secret exists.
- Removed `monitoring/` from the apps umbrella so it's gated independently.
- HelmRelease values now point Grafana at gf-pg-rw.monitoring.svc with
user/password sourced from the CNPG-managed gf-pg-app secret.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rebuild #701 saw mon-b/mon-c canary pods pull docker.io/rook/ceph:v1.19.4
in 3m52s and 5m03s respectively (568 MB image at ~1.9 MB/s). The pull
fits Docker Hub anonymous rate-limit signature: aggressive throttling
until the canary expires. The 5-minute window slipped past kube-scheduler's
event-based requeue for the real mon's pod, leaving mon-b Pending after
the canary cleared.
Rook publishes the same image to quay.io (no rate limit). Override
`image.repository` so the operator and its canary pods pull from there;
the actual mon containers already use `quay.io/ceph/ceph:v19` and pull
in ~10s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`local` (dir storage on rpool's filesystem) put VM disks as files
behind ZFS's variable recordsize/snapshot/CoW path. Etcd's small
fsync-heavy WAL writes saw enough added latency that during the
bootstrap I/O storm (Cilium + Flux + many lease grabbers) the etcd
leader couldn't service raft heartbeats inside its election window,
triggering "leader changed" / "request timed out" cascades.
Rook's mon-a Deployment create landed in that window, ate an etcd
timeout, and the operator never retries the create.
local-zfs (ZFS-pool storage = ZVOLs) bypasses the dir/file path —
fewer layers, lower fsync latency. Etcd was stable here pre-#699.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PVE rejects hostpci on the default i440fx machine type:
TASK ERROR: q35 machine model is not enabled at PVE/QemuServer/PCI.pm
q35 exposes a real PCIe root complex; the controlplane VMs need it
once they have an NVMe controller passed through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The earlier path_in_datastore (raw filesystem path) approach hit a
hardcoded PVE check: only root@pam can pass arbitrary filesystem
paths to qm. The Ansible API token can't, regardless of role.
Hardware Mappings sidestep this — a cluster-level PCI mapping
"osd-ssd" was created on PVE pointing to each node's freed NVMe;
the API token is allowed to attach mapped devices.
- proxmox_vm module: add `hostpci` variable + dynamic block, indexed
via list position (hostpci0, hostpci1, ...)
- vm_150/151/152: drop the virtio1 disk entry, add `hostpci = [{ mapping = "osd-ssd" }]`
- CephCluster: device renamed `vdb` -> `nvme0n1`. With PCI passthrough
the NVMe enumerates inside Talos via the nvme driver, not virtio-blk.
The path_in_datastore plumbing in the module stays — it's harmless
when unused and may serve other use cases later.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
local-zfs is now the dedicated raw-passthrough device for the Ceph OSD
on each node, so it can no longer host other VM disks. Move the 32 GB
Talos boot disks (k8s_1/k8s_2/k8s_3/k8s_edge) onto `local`, the PVE
host-filesystem storage on rpool.
Also fix `\${mod}` in the rebuild-k8s fallback retry — Woodpecker
interpolates \${var} before bash sees it, collapsing the address to
"module..proxmox_virtual_environment_vm.this". Use \$mod (no braces)
so bash gets the variable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without -target, `tofu apply -replace=...` plans the entire
configuration. Drift on unrelated VMs/LXCs (e.g. from manual
qm move-disk during the OSD-passthrough storage migration) gets
pulled into the apply and either hangs or destroys things outside
the rebuild's intended scope. Pipeline #690 hit this — the apply
tried to recreate gitea, authentik, audiobookshelf, and woodpecker
LXCs alongside the K8s VM replaces.
Add `-target=module.k8s_{1,2,3,edge}` to both the bulk apply and
the per-module fallback retries so the operation is bounded to the
K8s nodes regardless of unrelated state drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each PVE node's second NVMe was evacuated from rpool via `zpool remove`
and wiped. Talos VMs 150/151/152 now consume their respective Crucial P3
directly via path_in_datastore, eliminating the Bluestore-on-ZVOL-on-ZFS
double-allocator path and ARC-vs-osd_memory_target contention.
Module gains an optional path_in_datastore disk field. When set,
datastore_id is forced to "" and size is omitted; existing ZVOL disks
are unaffected. Disk is in lifecycle.ignore_changes, so this only takes
effect on the next rebuild-k8s `-replace`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zone-configurator:
- image: everythingsmarthome/everything-presence-mmwave-configurator:latest
- port 42069 (web) + 38080 (firmware OTA to ESP devices on IoT VLAN)
- hostNetwork on edge so ESP devices can pull firmware
- HA_BASE_URL + HA_LONG_LIVED_TOKEN env (token reused from
home-assistant-prometheus-token)
- 256 MiB PVC at /config
speech-to-phrase:
- real CLI contract: --hass-websocket-uri, --hass-token, --retrain-on-start
- volumes /models (2 GiB) + /train (1 GiB) per upstream docs
- dropped the made-up --train-dir/--tools-dir/--models-dir flags
Both Flux Kustomizations flipped to wait:true now that the contract is
correct. OTBR stays wait:false until the second SLZB's hostname is
supplied (config, not an image issue).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six sibling Deployments under the home-assistant namespace, plus six
Flux Kustomizations wired into clusters/homelab/home-assistant.yaml:
matter-server — ghcr.io/home-assistant-libs/python-matter-server
hostNetwork on edge for mDNS, PVC for fabric data
otbr — openthread/otbr; RADIO_URL placeholder for the 2nd
SLZB (TBD-SLZB-HOSTNAME); hostNetwork + NET_ADMIN
zone-configurator — Everything Presence Lite zone setup web UI
whisper — rhasspy/wyoming-whisper, model PVC, German default
openwakeword — rhasspy/wyoming-openwakeword, ok_nabu preload
speech-to-phrase — Wyoming STP; consumes HA token from existing
prometheus-token Secret + WS to home-assistant svc
FIXMEs marked inline for: zone-configurator image path, speech-to-phrase
image path, otbr SLZB hostname. Several Kustomizations land with
wait:false until those are confirmed so they don't block the chain.
Hostnames left as-is (whisper, openwakeword, etc.) — HA's restored
.storage references the old core-* names; user reconfigures via UI
post-migration per discussion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore failure root cause: HA's livenessProbe (initialDelay 120s + 3
failures @ 30s = 210s) fired in the middle of decrypting the 1 GiB
encrypted backup, SIGTERM'd HA mid-restore, and on restart HA wrote
a default configuration.yaml. The old seed gate `[ -f
/config/configuration.yaml ]` then matched that default file and
skipped the re-restore, leaving HA permanently on stub config.
Two fixes:
- startupProbe with 10 min ceiling (initialDelay 30 + 60 × 10s) so
liveness/readiness only kick in after HA finishes its initial
boot — including a long restore.
- Seed sentinel moved to /config/backups/.k8s-seed-applied. Lives
in the only dir HA's restore preserves; uses a name HA will never
write itself, so partial-restore-then-default-config can't fool it.
Job bumped to -v4 so Flux replaces v3 cleanly. The bad-state PVC
needs to be reset by deleting the home-assistant namespace once Flux
has the new code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Old sentinel /config/.restore-done was written by the seed Job and
wiped by HA's native restore (which only preserves /config/backups/).
Every Flux reconcile therefore re-ran the full seed — re-staging the
tar and re-writing .HA_RESTORE, which made HA wipe /config and restore
all over again on its next boot. Eternal restore loop.
Gate on /config/configuration.yaml instead: if it exists, HA has
restored before and the seed is a no-op. Bumped Job name to -v3 so
Flux cleanly replaces the stuck v2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HA is on hostNetwork, so the kernel SNATs pod→hostNetwork traffic to
the sending node's primary IP — HA sees requests from K8s node IPs,
not the ingress-nginx pod IPs the sidecar was writing.
Sidecar now resolves Endpoints → unique nodeNames → Node InternalIPs
via the K8s API, and uses THOSE as trusted_proxies. Adds a
ClusterRole+Binding for read-only node access.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ansible copy doesn't auto-create parent dirs, and
/etc/systemd/system/influxdb.service.d doesn't exist yet on VM 108.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rolls back every file added for the Coder deployment this session —
app manifests, Flux Kustomization, namespace, CiliumNetworkPolicy,
ingress-nginx egress entry, desktop workspace template.
Flux prune will tear down the running resources (HelmRelease, CNPG
Cluster, PVCs, Ingress, TLS cert). Garage bucket data at
s3://k8s-cnpg/coder-pg and the stray Cloudflare A record for
coder.renner.uno remain as orphans.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of the "unexpected EOF while saving SQL backup" turned out
to be server-side, not client-side: the Ubuntu influxdb2 systemd
unit denies the service a writable /tmp, so the backup handler
fails with `mkdir /tmp/XXXX: permission denied`, streams a JSON
"internal error" into the multipart SQL part, and the CLI chokes.
Drop-in points TMPDIR at /var/lib/influxdb/tmp (owned by the
influxdb user). Idempotent — creates dir + drop-in + daemon-reload
+ service restart only if the drop-in changed. Runs before the
backup task so every pipeline run converges on the working config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HA's image uses s6-overlay which aborts with "can only run as pid 1"
when the pod shares its process namespace — so pkill from a sidecar
cannot work. Trigger HA reload by self-deleting the pod instead; the
Deployment controller recreates it and HA reads the patched
configuration.yaml on the next boot. Adds a Role+RoleBinding in the
home-assistant ns granting pods/delete to the sidecar's SA.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2.7.4 and 2.7.5 CLIs both fail the same way against VM 108 but the
2.7.4 CLI worked against the in-cluster server. That points at the
multipart response VM 108 is sending, not the CLI. Capture the
response headers, first-line boundary, size, sha256, and last 500
bytes (base64) so we can see whether the multipart is actually
well-formed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2.7.12 doesn't exist as a standalone influxdb2-client tarball at
download.influxdata.com — only 2.7.5 (the version the install docs
still show). Newer 2.7.x releases are server-only. Try 2.7.5 as the
one-step-up option; if its multipart parser has the same bug we'll
need to either extract the CLI from the v2.8.0 server tarball or
replace the backup with a raw-curl flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Terraform template for dynamically provisioned browser-accessible
XFCE desktops. Each workspace spawns a K8s Deployment in the coder
namespace with a per-user ceph-block PVC for /home/coder that
survives stop/start but is destroyed with the workspace.
Not applied by Flux — push via `coder templates push desktop -d
kubernetes/apps/coder/templates/desktop` once the control plane is
reachable and an admin token exists.
- Base image: codercom/enterprise-base:ubuntu (agent preinstalled)
- Desktop: KasmVNC + XFCE via registry.coder.com/modules/kasmvnc
- Params: cpu (1/2/4/8), memory (2/4/8/16 GiB), home_size (5-200 GiB)
- Non-root (uid 1000), seccomp RuntimeDefault, all caps dropped
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The init container looked for /config/.restore-done from the seed Job,
but HA's native restore wipes everything outside /config/backups/ —
so on every pod restart after the initial restore, the init container
would block forever.
Flux's dependsOn: home-assistant-seed on the home-assistant Kustomization
already gates HA on seed success, so the init-level gating is redundant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
influx 2.7.4 fails parsing the /api/v2/backup/metadata multipart
response with "unexpected EOF while saving SQL backup"; raw curl on
the same endpoint returns 110 KB of valid body, so server-side is
fine. Bump the CLI we download on VM 108 to 2.7.12 (latest 2.7.x
client). Server stays 2.7.4; backup HTTP protocol is stable across
the 2.7.x line.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Coder pods initiate OIDC discovery against auth.renner.uno, which
resolves to the ingress-nginx LB. Cilium policy evaluates destination
at the pod endpoint, not the LB VIP, so we need to explicitly allow
egress to the ingress-nginx endpoint. Matches audiobookshelf-ns.yaml.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The per-ingress-nginx egress CNP enumerates app namespaces; coder was
missing, so requests to coder.renner.uno dead-ended at
"upstream timed out" even though coder's own CNP allowed the inbound
side. Add coder to the allowlist, matching the other apps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restored configuration.yaml carries trusted_proxies=10.2.45.0/24 (Caddy DMZ)
which is correct for VM 103 but wrong for the K8s ingress path — HA
replies 400 to forwarded requests whose direct source IP (ingress-nginx
pod IP in 10.244.0.0/16) isn't in the trusted list. Trusting the whole
pod /16 is too broad.
This commit keeps trusted_proxies narrow + dynamic:
- ServiceAccount with cross-ns Role on Endpoints/ingress-nginx-controller
- shareProcessNamespace so the sidecar can pkill HA for s6 to restart
- Sidecar polls Endpoints every 30s, patches http.trusted_proxies in
/config/configuration.yaml when the resolved pod IPs drift, signals HA
- Blocks until .HA_RESTORE_RESULT exists so the initial seed+restore
completes before the sidecar starts mutating config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without a per-namespace CNP, the cluster-wide default-deny blocks
ingress-nginx from reaching coder pods, so requests to coder.renner.uno
end in upstream connection timeouts (504) from ingress-nginx.
Model on authentik-ns.yaml: allow ingress from same-ns + ingress-nginx
+ monitoring + node-local; allow egress to same-ns + rook-ceph + kube-dns
+ node/world (world needed for OIDC discovery via the public auth.renner.uno
hostname, which resolves to the in-cluster LB IP but is served through
`host`/`world` entities from the pod's perspective).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dropping the export-lp / influx write path. The earlier pivot to LP
was a workaround for an `influx backup` failure on VM 108 that I did
not actually debug — I tested influx backup against the in-cluster
InfluxDB 2.7.4 today, exit 0, produced valid TSM tarballs. So
the tool works; the VM 108 issue was likely token/config, not a
2.7.x bug as the old commit message claimed.
LP round-trip was also broken — influxd inspect export-lp emits
string values with embedded raw newlines that influx write 400s on
("unbalanced quotes"). Bug is in the LP export/import contract
itself. `influx backup`/`restore` uses the binary TSM format and
side-steps it entirely.
Ansible side:
- Restores the influx backup flow that was abandoned pre-d30ac63.
- Keeps the debug probes (ping + raw /api/v2/backup/metadata curl)
so if VM 108 fails we have the HTTP response, not a CLI error.
- Drops the stop-influxd block: backup is online.
- Output tarball: s3://k8s-migration/influxdb/influxdb-backup.tar.gz.
K8s seed Job:
- Drops the -vN suffix. One stable spec. Cluster rebuilds get a
fresh Job each time; pod-template immutability stops being a
concern.
- Downloads tarball → extracts → deletes the chart-bootstrapped
empty bucket → runs `influx restore --bucket suedstr28`.
- 30 GiB ephemeral ceph-block PVC for scratch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
k8s-gateway honors `external-dns.alpha.kubernetes.io/exclude: "true"`
as well, so adding it to avoid a Cloudflare record also blocked
cluster-internal DNS from publishing coder.renner.uno. Result: the
pod appeared healthy but `coder.renner.uno` was unreachable in-cluster
(fell through k8s-gateway → upstream → stray Cloudflare record → 504
via cloudflared/Caddy HA).
The Cloudflare external-dns instance already filters on the
`cloudflare-proxied` annotation, which this Ingress doesn't have, so
dropping `exclude` is sufficient: no public record, and k8s-gateway
resumes publishing for cluster-internal resolution.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HA Core's homeassistant/backup_restore.py handles encrypted backups itself:
drop the .tar in /config/backups/ and write /config/.HA_RESTORE with a
JSON manifest {path, password, remove_after_restore, restore_database,
restore_homeassistant}. On next startup HA decrypts, version-checks,
reconciles registries, and restores — all inside HA, then wipes the
trigger to prevent boot loops.
Previous design tried to extract + decrypt the securetar-encrypted inner
payload in the seed Job with a third-party Go tool. Wrong. HA owns this.
Job renamed to -v2 so Flux prunes the stale -v1.
Before Flux reconciles this, rotate the password stub:
sops kubernetes/apps/home-assistant-seed/backup-password.sops.yaml
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the external-dns cloudflare-proxied annotation (inherited from
other apps' pattern) with `exclude: true` so external-dns doesn't
publish a public Cloudflare record for coder.renner.uno. Coder is for
internal developer use only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
k.rennerandClaude Opus 4.7