ci(kaniko): move alpine-jq to the maintained fork

#552 concluded that kaniko is terminal and that the nine build steps need
migrating to buildah or BuildKit. The premise is wrong. Google archived
GoogleContainerTools/kaniko on 2025-06-03, but the original authors forked it to
chainguard-dev/kaniko and kept shipping: releases run to v1.25.18, well past the
v1.24.0 the Google repo ended on. Upstream moved, it did not die.

So this is an image swap, not a migration. Same /kaniko/executor, same flags,
nothing else in the step changes. The fork publishes no binaries of its own, so
the image comes from gitlab-ci-utils, which builds it from that fork for amd64
and arm64 with versioned tags — the generic :debug tag exists too and is
deliberately not used, since every other image reference here is pinned.

Verified by pulling the image and running it rather than by reading its README:
/kaniko/executor is at the same path and reports v1.25.18, /bin/sh is a symlink
into busybox, and printf, base64, tr, mkdir and date are all present — the
utilities the auth and tag lines depend on. Anonymous pull works, so no registry
credential is needed for it.

Only this one step moves. A change to this file rebuilds every step regardless,
so a single run puts the new executor on alpine-jq beside seven unchanged kaniko
builds and shows both in the same log. alpine-jq is the smallest context here,
one FROM and one RUN, and the remaining seven plus ci-claude-refresh.yml follow
once this is green.

Two counts in #552 are also wrong and worth correcting there: the pin was
v1.23.2-debug everywhere, not v1.24.0-debug — #551 bumped alpine, not kaniko —
and there are eight steps in this file plus one in ci-claude-refresh.yml, so
nine invocations rather than ten.

Refs #552

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.woodpecker/build-ci-images.yml

@@ -136,8 +136,23 @@ steps:
   # not for who uses it — its first consumer is the zigbee2mqtt seed-data init
   # container, but nothing about it is zigbee-specific. Exists so tools come from
   # the image rather than an `apk add` on every container start.
+  # First step moved to the maintained kaniko fork (#552). Google archived
+  # GoogleContainerTools/kaniko on 2025-06-03, but the original authors forked
+  # it to chainguard-dev/kaniko and kept releasing: this is v1.25.18, well past
+  # the v1.24.0 the Google repo ended on. The fork publishes no binaries of its
+  # own, so the image comes from gitlab-ci-utils, which builds it for amd64 and
+  # arm64 with versioned tags.
+  #
+  # Same /kaniko/executor path, same flags, and /bin/sh, base64, tr, mkdir and
+  # date are all present — verified by pulling the image and running it, not by
+  # reading the README.
+  #
+  # Deliberately only this step for now. A change to this file rebuilds every
+  # step anyway, so one run gives the new executor on alpine-jq beside seven
+  # unchanged kaniko builds — an A/B in a single pipeline. The other seven and
+  # ci-claude-refresh.yml follow once this one is green.
   - name: build-alpine-jq
-    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    image: registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug
     directory: ci/images/alpine-jq
     depends_on: []
     when:

kubernetes/RUNBOOK-cold-rebuild.md

@@ -11,11 +11,14 @@ are no VMs to recreate, and OpenTofu owns only DNS, object storage and UniFi.
 Step 1 is therefore two unrelated halves — a `tofu apply` for the platform
 resources, and physically booting three machines from install media.
 
-**Read this first: the rebuild cannot be done remotely today.** Management is a
-tagged VLAN, and a node in maintenance mode has no VLAN configuration — it can
-only DHCP untagged, so it comes up unreachable. MS-01 has no IPMI. Every node
-therefore needs a keyboard and a monitor until #576 puts VLAN 40 native on the
-I226-V port. Plan for physical access before starting.
+**Someone has to be at the machines to boot them from USB** — MS-01 has no IPMI
+and nothing here does PXE. Everything after that is remote.
+
+A node in maintenance mode configures no VLANs and DHCPs untagged, so it comes
+up on whatever untagged network its ports carry — not on management, which is
+a tagged VLAN on the X710 trunk. It is reachable there: that is the address
+`apply-config --insecure` talks to in Step 2, and it is how all three
+bare-metal conversions were driven. Find the leases on the UDM.
 
 ## The circular-bootstrap problem this runbook exists to break
 
@@ -77,9 +80,10 @@ These must exist outside the cluster or the rebuild cannot complete:
       Proxmox credentials — there is no Proxmox any more.
 - [ ] **Install media** written and to hand: the Image Factory ISO named in
       Step 1, on a USB stick, for all three machines.
-- [ ] **Physical access to all three nodes** — keyboard and monitor. Not
-      optional today: see the note at the top. Budget for standing in front of
-      the machines, not for driving this from a desk.
+- [ ] **Someone at the machines to boot them from USB.** No IPMI, no PXE. A
+      monitor is useful for reading boot errors but is not required to find the
+      node afterwards — it DHCPs into management on its own (see the note at
+      the top). Everything past the boot is remote.
 
 ## Step 1a — tofu: DNS, object storage, UniFi
 
@@ -105,9 +109,9 @@ https://factory.talos.dev/image/0f6fb8bd8bb53013e9743b7c7b099d96fb9c32fd2464ced3
 ```
 
 Write it to a USB stick and boot each MS-01 from it. Each comes up in
-maintenance mode with a DHCP address on the **untagged** network — not on
-management, because that is a tagged VLAN a maintenance-mode node does not
-configure. Read the address off the console; that is what Step 2 talks to.
+maintenance mode with a DHCP address on an untagged network — not on
+management, which is tagged. Read the leases off the UDM; those addresses are
+what Step 2 applies the first config to.
 
 The schematic `0f6fb8bd…` carries `net.ifnames=1` — without it the predictable
 interface names that `talos/patches/k8s-{1,2,3}.yaml` match their VLAN parents
@@ -326,12 +330,12 @@ it: repo → **Settings → Repository → Mirror Settings** in Gitea, or
   in-cluster commits made **after** Flux/Gitea broke are not on the mirror
   (Gitea couldn't push them if Gitea itself was down). This is the intended
   recovery point, but be aware of it when reconciling drift.
-- **The rebuild cannot be driven remotely.** Management is a tagged VLAN and a
-  maintenance-mode node cannot configure VLANs, so a freshly booted machine is
-  unreachable until its config is applied — and MS-01 has no IPMI. Every node
-  needs a keyboard and a monitor. #576 fixes this by putting VLAN 40 native on
-  the I226-V port; until then, physical access is a hard prerequisite rather
-  than a convenience.
+- **Booting is physical; the rest is not.** MS-01 has no IPMI and nothing here
+  does PXE, so someone has to put a USB stick in three machines. From there it
+  is remote — a maintenance-mode node DHCPs untagged and is reachable at that
+  address, which is how all three bare-metal conversions were driven.
+  #576 claims a freshly booted node "comes up unreachable and must be installed
+  from a console". That is not what happens and has never been observed here.
 - **`bootstrap.sh talos` is dead** — it cannot generate a config (#581), and
   Step 2 no longer uses it. Its `flux` and `wait-apps` phases are still used in
   Steps 3, 5 and 6 and work; do not read this as the whole script being gone.

kubernetes/infrastructure/post-crds/velero-schedules.yaml

@@ -203,16 +203,16 @@ metadata:
   labels:
     app.kubernetes.io/part-of: homelab-backup
 spec:
-  # Daily at 06:45 local (next free stagger slot). All nine PVCs live
-  # in the home-assistant namespace regardless of which app/ dir owns
+  # Daily at 06:45 local (next free stagger slot). The app PVCs live in
+  # the home-assistant namespace regardless of which app/ dir owns
   # their manifest (matter-server, otbr, whisper, speech-to-phrase,
   # zone-configurator all set `namespace: home-assistant` via their
   # Kustomization, not per-resource): home-assistant-config (.storage
   # entity registry, recorder DB, secrets.yaml), esphome-data,
   # shelly-manager-data, matter-server-data, otbr-data, whisper-data,
-  # speech-to-phrase-models, speech-to-phrase-train,
-  # zone-configurator-config. No CNPG cluster in this namespace, so no
-  # label exclusion is needed. On restore, home-assistant-config has its
+  # speech-to-phrase-models, speech-to-phrase-train, zigbee2mqtt-data,
+  # zwavejs-store, zone-configurator-config. On restore,
+  # home-assistant-config has its
   # own path (the HA pod's `seed` initContainer pulling the native HAOS
   # backup from Garage, see the home-assistant Kustomization comment in
   # kubernetes/clusters/homelab/home-assistant.yaml); layering a Velero
@@ -229,6 +229,20 @@ spec:
     storageLocation: default
     ttl: 720h0m0s
     snapshotMoveData: true
+    # The ha-pg CNPG cluster now runs in this namespace (it did not when
+    # this schedule was written — the old comment here still claimed
+    # "no CNPG cluster in this namespace"). Its PVC ha-pg-1 carries the
+    # cnpg.io/cluster label and was being swept in: a crash-consistent
+    # snapshot of a running Postgres data directory is not a restorable
+    # backup, the same trap the wordpress-daily comment below describes.
+    # ha-pg is covered by Barman — daily base backup at 02:50 UTC
+    # (kubernetes/apps/home-assistant/ha-pg-scheduled-backup.yaml) plus
+    # continuous WAL archiving — so this exclusion removes a redundant
+    # 20 GiB upload, matching paperless/authentik/immich/trmnl above.
+    labelSelector:
+      matchExpressions:
+        - key: cnpg.io/cluster
+          operator: DoesNotExist
 ---
 apiVersion: velero.io/v1
 kind: Schedule