Zwei Resilienz-Fixes aus den Rebuild-Proben: was ein Kaltstart braucht, bevor Gitea existiert.
CI-Images nach konairius/, von der Pipeline mit einem Digest auf Gitea UND ghcr.io gepusht. Manifeste zeigen weiter auf Gitea; der Node-Fallback probiert Gitea zuerst, ghcr als zweites — containerd behält den Repository-Pfad beim Fallthrough. Steady-State verlässt das Netz nie.
system-node-critical für Multus/whereabouts/cni-installer: upstream Priorität 0 ließ die Pods bei Speicherdruck Pending stehen — kein Multus, keine Attachments, nichts meldet warum.
Zu den Review-Blockern:
Fallback-Mechanismus nicht im Repo: korrekt — machine.registries.mirrors schreibt die bootstrap.sh aus #798 (registry_auth_patch, fünfter --config-patch). Merge-Reihenfolge: #798 zuerst. Auf dem laufenden Cluster ist der Fallback ohnehin nur beim Rebuild relevant; steady-state zieht Gitea direkt.
gitea_admin kann nicht nach konairius/ pushen: der Header-Kommentar war stale und ist korrigiert — konairius ist eine Gitea-Organisation (vom Betreiber am 2026-09-03 für den Mirror-Pfad angelegt, gitea_admin ist Mitglied). Der Pipeline-Lauf vom 2026-09-04 hat die drei Images mit identischem Digest auf beide Registries gepusht; die ghcr-Seite ist zusätzlich durch die Hetzner-Läufe bewiesen (Pull über den Fallback). ghcr_mirror_token existiert als globales Woodpecker-Secret (vom Betreiber angelegt).
ci/images/vmtest fehlte: das Containerfile ist jetzt hier im PR, beim Pipeline-Step, statt in #798.
Vor dem Merge einmal ansehen:gitea.renner.uno/konairius/{cni-plugins,alpine-jq,alpine-openssl}:2026-09-04 — claude-bot fehlt der Package-Scope zur API-Verifikation.
Rollout-Effekt live: drei CNI-DaemonSets starten einmal neu; Init-Container ziehen den neuen Tag beim nächsten Pod-Neustart. paperless/renderer/z2m sind gegen aktuellen master rebasiert (AI-Config und Image-Automation unangetastet).
Zwei Resilienz-Fixes aus den Rebuild-Proben: was ein Kaltstart braucht, bevor Gitea existiert.
- **CI-Images nach `konairius/`**, von der Pipeline mit einem Digest auf Gitea UND ghcr.io gepusht. Manifeste zeigen weiter auf Gitea; der Node-Fallback probiert Gitea zuerst, ghcr als zweites — containerd behält den Repository-Pfad beim Fallthrough. Steady-State verlässt das Netz nie.
- **`system-node-critical` für Multus/whereabouts/cni-installer**: upstream Priorität 0 ließ die Pods bei Speicherdruck Pending stehen — kein Multus, keine Attachments, nichts meldet warum.
**Zu den Review-Blockern:**
1. *Fallback-Mechanismus nicht im Repo*: korrekt — `machine.registries.mirrors` schreibt die bootstrap.sh aus **#798** (`registry_auth_patch`, fünfter `--config-patch`). **Merge-Reihenfolge: #798 zuerst.** Auf dem laufenden Cluster ist der Fallback ohnehin nur beim Rebuild relevant; steady-state zieht Gitea direkt.
2. *gitea_admin kann nicht nach konairius/ pushen*: der Header-Kommentar war stale und ist korrigiert — `konairius` ist eine Gitea-**Organisation** (vom Betreiber am 2026-09-03 für den Mirror-Pfad angelegt, gitea_admin ist Mitglied). Der Pipeline-Lauf vom 2026-09-04 hat die drei Images mit identischem Digest auf beide Registries gepusht; die ghcr-Seite ist zusätzlich durch die Hetzner-Läufe bewiesen (Pull über den Fallback). `ghcr_mirror_token` existiert als globales Woodpecker-Secret (vom Betreiber angelegt).
3. *ci/images/vmtest fehlte*: das Containerfile ist jetzt hier im PR, beim Pipeline-Step, statt in #798.
**Vor dem Merge einmal ansehen:** `gitea.renner.uno/konairius/{cni-plugins,alpine-jq,alpine-openssl}:2026-09-04` — claude-bot fehlt der Package-Scope zur API-Verifikation.
Rollout-Effekt live: drei CNI-DaemonSets starten einmal neu; Init-Container ziehen den neuen Tag beim nächsten Pod-Neustart. paperless/renderer/z2m sind gegen aktuellen master rebasiert (AI-Config und Image-Automation unangetastet).
Gitea runs inside the cluster, so during a cold rebuild every image that
only exists at gitea.renner.uno is unpullable — a rehearsal died with the
cni-plugins installer in ImagePullBackOff, no macvlan binary, no ceph-san
attachment, CephCluster stuck at "Configuring Ceph Mons", OpenBao waiting
forever on a PVC. The failure surfaced four layers from its cause.
The CI images move to konairius/, published by the build pipeline to
Gitea AND ghcr.io with one digest. The manifests keep pointing at Gitea;
the node-level fallback (machine.registries.mirrors, written by
bootstrap.sh at install time) tries Gitea first and ghcr second, and
containerd keeps the repository path when it falls through — so the same
path resolves on both. Steady state never leaves the network.
The Multus, whereabouts and cni-plugins-installer DaemonSets also gain
priorityClassName system-node-critical. Upstream ships them at priority
0: on a rebuild two nodes sat at 99% memory requests when the DaemonSet
rolled, both multus pods stayed Pending, and every pod needing a
secondary attachment waited on an annotation that could never come — with
nothing reporting why.
Rollout effect on the live cluster: the three CNI DaemonSets restart once
(existing pods keep their networking; only new attachments wait), and the
three init containers pull the new tag on their next pod restart. The
ghcr side is proven by the Hetzner throwaway runs, which pull through the
fallback; eyeball gitea.renner.uno/konairius packages before merging.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🤖Automated review — model CLI default (complexity: complex), commit 23a6557aea77208f85ccda5bb1288ccc4859c16f.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
This PR aims to make cold-cluster-rebuild recovery work by (a) mirroring four CI-built images to a konairius/ namespace on both Gitea and ghcr.io so containerd can fall back to ghcr.io when Gitea isn't up yet, and (b) giving the three CNI DaemonSets (multus, whereabouts, cni-plugins-installer) system-node-critical so they aren't left Pending under memory pressure. The CNI priority-class change is sound and consistent with existing conventions (cni-dhcp-daemon already carries the same class, and the stated priority-value ordering vs. Talos's static pods is correct). The image-mirroring half, however, rests on a fallback mechanism that does not exist anywhere in this repository, and it reuses a Gitea credential that the repo's own comments say cannot publish to that namespace — both would leave the cold-rebuild scenario this PR claims to fix still broken. There is also an unrelated, currently-broken CI step bundled into the same commit.
🔴 Blocking
The node-side registry fallback the whole fix depends on isn't configured anywhere. Every changed manifest (kubernetes/apps/home-assistant/zigbee2mqtt-deployment.yaml:64, kubernetes/apps/paperless/paperless.yaml:154, kubernetes/apps/trmnl/renderer.yaml:184, kubernetes/infrastructure/multus/cni-plugins-installer.yaml:65, and .woodpecker/build-ci-images.yml:187) asserts that "machine.registries.mirrors, applied by bootstrap.sh" makes containerd try Gitea first and fall through to ghcr.io. kubernetes/bootstrap.sh:266-272 applies exactly four Talos config patches to every node: cluster.yaml, site-${SITE}.yaml, containerd-layer-retention.yaml, and ${node}.yaml. None of these four files (checked all of them, plus every other file under kubernetes/talos/patches/) contains a registries/mirrors block of any kind — containerd-layer-retention.yaml:5 in fact says explicitly "so no machine.registries.mirrors block is needed here", which is the opposite of what this PR's comments now claim is in effect. Without that mirror config, containerd has no fallback endpoint at all; pulling gitea.renner.uno/konairius/... during a cold rebuild will fail exactly as it did before this change (Gitea still doesn't exist yet), just under a different repository path. The rehearsal the PR body describes may have used a manually-applied node config that was never committed — if so, that's the actual missing piece of this PR.
The new Gitea push destinations likely fail authorization..woodpecker/build-ci-images.yml:221-226 (and the equivalent blocks for cni-plugins and alpine-openssl) add --destination gitea.renner.uno/konairius/<image> using the sameGITEA_USER: gitea_admin / ci_images_token credential used for the existing gitea_admin/ destinations — nothing in the diff changes which account owns that token. The file's own header comment, four lines above the when: block this PR doesn't touch (.woodpecker/build-ci-images.yml:21-23), states as a known fact: "Gitea scopes packages to a user or org namespace and these images live under gitea_admin/* — a different account cannot publish there." If konairius is a distinct Gitea user (as it is on GitHub, per bootstrap.sh:72) rather than an org gitea_admin belongs to, the konairius/ pushes in all three modified kaniko steps will fail with a permission error, and the PR's own pre-merge checklist item ("Gitea-Seite konnte claude-bot ... nicht verifizieren") doesn't cover this — it only asks to check the images exist, not whether the push can succeed at all.
build-vmtest (.woodpecker/build-ci-images.yml:9-32) references a directory that does not exist.directory: ci/images/vmtest with --dockerfile Containerfile in that directory — ci/images/vmtest/ is absent from the checkout (git log shows no history for that path either), while every sibling directory referenced by other steps (ci/images/alpine-jq, ci/images/cni-plugins, etc.) does exist. This step is unrelated to the PR's stated topic (image resilience / CNI priority) and isn't mentioned in the PR body at all. Because the top-level when: block triggers every step unconditionally on event: deployment (line 34-35, no path filter applies there), the next full build-ci-images deployment run will fail on this step. This looks like an accidental carry-over from a rebase rather than an intended part of this change.
🟡 Suggestions
Given the two points above, it would be worth actually re-running the cold-rebuild rehearsal against this branch's committed state (not a manually-patched node) before merging — the PR body's confidence ("Rollout-Effekt live") describes a rehearsal, but the committed bootstrap.sh/patch files don't produce the mechanism being relied on.
If ghcr_mirror_token is a new Woodpecker secret, confirm it's actually been created in Woodpecker's secret store — the diff only adds from_secret: ghcr_mirror_token references (.woodpecker/build-ci-images.yml:209, 262, 304), nothing provisions it.
Tests
No automated tests are applicable to this GitOps repo beyond the CI pipeline itself and the manual rehearsal described in the PR body. Given the two correctness concerns above, the pipeline run and a real cold-rebuild rehearsal against this exact commit (not a hand-patched node) are the tests that matter here, and neither is confirmed to have run against what's actually committed.
Recommendation
Request changes — 3 blocking issues.
🤖 **Automated review** — model `CLI default` (complexity: complex), commit `23a6557aea77208f85ccda5bb1288ccc4859c16f`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
This PR aims to make cold-cluster-rebuild recovery work by (a) mirroring four CI-built images to a `konairius/` namespace on both Gitea and ghcr.io so containerd can fall back to ghcr.io when Gitea isn't up yet, and (b) giving the three CNI DaemonSets (`multus`, `whereabouts`, `cni-plugins-installer`) `system-node-critical` so they aren't left Pending under memory pressure. The CNI priority-class change is sound and consistent with existing conventions (`cni-dhcp-daemon` already carries the same class, and the stated priority-value ordering vs. Talos's static pods is correct). The image-mirroring half, however, rests on a fallback mechanism that does not exist anywhere in this repository, and it reuses a Gitea credential that the repo's own comments say cannot publish to that namespace — both would leave the cold-rebuild scenario this PR claims to fix still broken. There is also an unrelated, currently-broken CI step bundled into the same commit.
## 🔴 Blocking
- **The node-side registry fallback the whole fix depends on isn't configured anywhere.** Every changed manifest (`kubernetes/apps/home-assistant/zigbee2mqtt-deployment.yaml:64`, `kubernetes/apps/paperless/paperless.yaml:154`, `kubernetes/apps/trmnl/renderer.yaml:184`, `kubernetes/infrastructure/multus/cni-plugins-installer.yaml:65`, and `.woodpecker/build-ci-images.yml:187`) asserts that "`machine.registries.mirrors`, applied by bootstrap.sh" makes containerd try Gitea first and fall through to ghcr.io. `kubernetes/bootstrap.sh:266-272` applies exactly four Talos config patches to every node: `cluster.yaml`, `site-${SITE}.yaml`, `containerd-layer-retention.yaml`, and `${node}.yaml`. None of these four files (checked all of them, plus every other file under `kubernetes/talos/patches/`) contains a `registries`/`mirrors` block of any kind — `containerd-layer-retention.yaml:5` in fact says explicitly "*so no `machine.registries.mirrors` block is needed here*", which is the opposite of what this PR's comments now claim is in effect. Without that mirror config, containerd has no fallback endpoint at all; pulling `gitea.renner.uno/konairius/...` during a cold rebuild will fail exactly as it did before this change (Gitea still doesn't exist yet), just under a different repository path. The rehearsal the PR body describes may have used a manually-applied node config that was never committed — if so, that's the actual missing piece of this PR.
- **The new Gitea push destinations likely fail authorization.** `.woodpecker/build-ci-images.yml:221-226` (and the equivalent blocks for `cni-plugins` and `alpine-openssl`) add `--destination gitea.renner.uno/konairius/<image>` using the *same* `GITEA_USER: gitea_admin` / `ci_images_token` credential used for the existing `gitea_admin/` destinations — nothing in the diff changes which account owns that token. The file's own header comment, four lines above the `when:` block this PR doesn't touch (`.woodpecker/build-ci-images.yml:21-23`), states as a known fact: *"Gitea scopes packages to a user or org namespace and these images live under `gitea_admin/*` — a different account cannot publish there."* If `konairius` is a distinct Gitea user (as it is on GitHub, per `bootstrap.sh:72`) rather than an org `gitea_admin` belongs to, the `konairius/` pushes in all three modified kaniko steps will fail with a permission error, and the PR's own pre-merge checklist item ("Gitea-Seite konnte claude-bot ... nicht verifizieren") doesn't cover this — it only asks to check the images exist, not whether the push can succeed at all.
- **`build-vmtest` (`.woodpecker/build-ci-images.yml:9-32`) references a directory that does not exist.** `directory: ci/images/vmtest` with `--dockerfile Containerfile` in that directory — `ci/images/vmtest/` is absent from the checkout (`git log` shows no history for that path either), while every sibling directory referenced by other steps (`ci/images/alpine-jq`, `ci/images/cni-plugins`, etc.) does exist. This step is unrelated to the PR's stated topic (image resilience / CNI priority) and isn't mentioned in the PR body at all. Because the top-level `when:` block triggers **every** step unconditionally on `event: deployment` (line 34-35, no path filter applies there), the next full `build-ci-images` deployment run will fail on this step. This looks like an accidental carry-over from a rebase rather than an intended part of this change.
## 🟡 Suggestions
- Given the two points above, it would be worth actually re-running the cold-rebuild rehearsal against this branch's committed state (not a manually-patched node) before merging — the PR body's confidence ("Rollout-Effekt live") describes a rehearsal, but the committed `bootstrap.sh`/patch files don't produce the mechanism being relied on.
- If `ghcr_mirror_token` is a new Woodpecker secret, confirm it's actually been created in Woodpecker's secret store — the diff only adds `from_secret: ghcr_mirror_token` references (`.woodpecker/build-ci-images.yml:209`, `262`, `304`), nothing provisions it.
## Tests
No automated tests are applicable to this GitOps repo beyond the CI pipeline itself and the manual rehearsal described in the PR body. Given the two correctness concerns above, the pipeline run and a real cold-rebuild rehearsal against this exact commit (not a hand-patched node) are the tests that matter here, and neither is confirmed to have run against what's actually committed.
## Recommendation
Request changes — 3 blocking issues.
<!-- pr-review:sha=23a6557aea77208f85ccda5bb1288ccc4859c16f -->
🤖Automated review — model CLI default (complexity: complex), commit 26ae7502197c665a5de5f4dcced0e4779eb8bedd.
Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.
Summary
Two independent resilience fixes for cold-cluster-rebuild scenarios: (1) four runtime images (alpine-jq, cni-plugins, alpine-openssl, and a new ci-vmtest) gain a konairius/ namespace pushed to both Gitea and ghcr.io, with four consumer manifests re-pointed at konairius/...:2026-09-04 so a node can fall back to ghcr when Gitea itself is down; (2) system-node-critical is added to the Multus/whereabouts/cni-plugins-installer DaemonSets so they aren't left Pending under memory pressure. The reasoning in the comments checks out against the rest of the repo (TALOS_VERSION pin matches cluster.yaml, the older talosctl pin in ci-k8s is real, cni-dhcp-daemon.yaml already carries system-node-critical). The one thing I can't verify from a static checkout — and neither could the PR author, per the PR body — is whether the four konairius/...:2026-09-04 images actually exist on both registries yet; the four manifest changes are a hard dependency on that.
🔴 Blocking
None found as concrete defects in this diff. The two items below are worth resolving before merge but aren't things I can prove will misbehave from static review alone.
🟡 Suggestions
.woodpecker/build-ci-images.yml:92 — build-vmtest uses a different, archived kaniko image. Every other step in this file uses registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug, and the file's own header comment (lines 4–13) explains at length why: Google archived GoogleContainerTools/kaniko on 2025-06-03, and #552 moved every step off it. build-vmtest uses gcr.io/kaniko-project/executor:v1.23.2-debug — the exact archived image the header warns about — with no comment explaining the deviation. It will likely still pull and work today (the tag predates the archival), but it quietly reintroduces the dependency #552 removed, for no stated reason, in a step added by this same PR.
Unverified image existence is the crux of this fix. The PR body says the 2026-09-04 pipeline run pushed the three existing images to konairius/ on both registries, but also that claude-bot lacks the package scope to confirm via the Gitea API, and I have no network access from this checkout to check either registry directly. zigbee2mqtt-deployment.yaml:63, paperless.yaml:157, renderer.yaml:183, and cni-plugins-installer.yaml:53 all now hard-pin konairius/...:2026-09-04 with no fallback to the previous gitea_admin/...:2026-07-30 tag. If any one of the three images didn't actually land in konairius/ on Gitea, those pods go straight to ImagePullBackOff on next rollout — in steady state, not just during a rebuild — which is a regression from the working gitea_admin/ tags they replace. Worth a manual crane digest/docker manifest inspect check against both registries before merging, exactly as the PR body already flags.
ci/images/vmtest/Containerfile — no consumer yet. The image is built and pushed here, but nothing in this PR (or, as far as I can find, in the repo) runs bootstrap.sh vmtest as a cluster Job against it — ci-vmtest isn't referenced anywhere outside this Containerfile and the new pipeline step. That's consistent with the PR body's framing (the Job itself apparently lives in #798), so not a defect, just flagging that this half of the change is inert until that lands.
🟢 Nits
The talosctl/flux/kopia sha256 pins in ci/images/vmtest/Containerfile (lines 259, 272, 287 of the diff) follow the repo's existing checksum-verification convention correctly, but I had no network access to check them against the actual upstream release checksums — worth a quick manual diff against the published sha256sum.txt/checksums.txt for each release, since a wrong pin here fails closed (safe) but a maliciously correct-looking wrong pin would not be caught by anything else in this PR.
Tests
No automated test exercises the registry-fallback path or confirms the konairius/ images resolve — by the repo's own admission (ci-manifests-test.yml header), manifest validation here is deliberately static/offline (kustomize build + lint), so it will catch YAML mistakes but not a missing image tag. That's consistent with the rest of the repo's testing story, not a gap introduced by this PR, but it means the image-existence risk above ships unguarded by CI and depends on the manual check the PR body already calls for.
Recommendation
Looks reasonable — no blocking issues; comments above (especially confirming the konairius/ images exist before merge, and reconsidering the build-vmtest kaniko image) are worth resolving first.
🤖 **Automated review** — model `CLI default` (complexity: complex), commit `26ae7502197c665a5de5f4dcced0e4779eb8bedd`.
> Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review.
---
## Summary
Two independent resilience fixes for cold-cluster-rebuild scenarios: (1) four runtime images (`alpine-jq`, `cni-plugins`, `alpine-openssl`, and a new `ci-vmtest`) gain a `konairius/` namespace pushed to both Gitea and ghcr.io, with four consumer manifests re-pointed at `konairius/...:2026-09-04` so a node can fall back to ghcr when Gitea itself is down; (2) `system-node-critical` is added to the Multus/whereabouts/cni-plugins-installer DaemonSets so they aren't left Pending under memory pressure. The reasoning in the comments checks out against the rest of the repo (TALOS_VERSION pin matches `cluster.yaml`, the older talosctl pin in `ci-k8s` is real, `cni-dhcp-daemon.yaml` already carries `system-node-critical`). The one thing I can't verify from a static checkout — and neither could the PR author, per the PR body — is whether the four `konairius/...:2026-09-04` images actually exist on both registries yet; the four manifest changes are a hard dependency on that.
## 🔴 Blocking
None found as concrete defects in this diff. The two items below are worth resolving before merge but aren't things I can prove will misbehave from static review alone.
## 🟡 Suggestions
- **`.woodpecker/build-ci-images.yml:92` — `build-vmtest` uses a different, archived kaniko image.** Every other step in this file uses `registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug`, and the file's own header comment (lines 4–13) explains at length why: Google archived `GoogleContainerTools/kaniko` on 2025-06-03, and #552 moved every step off it. `build-vmtest` uses `gcr.io/kaniko-project/executor:v1.23.2-debug` — the exact archived image the header warns about — with no comment explaining the deviation. It will likely still pull and work today (the tag predates the archival), but it quietly reintroduces the dependency #552 removed, for no stated reason, in a step added by this same PR.
- **Unverified image existence is the crux of this fix.** The PR body says the 2026-09-04 pipeline run pushed the three existing images to `konairius/` on both registries, but also that claude-bot lacks the package scope to confirm via the Gitea API, and I have no network access from this checkout to check either registry directly. `zigbee2mqtt-deployment.yaml:63`, `paperless.yaml:157`, `renderer.yaml:183`, and `cni-plugins-installer.yaml:53` all now hard-pin `konairius/...:2026-09-04` with no fallback to the previous `gitea_admin/...:2026-07-30` tag. If any one of the three images didn't actually land in `konairius/` on Gitea, those pods go straight to `ImagePullBackOff` on next rollout — in steady state, not just during a rebuild — which is a regression from the working `gitea_admin/` tags they replace. Worth a manual `crane digest`/`docker manifest inspect` check against both registries before merging, exactly as the PR body already flags.
- **`ci/images/vmtest/Containerfile` — no consumer yet.** The image is built and pushed here, but nothing in this PR (or, as far as I can find, in the repo) runs `bootstrap.sh vmtest` as a cluster Job against it — `ci-vmtest` isn't referenced anywhere outside this Containerfile and the new pipeline step. That's consistent with the PR body's framing (the Job itself apparently lives in #798), so not a defect, just flagging that this half of the change is inert until that lands.
## 🟢 Nits
- The `talosctl`/`flux`/`kopia` sha256 pins in `ci/images/vmtest/Containerfile` (lines 259, 272, 287 of the diff) follow the repo's existing checksum-verification convention correctly, but I had no network access to check them against the actual upstream release checksums — worth a quick manual diff against the published `sha256sum.txt`/`checksums.txt` for each release, since a wrong pin here fails closed (safe) but a maliciously *correct*-looking wrong pin would not be caught by anything else in this PR.
## Tests
No automated test exercises the registry-fallback path or confirms the `konairius/` images resolve — by the repo's own admission (`ci-manifests-test.yml` header), manifest validation here is deliberately static/offline (`kustomize build` + lint), so it will catch YAML mistakes but not a missing image tag. That's consistent with the rest of the repo's testing story, not a gap introduced by this PR, but it means the image-existence risk above ships unguarded by CI and depends on the manual check the PR body already calls for.
## Recommendation
Looks reasonable — no blocking issues; comments above (especially confirming the `konairius/` images exist before merge, and reconsidering the `build-vmtest` kaniko image) are worth resolving first.
<!-- pr-review:sha=26ae7502197c665a5de5f4dcced0e4779eb8bedd -->
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Zwei Resilienz-Fixes aus den Rebuild-Proben: was ein Kaltstart braucht, bevor Gitea existiert.
konairius/, von der Pipeline mit einem Digest auf Gitea UND ghcr.io gepusht. Manifeste zeigen weiter auf Gitea; der Node-Fallback probiert Gitea zuerst, ghcr als zweites — containerd behält den Repository-Pfad beim Fallthrough. Steady-State verlässt das Netz nie.system-node-criticalfür Multus/whereabouts/cni-installer: upstream Priorität 0 ließ die Pods bei Speicherdruck Pending stehen — kein Multus, keine Attachments, nichts meldet warum.Zu den Review-Blockern:
machine.registries.mirrorsschreibt die bootstrap.sh aus #798 (registry_auth_patch, fünfter--config-patch). Merge-Reihenfolge: #798 zuerst. Auf dem laufenden Cluster ist der Fallback ohnehin nur beim Rebuild relevant; steady-state zieht Gitea direkt.konairiusist eine Gitea-Organisation (vom Betreiber am 2026-09-03 für den Mirror-Pfad angelegt, gitea_admin ist Mitglied). Der Pipeline-Lauf vom 2026-09-04 hat die drei Images mit identischem Digest auf beide Registries gepusht; die ghcr-Seite ist zusätzlich durch die Hetzner-Läufe bewiesen (Pull über den Fallback).ghcr_mirror_tokenexistiert als globales Woodpecker-Secret (vom Betreiber angelegt).Vor dem Merge einmal ansehen:
gitea.renner.uno/konairius/{cni-plugins,alpine-jq,alpine-openssl}:2026-09-04— claude-bot fehlt der Package-Scope zur API-Verifikation.Rollout-Effekt live: drei CNI-DaemonSets starten einmal neu; Init-Container ziehen den neuen Tag beim nächsten Pod-Neustart. paperless/renderer/z2m sind gegen aktuellen master rebasiert (AI-Config und Image-Automation unangetastet).
🤖 Automated review — model
CLI default(complexity: complex), commit23a6557aea77208f85ccda5bb1288ccc4859c16f.Summary
This PR aims to make cold-cluster-rebuild recovery work by (a) mirroring four CI-built images to a
konairius/namespace on both Gitea and ghcr.io so containerd can fall back to ghcr.io when Gitea isn't up yet, and (b) giving the three CNI DaemonSets (multus,whereabouts,cni-plugins-installer)system-node-criticalso they aren't left Pending under memory pressure. The CNI priority-class change is sound and consistent with existing conventions (cni-dhcp-daemonalready carries the same class, and the stated priority-value ordering vs. Talos's static pods is correct). The image-mirroring half, however, rests on a fallback mechanism that does not exist anywhere in this repository, and it reuses a Gitea credential that the repo's own comments say cannot publish to that namespace — both would leave the cold-rebuild scenario this PR claims to fix still broken. There is also an unrelated, currently-broken CI step bundled into the same commit.🔴 Blocking
The node-side registry fallback the whole fix depends on isn't configured anywhere. Every changed manifest (
kubernetes/apps/home-assistant/zigbee2mqtt-deployment.yaml:64,kubernetes/apps/paperless/paperless.yaml:154,kubernetes/apps/trmnl/renderer.yaml:184,kubernetes/infrastructure/multus/cni-plugins-installer.yaml:65, and.woodpecker/build-ci-images.yml:187) asserts that "machine.registries.mirrors, applied by bootstrap.sh" makes containerd try Gitea first and fall through to ghcr.io.kubernetes/bootstrap.sh:266-272applies exactly four Talos config patches to every node:cluster.yaml,site-${SITE}.yaml,containerd-layer-retention.yaml, and${node}.yaml. None of these four files (checked all of them, plus every other file underkubernetes/talos/patches/) contains aregistries/mirrorsblock of any kind —containerd-layer-retention.yaml:5in fact says explicitly "so nomachine.registries.mirrorsblock is needed here", which is the opposite of what this PR's comments now claim is in effect. Without that mirror config, containerd has no fallback endpoint at all; pullinggitea.renner.uno/konairius/...during a cold rebuild will fail exactly as it did before this change (Gitea still doesn't exist yet), just under a different repository path. The rehearsal the PR body describes may have used a manually-applied node config that was never committed — if so, that's the actual missing piece of this PR.The new Gitea push destinations likely fail authorization.
.woodpecker/build-ci-images.yml:221-226(and the equivalent blocks forcni-pluginsandalpine-openssl) add--destination gitea.renner.uno/konairius/<image>using the sameGITEA_USER: gitea_admin/ci_images_tokencredential used for the existinggitea_admin/destinations — nothing in the diff changes which account owns that token. The file's own header comment, four lines above thewhen:block this PR doesn't touch (.woodpecker/build-ci-images.yml:21-23), states as a known fact: "Gitea scopes packages to a user or org namespace and these images live undergitea_admin/*— a different account cannot publish there." Ifkonairiusis a distinct Gitea user (as it is on GitHub, perbootstrap.sh:72) rather than an orggitea_adminbelongs to, thekonairius/pushes in all three modified kaniko steps will fail with a permission error, and the PR's own pre-merge checklist item ("Gitea-Seite konnte claude-bot ... nicht verifizieren") doesn't cover this — it only asks to check the images exist, not whether the push can succeed at all.build-vmtest(.woodpecker/build-ci-images.yml:9-32) references a directory that does not exist.directory: ci/images/vmtestwith--dockerfile Containerfilein that directory —ci/images/vmtest/is absent from the checkout (git logshows no history for that path either), while every sibling directory referenced by other steps (ci/images/alpine-jq,ci/images/cni-plugins, etc.) does exist. This step is unrelated to the PR's stated topic (image resilience / CNI priority) and isn't mentioned in the PR body at all. Because the top-levelwhen:block triggers every step unconditionally onevent: deployment(line 34-35, no path filter applies there), the next fullbuild-ci-imagesdeployment run will fail on this step. This looks like an accidental carry-over from a rebase rather than an intended part of this change.🟡 Suggestions
bootstrap.sh/patch files don't produce the mechanism being relied on.ghcr_mirror_tokenis a new Woodpecker secret, confirm it's actually been created in Woodpecker's secret store — the diff only addsfrom_secret: ghcr_mirror_tokenreferences (.woodpecker/build-ci-images.yml:209,262,304), nothing provisions it.Tests
No automated tests are applicable to this GitOps repo beyond the CI pipeline itself and the manual rehearsal described in the PR body. Given the two correctness concerns above, the pipeline run and a real cold-rebuild rehearsal against this exact commit (not a hand-patched node) are the tests that matter here, and neither is confirmed to have run against what's actually committed.
Recommendation
Request changes — 3 blocking issues.
23a6557aeato26ae750219🤖 Automated review — model
CLI default(complexity: complex), commit26ae7502197c665a5de5f4dcced0e4779eb8bedd.Summary
Two independent resilience fixes for cold-cluster-rebuild scenarios: (1) four runtime images (
alpine-jq,cni-plugins,alpine-openssl, and a newci-vmtest) gain akonairius/namespace pushed to both Gitea and ghcr.io, with four consumer manifests re-pointed atkonairius/...:2026-09-04so a node can fall back to ghcr when Gitea itself is down; (2)system-node-criticalis added to the Multus/whereabouts/cni-plugins-installer DaemonSets so they aren't left Pending under memory pressure. The reasoning in the comments checks out against the rest of the repo (TALOS_VERSION pin matchescluster.yaml, the older talosctl pin inci-k8sis real,cni-dhcp-daemon.yamlalready carriessystem-node-critical). The one thing I can't verify from a static checkout — and neither could the PR author, per the PR body — is whether the fourkonairius/...:2026-09-04images actually exist on both registries yet; the four manifest changes are a hard dependency on that.🔴 Blocking
None found as concrete defects in this diff. The two items below are worth resolving before merge but aren't things I can prove will misbehave from static review alone.
🟡 Suggestions
.woodpecker/build-ci-images.yml:92—build-vmtestuses a different, archived kaniko image. Every other step in this file usesregistry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug, and the file's own header comment (lines 4–13) explains at length why: Google archivedGoogleContainerTools/kanikoon 2025-06-03, and #552 moved every step off it.build-vmtestusesgcr.io/kaniko-project/executor:v1.23.2-debug— the exact archived image the header warns about — with no comment explaining the deviation. It will likely still pull and work today (the tag predates the archival), but it quietly reintroduces the dependency #552 removed, for no stated reason, in a step added by this same PR.Unverified image existence is the crux of this fix. The PR body says the 2026-09-04 pipeline run pushed the three existing images to
konairius/on both registries, but also that claude-bot lacks the package scope to confirm via the Gitea API, and I have no network access from this checkout to check either registry directly.zigbee2mqtt-deployment.yaml:63,paperless.yaml:157,renderer.yaml:183, andcni-plugins-installer.yaml:53all now hard-pinkonairius/...:2026-09-04with no fallback to the previousgitea_admin/...:2026-07-30tag. If any one of the three images didn't actually land inkonairius/on Gitea, those pods go straight toImagePullBackOffon next rollout — in steady state, not just during a rebuild — which is a regression from the workinggitea_admin/tags they replace. Worth a manualcrane digest/docker manifest inspectcheck against both registries before merging, exactly as the PR body already flags.ci/images/vmtest/Containerfile— no consumer yet. The image is built and pushed here, but nothing in this PR (or, as far as I can find, in the repo) runsbootstrap.sh vmtestas a cluster Job against it —ci-vmtestisn't referenced anywhere outside this Containerfile and the new pipeline step. That's consistent with the PR body's framing (the Job itself apparently lives in #798), so not a defect, just flagging that this half of the change is inert until that lands.🟢 Nits
talosctl/flux/kopiasha256 pins inci/images/vmtest/Containerfile(lines 259, 272, 287 of the diff) follow the repo's existing checksum-verification convention correctly, but I had no network access to check them against the actual upstream release checksums — worth a quick manual diff against the publishedsha256sum.txt/checksums.txtfor each release, since a wrong pin here fails closed (safe) but a maliciously correct-looking wrong pin would not be caught by anything else in this PR.Tests
No automated test exercises the registry-fallback path or confirms the
konairius/images resolve — by the repo's own admission (ci-manifests-test.ymlheader), manifest validation here is deliberately static/offline (kustomize build+ lint), so it will catch YAML mistakes but not a missing image tag. That's consistent with the rest of the repo's testing story, not a gap introduced by this PR, but it means the image-existence risk above ships unguarded by CI and depends on the manual check the PR body already calls for.Recommendation
Looks reasonable — no blocking issues; comments above (especially confirming the
konairius/images exist before merge, and reconsidering thebuild-vmtestkaniko image) are worth resolving first.