fix(bird): reference the image by digest, the dated tag resolves stale #853

2026-09-16T08:54:13Z

claude-bot commented

2026-09-16 08:54:13 +00:00

The DaemonSet from #852 is live and failing: Init:Error, /bin/sh: kubectl: not found.

What happened

The merge build pushed sha256:3bc85006… for bird:2026-09-16. Every pod resolved that tag to sha256:75e19214… — the first build of the same day, from before kubectl was added.

imagePullPolicy: Always does not help. It re-pulls, but the tag itself resolves to the stale digest. Evidence:

three pods, three nodes, 2–3 restarts each: all 75e19214
deleted all three; the fresh ones resolved to 75e19214 again
a one-off pod pulling @sha256:3bc85006… directly ran kubectl v1.33.1 and BIRD 2.17.2

So the registry holds the right image; only tag resolution is stale.

Why the dated tag cannot carry this

TAG=$(date +%Y-%m-%d) is mutable within the day, and this image was rebuilt three times today. Any same-day rebuild produces a tag that some nodes have already resolved.

Referencing the digest makes the deployment deterministic.

The cost, stated

The digest has to be updated whenever the image is rebuilt. The obvious alternative — tagging by commit — does not work for a manifest that lives in the same commit as the build. This is a narrow pin for one image, not a change to the repo's dated-tag convention.

Refs #576

🤖 Generated with Claude Code

The DaemonSet from #852 is live and failing: `Init:Error`, `/bin/sh: kubectl: not found`. ## What happened The merge build pushed `sha256:3bc85006…` for `bird:2026-09-16`. Every pod resolved that tag to `sha256:75e19214…` — the **first** build of the same day, from before `kubectl` was added. `imagePullPolicy: Always` does not help. It re-pulls, but the tag itself resolves to the stale digest. Evidence: - three pods, three nodes, 2–3 restarts each: all `75e19214` - deleted all three; the fresh ones resolved to `75e19214` again - a one-off pod pulling `@sha256:3bc85006…` directly ran `kubectl v1.33.1` and `BIRD 2.17.2` So the registry holds the right image; only tag resolution is stale. ## Why the dated tag cannot carry this `TAG=$(date +%Y-%m-%d)` is mutable within the day, and this image was rebuilt three times today. Any same-day rebuild produces a tag that some nodes have already resolved. Referencing the digest makes the deployment deterministic. ## The cost, stated The digest has to be updated whenever the image is rebuilt. The obvious alternative — tagging by commit — does not work for a manifest that lives in the same commit as the build. This is a narrow pin for one image, not a change to the repo's dated-tag convention. Refs #576 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-09-16 08:54:13 +00:00

fix(bird): reference the image by digest, the dated tag resolves stale

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

ci/woodpecker/pr/ci-manifests-test Pipeline was successful

Details

ci/woodpecker/pr/pr-review Pipeline was successful

Details

c1ccf228b7

The DaemonSet came up with Init:Error, kubectl: not found. The build had pushed sha256:3bc85006 for bird:2026-09-16, but every pod resolved that tag to sha256:75e19214 — the first build of the same day, from before kubectl was added.

imagePullPolicy: Always does not help: it re-pulls, but the tag itself resolves to the stale digest. Deleting the pods changed nothing; three fresh pods on three nodes all resolved to the old digest. A one-off pod pulling the digest directly ran kubectl 1.33.1 and BIRD 2.17.2, so the registry holds the right image and only tag resolution is stale.

The dated tag cannot carry a same-day rebuild, and this image was rebuilt three times today. Referencing the digest makes the deployment deterministic. The cost is that the reference has to be updated whenever the image is rebuilt; the alternative, tagging by commit, does not work for a manifest living in the same commit.

Refs #576

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

k.renner merged commit df9019d90e into master

2026-09-16 08:54:38 +00:00

k.renner referenced this issue from a commit

2026-09-16 08:54:38 +00:00

Merge pull request 'fix(bird): reference the image by digest, the dated tag resolves stale' (#853) from fix/bird-digest into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#853