claude-botandClaude Opus 5.5
cdf0b5ffca
Cold-rebuild and HA-rollback runbooks, README and the image-automation README name main and konairius/homelab. The plan records the Flux ref patch and the cron API calls Cut-over 2 needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
72 lines
3.0 KiB
Markdown
72 lines
3.0 KiB
Markdown
# Runbook — roll back a Home Assistant image bump
|
|
|
|
Use when an automatic Home Assistant update (a
|
|
`chore(image-automation): bump ghcr.io/home-assistant/home-assistant:<new>`
|
|
commit on `main`) breaks Home Assistant. The owner approves the rollback
|
|
and its timing. Atlas writes the PR; Argus merges it.
|
|
|
|
## Why a plain `git revert` is not enough
|
|
|
|
Image automation (`kubernetes/infrastructure/image-automation/`) re-scans
|
|
the registry every hour and rewrites the tag every 30 minutes to the
|
|
highest release matching the `home-assistant` ImagePolicy. A plain revert
|
|
of the bump commit would be bumped forward again within about an hour.
|
|
|
|
So the rollback commit does two things at once: it reverts the tag **and**
|
|
drops the `# {"$imagepolicy": ...}` marker from that image line. Without
|
|
the marker the Setters strategy has nothing to rewrite, so the old tag
|
|
stays pinned. It is still one commit, and reverting that commit restores
|
|
automation.
|
|
|
|
## Before you roll back
|
|
|
|
1. Find the bump commit:
|
|
```sh
|
|
git log --oneline -5 origin/main -- kubernetes/apps/home-assistant/deployment.yaml
|
|
```
|
|
2. Check the release notes of the broken version for a **database schema**
|
|
change. Home Assistant migrates the recorder schema (CNPG cluster
|
|
`ha-pg`) on start and cannot downgrade it. An older version started on a
|
|
newer schema runs without recorder history until the schema matches
|
|
again. If the bump crossed a schema change, tell the owner before going
|
|
on: the choice is roll back without history, or restore the recorder
|
|
database from backup as a separate step.
|
|
|
|
## Roll back (one commit, one PR)
|
|
|
|
```sh
|
|
git worktree add -b atlas/ha-rollback-<old> "$WT" origin/main && cd "$WT"
|
|
git revert --no-commit <bump-sha>
|
|
# In kubernetes/apps/home-assistant/deployment.yaml, delete the trailing
|
|
# # {"$imagepolicy": "flux-system:home-assistant"}
|
|
# from the home-assistant image line. Leave every other line alone.
|
|
git diff --cached --stat # expect: 1 file, 1 line changed
|
|
git commit -m "fix(home-assistant): pin <old>, pause image automation (bump to <new> broke HA)"
|
|
git push -u origin HEAD
|
|
```
|
|
|
|
The PR description states: broken version, pinned version, symptom, schema
|
|
check result, and "revert this commit to resume automatic updates".
|
|
HA-impact review and merge follow [README.md](../README.md#2-ha-impact-review-before-merge).
|
|
|
|
On merge Flux applies the Deployment (strategy `Recreate`): Home Assistant
|
|
restarts once, about one to two minutes of downtime.
|
|
|
|
## Verify
|
|
|
|
```sh
|
|
kubectl -n home-assistant get deploy home-assistant \
|
|
-o jsonpath='{.spec.template.spec.containers[?(@.name=="home-assistant")].image}'
|
|
kubectl -n home-assistant rollout status deploy/home-assistant
|
|
```
|
|
|
|
Read-only commands only (see REN-240 protection rule 1). One hour after
|
|
merge, confirm no new `chore(image-automation)` commit touched
|
|
`deployment.yaml`.
|
|
|
|
## Resume automatic updates
|
|
|
|
When a fixed Home Assistant release is out, revert the rollback commit. The
|
|
marker comes back, and image automation bumps to the newest release on its
|
|
next pass. That bump is itself a Home Assistant restart.
|