refactor(influxdb): promote seed-influxdb to a first-class apps/influxdb KS #387

2026-07-24T11:54:35Z

claude-bot commented

2026-07-24 11:54:35 +00:00

Third step of dissolving the transitional seed layer, and the first structurally different one. InfluxDB has no separate app KS — seed-influxdb was its full lifecycle owner — so this becomes a brand-new apps/influxdb Kustomization rather than folding into an existing app.

Changes

git mv kubernetes/seed/influxdb/ -> kubernetes/apps/influxdb/ (HelmRelease + HelmRepository, ExternalSecret, ha/grafana token Jobs + RBAC).
New kubernetes/clusters/homelab/influxdb.yaml (KS influxdb, path apps/influxdb, deps infrastructure/crds/ceph — dropped the boilerplate cnpg-barman dep, no CNPG here). Deleted seed-influxdb.yaml.
e2e-test.sh: replaced the seed-influxdb readiness check with influxdb.

Why it's safe (no uninstall, no data risk)

InfluxDB's data lives in the chart-provisioned influxdb-influxdb2 PVC via the HelmRelease's StatefulSet. Flux owns the HelmRelease CR (not the PVC), so pruning it would helm-uninstall the server. Mitigation — annotate HelmRelease + HelmRepository + ExternalSecret with kustomize.toolkit.fluxcd.io/prune: disabled:

retiring seed-influxdb skips them during GC; the new influxdb KS adopts them in place.
no spec change on the HelmRelease (only a metadata annotation) → helm-controller performs no upgrade; the running release is untouched.
the token Jobs (force: enabled) are idempotent (reuse an existing auth by description), and the server never restarts, so their force-recreate just re-affirms the existing tokens.

Verified

HelmRelease influxdb Ready + owned by seed-influxdb live; data PVC influxdb-influxdb2 Bound (owner <none>, chart-created → Flux never prunes it).
kubectl kustomize apps/influxdb builds all objects with the correct prune: disabled / force: enabled annotations; bash -n e2e-test.sh OK.

Post-merge check

seed-influxdb KS gone; influxdb KS Ready; HelmRelease survives + ownership flips to influxdb; influxdb-influxdb2-0 pod still Running (no restart); token secrets intact.

Separate note (not in this PR)

influxdb-seed-staging (50Gi, prune: disabled, already orphaned — not in any live inventory) is leftover migration-restore staging. Safe to delete to reclaim 50Gi of Ceph, but destructive, so flagged for you rather than done here.

Next after this: woodpecker (CNPG — first database move).

Third step of dissolving the transitional `seed` layer, and the **first structurally different** one. InfluxDB has no separate app KS — `seed-influxdb` was its full lifecycle owner — so this becomes a brand-new `apps/influxdb` Kustomization rather than folding into an existing app. ### Changes - `git mv kubernetes/seed/influxdb/ -> kubernetes/apps/influxdb/` (HelmRelease + HelmRepository, ExternalSecret, ha/grafana token Jobs + RBAC). - New `kubernetes/clusters/homelab/influxdb.yaml` (KS `influxdb`, path `apps/influxdb`, deps infrastructure/crds/ceph — dropped the boilerplate `cnpg-barman` dep, no CNPG here). Deleted `seed-influxdb.yaml`. - `e2e-test.sh`: replaced the `seed-influxdb` readiness check with `influxdb`. ### Why it's safe (no uninstall, no data risk) InfluxDB's data lives in the chart-provisioned `influxdb-influxdb2` PVC via the HelmRelease's StatefulSet. Flux owns the **HelmRelease CR** (not the PVC), so pruning it would **helm-uninstall** the server. Mitigation — annotate **HelmRelease + HelmRepository + ExternalSecret** with `kustomize.toolkit.fluxcd.io/prune: disabled`: - retiring `seed-influxdb` skips them during GC; the new `influxdb` KS adopts them in place. - no spec change on the HelmRelease (only a metadata annotation) → helm-controller performs **no** upgrade; the running release is untouched. - the token Jobs (`force: enabled`) are idempotent (reuse an existing auth by description), and the server never restarts, so their force-recreate just re-affirms the existing tokens. ### Verified - HelmRelease `influxdb` Ready + owned by `seed-influxdb` live; data PVC `influxdb-influxdb2` Bound (owner `<none>`, chart-created → Flux never prunes it). - `kubectl kustomize apps/influxdb` builds all objects with the correct `prune: disabled` / `force: enabled` annotations; `bash -n e2e-test.sh` OK. ### Post-merge check `seed-influxdb` KS gone; `influxdb` KS Ready; HelmRelease survives + ownership flips to `influxdb`; `influxdb-influxdb2-0` pod still Running (no restart); token secrets intact. ### Separate note (not in this PR) `influxdb-seed-staging` (50Gi, `prune: disabled`, **already orphaned** — not in any live inventory) is leftover migration-restore staging. Safe to delete to reclaim 50Gi of Ceph, but destructive, so flagged for you rather than done here. Next after this: `woodpecker` (CNPG — first database move).

claude-bot added 1 commit 2026-07-24 11:54:35 +00:00

refactor(influxdb): promote seed-influxdb to a first-class apps/influxdb KS

ci/woodpecker/pr/pr-review Pipeline was successful

Details

18db205063

Third step of dissolving the transitional 'seed' layer. InfluxDB has no
separate app KS — seed-influxdb was its full lifecycle owner — so unlike the
PVC-only moves this one becomes a brand-new apps/influxdb Kustomization.

- git mv kubernetes/seed/influxdb/ -> kubernetes/apps/influxdb/ (HelmRelease +
  HelmRepository, ExternalSecret, and the ha/grafana token Jobs + their RBAC).
- Add kubernetes/clusters/homelab/influxdb.yaml (KS 'influxdb', path
  apps/influxdb, deps infrastructure/crds/ceph — dropped the boilerplate
  cnpg-barman dep, InfluxDB has no CNPG). Delete seed-influxdb.yaml.
- e2e-test.sh: replace the seed-influxdb readiness check with 'influxdb'.

Safe ownership transfer (no uninstall, no data risk): the data lives in the
chart-provisioned influxdb-influxdb2 PVC via the HelmRelease's StatefulSet.
Flux owns the HelmRelease CR (not the PVC), so pruning it would helm-uninstall
the server. Annotated the HelmRelease + HelmRepository + ExternalSecret with
kustomize.toolkit.fluxcd.io/prune: disabled, so retiring seed-influxdb skips
them during GC and the new influxdb KS adopts them in place — no spec change,
so helm-controller performs no upgrade. The token Jobs (force: enabled) are
idempotent (reuse an existing auth by description) and the server never
restarts, so their force-recreate just re-affirms the existing tokens.

Verified: HelmRelease Ready + owned by seed-influxdb live; data PVC
influxdb-influxdb2 Bound (owner <none>, chart-created); kubectl kustomize
apps/influxdb builds all objects with the right annotations; bash -n e2e OK.

Note: influxdb-seed-staging (50Gi, prune-disabled, already orphaned — not in
any live inventory) is leftover migration-restore staging, untouched here;
flagged separately for cleanup.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

review-bot requested changes

2026-07-24 12:00:47 +00:00

review-bot left a comment

🤖 Automated review — model claude-sonnet-5 (complexity: normal), commit 18db2050637d24273ecae52167361b0aaf6821f2.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

Summary

This PR retires the transitional seed-influxdb Flux Kustomization and promotes its
contents (HelmRelease/HelmRepository, ExternalSecret, token Jobs) to a first-class
apps/influxdb KS, adding kustomize.toolkit.fluxcd.io/prune: disabled annotations to
protect the live HelmRelease/HelmRepository/ExternalSecret during the ownership handoff.
The git mv and annotation additions are clean, but the PR misses one live script
reference that will break cold-bootstrap, and the safety story for the ownership
handoff has a timing gap the PR body doesn't address.

🔴 Blocking

kubernetes/bootstrap.sh:334 still calls
wait_kustomization seed-influxdb "InfluxDB seed (HR install + token Jobs)" 600.
This PR deletes the seed-influxdb Kustomization (kubernetes/clusters/homelab/seed-influxdb.yaml
is removed) and renames it to influxdb, but bootstrap.sh was not updated to match.
On the next cold-cluster bootstrap, this step will poll a Kustomization that no longer
exists, spin through the full 600s timeout, print "not Ready, continuing anyway", and
move on without ever actually gating on InfluxDB's real readiness — which likely also
means home-assistant/monitoring (whose token-Job-produced Secrets ha-influxdb-token
and the grafana token depend on InfluxDB being ready) get raced. Fix: change the name
to influxdb (keep or adjust the label/timeout as appropriate).
Handoff ordering isn't guaranteed safe, contrary to the PR's "no risk" framing.
The prune: disabled annotations (on the HelmRelease, HelmRepository, and
ExternalSecret) are added in the same commit that both (a) removes
kubernetes/seed/influxdb/ — so the old seed-influxdb KS can never itself
reconcile the annotated manifest, since its source path disappears at the same time —
and (b) deletes kubernetes/clusters/homelab/seed-influxdb.yaml, which causes the
parent flux-system Kustomization (prune: true, kubernetes/clusters/homelab/flux-system/kustomization.yaml)
to delete the seed-influxdb KS object and trigger its finalizer-driven GC of its
resource inventory. The annotation only protects those objects if it is already
present on the live resources by the time that GC sweep runs a Get + prune check
on them — and the only thing that can stamp that annotation onto the already-live
objects is the new influxdb KS's first successful reconcile (a fresh git
checkout + build + apply, which necessarily takes at least a few seconds). There is
nothing in this diff that sequences "new KS reconciles and re-applies these objects"
before "old KS's deletion finalizer GCs them" — those are two independent, concurrently
scheduled controller reconciles. If the old KS's GC wins the race, it will prune the
live HelmRelease/HelmRepository/ExternalSecret (since they don't yet carry the
annotation), and pruning the HelmRelease triggers a helm uninstall, tearing down the
running InfluxDB StatefulSet — the exact outcome the PR says can't happen.
I can't verify Flux's precise scheduling/ordering guarantees here without a live
cluster, so I can't say this will fail, only that nothing in the manifests enforces
the ordering the safety argument depends on, and the PR's own "Post-merge check"
section defers verifying exactly this ownership-flip behavior — i.e. it hasn't been
tested yet. Flux's documented safe pattern for this kind of ownership transfer is to
land the prune: disabled annotation first, confirm via kubectl get ... -o jsonpath='{.metadata.annotations}' that it's live on the running objects, and only
then, in a follow-up change, delete the old KS / move the path. Consider splitting
this into two merges (annotate-and-wait, then move-and-delete) rather than doing both
atomically, or at minimum confirm on a staging/non-critical KS that the race resolves
the safe way before relying on it here.

🟡 Suggestions

kubernetes/infrastructure/network-policies/home-assistant-ns.yaml:214 and
kubernetes/apps/monitoring/helmrelease-prometheus.yaml:206 still reference
kubernetes/seed/influxdb/ha-token-job.yaml and kubernetes/seed/influxdb/grafana- token-job.yaml respectively, in comments. Not functionally broken, but worth updating
to kubernetes/apps/influxdb/... while other path references are being fixed up in
this PR, so they don't go stale/misleading.

🟢 Nits

kubernetes/apps/influxdb/helmrelease.yaml:21-24 (unchanged context, pre-existing):
the comment says "at cutover the whole directory moves to apps/influxdb/" as a future
event — that cutover is this PR, so the comment is now describing the past. Also
references a sibling "influxdb-seed Job (seed-job.yaml)" that doesn't exist anywhere
in the repo (grepped influxdb-seed|seed-job — only a hit in velero-schedules.yaml
which is unrelated). This predates this PR and isn't something it needs to fix, but
since the file is being touched/renamed here it'd be a good time to clean it up.

Tests

No automated tests apply to this kind of manifest move; the repo's equivalent is
kubernetes/e2e-test.sh, which this PR does update correctly (swaps the
seed-influxdb readiness check for influxdb). The one gap is bootstrap.sh (see
Blocking above), which exercises the same Kustomization-readiness concept for
cold-bootstrap but wasn't caught by e2e-test.sh since that script runs post-bootstrap.
bash -n syntax-checking (mentioned as done in the PR body) wouldn't catch a stale
Kustomization name reference — only an actual bootstrap run or a grep for
seed-influxdb across the repo would.

Recommendation

Request changes — 2 blocking issues (stale bootstrap.sh reference; unverified/likely
unsafe ownership-handoff ordering for the prune-disabled annotations).

🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `18db2050637d24273ecae52167361b0aaf6821f2`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary This PR retires the transitional `seed-influxdb` Flux Kustomization and promotes its contents (HelmRelease/HelmRepository, ExternalSecret, token Jobs) to a first-class `apps/influxdb` KS, adding `kustomize.toolkit.fluxcd.io/prune: disabled` annotations to protect the live HelmRelease/HelmRepository/ExternalSecret during the ownership handoff. The `git mv` and annotation additions are clean, but the PR misses one live script reference that will break cold-bootstrap, and the safety story for the ownership handoff has a timing gap the PR body doesn't address. ## 🔴 Blocking - **`kubernetes/bootstrap.sh:334`** still calls `wait_kustomization seed-influxdb "InfluxDB seed (HR install + token Jobs)" 600`. This PR deletes the `seed-influxdb` Kustomization (`kubernetes/clusters/homelab/seed-influxdb.yaml` is removed) and renames it to `influxdb`, but `bootstrap.sh` was not updated to match. On the next cold-cluster bootstrap, this step will poll a Kustomization that no longer exists, spin through the full 600s timeout, print "not Ready, continuing anyway", and move on without ever actually gating on InfluxDB's real readiness — which likely also means `home-assistant`/`monitoring` (whose token-Job-produced Secrets `ha-influxdb-token` and the grafana token depend on InfluxDB being ready) get raced. Fix: change the name to `influxdb` (keep or adjust the label/timeout as appropriate). - **Handoff ordering isn't guaranteed safe, contrary to the PR's "no risk" framing.** The `prune: disabled` annotations (on the HelmRelease, HelmRepository, and ExternalSecret) are added in the *same commit* that both (a) removes `kubernetes/seed/influxdb/` — so the old `seed-influxdb` KS can never itself reconcile the annotated manifest, since its source path disappears at the same time — and (b) deletes `kubernetes/clusters/homelab/seed-influxdb.yaml`, which causes the parent `flux-system` Kustomization (`prune: true`, `kubernetes/clusters/homelab/flux-system/kustomization.yaml`) to delete the `seed-influxdb` KS object and trigger its finalizer-driven GC of its resource inventory. The annotation only protects those objects if it is already present on the *live* resources by the time that GC sweep runs a `Get` + prune check on them — and the only thing that can stamp that annotation onto the already-live objects is the **new** `influxdb` KS's first successful reconcile (a fresh git checkout + build + apply, which necessarily takes at least a few seconds). There is nothing in this diff that sequences "new KS reconciles and re-applies these objects" before "old KS's deletion finalizer GCs them" — those are two independent, concurrently scheduled controller reconciles. If the old KS's GC wins the race, it will prune the live HelmRelease/HelmRepository/ExternalSecret (since they don't yet carry the annotation), and pruning the HelmRelease triggers a `helm uninstall`, tearing down the running InfluxDB StatefulSet — the exact outcome the PR says can't happen. I can't verify Flux's precise scheduling/ordering guarantees here without a live cluster, so I can't say this *will* fail, only that nothing in the manifests enforces the ordering the safety argument depends on, and the PR's own "Post-merge check" section defers verifying exactly this ownership-flip behavior — i.e. it hasn't been tested yet. Flux's documented safe pattern for this kind of ownership transfer is to land the `prune: disabled` annotation first, confirm via `kubectl get ... -o jsonpath='{.metadata.annotations}'` that it's live on the running objects, and only *then*, in a follow-up change, delete the old KS / move the path. Consider splitting this into two merges (annotate-and-wait, then move-and-delete) rather than doing both atomically, or at minimum confirm on a staging/non-critical KS that the race resolves the safe way before relying on it here. ## 🟡 Suggestions - `kubernetes/infrastructure/network-policies/home-assistant-ns.yaml:214` and `kubernetes/apps/monitoring/helmrelease-prometheus.yaml:206` still reference `kubernetes/seed/influxdb/ha-token-job.yaml` and `kubernetes/seed/influxdb/grafana- token-job.yaml` respectively, in comments. Not functionally broken, but worth updating to `kubernetes/apps/influxdb/...` while other path references are being fixed up in this PR, so they don't go stale/misleading. ## 🟢 Nits - `kubernetes/apps/influxdb/helmrelease.yaml:21-24` (unchanged context, pre-existing): the comment says "at cutover the whole directory moves to apps/influxdb/" as a future event — that cutover is this PR, so the comment is now describing the past. Also references a sibling "`influxdb-seed` Job (seed-job.yaml)" that doesn't exist anywhere in the repo (grepped `influxdb-seed|seed-job` — only a hit in `velero-schedules.yaml` which is unrelated). This predates this PR and isn't something it needs to fix, but since the file is being touched/renamed here it'd be a good time to clean it up. ## Tests No automated tests apply to this kind of manifest move; the repo's equivalent is `kubernetes/e2e-test.sh`, which this PR does update correctly (swaps the `seed-influxdb` readiness check for `influxdb`). The one gap is `bootstrap.sh` (see Blocking above), which exercises the same Kustomization-readiness concept for cold-bootstrap but wasn't caught by `e2e-test.sh` since that script runs post-bootstrap. `bash -n` syntax-checking (mentioned as done in the PR body) wouldn't catch a stale Kustomization *name* reference — only an actual bootstrap run or a grep for `seed-influxdb` across the repo would. ## Recommendation Request changes — 2 blocking issues (stale `bootstrap.sh` reference; unverified/likely unsafe ownership-handoff ordering for the prune-disabled annotations).

claude-bot commented

2026-07-24 12:03:31 +00:00

Closing in favour of a safer two-step split, per the review.

The review is correct that adding prune: disabled in the same commit that deletes seed-influxdb has a genuine race: the annotation only protects the live HelmRelease once the new KS reconciles and stamps it on, and nothing sequences that before the old KS's deletion-GC. A lost race = helm uninstall of a live database. Not worth gambling on Flux scheduling.

Replacement plan (Flux's documented ownership-transfer pattern):

PR-A — add prune: disabled to the existing seed/influxdb/ HelmRelease/HelmRepository/ExternalSecret in place (no move). Merge, then verify the annotation is live on the running objects.
PR-B — the actual move (git mv -> apps/influxdb, new influxdb KS, delete seed-influxdb), plus the missed bootstrap.sh:334 rename and the stale comment-path fixes.

PR-A opening now.

Closing in favour of a safer two-step split, per the review. The review is correct that adding `prune: disabled` in the same commit that deletes `seed-influxdb` has a genuine race: the annotation only protects the live HelmRelease once the *new* KS reconciles and stamps it on, and nothing sequences that before the old KS's deletion-GC. A lost race = helm uninstall of a live database. Not worth gambling on Flux scheduling. Replacement plan (Flux's documented ownership-transfer pattern): 1. **PR-A** — add `prune: disabled` to the *existing* `seed/influxdb/` HelmRelease/HelmRepository/ExternalSecret in place (no move). Merge, then verify the annotation is live on the running objects. 2. **PR-B** — the actual move (git mv -> apps/influxdb, new `influxdb` KS, delete `seed-influxdb`), plus the missed `bootstrap.sh:334` rename and the stale comment-path fixes. PR-A opening now.

claude-bot closed this pull request

2026-07-24 12:03:32 +00:00

claude-bot referenced this pull request

2026-07-24 12:05:14 +00:00

chore(influxdb): pre-stage prune:disabled on live seed-influxdb objects (step 1/2) #388

claude-bot referenced this issue from a commit

2026-07-24 12:56:13 +00:00

refactor(influxdb): move seed-influxdb to apps/influxdb (step 2/2)

claude-bot referenced this pull request

2026-07-24 12:56:27 +00:00

refactor(influxdb): move seed-influxdb to apps/influxdb (step 2/2) #390