konairius/homelab
10
0
Fork 0
Code Issues 11 Pull Requests 1 Packages Projects Releases 1 Wiki Activity

feat(k8s-3): bare-metal config for neptune #604

Merged
k.renner merged 1 commits from k8s-3-baremetal into master 2026-08-25 23:45:38 +00:00
Conversation 1 Commits 1 Files Changed 2
+70 -20
claude-bot commented 2026-08-25 23:43:37 +00:00
Member
Copy Link
Copy Source

neptune runs Talos directly now, so k8s-3 moves from the VM-era layout to the multi-document form k8s-1 and k8s-2 already use. The old config selected interfaces by busPath against virtio NICs and installed to /dev/vda — neither exists on bare metal.

This config is already live on the node; the PR brings git in line with it.

Why serials, not device names

Talos enumerated this machine differently than Proxmox did, and differently again between two consecutive boots:

Proxmox:      nvme1n1 = Samsung   nvme2n1 = Crucial
Talos boot 1: nvme0n1 = Kingston  nvme1n1 = Crucial   nvme2n1 = Samsung
Talos boot 2: nvme0n1 = Crucial   nvme1n1 = Kingston  nvme2n1 = Samsung

A /dev/nvme0n1 reference would have installed onto whichever drive happened to land there. Selecting by serial made the install deterministic, and EPHEMERAL landed on the Crucial as intended.

serial drive role
50026B7382B48B90 Kingston system disk
2326E6E8E67D Crucial EPHEMERAL — 83% of rated endurance, 100% spare, no media errors
S437NF0M501176 PM983 OSD — 3% used, no media errors

Serials, interface names and drive health were read from the machine itself: the Kingston and Crucial off the running Proxmox host with smartctl before the wipe, then confirmed from Talos maintenance mode. Interface names were verified rather than assumed from the other two nodes — assuming them is what stranded jupiter.

Install

Two-stage per the runbook, and it worked first attempt with no hang — the confirmation that the ens19 diagnosis was correct rather than coincidental. Stage 1 carried the minimal network config plus EPHEMERAL; the full config went on afterwards behind --mode=try, then persisted. Neither apply needed a reboot.

Also

k8s-2.yaml loses the sentence claiming k8s-3 is still a Proxmox VM. All three control-plane nodes are bare metal and no Proxmox remains in the homelab.

Not in this PR

The Ceph OSD re-add for neptune, and restoring cephfs-metadata to size 3 / .mgr to 3/2 / osd_mclock_profile to balanced.

neptune runs Talos directly now, so `k8s-3` moves from the VM-era layout to the multi-document form `k8s-1` and `k8s-2` already use. The old config selected interfaces by `busPath` against virtio NICs and installed to `/dev/vda` — neither exists on bare metal. This config is already live on the node; the PR brings git in line with it. ## Why serials, not device names Talos enumerated this machine differently than Proxmox did, and **differently again between two consecutive boots**: ``` Proxmox: nvme1n1 = Samsung nvme2n1 = Crucial Talos boot 1: nvme0n1 = Kingston nvme1n1 = Crucial nvme2n1 = Samsung Talos boot 2: nvme0n1 = Crucial nvme1n1 = Kingston nvme2n1 = Samsung ``` A `/dev/nvme0n1` reference would have installed onto whichever drive happened to land there. Selecting by serial made the install deterministic, and `EPHEMERAL` landed on the Crucial as intended. | serial | drive | role | | --- | --- | --- | | `50026B7382B48B90` | Kingston | system disk | | `2326E6E8E67D` | Crucial | EPHEMERAL — 83% of rated endurance, 100% spare, no media errors | | `S437NF0M501176` | PM983 | OSD — 3% used, no media errors | Serials, interface names and drive health were read from the machine itself: the Kingston and Crucial off the running Proxmox host with `smartctl` before the wipe, then confirmed from Talos maintenance mode. Interface names were verified rather than assumed from the other two nodes — assuming them is what stranded jupiter. ## Install Two-stage per the runbook, and it worked **first attempt with no hang** — the confirmation that the `ens19` diagnosis was correct rather than coincidental. Stage 1 carried the minimal network config plus `EPHEMERAL`; the full config went on afterwards behind `--mode=try`, then persisted. Neither apply needed a reboot. ## Also `k8s-2.yaml` loses the sentence claiming k8s-3 is still a Proxmox VM. All three control-plane nodes are bare metal and no Proxmox remains in the homelab. ## Not in this PR The Ceph OSD re-add for neptune, and restoring `cephfs-metadata` to size 3 / `.mgr` to 3/2 / `osd_mclock_profile` to `balanced`.
claude-bot added 1 commit 2026-08-25 23:43:37 +00:00
feat(k8s-3): bare-metal config for neptune
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
Details
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
Details
ci/woodpecker/pr/pr-review Pipeline was successful
Details
1306d2ad89 
neptune now runs Talos directly, so k8s-3 moves from the VM-era layout to the
multi-document form k8s-1 and k8s-2 already use. The old config selected
interfaces by busPath against virtio NICs and installed to /dev/vda; neither
exists on bare metal.

Disks are selected by serial rather than kernel name. That is not defensive
styling: Talos enumerated this machine's drives differently than Proxmox did,
and differently again between two consecutive boots — nvme0n1 was the Kingston
on one boot and the Crucial on the next. A device-name reference would have
installed onto the wrong drive.

  nvme  50026B7382B48B90  Kingston  system disk
  nvme  2326E6E8E67D      Crucial   EPHEMERAL, 83% of rated endurance
  nvme  S437NF0M501176    PM983     OSD, 3% used

Serials, interface names and drive health were read from the machine itself —
the Kingston and Crucial off the running Proxmox host before the wipe, and
confirmed again from Talos maintenance mode.

k8s-2.yaml loses the sentence claiming k8s-3 is still a Proxmox VM. No Proxmox
remains in the homelab.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
k.renner merged commit bd6e101bfb into master 2026-08-25 23:45:38 +00:00
review-bot left a comment
Member
Copy Link
Copy Source

🤖 Automated review — model CLI default (complexity: complex), commit 1306d2ad8957326e5e26b11ffdb502899a50ae6b.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

Summary

Migrates k8s-3.yaml (neptune) from the VM-era machine.network.interfaces/install.disk form to the multi-document Talos config already used by k8s-1.yaml/k8s-2.yaml, and drops the now-stale "k8s-3 is still a Proxmox VM" sentence from k8s-2.yaml. I compared the new documents field-by-field against the sibling k8s-1.yaml/k8s-2.yaml files and the disk/serial table in the PR description; the mapping from the old interface block to the new documents is complete and correct, and no blocking issues.

🟡 Suggestions

  • kubernetes/talos/patches/k8s-3.yaml:45-49,50-56 — the new LinkConfig enp2s0f0np0 and VLANConfig mgmt0 documents carry no comment, whereas the equivalent documents in both k8s-1.yaml:66-85,87-95 and k8s-2.yaml:46-58,60-66 explain the trunk/no-addressing rationale and what mgmt0 carries (node IP, etcd, API, Ceph public net). The k8s-3 file's header says "see k8s-1.yaml for the reasoning," so omitting them isn't wrong, but it breaks from the pattern k8s-2.yaml already established of keeping a condensed version of each comment. Worth a one-line comment on each for consistency with its siblings.
  • kubernetes/RUNBOOK-baremetal-conversion.md:5 — "Written after doing jupiter on 2026-08-25; saturn and neptune remain" is now stale once this PR lands, since neptune (per the PR title/body) is the last of the three converted. Not touched by this diff, so non-blocking, but worth a follow-up edit so the runbook doesn't read as still-pending work.

🟢 Nits

  • kubernetes/talos/patches/k8s-3.yaml:1-4 header drops "MACs" from the "only the serials, MACs and addresses differ" phrasing that k8s-2.yaml:1-3 still carries — accurate here since no document in either file actually selects by MAC, but the two headers are now worded slightly differently for no functional reason.

Verified correctness checks

  • Disk selectors: install.diskSelector.serial: 50026B7382B48B90 (k8s-3.yaml:53) and VolumeConfig EPHEMERAL disk.serial == '2326E6E8E67D' (k8s-3.yaml:77) match the Kingston/system-disk and Crucial/EPHEMERAL rows in the PR's serial table, and are distinct from k8s-1's (...B86/2332E86B749A) and k8s-2's (...5F6E/2332E86B74BC) serials — no accidental copy-paste collision, which would have been a serious bug here (wrong-drive install/wipe).
  • Ceph network address: 10.99.0.3/29 (k8s-3.yaml:118) is distinct from k8s-1's .1 and k8s-2's .2 on the same /29 — no IP collision.
  • Old→new interface mapping: the removed busPath: "0000:06:12.0" + vip interface maps to mgmt0 (VLANConfig + DHCPv4Config + Layer2VIPConfig 10.2.40.60); the removed busPath: "0000:06:13.0" (v30iot, link-only) maps to VLANConfig ens19, consistent with the retained sysctls entry for ens19; the removed driver: i40e / 10.99.0.3/29 / mtu: 9000 interface maps to LinkConfig enp2s0f1np1 with the same address and MTU preserved. All three are accounted for.
  • VIP address 10.2.40.60 is intentionally shared across all three nodes' Layer2VIPConfig (etcd-elected VIP) — not a collision.
  • k8s-2.yaml edit is a comment-only removal of the outdated "k8s-3 is still a Proxmox VM" claim; no functional change.

Tests

No automated validation exists for these Talos config patches in this repo (no CI step runs talosctl validate against them), consistent with how k8s-1.yaml/k8s-2.yaml were previously merged. The PR body states the config is already live on neptune and was applied successfully via the two-stage process in the runbook, which is the de facto verification for this kind of change — reasonable given the repo's existing conventions.

Recommendation

Looks reasonable — no blocking issues; comments above are optional.

🤖 **Automated review** — model `CLI default` (complexity: complex), commit `1306d2ad8957326e5e26b11ffdb502899a50ae6b`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary Migrates `k8s-3.yaml` (neptune) from the VM-era `machine.network.interfaces`/`install.disk` form to the multi-document Talos config already used by `k8s-1.yaml`/`k8s-2.yaml`, and drops the now-stale "k8s-3 is still a Proxmox VM" sentence from `k8s-2.yaml`. I compared the new documents field-by-field against the sibling `k8s-1.yaml`/`k8s-2.yaml` files and the disk/serial table in the PR description; the mapping from the old interface block to the new documents is complete and correct, and no blocking issues. ## 🟡 Suggestions - `kubernetes/talos/patches/k8s-3.yaml:45-49,50-56` — the new `LinkConfig enp2s0f0np0` and `VLANConfig mgmt0` documents carry no comment, whereas the equivalent documents in both `k8s-1.yaml:66-85,87-95` and `k8s-2.yaml:46-58,60-66` explain the trunk/no-addressing rationale and what mgmt0 carries (node IP, etcd, API, Ceph public net). The k8s-3 file's header says "see k8s-1.yaml for the reasoning," so omitting them isn't wrong, but it breaks from the pattern k8s-2.yaml already established of keeping a condensed version of each comment. Worth a one-line comment on each for consistency with its siblings. - `kubernetes/RUNBOOK-baremetal-conversion.md:5` — "Written after doing jupiter on 2026-08-25; saturn and neptune remain" is now stale once this PR lands, since neptune (per the PR title/body) is the last of the three converted. Not touched by this diff, so non-blocking, but worth a follow-up edit so the runbook doesn't read as still-pending work. ## 🟢 Nits - `kubernetes/talos/patches/k8s-3.yaml:1-4` header drops "MACs" from the "only the serials, MACs and addresses differ" phrasing that `k8s-2.yaml:1-3` still carries — accurate here since no document in either file actually selects by MAC, but the two headers are now worded slightly differently for no functional reason. ## Verified correctness checks - **Disk selectors**: `install.diskSelector.serial: 50026B7382B48B90` (k8s-3.yaml:53) and `VolumeConfig EPHEMERAL` `disk.serial == '2326E6E8E67D'` (k8s-3.yaml:77) match the Kingston/system-disk and Crucial/EPHEMERAL rows in the PR's serial table, and are distinct from k8s-1's (`...B86`/`2332E86B749A`) and k8s-2's (`...5F6E`/`2332E86B74BC`) serials — no accidental copy-paste collision, which would have been a serious bug here (wrong-drive install/wipe). - **Ceph network address**: `10.99.0.3/29` (k8s-3.yaml:118) is distinct from k8s-1's `.1` and k8s-2's `.2` on the same /29 — no IP collision. - **Old→new interface mapping**: the removed `busPath: "0000:06:12.0"` + `vip` interface maps to `mgmt0` (VLANConfig + DHCPv4Config + Layer2VIPConfig 10.2.40.60); the removed `busPath: "0000:06:13.0"` (v30iot, link-only) maps to `VLANConfig ens19`, consistent with the retained `sysctls` entry for `ens19`; the removed `driver: i40e` / `10.99.0.3/29` / `mtu: 9000` interface maps to `LinkConfig enp2s0f1np1` with the same address and MTU preserved. All three are accounted for. - **VIP address** `10.2.40.60` is intentionally shared across all three nodes' `Layer2VIPConfig` (etcd-elected VIP) — not a collision. - **`k8s-2.yaml` edit** is a comment-only removal of the outdated "k8s-3 is still a Proxmox VM" claim; no functional change. ## Tests No automated validation exists for these Talos config patches in this repo (no CI step runs `talosctl validate` against them), consistent with how `k8s-1.yaml`/`k8s-2.yaml` were previously merged. The PR body states the config is already live on neptune and was applied successfully via the two-stage process in the runbook, which is the de facto verification for this kind of change — reasonable given the repo's existing conventions. ## Recommendation Looks reasonable — no blocking issues; comments above are optional. <!-- pr-review:sha=1306d2ad8957326e5e26b11ffdb502899a50ae6b -->
Sign in to join this conversation.
Reviewers
No Reviewers
review-bot
Labels
Clear labels
apollo
Hermes expert: Immich, Audiobookshelf, Libation, Navidrome, Music Assistant
 atlas
Hermes expert: Talos, Flux, network, Authentik, Matrix, monitoring
 clio
Hermes expert: Paperless, wiki, ical-proxy, TRMNL, websites
 daedalus
Hermes expert: Gitea, Woodpecker, Renovate, review-bot, Hermes
 hestia
Hermes expert: Home Assistant, IoT, MQTT, ESPHome, voice
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
alert-monitor-bot claude-bot (Claude Bot) flux hermes-bot issue-fix-bot k.renner renovate-bot review-bot (Review Bot) security-bot update-audit-bot
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: konairius/homelab#604