The pod has a Multus net1 attachment on the IoT VLAN (10.2.30.229)
that bypasses Cilium policy entirely. With nginx on 0.0.0.0:8124, any
device on 10.2.30.0/24 could reach the proxy with no CNP gating —
defeating the loopback-only http.trusted_proxies XFF trust anchor.
Render the listen address from status.podIP via the nginx entrypoint's
envsubst pass, scoped to POD_IP only so nginx runtime vars stay intact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The opentofu-bootstrap/ project existed solely to provision the RustFS
LXCs (rustfs-1/2/3 on jupiter/saturn/neptune). With RustFS retired
2026-05-09 and the previous commit removing all main-project references,
this bootstrap project has no remaining purpose. Local state file on
the operator's disk (gitignored) becomes orphaned but affects nothing
since no other code references it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Files dropped into the paperless-consume RWX cephfs volume by a pod
scheduled on a different node than paperless's consumer never fire
inotify IN_CREATE — cephfs only propagates filesystem events to
inotify watchers on the same kernel. Production FTP ingestion has
been working by scheduler luck (ftp-scanner happens to co-locate
with paperless on k8s-2).
10s polling closes the gap. Inotify still fires for same-node
writes, so latency for local writes is unchanged. Cross-node writes
get picked up within ~10s + the standard 5s × 5 stability window.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
RustFS was retired 2026-05-09 (per the comment in
opentofu-bootstrap/rustfs.tf). The S3 endpoint at s3.renner.uno:9000
has been unreachable since then, but a lot of references in code
remained — health checks pinged a dead endpoint, alerts fired
permanently, and the ansible role + playbook + tofu locals would have
broken the next time anyone tried to use them. Cleaning up:
DELETED (pure RustFS, no other consumers):
- opentofu/rustfs.tf.disabled
- opentofu/providers.tf.backup
- ansible/inventory/group_vars/proxmox_rustfs.yml
- ansible/plays/install-rustfs.yml
- ansible/plays/archived/migrate-rustfs-storage.yml
- ansible/roles/rustfs/ (whole role)
EDITED (RustFS sections removed):
- opentofu/variables.tf: drop rustfs_count
- opentofu/locals.tf: drop rustfs_instances local
- opentofu/backup_jobs.tf: drop the rustfs_count VMID-range
- kubernetes/apps/gatus/configmap.yaml: drop the dead s3.renner.uno
health check
- ansible/roles/gatus/templates/config.yaml.j2: same removal in the
ansible-managed gatus template
- ansible/roles/prometheus-server/templates/alert-rules.yml.j2:
drop the RustFSUnhealthy alert
- ansible/roles/grafana-observability/templates/alert-rules.yaml.j2:
drop the rustfs-unhealthy alert
- ansible/plays/check-chaos-testing.yml: drop the RustFS test block
and the rustfs_vip var
NOT TOUCHED (caddy-ha retirement scope, separate cleanup):
- ansible/roles/caddy-ha/defaults/main.yml (still has
caddy_ha_s3_endpoint = "s3.renner.uno:9000")
- ansible/plays/install-crowdsec-caddy.yml (still loads
vars/rustfs_vault.yml)
- ansible/plays/vars/rustfs_vault.yml (loaded by the play above)
- scripts/restore-authentik-db.sh (manual one-off restore script
that still references s3.renner.uno)
- opentofu-bootstrap/{rustfs.tf,variables.tf} (separate tofu
project with local state; cleanup needs manual apply)
The Woodpecker secrets tf_rfs_access and tf_rfs_secret are no longer
consumed by anything in the repo and can be removed manually from the
Woodpecker UI when convenient.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous commit dropped serviceAccountName from the manifest, but
server-side apply did not prune the field on the live Deployment;
new ReplicaSet inherited the stale ref to the deleted
home-assistant-trusted-proxies SA and could not create pods.
Setting serviceAccountName: default explicitly reclaims the field
under Flux's ownership at the implicit Kubernetes default value.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the previous commit removed the aws_s3_bucket.{tfstate,migration,
caddy_certificates} resources, nothing in the .tf code references the
AWS provider that pointed at the decommissioned RustFS endpoint
(http://s3.renner.uno:9000). Removing the now-dead config:
- providers.tf: drop required_providers.aws and the provider "aws"
block (with the http://s3.renner.uno:9000 endpoint).
- variables.tf: drop aws_provider_access_key and aws_provider_secret_key
(only consumed by the removed provider block).
- .woodpecker/deploy-infrastructure.yml: drop the corresponding
TF_VAR_aws_provider_* env entries from both plan and apply steps
(sourced from tf_rfs_access/tf_rfs_secret secrets — those Woodpecker
secrets can be removed manually if unused elsewhere).
Backend "s3" config in providers.tf stays — it talks to the NAS-hosted
Garage instance (s3.nas-1.renner.uno) via AWS_ACCESS_KEY_ID/SECRET env
vars from tf_garage_* secrets, which is separate from the dead RustFS
provider being removed here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the trusted_proxies reconciler with an nginx sidecar that
fronts HA on :8124 and proxies to 127.0.0.1:8123. HA's TCP peer is
always loopback, so http.trusted_proxies = [127.0.0.1, ::1] is static
and no longer needs an HA restart every time an ingress-nginx pod
rolls. The CNP restricts :8124 to ingress-nginx pods, preserving the
narrow XFF trust boundary without depending on individual pod IPs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Server pods OOMKilled twice (exitCode 137) at the chart-suggested
512Mi/512Mi sizing — gunicorn router + worker processes plus blueprint
loading climb to ~485Mi steady-state on a 2-replica deployment, with
no headroom for bursts. Workers were sitting at ~430Mi, well within
the danger zone for the next spike.
Bumped both: requests 768Mi, limits 1Gi. The chart's commented-out
suggestions are too low for a real multi-replica setup; documented
for the wiki.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both records were marked "will be created" in the deploy-infrastructure
plan despite being in state with concrete UniFi IDs earlier — meaning
they got deleted from UniFi externally and tofu's refresh dropped them
from state. Per direction, they should stay deleted rather than be
recreated.
- dns_renner_uno.tf: remove "emby" entry from renner_uno_records map
- vm_103_homeassistant.tf: empty the cnames list on the HA VM module
After this change the next plan should show no action on these (gone
from .tf, gone from state).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three aws_s3_bucket resources (tfstate, migration, caddy_certificates)
and the unifi_dns_record.s3_vip pointed at s3.renner.uno (10.2.40.101),
the now-decommissioned RustFS VIP. Tofu plan was hanging in CI because
the AWS provider in providers.tf still points at that dead endpoint and
the parallel refresh of the three buckets times out at TCP, holding the
plan open indefinitely (confirmed reproducibly across pipelines 909 and
910 — same stall point after the three aws_s3_bucket refresh-start log
lines).
The buckets themselves are gone with RustFS so there's nothing to
destroy via the AWS API — state rm is the right primitive here. The
follow-up pipeline run will pass:
TOFU_STATE_RM='aws_s3_bucket.tfstate aws_s3_bucket.migration aws_s3_bucket.caddy_certificates'
which calls tofu state rm on each before tofu plan, dropping the orphan
entries without an API call to the dead endpoint. unifi_dns_record.s3_vip
stays in state and gets a normal destroy via the (responsive) UniFi
provider.
The AWS provider block in providers.tf is unused after this but left in
place to avoid scope-creeping into the broader RustFS-leftover cleanup;
that's a separate PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The upstream Helm chart ships built-in HPA support for both
deployments; using it instead of a standalone manifest keeps the
config in one place and tracks chart upgrades.
Both: min=2, max=4, target 70% CPU. Resources set to the chart's
documented suggested values (100m / 512Mi). Worker is queue-driven
(Celery + Redis) so replicas distribute work naturally; server is
HTTP-served via ingress-nginx, which maintains many concurrent
upstream connections, so distribution across pods works (this is
different from the immich-server → immich-ml case where a single
persistent connection pinned all traffic to one pod).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This reverts commit 976d8b4.
Upstream design doesn't load-balance across multiple ml backends:
the multi-URL field in System Settings → Machine Learning is
documented as sequential failover (try URL1, then URL2 if URL1
fails), not round-robin. With a single URL, immich-server pins all
inference traffic to one backend via a persistent HTTP connection,
so HPA-spawned replicas just sit warm without doing work. Confirmed
live (cur=2, one pod at 2905m CPU, the other at 2m).
For more inference throughput, upstream's documented lever is the
MACHINE_LEARNING_WORKERS env var (vertical, in-pod), with the
caveat that each worker duplicates the model in memory.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bursty CPU profile (library scans, facial recognition, smart search
queries) that idles between. Cache PVC is RWX, so multi-replica is
safe. Cap at 2 to limit aggregate model RAM. Slow scale rates and
long scaleDown stabilization (10m) avoid churning the model cache,
which takes ~30s of cold-start to rebuild.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The stabilization window was rejecting transient spikes — a single
high-CPU sample (832% on gotenberg during a 40-doc ingestion burst)
got smoothed to no-op. With the window at 0, any single sample over
the 70% target triggers scale-up immediately. Per-pod rate policy
(1 pod / 60-90s) still throttles runaway scaling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
One-off notes from the May 2 rebuild — Ceph baseline, InfluxDB
relocation, OTBR diagnosis, pipeline 871 report. No longer needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Spegel now handles all in-cluster image mirroring via P2P pull-through
caching, so the standalone distribution/distribution LXC is redundant.
This commit:
- removes the OpenTofu module — next deploy-infrastructure pipeline
will plan a destroy for module.registry_mirror
- removes the ansible role, play, and install-registry-mirror.yml
Woodpecker pipeline (all only invoked during LXC create/update)
- strips --registry-mirror / --insecure-registry kaniko flags from
.woodpecker/build-ci-images.yml — CI builds now pull docker.io
directly (single rebuild fits well under 100 anonymous pulls/6h)
- updates the rook-ceph operator chart-default comment so it
references Spegel instead of the LXC
The Talos patch file kubernetes/talos/patches/registry-mirror.yaml
keeps its name (now contains the discard_unpacked_layers .part for
Spegel); bootstrap.sh references it by that name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both are stateless ingestion workers with no PVCs, idle most of the
time and bursty during document intake. CPU-utilization HPA at 70%
of requests, min=1 max=3. Tika gets longer stabilization windows
to absorb JVM warmup and GC noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of "hostPort doesn't bind" identified via Cilium's docs, not
the CNI-chain hallucination I previously committed. From
https://docs.cilium.io/en/v1.19/network/kubernetes/kubeproxy-free/ :
"Cilium's eBPF kube-proxy replacement does not allow hostPort port
configurations for Pods that overlap with the configured NodePort
range. In such case, the hostPort setting will be ignored and a
warning emitted."
Cilium's default NodePort range is [30000, 32767] (no override in this
cluster's cilium-config). Spegel's chart default hostPort 30020 sits
inside that range, so Cilium silently drops it. CNI Chaining: none per
cilium-dbg status, so Multus/portmap were never relevant.
Fix: set hostPort to 29999, one below the NodePort range. Drop
usePreferSameNodeTrafficDistribution — with a working hostPort,
containerd's connect to 127.0.0.1:29999 always lands on the local
node's Spegel pod by definition (hostPort is per-node, not LB-routed),
so PreferSameNode is not needed.
This puts the deployment back on the documented Spegel path
(https://spegel.dev/docs/getting-started/#talos) with no deviations
beyond the one upstream-docs prescription.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous Talos patch was assembled from training data and a handful of
doc snippets, not from the documented integration end-to-end. Reading
https://spegel.dev/docs/getting-started/#talos verbatim, the only Talos-
specific pieces are:
1. A machine.files entry dropping a containerd .part file that
sets discard_unpacked_layers = false.
2. The Helm value spegel.containerdRegistryConfigPath set to
/etc/cri/conf.d/hosts.
3. The spegel namespace labeled pod-security.kubernetes.io/enforce=
privileged (already in place via namespaces.yaml).
No machine.registries.mirrors block is needed — Spegel writes its own
per-registry hosts.toml entries at runtime when containerdMirrorAdd is
left at its true default, and that's the documented path. Previous
patch had a manual mirrors block AND explicitly disabled
containerdMirrorAdd, which is the inverse of what the docs prescribe.
usePreferSameNodeTrafficDistribution=true is kept as a documented but
environment-forced deviation: this cluster's Cilium CNI conflist has
no portmap plugin (cni.exclusive=false for Multus coexistence), so
DaemonSet hostPort declarations are silently dropped at the CNI layer.
Verified via talosctl netstat (port not bound) and cilium-dbg service
list (no HostPort entry). Comment in the helmrelease values explains.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
INFO suppresses per-request handler traces, leaving only metrics as
evidence that pulls actually flow through Spegel. DEBUG logs each
served /v2/* request so verification pulls are inspectable via
kubectl logs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cilium's CNI conflist on this cluster has only the cilium-cni plugin
(cni.exclusive=false for Multus coexistence), with no portmap plugin
in the chain. DaemonSet hostPort declarations are silently dropped at
the CNI layer — verified via talosctl netstat (port 30020 not bound)
and cilium-dbg service list (no HostPort entry).
Spegel's chart anticipates this with usePreferSameNodeTrafficDistribution,
which drops hostPort from the DaemonSet and adds trafficDistribution:
PreferSameNode to the NodePort Service. Cilium's socketLB (bpf-lb-sock=
true, host-namespace included since bpf-lb-sock-hostns-only is unset)
rewrites containerd's connect(127.0.0.1, 30021) at the syscall layer to
a local backend pod via the already-registered NodePort 30021 entry.
Talos patch endpoint flips 30020 -> 30021 to match the NodePort.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Spegel DaemonSet using each node's containerd content store, with Talos
machine.registries.mirrors routing all four mirrored registries at
127.0.0.1:30020. machine.files drops a containerd .part flipping
discard_unpacked_layers=false (Spegel prerequisite, containerd v3 plugin
path).
The LXC's Spegel rejection was specific to the ~10-rebuilds/day pipeline
iteration phase. With rebuilds now post-disaster only, a single rebuild's
docker.io pulls fit under the 100/6h anonymous ceiling, so the warm-cache
survivability argument no longer outweighs the operational cost of a
separate LXC.
Cutover is manual via per-node \`talosctl apply-config --config-patch
@kubernetes/talos/patches/registry-mirror.yaml -n <node> --mode=reboot\`
since discard_unpacked_layers needs a containerd restart. LXC 250
decommission follows in a separate PR once Spegel is verified serving
mirrored pulls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulling the notifier back out — the channel itself worked (CNPs and
token were correct) but the notifier mounted libation-state read-only
and SQLite needs the directory writable for its WAL/journal even on a
SELECT. The pod failed at the sqlite3 open and the Job hit backoff
faster than logs could be retrieved.
Reverting all four notification-related additions (sidecar, HA token
secret in audiobookshelf ns, cross-namespace ingress allow on HA,
egress allow on libation). The libation CronJob goes back to its
plain "download books, exit" form; ABS's file watcher still picks up
each new folder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The audiobookshelf default-deny-ingress CNP carries an egress allowlist
that doesn't include home-assistant, so libation's notifier SYN was
hanging unanswered. Adding a layered CNP that selects libation pods
and adds home-assistant:8123 to their permitted egress targets.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After each scan+liberate pass, a notifier sidecar queries the libation
SQLite DB for books whose UserDefinedItem.BookStatus flipped to
Liberated during the run window and POSTs the titles to Home Assistant.
LastDownloaded is the truth column — only set on successful download,
untouched on license-denied attempts — so the query naturally skips
the 3 known-denied books that re-attempt every run.
Cross-namespace plumbing:
- ha-api-token.sops.yaml: clone of the existing home-assistant-api-token
secret in the audiobookshelf namespace (k8s secrets are namespace-
scoped, no reflector deployed).
- ha-ingress-policy.yaml: CiliumNetworkPolicy in the home-assistant
namespace allowing tcp/8123 ingress from the libation pod. Owned by
the libation Kustomization so decommissioning libation removes the
allow rule as well.
HA_NOTIFY_SERVICE defaults to notify/persistent_notification (always
exists in HA core); switch to notify/mobile_app_<device> via kustomize
patch to push to a phone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the first cluster scan, four library books (B07WC6QG79, B07JVHDWW8,
B0112N4LL2, B09KGX87QF) were updated in LibationContext.db with
BookStatus=1. Those books were never downloaded on the workstation
(so the workstation DB had them as NotLiberated) but already exist on
the audiobookshelf-books PVC from the S3 seed during the original
LXC→K8s migration. Leaving them NotLiberated caused liberate to
attempt re-downloads and would have produced duplicate folders in /audiobooks.
Refreshing the bootstrap secret with the post-patch DB so that if the
libation-config PVC is ever rebuilt, the same skip behavior applies on
re-seed. AccountsSettings.json now also reflects the refreshed token
state written by the first cluster run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bootstrap Secret mounts read-only (defaultMode: 0400), and the
init container's plain `cp` was propagating that mode into the
libation-config PVC. Result: LibationCli failed on the first scan
with "Libation cannot write its SQLite database (migrations need
write access)" against /config-internal/LibationContext.db.
`install -m 660` sets owner+group rw explicitly; fsGroup 1000 already
gives every container gid 1000, so 0660 is enough for the libation
container to update both AccountsSettings.json (token refresh) and
LibationContext.db (library state) across runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the workstation-side Libation install with a daily CronJob that
runs the official rmcrackan/libation image headlessly. SLEEP_TIME=-1
makes the entrypoint do one scan+liberate pass and exit; the schedule
is handled by Kubernetes.
Lives in the audiobookshelf namespace because k8s PVCs are namespace-
scoped and we share /audiobooks with the audiobookshelf Deployment.
A small RWO ceph-block PVC (libation-config) holds AccountsSettings.json,
LibationContext.db, and the cover/search caches; an init container
seeds it once from the SOPS-encrypted bootstrap secret (sentinel-file
gated so token refreshes and DB updates persist across runs).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous /24 installed `10.2.30.0/24 dev net1` as a connected route in HA's pod
namespace. k8s-edge's reported InternalIP is `10.2.30.137` (its v30iot address), so
music-assistant's hostNetwork pod registers that as the service endpoint. HA's
ClusterIP → DNAT path then resolved to `10.2.30.137:8095`, the kernel re-routed via
`dev net1` (10.2.30.0/24 match), and the packet went out net1 with src=10.244.x.x —
wrong subnet for v30iot, kernel returned "Host unreachable". HA→music-assistant broke.
/32 keeps net1 attached for mDNS multicast reception (group membership, not unicast
routing) without claiming the IoT VLAN subnet inside the pod's route table. Unicast
cluster-internal traffic stays on eth0 where Cilium can manage it; Shelly HTTP unicast
to 10.2.30.x continues via eth0 → cluster routing → gateway (the path that's worked
the entire time, and what the user pointed out is correct).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HA's built-in `shelly` integration uses zeroconf to discover devices and to push the
`host` field of each config_entry whenever a device re-advertises (DHCP renumbering,
firmware update, reboot, …). With HA on the cluster pod CIDR, the IoT-VLAN multicast
never reached HA's network namespace, so the 43 Shelly entries were stuck on the IPs HA
saw the day it first discovered them — 42 of 43 are stale right now.
- network-attachment.yaml: macvlan on k8s-edge eth1 (HA stays pinned to edge for USB
passthrough), static `10.2.30.229/24`. Next after matter-server/.226, otbr/.227,
shelly-manager/.228 in the iot Multus block.
- deployment: adds the `k8s.v1.cni.cncf.io/networks: home-assistant-iot` annotation;
eth0 remains primary so trusted_proxies, ip_ban, and the source-IP-preservation
rationale all keep holding.
Existing IP-based shelly config_entries get auto-updated by HA's discovery flow when
each device next emits an mDNS announcement (reboots, link changes, periodic
TTL-refresh). No manual entry rewrite needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The k8s-edge taint `workload-class=pinned:NoSchedule` already keeps any pod that doesn't
explicitly tolerate it off that node — and shelly-manager doesn't. The extra
`nodeAffinity NotIn[edge]` was belt-and-braces and just noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The refactored app (k.renner/shelly-manager 0fbd983) addresses Shellies by their mDNS
hostname and resolves them at request time. mDNS is UDP multicast and doesn't cross L3,
so the pod needs real L2 presence on the IoT VLAN — same problem matter-server/otbr
already solved with a macvlan NAD.
- shelly-manager-network-attachment.yaml: macvlan with static `10.2.30.228/24`, next
free after matter-server/.226 and otbr/.227.
- deployment: adds the `k8s.v1.cni.cncf.io/networks` annotation, nodeAffinity excludes
`zone=edge` because v30iot's kernel name varies — eth2 on k8s-{1,2,3} (the workers)
vs eth1 on k8s-edge (which has no X710 passthrough). Three workers is plenty of
headroom for a single replica.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit assumed the new v30iot NIC would land as eth1 and the existing mgmt
as eth0 across all three nodes. Live state disagrees:
- k8s-1: eth0 mgmt, eth1 X710, eth2 v30iot
- k8s-2: eth0 X710, eth1 mgmt, eth2 v30iot
- k8s-3: eth0 mgmt, eth1 X710, eth2 v30iot
Kernel ethN ordering depends on PCI bus discovery and isn't portable. busPath is — both
virtio NICs land at the same PCI slots across all three nodes:
- 0000:06:12.0 mgmt (the original virtio)
- 0000:06:13.0 v30iot (the new extra_network_bridges entry)
X710 keeps `driver: i40e` (unambiguous regardless of kernel name).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously only k8s-edge had a v30iot NIC, so any IoT-pool LoadBalancer lease that
elected a different node would GARP on the wrong interfaces and the LB IP would be
unreachable from devices on the IoT VLAN. Attaching v30iot to all k8s VMs lets Cilium L2
elect any node and still win ARP on v30iot from that node.
- opentofu/vm_15{0,1,2}_*.tf: extra_network_bridges = ["v30iot"]
- talos/patches/k8s-{1,2,3}.yaml: switch mgmt selector from `driver: virtio_net` to
`interface: eth0` (driver selector would now match both virtio NICs); add `interface:
eth1` for v30iot with routeMetric=4096 to keep the default route on eth0 (same pattern
as k8s-edge). X710 (i40e) keeps its driver selector and stays correct even if its
kernel name shifts to eth2 after the new virtio NIC inserts.
- infrastructure/post-crds/cilium-l2.yaml: drop nodeSelector pinning the iot policy to
k8s-edge — every node is now a valid announcer.
Apply path: deploy-infrastructure pipeline → tofu plan + manual apply (live virtio
hot-plug, no VM destroy); Talos picks up new NIC + machineconfig on next reconcile.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Blazor Server holds an open SignalR WebSocket for UI updates; the default 60s
proxy_read_timeout drops it mid-scan and the page reflects nothing. Matching the
vscode-ingress pattern (3600s + proxy_buffering off).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lives next to HA/mosquitto rather than in the generic `apps` ns since its only consumers
are HA-side. Hostname follows the vscode.home.renner.uno convention for LAN-only admin
UIs; the home-assistant Flux Kustomization owns this directory so no separate Kustomization
needed. Old kubernetes/apps/shelly-manager/ removed; the apps kustomization no longer
references it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous revert chain dropped the directed .80-.89 move along with the wrong-
assumption changes. Restoring the pool move and the .80/.81 pins for home-assistant-iot
and mosquitto-iot, leaving the DNS annotation and the (correctly reverted) default L2
policy edit alone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the user — .80-.89 is the verified-clean block (.77 and .90 brackets are live, the
block itself is empty). Repins home-assistant to .80 and mosquitto to .81.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous .224-.239 range overlapped with statically-/DHCP-assigned IoT devices at
.224 (Shelly Plug-S), .225, and .230. Cilium L2 GARP doesn't win those ARP races
durably, so the LB IPs sometimes route to the squatters and get TCP RST. Verified
.231-.247 free via ping sweep on 2026-05-13. Re-pins home-assistant to .231 and
mosquitto to .232; .226/.227 (matter-server/otbr Multus statics) keep their addresses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The default L2AnnouncementPolicy had no serviceSelector, so iot-vlan services matched both
default and iot policies. Leader election then included every node, and the lease for
home-assistant-iot landed on k8s-3 — a node with no v30iot interface — making
10.2.30.224 unreachable. Adding a `iot-vlan DoesNotExist` selector keeps the default
policy off IoT services so only k8s-edge can hold the lease.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Aligns the published name with the broker cert SAN (`mqtt.renner.uno`) so TLS on :8883
works without a name mismatch. external-dns-unifi handles `renner.uno` too — the record
stays internal-only (no Cloudflare label).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds external-dns.alpha.kubernetes.io/hostname to the mosquitto-iot Service so
external-dns-unifi creates an A record at 10.2.30.225 — workstations no longer have to
type the bare IP, and the broker cert SAN finally lines up with a resolvable name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verified by image-reflector successfully scanning tags with empty-auths and by a one-off
kubelet pull of `:main` running without ImagePullBackOff. Gitea's bearer-token flow grants
anonymous read on public packages, so the imagePullSecret + secretRef machinery was
over-engineering. Removes both placeholder SOPS Secrets, the Deployment pull-secret
reference, and the ImageRepository secretRef.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
k.rennerandClaude Opus 4.7
security-bot