Addresses the safely-codeable items of the #202 hardening bundle:
- snapshot-pvc: add kustomize.toolkit.fluxcd.io/prune: disabled so the sole
supported restore artifact (raft snapshots PVC) survives a manifest
rename/move, matching sibling data PVCs.
- policies/eso-read.hcl: version-control the eso-read policy that only lived
as a heredoc in the migration doc + live config; reference-only (OpenBao
policies aren't Flux-reconciled), with apply/verify commands inline.
- clustersecretstore: add conditions restricting the store to the exact
current consumer namespaces, closing the "any namespace can read all of
homelab/*" gap.
- network-policies/openbao-ns: add an egress stanza (default-deny egress,
allowing intra-ns, kube-dns, kube-apiserver/host, world, ICMP) to block
lateral in-cluster egress from the secrets backend.
Listener TLS/mTLS and per-app KV path re-segregation are the disruptive
remaining items and are raised as owner decisions on #202. Single-replica
raft is a documented/accepted tradeoff, not touched.
Fixes#202
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gitea (LXC 114), the woodpecker server (231) and the woodpecker agents
(910/911) were migrated into the cluster (Phase 2: Service flipped
ExternalName -> ClusterIP, now in-cluster StatefulSets/Deployment; see
kubernetes/apps/MIGRATION-gitea-woodpecker.md) and the Proxmox guests
decommissioned -- but their OpenTofu declarations were left behind with
onboot/start = true. An unscoped `tofu apply` (deploy-infrastructure.yml)
would attempt to recreate the retired stack, and backup_jobs.tf still
advertised the dead VMIDs as PBS-protected.
- delete opentofu/woodpecker_agents.tf, lxc_231_woodpecker.tf,
lxc_114_gitea_srv_renner_uno.tf
- drop VMIDs 114/231 and the 910.. agent loop from backup_jobs.tf
- remove the now-orphaned woodpecker_agents local + woodpecker_agent_count
variable (only those files referenced them)
- delete the dead .woodpecker/deploy-woodpecker-agent.yml and
install-woodpecker-server.yml pipelines (their ansible plays/roles no
longer exist)
REQUIRES `tofu state rm` of the four module resources BEFORE this merges
/ any apply runs, otherwise the plan destroy-plans them. See PR body.
Refs #199
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
object_storage.tf provisioned all four Hetzner buckets (CNPG WAL/base
backups, Velero PVC backups, cluster bootstrap secrets, tfstate) with
object_locking = false and no versioning, so a single leaked/misused S3
credential could irrecoverably delete backups and state.
Add minio_s3_bucket_versioning for all four buckets: a plain
DeleteObject no longer destroys data, the prior version survives until
purged by version ID.
object_locking is NOT flipped to true here: per the aminueza/minio
provider source, object_locking is only consumed on bucket Create, has
no Update handling and no ForceNew, so setting it on these
already-live buckets would be silently ignored -- not a real fix.
Enabling it requires destroying and recreating each bucket (full data
migration), which needs an operator decision; left as a follow-up
(see gitea issue #191 comments).
Also add minio_s3_bucket_lifecycle noncurrent-version expiration
(30 days) on the two high-churn backup buckets (cnpg, velero) to match
the retention window already enforced at the app layer (CNPG Barman
ObjectStore retentionPolicy, Velero Schedule ttl), so old S3 versions
don't accumulate indefinitely. secrets/tfstate buckets are low-churn
and left uncapped.
Fixes#191
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
The three backed-up CNPG clusters (ak-pg/im-pg/pprl-pg) bootstrap via
bootstrap.initdb, so a cold rebuild silently comes up with EMPTY databases:
barman WAL+base backups archive to Hetzner but are never wired back into a
rebuild, and the restore path was undocumented (issue #190).
- Add kubernetes/seed/RECOVERY.md: a tested manual recovery runbook with
per-cluster bootstrap.recovery + externalClusters blocks, precondition
checks, verify/revert steps, and a non-destructive CNPG restore-test drill.
- Explain the safety model: CNPG bootstrap is inert on a live cluster, so an
initdb->recovery swap cannot clobber live data; recovery has no initdb
fallback (hangs on an empty bucket), which is why permanent recovery is an
owner decision, not flipped here.
- Fix the seed-authentik Flux KS comment that falsely claimed rebuilds
auto-recover from Garage (decommissioned; backups are on Hetzner now).
- Point seed-immich / seed-paperless KS comments at the runbook.
Refs #190
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
netbird-ingress.yaml's ImagePolicy used ">=0.73.0" — unbounded above,
unlike every other of the 23 enrolled policies which pin to a major
(or major.minor). Since image-automation pushes bumps straight to
master with no PR gate, an unbounded range lets any future upstream
release (including a breaking 1.x) land automatically.
Bound it to "0.x", consistent with cnpg-barman-plugin's precedent for
pre-1.0 images: the currently deployed v0.74.4-rootless stays in
range, and a 1.x jump now requires a manual policy widen. Also adds
netbird-ingress to the README's enrolled-policies table, where it was
missing despite being wired into the kustomization.
Fixes#204 (partial — see issue comment for the scoped-token and
PR-gating items, which need an owner decision).
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
remove_ceph_cpool.yml (fleet-wide zfs destroy on pve_cluster) and
cleanup-gitea-mess.yml (deletes gitea service/binary/config/user on
all:!gitea.srv.renner.uno) are archived but still directly runnable
with no confirmation guard. Add the same fail-task safety-switch
pattern already used in plays/update-dist-upgrade.yml, requiring
-e 'confirm_destroy=true' before either play executes.
Fixes#207
Issue #195 flagged KasmWeb (browser.renner.uno) for having no Authentik
gate and hardcoded weak credentials. Owner comment overrides the
originally-proposed auth-gate fix: "can be decomissiond". Removes the
app dir, its apps/kustomization.yaml entry, the orphaned firefox
image-automation policy, and updates comments in the shared
insecure-backend ServersTransport and external-dns-unifi HelmRelease
that referenced test-browser as a consumer/example.
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
home-assistant, esphome, shelly-manager, matter-server, otbr, whisper,
speech-to-phrase, and zone-configurator all deploy into the
home-assistant namespace (per their Kustomization's `namespace:`
field, not their kubernetes/apps/ dir), and none of their PVCs were
covered by any Velero schedule -- including home-assistant-config,
which holds the .storage entity registry, recorder DB, and
secrets.yaml. Adds home-assistant-daily at the next free 06:45
stagger slot, PVC-scoped like the existing schedules.
Fixes#192
Confirmed-dead, low-risk removals only. Ambiguous items (ansible immich
role, ansible-side proxmox-dns wiring) are left in place pending owner
confirmation; tofu LXC drift is owned by #199.
- cve_list.txt: orphaned agent output (added in f22927b), referenced by
nothing; security-scan.yml uses trivy JSON outputs, not this file.
- .woodpecker/deploy-woodpecker-agent.yml: calls
ansible/plays/install-woodpecker-agent.yml and role woodpecker-agent/,
both of which no longer exist; push-trigger paths also gone.
- .woodpecker/install-woodpecker-server.yml: calls
ansible/plays/install-woodpecker-server.yml, which no longer exists.
- kubernetes/apps/woodpecker/server.yaml: fix stale comment claiming the
agent is replicas:0 (phase 2) with LXC agents still building —
contradicted by agent.yaml replicas:1.
Fixes#208 (partial)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
deploy-infrastructure.yml's plan step ran tofu plan (which evaluates
provider/data-source code) on push to any branch, with proxmox/hetzner/unifi
secrets mounted — a malicious .tf on a feature branch could exfiltrate them
pre-merge. deploy-netbird-server.yml was missing the branch: master gate its
siblings (deploy-netbird-proxy.yml, deploy-woodpecker-agent.yml) already
have, so a push to any branch touching netbird-server files triggered a real
ansible deploy with SSH key, vault pass, and proxmox token.
Fixes#196
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
shelly-manager will build+flash BLE proxies by spawning short-lived
esphome Jobs (on the shelly-manager-iot macvlan → mDNS → OTA:3232) and
streaming their logs. Give it a namespace-scoped identity to do so:
- ServiceAccount shelly-manager + Role shelly-manager-firmware
(jobs: create/get/list/watch/delete; pods: get/list/watch;
pods/log: get) + RoleBinding. No cluster-wide, secret, or exec rights.
- Wire serviceAccountName into the Deployment.
- Sync esphome/shelly-ble-proxy.yaml logger to the lean level already
OTA'd to the device (esp32_ble_client DEBUG, not VERBOSE) so the PVC
stops filling; VERBOSE is now an on-demand diagnostic via the pipeline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Provisioning a Shelly over BLE fails at the GATT connect and the cause is
inside the ESP's BLE stack — invisible from shelly-manager's side. To
diagnose and iterate without a USB re-flash per change:
* ota: (esphome platform) so future config changes push over WiFi from the
ESPHome Device Builder. Bootstrapping costs one USB flash; set
shelly_ble_proxy_ota_password in the dashboard secrets first.
* logger DEBUG with esp32_ble_client=VERBOSE / esp32_ble_tracker,
bluetooth_proxy=DEBUG, so a connect attempt logs its progress and the
failure reason.
Takes effect on the (one) USB re-flash; thereafter iterate remotely.
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
openbao.k.renner.uno -> openbao.renner.uno. The k. subdomain is a
migration-era artifact, never a production convention, and is banned
per owner directive 2026-07-14 — the wiki note that claimed a
dual-hostname convention was wrong and is being corrected.
Non-public reachability is a real control, not a hostname: the public
edge serves only explicitly configured SNI (openbao is not configured),
and the new CiliumNetworkPolicy default-denies the namespace with
explicit grants (traefik->8200, ESO->8200, intra-ns for the snapshot
CronJob) matching the <app>-ns.yaml pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
The renner.uno fallthrough forwarder still pointed at the Proxmox DNS HA VIP
10.2.40.53, which is decommissioned/unreachable. Every non-Ingress renner.uno
name (device/service hostnames like udm.renner.uno) therefore dead-ended in
SERVFAIL/timeout. A routine home-assistant pod restart on 2026-07-14 forced HA
to re-resolve its device hostnames and left ~1374 entities unavailable / ~30
config entries stuck (ZHA, Shelly, UniFi, Bluetooth, cloud integrations).
Point the fallthrough at the UDM (10.2.40.1 / fd00::1), which serves the
split-horizon renner.uno zone (verified: it answers udm.renner.uno). Also
scrub the stale 10.2.40.53 references in the CoreDNS comment, bootstrap.sh
DNS_SERVER, and the Gatus monitor (which was pointlessly probing the dead VIP).
A live hotfix has already been applied to the k8s-gateway ConfigMap to restore
resolution; this makes it durable so Flux does not revert it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PwcQMntG9y9XRkNMPDq2fe
bao operator raft snapshot save nightly at 03:00 to a dedicated PVC
(keep 7), authenticated via a new k8s-auth role openbao-snapshot ->
policy raft-snapshot (read on sys/storage/raft/snapshot only; applied
2026-07-14). Velero openbao-daily (06:15) ships ONLY the snapshot PVC
offsite via label selector — the live raft data PVC stays unbacked
(crash-consistent files; the .snap is the restore artifact). A failed
Job doubles as the sealed-after-restart alarm.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
shelly-manager now auto-discovers esphome bluetooth_proxy nodes on the IoT
VLAN via mDNS (_esphomelib._tcp) — see k.renner/shelly-manager#14. Replace
the static Ble__Proxies__0__{Name,Host,Port} env (which addressed the proxy
by shelly-ble-proxy-1.iot.renner.uno — poisoned over IPv6 by the
*.renner.uno wildcard) with a single shared fleet Noise key exposed as
Ble__Discovery__Key, from the same shelly-ble-proxy-key secret.
Merge after #14's image is live so discovery is present when the static
entry goes away.
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
authentik-secrets + samba-bridge config: values in OpenBao KV, both
round-trip-verified text. The identity provider was deliberately last —
every other consumer proved the pipeline first. ZERO sops files remain
under kubernetes/; authentik KS decryption dropped. Phase 4 (age-key
retirement) is now unblocked pending soak.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
seed/{authentik,immich,paperless} barman+hetzner creds and
seed/influxdb bootstrap: values in OpenBao KV (homelab/seed/*), all
round-trip-verified text. Future cold rebuilds no longer need the age
key for the seed phase. Decryption blocks dropped from every seed-* KS
(incl. dormant seed-audiobookshelf/seed-music-assistant and the
suspended, deprecated seed KS).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
home-assistant (5), immich, paperless (2): values in OpenBao KV, all
verified text-safe via base64 round-trip before load. ExternalSecrets
keep the old Secret names. Decryption blocks dropped from
home-assistant/immich/paperless KS plus the dormant one on
speech-to-phrase.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
evcc, otbr, libation, external-dns (apps), monitoring (4), woodpecker:
values in OpenBao KV (homelab/apps/*, loaded 2026-07-13), manifests are
now ExternalSecrets keeping the old Secret names. libation's payload is
a binary SQLite db — stored base64 in KV with decodingStrategy: Base64
on the ES. Decryption blocks dropped from apps/evcc/otbr/libation/
monitoring KS (paths sops-free) plus the dormant one on grafana-db.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
external-dns-unifi, netbird-ingress, velero (both), cloudflare-api-token:
values live in OpenBao KV (homelab/infrastructure/*, loaded 2026-07-13),
manifests become ExternalSecrets targeting the same Secret names so
consumers see no change. Both infra KS paths are now sops-free, so their
decryption blocks go too. Rollback per file = git revert (age key still
active until phase 4).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRgBFn9m8a2ncFMLusikH6
claude-botandClaude Opus 4.8
security-bot
k.renner