Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5.5 4852fde2a2 Merge main into worktree-restore-test-real
Hermes Matrix settings moved to the hermes-matrix-config ConfigMap on main; its homeserver reads ${SITE_DOMAIN}. The Home Assistant MCP URL reads ${SITE_DOMAIN}, $${HA_TOKEN} is escaped for Flux. alert-monitor gets GITEA_API on the site domain, the Hermes probe and HermesDown read ${SITE_DOMAIN}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 07:59:14 +02:00
claude-botandClaude Opus 5.5 d4860aae82 feat(site): Hermes reaches Home Assistant on the site domain
The ha-updates skill from #1400 calls Home Assistant at HA_URL; the
agent container now sets it to https://home.${SITE_DOMAIN}. The script's
fallback and the prose in SOUL.md and the skill stay literal and are
allow-listed: they are configMapGenerator files, as SOUL.md already is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 20:59:22 +02:00
claude-botandClaude Opus 5.5 1d5d1bd066 Merge main into worktree-restore-test-real
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 20:55:01 +02:00
claude-botandClaude Opus 5.5 7a7ff239ff Merge old worktree-restore-test-real; tree is feat/site-base-domain-stack-2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 20:51:15 +02:00
claude-botandClaude Opus 5.5 e141ec3414 feat(site): NetBird network, router group and setup key per site
netbird-service-sync gets NETWORK_NAME and ROUTER_GROUP_NAME from
site-settings (homelab: the controller.py defaults, Ruling 11), the
netbird-ingress ExternalSecret reads ${SITE_NETBIRD_INGRESS_KEY}. The
restore test's setup key comes from a second rotator,
netbird-secret-rotate-rtest, writing homelab/infrastructure/netbird-ingress-rtest
in production OpenBao so the raft snapshot carries it into the test. It
ships suspended with NB_AUTO_GROUPS as a marked placeholder (Ruling 2)
until the owner creates the NetBird group k8s-ingress-router-rtest.

Owner steps, none automated: create the group and set its ID; seed the KV
path (bao kv put homelab/infrastructure/netbird-ingress-rtest
setup-key=placeholder, field name as netbird-ingress: the DaemonSet reads
setup-key and the rotator's netbird-key mode writes BAO_FIELD default
setup-key), because the Job patches and a patch needs an existing secret;
bao policy write rotator-netbird (the policy is not Flux-applied); one Job
from the CronJob after merge.

Expected --changed homelab differences against the previous PR: the new
suspended CronJob in infrastructure, and the two explicit env values on
netbird-service-sync (one pod restart, behaviour unchanged).

Precondition: SITE_NETBIRD_NETWORK, SITE_NETBIRD_ROUTER_GROUP and
SITE_NETBIRD_INGRESS_KEY exist in the live site-settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 20:26:01 +02:00
claude-botandClaude Opus 5.5 5ae8da89e8 feat(site): service CIDR and DNS gateway address per site
serviceSubnets moves from talos/patches/cluster.yaml into each site patch
(homelab and qemu keep 10.96.0.0/12, hcloud gets 10.112.0.0/12; the IPv6
range fd10:96::/112 stays shared, Ruling 10). The hcloud resolver asks
10.112.0.11 first. k8s-gateway's clusterIP and the CoreDNS forward read
${SITE_DNS_GATEWAY_IP}, netbird-service-sync's SERVICE_CIDR reads
${SITE_SERVICE_CIDR}; infrastructure and infrastructure-inhouse substitute
since the wiring PR. bootstrap.sh needs no change: its k8s-gateway values
go through site-render.sh --subst since the base-domain PR, which now also
fills clusterIP.

Production: the merged Talos config is unchanged (the subnet only moves
between patch files) and --changed homelab against the previous PR is
clean (the CoreDNS comment names the variable, so it renders identically).

Precondition: SITE_SERVICE_CIDR and SITE_DNS_GATEWAY_IP exist in the live
site-settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 20:26:01 +02:00
claude-botandClaude Opus 5.5 2a874cf31c feat(site): base domain and ACME server per site
Every name a site serves (Ingress and TLS hosts, external-dns hostnames
and domain filters, Certificates, app URLs, authentik redirect URIs and
hosts, monitoring probes and links, CoreDNS and k8s-gateway zones) reads
${SITE_DOMAIN}; the ClusterIssuers read ${SITE_ACME_SERVER}, and
ACME_SERVER leaves cluster-settings. The paperless docs redirect regex
uses ${SITE_DOMAIN_RE} (Ruling 9). Home Assistant's config-sync GITEA_URL
(from #1326) is a served name too, and so are Hermes' hosts, dashboard
URL, OIDC issuer, tea login and git credential URL and the wiki's OIDC
authority, with their authentik redirect URIs. Names nothing in the
cluster serves stay literal and are listed in
testdata/site-render/allowed-production-names.txt (now also kuma.renner.uno
on the NetBird VPS, the uptime.renner.uno API group, Element Call's
rtc/turn.matrix names, the konairius Flux source, and two prose lines of
Hermes' SOUL.md: it is a configMapGenerator file, and substituting it
would change the hashed ConfigMap name and restart production);
hcloud_has_no_production_names renders every Kustomization for hcloud and
fails on any other renner.uno. KumaMonitor objects are not checked: they
are the production Uptime Kuma's monitors and keep production URLs;
kuma-sync, which alone reads them, is suspended on test sites.

bootstrap.sh installs k8s-gateway before Flux from the HelmRelease values,
which now carry ${SITE_DOMAIN}: the values go through
site-render.sh --subst <site> kubernetes/infrastructure (the same
environment the infrastructure Kustomization gets, --strict), so no
${...} reaches helm.

Expected --changed homelab differences against the previous PR (Ruling 1,
one red site-render run on this PR): authentik gate-sync gains
COOKIE_DOMAIN=renner.uno, gitea gains GITEA__server__ROOT_URL=
https://gitea.renner.uno (both production's values), monitoring-rules
refolds two long strings (parsed values unchanged), and
infrastructure's cluster-settings loses ACME_SERVER.

Precondition: SITE_DOMAIN, SITE_DOMAIN_RE and SITE_ACME_SERVER exist in the
live site-settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 20:26:01 +02:00
claude-botandClaude Opus 5.5 3f08b74d88 feat(flux): features a site lacks are suspended through site-settings
llama-server and wiki follow SITE_SUSPEND_IGPU, matter-server,
home-assistant-iot and music-assistant follow SITE_SUSPEND_IOT_VLAN,
external-dns-unifi follows SITE_SUSPEND_UNIFI, kuma-sync follows
SITE_SUSPEND_UPTIME_MONITORING (uptime monitoring does not run on test
sites; the KumaMonitor objects stay with their apps and are inert without
the controller). The root substitutes site-settings since the wiring PR,
so the values reach these Kustomization objects; homelab renders an
explicit suspend: false, which --changed treats as unchanged. Tests: the
seven follow each site, otbr stays suspended, and no unsuspended
Kustomization depends on a suspended one.

Precondition: SITE_SUSPEND_IGPU, SITE_SUSPEND_IOT_VLAN, SITE_SUSPEND_UNIFI
and SITE_SUSPEND_UPTIME_MONITORING exist in the live site-settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:52:12 +02:00
claude-botandClaude Opus 5.5 40f263e25f feat(flux): site-settings wiring; nothing substituted yet
Every Kustomization that later carries a ${SITE_*} variable substitutes
site-settings from this commit on, before any variable reaches it: live
Flux substitutes without --strict, so a variable that lands before its
Kustomization substitutes is applied literally, and one whose ConfigMap is
missing renders empty. The root sync (flux-system, via the kustomize patch
in flux-system/kustomization.yaml) and the restore-test root substitute
too, for the suspend values of the next PR; kuma-sync substitutes for
SITE_SUSPEND_UPTIME_MONITORING (uptime monitoring does not run on test
sites).

The site-settings inline manifest (all 12 keys) goes into the three site
patches. Production precondition: the ConfigMap exists on the cluster
before merge; the procedure is in
docs/superpowers/plans/2026-10-18-site-base-domain-rollout.md.

Objects with unrelated ${...} in trees that now substitute carry
kustomize.toolkit.fluxcd.io/substitute: disabled (Ruling 8: metadata only,
no pod restart). Where a later PR also puts a site value into the same
object, the unrelated reference is escaped instead ($${1},
$${INFLUXDB_TOKEN}), which renders identically.

site-render --changed homelab against the previous PR shows only the 13
substitute-disabled annotations and the site-settings entry appended to
infrastructure-crds' substituteFrom; every other path, hermes and
kuma-sync included, renders identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:52:12 +02:00
claude-botandClaude Opus 5.5 8c862298ab ci(manifests): render check for per-site values
site-render.sh renders a path the way Flux does with postBuild: kustomize
(with Flux's generated kustomization for directories without one), then
flux envsubst --strict with the Kustomization's substituteFrom ConfigMaps
in listed order (site-settings from the site's Talos patch) and its
inline substitute last. --changed renders the root and every Kustomization
naming site-settings at the base ref and at HEAD with the same site values
and fails on any difference, so a PR that moves literals into ${SITE_*}
variables proves the homelab render unchanged.

Both yq flavours: mikefarah yq (Alpine's yq, the CI image) prints a ---
line between results of different documents and evaluates string
interpolation on empty input, so results go through yqr (drops the
separators) and values are joined with tostring. BusyBox: grep has no
--include and sed is not GNU sed, so neither is relied on. The
substitute: disabled annotation counts only on the object's own
metadata, not a pod template's. --subst substitutes stdin as a
Kustomization would, for tooling that installs values outside Flux.

The CI step keeps ci-k8s:2026-09-08 until the image with yq exists; the
tag bump follows that build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:52:12 +02:00
claude-botandClaude Opus 5.5 c0994341ac ci(images): yq in ci-k8s
The site-render check (next PR) reads Talos site patches and Flux
Kustomizations with yq; ci-k8s has flux and kubectl but no yq. Alpine's
yq package is mikefarah yq. Ships alone so build-ci-images produces the
dated tag before the check that needs it is wired into a pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:52:12 +02:00
claude-botandClaude Opus 5.5 034b7aae9f docs: per-site base domain and service CIDR for the restore test (design + plan)
The restore-test site gets its own base domain (rtest.renner.uno), service
CIDR, NetBird identity and ACME server as site values in the Talos site
patches, and the features it lacks (iGPU, IoT VLAN, UniFi gateway) are
suspended there. Production renders unchanged. Approved by the owner
2026-10-18.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:52:12 +02:00
claude-botandClaude Opus 5.5 9c35f95fb3 Merge master into worktree-restore-test-real
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
review-bot review failed: context deadline exceeded
Conflict in kubernetes/apps/woodpecker/objectstore.yaml: master (#1295)
pointed wp-pg-store-source at wp-pg-woodpecker; this branch moved both
woodpecker ObjectStores to kubernetes/cnpg-object-stores/wp-pg.yaml, whose
source path comes from cnpg-paths and covers wp-pg-woodpecker through
cnpg.renner.uno/source-prefixes. Resolved by keeping the deletion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 10:28:34 +02:00
claude-botandClaude Opus 5.5 20b0621e2e fix(home-assistant): shelly-manager-auth stays with mqtt-account-controller
420bd35e moved shelly-manager-auth-externalsecret.yaml to home-assistant-iot
with the rest of shelly-manager. mqtt-account-controller, which stays in
home-assistant, mounts that Secret as well (volume shelly-manager-key, item
api-key, not optional). That made a cycle:

- mqtt-account-controller does not start without shelly-manager-auth;
- it creates the MQTT client certificates HA, mosquitto and zigbee2mqtt
  mount (mqtt-home-assistant-tls, mqtt-mosquitto-probe-tls,
  mqtt-zigbee2mqtt-bridge-tls), so they do not start either;
- home-assistant therefore never turns Ready, and home-assistant-iot, which
  would create shelly-manager-auth, depends on it.

Restore test run 29 (2026-10-16) sat in it: every other Secret in the
namespace existed, mqtt-account-controller reported
  MountVolume.SetUp failed for volume "shelly-manager-key" :
  secret "shelly-manager-auth" not found
and evcc, trmnl, zone-configurator, speech-to-phrase and home-assistant-iot
waited behind it.

The ExternalSecret goes back to home-assistant; shelly-manager in
home-assistant-iot runs after it and finds the Secret. Every Secret the
home-assistant render mounts now has its producer in the same Kustomization
(mqtt-dynsec-seed is optional).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 23:04:15 +02:00
claude-botandClaude Opus 5.5 afdeecc161 fix(monitoring-rules): drop the radius metrics Service; authentik's operator creates it
authentik-outpost-servicemonitors.yaml hand-wrote a headless Service
ak-outpost-local-radius-metrics (9300/TCP) and a ServiceMonitor for it,
because the radius outpost controller used not to expose its metrics port.
authentik 2026.8.3 does: the operator creates that same Service
(http-metrics 9300, managed-by goauthentik.io, service-type metrics) and its
own ServiceMonitor ak-outpost-local-radius selecting exactly those labels.

Restore test run 29 (2026-10-16): on a cold cluster the operator created the
Service first, with a ClusterIP. clusterIP is immutable, so the dry-run of
the hand-written clusterIP: None copy failed and took the whole
monitoring-rules Kustomization with it:

  Service/authentik/ak-outpost-local-radius-metrics dry-run failed (Invalid):
  spec.clusterIPs[0]: Invalid value: ["None"]: may not change once set

In production Flux had created it first; the operator adopted it, and Flux's
label managed-by: homelab now keeps the operator's ServiceMonitor from
matching. Removing the file lets the operator own both; when Flux prunes the
Service, the operator recreates it with its own labels.

No alert rule refers to the radius job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:51:46 +02:00
claude-botandClaude Opus 5.5 0933ccdf7b fix(restore-test): CNPG archives, backs up and prunes as in production
Patches 3c (isWALArchiver: false), 3d (no retentionPolicy) and 3e
(ScheduledBackups suspended) kept the test cluster from writing into the
production CNPG bucket. Since cffb7e02 it cannot: bootstrap.sh writes
cnpg-paths with every archive path in renner-uno-k8s-cnpg-rtest, the
cnpg-object-stores Kustomization fails rather than render without it, and
the source stores, the only ones in the production bucket, carry no
retentionPolicy.

The patches now only stopped the test from exercising what a real recovery
does next: archiving WAL into a new prefix and taking the immediate base
backup there. Removing them leaves three fewer deviations from production.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:19:54 +02:00
claude-botandClaude Opus 5.5 9b488b037e fix(flux): services that need OpenBao secrets apply after the secret store
crowdsec, external-dns-unifi, netbird-ingress and velero were in the
infrastructure Kustomization, while the ExternalSecrets that create their
Secrets were in infrastructure-crds, which depends on infrastructure. On a
cold start each was therefore installed before its Secret could exist: the
pods sat in CreateContainerConfigError until OpenBao was unsealed and ESO
synced. For the HelmReleases (crowdsec, external-dns-unifi, velero) Helm's
wait timed out meanwhile; with remediation.retries: 3 they give up after
four attempts and stay failed even once the Secret appears. Velero failing
that way stops every restore.

| workload | Secrets |
|---|---|
| crowdsec | crowdsec-bouncer-key (crowdsec, ingress), crowdsec-edge-bouncer-key |
| external-dns-unifi | external-dns-unifi-secret |
| netbird-ingress | netbird-ingress-setup-key |
| velero (+ backup-verify CronJob) | cloud-credentials, cloud-credentials-hetzner, hcloud-backup-verify |

Each now has a Kustomization of its own (clusters/homelab/
infrastructure-services.yaml), depending on infrastructure and
infrastructure-crds, with its ExternalSecrets beside it. The openbao
ClusterSecretStore in infrastructure-crds turns Ready only once OpenBao is
unsealed (a5c5d0e7), so the Secret exists before the consumer installs. One
Kustomization each so that a broken service holds back only itself.

Velero's Schedules need the velero.io CRDs from its HelmRelease, so they move
from post-crds to infrastructure/velero-schedules with a Kustomization after
velero. velero-backups-synced depends on velero (the BackupStorageLocation
and its credentials) instead of infrastructure-crds.

post-crds keeps cloudflare-api-token (consumed by its own ClusterIssuer) and
flux-git. The directories themselves stay where they are, so paths such as
the CI test infrastructure/velero/backup-verify/tests/run.sh are unchanged.

Verified: infrastructure, post-crds and the six service paths build;
infrastructure renders no ExternalSecret.

Production rollout: moving objects out of the pruned infrastructure and
infrastructure-crds Kustomizations is two-phase (add the new Kustomizations
first, then drop the old entries). Pruning the velero HelmRelease would
uninstall Velero.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:01:55 +02:00
claude-botandClaude Opus 5.5 12a0f8c000 fix(flux): renovate depends on gitea
Renovate's platform is Gitea: endpoint https://git.renner.uno/api/v1 in
apps/renovate/configmap.yaml, the same instance as gitea.renner.uno. It has
no function without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:39:07 +02:00
claude-botandClaude Opus 5.5 9dc3f5b4a0 fix(home-assistant): 30m timeout, not 4h
The 4h was sized for a seed initContainer that waited up to 3h for an HAOS
backup to appear in Garage, then downloaded and staged it. That step no
longer exists: Velero restores the config PVC in restore-home-assistant
before this Kustomization runs, and Garage is retired. The HA container's
startupProbe allows 10 minutes.

The slowest part left is ha-pg recovering from barman: its newest base backup
(ha-pg/ha-pg/base/20261016T025000) is 1.7 GiB compressed, then WAL replay and
the second instance's clone. 30m, as the other CNPG apps have; the next
restore test measures it.

With wait: true, the 4h also held one of kustomize-controller's workers that
long on a failed cold start, and with it evcc, trmnl, zone-configurator and
speech-to-phrase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:38:07 +02:00
claude-botandClaude Opus 5.5 4befc7c492 Revert "feat(talos): hcloud nodes carry ens19 as a dummy link"
This reverts commit ec8f496d. The v30iot workloads now select
net.renner.uno/iot-vlan, which no hcloud node carries, so nothing on that
site attaches to ens19 any more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:36:09 +02:00
claude-botandClaude Opus 5.5 420bd35e7f feat(iot): v30iot workloads run only where a node declares the IoT VLAN; HA no longer waits on them
Five workloads attach a macvlan on ens19 (v30iot) with DHCP IPAM:
shelly-manager and the mDNS relay (inside the home-assistant Kustomization),
matter-server, music-assistant and otbr. On a node without that VLAN the CNI
add fails and the pod never starts.

- k8s-1..3 carry net.renner.uno/iot-vlan: "true" beside the GPU label; each
  of them declares ens19 in its VLANConfig. The five Deployments select on
  it. The comments they replace described the retired k8s-edge taint.
- shelly-manager and the mDNS relay move to apps/home-assistant-iot, with a
  Kustomization of their own (depends on restore-home-assistant, gitea,
  home-assistant and the infrastructure Kustomizations). home-assistant's
  wait: true no longer covers them, so HA and everything waiting on it
  (evcc, trmnl, zone-configurator, speech-to-phrase) are Ready without the
  VLAN, and a broken IoT bridge no longer holds them back in production
  either.

On a site without the label the IoT Kustomizations fail at their timeout,
naming the Pending pods; nothing else depends on them.

Verified: home-assistant, home-assistant-iot, matter-server,
music-assistant, otbr, authentik and monitoring-rules build; the
home-assistant render has no IoT attachment left; every IoT workload carries
the selector; k8s-1 with site-homelab passes talosctl validate --strict.

Production rollout:
1. Label the nodes first (talosctl patch the nodeLabels), or the five pods go
   Pending.
2. The move between Kustomizations is two-phase (add under
   home-assistant-iot, then drop from home-assistant). shelly-manager's PVC
   already carries prune: disabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:36:01 +02:00
claude-botandClaude Opus 5.5 cffb7e027a feat(cnpg): every cluster incarnation archives to a new prefix; recovery reads the newest
A CNPG cluster restored from barman archived back into the location it
recovered from. plugin-barman-cloud's restore job checks the archive target
(plugins[] ObjectStore + serverName) and stops with "Expected empty archive"
when it holds WAL, which after a disaster it always does. It has no
exception for target == recovery source (internal/cnpgi/restore/restore.go,
v0.15). Run 28 hit it on wp-pg; the other six never got that far.

The check stays. Instead, archive and source become two ObjectStores whose
paths are chosen per incarnation:

- kubernetes/cnpg-object-stores/: all 14 ObjectStores (<x>-store archives,
  <x>-store-source is read by bootstrap.recovery), in a Kustomization of their
  own with postBuild.substituteFrom: cnpg-paths. Nothing else lives there:
  substitution blanks every ${...}, which is why it is not on the app
  Kustomizations (home-assistant's scripts, the Grafana dashboards). The
  ConfigMap is not optional; without it the paths would render empty.
- bootstrap.sh writes cnpg-paths before Flux starts, once per incarnation
  (an existing one is kept):
    CNPG_<X>_SOURCE   the prefix among <x>, <x>-g<14 digits> and the store's
                      cnpg.renner.uno/source-prefixes whose <x>/base/ holds
                      the newest base backup
    CNPG_<X>_ARCHIVE  <x>-g<UTC timestamp>, new and so empty
  Every ScheduledBackup has immediate: true, so a new incarnation has a base
  backup in its own prefix minutes after it starts, and a later recovery
  picks it. S3 access is curl --aws-sigv4 with the restore key (already in
  bootstrap.sh's secret list), so the CI image needs no new tool.
- The restore test archives to renner-uno-k8s-cnpg-rtest, created by the
  flux phase and emptied and deleted by hcloudclean. A test incarnation with a
  base backup in the production bucket would be its newest, and the next real
  recovery would read the test's data.
- The six clusters that recovered from their archive store now name
  <x>-store-source in externalClusters. serverName stays the cluster name
  everywhere. The app Kustomizations depend on cnpg-object-stores, which
  depends on infrastructure-cnpg-barman.

Verified against the bucket, read-only, 2026-10-16:
- the restore key lists renner-uno-k8s-cnpg and can create and delete a
  bucket (renner-uno-k8s-cnpg-rtest created and deleted again);
- the listing parser on real responses, top level and an encoded nested
  prefix; for wp-pg it picks wp-pg-woodpecker (newest 20261016T024000) over
  wp-pg (20260818T023502);
- leftover test serverNames (ak-pg-rtest, im-pg-rtest, ha-pg-restore-test,
  pprleval-pg) hold no base backups and match no candidate pattern.
shellcheck clean; all seven apps and cnpg-object-stores build; the render
holds exactly the 14 expected variables.

Production rollout, one time:
1. Create flux-system/cnpg-paths with the current prefixes as both source and
   archive (ak-pg, ha-pg, im-pg, mx-pg, pprl-pg, tr-pg; wp-pg-woodpecker), so
   the running clusters keep archiving where they do.
2. Move the ObjectStores in two phases (add in cnpg-object-stores, then drop
   from the apps). All 14 now carry prune: disabled; before this only ak, im
   and pprl did.
3. Whether changing externalClusters on a live Cluster restarts its pods is
   not verified; check on one cluster first.

Older generations are not cleaned up: retention applies only to the current
archive prefix. A cleanup script is planned separately.

RECOVERY.md: the table describes the new layout; the manual restore drill
reads <x>-store-source. That drill wrote 3 WAL files to ak-pg/ak-pg-rtest on
2026-07-25 despite isWALArchiver: false, so its "only ever reads" claim
does not hold; not changed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:27:21 +02:00
claude-botandClaude Opus 5.5 43dfa6158c fix(woodpecker): recover wp-pg from the prefix it archives to
wp-pg-store-source, the recovery source of wp-pg's bootstrap.recovery,
pointed at s3://renner-uno-k8s-cnpg/wp-pg: the prefix of the retired
apps/wp-pg. Since the namespace move wp-pg archives to wp-pg-woodpecker
(wp-pg-store). A cold rebuild would therefore have restored woodpecker as of
2026-08-18 and reported success.

Bucket, listed read-only 2026-10-16:
- wp-pg/wp-pg/base: newest 20260818T023502
- wp-pg-woodpecker/wp-pg/base: 20260916T024000 .. 20261016T024000 (daily),
  wals/ through timeline 6 (00000006.history from 2026-09-23)

The source now reads wp-pg-woodpecker; serverName stays wp-pg, the name the
live cluster archives under. bootstrap.recovery is honoured only when PGDATA
is absent, so the running cluster is unaffected. The old prefix wp-pg is no
longer referenced and keeps its objects; nothing applies retention to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:13:56 +02:00
claude-botandClaude Opus 5.5 cae4d83f1b fix(flux): trmnl depends on home-assistant, home-assistant on influxdb
- trmnl: the renderer reads HA's API
  (http://home-assistant-bgp.home-assistant.svc.cluster.local:8123).
- home-assistant: its InfluxDB integration writes to influxdb
  (influxdb_token in HA's secrets.yaml).

No cycle: influxdb depends only on its restore and infrastructure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:54:21 +02:00
claude-botandClaude Opus 5.5 7d9dadccba fix(flux): woodpecker depends on gitea
Gitea is woodpecker's forge and its OAuth login provider
(WOODPECKER_GITEA_URL https://gitea.renner.uno); the server has no function
without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:51:28 +02:00
claude-botandClaude Opus 5.5 0486ea2d99 fix(flux): gitea-mcp depends on gitea
gitea-mcp is a front end to Gitea's API (-H https://gitea.renner.uno) and
does nothing without it. Its image is on docker.gitea.com, which is why the
in-house-image sweep did not add this edge. /healthz reports the process
only, so without the edge the Kustomization went Ready with no Gitea behind
it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:49:34 +02:00
claude-botandClaude Opus 5.5 abf549bb9a fix(scripts): flux-graph.sh draws what Flux actually checks
A Kustomization with healthChecks and no wait is health-checked (as
infrastructure is), and one with neither has no check at all, whatever its
healthCheckExprs say. Suspended Kustomizations are filled grey and labelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:40:44 +02:00
claude-botandClaude Opus 5.5 813822462a fix(flux): every app Kustomization waits for its workloads; otbr suspended
With wait: false and no healthChecks, Ready means only that the manifests
were applied: dependents are released before the app runs, and a broken app
reports green. kustomize-controller's checkHealth returns before any check in
that case (len(Spec.HealthChecks) == 0 && !Spec.Wait), so healthCheckExprs
are not evaluated either.

- home-assistant: wait: true. The reason given (the USB meter, the canary)
  no longer holds; HA reads the meter over MQTT. evcc, zone-configurator and
  speech-to-phrase depend on it, and evcc needs mqtt-evcc-client-tls from it.
- llama-server, wiki: wait: true, reverting the wait: false of 7186e4d0. On a
  site without the iGPU they fail rather than report Ready.
- wordpress: wait: true. It set no wait, so its MariaDB healthCheckExprs
  were never evaluated.
- otbr: suspended. It is not in use.

infrastructure keeps wait: false with its explicit healthChecks, which is a
health gate on the operators, not the absence of one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:40:44 +02:00
claude-botandClaude Opus 5.5 09bf50731d feat(scripts): flux-graph.sh draws the Kustomization dependency graph
Reads every kustomize.toolkit.fluxcd.io/v1 Kustomization in
kubernetes/clusters/homelab and emits Graphviz DOT, one edge per dependsOn
entry, pointing from the dependency to its dependent. Each node carries its
wait mode and timeout. Infrastructure is green, the Velero restore chain
blue, wait: false dashed.

With no argument or a .dot path it writes the full graph; with a .svg or .png
path it renders the transitive reduction (tred), which drops edges implied by
a longer path.

Generated rather than checked in, so it cannot drift from the manifests.
Works with either yq (mikefarah's, as in the CI images, or the Python
wrapper): yq only converts YAML to JSON, jq builds the DOT.

Verified: the DOT matches the earlier hand-run yq output byte for byte;
.png and .svg render; shellcheck clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:35:12 +02:00
claude-botandClaude Opus 5.5 26976414e4 fix(authentik): its ServiceMonitor lives in monitoring-rules
The authentik chart renders server/servicemonitor.yaml whenever
server.metrics.serviceMonitor.enabled is set, without checking
.Capabilities for monitoring.coreos.com (verified in chart 2026.8.3). The
authentik Kustomization does not depend on monitoring, so on an empty cluster
the Helm install fails until kube-prometheus-stack has installed the CRDs.

The chart's ServiceMonitor is switched off and declared standalone in
monitoring-rules, as cilium and velero already do. The metrics Service is
gated only on server.metrics.enabled and stays. The selector matches the live
Service authentik-server-metrics (port metrics, 9300).

The new name, authentik-server-metrics, differs from the chart's
authentik-server, so the two never contend for one object while production
rolls over.

Swept every manifest for other chart-rendered monitoring CRs outside the
monitoring Kustomization: cilium and velero already off, blackbox-exporter
renders none (no targets, selfMonitor off).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:26:28 +02:00
claude-botandClaude Opus 5.5 ec8f496d95 feat(talos): hcloud nodes carry ens19 as a dummy link
The v30iot NetworkAttachmentDefinitions of matter-server, music-assistant,
otbr, shelly-manager and the mdns relay name master: ens19. On the hcloud
site no such link existed, so macvlan creation failed outright. Each hcloud
node now declares ens19 as a DummyLinkConfig: no address, nothing behind it.

The NADs lease by DHCP, and nothing answers on a dummy link, so the pods get
their macvlan but the dhcp IPAM step still times out.

Validated: talosctl gen config (cluster + site-hcloud + hcloud-1) then
talosctl validate --mode cloud --strict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:25:21 +02:00
claude-botandClaude Opus 5.5 7186e4d035 feat(gpu): iGPU workloads run only where a node declares the iGPU
llama-server, llama-embed, wiki-embed (gpu.intel.com/i915) and the wiki
behind them are non-essential and bound to the Intel iGPU. On a site
without one (Hetzner, restore rehearsal runs 27/28) their pods stayed
Pending, llama-server failed its health check and the Kustomizations held
kustomize-controller workers; the GPU plugin and exporter DaemonSets sat
on every node with nothing to serve.

The node patches k8s-1..3 label their machines
hardware.renner.uno/intel-gpu=true. The GPU plugin, the exporter, the
three llama Deployments and the wiki select on it, and llama-server and
wiki are no longer waited on. Elsewhere the DaemonSets have no pods and the
Deployments stay unscheduled without blocking anything.

Production rollout order: the label must be on the nodes (talosctl patch,
no reboot) before these manifests reconcile, or the GPU plugin and the
llama/wiki pods are removed or left Pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:51:49 +02:00
claude-botandClaude Opus 5.5 e9ca0c4e90 fix(mqtt-account-controller): start without the HA client certificate
The controller mounted mqtt-home-assistant-tls as a required volume, and is
the only creator of that Secret (it applies the cert-manager Certificates of
the register's cert accounts). On a new cluster - Velero restores volumes,
not Secrets - its pod could never start, and mosquitto, Home Assistant,
Zigbee2MQTT and evcc stayed in ContainerCreating behind the Secrets it would
have created.

Optional now. Verified in konairius/mqtt-account-controller main: the cert
dir is read only by the HA cert sync (02:00-05:00 window, errors logged);
the broker is dialled lazily; Secret.Read treats a missing Secret as empty;
Reconcile runs reconcileCerts even when the broker step fails. So the
controller starts, creates the Certificates, the consumers start, and the
next loop reconciles the broker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:44:26 +02:00
claude-botandClaude Opus 5.5 f3ba845e52 fix(restore-test): no writes to production NetBird from the test cluster
netbird-service-sync in the test cluster reconciled production NetBird
(netbird.renner.uno) with the PAT from the restored OpenBao: it rewrote the
public proxy services to the test cluster's Traefik ClusterIP, and
production's own controller re-created 14 and updated 5 of them every
minute from 09:53 to 10:39 UTC during run 28. The netbird-ingress routing
peers likewise joined the production mesh, announcing the same service and
LoadBalancer ranges as production.

Two safety patches in flux-root.yaml: netbird-service-sync runs with zero
replicas (3i), and netbird-ingress gets a nodeSelector no node carries (3j).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:43:19 +02:00
claude-botandClaude Opus 5.5 a02affc872 fix(velero): one restore Kustomization per schedule; each app waits for its own
Every app with backed-up volumes depended on one restore Kustomization that
was Ready only after all twelve Velero restores finished. authentik waited
for immich's photo library, home-assistant for paperless; in restore
rehearsal run 28 that was 45 minutes, during which the restore
Kustomization also held one of kustomize-controller's four workers.

kubernetes/restore/<schedule>/ now holds one Restore each, applied by a
restore-<schedule> Kustomization (clusters/homelab/restore.yaml) with the
same health check, after restore-gitea so Gitea's data still goes first.
Each app depends on its own: apps, audiobookshelf (libation through it),
authentik, home-assistant (matter-server, otbr, whisper too, same
namespace and schedule), immich, influxdb, matrix, music-assistant,
navidrome, paperless, trmnl, wordpress. Timeouts follow the data: immich
6h, audiobookshelf 3h, the rest 1h.

Not on production yet: the restore Kustomizations exist only on this
branch, so no Restore objects move between pruned Kustomizations there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:35:24 +02:00
claude-botandClaude Opus 5.5 0ac74b7b5b fix(flux): evcc depends on home-assistant
evcc reads its MQTT client certificate from mqtt-evcc-client-tls
(apps/evcc/deployment.yaml:48-59), which the mqtt-account-controller in
the home-assistant Kustomization delivers into the evcc namespace
(apps/home-assistant/mqtt-accounts.yaml:40-43), and connects to
mosquitto.home-assistant. On a cold cluster evcc started first and sat in
CreateContainerConfigError (restore rehearsal runs 27, 28). No cycle:
home-assistant does not reach evcc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:57:46 +02:00
claude-botandClaude Opus 5.5 4ebc00e57a fix(bird): runs only where the site routes pods into table 47
bird fills routing table 47, which exists only where the site patch routes
pod traffic into it (site-homelab.yaml RoutingRuleConfig), and renders its
config from node annotations that only that site and the k8s-N patches set
(pod0, egress gateway). On Hetzner the template rendered <no value> and the
init container crash-looped ("syntax error, unexpected '<'", run 28).

site-homelab.yaml labels its nodes net.renner.uno/pod-table=47 and the
DaemonSet selects on it; on a site without table 47 it has no pods.

Production rollout order: the label must be on the nodes (talosctl patch,
no reboot) before the DaemonSet change reconciles, or bird is removed from
every node and pod traffic falls through to the main table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:57:25 +02:00
claude-botandClaude Opus 5.5 37352808c6 fix(flux): monitoring depends on influxdb
Grafana mounts the grafana-influxdb-token Secret, which a Job in the
influxdb app writes (apps/influxdb/grafana-token-job.yaml). On a cold
cluster monitoring started first; Grafana sat in CreateContainerConfigError
and the monitoring Kustomization held a kustomize-controller worker on its
health check (restore rehearsal run 28). No cycle: influxdb does not use
monitoring's CRDs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:48:16 +02:00
claude-botandClaude Opus 5.5 36b42677d6 fix(flux): in-house-image infrastructure waits for Gitea; fallback images don't
bird and netbird-service-sync run images from Gitea's registry with no
copy on ghcr.io. In the infrastructure Kustomization they started long
before the restored Gitea served them and sat in ImagePullBackOff (runs 27,
28). infrastructure cannot depend on gitea, which depends on it, so both
move to a new Kustomization, infrastructure-inhouse, that depends on
infrastructure, infrastructure-crds and gitea. Their namespace stays in
infrastructure.

Images published to ghcr.io too have a fallback in the nodes' registry
mirror and need no ordering: apps (ical-proxy: GitHub CI pushes
ghcr.io/konairius/ical-proxy main-<sha>) and trmnl (alpine-jq) lose the
gitea edge from the previous commit; multus cni-plugins stays in
infrastructure. wiki keeps it: no ghcr copy of konairius/wiki was found.

The secret-rotator and backup-verify CronJobs also use Gitea images but
only run on schedule; they are left in place.

Production rollout must be two-phase (moving objects between pruned
Kustomizations): first add infrastructure-inhouse with both paths while
infrastructure still lists them, then remove them from infrastructure.
Applied as one change, infrastructure prunes bird before the new
Kustomization re-creates it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:43:34 +02:00
claude-botandClaude Opus 5.5 abc2a1b0a5 fix(flux): apps with in-house images depend on gitea
Their images are pulled from Gitea's registry. On a cold cluster they
started before the restored Gitea served it and sat in ImagePullBackOff,
retried on the kubelet's back-off (restore rehearsal runs 27, 28). They now
wait for the gitea Kustomization: apps (ical-proxy), authentik (server,
LDAP outpost, gate-sync), home-assistant (shelly-manager,
mqtt-account-controller), paperless (ftp-scanner), review-bot, trmnl
(renderer), wiki.

No cycle: gitea depends only on restore-gitea and the infrastructure
layers. The infrastructure Kustomization itself also runs in-house images
(bird, netbird-service-sync, secret-rotator, backup-verify, cni-plugins);
it cannot depend on gitea, which depends on it, and is left for a
separate change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:40:25 +02:00
claude-botandClaude Opus 5.5 d25366a80c fix(bootstrap): read the hcloud node range from the machine document only
site-hcloud.yaml now holds a ResolverConfig document too; the unselected
query printed "10.90.0.0/16" and "null" and hcloud network create
rejected the range (run 28).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:12:18 +02:00
claude-botandClaude Opus 5.5 4a4390ef46 feat(dns): the nodes resolve through k8s-gateway at every site, from the first pod on
Node-level image pulls of gitea.renner.uno/* go through the node's resolver,
not the cluster's. Only site-homelab.yaml pointed it at k8s-gateway (PR
#1292); on Hetzner the nodes asked Hetzner's resolvers, got the public
proxy (blocked by the test firewall) and fell through to ghcr, which only
mirrors konairius/* - bird, netbird-service-sync and review-bot stayed in
ImagePullBackOff (restore rehearsal run 27) while the restored Gitea had
them.

PR #1292 also left a cold-start gap (run 23): until k8s-gateway exists the
node's first resolver does not answer, every lookup waits out the 2 s read
timeout, CoreDNS gives up first and pods resolve nothing - Flux included.

- Each site patch states its node resolver list (ResolverConfig: k8s-gateway
  first, then the site's resolver: UDM at home, Hetzner's on hcloud) and
  k8s-gateway's upstream as a k8s-gateway-site ConfigMap (deployment
  dnsPolicy/dnsConfig), read by the HelmRelease via valuesFrom. The Corefile
  forwards to the pod's resolv.conf, never to the node's resolver (loop).
- bootstrap.sh installs k8s-gateway right after the nodes are Ready, from
  the HelmRelease under the site values, as Flux merges them; Flux adopts
  the release. k8s-gateway answers before anything else needs DNS.

helm template (chart 4.0.1) renders forward . /etc/resolv.conf with the
site's nameservers in dnsConfig for homelab and hcloud. talosctl validate
passes for k8s-1 and hcloud-1.

Rollout order for production: the k8s-gateway-site ConfigMap must exist
before this HelmRelease reconciles (valuesFrom is not optional).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:04:05 +02:00
claude-botandClaude Opus 5.5 a5c5d0e79d fix(flux): infrastructure is Ready only once OpenBao is unsealed
The infrastructure Kustomization health-checked the openbao HelmRelease,
which is Ready as soon as the chart is installed, sealed or not. On a cold
cluster infrastructure-crds then applied the openbao ClusterSecretStore
while OpenBao was unreachable and then sealed ("503 Vault is sealed").
External Secrets backs the store off exponentially, so every
ExternalSecret stayed "ClusterSecretStore not ready" for minutes after
OpenBao was restored and unsealed; Velero's pod could not mount
cloud-credentials-hetzner meanwhile (restore rehearsal run 27; ~15 min in
run 22).

The openbao StatefulSet is now a health check too. Its readiness probe is
`bao status`, which fails while sealed, and podManagementPolicy is
OrderedReady, so 3/3 ready means every replica is unsealed. No cycle:
OpenBao's PVC comes from infrastructure-ceph-cluster, which does not
depend on infrastructure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 10:34:42 +02:00
claude-botandClaude Opus 5.5 ed8241384d feat(cilium): native-routing CIDRs are a site fact too
ipv4/ipv6NativeRoutingCIDR exempt traffic into those ranges from
masquerading (Cilium: handed to the kernel "without applying any SNAT").
At home that is the point: 10.0.0.0/8 and fd00::/8 let LAN services see
real pod IPs, routed back by the UDM via BGP. On Hetzner the same range
covers the node segment 10.90.0.0/16, whose gateway drops packets with a
source not assigned to the sending server (uRPF, Hetzner networks FAQ).

Run 26: a pod on hcloud-2 reached only its own node's kube-apiserver
(10.90.0.3:6443 -> 401); 10.90.0.2 and 10.90.0.4 timed out, the pod's SYNs
went to-stack with the pod address. Flux controllers on hcloud-2 crash-
looped and flux install failed.

The CIDRs move into each site's cilium-site values: homelab and qemu keep
10.0.0.0/8 and fd00::/8, hcloud exempts only the pod CIDRs. Production
still renders identically (helm template, TLS secrets excluded); talosctl
validate passes for k8s-1 and hcloud-1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 10:14:15 +02:00
claude-botandClaude Opus 5.5 16b6aee04d feat(cilium): pod routing is a site fact, from the site patch
routingMode, autoDirectNodeRoutes and nodePort.directRoutingDevice describe
the site's network, not the cluster: at home pod traffic crosses nodes
natively on pod0 (VLAN 48, one L2 segment). On Hetzner Cloud that cannot
work: networks are layer 3, route only server addresses, and carry IPv4
only (docs.hetzner.com networks FAQ), so cross-node pod traffic had no
path (run 25: every DNS query to a CoreDNS pod on another node timed out).

Each Talos site patch now carries a cilium-site ConfigMap as an inline
manifest with its part of Cilium's values; the HelmRelease reads it via
valuesFrom and bootstrap.sh merges it the same way (site values first,
HelmRelease values on top, as Flux merges them):
- homelab: native, autoDirectNodeRoutes, directRoutingDevice pod0
- hcloud: tunnel (VXLAN) over the node segment
- qemu: native on the provisioner's single L2 bridge

Production renders unchanged: helm template of cilium 1.20.3 with master's
values vs. homelab site values + the new values gives identical manifests
(random TLS secrets excluded). talosctl validate passes for k8s-1 (metal)
and hcloud-1 (cloud).

Rollout order for production: the ConfigMap must exist before this
HelmRelease reconciles (valuesFrom is not optional). Talos creates inline
manifests on a config change but never updates them; changes go through
talosctl upgrade-k8s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:59:21 +02:00
claude-botandClaude Opus 5.5 4922d8777e fix(bootstrap): split the hcloudtest tcp egress rule at 100 prefixes
Hetzner rejects a firewall rule with more than 100 destination prefixes
("rules[2].destination_ips: value required to be smaller"; the limit is
not in the API spec or the FAQ — measured on an unattached firewall: 100
accepted, 101 rejected). Run 24 failed on the single rule carrying all
245 complement prefixes. The tcp allow-list is now one rule per 100.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:19:52 +02:00
claude-botandClaude Opus 5.5 f58e06fc76 fix(bootstrap): create Cilium's CNI subdirectory and install Multus on a cold cluster
Cilium's cni.confPath is /etc/cni/net.d/multus, a directory Multus creates
(#832). On an empty cluster neither exists: /etc/cni is read-only to the
kubelet on Talos, so the agent's DirectoryOrCreate mount cannot make it,
and the install waits forever. Without Multus the kubelet reports "cni
plugin not initialized" and nothing schedules, Flux included.

A hostNetwork pod per node creates the directory (there is no pod network
before Cilium), and Multus is applied from the manifests Flux owns right
after Cilium. Found in the Hetzner restore rehearsal, where both steps ran
successfully in runs 22 and 23; this is the production bootstrap path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:07:00 +02:00
claude-botandClaude Opus 5.5 b550e84fcc feat(restore-test): Flux root with the test's deviations, and the egress check
flux-root.yaml applies the production tree from the GitHub mirror; its
patches are the source rewrite, no Ceph cluster network (Hetzner networks
are layer 3), and the safety set: Velero ReadOnly, no CNPG WAL archiving,
retention or base backups, no MariaDB dumps, rotators suspended, Gitea
without world egress.

egress-check.yaml asserts from a pod: public DNS, github.com https and ssh,
ghcr.io and object storage reachable; every proxy.renner.uno address closed
on 443. stderr is reported, not discarded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:05:13 +02:00
claude-botandClaude Opus 5.5 bb6874f59c feat(bootstrap): hcloudtest boots production's config, contained by egress firewall
hcloudtest runs SITE=hcloud on cpx52 with the OSD on the system disk, and
nothing stands in for the home network. Ported from worktree-hcloud-wg-site:
the Talos snapshot builder (removes its server on failure), this host's
inbound firewall replacing earlier runs', the Hetzner token from OpenBao, the
FLUX_SOURCE=restore-test phase (flux install, per-run read-only deploy key on
the GitHub mirror, restore-test-git Secret, flux-root.yaml on FLUX_BRANCH),
its safety prechecks, access files under ~/.local/state/homelab-hcloudtest
instead of access-staging, hcloudclean deleting ssh keys and restore-test
deploy keys, and the 30-minute OpenBao status wait.

Containment: an outbound firewall on the nodes, applied before any config,
admits udp and icmp anywhere and tcp to the complement of proxy.renner.uno's
addresses, resolved over DoH at run time (cidr_complement, in bash; fails
above Hetzner's 500 effective rules). The egress check pod runs at the end
of hcloudtest and again before the restore test writes any secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:05:13 +02:00
claude-botandClaude Opus 5.5 af7c48e29f fix(talos): etcd subnet in the site patches; hcloud OSD on the system disk
cluster.yaml named VLAN 46 as etcd's advertised subnet, a fact of the home
network. It moves unchanged to site-homelab.yaml (the rendered production
config is byte-identical); site-hcloud.yaml advertises etcd on the node
segment bootstrap.sh provisions, 10.90.0.0/16.

cpx52 carries 480 GB local disk and the project's volume quota cannot hold
three OSD volumes: EPHEMERAL is capped at 60GiB and osd-data takes at least
345GiB of the system disk (ported from 92c7ee73, 8d42d89c).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:58:41 +02:00
claude-botandClaude Opus 5.5 33077cd1bc fix(velero): restore Gitea first, the rest after it
Gitea is the registry for the in-house images. With one shared restore gate it waited for every volume, Immich's included, while everything pulling from it looped on image pull errors. restore-gitea restores only Gitea and the gitea Kustomization depends on it alone; the other Restores depend on it, so Velero moves Gitea's data first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:54:18 +02:00
claude-botandClaude Opus 5.5 99af3d6efc fix(rook): install the ceph-csi-drivers chart Rook requires
Since Rook 1.20 the CSI Driver and OperatorConfig resources come from the
ceph-csi-drivers chart (rook Documentation/Helm-Charts/csi-drivers-chart.md:
"Rook requires the installation of the Ceph-CSI Driver chart"). The chart
was never installed here: the live cluster's Drivers are the ones the
operator migrated once during the 1.19->1.20 upgrade (2026-05-09), and
PR #12 only rendered the ServiceAccounts and RBAC. A fresh cluster
therefore has no CSI driver and provisions no volume: found by the
Hetzner restore test, where every PVC stayed Pending ("Waiting for a
volume to be created either by the external provisioner
'rook-ceph.rbd.csi.ceph.com'").

Values: Rook's recommended values for 1.21.0 plus the live shape:
2 controller replicas with Recreate, the workload-class=pinned
toleration on the node plugins, a snapshotter for RBD (Velero's CSI
snapshots of ceph-block) and group snapshots for CephFS.

The chart's ServiceAccounts are named per driver
(rook-ceph-rbd-csi-ceph-com-ctrlplugin-sa, ...), so they do not collide
with the static ones; the static ServiceAccounts and RBAC stay until the
live CSI pods run on the chart's, and go in a second change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:54:18 +02:00
claude-botandClaude Opus 5.5 dcc034be92 feat(velero): the cluster restores its app volumes itself
One Velero Restore per Schedule (kubernetes/restore/restores.yaml),
applied by Flux, restoring the schedule's newest completed backup. Every
app Kustomization with backed-up volumes depends on the new `restore`
Kustomization, so on a fresh cluster no app starts on an empty volume.
Restores the capability 952e6150 removed without an owner decision.

Ordering and failure handling, from Velero release-1.17 source:
- A Restore whose schedule has no synced backup yet goes FailedValidation
  for good (restore_controller.go). `restore` therefore depends on
  `velero-backups-synced`, which is Ready once the BackupStorageLocation
  has completed a sync (status.lastSyncedTime).
- After that sync, FailedValidation with "No backups found for schedule"
  means there is nothing to restore and counts as done; any other
  FailedValidation, Failed or PartiallyFailed fails the Kustomization.
- A claim that already exists is skipped (csi pvc_action: "PVC already
  exists"), and the Restore completes with a warning. On the running
  cluster the Restores touch nothing.
- Velero deletes a Restore with the backup it came from (30-day TTL);
  Flux recreates it, and it again skips the existing claims.
- All StorageClasses bind Immediate, so the data mover does not wait for
  a consuming pod.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:54:18 +02:00
claude-botandClaude Opus 5.5 fb7c12a92a fix(gitea): no pinned volumeName on data-gitea-0
The claim named its production PV, which does not exist on a rebuilt
cluster: the claim stayed Pending there, and a restored claim bound to a
new volume made the gitea Kustomization fail on the immutable field.

volumeName leaves git and the gitea Kustomization ignores
/spec/volumeName on this claim. kustomize-controller is the field's
only manager on the live claim, so its ignore rule adopts the live value
instead of clearing it; size growth still applies.

To verify after merge: the gitea Kustomization reconciles Ready and the
live claim still names pvc-e7c56db2-481d-40cb-b94a-b264b7b4b5be.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:54:18 +02:00
claude-botandClaude Opus 5.5 5927e4bc0e fix(talos): IPv4 first in nodeIP.validSubnets, as the live nodes run
Talos keeps the validSubnets order (siderolabs/net FilterIPs) and the first
entry becomes the primary node IP. Git had fd02:46::/64 first since
be643533 while k8s-1 runs with 10.2.46.0/24 first. A cluster built from git
gets an IPv6 primary and kube-apiserver exits with
'service IP family "10.96.0.0/12" must match public address family
"fd02:46::11"' — reproduced on a Hetzner rebuild rehearsal 2026-10-15.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:54:18 +02:00
135 changed files with 2030 additions and 222 deletions
+17
View File
@@ -37,6 +37,10 @@ when:
- ".woodpecker/**"
- "scripts/ci/promtool-rules.sh"
- "scripts/ci/yaml-citations.sh"
- "scripts/ci/site-render.sh"
- "scripts/ci/site-render-lib.sh"
- "scripts/ci/test-site-render.sh"
- "scripts/ci/testdata/site-render/**"
# THE THIRD DISCOVERY CHECK WENT TO A SCRIPT. kustomize-build and shellcheck
# both hand-roll the same shape — `find`, a floor guard against empty discovery,
@@ -341,3 +345,16 @@ steps:
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-09-08
commands:
- sh scripts/ci/yaml-citations.sh
# Per-site values (${SITE_*}, substituted by Flux from site-settings) must
# render, and must not change what the homelab site renders today.
- name: site-render
depends_on: [] # see note above `steps:`
image: gitea.renner.uno/gitea_admin/ci-k8s:2026-09-08
commands:
- bash scripts/ci/test-site-render.sh
- |
if [ "$CI_PIPELINE_EVENT" = "pull_request" ]; then
git fetch --depth=1 origin "$CI_COMMIT_TARGET_BRANCH"
bash scripts/ci/site-render.sh --changed homelab FETCH_HEAD
fi
+2 -1
View File
@@ -14,7 +14,8 @@ RUN apk add --no-cache \
git \
openssh-client \
openssl \
sops
sops \
yq
RUN curl -sfL -o /usr/local/bin/talosctl \
"https://github.com/siderolabs/talos/releases/download/${TALOS_VERSION}/talosctl-linux-amd64" \
@@ -0,0 +1,125 @@
# Site settings: production rollout
The stack (PR0 ci-k8s image, PR1 render check, PR2 wiring, PR3 suspends,
PR4 base domain, PR5 service CIDR, PR6 NetBird identity) needs the
`flux-system/site-settings` ConfigMap on the production cluster before PR2
merges: PR2 makes the root sync and the child Kustomizations name it in
`postBuild.substituteFrom`, and Flux fails a Kustomization whose
substituteFrom ConfigMap is missing.
Talos creates the ConfigMap from the `site-settings` inline manifest in
`kubernetes/talos/patches/site-homelab.yaml`. The owner runs every step below;
nothing here is automated.
## 1. Patch file
All 12 keys, exactly as in `site-homelab.yaml`:
```sh
cat > site-settings-patch.yaml <<'EOF'
cluster:
inlineManifests:
- name: site-settings
contents: |
apiVersion: v1
kind: Namespace
metadata:
name: flux-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: site-settings
namespace: flux-system
data:
SITE_DOMAIN: renner.uno
SITE_DOMAIN_RE: renner\.uno
SITE_ACME_SERVER: https://acme-v02.api.letsencrypt.org/directory
SITE_SERVICE_CIDR: 10.96.0.0/12
SITE_DNS_GATEWAY_IP: 10.96.0.11
SITE_NETBIRD_NETWORK: k8s public ingress
SITE_NETBIRD_ROUTER_GROUP: k8s-ingress-router
SITE_NETBIRD_INGRESS_KEY: infrastructure/netbird-ingress
SITE_SUSPEND_IGPU: "false"
SITE_SUSPEND_IOT_VLAN: "false"
SITE_SUSPEND_UNIFI: "false"
SITE_SUSPEND_UPTIME_MONITORING: "false"
EOF
```
Check it against the repo (no output means equal):
```sh
diff \
<(yq -r 'select(.cluster) | .cluster.inlineManifests[] | select(.name == "site-settings") | .contents' kubernetes/talos/patches/site-homelab.yaml) \
<(yq -r '.cluster.inlineManifests[0].contents' site-settings-patch.yaml)
```
## 2. Dry run, k8s-1
Only changed lines are printed, so no other part of the machine config (and no
secret in it) reaches the terminal:
```sh
talosctl -n k8s-1.srv.renner.uno patch machineconfig \
--patch @site-settings-patch.yaml --mode=no-reboot --dry-run \
| grep -E '^[+-]'
```
Expected: only `+` lines of the `site-settings` inline manifest (plus the
`---`/`+++` diff headers). Anything else: stop.
## 3. Apply, one node at a time
```sh
talosctl -n k8s-1.srv.renner.uno patch machineconfig \
--patch @site-settings-patch.yaml --mode=no-reboot
```
Verify (section 4), then repeat steps 2 and 3 for `k8s-2.srv.renner.uno`, then
`k8s-3.srv.renner.uno`. `--mode=no-reboot` fails instead of rebooting if
Talos ever wanted a reboot for this change.
## 4. Verify
```sh
kubectl -n flux-system get cm site-settings -o yaml
```
`data` must equal the homelab column of `site-homelab.yaml`: the 12 keys and
values of section 1, nothing else. All three nodes must carry the patch before
PR2 merges (a node rebuilt from its own config must not drop the entry).
## 5. Later changes
Talos only creates inline manifests: it does not update an object that already
exists. Changing a value or adding a key later needs either the delete
pattern in the machine config (`$patch: delete` on the `site-settings` entry
and the new entry, in one `talosctl patch` call) or `talosctl upgrade-k8s`
to sync the bootstrap manifests. Editing the live ConfigMap is drift: the next
rebuild restores the Talos value.
## 6. Before each PR merges
Every key the PR substitutes must exist live:
| PR | Keys |
|---|---|
| PR2 wiring | the ConfigMap itself (no key substituted yet) |
| PR3 suspends | `SITE_SUSPEND_IGPU`, `SITE_SUSPEND_IOT_VLAN`, `SITE_SUSPEND_UNIFI`, `SITE_SUSPEND_UPTIME_MONITORING` |
| PR4 base domain | `SITE_DOMAIN`, `SITE_DOMAIN_RE`, `SITE_ACME_SERVER` |
| PR5 service CIDR | `SITE_SERVICE_CIDR`, `SITE_DNS_GATEWAY_IP` |
| PR6 NetBird | `SITE_NETBIRD_NETWORK`, `SITE_NETBIRD_ROUTER_GROUP`, `SITE_NETBIRD_INGRESS_KEY` |
```sh
for k in SITE_DOMAIN SITE_DOMAIN_RE SITE_ACME_SERVER; do # the PR's keys
printf '%s=%s\n' "$k" "$(kubectl -n flux-system get cm site-settings -o go-template="{{index .data \"$k\"}}")"
done
```
Each line must show the homelab value; an empty value or `<no value>` blocks
the merge. Flux on the cluster substitutes without `--strict`: a missing key
becomes an empty string in the applied object, it does not fail.
The site-render CI step (`--changed homelab` against the target branch) shows
each PR's render differences; the PR body lists the expected ones.
@@ -0,0 +1,163 @@
# Site-specific base domain and service CIDR Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** The Hetzner restore-test site runs production's tree under its own base domain, service CIDR, NetBird identity and ACME server, with the features it lacks suspended, so it can be reached through the NetBird proxy without touching production.
**Architecture:** Each Talos site patch carries one more inline manifest, ConfigMap `flux-system/site-settings` (plus the `flux-system` Namespace), holding the site's values as `SITE_*` variables. Flux reads it through `postBuild.substituteFrom`: the root sync substitutes it into the Kustomization objects (for `spec.suspend`), and every Kustomization whose manifests carry a site value substitutes it into its manifests. Production's values reproduce today's literals, so its rendered output is byte-identical; a render check enforces that per PR.
**Tech Stack:** Flux v2 (kustomize-controller v1.9.5, flux CLI ≥ v2.8 for `flux envsubst --strict`), kustomize via `kubectl kustomize`, Talos inline manifests, bash, yq (both flavours must work), Woodpecker CI.
**Spec:** `docs/superpowers/specs/2026-10-18-site-base-domain-design.md`
## Global Constraints
- Only what must differ, differs. Every `SITE_*` variable exists because of a collision named in the spec.
- Production (`site-homelab`) renders byte-identical to `origin/master` for every changed path, except the two intended pod-spec additions named in Task 3 (Gitea `GITEA__server__ROOT_URL`, gate-sync `COOKIE_DOMAIN`), each set to the value already in effect.
- Site values live in the Talos site patches (`kubernetes/talos/patches/site-{homelab,hcloud,qemu}.yaml`). No test-only patch in `restore-test/flux-root.yaml` may introduce a site difference.
- Never force-push; stacked PRs, each based on the previous; never self-merge.
- Moving objects between pruned Kustomizations is not part of this plan; if a task finds it needs one, stop and use the #1307 prune handover.
- Talos changes reach production as incremental `talosctl patch machineconfig` (never the whole repo file), `--dry-run` output filtered so no machine-config secret is printed, k8s-1 first, then k8s-2, k8s-3.
- Matrix stays hardcoded (`server_name: matrix.renner.uno`); out of scope.
- Comments state facts, not history. No prose in config files.
Site values (exact):
| Variable | homelab | hcloud | qemu |
|---|---|---|---|
| `SITE_DOMAIN` | `renner.uno` | `rtest.renner.uno` | `renner.uno` |
| `SITE_ACME_SERVER` | `https://acme-v02.api.letsencrypt.org/directory` | `https://acme-staging-v02.api.letsencrypt.org/directory` | `https://acme-v02.api.letsencrypt.org/directory` |
| `SITE_SERVICE_CIDR` | `10.96.0.0/12` | `10.112.0.0/12` | `10.96.0.0/12` |
| `SITE_DNS_GATEWAY_IP` | `10.96.0.11` | `10.112.0.11` | `10.96.0.11` |
| `SITE_NETBIRD_NETWORK` | `k8s public ingress` | `k8s public ingress rtest` | `k8s public ingress` |
| `SITE_NETBIRD_ROUTER_GROUP` | `k8s-ingress-router` | `k8s-ingress-router-rtest` | `k8s-ingress-router` |
| `SITE_NETBIRD_INGRESS_KEY` | `infrastructure/netbird-ingress` | `infrastructure/netbird-ingress-rtest` | `infrastructure/netbird-ingress` |
| `SITE_SUSPEND_IGPU` | `false` | `true` | `true` |
| `SITE_SUSPEND_IOT_VLAN` | `false` | `true` | `true` |
| `SITE_SUSPEND_UNIFI` | `false` | `true` | `true` |
| `SITE_SUSPEND_UPTIME_MONITORING` | `false` | `true` | `true` |
`10.112.0.0/12` overlaps nothing in the repo (checked: Ceph SAN `10.99.0.0/29` sits inside `10.96.0.0/12`, Hetzner nodes use `10.90.0.0/16`, NetBird `100.64.0.0/10`).
## Review Focus
1. A manifest in a newly substituting Kustomization contains unrelated `${...}` (shell scripts, Grafana dashboards, proxy configs, CEL) — Flux would blank it silently. Expected: `flux envsubst --strict` in the render check fails on it; the object carries `kustomize.toolkit.fluxcd.io/substitute: disabled` or the `$` is escaped as `$${...}`. Test: Task 1's `test-site-render.sh` case `strict_fails_on_unknown_var`.
2. A suspended Kustomization is a `dependsOn` target of one that is not suspended on the same site — the dependent waits forever. Expected: nothing unsuspended depends on a suspended one. Test: Task 2 case `no_dependents_of_suspended`.
3. A served name left hardcoded on the hcloud site (an Ingress host, redirect URI, probe target) — the test publishes or calls a production name. Expected: the hcloud render contains `renner.uno` only as `rtest.renner.uno`, in the allow-list (label/annotation keys, image references, the Git source, email addresses, external hosts, Matrix). Test: Task 3 case `hcloud_has_no_production_names`.
4. The restore-test root (`restore-test/flux-root.yaml`) lacks the substitution the production root has, so suspends and domains render empty on the test. Expected: both roots substitute `site-settings`. Test: Task 2 case `restore_root_substitutes`.
5. The test's NetBird peers gain access to production through NetBird's "All" group policies. Expected: verified against the live NetBird policies before run 31 (Task 6 Step 2).
---
### Task 1: Render check for site values
**Files:**
- Create: `scripts/ci/site-render.sh`
- Create: `scripts/ci/test-site-render.sh`
- Create: `scripts/ci/testdata/site-render/` (fixtures)
- Modify: `.woodpecker/ci-manifests-test.yml` (new step `site-render`)
**Interfaces:**
- Produces: `scripts/ci/site-render.sh <site> <path>` — prints `kubectl kustomize <path>` piped through `flux envsubst --strict` with the `SITE_*` values read from the `site-settings` inline manifest of `kubernetes/talos/patches/site-<site>.yaml`; exit non-zero on an undefined variable. With `--root`, renders `kubernetes/clusters/homelab` the way the root sync does.
- Produces: `scripts/ci/site-render.sh --changed <site> <base-ref>` — for every Flux Kustomization path whose object has `postBuild.substituteFrom` naming `site-settings`, diff the render at HEAD against the plain render at `<base-ref>`; exit 1 on a difference, printing it.
- [ ] **Step 1: Write the failing tests** in `scripts/ci/test-site-render.sh` (same style as `scripts/ci/test-yaml-citations.sh`), with fixtures under `scripts/ci/testdata/site-render/`:
- `renders_site_value`: fixture kustomization with `host: app.${SITE_DOMAIN}` and a fixture site patch with `SITE_DOMAIN: example.test` → output contains `host: app.example.test`.
- `strict_fails_on_unknown_var`: fixture containing `${FOO}` → exit status non-zero, stderr names `FOO`.
- `escaped_dollar_survives`: fixture containing `$${FOO}` → output contains `${FOO}`.
- `disabled_object_untouched`: object annotated `kustomize.toolkit.fluxcd.io/substitute: disabled` containing `${FOO}` → output keeps `${FOO}`, exit 0.
- [ ] **Step 2: Run** `bash scripts/ci/test-site-render.sh` — expected FAIL (`site-render.sh` missing).
- [ ] **Step 3: Implement `site-render.sh`.** Read the ConfigMap from the site patch with the same yq expression bootstrap.sh uses for `cilium-site` (`select(.cluster) | .cluster.inlineManifests[] | select(.name == "site-settings") | .contents`), no `-y`; export its `data` keys; honour the `substitute: disabled` annotation by rendering those objects untouched (split the stream with yq, substitute the rest). Never `2>/dev/null`.
- [ ] **Step 4: Run** `bash scripts/ci/test-site-render.sh` — expected PASS; `shellcheck -S warning scripts/ci/site-render.sh scripts/ci/test-site-render.sh` clean.
- [ ] **Step 5: Add CI step** `site-render` to `.woodpecker/ci-manifests-test.yml` running the test script, then `site-render.sh --changed homelab "$CI_COMMIT_TARGET_BRANCH"` on pull requests. Image must carry flux and yq (check `ci/images/ci-k8s`; add flux there if missing, in this task).
- [ ] **Step 6: Commit** `ci(manifests): render check for per-site values`.
### Task 2: site-settings ConfigMap, root substitution, feature suspension
**Files:**
- Modify: `kubernetes/talos/patches/site-homelab.yaml`, `site-hcloud.yaml`, `site-qemu.yaml` (new inline manifest `site-settings`: Namespace `flux-system` + ConfigMap with all `SITE_*` keys from the table)
- Modify: `kubernetes/clusters/homelab/flux-system/kustomization.yaml` (patch adding `postBuild.substituteFrom: [{kind: ConfigMap, name: site-settings}]` to Kustomization `flux-system`; bootstrap keeps this file)
- Modify: `kubernetes/restore-test/flux-root.yaml` (same `postBuild` on Kustomization `restore-test`)
- Modify: `kubernetes/clusters/homelab/llama-server.yaml`, `wiki.yaml` (`suspend: ${SITE_SUSPEND_IGPU}`), `home-assistant.yaml` (`matter-server`, `home-assistant-iot`: `${SITE_SUSPEND_IOT_VLAN}`), `music-assistant.yaml` (`${SITE_SUSPEND_IOT_VLAN}`), `infrastructure-services.yaml` (`external-dns-unifi`: `${SITE_SUSPEND_UNIFI}`), `kuma-sync.yaml` (`${SITE_SUSPEND_UPTIME_MONITORING}`; owner, 2026-10-20: uptime monitoring does not run on the test cluster)
- Test: `scripts/ci/test-site-render.sh`
**Interfaces:**
- Consumes: `site-render.sh` (Task 1).
- Produces: ConfigMap `flux-system/site-settings` with the keys in the table; root substitution available to every Kustomization object in `kubernetes/clusters/homelab`.
- [ ] **Step 1: Write the failing tests:**
- `root_suspends_per_site`: `site-render.sh --root hcloud` shows `suspend: true` on the six Kustomizations listed above; `--root homelab` shows `suspend: false` on them and `otbr` unchanged (`true`).
- `restore_root_substitutes`: `kubernetes/restore-test/flux-root.yaml`'s `restore-test` Kustomization has `postBuild.substituteFrom` naming `site-settings` (yq query, exit 0).
- `no_dependents_of_suspended`: for each site, no Kustomization with `suspend: false` lists a suspended one in `dependsOn`.
- `homelab_root_unchanged`: `--root homelab` render equals the plain render at `origin/master` except `suspend: false` on the six objects.
- [ ] **Step 2: Run tests** — expected FAIL.
- [ ] **Step 3: Add the inline manifest and the suspend fields** listed under Files. Production values make `suspend: false` explicit; Flux treats it like the absent field.
- [ ] **Step 4: Run tests** — expected PASS; `site-render.sh --root homelab` exits 0 (no undefined variable anywhere in `kubernetes/clusters/homelab`).
- [ ] **Step 5: Commit** `feat(flux): site-settings from the Talos site patch; features a site lacks are suspended`.
- [ ] **Step 6: Production rollout before merge** (PR body): incremental Talos patch adding only the `site-settings` inline manifest, dry-run filtered, k8s-1 then k8s-2, k8s-3; verify `kubectl -n flux-system get cm site-settings -o yaml` equals the homelab column.
### Task 3: Base domain and ACME server per site
**Files (served names only; the spec's inventory lists them, about 120 files):**
- Modify: every Ingress/TLS host, cert-manager `dnsZones` and `dnsNames`, external-dns `domainFilters` and hostname annotations, k8s-gateway zone (`infrastructure/k8s-gateway/helmrelease.yaml:56,61`), CoreDNS zone (`infrastructure/coredns-config/configmap.yaml:28`), app URLs, authentik blueprint redirect URIs, `authentik_host_browser`, brand domain, LDAP names, monitoring probes, gatus, uptime-monitor — `renner.uno` → `${SITE_DOMAIN}`.
- Modify: `infrastructure/post-crds/clusterissuer.yaml` `${ACME_SERVER}` → `${SITE_ACME_SERVER}`; `infrastructure/cluster-settings/cluster-settings.yaml` drops `ACME_SERVER`; `clusters/homelab/infrastructure.yaml` `infrastructure-crds` adds `site-settings` to `substituteFrom`.
- Modify: `apps/gitea/statefulset.yaml` add `GITEA__server__ROOT_URL: https://gitea.${SITE_DOMAIN}`.
- Modify: `apps/authentik/gate-sync.yaml` add `COOKIE_DOMAIN: ${SITE_DOMAIN}`.
- Modify: each `kubernetes/clusters/homelab/*.yaml` Kustomization whose path now contains `${SITE_DOMAIN}` — add `postBuild.substituteFrom: [{kind: ConfigMap, name: site-settings}]`.
- Modify: objects in those paths with unrelated `${...}` (list in the spec's review and `git grep -l '\${[A-Za-z_]'`: home-assistant deployment/proxy.conf/config-sync.sh/zigbee2mqtt scripts, home-assistant-iot shelly proxy conf, otbr, monitoring dashboards and helmrelease-prometheus, multus cni-plugins-installer, openbao snapshot-cronjob, rook toolbox, image-automation lidarr, velero backup-verify) — annotate `kustomize.toolkit.fluxcd.io/substitute: disabled` where the object needs no site value, else escape as `$${...}`.
- Unchanged by rule: label/annotation keys (`dns.renner.uno/*`, `auth.renner.uno/*`, `net.renner.uno/*`, `cnpg.renner.uno/*`, …), image references, `gotk-sync.yaml` Git URL, email addresses, names of hosts outside the cluster (udm, nas-1/2, pve, jupiter/saturn/neptune, `netbird.renner.uno`, `slzb-*`, `*.iot.renner.uno`, `edge-1`), Matrix.
**Interfaces:**
- Consumes: `site-settings` keys `SITE_DOMAIN`, `SITE_ACME_SERVER` (Task 2); `site-render.sh` (Task 1).
- [ ] **Step 1: Write the failing test** `hcloud_has_no_production_names` in `test-site-render.sh`: render every substituting path with `hcloud`; every `renner.uno` occurrence must be `rtest.renner.uno` or match the allow-list regex file `scripts/ci/testdata/site-render/allowed-production-names.txt` (one regex per line, each with a comment naming why it stays).
- [ ] **Step 2: Run** — expected FAIL with the list of leftover names.
- [ ] **Step 3: Replace** per the Files list, in reviewable commits by area: infrastructure (DNS, cert-manager, ACME), authentik, apps, monitoring.
- [ ] **Step 4: Run** `test-site-render.sh` — PASS; `site-render.sh --changed homelab origin/master` — exits 0 except the two intended env additions, which it prints; confirm their values equal production's (`gitea.renner.uno`, `renner.uno`).
- [ ] **Step 5: Commit** per area, `feat(site): <area> reads the base domain from site-settings`.
### Task 4: Service CIDR per site
**Files:**
- Modify: `kubernetes/talos/patches/cluster.yaml:80-81` (remove `serviceSubnets`), `site-homelab.yaml`, `site-qemu.yaml` (`serviceSubnets: [10.96.0.0/12]`), `site-hcloud.yaml` (`[10.112.0.0/12]`, `ResolverConfig` nameserver `10.112.0.11` at :78)
- Modify: `infrastructure/k8s-gateway/helmrelease.yaml:104` `clusterIP: ${SITE_DNS_GATEWAY_IP}`; `infrastructure/coredns-config/configmap.yaml:35` `forward . ${SITE_DNS_GATEWAY_IP}`; `infrastructure-inhouse/netbird-service-sync/deployment.yaml:99` `${SITE_SERVICE_CIDR}`; their Kustomizations get `site-settings` in `substituteFrom` if Task 3 did not add it.
- Modify: `kubernetes/bootstrap.sh` k8s-gateway install (~1459): pass the site's `SITE_DNS_GATEWAY_IP` as the Service clusterIP (read from `site-settings` with the same yq expression as Task 1).
**Interfaces:**
- Consumes: `SITE_SERVICE_CIDR`, `SITE_DNS_GATEWAY_IP`.
- [ ] **Step 1: Failing tests:** `service_cidr_per_site` — `yq` over the merged Talos patches for each site yields the table's CIDR exactly once; `hcloud` render of k8s-gateway has `clusterIP: 10.112.0.11` and CoreDNS forwards there.
- [ ] **Step 2: Run** — FAIL.
- [ ] **Step 3: Implement** per Files.
- [ ] **Step 4: Run** — PASS; homelab render unchanged; `shellcheck -S warning kubernetes/bootstrap.sh`.
- [ ] **Step 5: Commit** `feat(site): service CIDR and DNS gateway address per site`.
- [ ] **Step 6: Production rollout:** none on the nodes — production's `serviceSubnets` value is unchanged, only moved between patch files; the live machine config already holds it.
### Task 5: NetBird identity per site
**Files:**
- Modify: `infrastructure-inhouse/netbird-service-sync/deployment.yaml` env `NETWORK_NAME: ${SITE_NETBIRD_NETWORK}`, `ROUTER_GROUP_NAME: ${SITE_NETBIRD_ROUTER_GROUP}`.
- Modify: `infrastructure/netbird-ingress/externalsecret.yaml` `key: ${SITE_NETBIRD_INGRESS_KEY}`; `clusters/homelab/infrastructure-services.yaml` `netbird-ingress` gets `site-settings` substitution.
- Create: `infrastructure/openbao/rotation/netbird-ingress-rtest.yaml` — a second CronJob like `netbird-ingress.yaml`, `BAO_PATH: homelab/infrastructure/netbird-ingress-rtest`, `NB_AUTO_GROUPS: <id of group k8s-ingress-router-rtest>`; add it to `infrastructure/openbao/rotation/kustomization.yaml`; extend the `rotator-netbird` OpenBao policy (`infrastructure/openbao/policies/`) to write that path.
**Interfaces:**
- Consumes: `SITE_NETBIRD_*` (Task 2).
- Produces: OpenBao key `homelab/infrastructure/netbird-ingress-rtest` holding a setup key whose only auto-group is `k8s-ingress-router-rtest`.
- [ ] **Step 1: Owner step (blocking):** the owner creates NetBird group `k8s-ingress-router-rtest` with no policies granting it access to production resources and gives its ID. Do not create it with credentials read from OpenBao.
- [ ] **Step 2: Failing tests:** hcloud render of netbird-service-sync has `NETWORK_NAME=k8s public ingress rtest`, `ROUTER_GROUP_NAME=k8s-ingress-router-rtest`; the netbird-ingress ExternalSecret extracts `infrastructure/netbird-ingress-rtest`; homelab render unchanged.
- [ ] **Step 3: Implement** per Files.
- [ ] **Step 4: Run tests** — PASS.
- [ ] **Step 5: Commit** `feat(site): NetBird network, router group and setup key per site`.
- [ ] **Step 6: Production rollout after merge:** create one Job from the new CronJob (`kubectl -n openbao create job --from=cronjob/netbird-secret-rotate-rtest …`), confirm the OpenBao key exists (metadata only, never the value), so the next raft snapshot carries it into the test.
### Task 6: Restore test with the new site (run 30, then run 31)
**Files:**
- Modify: `kubernetes/restore-test/flux-root.yaml` — run 31 only: remove patches 3f and 3g; renumber.
- [ ] **Step 0: Before run 30:** list every object carrying the label `external-dns/cloudflare=true` (the Cloudflare instance's `labelFilter`) and the names it would publish on hcloud. With `--default-targets` set to a `*.cfargotunnel.com` name and `--force-default-targets` (`apps/external-dns/helmrelease.yaml`), each such record would shadow the public wildcard with an unreachable target; if any exists, stop and bring it to the owner.
- [ ] **Step 1: Run 30** with patches 3f/3g still in place, hands-off, stall detector armed. Pass criteria: no undefined-variable errors; the six features show suspended; every other Kustomization Ready; all names on the test are `*.rtest.renner.uno`; certificates come from LE staging; production unaffected (production probes and `flux get ks` on production stay green); stale production URLs in restored data fail from the test (a pod on the test reaching `https://ci.renner.uno` and `https://auth.renner.uno` times out at the egress firewall). Record results in the wiki.
- [ ] **Step 2: Before run 31:** owner and builder read the live NetBird policies for group "All" and `k8s-ingress-router-rtest`; the test's peers must reach nothing of production. Check whether the NetBird proxy accepts services under `rtest.renner.uno` without a separately registered custom domain; if registration is needed, it is the owner's step.
- [ ] **Step 3: Remove patches 3f/3g**, commit `fix(restore-test): NetBird sync and routing peers run on the test site`, PR.
- [ ] **Step 4: Run 31** hands-off. Production checks every 5 min (NetBird service list for production hosts unchanged, `netbird-service-sync` log in production, public probes). Stop and tear down on the first production deviation. Pass criteria additionally: the owner opens `https://<app>.rtest.renner.uno` from the internet and logs in with the password, and an app behind the passkey step-up (`passkey-stepup.yaml`: e.g. Gitea) lets them register a new passkey on the test host after password login.
@@ -0,0 +1,231 @@
# Site-specific base domain and address ranges for the restore test
Date: 2026-10-18. Status: design, awaiting the owner's approval.
## Intent
The Hetzner restore test (`bootstrap.sh hcloudtest` + `FLUX_SOURCE=restore-test bootstrap.sh flux`)
boots production's configuration on throwaway machines and restores production's data into it.
Today it has to run with parts of production switched off (`kubernetes/restore-test/flux-root.yaml`
patches 3f and 3g, external-dns left unresolved), because it uses production's names and would
overwrite production's NetBird services, routing and DNS. In run 29 production ingress broke during
the test and the cause was never found.
The fix is to give the test cluster its own identity where, and only where, it collides with
production: a subdomain as its base domain and its own address ranges. Then the test can run with
everything production runs, including the internet path through the NetBird proxy, and the owner can
open the restored services from the internet to check that they actually work.
## Goals
- The base domain is a site fact. Production (`site-homelab`) keeps `renner.uno`. The restore-test
site (`site-hcloud`) uses a subdomain of `renner.uno` (working name `rtest.renner.uno`; the builder
confirms the final label with the owner only if it matters).
- The address range that collides is a site fact: the service CIDR, and every fixed address derived
from it. Pod CIDR and LoadBalancer pool are not routed into NetBird for the test site (see 3), so
they collide nowhere and stay identical.
- The NetBird identity of the cluster is per site, so production and test live side by side in the
same NetBird account.
- The test cluster is reachable from the internet through the same NetBird proxy as production, under
`*.<test domain>` names, and works like production there.
- With these in place, the restore-test patches that exist only because of name or range collisions
are removed (3f netbird-service-sync at zero replicas, 3g netbird-ingress on no node), and the
Cloudflare external-dns runs on the test site. external-dns-unifi does not, and neither do the
features the Hetzner site lacks: they are suspended there (see Decided).
## Non-goals
- Making production serve several domains.
- Matrix (Synapse, MAS, Element): out of scope (owner, 2026-10-18). Synapse refuses to start if
`server_name` changes (upstream `synapse/storage/databases/main/__init__.py`), so it stays
`matrix.renner.uno`; Matrix may be broken or unreachable on the test site.
- Passkeys on the test site: authentik's WebAuthn RP ID is the request host
(`authentik/stages/authenticator_webauthn/utils.py:6-11`, version 2026.8.3, not configurable), so
passkeys registered on `auth.renner.uno` do not work on the test host. The owner logs in with
the password; passkeys are not a design criterion (owner, 2026-10-18).
- Rewriting production URLs inside restored data (see Constraints, "Stale production URLs").
## Anti-goals (a failure even if it technically works)
- Any difference between the sites that is not needed. "Only things that must differ should differ"
(owner). Each per-site value must be justified by a collision.
- A harness or script that changes the test nodes or network to make the test pass. The test must
boot production's configuration; per-site facts live in the site and node files
(`site-homelab` describes the local cluster, `site-hcloud` the Hetzner cluster, node specifics in
their own files — owner's rule).
- The test cluster writing anything to production: production's NetBird services, routes, peers or
groups; production's DNS names; production's Woodpecker, Gitea or other services.
- The cluster depending on external DNS for its own services (owner's design rule).
- Any change in production's behaviour. For production, the change is a pure move of values into
the site files; rendered manifests stay identical.
## What has to differ per site, and why
### 1. Base domain
Collision: the test's Ingress hosts, NetBird proxy services (keyed by domain) and DNS records would be
production's.
Where the domain is written (inventory at origin/master, 2026-10-18, about 160 files contain
`renner.uno`; only the served names below change per site):
- Ingress and TLS hosts: about 34 files under `kubernetes/apps` and `kubernetes/infrastructure`
(e.g. `apps/gitea/ingress.yaml:19,29`), including nested names `*.home.renner.uno`.
- cert-manager: `infrastructure/post-crds/clusterissuer.yaml` (`dnsZones: renner.uno` at 19, 39, 66;
the ACME email at 8, 28, 55 is an address, not a served name); Certificates with `dnsNames`
(`apps/authentik/ldap-outpost-cert.yaml:28`, `apps/home-assistant/mosquitto-certificate.yaml:12`,
`apps/monitoring/helmrelease-alloy-syslog.yaml:12`).
- DNS: both external-dns instances (`apps/external-dns/helmrelease.yaml:51` Cloudflare;
`infrastructure/external-dns-unifi/helmrelease.yaml:84`), the `external-dns.kubernetes.io/hostname`
annotations (6 files), k8s-gateway zone (`infrastructure/k8s-gateway/helmrelease.yaml:56,61`),
CoreDNS zone (`infrastructure/coredns-config/configmap.yaml:28`).
- Application config with URLs: Gitea, Woodpecker, Immich, Paperless, WordPress, TRMNL, gitea-mcp,
review-bot, wiki, renovate, Grafana and the monitoring probes, gatus, uptime-monitor.
- authentik blueprints (`apps/authentik/blueprints-configmap.yaml`): OAuth redirect URIs (~813-951),
LDAP name and `tls_server_name`, `authentik_host_browser`, brand domain (1170).
- authentik gate-sync: `AUTHENTIK_HOST_BROWSER` (`apps/authentik/gate-sync.yaml:127`) and
`COOKIE_DOMAIN`, which is unset and defaults to `renner.uno` in the controller (`controller.py:57`);
it must be set explicitly.
- Gitea `ROOT_URL` lives in `app.ini` on the PVC (restored data), not in git; git must set it
(`GITEA__server__ROOT_URL`) so the test serves its own URL.
Stays the same on every site (not served names): label and annotation keys such as
`dns.renner.uno/public`, `auth.renner.uno/*`, `net.renner.uno/*`, `cnpg.renner.uno/*`; image
references and the registry mirror (`gitea.renner.uno` as a pull source); Flux's Git source;
email addresses; comments and docs. Names of things outside the cluster (UDM, NAS, PVE hosts,
IoT devices, `netbird.renner.uno`) stay too.
### 2. Service CIDR
Collision: the NetBird proxy reaches a cluster through a route to its service CIDR and connects to
Traefik's ClusterIP inside it (`ci/images/netbird-service-sync/controller.py`, `SERVICE_CIDR`,
`service_body()`). Two clusters with the same CIDR give the proxy two routes to the same range; it
cannot tell the clusters apart.
Currently shared, to become site facts:
- `serviceSubnets: 10.96.0.0/12` in `kubernetes/talos/patches/cluster.yaml:80-81`.
- Fixed service addresses: k8s-gateway `10.96.0.11` (`infrastructure/k8s-gateway/helmrelease.yaml:104`),
CoreDNS forward to it (`infrastructure/coredns-config/configmap.yaml:35`), the Talos
`ResolverConfig` in the site patches, kube-dns `10.96.0.10`.
- `netbird-service-sync` `SERVICE_CIDR` env in its Deployment.
Stay identical (no collision, because the test site routes nothing else into NetBird): pod CIDR
`10.244.0.0/16`, LoadBalancer pool `10.2.99.0/24` / `fd99::/64`
(`infrastructure/post-crds/cilium-bgp.yaml:22-25`) and the VIPs in it (Traefik `10.2.99.86`, Home
Assistant `10.2.99.87`). The builder confirms nothing else routes these between the sites; if
something does, that is a collision and the range becomes a site fact too.
Production keeps its current value. Its service CIDR cannot change on a running cluster anyway.
### 3. NetBird identity
Collision (verified in `ci/images/netbird-service-sync/controller.py`): the controller finds its
network by name (`NETWORK_NAME`, default `k8s public ingress`, line 44) and its resource by the
service CIDR; it treats every TLS service targeting that resource as its own and deletes those no
local Ingress carries (`is_managed` line 212, delete line 257). A second cluster with the same name
would delete production's services. `netbird-ingress` peers enrol with a setup key whose auto-group
is `k8s-ingress-router`, the router group of production's network; test peers would become routers
for production's traffic.
Per site: `NETWORK_NAME`, `ROUTER_GROUP_NAME` (and the policy name derived from the network name), the
`netbird-ingress` setup key and its auto-group.
Decided (owner, 2026-10-18): the test site gets its own setup key that only allows access to where it
is needed. The secret rotator creates it (production's key is minted by
`infrastructure/openbao/rotation/netbird-ingress.yaml`, `NB_AUTO_GROUPS`). The test's peers get no
production group, no production policy, no mesh routes and no mesh DNS: only what the proxy path to
the test's own services needs. Production's routes and nameserver groups for `k8s-ingress-router` are
set up by hand in NetBird, not in the repo; the test's peers must not be in that group. Peers join
NetBird's "All" group automatically; the builder checks what "All" grants in production and makes
sure the test's peers gain no access to production through it.
## Constraints
- Production config is the spec. Per-site values are expressed in the site files (`site-homelab.yaml`,
`site-hcloud.yaml`, `site-qemu.yaml`), the way `cilium-site` and `k8s-gateway-site` already are
(inline ConfigMaps read via `valuesFrom`, Talos-created). The builder chooses the mechanism by which
Flux-applied manifests read the site's domain and ranges (e.g. postBuild substitution from a
site ConfigMap). Known trap: postBuild substitution blanks every undefined `${var}`, and some app
manifests contain unrelated `${...}` syntax (`infrastructure/cluster-settings/cluster-settings.yaml:6-9`).
- For production the change is a no-op: rendered manifests and live objects stay identical. Prove it
(e.g. render before/after) per PR.
- Changes reach production as reviewable, stacked PRs. Moving objects between pruned Kustomizations
needs the prune handover used in #1307/#1317. Never force-push.
- The test may create its own entries in production's NetBird account and Cloudflare zone, limited to
its own names (owner, 2026-10-18). It must not modify or delete production's.
- Egress containment stays: the test's Hetzner firewall blocks TCP to production's public edge
(`proxy.renner.uno`, `bootstrap.sh` `hcloud_apply_egress`). NetBird management, signal and relay run
on `netbird.renner.uno`, a different address, and stay reachable. Inbound traffic arrives through the
NetBird tunnel. The builder must check that the test's routing peer works with TCP to the proxy
hosts blocked (the proxy hosts' extra relays on TCP 33080 are then unreachable; UDP passes).
- Stale production URLs in restored data stay as they are: Gitea's Woodpecker webhooks
(`https://ci.renner.uno/api/hook?access_token=…`, valid tokens), Home Assistant's `.storage`
(`external_url`, integrations), Immich's OIDC issuer in OpenBao `apps/immich`, WordPress post
content. They must not reach production: the test resolves its own names in-cluster
(k8s-gateway), and production names resolve publicly to the proxy, which the egress firewall
blocks. The builder verifies this holds for each of these.
- Let's Encrypt: the test site uses Let's Encrypt staging (owner, 2026-10-18: "staging certs are
fine"), so test runs never count against the `renner.uno` limits production's renewals share (about
40 Certificates per run). The ACME server therefore becomes a site fact; today `ACME_SERVER` is in
the Flux-applied `infrastructure/cluster-settings/cluster-settings.yaml`, substituted into the
ClusterIssuer. The owner accepts browser warnings when inspecting the test.
- To verify, not assumed: what the Cloudflare external-dns instance would write for the test names.
It runs `policy: upsert-only` with `--default-targets` set to a `*.cfargotunnel.com` name and
`--force-default-targets` (`apps/external-dns/helmrelease.yaml`); the public wildcard `*.renner.uno`
already resolves every test name to the proxy. A record it creates must not shadow the wildcard
with an unreachable target.
- Passkey step-up: some apps require a passkey step-up after login (blueprint `passkey-stepup.yaml`:
Home Assistant, NetBird, Audiobookshelf, Immich, Gitea, Grafana). On the test site the owner would
register a new passkey after password login; the builder checks the flow allows that.
## Decided (owner)
- Subdomain of `renner.uno` as the test base domain.
- Reachable from the internet through the NetBird proxy, like production; 1:1 with production
otherwise.
- The test may create its own NetBird and Cloudflare entries, limited to its names.
- Address ranges are site facts in the site files.
- Only what must differ, differs.
- Matrix out of scope; passkeys not a criterion, password login on the test.
- The test's NetBird peers use their own rotator-minted setup key with access only where needed.
- Features a site does not have are not applied there at all: their Kustomizations are suspended on
that site, so Flux does not try them and `flux get ks` shows them as suspended instead of failing
at a timeout ("noise", owner). The site file declares what the site has, like its other site facts:
the Intel iGPU, the IoT VLAN, and the UniFi gateway. Talos creates these as a ConfigMap (the way
`cilium-site` works), and Flux's root sync reads it to set `spec.suspend` on the affected
Kustomizations. Affected today: llama-server and wiki (iGPU), home-assistant-iot, matter-server and
music-assistant (IoT VLAN), external-dns-unifi (UniFi gateway; it does not run on the test
cluster). otbr is suspended on every site already. The builder confirms the list against the
current node-label gating (`hardware.renner.uno/intel-gpu`, `net.renner.uno/iot-vlan`, #1313); the
node labels stay, they say which nodes of a site carry the hardware.
- Uptime monitoring does not run on the test cluster (owner, 2026-10-20): the kuma-sync
Kustomization follows `SITE_SUSPEND_UPTIME_MONITORING` like the other site exceptions. Its
KumaMonitor objects stay with their apps and keep production URLs: they are monitors of the
production Uptime Kuma, inert without the controller.
- Let's Encrypt staging on the test site.
## Calls made in the design (not the owner's; check these)
- Only the service CIDR becomes a site range; pod CIDR and LoadBalancer pool stay identical, because
with the restricted setup key nothing routes them between the sites.
- The run-29 production ingress break was never explained. Run 30 turns on the parts that touch
production's public path, so the plan enables them in stages and checks production after each:
first `netbird-service-sync` for one pass (`ONCE`) against the test's own network, then the
routing peers, and stops on the first sign of production impact.
- The builder checks whether the NetBird proxy accepts services under the test subdomain without a
separately registered custom domain (production services are custom domains `<app>.renner.uno`,
wiki `[[netbird-reverse-proxy-custom-domains]]`). If a one-time registration is needed, it is the
owner's step to approve.
## Left to the builder
- The exact test domain label and the test's service CIDR (must not overlap production's, the LAN, or
NetBird's ranges).
- The substitution / site-value mechanism and how to keep it safe against unrelated `${...}`,
including on the root sync that sets `spec.suspend` (production must render unchanged: every
feature present, nothing suspended that is not suspended today).
- How the test's NetBird setup key and router group are created, within the constraints.
- How the Let's Encrypt limit is respected.
- Order and split of PRs, and the restore-test run that proves it (run 30), including inspecting the
restored services through the internet.
+2 -2
View File
@@ -12,7 +12,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: audiobooks.renner.uno
- host: audiobooks.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -25,4 +25,4 @@ spec:
tls:
- secretName: audiobookshelf-tls
hosts:
- audiobooks.renner.uno
- audiobooks.${SITE_DOMAIN}
@@ -406,7 +406,7 @@ data:
- id: ldap-outpost-cert
model: authentik_crypto.certificatekeypair
identifiers:
name: ldap.renner.uno
name: ldap.${SITE_DOMAIN}
state: present
attrs:
certificate_data: !Env LDAP_OUTPOST_TLS_CRT
@@ -427,7 +427,7 @@ data:
state: present
attrs:
certificate: !KeyOf ldap-outpost-cert
tls_server_name: ldap.renner.uno
tls_server_name: ldap.${SITE_DOMAIN}
# Synology DSM (nas-1) LDAP integration. DSM's Directory Client
# binds to the LDAP Outpost via the standalone LoadBalancer Service
@@ -522,7 +522,7 @@ data:
# one cert per handshake). DSM still binds on plaintext 389
# today; this just makes 636 ready when DSM cuts over.
certificate: !KeyOf ldap-outpost-cert
tls_server_name: ldap.renner.uno
tls_server_name: ldap.${SITE_DOMAIN}
- id: synology-application
model: authentik_core.application
@@ -608,7 +608,7 @@ data:
gid_start_number: 5000
mfa_support: false
certificate: !KeyOf ldap-outpost-cert
tls_server_name: ldap.renner.uno
tls_server_name: ldap.${SITE_DOMAIN}
- id: unifi-application
model: authentik_core.application
@@ -703,7 +703,7 @@ data:
log_level: info
authentik_host: "http://authentik-server.authentik.svc.cluster.local"
authentik_host_insecure: false
authentik_host_browser: "https://auth.renner.uno/"
authentik_host_browser: "https://auth.${SITE_DOMAIN}/"
refresh_interval: "minutes=5"
# Patched LDAP outpost: adds "sambaSamAccount" AND
# "sambaDomain" to the hardcoded GetUserOCs() set so
@@ -737,7 +737,7 @@ data:
log_level: info
authentik_host: "http://authentik-server.authentik.svc.cluster.local"
authentik_host_insecure: false
authentik_host_browser: "https://auth.renner.uno/"
authentik_host_browser: "https://auth.${SITE_DOMAIN}/"
refresh_interval: "minutes=5"
container_image: null
object_naming_template: "ak-outpost-%(name)s"
@@ -820,7 +820,7 @@ data:
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
client_type: confidential
redirect_uris:
- url: https://home.renner.uno/auth/openid/callback
- url: https://home.${SITE_DOMAIN}/auth/openid/callback
matching_mode: strict
- url: http://homeassistant.srv.renner.uno:8123/auth/openid/callback
matching_mode: strict
@@ -858,9 +858,9 @@ data:
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
client_type: confidential
redirect_uris:
- url: https://audiobooks.renner.uno/auth/openid/callback
- url: https://audiobooks.${SITE_DOMAIN}/auth/openid/callback
matching_mode: strict
- url: https://audiobooks.renner.uno/auth/openid/mobile-redirect
- url: https://audiobooks.${SITE_DOMAIN}/auth/openid/mobile-redirect
matching_mode: strict
- model: authentik_providers_oauth2.oauth2provider
@@ -875,9 +875,9 @@ data:
redirect_uris:
- url: app.immich:///oauth-callback
matching_mode: strict
- url: https://immich.renner.uno/auth/login
- url: https://immich.${SITE_DOMAIN}/auth/login
matching_mode: strict
- url: https://immich.renner.uno/user-settings
- url: https://immich.${SITE_DOMAIN}/user-settings
matching_mode: strict
- model: authentik_providers_oauth2.oauth2provider
@@ -890,7 +890,7 @@ data:
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
client_type: confidential
redirect_uris:
- url: https://gitea.renner.uno/user/oauth2/authentik/callback
- url: https://gitea.${SITE_DOMAIN}/user/oauth2/authentik/callback
matching_mode: strict
- model: authentik_providers_oauth2.oauth2provider
@@ -903,7 +903,7 @@ data:
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
client_type: confidential
redirect_uris:
- url: https://dash.renner.uno/login/generic_oauth
- url: https://dash.${SITE_DOMAIN}/login/generic_oauth
matching_mode: strict
# Matrix (REN-567): upstream login for MAS (account.matrix.renner.uno).
@@ -1013,7 +1013,7 @@ data:
- !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-email]]
- !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-offline_access]]
redirect_uris:
- url: https://hermes.renner.uno/auth/callback
- url: https://hermes.${SITE_DOMAIN}/auth/callback
matching_mode: strict
# The slug is the issuer path: https://auth.renner.uno/application/o/hermes/
@@ -1025,7 +1025,7 @@ data:
name: Hermes
slug: hermes
provider: !KeyOf hermes-provider
meta_launch_url: https://hermes.renner.uno
meta_launch_url: https://hermes.${SITE_DOMAIN}
meta_description: "Hermes Agent dashboard"
# Only the Hermes Users group passes.
@@ -1063,7 +1063,7 @@ data:
- !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-openid]]
- !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-profile]]
redirect_uris:
- url: https://wiki.renner.uno/signin-oidc
- url: https://wiki.${SITE_DOMAIN}/signin-oidc
matching_mode: strict
# The slug is the issuer path: https://auth.renner.uno/application/o/wiki/
@@ -1075,7 +1075,7 @@ data:
name: Wiki
slug: wiki
provider: !KeyOf wiki-provider
meta_launch_url: https://wiki.renner.uno
meta_launch_url: https://wiki.${SITE_DOMAIN}
meta_description: "Homelab wiki"
# OpenProject, decommissioned. Application first: Application.provider
+3 -1
View File
@@ -124,7 +124,9 @@ spec:
- name: AK_URL
value: http://authentik-server.authentik.svc.cluster.local
- name: AUTHENTIK_HOST_BROWSER
value: https://auth.renner.uno/
value: https://auth.${SITE_DOMAIN}/
- name: COOKIE_DOMAIN
value: ${SITE_DOMAIN}
- name: SERVICE_CONNECTION
value: Local Kubernetes Cluster
- name: DEFAULT_AUTHORIZATION_FLOW
+2 -2
View File
@@ -167,11 +167,11 @@ spec:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
hosts:
- auth.renner.uno
- auth.${SITE_DOMAIN}
tls:
- secretName: authentik-tls
hosts:
- auth.renner.uno
- auth.${SITE_DOMAIN}
worker:
resources:
@@ -25,4 +25,4 @@ spec:
name: letsencrypt-cluster
kind: ClusterIssuer
dnsNames:
- ldap.renner.uno
- ldap.${SITE_DOMAIN}
@@ -40,7 +40,7 @@ metadata:
# so this Service produces no public record — LDAP stays
# LAN-internal, which matches the threat model of the cleartext
# bind on 389.
external-dns.kubernetes.io/hostname: ldap.renner.uno
external-dns.kubernetes.io/hostname: ldap.${SITE_DOMAIN}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+2 -2
View File
@@ -9,10 +9,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- evcc.renner.uno
- evcc.${SITE_DOMAIN}
secretName: evcc-tls
rules:
- host: evcc.renner.uno
- host: evcc.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -48,7 +48,7 @@ spec:
- --default-targets=d2386ccd-a665-41e7-8631-603fdafcb006.cfargotunnel.com
- --force-default-targets
domainFilters:
- renner.uno
- ${SITE_DOMAIN}
sources:
- ingress
txtOwnerId: k8s-homelab
+10 -10
View File
@@ -79,7 +79,7 @@ data:
# Web Services
- name: Web Service 1
group: services
url: https://auth.renner.uno/-/health/ready/
url: https://auth.${SITE_DOMAIN}/-/health/ready/
interval: 60s
ui:
hide-hostname: true
@@ -93,7 +93,7 @@ data:
- name: Web Service 2
group: services
url: https://gitea.renner.uno/api/v1/settings/api
url: https://gitea.${SITE_DOMAIN}/api/v1/settings/api
interval: 60s
ui:
hide-hostname: true
@@ -107,7 +107,7 @@ data:
- name: Web Service 3
group: services
url: https://immich.renner.uno/api/server/ping
url: https://immich.${SITE_DOMAIN}/api/server/ping
interval: 60s
ui:
hide-hostname: true
@@ -121,7 +121,7 @@ data:
- name: Web Service 4
group: services
url: https://ci.renner.uno/healthz
url: https://ci.${SITE_DOMAIN}/healthz
interval: 60s
ui:
hide-hostname: true
@@ -134,7 +134,7 @@ data:
- name: Web Service 5
group: services
url: https://dash.renner.uno/api/health
url: https://dash.${SITE_DOMAIN}/api/health
interval: 60s
ui:
hide-hostname: true
@@ -148,7 +148,7 @@ data:
- name: Web Service 6
group: services
url: https://home.renner.uno
url: https://home.${SITE_DOMAIN}
interval: 60s
ui:
hide-hostname: true
@@ -166,7 +166,7 @@ data:
- name: Web Service 8
group: services
url: https://audiobooks.renner.uno/healthcheck
url: https://audiobooks.${SITE_DOMAIN}/healthcheck
interval: 60s
ui:
hide-hostname: true
@@ -179,7 +179,7 @@ data:
- name: Web Service 9
group: services
url: https://music.renner.uno
url: https://music.${SITE_DOMAIN}
interval: 60s
ui:
hide-hostname: true
@@ -197,7 +197,7 @@ data:
# unauthenticated request may redirect.
- name: Web Service 10
group: services
url: https://trmnl.renner.uno
url: https://trmnl.${SITE_DOMAIN}
interval: 60s
ui:
hide-hostname: true
@@ -220,7 +220,7 @@ data:
# only because their targets share a namespace with the prober.
- name: Web Service 11
group: services
url: https://paperless.renner.uno
url: https://paperless.${SITE_DOMAIN}
interval: 60s
ui:
hide-hostname: true
+2 -2
View File
@@ -13,10 +13,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- status.renner.uno
- status.${SITE_DOMAIN}
secretName: gatus-tls
rules:
- host: status.renner.uno
- host: status.${SITE_DOMAIN}
http:
paths:
- path: /
+2 -2
View File
@@ -35,10 +35,10 @@ spec:
# Only for the 401 on tokenless requests; clients send their own Gitea PAT.
- -oauth
- -oauth-public-url
- https://gitea-mcp.renner.uno
- https://gitea-mcp.${SITE_DOMAIN}
# Also advertised to clients as the OAuth authorization server, so it must be the public URL.
- -H
- https://gitea.renner.uno
- https://gitea.${SITE_DOMAIN}
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
+2 -2
View File
@@ -12,7 +12,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: gitea-mcp.renner.uno
- host: gitea-mcp.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -25,4 +25,4 @@ spec:
tls:
- secretName: gitea-mcp-tls
hosts:
- gitea-mcp.renner.uno
- gitea-mcp.${SITE_DOMAIN}
+4 -4
View File
@@ -16,7 +16,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: git.renner.uno
- host: git.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -26,7 +26,7 @@ spec:
name: gitea
port:
number: 3000
- host: gitea.renner.uno
- host: gitea.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -39,5 +39,5 @@ spec:
tls:
- secretName: gitea-tls
hosts:
- git.renner.uno
- gitea.renner.uno
- git.${SITE_DOMAIN}
- gitea.${SITE_DOMAIN}
+4 -2
View File
@@ -63,7 +63,9 @@ spec:
- name: GITEA__server__SSH_PORT
value: "22"
- name: GITEA__server__SSH_DOMAIN
value: gitea.renner.uno
value: gitea.${SITE_DOMAIN}
- name: GITEA__server__ROOT_URL
value: https://gitea.${SITE_DOMAIN}
# The LXC ran RUN_USER=gitea, so every existing remote/clone URL is
# gitea@gitea.renner.uno. The rootless image runs as `git`, so the
# built-in SSH server would only accept `git@` and reject `gitea@`.
@@ -117,7 +119,7 @@ spec:
value: "true"
# Webhook targets Gitea may call; every configured webhook host.
- name: GITEA__security__ALLOWED_HOST_LIST
value: "ci.renner.uno,wiki.ai.svc.cluster.local,review-bot.review-bot.svc.cluster.local"
value: "ci.${SITE_DOMAIN},wiki.ai.svc.cluster.local,review-bot.review-bot.svc.cluster.local"
ports:
- name: http
containerPort: 3000
+12 -6
View File
@@ -191,7 +191,7 @@ spec:
- config
- set
- mcp_servers.homeassistant
- '{"url":"https://home.renner.uno/api/mcp","headers":{"Authorization":"Bearer ${HA_TOKEN}"},"trust":"untrusted"}'
- '{"url":"https://home.${SITE_DOMAIN}/api/mcp","headers":{"Authorization":"Bearer $${HA_TOKEN}"},"trust":"untrusted"}'
env:
- name: HERMES_HOME
value: /opt/data
@@ -219,7 +219,7 @@ spec:
- -c
- |
H=/opt/hermes/.venv/bin/hermes
# Mirror default's mcp_servers.homeassistant (config-mcp-homeassistant); ${HA_TOKEN} stays literal.
# Mirror default's mcp_servers.homeassistant (config-mcp-homeassistant); $${HA_TOKEN} stays literal.
HA_MCP=$(PYTHONPATH=/opt/hermes /opt/hermes/.venv/bin/python -c '
import json
import hermes_yaml as yaml
@@ -359,7 +359,7 @@ spec:
{
echo 'logins:'
echo ' - name: hermes-bot'
echo ' url: https://gitea.renner.uno'
echo ' url: https://gitea.${SITE_DOMAIN}'
echo " token: $(cat /var/run/secrets/gitea/api-token)"
echo ' default: true'
echo ' user: hermes-bot'
@@ -385,11 +385,11 @@ spec:
- name: HERMES_DASHBOARD
value: "1"
- name: HERMES_DASHBOARD_PUBLIC_URL
value: https://hermes.renner.uno
value: https://hermes.${SITE_DOMAIN}
# Dashboard login: Authentik OIDC, public PKCE client (blueprint
# in apps/authentik/blueprints-configmap.yaml, group Hermes Users).
- name: HERMES_DASHBOARD_OIDC_ISSUER
value: https://auth.renner.uno/application/o/hermes/
value: https://auth.${SITE_DOMAIN}/application/o/hermes/
- name: HERMES_DASHBOARD_OIDC_CLIENT_ID
value: hermes-dashboard
# offline_access: Authentik issues a refresh token only when requested.
@@ -399,6 +399,12 @@ spec:
value: "true"
- name: API_SERVER_HOST
value: 0.0.0.0
# Home Assistant for the ha-updates skill (scripts/ha.py).
- name: HA_URL
value: https://home.${SITE_DOMAIN}
# Gitea issues API for the alert-monitor skill (scripts/alerts.py).
- name: GITEA_API
value: https://gitea.${SITE_DOMAIN}/api/v1/repos/konairius/homelab
# bao: log in with the pod's ServiceAccount token, role hermes
# (kubernetes/infrastructure/openbao/policies/hermes.hcl).
- name: BAO_ADDR
@@ -411,7 +417,7 @@ spec:
- name: GIT_CONFIG_COUNT
value: "3"
- name: GIT_CONFIG_KEY_0
value: credential.https://gitea.renner.uno.helper
value: credential.https://gitea.${SITE_DOMAIN}.helper
- name: GIT_CONFIG_VALUE_0
value: '!f() { test "$1" = get && echo username=hermes-bot && echo "password=$(cat /var/run/secrets/gitea/api-token)"; }; f'
- name: GIT_CONFIG_KEY_1
+4 -4
View File
@@ -17,10 +17,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- hermes.renner.uno
- hermes.${SITE_DOMAIN}
secretName: hermes-tls
rules:
- host: hermes.renner.uno
- host: hermes.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -44,10 +44,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- hermes-api.renner.uno
- hermes-api.${SITE_DOMAIN}
secretName: hermes-api-tls
rules:
- host: hermes-api.renner.uno
- host: hermes-api.${SITE_DOMAIN}
http:
paths:
- path: /
+1 -1
View File
@@ -16,7 +16,7 @@ configMapGenerator:
- name: hermes-matrix-config
namespace: hermes
literals:
- MATRIX_HOMESERVER=https://matrix.renner.uno
- MATRIX_HOMESERVER=https://matrix.${SITE_DOMAIN}
- MATRIX_USER_ID=@hermes:matrix.renner.uno
- MATRIX_ALLOWED_USERS=@k.renner:matrix.renner.uno
- MATRIX_E2EE_MODE=required
@@ -31,5 +31,8 @@ configMapGenerator:
namespace: home-assistant
files:
- api.conf.template=shelly-manager-api-proxy.conf
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
generatorOptions:
disableNameSuffixHash: true
@@ -18,7 +18,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: shelly-manager.home.renner.uno
- host: shelly-manager.home.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -31,4 +31,4 @@ spec:
tls:
- secretName: shelly-manager-tls
hosts:
- shelly-manager.home.renner.uno
- shelly-manager.home.${SITE_DOMAIN}
@@ -18,7 +18,7 @@ metadata:
lb-pool: bgp
annotations:
lbipam.cilium.io/ips: "10.2.99.87,fd99::87"
external-dns.kubernetes.io/hostname: ha.renner.uno
external-dns.kubernetes.io/hostname: ha.${SITE_DOMAIN}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
@@ -352,7 +352,7 @@ spec:
name: home-assistant-config-sync-token
key: token
- name: GITEA_URL
value: https://gitea.renner.uno
value: https://gitea.${SITE_DOMAIN}
- name: GITEA_REPO
value: konairius/homeassistant
- name: GITEA_USER
@@ -18,7 +18,7 @@ spec:
# proxy.
ingressClassName: authentik-gate
rules:
- host: esphome.home.renner.uno
- host: esphome.home.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -31,4 +31,4 @@ spec:
tls:
- secretName: esphome-tls
hosts:
- esphome.home.renner.uno
- esphome.home.${SITE_DOMAIN}
+2 -2
View File
@@ -12,7 +12,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: home.renner.uno
- host: home.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -25,4 +25,4 @@ spec:
tls:
- secretName: home-assistant-tls
hosts:
- home.renner.uno
- home.${SITE_DOMAIN}
@@ -67,6 +67,9 @@ configMapGenerator:
namespace: home-assistant
files:
- default.conf.template=proxy.conf
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
# config-sync sidecar script. Hash suffix disabled (name stays stable), so
# editing config-sync.sh does not auto-roll the pod — restart manually with
# `kubectl rollout restart deploy/home-assistant -n home-assistant`.
@@ -74,6 +77,9 @@ configMapGenerator:
namespace: home-assistant
files:
- sync.sh=config-sync.sh
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
# Z2M re-interview watchdog sidecar script. Hash suffix disabled (name stays
# stable), so editing zigbee2mqtt-interview-watch.sh does not auto-roll the
# pod — restart manually with
@@ -82,6 +88,9 @@ configMapGenerator:
namespace: home-assistant
files:
- watch.sh=zigbee2mqtt-interview-watch.sh
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
# Z2M discovery guard sidecar script (gitea issue #381). Hash suffix disabled
# (name stays stable), so editing zigbee2mqtt-discovery-guard.sh does not
# auto-roll the pod — restart manually with
@@ -90,6 +99,9 @@ configMapGenerator:
namespace: home-assistant
files:
- guard.sh=zigbee2mqtt-discovery-guard.sh
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
# Z2M external converters, copied into /app/data/external_converters by the
# seed-data init container on every start. Hash suffix disabled (name stays
# stable), so editing a converter does not auto-roll the pod — restart
@@ -9,7 +9,7 @@ spec:
name: letsencrypt-cluster
kind: ClusterIssuer
dnsNames:
- mqtt.renner.uno
- mqtt.${SITE_DOMAIN}
---
# Server certificate of the mutual-TLS listener :8884 (REN-802). Issued by
# mqtt-ca because certificate clients and mqtt-account-controller trust only
@@ -19,7 +19,7 @@ metadata:
# LAN can connect with `mqtt.renner.uno` (matches the broker cert
# SAN) instead of the bare IP. No Cloudflare label — kept off the
# public DNS; UniFi serves a split-DNS answer for LAN clients.
external-dns.kubernetes.io/hostname: mqtt.renner.uno
external-dns.kubernetes.io/hostname: mqtt.${SITE_DOMAIN}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
@@ -19,7 +19,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: vscode.home.renner.uno
- host: vscode.home.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -32,4 +32,4 @@ spec:
tls:
- secretName: home-assistant-vscode-tls
hosts:
- vscode.home.renner.uno
- vscode.home.${SITE_DOMAIN}
@@ -3,6 +3,8 @@ kind: Deployment
metadata:
name: zigbee2mqtt
namespace: home-assistant
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
labels:
app.kubernetes.io/name: zigbee2mqtt
spec:
@@ -16,7 +16,7 @@ spec:
# Traefik's websecure entrypoint passes it through.
ingressClassName: authentik-gate
rules:
- host: zigbee2mqtt.home.renner.uno
- host: zigbee2mqtt.home.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -29,4 +29,4 @@ spec:
tls:
- secretName: zigbee2mqtt-tls
hosts:
- zigbee2mqtt.home.renner.uno
- zigbee2mqtt.home.${SITE_DOMAIN}
+2 -2
View File
@@ -109,7 +109,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: ical-proxy.renner.uno
- host: ical-proxy.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -122,4 +122,4 @@ spec:
tls:
- secretName: ical-proxy-tls
hosts:
- ical-proxy.renner.uno
- ical-proxy.${SITE_DOMAIN}
+4 -4
View File
@@ -12,7 +12,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: immich.renner.uno
- host: immich.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -25,7 +25,7 @@ spec:
tls:
- secretName: immich-tls
hosts:
- immich.renner.uno
- immich.${SITE_DOMAIN}
---
# photos.renner.uno is LAN-registered (no proxy tunnel), so it lives on its
# own unlabeled Ingress: the dns.renner.uno/public label-filter is
@@ -42,7 +42,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: photos.renner.uno
- host: photos.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -55,4 +55,4 @@ spec:
tls:
- secretName: immich-photos-tls
hosts:
- photos.renner.uno
- photos.${SITE_DOMAIN}
+1 -1
View File
@@ -13,7 +13,7 @@ data:
IMMICH_MEDIA_LOCATION: "/data"
UPLOAD_LOCATION: "/data/uploads"
IMMICH_CONFIG_FILE: "/config/immich-config.json"
IMMICH_PUBLIC_URL: "https://immich.renner.uno"
IMMICH_PUBLIC_URL: "https://immich.${SITE_DOMAIN}"
IMMICH_MACHINE_LEARNING_URL: "http://immich-machine-learning:3003"
IMMICH_METRICS: "true"
IMMICH_TELEMETRY_INCLUDE: "all"
+2 -2
View File
@@ -28,10 +28,10 @@ spec:
ingressClassName: authentik-gate
tls:
- hosts:
- llm.renner.uno
- llm.${SITE_DOMAIN}
secretName: llama-server-tls
rules:
- host: llm.renner.uno
- host: llm.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -42,10 +42,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- wiki-embed.renner.uno
- wiki-embed.${SITE_DOMAIN}
secretName: wiki-embed-tls
rules:
- host: wiki-embed.renner.uno
- host: wiki-embed.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -17,6 +17,8 @@ kind: ServiceMonitor
metadata:
name: cilium-agent
namespace: kube-system
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
labels:
app.kubernetes.io/name: cilium-agent
app.kubernetes.io/part-of: cilium
@@ -21,10 +21,10 @@ spec:
targets:
staticConfig:
static:
- https://gitea.renner.uno
- https://immich.renner.uno
- https://auth.renner.uno
- https://home.renner.uno
- https://gitea.${SITE_DOMAIN}
- https://immich.${SITE_DOMAIN}
- https://auth.${SITE_DOMAIN}
- https://home.${SITE_DOMAIN}
# prometheus.renner.uno and alerts.renner.uno are Authentik-gated
# (PR #147 / #146). These EXTERNAL probes now measure the full
# gated path — DNS, TLS, Traefik, the gate outpost — following the
@@ -32,9 +32,9 @@ spec:
# not app health: with the gate up, they stay green even if the app
# behind it is dead (the probe never reaches the app). Real app
# health comes from the two INTERNAL /-/healthy probes below.
- https://prometheus.renner.uno
- https://alerts.renner.uno
- https://dash.renner.uno
- https://prometheus.${SITE_DOMAIN}
- https://alerts.${SITE_DOMAIN}
- https://dash.${SITE_DOMAIN}
# Offsite backup target (Hetzner Object Storage). Bare endpoint
# returns 200. CNPG WAL archiving + Velero both depend on it;
# this probe replaces the old GarageDown signal on nas-1.
@@ -56,11 +56,11 @@ spec:
# scans, the deploy targets — so an outage here is silent by
# construction: the thing that would have told you is the thing that
# is down.
- https://ci.renner.uno
- https://ci.${SITE_DOMAIN}
# Hermes. /api/status is public and answers 200 while the gateway
# runs. It runs the alert-monitor job that files the [alert] issues,
# so its outage is silent the same way Woodpecker's is.
- https://hermes.renner.uno/api/status
- https://hermes.${SITE_DOMAIN}/api/status
# OpenBao. Probes the health API rather than the UI on purpose: a
# SEALED instance still serves the UI (307 -> /ui/ -> 200) while every
# ExternalSecret in the cluster fails, which is exactly the
@@ -71,12 +71,12 @@ spec:
# standby, so the bare path would be permanently red. With the
# parameter, 200 means initialized, unsealed and serving, on whichever
# node answers; 503 sealed and 501 uninitialized both stay red.
- https://openbao.renner.uno/v1/sys/health?standbyok=true
- https://openbao.${SITE_DOMAIN}/v1/sys/health?standbyok=true
# ical-proxy serves an unauthenticated /health that answers 200 with
# {"status":"healthy"}. Unlike the gated hostnames this is real app
# health, not reachability — the root path returns 404 by design, since
# the service only answers on calendar paths.
- https://ical-proxy.renner.uno/health
- https://ical-proxy.${SITE_DOMAIN}/health
# APP-HEALTH probes for the gated monitoring UIs, from the correct
# in-cluster vantage (monitoring-vantage-point-rule): the gate is
# only on the ingress, so the ClusterIP /-/healthy endpoints are
@@ -30,7 +30,7 @@ spec:
# instance is the resolver and identical across these Probes, so the
# queried name has to come from a label.
labels:
dns_name: ha.renner.uno
dns_name: ha.${SITE_DOMAIN}
---
apiVersion: monitoring.coreos.com/v1
kind: Probe
@@ -52,7 +52,7 @@ spec:
- "[fd00::1]:53"
- "10.2.40.1:53"
labels:
dns_name: mqtt.renner.uno
dns_name: mqtt.${SITE_DOMAIN}
---
apiVersion: monitoring.coreos.com/v1
kind: Probe
@@ -74,7 +74,7 @@ spec:
- "[fd00::1]:53"
- "10.2.40.1:53"
labels:
dns_name: ldap.renner.uno
dns_name: ldap.${SITE_DOMAIN}
---
apiVersion: monitoring.coreos.com/v1
kind: Probe
@@ -96,7 +96,7 @@ spec:
- "[fd00::1]:53"
- "10.2.40.1:53"
labels:
dns_name: syslog.renner.uno
dns_name: syslog.${SITE_DOMAIN}
---
apiVersion: monitoring.coreos.com/v1
kind: Probe
@@ -118,7 +118,7 @@ spec:
- "[fd00::1]:53"
- "10.2.40.1:53"
labels:
dns_name: ftp.paperless.renner.uno
dns_name: ftp.paperless.${SITE_DOMAIN}
---
apiVersion: monitoring.coreos.com/v1
kind: Probe
@@ -140,4 +140,4 @@ spec:
- "[fd00::1]:53"
- "10.2.40.1:53"
labels:
dns_name: syncthing.paperless.renner.uno
dns_name: syncthing.paperless.${SITE_DOMAIN}
@@ -13,12 +13,12 @@ spec:
rules:
- alert: WoodpeckerDown
# Probe target defined in probes-blackbox.yaml. Root, ungated, 200.
expr: probe_success{job="blackbox-http", instance="https://ci.renner.uno"} == 0
expr: probe_success{job="blackbox-http", instance="https://ci.${SITE_DOMAIN}"} == 0
for: 5m
labels:
severity: warning
annotations:
summary: "Woodpecker is not answering on ci.renner.uno"
summary: "Woodpecker is not answering on ci.${SITE_DOMAIN}"
description: >-
Most scheduled jobs in this repo run here: issue-fix, the nightly
scans, and every deploy target. Nothing else reports this outage,
@@ -27,12 +27,12 @@ spec:
this alert exists.
- alert: HermesDown
# Probe target defined in probes-blackbox.yaml. Public status API, 200.
expr: probe_success{job="blackbox-http", instance="https://hermes.renner.uno/api/status"} == 0
expr: probe_success{job="blackbox-http", instance="https://hermes.${SITE_DOMAIN}/api/status"} == 0
for: 10m
labels:
severity: warning
annotations:
summary: "Hermes is not answering on hermes.renner.uno"
summary: "Hermes is not answering on hermes.${SITE_DOMAIN}"
description: >-
Hermes runs the alert-monitor job that files the [alert] issues,
and the ha-updates job. While it is down no new alert becomes an
@@ -36,7 +36,7 @@ spec:
severity: critical
annotations:
summary: "Public edge unreachable from the WAN (all members)"
description: "All IP-pinned WAN-path probes of the reverse-proxy edge (proxy-2 v4+v6, proxy-1 v6) have failed for 5 minutes — external clients cannot reach home.renner.uno and the other published apps. Check the netcup proxies (netbird-proxy containers), the mesh path to the cluster, and cluster Traefik."
description: "All IP-pinned WAN-path probes of the reverse-proxy edge (proxy-2 v4+v6, proxy-1 v6) have failed for 5 minutes — external clients cannot reach home.${SITE_DOMAIN} and the other published apps. Check the netcup proxies (netbird-proxy containers), the mesh path to the cluster, and cluster Traefik."
- alert: NetbirdPublicEdgeMemberDown
# A single edge member failing WAN reachability: DNS round-robin
@@ -15,7 +15,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: alerts.renner.uno
- host: alerts.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -28,4 +28,4 @@ spec:
tls:
- secretName: alertmanager-tls
hosts:
- alerts.renner.uno
- alerts.${SITE_DOMAIN}
@@ -56,12 +56,18 @@ configMapGenerator:
- name: dashboard-heatpump-roi
files:
- heatpump-roi.json
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
- name: dashboard-homeassistant-overview
files:
- homeassistant-overview.json
- name: dashboard-home-climate
files:
- home-climate.json
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
- name: dashboard-home-energy
files:
- home-energy.json
@@ -98,3 +104,6 @@ configMapGenerator:
- name: dashboard-pv-system-roi
files:
- pv-system-roi.json
options:
annotations:
kustomize.toolkit.fluxcd.io/substitute: disabled
@@ -9,7 +9,7 @@ spec:
name: letsencrypt-cluster
kind: ClusterIssuer
dnsNames:
- syslog.renner.uno
- syslog.${SITE_DOMAIN}
privateKey:
rotationPolicy: Always
---
@@ -231,7 +231,7 @@ metadata:
lb-pool: bgp
annotations:
io.cilium/lb-ipam-ips: "10.2.99.84,fd99::84"
external-dns.kubernetes.io/hostname: syslog.renner.uno
external-dns.kubernetes.io/hostname: syslog.${SITE_DOMAIN}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
@@ -56,9 +56,9 @@ spec:
http:
fail_if_not_ssl: true
tls_config:
server_name: home.renner.uno
server_name: home.${SITE_DOMAIN}
headers:
Host: home.renner.uno
Host: home.${SITE_DOMAIN}
# Plain TCP connect, used by the blackbox-tcp-relays Probe for the
# NetBird relay ws listeners on :33080. Address family is spelled
# out as ip6-then-fallback (blackbox's own tcp default, pinned so
@@ -87,7 +87,7 @@ spec:
prober: dns
timeout: 5s
dns:
query_name: ha.renner.uno
query_name: ha.${SITE_DOMAIN}
query_type: A
valid_rcodes: [NOERROR]
validate_answer_rrs:
@@ -97,7 +97,7 @@ spec:
prober: dns
timeout: 5s
dns:
query_name: mqtt.renner.uno
query_name: mqtt.${SITE_DOMAIN}
query_type: A
valid_rcodes: [NOERROR]
validate_answer_rrs:
@@ -107,7 +107,7 @@ spec:
prober: dns
timeout: 5s
dns:
query_name: ldap.renner.uno
query_name: ldap.${SITE_DOMAIN}
query_type: A
valid_rcodes: [NOERROR]
validate_answer_rrs:
@@ -117,7 +117,7 @@ spec:
prober: dns
timeout: 5s
dns:
query_name: syslog.renner.uno
query_name: syslog.${SITE_DOMAIN}
query_type: A
valid_rcodes: [NOERROR]
validate_answer_rrs:
@@ -127,7 +127,7 @@ spec:
prober: dns
timeout: 5s
dns:
query_name: ftp.paperless.renner.uno
query_name: ftp.paperless.${SITE_DOMAIN}
query_type: A
valid_rcodes: [NOERROR]
validate_answer_rrs:
@@ -137,7 +137,7 @@ spec:
prober: dns
timeout: 5s
dns:
query_name: syncthing.paperless.renner.uno
query_name: syncthing.paperless.${SITE_DOMAIN}
query_type: A
valid_rcodes: [NOERROR]
validate_answer_rrs:
@@ -75,7 +75,7 @@ spec:
podMonitorSelectorNilUsesHelmValues: false
# Rewrites the in-cluster URL embedded in alert generatorURLs and
# status pages so click-through from emails works.
externalUrl: https://prometheus.renner.uno
externalUrl: https://prometheus.${SITE_DOMAIN}
# Time retention is the one in force; the size cap is a backstop
# (REN-107, owner decision: 30 days on the existing disk). This was
# 90d/30GB, and at ~1 GB/day of ingest the 30GiB cap bound first: the
@@ -135,11 +135,11 @@ spec:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
hosts:
- dash.renner.uno
- dash.${SITE_DOMAIN}
tls:
- secretName: grafana-tls
hosts:
- dash.renner.uno
- dash.${SITE_DOMAIN}
# Postgres backend (gf-pg CNPG cluster). SQLite-on-RBD made schema
# migrations on cold rebuilds blow past the liveness probe budget;
# Postgres batches WAL fsyncs server-side and migrations finish in
@@ -201,7 +201,7 @@ spec:
defaultBucket: suedstr28
tlsSkipVerify: true
secureJsonData:
token: ${INFLUXDB_TOKEN}
token: $${INFLUXDB_TOKEN}
sidecar:
resources:
requests:
@@ -242,15 +242,15 @@ spec:
- name: grafana-influxdb-token
grafana.ini:
server:
root_url: https://dash.renner.uno
root_url: https://dash.${SITE_DOMAIN}
auth.generic_oauth:
enabled: true
name: Authentik
allow_sign_up: true
scopes: openid profile email
auth_url: https://auth.renner.uno/application/o/authorize/
token_url: https://auth.renner.uno/application/o/token/
api_url: https://auth.renner.uno/application/o/userinfo/
auth_url: https://auth.${SITE_DOMAIN}/application/o/authorize/
token_url: https://auth.${SITE_DOMAIN}/application/o/token/
api_url: https://auth.${SITE_DOMAIN}/application/o/userinfo/
use_pkce: true
role_attribute_path: "contains(groups[*], 'Infrastructure Admins') && 'Admin' || 'Viewer'"
resources:
@@ -276,7 +276,7 @@ spec:
# the "Source" link of every grouped alert, so the email link
# resolves externally instead of pointing at the in-cluster
# service hostname.
externalUrl: https://alerts.renner.uno
externalUrl: https://alerts.${SITE_DOMAIN}
storage:
volumeClaimTemplate:
spec:
+2 -2
View File
@@ -16,7 +16,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: loki.renner.uno
- host: loki.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -28,5 +28,5 @@ spec:
number: 3100
tls:
- hosts:
- loki.renner.uno
- loki.${SITE_DOMAIN}
secretName: loki-ingress-tls
@@ -15,7 +15,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: prometheus.renner.uno
- host: prometheus.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -28,4 +28,4 @@ spec:
tls:
- secretName: prometheus-tls
hosts:
- prometheus.renner.uno
- prometheus.${SITE_DOMAIN}
+2 -2
View File
@@ -12,7 +12,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: music.renner.uno
- host: music.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -25,4 +25,4 @@ spec:
tls:
- secretName: music-assistant-tls
hosts:
- music.renner.uno
- music.${SITE_DOMAIN}
+4 -4
View File
@@ -11,7 +11,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: navidrome.renner.uno
- host: navidrome.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -24,7 +24,7 @@ spec:
tls:
- secretName: navidrome-tls
hosts:
- navidrome.renner.uno
- navidrome.${SITE_DOMAIN}
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
@@ -50,7 +50,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: navidrome.renner.uno
- host: navidrome.${SITE_DOMAIN}
http:
paths:
- path: /rest
@@ -81,7 +81,7 @@ spec:
- websecure
routes:
- kind: Rule
match: Host(`navidrome.renner.uno`) && (PathPrefix(`/auth`) || PathPrefix(`/api`)) && !HeaderRegexp(`Cookie`, `authentik_proxy_`)
match: Host(`navidrome.${SITE_DOMAIN}`) && (PathPrefix(`/auth`) || PathPrefix(`/api`)) && !HeaderRegexp(`Cookie`, `authentik_proxy_`)
priority: 1000
middlewares:
- name: navidrome-strip-identity
+2 -2
View File
@@ -200,7 +200,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: lidarr.renner.uno
- host: lidarr.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -213,4 +213,4 @@ spec:
tls:
- secretName: lidarr-tls
hosts:
- lidarr.renner.uno
- lidarr.${SITE_DOMAIN}
@@ -14,8 +14,8 @@ metadata:
namespace: paperless
spec:
redirectRegex:
regex: ^https://docs\.renner\.uno/(.*)
replacement: https://paperless.renner.uno/${1}
regex: ^https://docs\.${SITE_DOMAIN_RE}/(.*)
replacement: https://paperless.${SITE_DOMAIN}/$${1}
permanent: true
---
apiVersion: networking.k8s.io/v1
@@ -33,7 +33,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: docs.renner.uno
- host: docs.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -46,4 +46,4 @@ spec:
tls:
- secretName: docs-renner-uno-tls
hosts:
- docs.renner.uno
- docs.${SITE_DOMAIN}
+1 -1
View File
@@ -119,7 +119,7 @@ metadata:
# label — FTP must not be exposed via the cargo tunnel (clear-text
# protocol, public exposure not appropriate). The brother scanner
# uses this hostname instead of the LB IP.
external-dns.kubernetes.io/hostname: ftp.paperless.renner.uno
external-dns.kubernetes.io/hostname: ftp.paperless.${SITE_DOMAIN}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+4 -4
View File
@@ -22,8 +22,8 @@ data:
# new users hit 403 on /api/ui_settings/ until an admin grants
# perms manually.
PAPERLESS_ACCOUNT_DEFAULT_GROUPS: "Familie"
PAPERLESS_URL: "https://paperless.renner.uno"
PAPERLESS_CSRF_TRUSTED_ORIGINS: "https://paperless.renner.uno,https://docs.renner.uno"
PAPERLESS_URL: "https://paperless.${SITE_DOMAIN}"
PAPERLESS_CSRF_TRUSTED_ORIGINS: "https://paperless.${SITE_DOMAIN},https://docs.${SITE_DOMAIN}"
# anchor: allowed-hosts
PAPERLESS_ALLOWED_HOSTS: "*"
PAPERLESS_TIME_ZONE: "Europe/Berlin"
@@ -365,7 +365,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: paperless.renner.uno
- host: paperless.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -378,4 +378,4 @@ spec:
tls:
- secretName: paperless-tls
hosts:
- paperless.renner.uno
- paperless.${SITE_DOMAIN}
+1 -1
View File
@@ -145,7 +145,7 @@ metadata:
lb-pool: bgp
annotations:
io.cilium/lb-ipam-ips: "10.2.99.85,fd99::85"
external-dns.kubernetes.io/hostname: syncthing.paperless.renner.uno
external-dns.kubernetes.io/hostname: syncthing.paperless.${SITE_DOMAIN}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+1 -1
View File
@@ -27,7 +27,7 @@ data:
config.json: |
{
"platform": "gitea",
"endpoint": "https://git.renner.uno/api/v1",
"endpoint": "https://git.${SITE_DOMAIN}/api/v1",
"gitAuthor": "Renovate Bot <renovate-bot@renner.uno>",
"autodiscover": true,
"onboarding": false,
+1 -1
View File
@@ -37,7 +37,7 @@ spec:
- name: REPOS
value: konairius/homelab,konairius/homeassistant,konairius/authentik-samba-bridge,konairius/shelly-manager
- name: GITEA_URL
value: https://gitea.renner.uno
value: https://gitea.${SITE_DOMAIN}
- name: AGENT_NAME
value: pr-review
- name: CONNECTOR_ID
+2 -2
View File
@@ -12,10 +12,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- review-bot.renner.uno
- review-bot.${SITE_DOMAIN}
secretName: review-bot-tls
rules:
- host: review-bot.renner.uno
- host: review-bot.${SITE_DOMAIN}
http:
paths:
- path: /status
+2 -2
View File
@@ -37,10 +37,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- trmnl-device.renner.uno
- trmnl-device.${SITE_DOMAIN}
secretName: trmnl-device-tls
rules:
- host: trmnl-device.renner.uno
- host: trmnl-device.${SITE_DOMAIN}
http:
paths:
- path: /
+2 -2
View File
@@ -13,10 +13,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- trmnl.renner.uno
- trmnl.${SITE_DOMAIN}
secretName: trmnl-tls
rules:
- host: trmnl.renner.uno
- host: trmnl.${SITE_DOMAIN}
http:
paths:
- path: /
+1 -1
View File
@@ -17,7 +17,7 @@ data:
# earlier value was http on the theory that the firmware would reject an
# internal CA; there is no internal CA here — letsencrypt-cluster issues
# publicly-trusted certs for LAN-only names via DNS-01.
API_URI: "https://trmnl-device.renner.uno"
API_URI: "https://trmnl-device.${SITE_DOMAIN}"
KEYVALUE_URL: "redis://terminus-valkey:6379"
# Rack::Attack rate-limit bypass. Beyond upstream's RFC1918 defaults this
# adds fc00::/7, which covers the pod CIDR (fd10:244::/48), the Cilium LB
@@ -248,13 +248,13 @@
apiVersion: endpointmonitor.stakater.com/v1alpha1
kind: EndpointMonitor
metadata:
name: www.renner.uno
name: www.${SITE_DOMAIN}
namespace: uptime-monitor
spec:
# `https://www.renner.uno` and not `/wp-admin` or `/wp-login.php`: those two
# paths are served by a SECOND Ingress that sits behind the Authentik outpost
# (../wordpress/ingress.yaml#gated-paths). `/` is the ungated one.
url: https://www.renner.uno/
url: https://www.${SITE_DOMAIN}/
uptimeRobotConfig:
# `http`, not `keyword`. The survey recommends a keyword check here
# eventually, and that is right — a WordPress instance can serve 200 with a
@@ -280,7 +280,7 @@ spec:
apiVersion: endpointmonitor.stakater.com/v1alpha1
kind: EndpointMonitor
metadata:
name: audiobooks.renner.uno
name: audiobooks.${SITE_DOMAIN}
namespace: uptime-monitor
spec:
# The path belongs in `spec.url` and NOT in `spec.healthEndpoint`, because
@@ -289,7 +289,7 @@ spec:
# `/healthcheck` is ungated, so the check reaches application health rather
# than mere reachability — it is the same endpoint the Gatus check uses
# (Web Service 8 in ../gatus/configmap.yaml#gatus-endpoints).
url: https://audiobooks.renner.uno/healthcheck
url: https://audiobooks.${SITE_DOMAIN}/healthcheck
uptimeRobotConfig:
monitorType: http
interval: 300
+2 -2
View File
@@ -65,7 +65,7 @@ spec:
- name: Wiki__Index__File__Directory
value: /data
- name: Wiki__NoteStore__Gitea__BaseUrl
value: https://gitea.renner.uno
value: https://gitea.${SITE_DOMAIN}
- name: Wiki__NoteStore__Gitea__Repository
value: konairius/notes
- name: Wiki__NoteStore__Gitea__Branch
@@ -74,7 +74,7 @@ spec:
value: http://wiki-embed.ai.svc.cluster.local:8080
# Human login: the `wiki` application in apps/authentik/blueprints-configmap.yaml.
- name: Auth__Oidc__Authority
value: https://auth.renner.uno/application/o/wiki/
value: https://auth.${SITE_DOMAIN}/application/o/wiki/
- name: Auth__Oidc__ClientId
value: wiki
# wiki-index BasicAuth users (htpasswd), checked by the service for the read API.
+4 -4
View File
@@ -12,10 +12,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- wiki.renner.uno
- wiki.${SITE_DOMAIN}
secretName: wiki-tls
rules:
- host: wiki.renner.uno
- host: wiki.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -50,10 +50,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- wiki-index.renner.uno
- wiki-index.${SITE_DOMAIN}
secretName: wiki-api-tls
rules:
- host: wiki-index.renner.uno
- host: wiki-index.${SITE_DOMAIN}
http:
paths:
- path: /
+2 -2
View File
@@ -13,7 +13,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: ci.renner.uno
- host: ci.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -26,4 +26,4 @@ spec:
tls:
- secretName: woodpecker-tls
hosts:
- ci.renner.uno
- ci.${SITE_DOMAIN}
+2 -2
View File
@@ -43,7 +43,7 @@ spec:
image: woodpeckerci/woodpecker-server:v3.19.0 # {"$imagepolicy": "flux-system:woodpecker-server"}
env:
- name: WOODPECKER_HOST
value: https://ci.renner.uno
value: https://ci.${SITE_DOMAIN}
- name: WOODPECKER_OPEN
value: "true"
- name: WOODPECKER_ADMIN
@@ -55,7 +55,7 @@ spec:
- name: WOODPECKER_GITEA
value: "true"
- name: WOODPECKER_GITEA_URL
value: https://gitea.renner.uno
value: https://gitea.${SITE_DOMAIN}
# Woodpecker v3 removed default privileged plugins. The image-build
# pipelines (e.g. shelly-manager) previously ran a privileged
# docker/buildx plugin under the docker backend; on the kubernetes
+2 -2
View File
@@ -52,8 +52,8 @@ spec:
key: password
- name: WORDPRESS_CONFIG_EXTRA
value: |
define('WP_HOME', 'https://www.renner.uno');
define('WP_SITEURL', 'https://www.renner.uno');
define('WP_HOME', 'https://www.${SITE_DOMAIN}');
define('WP_SITEURL', 'https://www.${SITE_DOMAIN}');
define('FORCE_SSL_ADMIN', true);
/* Traefik terminates TLS; without this WordPress sees plain
HTTP and builds http:// URLs into every page and redirect. */
+4 -4
View File
@@ -21,7 +21,7 @@ metadata:
spec:
ingressClassName: traefik
rules:
- host: www.renner.uno
- host: www.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -34,7 +34,7 @@ spec:
tls:
- secretName: wordpress-tls
hosts:
- www.renner.uno
- www.${SITE_DOMAIN}
---
# wp-admin, wp-login.php, xmlrpc.php and the outpost's own OAuth leg
# (/outpost.goauthentik.io) are gated by authentik-gate-sync (ingress class
@@ -61,9 +61,9 @@ spec:
tls:
- secretName: wordpress-tls
hosts:
- www.renner.uno
- www.${SITE_DOMAIN}
rules:
- host: www.renner.uno
- host: www.${SITE_DOMAIN}
http:
paths:
- path: /outpost.goauthentik.io
@@ -15,7 +15,7 @@ metadata:
spec:
ingressClassName: authentik-gate
rules:
- host: zone-configurator.home.renner.uno
- host: zone-configurator.home.${SITE_DOMAIN}
http:
paths:
- path: /
@@ -28,4 +28,4 @@ spec:
tls:
- secretName: zone-configurator-tls
hosts:
- zone-configurator.home.renner.uno
- zone-configurator.home.${SITE_DOMAIN}
+7 -2
View File
@@ -1905,13 +1905,18 @@ EOF
# patch), and CoreDNS forwards external names to the node. Until k8s-gateway
# answers, every lookup waits out the node's 2 s read timeout and CoreDNS
# gives up first: pods resolve nothing, Flux included. So it is installed
# here, from the values Flux will reconcile (the HelmRelease, under the
# here, from the values Flux will reconcile (the HelmRelease, substituted
# as its Flux Kustomization does with the site's site-settings, under the
# site's k8s-gateway-site values, as valuesFrom merges them), and Flux
# adopts the release.
step 0 "Installing k8s-gateway (the nodes' first resolver)..."
command -v flux >/dev/null || fail "flux not found (substitutes the k8s-gateway values)"
k8sgw_values=$(mktemp)
k8sgw_site_values=$(mktemp)
yq 'select(.kind == "HelmRelease") | .spec.values' "$K8SGW_HR" > "$k8sgw_values"
yq 'select(.kind == "HelmRelease") | .spec.values' "$K8SGW_HR" \
| SITE_PATCH_DIR="$TALOS_DIR/patches" bash "$REPO_ROOT/scripts/ci/site-render.sh" \
--subst "$SITE" kubernetes/infrastructure > "$k8sgw_values" \
|| fail "k8s-gateway values do not substitute with the site-settings of talos/patches/site-${SITE}.yaml"
yq -r 'select(.cluster) | .cluster.inlineManifests[] | select(.name == "k8s-gateway-site") | .contents' \
"$TALOS_DIR/patches/site-${SITE}.yaml" \
| yq -r '.data["values.yaml"]' > "$k8sgw_site_values"
+4
View File
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-13: path is sops-free (external-dns +
# woodpecker migrated) — secrets come from OpenBao via ESO
@@ -16,6 +16,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/audiobookshelf
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
decryption:
provider: sops
@@ -15,6 +15,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/authentik
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-14: path is sops-free — secrets come
# from OpenBao via ESO (MIGRATION-sops-openbao.md). This was the LAST
+4
View File
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/evcc
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-13: path is sops-free — secrets come
# from OpenBao via ESO (MIGRATION-sops-openbao.md).
@@ -3,3 +3,14 @@ kind: Kustomization
resources:
- gotk-components.yaml
- gotk-sync.yaml
patches:
- target:
kind: Kustomization
name: flux-system
patch: |
- op: add
path: /spec/postBuild
value:
substituteFrom:
- kind: ConfigMap
name: site-settings
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/gitea-mcp
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
+4
View File
@@ -13,6 +13,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/gitea
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# The claim's bound volume differs per cluster (apps/gitea/pvc.yaml).
ignore:
+4
View File
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/hermes
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
@@ -15,6 +15,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/home-assistant
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-13: path is sops-free — secrets come
# from OpenBao via ESO (MIGRATION-sops-openbao.md).
@@ -45,6 +49,7 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/matter-server
suspend: ${SITE_SUSPEND_IOT_VLAN}
prune: true
wait: true
dependsOn:
@@ -67,7 +72,12 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/home-assistant-iot
suspend: ${SITE_SUSPEND_IOT_VLAN}
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: restore-home-assistant # shelly-manager's PVC
@@ -115,6 +125,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/zone-configurator
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: home-assistant
+4
View File
@@ -16,6 +16,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/immich
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-13: path is sops-free — secrets come
# from OpenBao via ESO (MIGRATION-sops-openbao.md).
@@ -13,6 +13,10 @@ spec:
path: ./kubernetes/infrastructure-inhouse
prune: true
wait: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
dependsOn:
- name: infrastructure
- name: infrastructure-crds
@@ -17,6 +17,10 @@ spec:
name: flux-system
path: ./kubernetes/infrastructure/crowdsec
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
@@ -40,7 +44,12 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/infrastructure/external-dns-unifi
suspend: ${SITE_SUSPEND_UNIFI}
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
@@ -60,6 +69,10 @@ spec:
name: flux-system
path: ./kubernetes/infrastructure/netbird-ingress
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/infrastructure
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
# wait: false + explicit healthChecks (NOT wait: true). With wait: true, Flux
# ignores healthChecks and blocks this Kustomization's Ready on EVERY resource
# it applies (hundreds) — so one crashlooping workload froze it and, via
@@ -81,14 +85,14 @@ spec:
wait: true
# decryption block dropped 2026-07-13: no *.sops.yaml left under this
# path — secrets come from OpenBao via ESO (MIGRATION-sops-openbao.md).
# Substitute ${ACME_SERVER} (and CLUSTER_IN_DEVELOPMENT for any future
# gates) from the cluster-settings ConfigMap into the ClusterIssuer
# manifest — this is the single point where the dev/prod LE flag lands
# in cert-manager state.
# cluster-settings: CLUSTER_IN_DEVELOPMENT. site-settings: SITE_DOMAIN and
# SITE_ACME_SERVER for the ClusterIssuers.
postBuild:
substituteFrom:
- kind: ConfigMap
name: cluster-settings
- kind: ConfigMap
name: site-settings
dependsOn:
- name: infrastructure
---
@@ -14,7 +14,12 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/kuma-sync
suspend: ${SITE_SUSPEND_UPTIME_MONITORING}
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
healthCheckExprs:
- apiVersion: external-secrets.io/v1
@@ -14,7 +14,12 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/llama-server
suspend: ${SITE_SUSPEND_IGPU}
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
@@ -41,6 +41,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/monitoring
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-13: path is sops-free — secrets come
# from OpenBao via ESO (MIGRATION-sops-openbao.md).
@@ -73,6 +77,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/monitoring-rules
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: monitoring
@@ -15,7 +15,12 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/music-assistant
suspend: ${SITE_SUSPEND_IOT_VLAN}
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
decryption:
provider: sops
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/navidrome
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: restore-navidrome
@@ -16,6 +16,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/paperless
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# decryption block dropped 2026-07-13: path is sops-free — secrets come
# from OpenBao via ESO (MIGRATION-sops-openbao.md).
@@ -23,6 +23,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/renovate
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
healthCheckExprs:
- apiVersion: external-secrets.io/v1
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/review-bot
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
+4
View File
@@ -15,6 +15,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/trmnl
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: restore-trmnl
@@ -87,6 +87,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/uptime-monitor
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# No name/namespace qualifier, so this is evaluated against every
# ExternalSecret in this Kustomization's inventory. That is exactly one
+5
View File
@@ -12,7 +12,12 @@ spec:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/wiki
suspend: ${SITE_SUSPEND_IGPU}
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: infrastructure
@@ -17,6 +17,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/woodpecker
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
# sops-free path: woodpecker-secret + barman creds come from OpenBao via ESO
# (MIGRATION-sops-openbao.md).
@@ -12,6 +12,10 @@ spec:
name: flux-system
path: ./kubernetes/apps/wordpress
prune: true
postBuild:
substituteFrom:
- kind: ConfigMap
name: site-settings
wait: true
dependsOn:
- name: restore-wordpress
@@ -94,9 +94,13 @@ spec:
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: netbird-service-sync
# Must match cluster.network.serviceSubnets in talos/patches/cluster.yaml.
# Must match cluster.network.serviceSubnets in talos/patches/site-*.yaml.
- name: SERVICE_CIDR
value: "10.96.0.0/12"
value: "${SITE_SERVICE_CIDR}"
- name: NETWORK_NAME
value: "${SITE_NETBIRD_NETWORK}"
- name: ROUTER_GROUP_NAME
value: "${SITE_NETBIRD_ROUTER_GROUP}"
- name: TRAEFIK_SERVICE
value: ingress/traefik
- name: INTERVAL_SECONDS

Some files were not shown because too many files have changed in this diff Show More