The previous placeholder string contained underscores — not valid base64 — which the API
server rejects on Secret create, making the infrastructure / apps Kustomizations report
unhealthy until the real token is filled in. Using base64 of `{"auths":{}}` instead lets
the Secret apply cleanly; only the pod-level pull (or image-reflector tag listing) fails
until you encrypt real credentials in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New apps/shelly-manager Deployment + LAN-only Ingress under `shelly-manager.k.renner.uno`,
pulling from the in-house gitea registry. Adds a second broker user `shellies` scoped to
`shellies/#` + `homeassistant/#` so devices can be provisioned without sharing the HA
credential, and a third `admin` user for workstation debugging — each on its own bootstrap
Job + Secret so passwords rotate independently. Image automation tracks `build.<unix-ts>`
tags published by the shelly-manager CI; gitea registry creds live in both `apps` and
`flux-system` namespaces (SOPS placeholders pending tokens).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Proxmox web UI polls /api2/json/cluster/* and per-guest status/current
endpoints every ~2s; CrowdSec's http-probing scenario flagged the
pattern as scanning and banned residential IPv6 clients via the
ingress-nginx Lua bouncer (bare 403 across every host).
Adds a parser-stage whitelist matching only those polling paths on
pve.renner.uno.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The template var \${pv_install_date} substitutes as an int literal
(1684706400). Flux is strict about float vs int arithmetic, so
`float / int_literal` in the elapsed_years calc was failing and
breaking the Payback panel. Wrap with float(v: \${pv_install_date}).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the user directive that historical HA data lives in InfluxDB only,
the three time-spanning HA dashboards switch from PromQL against K8s
Prom (90d retention) to Flux against the in-cluster InfluxDB
(suedstr28 bucket, repointed in commit eb6399f).
- home-energy: 9 data panels (smart_meter L1/L2/L3/sum, energy total,
top consumers, lights). Measurement = unit_of_measurement
("W"/"kWh"/"V"/"A"/"%"); filter by entity_id tag.
- home-climate: 10 data panels (room temperatures via Shelly/Meter Pro,
humidity, CO2, heating system, hot water).
- pv-system-roi: 23 data panels including the multi-entity arithmetic
panels (Self-Use Savings, Total Value, Payback, ROI Progress) which
use Flux scalar findRecord + array.from for stats and pivot+map for
the cumulative-value timeseries.
Expect some queries to need iteration as the InfluxDB schema is
discovered live — particularly the friendly_name regex translations
on home-climate and the climate-domain heating panel which assumes
all _measurement="°C" with domain="climate" entries are wanted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VM 108 (the legacy InfluxDB at influxdb.srv.renner.uno:10.2.40.16) is
unreachable — no ICMP, no SSH, no port 8086. K8s Grafana's InfluxDB
datasource has been silently broken since the in-cluster InfluxDB
came online (commit 1daaa6f, 2026-05-05). The heatpump-roi dashboard
and any historical HA panel that queries InfluxDB are dead until this
repoint lands.
Datasource URL: http://influxdb-influxdb2.influxdb.svc.cluster.local:8086.
The legacy SOPS-stored token (`grafana-influxdb-token.sops.yaml`) is
useless against the in-cluster bolt store. Replaced with a seed Job
that mirrors the existing `influxdb-ha-token` pattern: ServiceAccount
in influxdb ns + cross-namespace Role on the `grafana-influxdb-token`
Secret in monitoring ns + Job that creates a read-only auth on the
suedstr28 bucket and writes the token to that Secret. The key name
inside the Secret stays `INFLUXDB_TOKEN` so the existing
`envFromSecrets` reference in the helmrelease keeps working.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2c. Three dashboards whose data path is structurally blocked in
the K8s-only setup:
- paperless-overview: paperless-ngx has no native /metrics; the
dashboard's `paperless_*` metric names match an upstream community
exporter that doesn't appear to ship a maintained public image.
- ntp-overview: dashboard queries `chrony_tracking_*` from a per-host
chrony_exporter. Talos K8s nodes don't expose chronyd via a socket a
DaemonSet can read without hostNetwork, which we forbid.
- synology-nas: requires a custom snmp.yml with the `synology` module
(not in upstream snmp_exporter's bundled config — confirmed
63110-line stock snmp.yml has zero `synology:` sections). Generating
a synology snmp.yml from Synology MIB files is out of scope for a
GitOps commit; the dashboard would also need updates for the v0.27+
multi-auth schema. Revisit if Synology metrics become a priority.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2b of the dashboard triage. Deploys prometheus-community's
blackbox-exporter chart in the monitoring namespace and creates Probe
CRs covering the legacy Ansible blackbox_http_targets list plus the
TCP probe for the legacy postgres-ha host. The chart's ServiceMonitor
exposes the exporter's own metrics; the Probe CRs drive periodic
endpoint probing.
Feeds blackbox-overview dashboard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2a of the dashboard triage. Wires three metric sources whose
dashboards previously had no data path:
- alloy-servicemonitor.yaml: scrapes the alloy DaemonSet's
http-metrics:12345 endpoint. Feeds alloy-overview.
- immich-servicemonitor.yaml: scrapes immich-server's metrics:8081
endpoint (already exposed by the existing Service). Feeds
immich-overview.
- helmrelease-prometheus.yaml: adds a static scrape for
gitea.srv.renner.uno:3000/metrics (anonymous, no auth token
required). Feeds gitea-overview.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 of the dashboard triage.
Deleted (no current data source, no plan to wire one):
- unifi-overview (unpoller removed after UDM crash)
- caddy-overview (Caddy retired 2026-05-09)
- docker-overview (Docker host era done; workloads on K8s)
- log-explorer (legacy host log shipping not in scope)
- dns-overview (dns-servers job lives but a dedicated dashboard isn't worth keeping; Explore tab covers ad-hoc)
Repointed datasource from `prometheus-legacy` to `prometheus` (the
legacy LXC 227 Prom is retired):
- node-overview, node-metrics — show K8s nodes only (legacy host
node-exporters were never re-scraped post-migration)
- infrastructure-overview — generic up{} + node_* metrics work on K8s
- proxmox-cluster — pve-exporter is now in-cluster (commit 33b452a)
Also deletes DASHBOARD-MIGRATION.md (the migration cut-over completed
with VM 109 retirement on 2026-05-09).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ingress-nginx + letsencrypt-cluster cert at prometheus.renner.uno
and alerts.renner.uno. No external-dns/cloudflare label, so only
external-dns-unifi publishes the hostnames — they resolve on LAN
and NXDOMAIN from the public internet.
externalUrl set on both so the click-through links embedded in
Alertmanager emails resolve to the external hostname instead of the
in-cluster service name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ingress-nginx + letsencrypt-cluster cert at prometheus.k.renner.uno
and alerts.k.renner.uno. The .k.renner.uno zone is published by
external-dns-unifi only, so neither hostname is reachable from the
public internet.
externalUrl set on both so the click-through links embedded in
Alertmanager emails resolve to the external hostname instead of the
in-cluster service name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kube-prom-stack default alert fires whenever the cluster can't
tolerate the failure of one node. On a small homelab with no spare
node headroom this is structurally unsatisfiable, not actionable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
readOnlyRootFilesystem=true leaves no writable directory under /tmp,
and pve_exporter's Python startup fails with FileNotFoundError from
tempfile._get_default_tempdir. Mount a small emptyDir at /tmp.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pointing unpoller v2.39.0 at the UDM caused the controller to crash.
Removing the deployment, secret, image-automation policy, and the
helmrelease pointer entirely. UniFi metrics stay offline until a
known-good poller/UDM combination is identified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pve-exporter image ENTRYPOINT is already `pve_exporter`; the args list
was duplicating the binary name as a literal argument, producing
"unrecognized arguments: pve_exporter".
unpoller v2.39.0 nil-panics in `datadogunifi.Run` when output sections
are missing from up.conf (regression from the 2.11.x era the Ansible
role targeted). Explicitly disable [influxdb]/[datadog]/[loki].
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous deploys (Ansible roles install-prometheus.yml against
hosts: prometheus.srv.renner.uno on LXC 227) died with that host's
retirement. Re-deploy both as K8s Deployments in the monitoring
namespace, reusing the existing Ansible-vault credentials for the
Proxmox root@pam!ansible token and the UniFi `Poller` read-only user
(now stored in SOPS-encrypted Secrets per the K8s convention).
pve-exporter (prompve/prometheus-pve-exporter:3.8.3) listens on 9221
and is queried as a blackbox-style exporter: the existing pve scrape
config keeps the three 10.2.40.20x targets and uses relabel_configs
to direct the actual HTTP call to pve-exporter.monitoring.svc.
unpoller (ghcr.io/unpoller/unpoller:v2.1.5) polls https://udm.renner.uno
and exposes :9130. Scraped via a ServiceMonitor; the dead static
unifi scrape config is removed from the helmrelease.
Both images are enrolled in image-automation: pve-exporter on semver
3.x with plain 3-component tags; unpoller on semver 2.x with v-prefix
extract that rejects -rc and arch-suffixed variants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The in-cluster ServiceMonitor was missing the `release: kube-prometheus-stack`
selector label, so the Prometheus resource never discovered it (same trap
the garage PrometheusRule documents). Adding the label brings the scrape
online via the existing in-cluster Service + bearer-token Secret.
Also removes the obsolete `homeassistant` additionalScrapeConfig that
pointed at the external IoT-VLAN IP with an empty bearer_token TODO —
HA lives in the cluster now, the external path is dead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Watchdog is an always-firing alert designed to feed an external
dead-man's-switch (PagerDuty DMS, Healthchecks.io, etc.); we don't
run one, and a 12h-repeat email of a continually-firing alert is
pure noise. Route alertname="Watchdog" to a null receiver.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scrape_samples_scraped is a gauge, not a counter, so rate() of a
stable value is 0 — the alert fired whenever Garage was healthy and
producing a steady sample count. Switch to a direct gauge==0
comparison so the alert reflects the annotated intent (endpoint up,
zero samples returned).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Alertmanager (kube-prometheus-stack) now sends to admin@renner.uno via
Proton SMTP submission (smtp.protonmail.ch:587 STARTTLS) using a per-
account token in the alertmanager-smtp SOPS-encrypted Secret.
A new infrastructure subtree, flux-notifications/, adds a Provider of
type alertmanager pointing at the in-cluster Alertmanager and two
Alert CRs: one info-severity for ImageUpdateAutomation bumps (with the
30m "no change" heartbeat excluded), and one error-severity catch-all
across every Flux source/sync kind. Flux events and Prometheus alerts
now share a single email funnel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ImageRepository/ImagePolicy pairs for gotenberg, redis (immich +
paperless), ceph, apache/tika, protonmail-bridge, kasm/firefox,
mmwave-configurator, and cnpg-barman-plugin. README's not-enrolled
list trimmed to the three genuinely-unfit (vsftpd, otbr, ical-proxy)
and gains a re-import-hazard section for cnpg-barman.
Pin replacements on moving tags (`:8`, `:7.4-alpine`, `:v19`, `:latest`)
resolve to current upstream digests, so no behavior change; firefox
bumps 1.17.0 -> 1.18.0 (next-newer rolling-daily).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
reload_core_config only re-processes the `homeassistant:` section of
configuration.yaml — it does not re-initialize the http integration.
The http component reads http.trusted_proxies once in async_setup() at
boot and has no live-reload hook. Result: every time the ingress-nginx
controller's pod IPs change (rollout, node drain, IPAM reshuffle), the
reconciler silently wrote the new IPs to disk but HA kept rejecting
every external request as "untrusted proxy" until the next manual HA
restart.
Switch the API call to homeassistant.restart so the sidecar's "I
detected drift" path actually applies the change. HA validates its
config pre-restart, exits cleanly, kubelet respawns the container, and
the fresh process reads the patched trusted_proxies on boot. The cost
is a ~30s HA outage per ingress-nginx pod-IP change — acceptable for
an event that happens rarely and was previously silently breaking HA
indefinitely.
Comments in deployment.yaml and rbac.yaml were stale (deployment.yaml
still described the long-removed pod-delete path; rbac.yaml claimed
"no Recreate needed" under reload_core_config). Updated both to
describe the actual mechanism.
The init-clone-crowdsec-bouncer setup mounted the emptyDir at /crowdsec,
which shadowed the lua-bouncer-plugin image's baked-in plugin files and
template config. With nothing in /crowdsec, the entrypoint's sed against
/crowdsec/crowdsec-bouncer.conf failed silently and no files ever landed
in /etc/nginx/lua/plugins/crowdsec, so nginx logged
load_plugin(): error loading plugin "plugins.crowdsec.main": module
'plugins.crowdsec.main' not found
The bouncer registered against LAPI (BOUNCER_KEY pre-registration) but
its last_pull stayed null since the rebuild — agents detected attackers
and LAPI accumulated CAPI decisions, but the controller never enforced
anything.
Aligns with the canonical recipe in
https://docs.crowdsec.net/docs/bouncers/ingress-nginx:
- mount the emptyDir at /lua_plugins so the image's /crowdsec stays
intact
- override command to run docker_start.sh and then cp the templated
files into the volume
- pin to v1.1.2 instead of the implicit :latest moving target
The gate fundamentally conflicted with MA's PWA model: Chrome fetches
the manifest, service worker, and icons with credentials omitted
(no crossorigin=use-credentials on MA's <link rel="manifest">), and
once installed the cached service worker bypasses fresh navigations
on cold-start so the Authentik cookie never lands in the WebAPK's
cookie jar. MA delegates auth to Home Assistant in-app; the gate's
marginal defense-in-depth wasn't worth the recurring breakage.
Ingress reverts to pointing at music-assistant directly. Cloudflared
keeps music.renner.uno wired to the tunnel. Blueprint marks the
provider/application/outpost state: absent so the worker tears down
ak-outpost-music-assistant-gate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Chrome fetches manifest.webmanifest, sw.js, favicon, and the icons it
references with credentials: 'omit' (no crossorigin=use-credentials
on MA's <link rel="manifest">), so the user's Authentik cookie never
reaches the gate on those requests. Without this skip, the gate 302s
the anonymous fetch and Chrome's installability check fails silently
- the install prompt stops being offered.
Skipped paths are static branding/manifest only; HTML, JS bundle,
/api, and /ws remain gated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dedicated paperless-gate-shaped outpost (music-assistant-gate) fronts
MA at the WAN edge. MA's own HA-delegated login inside the UI remains
the primary access decision; the gate adds defense in depth.
No egress CNP needed: MA is hostNetwork and the authentik baseline
already allows toEntities: [host, remote-node].
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The prepare-dirs initContainer chowned /consume/scanner to 1000:1000
mode 2775 and only granted u:1000:rwx via ACL. The earlier commit
assumed fauria/vsftpd's entrypoint would chown the subPath mount back
to ftp:ftp on every start (ACL surviving chown), restoring write
access for the ftp user. In practice the entrypoint's chown didn't
reach the mounted PVC subPath, so the tree stayed 1000:1000 with the
ftp user (uid 14, the vsftpd virtual-user → guest mapping target)
falling through to other::r-x. Every STOR returned 553 since the
ACL deploy.
Own the tree as ftp:ftp explicitly in the initContainer instead of
relying on the image's chown side effect. The setfacl line continues
to grant paperless u:1000:rwx (access + default) so the consumer can
read and delete files.
Replaces the legacy paperless-syncthing-1 docker container on VM 111
that the 2026-05-09 cutover stopped. Phones (Pixel 10 Pro) and
laptop (XPS 13-9310) sync into a syncthing/ subdirectory on the
existing paperless-consume PVC; paperless's recursive consumer picks
files up.
- StatefulSet runAsUser 1000 — matches paperless's uid, no ACL or
chown gymnastics required (the FTP scanner path already proved the
consume PVC works with mixed writers when uid lines up).
- Sync ports exposed via LoadBalancer at 10.2.40.85 with UniFi-only
DNS (syncthing.paperless.renner.uno); cloudflare label deliberately
omitted (sync is TCP+UDP, not HTTPS).
- UI (8384) is ClusterIP-only on first deploy. Set the admin password
via kubectl port-forward before publishing it; no Ingress yet.
- allow-syncthing-sync CNP opens 22000 TCP+UDP and 21027 UDP from
world/host/remote-node; the namespace default-deny otherwise blocks
external ingress.
When adding the paperless-consume folder in the Syncthing UI, set
folder type to 'Receive Only' on the cluster side — paperless's
consume-then-delete would otherwise propagate deletions back to the
phones.
ENABLE_HTTP_REMOTE_USER + ENABLE_HTTP_REMOTE_USER_API makes the
paperless.auth middleware log 'No authentication provided from
private IP …' on every API request without the X-Authentik-Username
header. The readinessProbe hitting /api/ every 10 s and the liveness
probe every 30 s flooded the log; the actual gate-outpost-fronted
traffic was lost in the noise.
paperless-ngx has no unauthenticated health endpoint, so a TCP probe
on the gunicorn port is the closest non-invasive equivalent. Trades
HTTP-level health for log readability — acceptable since gunicorn
binding the port is the same signal we previously got from /api/.
Bridge generates a self-signed IMAP cert with SAN=IP:127.0.0.1 only;
paperless-ngx's mail.py uses ssl.create_default_context() with no
hostname-check opt-out, so connecting via 'protonmail-bridge'
(K8s DNS) fails CN/SAN verification.
Loop closed inside the paperless pod:
- initContainer fetches the live bridge cert via openssl s_client
(STARTTLS, port 143) into an emptyDir; re-runs on every rollout so
cert regeneration only requires a paperless restart.
- bridge-tls-proxy sidecar (alpine/socat:1.8.0.3) listens on
127.0.0.1:11143 and forwards to protonmail-bridge:143. Hostname
verification matches because paperless connects via the literal
127.0.0.1 in the cert's SAN.
- paperless gets PAPERLESS_EMAIL_CERTIFICATE_LOCATION pointing at
the captured cert.
Runtime DB change still required: MailAccount.imap_server=127.0.0.1
and imap_port=11143.
The previous check ('at least one file exists') only proved the
restored fs was mountable, not that any meaningful fraction of the
data made the round trip. Now: per restored PVC, take a fresh CSI
snapshot of the live source PVC, mount it RO via the same noload SC
the data-mover uses, count regular files, and compare to the restored
count. Pass if the two are within 10% of each other — generous enough
to absorb a day of source drift, tight enough to catch silent
half-data losses.
Source-side artifacts (snapshot + tmp PVC + probe pod) live in the
source namespace and are torn down on every return path via a trap.
RBAC widened to grant cluster-wide volumesnapshots and PVC create.
LAN-only A record so the brother scanner can target a hostname instead
of the raw 10.2.40.82 LB IP. Picked up by the UniFi external-dns
instance via the standard `external-dns.alpha.kubernetes.io/hostname`
annotation; the cloudflare-scoped instance only follows the
external-dns/cloudflare label, which we deliberately don't set
(clear-text FTP must not traverse the cargo tunnel).
fauria/vsftpd's entrypoint runs `chown -R ftp:ftp /home/vsftpd` on
every container start, stomping the initContainer's chown to 1000:1000.
Files created by vsftpd end up uid 14:gid 50 mode 0600, so paperless
(uid 1000) gets EACCES when the consumer scans /usr/src/paperless/
consume/scanner/.
POSIX ACLs survive chown. The initContainer now also installs `acl`
and applies u:1000:rwx as both access ACL (existing entries) and
default ACL (inherited by all future uploads + mkdir). cephfs supports
ACLs by default.
vsftpd uses virtual users (guest_enable=YES, pam_service_name=vsftpd_virtual)
and chroots each into /home/vsftpd/$USER. Without CAP_SYS_CHROOT the
chroot() syscall returns EPERM and login fails immediately with
'500 OOPS: chroot' — confirmed by 0x4c3 CapBnd missing bit 18.
The Cilium L2 announcer leader-elects the lease holder independently of
pod locality. With ETP=Local, whenever the lease landed on a node
without a local backend (here: lease on k8s-2, pod on k8s-1) the LB IP
ARP-resolved successfully but every TCP SYN got RST'd. Same failure
mode the wiki documents from the alloy-syslog incident on 2026-05-02.
Cluster matches the convention used by every other cross-VLAN LB
service in the cluster; the only documented ETP=Local exception is
home-assistant on the iot VLAN where v30iot is L2-shared with the nodes.
The allow-ftp-scanner CNP permits world+host+remote-node, so the
masquerade-induced source-identity shift doesn't break the policy.
The ClusterRole was missing velero.io/datauploads, so kubectl get
dataupload returned no items (Forbidden silenced) and the script
skipped every service as 'no-PVC schedule'. Add the missing verb and
make the script differentiate 'truly zero DataUploads' from 'list
errored' so a future RBAC regression fails loudly instead.
Velero v1.18 does not populate .status.csiVolumeSnapshotsAttempted, so
the prior no-PVC skip path matched every backup and the script never
ran a real restore. Count DataUpload children labelled with the
backup's name instead.
Also accept PartiallyFailed source backups: Velero restores only the
DataUploads that completed, so a partial source still produces a
valid restore for the PVCs that did upload. Excluding them would
silently let the test miss the case where one PVC's DataUpload kept
failing (e.g. the ext4-noload window before that fix).
The annotation filter caused records on annotated ingresses to flap
once the UniFi external-dns instance also managed them: external-dns
turns annotations in its prefix namespace into ProviderSpecific on the
desired endpoint, the kashalls UniFi webhook does not implement
AdjustEndpoints to strip Cloudflare-only properties, and Records()
returns observed endpoints with empty ProviderSpecific — so plan
computed an Update every reconcile and the records cycled in UDM.
Labels are not translated into ProviderSpecific, so swapping the
selector from `external-dns.alpha.kubernetes.io/cloudflare-proxied`
to a plain `external-dns/cloudflare=true` label removes the round
trip. The Cloudflare instance still proxies all matched records
because of the existing `--cloudflare-proxied` extraArg default.
Touches the seven resources that use the selector: the cloudflare
external-dns HelmRelease and the six ingresses currently published
through the cargo tunnel (audiobookshelf, gatus, home-assistant,
immich, paperless, authentik-server).
Drops the Woodpecker pipeline. Test runs from a Job inside the cluster
on a weekly schedule (Sun 02:00 UTC), keeping the verification path
contained: no external CI dependency, no kubeconfig leaving the
cluster. Script lives next to the rest of the velero infra and is
mounted into the CronJob via configMapGenerator.
Discovers every velero.io Schedule labeled
app.kubernetes.io/part-of=homelab-backup, restores yesterday's actual
Completed Backup for each into <ns>-restore-test, mounts each restored
PVC, and confirms files exist. Sequential — peak storage cost is the
size of the largest single service. Backup phase=Completed alone never
proved the bytes were restorable; this closes that gap.
Temporary — investigating per-cycle update_old/update_new churn for a
subset of *.renner.uno records. external-dns at debug logs the plan
diff with the offending field, which is what we need.
Default 5s causes the external-dns container to time out on
GET /records (webhook fetches 129 records from UDM and the
JSON write doesn't complete in time), driving CrashLoopBackOff.
Same shape as the immich rollout, with the same auth-side reasoning:
ABS already has native OIDC against auth.renner.uno (the
NODE_EXTRA_CA_CERTS plumbing in deployment.yaml exists for exactly
that token-exchange path), so SSO is handled at the app layer. A
forward-auth gate doesn't fit because the upstream ABS PR for trusted-
header auth was closed (advplyr/audiobookshelf#1109) and the official
mobile app is documented as not working behind a redirect-style proxy
(advplyr/audiobookshelf-app#1155, "not planned"). What's added here
is just public reachability.
Two pieces:
1. external-dns annotation on the ABS Ingress so the cloudflare-scoped
external-dns instance creates CNAMEs to the cargo tunnel for both
hostnames in the rule list.
2. cloudflared configmap entry for audiobooks.renner.uno only. The
list is an explicit allow with a 404 catch-all, so the
.k.renner.uno LAN alias stays tunnel-404 — no public-internet use
case for it, and audiobooks.k.renner.uno is the local-DNS path
that bypasses CF's 100 MiB body cap for large uploads.
cloudflared mounts the configmap via subPath, so a manual
`kubectl -n ingress rollout restart deploy/cloudflared` is required
after Flux reconciles for live pods to pick up the new ingress list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous comment framed hostNetwork as "required for mDNS / uPnP /
DLNA / Chromecast discovery and for SlimProto." That's the wrong
abstraction layer. Discovery happens at the application layer; what
hostNetwork actually buys us is L2 adjacency with the audio clients
on the IoT VLAN (v30iot, attached via the edge node's second NIC).
The hard requirement is that Snapcast / SlimProto / Chromecast group
members must share an L2 broadcast domain — routing the audio path
through the CNI overlay breaks the sync timing budget. mDNS/uPnP/DLNA
visibility is a side-effect of L2 attachment, not the reason for it.
Comment-only change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backups of continuously-written ext4 RBD volumes (influxdb-influxdb2,
paperless/protonmail-bridge) wedge for 4h then time out: the data-mover
exposer pod cannot mount the snapshot RO because ext4 has a dirty
journal it cannot replay under read-only. The new storage class clones
ceph-block and adds noload so journal recovery is skipped on the
exposer's snapshot mount; source PVCs are unaffected.
Same shape as the paperless rollout, minus the Authentik gate trust
boundary — Immich has no trusted-header auth path (only OAuth2/OIDC
and password), so the gate-in-front-of-app pattern doesn't fit, and
the mobile/CLI x-api-key clients would break under a forward-auth
proxy. The OIDC integration against auth.renner.uno already covers
SSO; what's added here is just public reachability.
Two pieces:
1. external-dns annotation on the immich Ingress so the cloudflare-
scoped external-dns instance creates CNAMEs to the cargo tunnel
for all three hostnames in the rule list.
2. cloudflared configmap entry for immich.renner.uno only. The list
is an explicit allow with a 404 catch-all, so the .k.renner.uno
LAN alias and the legacy photos.renner.uno stay tunnel-404 — no
public-internet use case for either, and immich.k.renner.uno is
the local-DNS path mobile clients already use to bypass CF
entirely (relevant since CF's free/pro plans cap request bodies
at 100 MiB, which would otherwise break large video uploads).
cloudflared mounts the configmap via subPath, so a manual
`kubectl -n ingress rollout restart deploy/cloudflared` is required
after Flux reconciles for live pods to pick up the new ingress list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
k.rennerandClaude Opus 4.7
security-bot