Compare commits

..
Author SHA1 Message Date
claude-botandClaude Opus 5 9a521568c0 fix(dns): migrate external-dns hostname annotations off the alpha prefix
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
external-dns v0.22.0 changed the default --annotation-prefix from
external-dns.alpha.kubernetes.io/ to external-dns.kubernetes.io/ with no
fallback, and its release notes lead with the consequence: "This change can
delete all your DNS records."

The chart floated onto it at 2026-09-11 23:47:56 (appVersion 0.21.0 ->
0.22.0). Two hours later, at 2026-09-12 01:47:59, the unifi provider logged
`client deleted record` for every name that depended on the annotation:

  ha.renner.uno       mqtt.renner.uno     ldap.renner.uno
  syslog.renner.uno   ftp.paperless.renner.uno
  syncthing.paperless.renner.uno

A, AAAA and the TXT ownership records, and no `client created record` since.
Every one of those is a LoadBalancer Service, whose hostname external-dns
can only learn from the annotation — so the source went silent and
--policy=sync removed the records. Ingress-sourced names take their host
from spec.rules[].host and were untouched, which is exactly the split
observed: source_endpoints_total is 36 with source_errors_total 0.

These six annotations were the only alpha-prefixed external-dns annotations
in the repo, so moving them to the current prefix is the whole fix. Pinning
the prefix back with --annotation-prefix would keep us on a deprecated
default that upstream intends to drop.

Scanning has been broken since: the Brother resolves ftp.paperless.renner.uno
rather than using the address.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:00:30 +02:00
security-bot 3b3ed4fbc4 chore(image-automation): bump gitea.renner.uno/k.renner/vsftpd:build.1789630783
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-09-17 07:43:07 +00:00
k.renner c40466a3e6 Merge pull request 'fix(paperless): stop the ftp-scanner probe from crashing vsftpd' (#862) from fix/ftp-scanner-probe into master
ci/woodpecker/cron/issue-fix Pipeline was successful
Reviewed-on: k.renner/homelab#862
2026-09-17 07:39:25 +00:00
k.renner ff0b3434c1 Merge pull request 'fix(libation): persist refreshed Audible tokens back to the PVC' (#861) from fix/libation-accounts-persistence into master
Reviewed-on: k.renner/homelab#861
2026-09-17 07:39:07 +00:00
claude-botandClaude Opus 5 2cfebc2f8b refactor(paperless): drop the duplicated PASV address from ftp-scanner
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The LB address was declared twice: on the Service as io.cilium/lb-ipam-ips,
and again as a literal PASV_ADDRESS in the ConfigMap generator.

The copy was not doing anything. vsftpd listens on a single IPv6 socket, so
an IPv4 client arrives as ::ffff:a.b.c.d and vsftpd answers PASV with
0.0.0.0 regardless of the configured address — the only such session in 30
days of logs got `227 Entering Passive Mode (0,0,0,0,82,109)` while
PASV_ADDRESS=10.2.99.82 was set. IPv6 clients negotiate EPSV, which carries
a port and no address at all; that is what the scanner uses.

Removing the literal falls back to the image default of 0.0.0.0, which is
the reply clients already receive, so behaviour is unchanged and the address
now exists in exactly one place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 08:56:51 +02:00
claude-botandClaude Opus 5 3481e43032 fix(paperless): stop the ftp-scanner probe from crashing vsftpd
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
vsftpd 3.0.5 segfaults its standalone listener on roughly 0.3% of the
connections it accepts: the parent closes the accepted socket before it
registers the forked child's pid, so a child that exits inside that window
makes the SIGCHLD path look up an unregistered pid, get NULL, and
dereference it in hash_ip(). PID 1 is vsftpd, so the segfault takes the
container with it.

The tcpSocket probes added in 469cdbc2 opened ~8600 connections a day
against that, which turned a latent crash into a restart every ~25 minutes
and left the scanner hitting CrashLoopBackOff windows. Measured on the live
pod: 52 restarts in 21h, one node-side `vsftpd[...]: segfault at 0` per
restart, starting the hour the probes rolled out.

Assert the listening socket from inside the container instead. The probe
still catches the failure it was added for — vsftpd not serving port 21 —
without being a client of the process it watches.

vsftpd 3.0.5 is upstream's last release (2021) and Alpine ships it
unpatched in 3.21 (3.0.5-r2) and edge (3.0.5-r3), so there is no version
bump that fixes this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 08:52:14 +02:00
security-bot 98475a47e2 chore(image-automation): bump ghcr.io/advplyr/audiobookshelf:2.36.1
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-09-16 22:50:50 +00:00
security-bot 6f20955e3b chore(image-automation): bump ghcr.io/usetrmnl/terminus:0.74.0
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-09-16 19:51:35 +00:00
claude-botandClaude Opus 5 92ac021931 feat(libation): seed the token master key onto the PVC
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
TokenStorageMethod is Encrypted and the container has no OS secret store,
so LibationCli falls back to minting a portable master key at
/config-internal/libation-master.key -- a path that is not a mounted
volume. The key is therefore new on every run and discarded at exit,
which is why the entrypoint logs LAST-RESORT on each pass.

The entrypoint looks for the key at ${LIBATION_CONFIG_DIR}/
libation-master.key, i.e. on the PVC, and copies it into place when
found. The key is now stored in OpenBao alongside the other bootstrap
files, so seed it like them: without it a rebuilt PVC would hold tokens
encrypted under a key that no longer exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 18:47:24 +02:00
claude-botandClaude Opus 5 5bbca6a162 fix(libation): persist refreshed Audible tokens back to the PVC
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The image entrypoint stages config into /config-internal, which is not a
mounted volume, and it does so by copy:

    init_config_file() {
      ...
      cp ${FULLPATH} ${LIBATION_CONFIG_INTERNAL}/

Only LibationContext.db is symlinked to the PVC. AccountsSettings.json is
therefore read from the PVC at startup and written back nowhere, so every
token refresh Libation performs is discarded when the container exits. On
the live PVC the file still carried its 2026-05-14 seed timestamp, four
months after deployment, while the DB beside it was current.

Pass args instead of letting the entrypoint run its own loop. The
entrypoint completes its setup and then execs what it is given, so the
staged environment is intact; an EXIT trap copies the refreshed file back
to the PVC whether the run succeeds or fails.

This was found while recovering from Audible refusing content licenses
for this account. Re-registering the account fixed the refusals; without
this change the fresh registration would have been discarded on the next
run exactly like every refresh before it.

Also corrects the init container comment, which claimed the sentinel
preserved token refreshes. It preserves the DB; it never preserved the
tokens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 18:26:49 +02:00
k.renner edb07a344e Merge pull request 'fix(libation): pin image to 14.2.1 by digest' (#860) from fix/libation-pin-14.2.1 into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#860
2026-09-16 15:51:49 +00:00
claude-botandClaude Opus 5 260f548e67 fix(libation): pin image to 14.2.1 by digest
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The CronJob ran `rmcrackan/libation:latest` with imagePullPolicy Always
but was stuck on 13.7.11 (2026-08-24). Spegel answers containerd's tag
resolution from its peer cache -- a 200 from the mirror is a hit, so
upstream is never consulted and the tag never advances. Verified: the
pod's imageID was sha256:946177d3 (= tag 13.7.11) while Docker Hub's
:latest pointed at sha256:6f9d8ba1 (= 14.2.1), and querying a Spegel
pod's registry port directly returned the stale digest for that tag.

Audible changed device registration in early September; upstream shipped
an experimental fix in 14.1 and the real fix in 14.2, so the stale image
predates both. Since 2026-09-15 three newly purchased titles fail with
"Content license denied" and the library has not grown.

A digest reference is immune to the mirror's tag cache. Note that the
upstream maintainer's remedy also requires removing and re-adding the
Audible account with an interactive login, which this change does not
and cannot perform.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 17:30:37 +02:00
k.renner ef6bc24491 Merge pull request 'feat(ceph): serve reads from the local OSD (#576)' (#859) from feat/ceph-read-affinity into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#859
2026-09-16 12:25:51 +00:00
10 changed files with 51 additions and 36 deletions
@@ -70,7 +70,7 @@ spec:
mountPath: /ca
containers:
- name: audiobookshelf
image: ghcr.io/advplyr/audiobookshelf:2.36.0 # {"$imagepolicy": "flux-system:audiobookshelf"}
image: ghcr.io/advplyr/audiobookshelf:2.36.1 # {"$imagepolicy": "flux-system:audiobookshelf"}
ports:
- name: http
containerPort: 80
@@ -40,7 +40,7 @@ metadata:
# so this Service produces no public record — LDAP stays
# LAN-internal, which matches the threat model of the cleartext
# bind on 389.
external-dns.alpha.kubernetes.io/hostname: ldap.renner.uno
external-dns.kubernetes.io/hostname: ldap.renner.uno
spec:
type: LoadBalancer
externalTrafficPolicy: Local
@@ -18,7 +18,7 @@ metadata:
lb-pool: bgp
annotations:
lbipam.cilium.io/ips: "10.2.99.87,fd99::87"
external-dns.alpha.kubernetes.io/hostname: ha.renner.uno
external-dns.kubernetes.io/hostname: ha.renner.uno
spec:
type: LoadBalancer
externalTrafficPolicy: Local
@@ -19,7 +19,7 @@ metadata:
# LAN can connect with `mqtt.renner.uno` (matches the broker cert
# SAN) instead of the bare IP. No Cloudflare label — kept off the
# public DNS; UniFi serves a split-DNS answer for LAN clients.
external-dns.alpha.kubernetes.io/hostname: mqtt.renner.uno
external-dns.kubernetes.io/hostname: mqtt.renner.uno
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+27 -4
View File
@@ -42,9 +42,11 @@ spec:
type: RuntimeDefault
initContainers:
# Seed the libation-config PVC from the bootstrap secret on
# first run only — sentinel file gates re-runs so token
# refreshes in AccountsSettings.json and DB updates from
# subsequent liberate passes are preserved.
# first run only — sentinel file gates re-runs so DB updates
# from subsequent liberate passes are preserved. Refreshed
# Audible tokens are preserved by the container args, not by
# this sentinel: the entrypoint copies AccountsSettings.json
# into ephemeral /config-internal rather than symlinking it.
- name: seed-config
image: busybox:1.37.0
command: ["/bin/sh", "-eu", "-c"]
@@ -66,6 +68,11 @@ spec:
install -m 660 /bootstrap/AccountsSettings.json /libation-state/AccountsSettings.json
install -m 660 /bootstrap/Settings.json /libation-state/Settings.json
install -m 660 /bootstrap/LibationContext.db /libation-state/LibationContext.db
# Without this key the seeded tokens cannot be decrypted:
# TokenStorageMethod is Encrypted and the container has no
# OS secret store, so a missing key file makes the
# entrypoint mint a throwaway one on every run.
install -m 660 /bootstrap/libation-master.key /libation-state/libation-master.key
touch /libation-state/.bootstrapped
ls -la /libation-state
volumeMounts:
@@ -85,7 +92,23 @@ spec:
type: RuntimeDefault
containers:
- name: libation
image: docker.io/rmcrackan/libation:latest
# Pinned by digest: Spegel caches mutable tag resolutions, so
# :latest stayed on 13.7.11 despite imagePullPolicy Always.
image: docker.io/rmcrackan/libation:14.2.1@sha256:6f9d8ba19dc3803dd6ba295ab5a7abfb6ff3bc06286eebf4ecd8ccbe43245888
# The entrypoint stages config into ephemeral /config-internal
# by copy (only the DB is symlinked to the PVC), so Audible
# tokens refreshed during a run are lost at container exit.
# Passing args makes the entrypoint set up the environment and
# then exec this instead of its own loop; the trap copies the
# refreshed file back to the PVC on every exit path.
args:
- /bin/sh
- -c
- |
trap 'cp /config-internal/AccountsSettings.json /libation-state/AccountsSettings.json' EXIT
set -e
/libation/LibationCli scan
/libation/LibationCli liberate
env:
- name: LIBATION_CONFIG_DIR
value: /libation-state
@@ -232,7 +232,7 @@ metadata:
lb-pool: bgp
annotations:
io.cilium/lb-ipam-ips: "10.2.99.84,fd99::84"
external-dns.alpha.kubernetes.io/hostname: syslog.renner.uno
external-dns.kubernetes.io/hostname: syslog.renner.uno
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+10 -15
View File
@@ -43,7 +43,7 @@ spec:
mountPath: /consume
containers:
- name: vsftpd
image: gitea.renner.uno/k.renner/vsftpd:build.1784731469 # {"$imagepolicy": "flux-system:vsftpd"}
image: gitea.renner.uno/k.renner/vsftpd:build.1789630783 # {"$imagepolicy": "flux-system:vsftpd"}
ports:
- name: ftp-ctrl
containerPort: 21
@@ -70,24 +70,19 @@ spec:
- name: consume
mountPath: /home/scanner
subPath: scanner
# FTP has no health verb, so the control port accepting a connection
# is what can be asserted. It is enough for the failure that matters:
# if vsftpd dies the scanner cannot deposit anything and nothing else
# in the chain notices, because paperless only ever sees an empty
# consume directory — which is indistinguishable from nobody having
# scanned.
#
# Only the control port is probed. The passive range is negotiated
# per transfer and is not listening at rest.
# Asserts the control socket is listening without opening an FTP
# connection: vsftpd 3.0.5 segfaults its listener on a fraction of
# the connections it accepts. The passive range is negotiated per
# transfer and is not listening at rest, so it is not probed.
livenessProbe:
tcpSocket:
port: ftp-ctrl
exec:
command: ["sh", "-c", "netstat -lnt | grep -q ':21 '"]
initialDelaySeconds: 15
periodSeconds: 30
timeoutSeconds: 5
readinessProbe:
tcpSocket:
port: ftp-ctrl
exec:
command: ["sh", "-c", "netstat -lnt | grep -q ':21 '"]
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
@@ -124,7 +119,7 @@ metadata:
# label — FTP must not be exposed via the cargo tunnel (clear-text
# protocol, public exposure not appropriate). The brother scanner
# uses this hostname instead of the LB IP.
external-dns.alpha.kubernetes.io/hostname: ftp.paperless.renner.uno
external-dns.kubernetes.io/hostname: ftp.paperless.renner.uno
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+6 -9
View File
@@ -25,18 +25,15 @@ resources:
configMapGenerator:
# vsftpd config for the ftp-scanner deployment. The content-hash name
# suffix is intentionally LEFT ENABLED (unlike home-assistant's nginx
# config): vsftpd templates pasv_address from PASV_ADDRESS at container
# start, so a changed value must roll the pod. With the hash suffix,
# editing any value here renames the generated ConfigMap, which mutates
# the Deployment's pod template and triggers an automatic rolling
# restart. Without it, an IP change (e.g. an LB re-allocation) silently
# leaves the running pod advertising the stale PASV address, which
# breaks classic-PASV clients (older Brother scanners) while EPSV
# clients keep working.
# config): vsftpd reads these into vsftpd.conf at container start, so a
# changed value must roll the pod, which the renamed ConfigMap does.
#
# No PASV_ADDRESS: the LB address is declared once, on the Service
# (io.cilium/lb-ipam-ips). The image default 0.0.0.0 tells PASV clients
# to reuse the control-connection address.
- name: ftp-scanner-config
namespace: paperless
literals:
- PASV_ADDRESS=10.2.99.82
- PASV_MIN_PORT=21100
- PASV_MAX_PORT=21104
- FTP_USER=scanner
+1 -1
View File
@@ -145,7 +145,7 @@ metadata:
lb-pool: bgp
annotations:
io.cilium/lb-ipam-ips: "10.2.99.85,fd99::85"
external-dns.alpha.kubernetes.io/hostname: syncthing.paperless.renner.uno
external-dns.kubernetes.io/hostname: syncthing.paperless.renner.uno
spec:
type: LoadBalancer
externalTrafficPolicy: Local
+2 -2
View File
@@ -134,7 +134,7 @@ spec:
# is a device-bricking class of risk. Bumps are a reviewed, manual
# act — see kubernetes/infrastructure/image-automation/README.md.
- name: terminus
image: ghcr.io/usetrmnl/terminus:0.73.0 # {"$imagepolicy": "flux-system:terminus"}
image: ghcr.io/usetrmnl/terminus:0.74.0 # {"$imagepolicy": "flux-system:terminus"}
ports:
- name: http
containerPort: 2345
@@ -197,7 +197,7 @@ spec:
capabilities:
drop: ["ALL"]
- name: sidekiq
image: ghcr.io/usetrmnl/terminus:0.73.0 # {"$imagepolicy": "flux-system:terminus"}
image: ghcr.io/usetrmnl/terminus:0.74.0 # {"$imagepolicy": "flux-system:terminus"}
command: ["bundle", "exec", "sidekiq", "-r", "./config/sidekiq.rb"]
envFrom:
- configMapRef: