feat(uptime-monitor): deploy IngressMonitorController with zero monitors (REN-6)

Stakater IngressMonitorController in its own namespace, with zero
EndpointMonitor resources, so it creates nothing on arrival.

- Vendored EndpointMonitor CRD from chart 2.2.13, image pinned to v2.2.13
  and deliberately kept out of Flux image automation.
- Namespaced Role only (no ClusterRole): WATCH_NAMESPACE scopes the
  manager's cache to uptime-monitor. Verbs derived from what v2.2.13
  actually calls; every omission is justified inline in rbac.yaml.
- The ExternalSecret now renders IMC's config.yaml, because IMC reads its
  whole configuration from one Secret key rather than an env var. Config
  stays in git; only the key is injected.
- Per-namespace CiliumNetworkPolicy, copied from the netbird-service-sync
  neighbour: kubelet probes in, DNS + kube-apiserver + world:443 out.
- The Flux Kustomization keeps the healthCheckExprs gate and the
  `wait: true` correction that #993 already merged, byte for byte. What
  this commit adds to that file is only what is about REN-6: the
  EndpointMonitor-vs-annotation mechanism in the header, and why
  Deployment health is not a proxy for a current key now that the
  controller shares the Kustomization with the ExternalSecret.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.woodpecker/build-ci-images.yml

@@ -377,37 +377,6 @@ steps:
           --destination gitea.renner.uno/gitea_admin/netbird-service-sync:$TAG \
           --destination gitea.renner.uno/gitea_admin/netbird-service-sync:latest
 
-  # wiki-index: embedding index over the Notes repo's wiki/, fed from Gitea and
-  # queried by ~/Notes/bin/wiki_rank.py. Runtime artifact like netbird-service-sync.
-  - name: build-wiki-index
-    image: registry.gitlab.com/gitlab-ci-utils/container-images/kaniko:v1.25.18-debug
-    directory: ci/images/wiki-index
-    depends_on: []
-    when:
-      - event: deployment
-      - event: push
-        path: ["ci/images/wiki-index/**", ".woodpecker/build-ci-images.yml"]
-    environment:
-      GITEA_USER: gitea_admin
-      GITEA_TOKEN:
-        from_secret: ci_images_token
-    commands:
-      - mkdir -p /kaniko/.docker
-      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
-      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
-      - TAG=$(date +%Y-%m-%d)
-      # Also a per-commit tag: several builds a day (branch pushes and the
-      # merge) all rewrite the dated tag, and the deployment pin raced them
-      # on 2026-09-20. The commit tag is immutable and needs no digest lookup.
-      - SHORT=$(printf '%s' "$CI_COMMIT_SHA" | cut -c1-8)
-      - |
-        /kaniko/executor \
-          --context . \
-          --dockerfile Containerfile \
-          --destination gitea.renner.uno/gitea_admin/wiki-index:$TAG \
-          --destination gitea.renner.uno/gitea_admin/wiki-index:$TAG-$SHORT \
-          --destination gitea.renner.uno/gitea_admin/wiki-index:latest
-
   # bird: BGP daemon for the pod-network mesh over pod0 (#576). Runtime
   # artifact like heating-can; same registry so the audit tracks it.
   - name: build-bird

.woodpecker/deploy-netbird-server.yml

@@ -4,9 +4,6 @@ when:
     branch: master
     path:
       - "ansible/roles/netbird-server/**"
-      # install-netbird-server.yml also applies the netbird-proxy-watchdog role,
-      # so changes to it must redeploy this host too.
-      - "ansible/roles/netbird-proxy-watchdog/**"
       - "ansible/plays/install-netbird-server.yml"
       - ".woodpecker/deploy-netbird-server.yml"
   - event: deployment

.woodpecker/wiki-maintenance.yml

@@ -4,7 +4,7 @@
 # script (structural checks, no model), then consolidate (wiki-consolidate
 # agent: verifies a cohort of notes against sources, repos and the live
 # cluster and edits them). The pipeline commits and pushes as Claude Bot; the
-# push webhook then refreshes the wiki-index service.
+# push webhook then refreshes the wiki service.
 #
 # The agents address the vault as ~/Notes and the repos as ~/source/<repo>, so
 # each agent step links those into the workspace instead of rewriting them.

ansible/plays/install-netbird-proxy.yml

@@ -1,5 +1,5 @@
 ---
-# Deploy a NetBird reverse-proxy HA cluster member (a second proxy host).
+# Deploy the NetBird reverse-proxy HA cluster members (the public edge).
 # Usage: ansible-playbook -i inventory/ plays/install-netbird-proxy.yml
 #
 # Prerequisites:
@@ -7,12 +7,11 @@
 #   listed under [netbird_proxy] in inventory/pve_hosts.yml. Docker is
 #   installed by the (Arch-aware) docker-install role.
 # - DNS A/AAAA for the host's own name (e.g. proxy-1.renner.uno) -> its IP.
-# - Uses the existing vault_netbird_proxy_token (shared with VPS-1) — no new
-#   token to generate; a shared token + same NB_PROXY_DOMAIN forms the cluster.
-# - AFTER deploy + verify: add this host's IP to the proxy domain so traffic
-#   round-robins across the cluster. Repoint proxy.renner.uno away from its
-#   CNAME to netbird.renner.uno (management host) to A/AAAA records listing
-#   BOTH proxy IPs (VPS-1 + this host).
+# - Uses the existing vault_netbird_proxy_token (shared by every member) — no
+#   new token to generate; a shared token + same NB_PROXY_DOMAIN forms the
+#   cluster.
+# - AFTER deploy + verify: add this host's A/AAAA to proxy.renner.uno so
+#   traffic round-robins across the cluster.
 
 - name: Deploy NetBird reverse-proxy cluster member
   hosts: netbird_proxy

ansible/plays/install-netbird-server.yml

@@ -18,4 +18,3 @@
     - common
     - docker-install
     - netbird-server
-    - netbird-proxy-watchdog

ansible/plays/rotate-netbird-secrets.yml

@@ -43,10 +43,10 @@
 #
 # WHAT IS NOT, and must not be added:
 #
-#   vault_netbird_proxy_token    LOOKS symmetric — it is in proxy.env on both
-#                                the server and the proxies, all Ansible
-#                                templates — but the management server does not
-#                                validate it from any of them. It is registered
+#   vault_netbird_proxy_token    LOOKS symmetric — it is in proxy.env on the
+#                                proxies, an Ansible template — but the
+#                                management server does not validate it from
+#                                any file. It is registered
 #                                in the management DATABASE. Rotating it changes
 #                                only what the proxies present, so every proxy
 #                                is locked out: "proxy auth failed: too many

ansible/roles/netbird-proxy-watchdog/defaults/main.yml

@@ -1,5 +1,5 @@
 ---
-# Self-healing watchdog for the NetBird reverse proxy (VPS-1 and proxy-1).
+# Self-healing watchdog for the NetBird reverse proxy on the proxy hosts.
 #
 # Why it exists: the proxy's embedded NetBird client pins each proxied
 # domain's route to a specific routing-peer identity. The netbird-ingress
@@ -20,9 +20,7 @@ netbird_watchdog_probe_hosts:
   - home.renner.uno
   - immich.renner.uno
 
-# Address the probe connects to. 127.0.0.1 reaches the local proxy on both
-# members: VPS-1 routes it via Traefik's HostSNI(*) TCP passthrough, proxy-1
-# binds :443 directly.
+# Address the probe connects to. The proxy binds :443 on the host directly.
 netbird_watchdog_probe_address: 127.0.0.1
 
 netbird_watchdog_container: netbird-proxy

ansible/roles/netbird-proxy-watchdog/tasks/main.yml

@@ -1,7 +1,7 @@
 ---
 # Deploys the reverse-proxy watchdog (see defaults/main.yml for why).
 
-- name: Ensure curl is installed (probe dependency; proxy-1 is Arch, VPS-1 Debian)
+- name: Ensure curl is installed (probe dependency)
   ansible.builtin.package:
     name: curl
     state: present

ansible/roles/netbird-proxy/defaults/main.yml

@@ -1,8 +1,8 @@
 ---
 # NetBird reverse-proxy HA cluster member (proxy-only host, e.g. proxy-1).
-# Joins the same proxy cluster as the VPS-1 proxy: identical NB_PROXY_DOMAIN
-# + the same token. NetBird load-balances/fails the cluster over via DNS
-# round-robin on the proxy domain (see plays/install-netbird-proxy.yml).
+# Every member uses the same NB_PROXY_DOMAIN + token. NetBird load-balances/
+# fails the cluster over via DNS round-robin on the proxy domain (see
+# plays/install-netbird-proxy.yml).
 
 # MUST match the netbird-server role's netbird_version — the reverse-proxy
 # release line has to track the management server it enrolls against.
@@ -11,11 +11,13 @@ netbird_version: "0.73.2"
 # Management server this proxy enrolls against (public API, TLS).
 netbird_server_host: netbird.renner.uno
 
-# Proxy domain — MUST be identical across every cluster member.
+# Proxy domain — MUST be identical across every cluster member, and must NOT
+# be a subdomain of the management domain (netbird.renner.uno) per NetBird
+# docs (TLS/routing conflicts).
 netbird_proxy_domain: "proxy.renner.uno"
 
-# Reuse VPS-1's proxy token — a shared NB_PROXY_TOKEN across members with the
-# same NB_PROXY_DOMAIN is a valid HA cluster, so no separate token is needed.
+# A shared NB_PROXY_TOKEN across members with the same NB_PROXY_DOMAIN is a
+# valid HA cluster, so one token serves every member.
 netbird_proxy_token: "{{ vault_netbird_proxy_token }}"
 
 # Relay auth secret — MUST be identical across every relay instance and the

ansible/roles/netbird-proxy/templates/docker-compose.yml.j2

@@ -1,8 +1,7 @@
 # NetBird reverse-proxy — HA cluster member on a proxy-only host.
-# Just the reverse-proxy container, bound directly to :443. No Traefik:
-# unlike VPS-1 (where the proxy shares :443 with management/signal/relay via
-# SNI routing), this host runs nothing else, so the proxy owns the port and
-# terminates TLS + answers its own tls-alpn-01 ACME challenges directly.
+# Just the reverse-proxy container, bound directly to :443. No Traefik: the
+# proxy owns the port and terminates TLS + answers its own tls-alpn-01 ACME
+# challenges directly.
 services:
   proxy:
     image: netbirdio/reverse-proxy:{{ netbird_version }}

ansible/roles/netbird-server/defaults/main.yml

@@ -23,9 +23,3 @@ netbird_datastore_key: "{{ vault_netbird_datastore_key }}"
 
 # Relay secret for WebSocket relay authentication
 netbird_relay_secret: "{{ vault_netbird_relay_secret }}"
-
-# Reverse proxy configuration
-# IMPORTANT: Proxy domain must NOT be a subdomain of the management domain
-# (netbird.renner.uno) per NetBird docs - causes TLS/routing conflicts
-netbird_proxy_domain: "proxy.renner.uno"
-netbird_proxy_token: "{{ vault_netbird_proxy_token }}"

ansible/roles/netbird-server/handlers/main.yml

@@ -16,7 +16,3 @@
 - name: Restart netbird coturn
   ansible.builtin.shell: |
     cd /opt/netbird && docker compose up -d --force-recreate coturn
-
-- name: Restart netbird proxy
-  ansible.builtin.shell: |
-    cd /opt/netbird && docker compose up -d --force-recreate proxy

ansible/roles/netbird-server/tasks/main.yml

@@ -36,7 +36,6 @@
     - /opt/netbird/management
     - /opt/netbird/signal
     - /opt/netbird/letsencrypt
-    - /opt/netbird/proxy/certs
 
 - name: Deploy docker-compose.yml
   ansible.builtin.template:
@@ -59,23 +58,83 @@
     mode: "0644"
   notify: Restart netbird coturn
 
-- name: Deploy proxy.env
-  ansible.builtin.template:
-    src: proxy.env.j2
-    dest: /opt/netbird/proxy.env
-    mode: "0644"
-  notify: Restart netbird proxy
+# The public edge is the proxy hosts (plays/install-netbird-proxy.yml); this
+# host runs no reverse proxy. The tasks below remove the one it used to run,
+# its watchdogs first so they cannot restart it mid-removal, and can go once
+# the role has run against every host that ever carried it.
+- name: Check for reverse-proxy watchdog units
+  ansible.builtin.stat:
+    path: "/etc/systemd/system/{{ item }}"
+  loop:
+    - netbird-proxy-watchdog.timer
+    - netbird-proxy-flood-watchdog.service
+  register: _proxy_watchdog_units
+
+- name: Stop and disable reverse-proxy watchdogs
+  ansible.builtin.systemd:
+    name: "{{ item.item }}"
+    state: stopped
+    enabled: false
+  loop: "{{ _proxy_watchdog_units.results }}"
+  loop_control:
+    label: "{{ item.item }}"
+  when: item.stat.exists
+
+- name: Remove reverse-proxy watchdog units and scripts
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - /etc/systemd/system/netbird-proxy-watchdog.timer
+    - /etc/systemd/system/netbird-proxy-watchdog.service
+    - /etc/systemd/system/netbird-proxy-flood-watchdog.service
+    - /usr/local/bin/netbird-proxy-watchdog
+    - /usr/local/bin/netbird-proxy-flood-watchdog
+    - /run/netbird-proxy-watchdog
+  register: _proxy_watchdog_files
+
+- name: Reload systemd after removing the watchdog units
+  ansible.builtin.systemd:
+    daemon_reload: true
+  when: _proxy_watchdog_files.changed
 
 - name: Pull NetBird container images
   ansible.builtin.shell: |
     cd /opt/netbird && docker compose pull
   changed_when: false
 
+# --remove-orphans removes containers of services no longer in the compose
+# file (the reverse proxy).
 - name: Start NetBird containers
   ansible.builtin.shell: |
-    cd /opt/netbird && docker compose up -d
+    cd /opt/netbird && docker compose up -d --remove-orphans
   register: docker_compose_result
-  changed_when: "'Started' in docker_compose_result.stdout or 'Creating' in docker_compose_result.stdout"
+  changed_when: >-
+    'Started' in docker_compose_result.stdout or
+    'Creating' in docker_compose_result.stdout or
+    'Removed' in docker_compose_result.stdout or
+    'Removed' in docker_compose_result.stderr
+
+- name: Remove reverse-proxy volumes
+  ansible.builtin.shell: |
+    set -eu
+    for v in netbird_netbird_proxy_certs netbird_netbird_proxy_data netbird_netbird_proxy_state; do
+      if [ -n "$(docker volume ls -q --filter "name=^${v}$")" ]; then
+        docker volume rm "$v"
+        echo "removed $v"
+      fi
+    done
+  register: _proxy_volumes
+  changed_when: "'removed' in _proxy_volumes.stdout"
+
+- name: Remove reverse-proxy config and state
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - /opt/netbird/proxy.env
+    - /opt/netbird/proxy
+    - /opt/netbird/proxy-state
 
 # The management store has no cluster-side backup. Pulling one put a root
 # key on this host that the cluster held, and this server's own state is

ansible/roles/netbird-server/templates/docker-compose.yml.j2

@@ -187,10 +187,9 @@ services:
       - "traefik.http.routers.netbird-management.service=netbird-management"
       - "traefik.http.services.netbird-management.loadbalancer.server.port=33073"
       - "traefik.http.services.netbird-management.loadbalancer.server.scheme=h2c"
-      # gRPC reverse-proxy service — remote reverse-proxy instances (e.g.
-      # proxy-1) connect here via NB_PROXY_MANAGEMENT_ADDRESS=https://host:443.
-      # The local proxy uses management:33073 directly, so it never needed
-      # this route; remote ones get a 404 without it. Same backend (h2c 33073).
+      # gRPC reverse-proxy service — the proxy hosts (proxy-1, proxy-2)
+      # connect here via NB_PROXY_MANAGEMENT_ADDRESS=https://host:443 and get
+      # a 404 without it. Same backend (h2c 33073).
       - "traefik.http.routers.netbird-proxyservice.rule=Host(`{{ netbird_server_host }}`) && PathPrefix(`/management.ProxyService/`)"
       - "traefik.http.routers.netbird-proxyservice.entrypoints=websecure"
       - "traefik.http.routers.netbird-proxyservice.tls=true"
@@ -221,37 +220,6 @@ services:
         max-size: "500m"
         max-file: "2"
 
-  # Reverse Proxy Service (per docs.netbird.io/selfhosted/migration/enable-reverse-proxy)
-  proxy:
-    image: netbirdio/reverse-proxy:{{ netbird_version }}
-    container_name: netbird-proxy
-    restart: unless-stopped
-    networks:
-      - netbird
-    depends_on:
-      - management
-    env_file:
-      - ./proxy.env
-    volumes:
-      - netbird_proxy_certs:/certs
-      - /opt/netbird/proxy-state:/var/lib/netbird
-    labels:
-      - "traefik.enable=true"
-      - "traefik.tcp.routers.proxy-passthrough.entrypoints=websecure"
-      - "traefik.tcp.routers.proxy-passthrough.rule=HostSNI(`*`)"
-      - "traefik.tcp.routers.proxy-passthrough.tls.passthrough=true"
-      - "traefik.tcp.routers.proxy-passthrough.service=proxy-tls"
-      - "traefik.tcp.routers.proxy-passthrough.priority=1"
-      - "traefik.tcp.services.proxy-tls.loadbalancer.server.port=8443"
-    logging:
-      driver: "json-file"
-      options:
-        max-size: "500m"
-        max-file: "2"
-
 networks:
   netbird:
     driver: bridge
-
-volumes:
-  netbird_proxy_certs:

ansible/roles/netbird-server/templates/proxy.env.j2

@@ -1,8 +0,0 @@
-NB_PROXY_DOMAIN={{ netbird_proxy_domain }}
-NB_PROXY_TOKEN={{ netbird_proxy_token }}
-NB_PROXY_MANAGEMENT_ADDRESS=http://management:33073
-NB_PROXY_ALLOW_INSECURE=true
-NB_PROXY_ADDRESS=:8443
-NB_PROXY_ACME_CERTIFICATES=true
-NB_PROXY_ACME_CHALLENGE_TYPE=tls-alpn-01
-NB_PROXY_CERTIFICATE_DIRECTORY=/certs

ci/images/wiki-index/Containerfile

@@ -1,10 +0,0 @@
-# wiki-index: embedding index over the Notes repo's wiki/, fed from Gitea,
-# queried by ~/Notes/bin/wiki_rank.py. Standard library only; runs unprivileged.
-FROM python:3.13-slim
-
-RUN useradd --system --no-create-home --shell /usr/sbin/nologin wikiindex
-COPY wiki_index.py /opt/wiki_index.py
-
-ENV PYTHONUNBUFFERED=1
-USER wikiindex
-ENTRYPOINT ["python3", "/opt/wiki_index.py"]

ci/images/wiki-index/test_wiki_index.py

@@ -1,178 +0,0 @@
-#!/usr/bin/env python3
-"""Unit tests for wiki_index. Run: python3 -m unittest test_wiki_index"""
-
-import hashlib
-import hmac
-import json
-import tempfile
-import unittest
-from pathlib import Path
-
-import wiki_index as wi
-
-NOTE = """---
-title: 'UDM intercepts DNS'
-type: wiki
-tags: [dns, unifi]
-aliases: [udm dns interception, dig 1.1.1.1 wrong]
----
-
-# UDM intercepts DNS
-
-The UDM answers every UDP/53 query itself.
-
-## Details
-
-More.
-"""
-
-
-class ParseTests(unittest.TestCase):
-    def test_note_text(self):
-        title, text = wi.note_text("udm-dns", NOTE)
-        self.assertEqual(title, "UDM intercepts DNS")
-        self.assertIn("udm dns UDM intercepts DNS udm dns interception", text)
-        self.assertIn("dns unifi", text)
-        self.assertTrue(text.endswith("The UDM answers every UDP/53 query itself."))
-        self.assertLessEqual(len(text), wi.EMBED_CHARS)
-
-    def test_credentials_are_stripped(self):
-        original = wi.http
-        wi.http = lambda method, url, body=None, headers=None, timeout=60.0: (
-            {"auth": {"client_token": "t"}} if url.endswith("/login")
-            else {"data": {"data": {"gitea_token": "abc\n", "webhook_secret": " s3cret\n"}}})
-        original_sa = wi.SA_TOKEN
-        try:
-            with tempfile.TemporaryDirectory() as d:
-                wi.SA_TOKEN = Path(d) / "token"
-                wi.SA_TOKEN.write_text("jwt\n")
-                self.assertEqual(wi.Credentials().fetch(), {"gitea_token": "abc", "webhook_secret": "s3cret"})
-        finally:
-            wi.http, wi.SA_TOKEN = original, original_sa
-
-    def test_tree_paging_stops_on_short_last_page(self):
-        # Gitea: truncated=true on every page including the last, "tree": null past it.
-        pages = {
-            1: {"tree": [{"path": f"wiki/n{i}.md", "type": "blob", "sha": f"s{i}"} for i in range(1000)], "truncated": True, "total_count": 1192},
-            2: {"tree": [{"path": f"wiki/n{i}.md", "type": "blob", "sha": f"s{i}"} for i in range(1000, 1192)] + [{"path": "raw/x.md", "type": "blob", "sha": "r"}, {"path": "wiki/sub/y.md", "type": "blob", "sha": "y"}], "truncated": True, "total_count": 1192},
-            3: {"tree": None, "truncated": False, "total_count": 1192},
-        }
-        calls = []
-        original = wi.http
-
-        def fake_http(method, url, body=None, headers=None, timeout=60.0):
-            page = int(url.rsplit("page=", 1)[1])
-            calls.append(page)
-            return pages[page]
-
-        wi.http = fake_http
-        try:
-            tree = wi.Gitea("t").tree("abc")
-        finally:
-            wi.http = original
-        self.assertEqual(calls, [1, 2])
-        self.assertEqual(len(tree), 1192)
-        self.assertNotIn("raw/x.md", tree)
-        self.assertNotIn("wiki/sub/y.md", tree)
-        self.assertEqual(tree["wiki/n1191.md"], "s1191")
-
-    def test_signature(self):
-        body = b'{"ref":"refs/heads/master"}'
-        sig = hmac.new(b"s3cret", body, hashlib.sha256).hexdigest()
-        self.assertTrue(wi.verify_signature("s3cret", body, sig))
-        self.assertFalse(wi.verify_signature("other", body, sig))
-        self.assertFalse(wi.verify_signature("s3cret", body, ""))
-
-
-class FakeGitea:
-    def __init__(self, head, tree, files):
-        self._head, self._tree, self.files = head, tree, files
-        self.raw_calls = []
-
-    def head(self):
-        return self._head
-
-    def tree(self, commit):
-        return self._tree
-
-    def raw(self, path, ref):
-        self.raw_calls.append(path)
-        return self.files[path]
-
-
-class IndexTests(unittest.TestCase):
-    def setUp(self):
-        self.tmp = tempfile.TemporaryDirectory()
-        self.path = Path(self.tmp.name) / "index.json"
-        self.original_embed = wi.embed
-        self.embed_calls = []
-
-        def fake_embed(texts, batch=2):
-            self.embed_calls.append(list(texts))
-            return [wi.unit([1.0, float(len(t) % 7)]) for t in texts]
-
-        wi.embed = fake_embed
-
-    def tearDown(self):
-        wi.embed = self.original_embed
-        self.tmp.cleanup()
-
-    def test_sync_is_incremental_and_persists(self):
-        files = {"wiki/udm-dns.md": NOTE, "wiki/ceph.md": "---\ntitle: Ceph\n---\n# Ceph\n\nOSD.\n", "wiki/_index.md": "---\ntitle: Wiki\n---\n"}
-        tree = {"wiki/udm-dns.md": "aaa", "wiki/ceph.md": "bbb", "wiki/_index.md": "ccc"}
-        g = FakeGitea("c1", tree, files)
-        idx = wi.Index(self.path)
-        s = idx.sync(g)
-        self.assertEqual((s["changed"], s["removed"], s["notes"]), (2, 0, 2))
-        self.assertEqual(set(idx.notes), {"udm-dns", "ceph"})
-        self.assertEqual(idx.notes["udm-dns"]["title"], "UDM intercepts DNS")
-        self.assertTrue(self.path.exists())
-
-        # head unchanged: nothing fetched
-        s = idx.sync(g)
-        self.assertEqual(s.get("skipped"), "head unchanged")
-        self.assertEqual(len(g.raw_calls), 2)
-
-        # one note changed, one removed, new head
-        g2 = FakeGitea("c2", {"wiki/udm-dns.md": "aaa2"}, {"wiki/udm-dns.md": NOTE.replace("itself", "itself!")})
-        s = idx.sync(g2)
-        self.assertEqual((s["changed"], s["removed"], s["notes"]), (1, 1, 1))
-        self.assertEqual(g2.raw_calls, ["wiki/udm-dns.md"])
-
-        # restart loads the persisted index
-        idx2 = wi.Index(self.path)
-        self.assertEqual(idx2.commit, "c2")
-        self.assertEqual(set(idx2.notes), {"udm-dns"})
-
-    def test_rank_orders_by_cosine(self):
-        idx = wi.Index(self.path)
-        idx.notes = {"a": {"sha": "1", "title": "A", "vector": wi.unit([1.0, 0.0])},
-                     "b": {"sha": "2", "title": "B", "vector": wi.unit([0.0, 1.0])}}
-        wi.embed = lambda texts, batch=2: [wi.unit([0.2, 1.0])]
-        rows = idx.rank("q", k=5)
-        self.assertEqual([r["slug"] for r in rows], ["b", "a"])
-        self.assertGreater(rows[0]["score"], rows[1]["score"])
-
-    def test_status_reports_age_and_errors(self):
-        idx = wi.Index(self.path)
-        self.assertFalse(idx.status()["ready"])
-        g = FakeGitea("c1", {"wiki/a.md": "x"}, {"wiki/a.md": "---\ntitle: A\n---\n# A\n\nt\n"})
-        idx.sync(g)
-        st = idx.status()
-        self.assertTrue(st["ready"])
-        self.assertEqual(st["notes"], 1)
-        self.assertEqual(st["commit"], "c1")
-
-        class Broken(FakeGitea):
-            def head(self):
-                raise wi.ApiError("gitea down")
-
-        with self.assertRaises(wi.ApiError):
-            idx.sync(Broken("c1", {}, {}))
-        self.assertIn("gitea down", idx.status()["last_error"])
-        self.assertEqual(idx.status()["notes"], 1)  # last good index kept
-        self.assertTrue(idx.status()["ready"])  # and still served
-
-
-if __name__ == "__main__":
-    unittest.main()

ci/images/wiki-index/wiki_index.py

@@ -1,408 +0,0 @@
-#!/usr/bin/env python3
-"""wiki-index: embedding index over the Notes repo's wiki/, fed from Gitea.
-
-Source of truth is the notes repository on Gitea. The index holds, per wiki
-note, the git blob sha, title and a bge-m3 vector of slug/title/aliases/tags/
-first paragraph. A sync fetches the repo tree, embeds every note whose blob sha
-changed (via wiki-embed, in-cluster) and drops notes that left the tree; it
-runs at start, on every push webhook from Gitea, and every RESYNC_SECONDS.
-
-HTTP:
-  POST /rank     {"query": str, "k": int}   -> {"rows": [{"slug","title","score"}], "index": {...}}
-  GET  /health   -> 200 always (liveness) with {"ready", "notes", "commit", "synced_at", "age_s", "syncing", "last_error"}
-  GET  /ready    -> 200 when an index exists, else 503 (readiness)
-  POST /webhook  Gitea push event, HMAC-SHA256 in X-Gitea-Signature -> {"queued": true}
-
-Credentials (Gitea token, webhook secret) are read from OpenBao through the
-Kubernetes auth role on every use; they never land in a Secret or a log line.
-Standard library only.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import hmac
-import json
-import logging
-import os
-import re
-import sys
-import threading
-import time
-import urllib.error
-import urllib.parse
-import urllib.request
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-
-LOG = logging.getLogger("wiki-index")
-
-GITEA_URL = os.environ.get("GITEA_URL", "https://gitea.renner.uno").rstrip("/")
-NOTES_REPO = os.environ.get("NOTES_REPO", "k.renner/notes")
-NOTES_BRANCH = os.environ.get("NOTES_BRANCH", "master")
-WIKI_DIR = os.environ.get("WIKI_DIR", "wiki").strip("/")
-EMBED_URL = os.environ.get("EMBED_URL", "http://wiki-embed.ai.svc.cluster.local:8080").rstrip("/")
-EMBED_MODEL = os.environ.get("EMBED_MODEL", "bge-m3")
-EMBED_CHARS = int(os.environ.get("EMBED_CHARS", "1500"))
-INDEX_PATH = Path(os.environ.get("INDEX_PATH", "/data/index.json"))
-RESYNC_SECONDS = int(os.environ.get("RESYNC_SECONDS", "1800"))
-LISTEN_PORT = int(os.environ.get("LISTEN_PORT", "8080"))
-BAO_ADDR = os.environ.get("BAO_ADDR", "http://openbao.openbao.svc.cluster.local:8200").rstrip("/")
-BAO_ROLE = os.environ.get("BAO_ROLE", "wiki-index")
-BAO_SECRET_PATH = os.environ.get("BAO_SECRET_PATH", "homelab/data/apps/ai/wiki-index")
-SA_TOKEN = Path(os.environ.get("SA_TOKEN", "/var/run/secrets/kubernetes.io/serviceaccount/token"))
-
-
-class ApiError(Exception):
-    pass
-
-
-# ---------------------------------------------------------------- http helpers
-
-
-def http(method: str, url: str, body=None, headers=None, timeout: float = 60.0):
-    data = None
-    hdrs = dict(headers or {})
-    if body is not None:
-        data = json.dumps(body).encode("utf-8")
-        hdrs.setdefault("Content-Type", "application/json")
-    req = urllib.request.Request(url, data=data, method=method, headers=hdrs)
-    try:
-        with urllib.request.urlopen(req, timeout=timeout) as resp:
-            raw = resp.read()
-            ctype = resp.headers.get("Content-Type", "")
-    except urllib.error.HTTPError as e:
-        raise ApiError(f"{method} {url}: HTTP {e.code}: {e.read().decode('utf-8', 'replace')[:300]}") from e
-    except urllib.error.URLError as e:
-        raise ApiError(f"{method} {url}: {e.reason}") from e
-    if "json" in ctype:
-        return json.loads(raw.decode("utf-8"))
-    return raw
-
-
-# ---------------------------------------------------------------- credentials
-
-
-class Credentials:
-    """Gitea token and webhook secret from OpenBao, fetched per use."""
-
-    def __init__(self, static: dict | None = None):
-        self.static = static  # tests / local runs
-
-    def fetch(self) -> dict:
-        if self.static is not None:
-            return self.static
-        jwt = SA_TOKEN.read_text(encoding="utf-8").strip()
-        login = http("POST", f"{BAO_ADDR}/v1/auth/kubernetes/login", {"role": BAO_ROLE, "jwt": jwt})
-        token = login["auth"]["client_token"]
-        kv = http("GET", f"{BAO_ADDR}/v1/{BAO_SECRET_PATH}", headers={"X-Vault-Token": token})
-        data = {k: str(v).strip() for k, v in kv["data"]["data"].items()}  # `bao kv put key=@file` keeps the file's newline
-        for key in ("gitea_token", "webhook_secret"):
-            if not data.get(key):
-                raise ApiError(f"OpenBao {BAO_SECRET_PATH} has no key {key}")
-        return data
-
-
-# ---------------------------------------------------------------- gitea
-
-
-class Gitea:
-    def __init__(self, token: str, base: str = GITEA_URL, repo: str = NOTES_REPO, branch: str = NOTES_BRANCH):
-        self.base, self.repo, self.branch = base, repo, branch
-        self.headers = {"Authorization": f"token {token}"}
-
-    def api(self, path: str, **params):
-        q = ("?" + urllib.parse.urlencode(params)) if params else ""
-        return http("GET", f"{self.base}/api/v1/repos/{self.repo}{path}{q}", headers=self.headers)
-
-    def head(self) -> str:
-        return self.api(f"/branches/{urllib.parse.quote(self.branch, safe='')}")["commit"]["id"]
-
-    def tree(self, commit: str) -> dict[str, str]:
-        """path -> blob sha for every wiki note at that commit."""
-        out: dict[str, str] = {}
-        page, per_page, seen = 1, 1000, 0
-        while True:
-            t = self.api(f"/git/trees/{commit}", recursive="true", per_page=per_page, page=page)
-            entries = t.get("tree") or []  # Gitea answers "tree": null past the last page
-            for e in entries:
-                p = e.get("path", "")
-                if e.get("type") == "blob" and p.startswith(WIKI_DIR + "/") and p.endswith(".md") and "/" not in p[len(WIKI_DIR) + 1:]:
-                    out[p] = e["sha"]
-            seen += len(entries)
-            total = int(t.get("total_count") or 0)
-            if not entries or len(entries) < per_page or (total and seen >= total):
-                break
-            page += 1
-            if page > 50:
-                raise ApiError("tree paging runaway")
-        return out
-
-    def raw(self, path: str, ref: str) -> str:
-        data = http("GET", f"{self.base}/api/v1/repos/{self.repo}/raw/{urllib.parse.quote(path)}?ref={urllib.parse.quote(ref, safe='')}",
-                    headers=self.headers)
-        return data.decode("utf-8") if isinstance(data, bytes) else json.dumps(data)
-
-
-# ---------------------------------------------------------------- notes
-
-
-def parse_frontmatter(text: str) -> tuple[dict, str]:
-    m = re.match(r"^---[ \t]*\n(.*?)\n---[ \t]*(?:\n|$)", text, re.DOTALL)
-    if not m:
-        return {}, text
-    fm: dict = {}
-    for line in m.group(1).splitlines():
-        mm = re.match(r"^([A-Za-z_][\w-]*):\s*(.*)$", line)
-        if mm:
-            fm[mm.group(1)] = mm.group(2).strip()
-    return fm, text[m.end():]
-
-
-def as_list(value: str) -> list[str]:
-    s = (value or "").strip()
-    if s.startswith("[") and s.endswith("]"):
-        s = s[1:-1]
-    return [p.strip().strip("'\"") for p in s.split(",") if p.strip()]
-
-
-def first_paragraph(body: str) -> str:
-    para: list[str] = []
-    seen_h1 = False
-    for line in body.splitlines():
-        if line.startswith("# "):
-            seen_h1 = True
-            continue
-        if not seen_h1:
-            continue
-        if line.strip() == "" or line.startswith("#"):
-            if para:
-                break
-            continue
-        para.append(line.strip())
-    return " ".join(para)
-
-
-def note_text(slug: str, text: str) -> tuple[str, str]:
-    """(title, embedding text) for one note file."""
-    fm, body = parse_frontmatter(text)
-    title = (fm.get("title") or slug).strip().strip("'\"")
-    parts = [slug.replace("-", " "), title, " ".join(as_list(fm.get("aliases", ""))),
-             " ".join(as_list(fm.get("tags", ""))), first_paragraph(body)]
-    return title, " ".join(parts)[:EMBED_CHARS]
-
-
-# ---------------------------------------------------------------- embeddings
-
-
-def embed(texts: list[str], batch: int = 2) -> list[list[float]]:
-    out: list[list[float]] = []
-    for i in range(0, len(texts), batch):
-        resp = http("POST", f"{EMBED_URL}/v1/embeddings", {"model": EMBED_MODEL, "input": texts[i:i + batch]}, timeout=120)
-        rows = sorted(resp["data"], key=lambda d: d["index"])
-        out.extend(unit(d["embedding"]) for d in rows)
-    return out
-
-
-def unit(v: list[float]) -> list[float]:
-    n = sum(x * x for x in v) ** 0.5 or 1.0
-    return [round(x / n, 5) for x in v]
-
-
-# ---------------------------------------------------------------- index
-
-
-class Index:
-    def __init__(self, path: Path):
-        self.path = path
-        self.lock = threading.Lock()
-        self.notes: dict[str, dict] = {}  # slug -> {sha, title, vector}
-        self.commit = ""
-        self.synced_at = 0.0
-        self.syncing = False
-        self.last_error = ""
-        if path.exists():
-            try:
-                d = json.loads(path.read_text(encoding="utf-8"))
-                self.notes, self.commit, self.synced_at = d.get("notes", {}), d.get("commit", ""), d.get("synced_at", 0.0)
-                LOG.info("loaded index: %d notes at %s", len(self.notes), self.commit[:8])
-            except (OSError, ValueError) as e:
-                LOG.warning("index unreadable (%s); starting empty", e)
-
-    def save(self) -> None:
-        self.path.parent.mkdir(parents=True, exist_ok=True)
-        tmp = self.path.with_suffix(".tmp")
-        tmp.write_text(json.dumps({"notes": self.notes, "commit": self.commit, "synced_at": self.synced_at}), encoding="utf-8")
-        tmp.replace(self.path)
-
-    def sync(self, gitea: Gitea) -> dict:
-        """Bring the index to the branch head; returns a summary."""
-        with self.lock:
-            self.syncing = True
-        try:
-            head = gitea.head()
-            if head == self.commit and self.notes:
-                summary = {"commit": head, "changed": 0, "removed": 0, "notes": len(self.notes), "skipped": "head unchanged"}
-                self.synced_at = time.time()
-                self.save()
-                return summary
-            tree = gitea.tree(head)
-            slugs = {p[len(WIKI_DIR) + 1:-3]: (p, sha) for p, sha in tree.items() if not p.endswith("/_index.md")}
-            todo = [s for s, (p, sha) in slugs.items() if self.notes.get(s, {}).get("sha") != sha]
-            removed = [s for s in list(self.notes) if s not in slugs]
-            LOG.info("sync %s: %d notes, %d to embed, %d removed", head[:8], len(slugs), len(todo), len(removed))
-            fresh: dict[str, dict] = {}
-            for i in range(0, len(todo), 20):
-                chunk = todo[i:i + 20]
-                texts, titles = [], {}
-                for s in chunk:
-                    title, text = note_text(s, gitea.raw(slugs[s][0], head))
-                    titles[s] = title
-                    texts.append(text)
-                for s, vec in zip(chunk, embed(texts)):
-                    fresh[s] = {"sha": slugs[s][1], "title": titles[s], "vector": vec}
-            with self.lock:
-                for s in removed:
-                    self.notes.pop(s, None)
-                self.notes.update(fresh)
-                self.commit = head
-                self.synced_at = time.time()
-                self.last_error = ""
-                self.save()
-            return {"commit": head, "changed": len(todo), "removed": len(removed), "notes": len(self.notes)}
-        except Exception as e:
-            with self.lock:
-                self.last_error = str(e)[:300]
-            raise
-        finally:
-            with self.lock:
-                self.syncing = False
-
-    def rank(self, query: str, k: int = 60) -> list[dict]:
-        q = embed([query])[0]
-        with self.lock:
-            rows = [(s, n["title"], sum(a * b for a, b in zip(q, n["vector"]))) for s, n in self.notes.items()]
-        rows.sort(key=lambda r: -r[2])
-        return [{"slug": s, "title": t, "score": round(sc, 4)} for s, t, sc in rows[:k]]
-
-    def status(self) -> dict:
-        """`ready` = an index exists and can answer /rank; a failed sync keeps the last good index."""
-        with self.lock:
-            return {"ready": bool(self.notes), "notes": len(self.notes), "commit": self.commit,
-                    "synced_at": self.synced_at, "age_s": round(time.time() - self.synced_at) if self.synced_at else None,
-                    "syncing": self.syncing, "last_error": self.last_error or None}
-
-
-# ---------------------------------------------------------------- syncer
-
-
-class Syncer(threading.Thread):
-    """One worker: syncs on demand (webhook) and every RESYNC_SECONDS."""
-
-    def __init__(self, index: Index, creds: Credentials, interval: int):
-        super().__init__(daemon=True, name="syncer")
-        self.index, self.creds, self.interval = index, creds, interval
-        self.wake = threading.Event()
-
-    def trigger(self) -> None:
-        self.wake.set()
-
-    RETRY_SECONDS = 60
-
-    def run(self) -> None:
-        while True:
-            try:
-                data = self.creds.fetch()
-                summary = self.index.sync(Gitea(data["gitea_token"]))
-                LOG.info("sync done: %s", summary)
-                wait = self.interval
-            except Exception as e:  # keep serving the last good index; retry soon, not next interval
-                LOG.error("sync failed: %s", e)
-                wait = self.RETRY_SECONDS
-            self.wake.wait(wait)
-            self.wake.clear()
-
-
-def verify_signature(secret: str, body: bytes, signature: str) -> bool:
-    digest = hmac.new(secret.encode("utf-8"), body, hashlib.sha256).hexdigest()
-    return hmac.compare_digest(digest, (signature or "").strip())
-
-
-# ---------------------------------------------------------------- server
-
-
-def make_handler(index: Index, syncer: Syncer, creds: Credentials):
-    class Handler(BaseHTTPRequestHandler):
-        def log_message(self, fmt, *args):
-            LOG.info("%s %s", self.address_string(), fmt % args)
-
-        def send_json(self, code: int, obj) -> None:
-            data = json.dumps(obj).encode("utf-8")
-            self.send_response(code)
-            self.send_header("Content-Type", "application/json")
-            self.send_header("Content-Length", str(len(data)))
-            self.end_headers()
-            self.wfile.write(data)
-
-        def body(self) -> bytes:
-            n = int(self.headers.get("Content-Length") or 0)
-            return self.rfile.read(n) if n else b""
-
-        def do_GET(self):
-            if self.path == "/health":  # liveness: the process serves
-                return self.send_json(200, index.status())
-            if self.path == "/ready":  # readiness: an index exists
-                st = index.status()
-                return self.send_json(200 if st["ready"] else 503, st)
-            self.send_json(404, {"error": "not found"})
-
-        def do_POST(self):
-            if self.path == "/rank":
-                try:
-                    req = json.loads(self.body() or b"{}")
-                    query = str(req.get("query", "")).strip()
-                    k = max(1, min(int(req.get("k", 60)), 500))
-                    if not query:
-                        return self.send_json(400, {"error": "query required"})
-                    rows = index.rank(query, k)
-                    return self.send_json(200, {"rows": rows, "index": index.status()})
-                except ApiError as e:
-                    return self.send_json(502, {"error": str(e)})
-                except (ValueError, TypeError) as e:
-                    return self.send_json(400, {"error": str(e)})
-            if self.path == "/webhook":
-                raw = self.body()
-                try:
-                    secret = creds.fetch()["webhook_secret"]
-                except ApiError as e:
-                    return self.send_json(503, {"error": str(e)})
-                if not verify_signature(secret, raw, self.headers.get("X-Gitea-Signature", "")):
-                    return self.send_json(403, {"error": "bad signature"})
-                try:
-                    event = json.loads(raw or b"{}")
-                except ValueError:
-                    return self.send_json(400, {"error": "bad json"})
-                ref = event.get("ref", "")
-                if ref and ref != f"refs/heads/{NOTES_BRANCH}":
-                    return self.send_json(200, {"queued": False, "ref": ref})
-                syncer.trigger()
-                return self.send_json(200, {"queued": True})
-            self.send_json(404, {"error": "not found"})
-
-    return Handler
-
-
-def main() -> int:
-    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s", stream=sys.stdout)
-    index = Index(INDEX_PATH)
-    creds = Credentials()
-    syncer = Syncer(index, creds, RESYNC_SECONDS)
-    syncer.start()
-    server = ThreadingHTTPServer(("", LISTEN_PORT), make_handler(index, syncer, creds))
-    LOG.info("listening on :%d, repo %s@%s, embed %s", LISTEN_PORT, NOTES_REPO, NOTES_BRANCH, EMBED_URL)
-    server.serve_forever()
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())

docs/superpowers/plans/2026-09-25-paperclip.md

@@ -0,0 +1,976 @@
+# Paperclip Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Deploy Paperclip (server-local agent runs, option B) into the homelab cluster behind an Authentik gate, with CNPG Postgres and OpenBao-sourced secrets.
+
+**Architecture:** One Flux `Kustomization` (`paperclip`) applies `kubernetes/apps/paperclip/`: a single-replica Deployment on a ceph-block PVC, CNPG `pclp-pg` (PG17) with barman backups to Hetzner, and a Traefik Ingress that backends only the `paperclip-gate` Authentik outpost. Namespace, baseline CNP, ClusterSecretStore allowlist and image automation live in the existing `infrastructure` tree.
+
+**Tech Stack:** Flux v2 (kustomize-controller, image-reflector v1.2.5), CNPG + barman-cloud plugin, External Secrets → OpenBao, Cilium CNP, Traefik, Authentik blueprints.
+
+**Spec:** `docs/superpowers/specs/2026-09-25-paperclip-design.md`
+
+## Global Constraints
+
+- Worktree: `/home/krenner/source/homelab-worktrees/paperclip`, branch `feat/paperclip`. Never edit the main checkout.
+- Commits as claude-bot: `git -c user.name='Claude Bot' -c user.email='claude-bot@renner.uno' commit …`, message ends with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
+- Push: `git -c credential.helper='store --file=/home/krenner/.config/claude-bot/creds' push …`. PR via `tea pr create --login claude-bot`. Never self-merge.
+- One Bash command per call: no `&&`, `;`, `|` chaining, no `2>/dev/null`, no `python -c`.
+- Namespace `paperclip`; hostname `paperclip.renner.uno`; LAN-only (no `dns.renner.uno/public` label).
+- Image `ghcr.io/paperclipai/paperclip`, tag `nightly`, digest-pinned, `digestReflectionPolicy: Always`.
+- Postgres: CNPG `pclp-pg`, `ghcr.io/cloudnative-pg/postgresql:17`, 2 instances, `ceph-block`.
+- PVC `paperclip-data`, `ceph-block`, 20Gi, mounted at `/paperclip`.
+- OpenBao key `apps/paperclip` with fields `BETTER_AUTH_SECRET`, `PAPERCLIP_TOOL_ACTION_SIGNING_SECRET`, `PAPERCLIP_SECRETS_MASTER_KEY` (64-char hex), `CLAUDE_CODE_OAUTH_TOKEN`.
+- Env: `PAPERCLIP_DEPLOYMENT_MODE=authenticated`, `PAPERCLIP_DEPLOYMENT_EXPOSURE=private`, `PAPERCLIP_PUBLIC_URL=https://paperclip.renner.uno`, `HOST=0.0.0.0`, `PORT=3100`.
+- Manifest comments: one terse factual line; rationale goes in commit messages/PR.
+- No `kubectl apply`/`exec`/`rollout restart` against the live cluster; Flux applies after merge. Server-side `--dry-run=server` is allowed.
+
+## Review Focus
+
+1. **Pod replaced mid agent run** (image bump, secret rotation via Reloader, node drain) — `Recreate` kills in-flight runs; expected: Paperclip marks them failed/recoverable on restart, not wedged. Covered in Task 9 Step 6.
+2. **Shared OAuth token expired or missing** — expected: agent runs fail with a visible auth error in Paperclip, server stays healthy. Covered in Task 9 Step 7.
+3. **Direct access bypassing the gate** from another namespace pod — expected: connection refused/dropped by CNP. Covered in Task 9 Step 4.
+4. **Unauthenticated LAN request** to `paperclip.renner.uno` — expected: redirect to Authentik, never Paperclip's UI. Covered in Task 9 Step 3.
+5. **Nightly image with a breaking DB migration** — expected: a base backup exists before the first bump lands. Covered in Task 9 Step 5.
+
+---
+
+### Task 1: Upstream facts that decide two manifest values
+
+No repo changes; produces two decisions consumed by Task 6.
+
+**Interfaces:**
+- Produces: `SIGNUP_DISABLED` (`"true"` or `"false"`), `CONFIG_PARTIAL_OK` (yes/no), `ENTRYPOINT_NEEDS_ROOT` (yes/no).
+
+- [ ] **Step 1: Sign-up vs instance claim.** Read upstream `server/src/auth/better-auth.ts`, `server/src/routes/auth.ts`, and the claim flow for `authenticated`+`private` (search `claim` under `server/src`):
+
+Run: `gh api repos/paperclipai/paperclip/contents/server/src/routes/auth.ts -H "Accept: application/vnd.github.raw"`
+
+Decide: if the first-admin claim creates the account through a path that ignores `disableSignUp` (e.g. invite/claim token handler calling the adapter directly), `SIGNUP_DISABLED="true"`. Otherwise `SIGNUP_DISABLED="false"` — the Authentik gate is the access control, and a follow-up PR flips it to `"true"` after the admin is claimed.
+
+- [ ] **Step 2: Partial config.json.** Read `server/src/config.ts` and the config-file loader it calls (search `PAPERCLIP_CONFIG` / `readConfigFile`).
+
+Run: `gh api repos/paperclipai/paperclip/contents/server/src/config.ts -H "Accept: application/vnd.github.raw"`
+
+Decide `CONFIG_PARTIAL_OK=yes` if a file containing only `{"telemetry":{"enabled":false}}` parses (every other section optional/defaulted). If not, stop and report to the user — do not invent config fields.
+
+- [ ] **Step 3: Entrypoint privilege.**
+
+Run: `gh api repos/paperclipai/paperclip/contents/docker-entrypoint.sh -H "Accept: application/vnd.github.raw"`
+
+`ENTRYPOINT_NEEDS_ROOT=yes` if it calls `usermod`/`groupmod`/`chown`/`gosu`. Record the result for Task 6.
+
+- [ ] **Step 4: Anonymous health response.**
+
+Run: `gh api repos/paperclipai/paperclip/contents/server/src/routes/health.ts -H "Accept: application/vnd.github.raw"`
+
+Confirm `GET /api/health` returns HTTP 200 to an anonymous caller when healthy (probes use it). If it returns 401 anonymously, use a `tcpSocket` probe on 3100 in Task 6 instead.
+
+---
+
+### Task 2: Namespace, baseline CNP, ClusterSecretStore allowlist
+
+**Files:**
+- Modify: `kubernetes/infrastructure/namespaces/namespaces.yaml` (append)
+- Create: `kubernetes/infrastructure/network-policies/paperclip-ns.yaml`
+- Modify: `kubernetes/infrastructure/network-policies/kustomization.yaml`
+- Modify: `kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml` (sorted list)
+
+**Interfaces:**
+- Produces: namespace `paperclip`; ESO may reference `openbao` from it.
+
+- [ ] **Step 1: Append the namespace** to the end of `namespaces.yaml`:
+
+```yaml
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: paperclip
+```
+
+- [ ] **Step 2: Create `paperclip-ns.yaml`** (copy of `paperless-ns.yaml`'s `default-deny-ingress`, ns renamed, paperless-only CNPs dropped):
+
+```yaml
+# Traefik is not admitted: the only entry is the paperclip-gate outpost (apps/paperclip/network-policy.yaml).
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: default-deny-ingress
+  namespace: paperclip
+spec:
+  endpointSelector: {}
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": database
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": monitoring
+    - fromEntities:
+        - host
+        - remote-node
+    - fromEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": rook-ceph
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+    - toEntities:
+        - host
+        - remote-node
+        - world
+    - toEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }
+```
+
+- [ ] **Step 3: Register it** — in `network-policies/kustomization.yaml` add `  - paperclip-ns.yaml` directly after `  - paperless-ns.yaml`.
+
+- [ ] **Step 4: Allowlist the namespace** — in `openbao-clustersecretstore.yaml` insert `        - paperclip` between `        - monitoring` and `        - paperless` (list is sorted).
+
+- [ ] **Step 5: Render check.**
+
+Run: `kubectl kustomize kubernetes/infrastructure/network-policies`
+Expected: output contains `namespace: paperclip`, exit 0.
+
+Run: `kubectl apply --dry-run=server -f kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml`
+Expected: `clustersecretstore.external-secrets.io/openbao configured (server dry run)`.
+
+- [ ] **Step 6: Commit** `feat(paperclip): namespace, baseline CNP and OpenBao allowlist`.
+
+---
+
+### Task 3: OpenBao secret `apps/paperclip`
+
+Out-of-repo; must exist before merge or the ExternalSecret never becomes Ready.
+
+**Interfaces:**
+- Produces: OpenBao KV `apps/paperclip` with the four fields in Global Constraints.
+
+- [ ] **Step 1:** Ask wiki-synth for the claude-bot OpenBao write procedure (note `claude-bot-openbao-write-only-identity`): login method, KV mount, exact `bao kv put`/`patch` syntax for an `apps/<name>` key. Use exactly what it returns.
+
+- [ ] **Step 2: Write the generated fields** as claude-bot (write-only identity — never read values back). Values generated inline by the command:
+
+`BETTER_AUTH_SECRET` = `openssl rand -hex 32`, `PAPERCLIP_TOOL_ACTION_SIGNING_SECRET` = `openssl rand -hex 32`, `PAPERCLIP_SECRETS_MASTER_KEY` = `openssl rand -hex 32` (64 hex chars = 32 bytes, accepted format).
+
+Generate each with a separate `openssl rand -hex 32` call and pass it directly into the `bao kv put` from Step 1; do not echo or log them.
+
+- [ ] **Step 3: Ask the user** to add the shared token themselves (never handle it): "Run `claude setup-token`, then `bao kv patch <mount>/apps/paperclip CLAUDE_CODE_OAUTH_TOKEN=<token>`." Wait for confirmation.
+
+- [ ] **Step 4: Verify field names only** (no values): `bao kv metadata get <mount>/apps/paperclip` shows a current version ≥2. If claude-bot cannot read metadata, ask the user to confirm the four keys exist.
+
+---
+
+### Task 4: Image automation + initial digest
+
+**Files:**
+- Create: `kubernetes/infrastructure/image-automation/paperclip.yaml`
+- Modify: `kubernetes/infrastructure/image-automation/kustomization.yaml`
+
+**Interfaces:**
+- Produces: ImagePolicy `flux-system:paperclip`; `INITIAL_DIGEST` (`sha256:…`) for Task 6.
+
+- [ ] **Step 1: Create `paperclip.yaml`:**
+
+```yaml
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImageRepository
+metadata:
+  name: paperclip
+  namespace: flux-system
+spec:
+  image: ghcr.io/paperclipai/paperclip
+  interval: 1h
+---
+# Upstream publishes no semver tags; track the nightly tag by digest.
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImagePolicy
+metadata:
+  name: paperclip
+  namespace: flux-system
+spec:
+  imageRepositoryRef:
+    name: paperclip
+  digestReflectionPolicy: Always
+  filterTags:
+    pattern: '^nightly$'
+  policy:
+    alphabetical:
+      order: asc
+```
+
+- [ ] **Step 2: Register** — add `  - paperclip.yaml` after `  - paperless-ngx.yaml` in `image-automation/kustomization.yaml`.
+
+- [ ] **Step 3: Server dry-run.**
+
+Run: `kubectl apply --dry-run=server -f kubernetes/infrastructure/image-automation/paperclip.yaml`
+Expected: both objects `created (server dry run)`; no `unknown field "digestReflectionPolicy"` error.
+
+- [ ] **Step 4: Resolve the current nightly digest.**
+
+The workstation has `docker` only (no skopeo/crane/buildx, checked 2026-09-25); `docker manifest inspect` lists per-platform digests but not the index digest, so pull once:
+
+Run: `docker pull ghcr.io/paperclipai/paperclip:nightly`
+Run: `docker image inspect --format '{{index .RepoDigests 0}}' ghcr.io/paperclipai/paperclip:nightly`
+Expected: `ghcr.io/paperclipai/paperclip@sha256:<64 hex>`. Record the `sha256:…` part as `INITIAL_DIGEST`.
+Run: `docker image rm ghcr.io/paperclipai/paperclip:nightly`
+
+- [ ] **Step 5: Commit** `feat(paperclip): Flux image automation tracks the nightly digest`.
+
+---
+
+### Task 5: Postgres (CNPG) and secrets in the app directory
+
+**Files:**
+- Create: `kubernetes/apps/paperclip/cluster.yaml`, `objectstore.yaml`, `scheduled-backup.yaml`, `hetzner-creds-externalsecret.yaml`, `externalsecret.yaml`
+
+**Interfaces:**
+- Produces: Secret `pclp-pg-app` (CNPG, key `uri`), Service `pclp-pg-rw`, Secret `paperclip-secrets`.
+
+- [ ] **Step 1: `cluster.yaml`:**
+
+```yaml
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: pclp-pg
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # data guard
+spec:
+  instances: 2
+  smartShutdownTimeout: 10
+  stopDelay: 60
+  # PG17: upstream's external Postgres target (docker/docker-compose.yml).
+  imageName: ghcr.io/cloudnative-pg/postgresql:17
+  storage:
+    storageClass: ceph-block
+    size: 10Gi
+  plugins:
+    - name: barman-cloud.cloudnative-pg.io
+      isWALArchiver: true
+      parameters:
+        barmanObjectName: pclp-pg-store
+  bootstrap:
+    initdb:
+      database: paperclip
+      owner: paperclip
+  postgresql:
+    parameters:
+      shared_buffers: "256MB"
+      max_connections: "200"
+      checkpoint_timeout: "15min"
+      max_wal_size: "2GB"
+      wal_compression: "zstd"
+      archive_timeout: "30min"
+    synchronous:
+      method: any
+      number: 1
+      dataDurability: preferred
+  resources:
+    requests:
+      cpu: 200m
+      memory: 512Mi
+    limits:
+      memory: 1Gi
+```
+
+- [ ] **Step 2: `objectstore.yaml`:**
+
+```yaml
+apiVersion: barmancloud.cnpg.io/v1
+kind: ObjectStore
+metadata:
+  name: pclp-pg-store
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # backup target
+spec:
+  configuration:
+    destinationPath: s3://renner-uno-k8s-cnpg/pclp-pg
+    endpointURL: https://fsn1.your-objectstorage.com
+    s3Credentials:
+      accessKeyId:
+        name: hetzner-s3-creds
+        key: ACCESS_KEY_ID
+      secretAccessKey:
+        name: hetzner-s3-creds
+        key: ACCESS_SECRET_KEY
+      region:
+        name: hetzner-s3-creds
+        key: REGION
+    wal:
+      compression: gzip
+    data:
+      compression: gzip
+  retentionPolicy: "30d"
+```
+
+- [ ] **Step 3: `scheduled-backup.yaml`:**
+
+```yaml
+apiVersion: postgresql.cnpg.io/v1
+kind: ScheduledBackup
+metadata:
+  name: pclp-pg-daily
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # backup schedule
+spec:
+  schedule: "0 45 2 * * *"
+  immediate: true
+  backupOwnerReference: self
+  cluster:
+    name: pclp-pg
+  method: plugin
+  pluginConfiguration:
+    name: barman-cloud.cloudnative-pg.io
+```
+
+- [ ] **Step 4: `hetzner-creds-externalsecret.yaml`:**
+
+```yaml
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: hetzner-s3-creds
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # barman S3 creds
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: hetzner-s3-creds
+  dataFrom:
+    - extract:
+        key: infrastructure/hetzner-s3
+```
+
+- [ ] **Step 5: `externalsecret.yaml`:**
+
+```yaml
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: paperclip-secrets
+  namespace: paperclip
+spec:
+  refreshInterval: 10m
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: paperclip-secrets
+  dataFrom:
+    - extract:
+        key: apps/paperclip
+```
+
+- [ ] **Step 6: Commit** `feat(paperclip): CNPG pclp-pg with barman backups and OpenBao secrets`. (Rendered together with Task 6's kustomization.)
+
+---
+
+### Task 6: Server workload, storage, ingress, network policy
+
+**Files:**
+- Create: `kubernetes/apps/paperclip/pvc.yaml`, `paperclip.yaml`, `gate-service.yaml`, `ingress.yaml`, `network-policy.yaml`, `kustomization.yaml`
+
+**Interfaces:**
+- Consumes: `pclp-pg-app` (`uri`), `paperclip-secrets`, ImagePolicy `flux-system:paperclip`, `INITIAL_DIGEST` (Task 4), `SIGNUP_DISABLED` / `ENTRYPOINT_NEEDS_ROOT` / probe choice (Task 1).
+- Produces: pods labelled `app.kubernetes.io/name: paperclip` listening on 3100; Service `paperclip:3100`; ExternalName `ak-outpost-paperclip-gate:9000`.
+
+- [ ] **Step 1: `pvc.yaml`:**
+
+```yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: paperclip-data
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # data guard
+spec:
+  accessModes: [ReadWriteOnce]
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 20Gi
+```
+
+- [ ] **Step 2: `paperclip.yaml`** — replace `<INITIAL_DIGEST>` with the Task 4 value and `<SIGNUP_DISABLED>` with the Task 1 value. If Task 1 found `ENTRYPOINT_NEEDS_ROOT=no`, replace the container `securityContext` with `runAsNonRoot: true`, `runAsUser: 1000`, `runAsGroup: 1000`, `allowPrivilegeEscalation: false`, `capabilities: {drop: ["ALL"]}`. If Task 1 chose `tcpSocket`, replace each `httpGet` block with `tcpSocket: {port: http}`.
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: paperclip
+  namespace: paperclip
+  labels:
+    app.kubernetes.io/name: paperclip
+  annotations:
+    secret.reloader.stakater.com/reload: paperclip-secrets
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: paperclip
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: paperclip
+    spec:
+      enableServiceLinks: false
+      securityContext:
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+        seccompProfile:
+          type: RuntimeDefault
+      initContainers:
+        # Telemetry is only configurable in config.json.
+        - name: disable-telemetry
+          image: ghcr.io/paperclipai/paperclip:nightly@<INITIAL_DIGEST> # {"$imagepolicy": "flux-system:paperclip"}
+          command:
+            - sh
+            - -c
+            - |
+              set -eu
+              f=/paperclip/instances/default/config.json
+              mkdir -p "$(dirname "$f")"
+              if [ -s "$f" ]; then
+                jq '.telemetry.enabled = false' "$f" > "$f.tmp"
+              else
+                printf '%s\n' '{"telemetry":{"enabled":false}}' > "$f.tmp"
+              fi
+              mv "$f.tmp" "$f"
+          securityContext:
+            runAsNonRoot: true
+            runAsUser: 1000
+            runAsGroup: 1000
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+          volumeMounts:
+            - name: data
+              mountPath: /paperclip
+      containers:
+        - name: paperclip
+          image: ghcr.io/paperclipai/paperclip:nightly@<INITIAL_DIGEST> # {"$imagepolicy": "flux-system:paperclip"}
+          ports:
+            - name: http
+              containerPort: 3100
+          env:
+            - name: DATABASE_URL
+              valueFrom:
+                secretKeyRef:
+                  name: pclp-pg-app
+                  key: uri
+            - name: HOST
+              value: "0.0.0.0"
+            - name: PORT
+              value: "3100"
+            - name: PAPERCLIP_DEPLOYMENT_MODE
+              value: authenticated
+            - name: PAPERCLIP_DEPLOYMENT_EXPOSURE
+              value: private
+            - name: PAPERCLIP_PUBLIC_URL
+              value: https://paperclip.renner.uno
+            - name: PAPERCLIP_AUTH_DISABLE_SIGN_UP
+              value: "<SIGNUP_DISABLED>"
+          envFrom:
+            - secretRef:
+                name: paperclip-secrets
+          volumeMounts:
+            - name: data
+              mountPath: /paperclip
+          startupProbe:
+            httpGet:
+              path: /api/health
+              port: http
+            periodSeconds: 10
+            timeoutSeconds: 5
+            failureThreshold: 60
+          livenessProbe:
+            httpGet:
+              path: /api/health
+              port: http
+            periodSeconds: 30
+            timeoutSeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /api/health
+              port: http
+            periodSeconds: 10
+            timeoutSeconds: 5
+          resources:
+            requests:
+              cpu: 250m
+              memory: 1Gi
+            limits:
+              memory: 4Gi
+          securityContext:
+            runAsNonRoot: false
+            allowPrivilegeEscalation: true
+            capabilities:
+              drop: ["ALL"]
+              add: ["CHOWN", "DAC_OVERRIDE", "FOWNER", "SETGID", "SETUID"]
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: paperclip-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: paperclip
+  namespace: paperclip
+spec:
+  selector:
+    app.kubernetes.io/name: paperclip
+  ports:
+    - name: http
+      port: 3100
+      targetPort: http
+```
+
+- [ ] **Step 3: `gate-service.yaml`:**
+
+```yaml
+# ExternalName mirror so the Ingress can backend the paperclip-gate outpost in the authentik namespace.
+apiVersion: v1
+kind: Service
+metadata:
+  name: ak-outpost-paperclip-gate
+  namespace: paperclip
+spec:
+  type: ExternalName
+  externalName: ak-outpost-paperclip-gate.authentik.svc.cluster.local
+  ports:
+    - port: 9000
+      targetPort: 9000
+```
+
+- [ ] **Step 4: `ingress.yaml`:**
+
+```yaml
+# LAN-only (no dns.renner.uno/public label); backends the gate outpost, never the Service.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: paperclip
+  namespace: paperclip
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - paperclip.renner.uno
+      secretName: paperclip-tls
+  rules:
+    - host: paperclip.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: ak-outpost-paperclip-gate
+                port:
+                  number: 9000
+```
+
+- [ ] **Step 5: `network-policy.yaml`** (Traefik pod-side port is 8443, verified via `kubectl get pods -n ingress -l app.kubernetes.io/name=traefik`):
+
+```yaml
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: paperclip-gate-ingress
+  namespace: paperclip
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: paperclip
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+            goauthentik.io/outpost-name: paperclip-gate
+      toPorts:
+        - ports:
+            - port: "3100"
+              protocol: TCP
+---
+# Covers the server and every agent it spawns (same pod).
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: paperclip-egress
+  namespace: paperclip
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: paperclip
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+            cnpg.io/cluster: pclp-pg
+      toPorts:
+        - ports:
+            - port: "5432"
+              protocol: TCP
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+    # Gitea, Woodpecker, HA, Loki via their *.renner.uno names.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+      toPorts:
+        - ports:
+            - port: "8443"
+              protocol: TCP
+    - toEntities:
+        - kube-apiserver
+      toPorts:
+        - ports:
+            - port: "6443"
+              protocol: TCP
+    - toEntities:
+        - world
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
+```
+
+- [ ] **Step 6: `kustomization.yaml`:**
+
+```yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - externalsecret.yaml
+  - hetzner-creds-externalsecret.yaml
+  - cluster.yaml
+  - objectstore.yaml
+  - scheduled-backup.yaml
+  - pvc.yaml
+  - paperclip.yaml
+  - gate-service.yaml
+  - ingress.yaml
+  - network-policy.yaml
+```
+
+- [ ] **Step 7: Render check.**
+
+Run: `kubectl kustomize kubernetes/apps/paperclip`
+Expected: exit 0; output contains `kind: Cluster`, `kind: Deployment`, `claimName: paperclip-data`, and no literal `<INITIAL_DIGEST>` or `<SIGNUP_DISABLED>`.
+
+Run: `grep -rn "<INITIAL_DIGEST>\|<SIGNUP_DISABLED>" kubernetes/apps/paperclip`
+Expected: no output, exit 1.
+
+- [ ] **Step 8: Commit** `feat(paperclip): server Deployment, PVC, gated Ingress and CNPs`.
+
+---
+
+### Task 7: Authentik gate
+
+**Files:**
+- Modify: `kubernetes/apps/authentik/blueprints-configmap.yaml` (insert before the line `      # navidrome-gate: the Navidrome web UI.`)
+- Create: `kubernetes/apps/authentik/paperclip-gate-egress.yaml`
+- Modify: `kubernetes/apps/authentik/kustomization.yaml`
+
+**Interfaces:**
+- Produces: outpost pods labelled `goauthentik.io/outpost-name: paperclip-gate`, Service `ak-outpost-paperclip-gate.authentik:9000`.
+
+- [ ] **Step 1: Insert blueprint entries** (same indentation as the neighbouring `wiki-gate` block):
+
+```yaml
+      # paperclip-gate: Paperclip UI; the CNP admits only this outpost. Paperclip keeps its own login behind it.
+      - id: paperclip-users-group
+        model: authentik_core.group
+        identifiers:
+          name: Paperclip Users
+        state: present
+        attrs:
+          is_superuser: false
+          users:
+            - !Find [authentik_core.user, [username, k.renner]]
+
+      - id: paperclip-gate-provider
+        model: authentik_providers_proxy.proxyprovider
+        identifiers:
+          name: paperclip-gate
+        state: present
+        attrs:
+          authentication_flow: !Find [authentik_flows.flow, [slug, passkey-passwordless-authentication]]
+          authorization_flow: !Find [authentik_flows.flow, [slug, passkey-stepup-authorization]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          mode: proxy
+          internal_host: http://paperclip.paperclip.svc.cluster.local:3100
+          internal_host_ssl_validation: false
+          external_host: https://paperclip.renner.uno
+          cookie_domain: renner.uno
+          access_token_validity: hours=24
+          refresh_token_validity: days=30
+          intercept_header_auth: true
+
+      - model: authentik_core.application
+        identifiers:
+          slug: paperclip-gate
+        state: present
+        attrs:
+          name: Paperclip
+          slug: paperclip-gate
+          provider: !KeyOf paperclip-gate-provider
+          meta_launch_url: https://paperclip.renner.uno
+          meta_description: "Paperclip agent orchestration"
+
+      - model: authentik_policies.policybinding
+        identifiers:
+          target: !Find [authentik_core.application, [slug, paperclip-gate]]
+          group: !KeyOf paperclip-users-group
+          order: 0
+        state: present
+        attrs:
+          enabled: true
+          negate: false
+
+      - model: authentik_outposts.outpost
+        identifiers:
+          name: paperclip-gate
+        state: present
+        attrs:
+          type: proxy
+          service_connection: !KeyOf k8s-conn
+          providers:
+            - !KeyOf paperclip-gate-provider
+          config:
+            log_level: info
+            authentik_host: "http://authentik-server.authentik.svc.cluster.local"
+            authentik_host_insecure: false
+            authentik_host_browser: "https://auth.renner.uno/"
+            refresh_interval: "minutes=5"
+            container_image: null
+            object_naming_template: "ak-outpost-%(name)s"
+            kubernetes_namespace: authentik
+            kubernetes_replicas: 1
+            kubernetes_service_type: ClusterIP
+            kubernetes_disabled_components: ["ingress"]
+            kubernetes_image_pull_secrets: []
+            kubernetes_ingress_annotations: {}
+            kubernetes_ingress_class_name: null
+            kubernetes_ingress_secret_name: "authentik-outpost-tls"
+            kubernetes_json_patches: null
+
+```
+
+- [ ] **Step 2: `paperclip-gate-egress.yaml`:**
+
+```yaml
+# Lets the paperclip-gate outpost reach the Paperclip server; scoped to this outpost's pods.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: paperclip-gate-egress
+  namespace: authentik
+spec:
+  endpointSelector:
+    matchLabels:
+      goauthentik.io/outpost-name: paperclip-gate
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+            app.kubernetes.io/name: paperclip
+      toPorts:
+        - ports:
+            - port: "3100"
+              protocol: TCP
+```
+
+- [ ] **Step 3: Register** — add `  - paperclip-gate-egress.yaml` after `  - paperless-gate-egress.yaml` in `apps/authentik/kustomization.yaml`.
+
+- [ ] **Step 4: Render + dry-run.**
+
+Run: `kubectl kustomize kubernetes/apps/authentik`
+Expected: exit 0 (valid YAML incl. the blueprint ConfigMap).
+
+Run: `kubectl apply --dry-run=server -f kubernetes/apps/authentik/paperclip-gate-egress.yaml`
+Expected: `ciliumnetworkpolicy.cilium.io/paperclip-gate-egress created (server dry run)`.
+
+- [ ] **Step 5: Commit** `feat(authentik): paperclip-gate outpost and Paperclip Users group`.
+
+---
+
+### Task 8: Flux Kustomization
+
+**Files:**
+- Create: `kubernetes/clusters/homelab/paperclip.yaml`
+
+- [ ] **Step 1:**
+
+```yaml
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: paperclip
+  namespace: flux-system
+spec:
+  interval: 10m
+  retryInterval: 1m
+  timeout: 20m
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./kubernetes/apps/paperclip
+  prune: true
+  wait: true
+  dependsOn:
+    - name: infrastructure
+    - name: infrastructure-crds
+    - name: infrastructure-ceph-cluster
+    - name: infrastructure-cnpg-barman
+  healthCheckExprs:
+    - apiVersion: postgresql.cnpg.io/v1
+      kind: Cluster
+      failed: "has(status.phase) && status.phase.startsWith('Failed')"
+      current: "has(status.phase) && (status.phase == 'Cluster in healthy state' || status.readyInstances > 0)"
+```
+
+- [ ] **Step 2: Server dry-run.**
+
+Run: `kubectl apply --dry-run=server -f kubernetes/clusters/homelab/paperclip.yaml`
+Expected: `kustomization.kustomize.toolkit.fluxcd.io/paperclip created (server dry run)`.
+
+- [ ] **Step 3: Commit** `feat(paperclip): Flux Kustomization`.
+
+---
+
+### Task 9: PR, merge by user, live verification
+
+- [ ] **Step 1: Push and open the PR** (Task 3 must be complete).
+
+Run: `git -c credential.helper='store --file=/home/krenner/.config/claude-bot/creds' push -u origin feat/paperclip`
+Run: `tea pr create --login claude-bot --base master --head feat/paperclip --title "feat(paperclip): deploy Paperclip (server-local runs, trial)" --description "<summary of spec decisions + link to spec + bootstrap steps>\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)"`
+Then wait for the user to merge. Do not merge.
+
+- [ ] **Step 2: Reconciliation.** After merge:
+
+Run: `flux get kustomizations paperclip`
+Expected: `Ready True`. If False, read `kubectl describe kustomization paperclip -n flux-system` and the failing object's events; diagnose, don't poke.
+
+Run: `kubectl get cluster -n paperclip pclp-pg`
+Expected: `Cluster in healthy state`, 2 ready instances.
+
+Run: `kubectl get pods -n paperclip -l app.kubernetes.io/name=paperclip`
+Expected: `1/1 Running`.
+
+- [ ] **Step 3: Gate enforced (Review Focus 4).**
+
+Run: `curl -sS -o /dev/null -w '%{http_code} %{redirect_url}\n' https://paperclip.renner.uno/`
+Expected: `302` redirecting to `https://paperclip.renner.uno/outpost.goauthentik.io/…` or `auth.renner.uno`, never `200`.
+
+- [ ] **Step 4: Gate not bypassable (Review Focus 3).**
+
+`default` has no CNPs (verified 2026-09-25), so its egress is open and only paperclip's ingress policy can block the probe.
+
+Run: `kubectl get ciliumnetworkpolicies -n default`
+Expected: `No resources found` (else pick another CNP-free namespace).
+
+Run: `kubectl run cnp-probe -n default --rm -i --restart=Never --image=curlimages/curl:8.10.1 -- curl -sS -m 5 http://paperclip.paperclip.svc.cluster.local:3100/api/health`
+Expected: timeout (exit 28), no HTTP response.
+
+- [ ] **Step 5: First backup (Review Focus 5).**
+
+Run: `kubectl get backup -n paperclip`
+Expected: one `pclp-pg-daily-…` with phase `completed`.
+
+- [ ] **Step 6: Image policy.**
+
+Run: `kubectl get imagepolicy -n flux-system paperclip -o jsonpath='{.status.latestRef}'`
+Expected: `nightly` tag with a `sha256:` digest.
+
+- [ ] **Step 7: Hand-over to the user (browser steps, theirs):**
+  1. Open `https://paperclip.renner.uno`, pass Authentik, claim the instance as admin.
+  2. Create a test agent with the `claude-local` adapter and a trivial task ("reply with OK"); it must complete with the shared token (Review Focus 2: if it fails, report the run's error text — an expired token must show as an auth error, not a hang).
+  3. If `SIGNUP_DISABLED` was `"false"`, open the follow-up PR flipping `PAPERCLIP_AUTH_DISABLE_SIGN_UP` to `"true"` once the admin is claimed.
+  4. Review Focus 1 is observed on the first nightly bump: check the interrupted-run status in the UI.
+
+- [ ] **Step 8: Wiki.** Route the verified Paperclip facts (execution topology, env vars, image tags, digest-reflection usage) through wiki-ingest → wiki-distill once the deployment has proven out.

docs/superpowers/specs/2026-09-25-paperclip-design.md

@@ -0,0 +1,186 @@
+# Paperclip in the homelab cluster — design
+
+Date: 2026-09-25
+Status: draft, awaiting review
+
+## Goal
+
+Run [Paperclip](https://github.com/paperclipai/paperclip) (MIT, agent-orchestration
+server + UI) in the homelab cluster for real use: its agents do actual work against
+homelab systems. This spec covers the **platform** only. Bot identities and their
+credentials are out of scope (see "Deferred").
+
+## Decisions
+
+| Topic | Decision |
+|---|---|
+| Execution model | **B — server-local runs** (upstream default): agent CLIs are child processes of the server, in the server container. Trial only. |
+| Upgrade path | If Paperclip is kept: **C** — `@paperclipai/plugin-kubernetes` sandbox provider + kubernetes-sigs agent-sandbox controller (`agents.x-k8s.io`), per-run Pods and Secrets. Separate spec. |
+| Claude auth | **One shared Claude subscription AI connection**, created once via Paperclip's local sign-in (`claude auth login` into `ai-local-logins/<id>`); Paperclip stores and refreshes it. Server-env `CLAUDE_CODE_OAUTH_TOKEN` is not used: `stripAiAuthBindings` blanks it for managed runs, and upstream accepts no pasted setup-token (the device-code flow needs a Daytona sandbox). |
+| Bot credentials | Out of scope. Future "AIR-Manager" Paperclip role handles permission requests. |
+| Exposure | LAN-only, behind an Authentik forward-auth gate, then Paperclip's own login. |
+
+### Known trade-off of B (accepted for the trial)
+
+Upstream `runChildProcess()` spawns every local adapter with
+`{...sanitizeInheritedPaperclipEnv(process.env), ...opts.env}`; the sanitizer strips only
+`PAPERCLIP_*` keys. Every agent therefore sees `DATABASE_URL`, `BETTER_AUTH_SECRET`,
+the shared filesystem (including every other agent's workspace and the
+`ai-local-logins` credentials) and every agent-scoped secret it can read from disk. Under B, no bot may hold
+a credential we would not tolerate any other bot using. Per-bot isolation is C's job.
+
+## Architecture
+
+```
+LAN client ─▶ Traefik ─▶ ExternalName svc ─▶ ak-outpost-paperclip-gate (authentik ns)
+                                                   │ forward-auth OK
+                                                   ▼
+                                  paperclip Deployment (ns paperclip, 1 replica)
+                                   ├─ server :3100  (+ agent CLIs as child processes)
+                                   ├─ PVC /paperclip  (ceph-block)
+                                   └─▶ pclp-pg-rw (CNPG, PG17, 2 instances) ─▶ barman ▶ Hetzner S3
+```
+
+## Components
+
+All modeled on `kubernetes/apps/paperless/` unless stated.
+
+### Namespace and baseline policy
+- `Namespace paperclip` in `kubernetes/infrastructure/namespaces/namespaces.yaml`.
+- Baseline default-deny CNP `kubernetes/infrastructure/network-policies/paperclip-ns.yaml`
+  (copy of `paperless-ns.yaml`, renamed).
+- Add `paperclip` to the `openbao` ClusterSecretStore namespace allowlist.
+
+### Flux
+- `kubernetes/clusters/homelab/paperclip.yaml`: Flux `Kustomization`,
+  `path: ./kubernetes/apps/paperclip`, `prune: true`, `wait: true`, same `dependsOn` and
+  CNPG `healthCheckExprs` as `paperless.yaml`.
+- `kubernetes/apps/paperclip/kustomization.yaml`: plain resource list.
+
+### Server Deployment (`paperclip.yaml`)
+- Image `ghcr.io/paperclipai/paperclip:nightly@sha256:<digest>` with the Flux
+  image-automation marker for digest updates.
+- `replicas: 1`, `strategy: Recreate` (single writer on an RWO PVC).
+- Starts as root: upstream `docker-entrypoint.sh` remaps the `node` user, chowns
+  `/paperclip`, then drops to uid 1000 via `gosu`. Caps limited to CHOWN,
+  DAC_OVERRIDE, FOWNER, SETGID, SETUID; `fsGroup: 1000`.
+- `serviceAccountName: paperclip-prometheus-read` with
+  `automountServiceAccountToken: true`. All agents share the pod filesystem, so all
+  of them can read that token — it is mounted anyway because the Role behind it
+  grants exactly `get` on `services/proxy` for
+  `kube-prometheus-stack-prometheus:9090` in `monitoring` and nothing else
+  (`prometheus-read-rbac.yaml`, REN-36). It was `false` until then.
+- Port 3100; `Service paperclip` ClusterIP :3100.
+- Probes: startup and readiness on `GET /api/health` with `Host: localhost` (the
+  private-hostname guard rejects the pod IP); liveness is a TCP check, since
+  `/api/health` runs `SELECT 1` and a DB outage must not restart the server.
+- Env:
+  - `DATABASE_URL` assembled from CNPG `pclp-pg-app` Secret (`uri` key).
+  - `HOST=0.0.0.0`, `PORT=3100`
+  - `PAPERCLIP_DEPLOYMENT_MODE=authenticated`
+  - `PAPERCLIP_DEPLOYMENT_EXPOSURE=private`
+  - `PAPERCLIP_PUBLIC_URL=https://paperclip.renner.uno`
+  - `PAPERCLIP_AUTH_DISABLE_SIGN_UP=false` until the first admin is claimed: the
+    claim (`/bootstrap/claim`) requires an already signed-in Better Auth account.
+    A follow-up flips it to `true`.
+  - `PAPERCLIP_TELEMETRY_DISABLED=1` (read by `resolveTelemetryConfig`).
+  - `PAPERCLIP_API_URL=http://paperclip.paperclip.svc.cluster.local:3100` plus
+    `PAPERCLIP_ALLOWED_HOSTNAMES=paperclip.paperclip.svc.cluster.local`: upstream
+    otherwise hands agents the public URL, and the gate (`intercept_header_auth`)
+    rejects their bearer tokens. The Service name also works for sandbox pods under
+    option C; agent JWTs still authenticate each call.
+  - `envFrom: secretRef paperclip-secrets`
+- Reloader: `secret.reloader.stakater.com/reload: paperclip-secrets`.
+
+### Storage
+- `PVC paperclip-data`, `ceph-block`, 20Gi, mounted at `/paperclip` (`PAPERCLIP_HOME`
+  and `HOME`): instance config, workspaces, repo checkouts, per-agent `claude-runtime`
+  dirs.
+
+### Postgres
+- CNPG `Cluster pclp-pg`: 2 instances, `imageName: ghcr.io/cloudnative-pg/postgresql:17`,
+  `ceph-block` 10Gi, `bootstrap.initdb {database: paperclip, owner: paperclip}`,
+  barman-cloud plugin as WAL archiver.
+- `ObjectStore pclp-pg-store` → `s3://renner-uno-k8s-cnpg/pclp-pg`, creds via the
+  existing `hetzner-s3-creds` ExternalSecret pattern.
+- `ScheduledBackup` nightly, `method: plugin`, `backupOwnerReference: self`.
+- `kustomize.toolkit.fluxcd.io/prune: disabled` on Cluster, ObjectStore, ScheduledBackup.
+- PG17 deviates from the cluster-wide PG16 convention because upstream's external
+  Postgres target is `postgres:17` (`docker/docker-compose.yml`) and its embedded default
+  is PG18.
+
+### Secrets
+One ExternalSecret `paperclip-secrets` ← OpenBao `apps/paperclip` (whole-object
+extract):
+- `BETTER_AUTH_SECRET`
+- `PAPERCLIP_TOOL_ACTION_SIGNING_SECRET`
+- `PAPERCLIP_SECRETS_MASTER_KEY` (64-char hex). Held in OpenBao, not on the PVC, so
+  encrypted agent secrets in Postgres survive PVC loss.
+
+### Ingress and SSO
+- Hostname `paperclip.renner.uno`, Traefik `Ingress`, `cert-manager.io/cluster-issuer:
+  letsencrypt-cluster`, **no** `dns.renner.uno/public` label (LAN-only).
+- Authentik gate, per the paperless pattern:
+  - blueprint entries in `kubernetes/apps/authentik/blueprints-configmap.yaml`:
+    proxy provider `paperclip-gate` (`internal_host:
+    http://paperclip.paperclip.svc.cluster.local:3100`, `external_host:
+    https://paperclip.renner.uno`), application, group "Paperclip Users", policy binding;
+  - outpost `ak-outpost-paperclip-gate`;
+  - `kubernetes/apps/authentik/paperclip-gate-egress.yaml` (no `auth.renner.uno`
+    outpost Ingress: like wiki/shelly gates, the outpost serves its own callback on
+    `paperclip.renner.uno`);
+  - `ExternalName` Service in `paperclip` ns pointing at the outpost; the app Ingress
+    targets it on :9000.
+- Paperclip's own email/password login stays on (no header/OIDC login upstream).
+
+### Network policy (`kubernetes/apps/paperclip/network-policy.yaml`)
+Covers the server **and every agent** (same pod under B).
+- Ingress to the app pod: only from the `paperclip-gate` outpost (by label) on 3100.
+- Egress from the app pod:
+  - `pclp-pg` pods :5432
+  - kube-dns :53
+  - `ingress` namespace (Traefik) :8443 (pod-side websecure port) — reaches Gitea, Woodpecker, HA, Loki by their
+    `*.renner.uno` names
+  - `kube-apiserver` entity :6443
+  - `world` TCP :443 (Anthropic API, GitHub, npm)
+- No SSH egress; git over HTTPS only. The namespace baseline
+  (`paperclip-ns.yaml`) also restricts `world` to TCP 443 — Cilium unions allows
+  across CNPs, so an unrestricted baseline would override this list.
+- Known gap (house baseline): pods in `monitoring`/`database` and other pods in
+  `paperclip` can reach :3100 without the gate; Paperclip's own login still applies.
+
+### Image automation
+`kubernetes/infrastructure/image-automation/paperclip.yaml`: `ImageRepository`
+(`ghcr.io/paperclipai/paperclip`) + `ImagePolicy` filtering tag `^nightly$` with
+`digestReflectionPolicy: Always` (image-reflector-controller v1.2.5 in cluster; field
+available since v0.35.0). First use of digest reflection in this repo.
+
+## Guardrails
+- No bot receives cluster-admin or any binding in `flux-system`, `authentik`, `openbao`.
+- Git work goes through PRs; bots get no merge rights on protected branches.
+- Nothing is granted under B that we would not tolerate any agent misusing.
+
+## Bootstrap (one-time)
+1. Place `apps/paperclip` values in OpenBao (the three generated secrets).
+2. Merge; Flux deploys with sign-up enabled.
+3. Sign up through the gate and claim the instance as admin; then set
+   `PAPERCLIP_AUTH_DISABLE_SIGN_UP=true`.
+4. In onboarding "Connect a model", choose Claude subscription and run the shown login
+   **as the node user** (the entrypoint makes exec sessions root, and root-owned 0600
+   credentials are unreadable to the server):
+   `kubectl exec -it -n paperclip deploy/paperclip -- gosu node sh -c 'export CLAUDE_CONFIG_DIR=<shown path> && mkdir -p "$CLAUDE_CONFIG_DIR" && claude auth login'`
+   Repeat after loss of the PVC or database.
+
+## Verification
+- Flux Kustomization `paperclip` Ready; CNPG `pclp-pg` healthy; first ScheduledBackup
+  completes.
+- `curl https://paperclip.renner.uno/api/health` through the gate returns healthy after
+  Authentik login; unauthenticated request redirects to `auth.renner.uno`.
+- Direct pod access from another namespace is denied (CNP).
+- A test agent (claude-local) completes a trivial task using the shared Claude connection.
+- Flux ImagePolicy reports a `nightly@sha256:` digest.
+
+## Deferred
+- **Option C** (sandboxed runs) — required before any per-bot credential isolation.
+- **AIR-Manager** role and all bot identities/credentials.
+- Public exposure.

kubernetes/PROPOSAL-wiki-web-api.md

@@ -72,8 +72,7 @@ Rules that keep it modular, enforced by project references and a test:
 
 | port | what it abstracts | first adapter | examples of the next ones |
 |---|---|---|---|
-| `INoteStore` | read notes, raw sources and reports; enumerate by kind; observe changes (content hash per document) | Gitea repository API (`k.renner/notes`, blob SHA) | git checkout on a volume; a database (CNPG Postgres) with the vault imported; an Obsidian sync folder |
-| `IFactSink` | append a new raw source with provenance | Gitea contents API, new file on `main` | database insert; git commit on a checkout |
+| `INoteStore` | read notes, raw sources and reports at a head; create a new document (a fact becomes a raw note in the application layer, the store only moves text) | Gitea repository API (`k.renner/notes`, blob SHA) | git checkout on a volume; a database (CNPG Postgres) with the vault imported; an Obsidian sync folder |
 | `IEmbeddingModel` | text → vector, with a model id and dimension | llama-server `/v1/embeddings` (bge-m3 via wiki-embed, the model that won the German-document comparison on accuracy per cost) | the qwen3 embedding variants from that comparison; ONNX in-process; any OpenAI-compatible endpoint |
 | `IVectorIndex` | store vectors per (model id, document), query top-k | in-memory cosine with a file snapshot per model, as today | pgvector in the database; Qdrant |
 | `IChunker` | split a document into addressable units for typed questions | paragraph splitter (headings, fences), as in `wiki_ask.py` | sentence windows; semantic chunks |
@@ -99,7 +98,7 @@ Wiki.slnx
 src/
   Wiki.Core/               domain types, ports, frontmatter and link parsing
   Wiki.Application/        use cases; depends on Core only
-  Wiki.Adapters.Gitea/     INoteStore, IFactSink
+  Wiki.Adapters.Gitea/     INoteStore (read at a head, create)
   Wiki.Adapters.LlamaCpp/  IEmbeddingModel (OpenAI-compatible embeddings)
   Wiki.Adapters.TypeSafe/  IJudge (System One client)
   Wiki.Adapters.FileIndex/ IVectorIndex (memory + snapshot per model)
@@ -291,7 +290,7 @@ until phase 3 proves parity.
    Verify: `test_wiki_ask.py` passes against the API; a wiki-synth run with
    the definition unchanged. Then the Python service is retired and the
    `wiki-index` hostname points at the new one.
-4. **Facts.** API keys as an ExternalSecret, `IFactSink`, `/api/v1/facts`. Verify:
+4. **Facts.** API keys as an ExternalSecret, the deposit-fact use case, `/api/v1/facts`. Verify:
    a fact posted from the workstation shows as a `claude-bot` commit and in
    the next distill.
 5. **Questions.** Anthropic adapter, `/api/v1/questions` with streaming,

kubernetes/apps/audiobookshelf/deployment.yaml

@@ -122,7 +122,7 @@ spec:
             claimName: audiobookshelf-metadata
         - name: books
           persistentVolumeClaim:
-            claimName: audiobookshelf-books
+            claimName: audiobookshelf-books-bulk
         - name: le-ca
           emptyDir:
             medium: Memory

kubernetes/apps/audiobookshelf/pvcs.yaml

@@ -35,14 +35,14 @@ spec:
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
-  name: audiobookshelf-books
+  name: audiobookshelf-books-bulk
   namespace: audiobookshelf
   annotations:
     kustomize.toolkit.fluxcd.io/prune: disabled
 spec:
   accessModes:
     - ReadWriteMany
-  storageClassName: ceph-filesystem
+  storageClassName: ceph-filesystem-bulk
   resources:
     requests:
       storage: 300Gi

kubernetes/apps/authentik/blueprints-configmap.yaml

@@ -976,6 +976,280 @@ data:
             kubernetes_ingress_secret_name: "authentik-outpost-tls"
             kubernetes_json_patches: null
 
+      # wiki-gate: the wiki service (ns ai) behind the same passkey flows as
+      # llama-gate. The service trusts X-Authentik-Username only from pod
+      # addresses, and its CNP admits only this outpost, so the gate is the
+      # whole access control for wiki.renner.uno.
+      - id: wiki-gate-provider
+        model: authentik_providers_proxy.proxyprovider
+        identifiers:
+          name: wiki-gate
+        state: present
+        attrs:
+          authentication_flow: !Find [authentik_flows.flow, [slug, passkey-passwordless-authentication]]
+          authorization_flow: !Find [authentik_flows.flow, [slug, passkey-stepup-authorization]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          mode: proxy
+          internal_host: http://wiki.ai.svc.cluster.local:8080
+          internal_host_ssl_validation: false
+          external_host: https://wiki.renner.uno
+          cookie_domain: renner.uno
+          access_token_validity: hours=24
+          refresh_token_validity: days=30
+          intercept_header_auth: true
+
+      - model: authentik_core.application
+        identifiers:
+          slug: wiki-gate
+        state: present
+        attrs:
+          name: Wiki
+          slug: wiki-gate
+          provider: !KeyOf wiki-gate-provider
+          meta_launch_url: https://wiki.renner.uno
+          meta_description: "Notes wiki: search, notes, answers, maintenance"
+
+      - model: authentik_outposts.outpost
+        identifiers:
+          name: wiki-gate
+        state: present
+        attrs:
+          type: proxy
+          service_connection: !KeyOf k8s-conn
+          providers:
+            - !KeyOf wiki-gate-provider
+          config:
+            log_level: info
+            authentik_host: "http://authentik-server.authentik.svc.cluster.local"
+            authentik_host_insecure: false
+            authentik_host_browser: "https://auth.renner.uno/"
+            refresh_interval: "minutes=5"
+            container_image: null
+            object_naming_template: "ak-outpost-%(name)s"
+            kubernetes_namespace: authentik
+            kubernetes_replicas: 1
+            kubernetes_service_type: ClusterIP
+            kubernetes_disabled_components: ["ingress"]
+            kubernetes_image_pull_secrets: []
+            kubernetes_ingress_annotations: {}
+            kubernetes_ingress_class_name: null
+            kubernetes_ingress_secret_name: "authentik-outpost-tls"
+            kubernetes_json_patches: null
+
+      # paperclip-gate: Paperclip UI; the CNP admits only this outpost. Paperclip keeps its own login behind it.
+      - id: paperclip-users-group
+        model: authentik_core.group
+        identifiers:
+          name: Paperclip Users
+        state: present
+        attrs:
+          is_superuser: false
+          users:
+            - !Find [authentik_core.user, [username, k.renner]]
+
+      - id: paperclip-gate-provider
+        model: authentik_providers_proxy.proxyprovider
+        identifiers:
+          name: paperclip-gate
+        state: present
+        attrs:
+          authentication_flow: !Find [authentik_flows.flow, [slug, passkey-passwordless-authentication]]
+          authorization_flow: !Find [authentik_flows.flow, [slug, passkey-stepup-authorization]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          mode: proxy
+          internal_host: http://paperclip.paperclip.svc.cluster.local:3100
+          internal_host_ssl_validation: false
+          external_host: https://paperclip.renner.uno
+          cookie_domain: renner.uno
+          access_token_validity: hours=24
+          refresh_token_validity: days=30
+          intercept_header_auth: true
+
+      - model: authentik_core.application
+        identifiers:
+          slug: paperclip-gate
+        state: present
+        attrs:
+          name: Paperclip
+          slug: paperclip-gate
+          provider: !KeyOf paperclip-gate-provider
+          meta_launch_url: https://paperclip.renner.uno
+          meta_description: "Paperclip agent orchestration"
+
+      - model: authentik_policies.policybinding
+        identifiers:
+          target: !Find [authentik_core.application, [slug, paperclip-gate]]
+          group: !KeyOf paperclip-users-group
+          order: 0
+        state: present
+        attrs:
+          enabled: true
+          negate: false
+
+      - model: authentik_outposts.outpost
+        identifiers:
+          name: paperclip-gate
+        state: present
+        attrs:
+          type: proxy
+          service_connection: !KeyOf k8s-conn
+          providers:
+            - !KeyOf paperclip-gate-provider
+          config:
+            log_level: info
+            authentik_host: "http://authentik-server.authentik.svc.cluster.local"
+            authentik_host_insecure: false
+            authentik_host_browser: "https://auth.renner.uno/"
+            refresh_interval: "minutes=5"
+            container_image: null
+            object_naming_template: "ak-outpost-%(name)s"
+            kubernetes_namespace: authentik
+            kubernetes_replicas: 1
+            kubernetes_service_type: ClusterIP
+            kubernetes_disabled_components: ["ingress"]
+            kubernetes_image_pull_secrets: []
+            kubernetes_ingress_annotations: {}
+            kubernetes_ingress_class_name: null
+            kubernetes_ingress_secret_name: "authentik-outpost-tls"
+            kubernetes_json_patches: null
+
+      # navidrome-gate: the Navidrome web UI. Navidrome auto-creates the
+      # X-Authentik-Username user on first login; the first user is admin.
+      # /rest and /share bypass the gate at the Traefik router.
+      - id: navidrome-gate-provider
+        model: authentik_providers_proxy.proxyprovider
+        identifiers:
+          name: navidrome-gate
+        state: present
+        attrs:
+          authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          mode: proxy
+          internal_host: http://navidrome.navidrome.svc.cluster.local
+          internal_host_ssl_validation: false
+          external_host: https://navidrome.renner.uno
+          cookie_domain: renner.uno
+          access_token_validity: hours=24
+          refresh_token_validity: days=30
+          intercept_header_auth: true
+
+      - model: authentik_core.application
+        identifiers:
+          slug: navidrome-gate
+        state: present
+        attrs:
+          name: Navidrome
+          slug: navidrome-gate
+          provider: !KeyOf navidrome-gate-provider
+          meta_launch_url: https://navidrome.renner.uno
+          meta_description: "Music streaming (Subsonic)"
+
+      - model: authentik_outposts.outpost
+        identifiers:
+          name: navidrome-gate
+        state: present
+        attrs:
+          type: proxy
+          service_connection: !KeyOf k8s-conn
+          providers:
+            - !KeyOf navidrome-gate-provider
+          config:
+            log_level: info
+            authentik_host: "http://authentik-server.authentik.svc.cluster.local"
+            authentik_host_insecure: false
+            authentik_host_browser: "https://auth.renner.uno/"
+            refresh_interval: "minutes=5"
+            container_image: null
+            object_naming_template: "ak-outpost-%(name)s"
+            kubernetes_namespace: authentik
+            kubernetes_replicas: 1
+            kubernetes_service_type: ClusterIP
+            kubernetes_disabled_components: ["ingress"]
+            kubernetes_image_pull_secrets: []
+            kubernetes_ingress_annotations: {}
+            kubernetes_ingress_class_name: null
+            kubernetes_ingress_secret_name: "authentik-outpost-tls"
+            kubernetes_json_patches: null
+
+      # lidarr-gate: Lidarr runs with AuthenticationMethod=External, so the gate
+      # is its only access control.
+      - id: music-librarians-group
+        model: authentik_core.group
+        identifiers:
+          name: Music Librarians
+        state: present
+        attrs:
+          is_superuser: false
+          # Membership is owned here; blueprint apply reverts UI edits.
+          users:
+            - !Find [authentik_core.user, [username, k.renner]]
+
+      - id: lidarr-gate-provider
+        model: authentik_providers_proxy.proxyprovider
+        identifiers:
+          name: lidarr-gate
+        state: present
+        attrs:
+          authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          mode: proxy
+          internal_host: http://lidarr.navidrome.svc.cluster.local
+          internal_host_ssl_validation: false
+          external_host: https://lidarr.renner.uno
+          cookie_domain: renner.uno
+          access_token_validity: hours=24
+          refresh_token_validity: days=30
+          intercept_header_auth: true
+
+      - model: authentik_core.application
+        identifiers:
+          slug: lidarr-gate
+        state: present
+        attrs:
+          name: Lidarr
+          slug: lidarr-gate
+          provider: !KeyOf lidarr-gate-provider
+          meta_launch_url: https://lidarr.renner.uno
+          meta_description: "Music library gaps: discographies, Spotify artists"
+
+      - model: authentik_policies.policybinding
+        identifiers:
+          target: !Find [authentik_core.application, [slug, lidarr-gate]]
+          group: !KeyOf music-librarians-group
+          order: 0
+        state: present
+        attrs:
+          enabled: true
+          negate: false
+          failure_result: false
+
+      - model: authentik_outposts.outpost
+        identifiers:
+          name: lidarr-gate
+        state: present
+        attrs:
+          type: proxy
+          service_connection: !KeyOf k8s-conn
+          providers:
+            - !KeyOf lidarr-gate-provider
+          config:
+            log_level: info
+            authentik_host: "http://authentik-server.authentik.svc.cluster.local"
+            authentik_host_insecure: false
+            authentik_host_browser: "https://auth.renner.uno/"
+            refresh_interval: "minutes=5"
+            container_image: null
+            object_naming_template: "ak-outpost-%(name)s"
+            kubernetes_namespace: authentik
+            kubernetes_replicas: 1
+            kubernetes_service_type: ClusterIP
+            kubernetes_disabled_components: ["ingress"]
+            kubernetes_image_pull_secrets: []
+            kubernetes_ingress_annotations: {}
+            kubernetes_ingress_class_name: null
+            kubernetes_ingress_secret_name: "authentik-outpost-tls"
+            kubernetes_json_patches: null
+
       - model: authentik_outposts.outpost
         identifiers:
           name: authentik Embedded Outpost

kubernetes/apps/authentik/kustomization.yaml

@@ -7,6 +7,7 @@ resources:
   - helmrelease.yaml
   - paperless-gate-ingress.yaml
   - paperless-gate-egress.yaml
+  - paperclip-gate-egress.yaml
   - vscode-gate-egress.yaml
   - shelly-manager-gate-egress.yaml
   - esphome-gate-egress.yaml
@@ -16,6 +17,9 @@ resources:
   - prometheus-gate-egress.yaml
   - wordpress-gate-egress.yaml
   - llama-gate-egress.yaml
+  - wiki-gate-egress.yaml
+  - navidrome-gate-egress.yaml
+  - lidarr-gate-egress.yaml
   - ldap-outpost-cert.yaml
   - ldap-outpost-lb.yaml
   - ldap-outpost-cnp.yaml

kubernetes/apps/authentik/lidarr-gate-egress.yaml

@@ -0,0 +1,20 @@
+# Lets the lidarr-gate outpost reach Lidarr; the matching ingress rule is in
+# network-policies/navidrome-ns.yaml.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: lidarr-gate-egress
+  namespace: authentik
+spec:
+  endpointSelector:
+    matchLabels:
+      goauthentik.io/outpost-name: lidarr-gate
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": navidrome
+            app.kubernetes.io/name: lidarr
+      toPorts:
+        - ports:
+            - port: "8686"
+              protocol: TCP

kubernetes/apps/authentik/navidrome-gate-egress.yaml

@@ -0,0 +1,20 @@
+# Lets the navidrome-gate outpost reach Navidrome; the matching ingress rule
+# is in network-policies/navidrome-ns.yaml.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: navidrome-gate-egress
+  namespace: authentik
+spec:
+  endpointSelector:
+    matchLabels:
+      goauthentik.io/outpost-name: navidrome-gate
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": navidrome
+            app.kubernetes.io/name: navidrome
+      toPorts:
+        - ports:
+            - port: "4533"
+              protocol: TCP

kubernetes/apps/authentik/paperclip-gate-egress.yaml

@@ -0,0 +1,19 @@
+# Lets the paperclip-gate outpost reach the Paperclip server; scoped to this outpost's pods.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: paperclip-gate-egress
+  namespace: authentik
+spec:
+  endpointSelector:
+    matchLabels:
+      goauthentik.io/outpost-name: paperclip-gate
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+            app.kubernetes.io/name: paperclip
+      toPorts:
+        - ports:
+            - port: "3100"
+              protocol: TCP

kubernetes/apps/authentik/wiki-gate-egress.yaml

@@ -0,0 +1,21 @@
+# Lets the wiki-gate outpost reach the wiki service. The authentik namespace
+# baseline CNP otherwise stops any authentik pod from entering ai. Scoped to
+# this outpost's pod label; the matching ingress rule is in ai-ns.yaml.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: wiki-gate-egress
+  namespace: authentik
+spec:
+  endpointSelector:
+    matchLabels:
+      goauthentik.io/outpost-name: wiki-gate
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ai
+            app.kubernetes.io/name: wiki
+      toPorts:
+        - ports:
+            - port: "8080"
+              protocol: TCP

kubernetes/apps/gatus/configmap.yaml

@@ -30,6 +30,35 @@ metadata:
 #     Web Service 9 = Music Assistant
 #     Web Service 10 = TRMNL
 #     Web Service 11 = Paperless (gate-level only, see its entry)
+#
+# CORRECTION (REN-21, 2026-09-25) to the Web Service 11 comment further down,
+# which claims that an in-cluster probe of the paperless ClusterIP "needs a
+# CiliumNetworkPolicy first — the monitoring probes get away without one only
+# because their targets share a namespace with the prober".
+#
+# That is wrong, and it was wrong when written. It is stated here rather than
+# fixed in place because the sentence lives inside the `config.yaml: |` block
+# scalar below: any edit to it changes ConfigMap DATA, and
+# `configmap.reloader.stakater.com/reload: gatus-config`
+# (gatus/deployment.yaml:13) would then roll the gatus Deployment — a
+# production restart for a comment. This header is outside `data:`, so it is
+# inert; the served config is byte-identical.
+#
+#   * infrastructure/network-policies/paperless-ns.yaml:24-26 ALREADY admits
+#     the whole monitoring namespace on every port, and monitoring is where
+#     blackbox-exporter runs. Cilium unions the allow rules of every CNP
+#     selecting an endpoint, so no extra policy is required.
+#   * The in-cluster paperless probe therefore exists, added with no change
+#     to the paperless namespace:
+#     monitoring-rules/probes-blackbox.yaml, target
+#     http://paperless.paperless.svc.cluster.local:80/accounts/login/
+#   * The claim does NOT generalise to gatus itself. Gatus runs in `apps`,
+#     which is a different source namespace; a gatus-sourced in-cluster check
+#     would need its own allow, exactly as
+#     cnp-cross-namespace-allow-apps-gatus-to-monitored-services describes.
+#
+# The Web Service 11 entry below stays a reachability-only check on purpose.
+# The app-health half of the pair is the blackbox probe named above.
 data:
   config.yaml: |
     storage:

kubernetes/apps/gatus/deployment.yaml

@@ -46,7 +46,7 @@ spec:
           type: RuntimeDefault
       containers:
         - name: gatus
-          image: twinproduction/gatus:v5.36.0 # {"$imagepolicy": "flux-system:gatus"}
+          image: twinproduction/gatus:v5.37.0 # {"$imagepolicy": "flux-system:gatus"}
           securityContext:
             allowPrivilegeEscalation: false
             capabilities:

kubernetes/apps/home-assistant/kustomization.yaml

@@ -16,6 +16,9 @@ resources:
   # health broker credential: OpenBao-backed + Class-A rotated (first of the
   # eight users to migrate off mosquitto-bootstrap.yaml).
   - mosquitto-health-credentials-externalsecret.yaml
+  # open3e-int broker credential: open3e-esp32 gateway on the Vitocal internal
+  # CAN; OpenBao-backed + Class-A rotated.
+  - mosquitto-open3e-int-credentials-externalsecret.yaml
   - mosquitto-config.yaml
   - mosquitto-certificate.yaml
   - mosquitto-service.yaml

kubernetes/apps/home-assistant/mosquitto-deployment.yaml

@@ -12,7 +12,7 @@ metadata:
     #
     # Listed per-Secret rather than with the blanket auto annotation, so that
     # enrolling another broker user is a visible edit to this line.
-    secret.reloader.stakater.com/reload: mosquitto-health-credentials
+    secret.reloader.stakater.com/reload: mosquitto-health-credentials,mosquitto-open3e-int-credentials
 spec:
   replicas: 1
   strategy:
@@ -78,6 +78,7 @@ spec:
               mosquitto_passwd -b /mosquitto/auth/passwd admin "$ADMIN_PASSWORD"
               mosquitto_passwd -b /mosquitto/auth/passwd sml2mqtt "$SML2MQTT_PASSWORD"
               mosquitto_passwd -b /mosquitto/auth/passwd open3e "$OPEN3E_PASSWORD"
+              mosquitto_passwd -b /mosquitto/auth/passwd open3e-int "$OPEN3E_INT_PASSWORD"
               mosquitto_passwd -b /mosquitto/auth/passwd evcc "$EVCC_PASSWORD"
               mosquitto_passwd -b /mosquitto/auth/passwd health "$HEALTH_PASSWORD"
               mosquitto_passwd -b /mosquitto/auth/passwd zigbee2mqtt "$ZIGBEE2MQTT_PASSWORD"
@@ -124,6 +125,12 @@ spec:
               user open3e
               topic readwrite open3e/#
 
+              # open3e-esp32 gateway on the Vitocal internal CAN: own subtree
+              # under open3e/ so edge-1's listener on open3e/cmnd never sees
+              # its commands.
+              user open3e-int
+              topic readwrite open3e/int/#
+
               user evcc
               topic read open3e/#
               topic readwrite evcc/#
@@ -187,6 +194,13 @@ spec:
                 secretKeyRef:
                   name: mosquitto-open3e-credentials
                   key: password
+            # open3e-int: OpenBao-backed + Class-A rotated (see
+            # mosquitto-open3e-int-credentials-externalsecret.yaml).
+            - name: OPEN3E_INT_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: mosquitto-open3e-int-credentials
+                  key: password
             - name: EVCC_PASSWORD
               valueFrom:
                 secretKeyRef:

kubernetes/apps/home-assistant/mosquitto-open3e-int-credentials-externalsecret.yaml

@@ -0,0 +1,21 @@
+# Projects the mosquitto `open3e-int` broker credential (open3e-esp32 gateway
+# on the Vitocal internal CAN) from OpenBao into the Secret the broker
+# consumes. Rotator: kubernetes/infrastructure/openbao/rotation/mosquitto-open3e-int.yaml.
+#
+# creationPolicy is implicitly Owner: deleting this ExternalSecret deletes the
+# live Secret. Set creationPolicy: Orphan first to detach without destroying.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: mosquitto-open3e-int-credentials
+  namespace: home-assistant
+spec:
+  refreshInterval: 10m
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: mosquitto-open3e-int-credentials
+  dataFrom:
+    - extract:
+        key: apps/home-assistant/mosquitto-open3e-int-credentials

kubernetes/apps/immich/pvcs.yaml

@@ -28,6 +28,38 @@ spec:
     requests:
       storage: 300Gi
 ---
+# Media root on the EC 2+1 bulk pool; thumbs mount separately below.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: immich-library-bulk
+  namespace: immich
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled
+spec:
+  accessModes:
+    - ReadWriteMany
+  storageClassName: ceph-filesystem-bulk
+  resources:
+    requests:
+      storage: 300Gi
+---
+# Thumbnails stay replicated: read affinity serves them from the local OSD.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: immich-thumbs
+  namespace: immich
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled
+spec:
+  accessModes:
+    - ReadWriteMany
+  storageClassName: ceph-filesystem
+  resources:
+    requests:
+      storage: 30Gi
+---
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:

kubernetes/apps/immich/server.yaml

@@ -110,6 +110,8 @@ spec:
           volumeMounts:
             - name: library
               mountPath: /data
+            - name: thumbs
+              mountPath: /data/thumbs
             - name: immich-config-file
               mountPath: /config
               readOnly: true
@@ -145,7 +147,10 @@ spec:
       volumes:
         - name: library
           persistentVolumeClaim:
-            claimName: immich-library
+            claimName: immich-library-bulk
+        - name: thumbs
+          persistentVolumeClaim:
+            claimName: immich-thumbs
         - name: immich-config-file
           secret:
             secretName: immich-secrets

kubernetes/apps/libation/cronjob.yaml

@@ -4,7 +4,7 @@
 # the entrypoint exit after one pass instead of looping internally —
 # the CronJob handles scheduling.
 #
-# Books land in /audiobooks (the audiobookshelf-books PVC, shared
+# Books land in /audiobooks (the audiobookshelf-books-bulk PVC, shared
 # with the audiobookshelf Deployment). Config + DB live on
 # libation-config (RWO ceph-block, seeded once from
 # libation-bootstrap secret).
@@ -107,6 +107,9 @@ spec:
                 - |
                   trap 'cp /config-internal/AccountsSettings.json /libation-state/AccountsSettings.json' EXIT
                   set -e
+                  # Strip the Audible intro/outro; re-liberated books replace their file instead of gaining a " (1)" copy.
+                  jq '.StripAudibleBrandAudio = true | .OverwriteExisting = true' /config-internal/Settings.json > /config-internal/Settings.json.tmp
+                  mv /config-internal/Settings.json.tmp /config-internal/Settings.json
                   /libation/LibationCli scan
                   /libation/LibationCli liberate
               env:
@@ -135,7 +138,7 @@ spec:
                 claimName: libation-config
             - name: books
               persistentVolumeClaim:
-                claimName: audiobookshelf-books
+                claimName: audiobookshelf-books-bulk
             - name: bootstrap
               secret:
                 secretName: libation-bootstrap

kubernetes/apps/libation/pvc.yaml

@@ -5,7 +5,7 @@
 # SOPS-encrypted Secret on first boot (sentinel file gate); after that
 # Libation reads and writes it directly so token rotation persists.
 #
-# Books are written to the audiobookshelf-books PVC (already provisioned
+# Books are written to the audiobookshelf-books-bulk PVC (already provisioned
 # by the audiobookshelf seed Kustomization, RWX ceph-filesystem). We
 # share the same namespace so the PVC reference is direct — k8s PVCs
 # are namespace-scoped.

kubernetes/apps/monitoring-rules/external-secrets-podmonitor.yaml

@@ -0,0 +1,71 @@
+# Scrape the external-secrets (ESO) controller's own metrics. Until this
+# existed, nothing in this cluster could see a failed ExternalSecret sync:
+# `externalsecret_status_condition` held 0 series, so no rule could be written
+# about a key that silently stops syncing (REN-100).
+#
+# This is the scrape only. The alert rule is deliberately NOT in this change —
+# its `for:` window has to be tuned against real data, and that data does not
+# exist until this has been live for a while. See REN-103.
+#
+# A PodMonitor, NOT the chart's `serviceMonitor.enabled`, and NOT a hand-written
+# ServiceMonitor. Both alternatives were checked against chart 2.11.0:
+#
+#   * A hand-written ServiceMonitor has nothing to select. `metrics.service.enabled`
+#     defaults to false and infrastructure/external-secrets/helmrelease.yaml
+#     overrides nothing, so templates/service.yaml renders no Service for the
+#     controller. The container port exists (`name: metrics`, containerPort 8080)
+#     but nothing fronts it — the same trap that forced a PodMonitor for the Flux
+#     controllers.
+#   * Setting `serviceMonitor.enabled: true` would work, but it triggers a Helm
+#     UPGRADE of ESO, which re-renders the Deployment and re-applies its CRDs
+#     (`crds: CreateReplace` in the HelmRelease). ESO is what all 54 ExternalSecret
+#     manifests in this repo depend on. A PodMonitor gets the same series while
+#     touching the release not at all, which is the trade this repo already made
+#     for Velero's node-agent (velero-podmonitor.yaml) and for Flux.
+#
+# THIS NEEDS A NETWORK POLICY CHANGE, unlike the Flux PodMonitor: the cluster is
+# default-deny and external-secrets-ns.yaml admitted only intra-namespace traffic,
+# host/remote-node and ICMP — no `monitoring` grant, so this scrape would have
+# been dropped and the metric would have stayed absent. The matching ingress rule
+# is added to infrastructure/network-policies/external-secrets-ns.yaml in the
+# same pull request. Without it this file is decoration.
+apiVersion: monitoring.coreos.com/v1
+kind: PodMonitor
+metadata:
+  name: external-secrets
+  namespace: monitoring
+  labels:
+    app.kubernetes.io/name: external-secrets
+    # Not required for discovery — podMonitorSelector is empty, see
+    # apps/monitoring/helmrelease-prometheus.yaml — but set for consistency with
+    # the other monitors in this directory.
+    release: kube-prometheus-stack
+spec:
+  # Lives in `monitoring` rather than in `external-secrets`, matching
+  # external-dns-servicemonitor.yaml and the Flux PodMonitor. Every object in
+  # the external-secrets namespace is owned by the ESO HelmRelease; keeping this
+  # out of that namespace means this change cannot interact with the release.
+  namespaceSelector:
+    matchNames:
+      - external-secrets
+  selector:
+    matchLabels:
+      # The CONTROLLER only. The webhook and cert-controller carry
+      # `app.kubernetes.io/name: external-secrets-webhook` and
+      # `-cert-controller` respectively (see the chart's
+      # external-secrets-webhook.selectorLabels / -cert-controller.selectorLabels
+      # helpers), so an exact match on the unsuffixed name excludes both. That is
+      # intended: the `externalsecret_*` series are emitted by the controller, and
+      # the other two would only add idle targets.
+      app.kubernetes.io/name: external-secrets
+      app.kubernetes.io/instance: external-secrets
+  podMetricsEndpoints:
+    # Named container port, not a hardcoded 8080, so this follows
+    # `metrics.listen.port` if it is ever overridden in the HelmRelease.
+    - port: metrics
+      # 60s, not the chart's 30s default: `externalsecret_status_condition` is a
+      # slow-moving gauge over ~54 ExternalSecrets, and the alert built on it in
+      # REN-103 will use a multi-minute `for:` window. 30s would double the
+      # samples for no gain in detection time.
+      interval: 60s
+      scrapeTimeout: 10s

kubernetes/apps/monitoring-rules/kustomization.yaml

@@ -48,4 +48,8 @@ resources:
   - loki-servicemonitor.yaml
   - smartctl-exporter-servicemonitor.yaml
   - intel-gpu-exporter-servicemonitor.yaml
+  # ESO's controller metrics. Needs the `monitoring` ingress grant added to
+  # infrastructure/network-policies/external-secrets-ns.yaml in the same change,
+  # or it scrapes nothing. See the file.
+  - external-secrets-podmonitor.yaml
   - probes-blackbox.yaml

kubernetes/apps/monitoring-rules/probes-blackbox.yaml

@@ -88,6 +88,166 @@ spec:
         # /ready answers 200 unauthenticated and is in this namespace, so the
         # same intra-namespace allowance the two above rely on covers it.
         - http://loki.monitoring.svc.cluster.local:3100/ready
+        # APP-HEALTH probes for the two gated navidrome-namespace apps. Same
+        # rule as the block above (monitoring-vantage-point-rule), but these
+        # are the first targets here that leave the monitoring namespace.
+        #
+        # Both sit entirely behind an Authentik outpost — navidrome.renner.uno
+        # on ak-outpost-navidrome-gate (navidrome/ingress.yaml:21),
+        # lidarr.renner.uno on ak-outpost-lidarr-gate
+        # (navidrome/lidarr.yaml:218) — so an EXTERNAL probe of either name
+        # measures the gate, not the app, and stays green while the app behind
+        # a live gate is dead. That external signal is still wanted, and REN-2
+        # adds it from UptimeRobot; these are the other half of the pair.
+        #
+        # NO CiliumNetworkPolicy change is needed — but NOT because
+        # cnp-cross-namespace-allow-apps-gatus-to-monitored-services fails to
+        # apply here. It applies: its Generalization section names
+        # blackbox-exporter by hand. The reason we are safe is that the allow
+        # that note demands is ALREADY PRESENT — the navidrome baseline CNP
+        # admits the whole monitoring namespace on every port
+        # (infrastructure/network-policies/navidrome-ns.yaml:25-27), which is
+        # where blackbox-exporter runs. Run that check for the next namespace
+        # you add; do not skip it on the grounds that the note "is about
+        # gatus". Nothing in the navidrome namespace is edited by this change.
+        #
+        # EGRESS — the half the wiki note is silent on, and the half that is
+        # actually load-bearing here. Cilium enforces at BOTH ends, so the
+        # destination ingress allow above is worthless if the prober itself
+        # is in egress enforcement without a matching rule. It is not: an
+        # endpoint only enters egress enforcement once some policy selecting
+        # it declares an `egress:` block, and none does for monitoring. The
+        # namespace has exactly two Cilium policies and both are ingress-only:
+        # infrastructure/network-policies/monitoring-ns.yaml
+        # (default-deny-ingress) and monitoring-alloy-syslog.yaml
+        # (allow-syslog-world-ingress). blackbox-exporter runs in that same
+        # namespace (apps/monitoring/helmrelease-blackbox-exporter.yaml:5).
+        # Re-verified at 2b71dda1 across all 635 yaml files in the repo — 73
+        # Cilium objects, 38 carrying an egress or egressDeny block, none of
+        # them in the monitoring namespace, and the single
+        # CiliumClusterwideNetworkPolicy is scoped to `reserved.health` so it
+        # cannot select a pod. 17 other namespaces here already have an egress
+        # block. If a routine hardening pass ever adds one to monitoring, it
+        # MUST list the navidrome and paperless namespaces or these three
+        # cross-namespace targets go red while the *.monitoring.svc ones above
+        # keep working — a failure mode nobody will connect back to the
+        # hardening commit. The cnp-cross-namespace-* notes document only the
+        # ingress half; that silence is what makes this the trap.
+        #
+        # BLAST RADIUS if that egress dependency is ever broken. Nothing
+        # user-facing breaks: these are Prometheus-side Probe targets, not
+        # gatus checks, so status.renner.uno is unaffected and no workload is
+        # restarted. What breaks is the signal, and it breaks misleadingly.
+        # All three targets start failing BlackboxProbeFailing
+        # (prometheusrule-blackbox.yaml:34 — severity warning, for: 10m); none
+        # of the three is in that rule's exclusion list. BlackboxProbesAbsent
+        # does NOT fire, because the other 17 targets keep producing
+        # probe_success — so it reads as three apps going down, not as a
+        # policy change. WHO NOTICES: warning routes to the default `email`
+        # receiver, admin@renner.uno (helmrelease-prometheus.yaml alertmanager
+        # route, group_wait 30s, grouped by alertname), so the owner gets one
+        # email roughly 10-11 minutes after the policy lands. HOW FAST UNDONE:
+        # the REVERT recipe below is a one-line deletion; from merge it is
+        # live within ~11 minutes (GitRepository interval 1m0s,
+        # monitoring-rules Kustomization interval 10m), or immediately with
+        # `flux reconcile kustomization monitoring-rules --with-source`.
+        #
+        # /ping on both: unauthenticated, and already the pods' own kubelet
+        # probe path (navidrome/deployment.yaml:61-74,
+        # navidrome/lidarr.yaml:114-127). For lidarr that it is ungated is
+        # proven independently of the gate — lidarr.yaml:29 curls /ping in a
+        # loop BEFORE it has read the API key out of config.xml.
+        #
+        # STATUS CODE, unproven for all three of the new targets: module
+        # http_2xx follows redirects and passes only on a FINAL 2xx, while
+        # every piece of repo evidence behind these paths establishes only
+        # `< 400` — a kubelet httpGet accepts 200-399, and lidarr.yaml:29's
+        # `curl -f` without -L exits 0 on a 302. That evidence proves the
+        # paths are UNGATED, not that they answer 200. Check
+        # probe_http_status_code on the first scrapes; a target that settles
+        # on a 3xx gets REVERTED, not widened to accept 3xx.
+        #
+        # REVERT — an executed recipe, not a claim. Every command below was
+        # actually run against master 2b71dda1 on 2026-09-25; the results are
+        # what is recorded here.
+        #
+        # NO `git revert` works. All four candidates conflict in THIS file at
+        # 2b71dda1, because the comment blocks around these targets were
+        # rewritten after the probes landed (#979):
+        #
+        #   git revert -m 1 0a82d61f   merge of #977, paperless     CONFLICT
+        #   git revert -m 1 16238d39   merge of #976, nav + lidarr  CONFLICT
+        #   git revert cb9e73bd        the paperless commit         CONFLICT
+        #   git revert 2219c928        the nav + lidarr commit      CONFLICT
+        #
+        # Hand-resolving any of them is the trap: the natural --theirs
+        # resolution of the 16238d39 conflict also drops the paperless target
+        # below, leaving NO app-health probes and no signal that it happened.
+        #
+        # Delete the target lines forward instead. Only the target lines are
+        # part of the object; the comments are inert. Both of these were run
+        # and verified:
+        #
+        #   # all three app-health targets — blackbox-http 20 -> 17
+        #   sed -i '/svc\.cluster\.local:80\/ping$/d;/paperless\.paperless\.svc\.cluster\.local:80\/accounts\/login\/$/d' \
+        #     kubernetes/apps/monitoring-rules/probes-blackbox.yaml
+        #
+        #   # the paperless target alone — blackbox-http 20 -> 19
+        #   sed -i '/paperless\.paperless\.svc\.cluster\.local:80\/accounts\/login\/$/d' \
+        #     kubernetes/apps/monitoring-rules/probes-blackbox.yaml
+        #
+        # Verified after each run: the file still parses, blackbox-http drops
+        # to 17 and to 19 respectively, blackbox-edge-wan (3) and
+        # blackbox-tcp-relays (2) are untouched, and `git diff` contains
+        # nothing but the deleted target lines. Then drop whichever comment
+        # block is left orphaned, open a PR, and merge — timing as in the
+        # BLAST RADIUS paragraph above.
+        - http://navidrome.navidrome.svc.cluster.local:80/ping
+        - http://lidarr.navidrome.svc.cluster.local:80/ping
+        # APP-HEALTH probe for paperless. Same rule as the block above
+        # (monitoring-vantage-point-rule); the blind spot it closes is the one
+        # gatus already documents at apps/gatus/configmap.yaml:176-185 —
+        # through the paperless-gate outpost EVERY path, /api/ included,
+        # answers 302 to an unauthenticated caller, so no external probe of
+        # paperless.renner.uno can tell a live app from a live gate. That
+        # external signal is still wanted, and REN-2 adds it from UptimeRobot;
+        # this is the other half of the pair.
+        #
+        # That gatus comment says the in-cluster probe "needs a
+        # CiliumNetworkPolicy first". That sentence is simply WRONG, not
+        # merely inapplicable: paperless-ns.yaml:24-26 already admits the
+        # whole monitoring namespace, which is where blackbox-exporter runs,
+        # and Cilium unions the allow rules of every CNP selecting an
+        # endpoint (the mechanism apps/paperless/network-policy.yaml:8-12
+        # relies on). The EGRESS paragraph in the block above applies here
+        # identically. Nothing in the paperless namespace is edited by this
+        # change.
+        #
+        # The stale sentence is deliberately left in place in
+        # apps/gatus/configmap.yaml: it sits INSIDE the `config.yaml: |`
+        # block scalar, so editing it changes ConfigMap data and
+        # `configmap.reloader.stakater.com/reload` (gatus/deployment.yaml:13)
+        # rolls the gatus Deployment. The correction is written at the top of
+        # that file instead, outside `data:`, where it costs no restart.
+        #
+        # /accounts/login/ because paperless-ngx has no unauthenticated health
+        # endpoint at all. It is the app's own established "the webserver
+        # answers" signal — the same path its startup, liveness and readiness
+        # probes use (paperless.yaml:252-272) — and it renders a Django view
+        # rather than a static file. It does NOT cover the consumer/celery
+        # workers: see process-liveness-vs-protocol-liveness.
+        #
+        # The cluster-local Host header is accepted only because
+        # PAPERLESS_ALLOWED_HOSTS is "*" (paperless.yaml:27). Narrow that and
+        # Django answers 400 here and this probe goes permanently red.
+        #
+        # This is the target most likely to fail the STATUS CODE check in the
+        # block above — a Django login view can answer 302 — so check it
+        # first, and revert rather than widen if it does. Use the REVERT
+        # recipe in that block: `git revert -m 1 0a82d61f` conflicts here too
+        # at 2b71dda1, and the tested one-line deletion for this target alone
+        # is the second command listed there (20 -> 19 targets).
+        - http://paperless.paperless.svc.cluster.local:80/accounts/login/
 # WAN-path probes of the public edge BY PROXY IP. The name-based
 # home.renner.uno probe above resolves in-cluster (CoreDNS/k8s_gateway)
 # and never leaves the LAN — the vantage-point blind spot that let the

kubernetes/apps/monitoring-rules/prometheusrule-ceph.yaml

@@ -52,19 +52,61 @@ spec:
         - alert: CephPoolLostRedundancy
           # Names the pool, which POOL_NO_REDUNDANCY does not.
           #
-          # There is no pool-size metric, but bytes_used/stored IS the
-          # replication factor. The >1e6 floor drops pools too small to measure
-          # meaningfully, empty ones included.
+          # There is no pool-size metric, but for a REPLICATED pool
+          # bytes_used/stored IS the replication factor. The >1e6 floor drops
+          # pools too small to measure meaningfully, empty ones included.
           #
-          # No pool is exempt.
+          # ERASURE-CODED POOLS ARE EXCLUDED, AND MUST STAY EXCLUDED (REN-88).
+          #
+          # That ratio is not a replication factor for an EC pool, it is the
+          # coding overhead: k+m shards over k data shards. For the two ec:2+1
+          # pools here that is 3/2 = exactly 1.5 — sitting precisely on this
+          # rule's own threshold, on the wrong side of it half the time.
+          # Measured on `cephfs-bulk` (pool 15): used 480042086400 / stored
+          # 320028049408 = 1.5000000384, a margin of 12 KiB in 480 GB, so which
+          # way the comparison lands is decided by the last bits of a float
+          # division. It paged critical for 199 minutes on 2026-09-25 (12:37 to
+          # 15:56 UTC, two emails) with the cluster HEALTH_OK, zero degraded
+          # PGs and every OSD up, then cleared on its own. 50 of the 210
+          # samples in the preceding 30h were below 1.5; only that run held the
+          # 30m `for`. `migration.rgw.buckets.data` (pool 12) is the same ec:2+1
+          # shape and was one drift away from doing the same thing.
+          #
+          # The premise fails for EC too, not just the arithmetic: the old
+          # description claimed a single OSD loss destroys data, but 2+1 carries
+          # a parity shard and survives one OSD loss, same as replica-2.
+          #
+          # Do NOT "fix" this by nudging the threshold to 1.4 or 1.49. That
+          # keeps a knife-edge, just a quieter one, and it also narrows the real
+          # replica-2 -> replica-1 signal this rule exists to catch (1.0).
+          #
+          # EC pools keep full redundancy coverage without this rule: a real
+          # loss raises POOL_NO_REDUNDANCY, which is a live series in
+          # `ceph_health_detail` and is deliberately in scope for
+          # CephHealthWarn above. That check reads Ceph's own verdict instead of
+          # inferring one from a byte ratio, which is the right instrument here
+          # — an EC pool that loses a shard does not change its stored/used
+          # ratio at all, because Ceph does not delete the surviving shards.
+          # So there is no gap to reopen, and no second EC-specific threshold
+          # worth inventing.
+          #
+          # `on(pool_id)` is deliberately the whole matcher. ceph_pool_metadata
+          # carries labels ceph_pool_bytes_used does not (name, type,
+          # description, application, compression_mode), so default all-label
+          # matching would never join. Adding `cluster` would be defensive
+          # against a second Ceph cluster that does not exist, at the cost of
+          # the rule silently matching nothing if that label is ever dropped —
+          # an alert that fails open invisibly is the worse trade.
           expr: |
-            ceph_pool_bytes_used / ceph_pool_stored < 1.5 and ceph_pool_stored > 1e6
+            ceph_pool_bytes_used / ceph_pool_stored < 1.5
+              and on(pool_id) ceph_pool_metadata{type="replicated"} == 1
+              and ceph_pool_stored > 1e6
           for: 30m
           labels:
             severity: critical
           annotations:
-            summary: "Ceph pool {{ $labels.pool_id }} is running unreplicated"
-            description: "Pool {{ $labels.pool_id }} has a replication factor below 1.5, meaning a single OSD loss destroys data. No pool is meant to be unreplicated — an unreplicated pool also blocks ALL OSD maintenance cluster-wide, since `ceph osd ok-to-stop` refuses while its PGs would go offline. Check with `kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph osd pool ls detail`."
+            summary: "Ceph replicated pool {{ $labels.pool_id }} is running unreplicated"
+            description: "Replicated pool {{ $labels.pool_id }} has a replication factor below 1.5, meaning it is down to a single copy and one OSD loss destroys data. No replicated pool is meant to be unreplicated — it also blocks OSD maintenance for as long as it lasts, since `ceph osd ok-to-stop` refuses while its PGs would go offline. Erasure-coded pools are out of scope for this rule and are covered by POOL_NO_REDUNDANCY via CephHealthWarn (REN-88). Check with `kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph osd pool ls detail`."
 
         - alert: CephHealthError
           # HEALTH_ERR means data availability or redundancy is at risk.

kubernetes/apps/navidrome/deployment.yaml

@@ -0,0 +1,104 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: navidrome-config
+  namespace: navidrome
+data:
+  TZ: "Europe/Berlin"
+  ND_LOGLEVEL: "info"
+  ND_SCANNER_SCHEDULE: "0 * * * *"
+  ND_ENABLEINSIGHTSCOLLECTOR: "false"
+  # Identity from the navidrome-gate outpost; the CNP admits only the outpost
+  # and Traefik, and Traefik strips this header on the ungated routes.
+  ND_EXTAUTH_USERHEADER: "X-Authentik-Username"
+  ND_EXTAUTH_TRUSTEDSOURCES: "fd10:244::/48,10.244.0.0/16"
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: navidrome
+  namespace: navidrome
+  labels:
+    app.kubernetes.io/name: navidrome
+  annotations:
+    configmap.reloader.stakater.com/reload: navidrome-config
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: navidrome
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: navidrome
+    spec:
+      enableServiceLinks: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1000
+        runAsGroup: 1000
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: navidrome
+          image: deluan/navidrome:0.64.2 # {"$imagepolicy": "flux-system:navidrome"}
+          ports:
+            - name: http
+              containerPort: 4533
+          envFrom:
+            - configMapRef:
+                name: navidrome-config
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: music
+              mountPath: /music
+              readOnly: true
+          livenessProbe:
+            httpGet:
+              path: /ping
+              port: http
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            timeoutSeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /ping
+              port: http
+            initialDelaySeconds: 10
+            periodSeconds: 10
+            timeoutSeconds: 5
+          resources:
+            requests:
+              cpu: 100m
+              memory: 256Mi
+            limits:
+              memory: 1Gi
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: navidrome-data
+        - name: music
+          persistentVolumeClaim:
+            claimName: navidrome-music-bulk
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: navidrome
+  namespace: navidrome
+spec:
+  selector:
+    app.kubernetes.io/name: navidrome
+  ports:
+    - name: http
+      port: 80
+      targetPort: http

kubernetes/apps/navidrome/gate-service.yaml

@@ -0,0 +1,13 @@
+# ExternalName mirror so the navidrome-namespace Ingress can backend to the
+# navidrome-gate outpost in the authentik namespace.
+apiVersion: v1
+kind: Service
+metadata:
+  name: ak-outpost-navidrome-gate
+  namespace: navidrome
+spec:
+  type: ExternalName
+  externalName: ak-outpost-navidrome-gate.authentik.svc.cluster.local
+  ports:
+    - port: 9000
+      targetPort: 9000

kubernetes/apps/navidrome/ingress.yaml

@@ -0,0 +1,69 @@
+# Web UI behind the navidrome-gate outpost.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: navidrome
+  namespace: navidrome
+  labels:
+    dns.renner.uno/public: "true"
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  rules:
+    - host: navidrome.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: ak-outpost-navidrome-gate
+                port:
+                  number: 9000
+  tls:
+    - secretName: navidrome-tls
+      hosts:
+        - navidrome.renner.uno
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: navidrome-strip-identity
+  namespace: navidrome
+spec:
+  headers:
+    customRequestHeaders:
+      X-Authentik-Username: ""
+---
+# Subsonic API and public shares bypass the gate; clients authenticate with
+# their Navidrome password.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: navidrome-subsonic
+  namespace: navidrome
+  labels:
+    dns.renner.uno/public: "true"
+  annotations:
+    traefik.ingress.kubernetes.io/router.middlewares: navidrome-navidrome-strip-identity@kubernetescrd
+spec:
+  ingressClassName: traefik
+  rules:
+    - host: navidrome.renner.uno
+      http:
+        paths:
+          - path: /rest
+            pathType: Prefix
+            backend:
+              service:
+                name: navidrome
+                port:
+                  number: 80
+          - path: /share
+            pathType: Prefix
+            backend:
+              service:
+                name: navidrome
+                port:
+                  number: 80

kubernetes/apps/navidrome/kustomization.yaml

@@ -0,0 +1,8 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - pvcs.yaml
+  - deployment.yaml
+  - gate-service.yaml
+  - ingress.yaml
+  - lidarr.yaml

kubernetes/apps/navidrome/lidarr.yaml

@@ -0,0 +1,224 @@
+# Lidarr as a read-mostly "what is missing" view over the Navidrome library.
+# No download client or indexer is configured.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: lidarr-config
+  namespace: navidrome
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 5Gi
+---
+# Applied on every pod start, before the root folder exists: no renames, no tag
+# writes, no deletes. Adding /music as root folder starts Lidarr's library import.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: lidarr-reconcile
+  namespace: navidrome
+data:
+  reconcile.sh: |
+    set -eu
+    API=http://localhost:8686/api/v1
+    until curl -sf http://localhost:8686/ping >/dev/null; do sleep 5; done
+    KEY=$(sed -n 's:.*<ApiKey>\(.*\)</ApiKey>.*:\1:p' /config/config.xml)
+    if [ -z "$KEY" ]; then
+      echo "FATAL: no ApiKey in /config/config.xml" >&2
+      exit 1
+    fi
+    H="X-Api-Key: $KEY"
+
+    put() {
+      cur=$(curl -sf -H "$H" "$API/$1")
+      id=$(printf '%s' "$cur" | jq -r '.id')
+      printf '%s' "$cur" | jq -c "$2" \
+        | curl -sf -X PUT -H "$H" -H 'Content-Type: application/json' -d @- "$API/$1/$id" >/dev/null
+      echo "reconciled $1"
+    }
+
+    put config/naming '.renameTracks = false'
+    put config/metadataprovider '.writeAudioTags = "no" | .scrubAudioTags = false | .embedCoverArt = false'
+    put config/mediamanagement '.deleteEmptyFolders = false | .createEmptyArtistFolders = false
+      | .recycleBin = "" | .setPermissionsLinux = false | .importExtraFiles = false
+      | .skipFreeSpaceCheckWhenImporting = true'
+
+    roots=$(curl -sf -H "$H" "$API/rootfolder" | jq '[.[] | select(.path == "/music")] | length')
+    if [ "$roots" = "0" ]; then
+      qp=$(curl -sf -H "$H" "$API/qualityprofile" | jq '.[0].id')
+      mp=$(curl -sf -H "$H" "$API/metadataprofile" | jq '([.[] | select(.name == "Standard")][0].id) // .[0].id')
+      jq -nc --argjson qp "$qp" --argjson mp "$mp" '{
+          name: "Music", path: "/music",
+          defaultQualityProfileId: $qp, defaultMetadataProfileId: $mp,
+          defaultMonitorOption: "all", defaultNewItemMonitorOption: "all",
+          defaultTags: []
+        }' \
+        | curl -sf -X POST -H "$H" -H 'Content-Type: application/json' -d @- "$API/rootfolder" >/dev/null
+      echo "added root folder /music"
+    fi
+
+    echo "reconcile done"
+    while true; do sleep 3600; done
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: lidarr
+  namespace: navidrome
+  labels:
+    app.kubernetes.io/name: lidarr
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: lidarr
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: lidarr
+    spec:
+      enableServiceLinks: false
+      securityContext:
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: lidarr
+          image: lscr.io/linuxserver/lidarr:3.1.0.4875-ls42 # {"$imagepolicy": "flux-system:lidarr"}
+          env:
+            - name: PUID
+              value: "1000"
+            - name: PGID
+              value: "1000"
+            - name: TZ
+              value: Europe/Berlin
+            # Identity is enforced by the lidarr-gate outpost.
+            - name: LIDARR__AUTH__METHOD
+              value: External
+          ports:
+            - name: http
+              containerPort: 8686
+          volumeMounts:
+            - name: config
+              mountPath: /config
+            - name: music
+              mountPath: /music
+          livenessProbe:
+            httpGet:
+              path: /ping
+              port: http
+            initialDelaySeconds: 60
+            periodSeconds: 30
+            timeoutSeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /ping
+              port: http
+            initialDelaySeconds: 20
+            periodSeconds: 10
+            timeoutSeconds: 5
+          resources:
+            requests:
+              cpu: 100m
+              memory: 512Mi
+            limits:
+              memory: 4Gi
+          securityContext:
+            runAsNonRoot: false
+            allowPrivilegeEscalation: true
+            capabilities:
+              drop: ["ALL"]
+              add: ["CHOWN", "DAC_OVERRIDE", "FOWNER", "SETGID", "SETUID"]
+        - name: reconcile
+          image: lscr.io/linuxserver/lidarr:3.1.0.4875-ls42 # {"$imagepolicy": "flux-system:lidarr"}
+          command: ["sh", "/scripts/reconcile.sh"]
+          volumeMounts:
+            - name: config
+              mountPath: /config
+              readOnly: true
+            - name: reconcile
+              mountPath: /scripts
+              readOnly: true
+          resources:
+            requests:
+              cpu: 10m
+              memory: 16Mi
+            limits:
+              memory: 64Mi
+          securityContext:
+            runAsNonRoot: true
+            runAsUser: 1000
+            runAsGroup: 1000
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+      volumes:
+        - name: config
+          persistentVolumeClaim:
+            claimName: lidarr-config
+        - name: music
+          persistentVolumeClaim:
+            claimName: navidrome-music-bulk
+        - name: reconcile
+          configMap:
+            name: lidarr-reconcile
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: lidarr
+  namespace: navidrome
+spec:
+  selector:
+    app.kubernetes.io/name: lidarr
+  ports:
+    - name: http
+      port: 80
+      targetPort: http
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: ak-outpost-lidarr-gate
+  namespace: navidrome
+spec:
+  type: ExternalName
+  externalName: ak-outpost-lidarr-gate.authentik.svc.cluster.local
+  ports:
+    - port: 9000
+      targetPort: 9000
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: lidarr
+  namespace: navidrome
+  labels:
+    dns.renner.uno/public: "true"
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  rules:
+    - host: lidarr.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: ak-outpost-lidarr-gate
+                port:
+                  number: 9000
+  tls:
+    - secretName: lidarr-tls
+      hosts:
+        - lidarr.renner.uno

kubernetes/apps/navidrome/pvcs.yaml

@@ -0,0 +1,31 @@
+# SQLite database + artwork/transcode cache; single writer.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: navidrome-data
+  namespace: navidrome
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 10Gi
+---
+# Music library on the EC bulk pool.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: navidrome-music-bulk
+  namespace: navidrome
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled
+spec:
+  accessModes:
+    - ReadWriteMany
+  storageClassName: ceph-filesystem-bulk
+  resources:
+    requests:
+      storage: 300Gi

kubernetes/apps/paperclip/cluster.yaml

@@ -0,0 +1,43 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: pclp-pg
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # data guard
+spec:
+  instances: 2
+  smartShutdownTimeout: 10
+  stopDelay: 60
+  # PG17: upstream's external Postgres target (docker/docker-compose.yml).
+  imageName: ghcr.io/cloudnative-pg/postgresql:17
+  storage:
+    storageClass: ceph-block
+    size: 10Gi
+  plugins:
+    - name: barman-cloud.cloudnative-pg.io
+      isWALArchiver: true
+      parameters:
+        barmanObjectName: pclp-pg-store
+  bootstrap:
+    initdb:
+      database: paperclip
+      owner: paperclip
+  postgresql:
+    parameters:
+      shared_buffers: "256MB"
+      max_connections: "200"
+      checkpoint_timeout: "15min"
+      max_wal_size: "2GB"
+      wal_compression: "zstd"
+      archive_timeout: "30min"
+    synchronous:
+      method: any
+      number: 1
+      dataDurability: preferred
+  resources:
+    requests:
+      cpu: 200m
+      memory: 512Mi
+    limits:
+      memory: 1Gi

kubernetes/apps/paperclip/externalsecret.yaml

@@ -0,0 +1,15 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: paperclip-secrets
+  namespace: paperclip
+spec:
+  refreshInterval: 10m
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: paperclip-secrets
+  dataFrom:
+    - extract:
+        key: apps/paperclip

kubernetes/apps/paperclip/gate-service.yaml

@@ -0,0 +1,12 @@
+# ExternalName mirror so the Ingress can backend the paperclip-gate outpost in the authentik namespace.
+apiVersion: v1
+kind: Service
+metadata:
+  name: ak-outpost-paperclip-gate
+  namespace: paperclip
+spec:
+  type: ExternalName
+  externalName: ak-outpost-paperclip-gate.authentik.svc.cluster.local
+  ports:
+    - port: 9000
+      targetPort: 9000

kubernetes/apps/paperclip/hetzner-creds-externalsecret.yaml

@@ -0,0 +1,17 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: hetzner-s3-creds
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # barman S3 creds
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: hetzner-s3-creds
+  dataFrom:
+    - extract:
+        key: infrastructure/hetzner-s3

kubernetes/apps/paperclip/ingress.yaml

@@ -0,0 +1,25 @@
+# LAN-only (no dns.renner.uno/public label); backends the gate outpost, never the Service.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: paperclip
+  namespace: paperclip
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - paperclip.renner.uno
+      secretName: paperclip-tls
+  rules:
+    - host: paperclip.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: ak-outpost-paperclip-gate
+                port:
+                  number: 9000

kubernetes/apps/paperclip/kustomization.yaml

@@ -0,0 +1,14 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - externalsecret.yaml
+  - hetzner-creds-externalsecret.yaml
+  - cluster.yaml
+  - objectstore.yaml
+  - scheduled-backup.yaml
+  - pvc.yaml
+  - prometheus-read-rbac.yaml
+  - paperclip.yaml
+  - gate-service.yaml
+  - ingress.yaml
+  - network-policy.yaml

kubernetes/apps/paperclip/network-policy.yaml

@@ -0,0 +1,78 @@
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: paperclip-gate-ingress
+  namespace: paperclip
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: paperclip
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+            goauthentik.io/outpost-name: paperclip-gate
+      toPorts:
+        - ports:
+            - port: "3100"
+              protocol: TCP
+---
+# Covers the server and every agent it spawns (same pod).
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: paperclip-egress
+  namespace: paperclip
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: paperclip
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+            cnpg.io/cluster: pclp-pg
+      toPorts:
+        - ports:
+            - port: "5432"
+              protocol: TCP
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+    # Gitea, Woodpecker, HA, Loki via their *.renner.uno names.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+      toPorts:
+        - ports:
+            - port: "8443"
+              protocol: TCP
+    # Notes wiki API, authenticated by the paperclip API key.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ai
+            app.kubernetes.io/name: wiki
+      toPorts:
+        - ports:
+            - port: "8080"
+              protocol: TCP
+    - toEntities:
+        - kube-apiserver
+      toPorts:
+        - ports:
+            - port: "6443"
+              protocol: TCP
+    - toEntities:
+        - world
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP

kubernetes/apps/paperclip/objectstore.yaml

@@ -0,0 +1,26 @@
+apiVersion: barmancloud.cnpg.io/v1
+kind: ObjectStore
+metadata:
+  name: pclp-pg-store
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # backup target
+spec:
+  configuration:
+    destinationPath: s3://renner-uno-k8s-cnpg/pclp-pg
+    endpointURL: https://fsn1.your-objectstorage.com
+    s3Credentials:
+      accessKeyId:
+        name: hetzner-s3-creds
+        key: ACCESS_KEY_ID
+      secretAccessKey:
+        name: hetzner-s3-creds
+        key: ACCESS_SECRET_KEY
+      region:
+        name: hetzner-s3-creds
+        key: REGION
+    wal:
+      compression: gzip
+    data:
+      compression: gzip
+  retentionPolicy: "30d"

kubernetes/apps/paperclip/paperclip.yaml

@@ -0,0 +1,128 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: paperclip
+  namespace: paperclip
+  labels:
+    app.kubernetes.io/name: paperclip
+  annotations:
+    secret.reloader.stakater.com/reload: paperclip-secrets
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: paperclip
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: paperclip
+    spec:
+      enableServiceLinks: false
+      # Every agent shares this filesystem, so every agent can read this token.
+      # It is mounted anyway because it is worth exactly one thing: `get` on the
+      # Prometheus Service proxy, for monitoring reads (REN-36). See
+      # prometheus-read-rbac.yaml. Bots still bring their own Gitea and cluster
+      # credentials; this is not a general-purpose cluster credential.
+      serviceAccountName: paperclip-prometheus-read
+      automountServiceAccountToken: true
+      securityContext:
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: paperclip
+          image: ghcr.io/paperclipai/paperclip:nightly@sha256:9222efbf11863160669e13d5559e6a1be881886dd1758650b74c3993dc3ef2a6 # {"$imagepolicy": "flux-system:paperclip"}
+          ports:
+            - name: http
+              containerPort: 3100
+          env:
+            - name: DATABASE_URL
+              valueFrom:
+                secretKeyRef:
+                  name: pclp-pg-app
+                  key: uri
+            - name: HOST
+              value: "0.0.0.0"
+            - name: PORT
+              value: "3100"
+            - name: PAPERCLIP_DEPLOYMENT_MODE
+              value: authenticated
+            - name: PAPERCLIP_DEPLOYMENT_EXPOSURE
+              value: private
+            - name: PAPERCLIP_PUBLIC_URL
+              value: https://paperclip.renner.uno
+            # Agents reach the API via the Service, not through the Authentik gate.
+            - name: PAPERCLIP_API_URL
+              value: http://paperclip.paperclip.svc.cluster.local:3100
+            - name: PAPERCLIP_ALLOWED_HOSTNAMES
+              value: paperclip.paperclip.svc.cluster.local
+            - name: PAPERCLIP_AUTH_DISABLE_SIGN_UP
+              value: "true"
+            - name: PAPERCLIP_TELEMETRY_DISABLED
+              value: "1"
+          envFrom:
+            - secretRef:
+                name: paperclip-secrets
+          volumeMounts:
+            - name: data
+              mountPath: /paperclip
+          # The private-hostname guard 403s any Host but loopback and the public URL; the kubelet sends the pod IP.
+          startupProbe:
+            httpGet:
+              path: /api/health
+              port: http
+              httpHeaders:
+                - name: Host
+                  value: localhost
+            periodSeconds: 10
+            timeoutSeconds: 5
+            failureThreshold: 60
+          # /api/health runs SELECT 1; liveness stays DB-independent so a Postgres outage doesn't restart the server.
+          livenessProbe:
+            tcpSocket:
+              port: http
+            periodSeconds: 30
+            timeoutSeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /api/health
+              port: http
+              httpHeaders:
+                - name: Host
+                  value: localhost
+            periodSeconds: 10
+            timeoutSeconds: 5
+          # Every concurrent agent (Claude Code + MCP helpers) runs inside this container.
+          resources:
+            requests:
+              cpu: 250m
+              memory: 2Gi
+            limits:
+              memory: 32Gi
+          # Entrypoint remaps the node user and chowns /paperclip as root, then drops to it via gosu.
+          securityContext:
+            runAsNonRoot: false
+            allowPrivilegeEscalation: true
+            capabilities:
+              drop: ["ALL"]
+              add: ["CHOWN", "DAC_OVERRIDE", "FOWNER", "SETGID", "SETUID"]
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: paperclip-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: paperclip
+  namespace: paperclip
+spec:
+  selector:
+    app.kubernetes.io/name: paperclip
+  ports:
+    - name: http
+      port: 3100
+      targetPort: http

kubernetes/apps/paperclip/prometheus-read-rbac.yaml

@@ -0,0 +1,87 @@
+---
+# A read-only Prometheus path for the agents in the paperclip pod (REN-36).
+#
+# WHY THE APISERVER PROXY AND NOT A NETWORK RULE
+#
+# Prometheus runs with `--web.enable-lifecycle` (the chart leaves
+# `reloadStrategy` unset, so the operator defaults it to HTTP and appends the
+# flag), and :9090 carries no auth of its own. That puts `POST /-/quit` and
+# `POST /-/reload` on the same TCP port as `GET /api/v1/query`. An L4 allow on
+# 9090 cannot tell them apart, so it would hand this pod a kill switch for
+# cluster monitoring.
+#
+# The apiserver can tell them apart: it maps the HTTP method of a service-proxy
+# request onto an RBAC verb — GET->get, POST->create, PUT->update. A Role
+# holding only `get` therefore refuses `/-/quit` and `/-/reload` in the
+# authorizer, before the request is ever forwarded to Prometheus. Read-only by
+# construction, and `--web.enable-lifecycle` stops mattering.
+#
+# Shape copied from `../monitoring/ci-diagnostics-rbac.yaml` (the
+# `ci-diagnostics-service-proxy` Role), which has run this same path for the
+# alert-monitor since 2026-07-16.
+#
+# ONE VERB, NAMED SERVICES ONLY — ON PURPOSE
+#
+# Every process in the paperclip pod can read the mounted ServiceAccount token,
+# so the token is only worth what this Role grants. It grants `get` on the two
+# Services named below and nothing else. Widening it is a new card and a new
+# review, not an edit here — the Alertmanager name was added that way (REN-95,
+# for read-only alert-delivery checks; parent REN-94).
+#
+# `resourceNames` is matched verbatim by the authorizer, not as a selector:
+# `svc`, `svc:9090` and `http:svc:9090` are three different strings. Only the
+# port-qualified forms are granted, so the only URLs that work are
+#   /api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy/...
+#   /api/v1/namespaces/monitoring/services/kube-prometheus-stack-alertmanager:9093/proxy/...
+# A 403 on some other form is the grant working. Do not fix it by adding a
+# string.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: paperclip-prometheus-read
+  namespace: paperclip
+  labels:
+    app.kubernetes.io/name: paperclip
+# Named for the only thing it is worth. If a future change gives this account a
+# second grant, the name is a lie and the review should say so. Whether the
+# token is mounted is decided on the pod (`paperclip.yaml`), not here.
+#
+# REN-95 stretched the name: it now also proxies to Alertmanager. Kept as-is
+# deliberately — `serviceAccountName` is in the pod template, so renaming it
+# would restart the paperclip Deployment and evict running agents. Read the name
+# as "read-only kube-prometheus-stack access", and read the Role for the truth.
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: paperclip-prometheus-read
+  namespace: monitoring
+  labels:
+    app.kubernetes.io/name: paperclip
+rules:
+  - apiGroups: [""]
+    resources: ["services/proxy"]
+    verbs: ["get"]
+    resourceNames:
+      - "kube-prometheus-stack-prometheus:9090"
+      - "kube-prometheus-stack-alertmanager:9093"
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: paperclip-prometheus-read
+  namespace: monitoring
+  labels:
+    app.kubernetes.io/name: paperclip
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: paperclip-prometheus-read
+# The Role and this binding live in `monitoring`; the subject lives in
+# `paperclip`, so `namespace:` here is load-bearing. Omit it and the binding
+# silently resolves to a nonexistent `monitoring/paperclip-prometheus-read`,
+# which fails closed with a 403 indistinguishable from a wrong resourceName.
+subjects:
+  - kind: ServiceAccount
+    name: paperclip-prometheus-read
+    namespace: paperclip

kubernetes/apps/paperclip/pvc.yaml

@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: paperclip-data
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # data guard
+spec:
+  accessModes: [ReadWriteOnce]
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 20Gi

kubernetes/apps/paperclip/scheduled-backup.yaml

@@ -0,0 +1,16 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: ScheduledBackup
+metadata:
+  name: pclp-pg-daily
+  namespace: paperclip
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled # backup schedule
+spec:
+  schedule: "0 45 2 * * *"
+  immediate: true
+  backupOwnerReference: self
+  cluster:
+    name: pclp-pg
+  method: plugin
+  pluginConfiguration:
+    name: barman-cloud.cloudnative-pg.io

kubernetes/apps/trmnl/terminus.yaml

@@ -134,7 +134,7 @@ spec:
         # is a device-bricking class of risk. Bumps are a reviewed, manual
         # act — see kubernetes/infrastructure/image-automation/README.md.
         - name: terminus
-          image: ghcr.io/usetrmnl/terminus:0.74.0 # {"$imagepolicy": "flux-system:terminus"}
+          image: ghcr.io/usetrmnl/terminus:0.75.0 # {"$imagepolicy": "flux-system:terminus"}
           ports:
             - name: http
               containerPort: 2345
@@ -197,7 +197,7 @@ spec:
             capabilities:
               drop: ["ALL"]
         - name: sidekiq
-          image: ghcr.io/usetrmnl/terminus:0.74.0 # {"$imagepolicy": "flux-system:terminus"}
+          image: ghcr.io/usetrmnl/terminus:0.75.0 # {"$imagepolicy": "flux-system:terminus"}
           command: ["bundle", "exec", "sidekiq", "-r", "./config/sidekiq.rb"]
           envFrom:
             - configMapRef:

kubernetes/apps/uptime-monitor/crd-endpointmonitors.yaml

@@ -0,0 +1,464 @@
+# VENDORED, DO NOT EDIT BY HAND.
+#
+# Source: stakater/IngressMonitorController chart 2.2.13,
+#   charts/ingressmonitorcontroller/crds/endpointmonitor.stakater.com_endpointmonitors.yaml
+# Copied verbatim at commit aaffd8a866aa09a77030979d938e766895bde5d4.
+#
+# Vendored rather than installed via HelmRelease because this Kustomization
+# ships hand-written RBAC (see rbac.yaml) — the chart's own Role grants pods,
+# services and `secrets: list`, none of which this controller uses. Pulling the
+# chart for the CRD alone would drag those back in.
+#
+# To update: bump the image tag in deployment.yaml and re-copy this file from
+# the matching chart tag. Nothing here is Renovate-managed, so the tag and this
+# file move together, by hand, in one reviewable commit.
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: v0.16.1
+  name: endpointmonitors.endpointmonitor.stakater.com
+spec:
+  group: endpointmonitor.stakater.com
+  names:
+    kind: EndpointMonitor
+    listKind: EndpointMonitorList
+    plural: endpointmonitors
+    singular: endpointmonitor
+  scope: Namespaced
+  versions:
+  - name: v1alpha1
+    schema:
+      openAPIV3Schema:
+        description: EndpointMonitor is the Schema for the endpointmonitors API
+        properties:
+          apiVersion:
+            description: |-
+              APIVersion defines the versioned schema of this representation of an object.
+              Servers should convert recognized schemas to the latest internal value, and
+              may reject unrecognized values.
+              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
+            type: string
+          kind:
+            description: |-
+              Kind is a string value representing the REST resource this object represents.
+              Servers may infer this from the endpoint the client submits requests to.
+              Cannot be updated.
+              In CamelCase.
+              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
+            type: string
+          metadata:
+            type: object
+          spec:
+            description: EndpointMonitorSpec defines the desired state of EndpointMonitor
+            properties:
+              appInsightsConfig:
+                description: Configuration for AppInsights Monitor Provider
+                properties:
+                  frequency:
+                    description: 'Sets how often the test should run from each test
+                      location. Possible values: `300,600,900` seconds'
+                    type: integer
+                  retryEnable:
+                    description: 'If its `true`, falied test will be retry after a
+                      short interval. Possible values: `true, false`'
+                    type: boolean
+                  statusCode:
+                    description: Returned status code that is counted as a success
+                    type: integer
+                type: object
+              forceHttps:
+                description: Force monitor endpoint to use HTTPS
+                type: boolean
+              gcloudConfig:
+                description: Configuration for Google Cloud Monitor Provider
+                properties:
+                  projectId:
+                    description: Google Cloud Project ID
+                    type: string
+                type: object
+              grafanaConfig:
+                description: Configuration for Grafana Cloud Monitor Provider
+                properties:
+                  alertSensitivity:
+                    default: none
+                    description: |-
+                      The alertSensitivity value defaults to none if there are no alerts or can be set to low, medium,
+                      or high to correspond to the check alert levels.
+                    enum:
+                    - none
+                    - low
+                    - medium
+                    - high
+                    type: string
+                  frequency:
+                    description: The frequency value specifies how often the check
+                      runs in milliseconds
+                    format: int64
+                    type: integer
+                  probes:
+                    description: |-
+                      Probes are the monitoring agents responsible for simulating user interactions with your web applications
+                      or services. These agents periodically send requests to predefined URLs and record the responses,
+                      checking for expected outcomes and measuring performance.
+                    items:
+                      type: string
+                    type: array
+                  tenantId:
+                    format: int64
+                    type: integer
+                type: object
+              healthEndpoint:
+                type: string
+              pingdomConfig:
+                description: Configuration for Pingdom Monitor Provider
+                properties:
+                  alertContacts:
+                    description: '`-` separated contact id''s (e.g. "1234567_8_9-9876543_2_1")'
+                    type: string
+                  alertIntegrations:
+                    description: '`-` separated set list of integrations ids (e.g.
+                      "91166-12168")'
+                    type: string
+                  basicAuthUser:
+                    description: Required for basic-authentication
+                    type: string
+                  notifyWhenBackUp:
+                    description: Set to "false" to disable recovery notifications
+                    type: boolean
+                  paused:
+                    description: Set to "true" to pause checks
+                    type: boolean
+                  postDataEnvVar:
+                    description: |-
+                      Data that should be posted to the web page, for example submission data for a sign-up or login form.
+                      The data needs to be formatted in the same way as a web browser would send it to the web server.
+                      Because post data contains sensitive secret this field is only a reference to an environment variable.
+                    type: string
+                  requestHeaders:
+                    description: Custom request headers
+                    type: string
+                  requestHeadersEnvVar:
+                    description: |-
+                      Custom request headers that should be read from an environment variable as it possibly contains sensitive data.
+                      An example would be an API token.
+                    type: string
+                  resolution:
+                    description: The pingdom check interval in minutes
+                    type: integer
+                  sendNotificationWhenDown:
+                    description: How many failed check attempts before notifying
+                    type: integer
+                  shouldContain:
+                    description: Set to text string that has to be present in the
+                      HTML code of the page
+                    type: string
+                  sslDownDaysBefore:
+                    description: |-
+                      Consider down prior to certificate expiring
+                      Select the number of days prior to your certificate expiry date that you want to consider the check down.
+                      At this day your check will be considered down and if applicable a down alert will be sent.
+                    type: integer
+                  tags:
+                    description: Comma separated set of tags to apply to check (e.g.
+                      "testing,aws")
+                    type: string
+                  teamAlertContacts:
+                    description: '`-` separated team id''s (e.g. "1234567_8_9-9876543_2_1")'
+                    type: string
+                  verifyCertificate:
+                    description: |-
+                      Monitor SSL/TLS certificate
+                      Monitor the validity of your SSL/TLS certificate. With this enabled Uptime checks will be considered DOWN when
+                      the certificate becomes invalid or expires.
+                      SSL/TLS certificate monitoring is available for HTTP checks.
+                    type: boolean
+                type: object
+              pingdomTransactionConfig:
+                description: Configuration for Pingdom Transaction Monitor Provider
+                properties:
+                  alertContacts:
+                    description: '`-` separated contact id''s (e.g. "1234567_8_9-9876543_2_1")'
+                    type: string
+                  alertIntegrations:
+                    description: '`-` separated set list of integrations ids (e.g.
+                      "91166-12168")'
+                    type: string
+                  custom_message:
+                    description: Custom message that is part of the email and webhook
+                      alerts
+                    type: string
+                  interval:
+                    description: 'TMS test intervals in minutes. Allowed intervals:
+                      5,10,20,60,720,1440. The interval you''re allowed to set may
+                      vary depending on your current plan.'
+                    enum:
+                    - 5
+                    - 10
+                    - 20
+                    - 60
+                    - 720
+                    - 1440
+                    type: integer
+                  paused:
+                    description: 'Check status: active or inactive'
+                    type: boolean
+                  region:
+                    description: 'Name of the region where the check is executed.
+                      Supported regions: us-east, us-west, eu, au'
+                    enum:
+                    - us-east
+                    - us-west
+                    - eu
+                    - au
+                    type: string
+                  send_notification_when_down:
+                    description: Send notification when down X times
+                    format: int64
+                    type: integer
+                  severity_level:
+                    description: 'Check importance- how important are the alerts when
+                      the check fails. Allowed values: low, high'
+                    enum:
+                    - low
+                    - high
+                    type: string
+                  steps:
+                    description: steps to be executed as part of the check
+                    items:
+                      description: PingdomStep respresents a step of the script to
+                        run a transcaction check
+                      properties:
+                        args:
+                          additionalProperties:
+                            type: string
+                          description: |-
+                            contains the html element with assigned value
+                            the key element is always lowercase for example {"url": "https://www.pingdom.com"}
+                            see available values at https://pkg.go.dev/github.com/karlderkaefer/pingdom-golang-client@latest/pkg/pingdom/client/tmschecks#StepArg
+                          type: object
+                        function:
+                          description: |-
+                            contains the function that is executed as part of the step
+                            commands: go_to, click, fill, check, uncheck, sleep, select_radio, basic_auth, submit, wait_for_element, wait_for_contains
+                            validations: url, exists, not_exists, contains, not_contains, field_contains, field_not_contains, is_checked, is_not_checked, radio_selected, dropdown_selected, dropdown_not_selected
+                            see updated list https://docs.pingdom.com/api/#section/TMS-Steps-Vocabulary/Script-transaction-checks
+                          type: string
+                      required:
+                      - args
+                      - function
+                      type: object
+                    type: array
+                  tags:
+                    description: List of tags for a check. The tag name may contain
+                      the characters 'A-Z', 'a-z', '0-9', '_' and '-'. The maximum
+                      length of a tag is 64 characters.
+                    items:
+                      type: string
+                    type: array
+                  teamAlertContacts:
+                    description: '`-` separated team id''s (e.g. "1234567_8_9-9876543_2_1")'
+                    type: string
+                required:
+                - steps
+                type: object
+              providers:
+                description: Comma separated list of providers
+                type: string
+              statusCakeConfig:
+                description: Configuration for StatusCake Monitor Provider
+                properties:
+                  basicAuthSecret:
+                    description: Basic Auth Secret Name
+                    type: string
+                  basicAuthUser:
+                    description: Basic Auth User
+                    type: string
+                  checkRate:
+                    default: 300
+                    description: Set Check Rate for the monitor.
+                    type: integer
+                  confirmation:
+                    description: Confirmation value ranges from (0,10)
+                    maximum: 10
+                    minimum: 0
+                    type: integer
+                  contactGroup:
+                    description: Contact Group to be alerted.
+                    type: string
+                  enableSslAlert:
+                    description: Enable SSL Alert
+                    type: boolean
+                  findString:
+                    description: String to look for within the response. Considered
+                      down if not found
+                    type: string
+                  followRedirect:
+                    description: Enable ingress redirects
+                    type: boolean
+                  paused:
+                    description: Pause the service
+                    type: boolean
+                  pingUrl:
+                    description: Webhook for alerts
+                    type: string
+                  port:
+                    description: TCP Port
+                    type: integer
+                  rawPostData:
+                    description: RawPostData can be used to send parameters within
+                      the URL. Changes the request from a GET to a POST
+                    type: string
+                  realBrowser:
+                    description: Enable Real Browser
+                    type: boolean
+                  regions:
+                    description: Comma separated list of Node Location IDs
+                    type: string
+                  statusCodes:
+                    description: Comma separated list of HTTP codes to trigger error
+                      on
+                    type: string
+                  testTags:
+                    description: Comma separated list of tags
+                    type: string
+                  testType:
+                    description: Set Test type - HTTP, TCP, PING, or Heartbeat
+                    enum:
+                    - HTTP
+                    - TCP
+                    - PING
+                    - Heartbeat
+                    type: string
+                  timeout:
+                    description: Timeout is used to set a user agent string.
+                    maximum: 75
+                    minimum: 5
+                    type: integer
+                  triggerRate:
+                    description: Minutes to wait before sending an alert
+                    type: integer
+                  userAgent:
+                    description: UserAgent is used to set a user agent string.
+                    type: string
+                type: object
+                x-kubernetes-validations:
+                - message: checkRate must be between 30 and 172800 seconds for Heartbeat
+                    monitors
+                  rule: self.testType != 'Heartbeat' || (self.checkRate >= 30 && self.checkRate
+                    <= 172800)
+                - message: 'checkRate for uptime monitors must be one of: 0, 30, 60,
+                    300, 900, 1800, 3600, 86400'
+                  rule: self.testType == 'Heartbeat' || self.checkRate in [0, 30,
+                    60, 300, 900, 1800, 3600, 86400]
+              updownConfig:
+                description: Configuration for Updown Monitor Provider
+                properties:
+                  enable:
+                    description: Enable or disable checks
+                    type: boolean
+                  period:
+                    description: The pingdom check interval in seconds
+                    type: integer
+                  publishPage:
+                    description: Make status page public or not
+                    type: boolean
+                  requestHeaders:
+                    description: Additional request headers for API calls
+                    type: string
+                type: object
+              uptimeConfig:
+                description: Configuration for Uptime Monitor Provider
+                properties:
+                  checkType:
+                    description: The uptime check type that can be HTTP/DNS/ICMP etc.
+                    type: string
+                  contacts:
+                    description: Add one or more contact groups separated by `,`
+                    type: string
+                  interval:
+                    description: The uptime check interval in seconds
+                    type: integer
+                  locations:
+                    description: Add different locations for the check
+                    type: string
+                  tags:
+                    description: Add one or more tags for the check separated by `,`
+                    type: string
+                type: object
+              uptimeRobotConfig:
+                description: Configuration for UptimeRobot Monitor Provider
+                properties:
+                  alertContacts:
+                    description: The uptimerobot alertContacts to be associated with
+                      this monitor
+                    type: string
+                  customHTTPStatuses:
+                    description: |-
+                      Defines which http status codes are treated as up or down
+                      For ex: 200:0_401:1_503:1 (to accept 200 as down and 401 and 503 as up)
+                    type: string
+                  interval:
+                    description: The uptimerobot check interval in seconds
+                    minimum: 60
+                    type: integer
+                  keywordExists:
+                    description: Alert if value exist (yes) or doesn't exist (no)
+                      (Only if monitor-type is keyword)
+                    enum:
+                    - "yes"
+                    - "no"
+                    type: string
+                  keywordValue:
+                    description: keyword to check on URL (e.g.'search' or '404') (Only
+                      if monitor-type is keyword)
+                    type: string
+                  maintenanceWindows:
+                    description: Specify maintenanceWindows i.e. once or recurring
+                      “do-not-monitor periods”
+                    type: string
+                  monitorType:
+                    description: The uptimerobot monitor type (http or keyword)
+                    enum:
+                    - http
+                    - keyword
+                    type: string
+                  statusPages:
+                    description: The uptimerobot public status page ID to add this
+                      monitor to
+                    type: string
+                type: object
+              url:
+                description: URL to monitor
+                type: string
+              urlFrom:
+                description: URL to monitor from either an ingress or route reference
+                properties:
+                  ingressRef:
+                    description: IngressURLSource selects an Ingress to populate the
+                      URL with
+                    properties:
+                      name:
+                        type: string
+                    required:
+                    - name
+                    type: object
+                  routeRef:
+                    description: RouteURLSource selects a Route to populate the URL
+                      with
+                    properties:
+                      name:
+                        type: string
+                    required:
+                    - name
+                    type: object
+                type: object
+            type: object
+          status:
+            description: EndpointMonitorStatus defines the observed state of EndpointMonitor
+            type: object
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}

kubernetes/apps/uptime-monitor/deployment.yaml

@@ -0,0 +1,118 @@
+---
+# Stakater IngressMonitorController (REN-6).
+#
+# It ships with zero EndpointMonitor resources, which is the whole point of this
+# first deployment: the reconciler is keyed on one custom resource per event
+# (endpointmonitor_controller.go:62-107) and never enumerates the UptimeRobot
+# account, so with no EndpointMonitors it reconciles nothing and makes no API
+# call to UptimeRobot at all. The 9 monitors already in that account are not
+# read, not updated and not deleted.
+#
+# Plain manifests, not the upstream HelmRelease: the chart's RBAC grants pods,
+# services and `secrets: list`, and it also creates a Service and optional
+# cert-manager objects for a metrics endpoint that is disabled below. The
+# maintained thing we depend on is the upstream image, which is pinned here.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ingressmonitorcontroller
+  namespace: uptime-monitor
+  labels:
+    app.kubernetes.io/name: ingressmonitorcontroller
+  annotations:
+    # The API key is read ONCE, at startup (pkg/config/config.go
+    # `LoadControllerConfig` runs before the manager starts), so a rotated
+    # Secret is not picked up until the pod restarts. Reloader — which this
+    # cluster already runs — does that restart. Same idiom as
+    # apps/external-dns/helmrelease.yaml.
+    secret.reloader.stakater.com/reload: uptime-robot-api-key
+spec:
+  replicas: 1
+  strategy:
+    # Recreate, not RollingUpdate, and no --leader-elect. A rolling update would
+    # briefly run two reconcilers against the same UptimeRobot account; Recreate
+    # makes "at most one controller pod, ever" a property of the Deployment
+    # instead of something a Lease has to arbitrate. It costs a few seconds of
+    # controller downtime per upgrade, which for a monitor-syncing controller is
+    # nothing, and it removes the leases/events RBAC entirely.
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: ingressmonitorcontroller
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: ingressmonitorcontroller
+    spec:
+      serviceAccountName: ingressmonitorcontroller
+      automountServiceAccountToken: true
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 65532
+        runAsGroup: 65532
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: manager
+          # Pinned, and deliberately NOT under Flux image automation. rbac.yaml
+          # is derived from what this exact version calls; an automatic bump
+          # could add a call the Role does not cover, silently.
+          image: ghcr.io/stakater/ingressmonitorcontroller:v2.2.13
+          command: ["/manager"]
+          args:
+            - --health-probe-bind-address=:8081
+            # Metrics off. Enabling them means either TLS certs or
+            # `--metrics-secure=false`, and the authn/authz filter
+            # (cmd/main.go, `filters.WithAuthenticationAndAuthorization`) wants a
+            # ClusterRole for TokenReview/SubjectAccessReview — a cluster-scoped
+            # grant this namespace otherwise does not need. A ServiceMonitor for
+            # this controller is worth having, but it is its own change.
+            - --metrics-bind-address=0
+          env:
+            # Scopes the manager's cache to this namespace
+            # (cmd/main.go `buildDefaultNamespaces`), which is what makes the
+            # namespaced Role in rbac.yaml sufficient — there is no ClusterRole.
+            - name: WATCH_NAMESPACE
+              value: uptime-monitor
+            # The namespace the config Secret is looked up in. Read from the
+            # ServiceAccount token file when unset; set explicitly so the lookup
+            # is visible in the manifest.
+            - name: OPERATOR_NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+            # IMC reads its whole configuration from key `config.yaml` of this
+            # Secret — providers, API key and enableMonitorDeletion together.
+            # ESO renders that key; see externalsecret.yaml for the contents and
+            # for why the config lives in git while the key does not.
+            - name: CONFIG_SECRET_NAME
+              value: uptime-robot-api-key
+          # Both probes are served by the manager on :8081 and are dialled by
+          # the kubelet, which Cilium identifies as the `host` entity — the only
+          # ingress the namespace policy admits
+          # (infrastructure/network-policies/uptime-monitor-ns.yaml). The port is
+          # not exposed as a containerPort because nothing else may reach it.
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: 8081
+            initialDelaySeconds: 15
+            periodSeconds: 20
+          readinessProbe:
+            httpGet:
+              path: /readyz
+              port: 8081
+            initialDelaySeconds: 5
+            periodSeconds: 10
+          resources:
+            requests:
+              cpu: 10m
+              memory: 64Mi
+            limits:
+              memory: 128Mi
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+      terminationGracePeriodSeconds: 10

kubernetes/apps/uptime-monitor/externalsecret.yaml

@@ -0,0 +1,79 @@
+---
+# UptimeRobot API key for the IngressMonitorController (REN-2).
+#
+# OpenBao key infrastructure/uptime-robot, field `api_key`. `data` + `property`
+# rather than `dataFrom: extract` (the shape most ExternalSecrets in this repo
+# use) on purpose: extract copies EVERY field at the KV path into the cluster
+# Secret, so any field added there later — an account id, a second token —
+# would land in this namespace without anyone editing this file. Naming the one
+# field keeps the Secret to exactly what the controller needs. Same shape as
+# kubernetes/apps/wiki/externalsecrets.yaml.
+#
+# refreshInterval 1h matches the un-rotated ExternalSecrets here (velero,
+# wiki-*). The rotated ones use 10m so an automated roll propagates fast — if
+# kubernetes/infrastructure/openbao/rotation/ ever gains an uptime-robot
+# rotator, drop this to 10m. The controller Deployment already carries the
+# stakater/reloader annotation, so a new value restarts the pod that has the
+# old one cached.
+#
+# The `eso` role already reads this path: eso-read grants `homelab/data/*`
+# (kubernetes/infrastructure/openbao/policies/eso-read.hcl:31). No OpenBao
+# policy change is needed for this key.
+#
+# ---
+# Why this Secret holds a rendered `config.yaml` rather than a bare key (REN-6):
+# IngressMonitorController does not read an API key from an env var. It reads
+# its ENTIRE configuration — provider list, API key, enableMonitorDeletion —
+# from one Secret key named `config.yaml` (pkg/config/config.go,
+# `IngressMonitorControllerSecretConfigKey`), at startup, through the manager's
+# APIReader. So the choice is not "key or config"; it is which Secret the config
+# lives in. Templating it here keeps every non-secret decision in git and in
+# review, with only the key itself injected at sync time.
+#
+# The template replaces the Secret's data rather than merging, so `apiKey` is a
+# template input and the rendered `config.yaml` is the single output key.
+#
+# Two values are load-bearing:
+#
+#   apiURL — REQUIRED. `Setup()` assigns `monitor.url = p.ApiURL` with no
+#   default (pkg/monitors/uptimerobot/uptime-monitor.go:39-44); omit it and
+#   every request goes to a relative URL.
+#
+#   enableMonitorDeletion — true, so that removing an EndpointMonitor removes
+#   its monitor, which is what the mission asks for. It is bounded: deletion is
+#   reachable only via `GetByName`, which filters `m.FriendlyName == name` —
+#   exact equality — against the name the controller itself computed from the
+#   default template `{{.Name}}-{{.Namespace}}`. Every name it can compute ends
+#   in `-uptime-monitor`; none of the 9 monitors already in the account does.
+#   Set it to false for a create/update-only kill switch.
+#
+# monitorNameTemplate is deliberately absent: an empty value makes
+# GetNameTemplateFormat fall back to that same default (pkg/util/template-utils.go:9),
+# and writing it out here would mean escaping Go template syntax inside a Go
+# template for no gain.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: uptime-robot-api-key
+  namespace: uptime-monitor
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: uptime-robot-api-key
+    template:
+      engineVersion: v2
+      data:
+        config.yaml: |
+          providers:
+            - name: UptimeRobot
+              apiKey: "{{ .apiKey }}"
+              apiURL: https://api.uptimerobot.com/v2/
+          enableMonitorDeletion: true
+  data:
+    - secretKey: apiKey
+      remoteRef:
+        key: infrastructure/uptime-robot
+        property: api_key

kubernetes/apps/uptime-monitor/kustomization.yaml

@@ -0,0 +1,7 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - crd-endpointmonitors.yaml
+  - externalsecret.yaml
+  - rbac.yaml
+  - deployment.yaml

kubernetes/apps/uptime-monitor/rbac.yaml

@@ -0,0 +1,86 @@
+---
+# ServiceAccount + namespaced RBAC for IngressMonitorController (REN-6).
+#
+# There is NO ClusterRole here, on purpose. The Deployment sets
+# WATCH_NAMESPACE=uptime-monitor, which makes the manager build a
+# namespace-scoped cache (cmd/main.go `buildDefaultNamespaces`), so every read
+# the controller performs is confined to this namespace. A ClusterRole would
+# grant reach the process cannot use.
+#
+# The rules below were derived from what the pinned image actually calls, not
+# from the chart's rules or the upstream kubebuilder markers. Deliberately NOT
+# granted, and why:
+#
+#   - `""/pods`, `""/services` — read from exactly one place,
+#     `tryGetHealthEndpointFromIngress` (pkg/kube/wrappers/ingress-wrapper.go:126),
+#     which is reached only from the `urlFrom.ingressRef` path and only when the
+#     EndpointMonitor sets no explicit `healthEndpoint`. It is a best-effort
+#     guess at a readiness-probe path: both the Get and the List log the error
+#     and return `"", false` on failure, so a denial costs a log line, not a
+#     reconcile. Since the ingressRef path can only reach this namespace anyway
+#     (see the ingresses rule below) and nothing here serves HTTP, the grant
+#     would be inert. Add it if an Ingress ever lands in `uptime-monitor`.
+#   - `route.openshift.io/routes` — OpenShift only; `kube.IsOpenshift` is false
+#     here, so the Route scheme is never even registered (cmd/main.go init()).
+#   - `endpointmonitors/finalizers` — IMC uses no finalizers. Deletion is driven
+#     by the reconciler's IsNotFound branch, not by a finalizer
+#     (endpointmonitor_controller.go:74-81); `grep -rn Finalizer internal/controller/`
+#     has no hits.
+#   - leases / events — no leader election (see the Recreate strategy comment in
+#     deployment.yaml) and the reconciler holds no EventRecorder.
+#
+# Re-check this list whenever the image tag in deployment.yaml moves: it is
+# pinned and not Renovate-managed precisely so that never happens silently.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: ingressmonitorcontroller
+  namespace: uptime-monitor
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: ingressmonitorcontroller
+  namespace: uptime-monitor
+rules:
+  # Its own custom resource — the only thing the reconciler is keyed on.
+  - apiGroups: ["endpointmonitor.stakater.com"]
+    resources: ["endpointmonitors"]
+    verbs: ["get", "list", "watch"]
+  # Not exercised by v2.2.13 (`grep -rn "Status()" internal/controller/` is
+  # empty), but granted: it is this CRD's own subresource, so it confers no
+  # reach outside the resource already listed above, and it is the one write an
+  # operator of this shape is expected to gain.
+  - apiGroups: ["endpointmonitor.stakater.com"]
+    resources: ["endpointmonitors/status"]
+    verbs: ["get", "update", "patch"]
+  # Read-only. Used to resolve an EndpointMonitor's URL from an Ingress via
+  # `spec.urlFrom.ingressRef`. Note that pkg/kube/util/url.go:71 resolves that
+  # ref against the EndpointMonitor's OWN namespace, so this grant can only ever
+  # reach Ingresses in `uptime-monitor` — it cannot read an app's Ingress.
+  - apiGroups: ["networking.k8s.io"]
+    resources: ["ingresses"]
+    verbs: ["get", "list", "watch"]
+  # The controller loads its entire configuration — including the UptimeRobot
+  # API key — from one Secret key at startup (pkg/config/config.go
+  # `LoadControllerConfig`). It reads it through the manager's APIReader, i.e. a
+  # direct GET by name and not a cached list, which is why `get` alone is enough
+  # and why resourceNames can pin it to the single Secret ESO renders.
+  - apiGroups: [""]
+    resources: ["secrets"]
+    resourceNames: ["uptime-robot-api-key"]
+    verbs: ["get"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: ingressmonitorcontroller
+  namespace: uptime-monitor
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: ingressmonitorcontroller
+subjects:
+  - kind: ServiceAccount
+    name: ingressmonitorcontroller
+    namespace: uptime-monitor

kubernetes/apps/wiki-index/deployment.yaml

@@ -1,137 +0,0 @@
----
-# ServiceAccount is the OpenBao identity (auth/kubernetes/role/wiki-index); no
-# Kubernetes API access needed.
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: wiki-index
-  namespace: ai
-  labels:
-    app.kubernetes.io/name: wiki-index
----
-# Index (~17 MB JSON for 1700 notes). Survives restarts so a pod start does not
-# re-embed the wiki; a lost volume only costs one full sync (~5 min).
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: wiki-index-data
-  namespace: ai
-spec:
-  accessModes:
-    - ReadWriteOnce
-  storageClassName: ceph-block
-  resources:
-    requests:
-      storage: 1Gi
----
-# Embedding index over the Notes repo's wiki/ (ci/images/wiki-index). Syncs from
-# Gitea on push webhook and every RESYNC_SECONDS, embeds via wiki-embed, serves
-# POST /rank to ~/Notes/bin/wiki_rank.py through wiki-index.renner.uno.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: wiki-index
-  namespace: ai
-  labels:
-    app.kubernetes.io/name: wiki-index
-spec:
-  replicas: 1
-  strategy:
-    type: Recreate
-  selector:
-    matchLabels:
-      app.kubernetes.io/name: wiki-index
-  template:
-    metadata:
-      labels:
-        app.kubernetes.io/name: wiki-index
-    spec:
-      serviceAccountName: wiki-index
-      securityContext:
-        runAsNonRoot: true
-        runAsUser: 65534
-        runAsGroup: 65534
-        fsGroup: 65534
-        seccompProfile:
-          type: RuntimeDefault
-      containers:
-        - name: wiki-index
-          # Per-commit tag (build-ci-images.yml): immutable, no same-day overwrite.
-          image: gitea.renner.uno/gitea_admin/wiki-index:2026-09-20-4b529cd1
-          securityContext:
-            allowPrivilegeEscalation: false
-            readOnlyRootFilesystem: true
-            capabilities:
-              drop: ["ALL"]
-          env:
-            - name: GITEA_URL
-              value: https://gitea.renner.uno
-            - name: NOTES_REPO
-              value: k.renner/notes
-            - name: NOTES_BRANCH
-              value: main
-            - name: EMBED_URL
-              value: http://wiki-embed.ai.svc.cluster.local:8080
-            - name: BAO_ADDR
-              value: http://openbao.openbao.svc.cluster.local:8200
-            - name: BAO_ROLE
-              value: wiki-index
-            - name: BAO_SECRET_PATH
-              value: homelab/data/apps/ai/wiki-index
-            - name: INDEX_PATH
-              value: /data/index.json
-            - name: RESYNC_SECONDS
-              value: "1800"
-          ports:
-            - name: http
-              containerPort: 8080
-          # Pure-Python cosine over 1700 x 1024 floats per query; the index itself is ~60 MB in memory.
-          resources:
-            requests:
-              cpu: 100m
-              memory: 256Mi
-            limits:
-              cpu: "1"
-              memory: 512Mi
-          # /health: the process serves (liveness). /ready: an index exists; a cold
-          # start on an empty PVC embeds every note (~5 min) before traffic arrives.
-          startupProbe:
-            httpGet:
-              path: /health
-              port: http
-            periodSeconds: 5
-            failureThreshold: 12
-          readinessProbe:
-            httpGet:
-              path: /ready
-              port: http
-            periodSeconds: 10
-          livenessProbe:
-            httpGet:
-              path: /health
-              port: http
-            periodSeconds: 60
-            timeoutSeconds: 10
-            failureThreshold: 3
-          volumeMounts:
-            - name: data
-              mountPath: /data
-      volumes:
-        - name: data
-          persistentVolumeClaim:
-            claimName: wiki-index-data
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: wiki-index
-  namespace: ai
-  labels:
-    app.kubernetes.io/name: wiki-index
-spec:
-  selector:
-    app.kubernetes.io/name: wiki-index
-  ports:
-    - name: http
-      port: 8080
-      targetPort: http

kubernetes/apps/wiki-index/ingress.yaml

@@ -1,56 +0,0 @@
----
-# wiki-index.renner.uno — LAN-only (no dns.renner.uno/public label), machine
-# client (bin/wiki_rank.py), so Traefik BasicAuth like wiki-embed.renner.uno.
-# Gitea's webhook does not pass here; it targets the ClusterIP Service directly.
-apiVersion: traefik.io/v1alpha1
-kind: Middleware
-metadata:
-  name: wiki-index-basic-auth
-  namespace: ai
-spec:
-  basicAuth:
-    secret: wiki-index-basic-auth
-    realm: wiki-index
----
-# OpenBao key apps/ai/wiki-index-basic-auth, field `users` = htpasswd line.
-apiVersion: external-secrets.io/v1
-kind: ExternalSecret
-metadata:
-  name: wiki-index-basic-auth
-  namespace: ai
-spec:
-  refreshInterval: 1h
-  secretStoreRef:
-    kind: ClusterSecretStore
-    name: openbao
-  target:
-    name: wiki-index-basic-auth
-  dataFrom:
-    - extract:
-        key: apps/ai/wiki-index-basic-auth
----
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: wiki-index
-  namespace: ai
-  annotations:
-    cert-manager.io/cluster-issuer: letsencrypt-cluster
-    traefik.ingress.kubernetes.io/router.middlewares: ai-wiki-index-basic-auth@kubernetescrd
-spec:
-  ingressClassName: traefik
-  tls:
-    - hosts:
-        - wiki-index.renner.uno
-      secretName: wiki-index-tls
-  rules:
-    - host: wiki-index.renner.uno
-      http:
-        paths:
-          - path: /
-            pathType: Prefix
-            backend:
-              service:
-                name: wiki-index
-                port:
-                  number: 8080

kubernetes/apps/wiki/deployment.yaml

@@ -0,0 +1,141 @@
+# Index snapshots per embedding model and the document catalog. A lost volume
+# costs one full sync from Gitea (~10 min).
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: wiki-data
+  namespace: ai
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 1Gi
+---
+# Wiki service (konairius/Wiki): mirrors the Notes repo from Gitea, embeds
+# wiki/ via wiki-embed, serves /rank, /api/v1 and the MudBlazor UI. Adapters
+# are selected in appsettings (Wiki__* env overrides); secrets arrive as env
+# from ExternalSecrets, the pod has no OpenBao access.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: wiki
+  namespace: ai
+  annotations:
+    # Keys, tokens and the Anthropic key arrive via envFrom; a rotated Secret rolls the pod.
+    secret.reloader.stakater.com/reload: wiki-gitea,wiki-api-keys,wiki-typesafe,wiki-anthropic
+  labels:
+    app.kubernetes.io/name: wiki
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: wiki
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: wiki
+    spec:
+      automountServiceAccountToken: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 65534
+        runAsGroup: 65534
+        fsGroup: 65534
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: wiki
+          image: gitea.renner.uno/konairius/wiki:build.1790322994 # {"$imagepolicy": "flux-system:wiki"}
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          env:
+            - name: HOME
+              value: /tmp
+            - name: Wiki__Index__File__Directory
+              value: /data
+            - name: Wiki__NoteStore__Gitea__BaseUrl
+              value: https://gitea.renner.uno
+            - name: Wiki__NoteStore__Gitea__Repository
+              value: k.renner/notes
+            - name: Wiki__NoteStore__Gitea__Branch
+              value: main
+            - name: Wiki__Embeddings__Models__bge-m3__Url
+              value: http://wiki-embed.ai.svc.cluster.local:8080
+            # Pod networks: the CNP admits only the wiki-gate outpost and Gitea.
+            - name: Auth__ReverseProxy__TrustedProxies__0
+              value: fd10:244::/48
+            - name: Auth__ReverseProxy__TrustedProxies__1
+              value: 10.244.0.0/16
+          envFrom:
+            - secretRef:
+                name: wiki-gitea
+            - secretRef:
+                name: wiki-api-keys
+            - secretRef:
+                name: wiki-typesafe
+            - secretRef:
+                name: wiki-anthropic
+                optional: true
+          ports:
+            - name: http
+              containerPort: 8080
+          resources:
+            requests:
+              cpu: 200m
+              memory: 256Mi
+            limits:
+              cpu: "1"
+              memory: 1Gi
+          # /health: the process serves (liveness). /ready: catalog synced once and the
+          # active index loaded; a cold start syncs the whole vault first (~10 min).
+          startupProbe:
+            httpGet:
+              path: /ready
+              port: http
+            periodSeconds: 10
+            failureThreshold: 120
+          readinessProbe:
+            httpGet:
+              path: /ready
+              port: http
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: http
+            periodSeconds: 60
+            timeoutSeconds: 10
+            failureThreshold: 3
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: tmp
+              mountPath: /tmp
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: wiki-data
+        - name: tmp
+          emptyDir: {}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: wiki
+  namespace: ai
+  labels:
+    app.kubernetes.io/name: wiki
+spec:
+  selector:
+    app.kubernetes.io/name: wiki
+  ports:
+    - name: http
+      port: 8080
+      targetPort: http

kubernetes/apps/wiki/externalsecrets.yaml

@@ -0,0 +1,104 @@
+---
+# Gitea token (claude-bot, write:repository: the service creates raw/ files for
+# deposited facts) and the notes webhook secret. Secret keys are the
+# configuration paths the host reads.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: wiki-gitea
+  namespace: ai
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: wiki-gitea
+  data:
+    - secretKey: Wiki__NoteStore__Gitea__Token
+      remoteRef:
+        key: apps/ai/wiki-service
+        property: gitea_token
+    - secretKey: Wiki__Webhook__Secret
+      remoteRef:
+        key: apps/ai/wiki-index
+        property: webhook_secret
+---
+# One API key per machine identity: OpenBao apps/ai/wiki-api-keys, field = name.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: wiki-api-keys
+  namespace: ai
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: wiki-api-keys
+  data:
+    - secretKey: Auth__ApiKeys__workstation
+      remoteRef:
+        key: apps/ai/wiki-api-keys
+        property: workstation
+    - secretKey: Auth__ApiKeys__paperclip
+      remoteRef:
+        key: apps/ai/wiki-api-keys
+        property: paperclip
+---
+# TypeSafe System One key for the judge adapter (Wiki:Judges:Models:typesafe).
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: wiki-typesafe
+  namespace: ai
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: wiki-typesafe
+  data:
+    - secretKey: Wiki__Judges__Models__typesafe__ApiKey
+      remoteRef:
+        key: apps/ai/wiki-typesafe
+        property: api_key
+---
+# Answer model: Wiki:AnswerModels:Models:claude-sonnet. The pod starts without it (optional
+# secretRef); the questions endpoint answers 503 until the key is placed.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: wiki-anthropic
+  namespace: ai
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: wiki-anthropic
+  data:
+    - secretKey: Wiki__AnswerModels__Models__claude-sonnet__ApiKey
+      remoteRef:
+        key: apps/ai/wiki-anthropic
+        property: api_key
+---
+# BasicAuth users for the machine door; same OpenBao key as the Python service used.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: wiki-api-basic-auth
+  namespace: ai
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: wiki-api-basic-auth
+  dataFrom:
+    - extract:
+        key: apps/ai/wiki-index-basic-auth

kubernetes/apps/wiki/gate-service.yaml

@@ -0,0 +1,20 @@
+# ExternalName mirror so the ai-namespace Ingress can backend to the wiki-gate
+# outpost in the authentik namespace.
+#
+#   browser → traefik (wiki.renner.uno)
+#           → ak-outpost-wiki-gate (ExternalName → authentik ns)
+#           → outpost validates the session + passkey step-up, sets X-Authentik-Username
+#           → http://wiki.ai.svc.cluster.local:8080
+#
+# In-cluster clients (CI agents, the Gitea webhook) address the Service directly.
+apiVersion: v1
+kind: Service
+metadata:
+  name: ak-outpost-wiki-gate
+  namespace: ai
+spec:
+  type: ExternalName
+  externalName: ak-outpost-wiki-gate.authentik.svc.cluster.local
+  ports:
+    - port: 9000
+      targetPort: 9000

kubernetes/apps/wiki/ingress.yaml

@@ -0,0 +1,75 @@
+# wiki.renner.uno — LAN-only (no dns.renner.uno/public label). Traefik backends
+# the wiki-gate outpost, never the Service; the ai CNP admits only that outpost.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: wiki
+  namespace: ai
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - wiki.renner.uno
+      secretName: wiki-tls
+  rules:
+    - host: wiki.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: ak-outpost-wiki-gate
+                port:
+                  number: 9000
+---
+# wiki-index.renner.uno — the machine door (bin/wiki_rank.py, wiki_ask.py):
+# Traefik BasicAuth, and the identity header stripped so only the outpost path
+# can carry X-Authentik-Username to the service.
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: wiki-api-basic-auth
+  namespace: ai
+spec:
+  basicAuth:
+    secret: wiki-api-basic-auth
+    realm: wiki
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: wiki-api-strip-identity
+  namespace: ai
+spec:
+  headers:
+    customRequestHeaders:
+      X-Authentik-Username: ""
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: wiki-api
+  namespace: ai
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+    traefik.ingress.kubernetes.io/router.middlewares: ai-wiki-api-strip-identity@kubernetescrd,ai-wiki-api-basic-auth@kubernetescrd
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - wiki-index.renner.uno
+      secretName: wiki-api-tls
+  rules:
+    - host: wiki-index.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: wiki
+                port:
+                  number: 8080

kubernetes/apps/wiki/kustomization.yaml

@@ -1,5 +1,7 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
+  - externalsecrets.yaml
   - deployment.yaml
+  - gate-service.yaml
   - ingress.yaml

kubernetes/bootstrap.sh

@@ -1955,6 +1955,7 @@ if phase_runs wait-apps; then
     wait_kustomization paperless           "Paperless (incl. pprl-pg CNPG + data PVCs)"          600
     wait_kustomization home-assistant      "Home Assistant (HAOS backup → native restore)"       180
     wait_kustomization audiobookshelf      "Audiobookshelf (incl. library/metadata PVCs)"        900
+    wait_kustomization navidrome           "Navidrome (incl. data + music library PVCs)"         600
     wait_kustomization immich              "Immich (incl. im-pg CNPG + library PVC)"             1800
 
     # Wait for cert-manager to finish issuing every Certificate. Kustomization

kubernetes/clusters/homelab/libation.yaml

@@ -5,7 +5,7 @@ metadata:
   namespace: flux-system
 spec:
   # Headless Libation CronJob that downloads new Audible books into
-  # the audiobookshelf-books PVC. Lives in the audiobookshelf namespace
+  # the audiobookshelf-books-bulk PVC. Lives in the audiobookshelf namespace
   # because k8s PVCs are namespace-scoped and we share the books volume
   # with the audiobookshelf Deployment. Depends on audiobookshelf so
   # the books PVC exists before the libation-bootstrap secret/CronJob

kubernetes/clusters/homelab/navidrome.yaml

@@ -0,0 +1,19 @@
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: navidrome
+  namespace: flux-system
+spec:
+  interval: 10m
+  retryInterval: 1m
+  timeout: 15m
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./kubernetes/apps/navidrome
+  prune: true
+  wait: true
+  dependsOn:
+    - name: infrastructure
+    - name: infrastructure-crds
+    - name: infrastructure-ceph-cluster

kubernetes/clusters/homelab/paperclip.yaml

@@ -0,0 +1,44 @@
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: paperclip
+  namespace: flux-system
+spec:
+  interval: 10m
+  retryInterval: 1m
+  timeout: 20m
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./kubernetes/apps/paperclip
+  prune: true
+  wait: true
+  dependsOn:
+    # Deliberately NOT dependsOn: monitoring, even though
+    # apps/paperclip/prometheus-read-rbac.yaml puts a Role and a RoleBinding in
+    # namespace `monitoring` (REN-86).
+    #
+    # The only thing those two objects need first is the namespace, and the
+    # namespace does not come from the `monitoring` Kustomization — it is
+    # declared in infrastructure/namespaces/namespaces.yaml and applied by
+    # `infrastructure`, listed right below. `infrastructure` runs wait: false, so
+    # it applies the namespaces on its first pass and only its healthChecks hold
+    # its readiness, which is strictly later; by the time this Kustomization
+    # starts, `monitoring` exists. Role and RoleBinding are core RBAC kinds, and
+    # a RoleBinding is allowed to name a Role that does not exist yet, so
+    # nothing on this path waits on kube-prometheus-stack's CRDs or workloads.
+    #
+    # The edge would not be free either. `monitoring` dependsOn `grafana-db` and
+    # itself waits (wait: true, 30m) on kube-prometheus-stack, Loki, Alloy and
+    # blackbox-exporter. Adding it would stop the Kustomization that carries the
+    # paperclip pod from reconciling whenever any of that is unhealthy — a real
+    # coupling bought for a bootstrap-ordering problem that does not exist.
+    - name: infrastructure
+    - name: infrastructure-crds
+    - name: infrastructure-ceph-cluster
+    - name: infrastructure-cnpg-barman
+  healthCheckExprs:
+    - apiVersion: postgresql.cnpg.io/v1
+      kind: Cluster
+      failed: "has(status.phase) && status.phase.startsWith('Failed')"
+      current: "has(status.phase) && (status.phase == 'Cluster in healthy state' || status.readyInstances > 0)"

kubernetes/clusters/homelab/uptime-monitor.yaml

@@ -0,0 +1,108 @@
+# Automated uptime monitoring (REN-2): Stakater IngressMonitorController keeps
+# UptimeRobot monitors in sync with `EndpointMonitor` custom resources. (Not
+# Ingress annotations — `monitor.stakater.com/enabled` does nothing in IMC v2;
+# there is no Ingress reconciler in the source, and docs/migration-guide.md
+# calls the annotation mechanism obsolete. The opt-in property survives intact:
+# creating an EndpointMonitor per host is more explicit than an annotation, and
+# the default state is still nothing-monitored.) This Kustomization holds the
+# API key, the controller Deployment, its ServiceAccount and its RBAC (REN-6).
+#
+# A DEDICATED Kustomization rather than adding the ExternalSecret to
+# ./kubernetes/infrastructure/post-crds, which is where the other
+# infrastructure-layer ExternalSecrets live. That directory is applied by the
+# `infrastructure-crds` Kustomization, which runs `wait: true` with no explicit
+# healthChecks, so EVERY object under it must go healthy or the whole
+# Kustomization reports NotReady — and 19 other Kustomizations in this directory
+# dependsOn it. An ExternalSecret whose OpenBao path is not yet populated never
+# turns Ready, so putting this key there would put `homelab/infrastructure/uptime-robot`
+# in the critical path of most of the cluster. Here, the only thing a missing or
+# misnamed key can stall is this Kustomization, which nothing depends on.
+#
+# `wait: true` is kept below rather than dropped, but ON ITS OWN IT DOES NOT
+# MAKE A FAILED SYNC VISIBLE — an earlier revision of this comment claimed it
+# did, and that was wrong (REN-100). Flux ships no status reader for
+# `ExternalSecret`, so kstatus falls back to its generic rule, which looks only
+# at `status.observedGeneration` and at conditions of type `Reconciling` /
+# `Stalled`. ESO sets none of those three: a failed sync is reported as
+# `Ready=False, reason=SecretSyncedError`. kstatus therefore returns Current and
+# the Kustomization goes Ready with no Secret behind it.
+#
+# In-repo proof rather than a doc citation: `llama-server` is also `wait: true`
+# and carries the `ai/wiki-embed-basic-auth` ExternalSecret
+# (apps/llama-server/wiki-embed-ingress.yaml). That key sat in ESO retry backoff
+# through ~10 failures on 2026-09-20 and needed a manual `force-sync`. Its
+# Kustomization stayed Ready throughout, which is why a human had to notice.
+#
+# So `wait: true` is paired with the `healthCheckExprs` below, which is the
+# mechanism this repo already uses for CRs kstatus cannot read — CNPG `Cluster`
+# (paperless, immich, paperclip, authentik, woodpecker, monitoring), MariaDB
+# (wordpress) and the Ceph CRs (infrastructure). Now a key that fails to sync
+# holds this Kustomization NotReady, and it is safe to fail loudly here
+# precisely because the blast radius is this Kustomization alone: nothing
+# `dependsOn` it.
+#
+# That gate earns its keep now that the controller is here (REN-6) rather than
+# only the key. IMC GETs its whole configuration from the Secret once, at
+# startup, through the manager's APIReader, and never revalidates. Which failure
+# it papers over depends on which one happens:
+#
+#   * Secret absent (first sync never succeeded — `creationPolicy` is the default
+#     Owner, so no successful sync means no Secret at all): IMC panics in
+#     `LoadControllerConfig` and the pod CrashLoopBackOffs. That one is loud.
+#   * Secret present but stale, because a LATER refresh failed: the pod starts
+#     against the old value, reports Available, and the first sign of trouble is
+#     a rejected UptimeRobot call on some future reconcile. Nothing about
+#     Deployment health says the key is current.
+#
+# The second case is the live one — the Secret exists today — and the CEL rule
+# below is what turns it into a NotReady Kustomization instead of a green one.
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: uptime-monitor
+  namespace: flux-system
+spec:
+  interval: 10m
+  # 2m, not the 1m most app Kustomizations use: what stalls this Kustomization
+  # is an ExternalSecret against a key a human may still be placing in OpenBao,
+  # and a retry every minute against a 404 is just noise in the ESO and Flux
+  # logs.
+  retryInterval: 2m
+  timeout: 5m
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./kubernetes/apps/uptime-monitor
+  prune: true
+  wait: true
+  # No name/namespace qualifier, so this is evaluated against every
+  # ExternalSecret in this Kustomization's inventory. That is exactly one
+  # (apps/uptime-monitor/externalsecret.yaml) — the IMC Deployment added in
+  # REN-6 brought no second one — and any ExternalSecret added here later should
+  # be held to the same bar anyway.
+  #
+  # `external-secrets.io/v1` is the version every one of the 54 ExternalSecret
+  # manifests in this repo is written against, so it is the GVK Flux records in
+  # the inventory and the one that has to be matched here.
+  #
+  # Both expressions are guarded with `has(status.conditions)`, copying
+  # wordpress.yaml. Without the guard, CEL errors instead of evaluating while
+  # `status` is still empty — which is the state a freshly applied ExternalSecret
+  # is in, i.e. exactly the case this check exists for. Unguarded, the
+  # Kustomization would report a CEL evaluation error rather than a clean
+  # in-progress.
+  #
+  # No `inProgress` expression: Flux treats "neither failed nor current" as
+  # in-progress, which is the behaviour wanted here.
+  healthCheckExprs:
+    - apiVersion: external-secrets.io/v1
+      kind: ExternalSecret
+      failed: "has(status.conditions) && status.conditions.exists(c, c.type == 'Ready' && c.status == 'False')"
+      current: "has(status.conditions) && status.conditions.exists(c, c.type == 'Ready' && c.status == 'True')"
+  dependsOn:
+    # ESO itself, and the `uptime-monitor` namespace (declared centrally in
+    # kubernetes/infrastructure/namespaces/namespaces.yaml).
+    - name: infrastructure
+    # The `openbao` ClusterSecretStore, including the conditions.namespaces
+    # allowlist entry this namespace needs.
+    - name: infrastructure-crds

kubernetes/clusters/homelab/wiki.yaml

@@ -1,17 +1,17 @@
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
-  name: wiki-index
+  name: wiki
   namespace: flux-system
 spec:
   interval: 10m
   retryInterval: 1m
-  # First start on an empty PVC embeds every note (~5 min) before /health is 200.
-  timeout: 20m
+  # First start on an empty PVC syncs and embeds the whole vault before /ready is 200.
+  timeout: 25m
   sourceRef:
     kind: GitRepository
     name: flux-system
-  path: ./kubernetes/apps/wiki-index
+  path: ./kubernetes/apps/wiki
   prune: true
   wait: true
   dependsOn:

kubernetes/infrastructure/image-automation/README.md

@@ -49,7 +49,8 @@ existing `flux-system` GitRepository whose Secret was created by
 | redis-immich | `7.4.x` | `-alpine` suffix extract; tight to match LXC 225 compose |
 | redis-paperless | `8.x` | `-alpine` suffix extract |
 | ceph | `19.x` | v-prefix filter; rejects date-suffixed builds |
-| tika | `3.x` | 4-component → 3-component extract; theoretical loss of 4th-component precision (always `0` in history) |
+| tika | `3.x` | 4-component tags; `.0` image-revision pinned in the filter so the extracted key is unique |
+| lidarr | Lidarr `3.x` | LinuxServer `-ls<n>` rebuilds; synthetic `<build>.<ls>.0` semver key — major bounded in the filter, not the range |
 | protonmail-bridge | `3.x` | `-1` build-suffix extract |
 | firefox | `1.x` | `-rolling-daily` suffix extract; rejects arch-prefixed duplicates |
 | mmwave-configurator | `2.x` | Plain 3-component filter (rejects `addon-`/`-armv7`/`-arm64`/`-amd64` variants) |
@@ -120,6 +121,16 @@ unaffected.
   releases land in master immediately. Mitigation: ImagePolicy ranges
   must be tight (`semver: range: "2.x"` not `"*"`) so a major bump
   requires manual policy widening.
+- **`extract` must produce a total order over the tags `pattern`
+  matches.** Any part of the tag that varies has to be either in the
+  extracted key or excluded by the filter. A component that is matched
+  but not extracted makes two different tags share one key, and the
+  winner is then an arbitrary tie-break that can flap and downgrade a
+  running app with no human in the loop. This is not hypothetical: it
+  auto-committed a lidarr downgrade (ls42 → ls11) to master in
+  `b858b4aa` and rolled the pod. See `lidarr.yaml` for the two-field
+  case (synthetic semver key) and `tika.yaml` / `protonmail-bridge.yaml`
+  for the narrow-the-filter case.
 - Image findings still surface in the security-scan agent's report
   (Skipped / "Flux image-automation owns this") so they remain visible
   even when an image isn't yet enrolled in automation here.

kubernetes/infrastructure/image-automation/kustomization.yaml

@@ -9,11 +9,14 @@ resources:
   - gotenberg.yaml
   - home-assistant.yaml
   - immich.yaml
+  - lidarr.yaml
   - matter-server.yaml
   - mmwave-configurator.yaml
   - music-assistant.yaml
+  - navidrome.yaml
   - netbird-ingress.yaml
   - paperless-ngx.yaml
+  - paperclip.yaml
   - protonmail-bridge.yaml
   - mosquitto.yaml
   - redis.yaml
@@ -24,5 +27,6 @@ resources:
   - valkey.yaml
   - velero-plugin-for-aws.yaml
   - vsftpd.yaml
+  - wiki.yaml
   - woodpecker.yaml
   - wyoming.yaml

kubernetes/infrastructure/image-automation/lidarr.yaml

@@ -0,0 +1,57 @@
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImageRepository
+metadata:
+  name: lidarr
+  namespace: flux-system
+spec:
+  image: lscr.io/linuxserver/lidarr
+  interval: 1h
+---
+# Tags are <maj>.<min>.<patch>.<build>-ls<n>. Two numbers move, and BOTH
+# must be in the sort key:
+#   <build>  upstream Lidarr's CI build counter, monotonic (4658 -> 4711
+#            -> 4836 -> 4866 -> 4875 over the visible history).
+#   ls<n>    LinuxServer's rebuild counter. Bumped weekly for security
+#            rebuilds of the same upstream build. Not globally unique and
+#            NOT monotonic on its own: it resets on some upstream bumps
+#            (2.12.4.4658-ls50 -> 2.13.3.4711-ls1) and has been reused
+#            across builds (3.0.1.4866-ls11 and 3.1.0.4875-ls11).
+#
+# Extracting <build> alone is not a total order: all 32 published ls
+# rebuilds of build 4875 collapse to the same key and the winner is an
+# arbitrary tie-break. That fired in production on 2026-09-25 (b858b4aa),
+# auto-committing ls42 -> ls11 and rolling the pod.
+#
+# `numerical` cannot order two fields, and `alphabetical` over a
+# concatenation would mis-sort on width (ls9 vs ls11). So the key is a
+# synthetic semver <build>.<ls>.0 and the policy is semver. Comparison is
+# per-component and numeric, so it is width-safe and orders exactly as
+# (build, ls) lexicographic -- build dominates, which is what makes the
+# ls reset above harmless.
+#
+# The Lidarr major is bounded in filterTags, not in the range, because
+# the range operates on the synthetic key where "3" would mean build 3.
+# A Lidarr 4.x bump therefore needs a deliberate edit here, per the
+# "ranges must be tight" rule in README.md.
+#
+# The range floor is the build we are on. It never needs bumping (it only
+# rejects sorting backwards past 4875) and is belt-and-braces against a
+# future key mistake, not a version bound.
+#
+# Residual risk: if LinuxServer ever reset ls within one build number,
+# that would still read as a downgrade. It has not done so in the visible
+# history -- every reset coincided with a build change.
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImagePolicy
+metadata:
+  name: lidarr
+  namespace: flux-system
+spec:
+  imageRepositoryRef:
+    name: lidarr
+  filterTags:
+    pattern: '^3\.\d+\.\d+\.(?P<build>\d+)-ls(?P<ls>\d+)$'
+    extract: '${build}.${ls}.0'
+  policy:
+    semver:
+      range: ">=4875.0.0"

kubernetes/infrastructure/image-automation/navidrome.yaml

@@ -0,0 +1,20 @@
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImageRepository
+metadata:
+  name: navidrome
+  namespace: flux-system
+spec:
+  image: deluan/navidrome
+  interval: 1h
+---
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImagePolicy
+metadata:
+  name: navidrome
+  namespace: flux-system
+spec:
+  imageRepositoryRef:
+    name: navidrome
+  policy:
+    semver:
+      range: ">=0.64.0"

kubernetes/infrastructure/image-automation/paperclip.yaml

@@ -0,0 +1,25 @@
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImageRepository
+metadata:
+  name: paperclip
+  namespace: flux-system
+spec:
+  image: ghcr.io/paperclipai/paperclip
+  interval: 1h
+---
+# Upstream publishes no semver tags; track the nightly tag by digest.
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImagePolicy
+metadata:
+  name: paperclip
+  namespace: flux-system
+spec:
+  imageRepositoryRef:
+    name: paperclip
+  digestReflectionPolicy: Always
+  interval: 1h
+  filterTags:
+    pattern: '^nightly$'
+  policy:
+    alphabetical:
+      order: asc

kubernetes/infrastructure/image-automation/tika.yaml

@@ -7,14 +7,32 @@ spec:
   image: apache/tika
   interval: 1h
 ---
-# Upstream uses 4-component versions (3.3.0.0). Semver only sorts on the
-# first three; the filter discards the trailing component on extract.
-# In practice tika's 4th component has been `0` across the visible tag
-# history, so the loss of precision is theoretical. A future
-# 3.3.0.1-style patch would not auto-bump from 3.3.0.0; pin manually
-# if that ever happens. The filter also rejects `-full` variant tags
-# and `latest`/`-SNAPSHOT`/`-alpha-N` pre-releases. Range stays on 3.x
-# so the 4.0.0 GA (currently alpha/SNAPSHOT) is an explicit decision.
+# Upstream uses 4-component tags: <tika-version>.<image-revision>, e.g.
+# 3.3.1.0. Semver only sorts three components, so the extract has to drop
+# one. Dropping the revision on extract alone is not a total order --
+# 2.5.0.1/2.5.0.2, 2.6.0.0/2.6.0.1, 2.7.0.0/2.7.0.1 and 2.9.2.0/2.9.2.1
+# all collapse to one key, and the winner would be an arbitrary tie-break
+# that can flap and downgrade (same defect as lidarr.yaml, REN-91).
+#
+# The revision is therefore pinned in filterTags instead, the way
+# protonmail-bridge.yaml pins its `-1` build suffix: only `.0` matches, so
+# one tag maps to one key. Every 3.x release so far has been `.0`, so this
+# is a no-op today -- the winner is still 3.3.1.0.
+#
+# Failure mode if upstream ever ships a 3.x `.1`: the policy stops bumping
+# rather than picking wrongly. That is deliberate -- but note that nothing
+# reports the stall. The update-audit agent skips ImagePolicy-managed
+# images by design (ci/agents/update-audit/prompt.md, "skip those images
+# entirely"), the security-scan agent defers the other way ("Flux
+# image-automation owns this", README.md), and the Flux controllers are
+# not scraped, so there is no `gotk_reconcile_condition` series to alert
+# on either. REN-97 tracks closing that gap. Until it lands, a stall here
+# is found by hand; the fix is to bump the pin and decide then whether the
+# revision stream needs real tracking.
+#
+# The filter also rejects `-full` variant tags and `latest`/`-SNAPSHOT`/
+# `-alpha-N` pre-releases. Range stays on 3.x so the 4.0.0 GA (currently
+# alpha/SNAPSHOT) is an explicit decision.
 apiVersion: image.toolkit.fluxcd.io/v1
 kind: ImagePolicy
 metadata:
@@ -24,7 +42,7 @@ spec:
   imageRepositoryRef:
     name: tika
   filterTags:
-    pattern: '^(?P<ver>\d+\.\d+\.\d+)\.\d+$'
+    pattern: '^(?P<ver>\d+\.\d+\.\d+)\.0$'
     extract: '$ver'
   policy:
     semver:

kubernetes/infrastructure/image-automation/update-automation.yaml

@@ -8,8 +8,15 @@ spec:
   # Reuses the existing flux-system GitRepository — its Secret was
   # created by `flux bootstrap gitea --token-auth` and already has
   # write access to the repo. Push identity is the bootstrap token's
-  # owner (gitea_admin); commit author is overridden below to
-  # security-bot@renner.uno so commits stay visibly machine-authored.
+  # owner, which is the `flux` Gitea account (uid 9) — not
+  # gitea_admin, as this comment used to say. Evidence: the flux
+  # account's contribution history matches every
+  # `chore(image-automation)` commit on master 1:1, while
+  # gitea_admin's own contributions stop at 2026-07-21. That account
+  # therefore has to keep push access to master, or writeback fails
+  # silently — no commit is attempted unless a tracked image moves.
+  # The commit author overridden below to security-bot@renner.uno is
+  # the author field only; it says nothing about who pushes.
   sourceRef:
     kind: GitRepository
     name: flux-system

kubernetes/infrastructure/image-automation/wiki.yaml

@@ -0,0 +1,24 @@
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImageRepository
+metadata:
+  name: wiki
+  namespace: flux-system
+spec:
+  image: gitea.renner.uno/konairius/wiki
+  interval: 10m
+---
+# The Wiki release pipeline tags every main build `build.<unix-ts>`; latest push wins.
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImagePolicy
+metadata:
+  name: wiki
+  namespace: flux-system
+spec:
+  imageRepositoryRef:
+    name: wiki
+  filterTags:
+    pattern: '^build\.(?P<ts>\d+)$'
+    extract: '$ts'
+  policy:
+    numerical:
+      order: asc

kubernetes/infrastructure/namespaces/namespaces.yaml

@@ -78,6 +78,11 @@ metadata:
 ---
 apiVersion: v1
 kind: Namespace
+metadata:
+  name: navidrome
+---
+apiVersion: v1
+kind: Namespace
 metadata:
   name: immich
 ---
@@ -234,3 +239,24 @@ metadata:
   labels:
     app.kubernetes.io/name: woodpecker
     pod-security.kubernetes.io/enforce: privileged
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: paperclip
+---
+# IngressMonitorController, which syncs UptimeRobot monitors from
+# `EndpointMonitor` custom resources (REN-2; not Ingress annotations — those are
+# obsolete in IMC v2). Declared here rather than in the workload Kustomization
+# for the reason spelled out above `woodpecker`: the `infrastructure` KS creates
+# namespaces, and `infrastructure-crds` (post-crds, which includes
+# ../network-policies/) runs after it — so declaring it in the workload KS would
+# invert that order the moment this namespace gains a CiliumNetworkPolicy. It
+# has one now: ../network-policies/uptime-monitor-ns.yaml.
+#
+# No pod-security labels: the controller needs no hostPath, hostNetwork or
+# privilege, so the cluster's default `baseline` enforcement is enough.
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: uptime-monitor

kubernetes/infrastructure/network-policies/ai-ns.yaml

@@ -149,36 +149,34 @@ spec:
             - port: "8080"
               protocol: TCP
 ---
-# wiki-index: Traefik (LAN clients behind BasicAuth) and Gitea (push webhook)
-# may reach it; it reaches OpenBao for its credentials. Gitea itself is reached
-# through Traefik, which the namespace egress above already allows.
+# wiki: the wiki-gate outpost (humans, after Authentik), Traefik (the machine
+# door wiki-index.renner.uno, BasicAuth with X-Authentik-Username stripped),
+# Gitea (push webhook) and Paperclip agents (API key) may reach it. Gitea and wiki-embed are reached through
+# the namespace egress above; no OpenBao access.
 apiVersion: cilium.io/v2
 kind: CiliumNetworkPolicy
 metadata:
-  name: wiki-index
+  name: wiki
   namespace: ai
 spec:
   endpointSelector:
     matchLabels:
-      app.kubernetes.io/name: wiki-index
+      app.kubernetes.io/name: wiki
   ingress:
     - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+            goauthentik.io/outpost-name: wiki-gate
         - matchLabels:
             "k8s:io.kubernetes.pod.namespace": ingress
             app.kubernetes.io/name: traefik
         - matchLabels:
             "k8s:io.kubernetes.pod.namespace": gitea
             app.kubernetes.io/name: gitea
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+            app.kubernetes.io/name: paperclip
       toPorts:
         - ports:
             - port: "8080"
               protocol: TCP
-  egress:
-    - toEndpoints:
-        - matchLabels:
-            "k8s:io.kubernetes.pod.namespace": openbao
-            app.kubernetes.io/name: openbao
-      toPorts:
-        - ports:
-            - port: "8200"
-              protocol: TCP

kubernetes/infrastructure/network-policies/external-secrets-ns.yaml

@@ -10,6 +10,29 @@ spec:
     - fromEndpoints:
         - matchLabels:
             "k8s:io.kubernetes.pod.namespace": external-secrets
+    # Prometheus scrape of the ESO controller's metrics (REN-100). Until this
+    # existed a failed ExternalSecret sync was invisible to Prometheus: this
+    # namespace admitted only the rules either side of this one, so the scrape was
+    # dropped and `externalsecret_status_condition` held 0 series.
+    #
+    # Port-restricted, unlike the equivalent rule in velero-ns.yaml, which admits
+    # `monitoring` to the whole namespace. 8080 is the metrics port and the only
+    # thing `monitoring` has any business reaching here; the webhook (10250) and
+    # the health port stay closed to it. 8080 is the chart default
+    # (`metrics.listen.port`) and helmrelease.yaml does not override it — if that
+    # is ever overridden, this port and the PodMonitor's named `metrics` endpoint
+    # both have to follow.
+    #
+    # The consumer is apps/monitoring-rules/external-secrets-podmonitor.yaml.
+    # Additive: this rule cannot deny anything previously allowed, so ESO's
+    # existing reachability to OpenBao and the apiserver is unchanged.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": monitoring
+      toPorts:
+        - ports:
+            - port: "8080"
+              protocol: TCP
     # kube-apiserver (host identity in Cilium) drives the ExternalSecret/
     # SecretStore validating webhook and the controller's watch/list/create
     # calls. Without this, every ExternalSecret apply hangs the admission

kubernetes/infrastructure/network-policies/gitea-ns.yaml

@@ -65,11 +65,11 @@ spec:
               protocol: UDP
             - port: "53"
               protocol: TCP
-    # Push webhook of the notes repo to wiki-index.
+    # Push webhook of the notes repo to the wiki service.
     - toEndpoints:
         - matchLabels:
             "k8s:io.kubernetes.pod.namespace": ai
-            app.kubernetes.io/name: wiki-index
+            app.kubernetes.io/name: wiki
       toPorts:
         - ports:
             - port: "8080"

kubernetes/infrastructure/network-policies/kustomization.yaml

@@ -6,6 +6,7 @@ resources:
   - ingress-isolation.yaml
   - authentik-ns.yaml
   - paperless-ns.yaml
+  - paperclip-ns.yaml
   - audiobookshelf-ns.yaml
   - home-assistant-ns.yaml
   - spegel-ns.yaml
@@ -27,4 +28,6 @@ resources:
   - trmnl-ns.yaml
   - wordpress-ns.yaml
   - ai-ns.yaml
+  - navidrome-ns.yaml
+  - uptime-monitor-ns.yaml
   - allow-cilium-health.yaml

kubernetes/infrastructure/network-policies/navidrome-ns.yaml

@@ -0,0 +1,96 @@
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: default-deny-ingress
+  namespace: navidrome
+spec:
+  endpointSelector: {}
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": navidrome
+    # Traefik: the ungated /rest and /share routes (identity header stripped).
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+            goauthentik.io/outpost-name: navidrome-gate
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+            goauthentik.io/outpost-name: lidarr-gate
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": monitoring
+    - fromEntities:
+        - host
+        - remote-node
+    # ICMP — see home-assistant-ns.yaml.
+    - fromEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": navidrome
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+    # Metadata/artwork agents (Last.fm, Deezer, ListenBrainz).
+    - toEntities:
+        - world
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
+    - toEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }

kubernetes/infrastructure/network-policies/openbao-ns.yaml

@@ -31,16 +31,6 @@ spec:
         - ports:
             - port: "8200"
               protocol: TCP
-    # wiki-index (ns ai) logs in via auth/kubernetes/role/wiki-index and reads
-    # its Gitea token and webhook secret on every sync.
-    - fromEndpoints:
-        - matchLabels:
-            "k8s:io.kubernetes.pod.namespace": ai
-            app.kubernetes.io/name: wiki-index
-      toPorts:
-        - ports:
-            - port: "8200"
-              protocol: TCP
     # netbird-service-sync logs in via auth/kubernetes/role/netbird-service-sync
     # and reads the NetBird PAT on every pass.
     - fromEndpoints:

kubernetes/infrastructure/network-policies/paperclip-ns.yaml

@@ -0,0 +1,97 @@
+# Traefik is not admitted: the only entry is the paperclip-gate outpost (apps/paperclip/network-policy.yaml).
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: default-deny-ingress
+  namespace: paperclip
+spec:
+  endpointSelector: {}
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": database
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": monitoring
+    - fromEntities:
+        - host
+        - remote-node
+    - fromEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": authentik
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": rook-ceph
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+    - toEntities:
+        - host
+        - remote-node
+    # Cilium unions allows across CNPs, so world stays :443-only here too.
+    - toEntities:
+        - world
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
+    - toEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }

kubernetes/infrastructure/network-policies/uptime-monitor-ns.yaml

@@ -0,0 +1,54 @@
+---
+# Baseline policy for the `uptime-monitor` namespace (REN-6).
+#
+# Copied from netbird-service-sync-ns.yaml, which is the closest existing
+# neighbour in this directory: a controller that serves nothing and only calls
+# out — kube-apiserver plus one external HTTPS API. Same shape here.
+#
+# No Traefik ingress rule, because nothing in this namespace is behind an
+# Ingress. (For anyone reaching for a wiki note: the selector to admit Traefik
+# lives in `ingress` with label `app.kubernetes.io/name: traefik`, as in
+# paperclip-ns.yaml. An `ingress-nginx` selector matches nothing in this cluster
+# and would admit nothing, silently.)
+#
+# Deliberately no OpenBao egress: the controller never talks to OpenBao. ESO, in
+# the external-secrets namespace, resolves the key and writes the Secret; the
+# controller only reads that Secret through the API server.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: default-deny
+  namespace: uptime-monitor
+spec:
+  endpointSelector: {}
+  ingress:
+    # Kubelet liveness/readiness probes on :8081 arrive with the `host`
+    # identity. This is the only ingress the namespace admits — without it the
+    # controller pod fails its probes and CrashLoopBackOffs, with nothing in its
+    # own logs to say why.
+    - fromEntities:
+        - host
+        - remote-node
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+    # EndpointMonitor watch, Ingress reads, and the startup GET of the config
+    # Secret.
+    - toEntities:
+        - kube-apiserver
+    # api.uptimerobot.com. Not reached at all until an EndpointMonitor exists:
+    # the controller makes no API call at startup.
+    - toEntities:
+        - world
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP

kubernetes/infrastructure/openbao/policies/rotator-mosquitto-open3e-int.hcl

@@ -0,0 +1,30 @@
+# OpenBao policy: rotator-mosquitto-open3e-int
+#
+# Least-privilege policy for the mosquitto `open3e-int` broker-credential
+# rotation CronJob (kubernetes auth role `rotator-mosquitto-open3e-int`).
+# Per-secret isolation, same template as rotator-mosquitto-health.
+#
+# NOT applied by Flux. Apply ONCE with an admin token:
+#
+#   bao policy write rotator-mosquitto-open3e-int \
+#     kubernetes/infrastructure/openbao/policies/rotator-mosquitto-open3e-int.hcl
+#
+#   bao write auth/kubernetes/role/rotator-mosquitto-open3e-int \
+#     bound_service_account_names=mosquitto-open3e-int-secret-rotator \
+#     bound_service_account_namespaces=openbao \
+#     policies=rotator-mosquitto-open3e-int ttl=15m
+#
+# Do NOT set token_no_default_policy=true (kv-v2 patch preflight needs default).
+#
+# BOOTSTRAP: `random` mode rotates an existing field and never creates one.
+# Seed the path once with a `password` field before the first rotation:
+#
+#   bao kv put -mount=homelab apps/home-assistant/mosquitto-open3e-int-credentials \
+#     password=<generated>
+#
+# The consumer is the open3e-esp32 gateway on the heat pump's internal CAN
+# (NVS key mq_pass); a rotation takes effect on the device only after its
+# MQTT settings are updated (web UI or NVS image).
+path "homelab/data/apps/home-assistant/mosquitto-open3e-int-credentials" {
+  capabilities = ["read", "patch"]
+}

kubernetes/infrastructure/openbao/policies/wiki-index.hcl

@@ -1,21 +0,0 @@
-# OpenBao policy: wiki-index
-#
-# The wiki-index Deployment (namespace ai) reads its Gitea token and the
-# webhook HMAC secret on every use. Read-only on that one path.
-#
-# NOT applied by Flux. Apply ONCE with an admin token:
-#
-#   bao policy write wiki-index kubernetes/infrastructure/openbao/policies/wiki-index.hcl
-#   bao write auth/kubernetes/role/wiki-index \
-#     bound_service_account_names=wiki-index \
-#     bound_service_account_namespaces=ai \
-#     policies=wiki-index ttl=15m
-#
-# Secret: homelab/apps/ai/wiki-index, keys `gitea_token` (read:repository on
-# the notes repo) and `webhook_secret` (the Gitea webhook's secret).
-#
-# Do NOT set token_no_default_policy=true (kv-v2 preflight needs the default policy).
-
-path "homelab/data/apps/ai/wiki-index" {
-  capabilities = ["read"]
-}

kubernetes/infrastructure/openbao/rotation/kustomization.yaml

@@ -48,6 +48,7 @@ resources:
   - review-bot.yaml
   - claude-oauth.yaml
   - mosquitto-health.yaml
+  - mosquitto-open3e-int.yaml
 # NOT ROTATED, deliberately — recorded here because the whole point of this
 # directory is that unrotated credentials should be a visible choice:
 #

kubernetes/infrastructure/openbao/rotation/mosquitto-open3e-int.yaml

@@ -0,0 +1,79 @@
+---
+# Class-A rotation of the mosquitto `open3e-int` broker credential: the
+# open3e-esp32 gateway on the Vitocal internal CAN (ECU 0x6e0). Same
+# rotator -> OpenBao -> ESO -> Reloader chain as mosquitto-health.yaml.
+#
+# The consumer is an embedded device: after a rotation its MQTT password must
+# be updated on the device (web UI Einstellungen, or an NVS image over USB).
+#
+# ON-DEMAND: kubectl -n openbao create job --from=cronjob/mosquitto-open3e-int-secret-rotate mosquitto-open3e-int-secret-rotate-now
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: mosquitto-open3e-int-secret-rotator
+  namespace: openbao
+  labels:
+    app.kubernetes.io/name: mosquitto
+    app.kubernetes.io/component: secret-rotator
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: mosquitto-open3e-int-secret-rotate
+  namespace: openbao
+  labels:
+    app.kubernetes.io/name: mosquitto
+    app.kubernetes.io/component: secret-rotator
+spec:
+  # Quarterly, own month offset (paperless: 1,4,7,10; health: 2,5,8,11).
+  schedule: "0 4 1 3,6,9,12 *"
+  timeZone: Europe/Berlin
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 2
+  failedJobsHistoryLimit: 5
+  startingDeadlineSeconds: 3600
+  jobTemplate:
+    spec:
+      backoffLimit: 2
+      activeDeadlineSeconds: 300
+      ttlSecondsAfterFinished: 259200 # 3 d
+      template:
+        metadata:
+          labels:
+            app.kubernetes.io/name: mosquitto
+            app.kubernetes.io/component: secret-rotator
+        spec:
+          serviceAccountName: mosquitto-open3e-int-secret-rotator
+          restartPolicy: Never
+          securityContext:
+            runAsNonRoot: true
+            runAsUser: 65534
+            runAsGroup: 65534
+            seccompProfile:
+              type: RuntimeDefault
+          containers:
+            - name: rotate
+              image: gitea.renner.uno/gitea_admin/secret-rotator:2026-07-30
+              args: ["random"]
+              securityContext:
+                allowPrivilegeEscalation: false
+                readOnlyRootFilesystem: true
+                capabilities:
+                  drop: ["ALL"]
+              env:
+                - name: BAO_ADDR
+                  value: http://openbao.openbao.svc.cluster.local:8200
+                - name: BAO_ROLE
+                  value: rotator-mosquitto-open3e-int
+                - name: BAO_PATH
+                  value: homelab/apps/home-assistant/mosquitto-open3e-int-credentials
+                - name: BAO_FIELDS
+                  value: "password"
+                - name: RANDOM_BYTES
+                  value: "32"
+              resources:
+                requests:
+                  cpu: 25m
+                  memory: 16Mi
+                limits:
+                  memory: 48Mi