fix(authentik): allow authorization_code for matrix-mas (REN-567)

Authentik 2026.5 checks each authorize against the provider's grant_types, and a provider created by a blueprint starts with an empty list. Every Matrix login through Authentik failed with invalid_request (log: "Invalid grant_type for provider", grant_type=authorization_code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merge pull request 'feat(matrix): MAS with Authentik passkey login and QR sign-in (REN-567)' (#1121 ) from atlas/ren-567-mas into master
2026-10-04 21:19:26 +00:00 · 2026-10-04 20:51:03 +00:00 · 2026-10-04 20:11:32 +00:00 · 2026-10-04 20:08:04 +00:00 · 2026-10-04 19:43:36 +00:00 · 2026-10-04 15:24:30 +00:00
50 changed files with 2040 additions and 69 deletions
@@ -62,7 +62,7 @@ together with Paperclip. All tools are on the default `PATH`.

 | Image | `imageRegistry` | Adds |
 |---|---|---|
-| `gitea.renner.uno/konairius/agent-runtime-go/agent-runtime-claude:v1` | `gitea.renner.uno/konairius/agent-runtime-go` | Go toolchain, golangci-lint |
+| `gitea.renner.uno/konairius/agent-runtime-go/agent-runtime-claude:v1` | `gitea.renner.uno/konairius/agent-runtime-go` | Go toolchain, golangci-lint, govulncheck |
 | `gitea.renner.uno/konairius/agent-runtime-ops/agent-runtime-claude:v1` | `gitea.renner.uno/konairius/agent-runtime-ops` | kubectl, flux, kustomize, helm |

 The plugin keeps the name `agent-runtime-claude:v1` and swaps only host and
@@ -7,6 +7,13 @@
 #
 # Both binaries are static; they run on the Ubuntu 22.04 base unchanged.
 FROM golang:1.27.1-bookworm@sha256:69a7b9788769bec032d238959b61854e9ae87f57be9029ec04e9885fabf99195 AS go
+
+# govulncheck (REN-488) has no upstream image: build it in the go stage, static.
+# The version pins the module; go verifies it against sum.golang.org. Renovate
+# does not track it (no regex manager here): bump by hand.
+FROM go AS govulncheck
+RUN CGO_ENABLED=0 GOBIN=/out go install golang.org/x/vuln/cmd/govulncheck@v1.8.0
+
 FROM golangci/golangci-lint:v2.14.0@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS golangci-lint

 # upstream publishes no `v1` tag, only git-<sha> tags. Renovate keeps the
@@ -15,6 +22,7 @@ FROM ghcr.io/paperclipai/agent-runtime-claude:git-38d8f371722b315d2fb3bbaa512518

 COPY --from=go /usr/local/go /usr/local/go
 COPY --from=golangci-lint /usr/bin/golangci-lint /usr/local/bin/golangci-lint
+COPY --from=govulncheck /out/govulncheck /usr/local/bin/govulncheck
 # go and gofmt also in /usr/local/bin: a process started with the default PATH
 # (not the image ENV) still finds them.
 USER root
@@ -57,4 +65,4 @@ ENV PATH=/usr/local/go/bin:$PATH \
 # The PATH check uses the base image's default PATH, not the ENV above.
 RUN go version && golangci-lint version \
    && env -i PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
-       sh -ec 'for t in go gofmt golangci-lint claude-agent-acp; do command -v "$t"; done'
+       sh -ec 'for t in go gofmt golangci-lint govulncheck claude-agent-acp; do command -v "$t"; done'
@@ -20,6 +20,27 @@ paperclipai/paperclip#14769, also open: the plugin sets `remoteCwd:
 /workspace` when it acquires or resumes a lease.
 `pr-14769-probe-remote-cwd.patch` is that PR's diff (head `10dc7371`).

+Claude auto memory is lost after every run (REN-541): it lives under
+`CLAUDE_CONFIG_DIR`, which Paperclip makes fresh per run and deletes.
+`ren-541-claude-auto-memory.patch` makes the ACPX engine write
+`autoMemoryDirectory: <AGENT_HOME>/memory` into the per-run
+`.claude/settings.local.json`. `AGENT_HOME` is restored before and saved after
+every run, local and in k8s pods. The value is not part of the session
+fingerprint, so resume is unchanged. No upstream PR yet; rename the patch to
+`pr-<n>-…` once there is one.
+
+Claude sessions never resume with a managed AI connection (REN-563): the
+server gives every run a fresh temp home (`HOME`, `XDG_*`,
+`CLAUDE_CONFIG_DIR`, …) and deletes it after the run.
+`ren-563-acpx-claude-session-resume.patch` fixes both halves for local runs:
+the ACPX engine hashes a fixed placeholder for those paths in the session
+fingerprint (`adapterEnvHash`), as the server's own fingerprint already does,
+and links `CLAUDE_CONFIG_DIR/projects` (the Claude transcripts) to
+`<stateDir>/claude-projects`, which persists. Both must ship together: a
+stable fingerprint without the transcript makes the resumed turn fail with
+"Resource not found" (homelab#1111, reverted in #1113). Remote runs are
+unchanged. No upstream PR yet.
+
 ## Files

 - `upstream.env` — pinned commit and the version string stamped into the
@@ -39,10 +60,10 @@ Set `PAPERCLIP_COMMIT` to the new master commit and `PAPERCLIP_VERSION` to
 `git describe --tags --match 'v*' --long <commit>`, then run
 `sh prepare.sh /tmp/src`: it fails if a patch no longer applies.

-## Exit: when #13733 and #14769 are merged upstream
+## Exit: when every patch here is merged upstream

 Renovate cannot watch an upstream PR, so this is a manual step. Once
-upstream nightly contains both fixes:
+upstream nightly contains all of them:

 1. `kubernetes/infrastructure/image-automation/paperclip.yaml`: back to
   `ghcr.io/paperclipai/paperclip` with `filterTags.pattern: '^nightly$'`
@@ -0,0 +1,118 @@
+diff --git a/packages/adapter-utils/src/acpx-engine/execute.test.ts b/packages/adapter-utils/src/acpx-engine/execute.test.ts
+index a26f980..112e2eb 100644
+--- a/packages/adapter-utils/src/acpx-engine/execute.test.ts
+++ b/packages/adapter-utils/src/acpx-engine/execute.test.ts
+@@ -2544,6 +2544,37 @@ describe("shared ACPX engine runtime behavior", () => {
+     expect(note).toBeTruthy();
+   });
+ 
+  it("points Claude auto memory at the run's AGENT_HOME without changing the session fingerprint", async () => {
+    const root = await makeTempRoot();
+    const stateDir = path.join(root, "state");
+    const cwd = path.join(root, "worktree");
+    await fs.mkdir(cwd, { recursive: true });
+    const stableHome = path.join(root, "agent-home");
+    const runAgentHome = (runId: string) => path.join(root, "file-sync", "runs", runId, "live");
+    const readSettings = async () =>
+      JSON.parse(await fs.readFile(path.join(cwd, ".claude", "settings.local.json"), "utf8")) as {
+        autoMemoryDirectory?: string;
+        permissions?: { additionalDirectories?: string[] };
+      };
+
+    const first = await runExecutor(
+      { agent: "claude", stateDir, cwd },
+      { context: { paperclipWorkspace: { cwd, agentHome: runAgentHome("a"), agentHomeForPermissions: stableHome } } },
+    );
+    expect((await readSettings()).autoMemoryDirectory).toBe(path.join(runAgentHome("a"), "memory"));
+
+    const second = await runExecutor(
+      { agent: "claude", stateDir, cwd },
+      { context: { paperclipWorkspace: { cwd, agentHome: runAgentHome("b"), agentHomeForPermissions: stableHome } } },
+    );
+    const written = await readSettings();
+    expect(written.autoMemoryDirectory).toBe(path.join(runAgentHome("b"), "memory"));
+    expect(written.permissions?.additionalDirectories).toContain(stableHome);
+    expect(written.permissions?.additionalDirectories).not.toContain(runAgentHome("b"));
+    expect(second.result.sessionParams?.configFingerprint).toBeTypeOf("string");
+    expect(second.result.sessionParams?.configFingerprint).toBe(first.result.sessionParams?.configFingerprint);
+  });
+
+   it("merges Paperclip allowlist into an existing .claude/settings.local.json without losing user entries", async () => {
+     const root = await makeTempRoot();
+     const stateDir = path.join(root, "state");
+diff --git a/packages/adapter-utils/src/acpx-engine/execute.ts b/packages/adapter-utils/src/acpx-engine/execute.ts
+index b1dc569..e5681b8 100644
+--- a/packages/adapter-utils/src/acpx-engine/execute.ts
+++ b/packages/adapter-utils/src/acpx-engine/execute.ts
+@@ -1558,6 +1558,7 @@ interface PaperclipClaudeSettingsResult {
+   additionalDirectories: string[];
+   defaultMode: string;
+   overrodeDontAsk: boolean;
+  autoMemoryDirectory: string | null;
+ }
+ 
+ function uniqueSorted(values: Array<string | null | undefined>): string[] {
+@@ -1571,10 +1572,19 @@ function uniqueSorted(values: Array<string | null | undefined>): string[] {
+ // denies every non-allowlisted tool and never reaches `canUseTool`), and we
+ // widen the SDK's Read sandbox to include the Paperclip state dirs the agent
+ // needs to talk to its own control plane.
+//
+// It also points Claude's auto memory at `<AGENT_HOME>/memory`. The default
+// location lives under CLAUDE_CONFIG_DIR, which is a per-run temp home, so
+// memories were lost after every run. AGENT_HOME is restored before and saved
+// after each run, locally and in remote sandboxes. Claude ignores
+// `autoMemoryDirectory` only in checked-in project settings, not local ones.
+ async function writePaperclipClaudeSettings(input: {
+   cwd: string;
+   stateDir: string;
+   agentHome: string;
+  // The run's actual AGENT_HOME (the in-sandbox path for remote runs). It may
+  // differ from `agentHome`, which stays stable for session identity.
+  runAgentHome: string;
+   companyId: string;
+ }): Promise<PaperclipClaudeSettingsResult> {
+   const filePath = path.join(input.cwd, ".claude", "settings.local.json");
+@@ -1630,7 +1640,14 @@ async function writePaperclipClaudeSettings(input: {
+     additionalDirectories: mergedAdditionalDirectories,
+     defaultMode,
+   };
+-  const next: Record<string, unknown> = { ...existing, permissions: nextPermissions };
+  const autoMemoryDirectory = input.runAgentHome
+    ? path.posix.join(input.runAgentHome, "memory")
+    : null;
+  const next: Record<string, unknown> = {
+    ...existing,
+    permissions: nextPermissions,
+    ...(autoMemoryDirectory ? { autoMemoryDirectory } : {}),
+  };
+   await writeFileAtomically({
+     target: filePath,
+     contents: `${JSON.stringify(next, null, 2)}\n`,
+@@ -1642,6 +1659,7 @@ async function writePaperclipClaudeSettings(input: {
+     additionalDirectories: mergedAdditionalDirectories,
+     defaultMode,
+     overrodeDontAsk,
+    autoMemoryDirectory,
+   };
+ }
+ 
+@@ -2064,12 +2082,18 @@ async function buildRuntime(input: {
+       // by the company root already granted in these settings; the process env
+       // still receives this run's actual AGENT_HOME.
+       agentHome: asString(workspaceContext.agentHomeForPermissions, agentHome),
+      // Auto memory follows the actual AGENT_HOME so it is saved with the run.
+      // It is not part of the session fingerprint: a moved memory directory
+      // must not invalidate a resumable conversation.
+      runAgentHome: agentHome,
+       companyId: agent.companyId,
+     });
+     skillCommandNotes.push(
+       `Wrote Paperclip-managed Claude settings to ${paperclipClaudeSettings.filePath} (defaultMode=${paperclipClaudeSettings.defaultMode}${
+         paperclipClaudeSettings.overrodeDontAsk ? "; overrode user dontAsk" : ""
+-      }, +${paperclipClaudeSettings.additionalDirectories.length} read root(s), +${paperclipClaudeSettings.allow.length} allow rule(s)).`,
+      }, +${paperclipClaudeSettings.additionalDirectories.length} read root(s), +${paperclipClaudeSettings.allow.length} allow rule(s)${
+        paperclipClaudeSettings.autoMemoryDirectory ? `, autoMemoryDirectory=${paperclipClaudeSettings.autoMemoryDirectory}` : ""
+      }).`,
+     );
+   } else if (acpxAgent === "codex") {
+     // Step 2 — codex-home.seed: the codex managed-home + skills preparation.
@@ -0,0 +1,128 @@
+diff --git a/packages/adapter-utils/src/acpx-engine/execute.test.ts b/packages/adapter-utils/src/acpx-engine/execute.test.ts
+index 112e2eb..004baf0 100644
+--- a/packages/adapter-utils/src/acpx-engine/execute.test.ts
+++ b/packages/adapter-utils/src/acpx-engine/execute.test.ts
+@@ -1857,6 +1857,45 @@ describe("shared ACPX engine runtime behavior", () => {
+     expect(changed.result.sessionParams?.configFingerprint).not.toBe(first.result.sessionParams?.configFingerprint);
+   });
+ 
+  it("keeps the per-run managed AI home out of session identity and keeps Claude transcripts", async () => {
+    const root = await makeTempRoot();
+    const stateDir = path.join(root, "state");
+    const config = { agentCommand: "node ./fake-acp.js", stateDir };
+    async function withManagedHome(home: string, claudeConfigDir = path.join(home, "provider")) {
+      return runExecutor({
+        ...config,
+        managedAiConnection: { identity: "grant-1:shared:abc" },
+        env: {
+          HOME: home,
+          XDG_CONFIG_HOME: path.join(home, "config"),
+          XDG_DATA_HOME: path.join(home, "data"),
+          CODEX_HOME: path.join(home, "provider"),
+          GROK_HOME: path.join(home, "provider"),
+          CLAUDE_CONFIG_DIR: claudeConfigDir,
+        },
+      }, { context: { taskId: "issue-1" } });
+    }
+    const fp = (run: Awaited<ReturnType<typeof runExecutor>>) => run.result.sessionParams?.configFingerprint;
+    const first = await withManagedHome(path.join(root, "paperclip-ai-a"));
+    const second = await withManagedHome(path.join(root, "paperclip-ai-b"));
+    const overridden = await withManagedHome(path.join(root, "paperclip-ai-c"), path.join(root, "custom-claude"));
+    expect(fp(first)).toBeTypeOf("string");
+    expect(fp(second)).toBe(fp(first));
+    expect(fp(overridden)).not.toBe(fp(first));
+
+    // The transcript dir of each managed home points at one dir in the state
+    // dir, so a resumed session finds its transcript after the home is deleted.
+    const transcripts = path.join(stateDir, "claude-projects");
+    expect(await fs.readlink(path.join(root, "paperclip-ai-a", "provider", "projects"))).toBe(transcripts);
+    expect(await fs.readlink(path.join(root, "paperclip-ai-b", "provider", "projects"))).toBe(transcripts);
+    await expect(fs.lstat(path.join(root, "custom-claude", "projects"))).rejects.toThrow();
+
+    // Without a managed AI connection the same paths are user config and still count.
+    const plainA = await runExecutor({ ...config, env: { HOME: path.join(root, "home-a") } }, { context: { taskId: "issue-1" } });
+    const plainB = await runExecutor({ ...config, env: { HOME: path.join(root, "home-b") } }, { context: { taskId: "issue-1" } });
+    expect(fp(plainB)).not.toBe(fp(plainA));
+  });
+
+   it("busts the session fingerprint when a stable configured PAPERCLIP_* value rotates", async () => {
+     const root = await makeTempRoot();
+     const stateDir = path.join(root, "state");
+diff --git a/packages/adapter-utils/src/acpx-engine/execute.ts b/packages/adapter-utils/src/acpx-engine/execute.ts
+index e5681b8..0fd486c 100644
+--- a/packages/adapter-utils/src/acpx-engine/execute.ts
+++ b/packages/adapter-utils/src/acpx-engine/execute.ts
+@@ -1663,6 +1663,26 @@ async function writePaperclipClaudeSettings(input: {
+   };
+ }
+ 
+// Claude keeps conversation transcripts in CLAUDE_CONFIG_DIR/projects. A
+// managed AI home is a per-run temp dir that the server deletes after the run,
+// so the next run could not resume ("Resource not found"). Point `projects` at
+// a directory in the agent's state dir, like ~/.claude/projects on a normal
+// install. The server's recursive cleanup removes the link, not its target.
+async function linkManagedClaudeTranscripts(input: {
+  claudeConfigDir: string;
+  stateDir: string;
+}): Promise<string> {
+  const target = path.join(input.stateDir, "claude-projects");
+  await fs.mkdir(input.claudeConfigDir, { recursive: true, mode: 0o700 });
+  await fs.mkdir(target, { recursive: true, mode: 0o700 });
+  const link = path.join(input.claudeConfigDir, "projects");
+  const current = await fs.readlink(link).catch(() => null);
+  if (current === target) return target;
+  await fs.rm(link, { recursive: true, force: true });
+  await fs.symlink(target, link, "dir");
+  return target;
+}
+
+ // Cross the CLI's staging seam for a runner-backed remote sandbox: ship the
+ // workspace (and, in PR 2, the per-adapter managed-home `assets`) into the
+ // sandbox and obtain the in-sandbox `workspaceRemoteDir` plus the non-null
+@@ -2027,6 +2047,30 @@ async function buildRuntime(input: {
+     // are absent from tempKeysApplied and keep their compatibility protection.
+     if (!scratchKeys.has(key) || value !== scratch.dir) resolvedAdapterEnv[key] = value;
+   }
+  // A managed AI connection gets a fresh server-made temp home on every run
+  // (HOME, XDG_*, and the provider home under it). Hash a fixed placeholder for
+  // those paths so the session can resume; the server's own fingerprint does
+  // the same (managedAiSessionFingerprintConfig). Values that differ from the
+  // managed layout are real overrides and stay in the hash. A Claude session
+  // can only resume if its transcript survives the run, so this pairs with
+  // linkManagedClaudeTranscripts below. Remote runs keep their transcript in
+  // the sandbox and are left as they are.
+  const managedHome = config.managedAiConnection && !executionTargetIsRemote ? resolvedAdapterEnv.HOME : undefined;
+  if (managedHome) {
+    const managedLayout: Record<string, string> = {
+      HOME: "",
+      XDG_CONFIG_HOME: "config",
+      XDG_DATA_HOME: "data",
+      CODEX_HOME: "provider",
+      GROK_HOME: "provider",
+      CLAUDE_CONFIG_DIR: "provider",
+    };
+    for (const [key, sub] of Object.entries(managedLayout)) {
+      if (resolvedAdapterEnv[key] === (sub ? path.join(managedHome, sub) : managedHome)) {
+        resolvedAdapterEnv[key] = sub ? `<managed-ai-home>/${sub}` : "<managed-ai-home>";
+      }
+    }
+  }
+   if (authToken) env.PAPERCLIP_API_KEY = authToken;
+   // For the claude agent, set model via ANTHROPIC_MODEL at startup rather than
+   // via session/set_config_option — the ACP server's set_config_option handler
+@@ -2088,6 +2132,15 @@ async function buildRuntime(input: {
+       runAgentHome: agentHome,
+       companyId: agent.companyId,
+     });
+    // Same condition as the fingerprint placeholder above: a resumable session
+    // must find its transcript in the next run's managed home.
+    if (resolvedAdapterEnv.CLAUDE_CONFIG_DIR === "<managed-ai-home>/provider" && env.CLAUDE_CONFIG_DIR) {
+      const transcriptsDir = await linkManagedClaudeTranscripts({
+        claudeConfigDir: env.CLAUDE_CONFIG_DIR,
+        stateDir,
+      });
+      skillCommandNotes.push(`Linked Claude transcripts to ${transcriptsDir}.`);
+    }
+     skillCommandNotes.push(
+       `Wrote Paperclip-managed Claude settings to ${paperclipClaudeSettings.filePath} (defaultMode=${paperclipClaudeSettings.defaultMode}${
+         paperclipClaudeSettings.overrodeDontAsk ? "; overrode user dontAsk" : ""
@@ -2121,6 +2121,74 @@ data:
            - url: https://dash.renner.uno/login/generic_oauth
              matching_mode: strict

+      # Matrix (REN-567): upstream login for MAS (account.matrix.renner.uno).
+      # Git owns this provider, its application and the member list.
+      # Public client + PKCE, so there is no client secret to share.
+      # RS256 with the built-in self-signed keypair: a public client has no
+      # secret for HS256, and MAS verifies the ID token signature.
+      - id: matrix-users-group
+        model: authentik_core.group
+        identifiers:
+          name: Matrix Users
+        state: present
+        attrs:
+          is_superuser: false
+          # Every member can sign in to Matrix and gets an account named
+          # after their Authentik username. Add/remove members HERE.
+          users:
+            - !Find [authentik_core.user, [username, k.renner]]
+
+      - id: matrix-mas-provider
+        model: authentik_providers_oauth2.oauth2provider
+        identifiers:
+          name: Matrix (MAS)
+        state: present
+        attrs:
+          authentication_flow: !KeyOf passkey-passwordless-flow
+          authorization_flow: !KeyOf passkey-stepup-flow
+          invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
+          client_type: public
+          client_id: matrix-mas
+          # Authentik 2026.5 checks the grant type against this list, and a
+          # provider created by a blueprint starts with it empty, so every
+          # authorize fails with invalid_request ("Invalid grant_type for
+          # provider"). Providers made in the UI get the defaults.
+          grant_types:
+            - authorization_code
+            - refresh_token
+          signing_key: !Find [authentik_crypto.certificatekeypair, [name, authentik Self-signed Certificate]]
+          property_mappings:
+            - !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-openid]]
+            - !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-profile]]
+            - !Find [authentik_providers_oauth2.scopemapping, [managed, goauthentik.io/providers/oauth2/scope-email]]
+          redirect_uris:
+            # The path ends in the provider ULID from apps/matrix/mas.yaml.
+            - url: https://account.matrix.renner.uno/upstream/callback/01M4480KWZGN9HBNB5SSNGT2M1
+              matching_mode: strict
+
+      - model: authentik_core.application
+        identifiers:
+          slug: matrix
+        state: present
+        attrs:
+          name: Matrix
+          slug: matrix
+          provider: !KeyOf matrix-mas-provider
+          meta_launch_url: https://element.renner.uno
+          meta_description: "Matrix chat (Element)"
+
+      # Only the Matrix Users group passes.
+      - model: authentik_policies.policybinding
+        identifiers:
+          target: !Find [authentik_core.application, [slug, matrix]]
+          group: !KeyOf matrix-users-group
+          order: 0
+        state: present
+        attrs:
+          enabled: true
+          negate: false
+          failure_result: false
+
      # Superseded by the single passkey-webauthn-validation stage above.
      # Their flowstagebindings are FK-CASCADEd away with them; the
      # identification stage's webauthn_stage FK is SET_NULL and is
@@ -6,6 +6,8 @@ kind: Kustomization
 # and verified. See MIGRATION-gitea-woodpecker.md.
 resources:
  - statefulset.yaml
+  # The live size of the data volume; see the file (REN-583).
+  - pvc.yaml
  - service.yaml
  - ingressroute-ssh.yaml
  - ingress.yaml
@@ -0,0 +1,39 @@
+# The live data volume of gitea-0, owned here so its size is set in git
+# (REN-583). The StatefulSet controller created this claim from the
+# volumeClaimTemplates in statefulset.yaml, but that template is immutable on
+# an existing StatefulSet, so changing the size there cannot be applied. A
+# PVC's resources.requests.storage CAN grow in place (ceph-block has
+# allowVolumeExpansion: true; ceph-csi grows the RBD image and the filesystem
+# online, no pod restart). Flux applies this over the existing claim with
+# server-side apply and the resize follows.
+#
+# - Name, namespace, accessModes, storageClassName and volumeMode must match
+#   the live claim exactly: they are immutable, a mismatch fails the whole
+#   gitea Kustomization (no change to the live claim).
+# - volumeName must stay: kustomize-controller has owned spec.volumeName
+#   since the namespace move pre-created this claim (eb23d3ac, #878).
+#   Without it, server-side apply tries to clear the field and the API server
+#   rejects the whole claim as an immutable-spec change.
+# - Grow only. A smaller value is rejected by the API server.
+# - prune: disabled, so removing this file from git never deletes the claim
+#   and its data. To hand ownership back, delete the file; the claim stays.
+#
+# Size history: 20Gi -> 40Gi 2026-07-22 (#336, live resize by hand);
+# 40Gi -> 60Gi 2026-10-03 (REN-583, after the volume filled again on REN-580).
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: data-gitea-0
+  namespace: gitea
+  labels:
+    app.kubernetes.io/name: gitea
+  annotations:
+    kustomize.toolkit.fluxcd.io/prune: disabled
+spec:
+  accessModes: ["ReadWriteOnce"]
+  storageClassName: ceph-block
+  volumeMode: Filesystem
+  volumeName: pvc-e7c56db2-481d-40cb-b94a-b264b7b4b5be
+  resources:
+    requests:
+      storage: 60Gi
@@ -147,14 +147,10 @@ spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: ceph-block
        resources:
-          # Expanded 20Gi -> 40Gi on 2026-07-22 (ceph-block online resize) to
-          # clear a 100%-full outage (#336). The live PVC was resized directly.
-          # NOTE: volumeClaimTemplates is immutable on an existing StatefulSet,
-          # so bumping this value cannot be `kubectl apply`-ed in place — Flux
-          # will reject the whole object. Landing this size change requires a
-          # one-time `kubectl delete statefulset gitea -n gitea --cascade=orphan`
-          # (pod + PVC keep running) so Flux re-creates the STS from this
-          # template and re-adopts them. After the cleanup_packages GC reclaims
-          # the ~10G of orphaned blobs, real usage is ~9G, so 40Gi is headroom.
+          # This template only applies when the claim does not exist yet. The
+          # live size of data-gitea-0 is set in pvc.yaml (REN-583), not here:
+          # volumeClaimTemplates is immutable on an existing StatefulSet, so a
+          # change to this value cannot be applied and Flux would reject the
+          # whole object. Leave it at 40Gi.
          requests:
            storage: 40Gi
@@ -61,7 +61,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: shelly-manager
-          image: gitea.renner.uno/k.renner/shelly-manager:build.1784957186 # {"$imagepolicy": "flux-system:shelly-manager"}
+          image: gitea.renner.uno/k.renner/shelly-manager:build.1791047765 # {"$imagepolicy": "flux-system:shelly-manager"}
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
@@ -8,26 +8,11 @@
 # back into `kubernetes/apps/immich/` at retirement is the cleanup.
 #
 # Sizing:
-# - library: 300Gi — LXC 225's /mnt/media/immich is currently ~250 GiB.
-#   StorageClass ceph-filesystem has allowVolumeExpansion=true; grow with
-#   `kubectl patch pvc immich-library -p '{"spec":{"resources":{"requests":{"storage":"<new>"}}}}'`.
+# - library-bulk: 300Gi — the media library is ~250 GiB.
+#   StorageClass ceph-filesystem-bulk has allowVolumeExpansion=true; grow with
+#   `kubectl patch pvc immich-library-bulk -p '{"spec":{"resources":{"requests":{"storage":"<new>"}}}}'`.
 # - ml-cache: 20Gi — face-recognition + CLIP model cache. Survives pod
 #   restarts so ML doesn't re-download on every rollout.
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: immich-library
-  namespace: immich
-  annotations:
-    kustomize.toolkit.fluxcd.io/prune: disabled
-spec:
-  accessModes:
-    - ReadWriteMany
-  storageClassName: ceph-filesystem
-  resources:
-    requests:
-      storage: 300Gi
---
 # Media root on the EC 2+1 bulk pool; thumbs mount separately below.
 apiVersion: v1
 kind: PersistentVolumeClaim
@@ -0,0 +1,45 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: mx-pg
+  namespace: matrix
+spec:
+  instances: 2
+  smartShutdownTimeout: 10
+  stopDelay: 60
+  imageName: ghcr.io/cloudnative-pg/postgresql:16
+  enableSuperuserAccess: true
+  storage:
+    storageClass: ceph-block
+    size: 5Gi
+  plugins:
+    - name: barman-cloud.cloudnative-pg.io
+      isWALArchiver: true
+      parameters:
+        barmanObjectName: mx-pg-store
+  bootstrap:
+    initdb:
+      database: synapse
+      owner: synapse
+      # Synapse refuses to start on a database whose collation is not C
+      # (its text indexes depend on byte order). Fixed at initdb time; it
+      # cannot be changed on an existing database.
+      encoding: UTF8
+      localeCollate: C
+      localeCType: C
+  postgresql:
+    parameters:
+      shared_buffers: "128MB"
+      max_connections: "50"
+      # Same write-reduction set as the other CNPG clusters; rationale in
+      # apps/trmnl/cluster.yaml.
+      wal_compression: "zstd"
+      archive_timeout: "30min"
+      checkpoint_timeout: "15min"
+      max_wal_size: "2GB"
+  resources:
+    requests:
+      cpu: 100m
+      memory: 256Mi
+    limits:
+      memory: 512Mi
@@ -0,0 +1,137 @@
+# Element Web at element.renner.uno, locked to matrix.renner.uno: no custom
+# homeserver, no guests, no registration, no integration manager, no
+# analytics (no posthog block), no bug-report endpoint.
+#
+# feature_login_with_qr: "Sign in with QR code" on the welcome page, so a new
+# browser can be linked from a signed-in device. Linking a phone from the
+# desktop (Settings → Sessions → Link new device) needs no flag; it shows
+# when the server offers MSC4108 and the session has cross-signing.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: element-config
+  namespace: matrix
+data:
+  config.json: |
+    {
+      "default_server_config": {
+        "m.homeserver": {
+          "base_url": "https://matrix.renner.uno",
+          "server_name": "matrix.renner.uno"
+        }
+      },
+      "disable_custom_urls": true,
+      "disable_guests": true,
+      "disable_3pid_login": true,
+      "default_country_code": "DE",
+      "room_directory": { "servers": ["matrix.renner.uno"] },
+      "integrations_ui_url": "",
+      "integrations_rest_url": "",
+      "integrations_widgets_urls": [],
+      "bug_report_endpoint_url": "",
+      "show_labs_settings": false,
+      "features": {
+        "feature_login_with_qr": true
+      },
+      "setting_defaults": {
+        "UIFeature.registration": false,
+        "UIFeature.passwordReset": false,
+        "UIFeature.thirdPartyId": false,
+        "UIFeature.identityServer": false,
+        "UIFeature.deactivate": false
+      }
+    }
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: element-web
+  namespace: matrix
+  annotations:
+    configmap.reloader.stakater.com/reload: element-config
+  labels:
+    app.kubernetes.io/name: element-web
+    app.kubernetes.io/part-of: matrix
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: element-web
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: element-web
+        app.kubernetes.io/part-of: matrix
+    spec:
+      automountServiceAccountToken: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 101
+        runAsGroup: 101
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: element-web
+          image: ghcr.io/element-hq/element-web:v1.12.30
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          env:
+            # Unprivileged port: the image runs as nginx, not root.
+            - name: ELEMENT_WEB_PORT
+              value: "8080"
+          ports:
+            - name: http
+              containerPort: 8080
+          resources:
+            requests:
+              cpu: 10m
+              memory: 32Mi
+            limits:
+              memory: 128Mi
+          readinessProbe:
+            httpGet:
+              path: /config.json
+              port: http
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /config.json
+              port: http
+            periodSeconds: 30
+          volumeMounts:
+            # The entrypoint copies /app/config*.json to /tmp and serves it
+            # from there, so the ConfigMap lands at the source path.
+            - name: config
+              mountPath: /app/config.json
+              subPath: config.json
+              readOnly: true
+            # Writable paths for a read-only root: the entrypoint renders the
+            # nginx template into conf.d; pid and temp files go to /tmp.
+            - name: conf-d
+              mountPath: /etc/nginx/conf.d
+            - name: tmp
+              mountPath: /tmp
+      volumes:
+        - name: config
+          configMap:
+            name: element-config
+        - name: conf-d
+          emptyDir: {}
+        - name: tmp
+          emptyDir: {}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: element-web
+  namespace: matrix
+spec:
+  selector:
+    app.kubernetes.io/name: element-web
+  ports:
+    - name: http
+      port: 80
+      targetPort: http
@@ -0,0 +1,26 @@
+# OpenBao homelab/apps/matrix (seeded by Hermes):
+#   owner_user                 localpart of the owner account (k.renner)
+#   owner_password             the owner's password (MAS password login)
+#   registration_shared_secret random; Synapse's macaroon key fallback
+#   mas_matrix_secret          random; MAS <-> Synapse shared secret
+#   mas_encryption             32 bytes hex; MAS encrypts tokens at rest
+#   mas_rsa_key, mas_ec_key    PEM private keys; MAS signs its tokens
+#
+# The owner reads the login from OpenBao; it never appears in a comment, a
+# manifest or a log. owner_password is the password the account was created
+# with; changing it in OpenBao does not change the account password.
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: synapse-secrets
+  namespace: matrix
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: synapse-secrets
+  dataFrom:
+    - extract:
+        key: apps/matrix
@@ -0,0 +1,15 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: hetzner-s3-creds
+  namespace: matrix
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: hetzner-s3-creds
+  dataFrom:
+    - extract:
+        key: infrastructure/hetzner-s3
@@ -0,0 +1,148 @@
+# LAN-only: no dns.renner.uno/public label, so external-dns-unifi registers
+# all three names on the UDM and nothing publishes them through the NetBird proxy.
+# Phones reach the server on the home network (or NetBird). Public access is
+# a separate owner decision.
+#
+# Synapse: only the client-facing paths. /_synapse/admin stays off the
+# Ingress (in-cluster only); /_matrix/federation and /_matrix/key are not
+# served by the listener anyway.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: synapse
+  namespace: matrix
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - matrix.renner.uno
+      secretName: matrix-tls
+  rules:
+    - host: matrix.renner.uno
+      http:
+        paths:
+          # Legacy login/logout/refresh go to MAS (its compat layer). Longer
+          # paths win in Traefik, so these override /_matrix/client below.
+          - path: /_matrix/client/v3/login
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
+          - path: /_matrix/client/v3/logout
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
+          - path: /_matrix/client/v3/refresh
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
+          - path: /_matrix/client/r0/login
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
+          - path: /_matrix/client/r0/logout
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
+          - path: /_matrix/client/r0/refresh
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
+          - path: /_matrix/client
+            pathType: Prefix
+            backend:
+              service:
+                name: synapse
+                port:
+                  number: 8008
+          - path: /_matrix/media
+            pathType: Prefix
+            backend:
+              service:
+                name: synapse
+                port:
+                  number: 8008
+          - path: /_synapse/client
+            pathType: Prefix
+            backend:
+              service:
+                name: synapse
+                port:
+                  number: 8008
+          - path: /.well-known/matrix/client
+            pathType: Exact
+            backend:
+              service:
+                name: synapse
+                port:
+                  number: 8008
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: element-web
+  namespace: matrix
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - element.renner.uno
+      secretName: element-tls
+  rules:
+    - host: element.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: element-web
+                port:
+                  number: 80
+---
+# MAS: login pages, OAuth endpoints and account management.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: mas
+  namespace: matrix
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - account.matrix.renner.uno
+      secretName: mas-tls
+  rules:
+    - host: account.matrix.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: mas
+                port:
+                  number: 8080
@@ -0,0 +1,14 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - externalsecret.yaml
+  - synapse.yaml
+  - mas.yaml
+  - element.yaml
+  - ingress.yaml
+  # mx-pg CNPG Postgres + offsite Barman backup. The media store and signing
+  # key are on synapse-data, backed up by Velero (matrix-daily).
+  - cluster.yaml
+  - objectstore.yaml
+  - scheduled-backup.yaml
+  - hetzner-creds-externalsecret.yaml
@@ -0,0 +1,229 @@
+# Matrix Authentication Service (MAS) for matrix.renner.uno (REN-567).
+#
+# Synapse delegates all logins to MAS (matrix_authentication_service in
+# synapse.yaml). Why: Element X only signs in with a password or through
+# MAS. MAS gives it a one-tap passkey login through Authentik
+# (upstream_oauth2 below) and makes QR sign-in possible.
+#
+# Accounts: the owner account was moved from Synapse into MAS by syn2mas
+# (the migrate-to-mas init container in synapse.yaml), with its password
+# hash and sessions. The first Authentik login links to it by localpart
+# (on_conflict: add). The Authentik application admits only the Matrix
+# Users group, so no one else can create an account through it. Password
+# login stays on as a fallback; self-registration and recovery are off.
+#
+# Secrets: OpenBao homelab/apps/matrix (synapse-secrets). The database is
+# `mas` in mx-pg (Barman backup); MAS keeps no other state.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: mas-config
+  namespace: matrix
+data:
+  config.yaml: |
+    http:
+      public_base: https://account.matrix.renner.uno/
+      listeners:
+        - name: web
+          resources:
+            - name: discovery
+            - name: human
+            - name: oauth
+            - name: compat
+            - name: graphql
+            - name: assets
+          binds:
+            - address: "[::]:8080"
+        - name: internal
+          resources:
+            - name: health
+          binds:
+            - address: "[::]:8081"
+
+    # Password from the mounted mx-pg-app Secret; nothing in this file.
+    database:
+      host: mx-pg-rw
+      port: 5432
+      username: synapse
+      password_file: /pg/password
+      database: mas
+
+    matrix:
+      kind: synapse
+      homeserver: matrix.renner.uno
+      endpoint: http://synapse:8008/
+      secret_file: /secrets/mas_matrix_secret
+
+    secrets:
+      encryption_file: /secrets/mas_encryption
+      keys:
+        - key_file: /secrets/mas_rsa_key
+        - key_file: /secrets/mas_ec_key
+
+    passwords:
+      enabled: true
+      # Version 1 must be bcrypt with no pepper: syn2mas imports Synapse's
+      # bcrypt hashes under it. New passwords are hashed with argon2id.
+      schemes:
+        - version: 1
+          algorithm: bcrypt
+        - version: 2
+          algorithm: argon2id
+      minimum_complexity: 3
+
+    account:
+      password_registration_enabled: false
+      password_recovery_enabled: false
+      email_change_allowed: false
+      login_with_email_allowed: false
+      account_deactivation_allowed: false
+
+    # No mail server; MAS sends nothing.
+    email:
+      transport: blackhole
+
+    upstream_oauth2:
+      providers:
+        - id: 01M4480KWZGN9HBNB5SSNGT2M1
+          human_name: Authentik (passkey)
+          issuer: https://auth.renner.uno/application/o/matrix/
+          client_id: matrix-mas
+          # Public client + PKCE: no shared client secret to keep in sync.
+          token_endpoint_auth_method: none
+          pkce_method: always
+          scope: "openid profile email"
+          claims_imports:
+            # Authentik username = Matrix localpart. `add` links the first
+            # login to the migrated account of the same name.
+            localpart:
+              action: require
+              template: "{{ user.preferred_username }}"
+              on_conflict: add
+            displayname:
+              action: suggest
+              template: "{{ user.name }}"
+            email:
+              action: ignore
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: mas
+  namespace: matrix
+  annotations:
+    configmap.reloader.stakater.com/reload: mas-config
+    secret.reloader.stakater.com/reload: synapse-secrets
+  labels:
+    app.kubernetes.io/name: mas
+    app.kubernetes.io/part-of: matrix
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: mas
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: mas
+        app.kubernetes.io/part-of: matrix
+    spec:
+      automountServiceAccountToken: false
+      enableServiceLinks: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 65532
+        runAsGroup: 65532
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: mas
+          image: ghcr.io/element-hq/matrix-authentication-service:1.26.0
+          # The server applies database migrations and syncs the upstream
+          # providers from the config on start.
+          args: ["server", "--config", "/config/config.yaml"]
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          ports:
+            - name: http
+              containerPort: 8080
+            - name: internal
+              containerPort: 8081
+          resources:
+            requests:
+              cpu: 20m
+              memory: 64Mi
+            limits:
+              memory: 256Mi
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: internal
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: internal
+            periodSeconds: 30
+            timeoutSeconds: 5
+            failureThreshold: 4
+          volumeMounts:
+            - name: config
+              mountPath: /config
+              readOnly: true
+            - name: secrets
+              mountPath: /secrets
+              readOnly: true
+            - name: pg
+              mountPath: /pg
+              readOnly: true
+      volumes:
+        - name: config
+          configMap:
+            name: mas-config
+        - name: secrets
+          secret:
+            secretName: synapse-secrets
+            items:
+              - key: mas_matrix_secret
+                path: mas_matrix_secret
+              - key: mas_encryption
+                path: mas_encryption
+              - key: mas_rsa_key
+                path: mas_rsa_key
+              - key: mas_ec_key
+                path: mas_ec_key
+        - name: pg
+          secret:
+            secretName: mx-pg-app
+            items:
+              - key: password
+                path: password
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: mas
+  namespace: matrix
+spec:
+  selector:
+    app.kubernetes.io/name: mas
+  ports:
+    - name: http
+      port: 8080
+      targetPort: http
+---
+# `mas` database in mx-pg, owned by the synapse role (one app, one
+# namespace), so MAS reuses mx-pg-app. Barman backs it up with synapse.
+apiVersion: postgresql.cnpg.io/v1
+kind: Database
+metadata:
+  name: mx-pg-mas
+  namespace: matrix
+spec:
+  cluster:
+    name: mx-pg
+  name: mas
+  owner: synapse
@@ -0,0 +1,30 @@
+---
+apiVersion: barmancloud.cnpg.io/v1
+kind: ObjectStore
+metadata:
+  name: mx-pg-store
+  namespace: matrix
+spec:
+  configuration:
+    # Offsite Hetzner Object Storage (fsn1), same bucket as the other CNPG
+    # clusters. No SSE: Hetzner supports only SSE-C, unusable by barman-cloud
+    # (plugin #646), so at-rest encryption is unavailable; transit is TLS.
+    destinationPath: s3://renner-uno-k8s-cnpg/mx-pg
+    endpointURL: https://fsn1.your-objectstorage.com
+    s3Credentials:
+      accessKeyId:
+        name: hetzner-s3-creds
+        key: ACCESS_KEY_ID
+      secretAccessKey:
+        name: hetzner-s3-creds
+        key: ACCESS_SECRET_KEY
+      # Region is required (REGION=fsn1) — without it boto3's HeadBucket
+      # (barman-cloud-check-wal-archive during bootstrap.recovery) 400s.
+      region:
+        name: hetzner-s3-creds
+        key: REGION
+    wal:
+      compression: gzip
+    data:
+      compression: gzip
+  retentionPolicy: "30d"
@@ -0,0 +1,16 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: ScheduledBackup
+metadata:
+  name: mx-pg-daily
+  namespace: matrix
+spec:
+  # 03:00 local, same slot as the other CNPG clusters (the Velero PVC
+  # schedule for this namespace runs later, at 09:45).
+  schedule: "0 0 3 * * *"
+  immediate: true
+  backupOwnerReference: self
+  cluster:
+    name: mx-pg
+  method: plugin
+  pluginConfiguration:
+    name: barman-cloud.cloudnative-pg.io
@@ -0,0 +1,314 @@
+# Synapse homeserver for matrix.renner.uno (REN-567).
+#
+# Private: federation is off at three layers — federation_domain_whitelist
+# is empty, no listener serves the federation/keys resources, and the
+# namespace egress policy lets Synapse reach nothing outside the cluster
+# except the push gateway. Logins and accounts are delegated to MAS
+# (mas.yaml); registration is closed.
+#
+# server_name is permanent: it is part of every user and room ID, and
+# cannot be changed without a new, empty server. matrix.renner.uno needs no
+# .well-known delegation from the apex, which is the WordPress site.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: synapse-config
+  namespace: matrix
+data:
+  homeserver.yaml: |
+    server_name: matrix.renner.uno
+    public_baseurl: https://matrix.renner.uno/
+    # Lets Element X and other clients discover the homeserver from the
+    # server name. No server well-known: there is no federation to point at.
+    serve_client_wellknown: true
+    pid_file: /tmp/homeserver.pid
+    report_stats: false
+
+    listeners:
+      # Client API only (includes media and /.well-known). Leaving out the
+      # `federation` and `keys` resources means this server does not answer
+      # federation at all.
+      - port: 8008
+        type: http
+        tls: false
+        bind_addresses: ["::"]
+        x_forwarded: true
+        resources:
+          - names: [client]
+            compress: false
+
+    # The password comes from PGPASSWORD (libpq reads it), so no credential
+    # is written into this file.
+    database:
+      name: psycopg2
+      args:
+        host: mx-pg-rw
+        port: 5432
+        dbname: synapse
+        user: synapse
+        cp_min: 2
+        cp_max: 10
+
+    # Signing key is generated on first start if missing; it lives on the
+    # data PVC next to the media store, so Velero backs up both.
+    signing_key_path: /data/signing.key
+    media_store_path: /data/media_store
+    max_upload_size: 100M
+    # Also the macaroon key: Synapse falls back to it when
+    # macaroon_secret_key is unset, so rotating it logs every session out.
+    registration_shared_secret_path: /secrets/registration_shared_secret
+
+    # MAS handles login, sessions and accounts. Synapse asks it to check
+    # every access token; password login here must be off.
+    matrix_authentication_service:
+      enabled: true
+      endpoint: http://mas:8080/
+      secret_path: /secrets/mas_matrix_secret
+    password_config:
+      enabled: false
+
+    # Sign in with QR code (Element X): the rendezvous channel between the
+    # new and the signed-in device. Both require MAS. `open` lets the new,
+    # not-yet-signed-in device create the channel; the server is LAN-only.
+    experimental_features:
+      msc4108_enabled: true
+      msc4388_mode: open
+
+    # Federation off.
+    federation_domain_whitelist: []
+    trusted_key_servers: []
+    allow_public_rooms_over_federation: false
+
+    enable_registration: false
+    allow_guest_access: false
+    url_preview_enabled: false
+    user_directory:
+      search_all_users: true
+
+    # Phones get pushes through the Element push gateway on matrix.org.
+    # Only event and room IDs leave the cluster, never message text.
+    push:
+      include_content: false
+
+    # Appservice registrations (bridges) go here, one file each, mounted
+    # from a Secret. Empty until REN-565 Part 2 is decided.
+    app_service_config_files: []
+---
+# Media store and the server signing key. Not reconstructable: lost media
+# is gone, and a lost signing key invalidates the server's past signatures.
+# Backed up by Velero (matrix-daily); the database by Barman (mx-pg).
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: synapse-data
+  namespace: matrix
+  labels:
+    app.kubernetes.io/name: synapse
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 10Gi
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: synapse
+  namespace: matrix
+  annotations:
+    configmap.reloader.stakater.com/reload: synapse-config,mas-config
+    secret.reloader.stakater.com/reload: synapse-secrets
+  labels:
+    app.kubernetes.io/name: synapse
+    app.kubernetes.io/part-of: matrix
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: synapse
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: synapse
+        app.kubernetes.io/part-of: matrix
+    spec:
+      automountServiceAccountToken: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 991
+        runAsGroup: 991
+        fsGroup: 991
+        seccompProfile:
+          type: RuntimeDefault
+      initContainers:
+        # One-time move of the Synapse accounts (owner, password hash,
+        # sessions) into the MAS database. syn2mas needs Synapse offline;
+        # this runs before Synapse starts, and Recreate stops the old pod
+        # first. The marker on the data PVC makes later starts skip it.
+        # Debug image: the plain one has no shell for the marker check.
+        - name: migrate-to-mas
+          image: ghcr.io/element-hq/matrix-authentication-service:1.26.0-debug
+          command:
+            - /busybox/sh
+            - -ec
+            - |
+              if [ -f /data/.syn2mas-done ]; then
+                echo "syn2mas already done"; exit 0
+              fi
+              mas-cli syn2mas migrate \
+                --config /mas-config/config.yaml \
+                --synapse-config /config/homeserver.yaml \
+                --synapse-database-uri postgresql:
+              touch /data/.syn2mas-done
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          env:
+            # libpq-style variables for the Synapse database; the URI above
+            # is empty on purpose.
+            - name: PGHOST
+              value: mx-pg-rw
+            - name: PGUSER
+              value: synapse
+            - name: PGDATABASE
+              value: synapse
+            - name: PGPASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: mx-pg-app
+                  key: password
+          resources:
+            requests:
+              cpu: 10m
+              memory: 64Mi
+            limits:
+              memory: 256Mi
+          volumeMounts:
+            - name: config
+              mountPath: /config
+              readOnly: true
+            - name: mas-config
+              mountPath: /mas-config
+              readOnly: true
+            - name: secrets
+              mountPath: /secrets
+              readOnly: true
+            - name: pg
+              mountPath: /pg
+              readOnly: true
+            - name: data
+              mountPath: /data
+      containers:
+        - name: synapse
+          image: ghcr.io/element-hq/synapse:v1.162.0
+          # Bypass /start.py: it expects to run as root and template a config
+          # from env. Default mode is generate-missing-and-run, which creates
+          # the signing key on first start.
+          command:
+            - python
+            - -m
+            - synapse.app.homeserver
+            - --config-path
+            - /config/homeserver.yaml
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          env:
+            - name: PGPASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: mx-pg-app
+                  key: password
+          ports:
+            - name: http
+              containerPort: 8008
+          resources:
+            requests:
+              cpu: 100m
+              memory: 256Mi
+            limits:
+              memory: 1Gi
+          # First start runs every schema migration against an empty
+          # database; give it up to 10 min.
+          startupProbe:
+            httpGet:
+              path: /health
+              port: http
+            periodSeconds: 10
+            failureThreshold: 60
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: http
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: http
+            periodSeconds: 30
+            timeoutSeconds: 5
+            failureThreshold: 4
+          volumeMounts:
+            - name: config
+              mountPath: /config
+              readOnly: true
+            - name: secrets
+              mountPath: /secrets
+              readOnly: true
+            - name: data
+              mountPath: /data
+            - name: tmp
+              mountPath: /tmp
+      volumes:
+        - name: config
+          configMap:
+            name: synapse-config
+        - name: secrets
+          secret:
+            secretName: synapse-secrets
+            items:
+              - key: registration_shared_secret
+                path: registration_shared_secret
+              - key: mas_matrix_secret
+                path: mas_matrix_secret
+              - key: mas_encryption
+                path: mas_encryption
+              - key: mas_rsa_key
+                path: mas_rsa_key
+              - key: mas_ec_key
+                path: mas_ec_key
+        - name: mas-config
+          configMap:
+            name: mas-config
+        - name: pg
+          secret:
+            secretName: mx-pg-app
+            items:
+              - key: password
+                path: password
+        - name: data
+          persistentVolumeClaim:
+            claimName: synapse-data
+        - name: tmp
+          emptyDir: {}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: synapse
+  namespace: matrix
+spec:
+  selector:
+    app.kubernetes.io/name: synapse
+  ports:
+    - name: http
+      port: 8008
+      targetPort: http
@@ -32,7 +32,7 @@
 #
 # NO TLS AND NO AUTH ON THIS SCRAPE, and that is the chart's posture rather than
 # an omission here. In 2.11.0 `metrics.listen.secure.enabled` and
-# `metrics.listen.auth.enabled` are both false (values.yaml:365-369) and
+# `metrics.listen.auth.enabled` are both false in the chart's values.yaml and
 # infrastructure/external-secrets/helmrelease.yaml overrides neither, so the
 # controller serves /metrics over plain HTTP to anything that can reach :8080.
 # Hence no `scheme`, no `tlsConfig` and no `bearerTokenSecret` below: adding
@@ -77,4 +77,7 @@ resources:
  # customResourceState block in apps/monitoring/helmrelease-prometheus.yaml —
  # these two files must land and be reverted together.
  - prometheusrule-flux.yaml
+  # Gitea's data volume. It filled twice (#336, REN-580) and the chart's
+  # KubePersistentVolumeFillingUp warned 33 minutes before the second (REN-583).
+  - prometheusrule-gitea.yaml
  - probes-blackbox.yaml
@@ -261,6 +261,32 @@ spec:
        # at 2b71dda1, and the tested one-line deletion for this target alone
        # is the second command listed there (20 -> 19 targets).
        - http://paperless.paperless.svc.cluster.local:80/accounts/login/
+        # anchor: shelly-manager-app-health
+        # APP-HEALTH probe for shelly-manager (REN-531/REN-533). Same rule as
+        # the blocks above (monitoring-vantage-point-rule): shelly.renner.uno
+        # sits behind the shelly-manager-gate outpost, so only the in-cluster
+        # Service can tell a dead app from a live gate.
+        #
+        # /health is unauthenticated and answers a plain 200 — not a 3xx —
+        # per the app's own test (shelly-manager repo,
+        # AuthenticationTests.Health_IsUnauthenticated); it is also the pod's
+        # kubelet liveness/readiness path
+        # (../home-assistant/shelly-manager-deployment.yaml). AllowedHosts is
+        # "*", so the cluster-local Host header is accepted. Service :80 ->
+        # container :8080 (../home-assistant/shelly-manager-service.yaml).
+        #
+        # NO CiliumNetworkPolicy change: the home-assistant default-deny
+        # admits the whole monitoring namespace on every port
+        # (../../infrastructure/network-policies/home-assistant-ns.yaml,
+        # "Prometheus scrape" rule). If that rule is ever narrowed to
+        # Prometheus alone, add a blackbox-exporter grant on :8080 in a
+        # DEDICATED policy selecting shelly-manager — not on the {} selector,
+        # where :8080 is shared by three apps. The EGRESS paragraph above
+        # applies too: a future monitoring egress block must list
+        # home-assistant.
+        #
+        # REVERT: delete the target line below.
+        - http://shelly-manager.home-assistant.svc.cluster.local:80/health
 # WAN-path probes of the public edge BY PROXY IP. The name-based
 # home.renner.uno probe above resolves in-cluster (CoreDNS/k8s_gateway)
 # and never leaves the LAN — the vantage-point blind spot that let the
@@ -0,0 +1,42 @@
+# Gitea's data volume (data-gitea-0) running low (REN-583).
+#
+# It filled twice: 2026-07-22 (#336) and 2026-10-03 (REN-580), both times
+# because the package registry grew faster than cleanup_packages reclaimed it.
+# A full volume fails image pushes (HTTP 500) and Gitea's own SQLite writes.
+#
+# The chart's KubePersistentVolumeFillingUp did not give usable lead time on
+# REN-580: its warning needs under 15% free AND a 4-day linear projection to
+# zero AND one hour of both, so it fired at 20:26Z; the volume was full at
+# 20:59Z. Growth here comes in steps (a run of image builds added 8 GiB in a
+# few hours that day), which a linear projection lags.
+#
+# So this is a plain floor: under 25% free (15 GiB at 60Gi) for 15m. That is
+# about two of the 8 GiB steps seen on REN-580, and time for someone to run the
+# package cleanup or grow pvc.yaml in apps/gitea.
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+  name: gitea
+  namespace: monitoring
+  labels:
+    release: kube-prometheus-stack
+spec:
+  groups:
+    - name: gitea.rules
+      rules:
+        - alert: GiteaDataVolumeLow
+          expr: >-
+            kubelet_volume_stats_available_bytes{namespace="gitea",persistentvolumeclaim="data-gitea-0"}
+              / kubelet_volume_stats_capacity_bytes{namespace="gitea",persistentvolumeclaim="data-gitea-0"}
+            < 0.25
+          for: 15m
+          labels:
+            severity: warning
+          annotations:
+            summary: "Gitea data volume has {{ $value | humanizePercentage }} free"
+            description: >-
+              data-gitea-0 is under 25% free. When it fills, image pushes fail
+              with HTTP 500 and Gitea cannot write its database. Run the
+              cleanup_packages cron in Gitea's site admin and delete old
+              package versions, or grow the claim in apps/gitea/pvc.yaml
+              (ceph-block resizes online).
@@ -31,6 +31,7 @@ retired the depth rule in favour of one persistence rule (`> 5` for `20m`).
 | `home-assistant_test.yaml` | Hand-written suite for `prometheusrule-home-assistant.yaml` (REN-245). |
 | `ceph_test.yaml` | Hand-written suite for the `CephPoolLostRedundancyBlind` canary in `prometheusrule-ceph.yaml` (REN-93). |
 | `disk-wear_test.yaml` | Hand-written suite for the EPHEMERAL exclusion in `prometheusrule-disk-wear.yaml` (REN-419). |
+| `gitea_test.yaml` | Hand-written suite for `GiteaDataVolumeLow` in `prometheusrule-gitea.yaml` (REN-583). |
 | `mutations.txt` | Negative controls. Each must make the suite fail. |

 ## Adding a suite
@@ -0,0 +1,46 @@
+# Suite for prometheusrule-gitea.yaml (REN-583). Capacity 60 GiB; free space
+# in GiB. 15 GiB free is exactly 25% and must stay silent.
+rule_files:
+  - ../prometheusrule-gitea.yaml
+
+evaluation_interval: 1m
+
+tests:
+  # 20 GiB free, then 14 GiB free from minute 10 on: pending at 10m, fires
+  # at 25m (15m later), not before.
+  - interval: 1m
+    input_series:
+      - series: 'kubelet_volume_stats_capacity_bytes{namespace="gitea",persistentvolumeclaim="data-gitea-0"}'
+        values: '64424509440+0x40'
+      - series: 'kubelet_volume_stats_available_bytes{namespace="gitea",persistentvolumeclaim="data-gitea-0"}'
+        values: '21474836480+0x9 15032385536+0x30'
+    alert_rule_test:
+      - eval_time: 24m
+        alertname: GiteaDataVolumeLow
+        exp_alerts: []
+      - eval_time: 25m
+        alertname: GiteaDataVolumeLow
+        exp_alerts:
+          - exp_labels:
+              severity: warning
+              namespace: gitea
+              persistentvolumeclaim: data-gitea-0
+            exp_annotations:
+              summary: "Gitea data volume has 23.33% free"
+              description: "data-gitea-0 is under 25% free. When it fills, image pushes fail with HTTP 500 and Gitea cannot write its database. Run the cleanup_packages cron in Gitea's site admin and delete old package versions, or grow the claim in apps/gitea/pvc.yaml (ceph-block resizes online)."
+
+  # Exactly 25% free stays silent; another claim at 1% free is not this alert.
+  - interval: 1m
+    input_series:
+      - series: 'kubelet_volume_stats_capacity_bytes{namespace="gitea",persistentvolumeclaim="data-gitea-0"}'
+        values: '64424509440+0x40'
+      - series: 'kubelet_volume_stats_available_bytes{namespace="gitea",persistentvolumeclaim="data-gitea-0"}'
+        values: '16106127360+0x40'
+      - series: 'kubelet_volume_stats_capacity_bytes{namespace="other",persistentvolumeclaim="data-other-0"}'
+        values: '64424509440+0x40'
+      - series: 'kubelet_volume_stats_available_bytes{namespace="other",persistentvolumeclaim="data-other-0"}'
+        values: '644245094+0x40'
+    alert_rule_test:
+      - eval_time: 40m
+        alertname: GiteaDataVolumeLow
+        exp_alerts: []
@@ -167,3 +167,8 @@ disk-wear-critical-device-only|kubernetes/apps/monitoring-rules/prometheusrule-d
 # exclusion matches nothing.
 # Detected by: cases A and B.
 disk-wear-ephemeral-keeps-partition|kubernetes/apps/monitoring-rules/prometheusrule-disk-wear.yaml|s/n\[0-9\]+(p\[0-9\]+)?"/n[0-9]+"/
+
+# prometheusrule-gitea.yaml (REN-583). Detected by the 24m/25m pair and the
+# exactly-25% case in gitea_test.yaml.
+gitea-for-10m|kubernetes/apps/monitoring-rules/prometheusrule-gitea.yaml|/alert: GiteaDataVolumeLow/,/for:/s/for: 15m/for: 10m/
+gitea-floor-le|kubernetes/apps/monitoring-rules/prometheusrule-gitea.yaml|s/^\([[:space:]]*\)< 0\.25$/\1<= 0.25/
@@ -10,6 +10,7 @@ resources:
  - loki-basic-auth-externalsecret.yaml
  - loki-basic-auth-middleware.yaml
  - loki-ingress.yaml
+  - loki-read-gateway.yaml
  - alertmanager-gate-service.yaml
  - alertmanager-gate-ingress.yaml
  - prometheus-gate-service.yaml
@@ -18,6 +19,13 @@ resources:
  - smartctl-exporter.yaml
  - intel-gpu-exporter.yaml
  - dashboards/
+configMapGenerator:
+  # Hash suffix kept on purpose: an edit to the .conf rolls the gateway pod,
+  # which is stateless and off the ingest path.
+  - name: loki-read-gateway
+    namespace: monitoring
+    files:
+      - loki-read-gateway.conf
 # prometheusrule-garage.yaml lives in ./monitoring-rules/ and is applied by a
 # separate Flux Kustomization that dependsOn `monitoring` — otherwise the
 # server-side dry-run of the PrometheusRule fails on a fresh cluster
@@ -0,0 +1,60 @@
+# GET-only allow-list in front of Loki (REN-621, owner decision on REN-622).
+#
+# Loki runs SingleBinary with auth_enabled: false, and its HTTP port mixes
+# reads with control endpoints that also answer GET (/ingester/shutdown,
+# /flush). An RBAC `get` on services/proxy for loki:3100 would therefore be a
+# kill switch for log ingest. This server is the read-only boundary instead:
+# the paperclip read ServiceAccount may proxy to this Service only.
+#
+# Each allowed location forwards a URI built here, never the client's raw
+# request URI, so dot-segments or encoded slashes cannot reach another Loki
+# path. Anything not listed gets 403. /loki/api/v1/tail (websocket) is left
+# out on purpose. Adding a path is a new card and a new review.
+
+upstream loki {
+    server loki.monitoring.svc.cluster.local:3100;
+}
+
+server {
+    listen 8080;
+    server_tokens off;
+
+    proxy_http_version 1.1;
+    proxy_set_header Host loki;
+    proxy_read_timeout 60s;
+
+    location / {
+        return 403;
+    }
+
+    location = /healthz {
+        access_log off;
+        default_type text/plain;
+        return 200 "ok\n";
+    }
+
+    location = /loki/api/v1/query {
+        limit_except GET { deny all; }
+        proxy_pass http://loki/loki/api/v1/query$is_args$args;
+    }
+
+    location = /loki/api/v1/query_range {
+        limit_except GET { deny all; }
+        proxy_pass http://loki/loki/api/v1/query_range$is_args$args;
+    }
+
+    location = /loki/api/v1/labels {
+        limit_except GET { deny all; }
+        proxy_pass http://loki/loki/api/v1/labels$is_args$args;
+    }
+
+    location = /loki/api/v1/series {
+        limit_except GET { deny all; }
+        proxy_pass http://loki/loki/api/v1/series$is_args$args;
+    }
+
+    location ~ ^/loki/api/v1/label/(?<label_name>[A-Za-z_][A-Za-z0-9_]*)/values$ {
+        limit_except GET { deny all; }
+        proxy_pass http://loki/loki/api/v1/label/$label_name/values$is_args$args;
+    }
+}
@@ -0,0 +1,97 @@
+---
+# Read-only Loki gateway for the agents in the paperclip pod (REN-621,
+# owner decision on REN-622). The allow-list and the reason it exists are in
+# loki-read-gateway.conf. Callers reach it through the apiserver proxy only:
+#   /api/v1/namespaces/monitoring/services/loki-read-gateway:8080/proxy/loki/api/v1/query_range?...
+# granted by ../paperclip/prometheus-read-rbac.yaml.
+#
+# Stateless and off the ingest path: Alloy pushes straight to the `loki`
+# Service, so restarting or deleting this pod never touches log ingest.
+# Revert: delete this file, the .conf and their kustomization entries, and the
+# resourceName in the paperclip Role.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: loki-read-gateway
+  namespace: monitoring
+  labels:
+    app.kubernetes.io/name: loki-read-gateway
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: loki-read-gateway
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: loki-read-gateway
+    spec:
+      automountServiceAccountToken: false
+      enableServiceLinks: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 101
+        runAsGroup: 101
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: nginx
+          image: nginxinc/nginx-unprivileged:1.27-alpine
+          ports:
+            - name: http
+              containerPort: 8080
+              protocol: TCP
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          readinessProbe:
+            httpGet:
+              path: /healthz
+              port: http
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: http
+            periodSeconds: 30
+          resources:
+            requests:
+              cpu: 10m
+              memory: 16Mi
+            limits:
+              memory: 64Mi
+          volumeMounts:
+            # Replaces the image's default.conf; the entrypoint's ipv6 and
+            # envsubst steps skip a read-only conf.d and the server starts.
+            - name: config
+              mountPath: /etc/nginx/conf.d
+              readOnly: true
+            # pid file and proxy temp paths of the unprivileged image.
+            - name: tmp
+              mountPath: /tmp
+      volumes:
+        - name: config
+          configMap:
+            name: loki-read-gateway
+        - name: tmp
+          emptyDir:
+            medium: Memory
+            sizeLimit: 32Mi
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: loki-read-gateway
+  namespace: monitoring
+  labels:
+    app.kubernetes.io/name: loki-read-gateway
+spec:
+  selector:
+    app.kubernetes.io/name: loki-read-gateway
+  ports:
+    - name: http
+      port: 8080
+      targetPort: http
+      protocol: TCP
@@ -55,7 +55,7 @@ spec:
        # stdlib only; Flux keeps the tag in lockstep via the shared image
        # policy) — no extra dependency.
        - name: reconcile-publish-ip
-          image: ghcr.io/music-assistant/server:2.10.4 # {"$imagepolicy": "flux-system:music-assistant"}
+          image: ghcr.io/music-assistant/server:2.10.5 # {"$imagepolicy": "flux-system:music-assistant"}
          command: ["/app/venv/bin/python", "/scripts/reconcile.py"]
          volumeMounts:
            - name: data
@@ -65,7 +65,7 @@ spec:
              readOnly: true
      containers:
        - name: music-assistant
-          image: ghcr.io/music-assistant/server:2.10.4 # {"$imagepolicy": "flux-system:music-assistant"}
+          image: ghcr.io/music-assistant/server:2.10.5 # {"$imagepolicy": "flux-system:music-assistant"}
          env:
            - name: LOG_LEVEL
              value: info
@@ -71,7 +71,7 @@ apiVersion: v1
 kind: Secret
 type: kubernetes.io/service-account-token
 metadata:
-  name: agent-atlas-token
+  name: agent-atlas-token-v3
  namespace: paperclip
  labels:
    app.kubernetes.io/name: paperclip
@@ -93,7 +93,7 @@ apiVersion: v1
 kind: Secret
 type: kubernetes.io/service-account-token
 metadata:
-  name: agent-hestia-token
+  name: agent-hestia-token-v3
  namespace: paperclip
  labels:
    app.kubernetes.io/name: paperclip
@@ -104,7 +104,7 @@ apiVersion: v1
 kind: Secret
 type: kubernetes.io/service-account-token
 metadata:
-  name: agent-argus-token
+  name: agent-argus-token-v3
  namespace: paperclip
  labels:
    app.kubernetes.io/name: paperclip
@@ -115,7 +115,7 @@ apiVersion: v1
 kind: Secret
 type: kubernetes.io/service-account-token
 metadata:
-  name: agent-hermes-token
+  name: agent-hermes-token-v2
  namespace: paperclip
  labels:
    app.kubernetes.io/name: paperclip
@@ -186,11 +186,11 @@ rules:
    resources: ["secrets"]
    verbs: ["get"]
    resourceNames:
-      - agent-atlas-token
+      - agent-atlas-token-v3
      - agent-iris-token
-      - agent-hestia-token
-      - agent-argus-token
-      - agent-hermes-token
+      - agent-hestia-token-v3
+      - agent-argus-token-v3
+      - agent-hermes-token-v2
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
@@ -35,9 +35,10 @@ spec:
      method: any
      number: 1
      dataDurability: preferred
+  # Primary hit the 1Gi limit: 6 OOMKills 3-4 Oct, RSS up to ~820Mi (REN-603).
  resources:
    requests:
      cpu: 200m
-      memory: 512Mi
-    limits:
      memory: 1Gi
+    limits:
+      memory: 2Gi
@@ -82,6 +82,24 @@ spec:
        - ports:
            - port: "8686"
              protocol: TCP
+    # Hubble relay for `hubble observe` (REN-487); pod port, the Service maps
+    # 80 -> 4245. Paired with network-policies/kube-system-hubble-relay.yaml.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: hubble-relay
+      toPorts:
+        - ports:
+            - port: "4245"
+              protocol: TCP
+    # edge-1 SSH as the forced-command user atlas-zwave (zwave-js restart and
+    # logs only; REN-627, access REN-628). Not a node IP, so toCIDRSet matches.
+    - toCIDRSet:
+        - cidr: 10.2.30.108/32
+      toPorts:
+        - ports:
+            - port: "22"
+              protocol: TCP
    - toEntities:
        - kube-apiserver
      toPorts:
@@ -34,7 +34,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: paperclip
-          image: gitea.renner.uno/konairius/paperclip:build.1790806287@sha256:f0b4b4e09cc02fbc472faedf3edb6d39339ade7e2932c6a435f601209b92d646 # {"$imagepolicy": "flux-system:paperclip"}
+          image: gitea.renner.uno/konairius/paperclip:build.1791063392@sha256:44887a2a95b567ea6847609662964c3056b9721c924f943770a550570cabd9b4 # {"$imagepolicy": "flux-system:paperclip"}
          ports:
            - name: http
              containerPort: 3100
@@ -24,10 +24,11 @@
 #
 # Every process in the paperclip pod can read the mounted ServiceAccount token,
 # so the token is worth the sum of everything bound to this ServiceAccount. This
-# Role's own share of that is `get` on the two Services named below and nothing
+# Role's own share of that is `get` on the Services named below and nothing
 # else. Widening it is a new card and a new review, not an edit here — the
 # Alertmanager name was added that way (REN-95, for read-only alert-delivery
-# checks; parent REN-94).
+# checks; parent REN-94). The Loki read gateway was added that way too (REN-621,
+# owner decision REN-622).
 #
 # THIS IS NO LONGER THE ONLY GRANT ON THIS ServiceAccount
 #
@@ -43,6 +44,7 @@
 # port-qualified forms are granted, so the only URLs that work are
 #   /api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy/...
 #   /api/v1/namespaces/monitoring/services/kube-prometheus-stack-alertmanager:9093/proxy/...
+#   /api/v1/namespaces/monitoring/services/loki-read-gateway:8080/proxy/loki/api/v1/...
 # A 403 on some other form is the grant working. Do not fix it by adding a
 # string.
 apiVersion: v1
@@ -79,6 +81,11 @@ rules:
    resourceNames:
      - "kube-prometheus-stack-prometheus:9090"
      - "kube-prometheus-stack-alertmanager:9093"
+      # REN-621 (owner decision REN-622): the GET-only Loki gateway, never
+      # `loki:3100` itself. Loki answers GET /ingester/shutdown and /flush,
+      # so a `get` on it would not be read-only. See
+      # ../monitoring/loki-read-gateway.conf.
+      - "loki-read-gateway:8080"
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
@@ -0,0 +1,28 @@
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: matrix
+  namespace: flux-system
+spec:
+  # Synapse homeserver + Element Web (REN-567), with the mx-pg CNPG Cluster
+  # + ObjectStore + ScheduledBackup and the owner-account Job. Own KS so a
+  # failure here never holds up the shared apps KS.
+  interval: 10m
+  retryInterval: 1m
+  timeout: 30m
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./kubernetes/apps/matrix
+  prune: true
+  wait: true
+  dependsOn:
+    - name: infrastructure
+    - name: infrastructure-crds
+    - name: infrastructure-ceph-cluster
+    - name: infrastructure-cnpg-barman
+  healthCheckExprs:
+    - apiVersion: postgresql.cnpg.io/v1
+      kind: Cluster
+      failed: "has(status.phase) && status.phase.startsWith('Failed')"
+      current: "has(status.phase) && (status.phase == 'Cluster in healthy state' || status.readyInstances > 0)"
@@ -177,6 +177,15 @@ spec:
      hostRoot: /sys/fs/cgroup
    k8sServiceHost: localhost
    k8sServicePort: 7445
+    # Hubble itself is on in every agent (chart default); the relay aggregates
+    # the three nodes' flow buffers behind one gRPC endpoint so `hubble observe`
+    # works without exec into kube-system (REN-487). Renders no change to
+    # cilium-config or the agent DaemonSet, so the reloader annotation above
+    # does not roll the CNI. The relay has no client auth; who may reach it is
+    # network-policies/kube-system-hubble-relay.yaml.
+    hubble:
+      relay:
+        enabled: true
    l2announcements:
      enabled: true
    externalIPs:
@@ -146,6 +146,12 @@ kind: Namespace
 metadata:
  name: external-secrets
 ---
+# Matrix homeserver (Synapse) + Element Web, private, no federation (REN-567).
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: matrix
+---
 # Terminus (TRMNL BYOS server) driving the entrance-way e-ink panel.
 apiVersion: v1
 kind: Namespace
@@ -176,6 +176,10 @@ spec:
        - matchLabels:
            "k8s:io.kubernetes.pod.namespace": paperclip
            app.kubernetes.io/name: paperclip
+        # Agent sandbox pods (REN-487); egress side in paperclip-renner-ns.yaml.
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip-renner
+            paperclip.io/role: agent
      toPorts:
        - ports:
            - port: "8080"
@@ -0,0 +1,36 @@
+# Hubble relay serves every node's flow log over plaintext gRPC with no client
+# authentication, so the network is its only access control. Nothing else in
+# kube-system carries a policy; this one selects the relay pod alone and has
+# no egress block, so the relay's own path to the agents (hubble-peer, :4244
+# on each node) is untouched.
+#
+# Allowed: the paperclip pod (agents run `hubble observe`, REN-487; the egress
+# half is in apps/paperclip/network-policy.yaml) and the nodes, for the
+# kubelet's gRPC health probe on :4222. Flows are metadata the agents already
+# read through cluster-wide `view`. Sandbox pods in paperclip-renner are not
+# admitted.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: hubble-relay-ingress
+  namespace: kube-system
+spec:
+  endpointSelector:
+    matchLabels:
+      k8s-app: hubble-relay
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": paperclip
+            app.kubernetes.io/name: paperclip
+      toPorts:
+        - ports:
+            - port: "4245"
+              protocol: TCP
+    - fromEntities:
+        - host
+        - remote-node
+      toPorts:
+        - ports:
+            - port: "4222"
+              protocol: TCP
@@ -27,6 +27,7 @@ resources:
  - gitea-ns.yaml
  - netbird-service-sync-ns.yaml
  - trmnl-ns.yaml
+  - matrix-ns.yaml
  - wordpress-ns.yaml
  - ai-ns.yaml
  - navidrome-ns.yaml
@@ -34,3 +35,4 @@ resources:
  - renovate-ns.yaml
  - agent-sandbox-system-ns.yaml
  - allow-cilium-health.yaml
+  - kube-system-hubble-relay.yaml
@@ -0,0 +1,139 @@
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: default-deny-ingress
+  namespace: matrix
+spec:
+  endpointSelector: {}
+  ingress:
+    # Intra-namespace — synapse ↔ mx-pg, synapse ↔ mas, mas → mx-pg, and
+    # a later bridge (appservice) ↔ synapse.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": matrix
+    # matrix.renner.uno and element.renner.uno via Traefik.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+    # Prometheus scrape.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": monitoring
+    # Velero backups via PVC mount.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": velero
+    # CNPG operator (in `database` ns) reconciles mx-pg — needs to reach
+    # the postgres pods for health, failover, role checks.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": database
+    - fromEntities:
+        - host
+        - remote-node
+    # ICMP — all standard ICMPv4 + ICMPv6 from any cluster pod. See
+    # home-assistant-ns.yaml for the rationale. INGRESS ONLY.
+    - fromEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 143, family: IPv6 }
+---
+# Egress for the app pods (synapse, mas, element-web),
+# selected by part-of so the mx-pg CNPG pods keep unrestricted egress for
+# the barman-cloud WAL archive to Hetzner (same trade as trmnl-ns.yaml).
+#
+# Nothing here reaches the internet: that is the network half of
+# "federation off". The one exception is synapse-push-egress below.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: matrix-app-egress
+  namespace: matrix
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/part-of: matrix
+  egress:
+    # Intra-namespace — mx-pg-rw (postgres) and the synapse Service.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": matrix
+    # DNS. The L7 rule is what lets Cilium learn the FQDN mapping that
+    # synapse-push-egress needs.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+          rules:
+            dns:
+              - matchPattern: "*"
+---
+# Phone push notifications. Element and Element X register the Element
+# push gateway (https://matrix.org/_matrix/push/v1/notify); Synapse POSTs
+# event and room IDs there (push.include_content is false). If a phone uses
+# a different gateway (UnifiedPush/ntfy), its pushes fail closed here.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: synapse-push-egress
+  namespace: matrix
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: synapse
+  egress:
+    - toFQDNs:
+        - matchName: "matrix.org"
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
+---
+# MAS → Authentik (auth.renner.uno) for the upstream OIDC login: discovery,
+# token exchange and JWKS. auth.renner.uno resolves to the Traefik VIP;
+# Cilium applies the DNAT before egress policy, so the rule matches the
+# Traefik pods (same path as audiobookshelf → Authentik). Traefik is the
+# only reachable target; MAS gets no internet egress.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: mas-authentik-egress
+  namespace: matrix
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: mas
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+      toPorts:
+        - ports:
+            - port: "8443"
+              protocol: TCP
@@ -40,8 +40,12 @@ spec:
 # (10.2.46.11-13:6443, Service 10.96.0.1:443 translated by KPR) runs on the
 # nodes. The `kube-apiserver` entity matches it. Port is the post-DNAT 6443.
 #
-# Selects by agent id, so only the five agents with a kubectl identity get
-# the path; other agent pods stay closed. Not labelled
+# Selects every agent pod (REN-487, fixes REN-428). An agent-id selector
+# cannot work: the plugin writes the run id into `paperclip.io/agent-id`
+# (plugin.ts, "use params.runId as stand-in for agentId"), so it never matched
+# a real agent. The TCP path is not the grant: sandbox pods mount no
+# ServiceAccount token, so only an agent with a kubeconfig (REN-412) gets past
+# anonymous, and RBAC limits what it reads. Not labelled
 # `paperclip.io/managed-by`, so the plugin does not own or delete it.
 #
 # Revert: delete this document.
@@ -52,15 +56,8 @@ metadata:
  namespace: paperclip-renner
 spec:
  endpointSelector:
-    matchExpressions:
-      - key: paperclip.io/agent-id
-        operator: In
-        values:
-          - 27d813a4-ae75-4913-b25b-3516a893d614 # Atlas
-          - 6eb4d648-83b4-40cb-b16e-d05d3ef372cb # Iris
-          - 32b3c62a-a5dc-4499-b970-25f5c7f60e8c # Hestia
-          - b023e112-1466-46f1-9fc1-e87e5cd3f005 # Argus
-          - b1be203c-d260-4922-8b48-ab9e87e73fb0 # Hermes
+    matchLabels:
+      paperclip.io/role: agent
  egress:
    - toEntities:
        - kube-apiserver
@@ -68,3 +65,72 @@ spec:
        - ports:
            - port: "6443"
              protocol: TCP
+---
+# Agent pods reach the notes wiki API (REN-487; duty lists of Zeus, Clio,
+# Hermes). Ingress side: the `wiki` policy in ai-ns.yaml. Writes are gated
+# by the wiki's own API key, not by this path.
+#
+# Revert: delete this document and the paperclip-renner entry in ai-ns.yaml.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: agent-egress-wiki
+  namespace: paperclip-renner
+spec:
+  endpointSelector:
+    matchLabels:
+      paperclip.io/role: agent
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ai
+            app.kubernetes.io/name: wiki
+      toPorts:
+        - ports:
+            - port: "8080"
+              protocol: TCP
+---
+# Public hosts on the agents' duty lists (REN-487) that the plugin's
+# paperclip-egress-fqdn list does not cover:
+#   api.github.com          Hestia: HACS release and advisory checks
+#   release-assets / objects.githubusercontent.com
+#                           Argus: promtool download (github.com 302 target)
+#   vuln.go.dev             Hephaestus: govulncheck database
+#   uptime.renner.uno       Clio: public status feed (UptimeRobot)
+# 443 only. The DNS rule repeats the plugin's so the FQDN match does not
+# depend on that plugin-owned policy. Not labelled `paperclip.io/managed-by`.
+#
+# Revert: delete this document.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: agent-egress-duty-fqdns
+  namespace: paperclip-renner
+spec:
+  endpointSelector:
+    matchLabels:
+      paperclip.io/role: agent
+  egress:
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+          rules:
+            dns:
+              - matchPattern: "*"
+    - toFQDNs:
+        - matchName: api.github.com
+        - matchName: release-assets.githubusercontent.com
+        - matchName: objects.githubusercontent.com
+        - matchName: vuln.go.dev
+        - matchName: uptime.renner.uno
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
@@ -110,7 +110,7 @@ spec:
                # then either set GITEA_REQUIRE_PUSH against a repo review-bot can
                # push, or repoint this at another private repo it collaborates on.
                - name: GITEA_VERIFY_REPO
-                  value: k.renner/shelly-manager
+                  value: konairius/shelly-manager
                - name: GITEA_SCOPES
                  value: write:repository
                # review-bot posts reviews via the API (write:repository scope)
@@ -26,6 +26,7 @@ spec:
        - immich
        - influxdb
        - ingress
+        - matrix # synapse-secrets, mx-pg barman creds (apps/matrix)
        - monitoring
        - paperclip
        - paperless
@@ -342,3 +342,29 @@ spec:
    storageLocation: default
    ttl: 720h0m0s
    snapshotMoveData: true
+---
+apiVersion: velero.io/v1
+kind: Schedule
+metadata:
+  name: matrix-daily
+  namespace: velero
+  labels:
+    app.kubernetes.io/part-of: homelab-backup
+spec:
+  # Daily at 09:45 local (next free stagger slot after navidrome-daily at
+  # 09:15). synapse-data only: the media store and the server signing key.
+  # Selected by the label this repo sets, like wordpress-daily; the mx-pg
+  # database is covered by Barman.
+  schedule: "45 9 * * *"
+  useOwnerReferencesInBackup: false
+  template:
+    includedNamespaces:
+      - matrix
+    includedResources:
+      - persistentvolumeclaims
+    storageLocation: default
+    ttl: 720h0m0s
+    snapshotMoveData: true
+    labelSelector:
+      matchLabels:
+        app.kubernetes.io/name: synapse
@@ -59,11 +59,12 @@ spec:
      # ceph-cluster-csi pace.
      #
      # itemOperationTimeout caps the async DataUpload/DataDownload
-      # itself. The immich-library upload now runs ~3h50m–4h+, so at the
-      # old 4 h cap it was intermittently Canceled at exactly 4 h (no
-      # snapshot written → PartiallyFailed backup → restore-test fails
-      # with "snapshot not found"). The volume is still growing, so both
-      # timeouts are at 8 h to leave real headroom.
+      # itself. A full immich media upload (immich-library-bulk, ~250 GiB)
+      # runs ~3h50m–4h+, so at the old 4 h cap it was intermittently
+      # Canceled at exactly 4 h (no snapshot written → PartiallyFailed
+      # backup → restore-test fails with "snapshot not found"). The volume
+      # is still growing, so both timeouts are at 8 h to leave real
+      # headroom.
      dataMoverPrepareTimeout: 8h
      defaultItemOperationTimeout: 8h
      backupStorageLocation:
@@ -23,9 +23,9 @@
 #
 # Cost: each service is restored sequentially and torn down before the
 # next, so peak storage cost is the size of the largest single service
-# (immich library ~180 GB at time of writing). Immich alone can take
-# multiple hours (millions of small files), so the full sweep is sized
-# at up to 6 h via the cronjob's activeDeadlineSeconds.
+# (immich media, immich-library-bulk, ~250 GiB at time of writing). Immich
+# alone can take multiple hours (millions of small files), so the full
+# sweep is sized at up to 6 h via the cronjob's activeDeadlineSeconds.
 #
 # Capacity gate: before each restore, the raw size of everything the run
 # writes for it (restored PVCs plus source-side probe clones) is projected
@@ -53,7 +53,7 @@ FAILURES=()
 SKIPS=()
 RUN_ID=$(date -u +%s)
 RUN_START=$RUN_ID
-RESTORE_PHASE_TIMEOUT_S=${RESTORE_PHASE_TIMEOUT_S:-14400}   # 4 h — immich library (~180 GiB, millions of small files) does not finish in 2 h
+RESTORE_PHASE_TIMEOUT_S=${RESTORE_PHASE_TIMEOUT_S:-14400}   # 4 h — immich media (immich-library-bulk, ~250 GiB, millions of small files) does not finish in 2 h
 # Must equal the Job's activeDeadlineSeconds. A restore is started only if
 # its RESTORE_PHASE_TIMEOUT_S plus CLEANUP_RESERVE_S (parity probes and
 # teardown) ends before it.