feat(paperclip): deploy Paperclip (server-local runs, trial) #966

Merged
k.renner merged 10 commits from feat/paperclip into master 2026-09-25 11:24:24 +00:00
Member

Deploys Paperclip as a trial, following the spec in docs/superpowers/specs/2026-09-25-paperclip-design.md (plan: docs/superpowers/plans/2026-09-25-paperclip.md).

What

  • Namespace paperclip, baseline default-deny CNP (world egress :443 only), OpenBao ClusterSecretStore allowlist.
  • CNPG pclp-pg (PG17, 2 instances) with barman backups to Hetzner, nightly ScheduledBackup.
  • Single-replica server (Recreate) on a 20Gi ceph-block PVC; secrets from OpenBao apps/paperclip.
  • LAN-only paperclip.renner.uno behind a new Authentik paperclip-gate outpost (group Paperclip Users, passkey flows); Paperclip's own login stays behind it.
  • Flux image automation on nightly by digest (digestReflectionPolicy: Always, first use in this repo).

Trade-offs (accepted for the trial, option B)

  • Agents run as child processes of the server: they share its env and filesystem, so per-bot credential isolation is not real until option C (kubernetes sandbox provider).
  • Sign-up stays enabled until the admin is claimed (upstream claim needs a signed-in account); follow-up PR disables it.
  • Pods in monitoring/database can reach :3100 without the gate (house baseline); Paperclip login still applies.

Before merge

  • OpenBao homelab/apps/paperclip: BETTER_AUTH_SECRET, PAPERCLIP_TOOL_ACTION_SIGNING_SECRET, PAPERCLIP_SECRETS_MASTER_KEY
  • Owner adds CLAUDE_CODE_OAUTH_TOKEN via bao kv patch (server starts without it; agent runs fail until present)

After merge

Claim the admin at https://paperclip.renner.uno, run a trivial claude-local test agent, then the sign-up follow-up PR.

🤖 Generated with Claude Code

Deploys [Paperclip](https://github.com/paperclipai/paperclip) as a trial, following the spec in `docs/superpowers/specs/2026-09-25-paperclip-design.md` (plan: `docs/superpowers/plans/2026-09-25-paperclip.md`). ## What - Namespace `paperclip`, baseline default-deny CNP (world egress :443 only), OpenBao ClusterSecretStore allowlist. - CNPG `pclp-pg` (PG17, 2 instances) with barman backups to Hetzner, nightly ScheduledBackup. - Single-replica server (`Recreate`) on a 20Gi ceph-block PVC; secrets from OpenBao `apps/paperclip`. - LAN-only `paperclip.renner.uno` behind a new Authentik `paperclip-gate` outpost (group **Paperclip Users**, passkey flows); Paperclip's own login stays behind it. - Flux image automation on `nightly` by digest (`digestReflectionPolicy: Always`, first use in this repo). ## Trade-offs (accepted for the trial, option B) - Agents run as child processes of the server: they share its env and filesystem, so per-bot credential isolation is not real until option C (kubernetes sandbox provider). - Sign-up stays enabled until the admin is claimed (upstream claim needs a signed-in account); follow-up PR disables it. - Pods in `monitoring`/`database` can reach :3100 without the gate (house baseline); Paperclip login still applies. ## Before merge - [x] OpenBao `homelab/apps/paperclip`: BETTER_AUTH_SECRET, PAPERCLIP_TOOL_ACTION_SIGNING_SECRET, PAPERCLIP_SECRETS_MASTER_KEY - [ ] Owner adds `CLAUDE_CODE_OAUTH_TOKEN` via `bao kv patch` (server starts without it; agent runs fail until present) ## After merge Claim the admin at https://paperclip.renner.uno, run a trivial claude-local test agent, then the sign-up follow-up PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
claude-bot added 10 commits 2026-09-25 11:23:10 +00:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
digestReflectionPolicy: Always requires spec.interval on the ImagePolicy
(admission rejects it otherwise); set to 1h to match the ImageRepository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Sign-up stays enabled: upstream's first-admin claim requires an
  existing signed-in Better Auth account (routes/access.ts
  /bootstrap/claim), so disabling it would lock out the claim. A
  follow-up flips PAPERCLIP_AUTH_DISABLE_SIGN_UP after the claim.
- Telemetry is off via PAPERCLIP_TELEMETRY_DISABLED=1, which
  resolveTelemetryConfig honours; config.json can't carry it alone
  because the schema requires a complete file.
- Container starts as root: docker-entrypoint.sh remaps the node user
  and chowns /paperclip before dropping privileges with gosu.
- Image pinned to the amd64 manifest digest of nightly; Flux digest
  reflection takes over from there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Paperclip's private-hostname guard (authenticated+private) returns 403
  for any Host but loopback and the public URL. Kubelet probes send the
  pod IP, so the pod could never become Ready. Probes now send
  Host: localhost.
- Liveness is a TCP check: /api/health runs SELECT 1, so a Postgres
  outage would have restarted the server and killed in-flight runs.
- Baseline CNP allowed world on every port; Cilium unions allows across
  CNPs, so the app-level :443 restriction was not enforced. world is
  now :443-only in the baseline as well.
- automountServiceAccountToken: false. Every agent shares the pod
  filesystem; bots bring their own cluster credentials.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs(paperclip): align spec with implementation and review fixes
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
50cf3e977a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
k.renner merged commit 2c25acbfdd into master 2026-09-25 11:24:24 +00:00
review-bot left a comment
Member

🤖 Automated review — model CLI default (complexity: complex), commit 50cf3e977a1b2a58f65ac5611f9c388ddae2905c.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.


Summary

This PR deploys Paperclip (agent-orchestration server) into the homelab cluster: a new paperclip namespace with baseline default-deny CNP, a CNPG pclp-pg Postgres cluster with barman backups to Hetzner, a single-replica Deployment on a ceph-block PVC, an Authentik paperclip-gate outpost gating LAN-only ingress, and Flux image automation tracking the nightly tag by digest. I cross-checked every new/modified manifest against the closest existing analogs in the repo (paperless, wiki, other -gate-egress apps) — namespace, CNP, ExternalSecret, CNPG Cluster/ObjectStore/ScheduledBackup, ImagePolicy, and Authentik blueprint patterns all match established conventions closely (secret key names, dataFrom: extract, CNPG default secret field uri, cron-with-seconds schedule format, digestReflectionPolicy/interval pairing required by the installed image-reflector-controller v1.2.5 CRD, sorted OpenBao ClusterSecretStore allowlist). I did not find a concrete blocking defect.

🟡 Suggestions

  • kubernetes/apps/paperclip/paperclip.yaml (startup/readiness probes, lines ~1623-1647): The probes send Host: localhost to work around a claimed "private-hostname guard" in upstream Paperclip that 403s requests carrying the pod IP as Host. I can't verify from this checkout that upstream's guard actually accepts the literal string localhost (vs., say, 127.0.0.1 only) — that logic lives in the paperclipai/paperclip repo, not here. If the guard doesn't recognize localhost, readinessProbe/startupProbe will keep 403'ing, startupProbe will exhaust failureThreshold: 60 (~10 min) and the container will restart-loop without ever going Ready, while livenessProbe (TCP-only) keeps it alive — i.e. a silent-looking crash loop. This is exactly what Task 9 Step 2 of the plan doc is designed to catch (kubectl get pods -n paperclip … Expected: 1/1 Running), so it should surface immediately post-merge — just flagging it as the one part of the probe design I couldn't independently confirm.
  • docs/superpowers/plans/2026-09-25-paperclip.md Task 6 Step 2 (lines ~470-605): This template still shows a disable-telemetry initContainer and an httpGet livenessProbe, but the actually-committed kubernetes/apps/paperclip/paperclip.yaml dropped the initContainer in favor of PAPERCLIP_TELEMETRY_DISABLED=1 and switched liveness to tcpSocket. The companion spec doc (docs/superpowers/specs/2026-09-25-paperclip-design.md) was already updated to match (per the "align spec with implementation" commit), but the plan doc wasn't — harmless since it's a historical working checklist, not a living reference, but worth a note if anyone treats it as current.

🟢 Nits

  • kubernetes/infrastructure/image-automation/kustomization.yaml and kubernetes/infrastructure/network-policies/kustomization.yaml insert paperclip.yaml/paperclip-ns.yaml immediately after the paperless* entries, but paperclip < paperless alphabetically. Both lists are already only loosely ordered elsewhere in the file, so this doesn't break a convention that's actually enforced — the one list that does say "keep this list sorted" (openbao-clustersecretstore.yaml) got the insertion right.

Tests

No automated tests apply — this is a GitOps manifest change validated by Flux/kubectl dry-run and live reconciliation, which the PR's own plan document (Task 9) already scripts: flux get kustomizations paperclip, CNPG cluster health, gate-enforcement curl (expect 302, never 200), a CNP-bypass probe from a policy-free namespace (expect timeout), and first-backup completion. That's an appropriate verification plan for this kind of change; I'd treat the probe-behavior question above as the one item worth watching during that live verification.

Recommendation

Looks reasonable — no blocking issues; comments above are optional.

🤖 **Automated review** — model `CLI default` (complexity: complex), commit `50cf3e977a1b2a58f65ac5611f9c388ddae2905c`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary This PR deploys Paperclip (agent-orchestration server) into the homelab cluster: a new `paperclip` namespace with baseline default-deny CNP, a CNPG `pclp-pg` Postgres cluster with barman backups to Hetzner, a single-replica Deployment on a ceph-block PVC, an Authentik `paperclip-gate` outpost gating LAN-only ingress, and Flux image automation tracking the `nightly` tag by digest. I cross-checked every new/modified manifest against the closest existing analogs in the repo (`paperless`, `wiki`, other `-gate-egress` apps) — namespace, CNP, ExternalSecret, CNPG Cluster/ObjectStore/ScheduledBackup, ImagePolicy, and Authentik blueprint patterns all match established conventions closely (secret key names, `dataFrom: extract`, CNPG default secret field `uri`, cron-with-seconds schedule format, `digestReflectionPolicy`/`interval` pairing required by the installed image-reflector-controller v1.2.5 CRD, sorted OpenBao ClusterSecretStore allowlist). I did not find a concrete blocking defect. ## 🟡 Suggestions - **`kubernetes/apps/paperclip/paperclip.yaml` (startup/readiness probes, lines ~1623-1647):** The probes send `Host: localhost` to work around a claimed "private-hostname guard" in upstream Paperclip that 403s requests carrying the pod IP as Host. I can't verify from this checkout that upstream's guard actually accepts the literal string `localhost` (vs., say, `127.0.0.1` only) — that logic lives in the `paperclipai/paperclip` repo, not here. If the guard doesn't recognize `localhost`, `readinessProbe`/`startupProbe` will keep 403'ing, `startupProbe` will exhaust `failureThreshold: 60` (~10 min) and the container will restart-loop without ever going Ready, while `livenessProbe` (TCP-only) keeps it alive — i.e. a silent-looking crash loop. This is exactly what Task 9 Step 2 of the plan doc is designed to catch (`kubectl get pods -n paperclip … Expected: 1/1 Running`), so it should surface immediately post-merge — just flagging it as the one part of the probe design I couldn't independently confirm. - **`docs/superpowers/plans/2026-09-25-paperclip.md` Task 6 Step 2 (lines ~470-605):** This template still shows a `disable-telemetry` initContainer and an `httpGet` livenessProbe, but the actually-committed `kubernetes/apps/paperclip/paperclip.yaml` dropped the initContainer in favor of `PAPERCLIP_TELEMETRY_DISABLED=1` and switched liveness to `tcpSocket`. The companion spec doc (`docs/superpowers/specs/2026-09-25-paperclip-design.md`) was already updated to match (per the "align spec with implementation" commit), but the plan doc wasn't — harmless since it's a historical working checklist, not a living reference, but worth a note if anyone treats it as current. ## 🟢 Nits - `kubernetes/infrastructure/image-automation/kustomization.yaml` and `kubernetes/infrastructure/network-policies/kustomization.yaml` insert `paperclip.yaml`/`paperclip-ns.yaml` immediately after the `paperless*` entries, but `paperclip` < `paperless` alphabetically. Both lists are already only loosely ordered elsewhere in the file, so this doesn't break a convention that's actually enforced — the one list that does say "keep this list sorted" (`openbao-clustersecretstore.yaml`) got the insertion right. ## Tests No automated tests apply — this is a GitOps manifest change validated by Flux/kubectl dry-run and live reconciliation, which the PR's own plan document (Task 9) already scripts: `flux get kustomizations paperclip`, CNPG cluster health, gate-enforcement curl (expect 302, never 200), a CNP-bypass probe from a policy-free namespace (expect timeout), and first-backup completion. That's an appropriate verification plan for this kind of change; I'd treat the probe-behavior question above as the one item worth watching during that live verification. ## Recommendation Looks reasonable — no blocking issues; comments above are optional. <!-- pr-review:sha=50cf3e977a1b2a58f65ac5611f9c388ddae2905c -->
Sign in to join this conversation.