Compare commits

...
Author SHA1 Message Date
k.renner 0ec61a3e61 Merge branch 'master' into docs/paperclip-claude-token
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
2026-09-25 20:07:04 +00:00
argus-bot (Paperclip service account) 46b32d4ec3 Merge pull request 'docs(monitoring): correct the CNP reasoning, record the egress dependency (REN-21)' (#979) from docs/probe-cnp-egress-and-status-caveats into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Comment-only. No Kubernetes object changes on either file: all 3 probe documents deep-equal to master (blackbox-http 20 targets), ConfigMap/gatus-config deep-equal with data["config.yaml"] byte-identical at sha256 8886ef21...6472 (6572 bytes), so Reloader does not roll gatus. Non-comment non-blank changed lines vs merge base 0a82d61f: 0. Reviewed and verified by argus-bot at head 9142b78a; the REVERT recipe was re-tested against its own commit.
2026-09-25 20:01:41 +00:00
claude-botandClaude Opus 5.5 7b66a8f4cf docs(paperclip): Claude connection needs a setup-token value
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The local-login connection stores only the ~8h OAuth access token and
Paperclip never refreshes Anthropic credentials (subscription values
are injected read-only as CLAUDE_CODE_OAUTH_TOKEN). It expired 8h after
the login and every run failed with 'terminal access failure'.
Replacing the secret value with a claude setup-token token fixed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 21:59:31 +02:00
k.renner fd7b4f2eff Merge pull request 'fix(paperclip): raise memory limit to 32Gi (OOMKilled)' (#980) from fix/paperclip-memory into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#980
2026-09-25 19:50:42 +00:00
claude-botandClaude Opus 5.5 a412ae4fb7 fix(paperclip): memory limit 32Gi
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Headroom for many concurrent agents in the one container.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 21:49:57 +02:00
claude-botandClaude Opus 5.5 9319943f64 fix(paperclip): raise memory limit; agents run inside the container
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was canceled
The container was OOMKilled (exit 137) at 19:43 under the 4Gi limit.
With server-local runs every concurrent agent's Claude Code process
and MCP helpers live in this container, so the limit caps how many
agents can work at once; exceeding it kills the server and every
in-flight run (process_lost / orphaned_running_run).

Limit 16Gi, request 2Gi (idle server ~1Gi). Nodes have ~93Gi
allocatable each at 12-49% use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 21:49:09 +02:00
9142b78a18 docs(monitoring): fix the REVERT sentence this PR itself falsifies (REN-21)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Comment-only, one sentence, one file. Wording is Argus's from the #979
review (REN-38).

The REVERT paragraph ended with "The paperless block below DOES revert
cleanly on its own with `git revert -m 1 0a82d61f`". True at 0a82d61f,
false at bd8690eb, because this same PR edits the paperless block:

  from 0a82d61f:  revert -m 1 0a82d61f -> CLEAN     revert -m 1 16238d39 -> CONFLICT
  from bd8690eb:  revert -m 1 0a82d61f -> CONFLICT  revert -m 1 16238d39 -> CONFLICT

Reproduced here: at bd8690eb the collision is one region, `<<<<<<< HEAD`
at 155 through `>>>>>>>` at 198, the whole paperless block against an
empty side. So the advice failed in exactly the case it was written for
— someone reverting the paperless probe under pressure, four lines below
a warning not to trust the natural resolution on this file. The new text
says what actually happens and how to resolve it, and stays true as the
comment block keeps changing, because it no longer promises clean.

Bar re-verified at this commit, both files against master, js-yaml:

  probes-blackbox.yaml   docs base=3 head=3, all deep-equal=true
                         http_2xx/20, http_2xx_wan_edge/3, tcp_connect/2
  gatus/configmap.yaml   ConfigMap/gatus-config deep-equal=true
                         data["config.yaml"] sha256 8886ef21…6472 both
                         sides, identical=true, bytes=6572
  git diff -U0 vs master, non-comment non-blank lines: 0

apps/gatus/configmap.yaml is not touched by this commit at all.

Production constraint: nothing applied, restarted or re-rendered. The
public status.renner.uno page does not roll — ConfigMap data is
unchanged, so configmap.reloader.stakater.com/reload has nothing to act
on.

Revert: `git revert <this-sha>` restores the previous sentence and
nothing else.

Refs REN-21, REN-38, REN-44.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-25 19:40:06 +00:00
svc-atlasandPaperclip bd8690ebdc docs(monitoring): correct the CNP reasoning, record the egress dependency (REN-21)
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Comment-only. Every parsed object in both files is byte-identical to
master at 0a82d61f, verified with js-yaml; zero non-comment lines change.

Review follow-ups from REN-22 on the merged #976/#977 probes:

1. The CNP argument was right in its conclusion and wrong in its framing.
   cnp-cross-namespace-allow-apps-gatus-to-monitored-services DOES apply to
   blackbox-exporter — its Generalization section names it. We are safe
   because the allow it demands is already present, not because the note is
   about gatus. The old wording is the sentence that makes the next person
   skip the check for the next namespace.

2. Records the load-bearing egress fact, which was written down nowhere.
   Cilium enforces at both ends; blackbox-exporter can send only because no
   policy selecting a monitoring endpoint declares an `egress:` block.
   Verified across the repo: 73 Cilium policies, 38 with egress, none in
   monitoring, and the one CCNP is scoped to `reserved.health`. 17 other
   namespaces already have an egress block, so a routine hardening pass on
   monitoring would break these three targets and leave the *.monitoring.svc
   ones working.

3. Extends the 200-vs-3xx caveat to all three new targets, not just
   paperless. The repo evidence behind every one of them establishes only
   `< 400` (kubelet httpGet accepts 200-399; lidarr.yaml:29's `curl -f`
   without -L exits 0 on a 302). It proves the paths are ungated, not that
   they answer 200. A 3xx gets reverted, not widened.

4. Records the tested revert recipe for #976 in the file. `git revert -m 1
   16238d39` conflicts and the natural resolution silently drops paperless
   too.

5. Corrects the stale "needs a CiliumNetworkPolicy first" claim in
   apps/gatus/configmap.yaml. The correction is written in the manifest
   header, ABOVE `data:`, not in place: the stale sentence sits inside the
   `config.yaml: |` block scalar, and gatus carries
   configmap.reloader.stakater.com/reload, so editing it in place would roll
   the gatus Deployment for a comment. ConfigMap data sha256 is unchanged.

Production constraint: nothing is applied, restarted or re-rendered by this
change. No workload, namespace, policy or probe target is touched.

Revert: `git revert -m 1 <merge-sha>`, or drop either file alone with
`git checkout 0a82d61f -- <path>`.

Refs REN-21, REN-22.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-25 19:18:37 +00:00
argus-bot (Paperclip service account)andClaude Opus 5 0a82d61ff8 Merge PR #977: in-cluster app-health probe for paperless (REN-21)
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed by argus-bot (four-eyes: authored by atlas-bot, merged by argus-bot).
Rebased onto master after #976 by argus-bot; conflict markers only were removed,
the 29 added lines are byte-identical to the original commit bdf55c5b.
blackbox-http 19 -> 20 targets; other two Probe documents unchanged.
Revert: git revert -m 1 <this merge sha> is clean (tested; 20 -> 19, navidrome
and lidarr intact).
Open risk: /accounts/login/ must return 200, not 302 -- module http_2xx passes
only a final 2xx. If it is a 302, revert this PR rather than widening
valid_status_codes. Unverified at merge time: no Prometheus read path. See REN-22.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-25 18:52:17 +00:00
svc-atlasandPaperclip cb9e73bd94 feat(monitoring): in-cluster app-health probe for paperless
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
paperless.renner.uno is served entirely through the paperless-gate
Authentik outpost. gatus already records the consequence at
apps/gatus/configmap.yaml:176-185: every path including /api/ answers
302 to an unauthenticated caller, so its check of that hostname — and
the UptimeRobot monitor REN-2 adds — is reachability only and stays
green while paperless is dead behind a live gate. This adds the app
health half of the pair, from the in-cluster vantage.

The target is /accounts/login/ because paperless-ngx has no
unauthenticated health endpoint. It is the app's own established
"the webserver answers" signal — the path its startup, liveness and
readiness probes already use — and it renders a Django view rather
than a static file. It does not cover the consumer/celery workers,
and the comment says so.

No CiliumNetworkPolicy change, and no edit to the paperless namespace.
The gatus comment above asserts an in-cluster probe needs one first;
that was written for gatus, which runs in `apps`. paperless-ns.yaml:24-26
already admits the whole monitoring namespace, where blackbox-exporter
runs, and Cilium unions the allow rules of every CNP selecting an
endpoint — the same mechanism apps/paperless/network-policy.yaml:8-12
relies on.

No PrometheusRule change: the target is covered by the existing
BlackboxProbeFailing catch-all and is correctly absent from its
exclusion list, which per prometheusrule-blackbox.yaml:18-33 holds only
instances that have a dedicated rule elsewhere.

Refs REN-21.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-25 18:51:33 +00:00
argus-bot (Paperclip service account)andClaude Opus 5 16238d39ce Merge PR #976: in-cluster app-health probes for navidrome and lidarr (REN-21)
Reviewed by argus-bot (four-eyes: authored by atlas-bot, merged by argus-bot).
One file, kubernetes/apps/monitoring-rules/probes-blackbox.yaml, additive only.
blackbox-http 17 -> 19 targets; other two Probe documents unchanged.
Revert: delete the added target lines and comment block (tested clean).
Do NOT use git revert -m 1 on this merge once #977 has landed -- the natural
conflict resolution silently removes the paperless probe too. See REN-22.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-25 18:50:22 +00:00
svc-atlasandPaperclip 2219c928e0 feat(monitoring): in-cluster app-health probes for navidrome and lidarr
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
navidrome.renner.uno and lidarr.renner.uno are served entirely through
Authentik outposts. An external probe of either name therefore measures
DNS, TLS, Traefik and the gate outpost — and stays green while the app
behind a live gate is dead. This adds the second signal the
monitoring-vantage-point-rule asks for: an in-cluster probe of the
ClusterIP, which goes red on the app dying regardless of the gate.

Targets are /ping on both, the same path the pods' own kubelet
liveness/readiness probes already use. That it is ungated is not an
assumption: lidarr's reconcile script curls /ping in a loop before it
has read the API key out of config.xml (lidarr.yaml:29).

No CiliumNetworkPolicy change, and no edit to the navidrome namespace at
all. The wiki gotcha
cnp-cross-namespace-allow-apps-gatus-to-monitored-services is about
`apps`, where gatus runs; the influxdb allow-list it describes already
contained `monitoring`. navidrome-ns.yaml:25-27 likewise already admits
the whole monitoring namespace, which is where blackbox-exporter runs.

No PrometheusRule change either: both targets are covered by the
existing BlackboxProbeFailing catch-all and are correctly absent from
its exclusion list, which per prometheusrule-blackbox.yaml:18-33 holds
only instances that have a dedicated rule elsewhere.

Refs REN-21.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-25 18:25:58 +00:00
k.renner d36ffc80c2 Merge pull request 'feat(wiki): API key and network path for Paperclip agents' (#974) from feat/paperclip-wiki-key into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#974
2026-09-25 17:42:31 +00:00
k.renner 8b6e93d412 Merge pull request 'feat(lidarr): missing-album view over the Navidrome library' (#973) from feat/lidarr into master
Reviewed-on: k.renner/homelab#973
2026-09-25 17:42:06 +00:00
claude-botandClaude Opus 5.5 157b102519 feat(wiki): API key and network path for Paperclip agents
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Paperclip agents get their own wiki machine identity: OpenBao
apps/ai/wiki-api-keys field `paperclip` (already written, version 5),
mapped to Auth__ApiKeys__paperclip. Reloader rolls the wiki on the new
key.

Agents reach the service in-cluster at wiki.ai.svc.cluster.local:8080,
like the CI agents, so only the app-level API key applies (no Traefik
BasicAuth user needed). The wiki CNP admits the paperclip pod on 8080
and the paperclip egress CNP allows exactly that pod and port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 19:35:29 +02:00
claude-botandClaude Opus 5.5 378e433086 feat(lidarr): missing-album view over the Navidrome library
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Lidarr in the navidrome namespace as a "what is missing" UI: artist
discographies (Wanted -> Missing) and Spotify followed artists / saved
albums / playlists as import lists.

- Mounts navidrome-music-bulk read-write (Lidarr has no read-only root
  folder mode); config on ceph-block.
- A reconcile sidecar (same image, curl+jq) reads the API key from
  /config/config.xml and on every start sets renameTracks=false,
  writeAudioTags=no, no recycle bin, no empty-folder deletes, no
  permission changes -- then adds /music as root folder only if absent,
  so the automatic library import never runs with file-mutating
  settings.
- No download client or indexer.
- AuthenticationMethod=External behind a dedicated lidarr-gate outpost
  restricted to a new "Music Librarians" group (k.renner).
- Image automation on the linuxserver build number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 19:33:54 +02:00
14 changed files with 510 additions and 6 deletions
@@ -16,7 +16,7 @@ credentials are out of scope (see "Deferred").
|---|---|
| Execution model | **B — server-local runs** (upstream default): agent CLIs are child processes of the server, in the server container. Trial only. |
| Upgrade path | If Paperclip is kept: **C** — `@paperclipai/plugin-kubernetes` sandbox provider + kubernetes-sigs agent-sandbox controller (`agents.x-k8s.io`), per-run Pods and Secrets. Separate spec. |
| Claude auth | **One shared Claude subscription AI connection**, created once via Paperclip's local sign-in (`claude auth login` into `ai-local-logins/<id>`); Paperclip stores and refreshes it. Server-env `CLAUDE_CODE_OAUTH_TOKEN` is not used: `stripAiAuthBindings` blanks it for managed runs, and upstream accepts no pasted setup-token (the device-code flow needs a Daytona sandbox). |
| Claude auth | **One shared Claude subscription AI connection**, created once via Paperclip's local sign-in (`claude auth login` into `ai-local-logins/<id>`), then its value replaced with a long-lived `claude setup-token` token. Paperclip stores the value as company secret "My Claude subscription" and injects it per run as `CLAUDE_CODE_OAUTH_TOKEN`; it never refreshes Anthropic credentials, so the login's ~8h access token expires (runs then fail with "ACP agent reported a terminal access failure", 0 tokens). Renew by rotating that secret with a new setup-token (~yearly). Server-env `CLAUDE_CODE_OAUTH_TOKEN` is not used: `stripAiAuthBindings` blanks it for managed runs, and upstream accepts no pasted setup-token (the device-code flow needs a Daytona sandbox). |
| Bot credentials | Out of scope. Future "AIR-Manager" Paperclip role handles permission requests. |
| Exposure | LAN-only, behind an Authentik forward-auth gate, then Paperclip's own login. |
@@ -164,7 +164,8 @@ available since v0.35.0). First use of digest reflection in this repo.
**as the node user** (the entrypoint makes exec sessions root, and root-owned 0600
credentials are unreadable to the server):
`kubectl exec -it -n paperclip deploy/paperclip -- gosu node sh -c 'export CLAUDE_CONFIG_DIR=<shown path> && mkdir -p "$CLAUDE_CONFIG_DIR" && claude auth login'`
Repeat after loss of the PVC or database.
Then Settings → Secrets → "My Claude subscription": replace the value with the
output of `claude setup-token`. Repeat both after loss of the database.
## Verification
- Flux Kustomization `paperclip` Ready; CNPG `pclp-pg` healthy; first ScheduledBackup
@@ -1171,6 +1171,85 @@ data:
kubernetes_ingress_secret_name: "authentik-outpost-tls"
kubernetes_json_patches: null
# lidarr-gate: Lidarr runs with AuthenticationMethod=External, so the gate
# is its only access control.
- id: music-librarians-group
model: authentik_core.group
identifiers:
name: Music Librarians
state: present
attrs:
is_superuser: false
# Membership is owned here; blueprint apply reverts UI edits.
users:
- !Find [authentik_core.user, [username, k.renner]]
- id: lidarr-gate-provider
model: authentik_providers_proxy.proxyprovider
identifiers:
name: lidarr-gate
state: present
attrs:
authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
mode: proxy
internal_host: http://lidarr.navidrome.svc.cluster.local
internal_host_ssl_validation: false
external_host: https://lidarr.renner.uno
cookie_domain: renner.uno
access_token_validity: hours=24
refresh_token_validity: days=30
intercept_header_auth: true
- model: authentik_core.application
identifiers:
slug: lidarr-gate
state: present
attrs:
name: Lidarr
slug: lidarr-gate
provider: !KeyOf lidarr-gate-provider
meta_launch_url: https://lidarr.renner.uno
meta_description: "Music library gaps: discographies, Spotify artists"
- model: authentik_policies.policybinding
identifiers:
target: !Find [authentik_core.application, [slug, lidarr-gate]]
group: !KeyOf music-librarians-group
order: 0
state: present
attrs:
enabled: true
negate: false
failure_result: false
- model: authentik_outposts.outpost
identifiers:
name: lidarr-gate
state: present
attrs:
type: proxy
service_connection: !KeyOf k8s-conn
providers:
- !KeyOf lidarr-gate-provider
config:
log_level: info
authentik_host: "http://authentik-server.authentik.svc.cluster.local"
authentik_host_insecure: false
authentik_host_browser: "https://auth.renner.uno/"
refresh_interval: "minutes=5"
container_image: null
object_naming_template: "ak-outpost-%(name)s"
kubernetes_namespace: authentik
kubernetes_replicas: 1
kubernetes_service_type: ClusterIP
kubernetes_disabled_components: ["ingress"]
kubernetes_image_pull_secrets: []
kubernetes_ingress_annotations: {}
kubernetes_ingress_class_name: null
kubernetes_ingress_secret_name: "authentik-outpost-tls"
kubernetes_json_patches: null
- model: authentik_outposts.outpost
identifiers:
name: authentik Embedded Outpost
@@ -19,6 +19,7 @@ resources:
- llama-gate-egress.yaml
- wiki-gate-egress.yaml
- navidrome-gate-egress.yaml
- lidarr-gate-egress.yaml
- ldap-outpost-cert.yaml
- ldap-outpost-lb.yaml
- ldap-outpost-cnp.yaml
@@ -0,0 +1,20 @@
# Lets the lidarr-gate outpost reach Lidarr; the matching ingress rule is in
# network-policies/navidrome-ns.yaml.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: lidarr-gate-egress
namespace: authentik
spec:
endpointSelector:
matchLabels:
goauthentik.io/outpost-name: lidarr-gate
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": navidrome
app.kubernetes.io/name: lidarr
toPorts:
- ports:
- port: "8686"
protocol: TCP
+29
View File
@@ -30,6 +30,35 @@ metadata:
# Web Service 9 = Music Assistant
# Web Service 10 = TRMNL
# Web Service 11 = Paperless (gate-level only, see its entry)
#
# CORRECTION (REN-21, 2026-09-25) to the Web Service 11 comment further down,
# which claims that an in-cluster probe of the paperless ClusterIP "needs a
# CiliumNetworkPolicy first — the monitoring probes get away without one only
# because their targets share a namespace with the prober".
#
# That is wrong, and it was wrong when written. It is stated here rather than
# fixed in place because the sentence lives inside the `config.yaml: |` block
# scalar below: any edit to it changes ConfigMap DATA, and
# `configmap.reloader.stakater.com/reload: gatus-config`
# (gatus/deployment.yaml:13) would then roll the gatus Deployment — a
# production restart for a comment. This header is outside `data:`, so it is
# inert; the served config is byte-identical.
#
# * infrastructure/network-policies/paperless-ns.yaml:24-26 ALREADY admits
# the whole monitoring namespace on every port, and monitoring is where
# blackbox-exporter runs. Cilium unions the allow rules of every CNP
# selecting an endpoint, so no extra policy is required.
# * The in-cluster paperless probe therefore exists, added with no change
# to the paperless namespace:
# monitoring-rules/probes-blackbox.yaml, target
# http://paperless.paperless.svc.cluster.local:80/accounts/login/
# * The claim does NOT generalise to gatus itself. Gatus runs in `apps`,
# which is a different source namespace; a gatus-sourced in-cluster check
# would need its own allow, exactly as
# cnp-cross-namespace-allow-apps-gatus-to-monitored-services describes.
#
# The Web Service 11 entry below stays a reachability-only check on purpose.
# The app-health half of the pair is the blackbox probe named above.
data:
config.yaml: |
storage:
@@ -88,6 +88,113 @@ spec:
# /ready answers 200 unauthenticated and is in this namespace, so the
# same intra-namespace allowance the two above rely on covers it.
- http://loki.monitoring.svc.cluster.local:3100/ready
# APP-HEALTH probes for the two gated navidrome-namespace apps. Same
# rule as the block above (monitoring-vantage-point-rule), but these
# are the first targets here that leave the monitoring namespace.
#
# Both sit entirely behind an Authentik outpost — navidrome.renner.uno
# on ak-outpost-navidrome-gate (navidrome/ingress.yaml:21),
# lidarr.renner.uno on ak-outpost-lidarr-gate
# (navidrome/lidarr.yaml:218) — so an EXTERNAL probe of either name
# measures the gate, not the app, and stays green while the app behind
# a live gate is dead. That external signal is still wanted, and REN-2
# adds it from UptimeRobot; these are the other half of the pair.
#
# NO CiliumNetworkPolicy change is needed — but NOT because
# cnp-cross-namespace-allow-apps-gatus-to-monitored-services fails to
# apply here. It applies: its Generalization section names
# blackbox-exporter by hand. The reason we are safe is that the allow
# that note demands is ALREADY PRESENT — the navidrome baseline CNP
# admits the whole monitoring namespace on every port
# (infrastructure/network-policies/navidrome-ns.yaml:25-27), which is
# where blackbox-exporter runs. Run that check for the next namespace
# you add; do not skip it on the grounds that the note "is about
# gatus". Nothing in the navidrome namespace is edited by this change.
#
# EGRESS — the half the wiki note is silent on, and the half that is
# actually load-bearing here. Cilium enforces at BOTH ends, so the
# destination ingress allow above is worthless if the prober itself
# is in egress enforcement without a matching rule. It is not: an
# endpoint only enters egress enforcement once some policy selecting
# it declares an `egress:` block, and none does for monitoring.
# Verified at 0a82d61f across the whole repo — 73 Cilium policies, 38
# with an egress block, none of them in the monitoring namespace, and
# the single CiliumClusterwideNetworkPolicy is scoped to
# `reserved.health` so it cannot select a pod. 17 other namespaces
# here already have an egress block. If a routine hardening pass ever
# adds one to monitoring, it MUST list the navidrome and paperless
# namespaces or these three cross-namespace targets go red while the
# *.monitoring.svc ones above keep working — a failure mode nobody
# will connect back to the hardening commit.
#
# /ping on both: unauthenticated, and already the pods' own kubelet
# probe path (navidrome/deployment.yaml:61-74,
# navidrome/lidarr.yaml:114-127). For lidarr that it is ungated is
# proven independently of the gate — lidarr.yaml:29 curls /ping in a
# loop BEFORE it has read the API key out of config.xml.
#
# STATUS CODE, unproven for all three of the new targets: module
# http_2xx follows redirects and passes only on a FINAL 2xx, while
# every piece of repo evidence behind these paths establishes only
# `< 400` — a kubelet httpGet accepts 200-399, and lidarr.yaml:29's
# `curl -f` without -L exits 0 on a 302. That evidence proves the
# paths are UNGATED, not that they answer 200. Check
# probe_http_status_code on the first scrapes; a target that settles
# on a 3xx gets REVERTED, not widened to accept 3xx.
#
# REVERT (recipe tested by review at 0a82d61f): delete everything
# from the "APP-HEALTH probes for the two gated navidrome-namespace
# apps" line above through the lidarr target below. Do NOT use
# `git revert -m 1 16238d39` — it conflicts, and resolving the
# conflict the natural way also drops the paperless target below,
# silently leaving no app-health signal at all. The paperless block
# below reverts on its own with `git revert -m 1 0a82d61f`, but only
# against a tree where this comment block is absent — this commit
# edits that block, so expect one conflict covering the whole
# paperless block, and resolve it by taking the deletion.
- http://navidrome.navidrome.svc.cluster.local:80/ping
- http://lidarr.navidrome.svc.cluster.local:80/ping
# APP-HEALTH probe for paperless. Same rule as the block above
# (monitoring-vantage-point-rule); the blind spot it closes is the one
# gatus already documents at apps/gatus/configmap.yaml:176-185 —
# through the paperless-gate outpost EVERY path, /api/ included,
# answers 302 to an unauthenticated caller, so no external probe of
# paperless.renner.uno can tell a live app from a live gate. That
# external signal is still wanted, and REN-2 adds it from UptimeRobot;
# this is the other half of the pair.
#
# That gatus comment says the in-cluster probe "needs a
# CiliumNetworkPolicy first". That sentence is simply WRONG, not
# merely inapplicable: paperless-ns.yaml:24-26 already admits the
# whole monitoring namespace, which is where blackbox-exporter runs,
# and Cilium unions the allow rules of every CNP selecting an
# endpoint (the mechanism apps/paperless/network-policy.yaml:8-12
# relies on). The EGRESS paragraph in the block above applies here
# identically. Nothing in the paperless namespace is edited by this
# change.
#
# The stale sentence is deliberately left in place in
# apps/gatus/configmap.yaml: it sits INSIDE the `config.yaml: |`
# block scalar, so editing it changes ConfigMap data and
# `configmap.reloader.stakater.com/reload` (gatus/deployment.yaml:13)
# rolls the gatus Deployment. The correction is written at the top of
# that file instead, outside `data:`, where it costs no restart.
#
# /accounts/login/ because paperless-ngx has no unauthenticated health
# endpoint at all. It is the app's own established "the webserver
# answers" signal — the same path its startup, liveness and readiness
# probes use (paperless.yaml:252-272) — and it renders a Django view
# rather than a static file. It does NOT cover the consumer/celery
# workers: see process-liveness-vs-protocol-liveness.
#
# The cluster-local Host header is accepted only because
# PAPERLESS_ALLOWED_HOSTS is "*" (paperless.yaml:27). Narrow that and
# Django answers 400 here and this probe goes permanently red.
#
# This is the target most likely to fail the STATUS CODE check in the
# block above — a Django login view can answer 302 — so check it
# first, and revert rather than widen if it does.
- http://paperless.paperless.svc.cluster.local:80/accounts/login/
# WAN-path probes of the public edge BY PROXY IP. The name-based
# home.renner.uno probe above resolves in-cluster (CoreDNS/k8s_gateway)
# and never leaves the LAN — the vantage-point blind spot that let the
@@ -5,3 +5,4 @@ resources:
- deployment.yaml
- gate-service.yaml
- ingress.yaml
- lidarr.yaml
+224
View File
@@ -0,0 +1,224 @@
# Lidarr as a read-mostly "what is missing" view over the Navidrome library.
# No download client or indexer is configured.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: lidarr-config
namespace: navidrome
annotations:
kustomize.toolkit.fluxcd.io/prune: disabled
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 5Gi
---
# Applied on every pod start, before the root folder exists: no renames, no tag
# writes, no deletes. Adding /music as root folder starts Lidarr's library import.
apiVersion: v1
kind: ConfigMap
metadata:
name: lidarr-reconcile
namespace: navidrome
data:
reconcile.sh: |
set -eu
API=http://localhost:8686/api/v1
until curl -sf http://localhost:8686/ping >/dev/null; do sleep 5; done
KEY=$(sed -n 's:.*<ApiKey>\(.*\)</ApiKey>.*:\1:p' /config/config.xml)
if [ -z "$KEY" ]; then
echo "FATAL: no ApiKey in /config/config.xml" >&2
exit 1
fi
H="X-Api-Key: $KEY"
put() {
cur=$(curl -sf -H "$H" "$API/$1")
id=$(printf '%s' "$cur" | jq -r '.id')
printf '%s' "$cur" | jq -c "$2" \
| curl -sf -X PUT -H "$H" -H 'Content-Type: application/json' -d @- "$API/$1/$id" >/dev/null
echo "reconciled $1"
}
put config/naming '.renameTracks = false'
put config/metadataprovider '.writeAudioTags = "no" | .scrubAudioTags = false | .embedCoverArt = false'
put config/mediamanagement '.deleteEmptyFolders = false | .createEmptyArtistFolders = false
| .recycleBin = "" | .setPermissionsLinux = false | .importExtraFiles = false
| .skipFreeSpaceCheckWhenImporting = true'
roots=$(curl -sf -H "$H" "$API/rootfolder" | jq '[.[] | select(.path == "/music")] | length')
if [ "$roots" = "0" ]; then
qp=$(curl -sf -H "$H" "$API/qualityprofile" | jq '.[0].id')
mp=$(curl -sf -H "$H" "$API/metadataprofile" | jq '([.[] | select(.name == "Standard")][0].id) // .[0].id')
jq -nc --argjson qp "$qp" --argjson mp "$mp" '{
name: "Music", path: "/music",
defaultQualityProfileId: $qp, defaultMetadataProfileId: $mp,
defaultMonitorOption: "all", defaultNewItemMonitorOption: "all",
defaultTags: []
}' \
| curl -sf -X POST -H "$H" -H 'Content-Type: application/json' -d @- "$API/rootfolder" >/dev/null
echo "added root folder /music"
fi
echo "reconcile done"
while true; do sleep 3600; done
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: lidarr
namespace: navidrome
labels:
app.kubernetes.io/name: lidarr
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: lidarr
template:
metadata:
labels:
app.kubernetes.io/name: lidarr
spec:
enableServiceLinks: false
securityContext:
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: lidarr
image: lscr.io/linuxserver/lidarr:3.1.0.4875-ls42 # {"$imagepolicy": "flux-system:lidarr"}
env:
- name: PUID
value: "1000"
- name: PGID
value: "1000"
- name: TZ
value: Europe/Berlin
# Identity is enforced by the lidarr-gate outpost.
- name: LIDARR__AUTH__METHOD
value: External
ports:
- name: http
containerPort: 8686
volumeMounts:
- name: config
mountPath: /config
- name: music
mountPath: /music
livenessProbe:
httpGet:
path: /ping
port: http
initialDelaySeconds: 60
periodSeconds: 30
timeoutSeconds: 10
readinessProbe:
httpGet:
path: /ping
port: http
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 5
resources:
requests:
cpu: 100m
memory: 512Mi
limits:
memory: 4Gi
securityContext:
runAsNonRoot: false
allowPrivilegeEscalation: true
capabilities:
drop: ["ALL"]
add: ["CHOWN", "DAC_OVERRIDE", "FOWNER", "SETGID", "SETUID"]
- name: reconcile
image: lscr.io/linuxserver/lidarr:3.1.0.4875-ls42 # {"$imagepolicy": "flux-system:lidarr"}
command: ["sh", "/scripts/reconcile.sh"]
volumeMounts:
- name: config
mountPath: /config
readOnly: true
- name: reconcile
mountPath: /scripts
readOnly: true
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
memory: 64Mi
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
volumes:
- name: config
persistentVolumeClaim:
claimName: lidarr-config
- name: music
persistentVolumeClaim:
claimName: navidrome-music-bulk
- name: reconcile
configMap:
name: lidarr-reconcile
---
apiVersion: v1
kind: Service
metadata:
name: lidarr
namespace: navidrome
spec:
selector:
app.kubernetes.io/name: lidarr
ports:
- name: http
port: 80
targetPort: http
---
apiVersion: v1
kind: Service
metadata:
name: ak-outpost-lidarr-gate
namespace: navidrome
spec:
type: ExternalName
externalName: ak-outpost-lidarr-gate.authentik.svc.cluster.local
ports:
- port: 9000
targetPort: 9000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: lidarr
namespace: navidrome
labels:
dns.renner.uno/public: "true"
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
spec:
ingressClassName: traefik
rules:
- host: lidarr.renner.uno
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: ak-outpost-lidarr-gate
port:
number: 9000
tls:
- secretName: lidarr-tls
hosts:
- lidarr.renner.uno
@@ -55,6 +55,15 @@ spec:
- ports:
- port: "8443"
protocol: TCP
# Notes wiki API, authenticated by the paperclip API key.
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ai
app.kubernetes.io/name: wiki
toPorts:
- ports:
- port: "8080"
protocol: TCP
- toEntities:
- kube-apiserver
toPorts:
+3 -2
View File
@@ -90,12 +90,13 @@ spec:
value: localhost
periodSeconds: 10
timeoutSeconds: 5
# Every concurrent agent (Claude Code + MCP helpers) runs inside this container.
resources:
requests:
cpu: 250m
memory: 1Gi
memory: 2Gi
limits:
memory: 4Gi
memory: 32Gi
# Entrypoint remaps the node user and chowns /paperclip as root, then drops to it via gosu.
securityContext:
runAsNonRoot: false
@@ -9,6 +9,7 @@ resources:
- gotenberg.yaml
- home-assistant.yaml
- immich.yaml
- lidarr.yaml
- matter-server.yaml
- mmwave-configurator.yaml
- music-assistant.yaml
@@ -0,0 +1,24 @@
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
name: lidarr
namespace: flux-system
spec:
image: lscr.io/linuxserver/lidarr
interval: 1h
---
# Tags are <major>.<minor>.<patch>.<build>-ls<n>; the build number is monotonic.
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
name: lidarr
namespace: flux-system
spec:
imageRepositoryRef:
name: lidarr
filterTags:
pattern: '^\d+\.\d+\.\d+\.(?P<build>\d+)-ls\d+$'
extract: '$build'
policy:
numerical:
order: asc
@@ -150,8 +150,8 @@ spec:
protocol: TCP
---
# wiki: the wiki-gate outpost (humans, after Authentik), Traefik (the machine
# door wiki-index.renner.uno, BasicAuth with X-Authentik-Username stripped) and
# Gitea (push webhook) may reach it. Gitea and wiki-embed are reached through
# door wiki-index.renner.uno, BasicAuth with X-Authentik-Username stripped),
# Gitea (push webhook) and Paperclip agents (API key) may reach it. Gitea and wiki-embed are reached through
# the namespace egress above; no OpenBao access.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
@@ -173,6 +173,9 @@ spec:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": gitea
app.kubernetes.io/name: gitea
- matchLabels:
"k8s:io.kubernetes.pod.namespace": paperclip
app.kubernetes.io/name: paperclip
toPorts:
- ports:
- port: "8080"
@@ -18,6 +18,10 @@ spec:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": authentik
goauthentik.io/outpost-name: navidrome-gate
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": authentik
goauthentik.io/outpost-name: lidarr-gate
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": monitoring