fix(paperless): probe a real Django view instead of a TCP connect #703

Merged
k.renner merged 1 commits from paperless-http-probes into master 2026-08-28 11:03:00 +00:00
Member

Nach der Arbeitsteilung — gatus schaut von außen, Kubernetes überwacht die Pods — war Paperless auf der Kubernetes-Seite die schwächste Stelle: beide Probes waren tcpSocket. Das beweist nur, dass gunicorn eine Verbindung angenommen hat. Ein Worker-Deadlock oder eine kaputte Django-Konfiguration ließen den Pod weiterhin Ready.

Der bestehende Kommentar hatte recht — aber enger, als er annahm

Er begründete die TCP-Wahl damit, dass HTTP-Probes gegen /api/ den paperless.auth-Logger fluten: mit ENABLE_HTTP_REMOTE_USER loggt die Remote-User-Middleware jede unauthentifizierte API-Anfrage. Das war eine echte Einschränkung und ein bewusster Tausch, kein Versäumnis.

Die Middleware deckt aber nur die API-Oberfläche ab. /accounts/login/ ist eine normale Django-View. Gegen den laufenden Pod gemessen:

Pfad 4 Anfragen → neue paperless.auth-Zeilen
/accounts/login/ 0
/api/ +4

Der Lärm ist also vermeidbar, ohne auf eine HTTP-Probe zu verzichten.

Was die neue Probe beweist — und was nicht

/accounts/login/ rendert ein Template, also müssen URL-Routing, Settings und die Template-Engine funktionieren. Postgres beweist sie nicht — nichts Billiges auf dieser Oberfläche tut das, und der Kommentar sagt das jetzt, statt mehr zu suggerieren.

/healthz habe ich verworfen: es liefert dieselbe HTML wie die Login-Seite, ist also nur der SPA-Fallback und kein echter Health-Endpunkt.

Die Falle, die das sonst in einen CrashLoop geführt hätte

Das kubelet sendet die Pod-IP als Host-Header. Bei restriktivem ALLOWED_HOSTS antwortet Django 400, die Probe scheitert, der Pod crasht. PAPERLESS_ALLOWED_HOSTS steht auf "*" — heute also sicher. Der Kommentar hält fest, dass ein Einschränken dieses Werts den Probes einen expliziten Host-Header mitgeben müsste.

Timings unverändert: die 120 s Liveness-Verzögerung existiert für Paperless' langsamen Start.

Zur Validierung

Geprüft mit Flux' eigenem Field-Manager (--field-manager=kustomize-controller): serverside-applied, sauber.

Mit jedem anderen Manager meldet der Dry-Run das Deployment als invalid („may not specify more than 1 handler type"), weil tcpSocket weiterhin kustomize-controller gehört und von einem fremden Apply nicht geräumt wird. Das ist ein Artefakt der Prüfung, nicht des Manifests — mir wäre es beinahe als echter Fehler durchgegangen.

Danach

Der Rollout ersetzt den Pod. Ich prüfe, dass er Ready wird und dass paperless.auth danach nicht anfängt zu wachsen — ein laufender Pod allein beweist nicht, dass die Probe geräuschlos ist.

Nach der Arbeitsteilung — gatus schaut von außen, Kubernetes überwacht die Pods — war Paperless auf der Kubernetes-Seite die schwächste Stelle: beide Probes waren `tcpSocket`. Das beweist nur, dass gunicorn eine Verbindung angenommen hat. Ein Worker-Deadlock oder eine kaputte Django-Konfiguration ließen den Pod weiterhin `Ready`. ## Der bestehende Kommentar hatte recht — aber enger, als er annahm Er begründete die TCP-Wahl damit, dass HTTP-Probes gegen `/api/` den `paperless.auth`-Logger fluten: mit `ENABLE_HTTP_REMOTE_USER` loggt die Remote-User-Middleware **jede** unauthentifizierte API-Anfrage. Das war eine echte Einschränkung und ein bewusster Tausch, kein Versäumnis. Die Middleware deckt aber nur die **API-Oberfläche** ab. `/accounts/login/` ist eine normale Django-View. Gegen den laufenden Pod gemessen: | Pfad | 4 Anfragen → neue `paperless.auth`-Zeilen | |---|---| | `/accounts/login/` | **0** | | `/api/` | **+4** | Der Lärm ist also vermeidbar, ohne auf eine HTTP-Probe zu verzichten. ## Was die neue Probe beweist — und was nicht `/accounts/login/` rendert ein Template, also müssen URL-Routing, Settings und die Template-Engine funktionieren. **Postgres beweist sie nicht** — nichts Billiges auf dieser Oberfläche tut das, und der Kommentar sagt das jetzt, statt mehr zu suggerieren. `/healthz` habe ich verworfen: es liefert dieselbe HTML wie die Login-Seite, ist also nur der SPA-Fallback und kein echter Health-Endpunkt. ## Die Falle, die das sonst in einen CrashLoop geführt hätte Das kubelet sendet die **Pod-IP** als `Host`-Header. Bei restriktivem `ALLOWED_HOSTS` antwortet Django 400, die Probe scheitert, der Pod crasht. `PAPERLESS_ALLOWED_HOSTS` steht auf `"*"` — heute also sicher. Der Kommentar hält fest, dass ein Einschränken dieses Werts den Probes einen expliziten `Host`-Header mitgeben müsste. Timings unverändert: die 120 s Liveness-Verzögerung existiert für Paperless' langsamen Start. ## Zur Validierung Geprüft **mit Flux' eigenem Field-Manager** (`--field-manager=kustomize-controller`): `serverside-applied`, sauber. Mit jedem anderen Manager meldet der Dry-Run das Deployment als invalid („may not specify more than 1 handler type"), weil `tcpSocket` weiterhin `kustomize-controller` gehört und von einem fremden Apply nicht geräumt wird. Das ist ein Artefakt der Prüfung, nicht des Manifests — mir wäre es beinahe als echter Fehler durchgegangen. ## Danach Der Rollout ersetzt den Pod. Ich prüfe, dass er `Ready` wird **und** dass `paperless.auth` danach nicht anfängt zu wachsen — ein laufender Pod allein beweist nicht, dass die Probe geräuschlos ist.
claude-bot added 1 commit 2026-08-28 10:59:48 +00:00
fix(paperless): probe a real Django view instead of a TCP connect
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
3f6f1edcb3
Under the split where gatus watches from outside and Kubernetes watches the
pods, paperless was the weakest link on the Kubernetes side: both probes were
tcpSocket, which only proves gunicorn accepted a connection. A worker deadlock
or a broken Django config would keep the pod Ready.

The existing comment explained why: HTTP probes against /api/ flood
`paperless.auth`, because ENABLE_HTTP_REMOTE_USER makes the remote-user
middleware log every unauthenticated API request. That was a real constraint
and the TCP fallback was a deliberate trade, not an oversight.

It is also narrower than the comment assumed. The middleware only covers the
API surface, and /accounts/login/ is a plain Django view. Measured against the
running pod:

    4 requests to /accounts/login/  ->  0 new paperless.auth lines
    4 requests to /api/            ->  4 new paperless.auth lines

So the noise is avoidable without giving up an HTTP probe. /accounts/login/
renders a template, which means URL routing, settings and the template engine
all have to be working. It does not prove Postgres is reachable — nothing cheap
on this surface does, and the comment now says so rather than implying more.

ALLOWED_HOSTS was the other way this could have failed: the kubelet sends the
pod IP as Host, and a restrictive value would answer 400 and crashloop the pod.
PAPERLESS_ALLOWED_HOSTS is "*", so it is safe today, and the comment records
what narrowing it would require.

Timings are unchanged — the 120s liveness delay is there for paperless's slow
start and nothing here makes it start faster.

Validated with Flux's own field manager. Applying as any other manager reports
the deployment invalid with two handler types, because tcpSocket stays owned by
kustomize-controller and is not cleared by a foreign apply; that is an artifact
of the check, not of the manifest.
k.renner merged commit b0df454fe4 into master 2026-08-28 11:03:00 +00:00
review-bot left a comment
Member

🤖 Automated review — model claude-sonnet-5 (complexity: normal), commit 3f6f1edcb3f15bfd5fa68a41abb1b160436fa355.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.


Summary

Replaces tcpSocket liveness/readiness probes on the paperless container with httpGet probes against /accounts/login/ (kubernetes/apps/paperless/paperless.yaml:175-188), trading a bare TCP accept for a real Django view render while avoiding the paperless.auth log flood that /api/ would cause. The PR body documents live-pod measurements backing the log-noise claim and the ALLOWED_HOSTS/Host-header risk, and the reasoning is corroborated by kubernetes/apps/gatus/configmap.yaml:177-185, which independently documents that the external gatus check only proves gate reachability and explicitly calls out the gap this PR closes. No blocking issues found.

🟡 Suggestions

  • Readiness timeout may be tighter than before (kubernetes/apps/paperless/paperless.yaml:183-188): timeoutSeconds: 5 was fine for a TCP accept but now has to cover a full Django request (URL resolve, middleware, template render, and possibly a session-store round trip depending on paperless-ngx's SESSION_ENGINE/cache backend). Combined with a modest cpu: 200m request, a slow tick under load could flip readiness and pull the pod out of the Service rotation more than the old probe would have. Worth watching after rollout; if it flaps, bumping timeoutSeconds is cheap.
  • The comment says the new probe "does not prove Postgres is healthy; nothing cheap here does" — worth double-checking that claim once, since if paperless-ngx's session backend is DB-backed (vs. Redis-cached), AuthenticationMiddleware evaluating request.user on this view could indirectly touch Postgres, which would actually strengthen the probe but also mean a flaky DB turns readiness failures into liveness restarts. Not verifiable from this checkout (paperless-ngx source isn't vendored here); the PR's own "Danach" follow-up (watching paperless.auth post-rollout) is a good place to also confirm behavior when Postgres is briefly unavailable, if that's ever observed.

🟢 Nits

  • kubernetes/apps/paperless/paperless.yaml:158: the comment block's opening line — # /accounts/login/ rather than /api/ or a TCP connect. — reads as a sentence fragment (no subject/verb), unlike the removed comment's "TCP probes — HTTP probes against /api/ flooded..." opener. A leading clause like "Probe target is /accounts/login/..." would read cleaner.

Tests

No automated tests apply to this change (Kubernetes manifest / probe config in a GitOps repo); validation is operational (dry-run apply, rollout, and post-rollout log/readiness observation), which the PR description says was already done against the live pod using the correct field-manager. That's an appropriate validation method for this kind of change.

Recommendation

Looks reasonable — no blocking issues; comments above are optional.

🤖 **Automated review** — model `claude-sonnet-5` (complexity: normal), commit `3f6f1edcb3f15bfd5fa68a41abb1b160436fa355`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary Replaces `tcpSocket` liveness/readiness probes on the `paperless` container with `httpGet` probes against `/accounts/login/` (`kubernetes/apps/paperless/paperless.yaml:175-188`), trading a bare TCP accept for a real Django view render while avoiding the `paperless.auth` log flood that `/api/` would cause. The PR body documents live-pod measurements backing the log-noise claim and the `ALLOWED_HOSTS`/`Host`-header risk, and the reasoning is corroborated by `kubernetes/apps/gatus/configmap.yaml:177-185`, which independently documents that the external gatus check only proves gate reachability and explicitly calls out the gap this PR closes. No blocking issues found. ## 🟡 Suggestions - **Readiness timeout may be tighter than before** (`kubernetes/apps/paperless/paperless.yaml:183-188`): `timeoutSeconds: 5` was fine for a TCP accept but now has to cover a full Django request (URL resolve, middleware, template render, and possibly a session-store round trip depending on paperless-ngx's `SESSION_ENGINE`/cache backend). Combined with a modest `cpu: 200m` request, a slow tick under load could flip readiness and pull the pod out of the Service rotation more than the old probe would have. Worth watching after rollout; if it flaps, bumping `timeoutSeconds` is cheap. - The comment says the new probe "does not prove Postgres is healthy; nothing cheap here does" — worth double-checking that claim once, since if paperless-ngx's session backend is DB-backed (vs. Redis-cached), `AuthenticationMiddleware` evaluating `request.user` on this view could indirectly touch Postgres, which would actually strengthen the probe but also mean a flaky DB turns readiness failures into liveness restarts. Not verifiable from this checkout (paperless-ngx source isn't vendored here); the PR's own "Danach" follow-up (watching `paperless.auth` post-rollout) is a good place to also confirm behavior when Postgres is briefly unavailable, if that's ever observed. ## 🟢 Nits - `kubernetes/apps/paperless/paperless.yaml:158`: the comment block's opening line — `# /accounts/login/ rather than /api/ or a TCP connect.` — reads as a sentence fragment (no subject/verb), unlike the removed comment's "TCP probes — HTTP probes against /api/ flooded..." opener. A leading clause like "Probe target is `/accounts/login/`..." would read cleaner. ## Tests No automated tests apply to this change (Kubernetes manifest / probe config in a GitOps repo); validation is operational (dry-run apply, rollout, and post-rollout log/readiness observation), which the PR description says was already done against the live pod using the correct field-manager. That's an appropriate validation method for this kind of change. ## Recommendation Looks reasonable — no blocking issues; comments above are optional. <!-- pr-review:sha=3f6f1edcb3f15bfd5fa68a41abb1b160436fa355 -->
Sign in to join this conversation.