From d8242637591b93a5c82ad48758c2cfc8f4104ce3 Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Wed, 12 Aug 2026 18:48:38 +0200 Subject: [PATCH 1/2] fix(trmnl): give Chromium a writable HOME and stamp obtained_at MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two runtime faults from the first deploy. Neither is visible statically — both needed the pod running to find. 1. The browser never launched. Every render died with chrome_crashpad_handler: --database is required Chromium cannot start without a writable HOME: crashpad builds its database path from it. Diagnosed on the pod rather than guessed — HOME=/, uid=1000, / is root-owned 0755 and not writable, /tmp is. The cause is OUR securityContext, not the image: forcing runAsUser 1000 with no matching passwd entry makes HOME fall back to "/". Sets HOME=/tmp; crash dumps are throwaway, so they do not belong on the PVC. Worth being precise about, because review had flagged the dropped capabilities as the likely Chromium risk and the comment I added said they were fine. The capabilities were fine. Something adjacent to that securityContext broke it anyway, and the comment now says to check HOME before loosening the capability block. 2. Even once the browser works, nothing would have been pushed. The scheduler logged BYOS tokens ... expired beyond the refresh window and skipped the webhook entirely. It derives token age from obtained_at, and the template shipped a literal 0, so freshly-minted tokens read as dating from 1970. The init container now stamps it with jq's `now * 1000 | floor` at write time. Verified the filter produces a 13-digit epoch-ms value. Stock Terminus disables session expiry and issues effectively permanent tokens, so a write-time stamp is honest enough. Failure mode of the pair was quiet in the way this stack keeps being quiet: pod Running and Ready, /health passing, schedule firing once a minute, zero screens reaching Terminus. Co-Authored-By: Claude Opus 5 (1M context) --- kubernetes/apps/trmnl/renderer.yaml | 40 ++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/kubernetes/apps/trmnl/renderer.yaml b/kubernetes/apps/trmnl/renderer.yaml index 50dc8f59..714d3eb8 100644 --- a/kubernetes/apps/trmnl/renderer.yaml +++ b/kubernetes/apps/trmnl/renderer.yaml @@ -35,6 +35,13 @@ data: # example value "og_1bit" matches no model on this instance (the # 2-bit OG model here is id 11, name "og_plus"). Getting this wrong # shows up as Terminus rejecting the pushed screen with a 422. + # + # All three auth fields below are placeholders the init container + # replaces — including obtained_at, which must end up a real epoch-MS + # stamp. The 0 here is inert only because it is always overwritten; + # left as 0 the scheduler reads the tokens as ancient and refuses to + # push at all ("expired beyond the refresh window"), with no render + # attempt and no error beyond one startup warning. schedules.template.json: | [ { @@ -145,10 +152,20 @@ spec: - -c - | set -eu + # obtained_at is stamped at write time, in MILLISECONDS. + # It is not cosmetic: the scheduler derives token age from + # it, and a literal 0 in the template made the app treat + # freshly-minted tokens as ancient and refuse to push — + # "BYOS tokens ... expired beyond the refresh window" + # — with no render attempt and no error, just a warning at + # startup. Stamping it here is honest enough: stock + # Terminus disables session expiry and issues effectively + # permanent tokens, so age is not load-bearing. jq --arg at "$BYOS_ACCESS_TOKEN" \ --arg rt "$BYOS_REFRESH_TOKEN" \ '(.[].webhook_format.byosConfig.auth.access_token) = $at - | (.[].webhook_format.byosConfig.auth.refresh_token) = $rt' \ + | (.[].webhook_format.byosConfig.auth.refresh_token) = $rt + | (.[].webhook_format.byosConfig.auth.obtained_at) = (now * 1000 | floor)' \ /template/schedules.template.json > /data/schedules.json echo "wrote /data/schedules.json ($(wc -c < /data/schedules.json) bytes)" env: @@ -208,6 +225,21 @@ spec: value: "http://home-assistant-bgp.home-assistant.svc.cluster.local:8123" - name: TZ value: "Europe/Berlin" + # Chromium will not start without a writable HOME. Running + # as uid 1000 with no matching passwd entry leaves HOME="/", + # which is root-owned 0755, and the browser dies on launch + # with + # chrome_crashpad_handler: --database is required + # because crashpad cannot create its database directory. + # + # Diagnosed on the running pod, not guessed: HOME=/, uid=1000, + # / not writable, /tmp writable. Crash dumps are throwaway, so + # /tmp is the right home for them rather than the PVC. + # + # This is a consequence of OUR non-root securityContext, not + # of the image — so it must move together with runAsUser. + - name: HOME + value: "/tmp" # Dedicated long-lived token, NOT the operator's personal # one, so the renderer's access can be revoked on its own. # It is injected into the headless browser's localStorage, @@ -260,6 +292,12 @@ spec: # mounted either). Those flags are baked into the app's own # PUPPETEER_ARGS, not something we pass, so this holds as long as # the pinned tag does — worth re-checking on any bump. + # + # Confirmed on the running pod: the capability drop is genuinely + # not what broke the browser. The first deploy DID crash it, but + # on a writable-HOME problem caused by runAsUser — see the HOME + # env above. Do not "fix" a future Chromium launch failure by + # loosening this block before checking HOME and /tmp. securityContext: allowPrivilegeEscalation: false capabilities: -- 2.54.0 From 2dd3e8f0b28580a19cabff30d23d10dc788b9b47 Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Wed, 12 Aug 2026 18:53:55 +0200 Subject: [PATCH 2/2] fix(trmnl): bound the renderer's /tmp with a sized emptyDir MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the HOME=/tmp fix in this PR, from review. Pointing HOME at /tmp was correct but left it on the container's writable layer, which was harmless while Chromium crashed on launch and wrote nothing. With the browser actually starting once a minute for the life of the pod, crashpad state and profile data now accumulate there with nothing to bound them — resources.limits sets memory only, so there is no ephemeral-storage ceiling to catch it either. Backs it with an emptyDir at a 512Mi sizeLimit, matching the /tmp mount convention already used in pve-exporter-deployment.yaml. The point of the limit is the failure mode, not the size: unbounded it degrades into node disk pressure, which surfaces far from this pod and hurts whatever else lands on that node; bounded it is an eviction of the renderer, attributable on sight. Co-Authored-By: Claude Opus 5 (1M context) --- kubernetes/apps/trmnl/renderer.yaml | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/kubernetes/apps/trmnl/renderer.yaml b/kubernetes/apps/trmnl/renderer.yaml index 714d3eb8..0ee7b019 100644 --- a/kubernetes/apps/trmnl/renderer.yaml +++ b/kubernetes/apps/trmnl/renderer.yaml @@ -252,6 +252,16 @@ spec: volumeMounts: - name: data mountPath: /data + # Bounded scratch backing HOME. Chromium writes crashpad + # state and profile data here on every launch, and it now + # launches once a minute for the life of the pod — on the + # container's writable layer that is unbounded growth + # against node disk, with no ephemeral-storage limit to + # catch it. A sizeLimit converts a slow, cluster-wide + # node-disk-pressure problem into a bounded and clearly + # attributable eviction of this pod. + - name: tmp + mountPath: /tmp # HTTP probes here, unlike terminus.yaml's TCP-only ones, because # /health is confirmed rather than assumed: the upstream image's # own Dockerfile declares @@ -309,6 +319,9 @@ spec: - name: schedule-template configMap: name: trmnl-renderer-schedule + - name: tmp + emptyDir: + sizeLimit: 512Mi --- apiVersion: v1 kind: Service -- 2.54.0