Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5 a79a5417e3 docs(trmnl): record why the probe path and dropped caps are safe
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Both were verified against the upstream image while writing this, but
the evidence never made it into the manifest, so review flagged them as
unverified assumptions. Fair — the reasoning was invisible.

/health is not a guess: the image's own Dockerfile declares
`HEALTHCHECK ... curl -f http://localhost:10000/health`, so the route is
part of its published contract. That is the difference from
terminus.yaml, which got TCP-only probes precisely because Terminus
publishes no health endpoint.

Dropping ALL capabilities is safe despite the container running headless
Chromium because the app launches it with --no-sandbox
--disable-setuid-sandbox --no-zygote --disable-dev-shm-usage, baked into
its own PUPPETEER_ARGS. That last flag is also why no /dev/shm emptyDir
is mounted. Both hold only for the pinned tag, so the comment says to
re-check on a bump.

No behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:40:51 +02:00
claude-botandClaude Opus 5 b6ada58b7a fix(trmnl): reach HA over the in-cluster Service, not the public name
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Review caught a blocking bug, correctly. HOME_ASSISTANT_URL pointed at
http://ha.renner.uno:8123, which is unusable from inside a pod.

That hostname is served by the home-assistant-bgp LoadBalancer with
externalTrafficPolicy: Local, advertised to the LAN over BGP. It is
not backed by an Ingress, so CoreDNS's k8s_gateway delegation does not
intercept it — a pod resolving it gets the real external address
(10.2.99.87) and hairpins off the LB.

This cluster has hit precisely this before with precisely this
Service. kubernetes/infrastructure/openbao/rotation/home-assistant.yaml
reaches HA from a pod for token rotation and says so outright: "The
public name is unusable from a pod (ingress-LB hairpin)." The same
language recurs in the gitea and woodpecker network policies. I took
the hostname from the repo's own "internal http://ha.renner.uno:8123"
note, which is true for a workstation and false from a pod.

Now uses home-assistant-bgp.home-assistant.svc.cluster.local:8123,
verified to exist with ClusterIP 10.96.79.213 on 8123/TCP.

Fixing the URL alone was not enough. The home-assistant namespace's
default-deny-ingress had no rule admitting trmnl pods on 8123 — the
only 8123 rule is scoped to the openbao secret-rotator — so traffic
would still have been dropped. Adds a companion ingress rule modelled
on the rotator's, scoped to the renderer's pod label rather than the
whole trmnl namespace, since Terminus shares that namespace and has no
business reaching Home Assistant. Only the ingress half is needed:
trmnl has no default-deny egress selecting the renderer, unlike
openbao.

Worth naming the failure mode, because it is the fourth "fails
quietly" case in this stack: nothing crashes. The schedule fires every
minute, each render errors on connect, and the panel simply never
updates.

Also updates the trmnl-daily Velero comment, which enumerated PVCs and
so undersold itself once trmnl-renderer-data existed — the schedule
selects the namespace minus CNPG-labelled PVCs, so new PVCs are
already covered without editing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:34:53 +02:00
claude-botandClaude Opus 5 f466f00029 fix(trmnl): model_id is the integer 1, not the name "v2"
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Resolves the one field flagged as unverified on the renderer PR, now
that a Terminus API token exists to check it with.

GET /api/screens on the running instance returns model_id as a NUMBER
(an existing welcome screen carries model_id 11), and GET /api/models
gives TRMNL X as id 1, name "v2", 1872x1404, 16 colours. Upstream's
example value "og_1bit" matches no model on this server at all — the
2-bit OG model here is id 11, name "og_plus" — so that example was
illustrative rather than a real identifier.

Would have surfaced as Terminus rejecting every pushed screen with a
422, i.e. a permanently blank panel with the renderer looking healthy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:27:57 +02:00
claude-botandClaude Opus 5 7e2a439b9e feat(trmnl): deploy the headless dashboard renderer
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Phase 4. Screenshots the Home Assistant e-ink dashboard, dithers it to
16 greys and hands it to Terminus, which the panel then polls.

Upstream ships this as a Home Assistant add-on. Ours is a plain HA
container with no Supervisor, so it runs standalone — viable because
the app prefers HOME_ASSISTANT_URL/ACCESS_TOKEN env vars over
/data/options.json and authenticates with an ordinary long-lived
token. SUPERVISOR_TOKEN is deliberately left unset: it only flips an
internal isAddOn flag and would switch the app to add-on behaviour.

The non-obvious part is the direction of travel. With the byos-hanami
webhook in `uri` delivery mode the renderer does NOT push image bytes
— it POSTs a URL pointing at its own /output/<file> endpoint and
TERMINUS FETCHES THE IMAGE BACK. That is why this lives in the trmnl
namespace: the fetch is intra-namespace and already permitted by
default-deny-ingress. Splitting the two across namespaces would have
needed a new policy for a dependency that is easy to miss.

Fully declarative, no clicks. Schedules are a JSON file at
/data/schedules.json which the app re-reads every 60s, so the whole
schedule — viewport 1872x1404, floyd-steinberg at bitDepth 4, the
trmnl-eink theme, the dashboard path — is reviewable in git rather
than typed into the app's web UI. An init container splices the BYOS
tokens into the ConfigMap template with jq; jq rather than sed because
the tokens are JWTs going into a JSON string and sed would corrupt on
any character it treats as special (verified with a token containing
/ + &).

No HTTP in the init container on purpose. The app exposes a
/api/byos/login proxy that trades a Terminus username+password for
these tokens, but using it would mean shipping the Terminus PASSWORD
into the cluster and making every pod start depend on a network call.
Stock Terminus disables session expiry and issues effectively
permanent tokens, so they are minted once out of band and kept in
OpenBao; the password never enters the cluster.

Three things that are load-bearing and would fail quietly otherwise:

  * The PVC at /data is REQUIRED. The app picks its data directory by
    testing whether /data exists — with nothing mounted it silently
    falls back to /app/data in the container, so schedules and rendered
    output vanish on restart and the panel goes blank.
  * The Service is ClusterIP with NO Ingress. The app's web UI on this
    port has no authentication whatsoever, per upstream, so it must not
    be routable from the LAN.
  * The memory limit is 2Gi, not the 512Mi upstream's troubleshooting
    note suggests. The app's own watchdog restarts it past 900MB but
    only measures its OWN rss — Chromium is a separate process, and it
    is rendering a 1872x1404 page.

The image is arch-suffixed (trmnl-ha-amd64); upstream publishes no
multi-arch manifest. Pinned and not enrolled in image automation,
since a bad bump here surfaces as a corrupted or blank panel rather
than a failed rollout — rationale recorded in the image-automation
README alongside Terminus's.

PREREQUISITE: homelab/apps/trmnl/renderer must be seeded in OpenBao
with HA_ACCESS_TOKEN, BYOS_ACCESS_TOKEN and BYOS_REFRESH_TOKEN before
this merges, or the ExternalSecret never syncs, the init container
blocks, and the trmnl Kustomization fails its wait.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 05:58:26 +02:00
security-bot dc1ea300a8 chore(image-automation): bump ghcr.io/music-assistant/server:2.9.12
ci/woodpecker/cron/security-scan Pipeline failed
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-11 00:35:18 +00:00
k.renner 73c2674344 Merge pull request 'fix(trmnl): request an explicit device LB address, document the sidekiq race' (#481) from fix-trmnl-lb-address into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#481
2026-08-10 21:08:53 +00:00
claude-botandClaude Opus 5 931810e28b fix(trmnl): request an explicit device LB address, document the sidekiq race
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Two things the first reconcile surfaced.

Cilium IPAM allocated 10.2.99.0 for the device LoadBalancer. Nothing
requested it — the Service carried no lbipam.cilium.io/ips annotation, so
IPAM handed out the first free address in the pool block, which is the base
address of the 10.2.99.0-10.2.99.255 range. Each LB address is advertised
to the UDM as its own BGP route, so it is routable in principle, but a .0
host address is exactly the sort of thing constrained embedded IP stacks
get wrong, and the TRMNL firmware is the one client that has to work here.
Requesting .89 explicitly also brings this in line with every other LB
Service in the repo, all of which pin their address, and matches the
fd99::89 IPAM had already chosen for v6.

Omitting the annotation was my call on the original PR, on the reasoning
that the device is provisioned with a hostname so the address could move
freely. That reasoning still holds; it just is not worth spending it on an
address nobody picked.

Second, the sidekiq container exits 1 once on a fresh deploy with
`PG::UndefinedTable: relation "account" does not exist` — it loses the race
against the web container's migration, then comes up healthy on the
restart. Observed on the first deploy, converged in about a second. There
is no upstream ordering hook, and gating it behind an initContainer that
probes for a named table would couple us to Terminus's schema, so this is
left to Kubernetes' restart deliberately. Documented in place so the next
person reading `RESTARTS 1` can tell the benign case from a real one.

Verified on the live cluster before this commit: Kustomization Ready, tr-pg
"Cluster in healthy state" on PostgreSQL 18, both ExternalSecrets
SecretSynced, terminus and valkey Running.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 23:04:17 +02:00
security-bot 2d8324b3f9 chore(image-automation): bump valkey/valkey:9.1.1-alpine
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-10 21:00:08 +00:00
k.renner b529864453 Merge pull request 'feat(trmnl): deploy Terminus BYOS server for the entrance e-ink panel' (#480) from feat-trmnl-terminus into master
Reviewed-on: k.renner/homelab#480
2026-08-10 20:59:01 +00:00
claude-botandClaude Opus 5 ac2d728108 feat(trmnl): deploy Terminus BYOS server for the entrance e-ink panel
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Phase 1 of the TRMNL X dashboard plan: stand up the self-hosted Terminus
(BYOS) server the panel will poll. The Home Assistant dashboard, the e-ink
theme and the screenshot renderer are later phases and are not in here.

Phase 0 verification, all confirmed against upstream before writing any of
this:

  * TRMNL X IS a first-class model in Terminus (id v2 / label "TRMNL X",
    1872x1404, 16 colours, 4-bit) — the stop-and-report condition in the
    plan does not trigger. Note it is NOT in config/db/seeds.rb: the model
    list is populated by the MODEL_SYNCHRONIZER job pulling trmnl.com, so
    the Models list is empty until that job has run once.
  * The HA add-on can run standalone. Our HA is plain HA Core, not
    Supervised, but the add-on reads env vars in preference to
    /data/options.json and authenticates with an ordinary long-lived
    token — SUPERVISOR_TOKEN only flips an isAddOn boolean. So the
    fallback path in the plan (upstream container, not add-on) is viable.
    That deployment lands with Phase 4.

Deviations from the repo's house patterns, each deliberate:

  * tr-pg runs PostgreSQL 18, not the standard 16. Terminus ships a
    committed config/db/structure.sql containing `SET transaction_timeout
    = 0;`, a GUC that does not exist before PG 17, so schema load against
    a 16 server fails outright. This is the first non-16 CNPG cluster
    here. The existing 16 pin is inherited from 739a67e9 (Apr 2026) and
    has no documented rationale.
  * The device endpoint is cleartext HTTP on a Cilium BGP LoadBalancer
    (trmnl-device.renner.uno) rather than an Ingress. The firmware cannot
    be given the internal CA, and a TLS-less Ingress would be a new
    pattern for this repo — a dedicated LB address is not. The admin UI
    keeps a normal TLS Ingress at trmnl.renner.uno; both front the same
    Service. Same split as home.renner.uno / ha.renner.uno.
  * ghcr.io/usetrmnl/terminus is NOT enrolled in image automation despite
    publishing a usable tag stream. It serves the firmware and
    /api/display contract to a wall-mounted device, upstream is pre-1.0
    with no releases or changelog, and an unattended bump that breaks
    either leaves the panel dark. Bumps are manual and reviewed. The
    Valkey sidecar is enrolled — it touches no device-facing contract.
  * Egress is an FQDN allowlist (trmnl.com, plus the S3 host that serves
    firmware binaries), not toEntities: world. The S3 host is returned by
    the API at request time rather than being a source constant, so it can
    drift; the policy comment says where to look when firmware sync
    starts failing. This also means Terminus's Extensions/Poll feature,
    which fetches admin-configured URLs, will not work — accepted, since
    screen content is pushed in by the renderer.
  * The CNPG pods are deliberately left OUT of that egress policy so the
    barman WAL archive path is not behind a rule that fails closed.

Images verified to exist in their registries before pinning:
ghcr.io/usetrmnl/terminus:0.68.0, valkey/valkey:9.1-alpine,
ghcr.io/cloudnative-pg/postgresql:18.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 22:54:38 +02:00
k.renner c8bc6380a1 Merge pull request 'fix(heating-can): give supervisord a control socket' (#478) from heating-can-supervisorctl into master
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#478
2026-08-10 19:53:51 +00:00
claude-botandClaude Opus 5 e3dea169b0 fix(heating-can): give supervisord a control socket
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The image shipped without [unix_http_server], [rpcinterface:supervisor] or
[supervisorctl], so supervisorctl could not connect:

  Error: .ini file does not include supervisorctl section

That means no way to query or restart an individual service inside the
container -- docker logs was the only introspection. Per-service lifecycle
management was the whole argument for running a supervisor instead of one
process per container, and the first revision did not actually deliver it.

Unix socket only, inside the container. Nothing outside needs it, and this
container runs with host networking, so a TCP listener would be exposed on
the host.

The build-time self-test now asserts the section exists and supervisorctl
runs, so this cannot regress silently the way it shipped.

NOT fixed here, because my diagnosis was wrong: the repeated
"mqtt broker disconnected. reason_code = Unspecified error" is not a client
ID collision. E3onCAN has no -mcid option at all -- verified against the
pinned source -- and generates its own ID; the native broker log showed
E3onCANclient.py-e380-7f847281 alongside open3e-edge, so the two never
collided. I nearly "fixed" it by passing a flag that does not exist.

Telemetry is flowing (each reconnect succeeds), so this is churn rather than
an outage, but the cause is still unidentified. The next step is the broker's
own log on edge-1 -- mosquitto records the disconnect reason, which the
client side reports only as "Unspecified error".

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:49:00 +02:00
k.renner 6384223415 Merge pull request 'fix(heating-can): stop the native services before removing their units' (#477) from fix-native-teardown into master
Reviewed-on: k.renner/homelab#477
2026-08-10 19:48:34 +00:00
claude-botandClaude Opus 5 1b7e3dc100 fix(heating-can): stop the native services before removing their units
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The migration did not stop anything. It ran stop + disable + mask in one
call with failed_when: false, then deleted the unit files. Masking a unit
whose file still exists conflicts, so the call failed, failed_when hid the
failure, nothing was stopped -- and the next task deleted the unit files out
from under two running processes.

That left systemctl is-enabled reporting "not-found", which looks clean,
while is-active still said "active". I reported the cutover successful on
the strength of the former.

On 2026-08-10 the result was two open3e instances issuing UDS requests on
the same CAN bus from 19:28 to 19:37, taking over each other's
open3e-edge MQTT session -- the exact failure the block's own comment said
it existed to prevent.

Fixed in three parts:

  - Stop and disable FIRST, and do not swallow the result. Only units that
    actually exist are touched (stat first), so a fresh host does not need
    an error hidden to get through.

  - No masking. It writes a symlink at the same path the next task removes,
    so the two cancel out. With the unit files gone there is nothing to
    start.

  - Assert the invariant that actually matters -- no native process on the
    CAN bus -- rather than inferring it from systemd reporting success.
    pgrep, then fail loudly with the surviving command lines and why it
    matters.

Verified against the live host: ok=15 changed=1 failed=0, the assert passing
with "No native CAN processes remain; safe to start the container", and the
run now idempotent (the single change is the compose up, not the teardown).

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:45:47 +02:00
k.renner 1ee4c2b61c Merge pull request 'refactor(edge): split the plays by capability; pin open3e and register it with the audit' (#475) from edge-capability-groups into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#475
2026-08-10 19:26:11 +00:00
28 changed files with 1278 additions and 13 deletions
+56 -11
View File
@@ -34,17 +34,42 @@
# stopped and masked rather than merely disabled -- masked so a stray
# `systemctl start` or a leftover dependency cannot bring them back alongside
# the containers.
- name: Stop and mask the native CAN services
ansible.builtin.systemd:
name: "{{ item }}"
state: stopped
enabled: false
masked: true
- name: Find which native CAN units still exist
ansible.builtin.stat:
path: "/etc/systemd/system/{{ item }}.service"
loop:
- open3e
- e3oncan
failed_when: false
register: heating_can_native_units
# Stop and disable BEFORE anything else, and do NOT swallow the result.
#
# The first version did stop + disable + mask in one call with
# failed_when: false, then deleted the unit files. Masking a unit whose file
# still exists conflicts, so the call failed, failed_when hid it, nothing was
# stopped -- and the next task deleted the unit files out from under two
# running processes. systemctl is-enabled then reported "not-found" and looked
# clean while is-active still said "active".
#
# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
# kicking each other off the broker.
#
# Only units that actually exist are touched, so a fresh host does not need an
# error swallowed to get through this.
- name: Stop and disable the native CAN services
ansible.builtin.systemd:
name: "{{ item.item }}"
state: stopped
enabled: false
loop: "{{ heating_can_native_units.results }}"
loop_control:
label: "{{ item.item }}"
when: item.stat.exists
# Masking is deliberately NOT used. It writes a symlink at the same path the
# next task removes, so the two cancel out -- which is half of how the original
# failed. With the unit files gone there is nothing left to start.
- name: Remove the native unit files
ansible.builtin.file:
path: "/etc/systemd/system/{{ item }}.service"
@@ -54,10 +79,30 @@
- e3oncan
notify: Reload systemd
# The venvs are the point of this change -- leaving them would leave the thing
# that churned. Their configuration is NOT removed: devices.json, the overlay
# and the env file are still rendered below and bind-mounted into the
# containers.
# The invariant that actually matters is "no native process is on the CAN bus",
# and systemd reporting success is not the same claim. Two open3e instances
# polling one bus is the failure this whole block exists to prevent, so it is
# asserted directly rather than inferred from the calls above.
- name: Check for surviving native CAN processes
ansible.builtin.command:
argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
register: heating_can_survivors
changed_when: false
failed_when: false
- name: Assert no native CAN process survived
ansible.builtin.assert:
that:
- heating_can_survivors.stdout | default('') | length == 0
fail_msg: >-
A native open3e/e3oncan process is still running after the units were
stopped and removed:
{{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
Starting the container now would put two readers on the same CAN bus --
two open3e instances issue competing UDS requests and take over each
other's MQTT session. Kill the survivors, then re-run.
success_msg: No native CAN processes remain; safe to start the container.
- name: Remove the native venvs
ansible.builtin.file:
path: "{{ item }}"
+2
View File
@@ -103,6 +103,8 @@ RUN open3e --help > /dev/null \
&& supervisord --help > /dev/null \
&& grep -q '^\[program:open3e\]' /etc/supervisord.conf \
&& grep -q '^\[program:e3oncan\]' /etc/supervisord.conf \
&& grep -q '^\[supervisorctl\]' /etc/supervisord.conf \
&& supervisorctl --help > /dev/null \
&& echo "self-test OK: both services execute; supervisord has both programs"
# CAN_RAW needs no capabilities, so nothing here runs as root.
+18
View File
@@ -10,6 +10,24 @@
# expands %(ENV_NAME)s at start, so neither the compose file nor a wrapper
# script constructs open3e's or E3onCAN's argv -- those CLIs stay an
# implementation detail of this image.
# Control socket. Without these two sections supervisorctl cannot connect, so
# there is no way to query or restart an individual service inside the
# container -- which is the entire reason for running a supervisor rather than
# one process per container. The first revision shipped without them.
#
# Unix socket only, inside the container: nothing outside needs to reach it,
# and the container runs with host networking, so a TCP listener would be
# exposed on the host.
[unix_http_server]
file=/tmp/supervisor.sock
chmod=0700
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock
[supervisord]
nodaemon=true
user=open3e
+18
View File
@@ -160,6 +160,24 @@ data:
conditions:
- "[STATUS] == any(200, 301, 302)"
# Terminus admin UI. Checks the TLS route only — the cleartext
# device endpoint (trmnl-device.renner.uno) is the same pod, so a
# failure here covers both. Terminus exposes no health endpoint, so
# this is a root-page check: the login page answers 200, and an
# unauthenticated request may redirect.
- name: Web Service 10
group: services
url: https://trmnl.renner.uno
interval: 60s
ui:
hide-hostname: true
hide-url: true
hide-port: true
hide-conditions: true
hide-errors: true
conditions:
- "[STATUS] == any(200, 301, 302)"
# Removed 2026-05-21: Jellyfin (media.renner.uno), Wiki (wiki.renner.uno),
# InfluxDB (influx.renner.uno) — hostnames NXDOMAIN, no Ingress; services
# no longer deployed. Re-add when/if they come back.
@@ -52,7 +52,7 @@ spec:
# stdlib only; Flux keeps the tag in lockstep via the shared image
# policy) — no extra dependency.
- name: reconcile-publish-ip
image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
image: ghcr.io/music-assistant/server:2.9.12 # {"$imagepolicy": "flux-system:music-assistant"}
command: ["/app/venv/bin/python", "/scripts/reconcile.py"]
volumeMounts:
- name: data
@@ -62,7 +62,7 @@ spec:
readOnly: true
containers:
- name: music-assistant
image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
image: ghcr.io/music-assistant/server:2.9.12 # {"$imagepolicy": "flux-system:music-assistant"}
env:
- name: LOG_LEVEL
value: info
+41
View File
@@ -0,0 +1,41 @@
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: tr-pg
namespace: trmnl
spec:
instances: 1
smartShutdownTimeout: 10
stopDelay: 60
# PG 18, NOT the repo-standard 16. Terminus ships a committed
# config/db/structure.sql containing `SET transaction_timeout = 0;` — a GUC
# that does not exist before PostgreSQL 17, so schema load against a 16
# server dies with `unrecognized configuration parameter`. Upstream's
# compose.yml develops against 18.4, so 18 is the tested target; 17 is the
# bare minimum. Bare major tag matches the other clusters (minor security
# bumps flow automatically, major upgrades stay an explicit decision).
imageName: ghcr.io/cloudnative-pg/postgresql:18
enableSuperuserAccess: true
storage:
storageClass: ceph-block
size: 5Gi
plugins:
- name: barman-cloud.cloudnative-pg.io
isWALArchiver: true
parameters:
barmanObjectName: tr-pg-store
bootstrap:
initdb:
database: terminus
owner: terminus
postgresql:
parameters:
shared_buffers: "128MB"
max_connections: "50"
wal_compression: "on"
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 512Mi
+48
View File
@@ -0,0 +1,48 @@
# Device-facing endpoint. Cleartext HTTP, deliberately.
#
# The TRMNL firmware initiates every request and cannot be handed the
# internal CA, so terminating TLS here would simply break polling. Scope of
# the exposure: this Service is LAN-only (no public DNS label, no Cloudflare
# record), it carries no admin session — the device authenticates with its
# MAC in the `ID` header plus a per-device api_key — and the Terminus admin
# UI stays on the TLS route at trmnl.renner.uno.
#
# Same shape as home-assistant-bgp: a dedicated address on the Cilium BGP
# pool rather than a TLS-less Ingress, which would be a new pattern for this
# repo. The device is provisioned with the HOSTNAME, not the address.
#
# The address is requested explicitly, matching every other LB service here.
# Left to allocate freely, Cilium IPAM handed out the first free address in
# the pool block (10.2.99.0-10.2.99.255) — which is 10.2.99.0, the base
# address of the range. Each LB address is advertised to the UDM as its own
# BGP route so that is routable in principle, but a .0 host address is the
# kind of thing constrained embedded IP stacks get wrong, and the TRMNL
# firmware is the single client that has to work here. Not worth the risk
# for an address nobody chose on purpose.
#
# .89 continues the existing sequence (.81-.88 are in use) and matches the
# fd99::89 that IPAM had already picked for v6.
apiVersion: v1
kind: Service
metadata:
name: terminus-device
namespace: trmnl
labels:
app.kubernetes.io/name: terminus
lb-pool: bgp
annotations:
external-dns.alpha.kubernetes.io/hostname: trmnl-device.renner.uno
lbipam.cilium.io/ips: "10.2.99.89,fd99::89"
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ipFamilyPolicy: PreferDualStack
ipFamilies:
- IPv4
- IPv6
selector:
app.kubernetes.io/name: terminus
ports:
- name: http
port: 80
targetPort: http
+18
View File
@@ -0,0 +1,18 @@
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: terminus-secrets
namespace: trmnl
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: terminus-secrets
# Holds APP_SECRET only (session/cookie/CSRF key). DATABASE_URL is not in
# here — it comes straight off the CNPG-generated `tr-pg-app` Secret's
# `uri` key, so the password rotates with the cluster and is never copied.
dataFrom:
- extract:
key: apps/trmnl
@@ -0,0 +1,15 @@
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: hetzner-s3-creds
namespace: trmnl
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: hetzner-s3-creds
dataFrom:
- extract:
key: seed/trmnl/hetzner-creds
+28
View File
@@ -0,0 +1,28 @@
# Admin UI only — login, device registration, playlists, screen previews.
# The device never touches this route: the firmware speaks plain HTTP and
# would reject the internal CA, so it uses the cleartext LoadBalancer in
# device-service.yaml instead. Both front the same Service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: terminus
namespace: trmnl
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
spec:
ingressClassName: traefik
tls:
- hosts:
- trmnl.renner.uno
secretName: trmnl-tls
rules:
- host: trmnl.renner.uno
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: terminus
port:
number: 80
+20
View File
@@ -0,0 +1,20 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- externalsecret.yaml
- pvcs.yaml
- valkey.yaml
- terminus.yaml
- ingress.yaml
- device-service.yaml
# Headless HA screenshotter. Lives in this namespace because Terminus
# FETCHES rendered screens back from it (uri delivery mode), so the
# two need the intra-namespace path.
- renderer.yaml
- renderer-externalsecret.yaml
# tr-pg CNPG Postgres + offsite Barman backup. Note this cluster runs
# PostgreSQL 18, not the repo-standard 16 — see the comment in cluster.yaml.
- cluster.yaml
- objectstore.yaml
- scheduled-backup.yaml
- hetzner-creds-externalsecret.yaml
+30
View File
@@ -0,0 +1,30 @@
---
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: tr-pg-store
namespace: trmnl
spec:
configuration:
# Offsite Hetzner Object Storage (fsn1), same bucket as the other CNPG
# clusters. No SSE: Hetzner supports only SSE-C, unusable by barman-cloud
# (plugin #646), so at-rest encryption is unavailable; transit is TLS.
destinationPath: s3://renner-uno-k8s-cnpg/tr-pg
endpointURL: https://fsn1.your-objectstorage.com
s3Credentials:
accessKeyId:
name: hetzner-s3-creds
key: ACCESS_KEY_ID
secretAccessKey:
name: hetzner-s3-creds
key: ACCESS_SECRET_KEY
# Region is required (REGION=fsn1) — without it boto3's HeadBucket
# (barman-cloud-check-wal-archive during bootstrap.recovery) 400s.
region:
name: hetzner-s3-creds
key: REGION
wal:
compression: gzip
data:
compression: gzip
retentionPolicy: "30d"
+57
View File
@@ -0,0 +1,57 @@
---
# Uploads + synchronised fonts for the Terminus pod. Mounted twice by
# subPath (uploads/, fonts/) into both the web and sidekiq containers, which
# share one pod — so RWO is sufficient and the Deployment uses `Recreate`.
#
# Contents: rendered device screens and the font set pulled by the
# FONT_SYNCHRONIZER job. Persisting fonts here (rather than an emptyDir)
# means a pod restart does not depend on upstream being reachable.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: terminus-data
namespace: trmnl
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 5Gi
---
# Valkey RDB snapshots. The Sidekiq queue is reconstructable — the
# synchroniser jobs re-enqueue on boot — so this is small and losing it is
# an inconvenience, not data loss.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: terminus-valkey-data
namespace: trmnl
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 1Gi
---
# Renderer state: schedules.json plus the rendered screens under
# output/, which Terminus fetches back over HTTP in uri delivery mode.
#
# This mount is REQUIRED, not just nice to have. The app picks its data
# directory by testing whether /data exists — with nothing mounted it
# silently falls back to /app/data inside the container, so schedules
# and rendered output would vanish on every restart and the panel would
# go blank until the next render.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: trmnl-renderer-data
namespace: trmnl
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 2Gi
@@ -0,0 +1,27 @@
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: trmnl-renderer-secrets
namespace: trmnl
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: trmnl-renderer-secrets
# Kept separate from apps/trmnl (which holds Terminus's APP_SECRET)
# because these are credentials the RENDERER holds about other
# systems, with different rotation stories:
#
# HA_ACCESS_TOKEN — a dedicated Home Assistant long-lived token,
# not the operator's personal one, so the
# renderer can be revoked in isolation.
# BYOS_ACCESS_TOKEN — Terminus JWTs, minted once out of band from
# BYOS_REFRESH_TOKEN a Terminus login. The PASSWORD deliberately
# never enters the cluster; stock Terminus
# disables session expiry, so these are
# effectively permanent.
dataFrom:
- extract:
key: apps/trmnl/renderer
+293
View File
@@ -0,0 +1,293 @@
# Headless renderer: screenshots the Home Assistant e-ink dashboard,
# dithers it to 16 greys and hands it to Terminus.
#
# Upstream ships this as a Home Assistant add-on. Our HA is a plain
# container with no Supervisor, so it runs standalone here instead —
# viable because the app prefers HOME_ASSISTANT_URL/ACCESS_TOKEN env
# vars over /data/options.json and authenticates with an ordinary
# long-lived token. SUPERVISOR_TOKEN is deliberately NOT set: it only
# flips an internal isAddOn flag, and setting it would switch the app
# to add-on behaviour (fixed port, /data assumptions).
#
# DIRECTION OF TRAVEL, which is the non-obvious part: with the
# byos-hanami webhook in `uri` delivery mode the renderer does NOT
# push image bytes. It POSTs a URL pointing at its own /output/<file>
# endpoint, and TERMINUS FETCHES THE IMAGE BACK from it. That is why
# both live in this namespace — the pull is intra-namespace and is
# already permitted by the default-deny-ingress rule.
apiVersion: v1
kind: ConfigMap
metadata:
name: trmnl-renderer-schedule
namespace: trmnl
data:
# Template for /data/schedules.json. The init container splices the
# BYOS tokens in; everything non-secret is declared here so the
# schedule is reviewable in git rather than clicked into the app's
# web UI (which has no authentication and is not exposed).
#
# The app re-reads schedules.json every 60 s, so edits here take
# effect without a restart once the init container has rewritten it.
#
# model_id is 1 — TRMNL X (name "v2", 1872x1404, 16 colours) — and it
# is an INTEGER, not a name. Confirmed against the running server:
# GET /api/screens returns model_id as a number, and upstream's
# example value "og_1bit" matches no model on this instance (the
# 2-bit OG model here is id 11, name "og_plus"). Getting this wrong
# shows up as Terminus rejecting the pushed screen with a 422.
schedules.template.json: |
[
{
"id": "trmnl-entrance",
"name": "Entrance e-ink panel",
"enabled": true,
"cron": "* * * * *",
"interval_minutes": 1,
"ha_mode": true,
"dashboard_path": "/trmnl-entrance/home",
"target_url": null,
"viewport": { "width": 1872, "height": 1404 },
"device": "schedule_trmnl_x",
"crop": { "enabled": false, "x": 0, "y": 0, "width": 0, "height": 0 },
"format": "png",
"rotate": null,
"zoom": 1,
"wait": null,
"theme": "trmnl-eink",
"lang": null,
"dark": false,
"invert": false,
"timestamp": false,
"dithering": {
"enabled": true,
"method": "floyd-steinberg",
"palette": "gray-16",
"bitDepth": 4,
"gammaCorrection": true,
"blackLevel": 0,
"whiteLevel": 100,
"normalize": true,
"saturationBoost": false
},
"webhook_url": "http://terminus.trmnl.svc.cluster.local/api/screens",
"webhook_headers": {},
"webhook_format": {
"format": "byos-hanami",
"byosConfig": {
"label": "Eingang",
"name": "trmnl-entrance",
"model_id": 1,
"preprocessed": true,
"addon_base_url": "http://trmnl-renderer.trmnl.svc.cluster.local:10000",
"delivery_mode": "uri",
"auth": {
"enabled": true,
"access_token": "__BYOS_ACCESS_TOKEN__",
"refresh_token": "__BYOS_REFRESH_TOKEN__",
"obtained_at": 0
}
}
},
"createdAt": "2026-08-11T00:00:00.000Z",
"updatedAt": "2026-08-11T00:00:00.000Z"
}
]
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: trmnl-renderer
namespace: trmnl
labels:
app.kubernetes.io/name: trmnl-renderer
spec:
replicas: 1
# RWO PVC, and two renderers would push conflicting screens for the
# same device.
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: trmnl-renderer
template:
metadata:
labels:
app.kubernetes.io/name: trmnl-renderer
spec:
enableServiceLinks: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
initContainers:
# Writes /data/schedules.json from the ConfigMap template with
# the BYOS tokens injected.
#
# jq rather than sed: the tokens are JWTs and go into a JSON
# string, so they need JSON-safe escaping. A sed substitution
# would corrupt on any character sed treats as special.
#
# No HTTP here on purpose. The app does expose a /api/byos/login
# proxy that exchanges a Terminus username+password for these
# tokens, but that would mean shipping the Terminus PASSWORD
# into the cluster and doing a network call at every pod start.
# Stock Terminus disables session expiry and issues effectively
# permanent tokens, so they are minted once out of band and kept
# in OpenBao — the password never enters the cluster.
- name: render-schedule
image: gitea.renner.uno/gitea_admin/alpine-jq:2026-07-30
command:
- sh
- -c
- |
set -eu
jq --arg at "$BYOS_ACCESS_TOKEN" \
--arg rt "$BYOS_REFRESH_TOKEN" \
'(.[].webhook_format.byosConfig.auth.access_token) = $at
| (.[].webhook_format.byosConfig.auth.refresh_token) = $rt' \
/template/schedules.template.json > /data/schedules.json
echo "wrote /data/schedules.json ($(wc -c < /data/schedules.json) bytes)"
env:
- name: BYOS_ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: trmnl-renderer-secrets
key: BYOS_ACCESS_TOKEN
- name: BYOS_REFRESH_TOKEN
valueFrom:
secretKeyRef:
name: trmnl-renderer-secrets
key: BYOS_REFRESH_TOKEN
volumeMounts:
- name: data
mountPath: /data
- name: schedule-template
mountPath: /template
readOnly: true
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
containers:
# Pinned, not enrolled in image automation — same reasoning as
# Terminus itself. This drives a wall-mounted panel and upstream
# is pre-1.0; an unattended bump that changes the dithering
# pipeline or the webhook payload shows up as a corrupted
# screen, not as a failed rollout.
#
# Note the image is ARCH-SUFFIXED: upstream publishes no
# multi-arch manifest, only trmnl-ha-amd64 and trmnl-ha-aarch64.
- name: renderer
image: ghcr.io/usetrmnl/trmnl-ha-amd64:0.9.5
ports:
- name: http
containerPort: 10000
env:
# The in-cluster Service name, NOT ha.renner.uno and not
# home.renner.uno. Both public names are unusable from a
# pod: ha.renner.uno resolves to the home-assistant-bgp
# LoadBalancer's external address (10.2.99.87), which is not
# backed by an Ingress, so CoreDNS's k8s_gateway delegation
# does not intercept it and the pod hairpins off the LB.
#
# This cluster has hit that exact failure before — see the
# comment on kubernetes/infrastructure/openbao/rotation/
# home-assistant.yaml, which reaches HA the same way and
# states it outright: "The public name is unusable from a
# pod (ingress-LB hairpin)."
#
# FAILS QUIETLY if wrong: the schedule keeps firing every
# minute, each render errors on connect, and the panel just
# never updates. Nothing goes CrashLoopBackOff.
- name: HOME_ASSISTANT_URL
value: "http://home-assistant-bgp.home-assistant.svc.cluster.local:8123"
- name: TZ
value: "Europe/Berlin"
# Dedicated long-lived token, NOT the operator's personal
# one, so the renderer's access can be revoked on its own.
# It is injected into the headless browser's localStorage,
# which is how the app authenticates to the HA frontend.
- name: ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: trmnl-renderer-secrets
key: HA_ACCESS_TOKEN
volumeMounts:
- name: data
mountPath: /data
# HTTP probes here, unlike terminus.yaml's TCP-only ones, because
# /health is confirmed rather than assumed: the upstream image's
# own Dockerfile declares
# HEALTHCHECK ... curl -f http://localhost:10000/health
# so the route is part of the image's published contract. Terminus
# got TCP probes precisely because it publishes no such endpoint.
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 10
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
resources:
requests:
cpu: 250m
memory: 768Mi
# Chromium is a SEPARATE process from the app, and the app's
# own watchdog only measures its own RSS (restarting itself
# past 900 MB). The limit therefore has to cover the app's
# ceiling PLUS a browser rendering a 1872x1404 page, which
# is well above the 512 MB upstream's troubleshooting note
# suggests.
limits:
memory: 2Gi
# Chromium normally needs either SYS_ADMIN or a user namespace to
# start its own sandbox, which would make dropping ALL fatal. It
# is safe here because the app launches the browser with
# --no-sandbox --disable-setuid-sandbox --no-zygote (and
# --disable-dev-shm-usage, which is why no /dev/shm emptyDir is
# mounted either). Those flags are baked into the app's own
# PUPPETEER_ARGS, not something we pass, so this holds as long as
# the pinned tag does — worth re-checking on any bump.
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
volumes:
- name: data
persistentVolumeClaim:
claimName: trmnl-renderer-data
- name: schedule-template
configMap:
name: trmnl-renderer-schedule
---
apiVersion: v1
kind: Service
metadata:
name: trmnl-renderer
namespace: trmnl
labels:
app.kubernetes.io/name: trmnl-renderer
spec:
# ClusterIP only, and deliberately no Ingress. The app's web UI on
# this port has NO authentication of any kind — upstream says so
# outright — so it must not be routable from the LAN. Terminus
# reaches it in-namespace to fetch rendered screens; nothing else
# needs to.
selector:
app.kubernetes.io/name: trmnl-renderer
ports:
- name: http
port: 10000
targetPort: http
@@ -0,0 +1,16 @@
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
name: tr-pg-daily
namespace: trmnl
spec:
# 03:00 local, same slot as the other CNPG clusters (the Velero PVC
# schedule for this namespace runs later, at 08:15).
schedule: "0 0 3 * * *"
immediate: true
backupOwnerReference: self
cluster:
name: tr-pg
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
+184
View File
@@ -0,0 +1,184 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: terminus-config
namespace: trmnl
data:
TZ: "Europe/Berlin"
HANAMI_PORT: "2345"
# What the device is told to fetch from. Terminus bakes this into the
# absolute `image_url` it hands back on /api/display, so it MUST be the
# cleartext LAN name, not trmnl.renner.uno — the firmware speaks plain
# HTTP and would reject the internal CA. No trailing slash: the same
# string is typed into the captive portal at provisioning time.
API_URI: "http://trmnl-device.renner.uno"
KEYVALUE_URL: "redis://terminus-valkey:6379"
# Rack::Attack rate-limit bypass. Beyond upstream's RFC1918 defaults this
# adds fc00::/7, which covers the pod CIDR (fd10:244::/48), the Cilium LB
# pool (fd99::/120) and the LAN ULA — without it every IPv6 client,
# including the device and Traefik, is rate-limited as if it were remote.
RACK_ATTACK_ALLOWED_SUBNETS: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1,::1,fc00::/7"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: terminus
namespace: trmnl
labels:
app.kubernetes.io/name: terminus
spec:
replicas: 1
# RWO PVC shared by both containers in this pod, and APP_SETUP runs
# migrations on boot — two concurrent web pods would race on both.
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: terminus
template:
metadata:
labels:
app.kubernetes.io/name: terminus
spec:
enableServiceLinks: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
# Tag pinned deliberately and NOT enrolled in Flux image automation.
# Terminus is pre-1.0 with no GitHub Releases and no changelog; an
# unattended bump that changes the firmware or /api/display contract
# is a device-bricking class of risk. Bumps are a reviewed, manual
# act — see kubernetes/infrastructure/image-automation/README.md.
- name: terminus
image: ghcr.io/usetrmnl/terminus:0.68.0
ports:
- name: http
containerPort: 2345
envFrom:
- configMapRef:
name: terminus-config
env:
# Runs `hanami assets compile` + `hanami db migrate` in the
# entrypoint before exec'ing puma. Set on the web container
# only — the sidekiq sidecar must not race the migration.
- name: APP_SETUP
value: "true"
# CNPG generates this Secret; `uri` already carries the rotated
# password and points at the tr-pg-rw service.
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: tr-pg-app
key: uri
- name: APP_SECRET
valueFrom:
secretKeyRef:
name: terminus-secrets
key: APP_SECRET
volumeMounts:
- name: data
mountPath: /app/public/uploads
subPath: uploads
- name: data
mountPath: /app/public/fonts
subPath: fonts
# TCP probes, not HTTP: Terminus exposes no documented health
# endpoint, and every real route is behind either auth or
# Rack::Attack. Tighten to an HTTP GET once the route set is
# confirmed against the running app.
startupProbe:
tcpSocket:
port: http
periodSeconds: 10
# ~10 min: first boot compiles assets and migrates a fresh DB.
failureThreshold: 60
livenessProbe:
tcpSocket:
port: http
periodSeconds: 30
timeoutSeconds: 5
readinessProbe:
tcpSocket:
port: http
periodSeconds: 10
timeoutSeconds: 5
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
memory: 1Gi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
- name: sidekiq
image: ghcr.io/usetrmnl/terminus:0.68.0
command: ["bundle", "exec", "sidekiq", "-r", "./config/sidekiq.rb"]
envFrom:
- configMapRef:
name: terminus-config
# EXPECTED on a fresh deploy: this container loses the race against
# the web container's migration and exits 1 with
# `PG::UndefinedTable: relation "account" does not exist`, then
# comes up healthy on the restart once the schema exists.
# Observed on first deploy 2026-08-10; converged in ~1s. There is
# no upstream ordering hook, and gating it behind an initContainer
# that probes for a specific table would couple us to Terminus's
# schema. So this is left to Kubernetes' restart, deliberately —
# a single sidekiq restart shortly after a rollout is benign. More
# than a couple, or restarts long after a rollout, are NOT this.
env:
# No APP_SETUP here — see the web container.
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: tr-pg-app
key: uri
- name: APP_SECRET
valueFrom:
secretKeyRef:
name: terminus-secrets
key: APP_SECRET
volumeMounts:
- name: data
mountPath: /app/public/uploads
subPath: uploads
- name: data
mountPath: /app/public/fonts
subPath: fonts
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
volumes:
- name: data
persistentVolumeClaim:
claimName: terminus-data
---
apiVersion: v1
kind: Service
metadata:
name: terminus
namespace: trmnl
labels:
app.kubernetes.io/name: terminus
spec:
selector:
app.kubernetes.io/name: terminus
ports:
- name: http
port: 80
targetPort: http
+79
View File
@@ -0,0 +1,79 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: terminus-valkey
namespace: trmnl
labels:
app.kubernetes.io/name: terminus-valkey
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: terminus-valkey
template:
metadata:
labels:
app.kubernetes.io/name: terminus-valkey
spec:
securityContext:
runAsNonRoot: true
runAsUser: 999
fsGroup: 999
seccompProfile:
type: RuntimeDefault
containers:
# Valkey major version tracks upstream's compose.yml (9.x). Sidekiq
# is the only consumer; a major bump is a separate migration.
- name: valkey
image: valkey/valkey:9.1.1-alpine # {"$imagepolicy": "flux-system:valkey-trmnl"}
args:
- valkey-server
- --save
- "60"
- "1"
- --loglevel
- warning
ports:
- name: valkey
containerPort: 6379
volumeMounts:
- name: data
mountPath: /data
livenessProbe:
tcpSocket:
port: valkey
initialDelaySeconds: 10
readinessProbe:
exec:
command: ["valkey-cli", "ping"]
initialDelaySeconds: 5
periodSeconds: 10
resources:
requests:
cpu: 20m
memory: 64Mi
limits:
memory: 256Mi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
volumes:
- name: data
persistentVolumeClaim:
claimName: terminus-valkey-data
---
apiVersion: v1
kind: Service
metadata:
name: terminus-valkey
namespace: trmnl
spec:
selector:
app.kubernetes.io/name: terminus-valkey
ports:
- port: 6379
targetPort: valkey
+28
View File
@@ -0,0 +1,28 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: trmnl
namespace: flux-system
spec:
# Terminus (TRMNL BYOS server) plus its tr-pg CNPG Cluster + ObjectStore +
# ScheduledBackup, the Valkey queue, and the uploads/fonts PVC. Drives the
# entrance-way e-ink panel; the device polls the cleartext LoadBalancer.
interval: 10m
retryInterval: 1m
timeout: 30m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/trmnl
prune: true
wait: true
dependsOn:
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
- name: infrastructure-cnpg-barman
healthCheckExprs:
- apiVersion: postgresql.cnpg.io/v1
kind: Cluster
failed: "has(status.phase) && status.phase.startsWith('Failed')"
current: "has(status.phase) && (status.phase == 'Cluster in healthy state' || status.readyInstances > 0)"
@@ -96,6 +96,29 @@ The security-scan agent will continue to surface vuln findings on
these as "Skipped / Flux image-automation owns this" even without
a policy.
- `ghcr.io/usetrmnl/terminus` — publishes a usable `0.x.0` tag stream, so
unlike the three above this is **not** a tag-format problem. It is left
unenrolled because the blast radius is hardware: Terminus is pre-1.0,
cuts no GitHub Releases and ships no changelog, and it serves the
firmware and `/api/display` contract that the TRMNL X panel polls. An
unattended bump that changes either can leave a wall-mounted device
dark, with no rollback path that doesn't involve the captive portal.
Bumps are a reviewed, manual act: read the commit range, then edit the
tag in `kubernetes/apps/trmnl/terminus.yaml` (both containers).
The Valkey sidecar alongside it *is* enrolled (`valkey.yaml`) — it holds
only the Sidekiq queue and touches no device-facing contract.
- `ghcr.io/usetrmnl/trmnl-ha-amd64` — the TRMNL screenshot renderer, for
the same reason and one more. It owns the dithering pipeline and the
webhook payload shape, so a bad bump surfaces as a *corrupted or blank
panel*, not as a failed rollout — there is no readiness signal that
catches "the image is now unreadable". Separately, upstream publishes
**no multi-arch manifest**: only the arch-suffixed `trmnl-ha-amd64` and
`trmnl-ha-aarch64` repositories exist, so an ImagePolicy here would also
be pinning us to one architecture in a way the tag does not make
obvious. Bump manually in `kubernetes/apps/trmnl/renderer.yaml`.
## Re-import hazard: cnpg-barman-plugin
`kubernetes/infrastructure/cnpg-barman-plugin/manifest.yaml` is an
@@ -20,6 +20,7 @@ resources:
- redis.yaml
- shelly-manager.yaml
- tika.yaml
- valkey.yaml
- velero-plugin-for-aws.yaml
- vsftpd.yaml
- woodpecker.yaml
@@ -0,0 +1,27 @@
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
name: valkey-trmnl
namespace: flux-system
spec:
image: valkey/valkey
interval: 1h
---
# Alpine tags only, semver 9.x — matches the major version pinned in
# upstream Terminus's compose.yml. Valkey holds nothing but the Sidekiq
# queue here, so minor bumps are safe to take unattended; a 10.x cutover
# is a deliberate migration.
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
name: valkey-trmnl
namespace: flux-system
spec:
imageRepositoryRef:
name: valkey-trmnl
filterTags:
pattern: '^(?P<ver>\d+\.\d+\.\d+)-alpine$'
extract: '$ver'
policy:
semver:
range: "9.x"
@@ -145,3 +145,9 @@ apiVersion: v1
kind: Namespace
metadata:
name: external-secrets
---
# Terminus (TRMNL BYOS server) driving the entrance-way e-ink panel.
apiVersion: v1
kind: Namespace
metadata:
name: trmnl
@@ -371,3 +371,36 @@ spec:
- ports:
- port: "8123"
protocol: TCP
---
# The TRMNL e-ink renderer (trmnl ns) drives a headless browser against the
# entrance dashboard, so it needs the HA frontend on :8123 — same in-cluster
# path as the rotator above, and for the same reason: the public names
# hairpin off the home-assistant-bgp LoadBalancer and are unusable from a
# pod. It reaches HA at home-assistant-bgp.home-assistant.svc.cluster.local.
#
# Only the ingress half is needed here. Unlike the openbao rotator, the trmnl
# namespace has no default-deny EGRESS covering this pod — terminus-egress is
# scoped to the Terminus pods by label and does not select the renderer — so
# there is no source-side rule to add.
#
# Source scoped to the renderer pod specifically, not the whole trmnl
# namespace: Terminus itself sits in that namespace and has no business
# reaching Home Assistant.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: home-assistant-trmnl-renderer-ingress
namespace: home-assistant
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: home-assistant
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": trmnl
app.kubernetes.io/name: trmnl-renderer
toPorts:
- ports:
- port: "8123"
protocol: TCP
@@ -21,4 +21,5 @@ resources:
- openbao-ns.yaml
- external-secrets-ns.yaml
- woodpecker-ci-ns.yaml
- trmnl-ns.yaml
- allow-cilium-health.yaml
@@ -0,0 +1,177 @@
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: default-deny-ingress
namespace: trmnl
spec:
endpointSelector: {}
ingress:
# Intra-namespace — terminus ↔ sidekiq ↔ valkey ↔ tr-pg.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": trmnl
# Admin UI via Traefik (trmnl.renner.uno). The device does NOT come
# this way — see allow-device-http below.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
# Prometheus scrape.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": monitoring
# Velero backups via PVC mount.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": velero
# CNPG operator (in `database` ns) reconciles tr-pg — needs to reach
# the postgres pods for health, failover, role checks.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": database
- fromEntities:
- host
- remote-node
# ICMP — all standard ICMPv4 + ICMPv6 from any cluster pod. See
# home-assistant-ns.yaml for the rationale. INGRESS ONLY.
- fromEndpoints:
- matchExpressions:
- key: io.kubernetes.pod.namespace
operator: Exists
icmps:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
---
# The TRMNL panel polls /api/display over cleartext HTTP on the
# terminus-device LoadBalancer. The firmware cannot be handed the internal
# CA, so this bypasses Traefik entirely and the namespace default-deny has
# to be opened for it — same shape as paperless's allow-ftp-scanner.
#
# Scoped to the terminus pods and port 2345 only. Requests still have to
# carry a known device MAC in the `ID` header plus its api_key; this rule
# opens the port, not the application.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: allow-device-http
namespace: trmnl
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: terminus
ingress:
- fromEntities:
- world
- host
- remote-node
toPorts:
- ports:
- port: "2345"
protocol: TCP
---
# Egress allowlist for the Terminus pod only.
#
# Deliberately NOT applied to the tr-pg CNPG pods: selecting them here
# would make their egress restricted too, and the barman-cloud WAL archive
# path to Hetzner object storage is not worth putting behind an FQDN rule
# that fails closed. They keep unrestricted egress, matching immich/authentik.
#
# Terminus needs exactly three outbound destinations, all HTTPS:
# - trmnl.com → GET /api/models, /api/palettes (MODEL_SYNCHRONIZER,
# which is what seeds the TRMNL X model definition —
# 1872x1404, 16 colours — into the Models list),
# GET /api/firmware/latest (FIRMWARE_SYNCHRONIZER),
# GET /fonts/<name> (FONT_SYNCHRONIZER)
# - trmnl-fw.s3.us-east-2.amazonaws.com
# → the actual firmware binary. This host is NOT a
# constant in the source: it is the `url` field of
# the /api/firmware/latest response, so TRMNL can
# move bucket/region without a Terminus release and
# firmware sync would start failing closed here.
# If firmware sync breaks, re-check that response
# before assuming the policy is wrong.
#
# KNOWN LIMITATION: Terminus's "Extensions" (private plugin) feature polls
# arbitrary admin-configured URLs rendered from a Liquid template stored in
# the database. Those are unknowable from source and WILL be blocked by this
# allowlist. We do not use Extensions — screen content is pushed in from the
# Home Assistant renderer — so this is an accepted trade, not an oversight.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: terminus-egress
namespace: trmnl
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: terminus
egress:
# Intra-namespace — tr-pg-rw (postgres) and terminus-valkey.
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": trmnl
# DNS. The L7 `rules.dns` block is mandatory, not decorative: Cilium's
# DNS proxy only learns an FQDN→IP mapping for names it has actually
# observed being resolved, so without this the toFQDNs rules below
# never match anything.
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
rules:
dns:
- matchPattern: "*"
- toFQDNs:
- matchName: "trmnl.com"
- matchName: "trmnl-fw.s3.us-east-2.amazonaws.com"
toPorts:
- ports:
- port: "443"
protocol: TCP
- toEntities:
- host
- remote-node
@@ -28,6 +28,7 @@ spec:
- monitoring
- netbird-backup # netbird-backup-ssh: the VPS-pull backup key
- paperless
- trmnl
- velero
provider:
vault: # ESO's provider name; speaks OpenBao fine
@@ -283,3 +283,32 @@ spec:
storageLocation: default
ttl: 720h0m0s
snapshotMoveData: true
---
apiVersion: velero.io/v1
kind: Schedule
metadata:
name: trmnl-daily
namespace: velero
labels:
app.kubernetes.io/part-of: homelab-backup
spec:
# Daily at 08:15 local (next free stagger slot after netbird-backup-daily
# at 07:45). Selects the whole namespace minus the CNPG-labelled PVCs, so
# it picks up terminus-data (device screens + synchronised fonts),
# terminus-valkey-data and trmnl-renderer-data (the renderer's schedule
# and rendered output) without needing to be edited per PVC. The tr-pg
# CNPG cluster is covered separately by Barman, hence the exclusion.
schedule: "15 8 * * *"
useOwnerReferencesInBackup: false
template:
includedNamespaces:
- trmnl
includedResources:
- persistentvolumeclaims
storageLocation: default
ttl: 720h0m0s
snapshotMoveData: true
labelSelector:
matchExpressions:
- key: cnpg.io/cluster
operator: DoesNotExist