From 931810e28bad3b931b6bf0530b3c9c68ebd361ee Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Mon, 10 Aug 2026 23:04:17 +0200 Subject: [PATCH] fix(trmnl): request an explicit device LB address, document the sidekiq race MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things the first reconcile surfaced. Cilium IPAM allocated 10.2.99.0 for the device LoadBalancer. Nothing requested it — the Service carried no lbipam.cilium.io/ips annotation, so IPAM handed out the first free address in the pool block, which is the base address of the 10.2.99.0-10.2.99.255 range. Each LB address is advertised to the UDM as its own BGP route, so it is routable in principle, but a .0 host address is exactly the sort of thing constrained embedded IP stacks get wrong, and the TRMNL firmware is the one client that has to work here. Requesting .89 explicitly also brings this in line with every other LB Service in the repo, all of which pin their address, and matches the fd99::89 IPAM had already chosen for v6. Omitting the annotation was my call on the original PR, on the reasoning that the device is provisioned with a hostname so the address could move freely. That reasoning still holds; it just is not worth spending it on an address nobody picked. Second, the sidekiq container exits 1 once on a fresh deploy with `PG::UndefinedTable: relation "account" does not exist` — it loses the race against the web container's migration, then comes up healthy on the restart. Observed on the first deploy, converged in about a second. There is no upstream ordering hook, and gating it behind an initContainer that probes for a named table would couple us to Terminus's schema, so this is left to Kubernetes' restart deliberately. Documented in place so the next person reading `RESTARTS 1` can tell the benign case from a real one. Verified on the live cluster before this commit: Kustomization Ready, tr-pg "Cluster in healthy state" on PostgreSQL 18, both ExternalSecrets SecretSynced, terminus and valkey Running. Co-Authored-By: Claude Opus 5 (1M context) --- kubernetes/apps/trmnl/device-service.yaml | 16 ++++++++++++++-- kubernetes/apps/trmnl/terminus.yaml | 10 ++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/kubernetes/apps/trmnl/device-service.yaml b/kubernetes/apps/trmnl/device-service.yaml index c8c1bf15..f7f5e202 100644 --- a/kubernetes/apps/trmnl/device-service.yaml +++ b/kubernetes/apps/trmnl/device-service.yaml @@ -9,8 +9,19 @@ # # Same shape as home-assistant-bgp: a dedicated address on the Cilium BGP # pool rather than a TLS-less Ingress, which would be a new pattern for this -# repo. The device is provisioned with the HOSTNAME, not the address, so the -# lease can move without re-provisioning hardware. +# repo. The device is provisioned with the HOSTNAME, not the address. +# +# The address is requested explicitly, matching every other LB service here. +# Left to allocate freely, Cilium IPAM handed out the first free address in +# the pool block (10.2.99.0-10.2.99.255) — which is 10.2.99.0, the base +# address of the range. Each LB address is advertised to the UDM as its own +# BGP route so that is routable in principle, but a .0 host address is the +# kind of thing constrained embedded IP stacks get wrong, and the TRMNL +# firmware is the single client that has to work here. Not worth the risk +# for an address nobody chose on purpose. +# +# .89 continues the existing sequence (.81-.88 are in use) and matches the +# fd99::89 that IPAM had already picked for v6. apiVersion: v1 kind: Service metadata: @@ -21,6 +32,7 @@ metadata: lb-pool: bgp annotations: external-dns.alpha.kubernetes.io/hostname: trmnl-device.renner.uno + lbipam.cilium.io/ips: "10.2.99.89,fd99::89" spec: type: LoadBalancer externalTrafficPolicy: Local diff --git a/kubernetes/apps/trmnl/terminus.yaml b/kubernetes/apps/trmnl/terminus.yaml index e7587a2c..339d7ee3 100644 --- a/kubernetes/apps/trmnl/terminus.yaml +++ b/kubernetes/apps/trmnl/terminus.yaml @@ -124,6 +124,16 @@ spec: envFrom: - configMapRef: name: terminus-config + # EXPECTED on a fresh deploy: this container loses the race against + # the web container's migration and exits 1 with + # `PG::UndefinedTable: relation "account" does not exist`, then + # comes up healthy on the restart once the schema exists. + # Observed on first deploy 2026-08-10; converged in ~1s. There is + # no upstream ordering hook, and gating it behind an initContainer + # that probes for a specific table would couple us to Terminus's + # schema. So this is left to Kubernetes' restart, deliberately — + # a single sidekiq restart shortly after a rollout is benign. More + # than a couple, or restarts long after a rollout, are NOT this. env: # No APP_SETUP here — see the web container. - name: DATABASE_URL