fix(trmnl): bound the renderer's /tmp with a sized emptyDir

Follow-up to the HOME=/tmp fix in this PR, from review.

Pointing HOME at /tmp was correct but left it on the container's
writable layer, which was harmless while Chromium crashed on launch and
wrote nothing. With the browser actually starting once a minute for the
life of the pod, crashpad state and profile data now accumulate there
with nothing to bound them — resources.limits sets memory only, so
there is no ephemeral-storage ceiling to catch it either.

Backs it with an emptyDir at a 512Mi sizeLimit, matching the /tmp mount
convention already used in pve-exporter-deployment.yaml. The point of
the limit is the failure mode, not the size: unbounded it degrades into
node disk pressure, which surfaces far from this pod and hurts whatever
else lands on that node; bounded it is an eviction of the renderer,
attributable on sight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.woodpecker/build-ci-images.yml

@@ -217,3 +217,32 @@ steps:
           --dockerfile Containerfile \
           --destination gitea.renner.uno/gitea_admin/alpine-openssl:$TAG \
           --destination gitea.renner.uno/gitea_admin/alpine-openssl:latest
+
+  # Heating CAN readers: open3e (Vitocal/VX3) + E3onCAN (E380 meter), one image
+  # run as two services on edge-1. Replaces pip-into-venv on the host, where an
+  # unpinned VCS requirement re-cloned and restarted the heating bridge on every
+  # Ansible converge. Not a ci-* image -- it is a runtime artifact -- but it
+  # builds the same way and lands in the same registry so the audit sees it.
+  - name: build-heating-can
+    image: gcr.io/kaniko-project/executor:v1.23.2-debug
+    directory: ci/images/heating-can
+    depends_on: []
+    when:
+      - event: manual
+      - event: push
+        path: ["ci/images/heating-can/**", ".woodpecker/build-ci-images.yml"]
+    environment:
+      GITEA_USER: gitea_admin
+      GITEA_TOKEN:
+        from_secret: ci_images_token
+    commands:
+      - mkdir -p /kaniko/.docker
+      - AUTH=$(printf '%s:%s' "$GITEA_USER" "$GITEA_TOKEN" | base64 | tr -d '\n')
+      - printf '{"auths":{"gitea.renner.uno":{"auth":"%s"}}}' "$AUTH" > /kaniko/.docker/config.json
+      - TAG=$(date +%Y-%m-%d)
+      - |
+        /kaniko/executor \
+          --context . \
+          --dockerfile Containerfile \
+          --destination gitea.renner.uno/gitea_admin/heating-can:$TAG \
+          --destination gitea.renner.uno/gitea_admin/heating-can:latest

ansible/docs/edge-host.md

@@ -39,6 +39,8 @@ Build and boot the dedicated installer image — see
    (inventory, plays, this file) and its address is never recorded. Do not
    pin it to a reservation.
 4. Add the host to `[edge_host]` in `inventory/pve_hosts.yml`, by name.
+   That group is the base: every edge thin client belongs to it, and it
+   assumes no peripherals.
 5. From the workstation:
    ```
    ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
@@ -49,6 +51,26 @@ Build and boot the dedicated installer image — see
    the point: a run that touches only the new box tells you nothing about
    the others.
 
+## Peripherals
+
+Each attached device is its own inventory group and its own playbook. Group
+membership states which hardware a host physically has -- there are no flags
+and no `when:` on the roles, so moving a stick between boxes is a one-line
+inventory change.
+
+| device | group | playbook |
+|---|---|---|
+| Innomaker USB2CAN (Vitocal/VX3 + E380) | `heating_can` | `plays/install-heating-can.yml` |
+| Nabu Casa ZWA-2 Z-Wave radio | `zwave_gateway` | `plays/install-zwave-gateway.yml` |
+| FTDI serial power-meter head | `power_meter` | `plays/install-power-meter.yml` |
+
+Add the host to the matching group, then run that playbook. They depend on
+the base play having run: all three need the local MQTT bridge, and two of
+them need docker.
+
+A host with nothing attached is simply absent from all three groups and needs
+none of these commands.
+
 ## Secrets (before first play run)
 
 Create `inventory/group_vars/edge_host/vault.yml` with ansible-vault; the
@@ -121,8 +143,12 @@ Afterwards:
   deliberate for sml2mqtt, acceptable for zwave-js). If a zwave-js bump
   changes its Alpine major, also update `edge_zwave_alpine_base` and refresh
   the preloaded `.so` (see "Logging & eMMC" → zwave-js).
-- open3e updates: re-run the play; the pip task tracks the git default
-  branch (pin `edge_open3e_pkg` to a tag if that ever bites).
+- open3e / E3onCAN updates: both are pinned to commits in
+  `roles/heating-can/defaults/main.yml`, so re-running the play does NOT pick
+  up upstream changes -- that is deliberate, these drive the heat pump. The
+  weekly update-audit reports when either falls behind (see
+  `ci/agents/update-audit/prompt.md`); bump the pin and re-run.
+  Pinning to a tag is not an option for open3e: it publishes none.
 - Ad-hoc CAN scans: `systemctl stop open3e` first (single-instance rule),
   restart afterwards.
 - can-utils (`candump`) is AUR-only on Arch — install manually via

ansible/inventory/group_vars/edge_host/vars.yml

@@ -36,3 +36,18 @@ keepalived_ntp_sysctl:
 
 # Realtek NIC on the Wyse 3040 has no hardware timestamping.
 chrony_hwtimestamp: false
+
+# --- Local MQTT contract -------------------------------------------------
+# Shared between the broker and its clients, so it belongs to the inventory
+# rather than to either side's role. mqtt-bridge listens here; heating-can and
+# power-meter-sml publish here; and the bridge authenticates to the cluster
+# broker AS the open3e user, so it needs that identity too.
+#
+# Keeping these in one role's defaults only worked while a single play loaded
+# every role at once. Splitting the plays by capability exposed it: the bridge
+# could not render without a heating-can default, and the CAN services could
+# not render without an mqtt-bridge default.
+edge_mqtt_host: 127.0.0.1
+edge_mqtt_port: 1883
+edge_open3e_mqtt_user: open3e
+edge_open3e_mqtt_topic: open3e

ansible/inventory/pve_hosts.yml

@@ -41,9 +41,36 @@ nas-1.renner.uno ansible_user=ladmin ansible_python_interpreter=/usr/bin/python3
 [grafana]
 grafana.srv.renner.uno ansible_user=ladmin
 
+# Every bare-metal edge thin client. Gets the host-generic roles only:
+# flash-wear handling, the log shipper, the MQTT bridge, the VLAN
+# sub-interface. Nothing here assumes any peripheral is attached.
 [edge_host]
 edge-1.iot.renner.uno ansible_user=root
 
+# Capability groups. Membership states which hardware a host physically has,
+# and each service playbook targets the group for its own concern. No flags and
+# no `when:` on the roles -- moving a stick to another box is a one-line change
+# here, which is what it should be.
+#
+# These are children of [edge_host], declared below: a peripheral is added to an
+# edge client, it does not replace the base. Making that a :children relation
+# rather than a comment means a host added only to a capability group still
+# resolves group_vars/edge_host -- where the local MQTT contract lives -- so it
+# cannot silently come up without edge_mqtt_host and friends.
+[zwave_gateway]
+edge-1.iot.renner.uno
+
+[heating_can]
+edge-1.iot.renner.uno
+
+[power_meter]
+edge-1.iot.renner.uno
+
+[edge_host:children]
+zwave_gateway
+heating_can
+power_meter
+
 # NTP servers. Includes edge-1 so the role survives the PVE hosts being
 # retired for bare-metal Talos, which cannot run chrony or keepalived itself.
 # See plays/install-ntp-ha.yml for why keepalived is scoped more narrowly

ansible/plays/install-blackbox-exporter.yml

@@ -1,19 +0,0 @@
----
-# Deploy Blackbox Exporter to Prometheus server for endpoint probing
-# Usage: ansible-playbook -i inventory/ plays/install-blackbox-exporter.yml
-
-- name: Install Blackbox Exporter
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  roles:
-    - blackbox-exporter
-
-  post_tasks:
-    - name: Verify blackbox_exporter is running
-      uri:
-        url: "http://127.0.0.1:9115/metrics"
-        status_code: 200
-      register: exporter_health
-      retries: 3
-      delay: 5
-      until: exporter_health.status == 200

ansible/plays/install-edge-host.yml

@@ -3,10 +3,18 @@
 # hardware-bound services, because Talos supports neither SocketCAN nor USB
 # passthrough.
 #
-# One role per concern. Previously all of this lived in a single `edge-host`
-# role, which meant a host could not take the flash-wear handling or the log
-# shipper without also being handed the Z-Wave radio and the CAN bus. That is
-# what blocked a second thin client from joining as an NTP peer.
+# This play is the BASE only: what every edge thin client gets regardless of
+# what is plugged into it. The peripherals live in their own playbooks against
+# their own inventory groups:
+#
+#   plays/install-zwave-gateway.yml   [zwave_gateway]   ZWA-2 radio
+#   plays/install-heating-can.yml     [heating_can]     Innomaker USB2CAN
+#   plays/install-power-meter.yml     [power_meter]     FTDI meter head
+#
+# Splitting the roles without splitting the plays would have kept the coupling:
+# one play applying every role to one group is the monolith again, a layer up.
+# A host with no peripherals is simply absent from those groups -- no flags, no
+# `when:` on the roles.
 #
 # Self-contained: `certificates` and `common` are the parts of plays/common.yml
 # that apply to an edge host, carried here so bootstrapping one is a single
@@ -14,76 +22,20 @@
 # metal, not VMs) and monitoring-agent (Debian-oriented; these hosts ship logs
 # via edge-log-shipper).
 #
-# Order is dependency order, not alphabetical:
-#   mqtt-bridge      before its clients, so they have somewhere to publish
-#   edge-flash-wear  before docker, so the journal is volatile before
-#                    containers start logging into it
-#   socketcan        before heating-can, which consumes can0
-- name: Install Arch edge host
+# Order is dependency order: edge-flash-wear before docker, so the journal is
+# volatile before containers start logging into it.
+- name: Install Arch edge host (base)
   hosts: edge_host
   # Connection is already root (ansible_user=root), so this is a root-to-root
   # no-op today. Kept explicit, matching plays/common.yml, so the roles below
   # do not silently depend on the connection user staying privileged.
   become: true
 
-  # Defined on the play, not taken from a role's defaults: these are used by
-  # pre_tasks, which must not depend on role default resolution.
-  #
-  # heating-can, zwave-gateway and power-meter-sml each declare edge_base_dir
-  # in their own defaults too, so they stay usable standalone. Play vars win,
-  # so these are the effective values -- keep them in step if the path moves.
-  vars:
-    edge_base_dir: /opt/edge-host
-    edge_legacy_compose: "{{ edge_base_dir }}/docker-compose.yml"
-
-  # One-time migration off the shared compose project, BEFORE any role tries
-  # to start a container.
-  #
-  # zwave-js and sml2mqtt used to live in one project at edge_legacy_compose,
-  # both with a fixed container_name. Docker enforces container names globally,
-  # not per project, so while the old project still owns those names the new
-  # per-service projects cannot create them -- `docker compose up -d` fails with
-  # "Conflict: container name already in use", which is not the missing-USB-device
-  # error the roles tolerate, so the play would abort.
-  #
-  # It belongs here rather than in either role: it is a migration step for both
-  # of them, and putting it in one meant whichever role ran first hit the
-  # conflict. Guarded on the legacy file, so it runs once and then never again.
-  #
-  # check mode cannot verify this: command tasks are skipped, not simulated, so
-  # a --check run neither hits the conflict nor proves the teardown works.
-  pre_tasks:
-    - name: Retire the legacy shared compose project
-      ansible.builtin.command:
-        cmd: docker compose -f "{{ edge_legacy_compose }}" down
-      args:
-        removes: "{{ edge_legacy_compose }}"
-      failed_when: false
-
-    - name: Remove the legacy shared compose file
-      ansible.builtin.file:
-        path: "{{ edge_legacy_compose }}"
-        state: absent
-
-    - name: Remove the legacy zwave-js env from the shared project root
-      ansible.builtin.file:
-        path: "{{ edge_base_dir }}/zwavejs.env"
-        state: absent
-
   roles:
     - certificates
     - common
-
-    # Host-generic. An edge host with no attached peripherals takes these and
-    # nothing below them.
     - edge-flash-wear
     - docker-install
     - edge-log-shipper
     - mqtt-bridge
     - edge-vlan
-
-    # Hardware-bound. Each needs its own device present.
-    - socketcan
-    - heating-can
-    - zwave-gateway
-    - power-meter-sml

ansible/plays/install-heating-can.yml

@@ -0,0 +1,18 @@
+---
+# Heating system over the Viessmann E3 CAN bus, for hosts with the Innomaker
+# USB2CAN adapter attached.
+#
+# Targets [heating_can]: membership means the adapter is physically present.
+# Two roles because they are two concerns -- the interface is host network
+# state, the services consume it -- but one group, because both need the same
+# hardware.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: these
+# services publish to the local MQTT bridge it sets up.
+- name: Install heating CAN services
+  hosts: heating_can
+  become: true
+
+  roles:
+    - socketcan
+    - heating-can

ansible/plays/install-log-aggregation.yml

@@ -1,86 +0,0 @@
----
-# Setup Log Aggregation Infrastructure
-# This playbook configures the complete logging stack:
-# 1. Loki server for log storage and querying
-# 2. Alloy agents on all VMs/CTs for log collection
-# 3. Syslog receiver for remote devices (Synology NAS, network devices)
-#
-# Usage: ansible-playbook -i inventory/ plays/setup-log-aggregation.yml
-#
-# To limit to specific hosts:
-#   --limit loki.srv.renner.uno          # Only configure Loki server
-#   --limit proxmox_all_running          # Only configure log agents
-
-- name: Setup Loki Log Aggregation Server
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
-
-- name: Deploy Monitoring Agent to Loki Server (with syslog receiver)
-  hosts: proxmox_logging
-  become: yes
-  gather_facts: yes
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-    # Syslog receiver settings are defined in host_vars/loki.srv.renner.uno/vars.yml:
-    # - alloy_syslog_receiver_enabled: true
-    # - alloy_syslog_receiver_port: 1514
-    # - alloy_syslog_receiver_protocol: "udp"
-    # - alloy_syslog_receiver_format: "raw"
-    # - alloy_stability_level: "experimental"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Display syslog receiver info
-      debug:
-        msg: |
-          Syslog receiver is listening on UDP port {{ alloy_syslog_receiver_port | default(1514) }}
-          Configure remote devices to send syslog to: {{ inventory_hostname }}:{{ alloy_syslog_receiver_port | default(1514) }}
-      when: alloy_syslog_receiver_enabled | default(false)
-
-- name: Deploy Monitoring Agent to All Running VMs/CTs
-  hosts: proxmox_all_running:!proxmox_logging:!proxmox_noansible
-  become: yes
-  gather_facts: yes
-  serial: 10  # Deploy to 10 hosts at a time to avoid overwhelming the network
-
-  vars:
-    loki_endpoint: "http://loki.srv.renner.uno:3100"
-    prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
-
-  roles:
-    - monitoring-agent
-
-  post_tasks:
-    - name: Verify Alloy is running
-      systemd:
-        name: alloy
-        state: started
-      register: alloy_service
-
-    - name: Display agent status
-      debug:
-        msg: "Monitoring agent deployed successfully on {{ inventory_hostname }}"
-      when: alloy_service is defined

ansible/plays/install-loki.yml

@@ -1,26 +0,0 @@
----
-# Install Loki Log Aggregation Server
-# Usage: ansible-playbook -i inventory/ plays/install-loki.yml
-
-- name: Install Loki Log Aggregation Server
-  hosts: loki.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  roles:
-    - common
-    - loki-server
-
-  post_tasks:
-    - name: Verify Loki is running
-      uri:
-        url: "http://127.0.0.1:3100/ready"
-        status_code: 200
-      register: loki_health
-      retries: 5
-      delay: 10
-      until: loki_health.status == 200
-
-    - name: Display Loki status
-      debug:
-        msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"

ansible/plays/install-power-meter.yml

@@ -0,0 +1,14 @@
+---
+# Power-meter reader (sml2mqtt) for hosts with the FTDI serial meter head
+# attached.
+#
+# Targets [power_meter]: membership means the meter head is physically present.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker and publishes to the local MQTT bridge.
+- name: Install power-meter reader
+  hosts: power_meter
+  become: true
+
+  roles:
+    - power-meter-sml

ansible/plays/install-prometheus.yml

@@ -1,43 +0,0 @@
----
-# Install Prometheus Metrics Server
-# Usage: ansible-playbook -i inventory/ plays/install-prometheus.yml
-
-- name: Install Prometheus Metrics Server
-  hosts: prometheus.srv.renner.uno
-  become: yes
-  gather_facts: yes
-
-  vars:
-    # Synology NAS SNMP targets
-    snmp_synology_targets:
-      - name: nas-1
-        target: "10.2.40.50"
-      - name: nas-2
-        target: "10.3.40.57"
-
-    # SNMPv3 credentials from vault
-    snmp_use_v3: true
-    snmp_v3_username: "{{ vault_snmp_v3_username }}"
-    snmp_v3_auth_password: "{{ vault_snmp_v3_auth_password }}"
-    snmp_v3_priv_password: "{{ vault_snmp_v3_priv_password }}"
-
-  roles:
-    - common
-    - prometheus-server
-    - snmp-exporter
-    - pve-exporter
-    - unifi-poller
-
-  post_tasks:
-    - name: Verify Prometheus is running
-      uri:
-        url: "http://127.0.0.1:9090/-/ready"
-        status_code: 200
-      register: prometheus_health
-      retries: 5
-      delay: 10
-      until: prometheus_health.status == 200
-
-    - name: Display Prometheus status
-      debug:
-        msg: "Prometheus is running and healthy at http://{{ inventory_hostname }}:9090"

ansible/plays/install-zwave-gateway.yml

@@ -0,0 +1,15 @@
+---
+# Z-Wave gateway (zwave-js-ui) for hosts with the ZWA-2 USB radio attached.
+#
+# Targets [zwave_gateway]: membership means the radio is physically present.
+# Moving the stick to another edge host is a one-line inventory change.
+#
+# Requires the base play (plays/install-edge-host.yml) to have run: it needs
+# docker, and the flash-wear handling that the libeatmydata preload exists to
+# complement.
+- name: Install Z-Wave gateway
+  hosts: zwave_gateway
+  become: true
+
+  roles:
+    - zwave-gateway

ansible/plays/update.yml

@@ -16,7 +16,6 @@
 - import_playbook: install-grist.yml
 - import_playbook: install-music-assistant.yml
 - import_playbook: update-docker.yml
-- import_playbook: install-log-aggregation.yml
 - import_playbook: update-pve-rolling.yml
 
 - name: Update Summary

ansible/roles/blackbox-exporter/defaults/main.yml

@@ -1,26 +0,0 @@
----
-blackbox_exporter_version: "0.25.0"
-blackbox_exporter_port: 9115
-blackbox_exporter_user: "blackbox_exporter"
-blackbox_exporter_group: "blackbox_exporter"
-
-# Endpoints to probe
-blackbox_http_targets:
-  - name: "gitea"
-    url: "https://gitea.renner.uno"
-  - name: "immich"
-    url: "https://immich.renner.uno"
-  - name: "authentik"
-    url: "https://auth.renner.uno"
-  - name: "home-assistant"
-    url: "https://ha.renner.uno"
-  - name: "grafana"
-    url: "https://grafana.renner.uno"
-  - name: "prometheus"
-    url: "https://prometheus.renner.uno"
-
-blackbox_dns_targets: []
-
-blackbox_tcp_targets:
-  - name: "postgres-ha"
-    target: "10.2.40.135:5432"

ansible/roles/blackbox-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/blackbox-exporter/tasks/main.yml

@@ -1,80 +0,0 @@
----
-- name: Create blackbox_exporter group
-  group:
-    name: "{{ blackbox_exporter_group }}"
-    system: yes
-    state: present
-
-- name: Create blackbox_exporter user
-  user:
-    name: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    createhome: no
-    state: present
-
-- name: Create blackbox_exporter directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0755'
-  loop:
-    - /etc/blackbox_exporter
-    - /var/lib/blackbox_exporter
-
-- name: Download blackbox_exporter
-  get_url:
-    url: "https://github.com/prometheus/blackbox_exporter/releases/download/v{{ blackbox_exporter_version }}/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64.tar.gz"
-    dest: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    mode: '0644'
-
-- name: Extract blackbox_exporter
-  unarchive:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    dest: /tmp
-    remote_src: yes
-    creates: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
-
-- name: Install blackbox_exporter binary
-  copy:
-    src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64/blackbox_exporter"
-    dest: /usr/local/bin/blackbox_exporter
-    owner: root
-    group: root
-    mode: '0755'
-    remote_src: yes
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter configuration
-  template:
-    src: blackbox.yml.j2
-    dest: /etc/blackbox_exporter/blackbox.yml
-    owner: "{{ blackbox_exporter_user }}"
-    group: "{{ blackbox_exporter_group }}"
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Deploy blackbox_exporter systemd service
-  template:
-    src: blackbox_exporter.service.j2
-    dest: /etc/systemd/system/blackbox_exporter.service
-    mode: '0644'
-  notify: Restart blackbox_exporter
-
-- name: Enable and start blackbox_exporter
-  systemd:
-    name: blackbox_exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
-    - "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"

ansible/roles/blackbox-exporter/templates/blackbox.yml.j2

@@ -1,60 +0,0 @@
-modules:
-  http_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
-      valid_status_codes: []  # Defaults to 2xx
-      method: GET
-      follow_redirects: true
-      fail_if_ssl: false
-      fail_if_not_ssl: false
-      tls_config:
-        insecure_skip_verify: false
-
-  http_post_2xx:
-    prober: http
-    timeout: 5s
-    http:
-      method: POST
-      valid_status_codes: []
-
-  tcp_connect:
-    prober: tcp
-    timeout: 5s
-
-  # DNS module for external domain resolution (tests upstream/cache)
-  dns_external:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "google.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for internal domain resolution (tests local resolution)
-  dns_internal:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "k8s-1.srv.renner.uno"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  # DNS module for common cached domain
-  dns_cached:
-    prober: dns
-    timeout: 5s
-    dns:
-      query_name: "cloudflare.com"
-      query_type: "A"
-      valid_rcodes:
-        - NOERROR
-
-  icmp:
-    prober: icmp
-    timeout: 5s
-    icmp:
-      preferred_ip_protocol: "ip4"

ansible/roles/blackbox-exporter/templates/blackbox_exporter.service.j2

@@ -1,22 +0,0 @@
-[Unit]
-Description=Prometheus Blackbox Exporter
-Documentation=https://github.com/prometheus/blackbox_exporter
-Wants=network-online.target
-After=network-online.target
-
-[Service]
-Type=simple
-User={{ blackbox_exporter_user }}
-Group={{ blackbox_exporter_group }}
-ExecStart=/usr/local/bin/blackbox_exporter \
-    --config.file=/etc/blackbox_exporter/blackbox.yml \
-    --web.listen-address=:{{ blackbox_exporter_port }}
-Restart=on-failure
-RestartSec=5s
-
-# Allow ICMP probing (requires CAP_NET_RAW)
-AmbientCapabilities=CAP_NET_RAW
-CapabilityBoundingSet=CAP_NET_RAW
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/heating-can/defaults/main.yml

@@ -6,7 +6,6 @@
 # Service root. Kept per-role so the role is usable on its own.
 edge_base_dir: /opt/edge-host
 
-edge_open3e_pkg: git+https://github.com/open3e/open3e.git
 edge_open3e_mqtt_user: open3e
 edge_open3e_mqtt_topic: open3e
 edge_open3e_mqtt_format: "{device}_{didNumber:04d}_{didName}"
@@ -37,16 +36,25 @@ edge_open3e_devices:
   vitocal: { tx: "0x680", dpList: "vcal" }
   # dpList als Pfad: Overlay-Datei mit den Batteriemodul-DIDs (siehe
   # files/open3e-vx3plus.py) — die Stock-vx3-Liste filtert sie sonst weg.
-  vx3: { tx: "0x6a1", dpList: "/opt/edge-host/open3e/vx3plus.py" }
+  vx3: { tx: "0x6a1", dpList: "/config/vx3plus.py" }
   vcmu: { tx: "0x68c" }
 
-# --- E3onCAN: passive E380 energy-meter listener (no bus writes, so it
-# coexists with the single open3e instance). The meter broadcasts at
-# ~5 Hz; the gap throttles MQTT to one update per DID per interval.
-edge_e3oncan_repo: https://github.com/MyHomeMyData/E3onCAN.git
-edge_e3oncan_version: 461dcbdf97e8d72d2edb9929c63b39382af6c3a1  # V0.5.2
 edge_e3oncan_gap_seconds: 10
 edge_e3oncan_mqtt_format: "{device}_{didNumber:04d}_{didName}"
 
 # Secrets come from inventory/group_vars/edge_host/vault.yml:
 #   edge_open3e_mqtt_password
+
+# The image carrying both readers, built and pinned in CI
+# (ci/images/heating-can/Containerfile). The source pins live in the
+# Containerfile, in the artifact -- not here -- so what is deployed is a tag,
+# and `docker compose up -d` on an unchanged tag is a genuine no-op.
+#
+# Tracked by the weekly update-audit as an ordinary image, which is why this is
+# simpler to keep current than the two git refs it replaced: one publishes no
+# tags at all, the other was pinned to an untagged commit between releases.
+heating_can_image: gitea.renner.uno/gitea_admin/heating-can:latest
+
+# Compose project directory. Separate from the old native layout under
+# {{ edge_base_dir }}/open3e, which this role removes.
+heating_can_base_dir: "{{ edge_base_dir }}/heating-can"

ansible/roles/heating-can/handlers/main.yml

@@ -1,12 +1,10 @@
 ---
-- name: Restart open3e
+- name: Reload systemd
   ansible.builtin.systemd:
-    name: open3e
-    state: restarted
     daemon_reload: true
 
-- name: Restart e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    state: restarted
-    daemon_reload: true
+- name: Restart heating-can
+  ansible.builtin.command:
+    cmd: docker compose up -d --force-recreate
+    chdir: "{{ heating_can_base_dir }}"
+  changed_when: true

ansible/roles/heating-can/tasks/main.yml

@@ -1,116 +1,164 @@
 ---
 # Heating system over the Viessmann E3 CAN bus.
 #
-# Two readers of the same heating installation:
+# Two readers of the same heating installation, supervised inside one container
+# from one image (ci/images/heating-can):
 #   open3e    UDS request/response to the Vitocal heat pump and Vitocharge VX3
 #   e3oncan   passive CANopen listener for the E380 grid energy meter, which
-#             feeds the heat pump s own energy management
+#             feeds the heat pump's own energy management
 #
-# They coexist because e3oncan issues no bus writes. Both are native installs
-# (venv + systemd) rather than containers: they need a SocketCAN interface,
-# which is host network state. CAN_RAW sockets need no capabilities, so they
-# run as unprivileged system users.
+# They coexist because e3oncan issues no bus writes.
 #
-# The can0 interface itself belongs to the socketcan role -- an interface is
-# not the same concern as the services that consume it -- so this role must
-# run after it.
+# Nothing is installed here. The image is built in CI with both sources pinned,
+# published to the registry, and this host only pulls and runs it. That is what
+# removes the churn: these were pip-installed into venvs on the node, and pip
+# re-cloned and rebuilt a VCS requirement on every converge -- restarting the
+# heating bridge each time -- which pinning the SHA did not fix.
+#
+# network_mode: host is required, not incidental: SocketCAN interfaces live in
+# the host network namespace, so can0 is only reachable that way. It is also how
+# the local MQTT broker on 127.0.0.1 is reached. CAN_RAW needs no capabilities,
+# so both run unprivileged.
+#
+# The can0 interface itself belongs to the socketcan role; this role consumes it.
 
-- name: Install build/runtime packages for the CAN services
-  ansible.builtin.package:
-    name:
-      - python
-      - python-packaging
-      - git
-    state: present
-
-- name: Create open3e system user
-  ansible.builtin.user:
-    name: open3e
-    system: true
-    create_home: false
-    shell: /usr/bin/nologin
-
-# Owned by the service user: udsoncan writes its log file into the
-# working directory.
-- name: Create open3e directory
+- name: Create heating-can directory
   ansible.builtin.file:
-    path: "{{ edge_base_dir }}/open3e"
+    path: "{{ heating_can_base_dir }}"
     state: directory
-    owner: open3e
-    group: open3e
     mode: "0755"
 
-- name: Install open3e into a venv
-  ansible.builtin.pip:
-    name: "{{ edge_open3e_pkg }}"
-    virtualenv: "{{ edge_base_dir }}/open3e/venv"
-    virtualenv_command: python -m venv
-  notify: Restart open3e
+# --- Migration off the native install ---------------------------------------
+# Must happen before the containers start. Two open3e instances issuing UDS
+# requests on one CAN bus is the failure mode to avoid, so the old units are
+# stopped and masked rather than merely disabled -- masked so a stray
+# `systemctl start` or a leftover dependency cannot bring them back alongside
+# the containers.
+- name: Find which native CAN units still exist
+  ansible.builtin.stat:
+    path: "/etc/systemd/system/{{ item }}.service"
+  loop:
+    - open3e
+    - e3oncan
+  register: heating_can_native_units
+
+# Stop and disable BEFORE anything else, and do NOT swallow the result.
+#
+# The first version did stop + disable + mask in one call with
+# failed_when: false, then deleted the unit files. Masking a unit whose file
+# still exists conflicts, so the call failed, failed_when hid it, nothing was
+# stopped -- and the next task deleted the unit files out from under two
+# running processes. systemctl is-enabled then reported "not-found" and looked
+# clean while is-active still said "active".
+#
+# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
+# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
+# kicking each other off the broker.
+#
+# Only units that actually exist are touched, so a fresh host does not need an
+# error swallowed to get through this.
+- name: Stop and disable the native CAN services
+  ansible.builtin.systemd:
+    name: "{{ item.item }}"
+    state: stopped
+    enabled: false
+  loop: "{{ heating_can_native_units.results }}"
+  loop_control:
+    label: "{{ item.item }}"
+  when: item.stat.exists
+
+# Masking is deliberately NOT used. It writes a symlink at the same path the
+# next task removes, so the two cancel out -- which is half of how the original
+# failed. With the unit files gone there is nothing left to start.
+- name: Remove the native unit files
+  ansible.builtin.file:
+    path: "/etc/systemd/system/{{ item }}.service"
+    state: absent
+  loop:
+    - open3e
+    - e3oncan
+  notify: Reload systemd
+
+# The invariant that actually matters is "no native process is on the CAN bus",
+# and systemd reporting success is not the same claim. Two open3e instances
+# polling one bus is the failure this whole block exists to prevent, so it is
+# asserted directly rather than inferred from the calls above.
+- name: Check for surviving native CAN processes
+  ansible.builtin.command:
+    argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
+  register: heating_can_survivors
+  changed_when: false
+  failed_when: false
+
+- name: Assert no native CAN process survived
+  ansible.builtin.assert:
+    that:
+      - heating_can_survivors.stdout | default('') | length == 0
+    fail_msg: >-
+      A native open3e/e3oncan process is still running after the units were
+      stopped and removed:
+      {{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
+      Starting the container now would put two readers on the same CAN bus --
+      two open3e instances issue competing UDS requests and take over each
+      other's MQTT session. Kill the survivors, then re-run.
+    success_msg: No native CAN processes remain; safe to start the container.
+
+- name: Remove the native venvs
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "{{ edge_base_dir }}/open3e/venv"
+    - "{{ edge_base_dir }}/e3oncan/venv"
+    - "{{ edge_base_dir }}/e3oncan/src"
+    - "{{ edge_base_dir }}/open3e"
+    - "{{ edge_base_dir }}/e3oncan"
+
+# --- Configuration (bind-mounted into the containers) ------------------------
+- name: Deploy open3e devices.json
+  ansible.builtin.copy:
+    content: "{{ edge_open3e_devices | to_nice_json }}\n"
+    dest: "{{ heating_can_base_dir }}/devices.json"
+    mode: "0644"
+  notify: Restart heating-can
 
 - name: Deploy open3e vx3 datapoint overlay (battery-module DIDs)
   ansible.builtin.copy:
     src: open3e-vx3plus.py
-    dest: "{{ edge_base_dir }}/open3e/vx3plus.py"
+    dest: "{{ heating_can_base_dir }}/vx3plus.py"
     mode: "0644"
-  notify: Restart open3e
+  notify: Restart heating-can
 
-- name: Deploy open3e devices.json
-  ansible.builtin.copy:
-    content: "{{ edge_open3e_devices | to_nice_json }}\n"
-    dest: "{{ edge_base_dir }}/open3e/devices.json"
-    mode: "0644"
-  notify: Restart open3e
-
-- name: Deploy open3e environment (broker credential)
+# --- Compose project ---------------------------------------------------------
+- name: Deploy the broker credential env file
   ansible.builtin.template:
-    src: open3e.env.j2
-    dest: "{{ edge_base_dir }}/open3e/open3e.env"
-    owner: open3e
+    src: heating-can.env.j2
+    dest: "{{ heating_can_base_dir }}/heating-can.env"
     mode: "0600"
-  notify: Restart open3e
+  notify: Restart heating-can
 
-- name: Deploy open3e systemd unit
+- name: Deploy heating-can compose project
   ansible.builtin.template:
-    src: open3e.service.j2
-    dest: /etc/systemd/system/open3e.service
+    src: docker-compose.yml.j2
+    dest: "{{ heating_can_base_dir }}/docker-compose.yml"
     mode: "0644"
-  notify: Restart open3e
+  notify: Restart heating-can
 
-- name: Enable open3e
-  ansible.builtin.systemd:
-    name: open3e
-    enabled: true
-    state: started
-    daemon_reload: true
+- name: Pull the heating-can image
+  ansible.builtin.command:
+    cmd: docker compose pull
+    chdir: "{{ heating_can_base_dir }}"
+  changed_when: false
 
-
-# --- E3onCAN: direct E380 meter reads (passive; per-review decision not
-# to trust the VX3's relayed meter values). Same broker user as open3e;
-# topics land under the same open3e/ prefix as e380_<did>_<name>.
-- name: Check out E3onCAN (pinned)
-  ansible.builtin.git:
-    repo: "{{ edge_e3oncan_repo }}"
-    dest: "{{ edge_base_dir }}/e3oncan/src"
-    version: "{{ edge_e3oncan_version }}"
-  notify: Restart e3oncan
-
-- name: Create E3onCAN venv with requirements
-  ansible.builtin.pip:
-    requirements: "{{ edge_base_dir }}/e3oncan/src/requirements.txt"
-    virtualenv: "{{ edge_base_dir }}/e3oncan/venv"
-    virtualenv_command: python -m venv
-  notify: Restart e3oncan
-
-- name: Deploy e3oncan systemd unit
-  ansible.builtin.template:
-    src: e3oncan.service.j2
-    dest: /etc/systemd/system/e3oncan.service
-    mode: "0644"
-  notify: Restart e3oncan
-
-- name: Enable e3oncan
-  ansible.builtin.systemd:
-    name: e3oncan
-    enabled: true
-    state: started
-    daemon_reload: true
+# Unlike the zwave and meter services there is no missing-device tolerance here:
+# can0 is created by the socketcan role from a USB adapter that is part of this
+# host's purpose. If it is absent the heating telemetry is down and the run
+# should say so rather than pass quietly.
+- name: Start heating-can services
+  ansible.builtin.command:
+    cmd: docker compose up -d
+    chdir: "{{ heating_can_base_dir }}"
+  register: heating_can_up
+  changed_when: >-
+    'Started' in heating_can_up.stderr or 'Created' in heating_can_up.stderr
+    or 'Started' in heating_can_up.stdout or 'Creating' in heating_can_up.stdout

ansible/roles/heating-can/templates/docker-compose.yml.j2

@@ -0,0 +1,35 @@
+# Managed by ansible (roles/heating-can).
+#
+# One container. Both CAN readers live in the image and are supervised inside
+# it -- this file supplies configuration and nothing else. It does not know
+# open3e's or E3onCAN's command line.
+#
+# network_mode: host is required: SocketCAN interfaces live in the host network
+# namespace, so {{ edge_can_interface }} is only reachable this way, and it is
+# also how the local MQTT broker is reached. CAN_RAW needs no capabilities, so
+# the image runs unprivileged.
+services:
+  heating-can:
+    image: {{ heating_can_image }}
+    container_name: heating-can
+    restart: unless-stopped
+    network_mode: host
+    volumes:
+      - ./devices.json:/config/devices.json:ro
+      - ./vx3plus.py:/config/vx3plus.py:ro
+    env_file:
+      # Broker credential, 0600 -- see heating-can.env.j2.
+      - heating-can.env
+    environment:
+      CAN_INTERFACE: "{{ edge_can_interface }}"
+      MQTT_HOST: "{{ edge_mqtt_host }}"
+      MQTT_PORT: "{{ edge_mqtt_port }}"
+      MQTT_TOPIC: "{{ edge_open3e_mqtt_topic }}"
+      MQTT_FORMAT: "{{ edge_open3e_mqtt_format }}"
+      E3ONCAN_MQTT_FORMAT: "{{ edge_e3oncan_mqtt_format }}"
+      OPEN3E_CONFIG: /config/devices.json
+      OPEN3E_CLIENT_ID: open3e-edge
+      OPEN3E_POLL_SECONDS: "{{ edge_open3e_poll_seconds }}"
+      OPEN3E_DATAPOINTS: "{{ edge_open3e_datapoints }}"
+      E3ONCAN_DEVICE: e380
+      E3ONCAN_GAP_SECONDS: "{{ edge_e3oncan_gap_seconds }}"

ansible/roles/heating-can/templates/e3oncan.service.j2

@@ -1,32 +0,0 @@
-# Managed by ansible (roles/heating-can).
-# PASSIVE listener for the E380 energy meter's CANopen broadcasts — no
-# writes on the bus, so it may run alongside the (single) open3e
-# instance. Reads the meter directly instead of trusting the VX3's
-# relayed values.
-[Unit]
-Description=E3onCAN E380 energy-meter CAN listener (read-only)
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker e3oncan publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/e3oncan
-# Journald-friendly (python block-buffers pipes; lost on restarts).
-Environment=PYTHONUNBUFFERED=1
-ExecStart={{ edge_base_dir }}/e3oncan/venv/bin/python {{ edge_base_dir }}/e3oncan/src/E3onCANcollect.py \
-  -c {{ edge_can_interface }} \
-  -dev e380 \
-  -g {{ edge_e3oncan_gap_seconds }} \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser {{ edge_open3e_mqtt_user }} \
-  -mpass ${OPEN3E_MQTT_PASSWORD} \
-  -mfstr "{{ edge_e3oncan_mqtt_format }}" \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/heating-can/templates/heating-can.env.j2

@@ -0,0 +1,8 @@
+# Managed by ansible (roles/heating-can). 0600 -- carries the broker credential.
+#
+# Kept out of docker-compose.yml, which is 0644: the sibling roles on this host
+# do the same (power-meter-sml renders its config at 0600 while its compose file
+# stays world-readable), and the native unit this replaced used a 0600 env file
+# for exactly this value.
+MQTT_USER={{ edge_open3e_mqtt_user }}
+MQTT_PASSWORD={{ edge_open3e_mqtt_password }}

ansible/roles/heating-can/templates/open3e.env.j2

@@ -1,7 +0,0 @@
-# Managed by ansible (roles/heating-can). Read by open3e.service and
-# e3oncan.service.
-# Full user:password pair as one variable so the open3e unit can pass it
-# as a single -muser argument without exposing it in the unit file;
-# e3oncan takes user and password separately (-muser/-mpass).
-OPEN3E_MUSER={{ edge_open3e_mqtt_user }}:{{ edge_open3e_mqtt_password }}
-OPEN3E_MQTT_PASSWORD={{ edge_open3e_mqtt_password }}

ansible/roles/heating-can/templates/open3e.service.j2

@@ -1,39 +0,0 @@
-# Managed by ansible (roles/heating-can).
-# Read-mostly listener — no -w flag, no service 0x77. The ONE sanctioned
-# write path is the MQTT command topic (-l): HA publishes the DHW
-# one-time-charge command there (scope decision 2026-07-02, see vican
-# CLAUDE.md). Anything beyond that single write gets its own review.
-# -j is load-bearing: without it open3e flattens JSON payloads into
-# per-key subtopics and the HA package (homeassistant repo,
-# packages/open3e.yaml) receives nothing.
-# Exactly ONE open3e instance may be on the bus at a time (CAN collisions
-# stop open3e) — stop this service before running ad-hoc scans.
-[Unit]
-Description=open3e Viessmann E3 CAN-to-MQTT bridge (read-only)
-# Tie the service to the CAN device: stops on unplug, starts on plug via
-# the device unit rather than polling.
-BindsTo=sys-subsystem-net-devices-{{ edge_can_interface }}.device
-# mosquitto.service = the local MQTT bridge broker open3e publishes to (loopback).
-After=sys-subsystem-net-devices-{{ edge_can_interface }}.device network-online.target mosquitto.service
-Wants=network-online.target mosquitto.service
-
-[Service]
-User=open3e
-EnvironmentFile={{ edge_base_dir }}/open3e/open3e.env
-WorkingDirectory={{ edge_base_dir }}/open3e
-ExecStart={{ edge_base_dir }}/open3e/venv/bin/open3e \
-  -c {{ edge_can_interface }} \
-  -cnfg {{ edge_base_dir }}/open3e/devices.json \
-  -m {{ edge_mqtt_host }}:{{ edge_mqtt_port }}:{{ edge_open3e_mqtt_topic }} \
-  -muser ${OPEN3E_MUSER} \
-  -mcid open3e-edge \
-  -mfstr "{{ edge_open3e_mqtt_format }}" \
-  -r {{ edge_open3e_datapoints }} \
-  -t {{ edge_open3e_poll_seconds }} \
-  -l {{ edge_open3e_mqtt_topic }}/cmnd \
-  -j
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/loki-server/defaults/main.yml

@@ -1,44 +0,0 @@
----
-# Loki Server Configuration
-
-# Version and download
-loki_version: "3.4.2"
-loki_arch: "linux-amd64"
-loki_download_url: "https://github.com/grafana/loki/releases/download/v{{ loki_version }}/loki-{{ loki_arch }}.zip"
-loki_bin_path: "/usr/local/bin/loki"
-
-# Directories
-loki_data_dir: "/var/lib/loki"
-loki_config_dir: "/etc/loki"
-loki_config_file: "{{ loki_config_dir }}/loki-config.yaml"
-
-# Server configuration
-loki_http_listen_port: 3100
-loki_grpc_listen_port: 9096
-loki_http_listen_address: "0.0.0.0"
-
-# Retention settings (90 days)
-loki_retention_period: "2160h"  # 90 days
-loki_retention_delete_delay: "2h"
-
-# Storage configuration
-loki_storage_type: "filesystem"
-loki_chunks_directory: "{{ loki_data_dir }}/chunks"
-loki_rules_directory: "{{ loki_data_dir }}/rules"
-
-# Index configuration
-loki_index_prefix: "loki_index_"
-loki_index_period: "24h"
-
-# Limits
-loki_ingestion_rate_mb: 10
-loki_ingestion_burst_size_mb: 20
-loki_max_streams_per_user: 10000
-loki_max_entries_limit_per_query: 5000
-
-# Query settings
-loki_max_query_length: "720h"  # 30 days max query span
-
-# Systemd service
-loki_user: "loki"
-loki_group: "loki"

ansible/roles/loki-server/handlers/main.yml

@@ -1,10 +0,0 @@
----
-- name: Restart loki
-  systemd:
-    name: loki
-    state: restarted
-    daemon_reload: yes
-
-- name: Reload systemd
-  systemd:
-    daemon_reload: yes

ansible/roles/loki-server/tasks/main.yml

@@ -1,95 +0,0 @@
----
-# Loki Server Installation and Configuration
-
-- name: Install dependencies
-  apt:
-    name:
-      - unzip
-      - curl
-    state: present
-    update_cache: yes
-
-- name: Create loki user
-  user:
-    name: "{{ loki_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ loki_user }}"
-    group: "{{ loki_group }}"
-    mode: '0755'
-  loop:
-    - "{{ loki_config_dir }}"
-    - "{{ loki_data_dir }}"
-    - "{{ loki_chunks_directory }}"
-    - "{{ loki_rules_directory }}"
-    - "{{ loki_data_dir }}/index"
-    - "{{ loki_data_dir }}/index_cache"
-    - "{{ loki_data_dir }}/compactor"
-
-- name: Download Loki
-  get_url:
-    url: "{{ loki_download_url }}"
-    dest: "/tmp/loki.zip"
-    mode: '0644'
-
-- name: Unzip Loki
-  unarchive:
-    src: "/tmp/loki.zip"
-    dest: "/tmp"
-    remote_src: yes
-
-- name: Install Loki binary
-  copy:
-    src: "/tmp/loki-{{ loki_arch }}"
-    dest: "{{ loki_bin_path }}"
-    mode: '0755'
-    remote_src: yes
-  notify: Restart loki
-
-- name: Create Loki configuration
-  template:
-    src: loki-config.yaml.j2
-    dest: "{{ loki_config_file }}"
-    owner: "{{ loki_user }}"
-    group: "{{ loki_group }}"
-    mode: '0644'
-  notify: Restart loki
-
-- name: Create Loki systemd service
-  template:
-    src: loki.service.j2
-    dest: /etc/systemd/system/loki.service
-    mode: '0644'
-  notify:
-    - Reload systemd
-    - Restart loki
-
-- name: Enable and start Loki service
-  systemd:
-    name: loki
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Wait for Loki to be ready
-  uri:
-    url: "http://127.0.0.1:{{ loki_http_listen_port }}/ready"
-    status_code: 200
-  register: result
-  until: result.status == 200
-  retries: 30
-  delay: 5
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/loki.zip"
-    - "/tmp/loki-{{ loki_arch }}"

ansible/roles/loki-server/templates/loki-config.yaml.j2

@@ -1,72 +0,0 @@
-# Loki Configuration
-# Generated by Ansible
-
-auth_enabled: false
-
-server:
-  http_listen_address: {{ loki_http_listen_address }}
-  http_listen_port: {{ loki_http_listen_port }}
-  grpc_listen_port: {{ loki_grpc_listen_port }}
-  log_level: info
-
-common:
-  instance_addr: 127.0.0.1
-  path_prefix: {{ loki_data_dir }}
-  storage:
-    filesystem:
-      chunks_directory: {{ loki_chunks_directory }}
-      rules_directory: {{ loki_rules_directory }}
-  replication_factor: 1
-  ring:
-    kvstore:
-      store: inmemory
-
-query_range:
-  results_cache:
-    cache:
-      embedded_cache:
-        enabled: true
-        max_size_mb: 100
-
-schema_config:
-  configs:
-    - from: 2024-01-01
-      store: tsdb
-      object_store: filesystem
-      schema: v13
-      index:
-        prefix: {{ loki_index_prefix }}
-        period: {{ loki_index_period }}
-
-storage_config:
-  tsdb_shipper:
-    active_index_directory: {{ loki_data_dir }}/index
-    cache_location: {{ loki_data_dir }}/index_cache
-
-compactor:
-  working_directory: {{ loki_data_dir }}/compactor
-  compaction_interval: 10m
-  retention_enabled: true
-  retention_delete_delay: {{ loki_retention_delete_delay }}
-  retention_delete_worker_count: 150
-  delete_request_store: filesystem
-
-limits_config:
-  retention_period: {{ loki_retention_period }}
-  ingestion_rate_mb: {{ loki_ingestion_rate_mb }}
-  ingestion_burst_size_mb: {{ loki_ingestion_burst_size_mb }}
-  max_streams_per_user: {{ loki_max_streams_per_user }}
-  max_entries_limit_per_query: {{ loki_max_entries_limit_per_query }}
-  max_query_length: {{ loki_max_query_length }}
-  reject_old_samples: true
-  reject_old_samples_max_age: 168h  # 7 days
-
-ruler:
-  alertmanager_url: ""
-  storage:
-    type: local
-    local:
-      directory: {{ loki_rules_directory }}
-
-analytics:
-  reporting_enabled: false

ansible/roles/loki-server/templates/loki.service.j2

@@ -1,25 +0,0 @@
-[Unit]
-Description=Loki Log Aggregation System
-Documentation=https://grafana.com/docs/loki/latest/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ loki_user }}
-Group={{ loki_group }}
-ExecStart={{ loki_bin_path }} -config.file={{ loki_config_file }}
-ExecReload=/bin/kill -HUP $MAINPID
-Restart=on-failure
-RestartSec=10s
-LimitNOFILE=65536
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadWritePaths={{ loki_data_dir }}
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/mqtt-bridge/defaults/main.yml

@@ -5,8 +5,6 @@
 # — off the untrusted IoT VLAN — and buffers telemetry in RAM during cluster
 # blips. The local broker writes nothing to the eMMC (persistence off, logs to
 # the volatile journal). See edge-mqtt-bridge.conf.j2 and README "MQTT bridge".
-edge_mqtt_host: 127.0.0.1
-edge_mqtt_port: 1883
 # Upstream cluster broker the local bridge connects to (TLS; the public
 # Let's Encrypt cert is validated against the system CA bundle).
 edge_mqtt_cluster_host: mqtt.renner.uno

ansible/roles/prometheus-server/defaults/main.yml

@@ -1,48 +0,0 @@
----
-# Prometheus Server Configuration
-
-# Version and download
-prometheus_version: "3.2.1"
-prometheus_arch: "linux-amd64"
-prometheus_download_url: "https://github.com/prometheus/prometheus/releases/download/v{{ prometheus_version }}/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}.tar.gz"
-prometheus_bin_path: "/usr/local/bin/prometheus"
-prometheus_promtool_path: "/usr/local/bin/promtool"
-
-# Directories
-prometheus_data_dir: "/var/lib/prometheus"
-prometheus_config_dir: "/etc/prometheus"
-prometheus_config_file: "{{ prometheus_config_dir }}/prometheus.yml"
-prometheus_rules_dir: "{{ prometheus_config_dir }}/rules"
-prometheus_targets_dir: "{{ prometheus_config_dir }}/targets"
-
-# Server configuration
-prometheus_listen_address: "0.0.0.0"
-prometheus_listen_port: 9090
-prometheus_web_external_url: ""
-
-# Retention settings (90 days)
-prometheus_retention_time: "90d"
-prometheus_retention_size: "30GB"
-
-# Scrape settings
-prometheus_scrape_interval: "30s"
-prometheus_evaluation_interval: "30s"
-prometheus_scrape_timeout: "10s"
-
-# Remote write (for Alloy agents)
-prometheus_remote_write_enabled: true
-
-# Systemd service
-prometheus_user: "prometheus"
-prometheus_group: "prometheus"
-
-# Alert notification endpoints
-prometheus_alertmanager_url: ""
-
-# Email alerting (via Grafana)
-alerting_email_enabled: true
-alerting_email_to: ""
-
-# Home Assistant webhook for alerts
-alerting_ha_webhook_enabled: true
-alerting_ha_webhook_url: ""

ansible/roles/prometheus-server/handlers/main.yml

@@ -1,15 +0,0 @@
----
-- name: Restart prometheus
-  systemd:
-    name: prometheus
-    state: restarted
-    daemon_reload: yes
-
-- name: Reload prometheus
-  systemd:
-    name: prometheus
-    state: reloaded
-
-- name: Reload systemd
-  systemd:
-    daemon_reload: yes

ansible/roles/prometheus-server/tasks/main.yml

@@ -1,129 +0,0 @@
----
-# Prometheus Server Installation and Configuration
-
-- name: Install dependencies
-  apt:
-    name:
-      - curl
-    state: present
-    update_cache: yes
-
-- name: Create prometheus user
-  user:
-    name: "{{ prometheus_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0755'
-  loop:
-    - "{{ prometheus_config_dir }}"
-    - "{{ prometheus_data_dir }}"
-    - "{{ prometheus_rules_dir }}"
-    - "{{ prometheus_targets_dir }}"
-
-- name: Download Prometheus
-  get_url:
-    url: "{{ prometheus_download_url }}"
-    dest: "/tmp/prometheus.tar.gz"
-    mode: '0644'
-
-- name: Extract Prometheus
-  unarchive:
-    src: "/tmp/prometheus.tar.gz"
-    dest: "/tmp"
-    remote_src: yes
-
-- name: Install Prometheus binaries
-  copy:
-    src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item.src }}"
-    dest: "{{ item.dest }}"
-    mode: '0755'
-    remote_src: yes
-  loop:
-    - { src: "prometheus", dest: "{{ prometheus_bin_path }}" }
-    - { src: "promtool", dest: "{{ prometheus_promtool_path }}" }
-  notify: Restart prometheus
-
-- name: Check for console templates
-  stat:
-    path: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/consoles"
-  register: consoles_dir
-
-- name: Copy console templates
-  copy:
-    src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item }}/"
-    dest: "{{ prometheus_config_dir }}/{{ item }}/"
-    remote_src: yes
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-  loop:
-    - consoles
-    - console_libraries
-  when: consoles_dir.stat.exists
-
-- name: Create Prometheus configuration
-  template:
-    src: prometheus.yml.j2
-    dest: "{{ prometheus_config_file }}"
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0644'
-    validate: "{{ prometheus_promtool_path }} check config %s"
-  notify: Restart prometheus
-
-- name: Create alert rules
-  template:
-    src: alert-rules.yml.j2
-    dest: "{{ prometheus_rules_dir }}/alert-rules.yml"
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0644'
-  notify: Reload prometheus
-
-- name: Create node exporter targets file
-  template:
-    src: nodes.yml.j2
-    dest: "{{ prometheus_targets_dir }}/nodes.yml"
-    owner: "{{ prometheus_user }}"
-    group: "{{ prometheus_group }}"
-    mode: '0644'
-
-- name: Create Prometheus systemd service
-  template:
-    src: prometheus.service.j2
-    dest: /etc/systemd/system/prometheus.service
-    mode: '0644'
-  notify:
-    - Reload systemd
-    - Restart prometheus
-
-- name: Enable and start Prometheus service
-  systemd:
-    name: prometheus
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Wait for Prometheus to be ready
-  uri:
-    url: "http://127.0.0.1:{{ prometheus_listen_port }}/-/ready"
-    status_code: 200
-  register: result
-  until: result.status == 200
-  retries: 30
-  delay: 5
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/prometheus.tar.gz"
-    - "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}"

ansible/roles/prometheus-server/templates/alert-rules.yml.j2

@@ -1,100 +0,0 @@
-# Prometheus Alert Rules
-# Generated by Ansible
-
-groups:
-  - name: infrastructure
-    rules:
-      # Disk usage alerts
-      - alert: HighDiskUsage
-        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 15
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Disk usage is above 85% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
-
-      - alert: CriticalDiskUsage
-        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 5
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Critical disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Disk usage is above 95% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
-
-      # Memory alerts
-      - alert: HighMemoryUsage
-        expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 > 90
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High memory usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Memory usage is above 90% ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
-
-      # CPU alerts
-      - alert: HighCPUUsage
-        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
-        for: 10m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High CPU usage on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "CPU usage is above 85% for 10 minutes ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
-
-  - name: services
-    rules:
-      # Service down alerts
-      - alert: ServiceDown
-        expr: up == 0
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Service {{ '{{' }} $labels.job {{ '}}' }} down on {{ '{{' }} $labels.instance {{ '}}' }}"
-          description: "Service has been unreachable for more than 2 minutes"
-
-      # DNS VIP health
-      - alert: DNSVIPDown
-        expr: absent(up{job="dns-servers"} == 1) or count(up{job="dns-servers"} == 1) < 2
-        for: 1m
-        labels:
-          severity: critical
-        annotations:
-          summary: "DNS cluster degraded"
-          description: "Less than 2 DNS servers are healthy"
-
-  - name: security
-    rules:
-      # High auth failure rate (placeholder - needs log-based metrics from Loki)
-      - alert: HighAuthFailureRate
-        expr: rate(authentik_login_failed_total[5m]) > 5
-        for: 2m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High authentication failure rate"
-          description: "More than 5 auth failures per minute detected"
-
-  - name: loki
-    rules:
-      # Loki health
-      - alert: LokiDown
-        expr: up{job="loki"} == 0
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Loki log aggregation is down"
-          description: "Loki has been unreachable for more than 2 minutes"
-
-      # Loki ingestion rate
-      - alert: LokiHighIngestionRate
-        expr: rate(loki_distributor_bytes_received_total[5m]) > 10485760
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "High log ingestion rate in Loki"
-          description: "Log ingestion rate is above 10MB/s"

ansible/roles/prometheus-server/templates/nodes.yml.j2

@@ -1,26 +0,0 @@
-# Node Exporter targets - auto-generated by Ansible
-# Updates require re-running the prometheus playbook
-#
-# NOTE: Most hosts have Alloy agents that push metrics via remote_write.
-# This file only includes hosts that need direct scraping (standalone node_exporter).
-# Alloy-managed hosts are excluded to avoid duplicate "down" targets.
-
-# PVE cluster nodes - need direct scraping (no Alloy agent)
-- targets:
-    - 'jupiter.renner.uno:9100'
-  labels:
-    job: 'node'
-    group: 'pve_cluster'
-    hostname: 'jupiter'
-- targets:
-    - 'neptune.renner.uno:9100'
-  labels:
-    job: 'node'
-    group: 'pve_cluster'
-    hostname: 'neptune'
-- targets:
-    - 'saturn.renner.uno:9100'
-  labels:
-    job: 'node'
-    group: 'pve_cluster'
-    hostname: 'saturn'

ansible/roles/prometheus-server/templates/prometheus.service.j2

@@ -1,38 +0,0 @@
-[Unit]
-Description=Prometheus Monitoring System
-Documentation=https://prometheus.io/docs/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ prometheus_user }}
-Group={{ prometheus_group }}
-ExecStart={{ prometheus_bin_path }} \
-  --config.file={{ prometheus_config_file }} \
-  --storage.tsdb.path={{ prometheus_data_dir }} \
-  --storage.tsdb.retention.time={{ prometheus_retention_time }} \
-  --storage.tsdb.retention.size={{ prometheus_retention_size }} \
-  --web.listen-address={{ prometheus_listen_address }}:{{ prometheus_listen_port }} \
-{% if prometheus_remote_write_enabled %}
-  --web.enable-remote-write-receiver \
-{% endif %}
-{% if prometheus_web_external_url %}
-  --web.external-url={{ prometheus_web_external_url }} \
-{% endif %}
-  --web.enable-lifecycle
-
-ExecReload=/bin/kill -HUP $MAINPID
-Restart=on-failure
-RestartSec=10s
-LimitNOFILE=65536
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadWritePaths={{ prometheus_data_dir }}
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/prometheus-server/templates/prometheus.yml.j2

@@ -1,304 +0,0 @@
-# Prometheus Configuration
-# Generated by Ansible
-
-global:
-  scrape_interval: {{ prometheus_scrape_interval }}
-  evaluation_interval: {{ prometheus_evaluation_interval }}
-  scrape_timeout: {{ prometheus_scrape_timeout }}
-  external_labels:
-    cluster: homelab
-    datacenter: renner
-
-rule_files:
-  - "{{ prometheus_rules_dir }}/*.yml"
-
-{% if prometheus_alertmanager_url %}
-alerting:
-  alertmanagers:
-    - static_configs:
-        - targets:
-            - {{ prometheus_alertmanager_url }}
-{% endif %}
-
-scrape_configs:
-  # Prometheus self-monitoring
-  - job_name: 'prometheus'
-    static_configs:
-      - targets: ['localhost:{{ prometheus_listen_port }}']
-
-  # Loki metrics
-  - job_name: 'loki'
-    static_configs:
-      - targets: ['loki.srv.renner.uno:3100']
-
-  # File-based service discovery for dynamic targets
-  - job_name: 'alloy-nodes'
-    file_sd_configs:
-      - files:
-          - '{{ prometheus_targets_dir }}/*.yml'
-          - '{{ prometheus_targets_dir }}/*.json'
-        refresh_interval: 30s
-
-{% if prometheus_remote_write_enabled %}
-# Remote write receiver for Alloy agents
-# Agents push metrics to /api/v1/write
-{% endif %}
-
-# Static targets for infrastructure components
-  # Proxmox VE API metrics (via pve-exporter)
-  - job_name: 'pve'
-    scrape_interval: 60s
-    scrape_timeout: 55s
-    static_configs:
-      - targets:
-          - 'jupiter.renner.uno'
-          - 'neptune.renner.uno'
-          - 'saturn.renner.uno'
-    metrics_path: /pve
-    params:
-      module: [default]
-      cluster: ['1']
-      node: ['1']
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9221  # pve-exporter address
-      - target_label: group
-        replacement: proxmox
-      - target_label: type
-        replacement: hypervisor
-
-  # RustFS (S3-compatible storage)
-  # NOTE: RustFS uses OpenTelemetry push-model via --obs-endpoint, not Prometheus pull.
-  # Unlike MinIO, it doesn't expose /minio/v2/metrics/cluster endpoint.
-  # Metrics collection would require setting up RUSTFS_OBS_ENDPOINT to push to an OTEL collector.
-  # For now, node-level metrics are available via Alloy agents on the RustFS hosts.
-
-  # Home Assistant metrics
-  - job_name: 'homeassistant'
-    metrics_path: /api/prometheus
-    bearer_token: '{{ homeassistant_prometheus_token }}'
-    scheme: http
-    static_configs:
-      - targets: ['homeassistant.iot.renner.uno:8123']
-        labels:
-          group: 'smarthome'
-          type: 'homeassistant'
-
-  # UniFi Poller metrics
-  - job_name: 'unifi'
-    static_configs:
-      - targets: ['localhost:9130']
-        labels:
-          group: 'network'
-          type: 'unifi'
-
-  # Caddy reverse proxy metrics (HA cluster)
-  - job_name: 'caddy'
-    static_configs:
-      - targets:
-          - 'caddy.dmz.renner.uno:2019'
-          - 'caddy2.dmz.renner.uno:2019'
-        labels:
-          group: 'ingress'
-          type: 'reverse-proxy'
-
-  # PostgreSQL metrics (via postgres_exporter)
-  - job_name: 'postgresql'
-    static_configs:
-      - targets: ['localhost:9187']
-        labels:
-          group: 'database'
-          type: 'postgresql'
-
-  # Authentik identity provider metrics
-  - job_name: 'authentik'
-    static_configs:
-      - targets:
-          - 'authentik-app-1.srv.renner.uno:9300'
-          - 'authentik-app-2.srv.renner.uno:9300'
-        labels:
-          group: 'identity'
-          type: 'authentik'
-
-  # Node Exporter - system metrics from all hosts
-  - job_name: 'node'
-    file_sd_configs:
-      - files:
-          - '{{ prometheus_targets_dir }}/nodes.yml'
-        refresh_interval: 60s
-
-  # Chrony Exporter - NTP metrics from PVE cluster
-  # One static_config per host so each carries its own hostname label. The
-  # previous form listed all three targets together and recovered the hostname
-  # in relabel_configs by regex-matching __address__ against the node's IP --
-  # which silently coupled the labels to the addressing. Changing a target
-  # would have left the regex unmatched and the hostname label simply absent,
-  # with the scrape still succeeding.
-  - job_name: 'chrony'
-    static_configs:
-      - targets: ['jupiter.renner.uno:9123']
-        labels:
-          group: 'ntp'
-          type: 'time-server'
-          hostname: 'jupiter'
-      - targets: ['neptune.renner.uno:9123']
-        labels:
-          group: 'ntp'
-          type: 'time-server'
-          hostname: 'neptune'
-      - targets: ['saturn.renner.uno:9123']
-        labels:
-          group: 'ntp'
-          type: 'time-server'
-          hostname: 'saturn'
-
-  # Immich photo management metrics
-  - job_name: 'immich'
-    static_configs:
-      - targets: ['immich.srv.renner.uno:8081']
-        labels:
-          group: 'media'
-          type: 'photo-management'
-
-  # Blackbox Exporter - HTTP endpoint probing
-  - job_name: 'blackbox-http'
-    metrics_path: /probe
-    params:
-      module: [http_2xx]
-    static_configs:
-      - targets:
-          - https://gitea.renner.uno
-          - https://immich.renner.uno
-          - https://auth.renner.uno
-          - https://ha.renner.uno
-          - https://grafana.renner.uno
-          - https://prometheus.renner.uno
-        labels:
-          group: 'probe'
-          type: 'http'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - TCP probing
-  - job_name: 'blackbox-tcp'
-    metrics_path: /probe
-    params:
-      module: [tcp_connect]
-    static_configs:
-      - targets:
-          - 10.2.40.135:5432  # PostgreSQL HA VIP
-        labels:
-          group: 'probe'
-          type: 'tcp'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - DNS probing (external domains like google.com)
-  - job_name: 'blackbox-dns-external'
-    metrics_path: /probe
-    params:
-      module: [dns_external]
-    static_configs:
-      - targets:
-          - 10.2.40.53  # Proxmox DNS VIP
-          - 10.2.40.1   # UDM Gateway DNS
-        labels:
-          group: 'probe'
-          type: 'dns'
-          query_type: 'external'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - DNS probing (internal domains like k8s-1.srv.renner.uno)
-  - job_name: 'blackbox-dns-internal'
-    metrics_path: /probe
-    params:
-      module: [dns_internal]
-    static_configs:
-      - targets:
-          - 10.2.40.53  # Proxmox DNS VIP
-          - 10.2.40.1   # UDM Gateway DNS
-        labels:
-          group: 'probe'
-          type: 'dns'
-          query_type: 'internal'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-  # Blackbox Exporter - DNS probing (cached domains)
-  - job_name: 'blackbox-dns-cached'
-    metrics_path: /probe
-    params:
-      module: [dns_cached]
-    static_configs:
-      - targets:
-          - 10.2.40.53  # Proxmox DNS VIP
-          - 10.2.40.1   # UDM Gateway DNS
-        labels:
-          group: 'probe'
-          type: 'dns'
-          query_type: 'cached'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__param_target]
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9115
-
-{% if snmp_synology_targets is defined and snmp_synology_targets | length > 0 %}
-  # Synology NAS SNMP metrics
-  # snmp_exporter v0.27+ requires both synology and ucd_la_table modules
-  - job_name: 'synology'
-    scrape_interval: 60s
-    scrape_timeout: 55s
-    metrics_path: /snmp
-    params:
-      module: [synology, ucd_la_table, ucd_memory, if_mib]
-{% if snmp_use_v3 | default(false) %}
-      auth: [synology_v3]
-{% else %}
-      auth: [synology_v2]
-{% endif %}
-    static_configs:
-      - targets:
-{% for target in snmp_synology_targets %}
-          - '{{ target.target }}'  # {{ target.name }}
-{% endfor %}
-        labels:
-          group: 'storage'
-          type: 'nas'
-    relabel_configs:
-      - source_labels: [__address__]
-        target_label: __param_target
-      - source_labels: [__address__]
-        regex: '([^:]+)(:\d+)?'
-        replacement: '${1}'
-        target_label: instance
-      - target_label: __address__
-        replacement: localhost:9116  # snmp_exporter address
-{% endif %}

ansible/roles/promtail/defaults/main.yml

@@ -1,36 +0,0 @@
----
-# Promtail Configuration
-
-# Version and download
-promtail_version: "3.4.2"
-promtail_arch: "linux-amd64"
-promtail_download_url: "https://github.com/grafana/loki/releases/download/v{{ promtail_version }}/promtail-{{ promtail_arch }}.zip"
-promtail_bin_path: "/usr/local/bin/promtail"
-
-# Directories
-promtail_config_dir: "/etc/promtail"
-promtail_config_file: "{{ promtail_config_dir }}/promtail-config.yaml"
-promtail_positions_file: "/var/lib/promtail/positions.yaml"
-
-# Server configuration
-promtail_http_listen_port: 9080
-promtail_http_listen_address: "0.0.0.0"
-
-# Loki server
-promtail_loki_url: "http://loki.srv.renner.uno:3100/loki/api/v1/push"
-
-# Log scrape configurations
-# Override in playbook or host_vars for specific services
-promtail_scrape_configs:
-  - job_name: system
-    static_configs:
-      - targets:
-          - localhost
-        labels:
-          job: syslog
-          host: "{{ inventory_hostname }}"
-          __path__: /var/log/syslog
-
-# Systemd service
-promtail_user: "promtail"
-promtail_group: "promtail"

ansible/roles/promtail/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart promtail
-  systemd:
-    name: promtail
-    state: restarted
-    daemon_reload: yes

ansible/roles/promtail/tasks/main.yml

@@ -1,112 +0,0 @@
----
-- name: Install dependencies
-  apt:
-    name:
-      - unzip
-      - curl
-    state: present
-    update_cache: yes
-
-- name: Create promtail group
-  group:
-    name: "{{ promtail_group }}"
-    system: yes
-    state: present
-
-- name: Create promtail user
-  user:
-    name: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    home: /var/lib/promtail
-    create_home: yes
-    state: present
-
-- name: Add promtail user to systemd-journal group
-  user:
-    name: "{{ promtail_user }}"
-    groups: systemd-journal
-    append: yes
-
-- name: Create config directory
-  file:
-    path: "{{ promtail_config_dir }}"
-    state: directory
-    owner: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    mode: '0755'
-
-- name: Create positions directory
-  file:
-    path: "{{ promtail_positions_file | dirname }}"
-    state: directory
-    owner: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    mode: '0755'
-
-- name: Check if promtail binary exists
-  stat:
-    path: "{{ promtail_bin_path }}"
-  register: promtail_binary
-
-- name: Get installed promtail version
-  command: "{{ promtail_bin_path }} --version"
-  register: installed_version
-  changed_when: false
-  failed_when: false
-  when: promtail_binary.stat.exists
-
-- name: Download promtail
-  get_url:
-    url: "{{ promtail_download_url }}"
-    dest: /tmp/promtail.zip
-    mode: '0644'
-  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
-
-- name: Extract promtail binary
-  unarchive:
-    src: /tmp/promtail.zip
-    dest: /tmp/
-    remote_src: yes
-  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
-
-- name: Install promtail binary
-  copy:
-    src: "/tmp/promtail-{{ promtail_arch }}"
-    dest: "{{ promtail_bin_path }}"
-    remote_src: yes
-    mode: '0755'
-  when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
-  notify: Restart promtail
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - /tmp/promtail.zip
-    - "/tmp/promtail-{{ promtail_arch }}"
-
-- name: Configure promtail
-  template:
-    src: promtail-config.yaml.j2
-    dest: "{{ promtail_config_file }}"
-    owner: "{{ promtail_user }}"
-    group: "{{ promtail_group }}"
-    mode: '0644'
-  notify: Restart promtail
-
-- name: Create systemd service
-  template:
-    src: promtail.service.j2
-    dest: /etc/systemd/system/promtail.service
-    mode: '0644'
-  notify: Restart promtail
-
-- name: Enable and start promtail
-  systemd:
-    name: promtail
-    state: started
-    enabled: yes
-    daemon_reload: yes

ansible/roles/promtail/templates/promtail-config.yaml.j2

@@ -1,79 +0,0 @@
-server:
-  http_listen_port: {{ promtail_http_listen_port }}
-  http_listen_address: {{ promtail_http_listen_address }}
-  grpc_listen_port: 0
-
-positions:
-  filename: {{ promtail_positions_file }}
-
-clients:
-  - url: {{ promtail_loki_url }}
-    tenant_id: default
-
-scrape_configs:
-{% for config in promtail_scrape_configs %}
-  - job_name: {{ config.job_name }}
-{% if config.journal is defined %}
-    journal:
-      max_age: {{ config.journal.max_age | default('12h') }}
-      labels:
-        job: {{ config.job_name }}
-        host: {{ inventory_hostname }}
-{% if config.journal.matches is defined %}
-      matches: {{ config.journal.matches }}
-{% endif %}
-{% if config.relabel_configs is defined %}
-    relabel_configs:
-{% for relabel in config.relabel_configs %}
-      - source_labels: {{ relabel.source_labels | to_json }}
-        target_label: {{ relabel.target_label }}
-{% if relabel.regex is defined %}
-        regex: '{{ relabel.regex }}'
-{% endif %}
-{% if relabel.replacement is defined %}
-        replacement: '{{ relabel.replacement }}'
-{% endif %}
-{% endfor %}
-{% endif %}
-{% endif %}
-{% if config.static_configs is defined %}
-    static_configs:
-{% for static in config.static_configs %}
-      - targets:
-{% for target in static.targets %}
-          - {{ target }}
-{% endfor %}
-        labels:
-{% for key, value in static.labels.items() %}
-          {{ key }}: {{ value }}
-{% endfor %}
-{% endfor %}
-{% if config.pipeline_stages is defined %}
-    pipeline_stages:
-{% for stage in config.pipeline_stages %}
-{% if stage.multiline is defined %}
-      - multiline:
-          firstline: '{{ stage.multiline.firstline }}'
-{% if stage.multiline.max_wait_time is defined %}
-          max_wait_time: {{ stage.multiline.max_wait_time }}
-{% endif %}
-{% endif %}
-{% if stage.regex is defined %}
-      - regex:
-          expression: '{{ stage.regex.expression }}'
-{% endif %}
-{% if stage.labels is defined %}
-      - labels:
-{% for label in stage.labels %}
-          {{ label }}:
-{% endfor %}
-{% endif %}
-{% if stage.timestamp is defined %}
-      - timestamp:
-          source: {{ stage.timestamp.source }}
-          format: '{{ stage.timestamp.format }}'
-{% endif %}
-{% endfor %}
-{% endif %}
-{% endif %}
-{% endfor %}

ansible/roles/promtail/templates/promtail.service.j2

@@ -1,24 +0,0 @@
-[Unit]
-Description=Promtail Log Collector
-Documentation=https://grafana.com/docs/loki/latest/clients/promtail/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ promtail_user }}
-Group={{ promtail_group }}
-ExecStart={{ promtail_bin_path }} -config.file={{ promtail_config_file }}
-Restart=always
-RestartSec=5
-TimeoutStopSec=30
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadWritePaths=/var/lib/promtail
-CapabilityBoundingSet=CAP_DAC_READ_SEARCH
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/pve-exporter/defaults/main.yml

@@ -1,19 +0,0 @@
----
-# PVE Exporter defaults
-
-# Installation
-pve_exporter_version: "3.4.5"
-pve_exporter_install_dir: /opt/pve-exporter
-pve_exporter_config_dir: /etc/pve-exporter
-pve_exporter_venv_path: "{{ pve_exporter_install_dir }}/venv"
-
-# Service
-pve_exporter_user: pve-exporter
-pve_exporter_group: pve-exporter
-pve_exporter_port: 9221
-
-# Proxmox connection (override with vault variables)
-pve_exporter_pve_user: "{{ pve_user | default('root@pam') }}"
-pve_exporter_pve_token_name: "{{ pve_token_id | default('ansible') }}"
-pve_exporter_pve_token_value: "{{ pve_token_secret }}"
-pve_exporter_pve_verify_ssl: false

ansible/roles/pve-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart pve-exporter
-  systemd:
-    name: pve-exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/pve-exporter/tasks/main.yml

@@ -1,78 +0,0 @@
----
-# PVE Exporter installation and configuration
-
-- name: Install Python dependencies
-  apt:
-    name:
-      - python3
-      - python3-pip
-      - python3-venv
-    state: present
-    update_cache: yes
-
-- name: Create pve-exporter system user
-  user:
-    name: "{{ pve_exporter_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create installation directory
-  file:
-    path: "{{ pve_exporter_install_dir }}"
-    state: directory
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    mode: '0755'
-
-- name: Create configuration directory
-  file:
-    path: "{{ pve_exporter_config_dir }}"
-    state: directory
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    mode: '0750'
-
-- name: Create Python virtual environment
-  command: python3 -m venv {{ pve_exporter_venv_path }}
-  args:
-    creates: "{{ pve_exporter_venv_path }}/bin/python"
-
-- name: Set venv ownership
-  file:
-    path: "{{ pve_exporter_venv_path }}"
-    state: directory
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    recurse: yes
-
-- name: Install prometheus-pve-exporter
-  pip:
-    name: prometheus-pve-exporter
-    virtualenv: "{{ pve_exporter_venv_path }}"
-    state: present
-
-- name: Create configuration file
-  template:
-    src: pve.yml.j2
-    dest: "{{ pve_exporter_config_dir }}/pve.yml"
-    owner: "{{ pve_exporter_user }}"
-    group: "{{ pve_exporter_group }}"
-    mode: '0640'
-  notify: Restart pve-exporter
-
-- name: Create systemd service
-  template:
-    src: pve-exporter.service.j2
-    dest: /etc/systemd/system/pve-exporter.service
-    owner: root
-    group: root
-    mode: '0644'
-  notify: Restart pve-exporter
-
-- name: Enable and start pve-exporter service
-  systemd:
-    name: pve-exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes

ansible/roles/pve-exporter/templates/pve-exporter.service.j2

@@ -1,22 +0,0 @@
-[Unit]
-Description=Prometheus PVE Exporter
-Documentation=https://github.com/prometheus-pve/prometheus-pve-exporter
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ pve_exporter_user }}
-Group={{ pve_exporter_group }}
-ExecStart={{ pve_exporter_venv_path }}/bin/pve_exporter \
-  --config.file {{ pve_exporter_config_dir }}/pve.yml \
-  --web.listen-address 0.0.0.0:{{ pve_exporter_port }}
-Restart=on-failure
-RestartSec=10s
-
-# Security hardening
-NoNewPrivileges=yes
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/pve-exporter/templates/pve.yml.j2

@@ -1,8 +0,0 @@
-# PVE Exporter Configuration
-# Generated by Ansible
-
-default:
-  user: "{{ pve_exporter_pve_user }}"
-  token_name: "{{ pve_exporter_pve_token_name }}"
-  token_value: "{{ pve_exporter_pve_token_value }}"
-  verify_ssl: {{ pve_exporter_pve_verify_ssl | lower }}

ansible/roles/snmp-exporter/defaults/main.yml

@@ -1,33 +0,0 @@
----
-# SNMP Exporter defaults
-
-snmp_exporter_version: "0.30.1"
-snmp_exporter_listen_port: 9116
-snmp_exporter_user: "snmp_exporter"
-snmp_exporter_group: "snmp_exporter"
-
-snmp_exporter_install_dir: "/opt/snmp_exporter"
-snmp_exporter_config_dir: "/etc/snmp_exporter"
-
-# Synology NAS SNMP targets
-# Configure in host_vars or group_vars
-snmp_synology_targets: []
-# Example:
-# snmp_synology_targets:
-#   - name: nas-1
-#     target: 10.2.40.50
-#   - name: nas-2
-#     target: 10.2.40.51
-
-# SNMPv3 authentication (recommended for security)
-# These should be set in vault
-snmp_v3_username: "prometheus"
-snmp_v3_auth_protocol: "SHA"  # MD5 or SHA
-snmp_v3_auth_password: ""     # Set in vault
-snmp_v3_priv_protocol: "AES"  # DES or AES
-snmp_v3_priv_password: ""     # Set in vault
-snmp_v3_security_level: "authPriv"  # noAuthNoPriv, authNoPriv, authPriv
-
-# SNMPv2c fallback (less secure, but simpler)
-snmp_v2_community: "public"
-snmp_use_v3: true

ansible/roles/snmp-exporter/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart snmp_exporter
-  systemd:
-    name: snmp_exporter
-    state: restarted
-    daemon_reload: yes

ansible/roles/snmp-exporter/tasks/main.yml

@@ -1,114 +0,0 @@
----
-# SNMP Exporter installation and configuration
-
-- name: Create snmp_exporter group
-  group:
-    name: "{{ snmp_exporter_group }}"
-    system: yes
-    state: present
-
-- name: Create snmp_exporter user
-  user:
-    name: "{{ snmp_exporter_user }}"
-    group: "{{ snmp_exporter_group }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    home: "{{ snmp_exporter_install_dir }}"
-    create_home: no
-
-- name: Create directories
-  file:
-    path: "{{ item }}"
-    state: directory
-    owner: "{{ snmp_exporter_user }}"
-    group: "{{ snmp_exporter_group }}"
-    mode: '0755'
-  loop:
-    - "{{ snmp_exporter_install_dir }}"
-    - "{{ snmp_exporter_config_dir }}"
-
-- name: Check if snmp_exporter is already installed
-  stat:
-    path: "{{ snmp_exporter_install_dir }}/snmp_exporter"
-  register: snmp_exporter_binary
-
-- name: Get installed snmp_exporter version
-  command: "{{ snmp_exporter_install_dir }}/snmp_exporter --version"
-  register: installed_version
-  changed_when: false
-  failed_when: false
-  when: snmp_exporter_binary.stat.exists
-
-- name: Download snmp_exporter
-  get_url:
-    url: "https://github.com/prometheus/snmp_exporter/releases/download/v{{ snmp_exporter_version }}/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64.tar.gz"
-    dest: "/tmp/snmp_exporter.tar.gz"
-    mode: '0644'
-  when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
-
-- name: Extract snmp_exporter
-  unarchive:
-    src: "/tmp/snmp_exporter.tar.gz"
-    dest: "/tmp/"
-    remote_src: yes
-  when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
-
-- name: Install snmp_exporter binary
-  copy:
-    src: "/tmp/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64/snmp_exporter"
-    dest: "{{ snmp_exporter_install_dir }}/snmp_exporter"
-    remote_src: yes
-    owner: "{{ snmp_exporter_user }}"
-    group: "{{ snmp_exporter_group }}"
-    mode: '0755'
-  when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
-  notify: Restart snmp_exporter
-
-- name: Install default snmp.yml from release
-  copy:
-    src: "/tmp/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64/snmp.yml"
-    dest: "{{ snmp_exporter_config_dir }}/snmp.yml"
-    remote_src: yes
-    owner: "{{ snmp_exporter_user }}"
-    group: "{{ snmp_exporter_group }}"
-    mode: '0644'
-  when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
-  notify: Restart snmp_exporter
-
-- name: Add synology_v3 auth entry
-  lineinfile:
-    path: "{{ snmp_exporter_config_dir }}/snmp.yml"
-    insertafter: "^auths:"
-    line: "  synology_v3: {version: 3, security_level: {{ snmp_v3_security_level }}, username: {{ snmp_v3_username }}, password: {{ snmp_v3_auth_password }}, auth_protocol: {{ snmp_v3_auth_protocol }}, priv_protocol: {{ snmp_v3_priv_protocol }}, priv_password: {{ snmp_v3_priv_password }}}"
-  notify: Restart snmp_exporter
-
-- name: Add synology_v2 auth entry
-  lineinfile:
-    path: "{{ snmp_exporter_config_dir }}/snmp.yml"
-    insertafter: "^auths:"
-    line: "  synology_v2: {version: 2, community: {{ snmp_v2_community }}}"
-  notify: Restart snmp_exporter
-
-- name: Create snmp_exporter systemd service
-  template:
-    src: snmp_exporter.service.j2
-    dest: /etc/systemd/system/snmp_exporter.service
-    owner: root
-    group: root
-    mode: '0644'
-  notify: Restart snmp_exporter
-
-- name: Enable and start snmp_exporter service
-  systemd:
-    name: snmp_exporter
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - "/tmp/snmp_exporter.tar.gz"
-    - "/tmp/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64"

ansible/roles/snmp-exporter/templates/snmp-auth.yml.j2

@@ -1,18 +0,0 @@
-# SNMPv3 Authentication for Synology NAS
-# Appended to default snmp.yml by Ansible
-
-auths:
-{% if snmp_use_v3 | default(true) %}
-  synology_v3:
-    security_level: {{ snmp_v3_security_level }}
-    username: {{ snmp_v3_username }}
-    password: {{ snmp_v3_auth_password }}
-    auth_protocol: {{ snmp_v3_auth_protocol }}
-{% if snmp_v3_security_level == 'authPriv' %}
-    priv_protocol: {{ snmp_v3_priv_protocol }}
-    priv_password: {{ snmp_v3_priv_password }}
-{% endif %}
-{% else %}
-  synology_v2:
-    community: {{ snmp_v2_community }}
-{% endif %}

ansible/roles/snmp-exporter/templates/snmp_exporter.service.j2

@@ -1,26 +0,0 @@
-[Unit]
-Description=Prometheus SNMP Exporter
-Documentation=https://github.com/prometheus/snmp_exporter
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ snmp_exporter_user }}
-Group={{ snmp_exporter_group }}
-ExecStart={{ snmp_exporter_install_dir }}/snmp_exporter \
-  --config.file={{ snmp_exporter_config_dir }}/snmp.yml \
-  --web.listen-address=:{{ snmp_exporter_listen_port }}
-
-Restart=on-failure
-RestartSec=5s
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-PrivateTmp=yes
-ReadOnlyPaths=/
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/unifi-poller/defaults/main.yml

@@ -1,34 +0,0 @@
----
-# UniFi Poller defaults
-
-# Version to install
-unifi_poller_version: "2.11.2"
-
-# Installation paths
-unifi_poller_bin_path: /usr/local/bin/unpoller
-unifi_poller_config_dir: /etc/unpoller
-unifi_poller_config_file: "{{ unifi_poller_config_dir }}/up.conf"
-
-# Service user
-unifi_poller_user: unpoller
-unifi_poller_group: unpoller
-
-# UniFi Controller settings (override in vars/vault)
-unifi_controller_url: "https://192.168.1.1"
-unifi_controller_user: ""
-unifi_controller_pass: ""
-unifi_controller_verify_ssl: false
-unifi_controller_site: "default"
-
-# Prometheus exporter settings
-unifi_poller_prometheus_port: 9130
-unifi_poller_prometheus_path: /metrics
-
-# Polling intervals
-unifi_poller_interval: 30s
-unifi_poller_save_sites: true
-unifi_poller_save_ids: false
-unifi_poller_save_events: false
-unifi_poller_save_alarms: false
-unifi_poller_save_anomalies: false
-unifi_poller_save_dpi: true

ansible/roles/unifi-poller/handlers/main.yml

@@ -1,6 +0,0 @@
----
-- name: Restart unpoller
-  systemd:
-    name: unpoller
-    state: restarted
-    daemon_reload: yes

ansible/roles/unifi-poller/tasks/main.yml

@@ -1,88 +0,0 @@
----
-# UniFi Poller installation and configuration
-
-- name: Create unpoller system user
-  user:
-    name: "{{ unifi_poller_user }}"
-    system: yes
-    shell: /usr/sbin/nologin
-    create_home: no
-
-- name: Create configuration directory
-  file:
-    path: "{{ unifi_poller_config_dir }}"
-    state: directory
-    owner: "{{ unifi_poller_user }}"
-    group: "{{ unifi_poller_group }}"
-    mode: '0750'
-
-- name: Check if unpoller is already installed
-  stat:
-    path: "{{ unifi_poller_bin_path }}"
-  register: unpoller_binary
-
-- name: Get installed version
-  command: "{{ unifi_poller_bin_path }} --version"
-  register: installed_version
-  changed_when: false
-  failed_when: false
-  when: unpoller_binary.stat.exists
-
-- name: Download UniFi Poller
-  get_url:
-    url: "https://github.com/unpoller/unpoller/releases/download/v{{ unifi_poller_version }}/unpoller_{{ unifi_poller_version }}_linux_amd64.tar.gz"
-    dest: /tmp/unpoller.tar.gz
-    mode: '0644'
-  when: not unpoller_binary.stat.exists or (installed_version.stdout is defined and unifi_poller_version not in installed_version.stdout)
-
-- name: Extract UniFi Poller
-  unarchive:
-    src: /tmp/unpoller.tar.gz
-    dest: /tmp/
-    remote_src: yes
-  when: not unpoller_binary.stat.exists or (installed_version.stdout is defined and unifi_poller_version not in installed_version.stdout)
-
-- name: Install UniFi Poller binary
-  copy:
-    src: /tmp/unpoller
-    dest: "{{ unifi_poller_bin_path }}"
-    remote_src: yes
-    owner: root
-    group: root
-    mode: '0755'
-  when: not unpoller_binary.stat.exists or (installed_version.stdout is defined and unifi_poller_version not in installed_version.stdout)
-  notify: Restart unpoller
-
-- name: Create configuration file
-  template:
-    src: up.conf.j2
-    dest: "{{ unifi_poller_config_file }}"
-    owner: "{{ unifi_poller_user }}"
-    group: "{{ unifi_poller_group }}"
-    mode: '0640'
-  notify: Restart unpoller
-
-- name: Create systemd service
-  template:
-    src: unpoller.service.j2
-    dest: /etc/systemd/system/unpoller.service
-    owner: root
-    group: root
-    mode: '0644'
-  notify: Restart unpoller
-
-- name: Enable and start unpoller service
-  systemd:
-    name: unpoller
-    state: started
-    enabled: yes
-    daemon_reload: yes
-
-- name: Clean up downloaded files
-  file:
-    path: "{{ item }}"
-    state: absent
-  loop:
-    - /tmp/unpoller.tar.gz
-    - /tmp/unpoller
-  changed_when: false

ansible/roles/unifi-poller/templates/unpoller.service.j2

@@ -1,24 +0,0 @@
-[Unit]
-Description=UniFi Poller - Prometheus Exporter for UniFi Network
-Documentation=https://github.com/unpoller/unpoller
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-User={{ unifi_poller_user }}
-Group={{ unifi_poller_group }}
-ExecStart={{ unifi_poller_bin_path }} --config {{ unifi_poller_config_file }}
-Restart=on-failure
-RestartSec=10s
-
-# Security hardening
-NoNewPrivileges=yes
-ProtectSystem=strict
-ProtectHome=yes
-ReadOnlyPaths=/
-ReadWritePaths={{ unifi_poller_config_dir }}
-PrivateTmp=yes
-
-[Install]
-WantedBy=multi-user.target

ansible/roles/unifi-poller/templates/up.conf.j2

@@ -1,42 +0,0 @@
-# UniFi Poller Configuration
-# Generated by Ansible
-
-[poller]
-  # Turns on debug messages
-  debug = false
-  # Turns off messages
-  quiet = false
-
-[prometheus]
-  # Whether to export metrics
-  disable = false
-  # HTTP listen address for /metrics endpoint
-  http_listen = "0.0.0.0:{{ unifi_poller_prometheus_port }}"
-  # Path for metrics endpoint
-  report_errors = false
-
-[unifi]
-  # UniFi Controller configurations
-  [[unifi.controller]]
-    # URL for the UniFi Controller
-    url = "{{ unifi_controller_url }}"
-    # Username
-    user = "{{ unifi_controller_user }}"
-    # Password
-    pass = "{{ unifi_controller_pass }}"
-    # Sites to poll (leave empty for all sites)
-    sites = ["{{ unifi_controller_site }}"]
-    # Verify SSL certificates
-    verify_ssl = {{ unifi_controller_verify_ssl | lower }}
-    # Save site information
-    save_sites = {{ unifi_poller_save_sites | lower }}
-    # Save device IDs
-    save_ids = {{ unifi_poller_save_ids | lower }}
-    # Save events
-    save_events = {{ unifi_poller_save_events | lower }}
-    # Save alarms
-    save_alarms = {{ unifi_poller_save_alarms | lower }}
-    # Save anomalies
-    save_anomalies = {{ unifi_poller_save_anomalies | lower }}
-    # Save DPI data
-    save_dpi = {{ unifi_poller_save_dpi | lower }}

ci/agents/update-audit/prompt.md

@@ -78,6 +78,29 @@ For each, find the **current pin in the repo** (Read/Glob/Grep), then look up th
 6. **Ansible roles & OpenTofu providers** — `ansible/requirements.yml` (Galaxy
    roles/collections) and `opentofu/providers.tf` + `opentofu/.terraform.lock.hcl`.
    Report any with a newer published version.
+7. **Edge heating CAN services** — open3e (Vitocal heat pump, Vitocharge VX3)
+   and E3onCAN (E380 grid energy meter), shipped as one image built from
+   `ci/images/heating-can/Containerfile` and deployed by
+   `ansible/roles/heating-can`. Nothing else tracks them. They read and (for two
+   datapoints) write the heat pump over its CAN bus, so silent drift here
+   matters more than most.
+
+   Both upstream pins are `ARG`s in the Containerfile -- `OPEN3E_REF` and
+   `E3ONCAN_REF` -- not Ansible variables. Read them there, and note the
+   awkward bits so normal states are not reported as problems:
+   - open3e publishes **no tags at all** (its tags API returns an empty array),
+     so compare `OPEN3E_REF` against the default branch:
+     `https://api.github.com/repos/open3e/open3e/commits?per_page=1`. Report how
+     many commits behind and the dates. "No release found" is normal here.
+   - E3onCAN tags, but sparsely, and `E3ONCAN_REF` is an untagged commit later
+     than the newest tag. Compare against both
+     `https://api.github.com/repos/MyHomeMyData/E3onCAN/tags` and the default
+     branch, and say which the pin sits between.
+
+   The deployed image tag is `heating_can_image` in
+   `ansible/roles/heating-can/defaults/main.yml`. It is `:latest` today, so the
+   node follows whatever CI last built -- the pins above are what actually fix
+   the versions, and bumping them is what requires a rebuild.
 
 ## Method & rigor
 

ci/images/heating-can/Containerfile

@@ -0,0 +1,122 @@
+# Heating system over the Viessmann E3 CAN bus: open3e (Vitocal heat pump and
+# Vitocharge VX3) and E3onCAN (E380 grid energy meter).
+#
+# One image, two services. They read the same physical installation over the
+# same bus and are versioned together, so they are one artifact -- but they run
+# as two containers from this image so either can crash and restart on its own
+# without a supervisor as PID 1.
+#
+# Why an image at all: both were pip-installed into venvs on the host, open3e
+# straight from a git URL. pip cannot compare an installed VCS dist against a
+# requested one without fetching it, so `pip install` re-cloned and rebuilt on
+# every Ansible converge -- reporting changed, restarting the service, and
+# putting a gap in heating telemetry each time. Pinning the SHA did not fix
+# that (verified: pip still clones and rebuilds a pinned VCS requirement). A
+# tagged image does: `docker compose up -d` on an unchanged tag is a genuine
+# no-op.
+#
+# It also gives the weekly update-audit a normal artifact to track instead of
+# two awkward special cases -- open3e publishes no tags at all, and E3onCAN is
+# pinned to an untagged commit between releases.
+#
+# Runtime needs host networking: SocketCAN interfaces live in the host network
+# namespace, so can0 is only reachable with network_mode: host. That is also
+# how the local MQTT broker on 127.0.0.1:1883 is reached. CAN_RAW sockets need
+# no capabilities, so both run unprivileged.
+FROM python:3.13-slim AS build
+
+# Pins live here, in the artifact, rather than in Ansible variables.
+#
+# open3e: a commit, because the project publishes NO tags (its tags API returns
+# an empty array). Self-reported package version at this commit: 0.7.5.
+ARG OPEN3E_REF=6553c10453a6bae1210908f38a5af2481a46a3e5
+# E3onCAN: an untagged commit on the default branch (2026-06-21, "Fixed: MQTT
+# client ID collision caused stale retained values"). Later than the repo's
+# only tag, v0.5.1.
+ARG E3ONCAN_REF=461dcbdf97e8d72d2edb9929c63b39382af6c3a1
+
+# git is needed to FETCH the sources and exists only in this stage. The runtime
+# stage below copies the built venv and the E3onCAN source out, so neither git
+# nor the apt cache ends up in the published image.
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends git ca-certificates \
+ && rm -rf /var/lib/apt/lists/*
+
+RUN python -m venv /opt/venv
+ENV PATH="/opt/venv/bin:$PATH"
+
+RUN pip install --no-cache-dir supervisor \
+ && pip install --no-cache-dir "git+https://github.com/open3e/open3e.git@${OPEN3E_REF}"
+
+# E3onCAN is not packaged -- it is run as a script out of its checkout, so the
+# source has to travel into the runtime image alongside its dependencies.
+RUN git clone --quiet https://github.com/MyHomeMyData/E3onCAN.git /opt/e3oncan \
+ && git -C /opt/e3oncan checkout --quiet "${E3ONCAN_REF}" \
+ && pip install --no-cache-dir -r /opt/e3oncan/requirements.txt \
+ && rm -rf /opt/e3oncan/.git
+
+
+FROM python:3.13-slim
+
+ARG OPEN3E_REF=6553c10453a6bae1210908f38a5af2481a46a3e5
+ARG E3ONCAN_REF=461dcbdf97e8d72d2edb9929c63b39382af6c3a1
+
+LABEL org.opencontainers.image.title="heating-can" \
+      org.opencontainers.image.description="open3e + E3onCAN, Viessmann E3 CAN bus readers" \
+      org.opencontainers.image.source="https://gitea.renner.uno/k.renner/homelab" \
+      uno.renner.open3e.ref="${OPEN3E_REF}" \
+      uno.renner.e3oncan.ref="${E3ONCAN_REF}"
+
+# Everything is installed at build time. The runtime image carries the finished
+# venv and source only -- no git, no compiler, no package index.
+COPY --from=build /opt/venv /opt/venv
+COPY --from=build /opt/e3oncan /opt/e3oncan
+ENV PATH="/opt/venv/bin:$PATH"
+
+# Both services log to stdout, which docker's journald driver forwards to the
+# journal and edge-log-shipper ships to Loki. open3e's systemd unit did NOT set
+# this, so its output was block-buffered and effectively never reached the
+# journal -- a long-standing blind spot that this fixes for both.
+ENV PYTHONUNBUFFERED=1
+
+# One container, two services, so PID 1 is a supervisor. It restarts either
+# reader on its own if it dies, reaps children and forwards shutdown -- which a
+# bare process as PID 1 cannot do for a sibling.
+#
+# Configuration arrives as environment variables and supervisord expands them,
+# so nothing outside this image constructs open3e's or E3onCAN's argv. Those
+# CLIs are an implementation detail here; a compose file spelling them out
+# would break whenever either upstream renamed a flag.
+COPY supervisord.conf /etc/supervisord.conf
+
+# Build-time self-test. Fails the build if either service cannot actually run,
+# which is exactly what shipped before: supervisord pointed at
+# /opt/e3oncan/src/E3onCANcollect.py, a path that does not exist -- the file is
+# at the repo root. The image built green and would have run open3e while
+# e3oncan crash-looped, the E380 meter silently absent.
+#
+# Done here rather than as a CI step because this needs no registry
+# credentials and no runner able to pull a private image -- and because a
+# broken image should never be published in the first place.
+RUN open3e --help > /dev/null \
+ && python /opt/e3oncan/E3onCANcollect.py --help > /dev/null \
+ && supervisord --help > /dev/null \
+ && grep -q '^\[program:open3e\]' /etc/supervisord.conf \
+ && grep -q '^\[program:e3oncan\]' /etc/supervisord.conf \
+ && grep -q '^\[supervisorctl\]' /etc/supervisord.conf \
+ && supervisorctl --help > /dev/null \
+ && echo "self-test OK: both services execute; supervisord has both programs"
+
+# CAN_RAW needs no capabilities, so nothing here runs as root.
+RUN useradd --system --no-create-home --shell /usr/sbin/nologin open3e
+
+# A writable working directory, owned by the service user. udsoncan writes its
+# log file into the CWD, and the native install this replaces gave open3e's
+# directory `owner: open3e` for exactly that reason. /opt is root-owned from the
+# COPY layers, so running there would fail that write.
+RUN install -d -o open3e -g open3e -m 0755 /var/lib/open3e
+
+USER open3e
+WORKDIR /var/lib/open3e
+
+ENTRYPOINT ["supervisord", "-c", "/etc/supervisord.conf"]

ci/images/heating-can/supervisord.conf

@@ -0,0 +1,80 @@
+# Both CAN readers, supervised in one container.
+#
+# They read the same heating installation over the same bus and are versioned
+# and deployed together, so they are one container. A container runs one PID 1;
+# with two processes that PID 1 is a supervisor, which is what this is --
+# supervisord restarts either service on its own if it dies, reaps children,
+# and forwards shutdown.
+#
+# Configuration comes from the environment (see the compose file). supervisord
+# expands %(ENV_NAME)s at start, so neither the compose file nor a wrapper
+# script constructs open3e's or E3onCAN's argv -- those CLIs stay an
+# implementation detail of this image.
+# Control socket. Without these two sections supervisorctl cannot connect, so
+# there is no way to query or restart an individual service inside the
+# container -- which is the entire reason for running a supervisor rather than
+# one process per container. The first revision shipped without them.
+#
+# Unix socket only, inside the container: nothing outside needs to reach it,
+# and the container runs with host networking, so a TCP listener would be
+# exposed on the host.
+[unix_http_server]
+file=/tmp/supervisor.sock
+chmod=0700
+
+[rpcinterface:supervisor]
+supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
+
+[supervisorctl]
+serverurl=unix:///tmp/supervisor.sock
+
+[supervisord]
+nodaemon=true
+user=open3e
+logfile=/dev/null
+logfile_maxbytes=0
+pidfile=/tmp/supervisord.pid
+
+# open3e: UDS request/response to the Vitocal heat pump and Vitocharge VX3.
+[program:open3e]
+command=open3e
+    -c %(ENV_CAN_INTERFACE)s
+    -cnfg %(ENV_OPEN3E_CONFIG)s
+    -m %(ENV_MQTT_HOST)s:%(ENV_MQTT_PORT)s:%(ENV_MQTT_TOPIC)s
+    -muser %(ENV_MQTT_USER)s:%(ENV_MQTT_PASSWORD)s
+    -mcid %(ENV_OPEN3E_CLIENT_ID)s
+    -mfstr %(ENV_MQTT_FORMAT)s
+    -r %(ENV_OPEN3E_DATAPOINTS)s
+    -t %(ENV_OPEN3E_POLL_SECONDS)s
+    -l %(ENV_MQTT_TOPIC)s/cmnd
+    -j
+autostart=true
+autorestart=true
+startsecs=10
+stopsignal=INT
+# Straight to the container's stdout/stderr, so docker's journald driver picks
+# them up and edge-log-shipper ships them to Loki.
+stdout_logfile=/dev/fd/1
+stdout_logfile_maxbytes=0
+redirect_stderr=true
+
+# e3oncan: passive CANopen listener for the E380 grid energy meter, which feeds
+# the heat pump's own energy management. Issues no bus writes, which is why the
+# two can share the bus.
+[program:e3oncan]
+command=python /opt/e3oncan/E3onCANcollect.py
+    -c %(ENV_CAN_INTERFACE)s
+    -dev %(ENV_E3ONCAN_DEVICE)s
+    -g %(ENV_E3ONCAN_GAP_SECONDS)s
+    -m %(ENV_MQTT_HOST)s:%(ENV_MQTT_PORT)s:%(ENV_MQTT_TOPIC)s
+    -muser %(ENV_MQTT_USER)s
+    -mpass %(ENV_MQTT_PASSWORD)s
+    -mfstr %(ENV_E3ONCAN_MQTT_FORMAT)s
+    -j
+autostart=true
+autorestart=true
+startsecs=10
+stopsignal=INT
+stdout_logfile=/dev/fd/1
+stdout_logfile_maxbytes=0
+redirect_stderr=true

kubernetes/apps/gatus/configmap.yaml

@@ -160,6 +160,24 @@ data:
         conditions:
           - "[STATUS] == any(200, 301, 302)"
 
+      # Terminus admin UI. Checks the TLS route only — the cleartext
+      # device endpoint (trmnl-device.renner.uno) is the same pod, so a
+      # failure here covers both. Terminus exposes no health endpoint, so
+      # this is a root-page check: the login page answers 200, and an
+      # unauthenticated request may redirect.
+      - name: Web Service 10
+        group: services
+        url: https://trmnl.renner.uno
+        interval: 60s
+        ui:
+          hide-hostname: true
+          hide-url: true
+          hide-port: true
+          hide-conditions: true
+          hide-errors: true
+        conditions:
+          - "[STATUS] == any(200, 301, 302)"
+
       # Removed 2026-05-21: Jellyfin (media.renner.uno), Wiki (wiki.renner.uno),
       # InfluxDB (influx.renner.uno) — hostnames NXDOMAIN, no Ingress; services
       # no longer deployed. Re-add when/if they come back.

kubernetes/apps/home-assistant/deployment.yaml

@@ -38,7 +38,7 @@ spec:
       # gets the heavy HA pod (8Gi) off the soon-to-be-tiny edge box.
       containers:
         - name: home-assistant
-          image: ghcr.io/home-assistant/home-assistant:2026.8.0 # {"$imagepolicy": "flux-system:home-assistant"}
+          image: ghcr.io/home-assistant/home-assistant:2026.8.1 # {"$imagepolicy": "flux-system:home-assistant"}
           env:
             - name: TZ
               value: Europe/Berlin

kubernetes/apps/monitoring/pve-exporter-deployment.yaml

@@ -27,7 +27,7 @@ spec:
           type: RuntimeDefault
       containers:
         - name: pve-exporter
-          image: prompve/prometheus-pve-exporter:3.9.0 # {"$imagepolicy": "flux-system:pve-exporter"}
+          image: prompve/prometheus-pve-exporter:3.10.0 # {"$imagepolicy": "flux-system:pve-exporter"}
           # Image ENTRYPOINT is already `pve_exporter`; pass flags only.
           args:
             - --config.file=/etc/pve-exporter/pve.yml

kubernetes/apps/music-assistant/deployment.yaml

@@ -52,7 +52,7 @@ spec:
         # stdlib only; Flux keeps the tag in lockstep via the shared image
         # policy) — no extra dependency.
         - name: reconcile-publish-ip
-          image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
+          image: ghcr.io/music-assistant/server:2.9.13 # {"$imagepolicy": "flux-system:music-assistant"}
           command: ["/app/venv/bin/python", "/scripts/reconcile.py"]
           volumeMounts:
             - name: data
@@ -62,7 +62,7 @@ spec:
               readOnly: true
       containers:
         - name: music-assistant
-          image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
+          image: ghcr.io/music-assistant/server:2.9.13 # {"$imagepolicy": "flux-system:music-assistant"}
           env:
             - name: LOG_LEVEL
               value: info

kubernetes/apps/paperless/gotenberg.yaml

@@ -22,7 +22,7 @@ spec:
           type: RuntimeDefault
       containers:
         - name: gotenberg
-          image: gotenberg/gotenberg:8.34.0 # {"$imagepolicy": "flux-system:gotenberg"}
+          image: gotenberg/gotenberg:8.35.0 # {"$imagepolicy": "flux-system:gotenberg"}
           args:
             - "gotenberg"
             - "--chromium-disable-javascript=true"

kubernetes/apps/trmnl/cluster.yaml

@@ -0,0 +1,41 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: tr-pg
+  namespace: trmnl
+spec:
+  instances: 1
+  smartShutdownTimeout: 10
+  stopDelay: 60
+  # PG 18, NOT the repo-standard 16. Terminus ships a committed
+  # config/db/structure.sql containing `SET transaction_timeout = 0;` — a GUC
+  # that does not exist before PostgreSQL 17, so schema load against a 16
+  # server dies with `unrecognized configuration parameter`. Upstream's
+  # compose.yml develops against 18.4, so 18 is the tested target; 17 is the
+  # bare minimum. Bare major tag matches the other clusters (minor security
+  # bumps flow automatically, major upgrades stay an explicit decision).
+  imageName: ghcr.io/cloudnative-pg/postgresql:18
+  enableSuperuserAccess: true
+  storage:
+    storageClass: ceph-block
+    size: 5Gi
+  plugins:
+    - name: barman-cloud.cloudnative-pg.io
+      isWALArchiver: true
+      parameters:
+        barmanObjectName: tr-pg-store
+  bootstrap:
+    initdb:
+      database: terminus
+      owner: terminus
+  postgresql:
+    parameters:
+      shared_buffers: "128MB"
+      max_connections: "50"
+      wal_compression: "on"
+  resources:
+    requests:
+      cpu: 100m
+      memory: 256Mi
+    limits:
+      memory: 512Mi

kubernetes/apps/trmnl/device-service.yaml

@@ -0,0 +1,48 @@
+# Device-facing endpoint. Cleartext HTTP, deliberately.
+#
+# The TRMNL firmware initiates every request and cannot be handed the
+# internal CA, so terminating TLS here would simply break polling. Scope of
+# the exposure: this Service is LAN-only (no public DNS label, no Cloudflare
+# record), it carries no admin session — the device authenticates with its
+# MAC in the `ID` header plus a per-device api_key — and the Terminus admin
+# UI stays on the TLS route at trmnl.renner.uno.
+#
+# Same shape as home-assistant-bgp: a dedicated address on the Cilium BGP
+# pool rather than a TLS-less Ingress, which would be a new pattern for this
+# repo. The device is provisioned with the HOSTNAME, not the address.
+#
+# The address is requested explicitly, matching every other LB service here.
+# Left to allocate freely, Cilium IPAM handed out the first free address in
+# the pool block (10.2.99.0-10.2.99.255) — which is 10.2.99.0, the base
+# address of the range. Each LB address is advertised to the UDM as its own
+# BGP route so that is routable in principle, but a .0 host address is the
+# kind of thing constrained embedded IP stacks get wrong, and the TRMNL
+# firmware is the single client that has to work here. Not worth the risk
+# for an address nobody chose on purpose.
+#
+# .89 continues the existing sequence (.81-.88 are in use) and matches the
+# fd99::89 that IPAM had already picked for v6.
+apiVersion: v1
+kind: Service
+metadata:
+  name: terminus-device
+  namespace: trmnl
+  labels:
+    app.kubernetes.io/name: terminus
+    lb-pool: bgp
+  annotations:
+    external-dns.alpha.kubernetes.io/hostname: trmnl-device.renner.uno
+    lbipam.cilium.io/ips: "10.2.99.89,fd99::89"
+spec:
+  type: LoadBalancer
+  externalTrafficPolicy: Local
+  ipFamilyPolicy: PreferDualStack
+  ipFamilies:
+    - IPv4
+    - IPv6
+  selector:
+    app.kubernetes.io/name: terminus
+  ports:
+    - name: http
+      port: 80
+      targetPort: http

kubernetes/apps/trmnl/externalsecret.yaml

@@ -0,0 +1,18 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: terminus-secrets
+  namespace: trmnl
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: terminus-secrets
+  # Holds APP_SECRET only (session/cookie/CSRF key). DATABASE_URL is not in
+  # here — it comes straight off the CNPG-generated `tr-pg-app` Secret's
+  # `uri` key, so the password rotates with the cluster and is never copied.
+  dataFrom:
+    - extract:
+        key: apps/trmnl

kubernetes/apps/trmnl/hetzner-creds-externalsecret.yaml

@@ -0,0 +1,15 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: hetzner-s3-creds
+  namespace: trmnl
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: hetzner-s3-creds
+  dataFrom:
+    - extract:
+        key: seed/trmnl/hetzner-creds

kubernetes/apps/trmnl/ingress.yaml

@@ -0,0 +1,28 @@
+# Admin UI only — login, device registration, playlists, screen previews.
+# The device never touches this route: the firmware speaks plain HTTP and
+# would reject the internal CA, so it uses the cleartext LoadBalancer in
+# device-service.yaml instead. Both front the same Service.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: terminus
+  namespace: trmnl
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt-cluster
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - trmnl.renner.uno
+      secretName: trmnl-tls
+  rules:
+    - host: trmnl.renner.uno
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: terminus
+                port:
+                  number: 80

kubernetes/apps/trmnl/kustomization.yaml

@@ -0,0 +1,20 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - externalsecret.yaml
+  - pvcs.yaml
+  - valkey.yaml
+  - terminus.yaml
+  - ingress.yaml
+  - device-service.yaml
+  # Headless HA screenshotter. Lives in this namespace because Terminus
+  # FETCHES rendered screens back from it (uri delivery mode), so the
+  # two need the intra-namespace path.
+  - renderer.yaml
+  - renderer-externalsecret.yaml
+  # tr-pg CNPG Postgres + offsite Barman backup. Note this cluster runs
+  # PostgreSQL 18, not the repo-standard 16 — see the comment in cluster.yaml.
+  - cluster.yaml
+  - objectstore.yaml
+  - scheduled-backup.yaml
+  - hetzner-creds-externalsecret.yaml

kubernetes/apps/trmnl/objectstore.yaml

@@ -0,0 +1,30 @@
+---
+apiVersion: barmancloud.cnpg.io/v1
+kind: ObjectStore
+metadata:
+  name: tr-pg-store
+  namespace: trmnl
+spec:
+  configuration:
+    # Offsite Hetzner Object Storage (fsn1), same bucket as the other CNPG
+    # clusters. No SSE: Hetzner supports only SSE-C, unusable by barman-cloud
+    # (plugin #646), so at-rest encryption is unavailable; transit is TLS.
+    destinationPath: s3://renner-uno-k8s-cnpg/tr-pg
+    endpointURL: https://fsn1.your-objectstorage.com
+    s3Credentials:
+      accessKeyId:
+        name: hetzner-s3-creds
+        key: ACCESS_KEY_ID
+      secretAccessKey:
+        name: hetzner-s3-creds
+        key: ACCESS_SECRET_KEY
+      # Region is required (REGION=fsn1) — without it boto3's HeadBucket
+      # (barman-cloud-check-wal-archive during bootstrap.recovery) 400s.
+      region:
+        name: hetzner-s3-creds
+        key: REGION
+    wal:
+      compression: gzip
+    data:
+      compression: gzip
+  retentionPolicy: "30d"

kubernetes/apps/trmnl/pvcs.yaml

@@ -0,0 +1,57 @@
+---
+# Uploads + synchronised fonts for the Terminus pod. Mounted twice by
+# subPath (uploads/, fonts/) into both the web and sidekiq containers, which
+# share one pod — so RWO is sufficient and the Deployment uses `Recreate`.
+#
+# Contents: rendered device screens and the font set pulled by the
+# FONT_SYNCHRONIZER job. Persisting fonts here (rather than an emptyDir)
+# means a pod restart does not depend on upstream being reachable.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: terminus-data
+  namespace: trmnl
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 5Gi
+---
+# Valkey RDB snapshots. The Sidekiq queue is reconstructable — the
+# synchroniser jobs re-enqueue on boot — so this is small and losing it is
+# an inconvenience, not data loss.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: terminus-valkey-data
+  namespace: trmnl
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 1Gi
+---
+# Renderer state: schedules.json plus the rendered screens under
+# output/, which Terminus fetches back over HTTP in uri delivery mode.
+#
+# This mount is REQUIRED, not just nice to have. The app picks its data
+# directory by testing whether /data exists — with nothing mounted it
+# silently falls back to /app/data inside the container, so schedules
+# and rendered output would vanish on every restart and the panel would
+# go blank until the next render.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: trmnl-renderer-data
+  namespace: trmnl
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: ceph-block
+  resources:
+    requests:
+      storage: 2Gi

kubernetes/apps/trmnl/renderer-externalsecret.yaml

@@ -0,0 +1,27 @@
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: trmnl-renderer-secrets
+  namespace: trmnl
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: openbao
+  target:
+    name: trmnl-renderer-secrets
+  # Kept separate from apps/trmnl (which holds Terminus's APP_SECRET)
+  # because these are credentials the RENDERER holds about other
+  # systems, with different rotation stories:
+  #
+  #   HA_ACCESS_TOKEN     — a dedicated Home Assistant long-lived token,
+  #                         not the operator's personal one, so the
+  #                         renderer can be revoked in isolation.
+  #   BYOS_ACCESS_TOKEN   — Terminus JWTs, minted once out of band from
+  #   BYOS_REFRESH_TOKEN    a Terminus login. The PASSWORD deliberately
+  #                         never enters the cluster; stock Terminus
+  #                         disables session expiry, so these are
+  #                         effectively permanent.
+  dataFrom:
+    - extract:
+        key: apps/trmnl/renderer

kubernetes/apps/trmnl/renderer.yaml

@@ -0,0 +1,344 @@
+# Headless renderer: screenshots the Home Assistant e-ink dashboard,
+# dithers it to 16 greys and hands it to Terminus.
+#
+# Upstream ships this as a Home Assistant add-on. Our HA is a plain
+# container with no Supervisor, so it runs standalone here instead —
+# viable because the app prefers HOME_ASSISTANT_URL/ACCESS_TOKEN env
+# vars over /data/options.json and authenticates with an ordinary
+# long-lived token. SUPERVISOR_TOKEN is deliberately NOT set: it only
+# flips an internal isAddOn flag, and setting it would switch the app
+# to add-on behaviour (fixed port, /data assumptions).
+#
+# DIRECTION OF TRAVEL, which is the non-obvious part: with the
+# byos-hanami webhook in `uri` delivery mode the renderer does NOT
+# push image bytes. It POSTs a URL pointing at its own /output/<file>
+# endpoint, and TERMINUS FETCHES THE IMAGE BACK from it. That is why
+# both live in this namespace — the pull is intra-namespace and is
+# already permitted by the default-deny-ingress rule.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: trmnl-renderer-schedule
+  namespace: trmnl
+data:
+  # Template for /data/schedules.json. The init container splices the
+  # BYOS tokens in; everything non-secret is declared here so the
+  # schedule is reviewable in git rather than clicked into the app's
+  # web UI (which has no authentication and is not exposed).
+  #
+  # The app re-reads schedules.json every 60 s, so edits here take
+  # effect without a restart once the init container has rewritten it.
+  #
+  # model_id is 1 — TRMNL X (name "v2", 1872x1404, 16 colours) — and it
+  # is an INTEGER, not a name. Confirmed against the running server:
+  # GET /api/screens returns model_id as a number, and upstream's
+  # example value "og_1bit" matches no model on this instance (the
+  # 2-bit OG model here is id 11, name "og_plus"). Getting this wrong
+  # shows up as Terminus rejecting the pushed screen with a 422.
+  #
+  # All three auth fields below are placeholders the init container
+  # replaces — including obtained_at, which must end up a real epoch-MS
+  # stamp. The 0 here is inert only because it is always overwritten;
+  # left as 0 the scheduler reads the tokens as ancient and refuses to
+  # push at all ("expired beyond the refresh window"), with no render
+  # attempt and no error beyond one startup warning.
+  schedules.template.json: |
+    [
+      {
+        "id": "trmnl-entrance",
+        "name": "Entrance e-ink panel",
+        "enabled": true,
+        "cron": "* * * * *",
+        "interval_minutes": 1,
+        "ha_mode": true,
+        "dashboard_path": "/trmnl-entrance/home",
+        "target_url": null,
+        "viewport": { "width": 1872, "height": 1404 },
+        "device": "schedule_trmnl_x",
+        "crop": { "enabled": false, "x": 0, "y": 0, "width": 0, "height": 0 },
+        "format": "png",
+        "rotate": null,
+        "zoom": 1,
+        "wait": null,
+        "theme": "trmnl-eink",
+        "lang": null,
+        "dark": false,
+        "invert": false,
+        "timestamp": false,
+        "dithering": {
+          "enabled": true,
+          "method": "floyd-steinberg",
+          "palette": "gray-16",
+          "bitDepth": 4,
+          "gammaCorrection": true,
+          "blackLevel": 0,
+          "whiteLevel": 100,
+          "normalize": true,
+          "saturationBoost": false
+        },
+        "webhook_url": "http://terminus.trmnl.svc.cluster.local/api/screens",
+        "webhook_headers": {},
+        "webhook_format": {
+          "format": "byos-hanami",
+          "byosConfig": {
+            "label": "Eingang",
+            "name": "trmnl-entrance",
+            "model_id": 1,
+            "preprocessed": true,
+            "addon_base_url": "http://trmnl-renderer.trmnl.svc.cluster.local:10000",
+            "delivery_mode": "uri",
+            "auth": {
+              "enabled": true,
+              "access_token": "__BYOS_ACCESS_TOKEN__",
+              "refresh_token": "__BYOS_REFRESH_TOKEN__",
+              "obtained_at": 0
+            }
+          }
+        },
+        "createdAt": "2026-08-11T00:00:00.000Z",
+        "updatedAt": "2026-08-11T00:00:00.000Z"
+      }
+    ]
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: trmnl-renderer
+  namespace: trmnl
+  labels:
+    app.kubernetes.io/name: trmnl-renderer
+spec:
+  replicas: 1
+  # RWO PVC, and two renderers would push conflicting screens for the
+  # same device.
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: trmnl-renderer
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: trmnl-renderer
+    spec:
+      enableServiceLinks: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1000
+        runAsGroup: 1000
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+        seccompProfile:
+          type: RuntimeDefault
+      initContainers:
+        # Writes /data/schedules.json from the ConfigMap template with
+        # the BYOS tokens injected.
+        #
+        # jq rather than sed: the tokens are JWTs and go into a JSON
+        # string, so they need JSON-safe escaping. A sed substitution
+        # would corrupt on any character sed treats as special.
+        #
+        # No HTTP here on purpose. The app does expose a /api/byos/login
+        # proxy that exchanges a Terminus username+password for these
+        # tokens, but that would mean shipping the Terminus PASSWORD
+        # into the cluster and doing a network call at every pod start.
+        # Stock Terminus disables session expiry and issues effectively
+        # permanent tokens, so they are minted once out of band and kept
+        # in OpenBao — the password never enters the cluster.
+        - name: render-schedule
+          image: gitea.renner.uno/gitea_admin/alpine-jq:2026-07-30
+          command:
+            - sh
+            - -c
+            - |
+              set -eu
+              # obtained_at is stamped at write time, in MILLISECONDS.
+              # It is not cosmetic: the scheduler derives token age from
+              # it, and a literal 0 in the template made the app treat
+              # freshly-minted tokens as ancient and refuse to push —
+              #   "BYOS tokens ... expired beyond the refresh window"
+              # — with no render attempt and no error, just a warning at
+              # startup. Stamping it here is honest enough: stock
+              # Terminus disables session expiry and issues effectively
+              # permanent tokens, so age is not load-bearing.
+              jq --arg at "$BYOS_ACCESS_TOKEN" \
+                 --arg rt "$BYOS_REFRESH_TOKEN" \
+                 '(.[].webhook_format.byosConfig.auth.access_token) = $at
+                  | (.[].webhook_format.byosConfig.auth.refresh_token) = $rt
+                  | (.[].webhook_format.byosConfig.auth.obtained_at) = (now * 1000 | floor)' \
+                 /template/schedules.template.json > /data/schedules.json
+              echo "wrote /data/schedules.json ($(wc -c < /data/schedules.json) bytes)"
+          env:
+            - name: BYOS_ACCESS_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: trmnl-renderer-secrets
+                  key: BYOS_ACCESS_TOKEN
+            - name: BYOS_REFRESH_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: trmnl-renderer-secrets
+                  key: BYOS_REFRESH_TOKEN
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: schedule-template
+              mountPath: /template
+              readOnly: true
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+      containers:
+        # Pinned, not enrolled in image automation — same reasoning as
+        # Terminus itself. This drives a wall-mounted panel and upstream
+        # is pre-1.0; an unattended bump that changes the dithering
+        # pipeline or the webhook payload shows up as a corrupted
+        # screen, not as a failed rollout.
+        #
+        # Note the image is ARCH-SUFFIXED: upstream publishes no
+        # multi-arch manifest, only trmnl-ha-amd64 and trmnl-ha-aarch64.
+        - name: renderer
+          image: ghcr.io/usetrmnl/trmnl-ha-amd64:0.9.5
+          ports:
+            - name: http
+              containerPort: 10000
+          env:
+            # The in-cluster Service name, NOT ha.renner.uno and not
+            # home.renner.uno. Both public names are unusable from a
+            # pod: ha.renner.uno resolves to the home-assistant-bgp
+            # LoadBalancer's external address (10.2.99.87), which is not
+            # backed by an Ingress, so CoreDNS's k8s_gateway delegation
+            # does not intercept it and the pod hairpins off the LB.
+            #
+            # This cluster has hit that exact failure before — see the
+            # comment on kubernetes/infrastructure/openbao/rotation/
+            # home-assistant.yaml, which reaches HA the same way and
+            # states it outright: "The public name is unusable from a
+            # pod (ingress-LB hairpin)."
+            #
+            # FAILS QUIETLY if wrong: the schedule keeps firing every
+            # minute, each render errors on connect, and the panel just
+            # never updates. Nothing goes CrashLoopBackOff.
+            - name: HOME_ASSISTANT_URL
+              value: "http://home-assistant-bgp.home-assistant.svc.cluster.local:8123"
+            - name: TZ
+              value: "Europe/Berlin"
+            # Chromium will not start without a writable HOME. Running
+            # as uid 1000 with no matching passwd entry leaves HOME="/",
+            # which is root-owned 0755, and the browser dies on launch
+            # with
+            #   chrome_crashpad_handler: --database is required
+            # because crashpad cannot create its database directory.
+            #
+            # Diagnosed on the running pod, not guessed: HOME=/, uid=1000,
+            # / not writable, /tmp writable. Crash dumps are throwaway, so
+            # /tmp is the right home for them rather than the PVC.
+            #
+            # This is a consequence of OUR non-root securityContext, not
+            # of the image — so it must move together with runAsUser.
+            - name: HOME
+              value: "/tmp"
+            # Dedicated long-lived token, NOT the operator's personal
+            # one, so the renderer's access can be revoked on its own.
+            # It is injected into the headless browser's localStorage,
+            # which is how the app authenticates to the HA frontend.
+            - name: ACCESS_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: trmnl-renderer-secrets
+                  key: HA_ACCESS_TOKEN
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            # Bounded scratch backing HOME. Chromium writes crashpad
+            # state and profile data here on every launch, and it now
+            # launches once a minute for the life of the pod — on the
+            # container's writable layer that is unbounded growth
+            # against node disk, with no ephemeral-storage limit to
+            # catch it. A sizeLimit converts a slow, cluster-wide
+            # node-disk-pressure problem into a bounded and clearly
+            # attributable eviction of this pod.
+            - name: tmp
+              mountPath: /tmp
+          # HTTP probes here, unlike terminus.yaml's TCP-only ones, because
+          # /health is confirmed rather than assumed: the upstream image's
+          # own Dockerfile declares
+          #   HEALTHCHECK ... curl -f http://localhost:10000/health
+          # so the route is part of the image's published contract. Terminus
+          # got TCP probes precisely because it publishes no such endpoint.
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: http
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            timeoutSeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: http
+            initialDelaySeconds: 10
+            periodSeconds: 10
+            timeoutSeconds: 5
+          resources:
+            requests:
+              cpu: 250m
+              memory: 768Mi
+            # Chromium is a SEPARATE process from the app, and the app's
+            # own watchdog only measures its own RSS (restarting itself
+            # past 900 MB). The limit therefore has to cover the app's
+            # ceiling PLUS a browser rendering a 1872x1404 page, which
+            # is well above the 512 MB upstream's troubleshooting note
+            # suggests.
+            limits:
+              memory: 2Gi
+          # Chromium normally needs either SYS_ADMIN or a user namespace to
+          # start its own sandbox, which would make dropping ALL fatal. It
+          # is safe here because the app launches the browser with
+          # --no-sandbox --disable-setuid-sandbox --no-zygote (and
+          # --disable-dev-shm-usage, which is why no /dev/shm emptyDir is
+          # mounted either). Those flags are baked into the app's own
+          # PUPPETEER_ARGS, not something we pass, so this holds as long as
+          # the pinned tag does — worth re-checking on any bump.
+          #
+          # Confirmed on the running pod: the capability drop is genuinely
+          # not what broke the browser. The first deploy DID crash it, but
+          # on a writable-HOME problem caused by runAsUser — see the HOME
+          # env above. Do not "fix" a future Chromium launch failure by
+          # loosening this block before checking HOME and /tmp.
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: trmnl-renderer-data
+        - name: schedule-template
+          configMap:
+            name: trmnl-renderer-schedule
+        - name: tmp
+          emptyDir:
+            sizeLimit: 512Mi
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: trmnl-renderer
+  namespace: trmnl
+  labels:
+    app.kubernetes.io/name: trmnl-renderer
+spec:
+  # ClusterIP only, and deliberately no Ingress. The app's web UI on
+  # this port has NO authentication of any kind — upstream says so
+  # outright — so it must not be routable from the LAN. Terminus
+  # reaches it in-namespace to fetch rendered screens; nothing else
+  # needs to.
+  selector:
+    app.kubernetes.io/name: trmnl-renderer
+  ports:
+    - name: http
+      port: 10000
+      targetPort: http

kubernetes/apps/trmnl/scheduled-backup.yaml

@@ -0,0 +1,16 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: ScheduledBackup
+metadata:
+  name: tr-pg-daily
+  namespace: trmnl
+spec:
+  # 03:00 local, same slot as the other CNPG clusters (the Velero PVC
+  # schedule for this namespace runs later, at 08:15).
+  schedule: "0 0 3 * * *"
+  immediate: true
+  backupOwnerReference: self
+  cluster:
+    name: tr-pg
+  method: plugin
+  pluginConfiguration:
+    name: barman-cloud.cloudnative-pg.io

kubernetes/apps/trmnl/terminus.yaml

@@ -0,0 +1,184 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: terminus-config
+  namespace: trmnl
+data:
+  TZ: "Europe/Berlin"
+  HANAMI_PORT: "2345"
+  # What the device is told to fetch from. Terminus bakes this into the
+  # absolute `image_url` it hands back on /api/display, so it MUST be the
+  # cleartext LAN name, not trmnl.renner.uno — the firmware speaks plain
+  # HTTP and would reject the internal CA. No trailing slash: the same
+  # string is typed into the captive portal at provisioning time.
+  API_URI: "http://trmnl-device.renner.uno"
+  KEYVALUE_URL: "redis://terminus-valkey:6379"
+  # Rack::Attack rate-limit bypass. Beyond upstream's RFC1918 defaults this
+  # adds fc00::/7, which covers the pod CIDR (fd10:244::/48), the Cilium LB
+  # pool (fd99::/120) and the LAN ULA — without it every IPv6 client,
+  # including the device and Traefik, is rate-limited as if it were remote.
+  RACK_ATTACK_ALLOWED_SUBNETS: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1,::1,fc00::/7"
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: terminus
+  namespace: trmnl
+  labels:
+    app.kubernetes.io/name: terminus
+spec:
+  replicas: 1
+  # RWO PVC shared by both containers in this pod, and APP_SETUP runs
+  # migrations on boot — two concurrent web pods would race on both.
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: terminus
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: terminus
+    spec:
+      enableServiceLinks: false
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1000
+        runAsGroup: 1000
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        # Tag pinned deliberately and NOT enrolled in Flux image automation.
+        # Terminus is pre-1.0 with no GitHub Releases and no changelog; an
+        # unattended bump that changes the firmware or /api/display contract
+        # is a device-bricking class of risk. Bumps are a reviewed, manual
+        # act — see kubernetes/infrastructure/image-automation/README.md.
+        - name: terminus
+          image: ghcr.io/usetrmnl/terminus:0.68.0
+          ports:
+            - name: http
+              containerPort: 2345
+          envFrom:
+            - configMapRef:
+                name: terminus-config
+          env:
+            # Runs `hanami assets compile` + `hanami db migrate` in the
+            # entrypoint before exec'ing puma. Set on the web container
+            # only — the sidekiq sidecar must not race the migration.
+            - name: APP_SETUP
+              value: "true"
+            # CNPG generates this Secret; `uri` already carries the rotated
+            # password and points at the tr-pg-rw service.
+            - name: DATABASE_URL
+              valueFrom:
+                secretKeyRef:
+                  name: tr-pg-app
+                  key: uri
+            - name: APP_SECRET
+              valueFrom:
+                secretKeyRef:
+                  name: terminus-secrets
+                  key: APP_SECRET
+          volumeMounts:
+            - name: data
+              mountPath: /app/public/uploads
+              subPath: uploads
+            - name: data
+              mountPath: /app/public/fonts
+              subPath: fonts
+          # TCP probes, not HTTP: Terminus exposes no documented health
+          # endpoint, and every real route is behind either auth or
+          # Rack::Attack. Tighten to an HTTP GET once the route set is
+          # confirmed against the running app.
+          startupProbe:
+            tcpSocket:
+              port: http
+            periodSeconds: 10
+            # ~10 min: first boot compiles assets and migrates a fresh DB.
+            failureThreshold: 60
+          livenessProbe:
+            tcpSocket:
+              port: http
+            periodSeconds: 30
+            timeoutSeconds: 5
+          readinessProbe:
+            tcpSocket:
+              port: http
+            periodSeconds: 10
+            timeoutSeconds: 5
+          resources:
+            requests:
+              cpu: 200m
+              memory: 512Mi
+            limits:
+              memory: 1Gi
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+        - name: sidekiq
+          image: ghcr.io/usetrmnl/terminus:0.68.0
+          command: ["bundle", "exec", "sidekiq", "-r", "./config/sidekiq.rb"]
+          envFrom:
+            - configMapRef:
+                name: terminus-config
+          # EXPECTED on a fresh deploy: this container loses the race against
+          # the web container's migration and exits 1 with
+          # `PG::UndefinedTable: relation "account" does not exist`, then
+          # comes up healthy on the restart once the schema exists.
+          # Observed on first deploy 2026-08-10; converged in ~1s. There is
+          # no upstream ordering hook, and gating it behind an initContainer
+          # that probes for a specific table would couple us to Terminus's
+          # schema. So this is left to Kubernetes' restart, deliberately —
+          # a single sidekiq restart shortly after a rollout is benign. More
+          # than a couple, or restarts long after a rollout, are NOT this.
+          env:
+            # No APP_SETUP here — see the web container.
+            - name: DATABASE_URL
+              valueFrom:
+                secretKeyRef:
+                  name: tr-pg-app
+                  key: uri
+            - name: APP_SECRET
+              valueFrom:
+                secretKeyRef:
+                  name: terminus-secrets
+                  key: APP_SECRET
+          volumeMounts:
+            - name: data
+              mountPath: /app/public/uploads
+              subPath: uploads
+            - name: data
+              mountPath: /app/public/fonts
+              subPath: fonts
+          resources:
+            requests:
+              cpu: 100m
+              memory: 256Mi
+            limits:
+              memory: 512Mi
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: terminus-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: terminus
+  namespace: trmnl
+  labels:
+    app.kubernetes.io/name: terminus
+spec:
+  selector:
+    app.kubernetes.io/name: terminus
+  ports:
+    - name: http
+      port: 80
+      targetPort: http

kubernetes/apps/trmnl/valkey.yaml

@@ -0,0 +1,79 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: terminus-valkey
+  namespace: trmnl
+  labels:
+    app.kubernetes.io/name: terminus-valkey
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: terminus-valkey
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: terminus-valkey
+    spec:
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 999
+        fsGroup: 999
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        # Valkey major version tracks upstream's compose.yml (9.x). Sidekiq
+        # is the only consumer; a major bump is a separate migration.
+        - name: valkey
+          image: valkey/valkey:9.1.1-alpine # {"$imagepolicy": "flux-system:valkey-trmnl"}
+          args:
+            - valkey-server
+            - --save
+            - "60"
+            - "1"
+            - --loglevel
+            - warning
+          ports:
+            - name: valkey
+              containerPort: 6379
+          volumeMounts:
+            - name: data
+              mountPath: /data
+          livenessProbe:
+            tcpSocket:
+              port: valkey
+            initialDelaySeconds: 10
+          readinessProbe:
+            exec:
+              command: ["valkey-cli", "ping"]
+            initialDelaySeconds: 5
+            periodSeconds: 10
+          resources:
+            requests:
+              cpu: 20m
+              memory: 64Mi
+            limits:
+              memory: 256Mi
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: terminus-valkey-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: terminus-valkey
+  namespace: trmnl
+spec:
+  selector:
+    app.kubernetes.io/name: terminus-valkey
+  ports:
+    - port: 6379
+      targetPort: valkey

kubernetes/clusters/homelab/trmnl.yaml

@@ -0,0 +1,28 @@
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: trmnl
+  namespace: flux-system
+spec:
+  # Terminus (TRMNL BYOS server) plus its tr-pg CNPG Cluster + ObjectStore +
+  # ScheduledBackup, the Valkey queue, and the uploads/fonts PVC. Drives the
+  # entrance-way e-ink panel; the device polls the cleartext LoadBalancer.
+  interval: 10m
+  retryInterval: 1m
+  timeout: 30m
+  sourceRef:
+    kind: GitRepository
+    name: flux-system
+  path: ./kubernetes/apps/trmnl
+  prune: true
+  wait: true
+  dependsOn:
+    - name: infrastructure
+    - name: infrastructure-crds
+    - name: infrastructure-ceph-cluster
+    - name: infrastructure-cnpg-barman
+  healthCheckExprs:
+    - apiVersion: postgresql.cnpg.io/v1
+      kind: Cluster
+      failed: "has(status.phase) && status.phase.startsWith('Failed')"
+      current: "has(status.phase) && (status.phase == 'Cluster in healthy state' || status.readyInstances > 0)"

kubernetes/infrastructure/image-automation/README.md

@@ -96,6 +96,29 @@ The security-scan agent will continue to surface vuln findings on
 these as "Skipped / Flux image-automation owns this" even without
 a policy.
 
+- `ghcr.io/usetrmnl/terminus` — publishes a usable `0.x.0` tag stream, so
+  unlike the three above this is **not** a tag-format problem. It is left
+  unenrolled because the blast radius is hardware: Terminus is pre-1.0,
+  cuts no GitHub Releases and ships no changelog, and it serves the
+  firmware and `/api/display` contract that the TRMNL X panel polls. An
+  unattended bump that changes either can leave a wall-mounted device
+  dark, with no rollback path that doesn't involve the captive portal.
+  Bumps are a reviewed, manual act: read the commit range, then edit the
+  tag in `kubernetes/apps/trmnl/terminus.yaml` (both containers).
+
+  The Valkey sidecar alongside it *is* enrolled (`valkey.yaml`) — it holds
+  only the Sidekiq queue and touches no device-facing contract.
+
+- `ghcr.io/usetrmnl/trmnl-ha-amd64` — the TRMNL screenshot renderer, for
+  the same reason and one more. It owns the dithering pipeline and the
+  webhook payload shape, so a bad bump surfaces as a *corrupted or blank
+  panel*, not as a failed rollout — there is no readiness signal that
+  catches "the image is now unreadable". Separately, upstream publishes
+  **no multi-arch manifest**: only the arch-suffixed `trmnl-ha-amd64` and
+  `trmnl-ha-aarch64` repositories exist, so an ImagePolicy here would also
+  be pinning us to one architecture in a way the tag does not make
+  obvious. Bump manually in `kubernetes/apps/trmnl/renderer.yaml`.
+
 ## Re-import hazard: cnpg-barman-plugin
 
 `kubernetes/infrastructure/cnpg-barman-plugin/manifest.yaml` is an

kubernetes/infrastructure/image-automation/kustomization.yaml

@@ -20,6 +20,7 @@ resources:
   - redis.yaml
   - shelly-manager.yaml
   - tika.yaml
+  - valkey.yaml
   - velero-plugin-for-aws.yaml
   - vsftpd.yaml
   - woodpecker.yaml

kubernetes/infrastructure/image-automation/valkey.yaml

@@ -0,0 +1,27 @@
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImageRepository
+metadata:
+  name: valkey-trmnl
+  namespace: flux-system
+spec:
+  image: valkey/valkey
+  interval: 1h
+---
+# Alpine tags only, semver 9.x — matches the major version pinned in
+# upstream Terminus's compose.yml. Valkey holds nothing but the Sidekiq
+# queue here, so minor bumps are safe to take unattended; a 10.x cutover
+# is a deliberate migration.
+apiVersion: image.toolkit.fluxcd.io/v1
+kind: ImagePolicy
+metadata:
+  name: valkey-trmnl
+  namespace: flux-system
+spec:
+  imageRepositoryRef:
+    name: valkey-trmnl
+  filterTags:
+    pattern: '^(?P<ver>\d+\.\d+\.\d+)-alpine$'
+    extract: '$ver'
+  policy:
+    semver:
+      range: "9.x"

kubernetes/infrastructure/namespaces/namespaces.yaml

@@ -145,3 +145,9 @@ apiVersion: v1
 kind: Namespace
 metadata:
   name: external-secrets
+---
+# Terminus (TRMNL BYOS server) driving the entrance-way e-ink panel.
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: trmnl

kubernetes/infrastructure/network-policies/home-assistant-ns.yaml

@@ -371,3 +371,36 @@ spec:
         - ports:
             - port: "8123"
               protocol: TCP
+---
+# The TRMNL e-ink renderer (trmnl ns) drives a headless browser against the
+# entrance dashboard, so it needs the HA frontend on :8123 — same in-cluster
+# path as the rotator above, and for the same reason: the public names
+# hairpin off the home-assistant-bgp LoadBalancer and are unusable from a
+# pod. It reaches HA at home-assistant-bgp.home-assistant.svc.cluster.local.
+#
+# Only the ingress half is needed here. Unlike the openbao rotator, the trmnl
+# namespace has no default-deny EGRESS covering this pod — terminus-egress is
+# scoped to the Terminus pods by label and does not select the renderer — so
+# there is no source-side rule to add.
+#
+# Source scoped to the renderer pod specifically, not the whole trmnl
+# namespace: Terminus itself sits in that namespace and has no business
+# reaching Home Assistant.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: home-assistant-trmnl-renderer-ingress
+  namespace: home-assistant
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: home-assistant
+  ingress:
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": trmnl
+            app.kubernetes.io/name: trmnl-renderer
+      toPorts:
+        - ports:
+            - port: "8123"
+              protocol: TCP

kubernetes/infrastructure/network-policies/kustomization.yaml

@@ -21,4 +21,5 @@ resources:
   - openbao-ns.yaml
   - external-secrets-ns.yaml
   - woodpecker-ci-ns.yaml
+  - trmnl-ns.yaml
   - allow-cilium-health.yaml

kubernetes/infrastructure/network-policies/trmnl-ns.yaml

@@ -0,0 +1,177 @@
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: default-deny-ingress
+  namespace: trmnl
+spec:
+  endpointSelector: {}
+  ingress:
+    # Intra-namespace — terminus ↔ sidekiq ↔ valkey ↔ tr-pg.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": trmnl
+    # Admin UI via Traefik (trmnl.renner.uno). The device does NOT come
+    # this way — see allow-device-http below.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": ingress
+            app.kubernetes.io/name: traefik
+    # Prometheus scrape.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": monitoring
+    # Velero backups via PVC mount.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": velero
+    # CNPG operator (in `database` ns) reconciles tr-pg — needs to reach
+    # the postgres pods for health, failover, role checks.
+    - fromEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": database
+    - fromEntities:
+        - host
+        - remote-node
+    # ICMP — all standard ICMPv4 + ICMPv6 from any cluster pod. See
+    # home-assistant-ns.yaml for the rationale. INGRESS ONLY.
+    - fromEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: Exists
+      icmps:
+        - fields:
+            - { type: 0,   family: IPv4 }
+            - { type: 3,   family: IPv4 }
+            - { type: 4,   family: IPv4 }
+            - { type: 5,   family: IPv4 }
+            - { type: 8,   family: IPv4 }
+            - { type: 9,   family: IPv4 }
+            - { type: 10,  family: IPv4 }
+            - { type: 11,  family: IPv4 }
+            - { type: 12,  family: IPv4 }
+            - { type: 13,  family: IPv4 }
+            - { type: 14,  family: IPv4 }
+            - { type: 15,  family: IPv4 }
+            - { type: 16,  family: IPv4 }
+            - { type: 17,  family: IPv4 }
+            - { type: 18,  family: IPv4 }
+            - { type: 30,  family: IPv4 }
+            - { type: 1,   family: IPv6 }
+            - { type: 2,   family: IPv6 }
+            - { type: 3,   family: IPv6 }
+            - { type: 4,   family: IPv6 }
+            - { type: 128, family: IPv6 }
+            - { type: 129, family: IPv6 }
+            - { type: 130, family: IPv6 }
+            - { type: 131, family: IPv6 }
+            - { type: 132, family: IPv6 }
+            - { type: 133, family: IPv6 }
+            - { type: 134, family: IPv6 }
+            - { type: 135, family: IPv6 }
+            - { type: 136, family: IPv6 }
+            - { type: 137, family: IPv6 }
+            - { type: 138, family: IPv6 }
+            - { type: 141, family: IPv6 }
+            - { type: 142, family: IPv6 }
+            - { type: 143, family: IPv6 }
+            - { type: 148, family: IPv6 }
+            - { type: 149, family: IPv6 }
+            - { type: 151, family: IPv6 }
+            - { type: 152, family: IPv6 }
+            - { type: 153, family: IPv6 }
+---
+# The TRMNL panel polls /api/display over cleartext HTTP on the
+# terminus-device LoadBalancer. The firmware cannot be handed the internal
+# CA, so this bypasses Traefik entirely and the namespace default-deny has
+# to be opened for it — same shape as paperless's allow-ftp-scanner.
+#
+# Scoped to the terminus pods and port 2345 only. Requests still have to
+# carry a known device MAC in the `ID` header plus its api_key; this rule
+# opens the port, not the application.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: allow-device-http
+  namespace: trmnl
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: terminus
+  ingress:
+    - fromEntities:
+        - world
+        - host
+        - remote-node
+      toPorts:
+        - ports:
+            - port: "2345"
+              protocol: TCP
+---
+# Egress allowlist for the Terminus pod only.
+#
+# Deliberately NOT applied to the tr-pg CNPG pods: selecting them here
+# would make their egress restricted too, and the barman-cloud WAL archive
+# path to Hetzner object storage is not worth putting behind an FQDN rule
+# that fails closed. They keep unrestricted egress, matching immich/authentik.
+#
+# Terminus needs exactly three outbound destinations, all HTTPS:
+#   - trmnl.com            → GET /api/models, /api/palettes (MODEL_SYNCHRONIZER,
+#                            which is what seeds the TRMNL X model definition —
+#                            1872x1404, 16 colours — into the Models list),
+#                            GET /api/firmware/latest (FIRMWARE_SYNCHRONIZER),
+#                            GET /fonts/<name> (FONT_SYNCHRONIZER)
+#   - trmnl-fw.s3.us-east-2.amazonaws.com
+#                          → the actual firmware binary. This host is NOT a
+#                            constant in the source: it is the `url` field of
+#                            the /api/firmware/latest response, so TRMNL can
+#                            move bucket/region without a Terminus release and
+#                            firmware sync would start failing closed here.
+#                            If firmware sync breaks, re-check that response
+#                            before assuming the policy is wrong.
+#
+# KNOWN LIMITATION: Terminus's "Extensions" (private plugin) feature polls
+# arbitrary admin-configured URLs rendered from a Liquid template stored in
+# the database. Those are unknowable from source and WILL be blocked by this
+# allowlist. We do not use Extensions — screen content is pushed in from the
+# Home Assistant renderer — so this is an accepted trade, not an oversight.
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: terminus-egress
+  namespace: trmnl
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: terminus
+  egress:
+    # Intra-namespace — tr-pg-rw (postgres) and terminus-valkey.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": trmnl
+    # DNS. The L7 `rules.dns` block is mandatory, not decorative: Cilium's
+    # DNS proxy only learns an FQDN→IP mapping for names it has actually
+    # observed being resolved, so without this the toFQDNs rules below
+    # never match anything.
+    - toEndpoints:
+        - matchLabels:
+            "k8s:io.kubernetes.pod.namespace": kube-system
+            k8s-app: kube-dns
+      toPorts:
+        - ports:
+            - port: "53"
+              protocol: UDP
+            - port: "53"
+              protocol: TCP
+          rules:
+            dns:
+              - matchPattern: "*"
+    - toFQDNs:
+        - matchName: "trmnl.com"
+        - matchName: "trmnl-fw.s3.us-east-2.amazonaws.com"
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
+    - toEntities:
+        - host
+        - remote-node

kubernetes/infrastructure/post-crds/openbao-clustersecretstore.yaml

@@ -28,6 +28,7 @@ spec:
         - monitoring
         - netbird-backup # netbird-backup-ssh: the VPS-pull backup key
         - paperless
+        - trmnl
         - velero
   provider:
     vault: # ESO's provider name; speaks OpenBao fine

kubernetes/infrastructure/post-crds/velero-schedules.yaml

@@ -283,3 +283,32 @@ spec:
     storageLocation: default
     ttl: 720h0m0s
     snapshotMoveData: true
+---
+apiVersion: velero.io/v1
+kind: Schedule
+metadata:
+  name: trmnl-daily
+  namespace: velero
+  labels:
+    app.kubernetes.io/part-of: homelab-backup
+spec:
+  # Daily at 08:15 local (next free stagger slot after netbird-backup-daily
+  # at 07:45). Selects the whole namespace minus the CNPG-labelled PVCs, so
+  # it picks up terminus-data (device screens + synchronised fonts),
+  # terminus-valkey-data and trmnl-renderer-data (the renderer's schedule
+  # and rendered output) without needing to be edited per PVC. The tr-pg
+  # CNPG cluster is covered separately by Barman, hence the exclusion.
+  schedule: "15 8 * * *"
+  useOwnerReferencesInBackup: false
+  template:
+    includedNamespaces:
+      - trmnl
+    includedResources:
+      - persistentvolumeclaims
+    storageLocation: default
+    ttl: 720h0m0s
+    snapshotMoveData: true
+    labelSelector:
+      matchExpressions:
+        - key: cnpg.io/cluster
+          operator: DoesNotExist