Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5 aa37fb0a0a fix(wordpress): correct the network-policy claim and make the Velero selector safe
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the automated review on #492.

Blocking: two comments claimed the public/admin split was kept honest by
kubernetes/apps/wordpress/network-policy.yaml. That file does not exist — the
comment was written against the paperless template and never updated once the
namespace policy proved sufficient. The deeper point in the review is the real
one: a paperless-style workload-scoped policy would be a no-op here, because
traefik must reach the pod directly for every public page. The split is
enforced by the Ingress path rules and Authentik, not at the network layer,
and the comment now says so — an auditor looking for a network control would
otherwise have gone hunting for a file that was never there.

Velero: the schedule excluded the database PVC by app.kubernetes.io/instance,
a label the mariadb-operator was only assumed to set. A wrong assumption there
fails silently and sweeps the live database volume into the snapshot. Inverted
to select the content PVC by labels this repo sets on it, the same direction
openbao-daily already uses.

The Service-name assumption flagged as a suggestion is kept: the CRD's separate
primaryService/secondaryService fields indicate the base Service carries the
resource name, and a mismatch fails loudly at pod startup. Written down as a
first-deploy check rather than left implicit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014giKN9N6e5J2DF8QZmVh98
2026-08-16 22:01:01 +02:00
claude-botandClaude Opus 5 b784ce3d38 feat(wordpress): host www.renner.uno in the cluster behind an Authentik gate
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Migrates the Hetzner VM 'wordpress-hochzeit' into the cluster. The site is not
the frozen wedding archive it looks like: it carries an unpublished, fully
composed birth announcement from 2026-05-14, so wp-admin has to keep working
for a human author rather than being replaced by a Git workflow.

Split routing on one host: '/' goes straight to WordPress so the blog stays
publicly readable, while /wp-admin and /wp-login.php route through a dedicated
wordpress-gate outpost. The inverse -- gate the host and skip_path_regex the
public paths, as paperless-gate does -- would need a negative lookahead, and
the outpost's RE2 has none.

Gating all of /wp-admin is only safe because the dead wpgiftregistry plugin is
dropped in the migration: it was the sole emitter of ajaxurl on this site, and
of 121 admin-ajax hits in the current log window not one returned 200.

Core lives in the pinned image, not on the volume, so upgrading is a tag bump
in Git. Only wp-content is persisted.

Backups are split the same way CNPG/Barman and Velero are elsewhere: the
operator takes a logical dump to Hetzner Object Storage at 02:45, Velero takes
the wp-content PVC at 08:45 and skips the database volume -- a crash-consistent
snapshot of a running database is not a backup.

Group membership for the gate is owned by Git, as with the Home Assistant,
Shelly and zone-configurator gates. s.renner is the Authentik account for the
site's WordPress Editor, whose login there is still the pre-2019 s.birzer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014giKN9N6e5J2DF8QZmVh98
2026-08-16 21:48:49 +02:00
claude-botandClaude Opus 5 b976a67be8 feat(mariadb): add mariadb-operator and the wordpress backup bucket
WordPress speaks MySQL natively. The PostgreSQL route would mean pg4wp, an
SQL-rewriting shim whose README lists WordPress 6.5.3 as its newest tested
version while renner.uno runs 7.0.4 — every core update would become a
compatibility gamble.

Charts are OCI artifacts, so this follows the OCIRepository pattern from
spegel rather than the HelmRepository pattern used for CNPG. CRDs ship as a
separate chart that upstream requires first, hence the dependsOn. The version
range is capped at the year boundary: the project uses calendar versioning and
ships an upgrade guide with each minor.

renner-uno-k8s-mariadb carries the same 30d noncurrent-version expiry as the
CNPG and Velero buckets, matched to the Backup resource's maxRetention.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014giKN9N6e5J2DF8QZmVh98
2026-08-16 21:48:36 +02:00
k.renner 352f0b5cff Merge pull request 'fix(authentik): one passkey touch per login, password fallback when none enrolled' (#491) from fix-authentik-double-passkey-touch into master
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#491
2026-08-16 08:22:10 +00:00
claude-botandClaude Opus 5 803cdb3db8 feat(authentik): fall back to password when no passkey is enrolled
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The passkey apps were reachable only with a WebAuthn credential:
passkey-passwordless-authentication denies at its validation stage, and
passkey-stepup-authorization demands a passkey on every authorize. An
account with no passkey had no way in.

Branch both flows on enrollment with an expression policy,
passkey-user-has-webauthn. Enrolled accounts keep the mandatory passkey
at login and at every authorize. Accounts without one skip the WebAuthn
stage and get default-authentication-password followed by
default-authentication-mfa-validation — the same pair, in the same
order, that default-authentication-flow already runs, so the fallback is
not a weaker way in than auth.renner.uno's own login.

The policy runs against pending_user, so the branch follows the
identified account's enrollment and is not selectable by the requester.
Bindings keep evaluate_on_plan false and re_evaluate_policies true;
on-plan evaluation would run before a user is resolved.

The negated bindings carry failure_result: true. UserLoginStage defaults
a missing authentication backend to BACKEND_INBUILT rather than
refusing, so a flow that dropped both branches would log the user in on
a username alone. A PolicyException resolves to failure_result before
negate is applied, so a broken expression degrades to password + MFA
instead of to no factor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 09:45:24 +02:00
claude-botandClaude Opus 5 9e0ffae358 fix(authentik): share one WebAuthn stage across login and step-up
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Every fresh login to a passkey app costs two touches: one in
passkey-passwordless-authentication, a second immediately after in
passkey-stepup-authorization.

last_auth_threshold is enforced with a single browser cookie
(authentik_mfa) whose JWT is signed with, and carries, the validation
stage's PK. A stage only honours a cookie it wrote itself, and both
stages write to the same cookie name — so two stage objects each demand
their own touch and clobber each other's window.

Collapse them into one passkey-webauthn-validation stage bound into both
flows. not_configured_action is deny, since the authentication flow
reaches the stage with the user only identified and no password stage in
front of it. Passkey-less users enroll from user settings after a
password login; default-authenticator-webauthn-setup has a
configure_flow, so that path is live.

The 8h cookie now also covers the login flow, so a re-login in the same
browser inside the window is satisfied by the cookie alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 09:25:52 +02:00
security-bot 82573754b1 chore(image-automation): bump rhasspy/wyoming-whisper:3.6.0
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/update-all Pipeline failed
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-14 20:36:17 +00:00
security-bot ba6ca6d0af chore(image-automation): bump ghcr.io/home-assistant/home-assistant:2026.8.2
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-14 17:39:04 +00:00
security-bot 9e8c1fa6fe chore(image-automation): bump gotenberg/gotenberg:8.36.0
Updates managed by image-automation-controller.
2026-08-14 17:36:16 +00:00
k.renner 7a80df3cfc Merge pull request 'fix(ical-proxy): pin the image so a new build actually rolls' (#490) from pin-ical-proxy-image into master
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#490
2026-08-13 19:40:58 +00:00
claude-bot a2de5b5f56 fix(ical-proxy): pin the image so a new build actually rolls
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
:latest never rolls. The tag moves when a build publishes, but nothing
tells the Deployment: the pod template is unchanged, so Kubernetes has
no reason to replace anything.

Nothing else covers it either. This image is deliberately not enrolled in
Flux image automation — kubernetes/infrastructure/image-automation/README.md
lists ghcr.io/konairius/ical-proxy as having "only main, latest, and
main-<sha> commit builds; no release tags", so no ImagePolicy watches it.
The two pods have been running a 21-day-old image while :latest moved
underneath them, and the only way to pick up a new build was a manual
restart, which leaves no record anywhere.

Pinning makes the rollout a git change. Flux applies the new pod
template, Kubernetes rolls it, and what is RUNNING becomes recoverable
from this repo instead of "whatever :latest meant the last time a pod
happened to restart".

main-99a912b is the merge of konairius/ical-proxy#12, which adds the
summary_match and summary_replace parameters. The waste calendar needs
them to strip "| Abfuhrkalender - Landkreis Amberg-Sulzbach" from every
collection — the feed repeats the whole calendar's name on each event,
and no consumer can shorten it after the fact.

Bumping this tag is now a manual, reviewed edit. Format is
main-<short-sha>, from docker/metadata-action type=sha with the branch
as prefix.
2026-08-13 21:38:29 +02:00
k.renner 8573cdf220 Merge pull request 'fix(trmnl): let Reloader restart the renderer on config and secret changes' (#489) from fix-trmnl-renderer-reloader into master
ci/woodpecker/cron/security-scan Pipeline was successful
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#489
2026-08-12 20:23:03 +00:00
claude-botandClaude Opus 5 507e86cb83 fix(trmnl): let Reloader restart the renderer on config and secret changes
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The renderer's init container writes /data/schedules.json ONCE, at pod
start, and nothing re-reads either source afterwards. So a ConfigMap or
Secret change lands in git, updates the object, and the running
renderer keeps the old values indefinitely.

The app re-reads schedules.json every 60 seconds, which makes it look
self-updating. It is not — it re-reads the FILE, and only the init
container writes the file.

This was hit three times in practice, each needing `kubectl delete pod`
by hand:

  * switching the theme to Graphite E-ink Light — the pod kept
    rendering with a theme name that no longer existed
  * setting the schedule's lang field to "de"
  * every re-mint of the BYOS tokens, which is the Secret half

The theme case is the instructive one: nothing errored. HA silently
served the default theme for a name that did not resolve, so the panel
just looked wrong, and I was about to debug the design.

Annotates both sources. Named per-resource rather than the blanket
reloader.stakater.com/auto: "true", matching terminus.yaml and
mosquitto-deployment.yaml.

Terminus got this annotation earlier; the renderer did not, despite
having the same init-container pattern and hitting the same failure in
the same session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:22:04 +02:00
k.renner b2ed2beae4 Merge pull request 'fix(trmnl): render the panel in German via the schedule's lang field' (#488) from fix-trmnl-render-lang into master
Reviewed-on: k.renner/homelab#488
2026-08-12 20:18:58 +00:00
claude-botandClaude Opus 5 f8c968c5a7 fix(trmnl): render the panel in German via the schedule's lang field
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
lang was null, so the screenshot session rendered in English — and the
dashboard grew a German weekday array, a German month array and a
fifteen-entry weather-condition dict to translate around it by hand.

The field is documented as "UI language code (HA mode only)" and calls
Home Assistant's own _selectLanguage() on the page before capture;
upstream's tests use lang: 'de'. So localisation is HA's job and the
field was sitting in the schedule template, already written out, set to
null because I copied the shape without asking what it did.

Pairs with homeassistant#220, which deletes both hand-written mappings:
the weather badge points at weather.garten directly and the date is a
device_class: timestamp the frontend formats.

The instance stays language "en". This affects only the render session,
so the panel is German without changing anyone's UI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:16:43 +02:00
k.renner 595e5eb7a3 Merge pull request 'fix(trmnl): disable Terminus session expiry, the other half of the token fix' (#487) from fix-trmnl-session-expiry into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#487
2026-08-12 20:02:38 +00:00
claude-botandClaude Opus 5 7e293d3de7 fix(trmnl): disable Terminus session expiry, the other half of the token fix
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The renderer stopped pushing and the panel preview froze. Every webhook:

  HTTP 401: Unauthorized - This session has expired, please login again

Terminus has TWO independent expiries and the earlier fix only addressed
one. API_ACCESS_TOKEN_PERIOD governs the JWT's own lifetime and is now
10 years. SESSION_EXPIRATION_ENABLED / SESSION_INACTIVITY_LIMIT govern
the SESSION BEHIND that token and were left at their defaults, so a
valid, decade-long token stopped being accepted after about half an
hour of session inactivity.

Worth being precise, because the two look alike and are not: the first
failure was 400 "expired JWT access token", the token itself. This is
401 "This session has expired", the session. Different mechanism,
different error, same symptom — a panel that silently stops updating.

I noticed both expiries when diagnosing the first one, wrote that
Terminus "has two independent expiries", and then fixed one and moved
on. That is the actual mistake here, not the missing variable.

Disabling session expiry is the state the renderer's own code already
assumes ("stock Terminus disables session expiration"), which is false
by default and has to be made true here — the same shape as the token
period. And it is the deciding constraint for an unattended appliance:
a session that expires needs a human with a password to restore it, and
a hallway panel has neither.

Reloader will roll the pod; the ConfigMap annotation added earlier
covers this. Existing tokens must be re-minted afterwards — the current
session is already dead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:01:34 +02:00
k.renner 9423381102 Merge pull request 'fix(trmnl): point the renderer at Graphite E-ink Light' (#486) from fix-trmnl-graphite-theme into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#486
2026-08-12 19:36:24 +00:00
claude-botandClaude Opus 5 ae12ce760c fix(trmnl): point the renderer at Graphite E-ink Light
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The renderer's own docs recommend Graphite by name — "Use e-ink theme
(Graphite recommended)", with theme=Graphite in the example — and the
Home Assistant community converges on it independently. The dashboard
now ships graphite-eink-light.yaml (homeassistant repo) in place of the
theme I hand-wrote from a spec sheet.

Pairs with homeassistant#216. The theme name must match the key inside
the vendored file exactly ("Graphite E-ink Light"); a name that does not
resolve fails silently — HA serves the default theme and the panel just
looks wrong rather than erroring.

Note upstream's example also passes `dark`. Not used here: the e-ink
LIGHT variant is the one that gives black-on-white, which is what an
e-paper panel wants. `dark: false` stays.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 21:34:55 +02:00
k.renner 641e8bce84 Merge pull request 'fix(trmnl): make Terminus issue long-lived API tokens' (#485) from fix-trmnl-token-lifetime into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#485
2026-08-12 17:34:47 +00:00
claude-botandClaude Opus 5 463e493ec0 docs(trmnl): cite the source for API_ACCESS_TOKEN_PERIOD
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The setting was treated as an unverified assumption in review, and I
repeated that framing. It was not one.

Upstream's doc/configuration.adoc documents the variable and its
1800-second default in as many words, and this instance corroborated it
empirically: a token minted here was rejected roughly thirty minutes
later, which is the documented default behaving exactly as written.
Documentation plus matching observed behaviour is not an assumption.

The reviewer's uncertainty was reasonable — Terminus's source is not
vendored in this repo, so from the checkout alone the claim is
unsupported. The failing was mine for citing nothing at the point of
use, which is the same gap as the /health probe earlier: the evidence
existed and was not written where a reader could find it. Quotes the
documentation inline so the question is answerable from the manifest.

Note this does not make the post-merge check redundant. Whether Terminus
honours the variable is settled here; whether this ConfigMap edit
actually reached the running process via Reloader is a separate
question, and that is what decoding the new token's exp claim answers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:30:03 +02:00
claude-botandClaude Opus 5 8031a0ad7a docs(openbao): record the trmnl renderer tokens as deliberately unrotated
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Review's remaining point: with API_ACCESS_TOKEN_PERIOD at 10 years,
nothing will ever expire the renderer's BYOS token or prompt anyone to
rotate it. Left undocumented, "never rotated" is indistinguishable from
"forgotten".

Records it where it is discoverable — the rotation directory itself,
whose purpose is that unrotated credentials are a visible choice rather
than an omission. Notes why there is no rotator: minting a Terminus JWT
requires the account password, which is precisely the secret this design
declined to hold in-cluster, so a rotator would have to store the thing
the long-lived token exists to avoid. Rotation is therefore manual, tied
to the Terminus account password changing.

Also flags that HA_ACCESS_TOKEN in the same OpenBao path is a different
case: secret-rotator already has an `ha-token` mode (home-assistant.yaml
uses it), so enrolling that one is a real follow-up rather than a
limitation, and doing so would leave only the BYOS pair unrotated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:25:32 +02:00
claude-botandClaude Opus 5 18c8b74baa fix(trmnl): let Reloader roll terminus on ConfigMap changes
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Review caught that this PR, as it stood, would not have fixed the
outage it describes.

terminus-config is consumed via envFrom, and env vars are read once at
container start — Kubernetes does not hot-reload them. The ConfigMap is
a plain resource rather than a configMapGenerator, so editing it does
not change the pod template hash and Flux will not roll the pods
either. Stakater Reloader is deployed but acts only on annotated
workloads, and this Deployment carried no annotation.

Net effect would have been a silent no-op: git and the ConfigMap object
would show a 10-year token period while the running app kept issuing
30-minute tokens, and the re-mint step in the PR plan would have
produced another short-lived token that looked correct and expired
anyway. The manifest and reality would disagree with nothing to
indicate it.

Annotates for terminus-config specifically rather than the blanket
reloader.stakater.com/auto: "true", matching the reasoning already
recorded on mosquitto-deployment.yaml — enrolling anything further
stays a visible, reviewable edit to that line.

Verified Reloader is actually running (kube-system/reloader-reloader).

This also removes a manual step rather than documenting one, which is
the point: a fix that needs a human to remember `kubectl rollout
restart` is not deployed, it is a runbook entry waiting to be missed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:20:29 +02:00
claude-botandClaude Opus 5 e84ea02100 fix(trmnl): make Terminus issue long-lived API tokens
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The renderer renders correctly and pushes nothing. Every webhook is
rejected:

  [webhook] Webhook failed: 400 {"error":"expired JWT access token"}
  [webhook] BYOS auth: refresh failed (400) {"error":"expired JWT ..."}

Terminus issues API JWTs with API_ACCESS_TOKEN_PERIOD, default 1800s.
Minting a replacement via POST /api/jwt requires a still-VALID access
token in the Authorization header, so an expired token cannot refresh
itself — the second line above is the refresh attempt being rejected by
the very expiry it was trying to repair. There is no recovery path once
the window closes.

That makes "mint once, store in OpenBao" unworkable for an unattended
appliance, which is what this stack is. The renderer's own code assumes
"stock Terminus disables session expiration and issues effectively
permanent tokens"; that is not true of this deployment. Rather than work
around the assumption in the client, this makes it true: sets
API_ACCESS_TOKEN_PERIOD to 10 years.

The trade is deliberate. A long-lived JWT with full Terminus API rights
now sits in OpenBao. The alternative was putting the Terminus account
PASSWORD into the cluster so an init container could mint a short-lived
token on every pod start — a strictly worse secret to hold, for a
LAN-only service whose token is revocable by rotating the account.

Also corrects the obtained_at comment on the renderer, which justified
stamping "now" by repeating that same false permanence assumption.
Stamping now is safe ONLY because of this setting; against the 30-minute
default it is actively harmful, because the app would believe a stale
token had a fresh window, skip its proactive refresh, and strand itself.
The comment now says so, and says what to do if this setting is reverted.

The period is fixed at mint time, so this does not revive the existing
tokens — they must be re-minted after it deploys.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:15:43 +02:00
k.renner fc53334b31 Merge pull request 'fix(trmnl): give Chromium a writable HOME and stamp obtained_at' (#484) from fix-trmnl-renderer-runtime into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#484
2026-08-12 17:02:10 +00:00
claude-botandClaude Opus 5 2dd3e8f0b2 fix(trmnl): bound the renderer's /tmp with a sized emptyDir
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Follow-up to the HOME=/tmp fix in this PR, from review.

Pointing HOME at /tmp was correct but left it on the container's
writable layer, which was harmless while Chromium crashed on launch and
wrote nothing. With the browser actually starting once a minute for the
life of the pod, crashpad state and profile data now accumulate there
with nothing to bound them — resources.limits sets memory only, so
there is no ephemeral-storage ceiling to catch it either.

Backs it with an emptyDir at a 512Mi sizeLimit, matching the /tmp mount
convention already used in pve-exporter-deployment.yaml. The point of
the limit is the failure mode, not the size: unbounded it degrades into
node disk pressure, which surfaces far from this pod and hurts whatever
else lands on that node; bounded it is an eviction of the renderer,
attributable on sight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:53:55 +02:00
claude-botandClaude Opus 5 d824263759 fix(trmnl): give Chromium a writable HOME and stamp obtained_at
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Two runtime faults from the first deploy. Neither is visible statically
— both needed the pod running to find.

1. The browser never launched. Every render died with

     chrome_crashpad_handler: --database is required

   Chromium cannot start without a writable HOME: crashpad builds its
   database path from it. Diagnosed on the pod rather than guessed —
   HOME=/, uid=1000, / is root-owned 0755 and not writable, /tmp is.
   The cause is OUR securityContext, not the image: forcing
   runAsUser 1000 with no matching passwd entry makes HOME fall back
   to "/". Sets HOME=/tmp; crash dumps are throwaway, so they do not
   belong on the PVC.

   Worth being precise about, because review had flagged the dropped
   capabilities as the likely Chromium risk and the comment I added
   said they were fine. The capabilities were fine. Something adjacent
   to that securityContext broke it anyway, and the comment now says
   to check HOME before loosening the capability block.

2. Even once the browser works, nothing would have been pushed. The
   scheduler logged

     BYOS tokens ... expired beyond the refresh window

   and skipped the webhook entirely. It derives token age from
   obtained_at, and the template shipped a literal 0, so
   freshly-minted tokens read as dating from 1970. The init container
   now stamps it with jq's `now * 1000 | floor` at write time.
   Verified the filter produces a 13-digit epoch-ms value. Stock
   Terminus disables session expiry and issues effectively permanent
   tokens, so a write-time stamp is honest enough.

Failure mode of the pair was quiet in the way this stack keeps being
quiet: pod Running and Ready, /health passing, schedule firing once a
minute, zero screens reaching Terminus.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:48:38 +02:00
k.renner b0f0f9242b Merge pull request 'feat(trmnl): deploy the headless dashboard renderer' (#483) from feat-trmnl-renderer into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#483
2026-08-12 16:43:16 +00:00
claude-botandClaude Opus 5 a79a5417e3 docs(trmnl): record why the probe path and dropped caps are safe
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Both were verified against the upstream image while writing this, but
the evidence never made it into the manifest, so review flagged them as
unverified assumptions. Fair — the reasoning was invisible.

/health is not a guess: the image's own Dockerfile declares
`HEALTHCHECK ... curl -f http://localhost:10000/health`, so the route is
part of its published contract. That is the difference from
terminus.yaml, which got TCP-only probes precisely because Terminus
publishes no health endpoint.

Dropping ALL capabilities is safe despite the container running headless
Chromium because the app launches it with --no-sandbox
--disable-setuid-sandbox --no-zygote --disable-dev-shm-usage, baked into
its own PUPPETEER_ARGS. That last flag is also why no /dev/shm emptyDir
is mounted. Both hold only for the pinned tag, so the comment says to
re-check on a bump.

No behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:40:51 +02:00
claude-botandClaude Opus 5 b6ada58b7a fix(trmnl): reach HA over the in-cluster Service, not the public name
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Review caught a blocking bug, correctly. HOME_ASSISTANT_URL pointed at
http://ha.renner.uno:8123, which is unusable from inside a pod.

That hostname is served by the home-assistant-bgp LoadBalancer with
externalTrafficPolicy: Local, advertised to the LAN over BGP. It is
not backed by an Ingress, so CoreDNS's k8s_gateway delegation does not
intercept it — a pod resolving it gets the real external address
(10.2.99.87) and hairpins off the LB.

This cluster has hit precisely this before with precisely this
Service. kubernetes/infrastructure/openbao/rotation/home-assistant.yaml
reaches HA from a pod for token rotation and says so outright: "The
public name is unusable from a pod (ingress-LB hairpin)." The same
language recurs in the gitea and woodpecker network policies. I took
the hostname from the repo's own "internal http://ha.renner.uno:8123"
note, which is true for a workstation and false from a pod.

Now uses home-assistant-bgp.home-assistant.svc.cluster.local:8123,
verified to exist with ClusterIP 10.96.79.213 on 8123/TCP.

Fixing the URL alone was not enough. The home-assistant namespace's
default-deny-ingress had no rule admitting trmnl pods on 8123 — the
only 8123 rule is scoped to the openbao secret-rotator — so traffic
would still have been dropped. Adds a companion ingress rule modelled
on the rotator's, scoped to the renderer's pod label rather than the
whole trmnl namespace, since Terminus shares that namespace and has no
business reaching Home Assistant. Only the ingress half is needed:
trmnl has no default-deny egress selecting the renderer, unlike
openbao.

Worth naming the failure mode, because it is the fourth "fails
quietly" case in this stack: nothing crashes. The schedule fires every
minute, each render errors on connect, and the panel simply never
updates.

Also updates the trmnl-daily Velero comment, which enumerated PVCs and
so undersold itself once trmnl-renderer-data existed — the schedule
selects the namespace minus CNPG-labelled PVCs, so new PVCs are
already covered without editing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:34:53 +02:00
k.renner 0581bc334a Merge pull request 'chore(ansible): retire the monitoring plays for decommissioned hosts' (#479) from retire-dead-monitoring into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
Reviewed-on: k.renner/homelab#479
2026-08-12 16:30:30 +00:00
claude-botandClaude Opus 5 f466f00029 fix(trmnl): model_id is the integer 1, not the name "v2"
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Resolves the one field flagged as unverified on the renderer PR, now
that a Terminus API token exists to check it with.

GET /api/screens on the running instance returns model_id as a NUMBER
(an existing welcome screen carries model_id 11), and GET /api/models
gives TRMNL X as id 1, name "v2", 1872x1404, 16 colours. Upstream's
example value "og_1bit" matches no model on this server at all — the
2-bit OG model here is id 11, name "og_plus" — so that example was
illustrative rather than a real identifier.

Would have surfaced as Terminus rejecting every pushed screen with a
422, i.e. a permanently blank panel with the renderer looking healthy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 18:27:57 +02:00
security-bot 3f4846bbbc chore(image-automation): bump ghcr.io/music-assistant/server:2.9.13
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-12 14:35:43 +00:00
claude-botandClaude Opus 5 7e2a439b9e feat(trmnl): deploy the headless dashboard renderer
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Phase 4. Screenshots the Home Assistant e-ink dashboard, dithers it to
16 greys and hands it to Terminus, which the panel then polls.

Upstream ships this as a Home Assistant add-on. Ours is a plain HA
container with no Supervisor, so it runs standalone — viable because
the app prefers HOME_ASSISTANT_URL/ACCESS_TOKEN env vars over
/data/options.json and authenticates with an ordinary long-lived
token. SUPERVISOR_TOKEN is deliberately left unset: it only flips an
internal isAddOn flag and would switch the app to add-on behaviour.

The non-obvious part is the direction of travel. With the byos-hanami
webhook in `uri` delivery mode the renderer does NOT push image bytes
— it POSTs a URL pointing at its own /output/<file> endpoint and
TERMINUS FETCHES THE IMAGE BACK. That is why this lives in the trmnl
namespace: the fetch is intra-namespace and already permitted by
default-deny-ingress. Splitting the two across namespaces would have
needed a new policy for a dependency that is easy to miss.

Fully declarative, no clicks. Schedules are a JSON file at
/data/schedules.json which the app re-reads every 60s, so the whole
schedule — viewport 1872x1404, floyd-steinberg at bitDepth 4, the
trmnl-eink theme, the dashboard path — is reviewable in git rather
than typed into the app's web UI. An init container splices the BYOS
tokens into the ConfigMap template with jq; jq rather than sed because
the tokens are JWTs going into a JSON string and sed would corrupt on
any character it treats as special (verified with a token containing
/ + &).

No HTTP in the init container on purpose. The app exposes a
/api/byos/login proxy that trades a Terminus username+password for
these tokens, but using it would mean shipping the Terminus PASSWORD
into the cluster and making every pod start depend on a network call.
Stock Terminus disables session expiry and issues effectively
permanent tokens, so they are minted once out of band and kept in
OpenBao; the password never enters the cluster.

Three things that are load-bearing and would fail quietly otherwise:

  * The PVC at /data is REQUIRED. The app picks its data directory by
    testing whether /data exists — with nothing mounted it silently
    falls back to /app/data in the container, so schedules and rendered
    output vanish on restart and the panel goes blank.
  * The Service is ClusterIP with NO Ingress. The app's web UI on this
    port has no authentication whatsoever, per upstream, so it must not
    be routable from the LAN.
  * The memory limit is 2Gi, not the 512Mi upstream's troubleshooting
    note suggests. The app's own watchdog restarts it past 900MB but
    only measures its OWN rss — Chromium is a separate process, and it
    is rendering a 1872x1404 page.

The image is arch-suffixed (trmnl-ha-amd64); upstream publishes no
multi-arch manifest. Pinned and not enrolled in image automation,
since a bad bump here surfaces as a corrupted or blank panel rather
than a failed rollout — rationale recorded in the image-automation
README alongside Terminus's.

PREREQUISITE: homelab/apps/trmnl/renderer must be seeded in OpenBao
with HA_ACCESS_TOKEN, BYOS_ACCESS_TOKEN and BYOS_REFRESH_TOKEN before
this merges, or the ExternalSecret never syncs, the init container
blocks, and the trmnl Kustomization fails its wait.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 05:58:26 +02:00
security-bot dc1ea300a8 chore(image-automation): bump ghcr.io/music-assistant/server:2.9.12
ci/woodpecker/cron/security-scan Pipeline failed
ci/woodpecker/cron/e2e Pipeline failed
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-11 00:35:18 +00:00
k.renner 73c2674344 Merge pull request 'fix(trmnl): request an explicit device LB address, document the sidekiq race' (#481) from fix-trmnl-lb-address into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#481
2026-08-10 21:08:53 +00:00
claude-botandClaude Opus 5 931810e28b fix(trmnl): request an explicit device LB address, document the sidekiq race
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Two things the first reconcile surfaced.

Cilium IPAM allocated 10.2.99.0 for the device LoadBalancer. Nothing
requested it — the Service carried no lbipam.cilium.io/ips annotation, so
IPAM handed out the first free address in the pool block, which is the base
address of the 10.2.99.0-10.2.99.255 range. Each LB address is advertised
to the UDM as its own BGP route, so it is routable in principle, but a .0
host address is exactly the sort of thing constrained embedded IP stacks
get wrong, and the TRMNL firmware is the one client that has to work here.
Requesting .89 explicitly also brings this in line with every other LB
Service in the repo, all of which pin their address, and matches the
fd99::89 IPAM had already chosen for v6.

Omitting the annotation was my call on the original PR, on the reasoning
that the device is provisioned with a hostname so the address could move
freely. That reasoning still holds; it just is not worth spending it on an
address nobody picked.

Second, the sidekiq container exits 1 once on a fresh deploy with
`PG::UndefinedTable: relation "account" does not exist` — it loses the race
against the web container's migration, then comes up healthy on the
restart. Observed on the first deploy, converged in about a second. There
is no upstream ordering hook, and gating it behind an initContainer that
probes for a named table would couple us to Terminus's schema, so this is
left to Kubernetes' restart deliberately. Documented in place so the next
person reading `RESTARTS 1` can tell the benign case from a real one.

Verified on the live cluster before this commit: Kustomization Ready, tr-pg
"Cluster in healthy state" on PostgreSQL 18, both ExternalSecrets
SecretSynced, terminus and valkey Running.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 23:04:17 +02:00
security-bot 2d8324b3f9 chore(image-automation): bump valkey/valkey:9.1.1-alpine
ci/woodpecker/cron/alert-monitor Pipeline was successful
Updates managed by image-automation-controller.
2026-08-10 21:00:08 +00:00
k.renner b529864453 Merge pull request 'feat(trmnl): deploy Terminus BYOS server for the entrance e-ink panel' (#480) from feat-trmnl-terminus into master
Reviewed-on: k.renner/homelab#480
2026-08-10 20:59:01 +00:00
claude-botandClaude Opus 5 ac2d728108 feat(trmnl): deploy Terminus BYOS server for the entrance e-ink panel
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Phase 1 of the TRMNL X dashboard plan: stand up the self-hosted Terminus
(BYOS) server the panel will poll. The Home Assistant dashboard, the e-ink
theme and the screenshot renderer are later phases and are not in here.

Phase 0 verification, all confirmed against upstream before writing any of
this:

  * TRMNL X IS a first-class model in Terminus (id v2 / label "TRMNL X",
    1872x1404, 16 colours, 4-bit) — the stop-and-report condition in the
    plan does not trigger. Note it is NOT in config/db/seeds.rb: the model
    list is populated by the MODEL_SYNCHRONIZER job pulling trmnl.com, so
    the Models list is empty until that job has run once.
  * The HA add-on can run standalone. Our HA is plain HA Core, not
    Supervised, but the add-on reads env vars in preference to
    /data/options.json and authenticates with an ordinary long-lived
    token — SUPERVISOR_TOKEN only flips an isAddOn boolean. So the
    fallback path in the plan (upstream container, not add-on) is viable.
    That deployment lands with Phase 4.

Deviations from the repo's house patterns, each deliberate:

  * tr-pg runs PostgreSQL 18, not the standard 16. Terminus ships a
    committed config/db/structure.sql containing `SET transaction_timeout
    = 0;`, a GUC that does not exist before PG 17, so schema load against
    a 16 server fails outright. This is the first non-16 CNPG cluster
    here. The existing 16 pin is inherited from 739a67e9 (Apr 2026) and
    has no documented rationale.
  * The device endpoint is cleartext HTTP on a Cilium BGP LoadBalancer
    (trmnl-device.renner.uno) rather than an Ingress. The firmware cannot
    be given the internal CA, and a TLS-less Ingress would be a new
    pattern for this repo — a dedicated LB address is not. The admin UI
    keeps a normal TLS Ingress at trmnl.renner.uno; both front the same
    Service. Same split as home.renner.uno / ha.renner.uno.
  * ghcr.io/usetrmnl/terminus is NOT enrolled in image automation despite
    publishing a usable tag stream. It serves the firmware and
    /api/display contract to a wall-mounted device, upstream is pre-1.0
    with no releases or changelog, and an unattended bump that breaks
    either leaves the panel dark. Bumps are manual and reviewed. The
    Valkey sidecar is enrolled — it touches no device-facing contract.
  * Egress is an FQDN allowlist (trmnl.com, plus the S3 host that serves
    firmware binaries), not toEntities: world. The S3 host is returned by
    the API at request time rather than being a source constant, so it can
    drift; the policy comment says where to look when firmware sync
    starts failing. This also means Terminus's Extensions/Poll feature,
    which fetches admin-configured URLs, will not work — accepted, since
    screen content is pushed in by the renderer.
  * The CNPG pods are deliberately left OUT of that egress policy so the
    barman WAL archive path is not behind a rule that fails closed.

Images verified to exist in their registries before pinning:
ghcr.io/usetrmnl/terminus:0.68.0, valkey/valkey:9.1-alpine,
ghcr.io/cloudnative-pg/postgresql:18.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 22:54:38 +02:00
claude-botandClaude Opus 5 8e5b91e6d6 chore(ansible): retire the monitoring plays for decommissioned hosts
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Prometheus and Loki run in Kubernetes -- statefulsets
prometheus-kube-prometheus-stack-prometheus and the Loki release, both up 93
days. The standalone VMs these plays targeted are gone: neither
prometheus.srv.renner.uno nor loki.srv.renner.uno resolves at all, and the
proxmox_logging group install-log-aggregation.yml targets is not in the
inventory.

So these rendered scrape configs, exporter installs and log shippers for
hosts that do not exist. Removed:

  plays/install-prometheus.yml          -> prometheus.srv.renner.uno (gone)
  plays/install-blackbox-exporter.yml   -> prometheus.srv.renner.uno (gone)
  plays/install-loki.yml                -> loki.srv.renner.uno (gone)
  plays/install-log-aggregation.yml     -> proxmox_logging (no such group)
  roles/prometheus-server, snmp-exporter, pve-exporter, unifi-poller,
  roles/blackbox-exporter, loki-server, promtail

Each role was consumed only by the plays above; promtail was referenced by
nothing at all. Verified no dangling references remain.

plays/update.yml imported install-log-aggregation.yml, so deleting it alone
would have broken the fleet update play. That import is removed and
update.yml re-syntax-checked.

This also retires #473, which changed IPv4 literals to FQDNs in
prometheus.yml.j2 and nodes.yml.j2. That was a change to a template nothing
renders. Its PR body described the instance-label consequences for
dashboards and alert rules, which was fiction -- no dashboard reads those
labels. When it would not apply I reported it as blocked on Proxmox API
credentials; the actual reason was that the target host had not existed for
months, and `--list-hosts` returning 0 was the evidence I misread.

Worth checking separately, not assumed here: whether anything scrapes the
PVE node_exporter and chrony_exporter today. install-node-exporter-pve.yml
still installs them on pve_cluster, but the config that pointed a Prometheus
at them was in the templates just deleted. If that was the only reference,
those metrics have been dark since the VM went away -- and the FQDN change
in #473 would have masked the question rather than raising it.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 22:10:50 +02:00
k.renner c8bc6380a1 Merge pull request 'fix(heating-can): give supervisord a control socket' (#478) from heating-can-supervisorctl into master
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#478
2026-08-10 19:53:51 +00:00
claude-botandClaude Opus 5 e3dea169b0 fix(heating-can): give supervisord a control socket
ci/woodpecker/push/build-ci-images Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The image shipped without [unix_http_server], [rpcinterface:supervisor] or
[supervisorctl], so supervisorctl could not connect:

  Error: .ini file does not include supervisorctl section

That means no way to query or restart an individual service inside the
container -- docker logs was the only introspection. Per-service lifecycle
management was the whole argument for running a supervisor instead of one
process per container, and the first revision did not actually deliver it.

Unix socket only, inside the container. Nothing outside needs it, and this
container runs with host networking, so a TCP listener would be exposed on
the host.

The build-time self-test now asserts the section exists and supervisorctl
runs, so this cannot regress silently the way it shipped.

NOT fixed here, because my diagnosis was wrong: the repeated
"mqtt broker disconnected. reason_code = Unspecified error" is not a client
ID collision. E3onCAN has no -mcid option at all -- verified against the
pinned source -- and generates its own ID; the native broker log showed
E3onCANclient.py-e380-7f847281 alongside open3e-edge, so the two never
collided. I nearly "fixed" it by passing a flag that does not exist.

Telemetry is flowing (each reconnect succeeds), so this is churn rather than
an outage, but the cause is still unidentified. The next step is the broker's
own log on edge-1 -- mosquitto records the disconnect reason, which the
client side reports only as "Unspecified error".

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:49:00 +02:00
k.renner 6384223415 Merge pull request 'fix(heating-can): stop the native services before removing their units' (#477) from fix-native-teardown into master
Reviewed-on: k.renner/homelab#477
2026-08-10 19:48:34 +00:00
claude-botandClaude Opus 5 1b7e3dc100 fix(heating-can): stop the native services before removing their units
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The migration did not stop anything. It ran stop + disable + mask in one
call with failed_when: false, then deleted the unit files. Masking a unit
whose file still exists conflicts, so the call failed, failed_when hid the
failure, nothing was stopped -- and the next task deleted the unit files out
from under two running processes.

That left systemctl is-enabled reporting "not-found", which looks clean,
while is-active still said "active". I reported the cutover successful on
the strength of the former.

On 2026-08-10 the result was two open3e instances issuing UDS requests on
the same CAN bus from 19:28 to 19:37, taking over each other's
open3e-edge MQTT session -- the exact failure the block's own comment said
it existed to prevent.

Fixed in three parts:

  - Stop and disable FIRST, and do not swallow the result. Only units that
    actually exist are touched (stat first), so a fresh host does not need
    an error hidden to get through.

  - No masking. It writes a symlink at the same path the next task removes,
    so the two cancel out. With the unit files gone there is nothing to
    start.

  - Assert the invariant that actually matters -- no native process on the
    CAN bus -- rather than inferring it from systemd reporting success.
    pgrep, then fail loudly with the surviving command lines and why it
    matters.

Verified against the live host: ok=15 changed=1 failed=0, the assert passing
with "No native CAN processes remain; safe to start the container", and the
run now idempotent (the single change is the compose up, not the teardown).

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:45:47 +02:00
k.renner 1ee4c2b61c Merge pull request 'refactor(edge): split the plays by capability; pin open3e and register it with the audit' (#475) from edge-capability-groups into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#475
2026-08-10 19:26:11 +00:00
92 changed files with 2532 additions and 2133 deletions
@@ -1,19 +0,0 @@
---
# Deploy Blackbox Exporter to Prometheus server for endpoint probing
# Usage: ansible-playbook -i inventory/ plays/install-blackbox-exporter.yml
- name: Install Blackbox Exporter
hosts: prometheus.srv.renner.uno
become: yes
roles:
- blackbox-exporter
post_tasks:
- name: Verify blackbox_exporter is running
uri:
url: "http://127.0.0.1:9115/metrics"
status_code: 200
register: exporter_health
retries: 3
delay: 5
until: exporter_health.status == 200
-86
View File
@@ -1,86 +0,0 @@
---
# Setup Log Aggregation Infrastructure
# This playbook configures the complete logging stack:
# 1. Loki server for log storage and querying
# 2. Alloy agents on all VMs/CTs for log collection
# 3. Syslog receiver for remote devices (Synology NAS, network devices)
#
# Usage: ansible-playbook -i inventory/ plays/setup-log-aggregation.yml
#
# To limit to specific hosts:
# --limit loki.srv.renner.uno # Only configure Loki server
# --limit proxmox_all_running # Only configure log agents
- name: Setup Loki Log Aggregation Server
hosts: proxmox_logging
become: yes
gather_facts: yes
roles:
- common
- loki-server
post_tasks:
- name: Verify Loki is running
uri:
url: "http://127.0.0.1:3100/ready"
status_code: 200
register: loki_health
retries: 5
delay: 10
until: loki_health.status == 200
- name: Display Loki status
debug:
msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
- name: Deploy Monitoring Agent to Loki Server (with syslog receiver)
hosts: proxmox_logging
become: yes
gather_facts: yes
vars:
loki_endpoint: "http://loki.srv.renner.uno:3100"
prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
# Syslog receiver settings are defined in host_vars/loki.srv.renner.uno/vars.yml:
# - alloy_syslog_receiver_enabled: true
# - alloy_syslog_receiver_port: 1514
# - alloy_syslog_receiver_protocol: "udp"
# - alloy_syslog_receiver_format: "raw"
# - alloy_stability_level: "experimental"
roles:
- monitoring-agent
post_tasks:
- name: Display syslog receiver info
debug:
msg: |
Syslog receiver is listening on UDP port {{ alloy_syslog_receiver_port | default(1514) }}
Configure remote devices to send syslog to: {{ inventory_hostname }}:{{ alloy_syslog_receiver_port | default(1514) }}
when: alloy_syslog_receiver_enabled | default(false)
- name: Deploy Monitoring Agent to All Running VMs/CTs
hosts: proxmox_all_running:!proxmox_logging:!proxmox_noansible
become: yes
gather_facts: yes
serial: 10 # Deploy to 10 hosts at a time to avoid overwhelming the network
vars:
loki_endpoint: "http://loki.srv.renner.uno:3100"
prometheus_remote_write_endpoint: "http://prometheus.srv.renner.uno:9090/api/v1/write"
roles:
- monitoring-agent
post_tasks:
- name: Verify Alloy is running
systemd:
name: alloy
state: started
register: alloy_service
- name: Display agent status
debug:
msg: "Monitoring agent deployed successfully on {{ inventory_hostname }}"
when: alloy_service is defined
-26
View File
@@ -1,26 +0,0 @@
---
# Install Loki Log Aggregation Server
# Usage: ansible-playbook -i inventory/ plays/install-loki.yml
- name: Install Loki Log Aggregation Server
hosts: loki.srv.renner.uno
become: yes
gather_facts: yes
roles:
- common
- loki-server
post_tasks:
- name: Verify Loki is running
uri:
url: "http://127.0.0.1:3100/ready"
status_code: 200
register: loki_health
retries: 5
delay: 10
until: loki_health.status == 200
- name: Display Loki status
debug:
msg: "Loki is running and healthy at http://{{ inventory_hostname }}:3100"
-43
View File
@@ -1,43 +0,0 @@
---
# Install Prometheus Metrics Server
# Usage: ansible-playbook -i inventory/ plays/install-prometheus.yml
- name: Install Prometheus Metrics Server
hosts: prometheus.srv.renner.uno
become: yes
gather_facts: yes
vars:
# Synology NAS SNMP targets
snmp_synology_targets:
- name: nas-1
target: "10.2.40.50"
- name: nas-2
target: "10.3.40.57"
# SNMPv3 credentials from vault
snmp_use_v3: true
snmp_v3_username: "{{ vault_snmp_v3_username }}"
snmp_v3_auth_password: "{{ vault_snmp_v3_auth_password }}"
snmp_v3_priv_password: "{{ vault_snmp_v3_priv_password }}"
roles:
- common
- prometheus-server
- snmp-exporter
- pve-exporter
- unifi-poller
post_tasks:
- name: Verify Prometheus is running
uri:
url: "http://127.0.0.1:9090/-/ready"
status_code: 200
register: prometheus_health
retries: 5
delay: 10
until: prometheus_health.status == 200
- name: Display Prometheus status
debug:
msg: "Prometheus is running and healthy at http://{{ inventory_hostname }}:9090"
-1
View File
@@ -16,7 +16,6 @@
- import_playbook: install-grist.yml
- import_playbook: install-music-assistant.yml
- import_playbook: update-docker.yml
- import_playbook: install-log-aggregation.yml
- import_playbook: update-pve-rolling.yml
- name: Update Summary
@@ -1,26 +0,0 @@
---
blackbox_exporter_version: "0.25.0"
blackbox_exporter_port: 9115
blackbox_exporter_user: "blackbox_exporter"
blackbox_exporter_group: "blackbox_exporter"
# Endpoints to probe
blackbox_http_targets:
- name: "gitea"
url: "https://gitea.renner.uno"
- name: "immich"
url: "https://immich.renner.uno"
- name: "authentik"
url: "https://auth.renner.uno"
- name: "home-assistant"
url: "https://ha.renner.uno"
- name: "grafana"
url: "https://grafana.renner.uno"
- name: "prometheus"
url: "https://prometheus.renner.uno"
blackbox_dns_targets: []
blackbox_tcp_targets:
- name: "postgres-ha"
target: "10.2.40.135:5432"
@@ -1,6 +0,0 @@
---
- name: Restart blackbox_exporter
systemd:
name: blackbox_exporter
state: restarted
daemon_reload: yes
@@ -1,80 +0,0 @@
---
- name: Create blackbox_exporter group
group:
name: "{{ blackbox_exporter_group }}"
system: yes
state: present
- name: Create blackbox_exporter user
user:
name: "{{ blackbox_exporter_user }}"
group: "{{ blackbox_exporter_group }}"
system: yes
shell: /usr/sbin/nologin
createhome: no
state: present
- name: Create blackbox_exporter directories
file:
path: "{{ item }}"
state: directory
owner: "{{ blackbox_exporter_user }}"
group: "{{ blackbox_exporter_group }}"
mode: '0755'
loop:
- /etc/blackbox_exporter
- /var/lib/blackbox_exporter
- name: Download blackbox_exporter
get_url:
url: "https://github.com/prometheus/blackbox_exporter/releases/download/v{{ blackbox_exporter_version }}/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64.tar.gz"
dest: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
mode: '0644'
- name: Extract blackbox_exporter
unarchive:
src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
dest: /tmp
remote_src: yes
creates: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
- name: Install blackbox_exporter binary
copy:
src: "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64/blackbox_exporter"
dest: /usr/local/bin/blackbox_exporter
owner: root
group: root
mode: '0755'
remote_src: yes
notify: Restart blackbox_exporter
- name: Deploy blackbox_exporter configuration
template:
src: blackbox.yml.j2
dest: /etc/blackbox_exporter/blackbox.yml
owner: "{{ blackbox_exporter_user }}"
group: "{{ blackbox_exporter_group }}"
mode: '0644'
notify: Restart blackbox_exporter
- name: Deploy blackbox_exporter systemd service
template:
src: blackbox_exporter.service.j2
dest: /etc/systemd/system/blackbox_exporter.service
mode: '0644'
notify: Restart blackbox_exporter
- name: Enable and start blackbox_exporter
systemd:
name: blackbox_exporter
state: started
enabled: yes
daemon_reload: yes
- name: Clean up downloaded files
file:
path: "{{ item }}"
state: absent
loop:
- "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.tar.gz"
- "/tmp/blackbox_exporter-{{ blackbox_exporter_version }}.linux-amd64"
@@ -1,60 +0,0 @@
modules:
http_2xx:
prober: http
timeout: 5s
http:
valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
valid_status_codes: [] # Defaults to 2xx
method: GET
follow_redirects: true
fail_if_ssl: false
fail_if_not_ssl: false
tls_config:
insecure_skip_verify: false
http_post_2xx:
prober: http
timeout: 5s
http:
method: POST
valid_status_codes: []
tcp_connect:
prober: tcp
timeout: 5s
# DNS module for external domain resolution (tests upstream/cache)
dns_external:
prober: dns
timeout: 5s
dns:
query_name: "google.com"
query_type: "A"
valid_rcodes:
- NOERROR
# DNS module for internal domain resolution (tests local resolution)
dns_internal:
prober: dns
timeout: 5s
dns:
query_name: "k8s-1.srv.renner.uno"
query_type: "A"
valid_rcodes:
- NOERROR
# DNS module for common cached domain
dns_cached:
prober: dns
timeout: 5s
dns:
query_name: "cloudflare.com"
query_type: "A"
valid_rcodes:
- NOERROR
icmp:
prober: icmp
timeout: 5s
icmp:
preferred_ip_protocol: "ip4"
@@ -1,22 +0,0 @@
[Unit]
Description=Prometheus Blackbox Exporter
Documentation=https://github.com/prometheus/blackbox_exporter
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User={{ blackbox_exporter_user }}
Group={{ blackbox_exporter_group }}
ExecStart=/usr/local/bin/blackbox_exporter \
--config.file=/etc/blackbox_exporter/blackbox.yml \
--web.listen-address=:{{ blackbox_exporter_port }}
Restart=on-failure
RestartSec=5s
# Allow ICMP probing (requires CAP_NET_RAW)
AmbientCapabilities=CAP_NET_RAW
CapabilityBoundingSet=CAP_NET_RAW
[Install]
WantedBy=multi-user.target
+56 -11
View File
@@ -34,17 +34,42 @@
# stopped and masked rather than merely disabled -- masked so a stray
# `systemctl start` or a leftover dependency cannot bring them back alongside
# the containers.
- name: Stop and mask the native CAN services
ansible.builtin.systemd:
name: "{{ item }}"
state: stopped
enabled: false
masked: true
- name: Find which native CAN units still exist
ansible.builtin.stat:
path: "/etc/systemd/system/{{ item }}.service"
loop:
- open3e
- e3oncan
failed_when: false
register: heating_can_native_units
# Stop and disable BEFORE anything else, and do NOT swallow the result.
#
# The first version did stop + disable + mask in one call with
# failed_when: false, then deleted the unit files. Masking a unit whose file
# still exists conflicts, so the call failed, failed_when hid it, nothing was
# stopped -- and the next task deleted the unit files out from under two
# running processes. systemctl is-enabled then reported "not-found" and looked
# clean while is-active still said "active".
#
# The result on 2026-08-10 was two open3e instances issuing UDS requests on the
# same CAN bus for nine minutes, fighting over the open3e-edge client ID and
# kicking each other off the broker.
#
# Only units that actually exist are touched, so a fresh host does not need an
# error swallowed to get through this.
- name: Stop and disable the native CAN services
ansible.builtin.systemd:
name: "{{ item.item }}"
state: stopped
enabled: false
loop: "{{ heating_can_native_units.results }}"
loop_control:
label: "{{ item.item }}"
when: item.stat.exists
# Masking is deliberately NOT used. It writes a symlink at the same path the
# next task removes, so the two cancel out -- which is half of how the original
# failed. With the unit files gone there is nothing left to start.
- name: Remove the native unit files
ansible.builtin.file:
path: "/etc/systemd/system/{{ item }}.service"
@@ -54,10 +79,30 @@
- e3oncan
notify: Reload systemd
# The venvs are the point of this change -- leaving them would leave the thing
# that churned. Their configuration is NOT removed: devices.json, the overlay
# and the env file are still rendered below and bind-mounted into the
# containers.
# The invariant that actually matters is "no native process is on the CAN bus",
# and systemd reporting success is not the same claim. Two open3e instances
# polling one bus is the failure this whole block exists to prevent, so it is
# asserted directly rather than inferred from the calls above.
- name: Check for surviving native CAN processes
ansible.builtin.command:
argv: [pgrep, -af, "edge-host/(open3e|e3oncan)"]
register: heating_can_survivors
changed_when: false
failed_when: false
- name: Assert no native CAN process survived
ansible.builtin.assert:
that:
- heating_can_survivors.stdout | default('') | length == 0
fail_msg: >-
A native open3e/e3oncan process is still running after the units were
stopped and removed:
{{ heating_can_survivors.stdout_lines | default([]) | join('; ') }}.
Starting the container now would put two readers on the same CAN bus --
two open3e instances issue competing UDS requests and take over each
other's MQTT session. Kill the survivors, then re-run.
success_msg: No native CAN processes remain; safe to start the container.
- name: Remove the native venvs
ansible.builtin.file:
path: "{{ item }}"
@@ -1,44 +0,0 @@
---
# Loki Server Configuration
# Version and download
loki_version: "3.4.2"
loki_arch: "linux-amd64"
loki_download_url: "https://github.com/grafana/loki/releases/download/v{{ loki_version }}/loki-{{ loki_arch }}.zip"
loki_bin_path: "/usr/local/bin/loki"
# Directories
loki_data_dir: "/var/lib/loki"
loki_config_dir: "/etc/loki"
loki_config_file: "{{ loki_config_dir }}/loki-config.yaml"
# Server configuration
loki_http_listen_port: 3100
loki_grpc_listen_port: 9096
loki_http_listen_address: "0.0.0.0"
# Retention settings (90 days)
loki_retention_period: "2160h" # 90 days
loki_retention_delete_delay: "2h"
# Storage configuration
loki_storage_type: "filesystem"
loki_chunks_directory: "{{ loki_data_dir }}/chunks"
loki_rules_directory: "{{ loki_data_dir }}/rules"
# Index configuration
loki_index_prefix: "loki_index_"
loki_index_period: "24h"
# Limits
loki_ingestion_rate_mb: 10
loki_ingestion_burst_size_mb: 20
loki_max_streams_per_user: 10000
loki_max_entries_limit_per_query: 5000
# Query settings
loki_max_query_length: "720h" # 30 days max query span
# Systemd service
loki_user: "loki"
loki_group: "loki"
@@ -1,10 +0,0 @@
---
- name: Restart loki
systemd:
name: loki
state: restarted
daemon_reload: yes
- name: Reload systemd
systemd:
daemon_reload: yes
-95
View File
@@ -1,95 +0,0 @@
---
# Loki Server Installation and Configuration
- name: Install dependencies
apt:
name:
- unzip
- curl
state: present
update_cache: yes
- name: Create loki user
user:
name: "{{ loki_user }}"
system: yes
shell: /usr/sbin/nologin
create_home: no
- name: Create directories
file:
path: "{{ item }}"
state: directory
owner: "{{ loki_user }}"
group: "{{ loki_group }}"
mode: '0755'
loop:
- "{{ loki_config_dir }}"
- "{{ loki_data_dir }}"
- "{{ loki_chunks_directory }}"
- "{{ loki_rules_directory }}"
- "{{ loki_data_dir }}/index"
- "{{ loki_data_dir }}/index_cache"
- "{{ loki_data_dir }}/compactor"
- name: Download Loki
get_url:
url: "{{ loki_download_url }}"
dest: "/tmp/loki.zip"
mode: '0644'
- name: Unzip Loki
unarchive:
src: "/tmp/loki.zip"
dest: "/tmp"
remote_src: yes
- name: Install Loki binary
copy:
src: "/tmp/loki-{{ loki_arch }}"
dest: "{{ loki_bin_path }}"
mode: '0755'
remote_src: yes
notify: Restart loki
- name: Create Loki configuration
template:
src: loki-config.yaml.j2
dest: "{{ loki_config_file }}"
owner: "{{ loki_user }}"
group: "{{ loki_group }}"
mode: '0644'
notify: Restart loki
- name: Create Loki systemd service
template:
src: loki.service.j2
dest: /etc/systemd/system/loki.service
mode: '0644'
notify:
- Reload systemd
- Restart loki
- name: Enable and start Loki service
systemd:
name: loki
state: started
enabled: yes
daemon_reload: yes
- name: Wait for Loki to be ready
uri:
url: "http://127.0.0.1:{{ loki_http_listen_port }}/ready"
status_code: 200
register: result
until: result.status == 200
retries: 30
delay: 5
- name: Clean up downloaded files
file:
path: "{{ item }}"
state: absent
loop:
- "/tmp/loki.zip"
- "/tmp/loki-{{ loki_arch }}"
@@ -1,72 +0,0 @@
# Loki Configuration
# Generated by Ansible
auth_enabled: false
server:
http_listen_address: {{ loki_http_listen_address }}
http_listen_port: {{ loki_http_listen_port }}
grpc_listen_port: {{ loki_grpc_listen_port }}
log_level: info
common:
instance_addr: 127.0.0.1
path_prefix: {{ loki_data_dir }}
storage:
filesystem:
chunks_directory: {{ loki_chunks_directory }}
rules_directory: {{ loki_rules_directory }}
replication_factor: 1
ring:
kvstore:
store: inmemory
query_range:
results_cache:
cache:
embedded_cache:
enabled: true
max_size_mb: 100
schema_config:
configs:
- from: 2024-01-01
store: tsdb
object_store: filesystem
schema: v13
index:
prefix: {{ loki_index_prefix }}
period: {{ loki_index_period }}
storage_config:
tsdb_shipper:
active_index_directory: {{ loki_data_dir }}/index
cache_location: {{ loki_data_dir }}/index_cache
compactor:
working_directory: {{ loki_data_dir }}/compactor
compaction_interval: 10m
retention_enabled: true
retention_delete_delay: {{ loki_retention_delete_delay }}
retention_delete_worker_count: 150
delete_request_store: filesystem
limits_config:
retention_period: {{ loki_retention_period }}
ingestion_rate_mb: {{ loki_ingestion_rate_mb }}
ingestion_burst_size_mb: {{ loki_ingestion_burst_size_mb }}
max_streams_per_user: {{ loki_max_streams_per_user }}
max_entries_limit_per_query: {{ loki_max_entries_limit_per_query }}
max_query_length: {{ loki_max_query_length }}
reject_old_samples: true
reject_old_samples_max_age: 168h # 7 days
ruler:
alertmanager_url: ""
storage:
type: local
local:
directory: {{ loki_rules_directory }}
analytics:
reporting_enabled: false
@@ -1,25 +0,0 @@
[Unit]
Description=Loki Log Aggregation System
Documentation=https://grafana.com/docs/loki/latest/
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ loki_user }}
Group={{ loki_group }}
ExecStart={{ loki_bin_path }} -config.file={{ loki_config_file }}
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=10s
LimitNOFILE=65536
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths={{ loki_data_dir }}
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -1,48 +0,0 @@
---
# Prometheus Server Configuration
# Version and download
prometheus_version: "3.2.1"
prometheus_arch: "linux-amd64"
prometheus_download_url: "https://github.com/prometheus/prometheus/releases/download/v{{ prometheus_version }}/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}.tar.gz"
prometheus_bin_path: "/usr/local/bin/prometheus"
prometheus_promtool_path: "/usr/local/bin/promtool"
# Directories
prometheus_data_dir: "/var/lib/prometheus"
prometheus_config_dir: "/etc/prometheus"
prometheus_config_file: "{{ prometheus_config_dir }}/prometheus.yml"
prometheus_rules_dir: "{{ prometheus_config_dir }}/rules"
prometheus_targets_dir: "{{ prometheus_config_dir }}/targets"
# Server configuration
prometheus_listen_address: "0.0.0.0"
prometheus_listen_port: 9090
prometheus_web_external_url: ""
# Retention settings (90 days)
prometheus_retention_time: "90d"
prometheus_retention_size: "30GB"
# Scrape settings
prometheus_scrape_interval: "30s"
prometheus_evaluation_interval: "30s"
prometheus_scrape_timeout: "10s"
# Remote write (for Alloy agents)
prometheus_remote_write_enabled: true
# Systemd service
prometheus_user: "prometheus"
prometheus_group: "prometheus"
# Alert notification endpoints
prometheus_alertmanager_url: ""
# Email alerting (via Grafana)
alerting_email_enabled: true
alerting_email_to: ""
# Home Assistant webhook for alerts
alerting_ha_webhook_enabled: true
alerting_ha_webhook_url: ""
@@ -1,15 +0,0 @@
---
- name: Restart prometheus
systemd:
name: prometheus
state: restarted
daemon_reload: yes
- name: Reload prometheus
systemd:
name: prometheus
state: reloaded
- name: Reload systemd
systemd:
daemon_reload: yes
@@ -1,129 +0,0 @@
---
# Prometheus Server Installation and Configuration
- name: Install dependencies
apt:
name:
- curl
state: present
update_cache: yes
- name: Create prometheus user
user:
name: "{{ prometheus_user }}"
system: yes
shell: /usr/sbin/nologin
create_home: no
- name: Create directories
file:
path: "{{ item }}"
state: directory
owner: "{{ prometheus_user }}"
group: "{{ prometheus_group }}"
mode: '0755'
loop:
- "{{ prometheus_config_dir }}"
- "{{ prometheus_data_dir }}"
- "{{ prometheus_rules_dir }}"
- "{{ prometheus_targets_dir }}"
- name: Download Prometheus
get_url:
url: "{{ prometheus_download_url }}"
dest: "/tmp/prometheus.tar.gz"
mode: '0644'
- name: Extract Prometheus
unarchive:
src: "/tmp/prometheus.tar.gz"
dest: "/tmp"
remote_src: yes
- name: Install Prometheus binaries
copy:
src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item.src }}"
dest: "{{ item.dest }}"
mode: '0755'
remote_src: yes
loop:
- { src: "prometheus", dest: "{{ prometheus_bin_path }}" }
- { src: "promtool", dest: "{{ prometheus_promtool_path }}" }
notify: Restart prometheus
- name: Check for console templates
stat:
path: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/consoles"
register: consoles_dir
- name: Copy console templates
copy:
src: "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}/{{ item }}/"
dest: "{{ prometheus_config_dir }}/{{ item }}/"
remote_src: yes
owner: "{{ prometheus_user }}"
group: "{{ prometheus_group }}"
loop:
- consoles
- console_libraries
when: consoles_dir.stat.exists
- name: Create Prometheus configuration
template:
src: prometheus.yml.j2
dest: "{{ prometheus_config_file }}"
owner: "{{ prometheus_user }}"
group: "{{ prometheus_group }}"
mode: '0644'
validate: "{{ prometheus_promtool_path }} check config %s"
notify: Restart prometheus
- name: Create alert rules
template:
src: alert-rules.yml.j2
dest: "{{ prometheus_rules_dir }}/alert-rules.yml"
owner: "{{ prometheus_user }}"
group: "{{ prometheus_group }}"
mode: '0644'
notify: Reload prometheus
- name: Create node exporter targets file
template:
src: nodes.yml.j2
dest: "{{ prometheus_targets_dir }}/nodes.yml"
owner: "{{ prometheus_user }}"
group: "{{ prometheus_group }}"
mode: '0644'
- name: Create Prometheus systemd service
template:
src: prometheus.service.j2
dest: /etc/systemd/system/prometheus.service
mode: '0644'
notify:
- Reload systemd
- Restart prometheus
- name: Enable and start Prometheus service
systemd:
name: prometheus
state: started
enabled: yes
daemon_reload: yes
- name: Wait for Prometheus to be ready
uri:
url: "http://127.0.0.1:{{ prometheus_listen_port }}/-/ready"
status_code: 200
register: result
until: result.status == 200
retries: 30
delay: 5
- name: Clean up downloaded files
file:
path: "{{ item }}"
state: absent
loop:
- "/tmp/prometheus.tar.gz"
- "/tmp/prometheus-{{ prometheus_version }}.{{ prometheus_arch }}"
@@ -1,100 +0,0 @@
# Prometheus Alert Rules
# Generated by Ansible
groups:
- name: infrastructure
rules:
# Disk usage alerts
- alert: HighDiskUsage
expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 15
for: 5m
labels:
severity: warning
annotations:
summary: "High disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
description: "Disk usage is above 85% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
- alert: CriticalDiskUsage
expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 < 5
for: 2m
labels:
severity: critical
annotations:
summary: "Critical disk usage on {{ '{{' }} $labels.instance {{ '}}' }}"
description: "Disk usage is above 95% on {{ '{{' }} $labels.mountpoint {{ '}}' }} ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}% free)"
# Memory alerts
- alert: HighMemoryUsage
expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 > 90
for: 5m
labels:
severity: warning
annotations:
summary: "High memory usage on {{ '{{' }} $labels.instance {{ '}}' }}"
description: "Memory usage is above 90% ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
# CPU alerts
- alert: HighCPUUsage
expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
for: 10m
labels:
severity: warning
annotations:
summary: "High CPU usage on {{ '{{' }} $labels.instance {{ '}}' }}"
description: "CPU usage is above 85% for 10 minutes ({{ '{{' }} $value | printf \"%.1f\" {{ '}}' }}%)"
- name: services
rules:
# Service down alerts
- alert: ServiceDown
expr: up == 0
for: 2m
labels:
severity: critical
annotations:
summary: "Service {{ '{{' }} $labels.job {{ '}}' }} down on {{ '{{' }} $labels.instance {{ '}}' }}"
description: "Service has been unreachable for more than 2 minutes"
# DNS VIP health
- alert: DNSVIPDown
expr: absent(up{job="dns-servers"} == 1) or count(up{job="dns-servers"} == 1) < 2
for: 1m
labels:
severity: critical
annotations:
summary: "DNS cluster degraded"
description: "Less than 2 DNS servers are healthy"
- name: security
rules:
# High auth failure rate (placeholder - needs log-based metrics from Loki)
- alert: HighAuthFailureRate
expr: rate(authentik_login_failed_total[5m]) > 5
for: 2m
labels:
severity: warning
annotations:
summary: "High authentication failure rate"
description: "More than 5 auth failures per minute detected"
- name: loki
rules:
# Loki health
- alert: LokiDown
expr: up{job="loki"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: "Loki log aggregation is down"
description: "Loki has been unreachable for more than 2 minutes"
# Loki ingestion rate
- alert: LokiHighIngestionRate
expr: rate(loki_distributor_bytes_received_total[5m]) > 10485760
for: 5m
labels:
severity: warning
annotations:
summary: "High log ingestion rate in Loki"
description: "Log ingestion rate is above 10MB/s"
@@ -1,26 +0,0 @@
# Node Exporter targets - auto-generated by Ansible
# Updates require re-running the prometheus playbook
#
# NOTE: Most hosts have Alloy agents that push metrics via remote_write.
# This file only includes hosts that need direct scraping (standalone node_exporter).
# Alloy-managed hosts are excluded to avoid duplicate "down" targets.
# PVE cluster nodes - need direct scraping (no Alloy agent)
- targets:
- 'jupiter.renner.uno:9100'
labels:
job: 'node'
group: 'pve_cluster'
hostname: 'jupiter'
- targets:
- 'neptune.renner.uno:9100'
labels:
job: 'node'
group: 'pve_cluster'
hostname: 'neptune'
- targets:
- 'saturn.renner.uno:9100'
labels:
job: 'node'
group: 'pve_cluster'
hostname: 'saturn'
@@ -1,38 +0,0 @@
[Unit]
Description=Prometheus Monitoring System
Documentation=https://prometheus.io/docs/
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ prometheus_user }}
Group={{ prometheus_group }}
ExecStart={{ prometheus_bin_path }} \
--config.file={{ prometheus_config_file }} \
--storage.tsdb.path={{ prometheus_data_dir }} \
--storage.tsdb.retention.time={{ prometheus_retention_time }} \
--storage.tsdb.retention.size={{ prometheus_retention_size }} \
--web.listen-address={{ prometheus_listen_address }}:{{ prometheus_listen_port }} \
{% if prometheus_remote_write_enabled %}
--web.enable-remote-write-receiver \
{% endif %}
{% if prometheus_web_external_url %}
--web.external-url={{ prometheus_web_external_url }} \
{% endif %}
--web.enable-lifecycle
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=10s
LimitNOFILE=65536
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths={{ prometheus_data_dir }}
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -1,304 +0,0 @@
# Prometheus Configuration
# Generated by Ansible
global:
scrape_interval: {{ prometheus_scrape_interval }}
evaluation_interval: {{ prometheus_evaluation_interval }}
scrape_timeout: {{ prometheus_scrape_timeout }}
external_labels:
cluster: homelab
datacenter: renner
rule_files:
- "{{ prometheus_rules_dir }}/*.yml"
{% if prometheus_alertmanager_url %}
alerting:
alertmanagers:
- static_configs:
- targets:
- {{ prometheus_alertmanager_url }}
{% endif %}
scrape_configs:
# Prometheus self-monitoring
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:{{ prometheus_listen_port }}']
# Loki metrics
- job_name: 'loki'
static_configs:
- targets: ['loki.srv.renner.uno:3100']
# File-based service discovery for dynamic targets
- job_name: 'alloy-nodes'
file_sd_configs:
- files:
- '{{ prometheus_targets_dir }}/*.yml'
- '{{ prometheus_targets_dir }}/*.json'
refresh_interval: 30s
{% if prometheus_remote_write_enabled %}
# Remote write receiver for Alloy agents
# Agents push metrics to /api/v1/write
{% endif %}
# Static targets for infrastructure components
# Proxmox VE API metrics (via pve-exporter)
- job_name: 'pve'
scrape_interval: 60s
scrape_timeout: 55s
static_configs:
- targets:
- 'jupiter.renner.uno'
- 'neptune.renner.uno'
- 'saturn.renner.uno'
metrics_path: /pve
params:
module: [default]
cluster: ['1']
node: ['1']
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9221 # pve-exporter address
- target_label: group
replacement: proxmox
- target_label: type
replacement: hypervisor
# RustFS (S3-compatible storage)
# NOTE: RustFS uses OpenTelemetry push-model via --obs-endpoint, not Prometheus pull.
# Unlike MinIO, it doesn't expose /minio/v2/metrics/cluster endpoint.
# Metrics collection would require setting up RUSTFS_OBS_ENDPOINT to push to an OTEL collector.
# For now, node-level metrics are available via Alloy agents on the RustFS hosts.
# Home Assistant metrics
- job_name: 'homeassistant'
metrics_path: /api/prometheus
bearer_token: '{{ homeassistant_prometheus_token }}'
scheme: http
static_configs:
- targets: ['homeassistant.iot.renner.uno:8123']
labels:
group: 'smarthome'
type: 'homeassistant'
# UniFi Poller metrics
- job_name: 'unifi'
static_configs:
- targets: ['localhost:9130']
labels:
group: 'network'
type: 'unifi'
# Caddy reverse proxy metrics (HA cluster)
- job_name: 'caddy'
static_configs:
- targets:
- 'caddy.dmz.renner.uno:2019'
- 'caddy2.dmz.renner.uno:2019'
labels:
group: 'ingress'
type: 'reverse-proxy'
# PostgreSQL metrics (via postgres_exporter)
- job_name: 'postgresql'
static_configs:
- targets: ['localhost:9187']
labels:
group: 'database'
type: 'postgresql'
# Authentik identity provider metrics
- job_name: 'authentik'
static_configs:
- targets:
- 'authentik-app-1.srv.renner.uno:9300'
- 'authentik-app-2.srv.renner.uno:9300'
labels:
group: 'identity'
type: 'authentik'
# Node Exporter - system metrics from all hosts
- job_name: 'node'
file_sd_configs:
- files:
- '{{ prometheus_targets_dir }}/nodes.yml'
refresh_interval: 60s
# Chrony Exporter - NTP metrics from PVE cluster
# One static_config per host so each carries its own hostname label. The
# previous form listed all three targets together and recovered the hostname
# in relabel_configs by regex-matching __address__ against the node's IP --
# which silently coupled the labels to the addressing. Changing a target
# would have left the regex unmatched and the hostname label simply absent,
# with the scrape still succeeding.
- job_name: 'chrony'
static_configs:
- targets: ['jupiter.renner.uno:9123']
labels:
group: 'ntp'
type: 'time-server'
hostname: 'jupiter'
- targets: ['neptune.renner.uno:9123']
labels:
group: 'ntp'
type: 'time-server'
hostname: 'neptune'
- targets: ['saturn.renner.uno:9123']
labels:
group: 'ntp'
type: 'time-server'
hostname: 'saturn'
# Immich photo management metrics
- job_name: 'immich'
static_configs:
- targets: ['immich.srv.renner.uno:8081']
labels:
group: 'media'
type: 'photo-management'
# Blackbox Exporter - HTTP endpoint probing
- job_name: 'blackbox-http'
metrics_path: /probe
params:
module: [http_2xx]
static_configs:
- targets:
- https://gitea.renner.uno
- https://immich.renner.uno
- https://auth.renner.uno
- https://ha.renner.uno
- https://grafana.renner.uno
- https://prometheus.renner.uno
labels:
group: 'probe'
type: 'http'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115
# Blackbox Exporter - TCP probing
- job_name: 'blackbox-tcp'
metrics_path: /probe
params:
module: [tcp_connect]
static_configs:
- targets:
- 10.2.40.135:5432 # PostgreSQL HA VIP
labels:
group: 'probe'
type: 'tcp'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115
# Blackbox Exporter - DNS probing (external domains like google.com)
- job_name: 'blackbox-dns-external'
metrics_path: /probe
params:
module: [dns_external]
static_configs:
- targets:
- 10.2.40.53 # Proxmox DNS VIP
- 10.2.40.1 # UDM Gateway DNS
labels:
group: 'probe'
type: 'dns'
query_type: 'external'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115
# Blackbox Exporter - DNS probing (internal domains like k8s-1.srv.renner.uno)
- job_name: 'blackbox-dns-internal'
metrics_path: /probe
params:
module: [dns_internal]
static_configs:
- targets:
- 10.2.40.53 # Proxmox DNS VIP
- 10.2.40.1 # UDM Gateway DNS
labels:
group: 'probe'
type: 'dns'
query_type: 'internal'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115
# Blackbox Exporter - DNS probing (cached domains)
- job_name: 'blackbox-dns-cached'
metrics_path: /probe
params:
module: [dns_cached]
static_configs:
- targets:
- 10.2.40.53 # Proxmox DNS VIP
- 10.2.40.1 # UDM Gateway DNS
labels:
group: 'probe'
type: 'dns'
query_type: 'cached'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115
{% if snmp_synology_targets is defined and snmp_synology_targets | length > 0 %}
# Synology NAS SNMP metrics
# snmp_exporter v0.27+ requires both synology and ucd_la_table modules
- job_name: 'synology'
scrape_interval: 60s
scrape_timeout: 55s
metrics_path: /snmp
params:
module: [synology, ucd_la_table, ucd_memory, if_mib]
{% if snmp_use_v3 | default(false) %}
auth: [synology_v3]
{% else %}
auth: [synology_v2]
{% endif %}
static_configs:
- targets:
{% for target in snmp_synology_targets %}
- '{{ target.target }}' # {{ target.name }}
{% endfor %}
labels:
group: 'storage'
type: 'nas'
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__address__]
regex: '([^:]+)(:\d+)?'
replacement: '${1}'
target_label: instance
- target_label: __address__
replacement: localhost:9116 # snmp_exporter address
{% endif %}
-36
View File
@@ -1,36 +0,0 @@
---
# Promtail Configuration
# Version and download
promtail_version: "3.4.2"
promtail_arch: "linux-amd64"
promtail_download_url: "https://github.com/grafana/loki/releases/download/v{{ promtail_version }}/promtail-{{ promtail_arch }}.zip"
promtail_bin_path: "/usr/local/bin/promtail"
# Directories
promtail_config_dir: "/etc/promtail"
promtail_config_file: "{{ promtail_config_dir }}/promtail-config.yaml"
promtail_positions_file: "/var/lib/promtail/positions.yaml"
# Server configuration
promtail_http_listen_port: 9080
promtail_http_listen_address: "0.0.0.0"
# Loki server
promtail_loki_url: "http://loki.srv.renner.uno:3100/loki/api/v1/push"
# Log scrape configurations
# Override in playbook or host_vars for specific services
promtail_scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: syslog
host: "{{ inventory_hostname }}"
__path__: /var/log/syslog
# Systemd service
promtail_user: "promtail"
promtail_group: "promtail"
-6
View File
@@ -1,6 +0,0 @@
---
- name: Restart promtail
systemd:
name: promtail
state: restarted
daemon_reload: yes
-112
View File
@@ -1,112 +0,0 @@
---
- name: Install dependencies
apt:
name:
- unzip
- curl
state: present
update_cache: yes
- name: Create promtail group
group:
name: "{{ promtail_group }}"
system: yes
state: present
- name: Create promtail user
user:
name: "{{ promtail_user }}"
group: "{{ promtail_group }}"
system: yes
shell: /usr/sbin/nologin
home: /var/lib/promtail
create_home: yes
state: present
- name: Add promtail user to systemd-journal group
user:
name: "{{ promtail_user }}"
groups: systemd-journal
append: yes
- name: Create config directory
file:
path: "{{ promtail_config_dir }}"
state: directory
owner: "{{ promtail_user }}"
group: "{{ promtail_group }}"
mode: '0755'
- name: Create positions directory
file:
path: "{{ promtail_positions_file | dirname }}"
state: directory
owner: "{{ promtail_user }}"
group: "{{ promtail_group }}"
mode: '0755'
- name: Check if promtail binary exists
stat:
path: "{{ promtail_bin_path }}"
register: promtail_binary
- name: Get installed promtail version
command: "{{ promtail_bin_path }} --version"
register: installed_version
changed_when: false
failed_when: false
when: promtail_binary.stat.exists
- name: Download promtail
get_url:
url: "{{ promtail_download_url }}"
dest: /tmp/promtail.zip
mode: '0644'
when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
- name: Extract promtail binary
unarchive:
src: /tmp/promtail.zip
dest: /tmp/
remote_src: yes
when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
- name: Install promtail binary
copy:
src: "/tmp/promtail-{{ promtail_arch }}"
dest: "{{ promtail_bin_path }}"
remote_src: yes
mode: '0755'
when: not promtail_binary.stat.exists or (installed_version.stdout is defined and promtail_version not in installed_version.stdout)
notify: Restart promtail
- name: Clean up downloaded files
file:
path: "{{ item }}"
state: absent
loop:
- /tmp/promtail.zip
- "/tmp/promtail-{{ promtail_arch }}"
- name: Configure promtail
template:
src: promtail-config.yaml.j2
dest: "{{ promtail_config_file }}"
owner: "{{ promtail_user }}"
group: "{{ promtail_group }}"
mode: '0644'
notify: Restart promtail
- name: Create systemd service
template:
src: promtail.service.j2
dest: /etc/systemd/system/promtail.service
mode: '0644'
notify: Restart promtail
- name: Enable and start promtail
systemd:
name: promtail
state: started
enabled: yes
daemon_reload: yes
@@ -1,79 +0,0 @@
server:
http_listen_port: {{ promtail_http_listen_port }}
http_listen_address: {{ promtail_http_listen_address }}
grpc_listen_port: 0
positions:
filename: {{ promtail_positions_file }}
clients:
- url: {{ promtail_loki_url }}
tenant_id: default
scrape_configs:
{% for config in promtail_scrape_configs %}
- job_name: {{ config.job_name }}
{% if config.journal is defined %}
journal:
max_age: {{ config.journal.max_age | default('12h') }}
labels:
job: {{ config.job_name }}
host: {{ inventory_hostname }}
{% if config.journal.matches is defined %}
matches: {{ config.journal.matches }}
{% endif %}
{% if config.relabel_configs is defined %}
relabel_configs:
{% for relabel in config.relabel_configs %}
- source_labels: {{ relabel.source_labels | to_json }}
target_label: {{ relabel.target_label }}
{% if relabel.regex is defined %}
regex: '{{ relabel.regex }}'
{% endif %}
{% if relabel.replacement is defined %}
replacement: '{{ relabel.replacement }}'
{% endif %}
{% endfor %}
{% endif %}
{% endif %}
{% if config.static_configs is defined %}
static_configs:
{% for static in config.static_configs %}
- targets:
{% for target in static.targets %}
- {{ target }}
{% endfor %}
labels:
{% for key, value in static.labels.items() %}
{{ key }}: {{ value }}
{% endfor %}
{% endfor %}
{% if config.pipeline_stages is defined %}
pipeline_stages:
{% for stage in config.pipeline_stages %}
{% if stage.multiline is defined %}
- multiline:
firstline: '{{ stage.multiline.firstline }}'
{% if stage.multiline.max_wait_time is defined %}
max_wait_time: {{ stage.multiline.max_wait_time }}
{% endif %}
{% endif %}
{% if stage.regex is defined %}
- regex:
expression: '{{ stage.regex.expression }}'
{% endif %}
{% if stage.labels is defined %}
- labels:
{% for label in stage.labels %}
{{ label }}:
{% endfor %}
{% endif %}
{% if stage.timestamp is defined %}
- timestamp:
source: {{ stage.timestamp.source }}
format: '{{ stage.timestamp.format }}'
{% endif %}
{% endfor %}
{% endif %}
{% endif %}
{% endfor %}
@@ -1,24 +0,0 @@
[Unit]
Description=Promtail Log Collector
Documentation=https://grafana.com/docs/loki/latest/clients/promtail/
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ promtail_user }}
Group={{ promtail_group }}
ExecStart={{ promtail_bin_path }} -config.file={{ promtail_config_file }}
Restart=always
RestartSec=5
TimeoutStopSec=30
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/promtail
CapabilityBoundingSet=CAP_DAC_READ_SEARCH
[Install]
WantedBy=multi-user.target
@@ -1,19 +0,0 @@
---
# PVE Exporter defaults
# Installation
pve_exporter_version: "3.4.5"
pve_exporter_install_dir: /opt/pve-exporter
pve_exporter_config_dir: /etc/pve-exporter
pve_exporter_venv_path: "{{ pve_exporter_install_dir }}/venv"
# Service
pve_exporter_user: pve-exporter
pve_exporter_group: pve-exporter
pve_exporter_port: 9221
# Proxmox connection (override with vault variables)
pve_exporter_pve_user: "{{ pve_user | default('root@pam') }}"
pve_exporter_pve_token_name: "{{ pve_token_id | default('ansible') }}"
pve_exporter_pve_token_value: "{{ pve_token_secret }}"
pve_exporter_pve_verify_ssl: false
@@ -1,6 +0,0 @@
---
- name: Restart pve-exporter
systemd:
name: pve-exporter
state: restarted
daemon_reload: yes
-78
View File
@@ -1,78 +0,0 @@
---
# PVE Exporter installation and configuration
- name: Install Python dependencies
apt:
name:
- python3
- python3-pip
- python3-venv
state: present
update_cache: yes
- name: Create pve-exporter system user
user:
name: "{{ pve_exporter_user }}"
system: yes
shell: /usr/sbin/nologin
create_home: no
- name: Create installation directory
file:
path: "{{ pve_exporter_install_dir }}"
state: directory
owner: "{{ pve_exporter_user }}"
group: "{{ pve_exporter_group }}"
mode: '0755'
- name: Create configuration directory
file:
path: "{{ pve_exporter_config_dir }}"
state: directory
owner: "{{ pve_exporter_user }}"
group: "{{ pve_exporter_group }}"
mode: '0750'
- name: Create Python virtual environment
command: python3 -m venv {{ pve_exporter_venv_path }}
args:
creates: "{{ pve_exporter_venv_path }}/bin/python"
- name: Set venv ownership
file:
path: "{{ pve_exporter_venv_path }}"
state: directory
owner: "{{ pve_exporter_user }}"
group: "{{ pve_exporter_group }}"
recurse: yes
- name: Install prometheus-pve-exporter
pip:
name: prometheus-pve-exporter
virtualenv: "{{ pve_exporter_venv_path }}"
state: present
- name: Create configuration file
template:
src: pve.yml.j2
dest: "{{ pve_exporter_config_dir }}/pve.yml"
owner: "{{ pve_exporter_user }}"
group: "{{ pve_exporter_group }}"
mode: '0640'
notify: Restart pve-exporter
- name: Create systemd service
template:
src: pve-exporter.service.j2
dest: /etc/systemd/system/pve-exporter.service
owner: root
group: root
mode: '0644'
notify: Restart pve-exporter
- name: Enable and start pve-exporter service
systemd:
name: pve-exporter
state: started
enabled: yes
daemon_reload: yes
@@ -1,22 +0,0 @@
[Unit]
Description=Prometheus PVE Exporter
Documentation=https://github.com/prometheus-pve/prometheus-pve-exporter
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ pve_exporter_user }}
Group={{ pve_exporter_group }}
ExecStart={{ pve_exporter_venv_path }}/bin/pve_exporter \
--config.file {{ pve_exporter_config_dir }}/pve.yml \
--web.listen-address 0.0.0.0:{{ pve_exporter_port }}
Restart=on-failure
RestartSec=10s
# Security hardening
NoNewPrivileges=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -1,8 +0,0 @@
# PVE Exporter Configuration
# Generated by Ansible
default:
user: "{{ pve_exporter_pve_user }}"
token_name: "{{ pve_exporter_pve_token_name }}"
token_value: "{{ pve_exporter_pve_token_value }}"
verify_ssl: {{ pve_exporter_pve_verify_ssl | lower }}
@@ -1,33 +0,0 @@
---
# SNMP Exporter defaults
snmp_exporter_version: "0.30.1"
snmp_exporter_listen_port: 9116
snmp_exporter_user: "snmp_exporter"
snmp_exporter_group: "snmp_exporter"
snmp_exporter_install_dir: "/opt/snmp_exporter"
snmp_exporter_config_dir: "/etc/snmp_exporter"
# Synology NAS SNMP targets
# Configure in host_vars or group_vars
snmp_synology_targets: []
# Example:
# snmp_synology_targets:
# - name: nas-1
# target: 10.2.40.50
# - name: nas-2
# target: 10.2.40.51
# SNMPv3 authentication (recommended for security)
# These should be set in vault
snmp_v3_username: "prometheus"
snmp_v3_auth_protocol: "SHA" # MD5 or SHA
snmp_v3_auth_password: "" # Set in vault
snmp_v3_priv_protocol: "AES" # DES or AES
snmp_v3_priv_password: "" # Set in vault
snmp_v3_security_level: "authPriv" # noAuthNoPriv, authNoPriv, authPriv
# SNMPv2c fallback (less secure, but simpler)
snmp_v2_community: "public"
snmp_use_v3: true
@@ -1,6 +0,0 @@
---
- name: Restart snmp_exporter
systemd:
name: snmp_exporter
state: restarted
daemon_reload: yes
-114
View File
@@ -1,114 +0,0 @@
---
# SNMP Exporter installation and configuration
- name: Create snmp_exporter group
group:
name: "{{ snmp_exporter_group }}"
system: yes
state: present
- name: Create snmp_exporter user
user:
name: "{{ snmp_exporter_user }}"
group: "{{ snmp_exporter_group }}"
system: yes
shell: /usr/sbin/nologin
home: "{{ snmp_exporter_install_dir }}"
create_home: no
- name: Create directories
file:
path: "{{ item }}"
state: directory
owner: "{{ snmp_exporter_user }}"
group: "{{ snmp_exporter_group }}"
mode: '0755'
loop:
- "{{ snmp_exporter_install_dir }}"
- "{{ snmp_exporter_config_dir }}"
- name: Check if snmp_exporter is already installed
stat:
path: "{{ snmp_exporter_install_dir }}/snmp_exporter"
register: snmp_exporter_binary
- name: Get installed snmp_exporter version
command: "{{ snmp_exporter_install_dir }}/snmp_exporter --version"
register: installed_version
changed_when: false
failed_when: false
when: snmp_exporter_binary.stat.exists
- name: Download snmp_exporter
get_url:
url: "https://github.com/prometheus/snmp_exporter/releases/download/v{{ snmp_exporter_version }}/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64.tar.gz"
dest: "/tmp/snmp_exporter.tar.gz"
mode: '0644'
when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
- name: Extract snmp_exporter
unarchive:
src: "/tmp/snmp_exporter.tar.gz"
dest: "/tmp/"
remote_src: yes
when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
- name: Install snmp_exporter binary
copy:
src: "/tmp/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64/snmp_exporter"
dest: "{{ snmp_exporter_install_dir }}/snmp_exporter"
remote_src: yes
owner: "{{ snmp_exporter_user }}"
group: "{{ snmp_exporter_group }}"
mode: '0755'
when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
notify: Restart snmp_exporter
- name: Install default snmp.yml from release
copy:
src: "/tmp/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64/snmp.yml"
dest: "{{ snmp_exporter_config_dir }}/snmp.yml"
remote_src: yes
owner: "{{ snmp_exporter_user }}"
group: "{{ snmp_exporter_group }}"
mode: '0644'
when: not snmp_exporter_binary.stat.exists or (installed_version.stdout is defined and snmp_exporter_version not in installed_version.stdout)
notify: Restart snmp_exporter
- name: Add synology_v3 auth entry
lineinfile:
path: "{{ snmp_exporter_config_dir }}/snmp.yml"
insertafter: "^auths:"
line: " synology_v3: {version: 3, security_level: {{ snmp_v3_security_level }}, username: {{ snmp_v3_username }}, password: {{ snmp_v3_auth_password }}, auth_protocol: {{ snmp_v3_auth_protocol }}, priv_protocol: {{ snmp_v3_priv_protocol }}, priv_password: {{ snmp_v3_priv_password }}}"
notify: Restart snmp_exporter
- name: Add synology_v2 auth entry
lineinfile:
path: "{{ snmp_exporter_config_dir }}/snmp.yml"
insertafter: "^auths:"
line: " synology_v2: {version: 2, community: {{ snmp_v2_community }}}"
notify: Restart snmp_exporter
- name: Create snmp_exporter systemd service
template:
src: snmp_exporter.service.j2
dest: /etc/systemd/system/snmp_exporter.service
owner: root
group: root
mode: '0644'
notify: Restart snmp_exporter
- name: Enable and start snmp_exporter service
systemd:
name: snmp_exporter
state: started
enabled: yes
daemon_reload: yes
- name: Clean up downloaded files
file:
path: "{{ item }}"
state: absent
loop:
- "/tmp/snmp_exporter.tar.gz"
- "/tmp/snmp_exporter-{{ snmp_exporter_version }}.linux-amd64"
@@ -1,18 +0,0 @@
# SNMPv3 Authentication for Synology NAS
# Appended to default snmp.yml by Ansible
auths:
{% if snmp_use_v3 | default(true) %}
synology_v3:
security_level: {{ snmp_v3_security_level }}
username: {{ snmp_v3_username }}
password: {{ snmp_v3_auth_password }}
auth_protocol: {{ snmp_v3_auth_protocol }}
{% if snmp_v3_security_level == 'authPriv' %}
priv_protocol: {{ snmp_v3_priv_protocol }}
priv_password: {{ snmp_v3_priv_password }}
{% endif %}
{% else %}
synology_v2:
community: {{ snmp_v2_community }}
{% endif %}
@@ -1,26 +0,0 @@
[Unit]
Description=Prometheus SNMP Exporter
Documentation=https://github.com/prometheus/snmp_exporter
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ snmp_exporter_user }}
Group={{ snmp_exporter_group }}
ExecStart={{ snmp_exporter_install_dir }}/snmp_exporter \
--config.file={{ snmp_exporter_config_dir }}/snmp.yml \
--web.listen-address=:{{ snmp_exporter_listen_port }}
Restart=on-failure
RestartSec=5s
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadOnlyPaths=/
[Install]
WantedBy=multi-user.target
@@ -1,34 +0,0 @@
---
# UniFi Poller defaults
# Version to install
unifi_poller_version: "2.11.2"
# Installation paths
unifi_poller_bin_path: /usr/local/bin/unpoller
unifi_poller_config_dir: /etc/unpoller
unifi_poller_config_file: "{{ unifi_poller_config_dir }}/up.conf"
# Service user
unifi_poller_user: unpoller
unifi_poller_group: unpoller
# UniFi Controller settings (override in vars/vault)
unifi_controller_url: "https://192.168.1.1"
unifi_controller_user: ""
unifi_controller_pass: ""
unifi_controller_verify_ssl: false
unifi_controller_site: "default"
# Prometheus exporter settings
unifi_poller_prometheus_port: 9130
unifi_poller_prometheus_path: /metrics
# Polling intervals
unifi_poller_interval: 30s
unifi_poller_save_sites: true
unifi_poller_save_ids: false
unifi_poller_save_events: false
unifi_poller_save_alarms: false
unifi_poller_save_anomalies: false
unifi_poller_save_dpi: true
@@ -1,6 +0,0 @@
---
- name: Restart unpoller
systemd:
name: unpoller
state: restarted
daemon_reload: yes
-88
View File
@@ -1,88 +0,0 @@
---
# UniFi Poller installation and configuration
- name: Create unpoller system user
user:
name: "{{ unifi_poller_user }}"
system: yes
shell: /usr/sbin/nologin
create_home: no
- name: Create configuration directory
file:
path: "{{ unifi_poller_config_dir }}"
state: directory
owner: "{{ unifi_poller_user }}"
group: "{{ unifi_poller_group }}"
mode: '0750'
- name: Check if unpoller is already installed
stat:
path: "{{ unifi_poller_bin_path }}"
register: unpoller_binary
- name: Get installed version
command: "{{ unifi_poller_bin_path }} --version"
register: installed_version
changed_when: false
failed_when: false
when: unpoller_binary.stat.exists
- name: Download UniFi Poller
get_url:
url: "https://github.com/unpoller/unpoller/releases/download/v{{ unifi_poller_version }}/unpoller_{{ unifi_poller_version }}_linux_amd64.tar.gz"
dest: /tmp/unpoller.tar.gz
mode: '0644'
when: not unpoller_binary.stat.exists or (installed_version.stdout is defined and unifi_poller_version not in installed_version.stdout)
- name: Extract UniFi Poller
unarchive:
src: /tmp/unpoller.tar.gz
dest: /tmp/
remote_src: yes
when: not unpoller_binary.stat.exists or (installed_version.stdout is defined and unifi_poller_version not in installed_version.stdout)
- name: Install UniFi Poller binary
copy:
src: /tmp/unpoller
dest: "{{ unifi_poller_bin_path }}"
remote_src: yes
owner: root
group: root
mode: '0755'
when: not unpoller_binary.stat.exists or (installed_version.stdout is defined and unifi_poller_version not in installed_version.stdout)
notify: Restart unpoller
- name: Create configuration file
template:
src: up.conf.j2
dest: "{{ unifi_poller_config_file }}"
owner: "{{ unifi_poller_user }}"
group: "{{ unifi_poller_group }}"
mode: '0640'
notify: Restart unpoller
- name: Create systemd service
template:
src: unpoller.service.j2
dest: /etc/systemd/system/unpoller.service
owner: root
group: root
mode: '0644'
notify: Restart unpoller
- name: Enable and start unpoller service
systemd:
name: unpoller
state: started
enabled: yes
daemon_reload: yes
- name: Clean up downloaded files
file:
path: "{{ item }}"
state: absent
loop:
- /tmp/unpoller.tar.gz
- /tmp/unpoller
changed_when: false
@@ -1,24 +0,0 @@
[Unit]
Description=UniFi Poller - Prometheus Exporter for UniFi Network
Documentation=https://github.com/unpoller/unpoller
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User={{ unifi_poller_user }}
Group={{ unifi_poller_group }}
ExecStart={{ unifi_poller_bin_path }} --config {{ unifi_poller_config_file }}
Restart=on-failure
RestartSec=10s
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ReadOnlyPaths=/
ReadWritePaths={{ unifi_poller_config_dir }}
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -1,42 +0,0 @@
# UniFi Poller Configuration
# Generated by Ansible
[poller]
# Turns on debug messages
debug = false
# Turns off messages
quiet = false
[prometheus]
# Whether to export metrics
disable = false
# HTTP listen address for /metrics endpoint
http_listen = "0.0.0.0:{{ unifi_poller_prometheus_port }}"
# Path for metrics endpoint
report_errors = false
[unifi]
# UniFi Controller configurations
[[unifi.controller]]
# URL for the UniFi Controller
url = "{{ unifi_controller_url }}"
# Username
user = "{{ unifi_controller_user }}"
# Password
pass = "{{ unifi_controller_pass }}"
# Sites to poll (leave empty for all sites)
sites = ["{{ unifi_controller_site }}"]
# Verify SSL certificates
verify_ssl = {{ unifi_controller_verify_ssl | lower }}
# Save site information
save_sites = {{ unifi_poller_save_sites | lower }}
# Save device IDs
save_ids = {{ unifi_poller_save_ids | lower }}
# Save events
save_events = {{ unifi_poller_save_events | lower }}
# Save alarms
save_alarms = {{ unifi_poller_save_alarms | lower }}
# Save anomalies
save_anomalies = {{ unifi_poller_save_anomalies | lower }}
# Save DPI data
save_dpi = {{ unifi_poller_save_dpi | lower }}
+2
View File
@@ -103,6 +103,8 @@ RUN open3e --help > /dev/null \
&& supervisord --help > /dev/null \
&& grep -q '^\[program:open3e\]' /etc/supervisord.conf \
&& grep -q '^\[program:e3oncan\]' /etc/supervisord.conf \
&& grep -q '^\[supervisorctl\]' /etc/supervisord.conf \
&& supervisorctl --help > /dev/null \
&& echo "self-test OK: both services execute; supervisord has both programs"
# CAN_RAW needs no capabilities, so nothing here runs as root.
+18
View File
@@ -10,6 +10,24 @@
# expands %(ENV_NAME)s at start, so neither the compose file nor a wrapper
# script constructs open3e's or E3onCAN's argv -- those CLIs stay an
# implementation detail of this image.
# Control socket. Without these two sections supervisorctl cannot connect, so
# there is no way to query or restart an individual service inside the
# container -- which is the entire reason for running a supervisor rather than
# one process per container. The first revision shipped without them.
#
# Unix socket only, inside the container: nothing outside needs to reach it,
# and the container runs with host networking, so a TCP listener would be
# exposed on the host.
[unix_http_server]
file=/tmp/supervisor.sock
chmod=0700
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock
[supervisord]
nodaemon=true
user=open3e
@@ -1326,7 +1326,121 @@ data:
name: Caddy docs.renner.uno
state: absent
# Passkey-only (WebAuthn) login for selected apps, in two parts:
# ---------------------------------------------------------------
# wordpress-gate — www.renner.uno
#
# Unlike every other gate here, this one does NOT front a whole
# host. www.renner.uno is a public family blog; only /wp-admin and
# /wp-login.php are routed to this outpost (see
# kubernetes/apps/wordpress/ingress.yaml). The front end never
# touches it.
#
# The reverse carve-out — gate everything, skip_path_regex the
# public parts, as paperless-gate does for /static/ — is not
# possible here: the public set is every post URL, so the pattern
# would need a negative lookahead and the outpost's RE2 has none.
- id: wordpress-authors-group
model: authentik_core.group
identifiers:
name: WordPress Authors
state: present
attrs:
is_superuser: false
# Git owns membership, as with the other gates: this group is the
# explicit allowlist for who may publish on www.renner.uno.
# Add/remove members HERE, not in the UI — blueprint apply is
# authoritative for this M2M and reverts UI edits.
#
# s.renner is the Authentik account for the site's WordPress
# Editor, which still carries the pre-2019 login `s.birzer` — the
# two names do not match, and that is expected.
users:
- !Find [authentik_core.user, [username, k.renner]]
- !Find [authentik_core.user, [username, s.renner]]
- id: wordpress-gate-provider
model: authentik_providers_proxy.proxyprovider
identifiers:
name: wordpress-gate
state: present
attrs:
authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
mode: proxy
internal_host: http://wordpress.wordpress.svc.cluster.local
internal_host_ssl_validation: false
external_host: https://www.renner.uno
cookie_domain: renner.uno
access_token_validity: hours=24
refresh_token_validity: days=30
intercept_header_auth: true
# No skip_path_regex: wp-admin's assets are only ever fetched by an
# already-authenticated session, and wp-admin has no manifest or
# service worker — the credentials:'omit' PWA problem that forced
# carve-outs on vscode-gate and retired music-assistant-gate does
# not arise here.
- model: authentik_core.application
identifiers:
slug: wordpress-gate
state: present
attrs:
name: WordPress (Gate)
slug: wordpress-gate
provider: !KeyOf wordpress-gate-provider
meta_launch_url: https://www.renner.uno/wp-admin/
meta_description: "Forward-auth gate in front of wp-admin on www.renner.uno"
# Only WordPress Authors pass the gate — see the group above for the
# membership list. Both accounts that exist on the migrated site are
# covered, so the gate cannot lock out the editor the site is
# currently being kept alive for.
- model: authentik_policies.policybinding
identifiers:
target: !Find [authentik_core.application, [slug, wordpress-gate]]
group: !KeyOf wordpress-authors-group
order: 0
state: present
attrs:
enabled: true
negate: false
failure_result: false
- model: authentik_outposts.outpost
identifiers:
name: wordpress-gate
state: present
attrs:
type: proxy
service_connection: !KeyOf k8s-conn
providers:
- !KeyOf wordpress-gate-provider
config:
log_level: info
authentik_host: "http://authentik-server.authentik.svc.cluster.local"
authentik_host_insecure: false
authentik_host_browser: "https://auth.renner.uno/"
refresh_interval: "minutes=5"
container_image: null
object_naming_template: "ak-outpost-%(name)s"
kubernetes_namespace: authentik
kubernetes_replicas: 1
kubernetes_service_type: ClusterIP
kubernetes_disabled_components: ["ingress"]
kubernetes_image_pull_secrets: []
kubernetes_ingress_annotations: {}
kubernetes_ingress_class_name: null
kubernetes_ingress_secret_name: "authentik-outpost-tls"
kubernetes_json_patches: null
# Passkey login for selected apps. Passkey is mandatory for accounts
# that hold one and unavailable to accounts that do not, so both flows
# branch on enrollment (policy passkey-user-has-webauthn): enrolled →
# WebAuthn, not enrolled → password + MFA. Enrollment status is a
# property of the identified account, so the weaker branch is never
# something a requester can select.
#
# Two parts:
#
# 1. A passwordless AUTHENTICATION flow (username → passkey, no
# password prompt) bound per-provider, so fresh logins to these
@@ -1335,7 +1449,13 @@ data:
# 2. A step-up AUTHORIZATION flow — unlike the authentication flow,
# this runs on EVERY interactive authorize, so an existing
# password-only SSO session from another app cannot slip into
# these apps without presenting a passkey.
# these apps without presenting a passkey. Two things narrow that:
# the stage is skipped outright for accounts with no passkey (they
# have no way to satisfy it), and last_auth_threshold reduces it to
# "without presenting a passkey in this browser within the window"
# — one touch then covers every app, including a later
# password-only session for the same user. The cookie is keyed to
# that user's device, so it never carries across users.
passkey-stepup.yaml: |
version: 1
metadata:
@@ -1356,43 +1476,99 @@ data:
# flow this replaces.
authentication: require_authenticated
- id: passkey-stepup-webauthn-validation
# ONE stage object, shared by the step-up authorization flow and
# the passwordless authentication flow below.
#
# `last_auth_threshold` is enforced with a single browser cookie
# (`authentik_mfa`) whose JWT is both signed with and carries the
# stage's PK, so a validation stage only ever honours a cookie it
# wrote itself. Two separate stage objects therefore each demand
# their own touch, and — sharing the one cookie name — overwrite
# each other, so neither window survives. Sharing this object is
# what lets the login touch satisfy the per-app step-up.
#
# Declared here, ahead of every !KeyOf reference to it: the
# blueprint importer resolves entries top-to-bottom.
- id: passkey-webauthn-validation
model: authentik_stages_authenticator_validate.authenticatorvalidatestage
identifiers:
name: passkey-stepup-webauthn-validation
name: passkey-webauthn-validation
state: present
attrs:
device_classes:
- webauthn
# `configure`, not `deny`: g.renner has Home Assistant access
# but no enrolled WebAuthn device (live check 2026-07-07), so
# a hard deny would lock her out with no self-service path.
# `configure` injects the default WebAuthn enrollment stage
# inline — first login without a passkey becomes "enroll one
# now, then validate it". Still passkey-only: there is no way
# through this stage without a WebAuthn credential.
not_configured_action: configure
configuration_stages:
- !Find [authentik_stages_authenticator_webauthn.authenticatorwebauthnstage, [name, default-authenticator-webauthn-setup]]
# Skip re-validation if a passkey was already presented within
# the window. Non-zero for two reasons: (1) no double-touch
# when the login itself used a passkey seconds earlier;
# (2) NetBird's dashboard silently re-authorizes via
# /silent-auth (prompt=none) on token expiry — a flow that
# always demands interaction would fail those and force a
# visible re-login every access-token lifetime.
# `deny`, not `configure`: the authentication flow reaches this
# stage with the user merely *identified* (a typed username)
# and no password stage in front of it, so inline enrollment
# would let anyone who knows a passkey-less username enroll
# their own passkey and take over that account. The policy
# binding means only accounts that already hold a passkey reach
# this stage at all, so `deny` is defence in depth rather than
# a path anyone walks — accounts without one branch to password
# + MFA instead.
not_configured_action: deny
# Window within which an already-presented passkey satisfies a
# later run of this stage. Non-zero for two reasons: (1) the
# per-app step-up must not demand a second touch seconds after
# the login touch; (2) NetBird's dashboard silently
# re-authorizes via /silent-auth (prompt=none) on token expiry —
# a stage that always demands interaction fails those and
# forces a visible re-login every access-token lifetime.
#
# The cookie is browser-scoped, so inside the window a re-login
# in that same browser is satisfied by the cookie alone and the
# username is all that gets typed. Accepted for a single touch
# per app.
last_auth_threshold: hours=8
webauthn_user_verification: preferred
- model: authentik_flows.flowstagebinding
# Gate on "this account has a passkey", so enrollment status picks
# the branch. Enrolled accounts must use WebAuthn — at login and at
# every authorize — while accounts with no passkey fall back to
# password + MFA. An attacker cannot pick the weaker branch: the
# user is already identified when this runs, so the branch follows
# the victim's enrollment, not the requester's choice.
#
# request.user is the flow's pending_user, not the (anonymous)
# requester: ReevaluateMarker builds the engine with
# plan.context[PLAN_CONTEXT_PENDING_USER] (flows/markers.py). That
# requires the bindings below to keep evaluate_on_plan false and
# re_evaluate_policies true — on-plan evaluation happens before the
# identification stage has resolved a user.
- id: policy-has-passkey
model: authentik_policies_expression.expressionpolicy
identifiers:
name: passkey-user-has-webauthn
state: present
attrs:
expression: |
return ak_user_has_authenticator(request.user, "webauthn")
- id: binding-stepup-webauthn
model: authentik_flows.flowstagebinding
identifiers:
target: !KeyOf passkey-stepup-flow
stage: !KeyOf passkey-stepup-webauthn-validation
stage: !KeyOf passkey-webauthn-validation
order: 10
state: present
attrs:
evaluate_on_plan: false
re_evaluate_policies: true
- model: authentik_policies.policybinding
identifiers:
policy: !KeyOf policy-has-passkey
target: !KeyOf binding-stepup-webauthn
order: 0
state: present
attrs:
enabled: true
negate: false
failure_result: false
# Passwordless authentication flow: identification (username or
# email, NO password stage) → WebAuthn validation → login.
# email, NO password stage) → WebAuthn validation → login, with a
# password + MFA branch for accounts that have no passkey.
- id: passkey-passwordless-flow
model: authentik_flows.flow
identifiers:
@@ -1403,37 +1579,6 @@ data:
title: "Sign in with your passkey"
designation: authentication
# SEPARATE stage from the step-up one, and deliberately `deny`:
# at this point in an authentication flow the user has only been
# *identified* (typed a username), not authenticated — `configure`
# here would let anyone who knows a passkey-less username enroll
# their own passkey and take over that account. Inline enrollment
# for passkey-less users happens exclusively in the authorization
# step-up stage above, where require_authenticated guarantees a
# fully authenticated session. A passkey-less user therefore
# bootstraps via password login on auth.renner.uno, then gets the
# enrollment prompt on first app access.
#
# Declared BEFORE the identification stage because the latter
# references it via !KeyOf (webauthn_stage), and the blueprint
# importer resolves entries top-to-bottom.
- id: passkey-passwordless-webauthn-validation
model: authentik_stages_authenticator_validate.authenticatorvalidatestage
identifiers:
name: passkey-passwordless-webauthn-validation
state: present
attrs:
device_classes:
- webauthn
not_configured_action: deny
# Small non-zero window so a passkey presented via the
# identification stage's autofill (webauthn_stage below) is
# not challenged a second time by this stage at order 20 in
# the same flow run. The skip is session-scoped, so a fresh
# (attacker) session never inherits it.
last_auth_threshold: minutes=5
webauthn_user_verification: preferred
# webauthn_stage enables passkey autofill (conditional WebAuthn,
# authentik >= 2025.12): the browser offers enrolled passkeys in
# the username field's autofill — one tap, no username typed.
@@ -1449,7 +1594,7 @@ data:
user_fields:
- username
- email
webauthn_stage: !KeyOf passkey-passwordless-webauthn-validation
webauthn_stage: !KeyOf passkey-webauthn-validation
- model: authentik_flows.flowstagebinding
identifiers:
@@ -1458,12 +1603,87 @@ data:
order: 10
state: present
- model: authentik_flows.flowstagebinding
# Order 20 and 21/22 are the two mutually exclusive branches.
- id: binding-passwordless-webauthn
model: authentik_flows.flowstagebinding
identifiers:
target: !KeyOf passkey-passwordless-flow
stage: !KeyOf passkey-passwordless-webauthn-validation
stage: !KeyOf passkey-webauthn-validation
order: 20
state: present
attrs:
evaluate_on_plan: false
re_evaluate_policies: true
- model: authentik_policies.policybinding
identifiers:
policy: !KeyOf policy-has-passkey
target: !KeyOf binding-passwordless-webauthn
order: 0
state: present
attrs:
enabled: true
negate: false
failure_result: false
# The negated branch. failure_result: true is the safety net — the
# UserLoginStage defaults a missing authentication backend to
# BACKEND_INBUILT instead of refusing (user_login/stage.py), so a
# flow that drops BOTH branches would log the user in on a typed
# username alone. A PolicyException is resolved to failure_result
# before negate is applied (policies/process.py), so a broken
# expression pulls the password stage IN and degrades everyone to
# password + MFA rather than to no factor at all.
- id: binding-passwordless-password
model: authentik_flows.flowstagebinding
identifiers:
target: !KeyOf passkey-passwordless-flow
stage: !Find [authentik_stages_password.passwordstage, [name, default-authentication-password]]
order: 21
state: present
attrs:
evaluate_on_plan: false
re_evaluate_policies: true
- model: authentik_policies.policybinding
identifiers:
policy: !KeyOf policy-has-passkey
target: !KeyOf binding-passwordless-password
order: 0
state: present
attrs:
enabled: true
negate: true
failure_result: true
# Same MFA validation the default authentication flow runs at its
# order 30, so the fallback is not a weaker way into these apps
# than auth.renner.uno's own login already is. It carries
# not_configured_action: configure, which is safe here and nowhere
# else in this flow: the password stage above has established a
# first factor by the time it runs, so it doubles as the enrollment
# path for accounts holding no authenticator at all.
- id: binding-passwordless-mfa
model: authentik_flows.flowstagebinding
identifiers:
target: !KeyOf passkey-passwordless-flow
stage: !Find [authentik_stages_authenticator_validate.authenticatorvalidatestage, [name, default-authentication-mfa-validation]]
order: 22
state: present
attrs:
evaluate_on_plan: false
re_evaluate_policies: true
- model: authentik_policies.policybinding
identifiers:
policy: !KeyOf policy-has-passkey
target: !KeyOf binding-passwordless-mfa
order: 0
state: present
attrs:
enabled: true
negate: true
failure_result: true
- model: authentik_flows.flowstagebinding
identifiers:
@@ -1577,3 +1797,19 @@ data:
redirect_uris:
- url: https://dash.renner.uno/login/generic_oauth
matching_mode: strict
# Superseded by the single passkey-webauthn-validation stage above.
# Their flowstagebindings are FK-CASCADEd away with them; the
# identification stage's webauthn_stage FK is SET_NULL and is
# re-pointed above, so no dangling reference survives. Declared
# last: the importer applies entries top-to-bottom inside one
# transaction, so the replacements exist before these go.
- model: authentik_stages_authenticator_validate.authenticatorvalidatestage
identifiers:
name: passkey-stepup-webauthn-validation
state: absent
- model: authentik_stages_authenticator_validate.authenticatorvalidatestage
identifiers:
name: passkey-passwordless-webauthn-validation
state: absent
@@ -14,6 +14,7 @@ resources:
- zone-configurator-gate-egress.yaml
- alertmanager-gate-egress.yaml
- prometheus-gate-egress.yaml
- wordpress-gate-egress.yaml
- outpost-servicemonitors.yaml
- ldap-outpost-cert.yaml
- ldap-outpost-lb.yaml
@@ -0,0 +1,28 @@
---
# Allow the dedicated wordpress-gate outpost to egress into the wordpress
# namespace and reach WordPress (:80). Without this, the authentik-namespace
# baseline CNP has no egress rule to wordpress, so the gate outpost's
# forwarded requests are dropped. Scoped to the gate outpost's pod label so no
# other authentik component can reach it. Mirrors esphome-gate-egress.yaml.
#
# Note this gate only ever sees /wp-admin and /wp-login.php — the public site
# is routed straight to the same Service by a separate Ingress, so this rule
# is not the only path into the wordpress namespace, unlike the paperless case.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: wordpress-gate-egress
namespace: authentik
spec:
endpointSelector:
matchLabels:
goauthentik.io/outpost-name: wordpress-gate
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": wordpress
app.kubernetes.io/name: wordpress
toPorts:
- ports:
- port: "80"
protocol: TCP
+18
View File
@@ -160,6 +160,24 @@ data:
conditions:
- "[STATUS] == any(200, 301, 302)"
# Terminus admin UI. Checks the TLS route only — the cleartext
# device endpoint (trmnl-device.renner.uno) is the same pod, so a
# failure here covers both. Terminus exposes no health endpoint, so
# this is a root-page check: the login page answers 200, and an
# unauthenticated request may redirect.
- name: Web Service 10
group: services
url: https://trmnl.renner.uno
interval: 60s
ui:
hide-hostname: true
hide-url: true
hide-port: true
hide-conditions: true
hide-errors: true
conditions:
- "[STATUS] == any(200, 301, 302)"
# Removed 2026-05-21: Jellyfin (media.renner.uno), Wiki (wiki.renner.uno),
# InfluxDB (influx.renner.uno) — hostnames NXDOMAIN, no Ingress; services
# no longer deployed. Re-add when/if they come back.
@@ -38,7 +38,7 @@ spec:
# gets the heavy HA pod (8Gi) off the soon-to-be-tiny edge box.
containers:
- name: home-assistant
image: ghcr.io/home-assistant/home-assistant:2026.8.1 # {"$imagepolicy": "flux-system:home-assistant"}
image: ghcr.io/home-assistant/home-assistant:2026.8.2 # {"$imagepolicy": "flux-system:home-assistant"}
env:
- name: TZ
value: Europe/Berlin
+27 -1
View File
@@ -17,7 +17,33 @@ spec:
spec:
containers:
- name: ical-proxy
image: ghcr.io/konairius/ical-proxy:latest
# Pinned to a commit build rather than :latest.
#
# :latest never rolls. The tag moves when a build publishes, but
# nothing tells the Deployment about it: the pod template is
# unchanged, so Kubernetes has no reason to replace anything, and
# this image is deliberately NOT enrolled in Flux image automation
# — see kubernetes/infrastructure/image-automation/README.md, which
# lists ghcr.io/konairius/ical-proxy as having "only main, latest,
# and main-<sha> commit builds; no release tags", so there is no
# ImagePolicy watching it. The pods therefore sat on a 21-day-old
# image while :latest moved underneath them, and the only way to
# pick up a new build was a manual restart.
#
# A pinned tag makes the rollout a git change: Flux applies the new
# pod template and Kubernetes rolls it, which is also what makes
# what is RUNNING recoverable from this repo rather than "whatever
# :latest meant the last time a pod happened to restart".
#
# main-99a912b is the merge of konairius/ical-proxy#12, which adds
# the summary_match/summary_replace parameters this deployment
# needs to strip "| Abfuhrkalender - Landkreis Amberg-Sulzbach"
# from every waste collection.
#
# Bumping this is a manual, reviewed edit. The tag format is
# main-<short-sha> (docker/metadata-action type=sha with the branch
# as prefix).
image: ghcr.io/konairius/ical-proxy:main-99a912b
ports:
- containerPort: 8080
name: http
@@ -52,7 +52,7 @@ spec:
# stdlib only; Flux keeps the tag in lockstep via the shared image
# policy) — no extra dependency.
- name: reconcile-publish-ip
image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
image: ghcr.io/music-assistant/server:2.9.13 # {"$imagepolicy": "flux-system:music-assistant"}
command: ["/app/venv/bin/python", "/scripts/reconcile.py"]
volumeMounts:
- name: data
@@ -62,7 +62,7 @@ spec:
readOnly: true
containers:
- name: music-assistant
image: ghcr.io/music-assistant/server:2.9.11 # {"$imagepolicy": "flux-system:music-assistant"}
image: ghcr.io/music-assistant/server:2.9.13 # {"$imagepolicy": "flux-system:music-assistant"}
env:
- name: LOG_LEVEL
value: info
+1 -1
View File
@@ -22,7 +22,7 @@ spec:
type: RuntimeDefault
containers:
- name: gotenberg
image: gotenberg/gotenberg:8.35.0 # {"$imagepolicy": "flux-system:gotenberg"}
image: gotenberg/gotenberg:8.36.0 # {"$imagepolicy": "flux-system:gotenberg"}
args:
- "gotenberg"
- "--chromium-disable-javascript=true"
+41
View File
@@ -0,0 +1,41 @@
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: tr-pg
namespace: trmnl
spec:
instances: 1
smartShutdownTimeout: 10
stopDelay: 60
# PG 18, NOT the repo-standard 16. Terminus ships a committed
# config/db/structure.sql containing `SET transaction_timeout = 0;` — a GUC
# that does not exist before PostgreSQL 17, so schema load against a 16
# server dies with `unrecognized configuration parameter`. Upstream's
# compose.yml develops against 18.4, so 18 is the tested target; 17 is the
# bare minimum. Bare major tag matches the other clusters (minor security
# bumps flow automatically, major upgrades stay an explicit decision).
imageName: ghcr.io/cloudnative-pg/postgresql:18
enableSuperuserAccess: true
storage:
storageClass: ceph-block
size: 5Gi
plugins:
- name: barman-cloud.cloudnative-pg.io
isWALArchiver: true
parameters:
barmanObjectName: tr-pg-store
bootstrap:
initdb:
database: terminus
owner: terminus
postgresql:
parameters:
shared_buffers: "128MB"
max_connections: "50"
wal_compression: "on"
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 512Mi
+48
View File
@@ -0,0 +1,48 @@
# Device-facing endpoint. Cleartext HTTP, deliberately.
#
# The TRMNL firmware initiates every request and cannot be handed the
# internal CA, so terminating TLS here would simply break polling. Scope of
# the exposure: this Service is LAN-only (no public DNS label, no Cloudflare
# record), it carries no admin session — the device authenticates with its
# MAC in the `ID` header plus a per-device api_key — and the Terminus admin
# UI stays on the TLS route at trmnl.renner.uno.
#
# Same shape as home-assistant-bgp: a dedicated address on the Cilium BGP
# pool rather than a TLS-less Ingress, which would be a new pattern for this
# repo. The device is provisioned with the HOSTNAME, not the address.
#
# The address is requested explicitly, matching every other LB service here.
# Left to allocate freely, Cilium IPAM handed out the first free address in
# the pool block (10.2.99.0-10.2.99.255) — which is 10.2.99.0, the base
# address of the range. Each LB address is advertised to the UDM as its own
# BGP route so that is routable in principle, but a .0 host address is the
# kind of thing constrained embedded IP stacks get wrong, and the TRMNL
# firmware is the single client that has to work here. Not worth the risk
# for an address nobody chose on purpose.
#
# .89 continues the existing sequence (.81-.88 are in use) and matches the
# fd99::89 that IPAM had already picked for v6.
apiVersion: v1
kind: Service
metadata:
name: terminus-device
namespace: trmnl
labels:
app.kubernetes.io/name: terminus
lb-pool: bgp
annotations:
external-dns.alpha.kubernetes.io/hostname: trmnl-device.renner.uno
lbipam.cilium.io/ips: "10.2.99.89,fd99::89"
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ipFamilyPolicy: PreferDualStack
ipFamilies:
- IPv4
- IPv6
selector:
app.kubernetes.io/name: terminus
ports:
- name: http
port: 80
targetPort: http
+18
View File
@@ -0,0 +1,18 @@
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: terminus-secrets
namespace: trmnl
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: terminus-secrets
# Holds APP_SECRET only (session/cookie/CSRF key). DATABASE_URL is not in
# here — it comes straight off the CNPG-generated `tr-pg-app` Secret's
# `uri` key, so the password rotates with the cluster and is never copied.
dataFrom:
- extract:
key: apps/trmnl
@@ -0,0 +1,15 @@
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: hetzner-s3-creds
namespace: trmnl
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: hetzner-s3-creds
dataFrom:
- extract:
key: seed/trmnl/hetzner-creds
+28
View File
@@ -0,0 +1,28 @@
# Admin UI only — login, device registration, playlists, screen previews.
# The device never touches this route: the firmware speaks plain HTTP and
# would reject the internal CA, so it uses the cleartext LoadBalancer in
# device-service.yaml instead. Both front the same Service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: terminus
namespace: trmnl
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
spec:
ingressClassName: traefik
tls:
- hosts:
- trmnl.renner.uno
secretName: trmnl-tls
rules:
- host: trmnl.renner.uno
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: terminus
port:
number: 80
+20
View File
@@ -0,0 +1,20 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- externalsecret.yaml
- pvcs.yaml
- valkey.yaml
- terminus.yaml
- ingress.yaml
- device-service.yaml
# Headless HA screenshotter. Lives in this namespace because Terminus
# FETCHES rendered screens back from it (uri delivery mode), so the
# two need the intra-namespace path.
- renderer.yaml
- renderer-externalsecret.yaml
# tr-pg CNPG Postgres + offsite Barman backup. Note this cluster runs
# PostgreSQL 18, not the repo-standard 16 — see the comment in cluster.yaml.
- cluster.yaml
- objectstore.yaml
- scheduled-backup.yaml
- hetzner-creds-externalsecret.yaml
+30
View File
@@ -0,0 +1,30 @@
---
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: tr-pg-store
namespace: trmnl
spec:
configuration:
# Offsite Hetzner Object Storage (fsn1), same bucket as the other CNPG
# clusters. No SSE: Hetzner supports only SSE-C, unusable by barman-cloud
# (plugin #646), so at-rest encryption is unavailable; transit is TLS.
destinationPath: s3://renner-uno-k8s-cnpg/tr-pg
endpointURL: https://fsn1.your-objectstorage.com
s3Credentials:
accessKeyId:
name: hetzner-s3-creds
key: ACCESS_KEY_ID
secretAccessKey:
name: hetzner-s3-creds
key: ACCESS_SECRET_KEY
# Region is required (REGION=fsn1) — without it boto3's HeadBucket
# (barman-cloud-check-wal-archive during bootstrap.recovery) 400s.
region:
name: hetzner-s3-creds
key: REGION
wal:
compression: gzip
data:
compression: gzip
retentionPolicy: "30d"
+57
View File
@@ -0,0 +1,57 @@
---
# Uploads + synchronised fonts for the Terminus pod. Mounted twice by
# subPath (uploads/, fonts/) into both the web and sidekiq containers, which
# share one pod — so RWO is sufficient and the Deployment uses `Recreate`.
#
# Contents: rendered device screens and the font set pulled by the
# FONT_SYNCHRONIZER job. Persisting fonts here (rather than an emptyDir)
# means a pod restart does not depend on upstream being reachable.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: terminus-data
namespace: trmnl
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 5Gi
---
# Valkey RDB snapshots. The Sidekiq queue is reconstructable — the
# synchroniser jobs re-enqueue on boot — so this is small and losing it is
# an inconvenience, not data loss.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: terminus-valkey-data
namespace: trmnl
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 1Gi
---
# Renderer state: schedules.json plus the rendered screens under
# output/, which Terminus fetches back over HTTP in uri delivery mode.
#
# This mount is REQUIRED, not just nice to have. The app picks its data
# directory by testing whether /data exists — with nothing mounted it
# silently falls back to /app/data inside the container, so schedules
# and rendered output would vanish on every restart and the panel would
# go blank until the next render.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: trmnl-renderer-data
namespace: trmnl
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 2Gi
@@ -0,0 +1,27 @@
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: trmnl-renderer-secrets
namespace: trmnl
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: trmnl-renderer-secrets
# Kept separate from apps/trmnl (which holds Terminus's APP_SECRET)
# because these are credentials the RENDERER holds about other
# systems, with different rotation stories:
#
# HA_ACCESS_TOKEN — a dedicated Home Assistant long-lived token,
# not the operator's personal one, so the
# renderer can be revoked in isolation.
# BYOS_ACCESS_TOKEN — Terminus JWTs, minted once out of band from
# BYOS_REFRESH_TOKEN a Terminus login. The PASSWORD deliberately
# never enters the cluster; stock Terminus
# disables session expiry, so these are
# effectively permanent.
dataFrom:
- extract:
key: apps/trmnl/renderer
+385
View File
@@ -0,0 +1,385 @@
# Headless renderer: screenshots the Home Assistant e-ink dashboard,
# dithers it to 16 greys and hands it to Terminus.
#
# Upstream ships this as a Home Assistant add-on. Our HA is a plain
# container with no Supervisor, so it runs standalone here instead —
# viable because the app prefers HOME_ASSISTANT_URL/ACCESS_TOKEN env
# vars over /data/options.json and authenticates with an ordinary
# long-lived token. SUPERVISOR_TOKEN is deliberately NOT set: it only
# flips an internal isAddOn flag, and setting it would switch the app
# to add-on behaviour (fixed port, /data assumptions).
#
# DIRECTION OF TRAVEL, which is the non-obvious part: with the
# byos-hanami webhook in `uri` delivery mode the renderer does NOT
# push image bytes. It POSTs a URL pointing at its own /output/<file>
# endpoint, and TERMINUS FETCHES THE IMAGE BACK from it. That is why
# both live in this namespace — the pull is intra-namespace and is
# already permitted by the default-deny-ingress rule.
apiVersion: v1
kind: ConfigMap
metadata:
name: trmnl-renderer-schedule
namespace: trmnl
data:
# Template for /data/schedules.json. The init container splices the
# BYOS tokens in; everything non-secret is declared here so the
# schedule is reviewable in git rather than clicked into the app's
# web UI (which has no authentication and is not exposed).
#
# The app re-reads schedules.json every 60 s, so edits here take
# effect without a restart once the init container has rewritten it.
#
# lang is "de", not null. It calls Home Assistant's own
# _selectLanguage() for the screenshot session, so the rendered page
# is localised by HA rather than by hand. With it left null the
# dashboard grew a German weekday array, a German month array and a
# fifteen-entry weather-condition dict — reimplementing translations
# HA already ships. The instance itself stays language "en"; this
# only affects the render session.
#
# model_id is 1 — TRMNL X (name "v2", 1872x1404, 16 colours) — and it
# is an INTEGER, not a name. Confirmed against the running server:
# GET /api/screens returns model_id as a number, and upstream's
# example value "og_1bit" matches no model on this instance (the
# 2-bit OG model here is id 11, name "og_plus"). Getting this wrong
# shows up as Terminus rejecting the pushed screen with a 422.
#
# All three auth fields below are placeholders the init container
# replaces — including obtained_at, which must end up a real epoch-MS
# stamp. The 0 here is inert only because it is always overwritten;
# left as 0 the scheduler reads the tokens as ancient and refuses to
# push at all ("expired beyond the refresh window"), with no render
# attempt and no error beyond one startup warning.
schedules.template.json: |
[
{
"id": "trmnl-entrance",
"name": "Entrance e-ink panel",
"enabled": true,
"cron": "* * * * *",
"interval_minutes": 1,
"ha_mode": true,
"dashboard_path": "/trmnl-entrance/home",
"target_url": null,
"viewport": { "width": 1872, "height": 1404 },
"device": "schedule_trmnl_x",
"crop": { "enabled": false, "x": 0, "y": 0, "width": 0, "height": 0 },
"format": "png",
"rotate": null,
"zoom": 1,
"wait": null,
"theme": "Graphite E-ink Light",
"lang": "de",
"dark": false,
"invert": false,
"timestamp": false,
"dithering": {
"enabled": true,
"method": "floyd-steinberg",
"palette": "gray-16",
"bitDepth": 4,
"gammaCorrection": true,
"blackLevel": 0,
"whiteLevel": 100,
"normalize": true,
"saturationBoost": false
},
"webhook_url": "http://terminus.trmnl.svc.cluster.local/api/screens",
"webhook_headers": {},
"webhook_format": {
"format": "byos-hanami",
"byosConfig": {
"label": "Eingang",
"name": "trmnl-entrance",
"model_id": 1,
"preprocessed": true,
"addon_base_url": "http://trmnl-renderer.trmnl.svc.cluster.local:10000",
"delivery_mode": "uri",
"auth": {
"enabled": true,
"access_token": "__BYOS_ACCESS_TOKEN__",
"refresh_token": "__BYOS_REFRESH_TOKEN__",
"obtained_at": 0
}
}
},
"createdAt": "2026-08-11T00:00:00.000Z",
"updatedAt": "2026-08-11T00:00:00.000Z"
}
]
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: trmnl-renderer
namespace: trmnl
labels:
app.kubernetes.io/name: trmnl-renderer
annotations:
# BOTH sources need this, and for the same reason: the init
# container writes /data/schedules.json ONCE, at pod start. Nothing
# re-reads either source afterwards.
#
# The app re-reads schedules.json every 60s, which makes it look
# self-updating — but only the FILE, and the file is only rewritten
# by the init container. So a ConfigMap or Secret change lands in
# git, updates the object, and the running renderer keeps the old
# values indefinitely.
#
# Both halves were hit repeatedly in practice and each time needed
# `kubectl delete pod` by hand: the theme name, the schedule's lang
# field, and every re-mint of the BYOS tokens. A fix that depends on
# someone remembering to restart a pod is not deployed.
#
# Named per-resource rather than the blanket
# reloader.stakater.com/auto: "true", matching terminus.yaml and
# mosquitto-deployment.yaml.
configmap.reloader.stakater.com/reload: trmnl-renderer-schedule
secret.reloader.stakater.com/reload: trmnl-renderer-secrets
spec:
replicas: 1
# RWO PVC, and two renderers would push conflicting screens for the
# same device.
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: trmnl-renderer
template:
metadata:
labels:
app.kubernetes.io/name: trmnl-renderer
spec:
enableServiceLinks: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
initContainers:
# Writes /data/schedules.json from the ConfigMap template with
# the BYOS tokens injected.
#
# jq rather than sed: the tokens are JWTs and go into a JSON
# string, so they need JSON-safe escaping. A sed substitution
# would corrupt on any character sed treats as special.
#
# No HTTP here on purpose. The app does expose a /api/byos/login
# proxy that exchanges a Terminus username+password for these
# tokens, but that would mean shipping the Terminus PASSWORD
# into the cluster and doing a network call at every pod start.
# Tokens are therefore minted once out of band and kept in
# OpenBao — the password never enters the cluster.
#
# That only works because Terminus is configured with
# API_ACCESS_TOKEN_PERIOD = 10 years (see terminus.yaml). It is
# NOT, as an earlier version of this comment claimed, because
# stock Terminus issues permanent tokens — the stock default is
# 1800s, and this deployment demonstrated it by having the
# webhook start failing with "expired JWT access token" about
# thirty minutes after seeding.
- name: render-schedule
image: gitea.renner.uno/gitea_admin/alpine-jq:2026-07-30
command:
- sh
- -c
- |
set -eu
# obtained_at is stamped at write time, in MILLISECONDS.
# It is not cosmetic: the scheduler derives token age from
# it, and a literal 0 made the app treat freshly-minted
# tokens as ancient and skip the webhook entirely —
# "BYOS tokens ... expired beyond the refresh window"
#
# Stamping it "now" is only safe because Terminus is
# configured with API_ACCESS_TOKEN_PERIOD = 10 years (see
# terminus.yaml). Against the 30-minute default it would be
# actively harmful: the app would believe a stale token had
# 30 fresh minutes left, skip its proactive refresh, and
# then be unable to recover, because refreshing requires a
# still-valid token. If that setting is ever reverted, this
# must become a real mint time carried alongside the tokens.
jq --arg at "$BYOS_ACCESS_TOKEN" \
--arg rt "$BYOS_REFRESH_TOKEN" \
'(.[].webhook_format.byosConfig.auth.access_token) = $at
| (.[].webhook_format.byosConfig.auth.refresh_token) = $rt
| (.[].webhook_format.byosConfig.auth.obtained_at) = (now * 1000 | floor)' \
/template/schedules.template.json > /data/schedules.json
echo "wrote /data/schedules.json ($(wc -c < /data/schedules.json) bytes)"
env:
- name: BYOS_ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: trmnl-renderer-secrets
key: BYOS_ACCESS_TOKEN
- name: BYOS_REFRESH_TOKEN
valueFrom:
secretKeyRef:
name: trmnl-renderer-secrets
key: BYOS_REFRESH_TOKEN
volumeMounts:
- name: data
mountPath: /data
- name: schedule-template
mountPath: /template
readOnly: true
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
containers:
# Pinned, not enrolled in image automation — same reasoning as
# Terminus itself. This drives a wall-mounted panel and upstream
# is pre-1.0; an unattended bump that changes the dithering
# pipeline or the webhook payload shows up as a corrupted
# screen, not as a failed rollout.
#
# Note the image is ARCH-SUFFIXED: upstream publishes no
# multi-arch manifest, only trmnl-ha-amd64 and trmnl-ha-aarch64.
- name: renderer
image: ghcr.io/usetrmnl/trmnl-ha-amd64:0.9.5
ports:
- name: http
containerPort: 10000
env:
# The in-cluster Service name, NOT ha.renner.uno and not
# home.renner.uno. Both public names are unusable from a
# pod: ha.renner.uno resolves to the home-assistant-bgp
# LoadBalancer's external address (10.2.99.87), which is not
# backed by an Ingress, so CoreDNS's k8s_gateway delegation
# does not intercept it and the pod hairpins off the LB.
#
# This cluster has hit that exact failure before — see the
# comment on kubernetes/infrastructure/openbao/rotation/
# home-assistant.yaml, which reaches HA the same way and
# states it outright: "The public name is unusable from a
# pod (ingress-LB hairpin)."
#
# FAILS QUIETLY if wrong: the schedule keeps firing every
# minute, each render errors on connect, and the panel just
# never updates. Nothing goes CrashLoopBackOff.
- name: HOME_ASSISTANT_URL
value: "http://home-assistant-bgp.home-assistant.svc.cluster.local:8123"
- name: TZ
value: "Europe/Berlin"
# Chromium will not start without a writable HOME. Running
# as uid 1000 with no matching passwd entry leaves HOME="/",
# which is root-owned 0755, and the browser dies on launch
# with
# chrome_crashpad_handler: --database is required
# because crashpad cannot create its database directory.
#
# Diagnosed on the running pod, not guessed: HOME=/, uid=1000,
# / not writable, /tmp writable. Crash dumps are throwaway, so
# /tmp is the right home for them rather than the PVC.
#
# This is a consequence of OUR non-root securityContext, not
# of the image — so it must move together with runAsUser.
- name: HOME
value: "/tmp"
# Dedicated long-lived token, NOT the operator's personal
# one, so the renderer's access can be revoked on its own.
# It is injected into the headless browser's localStorage,
# which is how the app authenticates to the HA frontend.
- name: ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: trmnl-renderer-secrets
key: HA_ACCESS_TOKEN
volumeMounts:
- name: data
mountPath: /data
# Bounded scratch backing HOME. Chromium writes crashpad
# state and profile data here on every launch, and it now
# launches once a minute for the life of the pod — on the
# container's writable layer that is unbounded growth
# against node disk, with no ephemeral-storage limit to
# catch it. A sizeLimit converts a slow, cluster-wide
# node-disk-pressure problem into a bounded and clearly
# attributable eviction of this pod.
- name: tmp
mountPath: /tmp
# HTTP probes here, unlike terminus.yaml's TCP-only ones, because
# /health is confirmed rather than assumed: the upstream image's
# own Dockerfile declares
# HEALTHCHECK ... curl -f http://localhost:10000/health
# so the route is part of the image's published contract. Terminus
# got TCP probes precisely because it publishes no such endpoint.
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 10
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
resources:
requests:
cpu: 250m
memory: 768Mi
# Chromium is a SEPARATE process from the app, and the app's
# own watchdog only measures its own RSS (restarting itself
# past 900 MB). The limit therefore has to cover the app's
# ceiling PLUS a browser rendering a 1872x1404 page, which
# is well above the 512 MB upstream's troubleshooting note
# suggests.
limits:
memory: 2Gi
# Chromium normally needs either SYS_ADMIN or a user namespace to
# start its own sandbox, which would make dropping ALL fatal. It
# is safe here because the app launches the browser with
# --no-sandbox --disable-setuid-sandbox --no-zygote (and
# --disable-dev-shm-usage, which is why no /dev/shm emptyDir is
# mounted either). Those flags are baked into the app's own
# PUPPETEER_ARGS, not something we pass, so this holds as long as
# the pinned tag does — worth re-checking on any bump.
#
# Confirmed on the running pod: the capability drop is genuinely
# not what broke the browser. The first deploy DID crash it, but
# on a writable-HOME problem caused by runAsUser — see the HOME
# env above. Do not "fix" a future Chromium launch failure by
# loosening this block before checking HOME and /tmp.
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
volumes:
- name: data
persistentVolumeClaim:
claimName: trmnl-renderer-data
- name: schedule-template
configMap:
name: trmnl-renderer-schedule
- name: tmp
emptyDir:
sizeLimit: 512Mi
---
apiVersion: v1
kind: Service
metadata:
name: trmnl-renderer
namespace: trmnl
labels:
app.kubernetes.io/name: trmnl-renderer
spec:
# ClusterIP only, and deliberately no Ingress. The app's web UI on
# this port has NO authentication of any kind — upstream says so
# outright — so it must not be routable from the LAN. Terminus
# reaches it in-namespace to fetch rendered screens; nothing else
# needs to.
selector:
app.kubernetes.io/name: trmnl-renderer
ports:
- name: http
port: 10000
targetPort: http
@@ -0,0 +1,16 @@
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
name: tr-pg-daily
namespace: trmnl
spec:
# 03:00 local, same slot as the other CNPG clusters (the Velero PVC
# schedule for this namespace runs later, at 08:15).
schedule: "0 0 3 * * *"
immediate: true
backupOwnerReference: self
cluster:
name: tr-pg
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
+256
View File
@@ -0,0 +1,256 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: terminus-config
namespace: trmnl
data:
TZ: "Europe/Berlin"
HANAMI_PORT: "2345"
# What the device is told to fetch from. Terminus bakes this into the
# absolute `image_url` it hands back on /api/display, so it MUST be the
# cleartext LAN name, not trmnl.renner.uno — the firmware speaks plain
# HTTP and would reject the internal CA. No trailing slash: the same
# string is typed into the captive portal at provisioning time.
API_URI: "http://trmnl-device.renner.uno"
KEYVALUE_URL: "redis://terminus-valkey:6379"
# Rack::Attack rate-limit bypass. Beyond upstream's RFC1918 defaults this
# adds fc00::/7, which covers the pod CIDR (fd10:244::/48), the Cilium LB
# pool (fd99::/120) and the LAN ULA — without it every IPv6 client,
# including the device and Traefik, is rate-limited as if it were remote.
RACK_ATTACK_ALLOWED_SUBNETS: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.1,::1,fc00::/7"
# 10 years, against a default of 1800s (30 minutes).
#
# Documented upstream in doc/configuration.adoc:
# API_ACCESS_TOKEN_PERIOD — "The amount of time, in seconds, the JWT
# access token remains valid before needing to refresh your access
# token." Default: 1,800 seconds (30 minutes).
# Corroborated on this instance: a token minted here was rejected about
# thirty minutes later, i.e. the documented default behaving as written.
#
# This exists for the renderer, which pushes screens to /api/screens with
# a JWT held in OpenBao. At the default period that arrangement cannot
# work unattended, and it fails in a way that has no recovery: minting a
# replacement token via POST /api/jwt requires a still-VALID access token
# in the Authorization header, so once one expires it cannot be refreshed
# from itself. Observed exactly that on first deploy —
# [webhook] Webhook failed: 400 {"error":"expired JWT access token"}
# [webhook] BYOS auth: refresh failed (400) {"error":"expired JWT ..."}
# — the refresh attempt being rejected by the very expiry it was trying
# to repair.
#
# The renderer's own code assumes "stock Terminus disables session
# expiration and issues effectively permanent tokens". That is simply not
# true of this deployment, so the assumption is made true here rather
# than worked around in the client.
#
# The trade is deliberate: a long-lived JWT with full Terminus API rights
# now sits in OpenBao. Accepted because the alternative was putting the
# Terminus account PASSWORD into the cluster so an init container could
# mint a short-lived token on every pod start, which is a strictly worse
# secret to hold. The token is revocable by rotating the account's
# credentials; it grants access only to this LAN-only instance.
#
# NOTE: the period is fixed at mint time. Changing this does not extend
# tokens that already exist — they must be re-minted afterwards.
API_ACCESS_TOKEN_PERIOD: "315360000"
# The OTHER half of the same problem, and the reason the first fix
# did not hold. Terminus has TWO independent expiries: the JWT's own
# lifetime (above) and the SESSION behind it. Raising only the token
# period left the session at its defaults, so pushes kept working for
# a while and then failed with
# HTTP 401 {"error":"This session has expired, please login again"}
# which is a different error, from a different mechanism, than the
# "expired JWT access token" the first fix addressed.
#
# Documented in doc/configuration.adoc:
# SESSION_EXPIRATION_ENABLED — "Determines if your browser session
# or JWT API token is expired or not." Default: true.
# SESSION_INACTIVITY_LIMIT 1800s, SESSION_LIFETIME_LIMIT 86400s.
#
# Disabling it is exactly the state the renderer's own code assumes
# ("stock Terminus disables session expiration and issues ~100-year
# tokens") — an assumption that is false by default and has to be
# made true here, same as the token period.
#
# Same trade as above, and it is the deciding one for an unattended
# appliance: a session that expires needs a human with a password to
# restore it, and this panel has neither.
SESSION_EXPIRATION_ENABLED: "false"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: terminus
namespace: trmnl
labels:
app.kubernetes.io/name: terminus
annotations:
# terminus-config is consumed via envFrom, and env vars are read ONCE at
# container start — Kubernetes does not hot-reload them. The ConfigMap is
# a plain resource, not a configMapGenerator, so editing it does not
# change the pod template hash and Flux will not roll the pods either.
# Without this annotation a ConfigMap-only change (such as raising
# API_ACCESS_TOKEN_PERIOD) applies to git and to the ConfigMap object
# while the running app keeps the old value indefinitely — a silent
# no-op, with the manifest and reality disagreeing.
#
# Named per-ConfigMap rather than the blanket `reloader.stakater.com/
# auto: "true"`, matching mosquitto-deployment.yaml: enrolling anything
# further stays a visible, reviewable edit to this line.
configmap.reloader.stakater.com/reload: terminus-config
spec:
replicas: 1
# RWO PVC shared by both containers in this pod, and APP_SETUP runs
# migrations on boot — two concurrent web pods would race on both.
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: terminus
template:
metadata:
labels:
app.kubernetes.io/name: terminus
spec:
enableServiceLinks: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
# Tag pinned deliberately and NOT enrolled in Flux image automation.
# Terminus is pre-1.0 with no GitHub Releases and no changelog; an
# unattended bump that changes the firmware or /api/display contract
# is a device-bricking class of risk. Bumps are a reviewed, manual
# act — see kubernetes/infrastructure/image-automation/README.md.
- name: terminus
image: ghcr.io/usetrmnl/terminus:0.68.0
ports:
- name: http
containerPort: 2345
envFrom:
- configMapRef:
name: terminus-config
env:
# Runs `hanami assets compile` + `hanami db migrate` in the
# entrypoint before exec'ing puma. Set on the web container
# only — the sidekiq sidecar must not race the migration.
- name: APP_SETUP
value: "true"
# CNPG generates this Secret; `uri` already carries the rotated
# password and points at the tr-pg-rw service.
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: tr-pg-app
key: uri
- name: APP_SECRET
valueFrom:
secretKeyRef:
name: terminus-secrets
key: APP_SECRET
volumeMounts:
- name: data
mountPath: /app/public/uploads
subPath: uploads
- name: data
mountPath: /app/public/fonts
subPath: fonts
# TCP probes, not HTTP: Terminus exposes no documented health
# endpoint, and every real route is behind either auth or
# Rack::Attack. Tighten to an HTTP GET once the route set is
# confirmed against the running app.
startupProbe:
tcpSocket:
port: http
periodSeconds: 10
# ~10 min: first boot compiles assets and migrates a fresh DB.
failureThreshold: 60
livenessProbe:
tcpSocket:
port: http
periodSeconds: 30
timeoutSeconds: 5
readinessProbe:
tcpSocket:
port: http
periodSeconds: 10
timeoutSeconds: 5
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
memory: 1Gi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
- name: sidekiq
image: ghcr.io/usetrmnl/terminus:0.68.0
command: ["bundle", "exec", "sidekiq", "-r", "./config/sidekiq.rb"]
envFrom:
- configMapRef:
name: terminus-config
# EXPECTED on a fresh deploy: this container loses the race against
# the web container's migration and exits 1 with
# `PG::UndefinedTable: relation "account" does not exist`, then
# comes up healthy on the restart once the schema exists.
# Observed on first deploy 2026-08-10; converged in ~1s. There is
# no upstream ordering hook, and gating it behind an initContainer
# that probes for a specific table would couple us to Terminus's
# schema. So this is left to Kubernetes' restart, deliberately —
# a single sidekiq restart shortly after a rollout is benign. More
# than a couple, or restarts long after a rollout, are NOT this.
env:
# No APP_SETUP here — see the web container.
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: tr-pg-app
key: uri
- name: APP_SECRET
valueFrom:
secretKeyRef:
name: terminus-secrets
key: APP_SECRET
volumeMounts:
- name: data
mountPath: /app/public/uploads
subPath: uploads
- name: data
mountPath: /app/public/fonts
subPath: fonts
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
volumes:
- name: data
persistentVolumeClaim:
claimName: terminus-data
---
apiVersion: v1
kind: Service
metadata:
name: terminus
namespace: trmnl
labels:
app.kubernetes.io/name: terminus
spec:
selector:
app.kubernetes.io/name: terminus
ports:
- name: http
port: 80
targetPort: http
+79
View File
@@ -0,0 +1,79 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: terminus-valkey
namespace: trmnl
labels:
app.kubernetes.io/name: terminus-valkey
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: terminus-valkey
template:
metadata:
labels:
app.kubernetes.io/name: terminus-valkey
spec:
securityContext:
runAsNonRoot: true
runAsUser: 999
fsGroup: 999
seccompProfile:
type: RuntimeDefault
containers:
# Valkey major version tracks upstream's compose.yml (9.x). Sidekiq
# is the only consumer; a major bump is a separate migration.
- name: valkey
image: valkey/valkey:9.1.1-alpine # {"$imagepolicy": "flux-system:valkey-trmnl"}
args:
- valkey-server
- --save
- "60"
- "1"
- --loglevel
- warning
ports:
- name: valkey
containerPort: 6379
volumeMounts:
- name: data
mountPath: /data
livenessProbe:
tcpSocket:
port: valkey
initialDelaySeconds: 10
readinessProbe:
exec:
command: ["valkey-cli", "ping"]
initialDelaySeconds: 5
periodSeconds: 10
resources:
requests:
cpu: 20m
memory: 64Mi
limits:
memory: 256Mi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
volumes:
- name: data
persistentVolumeClaim:
claimName: terminus-valkey-data
---
apiVersion: v1
kind: Service
metadata:
name: terminus-valkey
namespace: trmnl
spec:
selector:
app.kubernetes.io/name: terminus-valkey
ports:
- port: 6379
targetPort: valkey
+1 -1
View File
@@ -18,7 +18,7 @@ spec:
spec:
containers:
- name: whisper
image: rhasspy/wyoming-whisper:3.5.0 # {"$imagepolicy": "flux-system:wyoming-whisper"}
image: rhasspy/wyoming-whisper:3.6.0 # {"$imagepolicy": "flux-system:wyoming-whisper"}
args:
- --model=base-int8
- --language=de
+113
View File
@@ -0,0 +1,113 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress
namespace: wordpress
annotations:
secret.reloader.stakater.com/reload: wordpress-secrets
spec:
replicas: 1
strategy:
# RWO volume, single writer — the rolling default would deadlock on the
# new pod waiting for a volume the old pod still holds.
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: wordpress
template:
metadata:
labels:
app.kubernetes.io/name: wordpress
spec:
securityContext:
# wp-content must be writable by Apache (www-data, uid/gid 33) for
# media uploads and plugin installs.
fsGroup: 33
containers:
- name: wordpress
image: wordpress:7.0.4-php8.3-apache
ports:
- name: http
containerPort: 80
env:
# The operator names the base Service after the MariaDB resource;
# the CRD's separate primaryService/secondaryService fields are the
# suffixed variants for replicated setups, which this is not
# (replicas: 1). Not verified against a running cluster — if it is
# wrong the pod fails loudly on connect at startup rather than
# degrading quietly. Confirm with `kubectl get svc -n wordpress`
# on first deploy.
- name: WORDPRESS_DB_HOST
value: wordpress-mariadb.wordpress.svc.cluster.local:3306
- name: WORDPRESS_DB_NAME
value: wordpress
- name: WORDPRESS_DB_USER
value: wordpress
- name: WORDPRESS_DB_PASSWORD
valueFrom:
secretKeyRef:
# Generated by the operator from the MariaDB CR — never
# stored in Git or OpenBao.
name: wordpress-mariadb-app
key: password
- name: WORDPRESS_CONFIG_EXTRA
value: |
define('WP_HOME', 'https://www.renner.uno');
define('WP_SITEURL', 'https://www.renner.uno');
define('FORCE_SSL_ADMIN', true);
/* Traefik terminates TLS; without this WordPress sees plain
HTTP and builds http:// URLs into every page and redirect. */
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$_SERVER['HTTPS'] = 'on';
}
/* Core lives in the image, not on the volume — the updater
would only fail against a read-only docroot. Plugin and
theme installs are unaffected. */
define('WP_AUTO_UPDATE_CORE', false);
/* xmlrpc.php is blocked at the ingress; this closes the same
door from the inside. Nothing here uses it: no Jetpack,
no mobile app, no pingbacks worth keeping. */
add_filter('xmlrpc_enabled', '__return_false');
envFrom:
# The eight WordPress salts. Rotating them in OpenBao invalidates
# every session — which is the intended lever, not a side effect.
- secretRef:
name: wordpress-secrets
volumeMounts:
- name: content
mountPath: /var/www/html/wp-content
readinessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 60
periodSeconds: 30
resources:
requests:
cpu: 50m
memory: 256Mi
limits:
memory: 768Mi
volumes:
- name: content
persistentVolumeClaim:
claimName: wordpress-content
---
apiVersion: v1
kind: Service
metadata:
name: wordpress
namespace: wordpress
spec:
selector:
app.kubernetes.io/name: wordpress
ports:
- name: http
port: 80
targetPort: http
@@ -0,0 +1,46 @@
# The eight WordPress salts (AUTH_KEY, SECURE_AUTH_KEY, LOGGED_IN_KEY,
# NONCE_KEY, AUTH_SALT, SECURE_AUTH_SALT, LOGGED_IN_SALT, NONCE_SALT).
# Expected keys in OpenBao at homelab/apps/wordpress, named exactly as the
# image's env vars: WORDPRESS_AUTH_KEY, WORDPRESS_SECURE_AUTH_KEY, ...
#
# Deliberately NOT carried over from the old wp-config.php: the Hetzner host
# is being decommissioned and its salts have been sitting on a VM whose
# neighbour site was compromised this month. New salts cost one logout.
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: wordpress-secrets
namespace: wordpress
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: wordpress-secrets
dataFrom:
- extract:
key: apps/wordpress
---
# S3 credentials for the MariaDB Backup.
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: hetzner-s3-creds
namespace: wordpress
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: hetzner-s3-creds
dataFrom:
# Reuses the paperless copy of the shared Hetzner S3 credentials (identical
# ACCESS_KEY_ID / ACCESS_SECRET_KEY / REGION) to avoid seeding a new OpenBao
# path for this migration — same approach woodpecker took. FOLLOW-UP: give
# wordpress its own seed/wordpress/hetzner-creds path once OpenBao can be
# written, and repoint this here (the value is the same, so it's a drop-in
# swap).
- extract:
key: seed/paperless/hetzner-creds
@@ -0,0 +1,15 @@
# A plain Ingress cannot point at a Service in another namespace, and the
# Authentik outposts all live in `authentik`. Same workaround as
# zone-configurator and paperless: mirror the outpost Service in as an
# ExternalName.
apiVersion: v1
kind: Service
metadata:
name: ak-outpost-wordpress-gate
namespace: wordpress
spec:
type: ExternalName
externalName: ak-outpost-wordpress-gate.authentik.svc.cluster.local
ports:
- port: 9000
targetPort: 9000
+78
View File
@@ -0,0 +1,78 @@
# Two routers on one host. The site itself must stay readable by anyone --
# it is a family blog -- while the editing surface sits behind Authentik.
#
# Why not one gated Ingress with skip_path_regex, the way paperless-gate
# carves out /static/: that works when the PUBLIC paths are enumerable. Here
# it is the reverse -- every post URL is public and only two paths are gated
# -- which would need a negative lookahead. The outpost is Go, and RE2 has no
# lookahead. So the split happens at the router instead.
#
# Traefik orders routers by rule length, so PathPrefix(`/wp-admin`) is
# evaluated before PathPrefix(`/`); no explicit priority is needed.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: wordpress
namespace: wordpress
labels:
dns.renner.uno/public: "true"
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
spec:
ingressClassName: traefik
rules:
- host: www.renner.uno
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: wordpress
port:
number: 80
tls:
- secretName: wordpress-tls
hosts:
- www.renner.uno
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: wordpress-admin
namespace: wordpress
labels:
# Same label as the public Ingress on purpose. external-dns-unifi runs
# with --label-filter=dns.renner.uno/public!=true, so an unlabelled
# Ingress on this host would quietly gain a UDM A-record pointing LAN
# clients straight at the Traefik VIP -- a second, unintended path to
# the same host that skips the public route entirely.
dns.renner.uno/public: "true"
# No cert-manager annotation and no tls block: the public Ingress above
# already owns the certificate for this host. Declaring it twice would
# create two Certificates racing for the same wordpress-tls Secret.
spec:
ingressClassName: traefik
rules:
- host: www.renner.uno
http:
paths:
# Covers admin-ajax.php too. That is only safe because the gift
# registry plugin -- the sole thing on this site that ever emitted
# an ajaxurl -- is being dropped in the migration. Verified against
# the live site: 121 admin-ajax hits in the current log window, not
# one of them a 200.
- path: /wp-admin
pathType: Prefix
backend:
service:
name: ak-outpost-wordpress-gate
port:
number: 9000
- path: /wp-login.php
pathType: Exact
backend:
service:
name: ak-outpost-wordpress-gate
port:
number: 9000
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- externalsecret.yaml
- mariadb.yaml
- pvc.yaml
- deployment.yaml
- gate-service.yaml
- ingress.yaml
+86
View File
@@ -0,0 +1,86 @@
# The database for www.renner.uno. Named `wordpress-mariadb`, not `wp-*`:
# `wp-pg` in the woodpecker namespace is Woodpecker's Postgres, and reusing
# the prefix for an unrelated engine invites exactly the wrong guess.
#
# Mirrors the CNPG convention: the operator lives in `database`, the database
# resource lives with its consumer (ak-pg in authentik, im-pg in immich).
apiVersion: k8s.mariadb.com/v1alpha1
kind: MariaDB
metadata:
name: wordpress-mariadb
namespace: wordpress
annotations:
# Data guard — a Kustomization rebuild must never take the database with it.
kustomize.toolkit.fluxcd.io/prune: disabled
spec:
# generate: true lets the operator create both Secrets on first reconcile.
# Neither password is ever typed by a human or parked in OpenBao, so there
# is nothing to rotate by hand and nothing to leak through Git.
rootPasswordSecretKeyRef:
name: wordpress-mariadb-root
key: password
generate: true
username: wordpress
passwordSecretKeyRef:
name: wordpress-mariadb-app
key: password
generate: true
database: wordpress
port: 3306
replicas: 1
storage:
size: 5Gi
storageClassName: ceph-block
# The source database is 527 KB. 5Gi is not a sizing estimate, it is the
# smallest allocation that leaves room for binlogs and a decade of posts.
myCnf: |
[mariadb]
bind-address=*
default_storage_engine=InnoDB
innodb_buffer_pool_size=256M
# WordPress writes utf8mb4 throughout; the source dump carries
# utf8mb4_unicode_520_ci on every table that survives the migration.
character_set_server=utf8mb4
collation_server=utf8mb4_unicode_520_ci
max_allowed_packet=64M
resources:
requests:
cpu: 100m
memory: 512Mi
limits:
memory: 1Gi
---
# Logical backup to Hetzner Object Storage. Velero only captures PVC data,
# which for a running database is crash-consistent, not transaction-consistent
# — that is a snapshot, not a backup you would want to restore from.
apiVersion: k8s.mariadb.com/v1alpha1
kind: Backup
metadata:
name: wordpress-mariadb-backup
namespace: wordpress
spec:
mariaDbRef:
name: wordpress-mariadb
schedule:
# 02:45, ahead of the Velero window that starts at 03:15.
cron: "45 2 * * *"
suspend: false
maxRetention: 720h # 30d, matching the CNPG ObjectStore retentionPolicy
compression: gzip
storage:
s3:
bucket: renner-uno-k8s-mariadb
prefix: wordpress
endpoint: fsn1.your-objectstorage.com
region: fsn1
accessKeyIdSecretKeyRef:
name: hetzner-s3-creds
key: ACCESS_KEY_ID
secretAccessKeySecretKeyRef:
name: hetzner-s3-creds
key: ACCESS_SECRET_KEY
tls:
enabled: true
+35
View File
@@ -0,0 +1,35 @@
# Only wp-content is persisted, not the whole docroot. WordPress core comes
# from the pinned image, so upgrading the site means bumping a tag in Git
# rather than clicking "update" in wp-admin — which is the entire point of
# moving off the Hetzner VM, where core was a mutable pile of PHP on disk.
#
# Consequence to be aware of: WordPress's own core updater will fail against
# a read-only core. Plugin and theme installs still work, because those land
# in wp-content.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wordpress-content
namespace: wordpress
labels:
# The Velero Schedule selects ON this label rather than excluding the
# database PVC by one the mariadb-operator is merely assumed to set. If
# that assumption were wrong the exclusion would fail silently and Velero
# would snapshot the live database volume; selecting a label this repo
# owns cannot fail that way. Same direction as openbao-daily, which picks
# its snapshot PVC by app.kubernetes.io/component.
app.kubernetes.io/name: wordpress
app.kubernetes.io/component: content
annotations:
kustomize.toolkit.fluxcd.io/prune: disabled
spec:
accessModes:
- ReadWriteOnce
# RWO is sufficient: a single replica with strategy Recreate. ceph-block is
# the cluster default; named explicitly so a default change can't silently
# move the media library onto another class.
storageClassName: ceph-block
resources:
requests:
# The migrated wp-content is 29 MB compressed, 13 MB of it uploads.
storage: 10Gi
@@ -48,6 +48,10 @@ spec:
kind: HelmRelease
name: cloudnative-pg # DB operator — seed jobs create databases
namespace: database
- apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
name: mariadb-operator # the MySQL half — WordPress can't use CNPG
namespace: database
# decryption block dropped 2026-07-13: no *.sops.yaml left under this
# path — secrets come from OpenBao via ESO (MIGRATION-sops-openbao.md).
---
+28
View File
@@ -0,0 +1,28 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: trmnl
namespace: flux-system
spec:
# Terminus (TRMNL BYOS server) plus its tr-pg CNPG Cluster + ObjectStore +
# ScheduledBackup, the Valkey queue, and the uploads/fonts PVC. Drives the
# entrance-way e-ink panel; the device polls the cleartext LoadBalancer.
interval: 10m
retryInterval: 1m
timeout: 30m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/trmnl
prune: true
wait: true
dependsOn:
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
- name: infrastructure-cnpg-barman
healthCheckExprs:
- apiVersion: postgresql.cnpg.io/v1
kind: Cluster
failed: "has(status.phase) && status.phase.startsWith('Failed')"
current: "has(status.phase) && (status.phase == 'Cluster in healthy state' || status.readyInstances > 0)"
@@ -0,0 +1,29 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: wordpress
namespace: flux-system
spec:
interval: 10m
retryInterval: 2m
timeout: 15m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/wordpress
prune: true
dependsOn:
# infrastructure gates on the mariadb-operator HelmRelease being Ready,
# so the MariaDB CRD exists and has a controller before this applies.
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
# kstatus has no built-in rule for k8s.mariadb.com resources, so without
# this the Kustomization would report Ready the moment the manifests are
# accepted — before the database is actually serving. Same reasoning as
# the CephCluster expressions in infrastructure.yaml.
healthCheckExprs:
- apiVersion: k8s.mariadb.com/v1alpha1
kind: MariaDB
failed: "has(status.conditions) && status.conditions.exists(c, c.type == 'Ready' && c.status == 'False' && c.reason == 'Failed')"
current: "has(status.conditions) && status.conditions.exists(c, c.type == 'Ready' && c.status == 'True')"
@@ -96,6 +96,29 @@ The security-scan agent will continue to surface vuln findings on
these as "Skipped / Flux image-automation owns this" even without
a policy.
- `ghcr.io/usetrmnl/terminus` — publishes a usable `0.x.0` tag stream, so
unlike the three above this is **not** a tag-format problem. It is left
unenrolled because the blast radius is hardware: Terminus is pre-1.0,
cuts no GitHub Releases and ships no changelog, and it serves the
firmware and `/api/display` contract that the TRMNL X panel polls. An
unattended bump that changes either can leave a wall-mounted device
dark, with no rollback path that doesn't involve the captive portal.
Bumps are a reviewed, manual act: read the commit range, then edit the
tag in `kubernetes/apps/trmnl/terminus.yaml` (both containers).
The Valkey sidecar alongside it *is* enrolled (`valkey.yaml`) — it holds
only the Sidekiq queue and touches no device-facing contract.
- `ghcr.io/usetrmnl/trmnl-ha-amd64` — the TRMNL screenshot renderer, for
the same reason and one more. It owns the dithering pipeline and the
webhook payload shape, so a bad bump surfaces as a *corrupted or blank
panel*, not as a failed rollout — there is no readiness signal that
catches "the image is now unreadable". Separately, upstream publishes
**no multi-arch manifest**: only the arch-suffixed `trmnl-ha-amd64` and
`trmnl-ha-aarch64` repositories exist, so an ImagePolicy here would also
be pinning us to one architecture in a way the tag does not make
obvious. Bump manually in `kubernetes/apps/trmnl/renderer.yaml`.
## Re-import hazard: cnpg-barman-plugin
`kubernetes/infrastructure/cnpg-barman-plugin/manifest.yaml` is an
@@ -20,6 +20,7 @@ resources:
- redis.yaml
- shelly-manager.yaml
- tika.yaml
- valkey.yaml
- velero-plugin-for-aws.yaml
- vsftpd.yaml
- woodpecker.yaml
@@ -0,0 +1,27 @@
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
name: valkey-trmnl
namespace: flux-system
spec:
image: valkey/valkey
interval: 1h
---
# Alpine tags only, semver 9.x — matches the major version pinned in
# upstream Terminus's compose.yml. Valkey holds nothing but the Sidekiq
# queue here, so minor bumps are safe to take unattended; a 10.x cutover
# is a deliberate migration.
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
name: valkey-trmnl
namespace: flux-system
spec:
imageRepositoryRef:
name: valkey-trmnl
filterTags:
pattern: '^(?P<ver>\d+\.\d+\.\d+)-alpine$'
extract: '$ver'
policy:
semver:
range: "9.x"
@@ -15,6 +15,7 @@ resources:
- rook-ceph/
- ceph-csi-drivers/
- cloudnative-pg/
- mariadb-operator/
- netbird-ingress/
- netbird-backup/
- coredns-config/
@@ -0,0 +1,81 @@
# MariaDB operator, for the one workload in this cluster that cannot use
# CloudNativePG: WordPress. WordPress speaks MySQL natively — the PostgreSQL
# route would mean pg4wp, an SQL-rewriting shim whose own README lists
# WordPress 6.5.3 as its newest tested version while renner.uno runs 7.0.4.
#
# The charts are OCI artifacts (ghcr.io), so this follows the OCIRepository
# pattern from infrastructure/spegel rather than the HelmRepository pattern
# used for CNPG.
#
# CRDs ship as a SEPARATE chart that must be installed first — upstream is
# explicit about the ordering, hence the dependsOn below rather than a single
# release with crds: CreateReplace.
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
name: mariadb-operator-crds
namespace: database
spec:
interval: 24h
url: oci://ghcr.io/mariadb-operator/charts/mariadb-operator-crds
ref:
# Calendar versioning: the major is the year. Upstream ships an upgrade
# guide with each minor (Helm repo deprecations, registry moves), so the
# range is capped at the year boundary instead of the usual open ">=".
semver: ">=26.6.0 <27.0.0"
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: mariadb-operator-crds
namespace: database
spec:
interval: 30m
chartRef:
kind: OCIRepository
name: mariadb-operator-crds
namespace: database
install:
remediation:
retries: 3
upgrade:
remediation:
retries: 3
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
name: mariadb-operator
namespace: database
spec:
interval: 24h
url: oci://ghcr.io/mariadb-operator/charts/mariadb-operator
ref:
semver: ">=26.6.0 <27.0.0"
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: mariadb-operator
namespace: database
spec:
interval: 30m
dependsOn:
- name: mariadb-operator-crds
namespace: database
chartRef:
kind: OCIRepository
name: mariadb-operator
namespace: database
install:
remediation:
retries: 3
upgrade:
remediation:
retries: 3
values:
# The operator watches all namespaces; the MariaDB CR itself lives with
# the app in kubernetes/apps/wordpress/, mirroring how CNPG Clusters sit
# in their consumer's namespace (ak-pg in authentik, im-pg in immich).
metrics:
enabled: false
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- helmrelease.yaml
@@ -145,3 +145,18 @@ apiVersion: v1
kind: Namespace
metadata:
name: external-secrets
---
# Terminus (TRMNL BYOS server) driving the entrance-way e-ink panel.
apiVersion: v1
kind: Namespace
metadata:
name: trmnl
---
# WordPress for www.renner.uno, migrated off the Hetzner VM `wordpress-hochzeit`.
# No privileged label needed: the cluster enforces `baseline`, which permits the
# official Apache image's root master process binding :80. It does trip the
# `restricted` audit/warn level — that is expected and not an error.
apiVersion: v1
kind: Namespace
metadata:
name: wordpress
@@ -371,3 +371,36 @@ spec:
- ports:
- port: "8123"
protocol: TCP
---
# The TRMNL e-ink renderer (trmnl ns) drives a headless browser against the
# entrance dashboard, so it needs the HA frontend on :8123 — same in-cluster
# path as the rotator above, and for the same reason: the public names
# hairpin off the home-assistant-bgp LoadBalancer and are unusable from a
# pod. It reaches HA at home-assistant-bgp.home-assistant.svc.cluster.local.
#
# Only the ingress half is needed here. Unlike the openbao rotator, the trmnl
# namespace has no default-deny EGRESS covering this pod — terminus-egress is
# scoped to the Terminus pods by label and does not select the renderer — so
# there is no source-side rule to add.
#
# Source scoped to the renderer pod specifically, not the whole trmnl
# namespace: Terminus itself sits in that namespace and has no business
# reaching Home Assistant.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: home-assistant-trmnl-renderer-ingress
namespace: home-assistant
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: home-assistant
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": trmnl
app.kubernetes.io/name: trmnl-renderer
toPorts:
- ports:
- port: "8123"
protocol: TCP
@@ -21,4 +21,6 @@ resources:
- openbao-ns.yaml
- external-secrets-ns.yaml
- woodpecker-ci-ns.yaml
- trmnl-ns.yaml
- wordpress-ns.yaml
- allow-cilium-health.yaml
@@ -0,0 +1,177 @@
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: default-deny-ingress
namespace: trmnl
spec:
endpointSelector: {}
ingress:
# Intra-namespace — terminus ↔ sidekiq ↔ valkey ↔ tr-pg.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": trmnl
# Admin UI via Traefik (trmnl.renner.uno). The device does NOT come
# this way — see allow-device-http below.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
# Prometheus scrape.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": monitoring
# Velero backups via PVC mount.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": velero
# CNPG operator (in `database` ns) reconciles tr-pg — needs to reach
# the postgres pods for health, failover, role checks.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": database
- fromEntities:
- host
- remote-node
# ICMP — all standard ICMPv4 + ICMPv6 from any cluster pod. See
# home-assistant-ns.yaml for the rationale. INGRESS ONLY.
- fromEndpoints:
- matchExpressions:
- key: io.kubernetes.pod.namespace
operator: Exists
icmps:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
---
# The TRMNL panel polls /api/display over cleartext HTTP on the
# terminus-device LoadBalancer. The firmware cannot be handed the internal
# CA, so this bypasses Traefik entirely and the namespace default-deny has
# to be opened for it — same shape as paperless's allow-ftp-scanner.
#
# Scoped to the terminus pods and port 2345 only. Requests still have to
# carry a known device MAC in the `ID` header plus its api_key; this rule
# opens the port, not the application.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: allow-device-http
namespace: trmnl
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: terminus
ingress:
- fromEntities:
- world
- host
- remote-node
toPorts:
- ports:
- port: "2345"
protocol: TCP
---
# Egress allowlist for the Terminus pod only.
#
# Deliberately NOT applied to the tr-pg CNPG pods: selecting them here
# would make their egress restricted too, and the barman-cloud WAL archive
# path to Hetzner object storage is not worth putting behind an FQDN rule
# that fails closed. They keep unrestricted egress, matching immich/authentik.
#
# Terminus needs exactly three outbound destinations, all HTTPS:
# - trmnl.com → GET /api/models, /api/palettes (MODEL_SYNCHRONIZER,
# which is what seeds the TRMNL X model definition —
# 1872x1404, 16 colours — into the Models list),
# GET /api/firmware/latest (FIRMWARE_SYNCHRONIZER),
# GET /fonts/<name> (FONT_SYNCHRONIZER)
# - trmnl-fw.s3.us-east-2.amazonaws.com
# → the actual firmware binary. This host is NOT a
# constant in the source: it is the `url` field of
# the /api/firmware/latest response, so TRMNL can
# move bucket/region without a Terminus release and
# firmware sync would start failing closed here.
# If firmware sync breaks, re-check that response
# before assuming the policy is wrong.
#
# KNOWN LIMITATION: Terminus's "Extensions" (private plugin) feature polls
# arbitrary admin-configured URLs rendered from a Liquid template stored in
# the database. Those are unknowable from source and WILL be blocked by this
# allowlist. We do not use Extensions — screen content is pushed in from the
# Home Assistant renderer — so this is an accepted trade, not an oversight.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: terminus-egress
namespace: trmnl
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: terminus
egress:
# Intra-namespace — tr-pg-rw (postgres) and terminus-valkey.
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": trmnl
# DNS. The L7 `rules.dns` block is mandatory, not decorative: Cilium's
# DNS proxy only learns an FQDN→IP mapping for names it has actually
# observed being resolved, so without this the toFQDNs rules below
# never match anything.
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
rules:
dns:
- matchPattern: "*"
- toFQDNs:
- matchName: "trmnl.com"
- matchName: "trmnl-fw.s3.us-east-2.amazonaws.com"
toPorts:
- ports:
- port: "443"
protocol: TCP
- toEntities:
- host
- remote-node
@@ -0,0 +1,167 @@
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: default-deny-ingress
namespace: wordpress
spec:
endpointSelector: {}
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": wordpress
# Unlike paperless, traefik IS allowed here. www.renner.uno is a public
# blog: the front end must be reachable without authentication, so the
# public router terminates directly in this namespace.
#
# Be clear about what this does and does not buy: the public/admin split
# is enforced by the two Ingress objects' path rules (`/` straight to the
# Service, /wp-admin and /wp-login.php to the gate outpost) and by
# Authentik — NOT at the network layer. paperless can point a
# workload-scoped CNP at its gate because nothing else may reach it;
# here traefik must reach the pod directly for every public page, so the
# equivalent rule would be a no-op. Anyone auditing this namespace for
# defence in depth should know the network layer is not where the admin
# surface is protected.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
# Scoped to the gate outpost rather than the whole authentik namespace:
# only that pod has business reaching wp-admin.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": authentik
goauthentik.io/outpost-name: wordpress-gate
# The mariadb-operator reconciles the MariaDB CR from the database
# namespace — same cross-namespace requirement CNPG consumers have.
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": database
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": monitoring
- fromEntities:
- host
- remote-node
# ICMP — all standard ICMPv4 + ICMPv6 from any cluster pod. See
# home-assistant-ns.yaml for the rationale.
- fromEndpoints:
- matchExpressions:
- key: io.kubernetes.pod.namespace
operator: Exists
icmps:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": wordpress
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": authentik
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": rook-ceph
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
# world covers two things WordPress genuinely needs outbound: the
# wordpress.org update/plugin API, and the Hetzner Object Storage
# endpoint the MariaDB Backup writes to.
- toEntities:
- host
- remote-node
- world
# ICMP — egress mirror.
- toEndpoints:
- matchExpressions:
- key: io.kubernetes.pod.namespace
operator: Exists
icmps:
- fields:
- { type: 0, family: IPv4 }
- { type: 3, family: IPv4 }
- { type: 4, family: IPv4 }
- { type: 5, family: IPv4 }
- { type: 8, family: IPv4 }
- { type: 9, family: IPv4 }
- { type: 10, family: IPv4 }
- { type: 11, family: IPv4 }
- { type: 12, family: IPv4 }
- { type: 13, family: IPv4 }
- { type: 14, family: IPv4 }
- { type: 15, family: IPv4 }
- { type: 16, family: IPv4 }
- { type: 17, family: IPv4 }
- { type: 18, family: IPv4 }
- { type: 30, family: IPv4 }
- { type: 1, family: IPv6 }
- { type: 2, family: IPv6 }
- { type: 3, family: IPv6 }
- { type: 4, family: IPv6 }
- { type: 128, family: IPv6 }
- { type: 129, family: IPv6 }
- { type: 130, family: IPv6 }
- { type: 131, family: IPv6 }
- { type: 132, family: IPv6 }
- { type: 133, family: IPv6 }
- { type: 134, family: IPv6 }
- { type: 135, family: IPv6 }
- { type: 136, family: IPv6 }
- { type: 137, family: IPv6 }
- { type: 138, family: IPv6 }
- { type: 141, family: IPv6 }
- { type: 142, family: IPv6 }
- { type: 143, family: IPv6 }
- { type: 148, family: IPv6 }
- { type: 149, family: IPv6 }
- { type: 151, family: IPv6 }
- { type: 152, family: IPv6 }
- { type: 153, family: IPv6 }
@@ -59,3 +59,22 @@ resources:
- review-bot.yaml
- claude-oauth.yaml
- mosquitto-health.yaml
# NOT ROTATED, deliberately — recorded here because the whole point of this
# directory is that unrotated credentials should be a visible choice:
#
# homelab/apps/trmnl/renderer → BYOS_ACCESS_TOKEN / BYOS_REFRESH_TOKEN
# Terminus API JWTs used by the e-ink renderer to push screens. Terminus
# is configured with API_ACCESS_TOKEN_PERIOD = 10 years
# (kubernetes/apps/trmnl/terminus.yaml), so nothing will ever expire
# these or prompt a rotation. That is deliberate: the alternative was
# holding the Terminus account password in-cluster to mint short-lived
# tokens per pod start. There is no rotator because minting requires the
# account password, which is exactly what we declined to store.
# Rotate BY HAND whenever the Terminus account password changes, by
# re-running POST /login and re-seeding both keys.
#
# HA_ACCESS_TOKEN in the same path is a dedicated Home Assistant
# long-lived token. `ha-token` mode in secret-rotator already handles
# this class (see home-assistant.yaml) — enrolling it is a genuine
# follow-up, not a limitation, and would leave only the BYOS pair
# unrotated.
@@ -28,7 +28,9 @@ spec:
- monitoring
- netbird-backup # netbird-backup-ssh: the VPS-pull backup key
- paperless
- trmnl
- velero
- wordpress
provider:
vault: # ESO's provider name; speaks OpenBao fine
server: http://openbao.openbao.svc.cluster.local:8200
@@ -283,3 +283,67 @@ spec:
storageLocation: default
ttl: 720h0m0s
snapshotMoveData: true
---
apiVersion: velero.io/v1
kind: Schedule
metadata:
name: trmnl-daily
namespace: velero
labels:
app.kubernetes.io/part-of: homelab-backup
spec:
# Daily at 08:15 local (next free stagger slot after netbird-backup-daily
# at 07:45). Selects the whole namespace minus the CNPG-labelled PVCs, so
# it picks up terminus-data (device screens + synchronised fonts),
# terminus-valkey-data and trmnl-renderer-data (the renderer's schedule
# and rendered output) without needing to be edited per PVC. The tr-pg
# CNPG cluster is covered separately by Barman, hence the exclusion.
schedule: "15 8 * * *"
useOwnerReferencesInBackup: false
template:
includedNamespaces:
- trmnl
includedResources:
- persistentvolumeclaims
storageLocation: default
ttl: 720h0m0s
snapshotMoveData: true
labelSelector:
matchExpressions:
- key: cnpg.io/cluster
operator: DoesNotExist
---
apiVersion: velero.io/v1
kind: Schedule
metadata:
name: wordpress-daily
namespace: velero
labels:
app.kubernetes.io/part-of: homelab-backup
spec:
# Daily at 08:45 local (next free stagger slot after trmnl-daily at 08:15).
# Covers the wordpress-content PVC: the media library and any plugins or
# themes installed through wp-admin. WordPress core is not in here — it
# comes from the pinned image, so Git is its backup.
schedule: "45 8 * * *"
useOwnerReferencesInBackup: false
template:
includedNamespaces:
- wordpress
includedResources:
- persistentvolumeclaims
storageLocation: default
ttl: 720h0m0s
snapshotMoveData: true
# Selects the content PVC by a label this repo sets, rather than
# excluding the database PVC by one the mariadb-operator is assumed to
# set — an exclusion built on a wrong assumption fails silently and
# sweeps the live database volume in. A crash-consistent snapshot of a
# running database is not a restorable backup; the operator's own Backup
# resource takes a logical dump to Hetzner Object Storage at 02:45, the
# same division of labour as CNPG/Barman elsewhere in this file.
# openbao-daily picks its PVC the same way.
labelSelector:
matchLabels:
app.kubernetes.io/name: wordpress
app.kubernetes.io/component: content
+4 -2
View File
@@ -42,6 +42,7 @@ locals {
"renner-uno-k8s-velero", # Velero PVC backups (Kopia data mover)
"renner-uno-k8s-secrets", # SOPS talosconfig/kubeconfig for cluster bootstrap
"renner-uno-tfstate", # OpenTofu remote state (migrated off the nas-1 Garage S3)
"renner-uno-k8s-mariadb", # mariadb-operator logical dumps (wordpress-mariadb)
]
}
@@ -95,8 +96,9 @@ locals {
# precedent in this repo — left uncapped (keep every version) since
# storage cost there is negligible.
hetzner_buckets_noncurrent_expiry_days = {
"renner-uno-k8s-cnpg" = 30
"renner-uno-k8s-velero" = 30
"renner-uno-k8s-cnpg" = 30
"renner-uno-k8s-velero" = 30
"renner-uno-k8s-mariadb" = 30 # Backup maxRetention: 720h (kubernetes/apps/wordpress/mariadb.yaml)
}
}