mosquitto: auto-update the broker image (image-automation) + guard the rollout #130

2026-07-08T09:11:21Z

k.renner commented

2026-07-08 09:11:21 +00:00

Keeps the MQTT broker current without hand-pinning, and makes a bad update loud + non-disruptive.

Auto-update — adds eclipse-mosquitto to the repo's Flux image-automation (like redis et al.): ImageRepository + ImagePolicy on semver range 2.x (auto-bump minor+patch; a 3.0 major stays a deliberate migration), plain-semver tags only. The deployment carries the pinned tag + $imagepolicy marker, so each bump lands as a git commit. Net: pinned/reproducible and auto-patched.

Guard the rollout — image-automation commits straight to master with no CI gate and the broker is a single pod, so a bad bump would be silent + disruptive:

gatus TCP-checks the broker ClusterIP on :1883 → a pod that fails to come up alerts within one interval. Reachability via a scoped egress (apps ns) + ingress (home-assistant ns, gatus pod on :1883 only), mirroring the existing evcc→mosquitto:1883 allow. (In-cluster :1883 is the broker's primary listener; the ClusterIP only exposes 1883. Plaintext deprecation applies only to the external IoT-VLAN LB path.)
strategy Recreate → RollingUpdate (maxUnavailable:0, maxSurge:1). With the PVC gone there is no RWO volume to fence, so the new broker comes up + passes readiness before the old one is torn down — no full-down window on a bump/reschedule; clients on the old pod just reconnect. The brief two-pod overlap is two independent stateless brokers behind the LB, fine with no shared state + self-healing discovery.

Keeps the MQTT broker current without hand-pinning, and makes a bad update loud + non-disruptive. **Auto-update** — adds `eclipse-mosquitto` to the repo's Flux image-automation (like redis et al.): `ImageRepository` + `ImagePolicy` on semver range `2.x` (auto-bump minor+patch; a 3.0 major stays a deliberate migration), plain-semver tags only. The deployment carries the pinned tag + `$imagepolicy` marker, so each bump lands as a git commit. Net: pinned/reproducible **and** auto-patched. **Guard the rollout** — image-automation commits straight to master with no CI gate and the broker is a single pod, so a bad bump would be silent + disruptive: - gatus TCP-checks the broker ClusterIP on `:1883` → a pod that fails to come up alerts within one interval. Reachability via a scoped egress (apps ns) + ingress (home-assistant ns, gatus pod on :1883 only), mirroring the existing evcc→mosquitto:1883 allow. (In-cluster :1883 is the broker's primary listener; the ClusterIP only exposes 1883. Plaintext deprecation applies only to the external IoT-VLAN LB path.) - `strategy` Recreate → RollingUpdate (`maxUnavailable:0`, `maxSurge:1`). With the PVC gone there is no RWO volume to fence, so the new broker comes up + passes readiness before the old one is torn down — no full-down window on a bump/reschedule; clients on the old pod just reconnect. The brief two-pod overlap is two independent stateless brokers behind the LB, fine with no shared state + self-healing discovery.

k.renner added 1 commit 2026-07-08 09:11:21 +00:00

mosquitto: pin the broker image by digest d1456fdab5

The now-critical, stateless broker ran on a floating `eclipse-mosquitto:2`
tag — a reschedule could pull a changed or broken image, and a Docker Hub
outage could block the pull entirely. Pin to 2.1.2 by digest: immutable
(reproducible restarts) and, cached by spegel across nodes, reschedulable
even if Docker Hub is unreachable. No renovate here, so bump tag+digest
manually to update.

Same digest the pod already runs, so applying is a no-op pull (brief
Recreate; Gen1 discovery self-heals via HA #112).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

k.renner force-pushed feat/mosquitto-pin-image from d1456fdab5 to 2155edd3b4

2026-07-08 09:32:55 +00:00

Compare

k.renner added 1 commit 2026-07-08 09:45:15 +00:00

mosquitto: alert on a bad image bump + roll without a downtime gap d0c8044a67

image-automation commits tag bumps straight to master with no CI gate, and
the broker is a single pod — so an image that fails to come up would be
silent. Two changes make that safe:

- gatus TCP-checks the broker's ClusterIP on :1883, so a pod that never comes
  up alerts within one interval. Reachability needed a scoped egress (apps ns)
  + ingress (home-assistant ns, gatus pod on :1883 only), mirroring the
  existing evcc -> mosquitto:1883 allow.
- strategy Recreate -> RollingUpdate (maxUnavailable:0, maxSurge:1). With the
  PVC gone there is no RWO volume to fence, so the new broker comes up and
  passes its readiness probe before the old one is torn down — no full-down
  window on a bump/reschedule; clients on the old pod just reconnect. The brief
  two-pod overlap is two independent stateless brokers behind the LB, fine with
  no shared state + self-healing discovery.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

k.renner changed title from ~~mosquitto: pin the broker image by digest~~ to mosquitto: auto-update the broker image (image-automation) + guard the rollout

2026-07-08 09:46:54 +00:00

k.renner merged commit 103f3b9284 into master

2026-07-08 09:48:15 +00:00

k.renner referenced this issue from a commit

2026-07-08 09:48:17 +00:00

Merge pull request 'mosquitto: auto-update the broker image (image-automation) + guard the rollout' (#130) from feat/mosquitto-pin-image into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#130