Commit Graph
9 Commits
Author SHA1 Message Date
atlas-botandClaude Opus 5.5 4daee5fcea chore(edge): retire sml2mqtt for good (REN-802)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
The SML power-meter reader was retired by owner decision on 2026-09-19:
grid data comes from the E380 on the CAN bus (wiki
sml-to-e380-grid-meter-migration). Its reader has been unplugged since then.

- register: edge-1 loses write sml2mqtt/# (live, no broker restart)
- bridge templates: no sml2mqtt topic, no legacy sml2mqtt bridge
- remove role power-meter-sml, play install-power-meter.yml, inventory
  group power_meter and the vault indirection
- edge-mqtt-bridge play: pre_tasks stop and remove the leftover compose
  project on edge-1 on the next pipeline run (idempotent)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:04:10 +00:00
atlas-botandClaude Opus 5.5 1b3c73dbe4 feat(edge): bridge credential pushed by mqtt-account-controller (REN-1307)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
edge-1's bridge password moves out of ansible-vault. A small receiver,
edge-mqtt-cred, speaks the controller's esp32 API over HTTPS with mTLS
(only CN mqtt-account-controller from the MQTT CA) and writes one bridge
`edge-1` for open3e and sml2mqtt. Until the first push the two
vault-password bridges stay in bridge.d/legacy.conf, so nothing changes
on the cluster side yet.

A Woodpecker pipeline runs the role-only play on merge, so no manual play
run is needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:02:39 +00:00
claude-botandClaude Opus 5 b76a948e28 chore(ansible): remove monitoring-agent, which shipped to endpoints that are gone
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
common.yml applied monitoring-agent (Alloy) to every inventory host, pointing at
loki.srv.renner.uno:3100 and prometheus.srv.renner.uno:9090. Both moved into the
cluster and neither name resolves any more, so nas-1, claude-ws, netbird and the
proxies have been shipping logs and metrics nowhere.

Repointing them was the obvious fix and the wrong one. The cluster endpoints are
authenticated — loki.renner.uno returns 401 and prometheus.renner.uno sits behind
an Authentik forward-auth gate — while monitoring-agent has no auth support at
all. Making it work would mean per-host credentials, htpasswd entries in the
cluster secret, and some answer for remote-write past the gate. Nobody reads
those metrics: edge-1 is the host that matters, and edge-2 when it exists.

edge hosts are unaffected. install-edge-host.yml is self-contained and
deliberately never carried monitoring-agent; they ship via edge-log-shipper,
which already does this correctly — authenticated push to the cluster ingress
with a per-host credential.

install-netbird-monitoring.yml goes too; applying monitoring-agent to
netbird_server was its only content.

Comments referring to the role are corrected in install-edge-host.yml,
install-claude-workstation.yml, the claude-workstation role and its RUNBOOK.
Three more described the old srv Loki as "firewalled off the IoT VLAN" — it is
not firewalled, it is gone. The reason edge hosts use the authenticated ingress
is unchanged, but the stated reason was no longer true.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 12:09:37 +02:00
claude-botandClaude Opus 5 150a83ee38 fix(edge): make the capability-group invariant structural, and unstale a note
Both from review on #475.

The rule "a host in a capability group must also be in [edge_host]" was
enforced by a comment, in a file that already uses :children a few lines
below for ntp_cluster. The three capability groups are now children of
edge_host, so the relation is structural: a host added only to [heating_can]
still resolves group_vars/edge_host, where the local MQTT contract lives, and
cannot silently come up without edge_mqtt_host and friends.

That matters more than it looks. The MQTT contract only moved to group_vars
in this PR because splitting the plays exposed that the roles could not
render without each other's defaults -- so an edge host outside [edge_host]
would hit exactly the failure that change was made to fix.

Verified all four groups still resolve to edge-1, and the redundant
ansible_user on the capability entries is dropped: it is inherited from the
[edge_host] declaration rather than repeated three times.

Also unstaled a Day-2 note that still read "the pip task tracks the git
default branch (pin edge_open3e_pkg to a tag if that ever bites)". Both
services are pinned to commits now, deliberately, and pinning to a tag is not
available for open3e -- it publishes none. Replaced with the actual workflow:
the weekly update-audit reports drift, bump the pin and re-run.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:16:37 +02:00
claude-botandClaude Opus 5 0b845e7c76 refactor(edge): split the plays by capability, not just the roles
Splitting edge-host into single-concern roles left the coupling one layer
up: install-edge-host.yml was still one play applying all eleven roles to
one group. A host in [edge_host] got the Z-Wave gateway and the CAN services
whether or not it had the hardware -- the monolith again, in the play.

Inventory groups now state which hardware a host physically has, and each
playbook targets the group for its own concern:

  [edge_host]       install-edge-host.yml       base: flash-wear, log
                                                shipper, MQTT bridge, VLAN,
                                                docker
  [heating_can]     install-heating-can.yml     Innomaker USB2CAN
  [zwave_gateway]   install-zwave-gateway.yml   Nabu Casa ZWA-2
  [power_meter]     install-power-meter.yml     FTDI meter head

No flags and no `when:` on the roles. Moving a stick to another box is a
one-line inventory change, which is what it should be. A host with nothing
attached is simply absent from the capability groups -- which is what lets a
second thin client join as an NTP peer without being handed peripherals it
does not have.

The split exposed a circular dependency the single play had been hiding: the
local MQTT contract was scattered across two roles' defaults, in both
directions. mqtt-bridge's config needs edge_open3e_mqtt_user, because the
bridge authenticates to the cluster broker AS that user; heating-can and
power-meter-sml need edge_mqtt_host, because that is where they publish.
Neither role could render without the other's defaults, and it only ever
worked because one play loaded every role at once.

Those four values are the interface between the broker and its clients, not
the property of either side, so they move to group_vars/edge_host -- where
the matching secrets (edge_open3e_mqtt_password, edge_sml2mqtt_mqtt_password)
already lived. Removed from the role defaults so there is one source of
truth. Each role keeps what is genuinely its own: mqtt-bridge the cluster
endpoint and CA path, heating-can the datapoints and poll interval.

Also dropped: the legacy shared-compose teardown pre_tasks. That migration
completed on the 2026-08-08 apply (verified: /opt/edge-host/docker-compose.yml
is gone), and carrying migration cruft in a base play would be the same
mixing this commit is undoing.

Verified by --check --diff against the live host, all four plays:

  install-edge-host      ok=33 changed=0 failed=0
  install-zwave-gateway  ok=8  changed=0 failed=0
  install-power-meter    ok=4  changed=0 failed=0
  install-heating-can    ok=18 changed=4 failed=0

Three of four are fully idempotent. heating-can's four are the known open3e
pip churn plus the E3onCAN venv, which reports changed only under check mode
(it was ok in the real apply) -- and their two restart handlers.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:16:17 +02:00
claude-botandClaude Opus 5 2d9c3ba213 refactor(edge): split the edge-host role into one role per concern
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
edge-host was a single role doing eight unrelated jobs: flash-wear
mitigation, log shipping, VLAN networking, an MQTT bridge, a CAN interface,
the heating CAN services, a Z-Wave gateway and a power-meter reader.

That is not just untidy. It is what blocked a second thin client from
joining as an NTP peer: the host-generic work (journald to RAM, Alloy, the
VLAN sub-interface) could not be applied without also handing the box a
Z-Wave radio and a CAN bus it does not have. The role bound its services to
specific USB device paths, so a second host would have tried to start
containers for hardware that is not there.

Now:

  edge-flash-wear   journald -> RAM, noatime/lazytime, TRIM, eMMC health
  edge-log-shipper  Alloy: journal -> Loki
  edge-vlan         tagged VLAN sub-interface
  mqtt-bridge       loopback mosquitto -> cluster broker over TLS
  socketcan         can0 only
  heating-can       open3e (Vitocal/VX3) + e3oncan (E380 meter)
  zwave-gateway     zwave-js-ui, own compose project
  power-meter-sml   sml2mqtt, own compose project

The E380 reader lives with open3e rather than in its own role: it is part
of the same heating installation -- it feeds the heat pump's own energy
management -- not a separate system that happens to share a bus. can0 does
stay separate, because an interface is not the same concern as the services
consuming it.

Each role installs its own packages, so it stands alone. An edge host with
no peripherals takes the first five and none of the rest.

docker-daemon.json moves to docker-install with its own docker_log_driver
variable and Restart docker handler, since configuring the docker daemon was
never an edge-host concern. Handlers split ten ways; defaults split per
role; all sixteen templates and files moved with git mv.

Two deliberate behaviour changes on the node, both accepted:

1. Compose split. zwave-js and sml2mqtt shared a compose file, so touching
   either recreated both. They are now separate projects in their own
   directories. container_name is fixed, so the old project must release the
   names first -- a guarded one-time task downs it and removes the file.
   First apply costs a brief down/up on both. Data paths are unchanged; the
   zwave store is not migrated.

2. Header repointing. Templates and scripts carried "Managed by
   roles/edge-host" headers, which render onto the host -- they would have
   sent an operator to a role that no longer exists. Updating them changes
   file content, so it restarts the services that consume those files.

Verified by --check --diff against the live host. 24 changed items, and
every one is accounted for:

    5  pre-existing, unchanged from the baseline before this work:
       common timesyncd; open3e venv; E3onCAN venv; and the open3e /
       e3oncan restart handlers. The venv tasks install from a git URL,
       which pip cannot verify, so they report changed on every run --
       a separate defect, not introduced here.
    5  the compose split: zwave env + compose and sml compose to their
       new paths, plus the two legacy-project retirement tasks.
   14  the header repointing: journald conf, eMMC health script, alloy
       config, mosquitto bridge config, 80-can0.network, open3e overlay /
       env / unit, e3oncan unit, zwave backup script + env, sml2mqtt
       config -- and the resulting restarts of journald, alloy, mosquitto,
       open3e and e3oncan.

Nothing changed outside those three groups.

One consequence check mode cannot show: 80-can0.network.j2 changed, so the
"Reload networkd" handler fires on a real run and reconfigures can0,
briefly dropping the bus under open3e and e3oncan. It is absent from the
check-mode output only because shell handlers are skipped there. The VLAN
templates did NOT change, so the riskier reconfigure of the parent uplink
on this headless host is not triggered.

Also fixed in passing: the keepalived-ntp guard told operators to recover
with `--limit <host> --tags vlan40`, and referred to the edge-host role.
Both corrected. The role README becomes ansible/docs/edge-host.md -- it
documents the host, not any single role now.

Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 11:33:57 +02:00
k.rennerandClaude Opus 4.7 50f9a280cc chore: Remove local markdown docs; wiki is single source of truth
ci/woodpecker/push/deploy-infrastructure Pipeline was successful
ci/woodpecker/push/security-scan Pipeline was successful
Documentation now lives in ~/Notes/wiki/ (Obsidian vault). The local
.md files duplicated wiki content and drifted from it. CLAUDE.md stays
(it's Claude Code agent instructions, not documentation) and had its
dangling "Related Documentation" + kubernetes/README.md +
dns-server/HA_DEPLOYMENT.md references stripped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 07:42:43 +02:00
k.rennerandClaude eb0199a255 feat: Improve Alloy syslog receiver for Synology NAS
- Add raw syslog format support for BSD/RFC 3164 style logs
- Parse hostname and app from Synology log messages
- Configure loki.srv.renner.uno as syslog receiver (UDP 1514)
- Add stability_level experimental for raw format support

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-01 19:10:28 +01:00
k.rennerandClaude 6cd44ac5c5 chore: Clean up and reorganize Ansible playbooks
Security Audit / Infrastructure Scan (IaC) (push) Failing after 1m22s
Deploy Infrastructure / plan (push) Successful in 1m26s
Security Audit / Docker Image Scan (docker.io/apache/tika:latest) (push) Failing after 5s
Security Audit / Docker Image Scan (docker.io/library/postgres:15) (push) Failing after 6s
Deploy Infrastructure / apply (push) Successful in 1m7s
Security Audit / Docker Image Scan (docker.io/library/redis:7) (push) Failing after 5s
Security Audit / Docker Image Scan (docker.io/gotenberg/gotenberg:8) (push) Failing after 1m23s
Security Audit / Docker Image Scan (ghcr.io/icereed/paperless-gpt:latest) (push) Failing after 4s
Deploy Infrastructure / integration-tests (push) Successful in 12s
Security Audit / Docker Image Scan (ghcr.io/paperless-ngx/paperless-ngx:latest) (push) Failing after 6s
Security Audit / Docker Image Scan (shenxn/protonmail-bridge:latest) (push) Failing after 6s
Naming Convention Standardization:
- Rename all setup-* playbooks to install-* or configure-*
- Rename all *-update.yml to update-*.yml for prefix consistency
- Fix underscore usage (ensure_template -> check-template)
- Standardize on prefixes: install-, update-, configure-, check-

Consolidations:
- Remove duplicate NetBird playbooks (3 removed)
- Replace update.yml with more complete update_all_software.yml
- Update internal import references in update.yml

Deletions:
- Remove unused certbot-cloudflare role (Caddy handles HTTPS)
- Remove empty vars/empty.yml placeholder

Archive:
- Move 7 completed migration/cleanup playbooks to archived/

Result: 54 -> 43 active playbooks with consistent naming

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-01 17:25:35 +01:00