Compare commits

...
Author SHA1 Message Date
claude-botandClaude Opus 5 c0791e2747 docs(zigbee2mqtt): record the WS90 ZCL reporting configuration
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The Wetterstation now carries a non-default reporting configuration: min
interval 5s on all ten reportable attributes, and tighter reportable-change
deadbands on temperature, humidity and pressure. None of it is expressible
in configuration.yaml - Zigbee2MQTT has no settings-schema field for
reporting configuration - so it lives only on the device and in Z2M's
database.db. This documents it and how to restore it.

min was 10s against an 8.8s transmit interval. Since min is a hold-off, every
change-triggered report was held past a transmit cycle and landed on the
next, so attributes reported at best every 17.6s and drifted out of phase.
The published stream showed no 8.8s periodicity at all. At min 5 each
attribute reports on its own transmit and the stream carries one publish per
cycle, measured at a mean gap of 8.69s. Publish volume fell from ~631/h to
~445/h because each publish is now a whole cycle rather than a fragment.

The deadbands come from the shared modernExtend helpers, which the WS90
definition composes without overrides. They are anti-chatter values sized for
battery-powered indoor sensors. Temperature was 1.00 C, giving a staircase
that moved about twice an hour; it is now 0.1 C, matching the resolution the
device actually reports. Pressure was 5 kPa - 50 hPa, which real barometric
pressure never traverses, so it only ever updated on the hourly heartbeat
that pressure_trend and weather_condition derive from. Note the same library
configures the identical ZCL attribute with a 5 hPa deadband in its older
reporting.ts helper, so the two code paths disagree by 10x.

Anything that makes Z2M re-run configure on the device reverts all of it, and
a fresh PVC will not have it. The restore commands are written out literally
rather than generated by a shell helper, because the nested quoting through
`kubectl exec -- sh -c` is where this goes wrong; each was executed and
verified as written, including the read-back, and re-applying is a no-op.

No behaviour change from this commit itself - the device was reconfigured at
runtime, this only records it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 19:15:22 +02:00
k.renner 2e00109f66 Merge pull request 'docs(zigbee2mqtt): the WS90 merge does not make the glitch detectable' (#561) from fix-ws90-merge-overclaim into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#561
2026-08-24 17:07:34 +00:00
claude-botandClaude Opus 5 06ffc3c910 docs(zigbee2mqtt): the WS90 merge does not make the glitch detectable
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
#560 claimed a merged episode "surfaces as a single all-zero frame". It
does not, and the difference decides whether a downstream filter is even
possible.

The zeros propagate attribute by attribute through the burst, so the
merged frame carries whatever subset had zeroed when that burst ended.
Across three episodes captured at nanosecond resolution the propagation
happened to complete, which is where the wrong generalisation came from.
Nothing guarantees it. A burst ending mid-propagation publishes a frame
with temperature at zero while humidity and pressure still read normally,
which is a physically consistent measurement - Z2M recomputes dew_point
from the false temperature - and is not separable from a genuine winter
reading.

The zeros also begin inside an otherwise normal transmit cycle, roughly
20ms after a valid reading in the same burst (measured: 27.7ms, 19.8ms,
22.0ms). They occupy a regular reporting slot, so no timing property
distinguishes them either.

Since the merge is last-wins and the zeros arrive at the tail of the
burst, it resolves that burst to the zeros and discards the valid reading
that led it. So debounce trades six bad samples per episode for one, at
the cost of one good sample. That is a bound on the corruption, not a
filter, and the comment now says so.

No behaviour change; the debounce value and device stanza are untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 06:41:14 +02:00
2 changed files with 169 additions and 4 deletions
@@ -179,16 +179,32 @@ data:
# Ecowitt WS90 weather station. Joined after the ZHA-mirrored block above
# was written, hence listed apart from it.
#
# This device also carries a non-default ZCL reporting configuration
# (per-attribute min interval and reportable-change deadbands). That is
# device-side state, not expressible here — Z2M has no settings field for
# it — so it is documented, with restore commands, in
# zigbee2mqtt-ws90-reporting.md. It is lost whenever Z2M re-runs
# `configure` on the device, and a fresh PVC will not have it.
#
# The station transmits every 8.8s, as a burst of seven per-cluster ZCL
# frames; Z2M publishes its full cumulative state once per frame. debounce
# merges a burst into one publish carrying the last value per attribute,
# and must stay below the transmit interval — a window that never sees
# silence never flushes.
#
# The firmware drops all readings to zero for a few seconds every few
# days. Merged, that surfaces as a single all-zero frame; unmerged, the
# zeros spread across the seven frames and none of them is individually
# distinguishable from a valid reading.
# Every few days the firmware zeroes its readings for ~8s, then skips a
# transmit cycle and recovers on the next. The zeros propagate attribute
# by attribute through a burst rather than arriving together, and they
# start inside an otherwise normal cycle — the last valid reading and the
# first zero are ~20ms apart.
#
# Merging cuts an episode from six published bad frames to one. It does
# NOT make that frame recognisable: it carries whatever subset had zeroed
# when the burst ended, which may be every field or only temperature, and
# a frame reading {"temperature":0,"humidity":92,"pressure":96.9} is not
# separable from a valid winter reading. Since the merge is last-wins and
# the zeros arrive at the tail, it also discards the valid reading that
# led the burst. This bounds the corruption; it does not filter it.
#
# Z2M reads this only at startup (requiresRestart), and this file is
# SEED-ONLY per the header — the running instance takes it from its PVC.
@@ -0,0 +1,149 @@
# WS90 Zigbee reporting configuration
Records the ZCL attribute-reporting configuration applied to the Wetterstation
(Ecowitt WS90 powered by Shelly, IEEE `0xc02cedfffe958f40`) and how to restore
it, because **this configuration lives on the device and in Z2M's
`database.db`, not in git**. Zigbee2MQTT has no settings-schema field for
reporting configuration, so there is nothing declarative to reconcile — Flux
cannot restore it and a fresh PVC will not have it.
## Applied configuration
Endpoint 1, all intervals in seconds. `change` is in the attribute's raw ZCL
units; the physical deadband is `change ÷ scale`.
| attribute | cluster | min | max | change | deadband | converter default |
| --- | --- | --- | --- | --- | --- | --- |
| temperature | `msTemperatureMeasurement` (1026) | 5 | 3600 | 10 | 0.1 °C | 10 / 3600 / 100 |
| humidity | `msRelativeHumidity` (1029) | 5 | 3600 | 50 | 0.5 % | 10 / 3600 / 100 |
| pressure | `msPressureMeasurement` (1027) | 5 | 3600 | 1 | 1 hPa | 10 / 3600 / 50 |
| illuminance | `msIlluminanceMeasurement` (1024) | 5 | 3600 | 5 | 5 lx | 10 / 3600 / 5 |
| windSpeed | `shellyWS90Wind` (0xfc01/0) | 5 | 3600 | 1 | — | 10 / 3600 / 1 |
| windDirection | `shellyWS90Wind` (0xfc01/4) | 5 | 3600 | 1 | — | 10 / 3600 / 1 |
| gustSpeed | `shellyWS90Wind` (0xfc01/7) | 5 | 3600 | 1 | — | 10 / 3600 / 1 |
| uvIndex | `shellyWS90UV` (0xfc02/0) | 5 | 3600 | 1 | — | 10 / 3600 / 1 |
| rainStatus | `shellyWS90Rain` (0xfc03/0) | 5 | 3600 | 1 | — | 10 / 3600 / 1 |
| precipitation | `shellyWS90Rain` (0xfc03/1) | 5 | 3600 | 1 | — | 10 / 3600 / 1 |
`genPowerCfg.batteryVoltage` (cluster 1, attribute 33) is deliberately left at
its hourly default.
The three `shellyWS90*` clusters are manufacturer-specific and require
`"options": {"manufacturerCode": 5264}` on every reconfiguration request.
## Why min is 5 and not 10
The station transmits every 8.8 s. `min` is a hold-off: a change-triggered
report is held until at least `min` seconds after the previous one. With
`min: 10` every report was therefore held past a transmit cycle and landed on
the next, so attributes reported at best every 17.6 s and drifted out of phase
with each other — the published stream showed no 8.8 s periodicity at all,
just a smear across 2–8 s.
At `min: 5` each attribute reports on its own transmit. The MQTT stream then
carries one publish per transmit cycle, measured at a mean gap of 8.69 s.
Total publish volume *fell* (~631/h to ~445/h) because each publish now
carries a whole cycle instead of a fragment of one.
Any value below 8.8 s achieves this. 5 leaves margin without approaching the
cycle time.
## Why the deadbands differ from the converter defaults
The defaults come from the shared `modernExtend` helpers in
zigbee-herdsman-converters (`m.temperature()`, `m.pressure()`, `m.humidity()`,
`m.illuminance()`), which the WS90 definition in `shelly.ts` composes without
overrides. They are anti-chatter values sized for battery-powered indoor
sensors, where report volume costs coin-cell life and mesh airtime. This
station is mains-and-solar and exists to resolve small changes, so the
trade-off runs the other way.
Each value above is matched to the resolution the device actually reports:
temperature and pressure arrive in 0.1-unit steps, humidity in whole percent.
Pressure is worth calling out. The converter default of `change: 50` with
`scale: 10` is a 5 kPa (50 hPa) deadband, which real barometric pressure never
traverses — so pressure only ever updated on the hourly `max` heartbeat, and
`pressure_trend` and `weather_condition` were derived from that. The same
library's older `reporting.ts` helper configures the identical ZCL attribute
with `change: 5`, a 5 hPa deadband, so the two code paths disagree by 10× and
the modernExtend value is the outlier.
## When this is lost
Anything that makes Z2M re-run `configure` on the device overwrites all of it
with the converter defaults:
- re-pairing or re-joining the device
- an explicit `zigbee2mqtt/bridge/request/device/configure`
- a converter update that changes the definition and triggers reconfiguration
It also will not exist on a fresh PVC. Re-apply with the commands below.
## Re-applying
Published from the `discovery-guard` sidecar, which already holds the
`zigbee2mqtt` broker credential (ACL `readwrite zigbee2mqtt/#`, which covers
`bridge/request/#`) so no credential has to leave the cluster:
Ten separate requests, one per attribute — written out literally rather than
generated by a shell helper, because the nested quoting through `kubectl exec
-- sh -c` is where this goes wrong. Each line below is in the exact form that
was executed and verified.
```sh
POD=$(kubectl get pod -n home-assistant -l app.kubernetes.io/name=zigbee2mqtt -o jsonpath='{.items[0].metadata.name}')
```
```sh
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"msTemperatureMeasurement\",\"attribute\":\"measuredValue\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":10}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"msRelativeHumidity\",\"attribute\":\"measuredValue\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":50}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"msPressureMeasurement\",\"attribute\":\"measuredValue\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"msIlluminanceMeasurement\",\"attribute\":\"measuredValue\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":5}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"shellyWS90Wind\",\"attribute\":\"windSpeed\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1,\"options\":{\"manufacturerCode\":5264}}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"shellyWS90Wind\",\"attribute\":\"windDirection\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1,\"options\":{\"manufacturerCode\":5264}}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"shellyWS90Wind\",\"attribute\":\"gustSpeed\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1,\"options\":{\"manufacturerCode\":5264}}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"shellyWS90UV\",\"attribute\":\"uvIndex\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1,\"options\":{\"manufacturerCode\":5264}}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"shellyWS90Rain\",\"attribute\":\"rainStatus\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1,\"options\":{\"manufacturerCode\":5264}}"'
kubectl exec -n home-assistant "$POD" -c discovery-guard -- sh -c 'mosquitto_pub -h "$MQTT_HOST" -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t "$BASE_TOPIC/bridge/request/device/configure_reporting" -m "{\"id\":\"0xc02cedfffe958f40\",\"endpoint\":1,\"cluster\":\"shellyWS90Rain\",\"attribute\":\"precipitation\",\"minimum_report_interval\":5,\"maximum_report_interval\":3600,\"reportable_change\":1,\"options\":{\"manufacturerCode\":5264}}"'
```
These are idempotent — re-sending a request that matches the current
configuration is a no-op on the device.
The device must be awake and reachable; each request performs a real
`ConfigureReporting` exchange and fails if it is not.
## Verifying
Read back what the device actually accepted — Z2M records the negotiated
result, so this reflects the device, not the request:
```sh
kubectl exec -n home-assistant "$POD" -c zigbee2mqtt -- \
grep 0xc02cedfffe958f40 /app/data/database.db |
jq -r '.endpoints | to_entries[] | .value.configuredReportings[]?
| "cluster=\(.cluster) attr=\(.attrId) min=\(.minRepIntval) max=\(.maxRepIntval) change=\(.repChange)"'
```
All ten rows above should read `min=5 max=3600`. On the wire, the publish
cadence on `zigbee2mqtt/Wetterstation` should sit at ~8.7 s and temperature
should move in 0.1 °C steps; 1 °C steps mean the defaults are back.
## Making it durable upstream
The lasting fix is for the WS90 definition to pass explicit `reporting`
overrides rather than inherit indoor-sensor defaults, which would fix every
WS90 rather than this one. `m.temperature()` and the other helpers already
accept a `reporting` argument. The pressure divergence between
`modernExtend.ts` and `reporting.ts` is a separate upstream bug worth
reporting on its own.