fix(chrony): step the clock at boot so a bad RTC self-corrects
chrony.conf.j2 had no makestep directive, so chronyd only ever slewed. A large offset is therefore never closed in practice. On 2026-08-04 edge-1 rebooted with its RTC ~16 days in the past. chronyd was synchronised to ptbtime1 at stratum 2 with a sub-millisecond RMS offset, yet reported "System time: 1376235.375 seconds slow of NTP time" and corrected none of it. With host time at 2026-07-20, every outbound TLS handshake failed x509 NotBefore validation against a cluster certificate issued 2026-07-23: tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2026-07-20T16:15:36+02:00 is before 2026-07-23T06:15:01Z That took out the local mosquitto -> cluster MQTT bridge (all Vitocal 250-A / Vitocharge VX3 / E380 heating telemetry went unavailable in Home Assistant for ~18h) and the Alloy -> Loki log shipper simultaneously. Because the same skew broke log shipping, the outage was invisible to log-based alerting. Recovery required a manual `chronyc makestep`. Adding the directive lets this correct itself at boot instead. The limit keeps steps confined to the first few updates after chronyd starts, so a long-running NTP server is never jumped out from under its workloads mid-flight. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -18,6 +18,18 @@ chrony_upstream_pools:
|
||||
# Local stratum if disconnected from upstream
|
||||
chrony_local_stratum: 10
|
||||
|
||||
# Step (rather than slew) the clock when the offset exceeds the threshold, but
|
||||
# only for the first N updates after chronyd starts. Without this, chronyd only
|
||||
# ever slews, and a large offset is never closed in practice: on 2026-08-04
|
||||
# edge-1 rebooted with an RTC ~16 days in the past and chronyd sat synchronised
|
||||
# to ptbtime1 reporting "System time: 1376235s slow" while correcting nothing.
|
||||
# Every outbound TLS handshake then failed x509 NotBefore validation, taking out
|
||||
# the MQTT bridge (heating telemetry) and the Loki log shipper at once.
|
||||
# Limiting to the first few updates keeps steps confined to boot, so a
|
||||
# long-running server is never jumped out from under its workloads.
|
||||
chrony_makestep_threshold: 1.0
|
||||
chrony_makestep_limit: 3
|
||||
|
||||
# Hardware timestamping. Fine on the PVE nodes' Intel NICs; the Wyse 3040
|
||||
# (edge-1) has a Realtek NIC that does not support it, and chronyd logs an
|
||||
# error per interface when asked for it. Overridden to false in
|
||||
|
||||
@@ -14,6 +14,11 @@ pool {{ pool }}
|
||||
# Record the rate at which the system clock gains/losses time
|
||||
driftfile /var/lib/chrony/chrony.drift
|
||||
|
||||
# Step the clock instead of slewing it when the offset is large, limited to the
|
||||
# first few updates after start so only boot-time corrections jump. Without a
|
||||
# makestep directive chronyd slews exclusively and never closes a large gap.
|
||||
makestep {{ chrony_makestep_threshold }} {{ chrony_makestep_limit }}
|
||||
|
||||
# Allow NTP client access from local networks
|
||||
{% for network in chrony_allowed_networks %}
|
||||
allow {{ network }}
|
||||
|
||||
Reference in New Issue
Block a user