From 8967db10011ecd5cc373c42fb482028392dda797 Mon Sep 17 00:00:00 2001 From: atlas-bot Date: Sat, 26 Sep 2026 08:08:29 +0000 Subject: [PATCH] =?UTF-8?q?test(uptime-monitor):=20restore=20the=20audiobo?= =?UTF-8?q?oks=20EndpointMonitor,=20and=20correct=20the=20403=20reading=20?= =?UTF-8?q?=E2=80=94=20REN-7?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Criterion 3 passed: the removal test (#1018, merged 101c11f4) deleted monitor 804092841 and the controller logged `Monitor Removed` at 03:57:58Z. This commit is the deliberate revert of that test, which Zeus sequenced to run only after the cap reading below. The restored object is byte-identical at the object level to the version that provably created 804092841. It also corrects a wrong claim this file asserted. The header stated that a monitor-count cap was EXCLUDED as the cause of the `www.renner.uno` 403, on the reasoning that a cap would arrive as HTTP 200 + `stat: fail`. The removal test refuted that: 03:26:27-03:56:27 account 10/10, seven consecutive `Status Code: 403` 03:57:58 804092841 deleted, account 9 04:01:27 `Monitor Added: www.renner.uno-uptime-monitor`, id 804092931, account back to 10 One freed slot is the only variable. UptimeRobot returns HTTP 403 when the account monitor limit is reached, so "account is full" is the first hypothesis for an IMC 403, not a discarded one. Capacity is now settled on fact for REN-158; REN-8's 18-monitor page against a cap of 10 stays blocked on it. Expect this restored create to be refused with 403 until the tier is raised — `www` took the freed slot. Left in git per the capacity ruling so the retry loop creates it unaided once capacity exists. `audiobooks.renner.uno` stays covered by its 60s Gatus check throughout, so there is no coverage gap. Also recorded: the real delete log string is `Removing monitor … from provider` / `Monitor Removed`, not the `Deleting Monitor` this file predicted; and the status page is in all-monitors mode, proven in both directions (9->10 on create, 10->9 on prune) with no API key, which settles REN-9's monitor-list mode. Verified: js-yaml loadAll gives 2 EndpointMonitors in `uptime-monitor`; the only functional changed line is the restored object, every other changed line is a comment; restored spec compared equal to `f20fe8bf^`. Not applied — Flux applies from master and I cannot merge my own work. Revert: revert this commit. Flux prunes the object and, because its monitor does not exist, nothing in UptimeRobot changes. Co-Authored-By: Claude Opus 5 --- .../pilot-endpointmonitors.yaml | 176 ++++++++++++------ 1 file changed, 115 insertions(+), 61 deletions(-) diff --git a/kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml b/kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml index d9435487..f4781f1c 100644 --- a/kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml +++ b/kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml @@ -1,32 +1,45 @@ --- # ============================================================================ -# REN-7 CRITERION 3 — THE REMOVAL TEST. THIS COMMIT DELETES THE `audiobooks` -# ENDPOINTMONITOR ON PURPOSE. IT IS NOT A CLEANUP AND NOT AN ABANDONMENT. +# REN-7 CRITERION 3 PASSED. THIS COMMIT PUTS THE `audiobooks` ENDPOINTMONITOR +# BACK — IT IS THE DELIBERATE REVERT OF THE REMOVAL TEST, NOT A NEW FEATURE. # -# The card requires proof in both directions: an EndpointMonitor creates a -# monitor, and DELETING the EndpointMonitor removes it again. "A monitor that -# cannot be removed is a leak." Create is proven — `audiobooks.renner.uno- -# uptime-monitor`, UptimeRobot id 804092841, created 2026-09-26 03:26:27, -# controller log `Monitor Added`. Removing the object here is the other half. +# The card requires proof in both directions, and both are now on record: # -# What must be observed after this merges: the public feed -# `https://uptime.renner.uno/api/getMonitorList/UgMXmP1iRV` goes from -# `psp.totalMonitors: 10` to `9`, and id 804092841 is absent. The controller -# log shows `Deleting Monitor: audiobooks.renner.uno-uptime-monitor`. +# CREATE 2026-09-26T03:26:27Z `Monitor Added: audiobooks.renner.uno- +# uptime-monitor`, UptimeRobot id 804092841, feed +# `psp.totalMonitors` 9 → 10. +# DELETE 2026-09-26T03:57:58Z object pruned by Flux after #1018 merged as +# 101c11f4; controller log `Removing monitor audiobooks.renner.uno- +# uptime-monitor from provider: UptimeRobot` then `Monitor Removed`; +# feed 10 → 9 with id 804092841 absent, confirmed independently. # -# PRECONDITION, CHECKED BEFORE THIS COMMIT WAS WRITTEN: v2.2.13 has no -# finalizer, so a delete while the controller is down orphans the monitor -# forever (see the caveat at the end of this header). Pod -# `ingressmonitorcontroller-5d577cc58-pgqwg` was `Ready=True`, `restarts=0`, -# up since 2026-09-26T00:43:45Z when this was authored. Anyone merging this -# must re-check that the pod is Ready first; if it is not, hold the merge. +# THE LOG STRING IS `Removing monitor … from provider` / `Monitor Removed`. +# An earlier revision of this header predicted `Deleting Monitor: …`, which +# appears nowhere in v2.2.13. Anyone grepping the predicted string finds +# nothing and wrongly concludes the delete failed. Grep `Monitor Removed`. # -# RE-CREATING IT IS THE REVERT: revert this commit and Flux re-applies the -# object, and the controller creates the monitor again under the same name. -# That round trip is also the `idempotent reconcile` check, so the revert is -# worth doing deliberately rather than only if something goes wrong. Expect a -# NEW UptimeRobot id — the friendly name is what binds object to monitor, not -# the id. +# Re-creating the object is idempotent by design: the friendly name binds +# object to monitor, not the id, so the new monitor carries a NEW id. +# +# PRECONDITION RE-CHECKED BEFORE THIS COMMIT WAS WRITTEN, and it is the one the +# no-finalizer caveat below demands: pod `ingressmonitorcontroller-5d577cc58- +# pgqwg` read from the Kubernetes API at 2026-09-26T08:01Z — `Ready=True`, +# `ContainersReady=True`, `restarts=0`, image v2.2.13, up since +# 2026-09-26T00:43:45Z, the same pod that served the whole create/delete +# window. That rule binds DELETES, so it does not gate this commit, but the +# reviewer should re-read it anyway before merging. +# +# EXPECT THIS CREATE TO BE REFUSED, AND THAT IS NOT A FAULT. The account cap is +# 10 monitors and it is full again (see the capacity section below). Restoring +# the object is what Zeus directed after the cap reading, on the reasoning that +# the retry loop then creates the monitor by itself the moment the tier is +# raised — no further PR needed. Until then this object logs one refused POST +# every 300s and nothing else: nothing pages, nothing degrades, and +# `audiobooks.renner.uno` stays covered by its 60s Gatus check +# (apps/gatus/configmap.yaml:163) throughout. +# +# REVERT: revert this commit. Flux prunes the object again and, because the +# monitor does not exist to begin with, nothing in UptimeRobot changes. # ============================================================================ # # REN-7 — the pilot batch. Two of the 14 monitors from the survey's §A0 page @@ -42,7 +55,9 @@ # appearing on it is authorised rather than a disclosure. That was a deliberate # selection criterion, not a coincidence — see the status-page note below. # -# Neither host has a monitor in the account today. The nine that exist are +# Neither host had a monitor in the account when the pilot was chosen; today +# `www.renner.uno-uptime-monitor` (id 804092931) exists because this pilot +# created it. The nine pre-existing ones are # `renner.uno`, `home.`, `immich.`, `auth.`, `docs.renner.uno` and the four # third-party ones; creating an EndpointMonitor for any of those would produce # a SECOND monitor beside the hand-made one rather than adopt it, because IMC @@ -111,19 +126,30 @@ # proof the alert path works. # # --- -# NO `statusPages` — DELIBERATELY, AS AN EXPERIMENT (Zeus, on REN-7) +# NO `statusPages` — AND THE EXPERIMENT THAT JUSTIFIED LEAVING IT UNSET IS NOW +# ANSWERED: THE PAGE IS IN ALL-MONITORS MODE. KEEP THE FIELD UNSET. # # IMC only adds a monitor to a public status page when # `uptimeRobotConfig.statusPages` is set; with the field absent # `handleStatusPagesConfig` returns without calling the API at all -# (uptime-monitor.go:318-329). Leaving it unset therefore asks the page itself -# the open REN-9 question: after these two monitors exist, do they appear on -# the `uptime.renner.uno` feed on their own (the page is in an all-monitors -# mode) or not (each monitor must be added explicitly)? +# (uptime-monitor.go:318-329). It was left unset to ask the page itself REN-9's +# question, and the pilot answered it in BOTH directions with no API key: # -# Baseline for that comparison, read keyless at 2026-09-26 from +# 9 → 10 on create, monitor appeared on the feed unbidden +# 10 → 9 on prune, monitor disappeared from it unbidden +# +# So `uptime.renner.uno` includes every monitor in the account automatically and +# nothing needs adding to it. Caveat for REN-9, and it is the fragile part: +# all-monitors mode is a setting in the UptimeRobot dashboard, NOT in git. If +# anyone switches that page to a fixed list, this stops being true silently and +# no manifest, review or alert here would catch it. +# +# Baseline that comparison was made against, read keyless at 2026-09-26 from # `https://uptime.renner.uno/api/getMonitorList/UgMXmP1iRV`: 9 monitors, ids -# 801710132/37/43/45/47/52/97 and 801714845/79. +# 801710132/37/43/45/47/52/97 and 801714845/79. Parser notes for whoever reads +# this feed next: there is no top-level `monitors` key — the list is at `data` +# (and again at `psp.monitors`), the count is `psp.totalMonitors`, and an id is +# `monitorId`, not `id`. # # --- # BLAST RADIUS @@ -167,29 +193,45 @@ # through the API rather than hoping a later reconcile finds it. # # --- -# STATE OF THE OBJECT BELOW: ITS CREATE IS BEING REJECTED, AND IT RETRIES -# FOREVER. TRACKED ON REN-155 — LEFT IN PLACE DELIBERATELY. +# CAPACITY: THE ACCOUNT CAP IS EXACTLY 10 MONITORS, PROVEN BY EXPERIMENT AND +# WITHOUT AN API KEY. THIS SUPERSEDES THIS FILE'S EARLIER READING OF THE 403. # -# `www.renner.uno-uptime-monitor` does not exist in UptimeRobot. Controller log, -# every 300s since 2026-09-26T03:26:27Z: +# An earlier revision of this header asserted the opposite — that "a duplicate +# URL, a monitor-count cap and a bad parameter would all arrive as 200 + +# `stat: fail`, not as 403", and therefore that the cap was EXCLUDED. That +# inference was wrong, and the correction is the useful finding here: # -# Creating Monitor: www.renner.uno-uptime-monitor -# AddMonitor Request failed. Status Code: 403 +# UPTIMEROBOT RETURNS HTTP 403 WHEN THE ACCOUNT MONITOR LIMIT IS REACHED. +# It does not use the 200 + `stat: fail` validation path for this. Anyone +# reasoning about an IMC 403 should treat "account is full" as the FIRST +# hypothesis, not a discarded one. # -# That is the `else` branch of `Add()` (uptime-monitor.go:187) — a NON-200, and -# therefore NOT the provider's own validation path, which returns HTTP 200 with -# `{"stat":"fail", ...}` and is logged at :174 with the message text. So the -# common guesses are excluded: a duplicate URL, a monitor-count cap and a bad -# parameter would all arrive as 200 + `stat: fail`, not as 403. The 403 body is -# discarded by that branch, so the provider's reason is not recoverable from the -# log; settling it needs the API key or the account dashboard. +# The proof is a natural experiment the removal test handed us for free, read +# from the controller log of the single pod that spans the whole window: # -# It is left in git rather than removed because it is harmless and it is the -# reproducer: `Add()` is fire-and-forget (`handleCreate` returns nil regardless, -# endpointmonitor_created.go:12-31), so there is no error, no status and no -# alert — only that log line, at INFO. Nothing pages, nothing degrades, and the -# object is the only remaining evidence. Anyone tempted to "clean it up" should -# close REN-155 first. +# 03:26:27 → 03:56:27 account at 10/10. `Creating Monitor: +# www.renner.uno-uptime-monitor` then `AddMonitor +# Request failed. Status Code: 403`, SEVEN consecutive +# 300s cycles. On cycles 2-7 that create is the only +# POST in the cycle, so it is not a burst/rate effect. +# 03:57:58 removal test deletes monitor 804092841 → account 9. +# 04:01:27 next ordinary reconcile, nothing else changed: +# `Monitor Added: www.renner.uno-uptime-monitor`, +# new id 804092931, feed back to 10. +# +# The only variable between the last 403 and the success is one free slot. A +# duplicate-URL or bad-parameter cause cannot explain a create that starts +# succeeding when an UNRELATED monitor is deleted. Capacity is settled on fact; +# the tier decision is REN-158's, and REN-8's §A0 page needs 18 monitors +# against this cap of 10, so no rollout batch may proceed until it is resolved. +# +# Why the object below stayed in git through those seven refusals: `Add()` is +# fire-and-forget (`handleCreate` returns nil regardless, +# endpointmonitor_created.go:12-31) and the failure is logged at INFO, not +# ERROR — so there is no error, no status and no alert, and the retrying object +# was the only evidence that the create was being refused at all. That property +# is worth remembering for REN-8: a rollout that exceeds the cap fails SILENTLY, +# monitor by monitor, with nothing to notice it but this one INFO line. apiVersion: endpointmonitor.stakater.com/v1alpha1 kind: EndpointMonitor metadata: @@ -212,15 +254,27 @@ spec: # matches the interval of the nine monitors already in the account. Written # out rather than left implicit so the value is reviewable here. interval: 300 -# The `audiobooks.renner.uno` EndpointMonitor was here. It is removed by this -# commit as REN-7's criterion-3 removal test — see the header block at the top -# of this file for what that proves, what to observe, and how to put it back. -# Its spec, for the revert: +--- +# Restored by this commit. This object already proved criterion 3 end to end: +# created as id 804092841 at 03:26:27Z, removed as `Monitor Removed` at +# 03:57:58Z when it was pruned from git. See the header for that evidence. # -# url: https://audiobooks.renner.uno/healthcheck # Gatus check, -# # apps/gatus/configmap.yaml:134 -# uptimeRobotConfig: { monitorType: http, interval: 300 } -# -# The path belongs in `spec.url` and not in `spec.healthEndpoint`, because -# `healthEndpoint` is only consulted on the ref-discovery path and is explicitly -# ignored when `spec.url` is set (pkg/kube/util/url.go:31-33). +# Its create will now be refused with HTTP 403 until the account tier is raised +# (REN-158), because `www.renner.uno` took the slot this object's deletion +# freed. That is expected and documented, not a regression to chase. +apiVersion: endpointmonitor.stakater.com/v1alpha1 +kind: EndpointMonitor +metadata: + name: audiobooks.renner.uno + namespace: uptime-monitor +spec: + # The path belongs in `spec.url` and NOT in `spec.healthEndpoint`, because + # `healthEndpoint` is only consulted on the ref-discovery path and is + # explicitly ignored when `spec.url` is set (pkg/kube/util/url.go:31-33). + # `/healthcheck` is ungated, so the check reaches application health rather + # than mere reachability — it is the same endpoint the Gatus check uses + # (apps/gatus/configmap.yaml:163). + url: https://audiobooks.renner.uno/healthcheck + uptimeRobotConfig: + monitorType: http + interval: 300 -- 2.54.0