konairius/homelab
10
0
Fork 0
Code Issues 8 Pull Requests 4 Packages Projects Releases 1 Wiki Activity

test(uptime-monitor): restore the audiobooks EndpointMonitor, and correct the 403 reading — REN-7 #1024

Merged
Ghost merged 1 commits from atlas/ren-7-restore-audiobooks into master 2026-09-26 08:18:38 +00:00
Conversation 2 Commits 1 Files Changed 1
+115 -61
1 changed files with 115 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml
+115 -61
View File
@@ -1,32 +1,45 @@
---
# ============================================================================
# REN-7 CRITERION 3 — THE REMOVAL TEST. THIS COMMIT DELETES THE `audiobooks`
# ENDPOINTMONITOR ON PURPOSE. IT IS NOT A CLEANUP AND NOT AN ABANDONMENT.
# REN-7 CRITERION 3 PASSED. THIS COMMIT PUTS THE `audiobooks` ENDPOINTMONITOR
# BACK — IT IS THE DELIBERATE REVERT OF THE REMOVAL TEST, NOT A NEW FEATURE.
#
# The card requires proof in both directions: an EndpointMonitor creates a
# monitor, and DELETING the EndpointMonitor removes it again. "A monitor that
# cannot be removed is a leak." Create is proven — `audiobooks.renner.uno-
# uptime-monitor`, UptimeRobot id 804092841, created 2026-09-26 03:26:27,
# controller log `Monitor Added`. Removing the object here is the other half.
# The card requires proof in both directions, and both are now on record:
#
# What must be observed after this merges: the public feed
# `https://uptime.renner.uno/api/getMonitorList/UgMXmP1iRV` goes from
# `psp.totalMonitors: 10` to `9`, and id 804092841 is absent. The controller
# log shows `Deleting Monitor: audiobooks.renner.uno-uptime-monitor`.
# CREATE 2026-09-26T03:26:27Z `Monitor Added: audiobooks.renner.uno-
# uptime-monitor`, UptimeRobot id 804092841, feed
# `psp.totalMonitors` 9 → 10.
# DELETE 2026-09-26T03:57:58Z object pruned by Flux after #1018 merged as
# 101c11f4; controller log `Removing monitor audiobooks.renner.uno-
# uptime-monitor from provider: UptimeRobot` then `Monitor Removed`;
# feed 10 → 9 with id 804092841 absent, confirmed independently.
#
# PRECONDITION, CHECKED BEFORE THIS COMMIT WAS WRITTEN: v2.2.13 has no
# finalizer, so a delete while the controller is down orphans the monitor
# forever (see the caveat at the end of this header). Pod
# `ingressmonitorcontroller-5d577cc58-pgqwg` was `Ready=True`, `restarts=0`,
# up since 2026-09-26T00:43:45Z when this was authored. Anyone merging this
# must re-check that the pod is Ready first; if it is not, hold the merge.
# THE LOG STRING IS `Removing monitor … from provider` / `Monitor Removed`.
# An earlier revision of this header predicted `Deleting Monitor: …`, which
# appears nowhere in v2.2.13. Anyone grepping the predicted string finds
# nothing and wrongly concludes the delete failed. Grep `Monitor Removed`.
#
# RE-CREATING IT IS THE REVERT: revert this commit and Flux re-applies the
# object, and the controller creates the monitor again under the same name.
# That round trip is also the `idempotent reconcile` check, so the revert is
# worth doing deliberately rather than only if something goes wrong. Expect a
# NEW UptimeRobot id — the friendly name is what binds object to monitor, not
# the id.
# Re-creating the object is idempotent by design: the friendly name binds
# object to monitor, not the id, so the new monitor carries a NEW id.
#
# PRECONDITION RE-CHECKED BEFORE THIS COMMIT WAS WRITTEN, and it is the one the
# no-finalizer caveat below demands: pod `ingressmonitorcontroller-5d577cc58-
# pgqwg` read from the Kubernetes API at 2026-09-26T08:01Z — `Ready=True`,
# `ContainersReady=True`, `restarts=0`, image v2.2.13, up since
# 2026-09-26T00:43:45Z, the same pod that served the whole create/delete
# window. That rule binds DELETES, so it does not gate this commit, but the
# reviewer should re-read it anyway before merging.
#
# EXPECT THIS CREATE TO BE REFUSED, AND THAT IS NOT A FAULT. The account cap is
# 10 monitors and it is full again (see the capacity section below). Restoring
# the object is what Zeus directed after the cap reading, on the reasoning that
# the retry loop then creates the monitor by itself the moment the tier is
# raised — no further PR needed. Until then this object logs one refused POST
# every 300s and nothing else: nothing pages, nothing degrades, and
# `audiobooks.renner.uno` stays covered by its 60s Gatus check
# (apps/gatus/configmap.yaml:163) throughout.
#
# REVERT: revert this commit. Flux prunes the object again and, because the
# monitor does not exist to begin with, nothing in UptimeRobot changes.
# ============================================================================
#
# REN-7 — the pilot batch. Two of the 14 monitors from the survey's §A0 page
@@ -42,7 +55,9 @@
# appearing on it is authorised rather than a disclosure. That was a deliberate
# selection criterion, not a coincidence — see the status-page note below.
#
# Neither host has a monitor in the account today. The nine that exist are
# Neither host had a monitor in the account when the pilot was chosen; today
# `www.renner.uno-uptime-monitor` (id 804092931) exists because this pilot
# created it. The nine pre-existing ones are
# `renner.uno`, `home.`, `immich.`, `auth.`, `docs.renner.uno` and the four
# third-party ones; creating an EndpointMonitor for any of those would produce
# a SECOND monitor beside the hand-made one rather than adopt it, because IMC
@@ -111,19 +126,30 @@
# proof the alert path works.
#
# ---
# NO `statusPages` — DELIBERATELY, AS AN EXPERIMENT (Zeus, on REN-7)
# NO `statusPages` — AND THE EXPERIMENT THAT JUSTIFIED LEAVING IT UNSET IS NOW
# ANSWERED: THE PAGE IS IN ALL-MONITORS MODE. KEEP THE FIELD UNSET.
#
# IMC only adds a monitor to a public status page when
# `uptimeRobotConfig.statusPages` is set; with the field absent
# `handleStatusPagesConfig` returns without calling the API at all
# (uptime-monitor.go:318-329). Leaving it unset therefore asks the page itself
# the open REN-9 question: after these two monitors exist, do they appear on
# the `uptime.renner.uno` feed on their own (the page is in an all-monitors
# mode) or not (each monitor must be added explicitly)?
# (uptime-monitor.go:318-329). It was left unset to ask the page itself REN-9's
# question, and the pilot answered it in BOTH directions with no API key:
#
# Baseline for that comparison, read keyless at 2026-09-26 from
# 9 → 10 on create, monitor appeared on the feed unbidden
# 10 → 9 on prune, monitor disappeared from it unbidden
#
# So `uptime.renner.uno` includes every monitor in the account automatically and
# nothing needs adding to it. Caveat for REN-9, and it is the fragile part:
# all-monitors mode is a setting in the UptimeRobot dashboard, NOT in git. If
# anyone switches that page to a fixed list, this stops being true silently and
# no manifest, review or alert here would catch it.
#
# Baseline that comparison was made against, read keyless at 2026-09-26 from
# `https://uptime.renner.uno/api/getMonitorList/UgMXmP1iRV`: 9 monitors, ids
# 801710132/37/43/45/47/52/97 and 801714845/79.
# 801710132/37/43/45/47/52/97 and 801714845/79. Parser notes for whoever reads
# this feed next: there is no top-level `monitors` key — the list is at `data`
# (and again at `psp.monitors`), the count is `psp.totalMonitors`, and an id is
# `monitorId`, not `id`.
#
# ---
# BLAST RADIUS
@@ -167,29 +193,45 @@
# through the API rather than hoping a later reconcile finds it.
#
# ---
# STATE OF THE OBJECT BELOW: ITS CREATE IS BEING REJECTED, AND IT RETRIES
# FOREVER. TRACKED ON REN-155 — LEFT IN PLACE DELIBERATELY.
# CAPACITY: THE ACCOUNT CAP IS EXACTLY 10 MONITORS, PROVEN BY EXPERIMENT AND
# WITHOUT AN API KEY. THIS SUPERSEDES THIS FILE'S EARLIER READING OF THE 403.
#
# `www.renner.uno-uptime-monitor` does not exist in UptimeRobot. Controller log,
# every 300s since 2026-09-26T03:26:27Z:
# An earlier revision of this header asserted the opposite — that "a duplicate
# URL, a monitor-count cap and a bad parameter would all arrive as 200 +
# `stat: fail`, not as 403", and therefore that the cap was EXCLUDED. That
# inference was wrong, and the correction is the useful finding here:
#
# Creating Monitor: www.renner.uno-uptime-monitor
# AddMonitor Request failed. Status Code: 403
# UPTIMEROBOT RETURNS HTTP 403 WHEN THE ACCOUNT MONITOR LIMIT IS REACHED.
# It does not use the 200 + `stat: fail` validation path for this. Anyone
# reasoning about an IMC 403 should treat "account is full" as the FIRST
# hypothesis, not a discarded one.
#
# That is the `else` branch of `Add()` (uptime-monitor.go:187) — a NON-200, and
# therefore NOT the provider's own validation path, which returns HTTP 200 with
# `{"stat":"fail", ...}` and is logged at :174 with the message text. So the
# common guesses are excluded: a duplicate URL, a monitor-count cap and a bad
# parameter would all arrive as 200 + `stat: fail`, not as 403. The 403 body is
# discarded by that branch, so the provider's reason is not recoverable from the
# log; settling it needs the API key or the account dashboard.
# The proof is a natural experiment the removal test handed us for free, read
# from the controller log of the single pod that spans the whole window:
#
# It is left in git rather than removed because it is harmless and it is the
# reproducer: `Add()` is fire-and-forget (`handleCreate` returns nil regardless,
# endpointmonitor_created.go:12-31), so there is no error, no status and no
# alert — only that log line, at INFO. Nothing pages, nothing degrades, and the
# object is the only remaining evidence. Anyone tempted to "clean it up" should
# close REN-155 first.
# 03:26:27 → 03:56:27 account at 10/10. `Creating Monitor:
# www.renner.uno-uptime-monitor` then `AddMonitor
# Request failed. Status Code: 403`, SEVEN consecutive
# 300s cycles. On cycles 2-7 that create is the only
# POST in the cycle, so it is not a burst/rate effect.
# 03:57:58 removal test deletes monitor 804092841 → account 9.
# 04:01:27 next ordinary reconcile, nothing else changed:
# `Monitor Added: www.renner.uno-uptime-monitor`,
# new id 804092931, feed back to 10.
#
# The only variable between the last 403 and the success is one free slot. A
# duplicate-URL or bad-parameter cause cannot explain a create that starts
# succeeding when an UNRELATED monitor is deleted. Capacity is settled on fact;
# the tier decision is REN-158's, and REN-8's §A0 page needs 18 monitors
# against this cap of 10, so no rollout batch may proceed until it is resolved.
#
# Why the object below stayed in git through those seven refusals: `Add()` is
# fire-and-forget (`handleCreate` returns nil regardless,
# endpointmonitor_created.go:12-31) and the failure is logged at INFO, not
# ERROR — so there is no error, no status and no alert, and the retrying object
# was the only evidence that the create was being refused at all. That property
# is worth remembering for REN-8: a rollout that exceeds the cap fails SILENTLY,
# monitor by monitor, with nothing to notice it but this one INFO line.
apiVersion: endpointmonitor.stakater.com/v1alpha1
kind: EndpointMonitor
metadata:
@@ -212,15 +254,27 @@ spec:
# matches the interval of the nine monitors already in the account. Written
# out rather than left implicit so the value is reviewable here.
interval: 300
# The `audiobooks.renner.uno` EndpointMonitor was here. It is removed by this
# commit as REN-7's criterion-3 removal test — see the header block at the top
# of this file for what that proves, what to observe, and how to put it back.
# Its spec, for the revert:
---
# Restored by this commit. This object already proved criterion 3 end to end:
# created as id 804092841 at 03:26:27Z, removed as `Monitor Removed` at
# 03:57:58Z when it was pruned from git. See the header for that evidence.
#
# url: https://audiobooks.renner.uno/healthcheck # Gatus check,
# # apps/gatus/configmap.yaml:134
# uptimeRobotConfig: { monitorType: http, interval: 300 }
#
# The path belongs in `spec.url` and not in `spec.healthEndpoint`, because
# `healthEndpoint` is only consulted on the ref-discovery path and is explicitly
# ignored when `spec.url` is set (pkg/kube/util/url.go:31-33).
# Its create will now be refused with HTTP 403 until the account tier is raised
# (REN-158), because `www.renner.uno` took the slot this object's deletion
# freed. That is expected and documented, not a regression to chase.
apiVersion: endpointmonitor.stakater.com/v1alpha1
kind: EndpointMonitor
metadata:
name: audiobooks.renner.uno
namespace: uptime-monitor
spec:
# The path belongs in `spec.url` and NOT in `spec.healthEndpoint`, because
# `healthEndpoint` is only consulted on the ref-discovery path and is
# explicitly ignored when `spec.url` is set (pkg/kube/util/url.go:31-33).
# `/healthcheck` is ungated, so the check reaches application health rather
# than mere reachability — it is the same endpoint the Gatus check uses
# (apps/gatus/configmap.yaml:163).
url: https://audiobooks.renner.uno/healthcheck
uptimeRobotConfig:
monitorType: http
interval: 300