test(uptime-monitor): restore the audiobooks EndpointMonitor, and correct the 403 reading — REN-7 #1024

Merged

Ghost merged 1 commits from atlas/ren-7-restore-audiobooks into master 2026-09-26 08:18:38 +00:00

Conversation 2 Commits 1 Files Changed 1

+115 -61

Ghost commented 2026-09-26 08:10:08 +00:00

Copy Link

Copy Source

What changed

Restores the audiobooks.renner.uno EndpointMonitor that #1018 deleted as REN-7's criterion-3 removal test, and corrects a claim in the file header that the test itself disproved. One file. The only functional changed line is the restored object — every other changed line is a YAML comment, verifiable with:

git diff -U0 origin/master -- kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml \
  | grep -E '^[+-]' | grep -vE '^(\+\+\+|---)' | sed -E 's/^.//' | grep -vE '^\s*(#|$)'

Why

Criterion 3 passed. #1018 (merged 101c11f4) pruned the object; the controller logged Monitor Removed: audiobooks.renner.uno-uptime-monitor at 03:57:58Z and the public feed went 10 → 9 with id 804092841 absent. Restoring the object is the deliberate revert of that test — Zeus sequenced it to happen only after the cap reading below, and that reading is now done.

The removal test doubled as a capacity experiment, and it overturned this file's own reasoning. The header asserted that a monitor-count cap was excluded as the cause of the www.renner.uno 403, because a cap would arrive as HTTP 200 + stat: fail. That was wrong:

time	account	event
03:26:27 → 03:56:27	10/10	Creating Monitor: www.renner.uno-uptime-monitor → AddMonitor Request failed. Status Code: 403, 7 consecutive 300s cycles
03:57:58	10 → 9	removal test deletes 804092841
04:01:27	9 → 10	Monitor Added: www.renner.uno-uptime-monitor, new id 804092931

One freed slot is the only variable, and on cycles 2–7 the refused create was the only POST in its cycle, so it is not a burst effect. UptimeRobot returns HTTP 403 when the account monitor limit is reached — so "account is full" is the first hypothesis for an IMC 403, not a discarded one. Capacity is now settled on fact for REN-158 rather than on the vendor's pricing page.

This restored create will be refused with 403 until the tier is raised, because www took the freed slot. That is expected, documented in the file, and per the capacity ruling the object stays in git so the retry loop creates the monitor unaided the moment capacity exists — no further PR needed. audiobooks.renner.uno remains covered by its 60s Gatus check (apps/gatus/configmap.yaml:163) throughout, so there is no coverage gap in the meantime.

Two other corrections to stale header text, both of which would mislead a future reader:

• The delete log string is Removing monitor … from provider / Monitor Removed, not the Deleting Monitor this file predicted. Grepping the predicted string finds nothing and looks like a failed delete.
• statusPages stays unset, and the experiment that justified leaving it unset is answered: the page is in all-monitors mode, proven in both directions (9 → 10 on create, 10 → 9 on prune) with no API key. Caveat now recorded: that mode is a dashboard setting, not git, so it can change silently.

How it was verified

• js-yaml loadAll → 2 non-empty documents, both EndpointMonitor in uptime-monitor, http / interval: 300.
• Restored object compared equal at the object level to f20fe8bf^ — the exact version that provably created monitor 804092841. The round trip is not a re-derivation.
• Controller precondition re-read from the Kubernetes API at 08:01Z: pod ingressmonitorcontroller-5d577cc58-pgqwg Ready=True, ContainersReady=True, restarts=0, v2.2.13, up since 00:43:45Z — the same pod that spans the whole create/delete window, so the log evidence above has no gap.
• Log and feed evidence read first-hand this run, keyless for the feed and read-only for the log.
• Not applied. Flux applies from master and I cannot merge my own work, so this is parse + object comparison, not an applied change. kustomize build was not run — kustomize is not installed in this run's environment; the prior 68/68 build figure is from an earlier run and is not re-asserted here.

The no-finalizer rule binds deletes, so it does not gate this commit. Re-check the pod is Ready before merging anyway.

Blast radius

One object added in uptime-monitor. Its create is refused, so nothing changes in UptimeRobot on merge — no new monitor, no new probe traffic, no page change. Not touched: no Ingress edited, no workload restarted, no ingress controller change, no DNS change, no RBAC change.

Revert

git revert this commit. Flux prunes the object again and, because its monitor does not exist to begin with, nothing in UptimeRobot changes. Strictly safer than the revert it undoes.

🤖 Generated with Claude Code

## What changed Restores the `audiobooks.renner.uno` `EndpointMonitor` that #1018 deleted as REN-7's criterion-3 removal test, and corrects a claim in the file header that the test itself disproved. One file. The only functional changed line is the restored object — **every other changed line is a YAML comment**, verifiable with: ``` git diff -U0 origin/master -- kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml \ | grep -E '^[+-]' | grep -vE '^(\+\+\+|---)' | sed -E 's/^.//' | grep -vE '^\s*(#|$)' ``` ## Why **Criterion 3 passed.** #1018 (merged `101c11f4`) pruned the object; the controller logged `Monitor Removed: audiobooks.renner.uno-uptime-monitor` at `03:57:58Z` and the public feed went `10 → 9` with id `804092841` absent. Restoring the object is the deliberate revert of that test — Zeus sequenced it to happen only *after* the cap reading below, and that reading is now done. **The removal test doubled as a capacity experiment, and it overturned this file's own reasoning.** The header asserted that a monitor-count cap was *excluded* as the cause of the `www.renner.uno` 403, because a cap would arrive as HTTP 200 + `stat: fail`. That was wrong: | time | account | event | |---|---|---| | `03:26:27` → `03:56:27` | 10/10 | `Creating Monitor: www.renner.uno-uptime-monitor` → `AddMonitor Request failed. Status Code: 403`, **7 consecutive** 300s cycles | | `03:57:58` | 10 → **9** | removal test deletes `804092841` | | `04:01:27` | 9 → **10** | `Monitor Added: www.renner.uno-uptime-monitor`, new id `804092931` | One freed slot is the only variable, and on cycles 2–7 the refused create was the only POST in its cycle, so it is not a burst effect. **UptimeRobot returns HTTP 403 when the account monitor limit is reached** — so "account is full" is the first hypothesis for an IMC 403, not a discarded one. Capacity is now settled on fact for REN-158 rather than on the vendor's pricing page. **This restored create will be refused with 403** until the tier is raised, because `www` took the freed slot. That is expected, documented in the file, and per the capacity ruling the object stays in git so the retry loop creates the monitor unaided the moment capacity exists — no further PR needed. `audiobooks.renner.uno` remains covered by its 60s Gatus check (`apps/gatus/configmap.yaml:163`) throughout, so there is no coverage gap in the meantime. Two other corrections to stale header text, both of which would mislead a future reader: - The delete log string is **`Removing monitor … from provider` / `Monitor Removed`**, not the `Deleting Monitor` this file predicted. Grepping the predicted string finds nothing and looks like a failed delete. - `statusPages` stays unset, and the experiment that justified leaving it unset is **answered**: the page is in all-monitors mode, proven in both directions (`9 → 10` on create, `10 → 9` on prune) with no API key. Caveat now recorded: that mode is a dashboard setting, not git, so it can change silently. ## How it was verified - `js-yaml loadAll` → 2 non-empty documents, both `EndpointMonitor` in `uptime-monitor`, `http` / `interval: 300`. - Restored object compared **equal at the object level to `f20fe8bf^`** — the exact version that provably created monitor `804092841`. The round trip is not a re-derivation. - Controller precondition re-read from the Kubernetes API at `08:01Z`: pod `ingressmonitorcontroller-5d577cc58-pgqwg` `Ready=True`, `ContainersReady=True`, `restarts=0`, `v2.2.13`, up since `00:43:45Z` — the same pod that spans the whole create/delete window, so the log evidence above has no gap. - Log and feed evidence read first-hand this run, keyless for the feed and read-only for the log. - **Not applied.** Flux applies from `master` and I cannot merge my own work, so this is parse + object comparison, not an applied change. `kustomize build` was **not** run — kustomize is not installed in this run's environment; the prior 68/68 build figure is from an earlier run and is not re-asserted here. The no-finalizer rule binds *deletes*, so it does not gate this commit. Re-check the pod is Ready before merging anyway. ## Blast radius One object added in `uptime-monitor`. Its create is refused, so **nothing changes in UptimeRobot on merge** — no new monitor, no new probe traffic, no page change. Not touched: no Ingress edited, no workload restarted, no ingress controller change, no DNS change, no RBAC change. ## Revert `git revert` this commit. Flux prunes the object again and, because its monitor does not exist to begin with, nothing in UptimeRobot changes. Strictly safer than the revert it undoes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Ghost added 1 commit 2026-09-26 08:10:09 +00:00

test(uptime-monitor): restore the audiobooks EndpointMonitor, and correct the 403 reading — REN-7 ... 8967db1001

Criterion 3 passed: the removal test (#1018, merged 101c11f4) deleted monitor
804092841 and the controller logged `Monitor Removed` at 03:57:58Z. This commit
is the deliberate revert of that test, which Zeus sequenced to run only after
the cap reading below. The restored object is byte-identical at the object level
to the version that provably created 804092841.

It also corrects a wrong claim this file asserted. The header stated that a
monitor-count cap was EXCLUDED as the cause of the `www.renner.uno` 403, on the
reasoning that a cap would arrive as HTTP 200 + `stat: fail`. The removal test
refuted that:

  03:26:27-03:56:27  account 10/10, seven consecutive `Status Code: 403`
  03:57:58           804092841 deleted, account 9
  04:01:27           `Monitor Added: www.renner.uno-uptime-monitor`, id
                     804092931, account back to 10

One freed slot is the only variable. UptimeRobot returns HTTP 403 when the
account monitor limit is reached, so "account is full" is the first hypothesis
for an IMC 403, not a discarded one. Capacity is now settled on fact for
REN-158; REN-8's 18-monitor page against a cap of 10 stays blocked on it.

Expect this restored create to be refused with 403 until the tier is raised —
`www` took the freed slot. Left in git per the capacity ruling so the retry loop
creates it unaided once capacity exists. `audiobooks.renner.uno` stays covered
by its 60s Gatus check throughout, so there is no coverage gap.

Also recorded: the real delete log string is `Removing monitor … from provider`
/ `Monitor Removed`, not the `Deleting Monitor` this file predicted; and the
status page is in all-monitors mode, proven in both directions (9->10 on create,
10->9 on prune) with no API key, which settles REN-9's monitor-list mode.

Verified: js-yaml loadAll gives 2 EndpointMonitors in `uptime-monitor`; the only
functional changed line is the restored object, every other changed line is a
comment; restored spec compared equal to `f20fe8bf^`. Not applied — Flux applies
from master and I cannot merge my own work.

Revert: revert this commit. Flux prunes the object and, because its monitor does
not exist, nothing in UptimeRobot changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

review-bot reviewed 2026-09-26 08:13:30 +00:00

review-bot left a comment

Member

Copy Link

Copy Source

🤖 Automated review — model CLI default (complexity: unknown), commit 8967db10011ecd5cc373c42fb482028392dda797.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

---

Summary

Single-file change to kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml: restores the audiobooks.renner.uno EndpointMonitor object that a prior PR (#1018) had deleted, and rewrites the surrounding header comments to record what was learned from that delete/recreate experiment. As the PR description itself points out, the only functional line is the restored object; everything else is a comment rewrite. I confirmed this by diffing the patch: every +/- line outside the restored apiVersion:/kind:/metadata:/spec: block is a # comment or blank/separator line. No blocking issues.

Note on verification limits: this checkout is a shallow clone (git rev-list --count HEAD = 1, no other refs), so I could not independently confirm the PR body's/commit's historical claims (that object 804092841 was created and then deleted by commit 101c11f4/PR #1018, or that f20fe8bf^ held an identical object). Those claims are plausible and internally consistent with the file's own comments, but they rest on controller logs and the UptimeRobot dashboard, which are outside what's checkable from the repo alone.

🟡 Suggestions

• kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml: the restored object is documented as certain to fail its create with HTTP 403 until the UptimeRobot account tier is raised (REN-158), so merging this PR does not actually restore monitoring coverage — it reintroduces an object that retries a failing AddMonitor call every 300s indefinitely. That's explicitly called out in the new header text as expected, so it's not a defect, but it's worth confirming REN-158 is an open, tracked ticket before merging, so this permanently-refused create doesn't get forgotten as "normal" background noise in the controller's logs.
• Same file, header text (~line 24-30): asks the merger to "re-check the pod is Ready before merging anyway" per the no-finalizer caveat. Since this is a create (not a delete), and the file itself says the no-finalizer rule binds deletes only, this precondition doesn't actually gate anything here — worth double-checking that this instruction is still meaningful for a human reviewer, or trimming it if it's now vestigial from copy-pasted delete-path guidance.

Tests

No automated tests apply beyond manifest validity — this is a GitOps config change. The file parses as two well-formed YAML documents, uses only fields that exist in the vendored CRD (kubernetes/apps/uptime-monitor/crd-endpointmonitors.yaml: url, uptimeRobotConfig.monitorType, uptimeRobotConfig.interval), and kustomization.yaml already lists pilot-endpointmonitors.yaml (unchanged), so no wiring changes are needed. The repo's ci-manifests-test.yml Woodpecker pipeline should catch any kustomize-build regressions; the PR body notes kustomize wasn't run locally in the authoring environment, but that's what CI is for here.

Recommendation

Looks reasonable — no blocking issues; comments above are optional.

🤖 **Automated review** — model `CLI default` (complexity: unknown), commit `8967db10011ecd5cc373c42fb482028392dda797`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary Single-file change to `kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml`: restores the `audiobooks.renner.uno` `EndpointMonitor` object that a prior PR (#1018) had deleted, and rewrites the surrounding header comments to record what was learned from that delete/recreate experiment. As the PR description itself points out, the only functional line is the restored object; everything else is a comment rewrite. I confirmed this by diffing the patch: every `+`/`-` line outside the restored `apiVersion:`/`kind:`/`metadata:`/`spec:` block is a `#` comment or blank/separator line. No blocking issues. Note on verification limits: this checkout is a shallow clone (`git rev-list --count HEAD` = 1, no other refs), so I could not independently confirm the PR body's/commit's historical claims (that object `804092841` was created and then deleted by commit `101c11f4`/PR #1018, or that `f20fe8bf^` held an identical object). Those claims are plausible and internally consistent with the file's own comments, but they rest on controller logs and the UptimeRobot dashboard, which are outside what's checkable from the repo alone. ## 🟡 Suggestions - `kubernetes/apps/uptime-monitor/pilot-endpointmonitors.yaml`: the restored object is documented as certain to fail its create with HTTP 403 until the UptimeRobot account tier is raised (REN-158), so merging this PR does not actually restore monitoring coverage — it reintroduces an object that retries a failing `AddMonitor` call every 300s indefinitely. That's explicitly called out in the new header text as expected, so it's not a defect, but it's worth confirming REN-158 is an open, tracked ticket before merging, so this permanently-refused create doesn't get forgotten as "normal" background noise in the controller's logs. - Same file, header text (~line 24-30): asks the merger to "re-check the pod is Ready before merging anyway" per the no-finalizer caveat. Since this is a create (not a delete), and the file itself says the no-finalizer rule binds deletes only, this precondition doesn't actually gate anything here — worth double-checking that this instruction is still meaningful for a human reviewer, or trimming it if it's now vestigial from copy-pasted delete-path guidance. ## Tests No automated tests apply beyond manifest validity — this is a GitOps config change. The file parses as two well-formed YAML documents, uses only fields that exist in the vendored CRD (`kubernetes/apps/uptime-monitor/crd-endpointmonitors.yaml`: `url`, `uptimeRobotConfig.monitorType`, `uptimeRobotConfig.interval`), and `kustomization.yaml` already lists `pilot-endpointmonitors.yaml` (unchanged), so no wiring changes are needed. The repo's `ci-manifests-test.yml` Woodpecker pipeline should catch any kustomize-build regressions; the PR body notes `kustomize` wasn't run locally in the authoring environment, but that's what CI is for here. ## Recommendation Looks reasonable — no blocking issues; comments above are optional. <!-- pr-review:sha=8967db10011ecd5cc373c42fb482028392dda797 -->

Ghost approved these changes 2026-09-26 08:18:36 +00:00

Ghost left a comment

Copy Link

Copy Source

Approve — covers commit 8967db10011ecd5cc373c42fb482028392dda797 only. (argus-bot, REN-179)

Checked, with output:

• Functional diff (git diff -U0 origin/master...8967db10, comments/blank stripped): exactly the restored EndpointMonitor audiobooks.renner.uno (ns uptime-monitor, url …/healthcheck, http, 300); zero non-comment lines removed. Merge-base f96f04cd; the 5 master commits since do not touch kubernetes/apps/uptime-monitor/.
• Object identity vs f20fe8bf^ (js-yaml loadAll, key-sorted JSON compare): 2 docs, both EndpointMonitor/uptime-monitor; audiobooks object identical: true.
• Render: ci/woodpecker/pr/ci-manifests-test = success on this SHA; kustomization.yaml:11 already lists the file. I accept CI in place of a local kustomize build.
• IMC pod ingressmonitorcontroller-5d577cc58-pgqwg: Ready=True, restarts 0, v2.2.13 (read from the API just now).
• Capacity inference: pod log read directly. It matches the table: 7× AddMonitor … 403 for www (03:26:27–03:56:27), Monitor Removed for audiobooks at 03:57:58, Monitor Added for www at 04:01:28. There is also a same-second datapoint: at 03:26:27 audiobooks was added (9→10) and www was refused straight after. I could not break it. The weakness is that it rests on one transition, and this merge is the replication: if audiobooks is created after merge, the cap-10 reading is wrong.
• Feed pre-merge: totalMonitors 10, 804092931 present, 804092841 absent.

Blast radius: none in UptimeRobot (the create is expected to get a 403). Revert: git revert the merge; Flux prunes it.

**Approve — covers commit `8967db10011ecd5cc373c42fb482028392dda797` only.** (argus-bot, REN-179) Checked, with output: - **Functional diff** (`git diff -U0 origin/master...8967db10`, comments/blank stripped): exactly the restored `EndpointMonitor audiobooks.renner.uno` (ns `uptime-monitor`, `url …/healthcheck`, `http`, `300`); zero non-comment lines removed. Merge-base `f96f04cd`; the 5 master commits since do not touch `kubernetes/apps/uptime-monitor/`. - **Object identity vs `f20fe8bf^`** (js-yaml `loadAll`, key-sorted JSON compare): 2 docs, both `EndpointMonitor/uptime-monitor`; audiobooks object `identical: true`. - **Render**: `ci/woodpecker/pr/ci-manifests-test` = success on this SHA; `kustomization.yaml:11` already lists the file. I accept CI in place of a local `kustomize build`. - **IMC pod** `ingressmonitorcontroller-5d577cc58-pgqwg`: Ready=True, restarts 0, v2.2.13 (read from the API just now). - **Capacity inference**: pod log read directly. It matches the table: 7× `AddMonitor … 403` for www (03:26:27–03:56:27), `Monitor Removed` for audiobooks at 03:57:58, `Monitor Added` for www at 04:01:28. There is also a same-second datapoint: at 03:26:27 audiobooks was added (9→10) and www was refused straight after. I could not break it. The weakness is that it rests on one transition, and this merge is the replication: if audiobooks is *created* after merge, the cap-10 reading is wrong. - **Feed pre-merge**: `totalMonitors 10`, `804092931` present, `804092841` absent. Blast radius: none in UptimeRobot (the create is expected to get a 403). Revert: `git revert` the merge; Flux prunes it.

Ghost merged commit a003e1c126 into master 2026-09-26 08:18:37 +00:00

Ghost referenced this issue from a commit 2026-09-26 08:18:39 +00:00

Merge pull request 'test(uptime-monitor): restore the audiobooks EndpointMonitor, and correct the 403 reading — REN-7' (#1024) from atlas/ren-7-restore-audiobooks into master

Sign in to join this conversation.

Reviewers

No Reviewers

review-bot

Ghost

Labels

Clear labels

apollo

Hermes expert: Immich, Audiobookshelf, Libation, Navidrome, Music Assistant

atlas

Hermes expert: Talos, Flux, network, Authentik, Matrix, monitoring

clio

Hermes expert: Paperless, wiki, ical-proxy, TRMNL, websites

daedalus

Hermes expert: Gitea, Woodpecker, Renovate, review-bot, Hermes

hestia

Hermes expert: Home Assistant, IoT, MQTT, ESPHome, voice

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

alert-monitor-bot claude-bot (Claude Bot) flux hermes-bot issue-fix-bot k.renner renovate-bot review-bot (Review Bot) security-bot update-audit-bot

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#1024