docs/traffic-classes
60
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  claude-botandClaude Opus 5
|
3a9ea5eefe |
feat(bird): iBGP mesh over pod0, routes into table 48 (#576)
Nothing reads table 48 yet. The mesh fills it so the routing rules land on a table that is already correct, and so a mistake here shows as an empty table rather than as broken pod traffic. No address is written down twice. Each node publishes its pod0 addresses as Talos nodeAnnotations, beside the VLANConfig that assigns them; the init container reads the node list and renders from it. bird.conf holds no address at all, and the pod CIDRs are not restated either — the own-CIDR filter keys on the outgoing interface plus a maximum-length exclusion, which separates the node's pod route from cilium_host's own host address without naming a prefix. Rendering is go-template-file, not string building: kubectl renders the two templates, a field selector drops the node itself server-side, and in-cluster config removes the token and CA plumbing. kubectl comes from the Alpine package, so there is no version literal to drift out of sync and no download to trust; it only ever reads core/v1 Nodes. jq and curl are gone. The init container ends with bird -p, so BIRD's own parser gates startup: a node whose annotation is missing renders as <no value> and fails there rather than crashlooping the daemon or forming a mesh with a hole. learn all rather than learn, since plain learn skips proto kernel. direct is explicit because iBGP defaults to multihop, which would make the gateway recursive. strict bind keeps the listener off mgmt0 and ingress0. Verified against the live cluster with all three nodes annotated via --mode=try: the templates rendered the expected defines and four peer protocols, and bird -p parsed the result inside the image with exit 0 — which also confirms net.len and the include ordering. kubectl kustomize builds. Refs #576 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
bb3987ce1c |
feat(bird): add curl and jq for the peer-list init container
The per-node peer list is rendered from the Kubernetes API rather than written into the repo a second time, so the init container needs an HTTP client that can verify against the API server's CA bundle. busybox wget cannot, and the query is not going to run unverified. Using the same image for the init container and the daemon keeps one lifecycle instead of borrowing a CI image at runtime. Refs #576 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
3302e2c5cd |
fix(bird): pin the base image by digest and self-test the binary
Every other Alpine image under ci/images/ pins alpine:3.21 by index digest, and alpine-jq states the reason in the file: the resolved version drifts between restarts of an otherwise pinned image. The first revision used a bare alpine:3.22 tag. 3.22 is still right — bird2 first appears in 3.22/community — so the tag stays and the index digest is added. heating-can runs --help checks before publishing so a wrong path fails the build rather than the first pod; bird documents --version, so it gets the same guarantee. Refs #576 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
3792485c2f |
feat(bird): image for the pod-network BGP mesh (1/3) (#576)
Pod east-west follows the peer's Kubernetes InternalIP on mgmt0, and Cilium offers no way to point it elsewhere — established by test in #838, and directRoutingDevice (#837) does not do it. Policy routing does: a rule on the pod source prefixes into a table whose routes carry pod0 next-hops, proven on k8s-1 by counting 4647 packets onto pod0. Those routes have to be generated, not written down: PodCIDRs are Kubernetes-assigned and do not survive a rebuild in place. BGP between the nodes over pod0 generates them, with the session source as the next-hop. BIRD rather than FRR. FRR's docs describe 'set table' only as setting a BGP table identifier, never a kernel table; binding bgpd to an address implies --no_kernel and would suppress the route installation the design exists for; and a gatewayless IPv6 device route has no next-hop for its route-maps to match, so the own-CIDR filter could not be uniform. BIRD documents 'kernel table', 'strict bind' without touching the kernel protocol, and an 'ifname' filter attribute that discriminates the local route from the learned ones in both families. No maintained standalone BIRD image exists: pierky/bird self-declares as lab-only and amd64-only, calico/bird is a Calico-internal artifact expecting confd, slipmesh/bird is five weeks old, srl-labs and ibhde are lab-scoped or on BIRD 1.6. A custom controller would have needed an image of its own anyway, so the build is common to both routes. Image only. The DaemonSet is 2/3 and the routing rules 3/3, so the table fills before anything reads it. Refs #576 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Fable 5.1
|
6b0617b4a7 |
chore(ci): bump CI tool pins to match the deployed cluster
From the update audit (#426): the tools CI drives the cluster with trailed what the cluster runs. ci/images/k8s/Containerfile - talosctl v1.12.6 -> v1.13.9, the Talos version on the nodes. An older talosctl can misjudge or reject config generated for a newer machine. - kubectl v1.36.2 -> v1.36.3, the cluster's Kubernetes version. - flux CLI 2.9.2 -> 2.9.5, the current 2.9.x. Checksums verified against the upstream release assets (talos sha256sum.txt, flux checksums.txt) and against a local download. .woodpecker/ci-manifests-test.yml - woodpecker-cli v3.17.0 -> v3.18.0, the version the server and agent already run; the lint step was one minor behind its own server. Merging triggers build-ci-images (path ci/images/**), which publishes ci-k8s:<date>. The six pipelines pinned to ci-k8s:2026-08-10 move to that tag in a follow-up once it exists. Refs #426 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
claude-botandClaude Fable 5.1
|
2c47c68a8b |
feat(netbird): carry the WAN client address through the passthrough edge
TLS-passthrough services hand Traefik a raw TCP stream, so every WAN request now logs the routing peer's pod IP (10.244.4.64/.112 in the access log) instead of the client. CrowdSec's detection parses exactly that log; with the HTTP-mode services gone it would attribute every external request to two pod IPs, and a re-armed bouncer would repeat the 2026-06-27 mass ban. The controller now sets options.proxy_protocol on every target (PROXY protocol v2, TCP/TLS modes only per NetBird) and treats a service without it as out of date, so the eleven existing services are updated on the next pass. Traefik's websecure entrypoint accepts the header from the pod CIDRs, mirroring forwardedHeaders.trustedIPs. Traefik takes connections with or without a header from a trusted IP, so mesh and LAN traffic that carries none is unaffected and the Traefik change can land before the controller image. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
claude-botandClaude Fable 5.1
|
c9ec97f6b6 |
feat(netbird): controller for reverse-proxy services from public Ingresses
netbird-proxy-services reconciles the NetBird reverse-proxy services that nobody managed so far: one TLS-passthrough service on :443 per host of every Ingress carrying dns.renner.uno/public, targeting Traefik ClusterIP through a subnet resource over the service CIDR. SNI and Host reach Traefik untouched, the proxy terminates nothing, no domain resource is involved -- so the dns-interceptor pinning bug that wedges the proxies on every routing-peer loss has nothing to pin on. Ownership is derived from NetBird, not from local state: managed iff mode is tls and the target is the controller subnet resource. The ten hand-made HTTP/domain services are reported as held and never touched; migration is deleting one of them, the controller creates the passthrough on its next pass. The PAT is read from OpenBao per pass through a read-only role (policies/netbird-proxy-services.hcl, applied once by hand like the rotator roles). Standard library only; unprivileged image built alongside heating-can. Validated against the live management (ONCE=1 through the rotator role): network, resource, router and policy created, 10 hosts held, one host that had no service at all (llm.renner.uno) created and answering behind its Authentik gate; second pass 0 created / 0 updated / 0 deleted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
26ae750219 |
fix(rebuild): images and CNI pods must survive a cluster without Gitea
Gitea runs inside the cluster, so during a cold rebuild every image that only exists at gitea.renner.uno is unpullable — a rehearsal died with the cni-plugins installer in ImagePullBackOff, no macvlan binary, no ceph-san attachment, CephCluster stuck at "Configuring Ceph Mons", OpenBao waiting forever on a PVC. The failure surfaced four layers from its cause. The CI images move to konairius/, published by the build pipeline to Gitea AND ghcr.io with one digest. The manifests keep pointing at Gitea; the node-level fallback (machine.registries.mirrors, written by bootstrap.sh at install time) tries Gitea first and ghcr second, and containerd keeps the repository path when it falls through — so the same path resolves on both. Steady state never leaves the network. The Multus, whereabouts and cni-plugins-installer DaemonSets also gain priorityClassName system-node-critical. Upstream ships them at priority 0: on a rebuild two nodes sat at 99% memory requests when the DaemonSet rolled, both multus pods stayed Pending, and every pod needing a secondary attachment waited on an annotation that could never come — with nothing reporting why. Rollout effect on the live cluster: the three CNI DaemonSets restart once (existing pods keep their networking; only new attachments wait), and the three init containers pull the new tag on their next pod restart. The ghcr side is proven by the Hetzner throwaway runs, which pull through the fallback; eyeball gitea.renner.uno/konairius packages before merging. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
e3dea169b0 |
fix(heating-can): give supervisord a control socket
The image shipped without [unix_http_server], [rpcinterface:supervisor] or [supervisorctl], so supervisorctl could not connect: Error: .ini file does not include supervisorctl section That means no way to query or restart an individual service inside the container -- docker logs was the only introspection. Per-service lifecycle management was the whole argument for running a supervisor instead of one process per container, and the first revision did not actually deliver it. Unix socket only, inside the container. Nothing outside needs it, and this container runs with host networking, so a TCP listener would be exposed on the host. The build-time self-test now asserts the section exists and supervisorctl runs, so this cannot regress silently the way it shipped. NOT fixed here, because my diagnosis was wrong: the repeated "mqtt broker disconnected. reason_code = Unspecified error" is not a client ID collision. E3onCAN has no -mcid option at all -- verified against the pinned source -- and generates its own ID; the native broker log showed E3onCANclient.py-e380-7f847281 alongside open3e-edge, so the two never collided. I nearly "fixed" it by passing a flag that does not exist. Telemetry is flowing (each reconnect succeeds), so this is churn rather than an outage, but the cause is still unidentified. The next step is the broker's own log on edge-1 -- mosquitto records the disconnect reason, which the client side reports only as "Unspecified error". Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
33d92e133c |
fix(audit): track the heating-can image pins, not the deleted Ansible vars
#476 merged first, which removed edge_open3e_pkg, edge_e3oncan_repo and edge_e3oncan_version in favour of the image. The audit entry added here still told the agent to read those, so it would have gone looking for variables that no longer exist and reported nothing -- the same silent non-auditing the entry was written to prevent. Repointed at where the pins actually live now: OPEN3E_REF and E3ONCAN_REF as ARGs in ci/images/heating-can/Containerfile. The awkward specifics carry over unchanged, since they are properties of the upstreams rather than of how we package them: open3e publishes no tags so the comparison must be against commits, and E3onCAN's pin is an untagged commit later than its newest tag. Added a note the previous version could not have made: heating_can_image is :latest, so the node follows whatever CI last built. The Containerfile ARGs are what actually fix the versions, and bumping one is what requires a rebuild -- otherwise a reader could reasonably conclude the tag is the pin. The pin commit itself is now a no-op against master and survives only as the audit registration; its conflict on heating-can/defaults resolved in favour of the image. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
dd6553ab07 |
fix(heating-can): pin open3e, and register both CAN services with the audit
open3e was installed from an unpinned git URL, so the heating bridge
silently tracked upstream main: whatever HEAD happened to be on the day a
converge ran was what ended up driving the heat pump. Now pinned to
6553c104, which was simultaneously upstream HEAD and the copy already
installed on edge-1, so the pin froze the running state and changed nothing.
A commit rather than a tag because open3e publishes none -- its tags API
returns an empty array. The package self-reports 0.7.5 at that commit, which
is useful context but is not a tag.
Honest about what this does NOT fix: pip still re-clones and rebuilds a VCS
requirement even when the SHA is pinned, because it cannot compare an
installed VCS dist against a requested URL without fetching it first.
Verified directly on the host:
pip install --dry-run "git+...@6553c104"
-> Cloning ... Resolved ... to commit 6553c104
Installing build dependencies: started
So the task still reports changed and still restarts open3e on every
converge. Pinning buys deliberate versioning, not idempotence.
Pinning without an update path only converts churn into silent rot, so both
services are now registered with the weekly update-audit -- which had been
auditing neither, despite E3onCAN having been pinned since July. The audit
prompt warns that anything not listed is silently not audited, and these two
were exactly that.
The new scope entry records the awkward specifics so the agent does not
report normal states as problems: open3e has no tags at all, so it must
compare commits against the default branch; E3onCAN does tag, but sparsely,
and its pin sits between the newest tag and HEAD.
Also corrected while there: edge_e3oncan_version was commented "# V0.5.2",
which does not exist upstream. The repo's only tag is v0.5.1 (6f43b3fb), and
461dcbdf is a later untagged commit from 2026-06-21, "Fixed: MQTT client ID
collision caused stale retained values; added publish error logging".
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
claude-botandClaude Opus 5
|
f9848d2bf5 |
fix(heating-can): correct the overlay path, secret mode and writable CWD
Review on #476 found a real regression plus three things worth fixing before this reaches edge-1. Blocking: edge_open3e_devices.vx3.dpList still pointed at the old host path /opt/edge-host/open3e/vx3plus.py. The compose file mounts that overlay at /config/vx3plus.py and nothing mounts anything under /opt/edge-host inside the container, so open3e would have looked for it at a path that does not exist -- silently falling back to the stock vx3 datapoint list and dropping the battery-module DIDs (vx3.1577-1582 and friends) the overlay exists to add. Now /config/vx3plus.py. Worth noting the build-time self-test could not have caught this: it proves the binaries execute, not that devices.json resolves against the mounted files at runtime. Writable working directory. WORKDIR was /opt, which the COPY layers create root-owned, while the process runs as open3e. udsoncan writes its log file into the CWD -- the native install gave open3e's directory `owner: open3e` for exactly that reason, and I dropped that when containerising. Now a dedicated /var/lib/open3e, owned by the service user. Secret mode. MQTT_PASSWORD was inline in docker-compose.yml, which is rendered 0644. The env file this replaced was 0600, and the sibling roles on this host keep secrets in a 0600 file beside a world-readable compose file. Restored: credentials move to heating-can.env at 0600, referenced via env_file. edge_e3oncan_mqtt_format had become dead -- both services were fed MQTT_FORMAT from edge_open3e_mqtt_format. The two default to the same string so behaviour was unchanged, but overriding it did nothing. Wired through as E3ONCAN_MQTT_FORMAT, which supervisord passes to e3oncan only. Also from review: dropped the registered-but-unread heating_can_native_stopped, and the migration now removes the emptied {{ edge_base_dir }}/open3e and /e3oncan directories rather than leaving them behind. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
fac46aa7ac |
fix(heating-can): self-test at build time instead of a CI step
The smoke-test step I added errored the whole pipeline before any step ran. It used the freshly built image as its own step image, but the Gitea registry is private: the build steps authenticate kaniko for PUSH, nothing gives the runner credentials to PULL. So the step could never have worked. Moved into the Containerfile as a build-time RUN, which is better anyway -- it needs no registry credentials, and a broken image is never published rather than published and then flagged. It asserts what the previous revision got wrong: both services actually execute, and supervisord's config carries both programs. That is exactly the failure that shipped green -- supervisord pointed at /opt/e3oncan/src/E3onCANcollect.py, which does not exist (the file is at the repo root), so the image would have run open3e while e3oncan crash-looped and the E380 meter went silently missing. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
6fb760edae |
fix(heating-can): correct the e3oncan path, and smoke-test the image in CI
The image shipped a path that does not exist. supervisord ran
`python /opt/e3oncan/src/E3onCANcollect.py`, but that file is at the repo
ROOT -- there is no src/ directory. The `src` came from the old Ansible task,
which cloned into `{{ edge_base_dir }}/e3oncan/src`; that was the clone
DESTINATION, not part of the upstream layout, and I carried it into the
Containerfile without checking. Confirmed against the upstream tree at the
pinned commit: E3onCANcollect.py sits at the root alongside requirements.txt,
which is why the build succeeded while the runtime path was wrong.
So the image built green and would have started open3e and failed e3oncan on
every restart -- the E380 meter silently absent, with supervisord retrying
forever.
Nothing caught that, because nothing ran the image. Now CI does: a smoke-test
step uses the freshly built image as its own step image and asserts both
services execute and supervisord's config carries both programs.
It cannot exercise the CAN bus or the broker, neither of which exists on a
runner, so it verifies exactly what is verifiable without hardware. That is
still the difference between "built" and "works" -- and it belongs on a
throwaway runner, not on the production edge host, which is where I wrongly
went looking for this the first time.
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
claude-botandClaude Opus 5
|
96fcb426aa |
build(heating-can): one container, both readers supervised inside it
Reverts my own reinterpretation. The instruction was to put both in one container because they belong together; I argued against it on crash-isolation grounds, then read the correction "two services, one image" as agreement with my version rather than as a restatement of the original. It was not. A container runs one PID 1. With two processes that PID 1 is a supervisor, which is how a container handles multiple services. supervisord now runs both readers: it restarts either on its own if it dies, reaps children and forwards shutdown -- none of which a bare process as PID 1 can do for a sibling, which was the actual answer to the concern I raised. Also gone: the dispatch entrypoint and the argv in the compose file. Both were the same mistake in different places -- the invocation living outside the artifact. open3e's and E3onCAN's command lines are an implementation detail of this image now; supervisord builds them from environment variables it expands itself. Nothing outside the image knows either CLI, so an upstream flag rename cannot break the compose file or an Ansible template. The compose file supplies configuration and nothing else: one service, one container, environment only. The credential env file is dropped -- the credentials are environment already. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
56a9859e50 |
refactor(heating-can): run the image, install nothing on the node
Nothing is installed on the host any more. The image is built in CI with both sources pinned, published to the registry, and edge-1 pulls and runs it. That is the whole point of building it -- installation happens at build time, once, not on every converge. Removed from the node: both venvs, the E3onCAN checkout, and the two systemd units. Kept and still rendered: devices.json, the vx3 datapoint overlay and the broker credentials, now bind-mounted into the containers rather than read from a venv working directory. Two services from one image, as before: open3e drives the Vitocal/VX3 over UDS, e3oncan listens passively for the E380 meter. Separate containers so either can crash and restart without a supervisor as PID 1. The migration stops AND masks the old units before the containers start. Masked rather than disabled because two open3e instances issuing UDS requests on one CAN bus is the failure mode to avoid, and a stray `systemctl start` or leftover dependency should not be able to bring the native one back alongside the container. No missing-device tolerance here, unlike the zwave and meter roles: can0 is this host's purpose. If it is absent the heating telemetry is down and the run should say so rather than pass quietly. heating_can_image replaces edge_open3e_pkg / edge_e3oncan_repo / edge_e3oncan_version. The source pins move into the Containerfile, so what is deployed here is a tag -- and `docker compose up -d` on an unchanged tag is a genuine no-op, which is what removes the restart-per-converge that pinning the pip URL could not. Verification is limited and worth stating: plays/install-heating-can.yml lives in #475 and is not on this branch, so only the master-side play syntax-checks here. A --check run cannot exercise this at all -- command tasks are skipped under check mode, so neither the pull nor the compose up would run. It cannot be meaningfully verified until the image has been built and #475 is merged. Depends on #475 (the capability-split playbooks, and the MQTT contract moving to group_vars). This branch is cut from master, so heating-can's defaults will conflict there; #475 should merge first and this be rebased onto it. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
4df1018e05 |
build(heating-can): make both services first-class commands in the image
The image had no ENTRYPOINT and no CMD, with a comment claiming that was deliberate because the commands are rendered by Ansible. That was rationalising a defect. An image with nothing runnable is not a self-contained artifact. The knowledge of how to START each service leaked into Ansible, and the two were not even symmetric: open3e installs a console script via pip, while E3onCAN is an unpackaged script in a checkout, so the compose file would have had to carry `python /opt/e3oncan/src/E3onCANcollect.py` -- an implementation detail of this image's internal layout, in a file that has no business knowing it. Move the checkout and every caller breaks. E3onCAN now gets a wrapper on PATH, so both services are invoked the same way and `docker run <image> e3oncan --help` works. CMD prints what the image provides and exits 64 (EX_USAGE) rather than silently doing nothing, so `docker run <image>` is self-describing. Which service to run, and its arguments, stays the caller's choice -- the compose file renders those, because the datapoint list and broker settings live there. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
0e4d516722 |
build(heating-can): image for the CAN readers, replacing pip-into-venv
Builds one image containing open3e (Vitocal heat pump, Vitocharge VX3) and E3onCAN (E380 grid energy meter), both pinned in the Containerfile. Why: both were pip-installed into venvs on edge-1, open3e straight from an unpinned git URL. pip cannot compare an installed VCS dist against a requested one without fetching it, so `pip install` re-cloned and rebuilt on every Ansible converge -- reporting changed, firing the restart handler, and putting a gap in heating telemetry each time. Pinning the SHA does not fix that; verified directly on the host that pip still clones and rebuilds a pinned VCS requirement. A tagged image does: `docker compose up -d` on an unchanged tag is a genuine no-op. It also gives the weekly update-audit a normal artifact instead of two awkward special cases. open3e publishes no tags at all, and E3onCAN is pinned to an untagged commit sitting between its newest tag and HEAD -- both of which the audit currently has to be told about in prose. One image, two services. They read the same physical installation over the same bus and are versioned together, so they are one artifact; but they will run as two containers from it, so either can crash and restart on its own without a supervisor as PID 1. Fixes a long-standing blind spot in passing: e3oncan.service set PYTHONUNBUFFERED=1 and open3e.service did not, so open3e's stdout was block-buffered and effectively never reached the journal or Loki. The image sets it for both. Build follows the existing kaniko pattern exactly -- path-scoped `when:` so only this image rebuilds, date tag plus latest, pushed to the Gitea registry alongside the ci-* images. Deliberately artifact-only. The heating-can role still deploys venvs and systemd units; moving it to a compose project is a separate change, once this image exists and has been pulled and run once. Rewriting the role that drives the heat pump against an image that has never been built would be the wrong order. Co-Authored-By: Claude Bot <claude-bot@renner.uno> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
8a580e341d |
docs(ci): record why woodpecker-cli cannot be enrolled in image-automation
I proposed widening ImageUpdateAutomation's `update.path` to `./` and adding a woodpecker-cli ImagePolicy so automation would own the lint pin the way it owns the server and agent. Checking that before building it killed the plan, and corrected two things I had asserted without evidence. WRONG #1 — my reason for hesitating. I claimed the 7 ansible-vault encrypted `.yml` files would break the walk and take all 33 automated images down with them. That was a guess about kyaml I never verified. Objects that are not Kubernetes manifests are SKIPPED, not fatal. WRONG #2 — and this is the one that matters: the plan cannot work at all. Image-automation only patches YAML that is a Kubernetes manifest. Maintainer, fluxcd/flux2#2826: "Flux can only patch Kubernetes manifests. This means the YAML manifest must have `apiVersion` and `kind` fields." The docs agree — markers go "in any Kubernetes custom resource stored in Git". A Woodpecker pipeline file is plain YAML with no apiVersion/kind, so a marker in it is silently ignored. That is a property of the FILE, not of `update.path`. Widening the path buys exactly nothing and would drag 197 unrelated ansible/esphome YAMLs into the walk. So no policy, no second automation, no path change — all three would have been motion that looked like a fix and changed nothing. The failure mode would have been the quiet kind this repo keeps getting bitten by: a marker sitting in the file looking maintained, never once updated. What actually closes the gap, given the pin must stay manual: * README gains a "Not enrolled (structurally impossible)" section holding the quote and the explicit "do not try widening the path" warning, so the next person does not re-derive this. * update-audit item 5 now covers `.woodpecker/*.yml` step images and is told to flag woodpecker-cli whenever it trails the DEPLOYED SERVER, not merely the newest CLI release — the server auto-bumps and the CLI does not, so that skew is the thing worth reporting. Also told to compare only the vX.Y.Z part (the tag carries `-alpine`; the plain tag is scratch with no shell and would break the step) and to ignore the `next-*`/`pull_*` dev tags, which are the newest by date on that repo and are not releases. * the step comment states the real constraint instead of my earlier wrong one, which told the reader that widening the path was the fix. Verified: v3.16.0-alpine is the newest 3.x alpine release tag, and matches the deployed server v3.16.0. Config still parses; step image unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
a6effdd4cf |
fix(paperless): drop runtime apk add from fetch-bridge-cert
Owner directive: tools belong in the image, not installed at container start. This init container ran `apk add --no-cache openssl` on EVERY paperless pod start. Unlike the multus CNI installer, this one genuinely needs an image: the openssl CLI is absent from the alpine base (verified — the base carries the libraries but not the command), and busybox has no `s_client` equivalent, so the STARTTLS certificate capture cannot be done with what is already there. Adds ci/images/alpine-openssl, named for what it is rather than for this consumer, so anything else needing s_client/x509 from a shell script can reuse it. Script logic is unchanged — only the apk add line is removed and the image swapped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
d62c5da5a0 |
fix(multus): include dhcp in the cni-plugins build assertion
Review point on #447. The build-time check asserted macvlan, static and host-local were executable but omitted dhcp, which the sibling cni-dhcp-daemon DaemonSet runs and which its own comment says ships from this installer ("the dhcp binary ships from the sibling cni-plugins-installer DaemonSet ... whole tarball extracted"). Verified that dependency in kubernetes/infrastructure/multus/ cni-dhcp-daemon.yaml before changing anything, rather than taking the review's word for it. Consequence of the omission: since the whole /plugins directory is copied regardless of what the loop checks, nothing is broken today — but the "fail loudly if upstream restructures the archive" guarantee only covered three of the four binaries this repo depends on. Upstream dropping dhcp while keeping the others would have built green and surfaced later as a broken DHCP daemon on whichever node rolled first. Confirmed all four are executable in the already-published image, so this tightens the assertion without changing what ships. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
8ac30b0ef7 |
fix(multus): bake CNI plugins into an image, drop the node-side fetch
Supersedes the wget-instead-of-curl approach in #446, which removed the `apk add` but left the real problems in place. The installer fetched the containernetworking/plugins release tarball from GitHub on the node and extracted it into /opt/cni/bin with NO integrity verification of any kind. That directory is executed as root in the CNI path on every node, so a tampered or MITM'd artifact would have obtained privileged execution cluster-wide with nothing to detect it. It also put github.com on the critical path of every node rebuild. Now the binaries are baked into ci/images/cni-plugins, where the download happens once and the sha256 IS verified, and the installer is a local file copy with zero network calls (verified: no wget/curl/apk/https in the rendered script). The pinned digest was cross-checked against upstream's own published cni-plugins-linux-amd64-v1.5.1.tgz.sha256 asset, not merely recorded from a local download — a download can be tampered with, so comparing against what upstream states is the point: 77baa2f669980a82255ffa2f2717de823992480271ee778aa51a9c60ae89ff9b Verified the gate actually gates: with the correct digest the layer logic downloads, verifies, extracts and asserts macvlan/static/host-local are executable; with a deliberately wrong digest `sha256sum -c` FAILS and the extract step is never reached. Install script changes beyond dropping the fetch: - Copies unconditionally instead of the old "already installed?" guard. The guard existed to avoid re-downloading; a local copy is cheap, so copying every time means a node always matches the version pinned in the manifest rather than keeping whatever it first installed. - Copies to a temp name in the same directory then rename(2)s over the target. `cp` onto a plugin the kubelet is currently executing can fail with ETXTBSY; rename never does and is atomic, so a concurrent CNI invocation sees either the old or the new binary, never a partial one. - LICENSE and README.md are no longer installed into /opt/cni/bin; the old tar extraction put them there. The chown 0:0 is kept but re-framed as a REPAIR. Its original cause is gone — extraction applied the archive's `./` uid/gid (1001/127, the github-actions runner) to the destination, and `cp` into an existing directory does not touch its ownership. It still runs because nodes provisioned by the old installer may already carry the wrong ownership, which breaks Cilium's apply-sysctl-overwrites init container. Verified end-to-end with the rendered script, including that repair path: pre-seeded owner 1001:127 -> 0:0 after run fresh install -> host-local/macvlan/static present, macvlan executes and reports "CNI macvlan plugin v1.5.1", exit 0 re-run -> idempotent, exit 0, 18 plugins, no .tmp files left Note the image carries ~80MB of extracted plugins (18 binaries), matching what the old installer put on each node. All are kept rather than trimming to the few in use, to avoid removing plugins already present on nodes as a side effect of this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
c3280310b5 |
refactor(zigbee2mqtt): use alpine-jq image, drop apk add from seed-data
Owner directives, both applied here: - "z2m-seed is a shit name for the image — name it for what it is, not for how it is used." Renamed to alpine-jq. Nothing in it is zigbee-specific; it is alpine plus jq, useful to anything that needs to read JSON from a shell script. The Containerfile now describes it that way, with the zigbee2mqtt case recorded as its first consumer rather than its identity. - "You do not need to pin the digest for our own images." Referenced by tag. This also collapses what was going to be two PRs into one, since the digest no longer has to exist before the Deployment can name it. The seed-data init container now uses that image instead of `apk add --no-cache jq` on every pod start. Two consequences of dropping digest pinning, both good: - The whole --image-name-with-digest-file / "PIN THIS:" apparatus is gone from the pipeline. That machinery existed only to hand a digest to a follow-up commit, and it is what broke every pipeline in the repo (#443) via an unquoted ": " in a YAML scalar. Less surface, same outcome. - :latest means Kubernetes defaults imagePullPolicy to Always, so a rebuilt image is picked up without editing the manifest. Kept the fail-loudly validation, since a wrong network_key costs the whole Zigbee network: explicit checks for an absent key (jq prints "null" for a missing path, so that is checked separately rather than being reported as merely "non-hex"), non-hex characters, and a length other than 32. Verified by running the manifest's RENDERED script, against the real coordinator backup and synthetic fixtures. The alpine-jq tag does not exist until this merges and builds, so the run used the existing image of identical content (alpine 3.21.7 + jq 1.7.1, verified present): real backup -> exit 0, network_key line byte-identical to the live value (sha256 27c5c77a945bcecc) non-hex key -> exit 1, "contains non-hex characters" 4-char key -> exit 1, "length 4, expected 32 hex chars" absent network_key -> exit 1, ".network_key.key absent from ..." device link_key also present in the doc -> correctly ignored; the network_key is used Validated with `woodpecker lint` (not just `yq`, which is what let #443 through), plus kustomize build and a server-side dry run. No stale z2m-seed references remain. NOTE: the now-orphaned z2m-seed package is still in the registry and should be deleted; that needs package-delete rights this account does not have. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
0d7fecad8d |
build(ci): add z2m-seed runtime image (alpine + jq)
Step 1 of 2 toward removing `apk add` from the zigbee2mqtt seed-data init container, which currently installs jq on every pod start. Owner directive: baking tools into a layered image is fine, adding them at runtime is not. This commit only adds the image and its build step; nothing consumes it yet, so it is inert on merge. The Deployment cannot be switched in the same commit because runtime images in this repo are pinned by DIGEST (as the secret-rotator CronJobs are), and the digest does not exist until this has built and pushed. The build therefore writes it out via kaniko's --image-name-with-digest-file and echoes it as "PIN THIS:", so the follow-up commit can copy the reference straight from the build log without a separate registry query. Why a real JSON parser rather than busybox sed/awk: "key" occurs 12 times in the actual backup — once per device with an APS link_key — and the FIRST occurrence is a device key, because "devices" sorts before "network_key". An extraction that is not correctly anchored yields a device link key, herdsman then forms a NEW network, and every device drops off. This was reproduced while prototyping the awk approach, which is what makes jq worth an image here rather than clever one-liners in the manifest. Follows the existing conventions: Containerfile alongside the other ci/images entries, base pinned by digest to the same alpine:3.21 the ci-k8s image uses, kaniko build gated on its own path so unrelated pushes do not rebuild it, and date + latest tags pushed as the other images do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 5
|
39a9519ed3 |
fix(triage): give the CI diagnostics SA read access to Loki
The alert triage agents could reach Prometheus and Alertmanager through the
apiserver service proxy, but not Loki:
kubectl auth can-i get "services/loki:3100" --subresource=proxy -n monitoring \
--as=system:serviceaccount:monitoring:ci-diagnostics -> no
(same check for kube-prometheus-stack-prometheus:9090 -> yes)
That gap has a sharp edge. A KubeJobFailed-class alert can only be cleared by
DELETING the failed Job — the rule's own annotation instructs this — and the
deletion destroys the pod and its logs. So the agents would inspect a failed
Job, find no live pod, declare the root cause "unconfirmable", and then delete
the last copy of the evidence.
The evidence was never actually gone: Alloy ships all pod logs to Loki with
retention_period: 2160h (90d). Issue #380 declared velero/restore-test-29740440
"probably unconfirmable" and its full failure output was in Loki the whole time
(FAIL: 3 — apps restore phase + two PVC parity checks).
Adds `loki` / `loki:3100` to the existing service-proxy Role (read-only GET,
same bare + :port-qualified pattern the other entries use, verified against the
live Service which exposes http-metrics:3100), and teaches both agent prompts
the query recipe so "the pod is gone" stops being accepted as a dead end.
No new privilege class: the agents already hold pods/log cluster-wide. This
mainly extends their reach to *deleted* pods.
Refs #427, #380
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
claude-botandClaude Opus 4.8
|
1992fabb12 |
fix(ci): make model-picker actually tool-less; gate tests on PRs
Addresses the
|
||
|
claude-botandClaude Opus 4.8
|
44a429685c |
fix(ci): round-3 pr-review hardening — REVIEW_TOKEN, no-die selection, tests
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the
|
||
|
claude-botandClaude Opus 4.8
|
a7890e0aa2 |
test(ci): extract + unit-test model validation; cut triage cost; doc nits
ci/woodpecker/pr/pr-review Pipeline was successful
Addresses the remaining pr-review items. - Extract the untrusted-model validation out of the pipeline YAML into scripts/ci/valid-model.sh (single safe-token + catalog-membership gate) and reuse it from both pick-model.sh and pr-review.yml, so the security-critical logic lives in one tested place. Add scripts/ci/test-model-selection.sh, an offline fixture test (no network / claude / Woodpecker) that pins the behaviour: valid ids pass; hallucinated, empty, space-, newline-, leading- dash-, and metachar-injected values are rejected; empty catalog rejects all. This is the regression test for the flag-injection fix. - pr-review triage cost: hand the triage the COMPLETE changed-file list (files.txt) plus a diff capped at 4000 lines (diff.triage.patch) instead of the full up-to-20k review diff, so classifying a PR on the default model stays cheap. stats.txt still carries the true totals; triage-prompt.md updated. - Doc nits: note the new helpers are POSIX `sh` (not `bash` like the older scripts/ci/*.sh), and that fetch-models.sh's json mode is a debug affordance. Deferred with rationale (also in the PR thread): - Model id in the four cron jobs' posted artifacts: selection (including the fallback-to-default) is already logged per run in the CI output, the operational surface for this; threading it through four different posted formats is a separate observability follow-up. - --settings sandbox on the picker: it is granted no tools (--allowedTools ""), so there is nothing for a Read-deny sandbox to constrain -- the other agents' sandboxes exist to scope tools they DO have. - Pagination past limit=1000: that is the API's max page size; >1000 models is not a real scenario, and the has_more warning turns even that into a visible log line rather than silent truncation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
f333dfd606 |
fix(ci): block flag injection via agent-supplied model id (pr-review)
ci/woodpecker/pr/pr-review Pipeline is running
The pr-review triage agent reads the attacker-controlled PR diff and writes
triage.json; its Write tool is not path-scoped, so a prompt-injected diff can
make it emit any `model` string. The catalog validation used
`grep -qxF -- "$MODEL"`, and grep -F treats a MULTI-LINE operand as
newline-separated patterns: a value like
"claude-sonnet-5\n--dangerously-skip-permissions" matches on its first line,
passes validation, and -- via the unquoted `${MODEL:+--model $MODEL}` expansion
-- word-splits into an extra argv entry, injecting a CLI flag into the review
agent's own `claude` invocation (defeating its --allowedTools allowlist and Read
sandbox, with CLAUDE_CODE_OAUTH_TOKEN in that process's env).
Removing map_model() (which returned one of three literal ids) is what opened
this; restore the invariant with a single-token guard before the expansion:
`case "$MODEL" in ""|*[!A-Za-z0-9._-]*) MODEL= ;; esac`. Verified it turns the
newline/space/hallucinated payloads into a plain `claude -p` (CLI default) while
still passing real ids. Mirrored the guard in pick-model.sh as defence in depth
(its grep -oE already keeps CHOICE single-line, so it was not exploitable).
Also from the review:
- pr-review.yml: correct the HOMELAB comment -- the fetch-models.sh call is
env -u REVIEW_TOKEN'd, so CLAUDE_CODE_OAUTH_TOKEN (not REVIEW_TOKEN) is what
branch-supplied shell there can see.
- pick-model.sh / triage-prompt.md: stop asking the picker to weigh "cost" the
catalog does not expose; reason from the visible capability signals (context /
max_output / effort|thinking|code_exec) as the scale proxy instead.
- fetch-models.sh: drop 2>/dev/null on curl so its -S error reaches the log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
claude-botandClaude Opus 4.8
|
b329c432c5 |
fix(ci): address pr-review feedback on dynamic model selection
ci/woodpecker/pr/pr-review Pipeline was successful
- fetch-models.sh: request ?limit=1000 so older/cheaper models are not dropped by the API's 20-item default page; warn on has_more. Add a --table mode so the catalog format lives in ONE place (pick-model.sh and pr-review.yml now consume it instead of each re-deriving the jq table). Clean up the HTTP-000 double-print and surface jq parse errors instead of hiding them. - pick-model.sh: run the picker with GITEA_TOKEN stripped from its env and without acceptEdits (it is granted no tools); consume fetch-models.sh --table. - pr-review.yml: print model `CLI default` instead of an empty backtick pair on the fallback path; document that the fetch-models.sh call runs branch-supplied shell (not a new exposure -- Woodpecker already runs the pipeline YAML from the PR branch). - triage-prompt.md: read stats + the changed-file list first and pull the full diff only when needed, so the triage (now on a capable default model) does not read a 20k-line diff just to classify a trivial PR. Verified against the live catalog: all 11 currently-listed ids run under `claude --model`, and the capability fields (context/effort/thinking/...) are present in the OAuth /v1/models response -- so the reviewer's blocking concern (a listed-but-CLI-unusable id) has no live trigger, and the fallback-to-default path still covers it if that ever changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
ffe3be4b54 |
feat(ci): select agent model from the live catalog, not hardcoded ids
ci/woodpecker/pr/pr-review Pipeline was successful
The pr-review, update-audit, security-scan, alert-monitor and issue-fix pipelines each pinned a dated model id (--model claude-sonnet-5, plus pr-review's haiku/sonnet/opus map_model allowlist). Those go stale every time a new model ships and bake in assumptions about which model families exist, so a newly released model could never be chosen without a code edit. Replace all of it with selection driven only by what is available right now: - scripts/ci/fetch-models.sh queries the catalog the CLI itself uses (Anthropic GET /v1/models, via the existing CLAUDE_CODE_OAUTH_TOKEN and the required anthropic-beta: oauth-2025-04-20 header) and emits each model with its objective capability signals -- context, max_output, effort, thinking, code_exec, release date -- with no id patterns and no family names. - scripts/ci/pick-model.sh lets a triage agent choose the best-fit id from that catalog for a one-line task description, weighing every listed model on its merits. pr-review's existing per-PR triage agent now does the same, reading the live catalog instead of emitting a fixed haiku/sonnet/opus tier. No model id -- and no model family -- is baked into any pipeline. A model released tomorrow, of any family, becomes eligible the day the API lists it. The bootstrap selection call runs on the CLI default model, and if the catalog is ever unreachable --model is simply omitted (CLI default) rather than falling back to a named model, so a run never dies over model selection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
a675bf6cf0 |
chore(ci): make ci-claude reproducible and keep it on latest (#351)
ci/woodpecker/pr/pr-review Pipeline was successful
The ci-claude image installed a fixed, pinned Claude Code version and every consumer pipeline pinned a stale dated image tag. Two problems: - The npm install layer changed on every build (install-time mtimes + the npm cache), so it never deduped in the Gitea registry — the single largest orphan pattern behind #336 (~260 MB orphaned per no-op rebuild). - A frozen version silently rots: old Claude Code releases eventually stop working against the API. Containerfile: default to @latest (npm auto-selects the musl native binary on this alpine base) and make the install layer byte-stable — clean the npm cache and normalize install-time mtimes on the installed tree and the directories the RUN touches. Two independent --no-cache builds now produce a byte-identical npm layer, so a rebuild of the same version dedups instead of orphaning a copy. New ci-claude-refresh.yml owns the image lifecycle: weekly (and on-demand) it resolves the latest published version, and only when it differs from what the consumers pin does it build ci-claude:<version> (+:latest) and open a PR rewiring the 5 consumer pipelines. Version tags make "did it change?" a race-free string compare and keep the cron from opening no-op PRs. Nothing auto-merges. build-ci-images.yml no longer builds ci-claude so the two never fight over tags. Requires owner setup before it does anything (documented in the pipeline header): a dedicated ci-claude-bump-bot identity (write:repository only) as the ci_claude_bump_bot_token secret, and a cron named "ci-claude-refresh". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
7e40d443de |
fix(ci): sandbox the pr-review agents' Read tool against token exfil
The triage + review agents run with REVIEW_TOKEN and CLAUDE_CODE_OAUTH_TOKEN in their environment, process attacker-controlled PR title/body/diff, and their output is posted verbatim as a public PR comment. Without a Read sandbox a prompt-injected diff could read /proc/self/environ (the tokens) or .git/config (Woodpecker's clone credential) and get it published. Every other agent pipeline already denies this via --settings; pr-review had none. Add ci/agents/pr-review/settings.json (mirrors issue-fix/update-audit, gitea #197) denying Read on /proc,/root,/home,/etc,.git, and pass --settings $PROMPTS/settings.json to both claude invocations (resolves alongside the prompts, so it works for the repos that fetch prompts from the homelab clone). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
2a65066672 |
fix(ci): review the workspace checkout, not a hand-cloned copy
On a pull_request event Woodpecker's built-in clone already checks the repo out at the PR head in the workspace, so the agents read that directly for context. Restores full-context review (the prior commit over-corrected to diff-only) while keeping the manual, token-bearing clone gone. Prompts point at the working-directory checkout instead of ./work/target. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
8aa3bbf761 |
refactor(ci): review from the diff, don't clone the target repo
The clone existed only to hand review_bot_token to git, which persisted it in work/target/.git/config — a file the review agent (Read/Glob/Grep, output posted publicly) could read. Nothing needed the checkout that the API-fetched diff + metadata don't already provide, so drop the clone entirely: no token on disk, no work/target, no cleanup. The PR + diff are still read over the API with ephemeral curl Authorization headers (never written to disk). Prompts updated to match: the agents review the diff alone (no ./work/target), and are told to flag, not guess, when a judgement needs code they can't see. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
ab105233d9 |
security(ci): add Read sandbox to alert-monitor and issue-fix agents
Both agent steps hold Gitea write credentials in their environment (issue-fix holds a push token too), but neither passed --settings, so their Read tool was unrestricted — unlike update-audit, which got a Read sandbox for exactly this reason (gitea issue #197). That left a complete exfil path: Read /proc/self/environ yields the token, Write puts it in the agent's output file, and the pipeline posts that output verbatim as a Gitea issue body / PR body on a PUBLIC repo. Add the same deny-list used by update-audit (/proc, /root, /home, /etc, .git) to both agents and wire --settings into each invocation. The repo checkout, .ci-alert-monitor/ and /tmp/alert-fix/ stay readable, which is all either agent needs. This is defense-in-depth only. The root cause — CI pipelines holding an admin-owned token at all — is tracked separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-botandClaude Opus 4.8
|
9ad08ad931 |
feat(ci): add pr-review agent — scheduled AI review of open PRs
New Woodpecker cron pipeline that mirrors the alert-monitor / issue-fix
fleet pattern (cron -> headless `claude` -> Gitea API), but reviews open
pull requests instead of triaging alerts:
- .woodpecker/pr-review.yml: hourly `pr-review` cron. Discovers open PRs
across k.renner/{homelab,shelly-manager,authentik-samba-bridge,
homeassistant}, skips drafts/WIP and PRs already reviewed at their
current head sha (dedup marker), caps at 5/run. For each: clones the PR
head, has a cheap Haiku triage agent pick the model by complexity
(haiku/sonnet/opus; security-sensitive paths force opus), runs the
review with that model, and posts a detailed report as a Gitea PR
review (COMMENT, or REQUEST_CHANGES on a real defect — never APPROVE).
- ci/agents/pr-review/triage-prompt.md: model-selection rubric.
- ci/agents/pr-review/prompt.md: the reviewer's task + report format.
Runs under a dedicated `review-bot` Gitea account (secret
review_bot_token) so reviews are visibly bot-authored and independently
revocable. Static review only — no cluster access; agent tools are
Read/Glob/Grep/Write with no Bash.
Not yet active: needs the review-bot account + token, the review_bot_token
Woodpecker secret, and the `pr-review` cron registered (see file header).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
claude-botandClaude Fable 5
|
c10c7af3c8 |
feat(alert-monitor): blip lookback — catch alerts that resolve between ticks
The live 15-min snapshot structurally misses anything that fires and resolves between ticks: on 2026-07-16 a 17-minute KubeHpaMaxedOut (authentik) and last evening's four-alert home-assistant incident were invisible to the monitor (owner noticed the blips; monitor did not). The fetch step now also queries Prometheus's ALERTS history over a 35-minute window (~2 ticks + jitter). The dedup step synthesizes pseudo-alerts for series not currently firing at all — checked against the PRE-filter snapshot so live-but-null-routed alerts don't reappear as blips — with a stable sha256-derived fingerprint so one noise verdict suppresses a flappy alert permanently. Blips then flow through the same title/noise dedup and triage as live alerts; the prompt explains their semantics (self-resolved; judge the underlying cause; null-routed per the repo route config = noise). Verified against live data: currently-firing KubeCPUOvercommit excluded, injected KubeHpaMaxedOut blip synthesized correctly; sha256sum confirmed present in ci-claude; lookback query tested via the service proxy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei |
||
|
claude-botandClaude Fable 5
|
abbc7a2d3f |
feat(issue-fix): pipeline-verified owner guidance re-opens assessed issues
The #259 lesson, both directions: the agent rightly refused unauthenticated instructions embedded in the issue body — and then had no channel for the owner's authentic reply 18 minutes later, so a human ended up doing the sweep the agent should have owned. - Pick step: an issue with an alert-fix:assessed marker becomes eligible again when a TRUSTED_AUTHORS comment is NEWER than the latest marker (timestamps compared in jq — busybox ash test has no string-<). Owner replies on the issue now drive the loop. - Trusted comments are appended to the agent's issue.md as 'Verified owner guidance'. Trust anchor is the hardcoded author list in this file — deliberately not the Gitea permission API, because the pipeline itself comments as gitea_admin and agent-authored text must never be able to masquerade as owner guidance. - Prompt: verified guidance is authoritative for intent and scope (may lift the surgical ceiling when it calls for a sweep), does not lift the edit-path allowlist, and factual mismatches with live state are surfaced, not silently resolved in either direction. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei |
||
|
claude-botandClaude Fable 5
|
90aeaf1ef8 |
feat(ci-agents): give triage/fix agents verified read-only live diagnostics
Both wrong assessments (#249 openbao 'sealed', #250 three untested hypotheses while one 'kubectl logs' showed the 401) share one root cause: the agents diagnose blind, so they narrate plausible fiction. Labeling unverified claims (previous commit) treats the symptom; this gives them the means to verify: - kubernetes/apps/monitoring/ci-diagnostics-rbac.yaml: ServiceAccount ci-diagnostics with a read-only ClusterRole (workloads, events, pods/log, Flux/ESO/cert-manager/monitoring/ceph CRDs — NO secrets) plus a namespaced services/proxy Role for the Prometheus and Alertmanager HTTP APIs. - scripts/ci/make-diag-kubeconfig.sh: mints a 1h token per run from the admin kubeconfig (no new long-lived secret), builds .ci-diag/kubeconfig, copies the static kubectl binary for the jq-only ci-claude image, smoke-tests with the restricted identity, and degrades to snapshot-only mode when the SA isn't reconciled. - alert-monitor.yml / issue-fix.yml: agents get exactly one extra tool, Bash(./.ci-diag/kubectl:*) — prefix match forbids pipes and any other binary; RBAC forbids mutation and secret reads. - prompts: verification is now mandatory, not aspirational — 'a hypothesis you could have settled with one read-only command but didn't is a defect in your output'; fix agent must test the issue's claims before building on them; minimal-quote rule against dumping logs into public issues. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei |
||
|
claude-botandClaude Fable 5
|
33eb6592c6 |
fix(ci-agents): forbid asserting unverified cluster state in triage/fix prompts
The first issue-fix run on #249 asserted the openbao pod was '0/1 and sealed' — it was Ready and unsealed the whole time; the alert fired because the chart's updateStrategy: OnDelete holds the 2.5.5→2.6.0 image bump until an operator deletes the pod. The agent has no cluster access and invented runtime state from the alert's generic annotation text. - issue-fix prompt: runtime claims must be attributed or framed as hypothesis, with read-only verification commands in the summary. - alert-monitor prompt: when context.txt does not confirm the alert's symptom, say so explicitly — the mismatch is often the real clue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei |
||
|
claude-botandClaude Fable 5
|
b7ff28d6a6 |
feat(ci): alert-monitor and issue-fix agent pipelines
Two new cron pipelines closing the loop from firing alert to fix PR:
- alert-monitor (*/15 min): reads active alerts from Alertmanager via the
apiserver service proxy (woodpecker-ci is netpol-blocked from monitoring
ClusterIPs; the Authentik gate blocks the public hostname), dedups
against open issues and a persistent noise-log issue, then a headless
Claude agent triages the new ones against the repo plus a cluster
snapshot. Confirmed problems become Gitea issues; noise is recorded in
the noise log so it is never re-triaged.
- issue-fix (hourly): picks the oldest open '[alert] ...' issue with no
fix branch and no assessed-marker comment, hands it to a fix agent
(same tool sandbox as security-fix), pushes alert-fix/issue-<n> and
opens a PR with 'Closes #<n>'. Not-repo-fixable verdicts are posted as
an issue comment instead.
- e2e.yml: scope the bare 'event: cron' to the three existing cron names
— otherwise the 15-minute cron would piggyback a full e2e run every 15
minutes.
The ci-k8s image has no jq, so the alert fetch step only talks to the
cluster; all JSON filtering/dedup/gating lives in the ci-claude step.
Cron registration (after this lands on master):
woodpecker repo cron add --repo gitea_admin/homelab \
--name alert-monitor --branch master --schedule '*/15 * * * *'
woodpecker repo cron add --repo gitea_admin/homelab \
--name issue-fix --branch master --schedule '41 * * * *'
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei
|
||
 k.renner
|
b1dec3e764 | Merge pull request 'fix(ci): sandbox update-audit agent against prompt-injection exfil' (#238) from fix/issue-197 into master | ||
|
claude-bot
|
e643533141 |
fix(ci): sandbox update-audit agent against prompt-injection exfil
Closes gitea issue #197. The update-audit agent fetches untrusted third-party release pages via WebFetch/WebSearch and its Write output is posted verbatim to a public Gitea issue, so injected content in a fetched page could try to redirect the agent's behavior. Harden the tool surface (verified empirically against the pinned Claude Code CLI): - Drop WebSearch: it cannot be domain-scoped like WebFetch, and the prompt already gives explicit source URLs or a well-known registry for everything it needs to look up. - Add ci/agents/update-audit/settings.json (loaded via --settings) that denies Read on /proc, /root, /home, /etc, and the repo's own .git/ — closing the credential-file and env-var-dump read paths a prompt-injected instruction could otherwise try. Verified this deny mechanism actually blocks reads, unlike Write path-scoping, which was tested and found NOT enforced under --permission-mode acceptEdits (documented in the pipeline comment; a full WebFetch domain allowlist was also considered and rejected, since Helm chart lookups legitimately hit whichever domain each HelmRelease's own trusted source URL points at). - prompt.md now tells the agent fetched content is untrusted data, not instructions, and to never echo it verbatim into the report. Not fixed here: GITEA_TOKEN is still the broad registry/release-writing token (no write:issue-only token exists yet) — that requires an operator to mint one; tracked via a comment on #197 and a code comment at the secret binding. |
||
|
claude-bot
|
2feec4b193 |
fix(ci): pin base image digests, verify tool checksums, pin claude-code version
ci/woodpecker/push/build-ci-images Pipeline was successful
Addresses #209. All four ci/images/*/Containerfile FROM lines were tag-pinned only (alpine:3.21, node:22-alpine), letting the base silently change between builds. Pin each to its current manifest-list digest. ci/images/k8s/Containerfile downloaded talosctl, kubectl, and velero via curl with no integrity check, and installed flux via `curl | bash` (itself an unpinned, unverified fetch-and-execute). Replace the flux install with a pinned-version tarball download verified against its published sha256 checksum, same as talosctl/kubectl/velero now are. kubectl's version was also resolved dynamically from dl.k8s.io's stable.txt at build time instead of being pinned; add KUBECTL_VERSION. ci/images/tofu/Containerfile's tofu.zip download gets the same sha256 verification. ci/images/claude/Containerfile's CLAUDE_CODE_VERSION defaulted to `latest` and was never overridden by build-ci-images.yml, so the image holding repo/issue-write tokens floated to whatever npm resolved at build time. Pin it to the current release (2.1.209). Checksums and digests verified against the respective registries/release APIs at fix time (2026-07-14). Co-Authored-By: Claude Bot <claude-bot@renner.uno> |
||
|
k.rennerandClaude Opus 4.8
|
0ce60164e1 |
ci: decommission migration-only rebuild-k8s; point access tooling at Hetzner
rebuild-k8s.yml was the one-shot PVE->K8s migration/rebuild pipeline; it is no longer needed and still pointed at the retired Garage S3 on the now-off nas-1. Remove it (DR restore is via Velero + flux bootstrap). The live cluster-access path is kept and migrated off Garage onto Hetzner Object Storage (renner-uno-k8s-secrets, fsn1.your-objectstorage.com, region fsn1, hetzner_s3_* keypair): - scripts/ci/fetch-kubeconfig.sh: Garage -> Hetzner; env vars renamed GARAGE_SECRETS_* -> S3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY. - .woodpecker/e2e.yml, e2e-test.yml: pass hetzner_s3_access_key/secret_key (the live cron e2e pipeline needs this — it was broken against dead nas-1). - scripts/import-cluster-access.sh: Garage -> Hetzner endpoint/bucket/region. NOTE: the workstation aws-cli "secrets" profile must now hold the Hetzner Object Storage keypair. Also: delete the orphaned test-garage-smoke play + pipeline, and fix the two dangling rebuild-k8s.yml references (zfs-tuning play comment; update-audit prompt Talos-pin file list). Assumes cluster-access objects (kubeconfig.sops/talosconfig.sops) were migrated into renner-uno-k8s-secrets; with rebuild-k8s gone, nothing auto-refreshes them, which is fine for a stable cluster. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
k.rennerandClaude Opus 4.8
|
1fb7bfacc6 |
cleanup(ingress): drop legacy Caddy/cloudflared/netbird-gateway tiers
The netbird-ingress DaemonSet, netbird reverse-proxy, and Traefik have fully superseded the old ingress tiers, which have been stopped (not deleted) since the 2026-05-09 PVE cutover. Remove the dead code: - Caddy HA: LXCs 232/233/234 (opentofu/ingress.tf), caddy-ha role, install-caddy-ha + install-crowdsec-caddy plays, caddy cred vars, proxmox_caddy_ha group_vars, build-caddy/deploy-caddy pipelines, ci-caddy-builder image. - cloudflared: LXC 117, cloudflared-lxc role, install-cloudflared play, cloudflare_tunnel/variables_cloudflare disabled tofu stubs. - netbird-gateway: LXC 260 (replaced by the in-cluster netbird-ingress DaemonSet), netbird-gateway role + play. - backup_jobs.tf: drop VMIDs 117/232/233/234/260 from the nightly vzdump include list (those LXCs are being destroyed). - netbird-server: point the setup-key hint at netbird-ingress/netbird-proxy. Kept deliberately: - crowdsec role (reusable; basis for the pending Traefik bouncer). - netbird-client role + play (loki/prometheus LXCs are tagged netbird_client, so it remains live architecture). Note: applying the opentofu change will plan a DESTROY of stopped LXCs 117/232/233/234/260 — review the plan before apply. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
k.rennerandClaude Opus 4.8
|
eb0c44daa8 |
ci: weekly infrastructure update-audit agent
Adds a Woodpecker cron pipeline that runs a read-only Claude Code agent to
audit the components NOT covered by existing automation, and posts the report
to a single persistent Gitea issue.
Scope (complements, does not duplicate, existing automation):
- Flux image-automation already bumps enrolled container tags.
- The daily security-scan agent already owns CVEs.
- This agent audits manually-pinned currency instead: Talos / Kubernetes /
Flux versions, Helm chart minor/major bumps held back by ~MAJOR.MINOR.0
constraints, images excluded from automation, and Ansible/OpenTofu drift.
The agent is read-only (allowedTools: Read,Glob,Grep,Write,WebFetch,WebSearch;
no Edit/Bash) so it can never mutate the repo. It writes a report to
/tmp/update-audit.md; the pipeline find-or-creates the "Infrastructure update
audit" Gitea issue and refreshes its body.
Cron uses Woodpecker's documented `cron:` name filter (rather than the injected
PIPELINE var the other pipelines use) so the schedule is CLI-creatable:
woodpecker repo cron add --repo gitea_admin/homelab \
--name update-audit --branch master --schedule '@weekly'
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
k.rennerandClaude Opus 4.7
|
9f48960589 |
revert(security-fix-agent): drop image-tag-bump role
ci/woodpecker/push/build-ci-images Pipeline was successful
Container image tag updates are Flux's job — the homelab cluster
already runs image-reflector-controller and image-automation-
controller (in flux-system/gotk-components.yaml). Setting up
ImageRepository / ImagePolicy / ImageUpdateAutomation resources
is the right home for "bump postgres:15.17 → 15.18 when a new
image is published"; doing it from a CVE-driven scan agent
duplicates that pipeline and competes with it.
Reverting:
- Agent prompt: drop the `## Image tag bumps` section; image
findings now go straight to the Skipped list with a "Flux
image-automation owns this" reason.
- Woodpecker yml: drop Bash(crane *) from the agent's tool
allowlist (no longer used) and the `pull: true` workaround
that existed only to pick up crane's mid-day rebuild.
- ci-claude Containerfile: remove the crane install. The image
is back to Node + Claude Code + git/jq/curl basics.
The agent is now scoped exclusively to IaC misconfigurations
(Trivy fs scan results). Image vulns surface in the log + summary
for visibility but no PR action.
Note: there are currently zero ImageRepository / ImagePolicy /
ImageUpdateAutomation resources in kubernetes/ — the controllers
are running idle. Wiring them up for the actual images is a
separate workstream.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
k.rennerandClaude Opus 4.7
|
cf98c3f044 |
fix(namespaces): move influxdb namespace into infrastructure/namespaces
The seed KS owns influxdb-seed-ssh Secret in the influxdb namespace, but the namespace was previously created by apps/influxdb/namespace.yaml — which dependsOn seed → seed never reconciles → circular. Hoist the namespace into infrastructure/namespaces/namespaces.yaml (consistent with all the other app namespaces) and drop it from the apps/influxdb Kustomization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |