homelab

2,432 Commits 735 Branches 2 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
claude-botandClaude Fable 5	90aeaf1ef8	feat(ci-agents): give triage/fix agents verified read-only live diagnostics Both wrong assessments (#249 openbao 'sealed', #250 three untested hypotheses while one 'kubectl logs' showed the 401) share one root cause: the agents diagnose blind, so they narrate plausible fiction. Labeling unverified claims (previous commit) treats the symptom; this gives them the means to verify: - kubernetes/apps/monitoring/ci-diagnostics-rbac.yaml: ServiceAccount ci-diagnostics with a read-only ClusterRole (workloads, events, pods/log, Flux/ESO/cert-manager/monitoring/ceph CRDs — NO secrets) plus a namespaced services/proxy Role for the Prometheus and Alertmanager HTTP APIs. - scripts/ci/make-diag-kubeconfig.sh: mints a 1h token per run from the admin kubeconfig (no new long-lived secret), builds .ci-diag/kubeconfig, copies the static kubectl binary for the jq-only ci-claude image, smoke-tests with the restricted identity, and degrades to snapshot-only mode when the SA isn't reconciled. - alert-monitor.yml / issue-fix.yml: agents get exactly one extra tool, Bash(./.ci-diag/kubectl:*) — prefix match forbids pipes and any other binary; RBAC forbids mutation and secret reads. - prompts: verification is now mandatory, not aspirational — 'a hypothesis you could have settled with one read-only command but didn't is a defect in your output'; fix agent must test the issue's claims before building on them; minimal-quote rule against dumping logs into public issues. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei	2026-07-16 10:30:29 +02:00
k.rennerandClaude Opus 4.8	0ce60164e1	ci: decommission migration-only rebuild-k8s; point access tooling at Hetzner rebuild-k8s.yml was the one-shot PVE->K8s migration/rebuild pipeline; it is no longer needed and still pointed at the retired Garage S3 on the now-off nas-1. Remove it (DR restore is via Velero + flux bootstrap). The live cluster-access path is kept and migrated off Garage onto Hetzner Object Storage (renner-uno-k8s-secrets, fsn1.your-objectstorage.com, region fsn1, hetzner_s3_* keypair): - scripts/ci/fetch-kubeconfig.sh: Garage -> Hetzner; env vars renamed GARAGE_SECRETS_* -> S3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY. - .woodpecker/e2e.yml, e2e-test.yml: pass hetzner_s3_access_key/secret_key (the live cron e2e pipeline needs this — it was broken against dead nas-1). - scripts/import-cluster-access.sh: Garage -> Hetzner endpoint/bucket/region. NOTE: the workstation aws-cli "secrets" profile must now hold the Hetzner Object Storage keypair. Also: delete the orphaned test-garage-smoke play + pipeline, and fix the two dangling rebuild-k8s.yml references (zfs-tuning play comment; update-audit prompt Talos-pin file list). Assumes cluster-access objects (kubeconfig.sops/talosconfig.sops) were migrated into renner-uno-k8s-secrets; with rebuild-k8s gone, nothing auto-refreshes them, which is fine for a stable cluster. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-27 12:17:33 +02:00
k.rennerandClaude Opus 4.7	b4975fabbb	ci: store cluster access in Garage instead of git ci/woodpecker/manual/security-scan Pipeline was successful Details ci/woodpecker/manual/e2e Pipeline failed Details The rebuild-k8s pipeline previously SOPS-encrypted talosconfig and kubeconfig and pushed them back to master. The git push raced with concurrent commits and forced workstation users to `git pull --rebase` after every rebuild. Replace the transport with Garage's k8s-secrets bucket (s3://k8s-secrets/cluster-access/), reusing the existing secrets-key that the install-garage-synology playbook already provisions. SOPS encryption is unchanged — only the storage layer moves. - New scripts/ci/fetch-kubeconfig.sh helper: 9 CI steps that previously inlined `sops -d kubernetes/access/kubeconfig.sops` now call this. - scripts/import-cluster-access.sh rewritten to pull both files from Garage; no more git fetch/checkout. - export-access step uploads via aws s3 cp; gitea_token + git config removed. - All consuming Woodpecker steps gain garage_secrets_key_id / garage_secrets_secret_key from_secret env vars, and bump to ci-k8s:2026-05-01 (the tag with aws-cli baked in). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-01 14:32:41 +02:00

Author

SHA1

Message

Date

claude-botandClaude Fable 5

90aeaf1ef8

feat(ci-agents): give triage/fix agents verified read-only live diagnostics

Both wrong assessments (#249 openbao 'sealed', #250 three untested
hypotheses while one 'kubectl logs' showed the 401) share one root
cause: the agents diagnose blind, so they narrate plausible fiction.
Labeling unverified claims (previous commit) treats the symptom; this
gives them the means to verify:

- kubernetes/apps/monitoring/ci-diagnostics-rbac.yaml: ServiceAccount
  ci-diagnostics with a read-only ClusterRole (workloads, events,
  pods/log, Flux/ESO/cert-manager/monitoring/ceph CRDs — NO secrets)
  plus a namespaced services/proxy Role for the Prometheus and
  Alertmanager HTTP APIs.
- scripts/ci/make-diag-kubeconfig.sh: mints a 1h token per run from
  the admin kubeconfig (no new long-lived secret), builds
  .ci-diag/kubeconfig, copies the static kubectl binary for the
  jq-only ci-claude image, smoke-tests with the restricted identity,
  and degrades to snapshot-only mode when the SA isn't reconciled.
- alert-monitor.yml / issue-fix.yml: agents get exactly one extra tool,
  Bash(./.ci-diag/kubectl:*) — prefix match forbids pipes and any other
  binary; RBAC forbids mutation and secret reads.
- prompts: verification is now mandatory, not aspirational — 'a
  hypothesis you could have settled with one read-only command but
  didn't is a defect in your output'; fix agent must test the issue's
  claims before building on them; minimal-quote rule against dumping
  logs into public issues.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NV7bZzZLeoFUEdAeCSLEei

2026-07-16 10:30:29 +02:00

k.rennerandClaude Opus 4.8

0ce60164e1

ci: decommission migration-only rebuild-k8s; point access tooling at Hetzner

rebuild-k8s.yml was the one-shot PVE->K8s migration/rebuild pipeline; it
is no longer needed and still pointed at the retired Garage S3 on the
now-off nas-1. Remove it (DR restore is via Velero + flux bootstrap).

The live cluster-access path is kept and migrated off Garage onto Hetzner
Object Storage (renner-uno-k8s-secrets, fsn1.your-objectstorage.com,
region fsn1, hetzner_s3_* keypair):
- scripts/ci/fetch-kubeconfig.sh: Garage -> Hetzner; env vars renamed
  GARAGE_SECRETS_* -> S3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY.
- .woodpecker/e2e.yml, e2e-test.yml: pass hetzner_s3_access_key/secret_key
  (the live cron e2e pipeline needs this — it was broken against dead nas-1).
- scripts/import-cluster-access.sh: Garage -> Hetzner endpoint/bucket/region.
  NOTE: the workstation aws-cli "secrets" profile must now hold the Hetzner
  Object Storage keypair.

Also: delete the orphaned test-garage-smoke play + pipeline, and fix the
two dangling rebuild-k8s.yml references (zfs-tuning play comment;
update-audit prompt Talos-pin file list).

Assumes cluster-access objects (kubeconfig.sops/talosconfig.sops) were
migrated into renner-uno-k8s-secrets; with rebuild-k8s gone, nothing
auto-refreshes them, which is fine for a stable cluster.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-27 12:17:33 +02:00

k.rennerandClaude Opus 4.7

b4975fabbb

ci: store cluster access in Garage instead of git

ci/woodpecker/manual/security-scan Pipeline was successful

Details

ci/woodpecker/manual/e2e Pipeline failed

Details

The rebuild-k8s pipeline previously SOPS-encrypted talosconfig and
kubeconfig and pushed them back to master. The git push raced with
concurrent commits and forced workstation users to `git pull --rebase`
after every rebuild.

Replace the transport with Garage's k8s-secrets bucket
(s3://k8s-secrets/cluster-access/), reusing the existing secrets-key
that the install-garage-synology playbook already provisions. SOPS
encryption is unchanged — only the storage layer moves.

- New scripts/ci/fetch-kubeconfig.sh helper: 9 CI steps that previously
  inlined `sops -d kubernetes/access/kubeconfig.sops` now call this.
- scripts/import-cluster-access.sh rewritten to pull both files from
  Garage; no more git fetch/checkout.
- export-access step uploads via aws s3 cp; gitea_token + git config
  removed.
- All consuming Woodpecker steps gain garage_secrets_key_id /
  garage_secrets_secret_key from_secret env vars, and bump to
  ci-k8s:2026-05-01 (the tag with aws-cli baked in).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-01 14:32:41 +02:00