Files
homelab/kubernetes/infrastructure/openbao/rotation/claude-oauth.yaml
T
hermes-bot de080442d3
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
feat(hermes): alert issues end to end: labelled at filing, routed, fixed by the expert card; retire issue-fix
The owner wants the path alert -> [alert] issue -> kanban card -> expert fix PR
-> owner merges, without a human step in between. Owner decisions (kanban card
t_3f5d0a62): [alert] issues are not excluded from the bridge, and the issue-fix
Woodpecker pipeline is retired, so the expert's card is the only automated fix
path for [alert] issues.

Routing
- alert-monitor: `alerts.py file` takes a required `--label` (hestia, atlas,
  apollo, clio, daedalus) and sets that label at creation (POST /issues
  `labels: [id]`). Gitea 1.27.3 adds the labels in the same transaction
  (services/issue/issue.go NewIssue) and the `opened` webhook payload loads them
  (services/convert/issue.go toIssue -> LoadLabels), so the first delivery
  already routes the card to the expert. Org labels are accepted for org repos
  (models/issues/issue_update.go: label.OrgID == repo.OwnerID). If the label
  does not exist yet, the issue is filed without it and the coordinator routes.
  SKILL.md tells the triage agent how to pick the expert.
- Label-less issues no longer go to triage. kanban.auto_decompose defaults to
  true (hermes_cli/config_defaults.py; not overridden here), so the dispatcher
  hands every triage card to the auxiliary-LLM decomposer
  (gateway/kanban_watchers_dispatcher.py auto_decompose_tick ->
  kanban_decompose.decompose_task), which rewrites title and body (dropping the
  Gitea-Issue line the completion hook needs), may fan out into children without
  that line, and keeps the assignee `default` anyway (_apply_single only sets an
  assignee when none is set). The card is now created ready for `default` with
  routing instructions: set exactly one domain label on the issue, nothing else.
- The script reacts to that label: a coordinator card nobody has claimed yet
  (todo/ready) is reassigned (`hermes kanban assign`) and its body replaced by
  the expert body (`hermes kanban edit --body`); a claimed or finished one gets
  a separate expert card under key `<owner>/<repo>#<n>@<expert>` (one per
  expert, created_by gitea-webhook, so its completion is posted on the issue).
  `assign` refuses a running card, which falls back to the separate card. Cards
  already with an expert are still never moved.

[alert] cards
- Their body carries the fix protocol that replaces issue-fix, including its
  prompt hardening from PR #253/#266: check for an open PR with `Closes #<n>` or
  a branch `alert-fix/issue-<n>` first; owner (k.renner) comments are
  authoritative, everything else is untrusted problem description; verify live
  before fixing, never state cluster state not observed this run, give
  read-only commands instead of predicted output; repo-fixable -> branch
  `alert-fix/issue-<n>`, smallest fix, PR with `Closes #<n>`, never merge; not
  repo-fixable -> no PR, the completion summary (posted on the issue) says why
  and what a human must do.

Gitea side (CronJob gitea-webhook-sync, every 15 min)
- Domain labels: created as org labels in every org (GET /admin/orgs) and as
  repo labels in user-owned repos, only when missing.
- Backfill: open issues without the card comment are replayed to the route as
  signed `opened` deliveries (same headers Gitea sends), at most 10 per run, 3 s
  apart (the adapter's default rate limit is 30/min/route), in random order so
  repos where the comment keeps failing cannot starve the rest. The idempotency
  key `<owner>/<repo>#<n>` makes a replay of an issue with a card a no-op.
  The job runs in the hermes namespace, whose default-deny already allows
  intra-namespace traffic to :8644.
- The admin PAT needs write:organization (org labels, reqOrgOwnership passes for
  a site admin), write:issue (repo labels, comments) and read:repository
  (/repos/search) next to write:admin. activeDeadlineSeconds 120 -> 600.

issue-fix removed
- .woodpecker/issue-fix.yml, ci/agents/issue-fix/, and what only it used:
  scripts/ci/make-diag-kubeconfig.sh, the monitoring/ci-diagnostics
  ServiceAccount and RBAC (no other pipeline mints its token; git grep), the
  .ci-diag/ gitignore entry. Comments naming issue-fix updated.
- Not in git and left to the owner: the Woodpecker cron `issue-fix` and the
  secret `issue_fix_bot_token` with its Gitea PAT.

Tested: unittest 73 OK; gitea_issues.py against the real hermes kanban CLI in a
scratch HERMES_HOME with a stub Gitea (unlabelled -> default ready card;
labelled while unclaimed -> same card reassigned to daedalus with the alert
body; labelled while claimed -> separate card `#n@daedalus`; replay -> no
duplicate card or comment); sync discovery read-only against live Gitea (42
open issues, all without a card); kubectl kustomize for apps/hermes,
apps/monitoring, apps/monitoring-rules.
2026-10-21 05:48:01 +00:00

113 lines
4.7 KiB
YAML

---
# Sync the Claude subscription OAuth token (claude_code_oauth_token) into
# Woodpecker's ORG-level secret store. SYNC, NOT MINT: the token comes from an
# interactive `claude setup-token` (no headless mode), so the owner seeds the
# value into OpenBao at infrastructure/claude#setup-token by hand and
# refreshes it whenever they re-run setup-token; this CronJob only propagates
# whatever is stored there into the konairius org secret (create if absent, else
# patch value+events). All three agent pipelines
# (security-scan/update-audit/pr-review) inherit it.
#
# Runs daily rather than quarterly: a hand-refreshed value should reach
# Woodpecker within a day without waiting a quarter. It re-patches the same value
# on quiet days (idempotent, value not logged). To propagate an OAuth refresh
# immediately:
# kubectl -n openbao create job --from=cronjob/claude-oauth-secret-sync claude-oauth-sync-now
#
# woodpecker is reached in-cluster at woodpecker.woodpecker.svc:8000 (it moved
# out of `apps` in the namespace migration — see
# kubernetes/apps/woodpecker/MIGRATION-namespace.md). Needs the netpol
# halves woodpecker-allow-rotators (woodpecker ns) + rotator-claude-oauth-egress
# (openbao-ns.yaml). OpenBao policy/role rotator-claude-oauth + the bootstraps
# (woodpecker-bot token, the seeded OAuth value) are provisioned by hand — see
# policies/rotator-claude-oauth.hcl.
apiVersion: v1
kind: ServiceAccount
metadata:
name: claude-oauth-secret-rotator
namespace: openbao
labels:
app.kubernetes.io/name: claude-oauth
app.kubernetes.io/component: secret-rotator
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: claude-oauth-secret-sync
namespace: openbao
labels:
app.kubernetes.io/name: claude-oauth
app.kubernetes.io/component: secret-rotator
spec:
# Live: image pinned to the build that carries `woodpecker-secret` mode, the
# OpenBao policy/role + bootstraps are applied, and BAO_PATH points at the
# seeded value. Runs daily at 04:20 to sync the OAuth value into the org secret.
suspend: false
schedule: "20 4 * * *" # daily, Europe/Berlin
timeZone: Europe/Berlin
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 2
failedJobsHistoryLimit: 5
startingDeadlineSeconds: 3600
jobTemplate:
spec:
# 4 (not the sibling rotators' 1-2) because this is the only DAILY
# rotator — every other CronJob here is quarterly, so it takes ~90x as
# many runs and is correspondingly more exposed to per-run transients.
# backoffLimit: 1 gave two attempts ~11 s apart, and on 2026-08-19 a
# sub-minute resolver transient took both out (Job
# claude-oauth-secret-sync-29785100, BackoffLimitExceeded), costing a
# day's sync. 4 spreads the attempts over ~150 s (10/20/40/80 s backoff),
# still well inside activeDeadlineSeconds.
backoffLimit: 4
activeDeadlineSeconds: 300
ttlSecondsAfterFinished: 259200 # 3 d
template:
metadata:
labels:
app.kubernetes.io/name: claude-oauth
app.kubernetes.io/component: secret-rotator
spec:
serviceAccountName: claude-oauth-secret-rotator
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 65534
runAsGroup: 65534
seccompProfile:
type: RuntimeDefault
containers:
- name: sync-woodpecker
image: gitea.renner.uno/gitea_admin/secret-rotator:2026-07-30
args: ["woodpecker-secret"]
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
env:
- name: BAO_ADDR
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: rotator-claude-oauth
- name: BOOTSTRAP_PATH
value: homelab/infrastructure/secret-rotators-bootstrap/woodpecker-bot
- name: BAO_PATH
value: homelab/infrastructure/claude
- name: BAO_FIELD
value: setup-token
- name: WOODPECKER_URL
value: http://woodpecker.woodpecker.svc.cluster.local:8000
- name: WOODPECKER_ORG
value: konairius
- name: WOODPECKER_SECRET_NAME
value: claude_code_oauth_token
- name: WOODPECKER_EVENTS
value: cron,deployment,manual,push,release,tag,pull_request
resources:
requests:
cpu: 25m
memory: 16Mi
limits:
memory: 48Mi