konairius/homelab
10
0
Fork 0
Code Issues 11 Pull Requests 3 Packages Projects Releases 1 Wiki Activity

refactor(rotation): port netbird-ingress to netbird-key mode #335

Merged
k.renner merged 1 commits from feat/port-netbird-rotator into master 2026-07-21 22:43:44 +00:00
Conversation 1 Commits 1 Files Changed 1
+26 -56
1 changed files with 26 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
kubernetes/infrastructure/openbao/rotation/netbird-ingress.yaml
+26 -56
View File
@@ -1,20 +1,18 @@
---
# Class-B automated rotation of the NetBird ingress setup-key.
# Class-B rotation of the netbird-ingress setup key. Mints a fresh reusable
# setup key via the secret-rotator binary (netbird-key mode):
# POST /api/setup-keys, then patches OpenBao. Create-new, not roll-in-place —
# the old key is left to expire (ephemeral, 1yr), so nothing is deleted and
# there is no window: the old key stays valid while the ingress re-enrolls.
#
# Unlike the Class-A jobs (which just generate a random value), this calls the
# self-hosted NetBird MANAGEMENT API to mint a fresh setup key, then writes it to
# OpenBao. openbao-ns pattern (intra-namespace OpenBao access; the public NetBird
# mgmt API is reachable via the openbao ns `world` egress). ESO syncs the
# netbird-ingress-setup-key Secret (ns ingress, 10m refresh); Reloader restarts
# the netbird-ingress DaemonSet so its peers re-enroll with the new key. Setup
# keys are disposable/reusable by design, so this is inherently low-risk.
# openbao-ns pattern; the NetBird management API (netbird.renner.uno, valid TLS)
# is reachable via the openbao ns `world` egress. ESO syncs the
# netbird-ingress-setup-key Secret (ns ingress, 10m); Reloader restarts the
# netbird-ingress DaemonSet, whose pods re-enroll on the new key.
#
# Tested + enabled 2026-07-19. The minted key MUST match the ingress key's
# settings: ephemeral=true (peers auto-reaped when offline — these DaemonSet peers
# re-enroll fresh on every restart) and allow_extra_dns_labels=true (the `DNS`
# capability), in the k8s-ingress-router group. It KEEPS a bounded 1y expiry on
# purpose: an auto-rotated key SHOULD expire, so a silently-broken rotation
# eventually surfaces (new enrollments fail) instead of a key living forever.
# An auto-rotated key SHOULD expire (bounded blast radius if a rotation ever
# silently breaks). The rotator NEVER deletes keys — a past manual delete-filter
# bug wiped unrelated keys, so automated deletion is deliberately absent.
apiVersion: v1
kind: ServiceAccount
metadata:
@@ -23,8 +21,6 @@ metadata:
labels:
app.kubernetes.io/name: netbird-ingress
app.kubernetes.io/component: secret-rotator
# OpenBao: auth/kubernetes/role/rotator-netbird binds this SA to rotator-netbird
# (read bootstrap PAT + patch the setup-key path).
---
apiVersion: batch/v1
kind: CronJob
@@ -35,7 +31,7 @@ metadata:
app.kubernetes.io/name: netbird-ingress
app.kubernetes.io/component: secret-rotator
spec:
suspend: false # tested + enabled 2026-07-19
suspend: false
schedule: "0 4 1 1,4,7,10 *" # quarterly, Europe/Berlin
timeZone: Europe/Berlin
concurrencyPolicy: Forbid
@@ -63,55 +59,29 @@ spec:
type: RuntimeDefault
containers:
- name: rotate
# Class-B jobs need an HTTP client + JSON. python:alpine provides both
# via stdlib (urllib/json) with NO runtime package install, so it runs
# cleanly under the non-root securityContext (apk/apt need root and fail
# here — the openbao ns enforces restricted PSA).
image: python:3.13-alpine
image: gitea.renner.uno/gitea_admin/secret-rotator@sha256:49bfddd8eeb5fd871cbeb05bc3b4e2fcaaab7d9e24cc3007caae6084b4634c18
args: ["netbird-key"]
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
env:
- name: BAO_ADDR
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: rotator-netbird
- name: BOOTSTRAP_PATH
value: homelab/infrastructure/secret-rotators-bootstrap/netbird
- name: BAO_PATH
value: homelab/infrastructure/netbird-ingress
- name: NB_MANAGEMENT_URL
value: "https://netbird.renner.uno"
# NetBird group ID(s) for the new key — the k8s-ingress-router group.
value: https://netbird.renner.uno
- name: NB_AUTO_GROUPS
value: "d8umic2oc9cs73d4e1f0"
command: ["python3", "-c"]
args:
- |
import json, os, urllib.request as u
bao = os.environ["BAO_ADDR"]; mgmt = os.environ["NB_MANAGEMENT_URL"]
groups = [g for g in os.environ["NB_AUTO_GROUPS"].split(",") if g]
jwt = open("/var/run/secrets/kubernetes.io/serviceaccount/token").read()
def call(method, url, headers=None, body=None, ct="application/json"):
data = json.dumps(body).encode() if body is not None else None
req = u.Request(url, data=data, method=method, headers=headers or {})
if data is not None:
req.add_header("Content-Type", ct)
with u.urlopen(req, timeout=30) as r:
return json.loads(r.read() or b"{}")
# 1. OpenBao login (k8s auth)
tok = call("POST", bao + "/v1/auth/kubernetes/login",
body={"role": "rotator-netbird", "jwt": jwt})["auth"]["client_token"]
# 2. read the bootstrap NetBird PAT
pat = call("GET", bao + "/v1/homelab/data/infrastructure/secret-rotators-bootstrap/netbird",
{"X-Vault-Token": tok})["data"]["data"]["pat"]
# 3. mint a new reusable setup key via the NetBird mgmt API
key = call("POST", mgmt + "/api/setup-keys", {"Authorization": "Token " + pat},
{"name": "ingress-rotated", "type": "reusable", "expires_in": 31536000,
"usage_limit": 0, "ephemeral": True, "allow_extra_dns_labels": True,
"auto_groups": groups})["key"]
# 4. patch the new key into OpenBao (KV v2 merge-patch)
call("PATCH", bao + "/v1/homelab/data/infrastructure/netbird-ingress",
{"X-Vault-Token": tok}, {"data": {"setup-key": key}}, "application/merge-patch+json")
print("rotated setup-key at homelab/infrastructure/netbird-ingress (value not logged)")
value: d8umic2oc9cs73d4e1f0
resources:
requests:
cpu: 25m
memory: 48Mi
memory: 16Mi
limits:
memory: 96Mi
memory: 48Mi