Compare commits

...
Author SHA1 Message Date
claude-botandClaude Fable 5.1 ea1032e184 fix(netbird-proxy-services): numeric runAsUser so runAsNonRoot can be verified
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
First rollout of #814 sat in CreateContainerConfigError: "container has
runAsNonRoot and image has non-numeric user (controller), cannot verify
user is non-root". The kubelet cannot resolve a named user; nobody/nogroup
in the pod spec is enough, the script needs no home and no writable path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 12:45:40 +02:00
k.renner 2856deb4f4 Merge pull request 'feat(netbird): deploy the reverse-proxy-services controller (2/2)' (#814) from netbird-proxy-services-deploy into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#814
2026-09-07 10:43:56 +00:00
k.renner 4d13acc9dd Merge pull request 'feat(netbird): controller for reverse-proxy services from public Ingresses (1/2: image)' (#813) from netbird-proxy-services into master
ci/woodpecker/push/build-ci-images Pipeline was successful
Reviewed-on: k.renner/homelab#813
2026-09-07 10:43:47 +00:00
claude-botandClaude Fable 5.1 1d21196060 feat(netbird): deploy the reverse-proxy-services controller
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Deployment for the controller of the previous commit: openbao namespace
like the rotators (its default-deny already allows OpenBao, DNS, the
apiserver and world), own ServiceAccount with a read-only ClusterRole
(Ingresses list/watch, Services get), one replica, Recreate. The image tag
is the branch build that pushed this exact code.

The OpenBao role is applied once by hand (policies/netbird-proxy-services
.hcl); until then the pod logs an auth error per minute and changes nothing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 12:11:14 +02:00
k.renner 7e8077714e Merge pull request 'fix(netbird-proxy): watchdog restart cooldown 600 s → 60 s' (#812) from proxy-watchdog-cooldown into master
ci/woodpecker/push/deploy-netbird-server Pipeline was successful
ci/woodpecker/push/deploy-netbird-proxy Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
Reviewed-on: k.renner/homelab#812
2026-09-07 09:43:20 +00:00
claude-botandClaude Fable 5.1 8e9eedfd16 fix(netbird-proxy): watchdog restart cooldown 600 s -> 60 s
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Every loss of the routing peer a proxy has pinned wedges its route
manager; the flood watchdog restarts the container within 2 s and the
public edge is back after one failed probe round. Inside the 600 s
cooldown the restart is suppressed and the URL watchdog three one-minute
rounds are the only recovery: 3.5 min of 000 on every public domain,
measured 2026-09-07 10:28 on proxy-2 ("restart suppressed, last restart
567s ago"). The morning node roll hit the same suppression on both
proxies at 06:58 and survived only because the routes recovered by luck.

Rolling reboots lose one peer every few minutes by design, so with 600 s
the second roll is always the unlucky one. A minute keeps the loop
protection (one restart per probe interval at most).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 10:38:28 +02:00
k.renner c7fd3e208a Merge pull request 'fix(infrastructure): LAPI ServersTransport behind the Traefik CRDs (phase 2/2, remove)' (#809) from lapi-transport-phase-2 into master
ci/woodpecker/cron/alert-monitor Pipeline was successful
ci/woodpecker/cron/issue-fix Pipeline was successful
Reviewed-on: k.renner/homelab#809
2026-09-07 06:09:31 +00:00
claude-botandClaude Fable 5.1 5d52e42887 fix(infrastructure): LAPI ServersTransport behind the Traefik CRDs (phase 2: remove)
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Phase 1 (#802) put the ServersTransport under post-crds and
infrastructure-crds has applied it. The copy in crowdsec/lapi-ingress.yaml
goes; the Ingress stays (core kind). With it, the infrastructure apply set
holds no traefik.io consumer any more, which is what a cold rebuild needs.

Prune: infrastructure reconciles first and skips the object if the last
applier label names infrastructure-crds; otherwise it deletes and
infrastructure-crds recreates it seconds later. Only the public LAPI
ingress used by the external bouncers is behind that transport.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 07:48:22 +02:00
6 changed files with 119 additions and 17 deletions
@@ -32,8 +32,13 @@ netbird_watchdog_fail_threshold: 3
# Minimum seconds between watchdog-issued restarts, so a genuinely broken
# backend can't put the proxy into a restart loop. Shared by both the URL-probe
# watchdog and the flood watchdog below.
netbird_watchdog_cooldown_seconds: 600
# watchdog and the flood watchdog below. One minute: the route-manager wedge
# recurs on every routing-peer loss, and rolling node reboots lose one peer
# every few minutes; a suppressed restart there leaves the wedge standing
# until the URL probe's three rounds fire (3.5 min of 000 on every domain,
# measured 2026-09-07). A minute still caps a loop at one restart per
# probe interval.
netbird_watchdog_cooldown_seconds: 60
# Log fingerprint of the reverse-proxy route-manager wedge, matched as a plain
# substring in the proxy's docker logs by the flood watchdog. On this build the
@@ -31,22 +31,12 @@
# serves read-only decision streams to a valid bouncer key and nothing else.
# The admin surface is cscli, in-cluster. If the source address ever becomes
# deterministic, an ipAllowList can be added on top.
---
#
# The LAPI serves HTTPS only once tls.enabled is set — there is no plaintext
# fallback and no second port; 8080 is the TLS port. Its certificate comes
# from the chart's own self-signed CA, so Traefik needs that CA to verify the
# backend. SANs include crowdsec-service.crowdsec.svc.cluster.local, which is
# what the Service resolves to.
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
name: crowdsec-lapi
namespace: crowdsec
spec:
serverName: crowdsec-service.crowdsec.svc.cluster.local
rootCAs:
- secret: crowdsec-root-secret
---
# backend; the ServersTransport carrying it is a traefik.io CRD consumer and
# lives in post-crds/crowdsec-lapi-serverstransport.yaml.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
@@ -18,6 +18,7 @@ resources:
- cloudnative-pg/
- mariadb-operator/
- netbird-ingress/
- netbird-proxy-services/
- coredns-config/
- k8s-gateway/
- external-dns-unifi/
@@ -0,0 +1,104 @@
# Reconciles NetBird reverse-proxy services from Ingresses that carry
# dns.renner.uno/public: one TLS-passthrough service per host on :443,
# targeting Traefik's ClusterIP through a subnet resource over the service
# CIDR (ci/images/netbird-proxy-services/controller.py).
#
# openbao namespace, like the rotators: its default-deny already allows egress
# to OpenBao, DNS, the apiserver and `world` (netbird.renner.uno), and the
# NetBird PAT is read from OpenBao on every pass through the
# netbird-proxy-services role — policies/netbird-proxy-services.hcl, applied
# once by hand like the rotator roles. Until that role exists the pod logs an
# auth error each minute and does nothing.
apiVersion: v1
kind: ServiceAccount
metadata:
name: netbird-proxy-services
namespace: openbao
labels:
app.kubernetes.io/name: netbird-proxy-services
---
# Read-only, cluster-wide: public Ingresses live in many namespaces, the
# Traefik Service in ingress/.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: netbird-proxy-services-read
rules:
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["services"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: netbird-proxy-services-read
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: netbird-proxy-services-read
subjects:
- kind: ServiceAccount
name: netbird-proxy-services
namespace: openbao
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: netbird-proxy-services
namespace: openbao
labels:
app.kubernetes.io/name: netbird-proxy-services
spec:
replicas: 1
# One writer against the management API at a time.
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: netbird-proxy-services
template:
metadata:
labels:
app.kubernetes.io/name: netbird-proxy-services
spec:
serviceAccountName: netbird-proxy-services
securityContext:
runAsNonRoot: true
# Numeric, or the kubelet refuses the image's named user with
# "cannot verify user is non-root". The script needs no home and no
# writable path, so nobody/nogroup is enough.
runAsUser: 65534
runAsGroup: 65534
seccompProfile:
type: RuntimeDefault
containers:
- name: controller
image: gitea.renner.uno/gitea_admin/netbird-proxy-services:2026-09-07
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
env:
- name: NB_MANAGEMENT_URL
value: https://netbird.renner.uno
- name: BAO_ADDR
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: netbird-proxy-services
# Must match cluster.network.serviceSubnets in talos/patches/cluster.yaml.
- name: SERVICE_CIDR
value: "10.96.0.0/12"
- name: TRAEFIK_SERVICE
value: ingress/traefik
- name: INTERVAL_SECONDS
value: "60"
resources:
requests:
cpu: 10m
memory: 32Mi
limits:
memory: 64Mi
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
@@ -31,7 +31,5 @@ resources:
- traefik-crowdsec-middleware.yaml
- traefik-rate-limit-middleware.yaml
- traefik-security-headers-middleware.yaml
# Phase 1 of a two-Kustomization move: also still in crowdsec/lapi-ingress.yaml
# until this layer has applied it once, so prune never runs ahead of apply.
- crowdsec-lapi-serverstransport.yaml
- ../network-policies/