Stateless Go app from ghcr.io/konairius/ical-proxy:latest.
2 replicas, test ingress at ical-proxy.k.renner.uno.
No DB, no OIDC, no secrets needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The restore Job needs to reach the Ceph RGW service in the
rook-ceph namespace to download the backup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CephObjectStore CRD doesn't support service.type. Create a standalone
LoadBalancer Service targeting the RGW pods instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sed-based JSON parsing failed silently. Use grep+cut instead
and add proper HTTP status code checking and error output.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trigger-backup Job runs in K8s, calls Woodpecker API to start the
backup pipeline with Ceph RGW credentials from the OBC, waits for
completion. Restore Job then downloads from RGW and restores.
FluxCD orchestrates: trigger-backup → restore → Authentik HelmRelease.
Bootstrap script no longer needs woodpecker CLI or S3 credentials.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The CephObjectStoreUser and ObjectBucketClaim take time to
provision after the Object Store is ready. Add explicit waits.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RustFS has an open bug (#2505) that loses IAM users on restart.
Use Ceph Object Gateway instead — already have Rook-Ceph running.
- CephObjectStore + CephObjectStoreUser + ObjectBucketClaim
- RGW exposed via Cilium LB at 10.2.40.81
- Restore Job reads credentials from auto-generated OBC secret
- Bootstrap script triggers backup pipeline with RGW credentials
- No manual credential management needed
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The MinIO IAM provider doesn't persist users reliably. Use manually
created credentials instead — stable across rebuilds.
Remove aminueza/minio provider, IAM resources, and credential output.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
minio/mc needs a writable home dir for config. Running as non-root
user 1000 can't write to /.mc.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Completed Jobs are immutable. Rename to v2 to trigger a new run
with the security context fixes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Operation not permitted error was caused by missing securityContext.
Add runAsNonRoot, seccompProfile, and drop all capabilities.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
authentik-db deploys first: CNPG cluster, restore Job, S3 credentials.
FluxCD waits for the restore Job to complete before deploying authentik
(HelmRelease). This ensures the DB is migrated before Authentik starts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The restore Job runs once via FluxCD, downloads the DB dump from S3,
and restores it into the CNPG cluster. Remove from kustomization
after migration is complete.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split into init container (minio/mc for S3 download) and main
container (postgres:16 for pg_restore). Shared via emptyDir volume.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SOPS-encrypted secret with MinIO migration user credentials.
Used by the restore-job.yaml to download the DB dump from S3.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Custom IAM policy caused provider bug during comparison.
Use MinIO's built-in readwrite policy instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use tf_rfs_access/tf_rfs_secret (already in CI secrets) instead of
adding new MinIO-specific secrets. No new env variables needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace broken AWS IAM resources with aminueza/minio provider.
Creates a dedicated MinIO user with access to the migration bucket.
Requires new Gitea CI secrets: minio_server, minio_user, minio_password.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IAM user with read/write access to the migration bucket only.
Credentials output as sensitive values for use in K8s secrets.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Downloads backup from RustFS S3 directly into the CNPG pod using
the MinIO client. No extra pods or complex orchestration needed.
Requires AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env vars.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create the migration bucket in RustFS via OpenTofu, not inline in
the pipeline. Remove bucket creation from the Woodpecker step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backup pipeline (CI): runs pg_dump via Ansible on the old DB server,
uploads dump to s3://migration/ bucket in RustFS. No extra secrets needed.
Restore script (workstation): downloads from S3, copies into K8s
CNPG pod, runs pg_restore. Uses local kubectl and aws-cli.
Preserves all users, passwords, 2FA, WebAuthn, and sessions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backup pipeline (CI): runs pg_dump via Ansible on the old DB server,
uploads dump to RustFS S3. No kubeconfig secret needed.
Restore script (workstation): downloads from S3, copies into K8s
CNPG pod, runs pg_restore. Uses local kubectl.
Preserves all users, passwords, 2FA, WebAuthn, and sessions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two-step pipeline:
1. Ansible playbook dumps the old Authentik PostgreSQL via pg_dump
2. kubectl restores the dump into the K8s CloudNativePG cluster
Preserves all users, passwords, 2FA devices, sessions, and config.
Requires kubeconfig_b64 secret in Gitea for K8s access.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Was added to try to fix ICMP on LB VIP, but that's a Cilium L2
limitation unrelated to traffic policy. Revert to default.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cilium L2 announced VIPs only forward service ports by default.
externalTrafficPolicy: Local makes the VIP a local address on the
node so the kernel responds to ICMP echo requests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Enable IPv6 in both the Talos inline Cilium manifest and the FluxCD
HelmRelease. Required for Cilium to announce IPv6 LB VIPs via NDP.
Requires cluster rebuild to apply Talos patch changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use io.cilium/lb-ipam-ips annotation to request both IPv4 (10.2.40.80)
and IPv6 ULA (fd40::80) for the ingress LoadBalancer service.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Format: "resource id" per line (space-separated), not "resource=id".
Avoids IFS conflicts with resource addresses containing dots.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allows importing existing Proxmox resources into tofu state before
plan/apply. Usage: --var "TOFU_IMPORT=resource=id" (newline-separated).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add IPv6 pod/service subnets (ULA fd10:244::/48, fd10:96::/112)
- Add IPv6 ULA range to Cilium LB IP pool (fd40::40-fd40::5f)
- Enable PreferDualStack on ingress-nginx service
Requires cluster rebuild to apply Talos config changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- set_password endpoint returns empty body on success
- Remove --silent flag from flux bootstrap (unsupported)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VMs stuck in UEFI after disk wipe. Need to be recreated via CI
with TOFU_REPLACE for module.k8s_1, module.k8s_2, module.k8s_3.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bootstrap now waits for the authentik kustomization after apps.
Include the Authentik API migration script for post-bootstrap setup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
auth_url stays external (browser redirect), but token_url and api_url
use the cluster-internal authentik-server service for server-to-server
OAuth calls from the monitoring namespace.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Grafana in the monitoring namespace was unreachable via ingress
because the nginx egress policy didn't include that namespace.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The chart template sets rollingUpdate params that conflict with Recreate.
RollingUpdate works fine with replicas=1 since the PVC detach race
was a one-time issue during the initial config change.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
K8s services must use *.k.renner.uno for testing before DNS cutover.
dash.renner.uno stays on the old infrastructure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RWO PVC can't attach to two pods simultaneously, so rolling updates
get stuck waiting for the volume. Recreate terminates old pod first.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
