The cloudflared ingress list is an explicit allow-list with a 404
catch-all, so external-dns publishing CNAMEs to the cargo tunnel
isn't enough on its own — cloudflared also needs the hostname mapped
to the in-cluster ingress-nginx Service. Add the two paperless URL-
facing hostnames; the .k.renner.uno aliases stay LAN-only since they
don't have a use case via the public tunnel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the `external-dns.alpha.kubernetes.io/cloudflare-proxied: "true"`
annotation that the cluster's cloudflare-scoped external-dns instance
filters on. external-dns publishes the four paperless hostnames as
CNAMEs to the cargo tunnel
(d2386ccd-…cfargotunnel.com per the existing config); cloudflared
forwards inbound traffic to ingress-nginx, which routes
paperless.renner.uno to the gate outpost — same gate flow as direct
LAN access, just with Cloudflare in front.
The .k.renner.uno hostnames also pick up Cloudflare records (the
external-dns scope filter is `domain-filter=renner.uno` which covers
sub-subdomains). They 301 to paperless.renner.uno anyway via the
existing if-redirect, so the redundancy is harmless. Splitting into
two Ingresses to publish only the bare-renner.uno hostnames would be
cleaner conceptually but adds an admission-webhook ordering problem
during reconcile, which we already avoided once.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without a default group, paperless's REMOTE_USER backend creates the
new user with no permissions — the SPA's first /api/ui_settings/ call
returns 403 ("You do not have permission to perform this action"),
which renders as a useless logged-in but blank UI. PAPERLESS_ACCOUNT_
DEFAULT_GROUPS=Familie covers the existing group that Konstantin and
Sarah are already members of, so new auto-created users get the same
viewing/editing surface as existing humans.
The existing Konstantin and Sarah accounts need to be renamed to match
their Authentik login names (k.renner / s.renner) — that's a manual
DB op (one-time, see follow-up note in the conversation) since the
classifier won't let me run direct writes against the prod paperless
DB from here. After the rename, those accounts pick up the gate-
forwarded REMOTE_USER and the Familie permissions are already on the
existing users.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Paperless splits remote-user auth into two flags:
- PAPERLESS_ENABLE_HTTP_REMOTE_USER covers the Django views (the web
UI's session-based auth path).
- PAPERLESS_ENABLE_HTTP_REMOTE_USER_API adds
PaperlessRemoteUserAuthentication to DRF's authentication classes,
so the API endpoints under /api/ trust the same header.
Without the API flag, the SPA frontend gets 401/403 from the gate-
forwarded request on /api/ui_settings/, /api/saved_views/, and every
other API call it makes after login, even though the web UI itself
authenticates successfully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
paperless's URL-facing entry is now the dedicated paperless-gate
outpost in the authentik namespace; ingress-nginx routes
paperless.renner.uno via an ExternalName Service that resolves
cross-namespace, so its TCP connection terminates in authentik and
never opens into the paperless namespace directly. Keeping
ingress-nginx in paperless's baseline allow-list let it bypass the
gate outpost and reach paperless's pod directly with arbitrary
X-Authentik-Username headers, which paperless's HTTP_REMOTE_USER
backend would unconditionally trust.
After this change the only ingress paths permitted to paperless's
pod are:
- in-namespace traffic (other paperless workloads).
- the gate outpost specifically (via the per-app CNP at
kubernetes/apps/paperless/network-policy.yaml).
- database (CNPG operator) and monitoring (Prometheus) namespaces
(cluster-wide baselines, unrelated to user traffic).
- host / remote-node entities (kubelet, node-level traffic).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous standard NetworkPolicy in paperless allowed ingress from
the gate outpost on paper, but the gate outpost still timed out at
134s when forwarding to paperless. Two reasons:
1. The cluster baseline CiliumNetworkPolicy in the paperless namespace
(`default-deny-ingress`, sets ingress allow-list paperless / ingress-
nginx / database / monitoring + host/remote-node) doesn't include
authentik. Standard NetworkPolicy allow rules don't compose
reliably with Cilium baseline CNPs for cross-namespace ingress —
directly verified: an in-namespace curl from ingress-nginx to
paperless succeeds (302 in 3 ms, allowed by the baseline CNP),
while the same curl from the gate outpost times out (no policy
path admits it).
2. The cluster baseline CNP in the authentik namespace
(`default-deny-ingress`, egress to authentik / rook-ceph / kube-dns
/ world) doesn't include paperless either, so even on the source
side the connection is blocked.
Replace the standard NP with two CiliumNetworkPolicies:
- `paperless/paperless-gate-ingress` (paperless namespace): selects
the paperless app pod and allows ingress from endpoints labeled
`goauthentik.io/outpost-name=paperless-gate` in the authentik
namespace. CNPs union additively with the baseline CNP so the gate
outpost is added to the allow set without broadening it.
- `authentik/paperless-gate-egress` (authentik namespace): selects
pods labeled `goauthentik.io/outpost-name=paperless-gate` and
allows egress to the paperless app pod specifically. Scoped to the
gate outpost's label, not the whole authentik namespace, so other
authentik workloads can't reach paperless.
Egress for paperless itself is also expressed as a CNP
(`paperless/paperless-egress`): in-namespace, kube-dns, and external
HTTPS only. Public-internet HTTPS uses Cilium's `world` entity which
already excludes cluster-internal IPs.
Together: the gate outpost is the only ingress path to paperless; the
gate outpost is the only thing in the authentik namespace that's even
permitted to talk to paperless; paperless's egress is restricted to
its known dependencies. Trust boundary lives on the gate outpost.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The earlier `port: 8000` restriction on the trust-boundary
NetworkPolicy resulted in the gate outpost timing out (134 s) when it
tried to forward authenticated requests to paperless via the service
port 80 → pod port 8000. Cilium's policy evaluation order with regard
to kube-proxy DNAT can match either the service port or the pod port
depending on configuration; in this cluster it's matching service
port 80, which was not in the allow-list.
The trust model is identity-based (the gate outpost pod is
trusted), not port-based, so dropping the port restriction is
correct. The outpost is the only path to paperless, and we trust
it on any port it cares to use.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splitting the four hostnames across two Ingresses tripped the
nginx-ingress admission webhook during reconcile (the still-old
`paperless` Ingress claimed all four hosts, blocking creation of the
new `paperless-aliases` Ingress that overlapped on three of them).
Replace with a single `paperless` Ingress whose configuration-snippet
uses an `if ($host != ...)` block to 301 the three alias hosts to the
canonical paperless.renner.uno before the request reaches the outpost.
One Ingress object, no admission conflict, same end-user behaviour.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
nginx-ingress's permanent-redirect annotation validator rejects the
$request_uri variable. Express the same redirect via configuration-
snippet's return 301 directive instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the gate ProxyProvider from forward_domain to proxy mode so the
outpost itself terminates and forwards every request:
ingress-nginx → ak-outpost-paperless-gate → paperless
In forward_domain, ingress-nginx held the trust boundary — it was what
forwarded X-Authentik-Username to paperless after auth_request, so a
compromised ingress-nginx could inject any username. In proxy mode the
outpost is the proxy: ingress-nginx routes the request to the
outpost's :9000 listener, the outpost validates the renner.uno-scoped
session cookie, and the outpost issues the upstream request to
paperless with the gate-attested headers. nginx never sees paperless's
listener.
NetworkPolicy `paperless-trust-boundary` (replaces the previous
nginx-only policy):
- Ingress: ONLY from pods labeled goauthentik.io/outpost-name=
paperless-gate in the authentik namespace, on TCP/8000. ingress-
nginx pods are now blocked from reaching paperless directly, so
even with a valid cookie they cannot bypass the outpost.
- Egress (newly tightened):
- kube-dns on UDP/TCP 53.
- In-namespace traffic to postgres / redis / tika / gotenberg /
protonmail-bridge on their respective ports.
- HTTPS (TCP 443) to public IP space, with RFC1918 / ULA
excluded so an attacker on a compromised paperless can't pivot
to internal-only services over 443.
Ingress rewiring:
- paperless.renner.uno → ak-outpost-paperless-gate ExternalName
(mirror in the paperless namespace), TLS via cert-manager.
- The other three hostnames (paperless.k.renner.uno, docs.renner.uno,
docs.k.renner.uno) move to a `paperless-aliases` Ingress with
permanent-redirect to https://paperless.renner.uno$request_uri.
Authentik proxy mode matches the request Host against a single
external_host per provider; collapsing to one canonical URL is
cleaner than maintaining four parallel ProxyProviders.
- Drop the four nginx forward-auth annotations (auth-url,
auth-signin, auth-response-headers, auth-snippet) and the
defense-in-depth header strip — none of that is in this path
anymore.
The auth.renner.uno/outpost.goauthentik.io ingress in the authentik
namespace stays — the outpost still serves /start and /callback there
during the OAuth dance (the redirect_uri is auto-derived to
paperless.renner.uno/outpost.goauthentik.io/callback now, so the
callback lands at paperless.renner.uno via the proxy ingress, but the
sign-in entry kept on the auth domain doesn't break anything and is
still useful if another forward-auth gate gets added later).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Header-based remote-user auth is only secure if the gate is the only
path to paperless — anything that can talk to the pod directly can
claim any X-Authentik-Username and Django will trust it. Lock it down
on two layers:
1. NetworkPolicy `paperless-ingress-from-nginx-only` restricts ingress
to the paperless pod (port 8000) to traffic from pods in the
`ingress` namespace with label app.kubernetes.io/name=ingress-nginx.
Egress is left open (postgres, redis, tika, gotenberg, protonmail,
external; tightening that is a follow-up).
2. nginx-ingress configuration-snippet on the paperless Ingress clears
any client-supplied X-Authentik-* request headers before they reach
the upstream. The auth-response-headers directive then re-adds them
from the gate outpost's auth response, so paperless only ever sees
values that originated at the gate. (auth-response-headers already
overrides client-supplied values via proxy_set_header ordering, but
the explicit nuke makes the trust boundary unmistakable in the
nginx config.)
Together: in-cluster lateral movement to paperless is blocked by the
NetworkPolicy; legitimate ingress-mediated requests have their
X-Authentik-* identity headers replaced with the gate-attested values
before paperless sees them. There is no other Service or Ingress that
exposes paperless's HTTP port.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Authentik outpost serializer now requires both fields (validated
against /authentik/outposts/api/serializers.py + the live worker's
Importer.validate()). The Caddy/LDAP/Local Radius entries were missing
them — the blueprint apply errored out at the Caddy entry, which
blocked every other state change in the apply (including the OIDC
deletions in the previous commit).
- `LDAP Outpost` (ldap) and `Local Radius` (radius) get the right type
+ provider lists mirroring live state, so blueprint reconcile
doesn't detach UI-managed providers.
- The `Caddy` outpost entry is dropped — its only provider (Caddy
docs.renner.uno) was retired earlier in this session and the outpost
is no longer in the cluster.
- Embedded outpost gets an explicit `type: proxy` for safety.
Validated against the live worker via Importer.validate() with
capture_logs disabled — returns True.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dual-layer setup (gate ProxyProvider + paperless OIDC OAuth2Provider
on a separate Application) was structurally fine but added a second
consent screen on first login for no real defense-in-depth gain. With
both layers gated by the same Paperless Users group, the second consent
just confirms what the gate already validated.
Refactor to gate-only:
- Authentik: state: absent for the OAuth2Provider Paperless, the
separate `paperless` Application that pinned it as primary, and the
early-session orphan `paperless` ProxyProvider. The `paperless-gate`
Application + ProxyProvider + Outpost stay; one Application now
covers paperless end-to-end.
- Paperless config: drop PAPERLESS_APPS=allauth.socialaccount... and
the PAPERLESS_SOCIALACCOUNT_PROVIDERS env binding, add
PAPERLESS_ENABLE_HTTP_REMOTE_USER=true and point
PAPERLESS_HTTP_REMOTE_USER_HEADER_NAME at HTTP_X_AUTHENTIK_USERNAME
so paperless reads the username straight from the gate's
X-authentik-username header.
- sops secret: drop the oidc-providers-json field; keep only the
paperless secret-key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plain OAuth2Provider entries don't get scope mappings auto-attached
the way ProxyProvider.set_oauth_defaults() does. Without them the
access token issued during paperless's allauth callback has no scope
values, /application/o/userinfo/ returns 403, and paperless renders
"An error occurred while attempting to login via your social network
account" — which is what the user just hit. Bind the three default
managed ScopeMappings (openid / profile / email).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each Authentik proxy outpost has its own session store. Sharing the
provider between the embedded outpost and the dedicated gate outpost
means the OAuth callback at /outpost.goauthentik.io/callback hits the
embedded outpost (which has no in-cluster /auth/nginx exposure), the
session is created in the embedded outpost's store, and the dedicated
gate outpost — which is what nginx auth_request actually queries —
never sees it. Result: an infinite redirect between /authorize and
/callback after every OAuth round-trip, surfaced as
ERR_TOO_MANY_REDIRECTS in the browser.
Add an Ingress in the authentik namespace that routes
auth.renner.uno/outpost.goauthentik.io/* to the dedicated outpost's
ClusterIP service, and detach the provider from the embedded outpost.
The dedicated outpost now serves /start, /callback, AND /auth/nginx
in the same process. The Authentik UI flow paths (/if, /application/o,
/api, /static, /flows, /media) continue to resolve via the chart's
authentik-server Ingress at the broader `/` rule, so 2FA still happens
on auth.renner.uno where credentials are bound.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ProxyProvider.set_oauth_defaults() derives redirect_uris from
external_host via _get_callback_url. Pointing the gate provider's
external_host at https://auth.renner.uno makes the OAuth callback URL
land on auth.renner.uno/outpost.goauthentik.io/callback — handled by
the embedded outpost (which has the gate provider attached). The
session cookie is renner.uno-scoped via cookie_domain so the dedicated
paperless-gate outpost picks it up on its in-cluster /auth/nginx
checks against the protected paperless hostnames.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WebAuthn / FIDO / TOTP credentials are bound to the origin they were
registered on (auth.renner.uno). Routing the OAuth flow through
paperless.renner.uno meant the browser couldn't release the credential
during the 2FA stage.
Restructure so the dedicated paperless-gate outpost only serves the
in-cluster /outpost.goauthentik.io/auth/nginx endpoint that nginx
auth_request hits, while every browser-facing leg of the flow (/start,
/callback, the /if/flow login UI, /application/o/authorize, /api,
/static, /flows, /media) happens on auth.renner.uno via the embedded
outpost. Both outposts have the gate's ProxyProvider attached, so the
embedded outpost can issue the OAuth code and the dedicated outpost
can validate the resulting cookie. The cookie is renner.uno-scoped so
it transfers between the two hosts.
Drop the paperless-namespace ExternalName services and the second
paperless-gate-outpost Ingress that was routing Authentik UI paths on
paperless.renner.uno — no longer needed. Update auth-signin to point
at auth.renner.uno.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Authentik authorize endpoint 302s into /if/flow/... (the web UI for
the login flow), and the UI in turn loads /api, /static, /flows, /media.
With authentik_host_browser=paperless.renner.uno, all of those paths
need to land on authentik-server, not on paperless. Add Prefix rules
for each so the browser-side OAuth/login flow can fully render and
complete on the gate hostname.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dedicated paperless-gate outpost has authentik_host_browser set to
paperless.renner.uno, so the OAuth authorize URL it 302s to during /start
is on paperless.renner.uno/application/o/authorize/. That path wasn't
routed anywhere on the gate ingress, so it fell through to the main
paperless Ingress, which has auth_request enabled, which sent the
browser back to /outpost.goauthentik.io/start, accumulating state in
the rd= parameter on each loop. After a few iterations the response
headers exceeded nginx's proxy_buffer_size and the ingress returned 502.
Add a second path rule on the paperless-gate-outpost Ingress for
/application/o/ → authentik-server (via an ExternalName service mirror
in the paperless namespace), so the OIDC authorize flow served by
authentik-server reaches the right backend.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Authentik blueprint schema (goauthentik.io/blueprints/schema.json,
$defs.model_authentik_outposts.outpost.properties.type) requires `type`
on Outpost entries (enum: proxy/ldap/radius/rac). Existing UI-created
outposts skip this in the blueprint because their type is preserved
from creation, but new outposts created from blueprint must set it
explicitly. Without it the serializer rejects the entry, the whole
blueprint apply rolls back, and nothing — not even the OAuth2Provider
above — gets created.
Validated with the live worker's importer: with `type: proxy` set,
validate now returns True instead of failing with the cascade error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the paperless Application's provider reference from !Find to
!KeyOf paperless-provider so it resolves against the OAuth2Provider
created earlier in the same blueprint (matters on first apply, before
the row is in the DB).
Drop the state: absent for the orphan `paperless` ProxyProvider —
Application.provider is on_delete=PROTECT, so the cleanup-vs-update
ordering deadlocks at validate time. Leftover orphan stays for manual
UI cleanup once the new wiring is confirmed working.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolve the cross-entry reference for the paperless app's OAuth2
primary via DB lookup instead of !KeyOf. The OAuth2Provider was
already created by an earlier blueprint apply, so !Find resolves
cleanly. !KeyOf for the new paperless-gate ProxyProvider stays
because that provider is born inside this blueprint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the user's intent — defense in depth with a separate gate server
that handles authn/authz before any request reaches paperless's own
attack surface. Architecture:
- New dedicated outpost `paperless-gate` (its own k8s Deployment
spawned by Authentik via the kubernetesserviceconnection).
- New ProxyProvider `paperless-gate` (forward_domain, cookie scoped to
renner.uno so all four paperless aliases share one session).
- New Application `paperless-gate` with the `Paperless Users` group
policy binding — every request through the gate is checked against
that group, including for forward_domain mode.
- Tear down the previous embedded-outpost experiment (paperless-proxy
Application + paperless ProxyProvider).
Paperless ingress wiring:
- Forward-auth annotations on the main ingress point auth_request at
the dedicated outpost's in-cluster service, and auth-signin at
https://paperless.renner.uno/outpost.goauthentik.io/start.
- A separate ingress (no auth annotations, to avoid the auth_request
loop on /start itself) routes /outpost.goauthentik.io/* on
paperless.renner.uno to the outpost service.
- ExternalName Service mirrors the authentik-namespace outpost into
the paperless namespace so the Ingress can backend cross-namespace.
The OAuth2Provider on the existing `paperless` Application stays as
its primary, unchanged — paperless's own django-allauth OIDC SSO
operates BEHIND the gate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The official Authentik integration guide for paperless-ngx
(integrations.goauthentik.io/documentation/paperless-ngx) prescribes
OAuth2/OpenID Connect as the application's primary provider with
redirect URI `https://<host>/accounts/oidc/authentik/login/callback/`
— no proxy/forward-auth involved. Paperless's own login page, backed
by Authentik OIDC, is the gate.
Revert the forward-auth detour:
- Drop nginx auth_request annotations from the paperless ingress.
- Replace the ProxyProvider blueprint entry with a declarative
OAuth2Provider matching the original credentials (so the existing
sops-encrypted client_id/client_secret keep working). Pin it as the
paperless Application's primary provider.
- State: absent the leftover ProxyProvider, paperless-proxy
Application, and its policy binding.
- Detach the embedded outpost's providers list (no forward-auth).
- Restore the sops oidc-providers-json client_id/client_secret to the
original values (corrected URL stays at auth.renner.uno).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The proxy provider's redirect_uris auto-included only the forward-auth
callback, so paperless's django-allauth OIDC client got 400 'Redirect
URI Error' when trying https://<host>/accounts/oidc/authentik/login/
callback/. Add a regex entry covering all four paperless hostnames,
plus keep the forward-auth callbacks so the proxy/auth_request path
still works.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Blueprint apply errored with `Application with this provider already
exists. (code='unique')` — the ProxyProvider was still primary on
`paperless-proxy` when the apply tried to re-attach it as primary on
`paperless`. Reorder so the `paperless-proxy` Application + binding are
deleted (state: absent) FIRST, then the provider gets re-bound to
`paperless`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous push errored on apply with `KeyOf: failed to find entry
with id of paperless-provider and a model instance` — the in-blueprint
!KeyOf reference wasn't resolving when both the application's primary
provider and the embedded outpost referenced it. Switch to !Find
against the DB (the ProxyProvider already exists there), which is the
more robust pattern when the target object is persistent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Authentik ProxyProvider is itself an OAuth2Provider — it serves
both /outpost.goauthentik.io/auth/nginx (forward-auth) and the standard
OIDC endpoints under /application/o/<slug>/... — so the proper pattern
is one Application with the ProxyProvider as primary, gating both flows
under the same PolicyBinding.
Tear down the temporary `paperless-proxy` split-Application and its
PolicyBinding, retire the now-redundant standalone `Paperless` OAuth2
provider, and repoint paperless's allauth `oidc-providers-json` at the
ProxyProvider's client_id / client_secret so its OIDC sign-in talks to
the same provider the forward-auth gate runs through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backchannel attachment of the ProxyProvider to the OIDC `paperless` app
left the embedded outpost's /auth/nginx returning 404 — the outpost
resolves forward-auth via the Application's primary provider only, not
backchannel providers, so the host wasn't routable. Surfaced as 500s on
every paperless hostname.
Create a separate `paperless-proxy` Application with the ProxyProvider as
primary, and bind the same `Paperless Users` group policy. The OIDC
`paperless` Application keeps its OAuth2 provider as primary and is
otherwise untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous blueprint hijacked the existing `paperless` Application by
overwriting its primary provider with the new forward-auth ProxyProvider,
which left paperless's OIDC OAuth2 provider (client_id=3jhutuBrnJ…)
orphaned (no Application bound) and broke the django-allauth integration
underneath the gate.
Switch to Authentik's intended pattern: the OAuth2 provider stays primary
on the Application (paperless's own SSO works), the ProxyProvider is
attached as a backchannel for the embedded outpost's forward-auth path.
The Paperless Users group binding governs both flows uniformly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sops-encrypted oidc-providers-json carried `https://auth.k.renner.uno/...`
as the OIDC discovery endpoint, but no ingress or DNS record exists for
`auth.k.renner.uno` — only the public `auth.renner.uno` is served by
authentik-server. Visiting paperless's "Sign in with Authentik" link
(/accounts/oidc/authentik/login/) crashed with a 500 because django-allauth
couldn't resolve the host. The pre-existing typo only became visible once
forward-auth started routing users through paperless's OIDC integration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add nginx auth_request annotations to the paperless ingress and a
matching forward_domain ProxyProvider + Application in the Authentik
blueprint, attached to the embedded outpost. cookie_domain=renner.uno
shares the session across all four paperless aliases (paperless.* and
docs.*, both .renner.uno and .k.renner.uno).
Also drop the half-built `Caddy docs.renner.uno` provider that was
producing a phantom ak-outpost-caddy ingress conflicting with the
paperless ingress on docs.renner.uno.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to the Caddy retirement work on master today. RustFS was
the S3 backend behind two consumers:
* caddy-certificates bucket -- dies with Caddy.
* legacy tofu-state bucket -- migrated to Garage some time ago
(per the providers.tf comment in opentofu/, the tofu state
backend already uses Garage via AWS_ACCESS_KEY_ID env).
With Caddy retired and tofu-state on Garage, nothing reads from
RustFS anymore. Flipping started + start_on_boot to false on
rustfs-{1,2,3} (vmid 800/801/802 across jupiter/saturn/neptune).
Apply path: bootstrap project uses local state and has no CI
pipeline by design ("Changes to this infrastructure should be rare
and applied manually" -- rustfs.tf:5). Run from a dev box:
cd opentofu-bootstrap && tofu plan && tofu apply
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan against my Caddy-retirement work surfaced two unrelated USB-
passthrough drifts on the HA VMs:
vm_103 (legacy HAOS, being shut down): tofu config still had the
stromzaehler + ZWA-2 USB devices declared, but live state has them
removed (rebinded to vm_153 k8s-edge at the HA-on-K8s cutover).
Plan was about to re-add them. Dropped the usb block.
vm_153 (running K8s edge node): block order and usb3 in config
didn't match live -- live has ZWA-2 first / stromzaehler second
with usb3=true; config had the inverse. Plan would have flipped
usb3 to false on a running VM (real change) and reordered. Updated
the config to match live so plan shows zero diff for vm_153.
No functional change: vm_103 stays stopped without the USB block;
vm_153's running USB passthrough is unchanged. This unblocks the
Caddy-retirement apply by leaving plan with only the intended scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pairs with the K8s file/photos commit. Empties every ingress_cnames
list still pointing at the (now stopped) Caddy VIP, so external-dns-unifi
can take ownership of the hostnames where a K8s Ingress is the new
owner, and stale hostnames simply stop resolving.
caddy_srv_renner_uno (ingress.tf) file/dir/caddy.renner.uno -> []
file: served by apps/file
dir, caddy: stale, no live backend
lxc_121 ical-proxy ical-proxy.renner.uno [K8s apps/ical-proxy]
lxc_212 audiobookshelf audiobooks.renner.uno [K8s apps/audiobookshelf]
lxc_225 immich photos.renner.uno [K8s apps/immich (alias)]
lxc_226 loki loki.renner.uno [K8s monitoring/loki-ingress]
lxc_229 music-assistant music.renner.uno [K8s apps/music-assistant]
vm_111 paperless paperless/docs.renner.uno [K8s apps/paperless]
Hostnames with no K8s replacement (services being retired entirely)
will stop resolving after apply:
lxc_227 prometheus prometheus.renner.uno
vm_108 influxdb influx/influxdb.renner.uno
vm_109 grafana grafana.renner.uno
Also removed the unifi_dns_record.caddy_vip A record
(ingress.renner.uno -> 10.2.45.100). Nothing CNAMEs at it any more,
and the keepalived VIP it pointed at goes away with the Caddy LXCs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two LAN-only Caddy hostnames absorbed into ingress-nginx:
* file.renner.uno -- new apps/file/ ExternalName Service +
Ingress, backend https://nas.renner.uno:7001 with
backend-protocol=HTTPS, proxy-ssl-verify=off. Same shape Caddy
used. file.renner.uno was the only "stale-by-name but still
serving 200" hostname after the migrated/dead audit, so it
needed a live K8s home before Caddy goes off.
* photos.renner.uno -- alias added to the existing apps/immich
Ingress (which was already serving immich.renner.uno and
immich.k.renner.uno). Caddy never had a route for photos --
instead the tofu CNAME on lxc_225 immich pointed it at the
Caddy VIP, where it was caught by the immich.renner.uno block.
Adding the alias here lets external-dns-unifi take ownership
of photos.renner.uno after the LXC's tofu ingress_cnames is
cleared in the companion commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DNS: removed pve/home/auth/dash from caddy_srv_renner_uno.ingress_cnames
in opentofu/ingress.tf. pve is now fronted by the new K8s ingress-nginx
path (apps/pve Service + EndpointSlice with the three cluster IPs);
home and auth have been served by ingress-nginx for a while and the
live UniFi A records are owned by external-dns-unifi -- the orphan
CNAMEs in the file were drift that would have been re-created on next
apply, breaking the working setup. dash backend is dead (502 via
Caddy, confirmed via curl).
Compute: every Proxmox guest under tofu management is now flipped to
stopped/onboot=false, except LXC 250 (registry-mirror) which keeps
flux/woodpecker image pulls working:
LXCs (start=false, onboot=false, hastate="stopped"):
117 cloudflared -- replaced by K8s cloudflared Deployment
121 ical-proxy -- replaced by K8s apps/ical-proxy
212 audiobookshelf -- replaced by K8s audiobookshelf
225 immich -- replaced by K8s immich
226 loki -- retired
227 prometheus -- replaced by K8s monitoring
229 music-assistant -- replaced by K8s music-assistant
232/233/234 caddy -- this whole retirement work
260 netbird-gateway -- retired
130/131/132/133 authentik HA -- replaced by K8s authentik
VMs (vm_state="stopped", onboot=false):
102 fortress
103 home-assistant -- replaced by K8s home-assistant
108 influxdb
109 grafana -- replaced by K8s monitoring
111 paperless -- replaced by K8s paperless
Module change: removed `started` from the proxmox_vm module's
ignore_changes list. The previous behaviour ("Don't force start/stop
state") meant tofu would silently drop vm_state changes, leaving the
five VMs above running indefinitely. The trade-off is that any VM
manually stopped for maintenance (e.g. a K8s node) will be auto-
restarted on the next tofu apply -- be aware.
`hastate` is dead-code in both proxmox_lxc and proxmox_vm modules
(declared as a variable, not consumed anywhere). The hastate="stopped"
edits are documentation only -- Proxmox HA state still has to be
managed via the Proxmox UI / pve-ha-manager / API. If pve-ha is
holding any of these in started state, it will fight the tofu stop.
LAN-side CNAMEs on the per-LXC ingress_cnames lists (e.g.
photos.renner.uno on lxc_225 immich) are intentionally left alone
in this commit -- they'll either be auto-cleared by external-dns-unifi
once the K8s Ingress claims the name, or fall stale pointing at a
dead Caddy VIP. Cleaning those up is a follow-up sweep.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace Caddy's reverse_proxy load balancer for pve.renner.uno (which
fanned out to https://{jupiter,neptune,saturn}.renner.uno:8006 with
health checks and Caddy's import loadbalancer) with an ingress-nginx
Service backed by a manual EndpointSlice listing all three PVE
cluster IPs. ingress-nginx's upstream pool with proxy-next-upstream
gives the same node-down failover behaviour Caddy provided.
Hardcoded IPs (10.2.40.201/202/203) are an explicit exception to the
"IPs aren't static" rule that drove the gitea/woodpecker ExternalName
shape -- corosync requires stable cluster member identity, so PVE
node IPs are static by construction. The endpointslice.yaml header
calls this out for the next person to read it.
backend-protocol=HTTPS + proxy-ssl-verify=off because PVE serves a
per-node self-signed cert on :8006; ingress-nginx terminates the
public TLS for pve.renner.uno via cert-manager DNS-01.
proxy-buffering=off and 1h read/send timeouts cover noVNC console
and long-running task-log streams.
Companion tofu commit removes pve.renner.uno from the caddy
ingress_cnames so external-dns-unifi can write the A record to
10.2.40.80.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to the K8s-side commit. Removing the CNAME lets
external-dns-unifi take ownership of ci.renner.uno on the next
reconcile and write an A record pointing at the ingress-nginx LB
(10.2.40.80) instead of the Caddy VIP (10.2.45.100).
Apply via the renner_homelab tofu pipeline; agents are unaffected
because they target woodpecker.srv.renner.uno:9000 directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same shape as the gitea commit before: ExternalName Service in apps
namespace pointing at woodpecker.srv.renner.uno:8000 (the HTTP UI),
plus an internal-only Ingress for ci.renner.uno. cert-manager DNS-01
issues the cert.
Agents (LXC 910/911) target woodpecker.srv.renner.uno:9000 directly
over gRPC -- not via Caddy, not via this Service, not via
ingress-nginx -- so there is nothing to migrate at the gRPC layer.
The Caddyfile entry for ci.renner.uno only proxied :8000 anyway.
Companion tofu commit removes the ci.renner.uno CNAME so
external-dns-unifi can take ownership of the LAN A record.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to the K8s-side commit that put gitea behind ingress-nginx
via an ExternalName Service. Removing the two CNAMEs (gitea.renner.uno
and git.renner.uno -> ingress.renner.uno) lets external-dns-unifi take
ownership of those names on the next reconcile and write A records
pointing at the ingress-nginx LB (10.2.40.80) instead of the Caddy VIP
(10.2.45.100).
Apply path: tofu plan + apply on the renner_homelab pipeline. After
apply, external-dns-unifi reconciles within ~1 min; LAN clients pick
up the new A records as their resolvers' caches expire (UniFi DNS
TTL=300s by default for upserted records).
Caddy still has its own Caddyfile entries for these hostnames -- those
become dead config once DNS no longer points at the Caddy VIP. Cleaning
them up is a separate ansible-side commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caddy on LXCs 232/233/234 currently terminates TLS for git.renner.uno
and gitea.renner.uno and reverse-proxies to the gitea LXC at
gitea.srv.renner.uno:3000. This adds an ingress-nginx Ingress for the
same two hostnames, internal-only (no cloudflare-proxied annotation),
fronted by a single ExternalName Service pointing at the LAN DNS
name -- not an IP, so DHCP renewal on the LXC is invisible to the
cluster.
cert-manager DNS-01 against the renner.uno Cloudflare zone gets the
cert without WAN reachability, matching how vscode.home.renner.uno
was issued. external-dns-unifi (domainFilters: [k.renner.uno,
renner.uno]) will only take ownership of the LAN A records once the
existing tofu-managed CNAMEs (ingress.renner.uno -> 10.2.45.100,
defined in opentofu/lxc_114_gitea_srv_renner_uno.tf:32-35) are
removed; that's a separate tofu commit + apply.
Caddy stays in the data path until that DNS swap. Once tofu drops
the CNAMEs and external-dns-unifi writes A records to 10.2.40.80,
git/gitea.renner.uno will resolve to ingress-nginx and Caddy will be
unused for these two hostnames.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vscode.home.renner.uno (LAN-only) edits HA's /config in a browser.
Sidecar in the HA pod rather than a separate Deployment because the
config PVC is RWO ceph-block; the alternative was migrating to RWX
ceph-filesystem, which would touch HA's running state for no other
benefit.
Auth is --auth=none — trust-the-network on the LAN. The Ingress
omits the cloudflare-proxied annotation, so external-dns-unifi
publishes a UniFi DNS A record for vscode.home.renner.uno -> the
ingress-nginx LB and external-dns-cloudflare leaves it alone.
DNS-01 via Cloudflare gets the cert without WAN reachability.
Pod restart caveat: rolling the home-assistant Deployment now
restarts HA + config-reconciler + code-server together, which is
fine on a Recreate strategy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add external-dns cloudflare-proxied annotation to the home-assistant
Ingress so a CNAME is provisioned to the tunnel, and add the matching
hostname rule to cloudflared so it forwards to ingress-nginx. Same
single-Ingress / both-hosts-with-annotation pattern that gatus and
authentik already use; no Authentik forward-auth gating, HA's own
auth is the only gate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The blueprint validator failed the apply with `KeyOf: failed to find
entry with id of k8s-conn and a model instance` because for a
freshly-created outpost the validator's KeyOf resolution doesn't see
entry 1's `_state.instance` mid-pass. Existing Caddy/LDAP/Radius
outposts dodge that by matching an existing DB row directly.
Hard-code the `Local Kubernetes Cluster` connection's literal UUID on
the new entry; the connection is `local: true` and never recreated, so
the UUID is stable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 2 of the gating split. Now that step 1 has applied and removed
home.renner.uno / docs.renner.uno from the live LAN Ingresses, the
new *-public Ingresses can be added without the validating webhook
flagging them as duplicate-host with the LAN Ingresses.
Both carry the Authentik forward-auth annotations pointing at
ak-outpost-nginx.authentik.svc.cluster.local:9000.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
k.rennerandClaude Opus 4.7