Compare commits

..
Author SHA1 Message Date
claude-botandClaude Fable 5.1 d82b96a751 fix(wiki-index): notes repository branch is main
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 10:09:43 +02:00
claude-botandClaude Fable 5.1 463cfa7bde feat(ai): deploy wiki-index, the embedding index service for the Notes repo
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
Deployment + Service + PVC + ServiceAccount in namespace ai, LAN-only
Ingress wiki-index.renner.uno behind Traefik BasicAuth (ExternalSecret
from OpenBao, same shape as wiki-embed), Flux Kustomization depending on
llama-server (wiki-embed is the embedding backend), pod-scoped
CiliumNetworkPolicy admitting Traefik and Gitea (push webhook) on 8080
and allowing egress to OpenBao 8200, and the OpenBao policy for the
service's Kubernetes auth role.

Image: gitea.renner.uno/gitea_admin/wiki-index:2026-09-20, built by the
step added in the previous PR; the tag is the build date.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 08:47:26 +02:00
k.renner 761bf9ba1e Merge pull request 'fix(eso): allow namespace ai on the openbao ClusterSecretStore' (#872) from fix/eso-allow-ai into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#872
2026-09-20 05:48:21 +00:00
claude-botandClaude Fable 5.1 7b446194b3 fix(eso): allow namespace ai on the openbao ClusterSecretStore
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
The wiki-embed-basic-auth ExternalSecret from #871 never synced:
"using cluster store openbao is not allowed from namespace ai: denied by
spec.condition". The store's namespace allowlist predates the ai
namespace. Adding it, sorted, as the manifest's own maintenance note
prescribes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 07:47:43 +02:00
k.renner 9eaafeb92f Merge pull request 'feat(ai): dedicated wiki-embed endpoint for the Notes ranker' (#871) from feat/wiki-embed into master
ci/woodpecker/cron/issue-fix Pipeline was successful
ci/woodpecker/cron/alert-monitor Pipeline was successful
Reviewed-on: k.renner/homelab#871
2026-09-20 05:38:26 +00:00
7 changed files with 275 additions and 0 deletions
+136
View File
@@ -0,0 +1,136 @@
---
# ServiceAccount is the OpenBao identity (auth/kubernetes/role/wiki-index); no
# Kubernetes API access needed.
apiVersion: v1
kind: ServiceAccount
metadata:
name: wiki-index
namespace: ai
labels:
app.kubernetes.io/name: wiki-index
---
# Index (~17 MB JSON for 1700 notes). Survives restarts so a pod start does not
# re-embed the wiki; a lost volume only costs one full sync (~5 min).
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wiki-index-data
namespace: ai
spec:
accessModes:
- ReadWriteOnce
storageClassName: ceph-block
resources:
requests:
storage: 1Gi
---
# Embedding index over the Notes repo's wiki/ (ci/images/wiki-index). Syncs from
# Gitea on push webhook and every RESYNC_SECONDS, embeds via wiki-embed, serves
# POST /rank to ~/Notes/bin/wiki_rank.py through wiki-index.renner.uno.
apiVersion: apps/v1
kind: Deployment
metadata:
name: wiki-index
namespace: ai
labels:
app.kubernetes.io/name: wiki-index
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: wiki-index
template:
metadata:
labels:
app.kubernetes.io/name: wiki-index
spec:
serviceAccountName: wiki-index
securityContext:
runAsNonRoot: true
runAsUser: 65534
runAsGroup: 65534
fsGroup: 65534
seccompProfile:
type: RuntimeDefault
containers:
- name: wiki-index
image: gitea.renner.uno/gitea_admin/wiki-index:2026-09-20
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
env:
- name: GITEA_URL
value: https://gitea.renner.uno
- name: NOTES_REPO
value: k.renner/notes
- name: NOTES_BRANCH
value: main
- name: EMBED_URL
value: http://wiki-embed.ai.svc.cluster.local:8080
- name: BAO_ADDR
value: http://openbao.openbao.svc.cluster.local:8200
- name: BAO_ROLE
value: wiki-index
- name: BAO_SECRET_PATH
value: homelab/data/apps/ai/wiki-index
- name: INDEX_PATH
value: /data/index.json
- name: RESYNC_SECONDS
value: "1800"
ports:
- name: http
containerPort: 8080
# Pure-Python cosine over 1700 x 1024 floats per query; the index itself is ~60 MB in memory.
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
# /health is 503 until the first sync produced an index. Cold start with an
# empty PVC embeds every note: budget 15 min.
startupProbe:
httpGet:
path: /health
port: http
periodSeconds: 10
failureThreshold: 90
readinessProbe:
httpGet:
path: /health
port: http
periodSeconds: 30
livenessProbe:
httpGet:
path: /health
port: http
periodSeconds: 60
timeoutSeconds: 10
failureThreshold: 10
volumeMounts:
- name: data
mountPath: /data
volumes:
- name: data
persistentVolumeClaim:
claimName: wiki-index-data
---
apiVersion: v1
kind: Service
metadata:
name: wiki-index
namespace: ai
labels:
app.kubernetes.io/name: wiki-index
spec:
selector:
app.kubernetes.io/name: wiki-index
ports:
- name: http
port: 8080
targetPort: http
+56
View File
@@ -0,0 +1,56 @@
---
# wiki-index.renner.uno — LAN-only (no dns.renner.uno/public label), machine
# client (bin/wiki_rank.py), so Traefik BasicAuth like wiki-embed.renner.uno.
# Gitea's webhook does not pass here; it targets the ClusterIP Service directly.
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: wiki-index-basic-auth
namespace: ai
spec:
basicAuth:
secret: wiki-index-basic-auth
realm: wiki-index
---
# OpenBao key apps/ai/wiki-index-basic-auth, field `users` = htpasswd line.
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: wiki-index-basic-auth
namespace: ai
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: openbao
target:
name: wiki-index-basic-auth
dataFrom:
- extract:
key: apps/ai/wiki-index-basic-auth
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: wiki-index
namespace: ai
annotations:
cert-manager.io/cluster-issuer: letsencrypt-cluster
traefik.ingress.kubernetes.io/router.middlewares: ai-wiki-index-basic-auth@kubernetescrd
spec:
ingressClassName: traefik
tls:
- hosts:
- wiki-index.renner.uno
secretName: wiki-index-tls
rules:
- host: wiki-index.renner.uno
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: wiki-index
port:
number: 8080
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- ingress.yaml
@@ -0,0 +1,22 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: wiki-index
namespace: flux-system
spec:
interval: 10m
retryInterval: 1m
# First start on an empty PVC embeds every note (~5 min) before /health is 200.
timeout: 20m
sourceRef:
kind: GitRepository
name: flux-system
path: ./kubernetes/apps/wiki-index
prune: true
wait: true
dependsOn:
- name: infrastructure
- name: infrastructure-crds
- name: infrastructure-ceph-cluster
# wiki-embed (the embedding backend) lives in the llama-server app.
- name: llama-server
@@ -148,3 +148,37 @@ spec:
- ports:
- port: "8080"
protocol: TCP
---
# wiki-index: Traefik (LAN clients behind BasicAuth) and Gitea (push webhook)
# may reach it; it reaches OpenBao for its credentials. Gitea itself is reached
# through Traefik, which the namespace egress above already allows.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: wiki-index
namespace: ai
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: wiki-index
ingress:
- fromEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": ingress
app.kubernetes.io/name: traefik
- matchLabels:
"k8s:io.kubernetes.pod.namespace": apps
app.kubernetes.io/name: gitea
toPorts:
- ports:
- port: "8080"
protocol: TCP
egress:
- toEndpoints:
- matchLabels:
"k8s:io.kubernetes.pod.namespace": openbao
app.kubernetes.io/name: openbao
toPorts:
- ports:
- port: "8200"
protocol: TCP
@@ -0,0 +1,21 @@
# OpenBao policy: wiki-index
#
# The wiki-index Deployment (namespace ai) reads its Gitea token and the
# webhook HMAC secret on every use. Read-only on that one path.
#
# NOT applied by Flux. Apply ONCE with an admin token:
#
# bao policy write wiki-index kubernetes/infrastructure/openbao/policies/wiki-index.hcl
# bao write auth/kubernetes/role/wiki-index \
# bound_service_account_names=wiki-index \
# bound_service_account_namespaces=ai \
# policies=wiki-index ttl=15m
#
# Secret: homelab/apps/ai/wiki-index, keys `gitea_token` (read:repository on
# the notes repo) and `webhook_secret` (the Gitea webhook's secret).
#
# Do NOT set token_no_default_policy=true (kv-v2 preflight needs the default policy).
path "homelab/data/apps/ai/wiki-index" {
capabilities = ["read"]
}
@@ -14,6 +14,7 @@ spec:
# ClusterSecretStore" and the secret never syncs. Keep this list sorted.
conditions:
- namespaces:
- ai # wiki-embed-basic-auth (apps/llama-server/wiki-embed-ingress.yaml)
- apps
- audiobookshelf
- authentik