diff --git a/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml b/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml index 3d3dae70..3e059817 100644 --- a/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml +++ b/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml @@ -73,3 +73,29 @@ spec: interval: 30s podTargetLabels: - role +--- +# wp-pg, restored into the `woodpecker` namespace (issue #433). It has never +# been scraped: `apps` had no PodMonitor for it AND its baseline network policy +# never allowed the monitoring namespace, so `cnpg_collector_up` has series for +# the four clusters above but not this one. The namespace move fixes the policy +# half; without this PodMonitor there is still no scrape target at all. +# +# Deliberately not adding one for the cluster still in `apps`: it is retired in +# stage 3, and a PodMonitor there would only be reconciled to be deleted again. +apiVersion: monitoring.coreos.com/v1 +kind: PodMonitor +metadata: + name: cnpg-wp-pg + namespace: woodpecker + labels: + app.kubernetes.io/name: cloudnative-pg + release: kube-prometheus-stack +spec: + selector: + matchLabels: + cnpg.io/cluster: wp-pg + podMetricsEndpoints: + - port: metrics + interval: 30s + podTargetLabels: + - role diff --git a/kubernetes/apps/woodpecker/MIGRATION-namespace.md b/kubernetes/apps/woodpecker/MIGRATION-namespace.md new file mode 100644 index 00000000..70b2b2ac --- /dev/null +++ b/kubernetes/apps/woodpecker/MIGRATION-namespace.md @@ -0,0 +1,204 @@ +# Woodpecker: `apps` → `woodpecker` namespace migration + +Moves the Woodpecker service — server, agent and the `wp-pg` CNPG backend — out +of the shared `apps` namespace into one of its own. Distinct from +`MIGRATION.md`, which covers the earlier SQLite → CNPG move. + +## Why + +`apps` holds gitea, gatus, external-dns, ical-proxy and woodpecker together, +and its `default-deny-ingress` uses `endpointSelector: {}`. Every allow rule it +carries therefore applies to *every* workload in the namespace. + +That is not academic. Issue #433 — `wp-pg` stuck at `Ready=False` with +`Instance Status Extraction Error` for three weeks — is caused by the CNPG +operator in `database` being unable to reach `https://:8000/pg/status`. +The only namespace-level fix is `fromEndpoints: namespace=database`, which +would expose gitea, gatus, external-dns and ical-proxy to the whole `database` +namespace to solve a problem affecting one pod on one port. + +`cnpg_collector_up` has series for authentik, immich, monitoring and paperless +but not for `wp-pg`, for the same reason: `apps` never allowed the monitoring +namespace either. + +A namespace per service keeps each grant on the workload that needs it. + +**`woodpecker-ci` is not the target.** It enforces `pod-security: privileged` +and exists solely for ephemeral pipeline step pods running repo-supplied code. +Putting the database there would let any build step reach it through the +intra-namespace allow. + +## Staging + +Rollback is trivial through stage 2. The `apps` cluster is untouched until +stage 3, so reverting means pointing the DSN back and scaling up. + +| Stage | Contents | CI impact | +|---|---|---| +| **1** (this PR) | `woodpecker` namespace, its network policies, ExternalSecret, ObjectStores, and `wp-pg` restored from barman | none — parallel, nothing points at it | +| **runbook** | verify the restore holds real data | none | +| **2** | move server + agent, flip DSN, RBAC subject, rotator policies, docs | **downtime** | +| **3** | delete the `apps` originals, add the prune guard to the survivor | none | + +## Stage 1 design notes + +**Two ObjectStores, deliberately.** `wp-pg-store-source` points at +`s3://renner-uno-k8s-cnpg/wp-pg` — the prefix the live cluster archives to — +and is read-only. `wp-pg-store` points at a **new** prefix, +`s3://renner-uno-k8s-cnpg/wp-pg-woodpecker`, for ongoing archiving. + +Both clusters writing one `destinationPath` would interleave their WAL streams +and corrupt both. Splitting them also leaves the pre-migration backup intact +and independently restorable. + +**`serverName: wp-pg`** on the `externalCluster` names the cluster that +*produced* the backup, not the one being created. + +**The namespace is declared in `infrastructure/namespaces/namespaces.yaml`,** +not alongside these workloads. Its CiliumNetworkPolicies ship with the +`infrastructure-crds` KS (`infrastructure/post-crds` includes +`../network-policies/`), which `dependsOn` `infrastructure` — so the namespace +exists before any policy targets it. Creating it in the `woodpecker` workload +KS would invert that order, since that KS runs after both. + +**Ingress from `woodpecker-ci` is deliberately absent in stage 1.** Step pods do +call the server back on `:8000`, but the CNPG instance manager serves its status +API on `:8000` too, and the baseline policy uses `endpointSelector: {}`. A +`toPorts` block narrows the port, not the destination workload — so putting that +rule on the baseline would hand arbitrary repo-supplied build steps a route to +wp-pg's instance manager. It lands in stage 2 as its own policy selecting the +server pod. + +**No `prune: disabled` guard on the restored cluster.** The original carries one +because it holds the only copy of the data. This one is reconstructable by +re-running the restore, and a prune guard on a migration artifact is how orphans +survive an abandoned migration. It gains the guard in stage 3. + +**Barman credentials still come from `seed/paperless/hetzner-creds`.** Issue +#397 wants woodpecker to own `seed/woodpecker/hetzner-creds`, and this would +have been the natural moment — but creating that path needs OpenBao write +access that was not available (`bao kv list seed` → 403). Repointing at a path +that may not exist would leave the restored cluster unable to archive WAL, +which is the thing this stage exists to prove. #397 stays open; the swap is one +line once the path exists. + +## Expect the `woodpecker` Kustomization to go not-Ready during the restore + +`kubernetes/clusters/homelab/woodpecker.yaml` sets `wait: true` and a +`healthCheckExprs` entry matching `postgresql.cnpg.io/v1 Cluster` **with no +name or namespace qualifier**. Flux evaluates it against every Cluster in that +Kustomization's inventory — which, after this stage, is both the live +`apps/wp-pg` and the restoring `woodpecker/wp-pg`. + +So while the restore runs, the whole `woodpecker` KS — the one that owns +production Woodpecker — reports not-Ready, and a reconcile timeout in that +window will trip the `flux-failures` Alert. **That is expected and is not +Woodpecker breaking.** Nothing `dependsOn: woodpecker`, so it does not cascade. + +If the restore is going to sit for a long time, or the noise is unwelcome, the +options are to suspend the KS while it completes, or to split the migration +resources into their own `wait: false` Kustomization. For a ~660 MB database the +window should be minutes. + +## Verifying stage 1 + +Restore is asynchronous. Watch it reach a healthy state: + +```sh +kubectl -n woodpecker get cluster wp-pg -w +kubectl -n woodpecker get pods +``` + +Expect `Cluster in healthy state`. If it sticks in bootstrap, the recovery job's +logs carry the reason: + +```sh +kubectl -n woodpecker logs -l cnpg.io/jobRole=full-recovery --tail=100 +``` + +**First check the generated app credentials target the right database.** CNPG +defaults `bootstrap.recovery.database` to `app`; the restored data uses +`woodpecker`. If this reads anything but `woodpecker`, the cluster has +provisioned an empty `app` database and the real data is unreachable through +the `-app` Secret and Service: + +```sh +kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.dbname}' | base64 -d; echo +kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.username}' | base64 -d; echo +``` + +**Then confirm the restore actually holds Woodpecker's data** — a cluster that +comes up healthy but empty is the failure mode worth catching: + +```sh +PGPASS=$(kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.password}' | base64 -d) +kubectl -n woodpecker exec -it wp-pg-1 -- env PGPASSWORD="$PGPASS" \ + psql -U woodpecker -d woodpecker -c '\dt' +kubectl -n woodpecker exec -it wp-pg-1 -- env PGPASSWORD="$PGPASS" \ + psql -U woodpecker -d woodpecker -c 'select count(*) from repos;' +``` + +Row counts should match the live cluster: + +```sh +PGPASS_OLD=$(kubectl -n apps get secret wp-pg-app -o jsonpath='{.data.password}' | base64 -d) +kubectl -n apps exec -it wp-pg-1 -- env PGPASSWORD="$PGPASS_OLD" \ + psql -U woodpecker -d woodpecker -c 'select count(*) from repos;' +``` + +They will not match exactly if builds ran after the backup the restore used — +that is expected, and stage 2 quiesces the server before the final cutover. + +Confirm the two fixes this move exists for: + +```sh +# #433: operator can now reach the instance +kubectl -n woodpecker get cluster wp-pg -o jsonpath='{.status.conditions}' +# metrics now scraped — needs BOTH the PodMonitor (cnpg-podmonitors.yaml) and +# the monitoring ingress allow; wp-pg had neither in `apps`, which is why it is +# absent from cnpg_collector_up while ak-pg/im-pg/gf-pg/pprl-pg are present +kubectl get --raw '/api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy/api/v1/query?query=cnpg_collector_up{namespace="woodpecker"}' +``` + +Confirm the scrape target is actually registered, not just that the metric is +missing for some other reason: + +```sh +kubectl -n woodpecker get podmonitor cnpg-wp-pg +``` + +**Check the policy semantics by hand.** `kubectl --dry-run=server` validates +manifest shape, not Cilium behaviour, so it cannot catch a rule that is +syntactically fine but scoped wider than intended. Confirm what actually reaches +wp-pg's instance-manager port — the operator should, a CI step pod should not: + +```sh +CILIUM=$(kubectl -n kube-system get pod -l k8s-app=cilium \ + --field-selector spec.nodeName=$(kubectl -n woodpecker get pod wp-pg-1 -o jsonpath='{.spec.nodeName}') \ + -o name | head -1) + +# Expect ALLOWED +kubectl -n kube-system exec ${CILIUM#pod/} -c cilium-agent -- cilium-dbg policy trace \ + --src-k8s-pod database:cloudnative-pg --dst-k8s-pod woodpecker:wp-pg-1 --dport 8000 + +# Expect DENIED — arbitrary build steps must not reach the database +kubectl -n kube-system exec ${CILIUM#pod/} -c cilium-agent -- cilium-dbg policy trace \ + --src-k8s-pod woodpecker-ci:any --dst-k8s-pod woodpecker:wp-pg-1 --dport 8000 +``` + +And that archiving goes to the new prefix, leaving the old one alone: + +```sh +kubectl -n woodpecker get cluster wp-pg -o jsonpath='{.status.conditions[?(@.type=="ContinuousArchiving")]}' +``` + +## Rolling back stage 1 + +Delete the four resources and the namespace. Nothing references them, and the +`apps` cluster is untouched: + +```sh +kubectl delete ns woodpecker +``` + +Then revert this PR so Flux stops recreating them. diff --git a/kubernetes/apps/woodpecker/kustomization.yaml b/kubernetes/apps/woodpecker/kustomization.yaml index a39d3b2f..55ce940f 100644 --- a/kubernetes/apps/woodpecker/kustomization.yaml +++ b/kubernetes/apps/woodpecker/kustomization.yaml @@ -29,3 +29,18 @@ resources: - objectstore.yaml - scheduled-backup.yaml - hetzner-creds-externalsecret.yaml + # --- namespace migration, stage 1 (issue #433) --- + # Woodpecker is moving out of the shared `apps` namespace into its own. + # These resources restore wp-pg into the target namespace from barman, + # running in parallel with the live cluster. Nothing points at them yet, so + # this stage carries no CI impact and is revertible by deleting them. + # Cutover (server/agent/DSN/RBAC) is stage 2; removing the `apps` originals + # is stage 3. Runbook: MIGRATION-namespace.md. + # + # The `woodpecker` Namespace itself is declared in infrastructure/namespaces, + # not here: it must exist before the `infrastructure-crds` KS applies the + # namespace's CiliumNetworkPolicies, and this KS runs after both of those. + - migration-externalsecret-hetzner.yaml + - migration-objectstore.yaml + - migration-cluster.yaml + - migration-scheduled-backup.yaml diff --git a/kubernetes/apps/woodpecker/migration-cluster.yaml b/kubernetes/apps/woodpecker/migration-cluster.yaml new file mode 100644 index 00000000..686dbaf1 --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-cluster.yaml @@ -0,0 +1,62 @@ +--- +# wp-pg restored into the `woodpecker` namespace from the barman backup of the +# live apps/wp-pg. Runs in parallel with the original until cutover (PR 2); +# nothing points at it yet, so it carries no traffic. +# +# Spec mirrors apps/wp-pg exactly — same instance count, image, storage class +# and Postgres parameters — so the only variable under test is the restore +# itself. The differences are deliberate and limited to: +# - bootstrap: recovery (not initdb), sourced from the old prefix +# - plugins.barmanObjectName: the NEW store, so it archives somewhere else +# +# NOT prune-disabled, unlike apps/wp-pg. That guard exists there because the +# original holds the only copy of the data through a KS ownership handoff. This +# one is reconstructable by re-running the restore, and a prune guard on a +# migration artifact is how orphans get left behind if the migration is +# abandoned. It gains the guard in PR 3, once it is the surviving cluster. +apiVersion: postgresql.cnpg.io/v1 +kind: Cluster +metadata: + name: wp-pg + namespace: woodpecker +spec: + instances: 1 + imageName: ghcr.io/cloudnative-pg/postgresql:16 + storage: + storageClass: ceph-block + size: 5Gi + plugins: + - name: barman-cloud.cloudnative-pg.io + isWALArchiver: true + parameters: + barmanObjectName: wp-pg-store + bootstrap: + recovery: + source: wp-pg-apps + # Must mirror the initdb values of the cluster this restores from + # (apps/wp-pg: database woodpecker, owner woodpecker). CNPG defaults + # `database` to `app` and `owner` to whatever `database` is, and those + # names do not exist in the physically restored data — it would provision + # an empty `app` database, point the generated wp-pg-app Secret and the + # -app Service at it, and leave the real data unreachable through them. + # Same pattern as every recovery example in kubernetes/RECOVERY.md. + database: woodpecker + owner: woodpecker + externalClusters: + - name: wp-pg-apps + plugin: + name: barman-cloud.cloudnative-pg.io + parameters: + barmanObjectName: wp-pg-store-source + # The cluster that produced the backup, not this one. + serverName: wp-pg + postgresql: + parameters: + shared_buffers: "128MB" + max_connections: "50" + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + memory: 512Mi diff --git a/kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml b/kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml new file mode 100644 index 00000000..a2a207cd --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml @@ -0,0 +1,28 @@ +--- +# Barman S3 credentials for the wp-pg cluster in its new `woodpecker` home. +# Secrets are namespace-scoped, so the copy in `apps` cannot be reused; this is +# the same material fetched into the new namespace. +# +# Still sourced from seed/paperless/hetzner-creds, deliberately. Issue #397 +# wants woodpecker to own seed/woodpecker/hetzner-creds instead, and this +# migration would have been the natural moment — but creating that path needs +# OpenBao write access that is not available here (`bao kv list seed` → 403). +# Repointing at a path that may not exist would leave the migrated cluster +# unable to archive WAL, which is exactly what the restore is meant to prove +# works. #397 stays open and is a drop-in swap of the `key:` below once the +# path exists; the values are identical. +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: hetzner-s3-creds + namespace: woodpecker +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: openbao + target: + name: hetzner-s3-creds + dataFrom: + - extract: + key: seed/paperless/hetzner-creds diff --git a/kubernetes/apps/woodpecker/migration-objectstore.yaml b/kubernetes/apps/woodpecker/migration-objectstore.yaml new file mode 100644 index 00000000..7ffa96fc --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-objectstore.yaml @@ -0,0 +1,64 @@ +--- +# RECOVERY SOURCE — read-only. Points at the prefix the live `apps/wp-pg` +# cluster has been archiving to since it was created. The restored cluster +# bootstraps from here and never writes to it. +# +# serverName on the consuming externalCluster must be `wp-pg`, the name of the +# cluster that produced these backups. +apiVersion: barmancloud.cnpg.io/v1 +kind: ObjectStore +metadata: + name: wp-pg-store-source + namespace: woodpecker +spec: + configuration: + destinationPath: s3://renner-uno-k8s-cnpg/wp-pg + endpointURL: https://fsn1.your-objectstorage.com + s3Credentials: + accessKeyId: + name: hetzner-s3-creds + key: ACCESS_KEY_ID + secretAccessKey: + name: hetzner-s3-creds + key: ACCESS_SECRET_KEY + region: + name: hetzner-s3-creds + key: REGION + wal: + compression: gzip + data: + compression: gzip + # No retentionPolicy: this store is a read source during the migration, and + # the old cluster in `apps` owns the lifecycle of the data underneath it. +--- +# ARCHIVING TARGET — a DIFFERENT prefix from the source, deliberately. +# +# Two clusters archiving WAL to one destinationPath interleave their streams +# and corrupt both. During the parallel-run window the old apps/wp-pg is still +# live and still archiving to s3://renner-uno-k8s-cnpg/wp-pg, so the restored +# cluster must not share it. Splitting the paths also keeps the pre-migration +# backup intact and independently restorable as a rollback. +apiVersion: barmancloud.cnpg.io/v1 +kind: ObjectStore +metadata: + name: wp-pg-store + namespace: woodpecker +spec: + configuration: + destinationPath: s3://renner-uno-k8s-cnpg/wp-pg-woodpecker + endpointURL: https://fsn1.your-objectstorage.com + s3Credentials: + accessKeyId: + name: hetzner-s3-creds + key: ACCESS_KEY_ID + secretAccessKey: + name: hetzner-s3-creds + key: ACCESS_SECRET_KEY + region: + name: hetzner-s3-creds + key: REGION + wal: + compression: gzip + data: + compression: gzip + retentionPolicy: "30d" diff --git a/kubernetes/apps/woodpecker/migration-scheduled-backup.yaml b/kubernetes/apps/woodpecker/migration-scheduled-backup.yaml new file mode 100644 index 00000000..f5080c8a --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-scheduled-backup.yaml @@ -0,0 +1,32 @@ +--- +# Base backups for the restored cluster, onto the new wp-pg-woodpecker prefix. +# +# Without this the cluster archives continuous WAL to that prefix but never +# takes a base backup there, which leaves two problems: the prefix has no +# independent recovery point of its own (recovery would have to chain all the +# way back through the source prefix), and wp-pg-store's retentionPolicy: 30d +# has no base backups to prune against, so the WAL chain grows unbounded for as +# long as stage 1 sits before cutover. +# +# 02:40 UTC, offset from the apps cluster's 02:35 and the other clusters' 02:30 +# so the parallel-run window does not put two woodpecker base backups against +# the Hetzner bucket at once. immediate: true takes the first as soon as the +# restore finishes, which is also the cheapest confirmation that the new +# credentials and the new ObjectStore both work end to end. +# +# Not prune-disabled, for the same reason as migration-cluster.yaml: this is a +# migration artifact until stage 3 promotes it. +apiVersion: postgresql.cnpg.io/v1 +kind: ScheduledBackup +metadata: + name: wp-pg-daily + namespace: woodpecker +spec: + schedule: "0 40 2 * * *" + immediate: true + backupOwnerReference: self + cluster: + name: wp-pg + method: plugin + pluginConfiguration: + name: barman-cloud.cloudnative-pg.io diff --git a/kubernetes/infrastructure/namespaces/namespaces.yaml b/kubernetes/infrastructure/namespaces/namespaces.yaml index 79058429..da92438e 100644 --- a/kubernetes/infrastructure/namespaces/namespaces.yaml +++ b/kubernetes/infrastructure/namespaces/namespaces.yaml @@ -160,3 +160,26 @@ apiVersion: v1 kind: Namespace metadata: name: wordpress +--- +# The Woodpecker service itself — server, agent and its wp-pg Postgres backend, +# moving out of the shared `apps` namespace (issue #433). Distinct from +# `woodpecker-ci`, which enforces pod-security: privileged and holds only +# ephemeral pipeline step pods. +# +# `apps` uses an endpointSelector:{} default-deny, so every allow it carries +# applies to gitea, gatus, external-dns and ical-proxy alike — granting the +# CNPG operator access to wp-pg there would have opened all of `apps` to all of +# `database`. A namespace per service keeps that grant on the one workload. +# +# Declared here rather than alongside the workloads so it is created by the +# `infrastructure` KS. The namespace's CiliumNetworkPolicies ship with the +# `infrastructure-crds` KS (infrastructure/post-crds includes +# ../network-policies/), which itself dependsOn `infrastructure` — so the +# namespace exists before any policy targets it. Creating it in the `woodpecker` +# workload KS would invert that order, since that KS runs after both. +# +# No pod-security labels: the cluster default applies. +apiVersion: v1 +kind: Namespace +metadata: + name: woodpecker diff --git a/kubernetes/infrastructure/network-policies/kustomization.yaml b/kubernetes/infrastructure/network-policies/kustomization.yaml index eef4eb76..0ef61488 100644 --- a/kubernetes/infrastructure/network-policies/kustomization.yaml +++ b/kubernetes/infrastructure/network-policies/kustomization.yaml @@ -21,6 +21,7 @@ resources: - openbao-ns.yaml - external-secrets-ns.yaml - woodpecker-ci-ns.yaml + - woodpecker-ns.yaml - trmnl-ns.yaml - wordpress-ns.yaml - allow-cilium-health.yaml diff --git a/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml new file mode 100644 index 00000000..0394fc25 --- /dev/null +++ b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml @@ -0,0 +1,120 @@ +--- +# Baseline ingress policy for the `woodpecker` namespace (wp-pg now; server and +# agent arrive in stage 2). +# +# Unlike the `apps` baseline this replaces, every allow here lands on a +# namespace holding one service, so a namespace-scoped grant is not a blanket +# grant across unrelated workloads. That is the point of the move: in `apps`, +# letting the CNPG operator reach wp-pg would also have opened gitea, gatus, +# external-dns and ical-proxy to the whole database namespace (issue #433). +# +# CAUTION when adding rules here: this policy has `endpointSelector: {}`, so +# every rule below applies to EVERY pod in the namespace. A `toPorts` block +# narrows the port, NOT the destination workload — that is governed solely by +# the enclosing policy's endpointSelector. Anything meant for one workload +# belongs in its own policy with its own selector, as +# `wp-pg-allow-cnpg-operator` below does. +# +# No egress section, so egress is unrestricted. Deliberate for now and revisited +# in stage 2: wp-pg needs DNS plus `world` for WAL archiving to Hetzner S3, and +# the server/agent add gitea, the Kubernetes API and outbound plugin pulls. +# Writing that ruleset once, with the full workload set present, beats guessing +# it here and silently breaking the WAL archiving this stage exists to verify. +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: default-deny-ingress + namespace: woodpecker +spec: + endpointSelector: {} + ingress: + # Intra-namespace: server <-> agent, server -> wp-pg. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": woodpecker + # Traefik terminates ci.renner.uno and forwards to the server (stage 2). + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": ingress + # Prometheus scrape. Namespace-wide and port-unrestricted, matching the + # other *-ns baselines; this is what covers wp-pg's :9187 metrics. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": monitoring + # Kubelet probes and hostNetwork siblings. + - fromEntities: + - host + - remote-node + # ICMP — from any cluster pod. See home-assistant-ns.yaml for the rationale. + # INGRESS ONLY: do NOT add an egress section to this CNP. + # + # Needed here specifically because wp-pg-allow-cnpg-operator below attaches + # an L4 rule (toPorts :8000) to the wp-pg pods, and Cilium blocks ICMP to an + # endpoint once any layer 4 policy selects it unless the traffic is + # CT-related. Without this, PMTUD Destination Unreachable and Packet Too Big + # toward wp-pg are dropped silently — the failure shows up as stalled + # transfers, not as a policy denial. + # + # Namespace-wide is right for this one, unlike the :8000 case noted below: + # these are diagnostic and path-MTU messages that expose no API surface, so + # granting them across the namespace concedes nothing a per-workload rule + # would withhold. + - fromEndpoints: + - matchExpressions: + - key: io.kubernetes.pod.namespace + operator: Exists + icmps: + - fields: + - { type: 0, family: IPv4 } + - { type: 3, family: IPv4 } + - { type: 8, family: IPv4 } + - { type: 11, family: IPv4 } + - { type: 12, family: IPv4 } + - { type: 1, family: IPv6 } + - { type: 2, family: IPv6 } + - { type: 3, family: IPv6 } + - { type: 4, family: IPv6 } + - { type: 128, family: IPv6 } + - { type: 129, family: IPv6 } + - { type: 133, family: IPv6 } + - { type: 134, family: IPv6 } + - { type: 135, family: IPv6 } + - { type: 136, family: IPv6 } + - { type: 143, family: IPv6 } + # + # NOT here: ingress from `woodpecker-ci`. Pipeline step pods do call the + # server back on :8000 to stream logs, but the CNPG instance manager serves + # its status API on :8000 too — so granting that on this endpointSelector:{} + # policy would hand arbitrary repo-supplied build steps a route to wp-pg's + # instance manager. It lands in stage 2 as its own policy selecting the + # server pod, the way woodpecker-allow-rotators already scopes its :8000 + # rule in apps/woodpecker/networkpolicy.yaml. +--- +# The CNPG operator polls each instance's status endpoint over HTTPS +# (https://:8000/pg/status) to populate Cluster status. Without it the +# Cluster reports "Instance Status Extraction Error: HTTP communication issue" +# while Postgres serves normally, and the operator loses failover and +# switchover decisions — issue #433, where the operator's dial timed out +# because the apps baseline never allowed the database namespace. +# +# Scoped to the wp-pg endpoints and that one port: the operator has no reason to +# reach the woodpecker server or agent, and the barman and mariadb operators +# sharing its namespace have no reason to reach anything here at all. +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: wp-pg-allow-cnpg-operator + namespace: woodpecker +spec: + endpointSelector: + matchLabels: + cnpg.io/cluster: wp-pg + ingress: + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": database + app.kubernetes.io/name: cloudnative-pg + toPorts: + - ports: + - port: "8000" + protocol: TCP