From 85268c1e79e338e7c02696e63101ae464400707f Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Tue, 18 Aug 2026 22:14:26 +0200 Subject: [PATCH 1/5] =?UTF-8?q?feat(woodpecker):=20stage=201=20=E2=80=94?= =?UTF-8?q?=20restore=20wp-pg=20into=20a=20dedicated=20namespace?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses #433. No cutover, no CI impact: this stands up the `woodpecker` namespace and restores wp-pg into it from barman, running in parallel with the live apps/wp-pg. Nothing points at it yet. Why a namespace move rather than a policy fix. wp-pg has been Ready=False with "Instance Status Extraction Error" for three weeks because the CNPG operator in `database` cannot reach https://:8000/pg/status. The apps default-deny-ingress uses endpointSelector:{}, so the only namespace-level remedy is to allow the whole database namespace in — which would expose gitea, gatus, external-dns and ical-proxy along with wp-pg, to fix one pod on one port. The same gap explains why cnpg_collector_up has series for authentik, immich, monitoring and paperless but not wp-pg: apps never allowed the monitoring namespace either. woodpecker-ci is deliberately not the target. It enforces pod-security: privileged and exists for ephemeral step pods running repo-supplied code; the database has no business sharing it. Two ObjectStores, deliberately. wp-pg-store-source reads the existing s3://renner-uno-k8s-cnpg/wp-pg prefix; wp-pg-store archives to a new wp-pg-woodpecker prefix. Two clusters writing one destinationPath would interleave their WAL streams and corrupt both, and splitting them keeps the pre-migration backup intact as rollback. The Namespace is declared in infrastructure/namespaces rather than beside these workloads: its network policies ship with the `infrastructure` KS, which the `woodpecker` KS dependsOn, so creating it in the workload KS would invert that order and deadlock a cold rebuild. The restored cluster deliberately carries no prune: disabled guard. The original has one because it holds the only copy of the data; this one is reconstructable, and a prune guard on a migration artifact is how orphans outlive an abandoned migration. It gains the guard in stage 3. Barman creds still resolve from seed/paperless/hetzner-creds. #397 wants a woodpecker-owned path and this was the natural moment, but creating it needs OpenBao write access that is not available here (bao kv list seed -> 403). Pointing at a path that may not exist would leave the restored cluster unable to archive WAL, which is what this stage exists to prove. Runbook, including how to verify the restore holds real data rather than merely coming up healthy: MIGRATION-namespace.md. Co-Authored-By: Claude Bot --- .../apps/woodpecker/MIGRATION-namespace.md | 138 ++++++++++++++++++ kubernetes/apps/woodpecker/kustomization.yaml | 15 ++ .../apps/woodpecker/migration-cluster.yaml | 53 +++++++ .../migration-externalsecret-hetzner.yaml | 28 ++++ .../woodpecker/migration-objectstore.yaml | 64 ++++++++ .../infrastructure/namespaces/namespaces.yaml | 23 +++ .../network-policies/kustomization.yaml | 1 + .../network-policies/woodpecker-ns.yaml | 82 +++++++++++ 8 files changed, 404 insertions(+) create mode 100644 kubernetes/apps/woodpecker/MIGRATION-namespace.md create mode 100644 kubernetes/apps/woodpecker/migration-cluster.yaml create mode 100644 kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml create mode 100644 kubernetes/apps/woodpecker/migration-objectstore.yaml create mode 100644 kubernetes/infrastructure/network-policies/woodpecker-ns.yaml diff --git a/kubernetes/apps/woodpecker/MIGRATION-namespace.md b/kubernetes/apps/woodpecker/MIGRATION-namespace.md new file mode 100644 index 00000000..d9344b6a --- /dev/null +++ b/kubernetes/apps/woodpecker/MIGRATION-namespace.md @@ -0,0 +1,138 @@ +# Woodpecker: `apps` → `woodpecker` namespace migration + +Moves the Woodpecker service — server, agent and the `wp-pg` CNPG backend — out +of the shared `apps` namespace into one of its own. Distinct from +`MIGRATION.md`, which covers the earlier SQLite → CNPG move. + +## Why + +`apps` holds gitea, gatus, external-dns, ical-proxy and woodpecker together, +and its `default-deny-ingress` uses `endpointSelector: {}`. Every allow rule it +carries therefore applies to *every* workload in the namespace. + +That is not academic. Issue #433 — `wp-pg` stuck at `Ready=False` with +`Instance Status Extraction Error` for three weeks — is caused by the CNPG +operator in `database` being unable to reach `https://:8000/pg/status`. +The only namespace-level fix is `fromEndpoints: namespace=database`, which +would expose gitea, gatus, external-dns and ical-proxy to the whole `database` +namespace to solve a problem affecting one pod on one port. + +`cnpg_collector_up` has series for authentik, immich, monitoring and paperless +but not for `wp-pg`, for the same reason: `apps` never allowed the monitoring +namespace either. + +A namespace per service keeps each grant on the workload that needs it. + +**`woodpecker-ci` is not the target.** It enforces `pod-security: privileged` +and exists solely for ephemeral pipeline step pods running repo-supplied code. +Putting the database there would let any build step reach it through the +intra-namespace allow. + +## Staging + +Rollback is trivial through stage 2. The `apps` cluster is untouched until +stage 3, so reverting means pointing the DSN back and scaling up. + +| Stage | Contents | CI impact | +|---|---|---| +| **1** (this PR) | `woodpecker` namespace, its network policies, ExternalSecret, ObjectStores, and `wp-pg` restored from barman | none — parallel, nothing points at it | +| **runbook** | verify the restore holds real data | none | +| **2** | move server + agent, flip DSN, RBAC subject, rotator policies, docs | **downtime** | +| **3** | delete the `apps` originals, add the prune guard to the survivor | none | + +## Stage 1 design notes + +**Two ObjectStores, deliberately.** `wp-pg-store-source` points at +`s3://renner-uno-k8s-cnpg/wp-pg` — the prefix the live cluster archives to — +and is read-only. `wp-pg-store` points at a **new** prefix, +`s3://renner-uno-k8s-cnpg/wp-pg-woodpecker`, for ongoing archiving. + +Both clusters writing one `destinationPath` would interleave their WAL streams +and corrupt both. Splitting them also leaves the pre-migration backup intact +and independently restorable. + +**`serverName: wp-pg`** on the `externalCluster` names the cluster that +*produced* the backup, not the one being created. + +**The namespace is declared in `infrastructure/namespaces/namespaces.yaml`,** +not alongside these workloads. Its CiliumNetworkPolicies ship with the +`infrastructure` KS, and the `woodpecker` KS `dependsOn` `infrastructure` — +creating the namespace in the workload KS would invert that order and deadlock +a cold rebuild. + +**No `prune: disabled` guard on the restored cluster.** The original carries one +because it holds the only copy of the data. This one is reconstructable by +re-running the restore, and a prune guard on a migration artifact is how orphans +survive an abandoned migration. It gains the guard in stage 3. + +**Barman credentials still come from `seed/paperless/hetzner-creds`.** Issue +#397 wants woodpecker to own `seed/woodpecker/hetzner-creds`, and this would +have been the natural moment — but creating that path needs OpenBao write +access that was not available (`bao kv list seed` → 403). Repointing at a path +that may not exist would leave the restored cluster unable to archive WAL, +which is the thing this stage exists to prove. #397 stays open; the swap is one +line once the path exists. + +## Verifying stage 1 + +Restore is asynchronous. Watch it reach a healthy state: + +```sh +kubectl -n woodpecker get cluster wp-pg -w +kubectl -n woodpecker get pods +``` + +Expect `Cluster in healthy state`. If it sticks in bootstrap, the recovery job's +logs carry the reason: + +```sh +kubectl -n woodpecker logs -l cnpg.io/jobRole=full-recovery --tail=100 +``` + +**Confirm the restore actually holds Woodpecker's data** — a cluster that comes +up healthy but empty is the failure mode worth catching: + +```sh +PGPASS=$(kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.password}' | base64 -d) +kubectl -n woodpecker exec -it wp-pg-1 -- env PGPASSWORD="$PGPASS" \ + psql -U woodpecker -d woodpecker -c '\dt' +kubectl -n woodpecker exec -it wp-pg-1 -- env PGPASSWORD="$PGPASS" \ + psql -U woodpecker -d woodpecker -c 'select count(*) from repos;' +``` + +Row counts should match the live cluster: + +```sh +PGPASS_OLD=$(kubectl -n apps get secret wp-pg-app -o jsonpath='{.data.password}' | base64 -d) +kubectl -n apps exec -it wp-pg-1 -- env PGPASSWORD="$PGPASS_OLD" \ + psql -U woodpecker -d woodpecker -c 'select count(*) from repos;' +``` + +They will not match exactly if builds ran after the backup the restore used — +that is expected, and stage 2 quiesces the server before the final cutover. + +Confirm the two fixes this move exists for: + +```sh +# #433: operator can now reach the instance +kubectl -n woodpecker get cluster wp-pg -o jsonpath='{.status.conditions}' +# metrics now scraped +kubectl get --raw '/api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy/api/v1/query?query=cnpg_collector_up{namespace="woodpecker"}' +``` + +And that archiving goes to the new prefix, leaving the old one alone: + +```sh +kubectl -n woodpecker get cluster wp-pg -o jsonpath='{.status.conditions[?(@.type=="ContinuousArchiving")]}' +``` + +## Rolling back stage 1 + +Delete the four resources and the namespace. Nothing references them, and the +`apps` cluster is untouched: + +```sh +kubectl delete ns woodpecker +``` + +Then revert this PR so Flux stops recreating them. diff --git a/kubernetes/apps/woodpecker/kustomization.yaml b/kubernetes/apps/woodpecker/kustomization.yaml index a39d3b2f..139d243b 100644 --- a/kubernetes/apps/woodpecker/kustomization.yaml +++ b/kubernetes/apps/woodpecker/kustomization.yaml @@ -29,3 +29,18 @@ resources: - objectstore.yaml - scheduled-backup.yaml - hetzner-creds-externalsecret.yaml + # --- namespace migration, stage 1 (issue #433) --- + # Woodpecker is moving out of the shared `apps` namespace into its own. + # These resources restore wp-pg into the target namespace from barman, + # running in parallel with the live cluster. Nothing points at them yet, so + # this stage carries no CI impact and is revertible by deleting them. + # Cutover (server/agent/DSN/RBAC) is stage 2; removing the `apps` originals + # is stage 3. Runbook: MIGRATION-namespace.md. + # + # The `woodpecker` Namespace itself is declared in infrastructure/namespaces, + # not here: its CiliumNetworkPolicies ship with the `infrastructure` KS, which + # this KS dependsOn, so creating the namespace here would invert that order + # and deadlock a cold rebuild. + - migration-externalsecret-hetzner.yaml + - migration-objectstore.yaml + - migration-cluster.yaml diff --git a/kubernetes/apps/woodpecker/migration-cluster.yaml b/kubernetes/apps/woodpecker/migration-cluster.yaml new file mode 100644 index 00000000..eae0582e --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-cluster.yaml @@ -0,0 +1,53 @@ +--- +# wp-pg restored into the `woodpecker` namespace from the barman backup of the +# live apps/wp-pg. Runs in parallel with the original until cutover (PR 2); +# nothing points at it yet, so it carries no traffic. +# +# Spec mirrors apps/wp-pg exactly — same instance count, image, storage class +# and Postgres parameters — so the only variable under test is the restore +# itself. The differences are deliberate and limited to: +# - bootstrap: recovery (not initdb), sourced from the old prefix +# - plugins.barmanObjectName: the NEW store, so it archives somewhere else +# +# NOT prune-disabled, unlike apps/wp-pg. That guard exists there because the +# original holds the only copy of the data through a KS ownership handoff. This +# one is reconstructable by re-running the restore, and a prune guard on a +# migration artifact is how orphans get left behind if the migration is +# abandoned. It gains the guard in PR 3, once it is the surviving cluster. +apiVersion: postgresql.cnpg.io/v1 +kind: Cluster +metadata: + name: wp-pg + namespace: woodpecker +spec: + instances: 1 + imageName: ghcr.io/cloudnative-pg/postgresql:16 + storage: + storageClass: ceph-block + size: 5Gi + plugins: + - name: barman-cloud.cloudnative-pg.io + isWALArchiver: true + parameters: + barmanObjectName: wp-pg-store + bootstrap: + recovery: + source: wp-pg-apps + externalClusters: + - name: wp-pg-apps + plugin: + name: barman-cloud.cloudnative-pg.io + parameters: + barmanObjectName: wp-pg-store-source + # The cluster that produced the backup, not this one. + serverName: wp-pg + postgresql: + parameters: + shared_buffers: "128MB" + max_connections: "50" + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + memory: 512Mi diff --git a/kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml b/kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml new file mode 100644 index 00000000..a2a207cd --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-externalsecret-hetzner.yaml @@ -0,0 +1,28 @@ +--- +# Barman S3 credentials for the wp-pg cluster in its new `woodpecker` home. +# Secrets are namespace-scoped, so the copy in `apps` cannot be reused; this is +# the same material fetched into the new namespace. +# +# Still sourced from seed/paperless/hetzner-creds, deliberately. Issue #397 +# wants woodpecker to own seed/woodpecker/hetzner-creds instead, and this +# migration would have been the natural moment — but creating that path needs +# OpenBao write access that is not available here (`bao kv list seed` → 403). +# Repointing at a path that may not exist would leave the migrated cluster +# unable to archive WAL, which is exactly what the restore is meant to prove +# works. #397 stays open and is a drop-in swap of the `key:` below once the +# path exists; the values are identical. +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: hetzner-s3-creds + namespace: woodpecker +spec: + refreshInterval: 1h + secretStoreRef: + kind: ClusterSecretStore + name: openbao + target: + name: hetzner-s3-creds + dataFrom: + - extract: + key: seed/paperless/hetzner-creds diff --git a/kubernetes/apps/woodpecker/migration-objectstore.yaml b/kubernetes/apps/woodpecker/migration-objectstore.yaml new file mode 100644 index 00000000..7ffa96fc --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-objectstore.yaml @@ -0,0 +1,64 @@ +--- +# RECOVERY SOURCE — read-only. Points at the prefix the live `apps/wp-pg` +# cluster has been archiving to since it was created. The restored cluster +# bootstraps from here and never writes to it. +# +# serverName on the consuming externalCluster must be `wp-pg`, the name of the +# cluster that produced these backups. +apiVersion: barmancloud.cnpg.io/v1 +kind: ObjectStore +metadata: + name: wp-pg-store-source + namespace: woodpecker +spec: + configuration: + destinationPath: s3://renner-uno-k8s-cnpg/wp-pg + endpointURL: https://fsn1.your-objectstorage.com + s3Credentials: + accessKeyId: + name: hetzner-s3-creds + key: ACCESS_KEY_ID + secretAccessKey: + name: hetzner-s3-creds + key: ACCESS_SECRET_KEY + region: + name: hetzner-s3-creds + key: REGION + wal: + compression: gzip + data: + compression: gzip + # No retentionPolicy: this store is a read source during the migration, and + # the old cluster in `apps` owns the lifecycle of the data underneath it. +--- +# ARCHIVING TARGET — a DIFFERENT prefix from the source, deliberately. +# +# Two clusters archiving WAL to one destinationPath interleave their streams +# and corrupt both. During the parallel-run window the old apps/wp-pg is still +# live and still archiving to s3://renner-uno-k8s-cnpg/wp-pg, so the restored +# cluster must not share it. Splitting the paths also keeps the pre-migration +# backup intact and independently restorable as a rollback. +apiVersion: barmancloud.cnpg.io/v1 +kind: ObjectStore +metadata: + name: wp-pg-store + namespace: woodpecker +spec: + configuration: + destinationPath: s3://renner-uno-k8s-cnpg/wp-pg-woodpecker + endpointURL: https://fsn1.your-objectstorage.com + s3Credentials: + accessKeyId: + name: hetzner-s3-creds + key: ACCESS_KEY_ID + secretAccessKey: + name: hetzner-s3-creds + key: ACCESS_SECRET_KEY + region: + name: hetzner-s3-creds + key: REGION + wal: + compression: gzip + data: + compression: gzip + retentionPolicy: "30d" diff --git a/kubernetes/infrastructure/namespaces/namespaces.yaml b/kubernetes/infrastructure/namespaces/namespaces.yaml index 79058429..e4dcc72b 100644 --- a/kubernetes/infrastructure/namespaces/namespaces.yaml +++ b/kubernetes/infrastructure/namespaces/namespaces.yaml @@ -160,3 +160,26 @@ apiVersion: v1 kind: Namespace metadata: name: wordpress +--- +# The Woodpecker service itself — server, agent and its wp-pg Postgres backend, +# moving out of the shared `apps` namespace (issue #433). Distinct from +# `woodpecker-ci`, which enforces pod-security: privileged and holds only +# ephemeral pipeline step pods. +# +# `apps` uses an endpointSelector:{} default-deny, so every allow it carries +# applies to gitea, gatus, external-dns and ical-proxy alike — granting the +# CNPG operator access to wp-pg there would have opened all of `apps` to all of +# `database`. A namespace per service keeps that grant on the one workload. +# +# Declared here rather than alongside the workloads so it is created by the +# `infrastructure` KS: the namespace's CiliumNetworkPolicies live in +# infrastructure/network-policies, and the `woodpecker` KS dependsOn +# `infrastructure`. Creating it in the workload KS would invert that order. +# +# No pod-security labels: the cluster default applies. +apiVersion: v1 +kind: Namespace +metadata: + name: woodpecker + labels: + app.kubernetes.io/name: woodpecker diff --git a/kubernetes/infrastructure/network-policies/kustomization.yaml b/kubernetes/infrastructure/network-policies/kustomization.yaml index eef4eb76..0ef61488 100644 --- a/kubernetes/infrastructure/network-policies/kustomization.yaml +++ b/kubernetes/infrastructure/network-policies/kustomization.yaml @@ -21,6 +21,7 @@ resources: - openbao-ns.yaml - external-secrets-ns.yaml - woodpecker-ci-ns.yaml + - woodpecker-ns.yaml - trmnl-ns.yaml - wordpress-ns.yaml - allow-cilium-health.yaml diff --git a/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml new file mode 100644 index 00000000..ad4a7d8d --- /dev/null +++ b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml @@ -0,0 +1,82 @@ +--- +# Baseline ingress policy for the `woodpecker` namespace (server, agent, wp-pg). +# +# Unlike the `apps` baseline this replaces, every allow here lands on a +# namespace holding one service, so a namespace-scoped grant is not a blanket +# grant across unrelated workloads. That is the point of the move: in `apps`, +# letting the CNPG operator reach wp-pg would also have opened gitea, gatus, +# external-dns and ical-proxy to the whole database namespace (issue #433). +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: default-deny-ingress + namespace: woodpecker +spec: + endpointSelector: {} + ingress: + # Intra-namespace: server <-> agent, server -> wp-pg. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": woodpecker + # Traefik terminates ci.renner.uno and forwards to the server. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": ingress + # Prometheus scrape. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": monitoring + # Pipeline step pods in the privileged CI namespace call back to the server + # to stream logs and report status. Scoped to the server's :8000 rather than + # granted namespace-wide — step pods run arbitrary repo-supplied code, so + # they must not reach wp-pg. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": woodpecker-ci + toPorts: + - ports: + - port: "8000" + protocol: TCP + # Kubelet probes and hostNetwork siblings. + - fromEntities: + - host + - remote-node +--- +# The CNPG operator polls each instance's status endpoint over HTTPS +# (https://:8000/pg/status) to populate Cluster status. Without it the +# Cluster reports "Instance Status Extraction Error: HTTP communication issue" +# while Postgres serves normally, and the operator loses failover and +# switchover decisions — issue #433, where the operator's dial timed out +# because the apps baseline never allowed the database namespace. +# +# Scoped to the wp-pg endpoints and that one port. The operator has no reason +# to reach the woodpecker server or agent, and the barman/mariadb operators +# sharing its namespace have no reason to reach anything here at all. +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: wp-pg-allow-cnpg-operator + namespace: woodpecker +spec: + endpointSelector: + matchLabels: + cnpg.io/cluster: wp-pg + ingress: + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": database + app.kubernetes.io/name: cloudnative-pg + toPorts: + - ports: + - port: "8000" + protocol: TCP + # CNPG instances expose Postgres metrics on :9187. The apps namespace never + # allowed this either, which is why cnpg_collector_up has series for + # authentik, immich, monitoring and paperless but not for wp-pg. + - fromEndpoints: + - matchLabels: + "k8s:io.kubernetes.pod.namespace": monitoring + toPorts: + - ports: + - port: "9187" + protocol: TCP -- 2.54.0 From b0ddbe0dfd7c07cd0458e352c7f4fb8ad792f3d7 Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Tue, 18 Aug 2026 22:25:32 +0200 Subject: [PATCH 2/5] fix(woodpecker): drop the woodpecker-ci rule off the blanket ingress policy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the blocking review finding on #504. The "step pods call the server back on :8000" rule was attached to the namespace baseline, which uses endpointSelector: {}. A toPorts block narrows the port, not the destination workload — that is governed solely by the enclosing policy's endpointSelector. So the rule granted woodpecker-ci access to :8000 on every pod in the namespace, and the CNPG instance manager serves its status API on :8000 (server.yaml:94 uses the same port). Arbitrary repo-supplied build steps would have had a route to wp-pg's instance manager. That is the same endpointSelector:{}-means-blanket-grant failure this PR exists to escape, reproduced inside the fix. Removed rather than rescoped: there is no woodpecker server in this namespace until stage 2, so the rule was premature as well as wrong. It lands there as its own policy selecting the server pod, the way woodpecker-allow-rotators already scopes its :8000 rule. The baseline now carries no port-scoped rules at all, so nothing in it can be mistaken for workload scoping, and a comment warns why that must stay true. Also from the review: - Corrected the KS attribution. The network policies ship with infrastructure-crds (infrastructure/post-crds includes ../network-policies/), not the top-level infrastructure KS. The ordering conclusion is unchanged — infrastructure-crds dependsOn infrastructure, and woodpecker dependsOn both — but the comments named the wrong one. - Documented why the baseline has no egress section, instead of leaving it to read as an oversight. Writing that ruleset in stage 2, with the server and agent present, beats guessing it now and breaking the WAL archiving this stage exists to verify. - Dropped the monitoring -> :9187 rule from wp-pg-allow-cnpg-operator. The baseline already allows the monitoring namespace unrestricted, so it added no coverage while implying scoping it did not provide. - Added a cilium-dbg policy trace step to the runbook. dry-run validates manifest shape, not Cilium semantics, and would not have caught this bug. Co-Authored-By: Claude Bot --- .../apps/woodpecker/MIGRATION-namespace.md | 34 ++++++++++-- kubernetes/apps/woodpecker/kustomization.yaml | 5 +- .../infrastructure/namespaces/namespaces.yaml | 8 +-- .../network-policies/woodpecker-ns.yaml | 54 ++++++++++--------- 4 files changed, 66 insertions(+), 35 deletions(-) diff --git a/kubernetes/apps/woodpecker/MIGRATION-namespace.md b/kubernetes/apps/woodpecker/MIGRATION-namespace.md index d9344b6a..43377895 100644 --- a/kubernetes/apps/woodpecker/MIGRATION-namespace.md +++ b/kubernetes/apps/woodpecker/MIGRATION-namespace.md @@ -56,9 +56,18 @@ and independently restorable. **The namespace is declared in `infrastructure/namespaces/namespaces.yaml`,** not alongside these workloads. Its CiliumNetworkPolicies ship with the -`infrastructure` KS, and the `woodpecker` KS `dependsOn` `infrastructure` — -creating the namespace in the workload KS would invert that order and deadlock -a cold rebuild. +`infrastructure-crds` KS (`infrastructure/post-crds` includes +`../network-policies/`), which `dependsOn` `infrastructure` — so the namespace +exists before any policy targets it. Creating it in the `woodpecker` workload +KS would invert that order, since that KS runs after both. + +**Ingress from `woodpecker-ci` is deliberately absent in stage 1.** Step pods do +call the server back on `:8000`, but the CNPG instance manager serves its status +API on `:8000` too, and the baseline policy uses `endpointSelector: {}`. A +`toPorts` block narrows the port, not the destination workload — so putting that +rule on the baseline would hand arbitrary repo-supplied build steps a route to +wp-pg's instance manager. It lands in stage 2 as its own policy selecting the +server pod. **No `prune: disabled` guard on the restored cluster.** The original carries one because it holds the only copy of the data. This one is reconstructable by @@ -120,6 +129,25 @@ kubectl -n woodpecker get cluster wp-pg -o jsonpath='{.status.conditions}' kubectl get --raw '/api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy/api/v1/query?query=cnpg_collector_up{namespace="woodpecker"}' ``` +**Check the policy semantics by hand.** `kubectl --dry-run=server` validates +manifest shape, not Cilium behaviour, so it cannot catch a rule that is +syntactically fine but scoped wider than intended. Confirm what actually reaches +wp-pg's instance-manager port — the operator should, a CI step pod should not: + +```sh +CILIUM=$(kubectl -n kube-system get pod -l k8s-app=cilium \ + --field-selector spec.nodeName=$(kubectl -n woodpecker get pod wp-pg-1 -o jsonpath='{.spec.nodeName}') \ + -o name | head -1) + +# Expect ALLOWED +kubectl -n kube-system exec ${CILIUM#pod/} -c cilium-agent -- cilium-dbg policy trace \ + --src-k8s-pod database:cloudnative-pg --dst-k8s-pod woodpecker:wp-pg-1 --dport 8000 + +# Expect DENIED — arbitrary build steps must not reach the database +kubectl -n kube-system exec ${CILIUM#pod/} -c cilium-agent -- cilium-dbg policy trace \ + --src-k8s-pod woodpecker-ci:any --dst-k8s-pod woodpecker:wp-pg-1 --dport 8000 +``` + And that archiving goes to the new prefix, leaving the old one alone: ```sh diff --git a/kubernetes/apps/woodpecker/kustomization.yaml b/kubernetes/apps/woodpecker/kustomization.yaml index 139d243b..71524777 100644 --- a/kubernetes/apps/woodpecker/kustomization.yaml +++ b/kubernetes/apps/woodpecker/kustomization.yaml @@ -38,9 +38,8 @@ resources: # is stage 3. Runbook: MIGRATION-namespace.md. # # The `woodpecker` Namespace itself is declared in infrastructure/namespaces, - # not here: its CiliumNetworkPolicies ship with the `infrastructure` KS, which - # this KS dependsOn, so creating the namespace here would invert that order - # and deadlock a cold rebuild. + # not here: it must exist before the `infrastructure-crds` KS applies the + # namespace's CiliumNetworkPolicies, and this KS runs after both of those. - migration-externalsecret-hetzner.yaml - migration-objectstore.yaml - migration-cluster.yaml diff --git a/kubernetes/infrastructure/namespaces/namespaces.yaml b/kubernetes/infrastructure/namespaces/namespaces.yaml index e4dcc72b..c81e5ffe 100644 --- a/kubernetes/infrastructure/namespaces/namespaces.yaml +++ b/kubernetes/infrastructure/namespaces/namespaces.yaml @@ -172,9 +172,11 @@ metadata: # `database`. A namespace per service keeps that grant on the one workload. # # Declared here rather than alongside the workloads so it is created by the -# `infrastructure` KS: the namespace's CiliumNetworkPolicies live in -# infrastructure/network-policies, and the `woodpecker` KS dependsOn -# `infrastructure`. Creating it in the workload KS would invert that order. +# `infrastructure` KS. The namespace's CiliumNetworkPolicies ship with the +# `infrastructure-crds` KS (infrastructure/post-crds includes +# ../network-policies/), which itself dependsOn `infrastructure` — so the +# namespace exists before any policy targets it. Creating it in the `woodpecker` +# workload KS would invert that order, since that KS runs after both. # # No pod-security labels: the cluster default applies. apiVersion: v1 diff --git a/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml index ad4a7d8d..ff185600 100644 --- a/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml +++ b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml @@ -1,11 +1,25 @@ --- -# Baseline ingress policy for the `woodpecker` namespace (server, agent, wp-pg). +# Baseline ingress policy for the `woodpecker` namespace (wp-pg now; server and +# agent arrive in stage 2). # # Unlike the `apps` baseline this replaces, every allow here lands on a # namespace holding one service, so a namespace-scoped grant is not a blanket # grant across unrelated workloads. That is the point of the move: in `apps`, # letting the CNPG operator reach wp-pg would also have opened gitea, gatus, # external-dns and ical-proxy to the whole database namespace (issue #433). +# +# CAUTION when adding rules here: this policy has `endpointSelector: {}`, so +# every rule below applies to EVERY pod in the namespace. A `toPorts` block +# narrows the port, NOT the destination workload — that is governed solely by +# the enclosing policy's endpointSelector. Anything meant for one workload +# belongs in its own policy with its own selector, as +# `wp-pg-allow-cnpg-operator` below does. +# +# No egress section, so egress is unrestricted. Deliberate for now and revisited +# in stage 2: wp-pg needs DNS plus `world` for WAL archiving to Hetzner S3, and +# the server/agent add gitea, the Kubernetes API and outbound plugin pulls. +# Writing that ruleset once, with the full workload set present, beats guessing +# it here and silently breaking the WAL archiving this stage exists to verify. apiVersion: cilium.io/v2 kind: CiliumNetworkPolicy metadata: @@ -18,29 +32,27 @@ spec: - fromEndpoints: - matchLabels: "k8s:io.kubernetes.pod.namespace": woodpecker - # Traefik terminates ci.renner.uno and forwards to the server. + # Traefik terminates ci.renner.uno and forwards to the server (stage 2). - fromEndpoints: - matchLabels: "k8s:io.kubernetes.pod.namespace": ingress - # Prometheus scrape. + # Prometheus scrape. Namespace-wide and port-unrestricted, matching the + # other *-ns baselines; this is what covers wp-pg's :9187 metrics. - fromEndpoints: - matchLabels: "k8s:io.kubernetes.pod.namespace": monitoring - # Pipeline step pods in the privileged CI namespace call back to the server - # to stream logs and report status. Scoped to the server's :8000 rather than - # granted namespace-wide — step pods run arbitrary repo-supplied code, so - # they must not reach wp-pg. - - fromEndpoints: - - matchLabels: - "k8s:io.kubernetes.pod.namespace": woodpecker-ci - toPorts: - - ports: - - port: "8000" - protocol: TCP # Kubelet probes and hostNetwork siblings. - fromEntities: - host - remote-node + # + # NOT here: ingress from `woodpecker-ci`. Pipeline step pods do call the + # server back on :8000 to stream logs, but the CNPG instance manager serves + # its status API on :8000 too — so granting that on this endpointSelector:{} + # policy would hand arbitrary repo-supplied build steps a route to wp-pg's + # instance manager. It lands in stage 2 as its own policy selecting the + # server pod, the way woodpecker-allow-rotators already scopes its :8000 + # rule in apps/woodpecker/networkpolicy.yaml. --- # The CNPG operator polls each instance's status endpoint over HTTPS # (https://:8000/pg/status) to populate Cluster status. Without it the @@ -49,8 +61,8 @@ spec: # switchover decisions — issue #433, where the operator's dial timed out # because the apps baseline never allowed the database namespace. # -# Scoped to the wp-pg endpoints and that one port. The operator has no reason -# to reach the woodpecker server or agent, and the barman/mariadb operators +# Scoped to the wp-pg endpoints and that one port: the operator has no reason to +# reach the woodpecker server or agent, and the barman and mariadb operators # sharing its namespace have no reason to reach anything here at all. apiVersion: cilium.io/v2 kind: CiliumNetworkPolicy @@ -70,13 +82,3 @@ spec: - ports: - port: "8000" protocol: TCP - # CNPG instances expose Postgres metrics on :9187. The apps namespace never - # allowed this either, which is why cnpg_collector_up has series for - # authentik, immich, monitoring and paperless but not for wp-pg. - - fromEndpoints: - - matchLabels: - "k8s:io.kubernetes.pod.namespace": monitoring - toPorts: - - ports: - - port: "9187" - protocol: TCP -- 2.54.0 From 1b5ad600d5c125bc3970e013fc16b85b4c8966db Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Tue, 18 Aug 2026 22:33:50 +0200 Subject: [PATCH 3/5] fix(woodpecker): set database/owner on the wp-pg recovery bootstrap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the second blocking review finding on #504. bootstrap.recovery set only `source`. The live CRD documents the defaults: `database` defaults to `app`, and `owner` defaults to whatever `database` is. Neither name exists in the physically restored data, which carries woodpecker/woodpecker from the backup. So CNPG would have restored the real data, then provisioned a separate empty `app` database and pointed the generated wp-pg-app Secret and the -app Service at that instead — a cluster reporting healthy, holding the right bytes, with the credentials aimed at the wrong database. The runbook's own verification (psql -U woodpecker -d woodpecker using the wp-pg-app password) would not have authenticated, so the failure would have surfaced as a confusing auth error rather than as what it was. kubernetes/RECOVERY.md sets both fields on all three recovery examples it documents (ak-pg, im-pg, pprl-pg) for exactly this reason; this one did not follow the pattern. Added a runbook step to read dbname/username back out of the wp-pg-app Secret before the row-count checks, so the mismatch is caught directly rather than inferred from a psql failure. Co-Authored-By: Claude Bot --- kubernetes/apps/woodpecker/MIGRATION-namespace.md | 15 +++++++++++++-- kubernetes/apps/woodpecker/migration-cluster.yaml | 9 +++++++++ 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/kubernetes/apps/woodpecker/MIGRATION-namespace.md b/kubernetes/apps/woodpecker/MIGRATION-namespace.md index 43377895..0419f6f9 100644 --- a/kubernetes/apps/woodpecker/MIGRATION-namespace.md +++ b/kubernetes/apps/woodpecker/MIGRATION-namespace.md @@ -98,8 +98,19 @@ logs carry the reason: kubectl -n woodpecker logs -l cnpg.io/jobRole=full-recovery --tail=100 ``` -**Confirm the restore actually holds Woodpecker's data** — a cluster that comes -up healthy but empty is the failure mode worth catching: +**First check the generated app credentials target the right database.** CNPG +defaults `bootstrap.recovery.database` to `app`; the restored data uses +`woodpecker`. If this reads anything but `woodpecker`, the cluster has +provisioned an empty `app` database and the real data is unreachable through +the `-app` Secret and Service: + +```sh +kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.dbname}' | base64 -d; echo +kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.username}' | base64 -d; echo +``` + +**Then confirm the restore actually holds Woodpecker's data** — a cluster that +comes up healthy but empty is the failure mode worth catching: ```sh PGPASS=$(kubectl -n woodpecker get secret wp-pg-app -o jsonpath='{.data.password}' | base64 -d) diff --git a/kubernetes/apps/woodpecker/migration-cluster.yaml b/kubernetes/apps/woodpecker/migration-cluster.yaml index eae0582e..686dbaf1 100644 --- a/kubernetes/apps/woodpecker/migration-cluster.yaml +++ b/kubernetes/apps/woodpecker/migration-cluster.yaml @@ -33,6 +33,15 @@ spec: bootstrap: recovery: source: wp-pg-apps + # Must mirror the initdb values of the cluster this restores from + # (apps/wp-pg: database woodpecker, owner woodpecker). CNPG defaults + # `database` to `app` and `owner` to whatever `database` is, and those + # names do not exist in the physically restored data — it would provision + # an empty `app` database, point the generated wp-pg-app Secret and the + # -app Service at it, and leave the real data unreachable through them. + # Same pattern as every recovery example in kubernetes/RECOVERY.md. + database: woodpecker + owner: woodpecker externalClusters: - name: wp-pg-apps plugin: -- 2.54.0 From e56f1e7cf6d84f74928aaaa83964f1c79cdfab53 Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Tue, 18 Aug 2026 22:44:36 +0200 Subject: [PATCH 4/5] fix(woodpecker): add the missing PodMonitor and base backup for wp-pg MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the third blocking review finding on #504. The PR claimed the namespace move fixes wp-pg's absent metrics. It does not, on its own. Prometheus requires the `release: kube-prometheus-stack` label that CNPG's own enablePodMonitor omits, so every scraped cluster has a hand-written PodMonitor in apps/monitoring-rules/cnpg-podmonitors.yaml — ak-pg, im-pg, gf-pg, pprl-pg. There has never been one for wp-pg. So wp-pg was missing both halves: no scrape target configured, and no monitoring ingress allow in the apps baseline. Fixing the policy half was necessary but not sufficient, and the runbook's cnpg_collector_up check would have returned empty either way — unable to confirm the thing it existed to confirm. Adds the PodMonitor in the woodpecker namespace. Also from the review: - Added a ScheduledBackup for the restored cluster. Every other CNPG cluster here pairs isWALArchiver with one. Without it the new wp-pg-woodpecker prefix takes continuous WAL but never a base backup, so it has no independent recovery point and wp-pg-store's 30d retention has nothing to prune against — the WAL chain grows unbounded for as long as stage 1 sits before cutover. Scheduled 02:40, clear of the apps cluster's 02:35 and the other clusters' 02:30. - Documented that the `woodpecker` KS goes not-Ready while the restore runs. Its healthCheckExprs matches Cluster by GVK with no name qualifier, so Flux evaluates it against both the live apps/wp-pg and the restoring woodpecker/wp-pg, and wait: true makes the KS that owns production Woodpecker report unhealthy. Expected, does not cascade since nothing dependsOn it, but it would read as an outage without a note. - Dropped the app.kubernetes.io/name label from the new Namespace; no other namespace in that file carries one. Co-Authored-By: Claude Bot --- .../monitoring-rules/cnpg-podmonitors.yaml | 26 +++++++++++++++ .../apps/woodpecker/MIGRATION-namespace.md | 29 ++++++++++++++++- kubernetes/apps/woodpecker/kustomization.yaml | 1 + .../migration-scheduled-backup.yaml | 32 +++++++++++++++++++ .../infrastructure/namespaces/namespaces.yaml | 2 -- 5 files changed, 87 insertions(+), 3 deletions(-) create mode 100644 kubernetes/apps/woodpecker/migration-scheduled-backup.yaml diff --git a/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml b/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml index 3d3dae70..3e059817 100644 --- a/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml +++ b/kubernetes/apps/monitoring-rules/cnpg-podmonitors.yaml @@ -73,3 +73,29 @@ spec: interval: 30s podTargetLabels: - role +--- +# wp-pg, restored into the `woodpecker` namespace (issue #433). It has never +# been scraped: `apps` had no PodMonitor for it AND its baseline network policy +# never allowed the monitoring namespace, so `cnpg_collector_up` has series for +# the four clusters above but not this one. The namespace move fixes the policy +# half; without this PodMonitor there is still no scrape target at all. +# +# Deliberately not adding one for the cluster still in `apps`: it is retired in +# stage 3, and a PodMonitor there would only be reconciled to be deleted again. +apiVersion: monitoring.coreos.com/v1 +kind: PodMonitor +metadata: + name: cnpg-wp-pg + namespace: woodpecker + labels: + app.kubernetes.io/name: cloudnative-pg + release: kube-prometheus-stack +spec: + selector: + matchLabels: + cnpg.io/cluster: wp-pg + podMetricsEndpoints: + - port: metrics + interval: 30s + podTargetLabels: + - role diff --git a/kubernetes/apps/woodpecker/MIGRATION-namespace.md b/kubernetes/apps/woodpecker/MIGRATION-namespace.md index 0419f6f9..70b2b2ac 100644 --- a/kubernetes/apps/woodpecker/MIGRATION-namespace.md +++ b/kubernetes/apps/woodpecker/MIGRATION-namespace.md @@ -82,6 +82,24 @@ that may not exist would leave the restored cluster unable to archive WAL, which is the thing this stage exists to prove. #397 stays open; the swap is one line once the path exists. +## Expect the `woodpecker` Kustomization to go not-Ready during the restore + +`kubernetes/clusters/homelab/woodpecker.yaml` sets `wait: true` and a +`healthCheckExprs` entry matching `postgresql.cnpg.io/v1 Cluster` **with no +name or namespace qualifier**. Flux evaluates it against every Cluster in that +Kustomization's inventory — which, after this stage, is both the live +`apps/wp-pg` and the restoring `woodpecker/wp-pg`. + +So while the restore runs, the whole `woodpecker` KS — the one that owns +production Woodpecker — reports not-Ready, and a reconcile timeout in that +window will trip the `flux-failures` Alert. **That is expected and is not +Woodpecker breaking.** Nothing `dependsOn: woodpecker`, so it does not cascade. + +If the restore is going to sit for a long time, or the noise is unwelcome, the +options are to suspend the KS while it completes, or to split the migration +resources into their own `wait: false` Kustomization. For a ~660 MB database the +window should be minutes. + ## Verifying stage 1 Restore is asynchronous. Watch it reach a healthy state: @@ -136,10 +154,19 @@ Confirm the two fixes this move exists for: ```sh # #433: operator can now reach the instance kubectl -n woodpecker get cluster wp-pg -o jsonpath='{.status.conditions}' -# metrics now scraped +# metrics now scraped — needs BOTH the PodMonitor (cnpg-podmonitors.yaml) and +# the monitoring ingress allow; wp-pg had neither in `apps`, which is why it is +# absent from cnpg_collector_up while ak-pg/im-pg/gf-pg/pprl-pg are present kubectl get --raw '/api/v1/namespaces/monitoring/services/kube-prometheus-stack-prometheus:9090/proxy/api/v1/query?query=cnpg_collector_up{namespace="woodpecker"}' ``` +Confirm the scrape target is actually registered, not just that the metric is +missing for some other reason: + +```sh +kubectl -n woodpecker get podmonitor cnpg-wp-pg +``` + **Check the policy semantics by hand.** `kubectl --dry-run=server` validates manifest shape, not Cilium behaviour, so it cannot catch a rule that is syntactically fine but scoped wider than intended. Confirm what actually reaches diff --git a/kubernetes/apps/woodpecker/kustomization.yaml b/kubernetes/apps/woodpecker/kustomization.yaml index 71524777..55ce940f 100644 --- a/kubernetes/apps/woodpecker/kustomization.yaml +++ b/kubernetes/apps/woodpecker/kustomization.yaml @@ -43,3 +43,4 @@ resources: - migration-externalsecret-hetzner.yaml - migration-objectstore.yaml - migration-cluster.yaml + - migration-scheduled-backup.yaml diff --git a/kubernetes/apps/woodpecker/migration-scheduled-backup.yaml b/kubernetes/apps/woodpecker/migration-scheduled-backup.yaml new file mode 100644 index 00000000..f5080c8a --- /dev/null +++ b/kubernetes/apps/woodpecker/migration-scheduled-backup.yaml @@ -0,0 +1,32 @@ +--- +# Base backups for the restored cluster, onto the new wp-pg-woodpecker prefix. +# +# Without this the cluster archives continuous WAL to that prefix but never +# takes a base backup there, which leaves two problems: the prefix has no +# independent recovery point of its own (recovery would have to chain all the +# way back through the source prefix), and wp-pg-store's retentionPolicy: 30d +# has no base backups to prune against, so the WAL chain grows unbounded for as +# long as stage 1 sits before cutover. +# +# 02:40 UTC, offset from the apps cluster's 02:35 and the other clusters' 02:30 +# so the parallel-run window does not put two woodpecker base backups against +# the Hetzner bucket at once. immediate: true takes the first as soon as the +# restore finishes, which is also the cheapest confirmation that the new +# credentials and the new ObjectStore both work end to end. +# +# Not prune-disabled, for the same reason as migration-cluster.yaml: this is a +# migration artifact until stage 3 promotes it. +apiVersion: postgresql.cnpg.io/v1 +kind: ScheduledBackup +metadata: + name: wp-pg-daily + namespace: woodpecker +spec: + schedule: "0 40 2 * * *" + immediate: true + backupOwnerReference: self + cluster: + name: wp-pg + method: plugin + pluginConfiguration: + name: barman-cloud.cloudnative-pg.io diff --git a/kubernetes/infrastructure/namespaces/namespaces.yaml b/kubernetes/infrastructure/namespaces/namespaces.yaml index c81e5ffe..da92438e 100644 --- a/kubernetes/infrastructure/namespaces/namespaces.yaml +++ b/kubernetes/infrastructure/namespaces/namespaces.yaml @@ -183,5 +183,3 @@ apiVersion: v1 kind: Namespace metadata: name: woodpecker - labels: - app.kubernetes.io/name: woodpecker -- 2.54.0 From 4996a6f738daba26179825eebed8e47af04e3206 Mon Sep 17 00:00:00 2001 From: Claude Bot Date: Tue, 18 Aug 2026 22:52:31 +0200 Subject: [PATCH 5/5] fix(woodpecker): allow ICMP to the new namespace's endpoints MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From the fourth review on #504 — no blocking issues, but this one is a real gap rather than a style point. The woodpecker baseline was the only one of the repo's namespace policies without an ICMP allow. That matters here specifically: wp-pg-allow-cnpg-operator attaches an L4 rule (toPorts :8000) to the wp-pg pods, and Cilium blocks ICMP to an endpoint once any layer 4 policy selects it unless the traffic is CT-related. So PMTUD Destination Unreachable and Packet Too Big toward wp-pg would have been dropped silently, surfacing as stalled transfers rather than as a policy denial. Uses the 16-type set the other baselines now carry after #495 trimmed them from 39, rather than reintroducing the old list. Granting this namespace-wide does not contradict the caution comment above it. The :8000 case had to be per-workload because that port is shared between the woodpecker server and the CNPG instance manager's admin API, so a blanket rule handed CI step pods a route to the database. ICMP echo and path-MTU messages expose no API surface, so a namespace-wide grant concedes nothing a scoped one would withhold — which is why every other baseline does it this way too. Co-Authored-By: Claude Bot --- .../network-policies/woodpecker-ns.yaml | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml index ff185600..0394fc25 100644 --- a/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml +++ b/kubernetes/infrastructure/network-policies/woodpecker-ns.yaml @@ -45,6 +45,42 @@ spec: - fromEntities: - host - remote-node + # ICMP — from any cluster pod. See home-assistant-ns.yaml for the rationale. + # INGRESS ONLY: do NOT add an egress section to this CNP. + # + # Needed here specifically because wp-pg-allow-cnpg-operator below attaches + # an L4 rule (toPorts :8000) to the wp-pg pods, and Cilium blocks ICMP to an + # endpoint once any layer 4 policy selects it unless the traffic is + # CT-related. Without this, PMTUD Destination Unreachable and Packet Too Big + # toward wp-pg are dropped silently — the failure shows up as stalled + # transfers, not as a policy denial. + # + # Namespace-wide is right for this one, unlike the :8000 case noted below: + # these are diagnostic and path-MTU messages that expose no API surface, so + # granting them across the namespace concedes nothing a per-workload rule + # would withhold. + - fromEndpoints: + - matchExpressions: + - key: io.kubernetes.pod.namespace + operator: Exists + icmps: + - fields: + - { type: 0, family: IPv4 } + - { type: 3, family: IPv4 } + - { type: 8, family: IPv4 } + - { type: 11, family: IPv4 } + - { type: 12, family: IPv4 } + - { type: 1, family: IPv6 } + - { type: 2, family: IPv6 } + - { type: 3, family: IPv6 } + - { type: 4, family: IPv6 } + - { type: 128, family: IPv6 } + - { type: 129, family: IPv6 } + - { type: 133, family: IPv6 } + - { type: 134, family: IPv6 } + - { type: 135, family: IPv6 } + - { type: 136, family: IPv6 } + - { type: 143, family: IPv6 } # # NOT here: ingress from `woodpecker-ci`. Pipeline step pods do call the # server back on :8000 to stream logs, but the CNPG instance manager serves -- 2.54.0