A WAL segment is a fixed 16 MiB file. archive_timeout forces a segment
switch on a timer so the archive never trails reality by more than that
interval, and the switched-out segment is written and archived at its full
16 MiB no matter how few bytes of real WAL it holds. CNPG defaults this to
5min, which permits up to 288 segments/day = 4.6 GB/day per cluster.
Measured on gf-pg: the database is 21 MB and generates 4.5 MB/day of actual
WAL, while its RBD volume takes 211 MB/hour -- against 192 MB/hour predicted
from 12 switches of 16 MiB. Essentially every byte written to this volume is
padding, roughly 1100x amplification.
This resolves an anomaly that has been open since the start of the endurance
work: gf-pg appeared to write gigabytes per day for a 21 MB database, and
neither WAL volume nor data changes could account for it. Segment padding
does, exactly.
Applied to gf-pg alone as a staged rollout. It is the lowest-risk cluster
and gives the cleanest signal -- its real WAL is small enough that any change
in volume is unambiguously attributable to this parameter. The same default
affects all seven clusters and is worth an estimated ~24 GB/day of logical
writes fleet-wide, but that only goes ahead once this one is verified.
The cost is RPO. On total loss of the cluster, up to 30min of Grafana state
becomes unrecoverable instead of 5min. That state is dashboards, annotations
and preferences, changing at ~4.5 MB/day, so the window is nearly always
empty. Clusters holding credentials or job state get evaluated separately
rather than inheriting this value by default.
archive_timeout is in CNPG's GlobalDefaultSettings, not its
FixedConfigurationParameters, so it is user-overridable -- verified against
pkg/postgres/configuration.go rather than assumed.
Context: drive-endurance work on three QLC NVMe OSDs at 81-87% wear.
`wal_compression: "on"` resolves to pglz, PostgreSQL's original 2016-era
algorithm. Every CNPG cluster here was set that way, so none of them were
getting the compression actually available to them: all seven builds accept
lz4 and zstd (verified via pg_settings.enumvals on each, including immich's
custom pgvector image and trmnl's PostgreSQL 18).
This matters because of what dominates recorder-style write volume. A page
is written to WAL in full -- all 8 KiB -- on its first modification after a
checkpoint, for torn-page protection. Measured on ha-pg, those full-page
images are 62.4% of WAL bytes. Compressing them is therefore the single
largest lever available on WAL volume, and pglz was the weakest option for
the job.
ha-pg additionally moves checkpoint_timeout 15min -> 30min. Full-page images
are emitted once per page per checkpoint interval, so doubling the interval
roughly halves how often each hot page pays the 8 KiB toll. max_wal_size
stays at 2GB deliberately: it must remain well above what one interval
generates, or checkpoints would trigger on size and the interval change
would do nothing. Recovery replays at most one interval of WAL -- under
200MB at the measured rate, i.e. seconds.
The other six clusters keep checkpoint_timeout at 15min. Their measured
write rates do not justify trading recovery time: gf-pg generates 4.5 MB/day
of WAL and wp-pg 62 MB/day, against ak-pg's 604 MB/day.
full_page_writes stays on everywhere. Disabling it would remove the
overhead entirely but RBD can tear a page across an object boundary, and
the trade is write volume against an unrecoverable database.
Both parameters are sighup-reloadable, so this does not restart any cluster.
Context: drive-endurance work on three QLC NVMe OSDs at 81-87% wear.
Prometheus held 298,236 active series. ~108,000 of them -- over a third -- were
apiserver/etcd `_bucket` histograms. Each fans out to one series per bucket x
verb x resource x scope, so a single metric was 23,810 series on its own. Every
Home Assistant metric combined is ~4,100 for comparison.
That cardinality is the dominant driver of Prometheus write volume (~11 GB/day of
TSDB + WAL, doubled again by Ceph replication) and none of it is observable value
on three nodes: per-bucket apiserver latency quantiles are for clusters with SLAs
and dozens of nodes.
Dropped, all verified to have ZERO rule dependencies by checking each metric
against the live alert and recording rules:
apiserver_request_duration_seconds_bucket rules=0 23,810 series
apiserver_request_body_size_bytes_bucket rules=0 23,200
etcd_request_duration_seconds_bucket rules=0 15,840
apiserver_watch_list_duration_seconds_bucket rules=0 11,538
apiserver_watch_cache_read_wait_seconds_bucket rules=0 8,232
apiserver_response_sizes_bucket rules=0 7,592
apiserver_watch_events_sizes_bucket rules=0 5,220
-------
95,432
KEPT deliberately: apiserver_request_sli_duration_seconds_bucket (12,144 series).
23 recording rules depend on it, including the apiserver availability SLO and the
KubeAPIErrorBudgetBurn burn-rate series. Dropping it would silently break that
alerting -- the manifest says so inline, so a later "tidy up" of the regex does
not remove it by accident.
Only `_bucket` series are dropped. The matching `_count` and `_sum` remain, so
request rates and mean latencies still work; what is lost is per-bucket
quantiles.
etcd_* is included even though kubeEtcd is disabled: those are the apiserver's
own etcd-client metrics on the apiserver endpoint, not a separate etcd scrape.
Expected: ~32% fewer active series, so roughly 11 -> 7 GB/day of Prometheus
writes, ~8 GB/day off the drives once replication is counted. Will verify against
prometheus_tsdb_head_series rather than assume.
Refs #434
Co-Authored-By: Claude Bot <noreply@anthropic.com>
An image reference is not a secret. Encoding it as base64 `data` had two real
costs: no reviewer sees a version change in a diff, and no $imagepolicy marker
can attach to a base64 blob.
The result was a two-minor version skew in the component that performs every
base backup and WAL archive for seven Postgres clusters (#535):
controller plugin-barman-cloud:v0.14.0 <- auto-bumped by Flux
sidecar plugin-barman-cloud-sidecar:v0.12.0 <- invisible, never moved
Not stale pods: ha-pg-1 was created hours ago during the recorder migration and
still received v0.12.0.
Three changes:
1. The Secret now uses stringData with the tag in plaintext, carrying an
$imagepolicy marker. The Secret KIND is kept because the controller reads it
via secretKeyRef; switching to a ConfigMap would need a matching edit to that
reference and diverge further from upstream. stringData is accepted
identically by the API server.
2. ImageRepository + ImagePolicy added for the sidecar image, with the same
filter and 0.x range as the controller, so the two cannot drift apart again.
Only the controller was tracked before.
3. Tag aligned to v0.14.0 to match the controller.
The Secret name carries a kustomize hash suffix from upstream's secretGenerator
but is static here, so editing the value does not change the name and the
deployment's secretKeyRef stays valid — verified in the render, which still shows
the name twice (Secret + reference).
Verified the tag EXISTS before pinning it: pulling
plugin-barman-cloud-sidecar:v0.14.0 succeeds (the pod fails later on a missing
`true` binary in that minimal image, which is the container command, not the
pull). Pinning a non-existent tag would have put all seven Postgres clusters into
ImagePullBackOff.
EXPECTED IMPACT: bumping the sidecar changes every Postgres pod spec, so all
seven CNPG clusters roll. CNPG handles this as a normal rolling restart. Backups
should be confirmed working afterwards — 113 Backup CRs exist today, newest ~3h
old and completed.
Note for future re-imports: upstream ships this Secret as base64 `data` with no
marker, so a wholesale re-import silently reintroduces the skew. The manifest now
says so inline.
Refs #535
Co-Authored-By: Claude Bot <noreply@anthropic.com>
Switches the Deployment's `data` volume from paperless-data (RWX,
ceph-filesystem) to paperless-data-block (RWO, ceph-block). `consume` and `media`
stay on CephFS -- the ftp-scanner writes to those from another node, so they
genuinely need RWX.
The copy is already done and verified byte-for-byte with paperless stopped:
src files: 75
dst files: 75
IDENTICAL: name, size and mtime match for every file
index segments: src 3, dst 3
mtime parity matters specifically because the Whoosh index segments carry their
own consistency expectations; `cp -a` preserved them. (The "can't preserve" lines
in the copy log were for the mount root /dst/. itself, not the files.)
Why: only this Deployment ever mounted the volume, and its contents are
single-writer state. Being on CephFS meant every access crossed the MDS --
measured with zero CI running, 22 requests/sec sustained and ~15 journal
events/sec against 2 directory commits per 180s, i.e. cap/inode churn rather
than file writes. That accounted for most of the constant 7-8 GB/day on
cephfs-metadata, which is a size 3 pool.
cephfs-metadata stays at size 3 deliberately: losing it destroys the entire
filesystem rather than the objects on one OSD, and the 275 GiB in cephfs-data0
depends on it.
paperless-data is retained (prune: disabled) as the rollback. It must NOT be
deleted until search has been confirmed returning results -- that is what proves
the index survived, not merely that paperless starts.
Refs #434
Co-Authored-By: Claude Bot <noreply@anthropic.com>
paperless-data is declared ReadWriteMany on ceph-filesystem, but only ONE pod
mounts it -- the paperless Deployment itself. Its contents are all single-writer
state: the 50 MB Whoosh index, classification_model.pickle,
celerybeat-schedule.db, log/, migration_lock. 75 files, nothing shared. Only
`consume` and `media` genuinely need RWX, because the ftp-scanner writes to those
from another node.
It is on CephFS by default, not by requirement, and every access goes through the
MDS. Measured 2026-08-20 with ZERO CI pipelines running:
paperless-data client request_load_avg 22 (all other sessions 0-1)
MDS over 180s 4,052 client requests = 22.5/s
2,718 journal events = 15/s
2 dir commits <- not file writes
Cap and inode churn from repeated access, not data being written. That accounts
for most of the constant 7-8 GB/day on cephfs-metadata, and since that pool is
size 3, roughly 21-24 GB/day reaches the drives.
cephfs-metadata deliberately STAYS at size 3. Losing it destroys the whole
filesystem rather than the objects on one OSD, and the 275 GiB in cephfs-data0
rides on it. Cutting writes at source is the fix; cutting replication is not.
PHASE 1 ONLY, and inert: the PVC is created empty and nothing mounts it. The data
copy and the deployment repoint are separate steps, documented in
MIGRATION-data-to-block.md along with why the copy needs paperless stopped (the
Whoosh index and celerybeat schedule are both live-written, so copying under an
active writer gives a torn read).
Honest sizing: ~21-24 GB/day off the drives against ~432 GB/day cluster-wide, so
about 5%. This is a correctness fix -- a single-writer volume does not belong on
a shared filesystem -- more than a large win. It also removes 22 req/sec of MDS
load, which benefits Woodpecker CI workspaces sharing the same MDS.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ten Postgres instances across seven CNPG clusters are ~48 GB/day of the
cluster's ~114 GB/day of client writes, and only ha-pg carried any
write-reduction tuning. The rest ran Postgres defaults, which favour throughput
over storage that wears out.
MEASURED, not assumed. ak-pg since 2026-05-17:
checkpoints_timed 27,624 = 291/day, i.e. one every 5 min (the default)
buffers_checkpoint 25,801,628 = ~206 GB flushed
database size 1,046 MB
That is the whole database rewritten roughly twice a day. The pathology is
specific to idle databases: sessions, tokens and job state re-dirty the same few
pages constantly, and checkpoint_timeout=5min forces each of them to disk 288
times a day instead of letting them coalesce. At 15min they are written a third
as often.
max_wal_size is raised alongside it. Without that, checkpoints trigger on size
instead and the timeout change does nothing. It also bounds crash recovery to at
most 2GB of WAL replay.
Applied to ak-pg, pprl-pg, tr-pg, wp-pg, im-pg, gf-pg. ha-pg already had it.
synchronous_commit is deliberately NOT changed here. It is off for ha-pg because
a state recorder is telemetry; these clusters hold credentials, documents and job
state, where losing the last few hundred ms of commits on an unclean shutdown is
not an acceptable trade for write volume.
Separately, ha-pg shared_buffers 256MB -> 512MB. After 15h on Postgres its
buffers_backend was 629,704 of 939,025 total page writes -- 67% written by
backends rather than the checkpointer or bgwriter, which means the working set
does not fit in the buffer pool and pages are written out repeatedly instead of
coalescing. Memory limit raised 1Gi -> 1536Mi to keep the same headroom (measured
383Mi in use, of which 256MB was shared_buffers).
Verified before commit by rendering every cluster and asserting the tuning landed
under spec.postgresql.parameters with the barman plugin block untouched. That
check exists because the first attempt at this matched the wrong `parameters:`
key -- the barman plugin's, at 6-space indent under plugins: -- and corrupted
wp-pg's barmanObjectName. Reverted and redone with an indent-aware match.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up to the trmnl renderer finding. An `emptyDir` without `medium: Memory`
is NODE STORAGE, which is easy to read as "ephemeral, therefore free" -- it is
ephemeral, but every byte still hits the node's disk.
Survey of the repo: 23 emptyDir declarations, only 2 RAM-backed. This converts
12 of the remaining ones.
HONEST SCOPE: this buys almost no write reduction. Ranking every pod by OS-disk
writes over 6h shows trmnl-renderer at 67 GB/day (fixed separately) and then
nothing emptyDir-driven -- the rest of the top of that list is Ceph mons
(hostPath, not emptyDir), kube-apiserver, and container logs from databases.
Every volume converted here measured 8.0K in the running pod:
/var/lib/code-server 8.0K
/mosquitto/auth 8.0K
/state (z2m guards) 8.0K
So the value is correctness and, for mosquitto, keeping credential material off
the node's filesystem: that volume holds the generated passwd/acl files.
Each converted volume gets an explicit sizeLimit (16Mi where none existed), so
tmpfs cannot silently consume a container's memory limit. All are orders of
magnitude above measured usage.
Behaviour is unchanged otherwise: an emptyDir persists for the pod's lifetime on
either medium, so the mosquitto shim's assumption that the passwd file survives
a container restart still holds.
DELIBERATELY NOT CONVERTED:
- kubernetes/talos/cilium.yaml and talos/patches/cilium-cni.yaml -- vendored
and generated from the chart; a hand edit would be lost on upgrade.
- trmnl/renderer.yaml -- converted separately, where the actual 34 GB/day was.
Cost: this rolls the affected pods once.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The renderer was the single largest writer in the cluster -- more than every
Ceph OSD and every database combined -- and none of it needed to touch disk.
Measured 2026-08-20 from the container's own cgroup io.stat:
253:0 (node /dev/vda5) 249,092,697,088 bytes over 7.3 days = ~34 GB/day
252:48 (its own PVC) 340 MB total = negligible
So the writes were not going to its volume at all; they were going to the
node's filesystem via a disk-backed emptyDir. `emptyDir: {}` without
`medium: Memory` is node storage, which is easy to read as "ephemeral, so
free".
The cause is cadence rather than volume. schedules.template.json renders on
"* * * * *", so Chrome is launched 1440 times a day and each launch creates,
fills and deletes a /tmp/puppeteer_dev_chrome_profile-* directory. At rest /tmp
holds under 8 MB -- it is pure churn for data that never outlives the render.
tmpfs pages count against the container's memory limit, so the 512Mi cap is
kept deliberately: measured usage is 129Mi against a 2Gi limit, leaving ample
headroom even if the cap were fully consumed.
Expected effect: ~34 GB/day off the OS disks, which measured 215 GB/day over a
4h window. This is a bigger cluster-wide reduction than the Home Assistant
recorder migration achieved, for a one-line change.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Net -44 lines. The previous version assembled the connection URI from
username/password/dbname, which forced percent-encoding, YAML-safe quoting via
json.dumps, a YAML round-trip validation, and python3 from HA's image.
None of that was necessary: CNPG already publishes a finished, percent-encoded
connection string in the app secret as `uri` / `fqdn-uri`. Reading that key
removes the whole class of problem the earlier review correctly flagged --
rather than solving it more carefully.
Verified against the live secret before simplifying: fqdn-uri contains only
[A-Za-z0-9:/@._%-], so naive double-quoting is provably safe and the YAML
validation was guarding against an input that cannot occur.
What is kept, because it is the part that actually matters: secrets.yaml is PVC
state holding every other HA secret (influxdb_token and friends), so the file is
filtered-and-appended rather than rewritten, moved atomically, and guarded
against shrinking. Tested in HA's own busybox shell -- comment and pre-existing
keys preserved, idempotent across runs, no duplicate key.
Also records why an env var cannot be used at all: HA removed `!env_var` from
its YAML loader, so `!secret` is the only mechanism for recorder.db_url, and
config-sync deliberately does not manage secrets.yaml.
Back to busybox; HA's image is no longer needed for this.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 2b is done: 12.77M statistics rows copied SQLite -> Postgres with HA
running and no downtime (read-only mount; SQLite WAL gives readers a consistent
snapshot).
statistics_meta 1449 OK
statistics 10799904 OK 153.9s
statistics_short_term 1969804 OK 27.0s
span in Postgres: 2024-03-13 -> 2026-08-19
1449 entities with long-term history, 2627859 rows carrying sum
database size 2252 MB against a 20Gi PVC
Records the high-water marks the cutover top-up must resume from, and two schema
details that cost a failed first attempt:
- has_mean/has_sum are INTEGER in SQLite, BOOLEAN in Postgres, and Postgres
will not coerce them.
- the legacy created/start/last_reset DATETIME columns are 100% NULL in both
bulk tables, which is what makes a tab-separated COPY safe.
Also documents why the schema was built from HA's own models with schema_changes
left EMPTY: HA's _initialize_database detects a new database via the
events.time_fired_ts index and writes SCHEMA_VERSION itself, so pre-stamping that
row would bypass HA's own initialisation.
ANALYZE was run on all three tables. After a bulk COPY the planner has no
statistics and HA seq-scans 10.8M rows on every history/energy view.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review catch on #524, and it was a genuine blocker.
The first version built "postgresql://${PGUSER}:${PGPASSWORD}@host/db" in
busybox sh with no encoding. Two defects:
1. The CNPG-generated password is random and may contain @ : / ? #, each of
which changes how the URL parses. This repo already hit this exact issue
with the same CNPG password-generation mechanism -- see
kubernetes/apps/woodpecker/server.yaml, which switched to a lib/pq
keyword DSN specifically to avoid escaping generated passwords. That
option does not exist here: HA's recorder.db_url must be a SQLAlchemy URL.
So the fix is percent-encoding.
2. Worse, and the reason this was worth blocking: a quote in the password
makes the YAML line malformed, so secrets.yaml stops parsing entirely and
HA loses EVERY secret, not just this one. The line-count guard could never
catch that -- the count is unchanged; the file is simply invalid.
Rewritten to run python3 from HA's own image (already pulled for the main
container, so no extra image) instead of busybox:
- urllib.parse.quote(..., safe="") on user, password and dbname.
- json.dumps for the YAML value: YAML is a JSON superset, so this emits a
correctly escaped double-quoted scalar even with quotes or backslashes.
- The file is still rewritten LINE-WISE rather than round-tripped through the
YAML parser, so hand-maintained comments and key order survive.
- The guard now validates the RESULT: parse the new content, assert no key
from the old file disappeared, and assert recorder_db_url round-trips to
exactly the URL that was built. That catches corruption the line count
cannot.
Verified in HA's image against a deliberately hostile password
(p@ss:w/o"rd#?&\x=1) before commit:
new version -> parses, comment preserved, idempotent, password decodes back
byte-exact, host and dbname parse correctly
naive version -> yaml.safe_load raises ParserError
i.e. the approach this replaces would have produced an unparseable
secrets.yaml.
Also updates MIGRATION-recorder-postgres.md for this step landing, including
why the ordering (init container before the configuration.yaml change) is
mandatory.
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 2a of the recorder move off SQLite (MIGRATION-recorder-postgres.md).
Adds an init container that writes `recorder_db_url` into /config/secrets.yaml
from the CNPG-generated ha-pg-app Secret.
INERT until the recorder is repointed. It only adds a key to secrets.yaml;
nothing reads it until `db_url: !secret recorder_db_url` lands in
configuration.yaml in the homeassistant repo. That ordering is REQUIRED in this
direction — HA refuses to start when a referenced secret is missing, so the
config change must not go first.
Why an init container rather than a hand-added line: the password is generated
by CNPG and can be rotated, so a hand-written entry goes stale silently and HA
then fails to start on a bad password. Re-rendering on every pod start is also
what makes a rotation take effect.
secrets.yaml is PVC runtime state that config-sync deliberately does not manage
(it holds influxdb_token and others), so this must EDIT the existing file, never
replace it. The script filters out any existing recorder_db_url line, appends
the current one, and guards the result: if the rewrite would drop any
non-recorder line it aborts rather than truncating the file holding every other
secret HA needs. Written to a temp path and moved atomically, chmod 600.
Verified in HA's own Alpine/busybox shell against a fixed sample before commit
-- pre-existing keys preserved, guard reports old=3 new=3, and a second run
produces a byte-identical 4-line file (idempotent, no duplicate key).
Refs #434
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven Postgres clusters back up to Hetzner and nothing alerted if that stopped.
The only related rule was HetznerBackupEndpointDown, which probes the S3
endpoint — it proves the bucket is reachable, not that anything was written to
it. Same failure shape as the WordPress backup that failed nightly for three
days against a bucket declared in tofu but never applied: the status surface
looked fine because nothing checked the thing that mattered.
CNPGWalArchiveBacklog is the load-bearing rule. `ready` counts WAL segments
waiting in pg_wal/archive_status; steady state is 0 on every cluster. A
sustained backlog means the archiver is failing, so point-in-time recovery is
silently broken from that moment and pg_wal grows until it fills the PVC and
takes Postgres down. Threshold >5 sustained for 30m: a segment is 16 MiB, so
that is ~80 MiB of un-archived WAL — well below anything that threatens the
volume, far above idle noise. It reads only the instance-level collector, so it
works despite #522.
CNPGBackupFailed fires on a base backup failure within 24h, guarded on
timestamp > 0 so the epoch-zero default is not read as "failed in 1970".
DELIBERATELY NOT ADDED: the backup-age alert, which is the one you would most
want. `(time() - cnpg_collector_last_available_backup_timestamp) > 48h` cannot
be used today because that metric reads exactly 0 on every instance in the
cluster (verified across gf-pg, ak-pg-1/2, pprl-pg-2/3, im-pg, wp-pg, ha-pg).
CNPG has no record of any successful backup anywhere, because the Backup CRs it
derives that from are deleted within a minute of completing — #522. Adding it
now would fire permanently for all seven clusters and be silenced within a day,
which is worse than not having it.
The backups themselves DO work; verified independently by listing the bucket
with barman-cloud-backup-list and restoring a WAL segment back out of S3. The
gap is in the record, not the data.
Both expressions were validated against live Prometheus before commit rather
than written from the metric names: the WAL selector matches 8 real series (a
zero match would mean a dead rule, the defect fixed in
prometheusrule-woodpecker-rotation.yaml), and neither expression fires
currently, which is correct since archiving is healthy and the only recorded
backup failure is ~91 days old.
Refs #522
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
summary:"CNPG WAL archiving is falling behind on {{ $labels.namespace }}/{{ $labels.pod }}"
description:"{{ $value }} WAL segments have been waiting to archive for over 30 minutes. Point-in-time recovery is broken for anything after the last archived segment, and pg_wal will grow until the PVC fills. Check the barman-cloud sidecar logs and the ObjectStore credentials: `kubectl -n {{ $labels.namespace }} logs {{ $labels.pod }} -c plugin-barman-cloud`."
- alert:CNPGBackupFailed
# cnpg_collector_last_failed_backup_timestamp is a unix timestamp, 0
# when no failure is recorded. Guard on > 0 so the epoch-zero default
summary:"CNPG backup failed in the last 24h on {{ $labels.namespace }}/{{ $labels.pod }}"
description:"A base backup for {{ $labels.namespace }}/{{ $labels.pod }} failed within the last 24 hours. Verify what is actually in the bucket rather than trusting CR status — see the barman-cloud-backup-list recipe in kubernetes/apps/home-assistant/MIGRATION-recorder-postgres.md."
# DELIBERATELY ABSENT: a backup-age alert, which is the one you would
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
claude-bot
k.renner
security-bot