fix(paperclip): one-shot Job to record lease termination receipts (REN-841) #1154

2026-10-06T13:55:12Z

Ghost commented

2026-10-06 13:55:12 +00:00

What

One-shot Job paperclip/pclp-lease-receipt-repair-ren841. It runs psql against pclp-pg-rw and adds metadata.remoteExecutionTermination (state destroyed) to the environment leases of two cancelled runs:

e08976cd-b520-4c42-8248-35b4b841a98d (REN-837)
144ae3bf-f67b-4421-8a4c-e16b2bc19973 (REN-825)

The UPDATE only touches leases that are remote, ephemeral, released with cleanup success, and still without a receipt. It prints the rows before and after.

Why

The @paperclipai/plugin-kubernetes onEnvironmentReleaseLease and onEnvironmentDestroyLease hooks return void, so the server records no termination receipt (remote-execution-termination.ts:11-16). Admission of the owner's queued comment needs a receipt on every remote lease (explicit-native-continuation.ts:228), so it returns remote_cleanup forever.

The CRs are gone. The agent-sandbox controller logged pc-01m48nfvzzhft28y8cm6srr1sy deleted at 13:13:26Z, 1 s after e08976cd finished, and pc-01m48em9azz4sx3e3yjbc5t87a at 11:16:35Z, 1 s after 144ae3bf finished. paperclip-renner has no pods. A plugin fix for future runs comes in a separate PR.

How verified

kubectl kustomize kubernetes/apps/paperclip renders the Job.
Run ids were checked against each card's executionBlocker.runId.
After the merge: the Job log shows the updated rows. The server's periodic resumeExecutionWaitComments then starts the queued owner comment on REN-837 within about a minute.

Revert

Revert this PR. A Job without a TTL is not re-run by Flux. The receipt is plain JSON in lease metadata, and leaving it in place is harmless. Once the Job has run, a follow-up PR removes the file.

🤖 Generated with Claude Code

## What One-shot Job `paperclip/pclp-lease-receipt-repair-ren841`. It runs psql against `pclp-pg-rw` and adds `metadata.remoteExecutionTermination` (state `destroyed`) to the environment leases of two cancelled runs: - `e08976cd-b520-4c42-8248-35b4b841a98d` (REN-837) - `144ae3bf-f67b-4421-8a4c-e16b2bc19973` (REN-825) The UPDATE only touches leases that are remote, ephemeral, released with cleanup `success`, and still without a receipt. It prints the rows before and after. ## Why The `@paperclipai/plugin-kubernetes` `onEnvironmentReleaseLease` and `onEnvironmentDestroyLease` hooks return `void`, so the server records no termination receipt (`remote-execution-termination.ts:11-16`). Admission of the owner's queued comment needs a receipt on every remote lease (`explicit-native-continuation.ts:228`), so it returns `remote_cleanup` forever. The CRs are gone. The agent-sandbox controller logged `pc-01m48nfvzzhft28y8cm6srr1sy` deleted at 13:13:26Z, 1 s after e08976cd finished, and `pc-01m48em9azz4sx3e3yjbc5t87a` at 11:16:35Z, 1 s after 144ae3bf finished. `paperclip-renner` has no pods. A plugin fix for future runs comes in a separate PR. ## How verified - `kubectl kustomize kubernetes/apps/paperclip` renders the Job. - Run ids were checked against each card's `executionBlocker.runId`. - After the merge: the Job log shows the updated rows. The server's periodic `resumeExecutionWaitComments` then starts the queued owner comment on REN-837 within about a minute. ## Revert Revert this PR. A Job without a TTL is not re-run by Flux. The receipt is plain JSON in lease metadata, and leaving it in place is harmless. Once the Job has run, a follow-up PR removes the file. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Ghost added 1 commit 2026-10-06 13:55:12 +00:00

fix(paperclip): one-shot Job to record lease termination receipts (REN-841)

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

ci/woodpecker/pr/pr-review Pipeline was successful

Details

ci/woodpecker/pr/ci-manifests-test Pipeline was successful

Details

ce80a15ab4

The kubernetes sandbox plugin returns no termination receipt, so the
leases of cancelled runs e08976cd (REN-837) and 144ae3bf (REN-825) were
released without one and the queued owner comments wait on
remote_cleanup forever. The controller log shows both Sandbox CRs
deleted. Write the missing receipt for exactly those leases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Ghost referenced this pull request

2026-10-06 13:57:54 +00:00

fix(paperclip): k8s plugin returns a termination receipt (REN-841) #1155

Ghost approved these changes

2026-10-06 14:28:16 +00:00

Ghost left a comment

Approved at ce80a15ab4. Checked: receipt JSON keys match hasRemoteTerminationReceipt/remoteTerminationReceipt at upstream a027f76 (schema, companyId, runId, leaseId, provider, providerLeaseId, state destroyed); column names match packages/db environment_leases; run ids equal REN-837/REN-825 executionBlocker.runId; Sandbox CRs gone (paperclip-renner holds only an unrelated sandbox); pg image digest equals running pclp-pg pods; same-namespace netpol allows the Job to pclp-pg:5432; CI success.

Approved at ce80a15ab476b71358c7fb21e7c8de37a43cc94b. Checked: receipt JSON keys match hasRemoteTerminationReceipt/remoteTerminationReceipt at upstream a027f76 (schema, companyId, runId, leaseId, provider, providerLeaseId, state destroyed); column names match packages/db environment_leases; run ids equal REN-837/REN-825 executionBlocker.runId; Sandbox CRs gone (paperclip-renner holds only an unrelated sandbox); pg image digest equals running pclp-pg pods; same-namespace netpol allows the Job to pclp-pg:5432; CI success.

Ghost merged commit 53bc71737d into master

2026-10-06 14:28:17 +00:00

Ghost referenced this issue from a commit

2026-10-06 14:28:19 +00:00

Merge pull request 'fix(paperclip): one-shot Job to record lease termination receipts (REN-841)' (#1154) from atlas/ren-841-lease-receipt-repair into master

Ghost referenced this pull request

2026-10-06 14:32:14 +00:00

chore(paperclip): remove the one-shot lease receipt repair Job (REN-841) #1156

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#1154