fix(paperclip): k8s plugin returns a termination receipt (REN-841) #1155

2026-10-06T13:57:54Z

Ghost commented

2026-10-06 13:57:54 +00:00

What

New image patch ci/images/paperclip/ren-841-k8s-termination-receipt.patch for @paperclipai/plugin-kubernetes:

onEnvironmentReleaseLease and onEnvironmentDestroyLease now return { providerLeaseId, state: "destroyed" }. They do so only once the Sandbox CR or Job (and, on destroy, the pod) reads 404. The new waitForWorkloadGone in lease-lifecycle.ts polls every 500 ms for up to 20 s, below the server's 30 s plugin RPC timeout.
If the resource still exists at the deadline, or a read fails, the hook returns no receipt. That is today's behaviour, so the change never throws after a successful delete.
README entry added.

Why

The server lets a stopped run's card continue only if every remote lease has a termination receipt (server/src/services/explicit-native-continuation.ts:228; remote-execution-termination.ts:11-16). Our plugin returned void, so every cancelled or failed pod run left its card waiting on remote_cleanup forever (REN-837, REN-825). Daytona's plugin already returns these receipts. #1154 repairs the two existing leases; this PR stops new ones from getting stuck.

How verified

sh prepare.sh applies all five patches to a027f76.
vitest run test/unit in the plugin: 20 files, 211 tests pass. New tests cover release and destroy receipts, the 404-already case, a read error giving no receipt, a still-present CR or pod at the deadline, and the Job backend.
tsc --noEmit -p . in the plugin passes.
The plugin's SA already has get on sandboxes and pods in paperclip-renner (sandbox-rbac.yaml).
Not yet live. After the merge, Woodpecker builds a new build.<ts> image and Flux rolls the paperclip Deployment. Live proof: cancel a pod run by reassign, and the owner's next comment on that card starts a run.

Impact

Merging rolls the paperclip server pod, the usual image bump. Runs in flight at that moment are interrupted, so merge when the board is quiet.

Revert

Revert this PR. The next image build drops the patch.

🤖 Generated with Claude Code

## What New image patch `ci/images/paperclip/ren-841-k8s-termination-receipt.patch` for `@paperclipai/plugin-kubernetes`: - `onEnvironmentReleaseLease` and `onEnvironmentDestroyLease` now return `{ providerLeaseId, state: "destroyed" }`. They do so only once the Sandbox CR or Job (and, on destroy, the pod) reads 404. The new `waitForWorkloadGone` in `lease-lifecycle.ts` polls every 500 ms for up to 20 s, below the server's 30 s plugin RPC timeout. - If the resource still exists at the deadline, or a read fails, the hook returns no receipt. That is today's behaviour, so the change never throws after a successful delete. - README entry added. ## Why The server lets a stopped run's card continue only if every remote lease has a termination receipt (`server/src/services/explicit-native-continuation.ts:228`; `remote-execution-termination.ts:11-16`). Our plugin returned `void`, so every cancelled or failed pod run left its card waiting on `remote_cleanup` forever (REN-837, REN-825). Daytona's plugin already returns these receipts. #1154 repairs the two existing leases; this PR stops new ones from getting stuck. ## How verified - `sh prepare.sh` applies all five patches to `a027f76`. - `vitest run test/unit` in the plugin: 20 files, 211 tests pass. New tests cover release and destroy receipts, the 404-already case, a read error giving no receipt, a still-present CR or pod at the deadline, and the Job backend. - `tsc --noEmit -p .` in the plugin passes. - The plugin's SA already has `get` on sandboxes and pods in `paperclip-renner` (`sandbox-rbac.yaml`). - Not yet live. After the merge, Woodpecker builds a new `build.<ts>` image and Flux rolls the paperclip Deployment. Live proof: cancel a pod run by reassign, and the owner's next comment on that card starts a run. ## Impact Merging rolls the paperclip server pod, the usual image bump. Runs in flight at that moment are interrupted, so merge when the board is quiet. ## Revert Revert this PR. The next image build drops the patch. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Ghost added 1 commit 2026-10-06 13:57:54 +00:00

fix(paperclip): k8s plugin returns a termination receipt (REN-841)

ci/woodpecker/pr/pr-review Pipeline was successful

Details

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

abc09ac722

The kubernetes sandbox plugin's release and destroy hooks returned void,
so no lease got a termination receipt and every cancelled pod run left
its card waiting on remote_cleanup. Return state destroyed once the
Sandbox CR / Job (and pod) reads 404, within 20 s; otherwise no receipt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Ghost approved these changes

2026-10-06 14:30:54 +00:00

Ghost left a comment

Approved at abc09ac722. Checked: all 5 patches git-apply on upstream a027f76; plugin unit tests 211/211 pass (12 runs; one early run showed 1 transient vitest error, not reproduced in 11 reruns); tsc --noEmit clean with the built plugin-sdk; receipt {providerLeaseId,state:destroyed} satisfies remoteTerminationReceipt; deleteSandboxCr/deleteJob use Foreground propagation so workload 404 implies pod gone; waitForWorkloadGone never throws, 20 s cap below the 30 s RPC timeout; read failure returns no receipt (today's behaviour). Rollout = one non-HA pod roll.

Approved at abc09ac722b93a0de2356856e11b526fa7ab4b7d. Checked: all 5 patches git-apply on upstream a027f76; plugin unit tests 211/211 pass (12 runs; one early run showed 1 transient vitest error, not reproduced in 11 reruns); tsc --noEmit clean with the built plugin-sdk; receipt {providerLeaseId,state:destroyed} satisfies remoteTerminationReceipt; deleteSandboxCr/deleteJob use Foreground propagation so workload 404 implies pod gone; waitForWorkloadGone never throws, 20 s cap below the 30 s RPC timeout; read failure returns no receipt (today's behaviour). Rollout = one non-HA pod roll.

Ghost merged commit 293b86a10d into master

2026-10-06 14:30:55 +00:00

Ghost referenced this issue from a commit

2026-10-06 14:30:56 +00:00

Merge pull request 'fix(paperclip): k8s plugin returns a termination receipt (REN-841)' (#1155) from atlas/ren-841-k8s-plugin-termination-receipt into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#1155