{"record":{"id":"0652120ef5ad5e8d","repo":"paperclipai/paperclip","slug":"paperclip-sandbox-callback-bridge-failed-to-abor","errorCode":null,"errorMessage":"[paperclip] sandbox callback bridge failed to abort queued requests after worker failure: ${failPendingError instanceof Error ? failPendingError.message : String(failPendingError)}","messagePattern":"\\[paperclip\\] sandbox callback bridge failed to abort queued requests after worker failure: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"packages/adapter-utils/src/sandbox-callback-bridge.ts","lineNumber":1535,"sourceCode":"          consecutivePollFailures += 1;\n          const message = `${buildWorkerFailureMessage(error)} (transient poll failure ${consecutivePollFailures}; retrying)`;\n          if (consecutivePollFailures === 1) {\n            // Put the first failure of a streak on the run trace; later repeats\n            // only warn, so a flapping channel does not spam failed spans.\n            await surfaceRunError(new Error(message));\n          } else {\n            console.warn(`[paperclip] ${message}`);\n          }\n          const backoffMs = Math.min(\n            pollIntervalMs * 2 ** consecutivePollFailures,\n            MAX_TRANSIENT_ITERATION_BACKOFF_MS,\n          );\n          await new Promise((resolve) => setTimeout(resolve, backoffMs));\n          continue;\n        }\n        // A file whose attempt is still in flight (or waiting on its 504\n        // backstop) is not actionable: `processRequestFile` would skip it via\n        // the guard map. Treat an all-guarded listing like an empty one and\n        // sleep a poll interval. Re-listing immediately would spin the loop —\n        // an exec storm against a real sandbox channel, and with an in-memory\n        // client a pure-microtask loop that starves every timer in the process\n        // (including the guard's own backstop and abort timers).\n        const actionableFileNames = fileNames.filter((fileName) => !inFlightRequestGuards.has(fileName));\n        if (actionableFileNames.length === 0) {\n          lastSuccessfulIterationAt = Date.now();\n          if (stopping) {\n            break;\n          }\n          await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));\n          continue;\n        }\n        for (const fileName of actionableFileNames) {\n          if (stopping && Date.now() >= stopDeadline) break;\n          inFlight += 1;\n          try {\n            // A request is run-time work, not startup work. Wrap it in a","sourceCodeStart":1517,"sourceCodeEnd":1553,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/packages/adapter-utils/src/sandbox-callback-bridge.ts#L1517-L1553","documentation":"The bridge worker loop caught a fatal error (e.g. per-iteration timeout while listing or processing request files), surfaced it as a run-trace span via surfaceRunError, then called failPendingRequests(message, { abandonInFlight: true }) to abort queued requests with 503s. That abort itself threw — so this warning reports the cleanup failure. Queued request files may remain, and their sandbox callers stay pending until their own deadlines; the run has already recorded the primary worker failure.","triggerScenarios":"Sandbox channel dies while requests are queued: the loop's listJsonFiles or a request write times out (worker failure), and the subsequent recovery sweep that writes 503 responses fails again on the same dead channel.","commonSituations":"Sandbox killed mid-run; shared storage unmounted; severe host I/O contention making every channel operation time out in sequence.","solutions":["Find the primary failure first: the run trace now contains the surfaced worker-failure span (surfaceRunError) — read it to see the root timeout/error.","Verify and restore sandbox/storage health, then stop and re-run the affected agent turn so a fresh bridge drains or 503s leftover requests.","Do not retry the run against the same sandbox until the channel responds (a quick list/write probe against the shared dir is a good smoke test).","Report with run logs if recovery keeps failing on a healthy channel — that combination indicates a defect."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"const probe = await withTimeout(input.client.listJsonFiles(directories.requestsDir), iterationTimeoutMs, \"probe\");\n// channel answers => safe to attempt failPendingRequests; otherwise expect stranded queue","typeGuard":null,"tryCatchPattern":"try {\n  await failPendingRequests(message, { abandonInFlight: true });\n} catch (failPendingError) {\n  // Best-effort cleanup failed on the same dead channel. The primary failure was\n  // already surfaced to the run trace — log and let callers hit their own deadlines.\n  console.warn(`[paperclip] sandbox callback bridge failed to abort queued requests: ${String(failPendingError)}`);\n}","preventionTips":["Surface the primary worker error to the run trace (surfaceRunError) so cleanup failure never hides the root cause.","Stop and re-run the agent turn after restoring sandbox health so a fresh bridge drains leftover requests.","Proactively probe the channel before recovery sweeps when a timeout just occurred.","Alert on this warn in run telemetry — it predicts stranded sandbox callers."],"tags":["sandbox","callback-bridge","worker-failure","recovery","cleanup"],"backgroundTag":"queued-request-abort-failed","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-08-18T22:49:45.177Z","contentChangedAt":"2026-08-18T22:49:45.177Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}