[paperclip] sandbox callback bridge kept queued request ${re

Error message

[paperclip] sandbox callback bridge kept queued request ${requestId} after every recovery 503 write failed: ${lastWriteError}

What it means

In the sandbox callback bridge's recovery path (failPendingRequests), a queued request that was never claimed must be answered with a terminal 503 response file. The write is retried up to MAX_BACKSTOP_WRITE_ATTEMPTS times, each bounded by the per-iteration timeout. If every attempt fails (lastWriteError), the bridge deliberately keeps the request .json file so a later recovery pass or caller retry can still deliver the 503 — dropping it would strand the sandbox caller until its own deadline. The warning names the retained requestId.

Source

Thrown at packages/adapter-utils/src/sandbox-callback-bridge.ts:1285

          wrote503 = true;
          break;
        } catch (error) {
          lastWriteError = error instanceof Error ? error.message : String(error);
          console.warn(
            `[paperclip] sandbox callback bridge failed to write recovery 503 for ${requestId} (attempt ${attempt}/${MAX_BACKSTOP_WRITE_ATTEMPTS}): ${lastWriteError}`,
          );
          if (attempt < MAX_BACKSTOP_WRITE_ATTEMPTS) {
            await new Promise((resolve) => setTimeout(resolve, BACKSTOP_WRITE_RETRY_MS));
          }
        }
      }
      if (wrote503) {
        // The 503 landed. Remove the request file, so the poll loop does not
        // re-process it.
        await input.client.remove(requestPath).catch(() => undefined);
      } else {
        // Every 503 write failed. Keep the request file for a later recovery pass.
        console.warn(
          `[paperclip] sandbox callback bridge kept queued request ${requestId} after every recovery 503 write failed: ${lastWriteError}`,
        );
      }
    }
  };

  // Surface a bridge-worker failure through the run trace, not only stdout. A
  // failed span under `input.runtimeSpan` records the error against the live run
  // span, so the run and the orchestrator see the hang. When no `runtimeSpan`
  // runner is wired (no injected tracer), the helper still writes a warn line,
  // so the failure is never silent.
  const surfaceRunError = async (error: Error) => {
    if (input.runtimeSpan) {
      try {
        await input.runtimeSpan(CALLBACK_BRIDGE_WORKER_FAILED_SPAN, async () => {
          throw error;
        });
      } catch {

View on GitHub (pinned to a7e689b3c3)

Solutions

Check sandbox health: is the sandbox process/container alive and the shared directory writable? Restart or resume it.
Let a later recovery pass handle the retained request — re-running the agent turn re-creates the bridge and re-enumerates the queue.
Inspect run logs for the sibling warnings (failed read / failed 503 write per attempt) to confirm the channel, not the code, is the failure point.
If the sandbox is unrecoverable, stop the run: the sandbox caller is designed to hit its own deadline and surface a timeout.

Defensive patterns

Strategy: retry

Validate before calling

// Smoke-test the channel before relying on recovery writes
try {
  await withTimeout(input.client.listJsonFiles(directories.requestsDir), 2_000, "channel probe");
} catch {
  // channel unresponsive: skip recovery sweep, let a later pass deliver 503s
}

Type guard

const hasQueuedRequests = (files: string[]): boolean => files.some((f) => f.endsWith(".json"));

Try / catch

for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
  try { await writeBridgeResponse(...); wrote = true; break; }
  catch (e) { lastWriteError = e; await delay(BACKOFF_MS); }
}
if (!wrote) {
  // keep the request file — never drop it; a later pass or caller retry handles it
  console.warn(`kept queued request ${requestId}: ${lastWriteError}`);
}

Prevention

Keep the sandbox (and its shared storage) alive and responsive while callbacks are in flight; pause/kill only at turn boundaries.
Bound every channel op with the per-iteration timeout so a dead channel fails fast instead of hanging the sweep.
Monitor for this warn: it means degraded mode — queued callers are waiting on a later recovery pass.
Never 'clean up' leftover request .json files manually during a live run; they are retained deliberately.

When it happens

Trigger: The sandbox file channel becomes unresponsive (frozen/paused sandbox, dead transport, host I/O failure) while callback requests sit in the requests directory — reads/list succeeded enough to enumerate the request, but every 503 write times out or errors.

Common situations: Sandbox container paused by the host; gVisor/FUSE-backed channel stalling; long GC pauses; storage outage during a run with in-flight callbacks.

Related errors

AI-assisted analysis of paperclipai/paperclip@a7e689b3c3 (2026-08-18). Data as JSON: /api/errors/b7940229f1ab4da0. Report an issue: GitHub.