{"record":{"id":"d29cf094e0b81519","repo":"paperclipai/paperclip","slug":"timed-out-acquiring-sandbox-callback-bridge-respon","errorCode":null,"errorMessage":"Timed out acquiring sandbox callback bridge response lock.","messagePattern":"Timed out acquiring sandbox callback bridge response lock\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/adapter-utils/src/sandbox-callback-bridge.ts","lineNumber":456,"sourceCode":"          } catch {\n            // pid file missing or unreadable — treat as stale lock\n          }\n          let holderAlive = false;\n          if (holderPid !== null) {\n            try {\n              process.kill(holderPid, 0);\n              holderAlive = true;\n            } catch {\n              holderAlive = false;\n            }\n          }\n          if (!holderAlive) {\n            await fs.rm(lockDir, { recursive: true, force: true }).catch(() => undefined);\n            continue;\n          }\n          attempts += 1;\n          if (attempts >= 600) {\n            throw new Error(\"Timed out acquiring sandbox callback bridge response lock.\");\n          }\n          await new Promise((resolve) => setTimeout(resolve, 50));\n        }\n      }\n\n      try {\n        if (options.requestPath) {\n          const requestExists = await pathExists(options.requestPath);\n          if (!requestExists) {\n            return { wrote: false };\n          }\n        }\n        const responseExists = await pathExists(responsePath);\n        if (responseExists) {\n          return { wrote: false };\n        }\n        await fs.writeFile(tempPath, body, \"utf8\");\n        await fs.rename(tempPath, responsePath);","sourceCodeStart":438,"sourceCodeEnd":474,"githubUrl":"https://github.com/paperclipai/paperclip/blob/67001ec6eb96ae601aa27bc91d9b2415d665334a/packages/adapter-utils/src/sandbox-callback-bridge.ts#L438-L474","documentation":"Thrown by the Node-side writeResponseFile (createLocalSandboxCallbackBridgeQueueClient) after 600 lock-acquisition attempts spaced 50ms apart (~30s total). The lock is a mkdir-based mutex with a PID-liveness check: a stale lock whose holder process is dead is reclaimed, but a live holder blocks the writer until the attempt budget is exhausted. The retry budget is hardcoded and intentionally finite to surface deadlocks rather than hang silently.","triggerScenarios":"Two bridge writers (or a writer and a wedged holder) racing for the same responsePath, with the holder still alive (process.kill(holderPid, 0) succeeds) for the full 30s window. A long-running handler that holds the lock dir without releasing it, or a PID-reuse scenario where the holder PID was reassigned to a different live process, will both exhaust the budget.","commonSituations":"Concurrent bridge handlers writing the same response file (misconfigured deduplication), a previous writer that crashed mid-section but left a stale PID file pointing at a still-running process, or system load so high that the holder does not get CPU time to release the lock within 30s. PID reuse on systems that recycle PIDs aggressively can also pin the lock to an unrelated live process.","solutions":["Inspect the lockDir at ${responsePath}.paperclip-write.lock/pid — if the recorded PID is unrelated to a real bridge writer, remove the lockDir manually to unblock.","Reduce concurrent writers: serialize handlers per responsePath so only one writer contends for the lock.","Investigate the prior holder: a handler stuck in fs.writeFile/fs.rename is the usual culprit; check Node event-loop blocking or filesystem latency.","If this recurs, instrument the retry loop to log holderPid and holderAlive transitions to distinguish stale-lock recovery failure from genuine contention."],"exampleFix":"// before: two concurrent handlers writing the same response\nawait Promise.all([\n  bridge.writeResponseFile(respPath, bodyA, { requestPath }),\n  bridge.writeResponseFile(respPath, bodyB, { requestPath }),\n]);\n\n// after: serialize per response path\nconst writer = createResponseSerializer(respPath);\nawait writer.enqueue(() => bridge.writeResponseFile(respPath, bodyA, { requestPath }));\nawait writer.enqueue(() => bridge.writeResponseFile(respPath, bodyB, { requestPath }));","handlingStrategy":"try-catch","validationCode":"async function preflightLockDir(responsePath: string): Promise<void> {\n  const lockDir = `${responsePath}.paperclip-write.lock`;\n  try {\n    await fs.mkdir(lockDir);\n    await fs.rmdir(lockDir);\n  } catch (error) {\n    const code = (error as NodeJS.ErrnoException)?.code;\n    if (code === \"EEXIST\") {\n      const pidRaw = await fs.readFile(`${lockDir}/pid`, \"utf8\").catch(() => \"\");\n      const pid = Number.parseInt(pidRaw.trim(), 10);\n      if (Number.isFinite(pid) && pid > 0) {\n        try { process.kill(pid, 0); throw new Error(`Stale-but-live lock holder ${pid} for ${responsePath}`); } catch { /* dead holder — will be reclaimed */ }\n    }\n    } else if (code !== \"ENOENT\") {\n      throw error;\n    }\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  await bridge.writeResponseFile(responsePath, body, { requestPath });\n} catch (error) {\n  if (error instanceof Error && error.message === \"Timed out acquiring sandbox callback bridge response lock.\") {\n    // Best-effort recovery: clear the lock dir if the recorded holder is no longer alive, then retry once.\n    const lockDir = `${responsePath}.paperclip-write.lock`;\n    await fs.rm(lockDir, { recursive: true, force: true }).catch(() => undefined);\n    await bridge.writeResponseFile(responsePath, body, { requestPath });\n  } else {\n    throw error;\n  }\n}","preventionTips":["Serialize writers per responsePath so only one handler contends for the lock.","Investigate handlers stuck in fs.writeFile/fs.rename — they are the usual lock-hoarding culprit.","Instrument the retry loop to log holderPid/holderAlive transitions when this recurs.","Watch for PID reuse on systems that recycle PIDs aggressively — a stale lock can appear held by an unrelated live process."],"tags":["bridge","concurrency","locking","timeout","callback-bridge"],"backgroundTag":null,"analyzedSha":"67001ec6eb96ae601aa27bc91d9b2415d665334a","analyzedAt":"2026-08-12T12:05:45.408Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}