{"record":{"id":"aac92d31524e287c","repo":"paperclipai/paperclip","slug":"createos-setup-failed-and-cleanup-is-unconfirmed-for-sandbox","errorCode":null,"errorMessage":"CreateOS setup failed and cleanup is unconfirmed for sandbox ${sandbox.id}.","messagePattern":"CreateOS setup failed and cleanup is unconfirmed for sandbox (.+?)\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"packages/plugins/sandbox-providers/createos/src/plugin.ts","lineNumber":49,"sourceCode":"    const data = await client.json(`/sandboxes/${sandbox.id}/exec`, \"POST\", {\n      cmd: \"/bin/bash\", args: [\"-lc\", `mkdir -p -- ${shellQuote(CWD)}`],\n    }, signal);\n    if (object(data.result).exit_code !== 0) throw new Error(\"CreateOS workspace preparation failed; the image must provide Bash.\");\n    const marker = randomUUID();\n    await client.upload(sandbox.id, MARKER, marker, signal);\n    return {\n      providerLeaseId: sandbox.id,\n      metadata: {\n        provider: \"createos\", apiUrl: config.apiUrl,\n        companyId: params.companyId, environmentId: params.environmentId,\n        remoteCwd: CWD, shellCommand: \"bash\", marker,\n        shape: config.shape, rootfs: config.rootfs, region: config.region,\n        reuseLease: config.reuseLease,\n      },\n    };\n  } catch (error) {\n    try { await client.destroySandbox(sandbox.id); }\n    catch { throw new Error(`CreateOS setup failed and cleanup is unconfirmed for sandbox ${sandbox.id}.`); }\n    throw error;\n  }\n}\n\n// Each worker owns its own transient lifecycle state. The host owns durable leases.\nexport function createPlugin() {\n  let ctx: PluginContext | null = null;\n  let shuttingDown = false;\n  type Active = { controller: AbortController; done: Promise<void> };\n  const active = new Map<string, Set<Active>>();\n  const closing = new Set<string>();\n  const unconfirmedCleanup = new Set<string>();\n\n  function key(params: PluginEnvironmentDriverBaseParams, id: string): string {\n    const config = parseConfig(params.config);\n    const account = createHash(\"sha256\").update(resolveApiKey(config)).digest(\"hex\");\n    return JSON.stringify([params.companyId, params.environmentId, config.apiUrl, account, id]);\n  }","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/packages/plugins/sandbox-providers/createos/src/plugin.ts#L31-L67","documentation":"During lease acquisition, CreateOS plugin provisions a sandbox, transitions it to running, prepares the workspace, and uploads a lease marker. If any of those steps fail, acquire tries to destroy the partially-created sandbox so no orphan is left behind. This error is thrown when that compensating destroySandbox call itself fails, meaning the setup failed AND the provider may still be billing for a live sandbox the host no longer references.","triggerScenarios":"acquire() fails partway (transition to 'running' fails, workspace mkdir exec fails, marker upload fails) and then the compensating client.destroySandbox(sandbox.id) also throws — e.g. the CreateOS API went down between the two calls, the API key lacks delete permission, or a transient network partition interrupts the destroy request.","commonSituations":"CreateOS API outage or flaky network during environment probe/acquire; sandbox stuck in a transitional state that cannot be deleted; API key created with create but not delete scope; rate limiting that lets create succeed but rejects the immediate destroy call.","solutions":["Check the CreateOS dashboard/API for a sandbox with the ID in the message and destroy it manually to avoid orphaned billing.","Retry acquire once connectivity is confirmed — the failed sandbox was never returned as a lease, so a fresh acquire creates a new sandbox.","Verify the API key has both sandbox-create and sandbox-delete permissions for the configured apiUrl.","If destroys consistently fail, probe the environment (onEnvironmentProbe) to check API health before acquiring leases."],"exampleFix":"// before: destroy failure hides the original error entirely\ncatch { throw new Error(`CreateOS setup failed and cleanup is unconfirmed for sandbox ${sandbox.id}.`); }\n// after: run a probe first and surface the original cause when destroying\nconst probe = await fetch(`${config.apiUrl}/sandboxes/${sandbox.id}`, { headers: { authorization: `Bearer ${resolveApiKey(config)}` } });\nif (probe.ok) console.warn(`sandbox ${sandbox.id} may still be live; destroy it manually`);\nthrow error; // rethrow the original setup failure for better diagnosis","handlingStrategy":"try-catch","validationCode":"// check API reachability and credentials before acquiring\nconst res = await fetch(`${config.apiUrl}/health`, { headers: { authorization: `Bearer ${apiKey}` } });\nif (!res.ok) throw new Error(`CreateOS API unhealthy (${res.status}); skipping acquire`);","typeGuard":null,"tryCatchPattern":"try {\n  lease = await acquireLease(params);\n} catch (e) {\n  if (e.message.includes(\"cleanup is unconfirmed\")) {\n    const sandboxId = e.message.match(/sandbox ([^\\s.]+)/)?.[1];\n    logger.error(`orphaned sandbox possible: ${sandboxId}; reconcile via CreateOS API`);\n  }\n  throw e;\n}","preventionTips":["Probe the environment before batch-acquiring leases to confirm API health.","Use an API key with both create and delete scopes.","Periodically reconcile live sandboxes against active leases to catch orphans.","Alert on any error containing 'cleanup is unconfirmed' — it implies possible billing for unused sandboxes."],"tags":["sandbox","cleanup","orphaned-resource","network"],"backgroundTag":"cleanup-unconfirmed","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-09-18T08:03:59.046Z","contentChangedAt":"2026-09-18T08:03:59.046Z","schemaVersion":2},"datasetVersion":"2026-09-22T11:17:16.035Z"}