{"record":{"id":"3d0f0e59465c27c5","repo":"paperclipai/paperclip","slug":"createos-command-probe-failed","errorCode":null,"errorMessage":"CreateOS command probe failed.","messagePattern":"CreateOS command probe failed\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/plugins/sandbox-providers/createos/src/plugin.ts","lineNumber":142,"sourceCode":"      closing.delete(scope);\n    }\n  }\n\n  return definePlugin({\n    async setup(context) { ctx = context; ctx.logger.info(\"CreateOS sandbox provider ready\"); },\n    async onHealth() { return { status: \"ok\", message: \"CreateOS provider loaded; probe an environment to check connectivity.\" }; },\n    async onEnvironmentValidateConfig(params) {\n      try { return { ok: true, normalizedConfig: { ...parseConfig(params.config) } }; }\n      catch (error) { return { ok: false, errors: [error instanceof Error ? error.message : \"Invalid CreateOS configuration.\"] }; }\n    },\n    async onEnvironmentProbe(params) {\n      let lease: PluginEnvironmentLease | null = null;\n      try {\n        lease = await acquire({ ...params, runId: \"probe\" });\n        const result = await execute(new CreateosClient(parseConfig(params.config)), {\n          ...params, lease, command: \"/bin/echo\", args: [\"paperclip-createos-ready\"], cwd: CWD,\n        }, AbortSignal.timeout(parseConfig(params.config).timeoutMs));\n        if (result.timedOut || result.exitCode !== 0 || !result.stdout.includes(\"paperclip-createos-ready\")) throw new Error(\"CreateOS command probe failed.\");\n        return { ok: true, summary: \"CreateOS sandbox creation and command execution succeeded.\" };\n      } catch (error) {\n        return { ok: false, summary: error instanceof Error ? error.message : \"CreateOS probe failed.\" };\n      } finally {\n        // Never leave a reusable probe sandbox behind or hide deletion failure.\n        if (lease?.providerLeaseId) await new CreateosClient(parseConfig(params.config)).destroySandbox(lease.providerLeaseId);\n      }\n    },\n    onEnvironmentAcquireLease: acquire,\n    async onEnvironmentResumeLease(params) {\n      if (!metadataMatches(params, params.leaseMetadata)) throw new Error(\"CreateOS lease does not belong to this environment.\");\n      const scope = key(params, params.providerLeaseId);\n      if (closing.has(scope) || unconfirmedCleanup.has(scope)) throw new Error(\"CreateOS lease cleanup must finish before resume.\");\n      const marker = params.leaseMetadata?.marker;\n      if (typeof marker !== \"string\" || !/^[0-9a-f-]{36}$/.test(marker)) return { providerLeaseId: null, metadata: { expired: true } };\n      const config = parseConfig(params.config);\n      if (params.leaseMetadata?.shape !== config.shape || params.leaseMetadata.rootfs !== config.rootfs || params.leaseMetadata.region !== config.region) {\n        return { providerLeaseId: null, metadata: { expired: true } };","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/packages/plugins/sandbox-providers/createos/src/plugin.ts#L124-L160","documentation":"Thrown by onEnvironmentProbe when the plugin's readiness probe fails. The probe acquires a temporary lease, runs '/bin/echo paperclip-createos-ready' in the sandbox, and requires the command to exit 0, not time out, and echo the exact marker on stdout. Any deviation (sandbox never became ready, command timed out, wrong output) raises this error; the underlying cause is discarded in favor of this fixed message.","triggerScenarios":"Calling onEnvironmentProbe (environment health check / plugin validation) when: sandbox creation succeeds but command execution times out (result.timedOut), the echo process exits non-zero, or stdout does not contain 'paperclip-createos-ready' (e.g. the sandbox's shell/init mangled the output or a different process captured stdout).","commonSituations":"CreateOS service degraded or overloaded so commands queue past timeoutMs; sandbox image without a working /bin/echo or broken shell init; overly small timeoutMs in config; region/shape misprovisioning causing slow cold starts.","solutions":["Re-run the probe; transient CreateOS slowness is the most common cause (command timeout rather than genuine failure).","Increase config.timeoutMs so the probe command has enough budget on cold starts.","Check CreateOS API health/logs for the sandbox — a non-zero exit or missing marker usually indicates an image or init problem.","If probes consistently fail, verify the sandbox image supports /bin/echo and that shell init does not swallow stdout."],"exampleFix":"// before: default timeout too small for cold starts\nconst config = { shape: 'small', region: 'us-1', timeoutMs: 5000 };\n// after\nconst config = { shape: 'small', region: 'us-1', timeoutMs: 30000 };","handlingStrategy":"retry","validationCode":"// Pre-flight config sanity before probing\nfunction probeConfigOk(config: { timeoutMs: number }): boolean {\n  return Number.isFinite(config.timeoutMs) && config.timeoutMs >= 10000;\n}","typeGuard":"function isProbeFailure(e: unknown): boolean {\n  return e instanceof Error && e.message === 'CreateOS command probe failed.';\n}","tryCatchPattern":"const probe = await plugin.onEnvironmentProbe(params); // returns {ok,summary}, rarely throws\nif (!probe.ok) {\n  await sleep(2000);\n  const retry = await plugin.onEnvironmentProbe(params);\n  if (!retry.ok) throw new Error(`CreateOS probe failed twice: ${retry.summary}`);\n}","preventionTips":["Set timeoutMs generously (>=30s) to absorb sandbox cold starts","Retry the probe once or twice before declaring the provider unhealthy","Verify the sandbox image supports /bin/echo and preserves stdout","Monitor CreateOS service health; cluster-wide probe failures indicate provider outage, not config"],"tags":["sandbox","health-check","probe","timeout"],"backgroundTag":"http-request-failed","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-09-18T08:03:59.046Z","contentChangedAt":"2026-09-18T08:03:59.046Z","schemaVersion":2},"datasetVersion":"2026-09-22T11:17:16.035Z"}