paperclipai/paperclip · error

CreateOS process output connection failed.

Error message

CreateOS process output connection failed.

What it means

The plugin failed to (re)establish the /processes/:id/connect connection more than 3 times (only 5xx/429 errors are retried, with 250 ms backoff). This means the output stream endpoint is persistently unreachable or erroring, so process output cannot be read.

Solutions

  1. Retry the whole execute call after verifying the CreateOS service is healthy (/health or equivalent).
  2. Check rate limiting: 429 responses exhaust the 3-retry budget quickly; reduce concurrency or add backoff at the caller level.
  3. Verify the configured CreateOS base URL, DNS, and network reachability to the sandbox host.
  4. Inspect server logs for 5xx on the /connect route during the failure window.
  5. Note the process tree is terminated by cleanup on this error; a rerun is safe.

Example fix

// before
await client.request(`${base}/${processId}/connect?after=${cursor}`); // fails, no budget left
// after (caller-side)
for (let i = 0; i < 3; i++) { try { return await execute(client, params, signal); } catch (e) { if (!/connection failed/.test(e.message)) throw e; await sleep(2 ** i * 250); } }
Defensive patterns

Strategy: retry

Validate before calling

// Pre-flight reachability check
const probe = await fetch(`${baseUrl}/sandboxes/${leaseId}/processes`, { method: "HEAD" }).catch(() => null);
if (!probe) throw new Error("CreateOS host unreachable before execute");

Try / catch

try { await execute(client, params, signal); }
catch (e) {
  if (e instanceof Error && e.message === "CreateOS process output connection failed.") {
    await sleep(1000); // service-level backoff, then re-run execute (cleanup already killed the tree)
  } else throw e;
}

Prevention

When it happens

Trigger: Four consecutive connect attempts fail with 5xx or 429 (or a network error classified as retryable), e.g. the CreateOS host is down, DNS fails, TLS fails, or the server is rate-limiting /connect beyond the retry budget.

Common situations: CreateOS service restart or outage during a command; aggressive rate limiting after many concurrent executes; firewall/DNS misconfiguration in the sandbox base URL; load balancer returning 503s for the connect route.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

AI-assisted analysis of paperclipai/paperclip@3f1d897a7c (2026-09-18). Data as JSON: /api/errors/8666f3fa6e9d2d13. Report an issue: GitHub.

Appendix: source

Thrown at packages/plugins/sandbox-providers/createos/src/execute.ts:145

    processId = identifier(created.process_id);
    // No interactive stdin for ordinary commands; any supplied input comes
    // from the staged file, avoiding a write-vs-fast-exit race.
    try { await client.json(`${base}/${processId}/stdin/close`, "POST", undefined, signal); }
    catch (error) {
      if (!(error instanceof CreateosApiError && error.status === 409)) throw error;
    }
    let reconnects = 0;
    for (;;) {
      signal.throwIfAborted();
      let response: Response;
      try {
        response = await client.request(`${base}/${processId}/connect?after=${cursor}`, { signal });
      } catch (error) {
        if (error instanceof CreateosApiError && error.status === 410) {
          throw new Error("CreateOS process output was evicted before it could be read.");
        }
        if (signal.aborted || (error instanceof CreateosApiError && error.status < 500 && error.status !== 429)) throw error;
        if (++reconnects > 3) throw new Error("CreateOS process output connection failed.");
        await delay(250, undefined, { signal });
        continue;
      }
      try {
        for await (const event of events(response)) {
          if (event.type === "heartbeat") continue;
          if (event.type === "data") {
            const seq = event.seq;
            if (typeof seq !== "number" || !Number.isSafeInteger(seq) || seq < 1) throw new Error("CreateOS process sequence is invalid.");
            if (seq <= cursor) continue;
            if (seq !== cursor + 1) throw new Error("CreateOS process output has a sequence gap.");
            if ((event.stream !== "stdout" && event.stream !== "stderr") ||
                typeof event.data_base64 !== "string" ||
                !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(event.data_base64)) {
              throw new Error("CreateOS process output is invalid.");
            }
            output.write(event.stream, Buffer.from(event.data_base64, "base64"));
            cursor = seq;

View on GitHub (pinned to 3f1d897a7c)