different-ai/openwork · error · Error

Could not capture screenshot after action.

Error message

Could not capture screenshot after action.

What it means

The CUA protocol requires a screenshot after every executed action to feed back to the model. The loop calls callTool("cua_screenshot", {}) and runs extractImage on the result; if no base64 image can be extracted, it throws because the next computer_call_output cannot be constructed without image data.

Source

Thrown at packages/handsfree/src/cua-runner.mjs:73

    if (!computerCall) return { ok: true, messages, turns: turn + 1 };

    for (const action of computerCall.actions || (computerCall.action ? [computerCall.action] : [])) {
      if (signal?.aborted) return { ok: true, messages, turns: turn + 1, aborted: true };
      if (action.type === "screenshot") continue;
      onProgress?.({ kind: "action", ...summarizeAction(action) });
      const actionResult = await executeCuaAction(callTool, action);
      const actionPayload = parseToolText(actionResult);
      if (actionPayload?.ok === false) {
        if (actionPayload.requiredNextAction === "snapshot") break;
        throw new Error(actionPayload.error || `Computer action failed: ${action.type}`);
      }
      await delay(150);
    }

    const screenshot = await callTool("cua_screenshot", {});
    const image = extractImage(screenshot);
    if (!image) throw new Error("Could not capture screenshot after action.");

    items.push({
      type: "computer_call_output",
      call_id: computerCall.call_id,
      acknowledged_safety_checks: computerCall.pending_safety_checks || [],
      output: { type: "input_image", image_url: `data:image/png;base64,${image}` },
    });
  }

  return { ok: true, messages, turns: maxTurns, truncated: true };
}

export async function executeCuaAction(callTool, action) {
  switch (action.type) {
    case "click":
      return callTool("cua_click", { x: action.x, y: action.y, button: action.button || "left", ...(action.keys?.length ? { keys: action.keys } : {}) });
    case "double_click":
      return callTool("cua_double_click", { x: action.x, y: action.y });

View on GitHub (pinned to 2b7df46e8a)

Solutions

1. Log the raw cua_screenshot tool result and check whether it contains an image at all vs. an error message.
2. Verify the browser session/page is alive (crashed tabs return no image); navigate or relaunch the page.
3. Extend extractImage to handle the actual content shape your MCP client returns (e.g. nested content arrays, image_url fields).
4. Add a retry of the screenshot call once before failing — transient capture failures are common.

try {
  await runCuaLoop(opts);
} catch (e) {
  if (e.message === "Could not capture screenshot after action.") {
    const shot = await callTool("cua_screenshot", {});
    if (!extractImage(shot)) await restartBrowserSession(); // page likely crashed
  } else throw e;
}

When it happens

Trigger: callTool("cua_screenshot") returned a result where extractImage finds no data URL/base64 string — the screenshot tool errored, returned a text-only error payload, or returned a payload in a format extractImage doesn't recognize.

Common situations: Headless/windowless browser where screenshot capture fails; page crashed (renderer gone) so capture returns nothing; MCP tool returning wrapped content whose image field isn't parsed by extractImage; screenshot hitting an interstitial (crash dialog, print preview).

Related errors

• latest-mac.yml is missing artifact path/url.
• latest-mac.yml is missing sha512.
• invalid_json
• ZIP data for ${entry.name} is out of bounds.
• ZIP uncompressed size mismatch for ${entry.name}.

AI-assisted analysis of different-ai/openwork@2b7df46e8a (2026-09-01). Data as JSON: /api/errors/f31934804f43e2d2. Report an issue: GitHub.