{"record":{"id":"f31934804f43e2d2","repo":"different-ai/openwork","slug":"could-not-capture-screenshot-after-action","errorCode":null,"errorMessage":"Could not capture screenshot after action.","messagePattern":"Could not capture screenshot after action\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/handsfree/src/cua-runner.mjs","lineNumber":73,"sourceCode":"\n    if (!computerCall) return { ok: true, messages, turns: turn + 1 };\n\n    for (const action of computerCall.actions || (computerCall.action ? [computerCall.action] : [])) {\n      if (signal?.aborted) return { ok: true, messages, turns: turn + 1, aborted: true };\n      if (action.type === \"screenshot\") continue;\n      onProgress?.({ kind: \"action\", ...summarizeAction(action) });\n      const actionResult = await executeCuaAction(callTool, action);\n      const actionPayload = parseToolText(actionResult);\n      if (actionPayload?.ok === false) {\n        if (actionPayload.requiredNextAction === \"snapshot\") break;\n        throw new Error(actionPayload.error || `Computer action failed: ${action.type}`);\n      }\n      await delay(150);\n    }\n\n    const screenshot = await callTool(\"cua_screenshot\", {});\n    const image = extractImage(screenshot);\n    if (!image) throw new Error(\"Could not capture screenshot after action.\");\n\n    items.push({\n      type: \"computer_call_output\",\n      call_id: computerCall.call_id,\n      acknowledged_safety_checks: computerCall.pending_safety_checks || [],\n      output: { type: \"input_image\", image_url: `data:image/png;base64,${image}` },\n    });\n  }\n\n  return { ok: true, messages, turns: maxTurns, truncated: true };\n}\n\nexport async function executeCuaAction(callTool, action) {\n  switch (action.type) {\n    case \"click\":\n      return callTool(\"cua_click\", { x: action.x, y: action.y, button: action.button || \"left\", ...(action.keys?.length ? { keys: action.keys } : {}) });\n    case \"double_click\":\n      return callTool(\"cua_double_click\", { x: action.x, y: action.y });","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/different-ai/openwork/blob/2b7df46e8ae1517d64c896c7793d2d52ec845669/packages/handsfree/src/cua-runner.mjs#L55-L91","documentation":"The CUA protocol requires a screenshot after every executed action to feed back to the model. The loop calls callTool(\"cua_screenshot\", {}) and runs extractImage on the result; if no base64 image can be extracted, it throws because the next computer_call_output cannot be constructed without image data.","triggerScenarios":"callTool(\"cua_screenshot\") returned a result where extractImage finds no data URL/base64 string — the screenshot tool errored, returned a text-only error payload, or returned a payload in a format extractImage doesn't recognize.","commonSituations":"Headless/windowless browser where screenshot capture fails; page crashed (renderer gone) so capture returns nothing; MCP tool returning wrapped content whose image field isn't parsed by extractImage; screenshot hitting an interstitial (crash dialog, print preview).","solutions":["Log the raw cua_screenshot tool result and check whether it contains an image at all vs. an error message.","Verify the browser session/page is alive (crashed tabs return no image); navigate or relaunch the page.","Extend extractImage to handle the actual content shape your MCP client returns (e.g. nested content arrays, image_url fields).","Add a retry of the screenshot call once before failing — transient capture failures are common."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  await runCuaLoop(opts);\n} catch (e) {\n  if (e.message === \"Could not capture screenshot after action.\") {\n    const shot = await callTool(\"cua_screenshot\", {});\n    if (!extractImage(shot)) await restartBrowserSession(); // page likely crashed\n  } else throw e;\n}","preventionTips":["Verify the screenshot MCP tool returns image content your parser handles","Monitor for crashed renderer targets in headless sessions","Add a single screenshot retry before failing the whole loop"],"tags":["screenshot","computer-use","tool-response","parsing"],"backgroundTag":"screenshot-capture-failed","analyzedSha":"2b7df46e8ae1517d64c896c7793d2d52ec845669","analyzedAt":"2026-09-01T07:59:23.713Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}