{"record":{"id":"afcf61a55ede0e64","repo":"Hmbown/CodeWhale","slug":"open-application-first-to-choose-which-application-receives","errorCode":null,"errorMessage":"open_application first to choose which application receives input","messagePattern":"open_application first to choose which application receives input","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/darwin.mjs","lineNumber":372,"sourceCode":"  //   2. otherwise refuse in background mode. Explicit foreground control\n  //      permits a global gesture only when the bound application owns the\n  //      window under the point. Restoring the cursor is not isolation.\n  // Every receipt says which of the two happened.\n  function mouseName(button) { return { left: \"left\", right: \"right\", middle: \"middle\" }[button] ?? \"left\"; }\n\n  function assertInScreen(x, y) {\n    if (!Number.isFinite(x) || !Number.isFinite(y)) throw new ExecError(\"coordinates must be finite numbers\");\n  }\n\n  function buttonCode(button) { return button === \"middle\" ? 2 : button === \"right\" ? 1 : 0; }\n\n  function requireSharedPointer() {\n    if (!state.foregroundInput) throw Object.assign(new ExecError(\"This action needs the shared macOS pointer and was not sent in background mode. Use an accessibility action or a separate computer; foreground control requires exclusive desktop use authorized by the user.\"), { code: \"shared_pointer_required\" });\n  }\n\n  /** Refuse a global gesture whose landing point belongs to another application. */\n  async function assertOwnsPoint(x, y) {\n    if (!state.inputApp) throw new ExecError(\"open_application first to choose which application receives input\");\n    const w = await native(\"window_at_point\", { x, y });\n    if (!w?.found) throw new ExecError(`no window at (${x}, ${y}) — take a fresh screenshot and choose a point inside the target window`);\n    if (w.owner_pid !== state.inputApp.pid) {\n      throw new ExecError(`(${x}, ${y}) is covered by a window owned by ${w.owner_name || \"another application\"} (pid ${w.owner_pid}) — use an accessibility element target or a separate computer; no pointer input was sent`);\n    }\n    return w;\n  }\n\n  /** What a global gesture cost the user: their cursor, and briefly their foreground. */\n  function pointerCost(r) {\n    return {\n      pointer_moved: true,\n      pointer_restored: !!r?.restored,\n      foreground_taken: !!r?.foreground_taken,\n      ...(r?.foreground_before ? { foreground_before: r.foreground_before } : {}),\n      ...(r?.foreground_after ? { foreground_after: r.foreground_after } : {}),\n      ...(Number.isFinite(r?.yield_ms) && r.yield_ms > 0 ? { yield_ms: r.yield_ms } : {}),\n    };","sourceCodeStart":354,"sourceCodeEnd":390,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/darwin.mjs#L354-L390","documentation":"assertOwnsPoint ensures a global pointer gesture only lands inside a window owned by the bound application. Before checking ownership it requires that an application is bound at all (state.inputApp); without open_application there is no owner to compare against, so any global gesture is refused.","triggerScenarios":"Calling a global pointer action (click at coordinates, drag, scroll at a point) that routes through assertOwnsPoint before open_application has ever been called in the session, or after the binding was cleared/reset.","commonSituations":"New sessions where scripts start with coordinate clicks; tool state reset between agent turns losing the binding; forgetting that element-less pointer actions require a bound app.","solutions":["Call open_application with the target application first, then re-observe and perform the pointer action.","If you only have a screenshot coordinate, still open/observe the owning app so ownership can be verified.","Use an accessibility element target from observe() as an alternative that establishes the app context.","Persist the binding across your workflow instead of resetting state between actions."],"exampleFix":"// before\nawait click({ x: 400, y: 300 }); // no bound app\n\n// after\nawait open_application(\"com.example.app\");\nawait observe();\nawait click({ x: 400, y: 300 });","handlingStrategy":"validation","validationCode":"if (!sessionState.inputApp) {\n  await open_application(targetApp);\n  await observe();\n}","typeGuard":null,"tryCatchPattern":"try {\n  await click({ x, y });\n} catch (e) {\n  if (String(e.message).includes('open_application first')) {\n    await open_application(targetApp);\n    await observe();\n    await click({ x, y });\n  } else throw e;\n}","preventionTips":["Make open_application + observe the mandatory first two steps of every pointer workflow.","Track session binding state and refuse pointer actions when it is null.","Do not reset backend state between actions in a multi-step gesture workflow.","Prefer element targets, which naturally carry their owning app context."],"tags":["macos","state","pointer","prerequisite"],"backgroundTag":"missing-required-argument","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T16:17:23.217Z"}