Hmbown/CodeWhale · error · ExecError

open_application first to choose which application receives…

Error message

open_application first to choose which application receives input

What it means

assertOwnsPoint ensures a global pointer gesture only lands inside a window owned by the bound application. Before checking ownership it requires that an application is bound at all (state.inputApp); without open_application there is no owner to compare against, so any global gesture is refused.

Solutions

Call open_application with the target application first, then re-observe and perform the pointer action.
If you only have a screenshot coordinate, still open/observe the owning app so ownership can be verified.
Use an accessibility element target from observe() as an alternative that establishes the app context.
Persist the binding across your workflow instead of resetting state between actions.

Example fix

// before
await click({ x: 400, y: 300 }); // no bound app

// after
await open_application("com.example.app");
await observe();
await click({ x: 400, y: 300 });

Defensive patterns

Strategy: validation

Validate before calling

if (!sessionState.inputApp) {
  await open_application(targetApp);
  await observe();
}

Try / catch

try {
  await click({ x, y });
} catch (e) {
  if (String(e.message).includes('open_application first')) {
    await open_application(targetApp);
    await observe();
    await click({ x, y });
  } else throw e;
}

Prevention

Make open_application + observe the mandatory first two steps of every pointer workflow.
Track session binding state and refuse pointer actions when it is null.
Do not reset backend state between actions in a multi-step gesture workflow.
Prefer element targets, which naturally carry their owning app context.

When it happens

Trigger: Calling a global pointer action (click at coordinates, drag, scroll at a point) that routes through assertOwnsPoint before open_application has ever been called in the session, or after the binding was cleared/reset.

Common situations: New sessions where scripts start with coordinate clicks; tool state reset between agent turns losing the binding; forgetting that element-less pointer actions require a bound app.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/afcf61a55ede0e64. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/darwin.mjs:372

  //   2. otherwise refuse in background mode. Explicit foreground control
  //      permits a global gesture only when the bound application owns the
  //      window under the point. Restoring the cursor is not isolation.
  // Every receipt says which of the two happened.
  function mouseName(button) { return { left: "left", right: "right", middle: "middle" }[button] ?? "left"; }

  function assertInScreen(x, y) {
    if (!Number.isFinite(x) || !Number.isFinite(y)) throw new ExecError("coordinates must be finite numbers");
  }

  function buttonCode(button) { return button === "middle" ? 2 : button === "right" ? 1 : 0; }

  function requireSharedPointer() {
    if (!state.foregroundInput) throw Object.assign(new ExecError("This action needs the shared macOS pointer and was not sent in background mode. Use an accessibility action or a separate computer; foreground control requires exclusive desktop use authorized by the user."), { code: "shared_pointer_required" });
  }

  /** Refuse a global gesture whose landing point belongs to another application. */
  async function assertOwnsPoint(x, y) {
    if (!state.inputApp) throw new ExecError("open_application first to choose which application receives input");
    const w = await native("window_at_point", { x, y });
    if (!w?.found) throw new ExecError(`no window at (${x}, ${y}) — take a fresh screenshot and choose a point inside the target window`);
    if (w.owner_pid !== state.inputApp.pid) {
      throw new ExecError(`(${x}, ${y}) is covered by a window owned by ${w.owner_name || "another application"} (pid ${w.owner_pid}) — use an accessibility element target or a separate computer; no pointer input was sent`);
    }
    return w;
  }

  /** What a global gesture cost the user: their cursor, and briefly their foreground. */
  function pointerCost(r) {
    return {
      pointer_moved: true,
      pointer_restored: !!r?.restored,
      foreground_taken: !!r?.foreground_taken,
      ...(r?.foreground_before ? { foreground_before: r.foreground_before } : {}),
      ...(r?.foreground_after ? { foreground_after: r.foreground_after } : {}),
      ...(Number.isFinite(r?.yield_ms) && r.yield_ms > 0 ? { yield_ms: r.yield_ms } : {}),
    };

View on GitHub (pinned to 73e0f67d83)