Hmbown/CodeWhale · error · ExecError

no window at ( , ) — take a fresh screenshot and choose a…

Error message

no window at (${x}, ${y}) — take a fresh screenshot and choose a point inside the target window

What it means

After confirming an app is bound, assertOwnsPoint asks the native helper for the window at the gesture point (window_at_point). If no window is found at (x, y), the point is empty desktop/menu-bar-free space or outside all windows, and the gesture is refused with guidance to re-screenshot. This guards against clicking into nothing and mistargeting.

Solutions

Take a fresh screenshot and pick new coordinates inside the target window, then retry.
Re-observe/open_application to refresh window geometry if the window moved or resized.
Convert image coordinates to screen points with the correct scale factor (Retina 2x) and display offset.
Prefer accessibility element targets, which track window layout automatically instead of fixed points.

Example fix

// before
await click({ x: 400, y: 300 }); // stale point, window moved

// after
const shot = await screenshot(); // fresh geometry
const pt = toScreenPoint(shot, 400, 300); // scale + offset applied
await click(pt);

Defensive patterns

Strategy: retry

Validate before calling

const w = await native('window_at_point', { x, y });
if (!w?.found) throw new Error('point is outside every window — take a fresh screenshot first');

Try / catch

try {
  await click({ x, y });
} catch (e) {
  if (String(e.message).startsWith('no window at')) {
    const shot = await screenshot();          // fresh geometry
    const pt = toScreenPoint(shot, x, y);     // re-apply scale/offset
    await click(pt);
  } else throw e;
}

Prevention

Always derive coordinates from the most recent screenshot, never from cached ones.
Apply Retina scale factors and multi-display offsets when converting image to screen coordinates.
Re-observe if the window may have moved, resized, or been minimized.
Prefer accessibility element targets, which follow layout changes automatically.

When it happens

Trigger: Pointer action at coordinates where window_at_point returns found=false — stale coordinates from an old screenshot after the window moved/closed/resized, coordinates in screen-space outside any window, multi-display coordinate confusion, or a minimized window.

Common situations: Windows moved by the user between screenshot and click; screenshot taken before a layout change; coordinates from a scaled/retina image not converted to screen points; clicking on desktop background; wrong display's coordinate space on multi-monitor setups.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/bd4c7bcf43cf1982. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/darwin.mjs:374

  //      window under the point. Restoring the cursor is not isolation.
  // Every receipt says which of the two happened.
  function mouseName(button) { return { left: "left", right: "right", middle: "middle" }[button] ?? "left"; }

  function assertInScreen(x, y) {
    if (!Number.isFinite(x) || !Number.isFinite(y)) throw new ExecError("coordinates must be finite numbers");
  }

  function buttonCode(button) { return button === "middle" ? 2 : button === "right" ? 1 : 0; }

  function requireSharedPointer() {
    if (!state.foregroundInput) throw Object.assign(new ExecError("This action needs the shared macOS pointer and was not sent in background mode. Use an accessibility action or a separate computer; foreground control requires exclusive desktop use authorized by the user."), { code: "shared_pointer_required" });
  }

  /** Refuse a global gesture whose landing point belongs to another application. */
  async function assertOwnsPoint(x, y) {
    if (!state.inputApp) throw new ExecError("open_application first to choose which application receives input");
    const w = await native("window_at_point", { x, y });
    if (!w?.found) throw new ExecError(`no window at (${x}, ${y}) — take a fresh screenshot and choose a point inside the target window`);
    if (w.owner_pid !== state.inputApp.pid) {
      throw new ExecError(`(${x}, ${y}) is covered by a window owned by ${w.owner_name || "another application"} (pid ${w.owner_pid}) — use an accessibility element target or a separate computer; no pointer input was sent`);
    }
    return w;
  }

  /** What a global gesture cost the user: their cursor, and briefly their foreground. */
  function pointerCost(r) {
    return {
      pointer_moved: true,
      pointer_restored: !!r?.restored,
      foreground_taken: !!r?.foreground_taken,
      ...(r?.foreground_before ? { foreground_before: r.foreground_before } : {}),
      ...(r?.foreground_after ? { foreground_after: r.foreground_after } : {}),
      ...(Number.isFinite(r?.yield_ms) && r.yield_ms > 0 ? { yield_ms: r.yield_ms } : {}),
    };
  }

View on GitHub (pinned to 73e0f67d83)