{"record":{"id":"bd4c7bcf43cf1982","repo":"Hmbown/CodeWhale","slug":"no-window-at-x-y-take-a-fresh-screenshot-and-choose-a-point","errorCode":null,"errorMessage":"no window at (${x}, ${y}) — take a fresh screenshot and choose a point inside the target window","messagePattern":"no window at \\((.+?), (.+?)\\) — take a fresh screenshot and choose a point inside the target window","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/darwin.mjs","lineNumber":374,"sourceCode":"  //      window under the point. Restoring the cursor is not isolation.\n  // Every receipt says which of the two happened.\n  function mouseName(button) { return { left: \"left\", right: \"right\", middle: \"middle\" }[button] ?? \"left\"; }\n\n  function assertInScreen(x, y) {\n    if (!Number.isFinite(x) || !Number.isFinite(y)) throw new ExecError(\"coordinates must be finite numbers\");\n  }\n\n  function buttonCode(button) { return button === \"middle\" ? 2 : button === \"right\" ? 1 : 0; }\n\n  function requireSharedPointer() {\n    if (!state.foregroundInput) throw Object.assign(new ExecError(\"This action needs the shared macOS pointer and was not sent in background mode. Use an accessibility action or a separate computer; foreground control requires exclusive desktop use authorized by the user.\"), { code: \"shared_pointer_required\" });\n  }\n\n  /** Refuse a global gesture whose landing point belongs to another application. */\n  async function assertOwnsPoint(x, y) {\n    if (!state.inputApp) throw new ExecError(\"open_application first to choose which application receives input\");\n    const w = await native(\"window_at_point\", { x, y });\n    if (!w?.found) throw new ExecError(`no window at (${x}, ${y}) — take a fresh screenshot and choose a point inside the target window`);\n    if (w.owner_pid !== state.inputApp.pid) {\n      throw new ExecError(`(${x}, ${y}) is covered by a window owned by ${w.owner_name || \"another application\"} (pid ${w.owner_pid}) — use an accessibility element target or a separate computer; no pointer input was sent`);\n    }\n    return w;\n  }\n\n  /** What a global gesture cost the user: their cursor, and briefly their foreground. */\n  function pointerCost(r) {\n    return {\n      pointer_moved: true,\n      pointer_restored: !!r?.restored,\n      foreground_taken: !!r?.foreground_taken,\n      ...(r?.foreground_before ? { foreground_before: r.foreground_before } : {}),\n      ...(r?.foreground_after ? { foreground_after: r.foreground_after } : {}),\n      ...(Number.isFinite(r?.yield_ms) && r.yield_ms > 0 ? { yield_ms: r.yield_ms } : {}),\n    };\n  }\n","sourceCodeStart":356,"sourceCodeEnd":392,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/darwin.mjs#L356-L392","documentation":"After confirming an app is bound, assertOwnsPoint asks the native helper for the window at the gesture point (window_at_point). If no window is found at (x, y), the point is empty desktop/menu-bar-free space or outside all windows, and the gesture is refused with guidance to re-screenshot. This guards against clicking into nothing and mistargeting.","triggerScenarios":"Pointer action at coordinates where window_at_point returns found=false — stale coordinates from an old screenshot after the window moved/closed/resized, coordinates in screen-space outside any window, multi-display coordinate confusion, or a minimized window.","commonSituations":"Windows moved by the user between screenshot and click; screenshot taken before a layout change; coordinates from a scaled/retina image not converted to screen points; clicking on desktop background; wrong display's coordinate space on multi-monitor setups.","solutions":["Take a fresh screenshot and pick new coordinates inside the target window, then retry.","Re-observe/open_application to refresh window geometry if the window moved or resized.","Convert image coordinates to screen points with the correct scale factor (Retina 2x) and display offset.","Prefer accessibility element targets, which track window layout automatically instead of fixed points."],"exampleFix":"// before\nawait click({ x: 400, y: 300 }); // stale point, window moved\n\n// after\nconst shot = await screenshot(); // fresh geometry\nconst pt = toScreenPoint(shot, 400, 300); // scale + offset applied\nawait click(pt);","handlingStrategy":"retry","validationCode":"const w = await native('window_at_point', { x, y });\nif (!w?.found) throw new Error('point is outside every window — take a fresh screenshot first');","typeGuard":null,"tryCatchPattern":"try {\n  await click({ x, y });\n} catch (e) {\n  if (String(e.message).startsWith('no window at')) {\n    const shot = await screenshot();          // fresh geometry\n    const pt = toScreenPoint(shot, x, y);     // re-apply scale/offset\n    await click(pt);\n  } else throw e;\n}","preventionTips":["Always derive coordinates from the most recent screenshot, never from cached ones.","Apply Retina scale factors and multi-display offsets when converting image to screen coordinates.","Re-observe if the window may have moved, resized, or been minimized.","Prefer accessibility element targets, which follow layout changes automatically."],"tags":["macos","coordinates","stale-screenshot","pointer"],"backgroundTag":"resource-not-found","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T16:17:23.217Z"}