{"record":{"id":"347d1b4abca6d5d9","repo":"Hmbown/CodeWhale","slug":"no-supported-accessibility-click-at-x-y-in-the-bound","errorCode":null,"errorMessage":"no supported accessibility click at (${x}, ${y}) in the bound application (${a11yReason}) — observe the available actions, use strategy \"app\" for a window-scoped pointer click, or a separate computer","messagePattern":"no supported accessibility click at \\((.+?), (.+?)\\) in the bound application \\((.+?)\\) — observe the available actions, use strategy \"app\" for a window-scoped pointer click, or a separate computer","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/darwin.mjs","lineNumber":433,"sourceCode":"   * the bound application's accessibility tree: when the point names a\n   * pressable element we perform its semantic action, which needs no pointer\n   * and no foreground. strategy=\"a11y\" requires that and fails closed;\n   * strategy=\"event\" goes straight to the guarded global gesture.\n   */\n  async function pointerClick(button, x, y, clicks, strategy = \"auto\") {\n    assertInScreen(x, y);\n    if (![\"auto\", \"a11y\", \"event\", \"app\"].includes(strategy)) throw new ExecError(`strategy must be auto, a11y, app or event (got ${JSON.stringify(strategy)})`);\n    let a11yReason = null;\n    if (strategy !== \"event\" && [\"left\", \"right\"].includes(button) && clicks === 1) {\n      if (button === \"right\") await requireBackgroundActions();\n      const hit = await native(\"hit_test\", { x, y, perform: true, ...(button === \"right\" ? { operation: \"context\" } : {}) });\n      if (hit?.action_sent) {\n        return { action_sent: true, strategy: \"a11y\", action: hit.action, pointer_moved: false, at: { x, y }, button, clicks,\n                 element: { role: hit.element?.role ?? null, label: hit.element?.label ?? null } };\n      }\n      a11yReason = hit?.reason ?? \"not_found\";\n      if (strategy === \"a11y\") {\n        throw new ExecError(`no supported accessibility click at (${x}, ${y}) in the bound application (${a11yReason}) — observe the available actions, use strategy \"app\" for a window-scoped pointer click, or a separate computer`);\n      }\n    } else if (strategy === \"a11y\") {\n      throw new ExecError(`strategy \"a11y\" is only available for a left single click on this backend; ${mouseName(button)} x${clicks} has no accessibility equivalent`);\n    }\n    if (strategy === \"app\" || (strategy === \"auto\" && !state.foregroundInput)) {\n      // Window-routed record delivery: AppKit accepts the events as genuine\n      // input, the cursor never moves. A momentary no-raise front lease is\n      // taken and restored inside the helper; it is reported, not hidden.\n      if ((await native(\"input_capabilities\"))?.window_record === 1) {\n        // Ownership is enforced by window containment inside the helper: the\n        // events are addressed to a window id of the bound app, so a covered\n        // background window is still safe — they cannot land on the coverer.\n        const r = await native(\"bg_pointer\", { steps: clickSteps(button, x, y, clicks),\n          ...(a11yReason === \"web_popup_requires_real_click\" ? { menu_poll_ms: 6000 } : {}) });\n        return { action_sent: true, strategy: \"window-record\", input_scope: \"application-window\",\n                 at: { x, y }, button, clicks, pointer_moved: false, front_lease: r.front_lease ?? true,\n                 ...leaseAccounting(r),\n                 ...(r.menu_lease_held ? { menu_lease_held: true } : {}),","sourceCodeStart":415,"sourceCodeEnd":451,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/darwin.mjs#L415-L451","documentation":"When strategy='a11y' is requested, pointerClick fails closed if the hit_test found no pressable accessibility element supporting the requested action at the given point. a11yReason (e.g. not_found, no_action) explains why; no pointer input was sent because a11y strategy forbids fallback to raw events.","triggerScenarios":"Calling pointerClick with strategy='a11y' at a point where the element is not an AX pressable control, has no supported AX action, or hit_test fails for reasons like element not found/offscreen.","commonSituations":"Clicking empty canvas, text areas without press actions, images, or custom-drawn controls that expose no AXPress; coordinates slightly off the control; the app exposes a poor accessibility tree (e.g. Electron/Java apps without AX enabled).","solutions":["Inspect available element actions (hit_test without perform) and target a point inside a control that supports a semantic action","Use strategy='app' for a window-scoped pointer click that stays inside the bound app without global events","Fall back to strategy='auto' to let the backend pick a delivery path","Enable/repair the target app's accessibility support or use a separate computer scoped to that app"],"exampleFix":"// before\nawait click(x, y, { strategy: \"a11y\" }); // control has no AX action\n// after\nawait click(x, y, { strategy: \"app\" }); // window-scoped pointer click","handlingStrategy":"fallback","validationCode":"const hit = await observe({ x, y }); // hit_test without perform\nconst canA11y = hit?.element && hit.actions?.includes(\"press\");","typeGuard":"const a11yClickable = (hit) => Boolean(hit?.element?.role && (hit.actions ?? []).includes(\"press\"));","tryCatchPattern":"try {\n  return await click(x, y, { strategy: \"a11y\" });\n} catch (e) {\n  if (e instanceof ExecError && /no supported accessibility click/.test(e.message)) {\n    return await click(x, y, { strategy: \"app\" });\n  }\n  throw e;\n}","preventionTips":["Observe the element and its actions before choosing strategy='a11y'","Only target controls known to expose AXPress/press actions","Keep 'app' or 'auto' as fallback for apps with weak accessibility trees"],"tags":["accessibility","a11y","click","macos"],"backgroundTag":"unsupported-operation","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T16:17:23.217Z"}