{"record":{"id":"ba6857c11dfa0fe6","repo":"Hmbown/CodeWhale","slug":"element-has-no-resolved-accessibility-identity","errorCode":null,"errorMessage":"element has no resolved accessibility identity","messagePattern":"element has no resolved accessibility identity","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/darwin.mjs","lineNumber":311,"sourceCode":"    }\n    const result = tryJson(r.stdout, null);\n    const interference = leaseVerdict(result);\n    if (interference !== null) result.user_input_during_lease = interference;\n    if (state.previewEnabled && state.inputApp && [\"type\", \"key_event\", \"pointer_sequence\", \"bg_pointer\", \"bg_key\", \"set_value\", \"select_text\", \"perform_action\", \"hit_test\", \"click_element\", \"scroll_element\", \"focus_element\"].includes(tool)) {\n      try { await updatePreview(); } catch (error) { result.preview_error = error.message; }\n    }\n    return result;\n  }\n\n  async function requireBackgroundActions() {\n    if ((await native(\"input_capabilities\"))?.background_actions !== 1) {\n      throw Object.assign(new ExecError(\"Update the Computer Use helper to use background focus, selection, context menus and scrolling.\"), { code: \"app_upgrade_required\" });\n    }\n  }\n\n  function assertBoundElement(target) {\n    if (!state.inputApp || target.app_ref?.pid !== state.inputApp.pid) throw new ExecError(\"element does not belong to the bound application — open_application and observe again\");\n    if (!Array.isArray(target.path) || !Number.isInteger(target.windowIndex) || !target.role) throw new ExecError(\"element has no resolved accessibility identity\");\n  }\n\n  async function nativeLease(tool, args) {\n    if (tool === \"pointer_sequence\") requireSharedPointer();\n    if (!exec.runInputLease) throw new ExecError(\"This executor cannot safely own held input; update Computer Use\");\n    if ((await native(\"input_capabilities\"))?.input_lease !== 1) throw new ExecError(\"The native helper needs an update for disconnect-safe held input\");\n    const helper = await nativeHelper();\n    try {\n      return await exec.runInputLease(helper, [JSON.stringify({ tool, args: { ...args, ...yieldArgs, input_app_ref: state.inputApp, foreground_input: state.foregroundInput, owner_pipe: true, input_lease: true } })]);\n    } catch (error) {\n      error.code = nativeErrorCode(error.message) ?? error.code;\n      throw error;\n    }\n  }\n\n  async function updatePreview(show = false) {\n    const win = await native(\"window_info\", { app_ref: state.inputApp });\n    const dir = path.join(stateDir(), \"preview\");","sourceCodeStart":293,"sourceCodeEnd":329,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/darwin.mjs#L293-L329","documentation":"assertBoundElement also requires that a target element carry a resolved accessibility identity: an array `path` of child indices, an integer `windowIndex`, and a `role`. A target lacking any of these cannot be located in the AX tree, so the backend refuses to act on it rather than guessing.","triggerScenarios":"Passing a hand-crafted or deserialized element object to an input tool where `path` is not an array, `windowIndex` is missing/non-integer, or `role` is empty/absent — e.g. constructing a target from a screenshot coordinate instead of from observe() output, or JSON round-tripping that dropped fields.","commonSituations":"Manually building element descriptors in scripts; persisting observe() results to disk and re-reading with lossy serialization; LLM-generated tool arguments that omit required element fields; sending a coordinate-style target where an element target is required.","solutions":["Obtain the target from observe()/screenshot annotation output instead of constructing it manually, so path/windowIndex/role are populated.","Validate the element object has Array path, integer windowIndex, and non-empty role before calling the tool.","Check that your serialization of elements preserves all fields (avoid hand-written JSON for targets).","If the element genuinely cannot be resolved, use a coordinate-based pointer action on a bound app window instead of an element target."],"exampleFix":"// before\nawait click({ element: { role: \"AXButton\" } }); // no path/windowIndex\n\n// after (validate first)\nif (!Array.isArray(el.path) || !Number.isInteger(el.windowIndex) || !el.role)\n  throw new Error(\"unresolved element — re-observe\");\nawait click({ element: el });","handlingStrategy":"validation","validationCode":"function isResolvedElement(el) {\n  return !!el && Array.isArray(el.path) && Number.isInteger(el.windowIndex) && typeof el.role === 'string' && el.role.length > 0;\n}","typeGuard":"function isResolvedElement(el) {\n  return !!el && Array.isArray(el.path) && Number.isInteger(el.windowIndex) && typeof el.role === 'string' && el.role.length > 0;\n}","tryCatchPattern":"if (!isResolvedElement(el)) {\n  const fresh = await observe(); // re-resolve\n  el = fresh[el?.label] ?? null;\n}\ntry {\n  await click({ element: el });\n} catch (e) {\n  if (String(e.message).includes('no resolved accessibility identity')) throw new Error('target unresolved — re-observe required');\n  throw e;\n}","preventionTips":["Only pass element objects returned by observe() — never hand-build them.","JSON round-trip elements with full field preservation; avoid lossy serialization.","Validate path/windowIndex/role before every element-targeted tool call.","When the model composes tool arguments, enforce a schema requiring path, windowIndex, and role."],"tags":["macos","accessibility","validation","missing-fields"],"backgroundTag":"schema-validation-failed","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T11:17:16.035Z"}