{"record":{"id":"c3be728c7424497d","repo":"Hmbown/CodeWhale","slug":"perform-action-needs-an-element-target-type-element-index","errorCode":null,"errorMessage":"perform_action needs an element target — {type:'element',index} from get_app_state","messagePattern":"perform_action needs an element target — (.+?) from get_app_state","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/darwin.mjs","lineNumber":1186,"sourceCode":"        // posts a complete press (down and up); the `down` field is unused.\n        await native(\"bg_key\", { code: 0, flags: 1 << 20 });\n        await new Promise((r) => setTimeout(r, 60));\n        await native(\"type\", { text: value });\n        const back = await native(\"get_value\", { target: args.target });\n        const verified = back?.value === value;\n        return { action_sent: true, strategy: \"focus-type-replace\", role: back?.role ?? null,\n                 after: back?.value ?? null, verified,\n                 ...(verified ? {} : { note: \"replacement did not verify against the control's own value; observe before relying on it\" }) };\n      }\n    },\n    focus: (args = {}) => native(\"focus_element\", args),\n    get_value: (args = {}) => native(\"get_value\", args),\n    select_text: async (args = {}) => {\n      if (args.target?.type !== \"element\") throw new ExecError(\"select_text needs an element target — {type:'element',index} from get_app_state\");\n      return native(\"select_text\", args);\n    },\n    perform_action: async (args = {}) => {\n      if (args.target?.type !== \"element\") throw new ExecError(\"perform_action needs an element target — {type:'element',index} from get_app_state\");\n      return native(\"perform_action\", args);\n    },\n    invoke_menu: async ({ path: menuPath } = {}) => invokeMenu(menuPath),\n    app_script: appScript,\n    read_clipboard: readClipboard,\n    write_clipboard: writeClipboard,\n    cursor_position: cursorPosition,\n    recordingStart,\n    recordingStop,\n    recordingStatus,\n    recordingList,\n    closeSession,\n    list_sessions: async () => ({\n      via: \"direct\",\n      count: 1,\n      sessions: [{\n        target: state.inputApp ? { pid: state.inputApp.pid, ...(state.inputApp.bundle_id ? { bundle_id: state.inputApp.bundle_id } : {}), ...(state.inputApp.name ? { name: state.inputApp.name } : {}) } : null,\n        mode: state.foregroundInput ? \"foreground\" : \"background\",","sourceCodeStart":1168,"sourceCodeEnd":1204,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/darwin.mjs#L1168-L1204","documentation":"perform_action on the darwin backend requires a semantic element target ({type:'element', index}); any other target type (or none) is rejected with this ExecError. The check exists so semantic actions always run against a freshly observed element rather than stale coordinates.","triggerScenarios":"perform_action({}) with no target, perform_action({target:{type:'coordinate',...}}), or passing a target object whose type is not 'element'.","commonSituations":"Scripting presses/menus after screen clicks by reusing coordinate targets; automation tools that cached coordinates across UI changes; callers assuming perform_action accepts free-form targets.","solutions":["Fetch get_app_state and pass {target:{type:'element', index}} for the control to act on.","If the UI changed since observation, re-run get_app_state to get fresh indexes before performing the action.","Add a pre-call check like `if (args.target?.type !== 'element')` in wrapper code to fail early with a clearer message."],"exampleFix":"// before\nawait backend.perform_action({ target: { type: 'coordinate', x: 300, y: 200 } });\n// after\nconst state = await backend.get_app_state();\nawait backend.perform_action({ target: { type: 'element', index: buttonIndexFromState } });","handlingStrategy":"validation","validationCode":"if (!args || args.target?.type !== 'element' || !Number.isInteger(args.target.index)) throw new TypeError('perform_action requires {target:{type:\"element\", index}} from get_app_state');","typeGuard":"const isElementTarget = (t) => !!t && typeof t === 'object' && t.type === 'element' && Number.isInteger(t.index);","tryCatchPattern":"try { await backend.perform_action(args); } catch (e) { if (String(e.message).includes('needs an element target')) { const s = await backend.get_app_state(); /* resolve fresh index, retry */ } else throw e; }","preventionTips":["Build a thin wrapper that resolves labels to fresh element indexes via get_app_state.","Re-observe after any UI-changing step before performing the next element action.","Keep coordinate clicks and semantic actions in distinct code paths."],"tags":["automation","argument-validation","element-target"],"backgroundTag":"invalid-argument-value","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}