Hmbown/CodeWhale · error · ExecError

perform_action needs an element target

Error message

perform_action needs an element target — {type:'element',index} from get_app_state

What it means

perform_action on the darwin backend requires a semantic element target ({type:'element', index}); any other target type (or none) is rejected with this ExecError. The check exists so semantic actions always run against a freshly observed element rather than stale coordinates.

Solutions

1. Fetch get_app_state and pass {target:{type:'element', index}} for the control to act on.
2. If the UI changed since observation, re-run get_app_state to get fresh indexes before performing the action.
3. Add a pre-call check like `if (args.target?.type !== 'element')` in wrapper code to fail early with a clearer message.

Example fix

// before
await backend.perform_action({ target: { type: 'coordinate', x: 300, y: 200 } });
// after
const state = await backend.get_app_state();
await backend.perform_action({ target: { type: 'element', index: buttonIndexFromState } });

if (!args || args.target?.type !== 'element' || !Number.isInteger(args.target.index)) throw new TypeError('perform_action requires {target:{type:"element", index}} from get_app_state');

const isElementTarget = (t) => !!t && typeof t === 'object' && t.type === 'element' && Number.isInteger(t.index);

try { await backend.perform_action(args); } catch (e) { if (String(e.message).includes('needs an element target')) { const s = await backend.get_app_state(); /* resolve fresh index, retry */ } else throw e; }

When it happens

Trigger: perform_action({}) with no target, perform_action({target:{type:'coordinate',...}}), or passing a target object whose type is not 'element'.

Common situations: Scripting presses/menus after screen clicks by reusing coordinate targets; automation tools that cached coordinates across UI changes; callers assuming perform_action accepts free-form targets.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• select_text needs an element target
• Automation admission belongs to a different task store; it…
• Automation admission execution ownership is unverified or…
• Automation name is required
• Automation prompt is required

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/c3be728c7424497d. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/darwin.mjs:1186

        // posts a complete press (down and up); the `down` field is unused.
        await native("bg_key", { code: 0, flags: 1 << 20 });
        await new Promise((r) => setTimeout(r, 60));
        await native("type", { text: value });
        const back = await native("get_value", { target: args.target });
        const verified = back?.value === value;
        return { action_sent: true, strategy: "focus-type-replace", role: back?.role ?? null,
                 after: back?.value ?? null, verified,
                 ...(verified ? {} : { note: "replacement did not verify against the control's own value; observe before relying on it" }) };
      }
    },
    focus: (args = {}) => native("focus_element", args),
    get_value: (args = {}) => native("get_value", args),
    select_text: async (args = {}) => {
      if (args.target?.type !== "element") throw new ExecError("select_text needs an element target — {type:'element',index} from get_app_state");
      return native("select_text", args);
    },
    perform_action: async (args = {}) => {
      if (args.target?.type !== "element") throw new ExecError("perform_action needs an element target — {type:'element',index} from get_app_state");
      return native("perform_action", args);
    },
    invoke_menu: async ({ path: menuPath } = {}) => invokeMenu(menuPath),
    app_script: appScript,
    read_clipboard: readClipboard,
    write_clipboard: writeClipboard,
    cursor_position: cursorPosition,
    recordingStart,
    recordingStop,
    recordingStatus,
    recordingList,
    closeSession,
    list_sessions: async () => ({
      via: "direct",
      count: 1,
      sessions: [{
        target: state.inputApp ? { pid: state.inputApp.pid, ...(state.inputApp.bundle_id ? { bundle_id: state.inputApp.bundle_id } : {}), ...(state.inputApp.name ? { name: state.inputApp.name } : {}) } : null,
        mode: state.foregroundInput ? "foreground" : "background",

View on GitHub (pinned to 73e0f67d83)