Hmbown/CodeWhale · error · ExecError

perform_action failed

Error message

perform_action failed: ${j.code}

What it means

perform_action on the win32 backend resolves a UIA element by base64-encoded tree path and invokes one of its supported patterns (Invoke, ExpandCollapse, Toggle, etc.). As with set_value, the PowerShell script catches exceptions and reports {ok:false, code:"<message>"}; the Node layer rethrows as ExecError('perform_action failed: <code>'). A specific in-script failure, 'action_not_found: <names>', is emitted when the requested action does not match any pattern the element exposes.

Solutions

  1. Parse the code in the message: if it starts with 'action_not_found:', retry with one of the pattern names it lists (they are the element's actually supported actions).
  2. Re-snapshot the accessibility tree and rebuild target.path before retrying — stale paths are the most common cause of the null-element exceptions.
  3. Verify the element type supports the action (buttons/links => Invoke, expanders => ExpandCollapse, checkboxes => Toggle); use coordinate clicks for patternless elements.
  4. If the target app is hung (timeout-style message), unblock its UI thread or restart it, then retry.
  5. Fall back to raw pointer events: compute the element's screen rect from the fresh snapshot and use click().

Example fix

// before
await backend.perform_action({ target, action: "expand" });
// after
try {
  await backend.perform_action({ target, action: "expand" });
} catch (e) {
  const m = /action_not_found: (.+)/.exec(e.message);
  if (m) {
    const supported = m[1].split(",");          // patterns the element really has
    await backend.perform_action({ target: await refresh(target), action: supported[0] });
  } else throw e;
}
Defensive patterns

Strategy: try-catch

Validate before calling

function actionSupported(elementSnapshot, action) {
  const map = { click: "Invoke", invoke: "Invoke", expand: "ExpandCollapse", toggle: "Toggle" };
  const wanted = map[String(action ?? "Invoke").toLowerCase()] ?? String(action);
  return (elementSnapshot?.patterns ?? []).some((p) => p.replace(/Pattern$/, "").toLowerCase() === wanted.toLowerCase());
}

Type guard

function hasElementPath(t) {
  return t != null && typeof t === "object" && Array.isArray(t.path) && t.path.length > 0 &&
    !("app_ref" in t) && !("windowIndex" in t) && !("window_id" in t);
}

Try / catch

try {
  await backend.perform_action({ target, action });
} catch (e) {
  const notFound = /action_not_found: (.+)/.exec(e.message);
  if (notFound) {
    const supported = notFound[1].split(",").map((s) => s.trim());
    return backend.perform_action({ target: await refreshTarget(target), action: supported[0] });
  }
  if (e instanceof ExecError && e.message.startsWith("perform_action failed:")) {
    return clickElementByRect(await refreshTarget(target)); // pointer fallback
  }
  throw e;
}

Prevention

When it happens

Trigger: Calling perform_action({ target, action }) when: the requested action maps to a pattern the element does not implement (script returns action_not_found plus the list of available pattern names), the element path is stale so $cur is null, GetCurrentPattern returns null and the pattern method is invoked on null, or the UIA call itself throws (element vanished, COM failure, app busy).

Common situations: Requesting action:'click' on an element with no Invoke pattern (static text, containers); acting on an element snapshot taken before a UI change (menu closed, item re-indexed so the path points at the wrong node); 45s timeout because the target app's UI thread is blocked; invoking ExpandCollapse on an already-toggled control that throws.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/d2d4e7a444f135aa. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/win32.mjs:519

foreach ($t in $targets) { $cur = $t; break }
foreach ($i in $indices) {
  if ($i -eq 0) { continue }
  $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);
  if ($i -ge $kids.Count) { Write-Output '{"ok": false, "code": "element_stale"}'; exit 0 }
  $cur = $kids[$i];
}
$want = '${act}';
try {
  $pats = $cur.GetSupportedPatterns();
  $names = @($pats | ForEach-Object { $_.ProgrammaticName -replace 'Pattern\$','' -replace 'Pattern$','' });
  $chosen = $names | Where-Object { $_ -ieq $want } | Select-Object -First 1;
  if (-not $chosen -and ($want -ieq 'click' -or $want -ieq 'invoke')) { $chosen = 'Invoke' }
  if (-not $chosen) { Write-Output ('{"ok": false, "code": "action_not_found: " + ($names -join ",") }'); exit 0 }
  $pt = $cur.GetCurrentPattern($pats | Where-Object { ($_.ProgrammaticName -replace 'Pattern$','') -ieq $chosen } | Select-Object -First 1);
  if ($chosen -eq 'Invoke') { $pt.Invoke() } elseif ($chosen -eq 'ExpandCollapse') { $pt.Expand() } elseif ($chosen -eq 'Toggle') { $pt.Toggle() } else { $pt.Invoke() }
  Write-Output '{"ok": true, "sent": true}';
} catch { Write-Output ('{"ok": false, "code": "' + $_.Exception.Message.Replace('"','') + '"}') }`, { timeoutMs: 45_000 });
      if (!j.ok) throw new ExecError(`perform_action failed: ${j.code}`);
      return { action_sent: true, strategy: "a11y", action };
    },
    read_clipboard: async () => {
      const j = await psJson(`$t = Get-Clipboard -Raw -ErrorAction SilentlyContinue;
Write-Output ('{"text": ' + ($t | ConvertTo-Json -Compress) + '}');`, { timeoutMs: 10_000 });
      return { text: j.text ?? "", encoding: "utf8" };
    },
    write_clipboard: async ({ text }) => {
      const b64 = Buffer.from(String(text ?? ""), "utf16le").toString("base64");
      await psOk(`Set-Clipboard -Value ([System.Text.Encoding]::Unicode.GetString([Convert]::FromBase64String('${b64}')));
Write-Output '{"ok": true}';`, { timeoutMs: 10_000 });
      return { written: String(text ?? "").length };
    },
    cursor_position: async () => {
      const j = await psJson(`${USER32_PRELUDE}
$p = New-Object User32+POINT;
[void][User32]::GetCursorPos([ref]$p);
Write-Output ('{"x": ' + $p.X + ', "y": ' + $p.Y + '}');`);

View on GitHub (pinned to 433685b202)