{"record":{"id":"d2d4e7a444f135aa","repo":"Hmbown/CodeWhale","slug":"perform-action-failed-j-code","errorCode":null,"errorMessage":"perform_action failed: ${j.code}","messagePattern":"perform_action failed: (.+?)","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/win32.mjs","lineNumber":519,"sourceCode":"foreach ($t in $targets) { $cur = $t; break }\nforeach ($i in $indices) {\n  if ($i -eq 0) { continue }\n  $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);\n  if ($i -ge $kids.Count) { Write-Output '{\"ok\": false, \"code\": \"element_stale\"}'; exit 0 }\n  $cur = $kids[$i];\n}\n$want = '${act}';\ntry {\n  $pats = $cur.GetSupportedPatterns();\n  $names = @($pats | ForEach-Object { $_.ProgrammaticName -replace 'Pattern\\$','' -replace 'Pattern$','' });\n  $chosen = $names | Where-Object { $_ -ieq $want } | Select-Object -First 1;\n  if (-not $chosen -and ($want -ieq 'click' -or $want -ieq 'invoke')) { $chosen = 'Invoke' }\n  if (-not $chosen) { Write-Output ('{\"ok\": false, \"code\": \"action_not_found: \" + ($names -join \",\") }'); exit 0 }\n  $pt = $cur.GetCurrentPattern($pats | Where-Object { ($_.ProgrammaticName -replace 'Pattern$','') -ieq $chosen } | Select-Object -First 1);\n  if ($chosen -eq 'Invoke') { $pt.Invoke() } elseif ($chosen -eq 'ExpandCollapse') { $pt.Expand() } elseif ($chosen -eq 'Toggle') { $pt.Toggle() } else { $pt.Invoke() }\n  Write-Output '{\"ok\": true, \"sent\": true}';\n} catch { Write-Output ('{\"ok\": false, \"code\": \"' + $_.Exception.Message.Replace('\"','') + '\"}') }`, { timeoutMs: 45_000 });\n      if (!j.ok) throw new ExecError(`perform_action failed: ${j.code}`);\n      return { action_sent: true, strategy: \"a11y\", action };\n    },\n    read_clipboard: async () => {\n      const j = await psJson(`$t = Get-Clipboard -Raw -ErrorAction SilentlyContinue;\nWrite-Output ('{\"text\": ' + ($t | ConvertTo-Json -Compress) + '}');`, { timeoutMs: 10_000 });\n      return { text: j.text ?? \"\", encoding: \"utf8\" };\n    },\n    write_clipboard: async ({ text }) => {\n      const b64 = Buffer.from(String(text ?? \"\"), \"utf16le\").toString(\"base64\");\n      await psOk(`Set-Clipboard -Value ([System.Text.Encoding]::Unicode.GetString([Convert]::FromBase64String('${b64}')));\nWrite-Output '{\"ok\": true}';`, { timeoutMs: 10_000 });\n      return { written: String(text ?? \"\").length };\n    },\n    cursor_position: async () => {\n      const j = await psJson(`${USER32_PRELUDE}\n$p = New-Object User32+POINT;\n[void][User32]::GetCursorPos([ref]$p);\nWrite-Output ('{\"x\": ' + $p.X + ', \"y\": ' + $p.Y + '}');`);","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/433685b2024e7bc4c99e1e2e326bcad39b4d9d65/crates/tui/plugins/computer-use/src/backends/win32.mjs#L501-L537","documentation":"perform_action on the win32 backend resolves a UIA element by base64-encoded tree path and invokes one of its supported patterns (Invoke, ExpandCollapse, Toggle, etc.). As with set_value, the PowerShell script catches exceptions and reports {ok:false, code:\"<message>\"}; the Node layer rethrows as ExecError('perform_action failed: <code>'). A specific in-script failure, 'action_not_found: <names>', is emitted when the requested action does not match any pattern the element exposes.","triggerScenarios":"Calling perform_action({ target, action }) when: the requested action maps to a pattern the element does not implement (script returns action_not_found plus the list of available pattern names), the element path is stale so $cur is null, GetCurrentPattern returns null and the pattern method is invoked on null, or the UIA call itself throws (element vanished, COM failure, app busy).","commonSituations":"Requesting action:'click' on an element with no Invoke pattern (static text, containers); acting on an element snapshot taken before a UI change (menu closed, item re-indexed so the path points at the wrong node); 45s timeout because the target app's UI thread is blocked; invoking ExpandCollapse on an already-toggled control that throws.","solutions":["Parse the code in the message: if it starts with 'action_not_found:', retry with one of the pattern names it lists (they are the element's actually supported actions).","Re-snapshot the accessibility tree and rebuild target.path before retrying — stale paths are the most common cause of the null-element exceptions.","Verify the element type supports the action (buttons/links => Invoke, expanders => ExpandCollapse, checkboxes => Toggle); use coordinate clicks for patternless elements.","If the target app is hung (timeout-style message), unblock its UI thread or restart it, then retry.","Fall back to raw pointer events: compute the element's screen rect from the fresh snapshot and use click()."],"exampleFix":"// before\nawait backend.perform_action({ target, action: \"expand\" });\n// after\ntry {\n  await backend.perform_action({ target, action: \"expand\" });\n} catch (e) {\n  const m = /action_not_found: (.+)/.exec(e.message);\n  if (m) {\n    const supported = m[1].split(\",\");          // patterns the element really has\n    await backend.perform_action({ target: await refresh(target), action: supported[0] });\n  } else throw e;\n}","handlingStrategy":"try-catch","validationCode":"function actionSupported(elementSnapshot, action) {\n  const map = { click: \"Invoke\", invoke: \"Invoke\", expand: \"ExpandCollapse\", toggle: \"Toggle\" };\n  const wanted = map[String(action ?? \"Invoke\").toLowerCase()] ?? String(action);\n  return (elementSnapshot?.patterns ?? []).some((p) => p.replace(/Pattern$/, \"\").toLowerCase() === wanted.toLowerCase());\n}","typeGuard":"function hasElementPath(t) {\n  return t != null && typeof t === \"object\" && Array.isArray(t.path) && t.path.length > 0 &&\n    !(\"app_ref\" in t) && !(\"windowIndex\" in t) && !(\"window_id\" in t);\n}","tryCatchPattern":"try {\n  await backend.perform_action({ target, action });\n} catch (e) {\n  const notFound = /action_not_found: (.+)/.exec(e.message);\n  if (notFound) {\n    const supported = notFound[1].split(\",\").map((s) => s.trim());\n    return backend.perform_action({ target: await refreshTarget(target), action: supported[0] });\n  }\n  if (e instanceof ExecError && e.message.startsWith(\"perform_action failed:\")) {\n    return clickElementByRect(await refreshTarget(target)); // pointer fallback\n  }\n  throw e;\n}","preventionTips":["Match the action to an element pattern that exists in the snapshot (Invoke for buttons, ExpandCollapse for expanders, Toggle for checkboxes).","Refresh target.path right before perform_action; stale paths are the dominant failure mode.","Use coordinate clicks for patternless elements (text, containers).","A hang/timeout usually means the target app's UI thread is blocked — fix the app, not the call."],"tags":["windows","uiautomation","accessibility","perform-action"],"backgroundTag":"api-error-response","analyzedSha":"433685b2024e7bc4c99e1e2e326bcad39b4d9d65","analyzedAt":"2026-09-15T12:24:24.634Z","contentChangedAt":"2026-09-15T12:24:24.634Z","schemaVersion":2},"datasetVersion":"2026-09-22T16:17:23.217Z"}