Hmbown/CodeWhale · error · ExecError

application window not found in UIA tree — pass…

Error message

application window not found in UIA tree — pass app_ref.name as the exact window title from list_windows or list_apps.title

What it means

get_app_state walks the UI Automation (UIA) tree via PowerShell to find a top-level window whose title matches app_ref.name. It throws this ExecError when no matching window was found (j.found false, possibly after truncation), telling the caller to pass the exact window title from list_windows or list_apps.title.

Solutions

1. Call list_windows or list_apps first and copy the title field exactly into app_ref.name
2. Re-list windows if the app's title changed (documents/tabs update titles dynamically)
3. Check the app actually has a visible top-level window; background processes won't appear in UIA
4. Catch the ExecError and fall back to list_apps so the agent can retry with a verified title

Example fix

// before
await getAppState({ appRef: { name: "notepad" } });
// after
const wins = await listWindows();
await getAppState({ appRef: { name: wins.find(w => w.app === "notepad").title } });

async function resolveWindowTitle(backend, processName) {
  const { windows } = await backend.list_windows();
  return windows.find(w => w.title.toLowerCase().includes(processName.toLowerCase()))?.title ?? null;
}

try {
  return await backend.get_app_state({ app_ref: { name: title } });
} catch (e) {
  if (String(e.message).includes("not found in UIA tree")) {
    const fresh = await backend.list_windows();
    const exact = fresh.windows.find(w => w.title === title);
    if (exact) return await backend.get_app_state({ app_ref: { name: exact.title } });
  }
  throw e;
}

When it happens

Trigger: Calling get_app_state with app_ref.name set to a process name ("chrome") instead of the exact window title ("New Tab - Google Chrome"); the window is closed or minimized to tray; case/spacing mismatch; the UIA search timed out or truncated before reaching the window.

Common situations: Confusing process names with window titles; stale titles after the page/tab changed; elevated (admin) windows invisible to a non-elevated UIA query; apps with no main window (background services).

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• set_value failed
• unsupported UIA action
• Windows get_app_state does not support window_id
• Windows get_app_state supports only app_ref
• Agent not found

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/6b300ddad08cf849. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/win32.mjs:400

  $stack.Push(@($t, @(0)));
  while ($stack.Count -gt 0) {
    $entry = $stack.Pop(); $cur = $entry[0]; $path = $entry[1];
    if ($els.Count -ge $max) { $truncated = $true; break }
    $rect = $cur.Current.BoundingRectangle;
    $acts = @();
    try { $acts = @($cur.GetSupportedPatterns() | ForEach-Object { $_.ProgrammaticName -replace 'PatternIdentifiers\\.Pattern$','' -replace 'Pattern$','' }) } catch {}
    $value = ''; $vp = $null;
    if (-not $cur.Current.IsPassword -and $cur.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$vp)) { $value = [string]$vp.Current.Value }
    [void]$els.Add([pscustomobject]@{ index = $els.Count; path = @($path); runtime_id = @($cur.GetRuntimeId()); window_runtime_id = $windowId; role = [string]$cur.Current.ControlType.ProgrammaticName; label = [string]$cur.Current.Name; value = $value.Substring(0, [Math]::Min(120, $value.Length)); enabled = $cur.Current.IsEnabled;
      x = [int]$rect.X; y = [int]$rect.Y; w = [int]$rect.Width; h = [int]$rect.Height; actions = $acts });
    $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);
    for ($i = $kids.Count - 1; $i -ge 0; $i--) { $stack.Push(@($kids[$i], ($path + $i))) }
  }
  break;
}
$result = [pscustomobject]@{ found = $found; name = $appName; truncated = $truncated; elements = @($els | ForEach-Object { [pscustomobject]@{ index = $_.index; path = @($_.path); runtime_id = $_.runtime_id; window_runtime_id = $_.window_runtime_id; role = ($_.role -replace 'ControlType.',''); label = $_.label; value = $_.value; enabled = $_.enabled; position = [pscustomobject]@{ x = $_.x; y = $_.y }; size = [pscustomobject]@{ w = $_.w; h = $_.h }; actions = $_.actions } }) };
Write-Output ($result | ConvertTo-Json -Depth 6 -Compress);`, { timeoutMs: 60_000 });
      if (!j.found) throw new ExecError("application window not found in UIA tree — pass app_ref.name as the exact window title from list_windows or list_apps.title");
      return j;
    },
    screenshot: async (args = {}) => {
      if (Object.hasOwn(args, "app_ref") || Object.hasOwn(args, "window_id")) throw unsupportedSelector("Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot");
      const { display = activeDisplay, region, path: outPath } = args;
      if (display != null && (!Number.isInteger(display) || display < 1)) throw new ExecError("display index must be a positive integer");
      if (region != null && (!Array.isArray(region) || region.length !== 4 || !region.every(Number.isInteger) || region[2] <= 0 || region[3] <= 0)) throw new ExecError("region must be integer [x,y,width,height] with positive size");
      const dir = recordingsDir();
      fs.mkdirSync(dir, { recursive: true });
      const file = path.resolve(outPath || path.join(dir, `shot-${crypto.randomBytes(6).toString("hex")}.png`));
      const meta = await psJson(`Add-Type -AssemblyName System.Windows.Forms; Add-Type -AssemblyName System.Drawing;
$bounds = [System.Windows.Forms.SystemInformation]::VirtualScreen;
${display == null ? "" : `$screens = [System.Windows.Forms.Screen]::AllScreens; if (${display} -gt $screens.Count) { throw 'display index is out of range' }; $bounds = $screens[${display - 1}].Bounds;`}
${region == null ? "" : `$crop = New-Object System.Drawing.Rectangle(${region.join(",")}); if (-not $bounds.Contains($crop)) { throw 'region is outside capture bounds' }; $bounds = $crop;`}
$bmp = New-Object System.Drawing.Bitmap($bounds.Width, $bounds.Height);
try {
  $g = [System.Drawing.Graphics]::FromImage($bmp);
  try { $g.CopyFromScreen($bounds.X, $bounds.Y, 0, 0, $bounds.Size); } finally { $g.Dispose(); }

View on GitHub (pinned to 73e0f67d83)