{"record":{"id":"6b300ddad08cf849","repo":"Hmbown/CodeWhale","slug":"application-window-not-found-in-uia-tree-pass-app-ref-name","errorCode":null,"errorMessage":"application window not found in UIA tree — pass app_ref.name as the exact window title from list_windows or list_apps.title","messagePattern":"application window not found in UIA tree — pass app_ref\\.name as the exact window title from list_windows or list_apps\\.title","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/win32.mjs","lineNumber":400,"sourceCode":"  $stack.Push(@($t, @(0)));\n  while ($stack.Count -gt 0) {\n    $entry = $stack.Pop(); $cur = $entry[0]; $path = $entry[1];\n    if ($els.Count -ge $max) { $truncated = $true; break }\n    $rect = $cur.Current.BoundingRectangle;\n    $acts = @();\n    try { $acts = @($cur.GetSupportedPatterns() | ForEach-Object { $_.ProgrammaticName -replace 'PatternIdentifiers\\\\.Pattern$','' -replace 'Pattern$','' }) } catch {}\n    $value = ''; $vp = $null;\n    if (-not $cur.Current.IsPassword -and $cur.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$vp)) { $value = [string]$vp.Current.Value }\n    [void]$els.Add([pscustomobject]@{ index = $els.Count; path = @($path); runtime_id = @($cur.GetRuntimeId()); window_runtime_id = $windowId; role = [string]$cur.Current.ControlType.ProgrammaticName; label = [string]$cur.Current.Name; value = $value.Substring(0, [Math]::Min(120, $value.Length)); enabled = $cur.Current.IsEnabled;\n      x = [int]$rect.X; y = [int]$rect.Y; w = [int]$rect.Width; h = [int]$rect.Height; actions = $acts });\n    $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);\n    for ($i = $kids.Count - 1; $i -ge 0; $i--) { $stack.Push(@($kids[$i], ($path + $i))) }\n  }\n  break;\n}\n$result = [pscustomobject]@{ found = $found; name = $appName; truncated = $truncated; elements = @($els | ForEach-Object { [pscustomobject]@{ index = $_.index; path = @($_.path); runtime_id = $_.runtime_id; window_runtime_id = $_.window_runtime_id; role = ($_.role -replace 'ControlType.',''); label = $_.label; value = $_.value; enabled = $_.enabled; position = [pscustomobject]@{ x = $_.x; y = $_.y }; size = [pscustomobject]@{ w = $_.w; h = $_.h }; actions = $_.actions } }) };\nWrite-Output ($result | ConvertTo-Json -Depth 6 -Compress);`, { timeoutMs: 60_000 });\n      if (!j.found) throw new ExecError(\"application window not found in UIA tree — pass app_ref.name as the exact window title from list_windows or list_apps.title\");\n      return j;\n    },\n    screenshot: async (args = {}) => {\n      if (Object.hasOwn(args, \"app_ref\") || Object.hasOwn(args, \"window_id\")) throw unsupportedSelector(\"Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot\");\n      const { display = activeDisplay, region, path: outPath } = args;\n      if (display != null && (!Number.isInteger(display) || display < 1)) throw new ExecError(\"display index must be a positive integer\");\n      if (region != null && (!Array.isArray(region) || region.length !== 4 || !region.every(Number.isInteger) || region[2] <= 0 || region[3] <= 0)) throw new ExecError(\"region must be integer [x,y,width,height] with positive size\");\n      const dir = recordingsDir();\n      fs.mkdirSync(dir, { recursive: true });\n      const file = path.resolve(outPath || path.join(dir, `shot-${crypto.randomBytes(6).toString(\"hex\")}.png`));\n      const meta = await psJson(`Add-Type -AssemblyName System.Windows.Forms; Add-Type -AssemblyName System.Drawing;\n$bounds = [System.Windows.Forms.SystemInformation]::VirtualScreen;\n${display == null ? \"\" : `$screens = [System.Windows.Forms.Screen]::AllScreens; if (${display} -gt $screens.Count) { throw 'display index is out of range' }; $bounds = $screens[${display - 1}].Bounds;`}\n${region == null ? \"\" : `$crop = New-Object System.Drawing.Rectangle(${region.join(\",\")}); if (-not $bounds.Contains($crop)) { throw 'region is outside capture bounds' }; $bounds = $crop;`}\n$bmp = New-Object System.Drawing.Bitmap($bounds.Width, $bounds.Height);\ntry {\n  $g = [System.Drawing.Graphics]::FromImage($bmp);\n  try { $g.CopyFromScreen($bounds.X, $bounds.Y, 0, 0, $bounds.Size); } finally { $g.Dispose(); }","sourceCodeStart":382,"sourceCodeEnd":418,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/win32.mjs#L382-L418","documentation":"get_app_state walks the UI Automation (UIA) tree via PowerShell to find a top-level window whose title matches app_ref.name. It throws this ExecError when no matching window was found (j.found false, possibly after truncation), telling the caller to pass the exact window title from list_windows or list_apps.title.","triggerScenarios":"Calling get_app_state with app_ref.name set to a process name (\"chrome\") instead of the exact window title (\"New Tab - Google Chrome\"); the window is closed or minimized to tray; case/spacing mismatch; the UIA search timed out or truncated before reaching the window.","commonSituations":"Confusing process names with window titles; stale titles after the page/tab changed; elevated (admin) windows invisible to a non-elevated UIA query; apps with no main window (background services).","solutions":["Call list_windows or list_apps first and copy the title field exactly into app_ref.name","Re-list windows if the app's title changed (documents/tabs update titles dynamically)","Check the app actually has a visible top-level window; background processes won't appear in UIA","Catch the ExecError and fall back to list_apps so the agent can retry with a verified title"],"exampleFix":"// before\nawait getAppState({ appRef: { name: \"notepad\" } });\n// after\nconst wins = await listWindows();\nawait getAppState({ appRef: { name: wins.find(w => w.app === \"notepad\").title } });","handlingStrategy":"fallback","validationCode":"async function resolveWindowTitle(backend, processName) {\n  const { windows } = await backend.list_windows();\n  return windows.find(w => w.title.toLowerCase().includes(processName.toLowerCase()))?.title ?? null;\n}","typeGuard":null,"tryCatchPattern":"try {\n  return await backend.get_app_state({ app_ref: { name: title } });\n} catch (e) {\n  if (String(e.message).includes(\"not found in UIA tree\")) {\n    const fresh = await backend.list_windows();\n    const exact = fresh.windows.find(w => w.title === title);\n    if (exact) return await backend.get_app_state({ app_ref: { name: exact.title } });\n  }\n  throw e;\n}","preventionTips":["Copy window titles verbatim from list_windows/list_apps, never guess","Re-list windows when titles change (tabs/documents)","Remember background/elevated processes may not appear in UIA","Retry after the window finishes opening (launch is async)"],"tags":["uia","window","not-found","windows"],"backgroundTag":"entity-not-found","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T16:17:23.217Z"}