{"record":{"id":"04758c3f79fec815","repo":"Hmbown/CodeWhale","slug":"windows-screenshot-does-not-support-app-ref-or-window-id","errorCode":null,"errorMessage":"Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot","messagePattern":"Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/win32.mjs","lineNumber":404,"sourceCode":"    $rect = $cur.Current.BoundingRectangle;\n    $acts = @();\n    try { $acts = @($cur.GetSupportedPatterns() | ForEach-Object { $_.ProgrammaticName -replace 'PatternIdentifiers\\\\.Pattern$','' -replace 'Pattern$','' }) } catch {}\n    $value = ''; $vp = $null;\n    if (-not $cur.Current.IsPassword -and $cur.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$vp)) { $value = [string]$vp.Current.Value }\n    [void]$els.Add([pscustomobject]@{ index = $els.Count; path = @($path); runtime_id = @($cur.GetRuntimeId()); window_runtime_id = $windowId; role = [string]$cur.Current.ControlType.ProgrammaticName; label = [string]$cur.Current.Name; value = $value.Substring(0, [Math]::Min(120, $value.Length)); enabled = $cur.Current.IsEnabled;\n      x = [int]$rect.X; y = [int]$rect.Y; w = [int]$rect.Width; h = [int]$rect.Height; actions = $acts });\n    $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);\n    for ($i = $kids.Count - 1; $i -ge 0; $i--) { $stack.Push(@($kids[$i], ($path + $i))) }\n  }\n  break;\n}\n$result = [pscustomobject]@{ found = $found; name = $appName; truncated = $truncated; elements = @($els | ForEach-Object { [pscustomobject]@{ index = $_.index; path = @($_.path); runtime_id = $_.runtime_id; window_runtime_id = $_.window_runtime_id; role = ($_.role -replace 'ControlType.',''); label = $_.label; value = $_.value; enabled = $_.enabled; position = [pscustomobject]@{ x = $_.x; y = $_.y }; size = [pscustomobject]@{ w = $_.w; h = $_.h }; actions = $_.actions } }) };\nWrite-Output ($result | ConvertTo-Json -Depth 6 -Compress);`, { timeoutMs: 60_000 });\n      if (!j.found) throw new ExecError(\"application window not found in UIA tree — pass app_ref.name as the exact window title from list_windows or list_apps.title\");\n      return j;\n    },\n    screenshot: async (args = {}) => {\n      if (Object.hasOwn(args, \"app_ref\") || Object.hasOwn(args, \"window_id\")) throw unsupportedSelector(\"Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot\");\n      const { display = activeDisplay, region, path: outPath } = args;\n      if (display != null && (!Number.isInteger(display) || display < 1)) throw new ExecError(\"display index must be a positive integer\");\n      if (region != null && (!Array.isArray(region) || region.length !== 4 || !region.every(Number.isInteger) || region[2] <= 0 || region[3] <= 0)) throw new ExecError(\"region must be integer [x,y,width,height] with positive size\");\n      const dir = recordingsDir();\n      fs.mkdirSync(dir, { recursive: true });\n      const file = path.resolve(outPath || path.join(dir, `shot-${crypto.randomBytes(6).toString(\"hex\")}.png`));\n      const meta = await psJson(`Add-Type -AssemblyName System.Windows.Forms; Add-Type -AssemblyName System.Drawing;\n$bounds = [System.Windows.Forms.SystemInformation]::VirtualScreen;\n${display == null ? \"\" : `$screens = [System.Windows.Forms.Screen]::AllScreens; if (${display} -gt $screens.Count) { throw 'display index is out of range' }; $bounds = $screens[${display - 1}].Bounds;`}\n${region == null ? \"\" : `$crop = New-Object System.Drawing.Rectangle(${region.join(\",\")}); if (-not $bounds.Contains($crop)) { throw 'region is outside capture bounds' }; $bounds = $crop;`}\n$bmp = New-Object System.Drawing.Bitmap($bounds.Width, $bounds.Height);\ntry {\n  $g = [System.Drawing.Graphics]::FromImage($bmp);\n  try { $g.CopyFromScreen($bounds.X, $bounds.Y, 0, 0, $bounds.Size); } finally { $g.Dispose(); }\n  $bmp.Save('${file.replace(/'/g, \"''\")}', [System.Drawing.Imaging.ImageFormat]::Png);\n} finally { $bmp.Dispose(); }\n@{ x = $bounds.X; y = $bounds.Y; w = $bounds.Width; h = $bounds.Height } | ConvertTo-Json -Compress;`, { timeoutMs: 30_000 });\n      if (!fs.existsSync(file) || ![meta.x, meta.y, meta.w, meta.h].every(Number.isFinite) || meta.w <= 0 || meta.h <= 0) throw new ExecError(\"screenshot did not return a valid raster\");","sourceCodeStart":386,"sourceCodeEnd":422,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/win32.mjs#L386-L422","documentation":"The Windows screenshot backend captures the desktop/display via System.Drawing; it cannot capture a single app window or region identified by app_ref or window_id. Presence of either key throws unsupportedSelector immediately.","triggerScenarios":"Calling the computer-use screenshot action on Windows with app_ref or window_id in the arguments.","commonSituations":"Porting per-window capture logic from macOS; agents trying to screenshot a specific app after get_app_state; scripts expecting window-scoped capture parity across backends.","solutions":["Omit app_ref and window_id to take a desktop screenshot.","Select a display with display (positive integer) or constrain with region: [x,y,width,height].","Crop the desktop image client-side to the target window's position/size obtained from list_windows.","For app-scoped visual inspection, use get_app_state's element list (positions/sizes) instead of a pixel capture."],"exampleFix":"// before\nawait computerUse({ action: \"screenshot\", app_ref: { name: \"Notepad\" } });\n// after\nawait computerUse({ action: \"screenshot\", display: 1 });","handlingStrategy":"validation","validationCode":"if (args && (\"app_ref\" in args || \"window_id\" in args)) throw new Error(\"win32 screenshot is desktop-only; use display/region\");","typeGuard":"const desktopOnlyShot = (a) => a == null || (!(\"app_ref\" in a) && !(\"window_id\" in a));","tryCatchPattern":"try { await screenshot(args) } catch (e) { if (String(e.message).includes(\"desktop screenshot\")) { delete args.app_ref; delete args.window_id; await screenshot(args); } else throw e; }","preventionTips":["Use display and region parameters to scope captures on Windows","Crop captured images with window rects from list_windows when you need per-window imagery","Document desktop-only capture as a Windows limitation in callers"],"tags":["windows","computer-use","screenshot","unsupported-selector"],"backgroundTag":"unsupported-operation","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T16:17:23.217Z"}