Hmbown/CodeWhale · error

Windows screenshot does not support app_ref or window_id…

Error message

Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot

What it means

The Windows screenshot backend captures the desktop/display via System.Drawing; it cannot capture a single app window or region identified by app_ref or window_id. Presence of either key throws unsupportedSelector immediately.

Solutions

  1. Omit app_ref and window_id to take a desktop screenshot.
  2. Select a display with display (positive integer) or constrain with region: [x,y,width,height].
  3. Crop the desktop image client-side to the target window's position/size obtained from list_windows.
  4. For app-scoped visual inspection, use get_app_state's element list (positions/sizes) instead of a pixel capture.

Example fix

// before
await computerUse({ action: "screenshot", app_ref: { name: "Notepad" } });
// after
await computerUse({ action: "screenshot", display: 1 });
Defensive patterns

Strategy: validation

Validate before calling

if (args && ("app_ref" in args || "window_id" in args)) throw new Error("win32 screenshot is desktop-only; use display/region");

Type guard

const desktopOnlyShot = (a) => a == null || (!("app_ref" in a) && !("window_id" in a));

Try / catch

try { await screenshot(args) } catch (e) { if (String(e.message).includes("desktop screenshot")) { delete args.app_ref; delete args.window_id; await screenshot(args); } else throw e; }

Prevention

When it happens

Trigger: Calling the computer-use screenshot action on Windows with app_ref or window_id in the arguments.

Common situations: Porting per-window capture logic from macOS; agents trying to screenshot a specific app after get_app_state; scripts expecting window-scoped capture parity across backends.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/04758c3f79fec815. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/win32.mjs:404

    $rect = $cur.Current.BoundingRectangle;
    $acts = @();
    try { $acts = @($cur.GetSupportedPatterns() | ForEach-Object { $_.ProgrammaticName -replace 'PatternIdentifiers\\.Pattern$','' -replace 'Pattern$','' }) } catch {}
    $value = ''; $vp = $null;
    if (-not $cur.Current.IsPassword -and $cur.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$vp)) { $value = [string]$vp.Current.Value }
    [void]$els.Add([pscustomobject]@{ index = $els.Count; path = @($path); runtime_id = @($cur.GetRuntimeId()); window_runtime_id = $windowId; role = [string]$cur.Current.ControlType.ProgrammaticName; label = [string]$cur.Current.Name; value = $value.Substring(0, [Math]::Min(120, $value.Length)); enabled = $cur.Current.IsEnabled;
      x = [int]$rect.X; y = [int]$rect.Y; w = [int]$rect.Width; h = [int]$rect.Height; actions = $acts });
    $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);
    for ($i = $kids.Count - 1; $i -ge 0; $i--) { $stack.Push(@($kids[$i], ($path + $i))) }
  }
  break;
}
$result = [pscustomobject]@{ found = $found; name = $appName; truncated = $truncated; elements = @($els | ForEach-Object { [pscustomobject]@{ index = $_.index; path = @($_.path); runtime_id = $_.runtime_id; window_runtime_id = $_.window_runtime_id; role = ($_.role -replace 'ControlType.',''); label = $_.label; value = $_.value; enabled = $_.enabled; position = [pscustomobject]@{ x = $_.x; y = $_.y }; size = [pscustomobject]@{ w = $_.w; h = $_.h }; actions = $_.actions } }) };
Write-Output ($result | ConvertTo-Json -Depth 6 -Compress);`, { timeoutMs: 60_000 });
      if (!j.found) throw new ExecError("application window not found in UIA tree — pass app_ref.name as the exact window title from list_windows or list_apps.title");
      return j;
    },
    screenshot: async (args = {}) => {
      if (Object.hasOwn(args, "app_ref") || Object.hasOwn(args, "window_id")) throw unsupportedSelector("Windows screenshot does not support app_ref or window_id; omit them for a desktop screenshot");
      const { display = activeDisplay, region, path: outPath } = args;
      if (display != null && (!Number.isInteger(display) || display < 1)) throw new ExecError("display index must be a positive integer");
      if (region != null && (!Array.isArray(region) || region.length !== 4 || !region.every(Number.isInteger) || region[2] <= 0 || region[3] <= 0)) throw new ExecError("region must be integer [x,y,width,height] with positive size");
      const dir = recordingsDir();
      fs.mkdirSync(dir, { recursive: true });
      const file = path.resolve(outPath || path.join(dir, `shot-${crypto.randomBytes(6).toString("hex")}.png`));
      const meta = await psJson(`Add-Type -AssemblyName System.Windows.Forms; Add-Type -AssemblyName System.Drawing;
$bounds = [System.Windows.Forms.SystemInformation]::VirtualScreen;
${display == null ? "" : `$screens = [System.Windows.Forms.Screen]::AllScreens; if (${display} -gt $screens.Count) { throw 'display index is out of range' }; $bounds = $screens[${display - 1}].Bounds;`}
${region == null ? "" : `$crop = New-Object System.Drawing.Rectangle(${region.join(",")}); if (-not $bounds.Contains($crop)) { throw 'region is outside capture bounds' }; $bounds = $crop;`}
$bmp = New-Object System.Drawing.Bitmap($bounds.Width, $bounds.Height);
try {
  $g = [System.Drawing.Graphics]::FromImage($bmp);
  try { $g.CopyFromScreen($bounds.X, $bounds.Y, 0, 0, $bounds.Size); } finally { $g.Dispose(); }
  $bmp.Save('${file.replace(/'/g, "''")}', [System.Drawing.Imaging.ImageFormat]::Png);
} finally { $bmp.Dispose(); }
@{ x = $bounds.X; y = $bounds.Y; w = $bounds.Width; h = $bounds.Height } | ConvertTo-Json -Compress;`, { timeoutMs: 30_000 });
      if (!fs.existsSync(file) || ![meta.x, meta.y, meta.w, meta.h].every(Number.isFinite) || meta.w <= 0 || meta.h <= 0) throw new ExecError("screenshot did not return a valid raster");

View on GitHub (pinned to 73e0f67d83)