Windows semantic actions require a fresh observed element…

Error message

Windows semantic actions require a fresh observed element with window and element runtime identities

What it means

On Windows, win32.mjs's UI Automation element actions can only re-resolve an element that was freshly observed: the selector must carry both the element's runtime_id and the owning window's window_runtime_id (non-empty integer arrays), a `path` array starting at 0 of non-negative integers, and must not carry a stale window handle (`window_id`) or a non-zero windowIndex. elementScript validates these invariants and throws unsupportedSelector with this message when the selector lacks or contradicts the observation identities, because the generated PowerShell would otherwise target the wrong or a nonexistent element.

Solutions

Re-observe the target element immediately before the action so the selector carries fresh runtime_id and window_runtime_id arrays.
Strip any `window_id` field and ensure windowIndex is 0 or absent from the selector passed to Windows semantic actions.
Validate the `path` array: it must start with 0 and contain only non-negative integers; fix the observation code producing it if not.
If selectors cross a process/serialization boundary, verify runtime_id and window_runtime_id survive as integer arrays (not strings or dropped fields).

Example fix

// before
await click({ path: [0, 3], window_id: 0x1a0b2c }); // stale handle-based selector
// throws: Windows semantic actions require a fresh observed element...
// after
const observed = await observe(); // fresh observation
await click({
  runtime_id: observed.runtime_id,           // e.g. [42, 123456]
  window_runtime_id: observed.window_runtime_id,
  path: observed.path,                        // starts at 0, integers
  windowIndex: 0
});

Defensive patterns

Strategy: type-guard

Validate before calling

// Before invoking any Windows semantic action:
function validWinSelector(t) {
  const idsOk = (a) => Array.isArray(a) && a.length > 0 && a.every(Number.isInteger);
  return !!t
    && idsOk(t.runtime_id)
    && idsOk(t.window_runtime_id)
    && Array.isArray(t.path) && t.path[0] === 0
    && t.path.every((i) => Number.isInteger(i) && i >= 0)
    && (t.windowIndex == null || t.windowIndex === 0)
    && !Object.hasOwn(t, "window_id");
}

Type guard

function isFreshObservedElement(t) {
  return validWinSelector(t); // same predicate as above
}

Try / catch

try {
  await click(target);
} catch (err) {
  if (String(err.message).includes('fresh observed element')) {
    const fresh = await observe(targetHints); // re-observe to get new runtime ids
    await click(fresh);
  } else throw err;
}

Prevention

Never cache element selectors across UI changes on Windows; re-observe before each action.
Do not hand-construct selectors — always take them from the observation API.
Strip legacy window_id fields when porting selectors from other platforms.
When serializing selectors between processes, assert runtime_id/window_runtime_id survive as integer arrays.
Validate selectors with a shared predicate before any semantic action call.

When it happens

Trigger: Calling a Windows semantic action (click/type/etc.) with an element whose selector came from a stale snapshot (window or element rebuilt since observation, so runtime ids changed), a hand-built selector missing runtime_id/window_runtime_id, a selector carrying a legacy `window_id` field or a nonzero windowIndex, or a `path` that is empty, doesn't start with 0, or contains non-integer/negative indices.

Common situations: A UI automation script caches element references across a window repaint or app restart, invalidating runtime ids; a developer reuses a macOS/Linux-style selector shape on Windows where the platform requires runtime identities; serialization round-trips drop or rename the runtime_id arrays; an older selector format with window_id is fed to the new backend.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/c5b1ded48ff02e57. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/win32.mjs:131

/** Coordinate clicks on this backend are always raw pointer events; strategy="a11y" must fail closed rather than silently degrade. */
function assertEventStrategy(strategy) {
  if (strategy != null && strategy !== "auto" && strategy !== "event") {
    throw new ExecError(`strategy "${strategy}" is macOS-only; this backend dispatches coordinate clicks as raw pointer events — use an element target for a semantic action`);
  }
}

function unsupportedSelector(message) {
  return Object.assign(new ExecError(message), { code: "unsupported_selector" });
}

// A cached child path alone is unsafe: sibling/window order can change.
// Runtime IDs bind both the root and leaf to the observation that supplied them.
function elementScript(target) {
  const ids = [target?.runtime_id, target?.window_runtime_id];
  if (ids.some(id => !Array.isArray(id) || !id.length || !id.every(Number.isInteger))
      || !Array.isArray(target?.path) || target.path[0] !== 0 || !target.path.every(i => Number.isInteger(i) && i >= 0)
      || (target.windowIndex != null && target.windowIndex !== 0) || Object.hasOwn(target, "window_id")) {
    throw unsupportedSelector("Windows semantic actions require a fresh observed element with window and element runtime identities");
  }
  const encoded = Buffer.from(JSON.stringify(target), "utf16le").toString("base64");
  return `${UIA_PRELUDE}
$target = [Text.Encoding]::Unicode.GetString([Convert]::FromBase64String('${encoded}')) | ConvertFrom-Json;
$root = [System.Windows.Automation.AutomationElement]::RootElement;
$windows = @($root.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition));
$matches = @($windows | Where-Object { ($_.GetRuntimeId() -join ',') -ceq ($target.window_runtime_id -join ',') });
if ($matches.Count -ne 1) { throw 'element_stale: observed window no longer exists' }
$cur = $matches[0];
if ($target.app_ref.name -and $cur.Current.Name -cne $target.app_ref.name) { throw 'element_stale: window identity changed' }
if ($target.app_ref.pid -and $cur.Current.ProcessId -ne $target.app_ref.pid) { throw 'element_stale: window process changed' }
for ($step = 1; $step -lt $target.path.Count; $step++) {
  $kids = $cur.FindAll([System.Windows.Automation.TreeScope]::Children, [System.Windows.Automation.Condition]::TrueCondition);
  $i = $target.path[$step];
  if ($i -ge $kids.Count) { throw 'element_stale: observed path changed' }
  $cur = $kids[$i];
}
if (($cur.GetRuntimeId() -join ',') -cne ($target.runtime_id -join ',')) { throw 'element_stale: observed element was replaced' }

View on GitHub (pinned to 73e0f67d83)