stablyai/orca · error · RuntimeError

coordinate action requires a visible window and coordinates

Error message

coordinate action requires a visible window and coordinates

What it means

Raised by screen_point (runtime.py:797-798) when none of the resolution paths are available: there is no live node rect (node is None or returned None), no saved_element to flag as stale, AND the window_rect/x/y fallback is incomplete (window_rect is None, or x is None, or y is None). The function cannot compute a screen coordinate from any source, so it refuses. This indicates the caller neither supplied an element reference nor complete explicit coordinates within a visible window.

Source

Thrown at native/computer-use-linux/runtime.py:798

        if label in priority:
            return index
        if fallback is None and any(term in label for term in ("click", "press", "activate")):
            fallback = index
    return fallback


def perform_action(node, index):
    return bool(index is not None and attempt(lambda: node.do_action(int(index)), False))


def screen_point(window_rect, saved_element=None, x=None, y=None, node=None):
    rect = screen_rect(node) if node is not None else None
    if rect is not None:
        return rect.x + rect.width / 2, rect.y + rect.height / 2
    if saved_element is not None:
        raise RuntimeError("stale element frame; run get-app-state again and use a fresh element index")
    if window_rect is None or x is None or y is None:
        raise RuntimeError("coordinate action requires a visible window and coordinates")
    return window_rect.x + float(x), window_rect.y + float(y)


def require_positive_integer(value, name):
    try:
        parsed = int(value)
    except (TypeError, ValueError):
        raise RuntimeError(f"{name} must be a positive integer")
    if parsed <= 0:
        raise RuntimeError(f"{name} must be a positive integer")
    return parsed


def require_positive_number(value, name):
    try:
        parsed = float(value)
    except (TypeError, ValueError):
        raise RuntimeError(f"{name} must be a positive number")

View on GitHub (pinned to 1136503c6a)

Solutions

1. Provide explicit x and y coordinates (operation.x, operation.y) for coordinate-based actions when no element is referenced.
2. Ensure the target window is visible (has a screen rect) before coordinate actions — restore it if minimized.
3. Reference an element index (operation.element) instead of coordinates to let the bridge compute the center of the element's rect.
4. Validate the operation JSON includes either {element} or {x AND y} before dispatching.

Example fix

// before — click with no element and missing coordinates
{ "app": "Firefox", "tool": "click" }  // no element, no x/y

// after — supply coordinates within the visible window
{ "app": "Firefox", "tool": "click", "x": 200, "y": 300 }

# Validate the operation has a coordinate source before dispatch
def has_coordinate_source(operation: dict) -> bool:
    return operation.get('element') is not None or (
        operation.get('x') is not None and operation.get('y') is not None
    )
# usage:
if tool in {'click', 'scroll', 'drag'} and not has_coordinate_source(operation):
    raise SystemExit(f'{tool} requires either an element or x/y coordinates')

def is_valid_coordinate_op(operation: dict) -> bool:
    return operation.get('element') is not None or (
        isinstance(operation.get('x'), (int, float))
        and isinstance(operation.get('y'), (int, float))
    )

try:
    point = screen_point(bounds, saved_element, x, y, node)
except RuntimeError as exc:
    if 'coordinate action requires' in str(exc):
        raise SystemExit(f'Operation is missing an element or x/y coordinates: {operation}')
    raise

When it happens

Trigger: click/scroll/drag operation where: find_element returned None for the operation's element field (or no element was given), the window returned no screen_rect (window has no component iface / zero size), AND the operation omitted x or y (or both), or window_rect was None. Essentially an underspecified coordinate operation.

Common situations: Agent dispatched a click with neither element nor coordinates (or partial coordinates like only x); window is minimized/invisible so screen_rect(window) returned None; an operation template that defaulted x/y to null reached the bridge.

Related errors

• {name} must be a positive integer
• stale element frame; run get-app-state again and use a fresh
• Wayland GPU sandbox validation must run on Linux.
• Linux computer use requires an active desktop session; missi
• No top-level AT-SPI window is available for {app}

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/0e54ffb192386f54. Report an issue: GitHub.