{"record":{"id":"0e54ffb192386f54","repo":"stablyai/orca","slug":"coordinate-action-requires-a-visible-window-and-co","errorCode":null,"errorMessage":"coordinate action requires a visible window and coordinates","messagePattern":"coordinate action requires a visible window and coordinates","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"native/computer-use-linux/runtime.py","lineNumber":798,"sourceCode":"        if label in priority:\n            return index\n        if fallback is None and any(term in label for term in (\"click\", \"press\", \"activate\")):\n            fallback = index\n    return fallback\n\n\ndef perform_action(node, index):\n    return bool(index is not None and attempt(lambda: node.do_action(int(index)), False))\n\n\ndef screen_point(window_rect, saved_element=None, x=None, y=None, node=None):\n    rect = screen_rect(node) if node is not None else None\n    if rect is not None:\n        return rect.x + rect.width / 2, rect.y + rect.height / 2\n    if saved_element is not None:\n        raise RuntimeError(\"stale element frame; run get-app-state again and use a fresh element index\")\n    if window_rect is None or x is None or y is None:\n        raise RuntimeError(\"coordinate action requires a visible window and coordinates\")\n    return window_rect.x + float(x), window_rect.y + float(y)\n\n\ndef require_positive_integer(value, name):\n    try:\n        parsed = int(value)\n    except (TypeError, ValueError):\n        raise RuntimeError(f\"{name} must be a positive integer\")\n    if parsed <= 0:\n        raise RuntimeError(f\"{name} must be a positive integer\")\n    return parsed\n\n\ndef require_positive_number(value, name):\n    try:\n        parsed = float(value)\n    except (TypeError, ValueError):\n        raise RuntimeError(f\"{name} must be a positive number\")","sourceCodeStart":780,"sourceCodeEnd":816,"githubUrl":"https://github.com/stablyai/orca/blob/1136503c6a231a16dce8f921f6fadb63d181e8db/native/computer-use-linux/runtime.py#L780-L816","documentation":"Raised by screen_point (runtime.py:797-798) when none of the resolution paths are available: there is no live node rect (node is None or returned None), no saved_element to flag as stale, AND the window_rect/x/y fallback is incomplete (window_rect is None, or x is None, or y is None). The function cannot compute a screen coordinate from any source, so it refuses. This indicates the caller neither supplied an element reference nor complete explicit coordinates within a visible window.","triggerScenarios":"click/scroll/drag operation where: find_element returned None for the operation's element field (or no element was given), the window returned no screen_rect (window has no component iface / zero size), AND the operation omitted x or y (or both), or window_rect was None. Essentially an underspecified coordinate operation.","commonSituations":"Agent dispatched a click with neither element nor coordinates (or partial coordinates like only x); window is minimized/invisible so screen_rect(window) returned None; an operation template that defaulted x/y to null reached the bridge.","solutions":["Provide explicit x and y coordinates (operation.x, operation.y) for coordinate-based actions when no element is referenced.","Ensure the target window is visible (has a screen rect) before coordinate actions — restore it if minimized.","Reference an element index (operation.element) instead of coordinates to let the bridge compute the center of the element's rect.","Validate the operation JSON includes either {element} or {x AND y} before dispatching."],"exampleFix":"// before — click with no element and missing coordinates\n{ \"app\": \"Firefox\", \"tool\": \"click\" }  // no element, no x/y\n\n// after — supply coordinates within the visible window\n{ \"app\": \"Firefox\", \"tool\": \"click\", \"x\": 200, \"y\": 300 }","handlingStrategy":"validation","validationCode":"# Validate the operation has a coordinate source before dispatch\ndef has_coordinate_source(operation: dict) -> bool:\n    return operation.get('element') is not None or (\n        operation.get('x') is not None and operation.get('y') is not None\n    )\n# usage:\nif tool in {'click', 'scroll', 'drag'} and not has_coordinate_source(operation):\n    raise SystemExit(f'{tool} requires either an element or x/y coordinates')","typeGuard":"def is_valid_coordinate_op(operation: dict) -> bool:\n    return operation.get('element') is not None or (\n        isinstance(operation.get('x'), (int, float))\n        and isinstance(operation.get('y'), (int, float))\n    )","tryCatchPattern":"try:\n    point = screen_point(bounds, saved_element, x, y, node)\nexcept RuntimeError as exc:\n    if 'coordinate action requires' in str(exc):\n        raise SystemExit(f'Operation is missing an element or x/y coordinates: {operation}')\n    raise","preventionTips":["Always provide either operation.element OR both operation.x and operation.y for coordinate actions.","Ensure the target window is visible (restore if minimized) so screen_rect(window) is not None.","Validate the operation JSON on the caller side before dispatching."],"tags":["linux","at-spi","coordinate","validation","input"],"backgroundTag":null,"analyzedSha":"1136503c6a231a16dce8f921f6fadb63d181e8db","analyzedAt":"2026-08-12T23:15:58.167Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}