{"record":{"id":"f2bc3087e8cfc415","repo":"Hmbown/CodeWhale","slug":"low-level-press-release-is-not-exposed-by-uitest-uiinput-use","errorCode":null,"errorMessage":"low-level press/release is not exposed by uitest uiInput; use left_click_drag","messagePattern":"low-level press/release is not exposed by uitest uiInput; use left_click_drag","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/harmonyos.mjs","lineNumber":229,"sourceCode":"      const buf = fs.readFileSync(file);\n      displayPixels = jpegSize(buf) ?? displayPixels;\n      return { file, bytes: buf.length, pixels: jpegSize(buf), scale: 1, points: jpegSize(buf) };\n    },\n    zoom: async ({ region, path: outPath }) => {\n      throw new ExecError(\"zoom is not supported on the harmony backend yet — screenshot + region on the host is the workaround\");\n    },\n    left_click: ({ target, strategy }) => { assertEventStrategy(strategy); return uiInput([\"click\", String(Math.round(target.x)), String(Math.round(target.y))]); },\n    double_click: ({ target }) => uiInput([\"doubleClick\", String(Math.round(target.x)), String(Math.round(target.y))]),\n    triple_click: async ({ target }) => {\n      await uiInput([\"doubleClick\", String(Math.round(target.x)), String(Math.round(target.y))]);\n      return uiInput([\"click\", String(Math.round(target.x)), String(Math.round(target.y))]);\n    },\n    right_click: async () => { throw new ExecError(\"uitest uiInput has no right-click; use longClick semantics via hold or left_click\"); },\n    middle_click: async () => { throw new ExecError(\"middle click is not exposed by uitest uiInput\"); },\n    mouse_move: async () => ({ action_sent: false, note: \"hover without press is not exposed by uitest uiInput\" }),\n    left_click_drag: ({ from_target: from, to }) =>\n      uiInput([\"swipe\", String(Math.round(from.x)), String(Math.round(from.y)), String(Math.round(to.x)), String(Math.round(to.y)), \"200\"], { timeoutMs: 30_000 }),\n    left_mouse_down: async () => { throw new ExecError(\"low-level press/release is not exposed by uitest uiInput; use left_click_drag\"); },\n    left_mouse_up: async () => { throw new ExecError(\"low-level press/release is not exposed by uitest uiInput; use left_click_drag\"); },\n    scroll: ({ target, direction = \"down\", amount = 300 }) => {\n      const dist = Math.max(60, Math.min(1200, amount * 24));\n      const dx = direction === \"left\" ? dist : direction === \"right\" ? -dist : 0;\n      const dy = direction === \"up\" ? dist : direction === \"down\" ? -dist : 0;\n      return uiInput([\"swipe\", String(Math.round(target.x)), String(Math.round(target.y)), String(Math.round(target.x + dx)), String(Math.round(target.y + dy)), \"400\"]);\n    },\n    type: async ({ text }) => {\n      if (!text) return { action_sent: false, note: \"empty text\" };\n      await uiInput([\"inputText\", \"300\", \"300\", escDeviceText(text)]).catch(async (e) => {\n        // Some builds require coordinates of the focused field; retry with a click-first pattern.\n        throw e;\n      });\n      return { action_sent: true, chars: text.length, note: \"inputText at 300,300 — click the field first for focused input\" };\n    },\n    key: ({ text }) => {\n      const KEYMAP = { enter: \"Enter\", return: \"Enter\", escape: \"Esc\", esc: \"Esc\", back: \"Back\", home: \"Home\", backspace: \"Back\", delete: \"Del\", tab: \"Tab\", left: \"DPAD_LEFT\", right: \"DPAD_RIGHT\", up: \"DPAD_UP\", down: \"DPAD_DOWN\", power: \"Power\", menu: \"Menu\" };\n      const k = KEYMAP[String(text).toLowerCase()] ?? String(text);","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/harmonyos.mjs#L211-L247","documentation":"The harmony backend exposes drags only through uitest's `swipe` sub-command (left_click_drag). There is no press-and-hold primitive, so `left_mouse_down` throws this ExecError directing callers to left_click_drag. It is an intentional fail-fast for the press half of the low-level mouse API.","triggerScenarios":"Calling the left_mouse_down action ({ action: \"left_mouse_down\" }) on the harmony backend from harmonyos.mjs; custom drag loops that sequence left_mouse_down → mouse_move → left_mouse_up.","commonSituations":"Porting a desktop drag implementation that builds drags from raw down/move/up events; agents emitting granular mouse primitives; interaction frameworks that assume a full low-level mouse API on every backend.","solutions":["Rewrite the gesture as a single left_click_drag({ from_target, to }) — the backend implements it as uitest swipe and is the fix the message names.","For press-and-hold semantics (not movement), use perform_action({ target, action: \"longClick\" }) or hold_key({ text: \"click\" }).","Buffer intermediate move points and collapse the down/move/up sequence into one drag call before dispatching to a harmony backend.","If fine-grained path control is required, chain multiple left_click_drag calls along the path, or add a press/release primitive to harmonyos.mjs only if uitest grows one."],"exampleFix":"// before\nawait backend.left_mouse_down({ target: from });\nawait backend.mouse_move({ target: to });\nawait backend.left_mouse_up({});\n\n// after\nawait backend.left_click_drag({ from_target: from, to });","handlingStrategy":"fallback","validationCode":"const supportsLowLevelMouse = (backend) => backend.kind !== \"harmonyos\";\nif (!supportsLowLevelMouse(backend)) {\n  throw new Error(\"use left_click_drag on harmony; no press primitive\");\n}","typeGuard":"const hasPressPrimitive = (backend) => typeof backend.left_click_drag === \"function\" && backend.kind !== \"harmonyos\";","tryCatchPattern":"try {\n  await backend.left_mouse_down({ target });\n} catch (e) {\n  if (/low-level press\\/release is not exposed/.test(e.message)) {\n    // redo the gesture as an atomic drag instead\n    await backend.left_click_drag({ from_target: from, to });\n  } else {\n    throw e;\n  }\n}","preventionTips":["Never build drags from down/move/up primitives when targeting HarmonyOS; emit left_click_drag directly.","Keep a capability flag (e.g. lowLevelMouse: false) alongside each backend and branch on it.","Detect the error once and remember the degraded capability for the session instead of retrying.","Test drag interactions on every backend your scripts support."],"tags":["harmonyos","mouse-down","uitest","drag","computer-use"],"backgroundTag":"unsupported-operation","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-22T21:17:16.096Z"}