{"record":{"id":"405689844205e0d0","repo":"Hmbown/CodeWhale","slug":"the-native-helper-needs-an-update-for-selected-window-text","errorCode":null,"errorMessage":"The native helper needs an update for selected-window text recognition","messagePattern":"The native helper needs an update for selected-window text recognition","errorType":"exception","errorClass":"ExecError","httpStatus":null,"severity":"error","filePath":"crates/tui/plugins/computer-use/src/backends/darwin.mjs","lineNumber":959,"sourceCode":"        throw Object.assign(new ExecError(\"set_window_frame needs window_id (a non-negative window index from list_windows)\"), { code: \"bad_args\" });\n      }\n      const r = await native(\"set_window_frame\", { app_ref, window_id, frame });\n      return { ...r, verified: r?.verified === true, note: r?.note ?? \"the after frame is the app's own readback; cross-check with list_windows before relying on it\" };\n    },\n    list_windows: listWindows,\n    open_application: openApplication,\n    get_app_state: async ({ app_ref, detail, depth, window_id, include_ocr = false, ocr_region } = {}) => {\n      const t0 = Date.now();\n      const t = await native(\"get_app_state\", { app_ref: app_ref === undefined ? state.inputApp ?? undefined : app_ref, detail, window_id });\n      if (process.env.CODEWHALE_CU_DEBUG_OBSERVE) console.error(`observe ${Date.now() - t0}ms elements=${t.elements?.length} truncated=${t.truncated}`);\n      if (!t.found) throw new ExecError(\"application not found — call list_apps for exact names/pids\");\n      if (include_ocr) {\n        // Resolve once through AX, then capture only that exact application's\n        // selected window. A changing foreground cannot redirect this image.\n        let raster;\n        try {\n          if (!Number.isSafeInteger(t.pid) || t.pid <= 0) throw new ExecError(\"The observed application did not provide an exact process identity for OCR\");\n          if ((await native(\"input_capabilities\"))?.window_ocr !== 1) throw new ExecError(\"The native helper needs an update for selected-window text recognition\");\n          // PNG here, against the JPEG default: this raster is fed to text\n          // recognition, not to a viewer, and lossless glyph edges are what\n          // Vision reads. A single window is small enough that the size the\n          // JPEG default exists to solve does not arise.\n          const ocrDir = path.join(recordingsDir(), \"captures\");\n          fs.mkdirSync(ocrDir, { recursive: true });\n          raster = await screenshot({\n            ...(ocr_region\n              ? { region: ocr_region }\n              : { app_ref: { pid: t.pid, ...(t.bundle_id ? { bundle_id: t.bundle_id } : {}) }, window_id }),\n            path: path.join(ocrDir, `ocr-${crypto.randomBytes(4).toString(\"hex\")}.png`),\n          });\n          const ocr = await native(\"recognize_text\", { file: raster.file });\n          if (ocr?.status === \"ok\" && ocr.pixels?.w === raster.pixels.w && ocr.pixels?.h === raster.pixels.h && Array.isArray(ocr.blocks)) {\n            t.ocr = { ...ocr, raster, blocks: ocr.blocks.map(block => ({ ...block, target: {\n              type: \"coordinate\", x: Math.floor(block.bounds.x + block.bounds.w / 2), y: Math.floor(block.bounds.y + block.bounds.h / 2),\n            } })) };\n          } else {","sourceCodeStart":941,"sourceCodeEnd":977,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/73e0f67d83c59909b571efdfc88c4bc28c309cb1/crates/tui/plugins/computer-use/src/backends/darwin.mjs#L941-L977","documentation":"Thrown in the same include_ocr path: after confirming a valid pid, the backend queries the native helper's input_capabilities and requires window_ocr === 1, meaning the installed native helper binary supports selected-window text recognition. An older or mismatched helper lacks this capability, so the backend fails loudly instead of silently OCR-ing the wrong region. It indicates a version-skew between the JS backend and its native macOS companion.","triggerScenarios":"Any capture with include_ocr=true on macOS where the connected native helper reports input_capabilities.window_ocr !== 1 — typically an outdated helper binary, a stale cached helper, or a helper built from an older source tree.","commonSituations":"Upgrading the npm/plugin package without rebuilding or reinstalling the native helper; multiple helper versions on PATH; a partial build that predates the window_ocr feature flag.","solutions":["Rebuild/reinstall the native macOS helper so it matches the plugin version that declares window_ocr support.","Check the helper version: call input_capabilities directly and confirm window_ocr === 1 before retrying.","Clear any cached helper binary and let the backend re-extract or re-launch the current one.","As a stopgap, capture without OCR and run text recognition on the PNG yourself."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"const caps = await native('input_capabilities');\nif (caps?.window_ocr !== 1) {\n  throw new Error('native helper lacks window_ocr; reinstall helper');\n}\nawait session.capture({ app, include_ocr: true });","typeGuard":"const supportsWindowOcr = (caps) => caps?.window_ocr === 1;","tryCatchPattern":"try {\n  await session.capture({ app, include_ocr: true });\n} catch (e) {\n  if (String(e.message).includes('native helper needs an update')) {\n    await reinstallNativeHelper();\n    return session.capture({ app, include_ocr: true });\n  } else throw e;\n}","preventionTips":["Reinstall/rebuild the native helper whenever the plugin package version changes.","Probe input_capabilities once at session start and fail fast on missing features.","Pin plugin and helper versions together in your install scripts."],"tags":["macos","ocr","native-helper","version-skew"],"backgroundTag":"missing-dependency","analyzedSha":"73e0f67d83c59909b571efdfc88c4bc28c309cb1","analyzedAt":"2026-09-22T01:30:00.501Z","contentChangedAt":"2026-09-22T01:30:00.501Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}