Hmbown/CodeWhale · error · ExecError

The native helper needs an update for selected-window text…

Error message

The native helper needs an update for selected-window text recognition

What it means

Thrown in the same include_ocr path: after confirming a valid pid, the backend queries the native helper's input_capabilities and requires window_ocr === 1, meaning the installed native helper binary supports selected-window text recognition. An older or mismatched helper lacks this capability, so the backend fails loudly instead of silently OCR-ing the wrong region. It indicates a version-skew between the JS backend and its native macOS companion.

Solutions

  1. Rebuild/reinstall the native macOS helper so it matches the plugin version that declares window_ocr support.
  2. Check the helper version: call input_capabilities directly and confirm window_ocr === 1 before retrying.
  3. Clear any cached helper binary and let the backend re-extract or re-launch the current one.
  4. As a stopgap, capture without OCR and run text recognition on the PNG yourself.
Defensive patterns

Strategy: validation

Validate before calling

const caps = await native('input_capabilities');
if (caps?.window_ocr !== 1) {
  throw new Error('native helper lacks window_ocr; reinstall helper');
}
await session.capture({ app, include_ocr: true });

Type guard

const supportsWindowOcr = (caps) => caps?.window_ocr === 1;

Try / catch

try {
  await session.capture({ app, include_ocr: true });
} catch (e) {
  if (String(e.message).includes('native helper needs an update')) {
    await reinstallNativeHelper();
    return session.capture({ app, include_ocr: true });
  } else throw e;
}

Prevention

When it happens

Trigger: Any capture with include_ocr=true on macOS where the connected native helper reports input_capabilities.window_ocr !== 1 — typically an outdated helper binary, a stale cached helper, or a helper built from an older source tree.

Common situations: Upgrading the npm/plugin package without rebuilding or reinstalling the native helper; multiple helper versions on PATH; a partial build that predates the window_ocr feature flag.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/405689844205e0d0. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/plugins/computer-use/src/backends/darwin.mjs:959

        throw Object.assign(new ExecError("set_window_frame needs window_id (a non-negative window index from list_windows)"), { code: "bad_args" });
      }
      const r = await native("set_window_frame", { app_ref, window_id, frame });
      return { ...r, verified: r?.verified === true, note: r?.note ?? "the after frame is the app's own readback; cross-check with list_windows before relying on it" };
    },
    list_windows: listWindows,
    open_application: openApplication,
    get_app_state: async ({ app_ref, detail, depth, window_id, include_ocr = false, ocr_region } = {}) => {
      const t0 = Date.now();
      const t = await native("get_app_state", { app_ref: app_ref === undefined ? state.inputApp ?? undefined : app_ref, detail, window_id });
      if (process.env.CODEWHALE_CU_DEBUG_OBSERVE) console.error(`observe ${Date.now() - t0}ms elements=${t.elements?.length} truncated=${t.truncated}`);
      if (!t.found) throw new ExecError("application not found — call list_apps for exact names/pids");
      if (include_ocr) {
        // Resolve once through AX, then capture only that exact application's
        // selected window. A changing foreground cannot redirect this image.
        let raster;
        try {
          if (!Number.isSafeInteger(t.pid) || t.pid <= 0) throw new ExecError("The observed application did not provide an exact process identity for OCR");
          if ((await native("input_capabilities"))?.window_ocr !== 1) throw new ExecError("The native helper needs an update for selected-window text recognition");
          // PNG here, against the JPEG default: this raster is fed to text
          // recognition, not to a viewer, and lossless glyph edges are what
          // Vision reads. A single window is small enough that the size the
          // JPEG default exists to solve does not arise.
          const ocrDir = path.join(recordingsDir(), "captures");
          fs.mkdirSync(ocrDir, { recursive: true });
          raster = await screenshot({
            ...(ocr_region
              ? { region: ocr_region }
              : { app_ref: { pid: t.pid, ...(t.bundle_id ? { bundle_id: t.bundle_id } : {}) }, window_id }),
            path: path.join(ocrDir, `ocr-${crypto.randomBytes(4).toString("hex")}.png`),
          });
          const ocr = await native("recognize_text", { file: raster.file });
          if (ocr?.status === "ok" && ocr.pixels?.w === raster.pixels.w && ocr.pixels?.h === raster.pixels.h && Array.isArray(ocr.blocks)) {
            t.ocr = { ...ocr, raster, blocks: ocr.blocks.map(block => ({ ...block, target: {
              type: "coordinate", x: Math.floor(block.bounds.x + block.bounds.w / 2), y: Math.floor(block.bounds.y + block.bounds.h / 2),
            } })) };
          } else {

View on GitHub (pinned to 73e0f67d83)