stablyai/orca · error · Error

The computer-use helper owner benchmark is macOS-only

Error message

The computer-use helper owner benchmark is macOS-only

What it means

Platform guard at the top of runBenchmark(). The computer-use helper under test is a native macOS .app bundle (Orca Computer Use.app), and the benchmark uses macOS-specific process inspection (ps eww, signal semantics). Running on Linux or Windows has no helper binary to test.

Solutions

1. Run the benchmark on macOS only
2. Gate the benchmark in CI with a platform condition (e.g., only run on macos-latest)
3. Add a wrapper that skips gracefully on non-darwin instead of throwing

Example fix

// before: unconditional invocation in CI matrix
- run: node config/scripts/macos-computer-helper-owner-loss-benchmark.mjs

// after: gate by platform
- run: node config/scripts/macos-computer-helper-owner-loss-benchmark.mjs
  if: runner.os == 'macOS'

// Check platform before invoking the benchmark
if (process.platform !== 'darwin') {
  console.warn('Skipping macOS-only benchmark on', process.platform)
  process.exit(0)
}

When it happens

Trigger: process.platform is not 'darwin' (e.g., 'linux' or 'win32') when runBenchmark() is called.

Common situations: Running the benchmark in Linux/Windows CI; developer accidentally invoking the script on a non-Mac machine; Docker container without macOS.

Related errors

• emulator_not_macos
• The production watchdog is macOS-only; run this benchmark…
• accessibility_error
• accessibility_error
• accessibility_error

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/d73a566a430b8a49. Report an issue: GitHub.

Appendix: source

Thrown at config/scripts/macos-computer-helper-owner-loss-benchmark.mjs:528

    })
    cleanupError = cleanup.error
    trialOutput = cleanup.output
  }
  if (!trialError && result?.status !== 0) {
    trialError = new Error(
      `Electron trial failed (${result.error?.message ?? result.signal ?? result.status}):\n${trialOutput}`
    )
  }
  if (!trialError && !serializedResult) {
    trialError = new Error(`Electron trial did not write a result:\n${trialOutput}`)
  }
  throwBenchmarkTrialFailures(trialError, cleanupError)
  return parsedResult
}

function runBenchmark() {
  if (process.platform !== 'darwin') {
    throw new Error('The computer-use helper owner benchmark is macOS-only')
  }
  const options = parseArgs(process.argv.slice(2))
  const dirty = execFileSync('git', ['status', '--porcelain'], {
    cwd: repoRoot,
    encoding: 'utf8'
  }).trim()
  if (dirty) {
    throw new Error('Commit or stash changes before running the provenance-bound benchmark')
  }
  buildArtifacts()
  if (!existsSync(sidecarPath) || !existsSync(helperPath)) {
    throw new Error('Fresh production sidecar/helper build did not produce the expected artifacts')
  }
  const executable = electronPath()
  const results = Array.from({ length: options.trials }, () => runTrial(executable, options.expect))
  const rssBytes = results.map((result) => result.connectedRssBytes)
  const cpuMilliseconds = results.map((result) => result.connectedCpuMilliseconds)
  const activeRequestTotals = results.map((result) => result.activeRequests.totalMs)

View on GitHub (pinned to 1136503c6a)