stablyai/orca · error · Error

The computer-use helper owner benchmark is macOS-only

Error message

The computer-use helper owner benchmark is macOS-only

What it means

Platform guard at the top of runBenchmark(). The computer-use helper under test is a native macOS .app bundle (Orca Computer Use.app), and the benchmark uses macOS-specific process inspection (ps eww, signal semantics). Running on Linux or Windows has no helper binary to test.

Source

Thrown at config/scripts/macos-computer-helper-owner-loss-benchmark.mjs:528

    })
    cleanupError = cleanup.error
    trialOutput = cleanup.output
  }
  if (!trialError && result?.status !== 0) {
    trialError = new Error(
      `Electron trial failed (${result.error?.message ?? result.signal ?? result.status}):\n${trialOutput}`
    )
  }
  if (!trialError && !serializedResult) {
    trialError = new Error(`Electron trial did not write a result:\n${trialOutput}`)
  }
  throwBenchmarkTrialFailures(trialError, cleanupError)
  return parsedResult
}

function runBenchmark() {
  if (process.platform !== 'darwin') {
    throw new Error('The computer-use helper owner benchmark is macOS-only')
  }
  const options = parseArgs(process.argv.slice(2))
  const dirty = execFileSync('git', ['status', '--porcelain'], {
    cwd: repoRoot,
    encoding: 'utf8'
  }).trim()
  if (dirty) {
    throw new Error('Commit or stash changes before running the provenance-bound benchmark')
  }
  buildArtifacts()
  if (!existsSync(sidecarPath) || !existsSync(helperPath)) {
    throw new Error('Fresh production sidecar/helper build did not produce the expected artifacts')
  }
  const executable = electronPath()
  const results = Array.from({ length: options.trials }, () => runTrial(executable, options.expect))
  const rssBytes = results.map((result) => result.connectedRssBytes)
  const cpuMilliseconds = results.map((result) => result.connectedCpuMilliseconds)
  const activeRequestTotals = results.map((result) => result.activeRequests.totalMs)

View on GitHub (pinned to 1136503c6a)

Solutions

Run the benchmark on macOS only
Gate the benchmark in CI with a platform condition (e.g., only run on macos-latest)
Add a wrapper that skips gracefully on non-darwin instead of throwing

Example fix

// before: unconditional invocation in CI matrix
- run: node config/scripts/macos-computer-helper-owner-loss-benchmark.mjs

// after: gate by platform
- run: node config/scripts/macos-computer-helper-owner-loss-benchmark.mjs
  if: runner.os == 'macOS'

Defensive patterns

Strategy: validation

Validate before calling

// Check platform before invoking the benchmark
if (process.platform !== 'darwin') {
  console.warn('Skipping macOS-only benchmark on', process.platform)
  process.exit(0)
}

Prevention

Gate benchmark execution in CI with a platform condition (e.g., if: runner.os == 'macOS')
Wrap the invocation in a platform check that skips gracefully rather than throwing
Document the macOS requirement prominently

When it happens

Trigger: process.platform is not 'darwin' (e.g., 'linux' or 'win32') when runBenchmark() is called.

Common situations: Running the benchmark in Linux/Windows CI; developer accidentally invoking the script on a non-Mac machine; Docker container without macOS.

Related errors

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/d73a566a430b8a49. Report an issue: GitHub.