stablyai/orca · error · Error

Commit or stash changes before running the provenance-bound

Error message

Commit or stash changes before running the provenance-bound benchmark

What it means

The benchmark records SHA256 hashes of source files and build artifacts for provenance (report.sources, report.artifacts at lines 558-567) tied to the git revision. A dirty working tree means the recorded hashes will not match the committed revision, breaking reproducibility. The check runs `git status --porcelain` and throws if any output is produced.

Source

Thrown at config/scripts/macos-computer-helper-owner-loss-benchmark.mjs:536

  }
  if (!trialError && !serializedResult) {
    trialError = new Error(`Electron trial did not write a result:\n${trialOutput}`)
  }
  throwBenchmarkTrialFailures(trialError, cleanupError)
  return parsedResult
}

function runBenchmark() {
  if (process.platform !== 'darwin') {
    throw new Error('The computer-use helper owner benchmark is macOS-only')
  }
  const options = parseArgs(process.argv.slice(2))
  const dirty = execFileSync('git', ['status', '--porcelain'], {
    cwd: repoRoot,
    encoding: 'utf8'
  }).trim()
  if (dirty) {
    throw new Error('Commit or stash changes before running the provenance-bound benchmark')
  }
  buildArtifacts()
  if (!existsSync(sidecarPath) || !existsSync(helperPath)) {
    throw new Error('Fresh production sidecar/helper build did not produce the expected artifacts')
  }
  const executable = electronPath()
  const results = Array.from({ length: options.trials }, () => runTrial(executable, options.expect))
  const rssBytes = results.map((result) => result.connectedRssBytes)
  const cpuMilliseconds = results.map((result) => result.connectedCpuMilliseconds)
  const activeRequestTotals = results.map((result) => result.activeRequests.totalMs)
  const activeRequestRates = results.map((result) => result.activeRequests.requestsPerSecond)
  const activeRequestMedians = results.map((result) => result.activeRequests.medianLatencyMs)
  const activeRequestP95s = results.map((result) => result.activeRequests.p95LatencyMs)
  const activeRequestMaxes = results.map((result) => result.activeRequests.maxLatencyMs)
  const postLossRssBytes = results.map((result) => result.postLossRssBytes)
  const report = {
    benchmark: 'macos-computer-helper-authenticated-owner-loss',
    revision: execFileSync('git', ['rev-parse', 'HEAD'], {

View on GitHub (pinned to 1136503c6a)

Solutions

1. Commit or stash all changes: `git stash` or `git add -A && git commit`
2. Add generated/build artifacts to .gitignore if they appear as modifications
3. Run `git status --porcelain` yourself before invoking to identify dirty files

Example fix

# before: running benchmark with uncommitted changes
git status --porcelain  # shows modified files

# after: clean working tree
git stash
# or
git add -A && git commit -m 'wip'
node benchmark.mjs --expect reaped

// Check git cleanliness before running
const dirty = execFileSync('git', ['status', '--porcelain'], { encoding: 'utf8' }).trim()
if (dirty) {
  throw new Error(`Working tree is dirty:\n${dirty}\nCommit or stash before running the benchmark.`)
}

When it happens

Trigger: git status --porcelain returns non-empty output at line 531 — any untracked, modified, or staged file in the repo triggers it.

Common situations: Generated files from a previous build showing as modified; untracked temp files; developer running the benchmark with local edits; build artifacts checked into git that differ after rebuild.

Related errors

• ${name} must be a positive integer, received ${value}
• resolved values differ for ${baseRef}:\n serial ${JSON.
• no benchmark file exists at both HEAD and HEAD~1 in this che
• advertised-url-watcher.ts no longer contains \`${marker}\`
• ${name} must be a positive integer, received ${value}

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/7ea659a580a537e2. Report an issue: GitHub.