What causes "Benchmark trial cleanup failed" in stablyai/orca?

Inspect AggregateError.errors to identify which specific stages failed

stablyai/orca · error · AggregateError

Benchmark trial cleanup failed

Error message

Benchmark trial cleanup failed

What it means

runBenchmarkCleanupStages executes an array of cleanup stage functions, catching each one's errors. If two or more stages throw, the errors are wrapped in an AggregateError with this message. A single stage failure re-throws that error directly.

Source

Thrown at config/scripts/macos-computer-helper-owner-loss-processes.mjs:172

    detached: true,
    killSignal: 'SIGKILL'
  })
}

export function runBenchmarkCleanupStages(stages) {
  const errors = []
  for (const stage of stages) {
    try {
      stage()
    } catch (error) {
      errors.push(error)
    }
  }
  if (errors.length === 1) {
    throw errors[0]
  }
  if (errors.length > 1) {
    throw new AggregateError(errors, 'Benchmark trial cleanup failed')
  }
}

export function throwBenchmarkTrialFailures(trialError, cleanupError) {
  if (trialError && cleanupError) {
    throw new AggregateError([trialError, cleanupError], 'Electron trial and cleanup failed')
  }
  if (trialError) {
    throw trialError
  }
  if (cleanupError) {
    throw cleanupError
  }
}

export function parseBenchmarkTrialResult(serializedResult) {
  return JSON.parse(serializedResult)
}

View on GitHub (pinned to 1136503c6a)

Solutions

Inspect AggregateError.errors to identify which specific stages failed
Run cleanup stages independently to isolate failures — some may succeed on retry after the system settles
Ensure file descriptors are closed before temp directory cleanup
Manually clean up leftover processes and temp dirs between failed runs

Example fix

// before: all-or-nothing cleanup obscures individual stage failures
try {
  runBenchmarkCleanupStages(stages)
} catch (e) {
  console.error(e.message)
}

// after: report each stage failure individually for diagnosis
try {
  runBenchmarkCleanupStages(stages)
} catch (e) {
  if (e instanceof AggregateError) {
    e.errors.forEach((err, i) => console.error(`Stage ${i}:`, err.message))
  } else {
    console.error(e.message)
  }
}

Defensive patterns

Strategy: try-catch

Try / catch

try {
  runBenchmarkCleanupStages(stages)
} catch (error) {
  if (error instanceof AggregateError) {
    // Report each stage failure — some may be benign (process already gone)
    error.errors.forEach((err, i) => {
      if (err.code === 'ESRCH') return
      console.error(`Cleanup stage ${i} failed:`, err.message)
    })
  } else {
    throw error
  }
}

Prevention

Design cleanup stages to be idempotent — re-running after partial failure should succeed
Close file descriptors before temp directory removal stages
Run stages in dependency order (kill processes before removing dirs)

When it happens

Trigger: Multiple cleanup stages each throw — e.g., killRecordedProcess fails (process already gone or signal error) AND killProcessMatchingCommand also fails (leftover processes) AND temp dir removal fails (files still open). errors.length > 1 at line 171.

Common situations: A crashed trial leaves the system in a dirty state where multiple independent cleanup steps all fail; file descriptors still open preventing temp dir deletion; processes in D state that can't be killed.

Related errors

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/496536e6f0859a09. Report an issue: GitHub.