What causes "Some benchmarks failed" in vitest-dev/vitest?

Inspect `error.errors` (the AggregateError's array) to see each individual failure and stack.

vitest-dev/vitest · error · AggregateError

Some benchmarks failed

Error message

Some benchmarks failed

What it means

The default tinybench-backed provider collects every task's error after a run. If exactly one task errored, that error is rethrown directly; if TWO or more errored, they're wrapped in an `AggregateError` with message `"Some benchmarks failed"` (see `default-provider.ts:38`). The individual causes live on `error.errors`. This surfaces from `bench().run()` or `bench.compare()` when multiple benchmark bodies throw.

Source

Thrown at packages/vitest/src/runtime/benchmark/default-provider.ts:38

        signal: test.context.signal,
        name: `${test.fullTestName} ${currentIndex}`,
        retainSamples: config.benchmark.retainSamples,
        ...options,
        now,
      })
      for (const { name, fn, fnOpts } of registrations) {
        tinybench.add(name, fn, fnOpts)
      }
      await tinybench.run()

      const errors = tinybench.tasks
        .filter(task => task.result.state === 'errored')
        .map(task => (task.result as { error: unknown }).error)
      if (errors.length === 1) {
        throw errors[0]
      }
      if (errors.length > 1) {
        throw new AggregateError(errors, 'Some benchmarks failed')
      }

      return tinybench.tasks.map(toBenchResult)
    },
  }
}

function toBenchResult(task: TinybenchTask): BenchResult {
  const result = task.result
  if (result.state !== 'completed') {
    throw new Error(`task "${task.name}" did not complete: received "${result.state}"`)
  }
  return {
    ...result,
    name: task.name,
  }
}

View on GitHub (pinned to d568f8ce37)

Solutions

Inspect `error.errors` (the AggregateError's array) to see each individual failure and stack.
Fix the first/root underlying error — often a shared dependency.
Temporarily isolate benchmarks (comment out others) to pinpoint which threw.

Defensive patterns

Strategy: try-catch

Validate before calling

// Validate shared setup/inputs before the benchmark run so individual
// benches don't all fail on the same missing dependency.
function assertBenchDeps(deps: Record<string, unknown>) {
  for (const [k, v] of Object.entries(deps)) {
    if (v == null) throw new Error(`Benchmark dependency missing: ${k}`)
  }
}
assertBench({ sortFn, dataset })

Try / catch

try {
  await bench.compare(a, b, c)
} catch (e) {
  if (e instanceof AggregateError) {
    for (const inner of e.errors) console.error(inner)
  }
  throw e
}

Prevention

Validate shared setup and dependencies before the benchmark run.
Run new benchmarks in isolation first to confirm each passes before combining.
Log AggregateError.errors to see every individual failure.

When it happens

Trigger: Two or more `bench()` functions in the same test throwing; a shared setup bug breaking every benchmark in a `compare`.

Common situations: A missing import/fixture referenced by several benchmarks; a code change that broke multiple benchmarked functions at once; an environment issue (missing env var) hitting all benchmarks.

Related errors

AI-assisted analysis of vitest-dev/vitest@d568f8ce37 (2026-08-03). Data as JSON: /data/errors/c111ad1edd9207ce.json. Report an issue: GitHub.