vitest-dev/vitest · error · SyntaxError

`bench.compare()` expects every argument to be the return va

Error message

`bench.compare()` expects every argument to be the return value of `bench` or `bench.from`.

What it means

Every positional argument to `bench.compare()` must be a `BenchRegistration` returned by `bench()` or `bench.from()` (detected via the internal `kRegistration` symbol). This `SyntaxError` (see `benchmark.ts:479`) fires when any argument is `null`, not an object, or lacks the symbol — e.g. passing a raw function, a string name, a `BenchResult`, or the result of `.run()`.

Source

Thrown at packages/vitest/src/runtime/benchmark.ts:479

    const isOptions = lastArg != null && typeof lastArg === 'object' && !(kRegistration in lastArg)
    const benchOptions = isOptions ? args.pop() as BenchRunOptions : undefined
    const registrations = args as BenchRegistration<any>[]

    // Mark every passed-in registration as consumed before validation so a
    // throwing `bench.compare()` (wrong arity, wrong shape) doesn't also
    // trigger the unrun-bench warning — the user's intent was to consume them.
    for (const reg of registrations) {
      if (reg != null && typeof reg === 'object' && kRegistration in reg) {
        pending.delete(reg)
      }
    }

    if (registrations.length < 2) {
      throw new SyntaxError(`\`bench.compare()\` requires at least 2 benchmarks, received ${registrations.length} instead. ${registrations.length === 1 ? 'Consider calling `bench().run()`. ' : 'Define benchmarks by calling `bench()`. '}See https://vitest.dev/guide/benchmarking#comparing-benchmarks`)
    }
    for (const reg of registrations) {
      if (reg == null || typeof reg !== 'object' || !(kRegistration in reg)) {
        throw new SyntaxError('`bench.compare()` expects every argument to be the return value of `bench` or `bench.from`.')
      }
    }

    const runnable: RunnableRegistration<any>[] = []
    const fromEntries: FromRegistration<any>[] = []
    for (const reg of registrations) {
      if (isFromRegistration(reg)) {
        fromEntries.push(reg)
      }
      else {
        runnable.push(reg as RunnableRegistration<any>)
      }
    }

    const taskMeta = new Map<string, TaskMeta>()
    for (const reg of runnable) {
      if (reg[kPerProject]) {
        taskMeta.set(reg.name, { perProject: true })

View on GitHub (pinned to d568f8ce37)

Solutions

1. Pass the raw registration objects, not their results: `const a = bench('a', fn); await bench.compare(a, b)`.
2. Ensure no `.run()` is called on registrations you intend to compare — `compare` runs them itself.
3. Filter out nullish entries before spreading: `bench.compare(...regs.filter(Boolean))`.

Example fix

// before
await bench.compare(
  bench('a', fnA).run(),
  bench.from('b', path)
)

// after
const a = bench('a', fnA)
const b = bench.from('b', path)
await bench.compare(a, b)

// Ensure every arg is a non-null object produced by bench()/bench.from().
// The internal kRegistration symbol isn't user-accessible, so infer by shape:
function looksLikeRegistration(v: unknown): v is { run: Function; name: string } {
  return v != null && typeof v === 'object'
    && typeof (v as any).run === 'function'
    && typeof (v as any).name === 'string'
}
const clean = args.filter(looksLikeRegistration)
if (clean.length !== args.length) {
  throw new Error('One or more bench.compare args are not registrations')
}
await bench.compare(...clean)

function isBenchRegistrationLike(v: unknown): v is { name: string; run: (o?: any) => Promise<any> } {
  return v != null && typeof v === 'object'
    && typeof (v as any).name === 'string'
    && typeof (v as any).run === 'function'
}

try {
  await bench.compare(...regs)
} catch (e) {
  if (e instanceof SyntaxError && /expects every argument/.test(e.message)) {
    // remove any non-registration entries and retry
  } else throw e
}

When it happens

Trigger: Passing `bench('a', fn).run()` (which is a `Promise<BenchResult>`) instead of the registration; passing a string name; passing `undefined`; passing a plain function.

Common situations: Accidentally awaiting/calling `.run()` before `compare`; destructuring the wrong variable; mixing up the registration object with its result.

Related errors

• `bench.compare()` requires at least 2 benchmarks, received $
• task "${name}" was not defined
• `bench.from()` requires a name (string or named function) as
• `bench.from()` expects a string path or a function returning
• benchmark provider did not return a result for "${name}"

AI-assisted analysis of vitest-dev/vitest@d568f8ce37 (2026-08-03). Data as JSON: /data/errors/250a7657136d639c.json. Report an issue: GitHub.