stablyai/orca · error · Error

test:e2e:terminal-perf:scale:report always uses --reporter=j

Error message

test:e2e:terminal-perf:scale:report always uses --reporter=json

What it means

The scale perf report gate always runs the underlying test with --reporter=json because it post-processes the JSON report to apply gating thresholds. Letting the caller override --reporter (e.g. to dot or html) would break the downstream parsing, so the parser rejects any --reporter or --reporter=... arg up front.

Source

Thrown at config/scripts/run-terminal-scale-perf-report-gate.mjs:37

    if (arg === '--report' || arg === '--report-path' || arg === '--output') {
      const next = forwardedArgs[index + 1]
      if (!next || next.startsWith('-')) {
        throw new Error(`${arg} requires a path`)
      }
      reportPath = next
      index += 1
      continue
    }
    if (
      arg.startsWith('--report=') ||
      arg.startsWith('--report-path=') ||
      arg.startsWith('--output=')
    ) {
      reportPath = arg.slice(arg.indexOf('=') + 1)
      continue
    }
    if (arg === '--reporter' || arg.startsWith('--reporter=')) {
      throw new Error('test:e2e:terminal-perf:scale:report always uses --reporter=json')
    }
    passthroughArgs.push(arg)
  }

  return { passthroughArgs, reportPath }
}

function runNodeScript(scriptPath, args, stdio, spawnSyncImpl, env) {
  return spawnSyncImpl(process.execPath, [scriptPath, ...args], {
    cwd: process.cwd(),
    env,
    stdio
  })
}

function exitCode(result) {
  if (result.signal) {
    console.error(`Terminal scale perf command exited with signal ${result.signal}`)

View on GitHub (pinned to 1136503c6a)

Solutions

Remove the --reporter arg — the gate must use JSON internally. To change human-readable output, configure it in playwright.config.ts reporters (the JSON reporter is layered by the gate, not the only one).
If you genuinely need a different reporter, invoke the underlying test:e2e:terminal-perf:scale task directly (not the :report gate) and run the gate logic separately.

Example fix

// before
pnpm run test:e2e:terminal-perf:scale:report -- --reporter=dot
// after
pnpm run test:e2e:terminal-perf:scale:report

Defensive patterns

Strategy: validation

Validate before calling

if (args.some((a) => a === '--reporter' || a.startsWith('--reporter='))) {
  console.error('This gate always uses --reporter=json; remove --reporter'); process.exit(2)
}

Prevention

Do not forward --reporter to the gate; the JSON reporter is required for threshold parsing.
Configure human-readable reporters in playwright.config.ts instead.
Document the gate's fixed reporter in the package.json script comment.

When it happens

Trigger: Passing `--reporter=dot`, `--reporter html`, or `--reporter` to the gate script. The parser detects the arg literally and aborts before forwarding it.

Common situations: A developer wants prettier CI output and adds --reporter=dot; a wrapper script blindly forwards all Playwright args; copy-paste from another Playwright invocation.

Related errors

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/4a4af3a91f18fdab. Report an issue: GitHub.