stablyai/orca · error · Error

Usage: node config/scripts/compare-benchmark-artifacts.mjs -

Error message

Usage: node config/scripts/compare-benchmark-artifacts.mjs --baseline <path> --candidate <path> [--title <title>] [--output <path>] [--json-output <path>] [--higher-is-better <metric-key> ...]

What it means

Thrown by parseBenchmarkComparisonArgs as the USAGE message in two cases: (1) an unrecognized flag was passed (the final `throw new Error(USAGE)` in the loop), or (2) --baseline or --candidate was not provided (the two post-loop checks). The full usage string documents every accepted flag.

Source

Thrown at config/scripts/compare-benchmark-artifacts.mjs:52

      continue
    }
    if (flag === '--json-output') {
      parsed.jsonOutputPath = readRequiredValue(args, ++index, '--json-output requires a path')
      continue
    }
    if (flag === '--higher-is-better') {
      let consumed = 0
      while (args[index + 1] != null && !args[index + 1].startsWith('--')) {
        parsed.higherIsBetter.add(args[index + 1])
        index += 1
        consumed += 1
      }
      if (consumed === 0) {
        throw new Error('--higher-is-better requires a metric key')
      }
      continue
    }
    throw new Error(USAGE)
  }

  if (!parsed.baselinePath) {
    throw new Error(USAGE)
  }
  if (!parsed.candidatePath) {
    throw new Error(USAGE)
  }
  return parsed
}

function readRequiredValue(args, index, message) {
  const value = args[index]
  if (value == null || value.startsWith('--')) {
    throw new Error(message)
  }
  return value
}

View on GitHub (pinned to 1136503c6a)

Solutions

1. Provide both required flags: --baseline <baseline.json> --candidate <candidate.json>.
2. Remove any unrecognized flags or check the spelling against the USAGE line.
3. Run the script with no args to print USAGE intentionally, then construct the real command.
4. In CI, assert both $BASELINE and $CANDIDATE are non-empty before invoking.

Example fix

// before
// node config/scripts/compare-benchmark-artifacts.mjs --base a.json --candidate b.json
// -> 'Usage: node config/scripts/compare-benchmark-artifacts.mjs --baseline <path> ...'

// after
// node config/scripts/compare-benchmark-artifacts.mjs --baseline a.json --candidate b.json

function assertRequiredArgs(parsed) {
  if (!parsed.baselinePath || !parsed.candidatePath) {
    throw new Error(USAGE)
  }
}
// call after parseBenchmarkComparisonArgs

try {
  runBenchmarkComparisonCli()
} catch (error) {
  console.error(error.message)
  process.exit(1)
}

When it happens

Trigger: Passing an unknown flag like --verbose or --format; misspelling --baseline as --base; omitting --baseline and/or --candidate entirely; passing --baseline without a value (caught earlier by readRequiredValue but a missing required flag falls through here).

Common situations: Operator ran the script with no args expecting a help message; a CI workflow template lost its --baseline substitution; copy-paste from an older version of the script that accepted different flags.

Related errors

• --higher-is-better requires a metric key
• Missing value for ${arg}
• Unknown argument: ${arg}
• --boundary must be child or worker
• --trials must be a positive integer

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/b461138866612326. Report an issue: GitHub.