can1357/oh-my-pi · error

Use --cache-pairs instead of --runs with --cache

Error message

Use --cache-pairs instead of --runs with --cache

What it means

The bench CLI supports two mutually exclusive workload modes: --runs (repeat the same prompt) and --cache (cold/warm stable-prefix pairs). Passing --runs together with --cache is contradictory because cache mode ignores run counts and instead uses --cache-pairs. The CLI validates flag combinations up front and throws this error instead of silently ignoring --runs.

Source

Thrown at packages/coding-agent/src/cli/bench-cli.ts:861

function assertCacheModeSupported(targets: BenchTarget[]): void {
	if (targets.some(({ model }) => model.api === "openai-codex-responses")) {
		throw new Error(
			"--cache is not supported for openai-codex-responses because Codex WebSocket chaining cannot produce independent prompt-cache pairs",
		);
	}
}

export async function runBenchCommand(command: BenchCommandArgs, deps: BenchDependencies = {}): Promise<BenchSummary> {
	const cacheMode = command.flags.cache === true;
	const cacheFlagsUsed =
		command.flags.cachePrefixFile !== undefined ||
		command.flags.cachePrefixBytes !== undefined ||
		command.flags.cachePairs !== undefined ||
		command.flags.cacheConcurrency !== undefined;
	if (!cacheMode && cacheFlagsUsed) throw new Error("Cache flags require --cache");
	if (cacheMode && command.flags.runs !== undefined)
		throw new Error("Use --cache-pairs instead of --runs with --cache");
	if (cacheMode && command.flags.prompt !== undefined) throw new Error("--cache builds its own stable-prefix prompts");
	if (cacheMode && command.flags.profile !== undefined) throw new Error("--profile cannot be combined with --cache");
	if (cacheMode && (command.flags.par ?? 1) > 1) {
		throw new Error("--par cannot parallelize cold/warm pairs; use --cache-concurrency instead");
	}
	const profileFlag = command.flags.profile;
	if (
		profileFlag !== undefined &&
		profileFlag !== "mix" &&
		profileFlag !== "chat" &&
		profileFlag !== "prefill" &&
		profileFlag !== "generation"
	) {
		throw new Error(`Unknown --profile "${profileFlag}" (expected mix, chat, prefill, or generation)`);
	}
	const profile: BenchProfile = profileFlag ?? "mix";
	if (!cacheMode && command.flags.prompt !== undefined && profile !== "chat" && profile !== "generation") {
		throw new Error("--prompt requires --profile chat or generation");

View on GitHub (pinned to 9690622007)

Solutions

1. Remove --runs from the command when using --cache
2. Use --cache-pairs N to control how many cold/warm pairs are measured in cache mode
3. Split into two separate bench invocations if you need both runs-based and cache-based measurements

Example fix

// before
omp bench sonnet --cache --runs 5
// after
omp bench sonnet --cache --cache-pairs 5

if (args.includes("--cache") && (args.includes("--runs") || args.runs !== undefined)) throw new Error("--runs is incompatible with --cache; use --cache-pairs");

try {
  await runBench(argv);
} catch (err) {
  if (err instanceof Error && err.message.includes("instead of --runs")) {
    console.error("Remove --runs when using --cache; use --cache-pairs.");
    process.exitCode = 2;
  } else throw err;
}

When it happens

Trigger: Running `omp bench --cache --runs 5 ...` or any command where --cache is set (directly or via cache sub-flags) and command.flags.runs is also defined.

Common situations: Reusing an old non-cache bench command and adding --cache to experiment with prompt-caching benchmarks; scripted bench matrices that append --runs unconditionally; copying docs from the runs-based mode while switching to cache mode.

Related errors

• --cache builds its own stable-prefix prompts
• --profile cannot be combined with --cache
• --par cannot parallelize cold/warm pairs; use --cache-concur
• Unknown --profile "${profileFlag}" (expected mix, chat, pref
• --prompt requires --profile chat or generation

AI-assisted analysis of can1357/oh-my-pi@9690622007 (2026-08-31). Data as JSON: /api/errors/6759027511f6caef. Report an issue: GitHub.