What causes "Model must be provider/model, got ${value}" in can1357/oh-my-pi?

Format the model as provider/model, e.g. openai/gpt-4o or anthropic/claude-sonnet-4

Model must be provider/model, got ${value}

Error message

Model must be provider/model, got ${value}

What it means

modelParts splits a model identifier on the first '/' into provider and model. A valid id must contain a slash that is neither the first nor last character; anything else (no slash, leading slash, trailing slash) is rejected with this message.

Source

Thrown at packages/metaharness/src/tb/trial.ts:56

function addDelegatedUsage(messages: AgentMessage[], usage: TrialUsage): void {
	for (const message of messages) {
		if (message.role !== "toolResult" || message.toolName !== "inspect_image" || !isRecord(message.details)) continue;
		const delegated = message.details.usage;
		if (!isRecord(delegated)) continue;
		if (typeof delegated.input === "number") usage.input += delegated.input;
		if (typeof delegated.output === "number") usage.output += delegated.output;
		if (typeof delegated.cacheRead === "number") usage.cacheRead += delegated.cacheRead;
		if (typeof delegated.cacheWrite === "number") usage.cacheWrite += delegated.cacheWrite;
		if (isRecord(delegated.cost) && typeof delegated.cost.total === "number") {
			usage.costUsd += delegated.cost.total;
		}
	}
}

function modelParts(value: string): { provider: string; model: string } {
	const slash = value.indexOf("/");
	if (slash <= 0 || slash === value.length - 1) throw new Error(`Model must be provider/model, got ${value}`);
	return { provider: value.slice(0, slash), model: value.slice(slash + 1) };
}

/** Run one Terminal-Bench task inside a fresh Vibemon microVM. */
export async function runTrial(opts: {
	task: TbTask;
	model: string;
	binaries: AgentBinaries;
	gateway: GatewayConfig;
	vmon: VmonConfig;
	trialDir: string;
	log?: (line: string) => void;
}): Promise<TrialResult> {
	const wallStartedAt = performance.now();
	const deadlineMs = (opts.task.agentTimeoutSec + opts.task.verifierTimeoutSec + 900) * 1_000;
	let vm: TrialVm | null = null;
	let client: RpcClient | null = null;
	let deadlineExpired = false;

View on GitHub (pinned to 9690622007)

Solutions

Format the model as provider/model, e.g. openai/gpt-4o or anthropic/claude-sonnet-4
Remove stray leading/trailing slashes from the model string
Check config files/env lists for truncated entries

Example fix

// before
tb run --models gpt-4o
// after
tb run --models openai/gpt-4o

Defensive patterns

Strategy: validation

Validate before calling

function assertModelId(v: string): void {
  const i = v.indexOf("/");
  if (i <= 0 || i === v.length - 1) throw new Error(`Model must be provider/model, got ${v}`);
}
for (const m of models) assertModelId(m);

Type guard

function isModelId(v: string): boolean {
  const i = v.indexOf("/");
  return i > 0 && i < v.length - 1;
}

Prevention

Always use provider/model form in configs and CLI flags
Lint config model lists with a provider/model regex
Avoid hand-edited lists; keep model strings next to provider names

When it happens

Trigger: runTrial/modelParts receives a --models entry like 'gpt-4o' (no provider), '/claude' (leading slash), or 'openai/' (trailing slash).

Common situations: Users accustomed to bare model names omit the provider prefix; YAML/config lists edited so a slash was dropped or duplicated; provider names like 'openai' forgotten for gateway routing.

Related errors

AI-assisted analysis of can1357/oh-my-pi@9690622007 (2026-08-31). Data as JSON: /api/errors/2a4c4dd7262055cf. Report an issue: GitHub.