What causes "caveman agent: pinned model must use provider/model format" in JuliusBrussee/caveman?

Set the value to full `provider/model` form, e.g. `CAVE_MODEL=openai/gpt-5.4-mini` or `anthropic/claude-haiku-4-5`.

caveman agent: pinned model must use provider/model format

Error message

caveman agent: pinned model must use provider/model format

What it means

Model resolution in the agent runtime validates any pinned model string (from `definition.model`, `CAVE_MODEL`, or `.caveman/provider.json`) against `provider/model` format. It throws this error when the string has no `/` at a valid position — either missing entirely, at index 0, or as the last character — because the runtime splits on the first `/` to look up provider and model id.

Solutions

Set the value to full `provider/model` form, e.g. `CAVE_MODEL=openai/gpt-5.4-mini` or `anthropic/claude-haiku-4-5`.
Fix `.caveman/provider.json` so `model` includes the provider prefix.
Fix the `model` field on the `AgentDefinition` if pinned there.
Unset `CAVE_MODEL` to fall back to credential-based baseline selection instead of pinning.

Example fix

// before
CAVE_MODEL=gpt-5.4-mini
// after
CAVE_MODEL=openai/gpt-5.4-mini

Defensive patterns

Strategy: validation

Validate before calling

function isProviderModelFormat(s: string | undefined): boolean {
  if (!s) return true; // unset falls back to baseline selection
  const i = s.indexOf("/");
  return i > 0 && i < s.length - 1;
}
if (!isProviderModelFormat(process.env.CAVE_MODEL)) throw new Error("CAVE_MODEL must be provider/model");

Type guard

const isPinnedModel = (v: unknown): v is string =>
  typeof v === "string" && /^[^/]+\/[^/].*$/.test(v) && !v.endsWith("/");

Try / catch

try {
  await agent.run(task);
} catch (e) {
  if ((e as Error).message.includes("pinned model must use provider/model format")) {
    console.error("Set CAVE_MODEL like openai/gpt-5.4-mini");
  } else throw e;
}

Prevention

Always write model pins as `provider/model` with a non-empty provider and model id.
Add a startup env check for CAVE_MODEL format before launching runs.
When copying model ids from provider docs, prepend the provider slug yourself.

When it happens

Trigger: Setting `CAVE_MODEL=gpt-4o` (no slash), `CAVE_MODEL=/gpt-4o` (leading slash), or `CAVE_MODEL=openai/` (trailing slash); likewise a `definition.model` or `provider.json` `model` field with the same shapes.

Common situations: Copy-pasting just a model id from provider docs into `CAVE_MODEL`; editing `.caveman/provider.json` by hand and dropping the provider prefix; shell quoting issues stripping part of the value.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/f12a830787f07f84. Report an issue: GitHub.

Appendix: source

Thrown at packages/agent/src/runtime.ts:3244

  }
  // Runtime accepts GOOGLE_API_KEY as alias for Pi's GEMINI_API_KEY-only
  // Google provider. Use only after Pi failed to prove a stronger regime, so a
  // future OAuth-capable Google provider cannot be relabeled as metered.
  if (provider === "google" && process.env.GOOGLE_API_KEY) return "metered";
  return "unknown";
}

function resolveModel(definition: AgentDefinition, models: Models, rootDir: string): Model<Api> {
  if (typeof definition.model === "object" && "provider" in definition.model) {
    return definition.model;
  }
  const requested = typeof definition.model === "string"
    ? definition.model
    : process.env.CAVE_MODEL ?? localProviderModel(rootDir);
  if (requested) {
    const slash = requested.indexOf("/");
    if (slash <= 0 || slash === requested.length - 1) {
      throw new Error("caveman agent: pinned model must use provider/model format");
    }
    const model = models.getModel(requested.slice(0, slash), requested.slice(slash + 1));
    if (!model) throw new Error(`caveman agent: unknown model ${JSON.stringify(requested)}`);
    return model;
  }

  const configured: Array<[string, string]> = [];
  if (process.env.ANTHROPIC_API_KEY) configured.push(["anthropic", "claude-haiku-4-5"]);
  if (process.env.OPENAI_API_KEY) configured.push(["openai", "gpt-5.4-mini"]);
  if (process.env.GEMINI_API_KEY || process.env.GOOGLE_API_KEY) configured.push(["google", "gemini-2.5-flash"]);
  if (configured.length !== 1) {
    throw new Error(
      configured.length === 0
        ? "caveman agent: no supported provider credential found"
        : "caveman agent: multiple provider credentials found; set CAVE_MODEL once",
    );
  }
  const [provider, id] = configured[0]!;

View on GitHub (pinned to 3ee70a1026)