paperclipai/paperclip · error

eval-session provider/driver mismatch

Error message

eval-session provider/driver mismatch

What it means

parseEvalSessionRequest validates that the eval-session request's optional 'driver' field matches the driver implied by the chosen 'provider'. Each provider maps to exactly one driver (codex -> codex_app_server, opencode -> opencode_server, claude_managed -> claude_managed_agents_api, aws_agentcore -> aws_agentcore_harness_api, acpx -> acpx_runtime). This fail-closed check exists because the request crosses an executable/CLI boundary, so a mismatched driver/provider pair would silently target the wrong runtime.

Source

Thrown at packages/paperclip-runner/src/cli/eval-session-contract.ts:234

  if (input.schema !== EVAL_SESSION_REQUEST_SCHEMA) {
    throw new Error("unsupported request schema");
  }
  const providerValue = input.provider ?? "codex";
  if (
    providerValue !== "codex" &&
    providerValue !== "opencode" &&
    providerValue !== "claude_managed" &&
    providerValue !== "aws_agentcore" &&
    providerValue !== "acpx"
  ) {
    throw new Error(
      "eval-session provider is unsupported by CapabilityLiveSessionService",
    );
  }
  const provider = providerValue;
  const driver = expectedEvalSessionDriver(provider);
  if (input.driver !== undefined && input.driver !== driver) {
    throw new Error("eval-session provider/driver mismatch");
  }
  // The original Evalbook v1 producer serialized absent provider-specific
  // options as JSON null. Preserve compatibility with those immutable request
  // artifacts while continuing to reject non-null values for the wrong lane.
  const acpxAgent = input.acpxAgent === null ? undefined : input.acpxAgent;
  if (acpxAgent === "pi") throw new Error("The Pi ACPX profile is not available");
  if (
    acpxAgent !== undefined &&
    acpxAgent !== "codex" &&
    acpxAgent !== "claude"
  ) {
    throw new Error("eval-session acpxAgent must be codex or claude");
  }
  if (provider !== "acpx" && acpxAgent !== undefined) {
    throw new Error("eval-session acpxAgent requires provider acpx");
  }
  const managedProfileInput = input.managedProfile === null
    ? undefined

View on GitHub (pinned to 01ad858492)

Solutions

1. Remove the explicit 'driver' field from the request and let the contract derive it from 'provider' (driver is optional).
2. Set driver to the exact value returned by expectedEvalSessionDriver(provider): codex->'codex_app_server', opencode->'opencode_server', claude_managed->'claude_managed_agents_api', aws_agentcore->'aws_agentcore_harness_api', acpx->'acpx_runtime'.
3. Regenerate the request from the current schema/version instead of reusing an old request file.
4. Call expectedEvalSessionDriver(provider) in your generator and assign its result to driver.

Example fix

// before
const request = { schema, provider: "opencode", driver: "codex_app_server", ... };
// after
import { expectedEvalSessionDriver } from "./eval-session-contract.js";
const request = { schema, provider: "opencode", driver: expectedEvalSessionDriver("opencode"), ... };
// or simply omit driver:
const request = { schema, provider: "opencode", ... };

import { expectedEvalSessionDriver } from "./eval-session-contract.js";
function assertDriverMatches(request: { provider?: string; driver?: string }): void {
  const provider = request.provider ?? "codex";
  const allowed = ["codex", "opencode", "claude_managed", "aws_agentcore", "acpx"] as const;
  if (!(allowed as readonly string[]).includes(provider)) {
    throw new Error(`unsupported provider: ${provider}`);
  }
  const expected = expectedEvalSessionDriver(provider as typeof allowed[number]);
  if (request.driver !== undefined && request.driver !== expected) {
    throw new Error(`driver ${request.driver} != expected ${expected} for provider ${provider}`);
  }
}

const DRIVER_BY_PROVIDER = {
  codex: "codex_app_server",
  opencode: "opencode_server",
  claude_managed: "claude_managed_agents_api",
  aws_agentcore: "aws_agentcore_harness_api",
  acpx: "acpx_runtime",
} as const;
function hasValidDriver(r: { provider?: string; driver?: string }): boolean {
  const p = (r.provider ?? "codex") as keyof typeof DRIVER_BY_PROVIDER;
  return r.driver === undefined || r.driver === DRIVER_BY_PROVIDER[p];
}

try {
  const parsed = parseEvalSessionRequest(raw);
} catch (err) {
  if (err instanceof Error && err.message === "eval-session provider/driver mismatch") {
    console.error(`Bad driver for provider ${raw.provider}: drop 'driver' or use expectedEvalSessionDriver(provider)`);
  } else throw err;
}

When it happens

Trigger: Calling parseEvalSessionRequest (via the request entry point) with a request object where input.driver is defined and does not equal expectedEvalSessionDriver(provider), e.g. {provider:'opencode', driver:'codex_app_server'} or a stale/hand-edited request file where driver was not updated after changing provider.

Common situations: Hand-editing or templating eval-session request JSON and changing provider but forgetting to update driver; reusing a request file from a previous run against a different provider; a generator script hardcoding driver values; typos in the driver constant.

Related errors

• eval-session acpxAgent requires provider acpx
• eval-session managedProfile requires provider claude_managed
• ${label} is not a regular file at ${canonical}.
• ${path} must be an object
• ${path} must be a non-empty string

AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/8f012a050d11b727. Report an issue: GitHub.