paperclipai/paperclip · error · Error

Live roster ${rosterId} has no cases

Error message

Live roster ${rosterId} has no cases

What it means

buildProtocolEvalCatalog loads every live roster JSON file under the rosters directory and validates its schema. After confirming the schema is 'paperclip-runner/live-roster/v1', it checks that roster.cases is a non-empty array. If cases is missing, not an array, or empty, the script refuses to build an eval campaign around a roster with nothing to run.

Source

Thrown at packages/paperclip-runner/scripts/runner-protocol-eval-campaign.mjs:153

        entry.isFile() &&
        entry.name.startsWith("live-") &&
        entry.name.endsWith(".json"),
    )
    .map((entry) => entry.name)
    .sort();
  const rosters = [];
  for (const rosterFile of rosterFiles) {
    const rosterPath = resolve(rosterRoot, rosterFile);
    const roster = await loadObject(rosterPath);
    const rosterId = safeId(roster.id, "roster ID");
    if (selected && !selected.has(rosterId) && !selected.has(rosterFile)) {
      continue;
    }
    if (roster.schema !== "paperclip-runner/live-roster/v1") {
      throw new Error(`Unsupported live roster schema in ${rosterFile}`);
    }
    if (!Array.isArray(roster.cases) || roster.cases.length === 0) {
      throw new Error(`Live roster ${rosterId} has no cases`);
    }
    const configPath = inside(
      programRoot,
      resolve(rosterRoot, String(roster.config ?? "")),
      `Config for ${rosterId}`,
    );
    const config = await loadObject(configPath);
    const credentialName = credentialForConfig(config);
    const cases = roster.cases.map((caseId) => safeId(caseId, "case ID"));
    if (new Set(cases).size !== cases.length) {
      throw new Error(`Live roster ${rosterId} repeats a case`);
    }
    rosters.push({
      rosterId,
      rosterFile,
      configFile: relative(programRoot, configPath).split(sep).join("/"),
      model: String(roster.model ?? config.model ?? "unknown"),
      provider: String(config.provider ?? "codex"),

View on GitHub (pinned to 01ad858492)

Solutions

1. Open the roster file named in the error path and add at least one valid case ID to the 'cases' array.
2. Verify the field is literally named 'cases' and is a JSON array of case IDs (strings).
3. If the roster is intentionally unused, remove the roster file so it is not picked up by the catalog builder.

Example fix

// before
{ "schema": "paperclip-runner/live-roster/v1", "model": "gpt-5", "cases": [] }
// after
{ "schema": "paperclip-runner/live-roster/v1", "model": "gpt-5", "cases": ["case-001", "case-002"] }

const roster = JSON.parse(await readFile(rosterFile, "utf8"));
if (roster.schema === "paperclip-runner/live-roster/v1" &&
    (!Array.isArray(roster.cases) || roster.cases.length === 0)) {
  throw new Error(`${rosterFile}: cases must be a non-empty array`);
}

function hasCases(r) {
  return Array.isArray(r.cases) && r.cases.length > 0;
}

When it happens

Trigger: A live roster JSON file with schema 'paperclip-runner/live-roster/v1' either omits the 'cases' field entirely, sets it to a non-array value (e.g. a string or object), or sets it to an empty array [].

Common situations: Hand-authoring a new roster file and forgetting to fill in cases; copying a roster template with the cases list stripped out; a script or merge that emptied the cases array; renaming the field (e.g. 'tests' instead of 'cases').

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• ${label} is not a regular file at ${canonical}.
• Claude Managed evals require exact model claude-sonnet-5
• AWS AgentCore evals require exact model global.anthropic.cla
• request.session.provider must match request.provider
• request.session.requestedModel must match request.model

AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/f58dc87bdb764e56. Report an issue: GitHub.