{"record":{"id":"e68b0eba7e1a6cab","repo":"paperclipai/paperclip","slug":"unknown-runner-live-eval-case-id","errorCode":null,"errorMessage":"unknown Runner live eval case: ${id}","messagePattern":"unknown Runner live eval case: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/paperclip-runner/src/eval/live-workflow-matrix.ts","lineNumber":273,"sourceCode":"\n/** Selects a stable paid subset without weakening validation of the full schedule. */\nexport function selectRunnerLiveEvalSchedule(\n  schedule: RunnerLiveEvalSchedule,\n  selection: RunnerLiveEvalSelection,\n): RunnerLiveEvalSchedule {\n  const candidateIds = new Set(selection.candidateIds ?? []);\n  const caseIds = new Set(selection.caseIds ?? []);\n  for (const id of candidateIds) {\n    if (!schedule.candidates.some((candidate) => candidate.id === id)) {\n      throw new Error(`unknown Runner live eval candidate: ${id}`);\n    }\n  }\n  const scheduledCaseIds = new Set(\n    schedule.entries.map((entry) => entry.caseId),\n  );\n  for (const id of caseIds) {\n    if (!scheduledCaseIds.has(id as RunnerWorkflowEvalCase[\"id\"])) {\n      throw new Error(`unknown Runner live eval case: ${id}`);\n    }\n  }\n  if (\n    selection.limit !== undefined &&\n    (!Number.isSafeInteger(selection.limit) || selection.limit <= 0)\n  ) {\n    throw new Error(\n      \"Runner live eval selection limit must be a positive integer\",\n    );\n  }\n  let entries = schedule.entries.filter(\n    (entry) =>\n      (candidateIds.size === 0 || candidateIds.has(entry.candidateId)) &&\n      (caseIds.size === 0 || caseIds.has(entry.caseId)),\n  );\n  if (selection.limit !== undefined)\n    entries = entries.slice(0, selection.limit);\n  if (entries.length === 0) {","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/packages/paperclip-runner/src/eval/live-workflow-matrix.ts#L255-L291","documentation":"selectRunnerLiveEvalSchedule validates the caseIds filter of a RunnerLiveEvalSelection against the case ids actually scheduled in schedule.entries. If a requested case id has no scheduled entry, it throws this error. This prevents selections that silently match nothing because a case was never scheduled.","triggerScenarios":"Passing selection.caseIds containing a case id that is not present in any schedule.entries entry — a typo, a case removed from the matrix, or an id from another schedule version.","commonSituations":"Referencing an eval case that was renamed in the matrix definition; running a selection file against a newer/older schedule where the case was dropped; hard-coded case lists in CI pipelines drifting from the schedule.","solutions":["List valid ids via new Set(schedule.entries.map((e) => e.caseId)) and fix the selection.","Synchronize the selection config with the current matrix case definitions.","Pre-validate caseIds against scheduled entries in your tooling before invoking the selector."],"exampleFix":"// before\nselectRunnerLiveEvalSchedule(schedule, { caseIds: ['removed-case'] });\n// after\nconst scheduled = new Set(schedule.entries.map((e) => e.caseId));\nselectRunnerLiveEvalSchedule(schedule, { caseIds: ['removed-case'].filter((id) => scheduled.has(id)) });","handlingStrategy":"validation","validationCode":"const scheduled = new Set(schedule.entries.map((e) => e.caseId));\nconst bad = (selection.caseIds ?? []).filter((id) => !scheduled.has(id));\nif (bad.length) throw new Error(`selection references unscheduled cases: ${bad.join(', ')}`);","typeGuard":"function casesAreScheduled(schedule: Schedule, ids?: string[]): ids is string[] {\n  const scheduled = new Set(schedule.entries.map((e) => e.caseId));\n  return (ids ?? []).every((id) => scheduled.has(id));\n}","tryCatchPattern":"try {\n  const plan = selectRunnerLiveEvalSchedule(schedule, selection);\n} catch (err) {\n  if ((err as Error).message.startsWith('unknown Runner live eval case')) {\n    throw new ConfigError(`caseIds invalid; scheduled ids: ${[...new Set(schedule.entries.map((e) => e.caseId))].join(', ')}`);\n  }\n  throw err;\n}","preventionTips":["Derive caseId lists from the schedule file instead of hard-coding them in CI configs.","Update selection configs in the same change that renames or removes matrix cases.","Add a config test that selects every declared case against the current schedule.","Use TypeScript union types for case ids where possible to catch typos at compile time."],"tags":["eval","validation","configuration","runner"],"backgroundTag":"invalid-argument-value","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-10T03:14:50.855Z","contentChangedAt":"2026-09-10T03:14:50.855Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}