{"record":{"id":"07551c4a573c477c","repo":"paperclipai/paperclip","slug":"unknown-runner-live-eval-candidate-id","errorCode":null,"errorMessage":"unknown Runner live eval candidate: ${id}","messagePattern":"unknown Runner live eval candidate: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/paperclip-runner/src/eval/live-workflow-matrix.ts","lineNumber":265,"sourceCode":"  expectedExecutions: number;\n}\n\nexport interface RunnerLiveEvalSelection {\n  candidateIds?: readonly string[];\n  caseIds?: readonly string[];\n  limit?: number;\n}\n\n/** Selects a stable paid subset without weakening validation of the full schedule. */\nexport function selectRunnerLiveEvalSchedule(\n  schedule: RunnerLiveEvalSchedule,\n  selection: RunnerLiveEvalSelection,\n): RunnerLiveEvalSchedule {\n  const candidateIds = new Set(selection.candidateIds ?? []);\n  const caseIds = new Set(selection.caseIds ?? []);\n  for (const id of candidateIds) {\n    if (!schedule.candidates.some((candidate) => candidate.id === id)) {\n      throw new Error(`unknown Runner live eval candidate: ${id}`);\n    }\n  }\n  const scheduledCaseIds = new Set(\n    schedule.entries.map((entry) => entry.caseId),\n  );\n  for (const id of caseIds) {\n    if (!scheduledCaseIds.has(id as RunnerWorkflowEvalCase[\"id\"])) {\n      throw new Error(`unknown Runner live eval case: ${id}`);\n    }\n  }\n  if (\n    selection.limit !== undefined &&\n    (!Number.isSafeInteger(selection.limit) || selection.limit <= 0)\n  ) {\n    throw new Error(\n      \"Runner live eval selection limit must be a positive integer\",\n    );\n  }","sourceCodeStart":247,"sourceCodeEnd":283,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/packages/paperclip-runner/src/eval/live-workflow-matrix.ts#L247-L283","documentation":"selectRunnerLiveEvalSchedule validates the candidateIds filter of a RunnerLiveEvalSelection against the candidate list of the compiled schedule. If any requested candidate id does not exist in schedule.candidates, it throws immediately rather than silently producing an empty or partial selection.","triggerScenarios":"Passing selection.candidateIds containing an id that is not in the schedule's candidates — e.g. a renamed or removed candidate, a typo, or an id from a different schedule version.","commonSituations":"Editing candidate definitions in the matrix and forgetting to update selection configs; copying a selection between environments/matrix versions; stale CI config referencing an old candidate id.","solutions":["Print the schedule's valid ids (schedule.candidates.map(c => c.id)) and correct the selection to use one of them.","Update the selection config after any rename/removal of candidates.","Validate selection ids against the schedule in your own config loader before calling selectRunnerLiveEvalSchedule."],"exampleFix":"// before\nselectRunnerLiveEvalSchedule(schedule, { candidateIds: ['old-candidate'] });\n// after\nconst valid = schedule.candidates.map((c) => c.id);\nselectRunnerLiveEvalSchedule(schedule, { candidateIds: ['old-candidate'].filter((id) => valid.includes(id)) });","handlingStrategy":"validation","validationCode":"const valid = new Set(schedule.candidates.map((c) => c.id));\nconst bad = (selection.candidateIds ?? []).filter((id) => !valid.has(id));\nif (bad.length) throw new Error(`selection references unknown candidates: ${bad.join(', ')}`);","typeGuard":"function candidatesExist(schedule: Schedule, ids?: string[]): ids is string[] {\n  const valid = new Set(schedule.candidates.map((c) => c.id));\n  return (ids ?? []).every((id) => valid.has(id));\n}","tryCatchPattern":"try {\n  const plan = selectRunnerLiveEvalSchedule(schedule, selection);\n} catch (err) {\n  if ((err as Error).message.startsWith('unknown Runner live eval candidate')) {\n    throw new ConfigError(`candidateIds invalid; valid ids: ${schedule.candidates.map((c) => c.id).join(', ')}`);\n  }\n  throw err;\n}","preventionTips":["Regenerate or review selection configs whenever candidate definitions change.","Keep candidate ids in a single shared constants module used by both matrix and selection.","Lint eval selection files against the current schedule in CI.","Copy selections only between compatible schedule versions."],"tags":["eval","validation","configuration","runner"],"backgroundTag":"invalid-argument-value","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-10T03:14:50.855Z","contentChangedAt":"2026-09-10T03:14:50.855Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}