{"record":{"id":"7404a237132c231e","repo":"JuliusBrussee/caveman","slug":"caveman-agent-eval-id-is-required","errorCode":null,"errorMessage":"caveman agent: eval id is required","messagePattern":"caveman agent: eval id is required","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/agent/src/primitives.ts","lineNumber":443,"sourceCode":"  readonly id: string;\n  readonly approved: boolean;\n  readonly required: boolean;\n  readonly input: unknown;\n  readonly tools: { mode: \"fixture\" | \"live\"; sandbox?: string };\n  readonly quality: readonly QualityGrader[];\n  readonly guardrails: readonly EvalGuardrail[];\n}\n\nexport function evalFixture(options: {\n  id: string;\n  approved?: boolean;\n  required?: boolean;\n  input: unknown;\n  tools?: { mode: \"fixture\" | \"live\"; sandbox?: string };\n  quality: QualityGrader[];\n  guardrails?: EvalGuardrail[];\n}): EvalDefinition {\n  if (options.id.trim() === \"\") throw new Error(\"caveman agent: eval id is required\");\n  if (options.quality.length === 0) throw new Error(\"caveman agent: eval needs at least one quality grader\");\n  const known = new Set([\"contains\", \"tool_called\", \"exact_match\", \"json_schema\"]);\n  for (const grader of options.quality) {\n    if (!known.has(grader.type)) {\n      throw new Error(`caveman agent: unknown grader ${(grader as { type: string }).type}`);\n    }\n  }\n  for (const guardrail of options.guardrails ?? []) {\n    if (guardrail.type === \"latency_threshold\" &&\n        (!Number.isSafeInteger(guardrail.p95_ms) || guardrail.p95_ms <= 0)) {\n      throw new Error(\"caveman agent: latency guardrail requires positive integer p95_ms\");\n    }\n    if (guardrail.type === \"error_rate\" &&\n        (!Number.isFinite(guardrail.max) || guardrail.max < 0 || guardrail.max > 1)) {\n      throw new Error(\"caveman agent: error-rate guardrail max must be in [0,1]\");\n    }\n  }\n  const tools = options.tools ?? { mode: \"fixture\" as const };","sourceCodeStart":425,"sourceCodeEnd":461,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/packages/agent/src/primitives.ts#L425-L461","documentation":"Thrown by the evalFixture() builder when options.id is empty or whitespace-only. Eval fixture ids identify the fixture in eval-complete selection and lock evidence, so a blank id is rejected at definition construction before any eval run starts.","triggerScenarios":"Calling evalFixture({ id: '' }) or evalFixture({ id: '  ' }) with an otherwise valid input and quality graders.","commonSituations":"Generating fixture ids from file stems where a file has no name; iterating a config list where one entry is missing its id key; sanitizing user labels down to an empty string before passing them as ids.","solutions":["Give every fixture a stable non-empty id, e.g. 'rag-happy-path'","When generating ids, fall back to an index or slug: id: raw?.trim() || `fixture-${i}`","Fail early in your fixture loader with a clear file/entry reference instead of letting the builder throw"],"exampleFix":"// before\nevalFixture({ id: entry.name ?? '', input: q, quality: [{ type: 'contains', fragments: ['ok'] }] });\n\n// after\nevalFixture({ id: entry.name?.trim() || `fixture-${index}`, input: q, quality: [{ type: 'contains', fragments: ['ok'] }] });","handlingStrategy":"validation","validationCode":"function fixtureId(raw: unknown, index: number): string {\n  const id = typeof raw === 'string' ? raw.trim() : '';\n  if (id === '') throw new Error(`fixture[${index}] is missing a non-empty id`);\n  return id;\n}","typeGuard":"function isFixtureId(value: unknown): value is string { return typeof value === 'string' && value.trim() !== ''; }","tryCatchPattern":null,"preventionTips":["Require fixture ids in the config schema (minLength: 1)","Fail in the loader with the file/entry reference, not in the builder","Use descriptive ids — they appear in eval selection evidence"],"tags":["validation","eval","required-field"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}