{"record":{"id":"7404a237132c231e","repo":"JuliusBrussee/caveman","slug":"caveman-agent-eval-id-is-required","errorCode":null,"errorMessage":"caveman agent: eval id is required","messagePattern":"caveman agent: eval id is required","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/agent/src/primitives.ts","lineNumber":443,"sourceCode":"  readonly id: string;\n  readonly approved: boolean;\n  readonly required: boolean;\n  readonly input: unknown;\n  readonly tools: { mode: \"fixture\" | \"live\"; sandbox?: string };\n  readonly quality: readonly QualityGrader[];\n  readonly guardrails: readonly EvalGuardrail[];\n}\n\nexport function evalFixture(options: {\n  id: string;\n  approved?: boolean;\n  required?: boolean;\n  input: unknown;\n  tools?: { mode: \"fixture\" | \"live\"; sandbox?: string };\n  quality: QualityGrader[];\n  guardrails?: EvalGuardrail[];\n}): EvalDefinition {\n  if (options.id.trim() === \"\") throw new Error(\"caveman agent: eval id is required\");\n  if (options.quality.length === 0) throw new Error(\"caveman agent: eval needs at least one quality grader\");\n  const known = new Set([\"contains\", \"tool_called\", \"exact_match\", \"json_schema\"]);\n  for (const grader of options.quality) {\n    if (!known.has(grader.type)) {\n      throw new Error(`caveman agent: unknown grader ${(grader as { type: string }).type}`);\n    }\n  }\n  for (const guardrail of options.guardrails ?? []) {\n    if (guardrail.type === \"latency_threshold\" &&\n        (!Number.isSafeInteger(guardrail.p95_ms) || guardrail.p95_ms <= 0)) {\n      throw new Error(\"caveman agent: latency guardrail requires positive integer p95_ms\");\n    }\n    if (guardrail.type === \"error_rate\" &&\n        (!Number.isFinite(guardrail.max) || guardrail.max < 0 || guardrail.max > 1)) {\n      throw new Error(\"caveman agent: error-rate guardrail max must be in [0,1]\");\n    }\n  }\n  const tools = options.tools ?? { mode: \"fixture\" as const };","sourceCodeStart":425,"sourceCodeEnd":461,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/agent/src/primitives.ts#L425-L461","documentation":"evalFixture() requires a non-empty id: it throws when options.id.trim() === \"\" (empty string or whitespace-only) or when id is missing/undefined. The id identifies the eval fixture in results and approvals, so the library refuses to build an EvalDefinition without one.","triggerScenarios":"evalFixture({ id: \"\", ... }), evalFixture({ id: \"   \", ... }), or omitting id altogether while supplying the required input and quality fields.","commonSituations":"Generating ids from a variable that is empty or whitespace-padded; building fixtures programmatically from table rows where the id column is blank; YAML/JSON config with id: \"\".","solutions":["Pass a non-empty, non-whitespace id string.","Trim and validate the id at config load time before constructing the fixture.","If ids are generated, add a fallback like a slugified name or a counter to guarantee uniqueness and non-emptiness."],"exampleFix":"// before\nevalFixture({ id: \"\", input: \"q\", quality: [exactMatch()] });\n// after\nevalFixture({ id: \"qa-basic-1\", input: \"q\", quality: [exactMatch()] });","handlingStrategy":"validation","validationCode":"function hasId(id) { return typeof id === \"string\" && id.trim().length > 0; }\nif (!hasId(fixture.id)) throw new Error(`eval fixture id required, got ${JSON.stringify(fixture.id)}`);","typeGuard":"function hasNonEmptyId(o: { id?: string }): o is { id: string } { return typeof o.id === \"string\" && o.id.trim().length > 0; }","tryCatchPattern":"let ev;\ntry { ev = evalFixture(opts); }\ncatch (e) { if (String(e.message).includes(\"eval id is required\")) ev = evalFixture({ ...opts, id: opts.id?.trim() || generatedId() }); else throw e; }","preventionTips":["Validate fixture configs (id non-empty) when loading from YAML/JSON.","Trim ids sourced from user input or spreadsheets.","Auto-generate deterministic ids when a source row lacks one."],"tags":["validation","eval","missing-id","argument-validation"],"backgroundTag":"empty-required-field","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}