{"record":{"id":"889fc4e059768323","repo":"JuliusBrussee/caveman","slug":"caveman-agent-error-rate-guardrail-max-must-be-in","errorCode":null,"errorMessage":"caveman agent: error-rate guardrail max must be in [0,1]","messagePattern":"caveman agent: error-rate guardrail max must be in \\[0,1\\]","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/agent/src/primitives.ts","lineNumber":458,"sourceCode":"  quality: QualityGrader[];\n  guardrails?: EvalGuardrail[];\n}): EvalDefinition {\n  if (options.id.trim() === \"\") throw new Error(\"caveman agent: eval id is required\");\n  if (options.quality.length === 0) throw new Error(\"caveman agent: eval needs at least one quality grader\");\n  const known = new Set([\"contains\", \"tool_called\", \"exact_match\", \"json_schema\"]);\n  for (const grader of options.quality) {\n    if (!known.has(grader.type)) {\n      throw new Error(`caveman agent: unknown grader ${(grader as { type: string }).type}`);\n    }\n  }\n  for (const guardrail of options.guardrails ?? []) {\n    if (guardrail.type === \"latency_threshold\" &&\n        (!Number.isSafeInteger(guardrail.p95_ms) || guardrail.p95_ms <= 0)) {\n      throw new Error(\"caveman agent: latency guardrail requires positive integer p95_ms\");\n    }\n    if (guardrail.type === \"error_rate\" &&\n        (!Number.isFinite(guardrail.max) || guardrail.max < 0 || guardrail.max > 1)) {\n      throw new Error(\"caveman agent: error-rate guardrail max must be in [0,1]\");\n    }\n  }\n  const tools = options.tools ?? { mode: \"fixture\" as const };\n  if (tools.mode === \"live\" && !tools.sandbox) {\n    throw new Error(\"caveman agent: live eval tools require an explicit sandbox\");\n  }\n  return Object.freeze({\n    kind: \"eval\",\n    id: options.id,\n    approved: options.approved ?? false,\n    required: options.required ?? true,\n    input: options.input,\n    tools,\n    quality: Object.freeze([...options.quality]),\n    guardrails: Object.freeze([...(options.guardrails ?? [])]),\n  });\n}\n","sourceCodeStart":440,"sourceCodeEnd":476,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/agent/src/primitives.ts#L440-L476","documentation":"evalFixture() validates error_rate guardrails: max must be a finite number in the closed interval [0, 1]. It throws when max is non-finite (NaN/Infinity), negative, or greater than 1, because an error rate is a ratio between 0 and 1 inclusive.","triggerScenarios":"guardrails: [{ type: \"error_rate\", max: 5 }] (expressing 5% as 5 instead of 0.05), max: -0.1, max: Infinity/NaN, or a non-numeric value.","commonSituations":"Specifying percentages as whole numbers (5 for 5%) instead of fractions; a config merge producing undefined that becomes NaN after arithmetic; unit confusion between percent and ratio.","solutions":["Express the rate as a fraction in [0,1], e.g. 0.05 for 5%.","Convert percentages in code: max: percentValue / 100.","Clamp/validate before the call: Number.isFinite(v) && v >= 0 && v <= 1."],"exampleFix":"// before\n{ type: \"error_rate\", max: 5 } // meant 5%\n// after\n{ type: \"error_rate\", max: 0.05 }","handlingStrategy":"validation","validationCode":"for (const g of guardrails) {\n  if (g.type === \"error_rate\" && !(Number.isFinite(g.max) && g.max >= 0 && g.max <= 1)) {\n    throw new Error(`error_rate max must be in [0,1], got ${g.max}`);\n  }\n}","typeGuard":"function isRatio(v: unknown): v is number { return typeof v === \"number\" && Number.isFinite(v) && v >= 0 && v <= 1; }","tryCatchPattern":"try { ev = evalFixture(opts); }\ncatch (e) { if (String(e.message).includes(\"error-rate\")) throw new Error(\"Use a fraction in [0,1] for error_rate.max (0.05 = 5%)\"); throw e; }","preventionTips":["Store error-rate thresholds as ratios, never percentages.","Convert user-facing percent config with value / 100.","Guard arithmetic so undefined/NaN never reaches max."],"tags":["validation","eval","guardrail","value-out-of-range"],"backgroundTag":"value-out-of-range","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}