{"record":{"id":"6c74130178a67100","repo":"JuliusBrussee/caveman","slug":"caveman-build-requiredfixturepassrate-must-be-in","errorCode":null,"errorMessage":"caveman build: requiredFixturePassRate must be in (0,1]","messagePattern":"caveman build: requiredFixturePassRate must be in \\(0,1\\]","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/agent/src/build.ts","lineNumber":47,"sourceCode":"      \"caveman build: dataResidency is not enforced yet; refusing to ignore residency policy\",\n    );\n  }\n  const config: BuildConfig = {\n    entry: options.entry,\n    evals: options.evals,\n    efficiency: options.efficiency ?? \"max\",\n    requiredFixturePassRate: options.requiredFixturePassRate ?? 1,\n    qualityRetention: options.qualityRetention ?? 0.98,\n    maxSearchCostUsd: options.maxSearchCostUsd ?? 2,\n    lock: options.lock ?? \"strict\",\n    sandbox: options.sandbox ?? \"required\",\n    ...(options.allowedModels === undefined ? {} : { allowedModels: [...options.allowedModels] }),\n    ...(options.deniedModels === undefined ? {} : { deniedModels: [...options.deniedModels] }),\n    ...(options.maxP95LatencyMs === undefined ? {} : { maxP95LatencyMs: options.maxP95LatencyMs }),\n    ...(options.forbiddenSafetyClasses === undefined ? {} : { forbiddenSafetyClasses: [...options.forbiddenSafetyClasses] }),\n  };\n  if (!(config.requiredFixturePassRate > 0 && config.requiredFixturePassRate <= 1)) {\n    throw new Error(\"caveman build: requiredFixturePassRate must be in (0,1]\");\n  }\n  if (!(config.qualityRetention > 0 && config.qualityRetention <= 1)) {\n    throw new Error(\"caveman build: qualityRetention must be in (0,1]\");\n  }\n  if (!(config.maxSearchCostUsd > 0)) {\n    throw new Error(\"caveman build: maxSearchCostUsd must be positive\");\n  }\n  return Object.freeze(config);\n}\n\nexport interface CavePlan {\n  schema_version: 1;\n  plan_id: string;\n  model: string;\n  reasoning: \"none\" | \"minimal\" | \"low\" | \"medium\" | \"high\";\n  segment_routes: Array<{\n    segment_kind: ContextKind;\n    segment_id?: string;","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/packages/agent/src/build.ts#L29-L65","documentation":"Thrown by defineBuild when requiredFixturePassRate is outside the interval (0, 1]. This is the fraction of eval fixtures the built artifact must pass: 0 would accept a build that passes nothing, values above 1 are impossible, and NaN (from a bad parse) fails the range check. The default is 1 (perfect fixture pass rate).","triggerScenarios":"Passing requiredFixturePassRate: 0, a negative number, 1.5, or NaN; computing the rate from a division that yields NaN; loading the value from env/config as an unvalidated number.","commonSituations":"Teams lowering the bar to 0 during experimentation to 'skip' eval gating (use a different mechanism); decimal typos like 11 for 1.1; rates parsed from strings that fail Number() conversion.","solutions":["Pass a value in (0, 1]: 1 for strict gating (the default), e.g. 0.95 to tolerate 5% fixture failures.","Validate env/config-sourced values with a range check and a clear error at load time.","If the intent was to skip eval gating entirely, check for a supported flag for that instead of abusing the pass rate."],"exampleFix":"// before\ndefineBuild({ entry, evals, requiredFixturePassRate: Number(process.env.FIXTURE_RATE) }); // NaN when unset\n\n// after\nconst rate = process.env.FIXTURE_RATE !== undefined ? Number(process.env.FIXTURE_RATE) : 1;\nif (!(rate > 0 && rate <= 1)) throw new Error(`FIXTURE_RATE must be in (0,1], got ${rate}`);\ndefineBuild({ entry, evals, requiredFixturePassRate: rate });","handlingStrategy":"validation","validationCode":"function fixtureRate(v: number | undefined): number {\n  const r = v ?? 1;\n  if (!(r > 0 && r <= 1)) throw new Error(`requiredFixturePassRate must be in (0,1], got ${r}`);\n  return r;\n}","typeGuard":"function isUnitInterval(v: unknown): v is number {\n  return typeof v === \"number\" && Number.isFinite(v) && v > 0 && v <= 1;\n}","tryCatchPattern":null,"preventionTips":["Use fractions (0.95), never percentages (95), for pass rates.","Validate env/config values with a (0,1] range check at load time.","To loosen gating, lower the rate gradually (e.g. 0.9) rather than to 0."],"tags":["build","config","validation","evals","numbers"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}