JuliusBrussee/caveman · error · Error

caveman build: requiredFixturePassRate must be in (0,1]

Error message

caveman build: requiredFixturePassRate must be in (0,1]

What it means

Thrown by defineBuild when requiredFixturePassRate is outside the interval (0, 1]. This is the fraction of eval fixtures the built artifact must pass: 0 would accept a build that passes nothing, values above 1 are impossible, and NaN (from a bad parse) fails the range check. The default is 1 (perfect fixture pass rate).

Source

Thrown at packages/agent/src/build.ts:47

      "caveman build: dataResidency is not enforced yet; refusing to ignore residency policy",
    );
  }
  const config: BuildConfig = {
    entry: options.entry,
    evals: options.evals,
    efficiency: options.efficiency ?? "max",
    requiredFixturePassRate: options.requiredFixturePassRate ?? 1,
    qualityRetention: options.qualityRetention ?? 0.98,
    maxSearchCostUsd: options.maxSearchCostUsd ?? 2,
    lock: options.lock ?? "strict",
    sandbox: options.sandbox ?? "required",
    ...(options.allowedModels === undefined ? {} : { allowedModels: [...options.allowedModels] }),
    ...(options.deniedModels === undefined ? {} : { deniedModels: [...options.deniedModels] }),
    ...(options.maxP95LatencyMs === undefined ? {} : { maxP95LatencyMs: options.maxP95LatencyMs }),
    ...(options.forbiddenSafetyClasses === undefined ? {} : { forbiddenSafetyClasses: [...options.forbiddenSafetyClasses] }),
  };
  if (!(config.requiredFixturePassRate > 0 && config.requiredFixturePassRate <= 1)) {
    throw new Error("caveman build: requiredFixturePassRate must be in (0,1]");
  }
  if (!(config.qualityRetention > 0 && config.qualityRetention <= 1)) {
    throw new Error("caveman build: qualityRetention must be in (0,1]");
  }
  if (!(config.maxSearchCostUsd > 0)) {
    throw new Error("caveman build: maxSearchCostUsd must be positive");
  }
  return Object.freeze(config);
}

export interface CavePlan {
  schema_version: 1;
  plan_id: string;
  model: string;
  reasoning: "none" | "minimal" | "low" | "medium" | "high";
  segment_routes: Array<{
    segment_kind: ContextKind;
    segment_id?: string;

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Pass a value in (0, 1]: 1 for strict gating (the default), e.g. 0.95 to tolerate 5% fixture failures.
2. Validate env/config-sourced values with a range check and a clear error at load time.
3. If the intent was to skip eval gating entirely, check for a supported flag for that instead of abusing the pass rate.

Example fix

// before
defineBuild({ entry, evals, requiredFixturePassRate: Number(process.env.FIXTURE_RATE) }); // NaN when unset

// after
const rate = process.env.FIXTURE_RATE !== undefined ? Number(process.env.FIXTURE_RATE) : 1;
if (!(rate > 0 && rate <= 1)) throw new Error(`FIXTURE_RATE must be in (0,1], got ${rate}`);
defineBuild({ entry, evals, requiredFixturePassRate: rate });

function fixtureRate(v: number | undefined): number {
  const r = v ?? 1;
  if (!(r > 0 && r <= 1)) throw new Error(`requiredFixturePassRate must be in (0,1], got ${r}`);
  return r;
}

function isUnitInterval(v: unknown): v is number {
  return typeof v === "number" && Number.isFinite(v) && v > 0 && v <= 1;
}

When it happens

Trigger: Passing requiredFixturePassRate: 0, a negative number, 1.5, or NaN; computing the rate from a division that yields NaN; loading the value from env/config as an unvalidated number.

Common situations: Teams lowering the bar to 0 during experimentation to 'skip' eval gating (use a different mechanism); decimal typos like 11 for 1.1; rates parsed from strings that fail Number() conversion.

Related errors

• caveman build: qualityRetention must be in (0,1]
• cave_budget_max_invalid
• cave_budget_denomination_ambiguous
• cave_budget_initial_invalid
• cave_budget_output_floor_invalid

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/6c74130178a67100. Report an issue: GitHub.