JuliusBrussee/caveman · error

caveman agent: latency guardrail requires positive integer p

Error message

caveman agent: latency guardrail requires positive integer p95_ms

What it means

Thrown by evalFixture() when a guardrail of type 'latency_threshold' has a p95_ms that is not a safe positive integer. The latency guardrail asserts a p95 latency budget in whole milliseconds, so 0, negatives, fractions, and NaN are rejected when the fixture is defined.

Source

Thrown at packages/agent/src/primitives.ts:454

  approved?: boolean;
  required?: boolean;
  input: unknown;
  tools?: { mode: "fixture" | "live"; sandbox?: string };
  quality: QualityGrader[];
  guardrails?: EvalGuardrail[];
}): EvalDefinition {
  if (options.id.trim() === "") throw new Error("caveman agent: eval id is required");
  if (options.quality.length === 0) throw new Error("caveman agent: eval needs at least one quality grader");
  const known = new Set(["contains", "tool_called", "exact_match", "json_schema"]);
  for (const grader of options.quality) {
    if (!known.has(grader.type)) {
      throw new Error(`caveman agent: unknown grader ${(grader as { type: string }).type}`);
    }
  }
  for (const guardrail of options.guardrails ?? []) {
    if (guardrail.type === "latency_threshold" &&
        (!Number.isSafeInteger(guardrail.p95_ms) || guardrail.p95_ms <= 0)) {
      throw new Error("caveman agent: latency guardrail requires positive integer p95_ms");
    }
    if (guardrail.type === "error_rate" &&
        (!Number.isFinite(guardrail.max) || guardrail.max < 0 || guardrail.max > 1)) {
      throw new Error("caveman agent: error-rate guardrail max must be in [0,1]");
    }
  }
  const tools = options.tools ?? { mode: "fixture" as const };
  if (tools.mode === "live" && !tools.sandbox) {
    throw new Error("caveman agent: live eval tools require an explicit sandbox");
  }
  return Object.freeze({
    kind: "eval",
    id: options.id,
    approved: options.approved ?? false,
    required: options.required ?? true,
    input: options.input,
    tools,
    quality: Object.freeze([...options.quality]),

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Pass a whole positive millisecond count, e.g. p95_ms: 2000
2. Convert and round other units at the boundary: p95_ms: Math.max(1, Math.round(sloSeconds * 1000))
3. Validate guardrail objects from config against the p95_ms > 0 integer rule before building fixtures

Example fix

// before
evalFixture({ id: 'e1', input: q, quality: q1, guardrails: [{ type: 'latency_threshold', p95_ms: 0.5 }] });

// after
evalFixture({ id: 'e1', input: q, quality: q1, guardrails: [{ type: 'latency_threshold', p95_ms: 500 }] });

function toP95Ms(raw: unknown): number {
  const n = typeof raw === 'number' ? raw : Number(raw);
  if (!Number.isSafeInteger(n) || n <= 0) throw new Error(`p95_ms must be a positive integer (milliseconds), got ${String(raw)}`);
  return n;
}

function isP95Ms(value: unknown): value is number { return typeof value === 'number' && Number.isSafeInteger(value) && value > 0; }

When it happens

Trigger: Passing guardrails: [{ type: 'latency_threshold', p95_ms: 0 }], p95_ms: -1000, p95_ms: 1500.5, or p95_ms from unparsed string config.

Common situations: Converting seconds-based SLOs ('0.5s') to milliseconds and passing 0.5*1000 as a float is fine — but passing 0.5 (forgot to multiply) or a seconds value (500 meaning ms) misconfigures the budget; strings from JSON SLO files.

Related errors

• caveman agent: error-rate guardrail max must be in [0,1]
• cave_live_eval_sandbox_profile_missing
• cave_live_eval_sandbox_profile_invalid
• cave_compaction_option_invalid
• caveman agent: memory recallBudget must be a non-negative in

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/6f1012cd2d39fcd2. Report an issue: GitHub.