JuliusBrussee/caveman · error · Error

cave_fixture_terminal_evidence_missing

Error message

cave_fixture_terminal_evidence_missing

What it means

A defensive catch-all around terminal-evidence construction in `runFixture`: the block builds the run's evidence record (token counts, cache stats, recovery, privacy/sandbox conformance flags, unknown-transform detection, output digest) from the completed `runAgentInternal` result. Any failure in that block is re-thrown as `cave_fixture_terminal_evidence_missing` with the original error attached as `cause`. The build cannot write a lock whose eval evidence is incomplete or malformed.

Source

Thrown at packages/agent/src/cli.ts:1222

      graders,
      latency_ms: result.latencyMs,
      provider_visible_tokens: result.inputTokens + result.cacheReadTokens + result.cacheWriteTokens,
      cache_prefix_sha256: result.cachePrefixSHA256,
      cache_boundary_known: result.cacheBoundaryKnown,
      cache_read_tokens: result.cacheReadTokens,
      cache_write_tokens: result.cacheWriteTokens,
      cache_bust: result.cacheBust,
      error: false,
      recovery_resolved: result.recoveryResolved && result.transformFailures.length === 0,
      privacy_passed: privacyConformance && contentBlindRunEvidence(result, fixture),
      sandbox_passed: sandboxConformance && (definition.tools.length === 0 ||
        (definition.sandbox === "required" && result.toolCalls.length > 0)),
      unknown_transform: plan.segment_routes.some((route) =>
        !result.evaluatedTransformIDs.includes(route.transform_id)),
      output_digest: sha256(result.text),
    };
  } catch (error) {
    throw new Error("cave_fixture_terminal_evidence_missing", { cause: error });
  }
}

async function loadEvalSandboxProfile(
  root: string,
  fixture: EvalDefinition,
): Promise<{
  network: boolean;
  childProcess: boolean;
  credentialEnv: readonly string[];
}> {
  const path = fixture.tools.sandbox;
  if (!path) throw new Error("cave_live_eval_sandbox_profile_missing");
  const fullPath = resolve(root, path);
  const relativePath = relative(root, fullPath);
  if (relativePath === ".." || relativePath.startsWith("../") ||
      relativePath.startsWith("..\\") || isAbsolute(relativePath)) {
    throw new Error("cave_live_eval_sandbox_profile_escapes_root");

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Catch the error and read `.cause` — the remediation depends entirely on the wrapped failure.
2. If the cause mentions a missing result field, align runtime and CLI versions (reinstall one package, not a mix).
3. Re-run with the same seed to reproduce; if the fixture run itself crashed mid-flight, fix the underlying run error first.
4. Report with the cause stack if a stock configuration reproduces it.

Example fix

// reading the true failure
try {
  await build(args);
} catch (error) {
  if (error instanceof Error && error.message === "cave_fixture_terminal_evidence_missing") {
    console.error(error.cause); // the actual exception
  }
}

try {
  await build(args);
} catch (error) {
  if (error instanceof Error && error.message === "cave_fixture_terminal_evidence_missing" && error.cause) {
    console.error("underlying failure:", error.cause); // fix the cause, not the wrapper
  } else throw error;
}

When it happens

Trigger: An exception while assembling the evidence object after a fixture run: a result field unexpectedly undefined (e.g. `result.transformFailures` missing), a crash in `contentBlindRunEvidence`, or a runtime result shape that deviates from what the evidence builder assumes. Inspect `error.cause` for the true failure.

Common situations: Framework/runtime version skew where `runAgentInternal` returns a different result shape than the CLI expects; a custom runner option altering result fields; NaN/undefined propagating into sha256/stableStringify inside the evidence builder.

Related errors

• cave_live_eval_sandbox_profile_missing
• cave_harness_transform_not_evaluated
• cave_harness_unknown_transform
• caveman build: dataResidency is not enforced yet; refusing t
• caveman build: requiredFixturePassRate must be in (0,1]

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/13cd139aac8fbc77. Report an issue: GitHub.