{"record":{"id":"13cd139aac8fbc77","repo":"JuliusBrussee/caveman","slug":"cave-fixture-terminal-evidence-missing","errorCode":null,"errorMessage":"cave_fixture_terminal_evidence_missing","messagePattern":"cave_fixture_terminal_evidence_missing","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/agent/src/cli.ts","lineNumber":1222,"sourceCode":"      graders,\n      latency_ms: result.latencyMs,\n      provider_visible_tokens: result.inputTokens + result.cacheReadTokens + result.cacheWriteTokens,\n      cache_prefix_sha256: result.cachePrefixSHA256,\n      cache_boundary_known: result.cacheBoundaryKnown,\n      cache_read_tokens: result.cacheReadTokens,\n      cache_write_tokens: result.cacheWriteTokens,\n      cache_bust: result.cacheBust,\n      error: false,\n      recovery_resolved: result.recoveryResolved && result.transformFailures.length === 0,\n      privacy_passed: privacyConformance && contentBlindRunEvidence(result, fixture),\n      sandbox_passed: sandboxConformance && (definition.tools.length === 0 ||\n        (definition.sandbox === \"required\" && result.toolCalls.length > 0)),\n      unknown_transform: plan.segment_routes.some((route) =>\n        !result.evaluatedTransformIDs.includes(route.transform_id)),\n      output_digest: sha256(result.text),\n    };\n  } catch (error) {\n    throw new Error(\"cave_fixture_terminal_evidence_missing\", { cause: error });\n  }\n}\n\nasync function loadEvalSandboxProfile(\n  root: string,\n  fixture: EvalDefinition,\n): Promise<{\n  network: boolean;\n  childProcess: boolean;\n  credentialEnv: readonly string[];\n}> {\n  const path = fixture.tools.sandbox;\n  if (!path) throw new Error(\"cave_live_eval_sandbox_profile_missing\");\n  const fullPath = resolve(root, path);\n  const relativePath = relative(root, fullPath);\n  if (relativePath === \"..\" || relativePath.startsWith(\"../\") ||\n      relativePath.startsWith(\"..\\\\\") || isAbsolute(relativePath)) {\n    throw new Error(\"cave_live_eval_sandbox_profile_escapes_root\");","sourceCodeStart":1204,"sourceCodeEnd":1240,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/packages/agent/src/cli.ts#L1204-L1240","documentation":"A defensive catch-all around terminal-evidence construction in `runFixture`: the block builds the run's evidence record (token counts, cache stats, recovery, privacy/sandbox conformance flags, unknown-transform detection, output digest) from the completed `runAgentInternal` result. Any failure in that block is re-thrown as `cave_fixture_terminal_evidence_missing` with the original error attached as `cause`. The build cannot write a lock whose eval evidence is incomplete or malformed.","triggerScenarios":"An exception while assembling the evidence object after a fixture run: a result field unexpectedly undefined (e.g. `result.transformFailures` missing), a crash in `contentBlindRunEvidence`, or a runtime result shape that deviates from what the evidence builder assumes. Inspect `error.cause` for the true failure.","commonSituations":"Framework/runtime version skew where `runAgentInternal` returns a different result shape than the CLI expects; a custom runner option altering result fields; NaN/undefined propagating into sha256/stableStringify inside the evidence builder.","solutions":["Catch the error and read `.cause` — the remediation depends entirely on the wrapped failure.","If the cause mentions a missing result field, align runtime and CLI versions (reinstall one package, not a mix).","Re-run with the same seed to reproduce; if the fixture run itself crashed mid-flight, fix the underlying run error first.","Report with the cause stack if a stock configuration reproduces it."],"exampleFix":"// reading the true failure\ntry {\n  await build(args);\n} catch (error) {\n  if (error instanceof Error && error.message === \"cave_fixture_terminal_evidence_missing\") {\n    console.error(error.cause); // the actual exception\n  }\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  await build(args);\n} catch (error) {\n  if (error instanceof Error && error.message === \"cave_fixture_terminal_evidence_missing\" && error.cause) {\n    console.error(\"underlying failure:\", error.cause); // fix the cause, not the wrapper\n  } else throw error;\n}","preventionTips":["Always inspect `error.cause` — the wrapper hides the real failure.","Keep runtime and CLI packages at the same release so result shapes agree.","Don't alter run results with custom runner options when building locks."],"tags":["eval","evidence","build","wrapper"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}