JuliusBrussee/caveman · error

cave_live_eval_sandbox_profile_escapes_root

cave_live_eval_sandbox_profile_escapes_root

Error message

cave_live_eval_sandbox_profile_escapes_root

What it means

loadEvalSandboxProfile() resolves the fixture's sandbox profile path against the project root and rejects any profile that resolves outside the root: if the root-relative path is '..', starts with '../' (or '..\\') or is absolute, it throws 'cave_live_eval_sandbox_profile_escapes_root'. This path-containment check guarantees live eval sandbox profiles live inside the project, preventing profiles (and the credentials they reference) from being loaded from arbitrary locations.

Solutions

1. Move the sandbox profile into the project (e.g. eval/sandbox.profile.json) and point fixture.tools.sandbox at the project-relative path
2. Copy the shared profile into this project instead of referencing it via ../
3. Reject or sanitize any programmatically supplied sandbox path that contains .. or is absolute

Example fix

// before
{ tools: { mode: "live", sandbox: "../shared/sandbox.profile.json" } }
// after
{ tools: { mode: "live", sandbox: "eval/sandbox.profile.json" } }

import { resolve, relative, isAbsolute } from "node:path";
function sandboxPathIsContained(root: string, sandbox: string): boolean {
  const rel = relative(root, resolve(root, sandbox));
  return !(rel === ".." || rel.startsWith("../") || rel.startsWith("..\\") || isAbsolute(rel));
}
for (const f of evals.filter((x) => x.tools.mode === "live")) {
  if (!sandboxPathIsContained(root, f.tools.sandbox!)) {
    throw new Error(`fixture '${f.id}' sandbox path escapes project root`);
  }
}

try {
  await loadEvalSandboxProfile(root, fixture);
} catch (error) {
  if ((error as Error).message === "cave_live_eval_sandbox_profile_escapes_root") {
    throw new Error(`fixture '${fixture.id}' sandbox profile must be inside the project root; copy the profile into the repo and use a relative path`);
  }
  throw error;
}

When it happens

Trigger: Declaring fixture.tools.sandbox as a path containing '..' segments (e.g. '../shared/profile.json' or '../../etc/profile.json'), an absolute filesystem path, or any path that escapes the project root, while running a live-mode fixture.

Common situations: Sharing one sandbox profile across projects via a parent-directory relative path; passing an absolute path like '/etc/caveman/profile.json'; symlinks/..-heavy paths constructed from user input in generated fixtures.

Understand the failure class

Background: Path traversal blocked: "path escapes the workspace" and "outside site root" errors when a path will not stay inside its allowed directory — this error's family across 26 libraries.

Related errors

• cave_sandbox_source_read_grant_escapes_staging
• cave_tool_sandbox_entry_escapes_root
• cave_tool_sandbox_source_escapes_root
• cave_host_sandbox_nested_under_required
• cave_live_eval_sandbox_profile_missing

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/5eea8ee6a8039361. Report an issue: GitHub.

Appendix: source

Thrown at packages/agent/src/cli.ts:1240

    throw new Error("cave_fixture_terminal_evidence_missing", { cause: error });
  }
}

async function loadEvalSandboxProfile(
  root: string,
  fixture: EvalDefinition,
): Promise<{
  network: boolean;
  childProcess: boolean;
  credentialEnv: readonly string[];
}> {
  const path = fixture.tools.sandbox;
  if (!path) throw new Error("cave_live_eval_sandbox_profile_missing");
  const fullPath = resolve(root, path);
  const relativePath = relative(root, fullPath);
  if (relativePath === ".." || relativePath.startsWith("../") ||
      relativePath.startsWith("..\\") || isAbsolute(relativePath)) {
    throw new Error("cave_live_eval_sandbox_profile_escapes_root");
  }
  const parsed = JSON.parse(await readFile(fullPath, "utf8")) as Record<string, unknown>;
  const keys = Object.keys(parsed).sort();
  const expected = ["child_process", "credential_env", "network", "schema_version"];
  if (keys.length !== expected.length || keys.some((key, index) => key !== expected[index]) ||
      parsed.schema_version !== 1 || typeof parsed.network !== "boolean" ||
      typeof parsed.child_process !== "boolean" || !Array.isArray(parsed.credential_env) ||
      parsed.credential_env.some((name) => typeof name !== "string" ||
        !/^[A-Z][A-Z0-9_]{0,127}$/.test(name))) {
    throw new Error("cave_live_eval_sandbox_profile_invalid");
  }
  validateSandboxCredentialEnv(parsed.credential_env as string[]);
  return {
    network: parsed.network,
    childProcess: parsed.child_process,
    credentialEnv: parsed.credential_env as string[],
  };
}

View on GitHub (pinned to 3ee70a1026)