JuliusBrussee/caveman · error · Error

cave_live_eval_sandbox_profile_escapes_root

Error message

cave_live_eval_sandbox_profile_escapes_root

What it means

Thrown by `loadEvalSandboxProfile` when the fixture's sandbox profile path, resolved against the project root, escapes the root: `relative(root, fullPath)` starts with `..`, equals `..`, or is absolute. The sandbox profile is trusted security configuration, so it must live inside the project; a path pointing outside (absolute path, or `../` traversal, including Windows backslash variants) is rejected before the JSON is ever read.

Source

Thrown at packages/agent/src/cli.ts:1240

    throw new Error("cave_fixture_terminal_evidence_missing", { cause: error });
  }
}

async function loadEvalSandboxProfile(
  root: string,
  fixture: EvalDefinition,
): Promise<{
  network: boolean;
  childProcess: boolean;
  credentialEnv: readonly string[];
}> {
  const path = fixture.tools.sandbox;
  if (!path) throw new Error("cave_live_eval_sandbox_profile_missing");
  const fullPath = resolve(root, path);
  const relativePath = relative(root, fullPath);
  if (relativePath === ".." || relativePath.startsWith("../") ||
      relativePath.startsWith("..\\") || isAbsolute(relativePath)) {
    throw new Error("cave_live_eval_sandbox_profile_escapes_root");
  }
  const parsed = JSON.parse(await readFile(fullPath, "utf8")) as Record<string, unknown>;
  const keys = Object.keys(parsed).sort();
  const expected = ["child_process", "credential_env", "network", "schema_version"];
  if (keys.length !== expected.length || keys.some((key, index) => key !== expected[index]) ||
      parsed.schema_version !== 1 || typeof parsed.network !== "boolean" ||
      typeof parsed.child_process !== "boolean" || !Array.isArray(parsed.credential_env) ||
      parsed.credential_env.some((name) => typeof name !== "string" ||
        !/^[A-Z][A-Z0-9_]{0,127}$/.test(name))) {
    throw new Error("cave_live_eval_sandbox_profile_invalid");
  }
  validateSandboxCredentialEnv(parsed.credential_env as string[]);
  return {
    network: parsed.network,
    childProcess: parsed.child_process,
    credentialEnv: parsed.credential_env as string[],
  };
}

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Move the sandbox profile inside the project root and reference it with a root-relative path.
2. For shared monorepo profiles, place them under the package root or copy/sync them in during build.
3. Avoid absolute paths in eval definitions — they also break portability across machines.

Example fix

// before
tools: { sandbox: "../../shared/sandbox.json" }

// after: copy the profile into the project
tools: { sandbox: "evals/sandbox.json" }

import { relative, isAbsolute, resolve } from "node:path";

function sandboxPathInRoot(root: string, p: string): boolean {
  const rel = relative(root, resolve(root, p));
  return rel !== ".." && !rel.startsWith("..") && !rel.startsWith("..\\") && !isAbsolute(rel);
}

try {
  await build(args);
} catch (error) {
  if (error instanceof Error && error.message === "cave_live_eval_sandbox_profile_escapes_root") {
    // move the profile under the project root and use a relative path
  } else throw error;
}

When it happens

Trigger: A fixture with `tools.sandbox: "/etc/sandbox.json"` or `"../../shared/sandbox.json"`. Also reachable via symlinks that make the resolved absolute path fall outside root — realpath resolution means a link pointing out of the workspace is out.

Common situations: Monorepo teams pointing evals at a shared profile outside the package directory; absolute paths authored on one machine breaking on another; symlinked config directories (dotfiles) whose realpath lives outside the repo.

Related errors

• caveman-code: path escapes the workspace: ${candidate}
• probe returned false
• cave_sandbox_conformance_failed
• cave_live_eval_sandbox_profile_missing
• cave_live_eval_sandbox_profile_invalid

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/5eea8ee6a8039361. Report an issue: GitHub.