JuliusBrussee/caveman · error · Error

cave_live_eval_sandbox_profile_invalid

Error message

cave_live_eval_sandbox_profile_invalid

What it means

Thrown by `loadEvalSandboxProfile` when the profile JSON parses but fails a strict schema check: the object must have exactly the four sorted keys `child_process, credential_env, network, schema_version` (no extras, none missing), `schema_version === 1`, `network` and `child_process` boolean, and `credential_env` an array of strings each matching `^[A-Z][A-Z0-9_]{0,127}$`. Any deviation — extra key, wrong type, malformed env name — throws before `validateSandboxCredentialEnv` even runs.

Source

Thrown at packages/agent/src/cli.ts:1250

  credentialEnv: readonly string[];
}> {
  const path = fixture.tools.sandbox;
  if (!path) throw new Error("cave_live_eval_sandbox_profile_missing");
  const fullPath = resolve(root, path);
  const relativePath = relative(root, fullPath);
  if (relativePath === ".." || relativePath.startsWith("../") ||
      relativePath.startsWith("..\\") || isAbsolute(relativePath)) {
    throw new Error("cave_live_eval_sandbox_profile_escapes_root");
  }
  const parsed = JSON.parse(await readFile(fullPath, "utf8")) as Record<string, unknown>;
  const keys = Object.keys(parsed).sort();
  const expected = ["child_process", "credential_env", "network", "schema_version"];
  if (keys.length !== expected.length || keys.some((key, index) => key !== expected[index]) ||
      parsed.schema_version !== 1 || typeof parsed.network !== "boolean" ||
      typeof parsed.child_process !== "boolean" || !Array.isArray(parsed.credential_env) ||
      parsed.credential_env.some((name) => typeof name !== "string" ||
        !/^[A-Z][A-Z0-9_]{0,127}$/.test(name))) {
    throw new Error("cave_live_eval_sandbox_profile_invalid");
  }
  validateSandboxCredentialEnv(parsed.credential_env as string[]);
  return {
    network: parsed.network,
    childProcess: parsed.child_process,
    credentialEnv: parsed.credential_env as string[],
  };
}

function contextIRIsContentBlind(ir: Awaited<ReturnType<typeof lowerContext>>["ir"]): boolean {
  const allowed = new Set([
    "id", "kind", "stability", "safety", "priority", "recovery", "cacheRegion",
    "privacy", "opaque", "ttlTurns", "provenanceDigest", "tokenCount", "bodyHandle",
  ]);
  return ir.segments.every((segment) =>
    Object.keys(segment).every((key) => allowed.has(key)) &&
    /^cave_local_sha256:[0-9a-f]{64}$/.test(segment.bodyHandle) &&
    /^[0-9a-f]{64}$/.test(segment.provenanceDigest));

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Rewrite the profile to exactly the four keys with schema_version 1 and correct types.
2. Normalize credential_env entries to UPPER_SNAKE_CASE (start A-Z, then A-Z0-9_, max 128 chars).
3. Validate profiles in a pre-test step or CI lint so drift is caught before `caveman build`.

Example fix

// before
{ "schema_version": 1, "network": "true", "child_process": false, "credential_env": ["api-key"], "notes": "tmp" }

// after
{ "schema_version": 1, "network": true, "child_process": false, "credential_env": ["API_KEY"] }

function isValidSandboxProfile(parsed: unknown): boolean {
  if (typeof parsed !== "object" || parsed === null) return false;
  const p = parsed as Record<string, unknown>;
  const keys = Object.keys(p).sort();
  const expected = ["child_process", "credential_env", "network", "schema_version"];
  return keys.length === 4 && keys.every((k, i) => k === expected[i]) &&
    p.schema_version === 1 && typeof p.network === "boolean" &&
    typeof p.child_process === "boolean" &&
    Array.isArray(p.credential_env) &&
    (p.credential_env as unknown[]).every((n) => typeof n === "string" && /^[A-Z][A-Z0-9_]{0,127}$/.test(n));
}

type SandboxProfile = { schema_version: 1; network: boolean; child_process: boolean; credential_env: string[] };
function isSandboxProfile(v: unknown): v is SandboxProfile {
  return isValidSandboxProfile(v);
}

try {
  await build(args);
} catch (error) {
  if (error instanceof Error && error.message === "cave_live_eval_sandbox_profile_invalid") {
    // rewrite the profile to the exact 4-key schema with UPPER_SNAKE_CASE env names
  } else throw error;
}

When it happens

Trigger: A sandbox profile with an added convenience key (e.g. `"notes": ...`), `schema_version: 2` or missing, `network: "false"` as a string, or a credential env name like `github_token` (lowercase) or one longer than 128 chars. Hand-edited profiles are the usual source.

Common situations: Authors adding comments/metadata into the JSON; profiles written for a future schema version; env-var lists copied from shell exports with lowercase names or leading digits.

Related errors

• cave_live_eval_sandbox_profile_missing
• cave_live_eval_sandbox_profile_escapes_root
• caveman agent: unknown sandbox mode ${JSON.stringify(sandbox
• caveman agent: tool Standard JSON Schema converter is invali
• caveman agent: eval id is required

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/83bdbdbdaa4c6617. Report an issue: GitHub.