{"record":{"id":"389dc6cb36a7a2a1","repo":"JuliusBrussee/caveman","slug":"eval-evidence-file-exceeds-4-mib-batch-limit","errorCode":null,"errorMessage":"eval evidence file exceeds 4 MiB batch limit","messagePattern":"eval evidence file exceeds 4 MiB batch limit","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/cli/src/index.ts","lineNumber":17145,"sourceCode":"  };\n  const fieldMap = flagFrom(argv, \"--field-map\", \"\");\n  if (fieldMap) headers[\"x-cave-field-map\"] = fieldMap;\n  const response = await fetch(`${cfg.baseURL}/api/v1/imports?format=${encodeURIComponent(format)}`, {\n    method: \"POST\",\n    headers,\n    body: data\n  });\n  print(await response.json());\n}\n\n// auditEvalImport turns newline-delimited, source-neutral eval records into one\n// bounded batch. Server stamps tenant scope, observed basis, and external-only\n// authority; CI cannot promote its own results into rollout authority.\nasync function auditEvalImport(argv: string[]) {\n  const file = positionalAfterOptions(argv.slice(1), new Set([\"--project\"]));\n  if (!file) throw new Error(`usage: ${invokedCommand(\"audit\")} eval-import <evidence.jsonl> [--project <uuid>] [--dry-run]`);\n  const data = await readFile(file);\n  if (data.byteLength > 4 * 1024 * 1024) throw new Error(\"eval evidence file exceeds 4 MiB batch limit\");\n\n  const items: Record<string, unknown>[] = [];\n  for (const [index, rawLine] of data.toString(\"utf8\").split(/\\r?\\n/).entries()) {\n    const line = rawLine.trim();\n    if (!line) continue;\n    let parsed: unknown;\n    try {\n      parsed = JSON.parse(line);\n    } catch {\n      throw new Error(`eval evidence line ${index + 1} is not valid JSON`);\n    }\n    if (!parsed || typeof parsed !== \"object\" || Array.isArray(parsed)) {\n      throw new Error(`eval evidence line ${index + 1} must be a JSON object`);\n    }\n    items.push(parsed as Record<string, unknown>);\n    if (items.length > 1000) throw new Error(\"eval evidence batch exceeds 1000 records\");\n  }\n  if (items.length === 0) throw new Error(\"eval evidence file contains no records\");","sourceCodeStart":17127,"sourceCodeEnd":17163,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/5184b3d11ac6a1acb7d44b9bfaa31698157cff97/packages/cli/src/index.ts#L17127-L17163","documentation":"Hard batch-size guard inside `caveman audit eval-import`: the evidence file must be at most 4 MiB (4 * 1024 * 1024 bytes). The limit keeps a single import request bounded; larger files are rejected client-side before any line is parsed or any upload starts.","triggerScenarios":"A JSONL export larger than 4 MiB — long eval runs with many records, or records that inline large payloads such as full prompts and completions.","commonSituations":"CI artifacts that grow over months until they cross the limit; verbose eval records embedding whole transcripts; concatenating several runs into one file before importing.","solutions":["Split the JSONL into files under 4 MiB and import each (also respect the 1000-record cap)","Trim bloated fields from records before export","Store references to large content instead of inlining it in evidence records"],"exampleFix":"# before\ncaveman audit eval-import big-evidence.jsonl\n# after\nsplit -l 500 big-evidence.jsonl chunk-\nfor f in chunk-*; do caveman audit eval-import \"$f\"; done","handlingStrategy":"validation","validationCode":"import { stat } from 'node:fs/promises';\nconst { size } = await stat(file);\nif (size > 4 * 1024 * 1024) throw new Error(`evidence too large (${size} bytes) — split into <4 MiB batches`);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check file size in CI before the import step","Standardize on batches of at most 1000 records and under 4 MiB","Stream-split long eval runs instead of producing one large artifact"],"tags":["cli","file-size","eval","import","limits"],"backgroundTag":"request-body-too-large","analyzedSha":"5184b3d11ac6a1acb7d44b9bfaa31698157cff97","analyzedAt":"2026-08-18T03:14:35.516Z","contentChangedAt":"2026-08-18T03:14:35.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}