{"record":{"id":"afb9c999e246529b","repo":"koala73/worldmonitor","slug":"batch-exceeds-company-monitoring-limits-maximpor-afb9c9","errorCode":null,"errorMessage":"batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportBatchBytes} bytes","messagePattern":"batch exceeds (.+?) bytes","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"shared/company-monitoring-contract.ts","lineNumber":322,"sourceCode":"  return normalized;\n}\n\nexport function normalizeCompanyImportBatch(inputs: CompanyImportRowInput[]): NormalizedCompanyImportRow[] {\n  if (!Array.isArray(inputs)) throw new Error('import batch must be a list');\n  if (inputs.length === 0) throw new Error('import batch requires at least one row');\n  if (inputs.length > COMPANY_MONITORING_LIMITS.maxImportRows) {\n    throw new Error(`batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportRows} rows`);\n  }\n\n  const normalized = inputs.map(normalizeCompanyImportRow).sort((left, right) => left.ordinal - right.ordinal);\n  const importId = normalized[0]?.clientImportId;\n  for (let index = 0; index < normalized.length; index += 1) {\n    const row = normalized[index]!;\n    if (row.clientImportId !== importId) throw new Error('batch rows must share one clientImportId');\n    if (row.ordinal !== index) throw new Error('batch ordinals must be contiguous from 0');\n  }\n  if (utf8Bytes(JSON.stringify(normalized)) > COMPANY_MONITORING_LIMITS.maxImportBatchBytes) {\n    throw new Error(`batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportBatchBytes} bytes`);\n  }\n  return normalized;\n}\n\nexport function assertCompanyMonitoringPayloadSize(byteLength: number): void {\n  if (!Number.isSafeInteger(byteLength) || byteLength < 0) throw new Error('request byte length is invalid');\n  if (byteLength > COMPANY_MONITORING_LIMITS.maxRequestBytes) {\n    throw new Error(`request exceeds ${COMPANY_MONITORING_LIMITS.maxRequestBytes} bytes`);\n  }\n}\n\nexport function assertCompanyMonitoringAccountContext(\n  context: { ownerAccountId?: string } | null | undefined,\n): string {\n  const ownerAccountId = context?.ownerAccountId;\n  if (typeof ownerAccountId !== 'string' || !ownerAccountId.trim()) {\n    throw new Error('account context is required');\n  }","sourceCodeStart":304,"sourceCodeEnd":340,"githubUrl":"https://github.com/koala73/worldmonitor/blob/eeab0a219fce0f02a00603b532dbae9041b934ac/shared/company-monitoring-contract.ts#L304-L340","documentation":"Thrown by normalizeCompanyImportBatch() when the UTF-8 byte size of JSON.stringify(normalized) exceeds COMPANY_MONITORING_LIMITS.maxImportBatchBytes (256 KiB). This is the last batch guard: even a row-count-legal batch (<=100 rows) can exceed the serialized byte cap, since each individual row may carry up to maxImportRowBytes (8 KiB).","triggerScenarios":"100 rows each near the 8 KiB per-row cap (100 x 8 KiB = 800 KiB >> 256 KiB); rows with long free-text fields (notes, descriptions, evidence text) that inflate the serialized form; Unicode content whose UTF-8 encoding is much larger than the JS string length suggests.","commonSituations":"A CSV import with a large notes column; importers that copy full article/evidence text into each row; CJK or emoji-heavy content measured with .length (UTF-16 code units) instead of byte length during client-side estimation.","solutions":["Reduce the number of rows per batch — with worst-case 8 KiB rows, roughly 30 rows keeps you under 256 KiB; compute your own safe chunk size from measured row sizes","Trim or truncate large text fields before import instead of shipping full documents inside rows","Pre-measure with the same method the contract uses: new TextEncoder().encode(JSON.stringify(rows)).length > 262144, and split before sending"],"exampleFix":"// before\nawait submitImportBatch(normalizeCompanyImportBatch(allRows)); // 90 rows x ~6 KiB -> >256 KiB, throws\n\n// after\nconst enc = new TextEncoder();\nconst MAX = COMPANY_MONITORING_LIMITS.maxImportBatchBytes;\nlet chunk: Row[] = [];\nfor (const row of allRows) {\n  const next = [...chunk, row];\n  if (enc.encode(JSON.stringify(next)).length > MAX) {\n    await submitImportBatch(normalizeCompanyImportBatch(chunk));\n    chunk = [];\n  }\n  chunk.push(row);\n}\nif (chunk.length) await submitImportBatch(normalizeCompanyImportBatch(chunk));","handlingStrategy":"validation","validationCode":"const enc = new TextEncoder();\nconst MAX_BATCH = COMPANY_MONITORING_LIMITS.maxImportBatchBytes; // 256 KiB\nfunction fitsBatchBudget(rows: unknown[]): boolean {\n  return enc.encode(JSON.stringify(rows)).length <= MAX_BATCH;\n}\n// chunk by measured bytes, not row count alone:\nfunction chunkByBytes<T>(rows: T[]): T[][] { /* grow chunk until fitsBatchBudget fails, then flush */ }","typeGuard":null,"tryCatchPattern":"catch (e) { if (e instanceof Error && e.message.endsWith(' bytes')) { shrinkBatchAndRetry(); } else throw e; }","preventionTips":["Measure with TextEncoder on the serialized JSON — string .length undercounts UTF-8, especially CJK/emoji","Cap free-text fields at ingestion, not at submit","Assume worst-case rows (8 KiB each) when picking a client-side chunk size"],"tags":["validation","import","payload-size","utf8","company-monitoring"],"backgroundTag":"request-payload-too-large","analyzedSha":"eeab0a219fce0f02a00603b532dbae9041b934ac","analyzedAt":"2026-08-21T16:51:25.751Z","contentChangedAt":"2026-08-21T16:51:25.751Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}