koala73/worldmonitor · error · Error

batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportBatchByte

Error message

batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportBatchBytes} bytes

What it means

Thrown by normalizeCompanyImportBatch() when the UTF-8 byte size of JSON.stringify(normalized) exceeds COMPANY_MONITORING_LIMITS.maxImportBatchBytes (256 KiB). This is the last batch guard: even a row-count-legal batch (<=100 rows) can exceed the serialized byte cap, since each individual row may carry up to maxImportRowBytes (8 KiB).

Source

Thrown at shared/company-monitoring-contract.ts:322

  return normalized;
}

export function normalizeCompanyImportBatch(inputs: CompanyImportRowInput[]): NormalizedCompanyImportRow[] {
  if (!Array.isArray(inputs)) throw new Error('import batch must be a list');
  if (inputs.length === 0) throw new Error('import batch requires at least one row');
  if (inputs.length > COMPANY_MONITORING_LIMITS.maxImportRows) {
    throw new Error(`batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportRows} rows`);
  }

  const normalized = inputs.map(normalizeCompanyImportRow).sort((left, right) => left.ordinal - right.ordinal);
  const importId = normalized[0]?.clientImportId;
  for (let index = 0; index < normalized.length; index += 1) {
    const row = normalized[index]!;
    if (row.clientImportId !== importId) throw new Error('batch rows must share one clientImportId');
    if (row.ordinal !== index) throw new Error('batch ordinals must be contiguous from 0');
  }
  if (utf8Bytes(JSON.stringify(normalized)) > COMPANY_MONITORING_LIMITS.maxImportBatchBytes) {
    throw new Error(`batch exceeds ${COMPANY_MONITORING_LIMITS.maxImportBatchBytes} bytes`);
  }
  return normalized;
}

export function assertCompanyMonitoringPayloadSize(byteLength: number): void {
  if (!Number.isSafeInteger(byteLength) || byteLength < 0) throw new Error('request byte length is invalid');
  if (byteLength > COMPANY_MONITORING_LIMITS.maxRequestBytes) {
    throw new Error(`request exceeds ${COMPANY_MONITORING_LIMITS.maxRequestBytes} bytes`);
  }
}

export function assertCompanyMonitoringAccountContext(
  context: { ownerAccountId?: string } | null | undefined,
): string {
  const ownerAccountId = context?.ownerAccountId;
  if (typeof ownerAccountId !== 'string' || !ownerAccountId.trim()) {
    throw new Error('account context is required');
  }

View on GitHub (pinned to eeab0a219f)

Solutions

  1. Reduce the number of rows per batch — with worst-case 8 KiB rows, roughly 30 rows keeps you under 256 KiB; compute your own safe chunk size from measured row sizes
  2. Trim or truncate large text fields before import instead of shipping full documents inside rows
  3. Pre-measure with the same method the contract uses: new TextEncoder().encode(JSON.stringify(rows)).length > 262144, and split before sending

Example fix

// before
await submitImportBatch(normalizeCompanyImportBatch(allRows)); // 90 rows x ~6 KiB -> >256 KiB, throws

// after
const enc = new TextEncoder();
const MAX = COMPANY_MONITORING_LIMITS.maxImportBatchBytes;
let chunk: Row[] = [];
for (const row of allRows) {
  const next = [...chunk, row];
  if (enc.encode(JSON.stringify(next)).length > MAX) {
    await submitImportBatch(normalizeCompanyImportBatch(chunk));
    chunk = [];
  }
  chunk.push(row);
}
if (chunk.length) await submitImportBatch(normalizeCompanyImportBatch(chunk));
Defensive patterns

Strategy: validation

Validate before calling

const enc = new TextEncoder();
const MAX_BATCH = COMPANY_MONITORING_LIMITS.maxImportBatchBytes; // 256 KiB
function fitsBatchBudget(rows: unknown[]): boolean {
  return enc.encode(JSON.stringify(rows)).length <= MAX_BATCH;
}
// chunk by measured bytes, not row count alone:
function chunkByBytes<T>(rows: T[]): T[][] { /* grow chunk until fitsBatchBudget fails, then flush */ }

Try / catch

catch (e) { if (e instanceof Error && e.message.endsWith(' bytes')) { shrinkBatchAndRetry(); } else throw e; }

Prevention

When it happens

Trigger: 100 rows each near the 8 KiB per-row cap (100 x 8 KiB = 800 KiB >> 256 KiB); rows with long free-text fields (notes, descriptions, evidence text) that inflate the serialized form; Unicode content whose UTF-8 encoding is much larger than the JS string length suggests.

Common situations: A CSV import with a large notes column; importers that copy full article/evidence text into each row; CJK or emoji-heavy content measured with .length (UTF-16 code units) instead of byte length during client-side estimation.

Related errors

AI-assisted analysis of koala73/worldmonitor@eeab0a219f (2026-08-21). Data as JSON: /api/errors/afb9c999e246529b. Report an issue: GitHub.