Hmbown/CodeWhale · error · Error

File exceeds the 64 MiB MVP import limit. Split the export…

Error message

File exceeds the 64 MiB MVP import limit. Split the export by trace.

What it means

importTrace enforces a hard 64 MiB limit on the raw input text (measured in UTF-8 bytes), overridable only downward via options.maxBytes. Oversized files are rejected before parsing so a huge export cannot exhaust memory in the browser/runtime; the message tells you to split the export per trace.

Solutions

Split the export into multiple files by trace ID and import each separately.
Raise or re-check the limit only if you set options.maxBytes yourself; the built-in 64 MiB cap is not configurable upward.
Reduce imported data at the source (sampling, shorter time window, fewer attributes) before export.
Pre-filter the text before calling importTrace to remove unneeded traces.

Example fix

// before
importTrace(hugeText); // > 64 MiB
// after
const chunks = splitByTrace(hugeText); // split export per trace ID
chunks.forEach(c => importTrace(c));

Defensive patterns

Strategy: validation

Validate before calling

if (new TextEncoder().encode(text).length > 64 * 1024 * 1024) {
  throw new Error('File too large: split by trace before importing.');
}

Try / catch

try {
  importTrace(text);
} catch (e) {
  if (e instanceof Error && e.message.includes('64 MiB')) {
    console.error('Split the export by trace ID and import each part.');
  }
}

Prevention

Check file size (bytes) before reading/importing.
Export per trace or per time slice instead of one giant file.
Remember the limit is UTF-8 byte length, not character count.
Set options.maxBytes only when you want a stricter cap; it cannot raise 64 MiB.

When it happens

Trigger: Passing a JSON or JSONL trace file whose UTF-8 byte length exceeds 64 MiB (or the smaller options.maxBytes if set) to importTrace.

Common situations: Long-running services exporting days of OTLP spans into one file; cumulative exports that kept growing; an options.maxBytes override set smaller than the actual file.

Understand the failure class

Background: "File too large" / "file size exceeds limit" errors: why libraries cap file sizes and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/15d9ec245b6712d2. Report an issue: GitHub.

Appendix: source

Thrown at pet/src/core/ingest.ts:224

      const ea = attributes(record.attributes), time = Number(ns(record.timeUnixNano, 'event.timeUnixNano') - origin) / 1e6;
      const ename = String(record.name ?? 'span event');
      events.push({ schemaVersion: 1, id: `${s.spanId}/event/${i}`, traceId: s.traceId, parentId: s.spanId,
        startTime: time, endTime: time, agentId: e.agentId, name: ename, category: categoryFor(ename, ea),
        status: ename === 'exception' ? 'error' : 'unknown', attributes: ea, raw: { event: record, spanId: s.spanId, resource: rec.resource, scope: rec.scope } });
    }
    if (events.length > maxEvents) throw new Error(`Import exceeds the ${maxEvents.toLocaleString()} event limit, including span events.`);
  }
  return { events, origins: new Map([...bases].map(([k, v]) => [k, v.toString()])), warnings };
}

/** Parse strictly: malformed lines or duplicate identities never disappear silently. */
export function importTrace(text: string, filename = 'Imported trace', options: ImportOptions = {}): Trace[] {
  const mode = options.privacy ?? 'redact', maxEvents = options.maxEvents ?? 250_000;
  if (!['redact', 'metadata', 'retain'].includes(mode)) throw new Error('Unknown privacy mode.');
  const maxTraces = options.maxTraces ?? 8;
  if (!Number.isInteger(maxEvents) || maxEvents < 1 || maxEvents > 250_000) throw new Error('maxEvents must be in [1, 250000].');
  if (!Number.isInteger(maxTraces) || maxTraces < 1 || maxTraces > 64) throw new Error('maxTraces must be in [1, 64].');
  if (new TextEncoder().encode(text).length > (options.maxBytes ?? 64 * 1024 * 1024)) throw new Error('File exceeds the 64 MiB MVP import limit. Split the export by trace.');
  const trimmed = text.replace(/^\uFEFF/, '').trim();
  if (!trimmed) throw new Error('The trace file is empty.');
  let document: unknown;
  try { document = JSON.parse(trimmed); }
  catch {
    document = trimmed.split(/\r?\n/).filter(l => l.trim()).map((line, i) => {
      try { return JSON.parse(line); } catch { throw new Error(`Invalid JSON on nonempty line ${i + 1}. Import cancelled; no rows were skipped.`); }
    });
  }
  // Transform before both normalization and raw retention, so raw cannot bypass redaction.
  const safe = mode === 'retain' ? (options.redactor ? options.redactor(document, '') : document) : redact(document, '', options.redactor);
  const root = obj(safe);
  if(root.format === 'whalesong.evidence/v1') return [evidenceToTrace(validateBundle(root, Math.min(maxEvents, 100_000)))];
  if (isCodewhaleSession(safe)) {
    const trace = fromCodewhaleSession(safe, filename, maxEvents);
    trace.privacy = mode;
    trace.events = trace.events.map(e => privacyEvent(e, mode));
    return [trace];

View on GitHub (pinned to 433685b202)