Hmbown/CodeWhale · error · Error

Input exceeds 64 MiB.

Error message

Input exceeds 64 MiB.

What it means

The load() helper stats the --input file and refuses to read it if it exceeds 64 MiB (64 * 1024 * 1024 bytes). This bound keeps importTrace's full-file readFile into memory bounded, since the converter parses the whole trace into a single string before importing.

Solutions

1. Split the input into traces or time windows under 64 MiB and convert each separately, selecting with --trace=<id>.
2. Trim or gzip-then-filter the file to only the events you need before converting.
3. Record from the live source with --runtime/--thread (streaming, segmented) instead of converting a giant captured file.
4. If the file is genuinely under 64 MiB, check the path — stat on a directory or a growing file may report more than expected.

Example fix

// before
node scripts/pet.mjs --input=huge-recording.jsonl --output=pet.jsonl
// after
split -b 60m huge-recording.jsonl part-
node scripts/pet.mjs --input=part-aa --output=pet-aa.jsonl

import { stat } from 'node:fs/promises';
const MAX = 64 * 1024 * 1024;
if ((await stat(inputFile)).size > MAX) {
  console.error('split or filter the trace before converting');
} else {
  // safe to invoke pet.mjs --input=...
}

try {
  await runPetCli(['--input=' + file, '--output=' + out]);
} catch (err) {
  if (err.message === 'Input exceeds 64 MiB.') {
    console.error(`split ${file} (<64 MiB) or convert per-trace windows`);
  }
  throw err;
}

When it happens

Trigger: Calling load() via scripts/pet.mjs --input=<file> where stat(file).size > 67108864 bytes — e.g. pointing at a very large or live-appending trace JSONL instead of a bounded export.

Common situations: Converting a long-running recording journal that grew past 64 MiB; passing the live OUTPUT.segment-NNNNNN.jsonl chain's directory or a merged dump; forgetting that trace exports should be bounded windows.

Understand the failure class

Background: "File too large" / "file size exceeds limit" errors: why libraries cap file sizes and how to fix them — this error's family across 46 libraries.

Related errors

• bundle at is bytes; the limit is bytes
• Complete local diff requires
• evidence metadata exceeds limit
• exceeds the tombstone size limit
• legacy import refused: symlink or size bound

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/9e63698e356d35c3. Report an issue: GitHub.

Appendix: source

Thrown at pet/scripts/pet.mjs:28

import { createPetRecorder } from './lib/pet-recorder.mjs';

const args = process.argv.slice(2);
const option = name => args.find(a => a.startsWith(`--${name}=`))?.slice(name.length + 3);
if (args.includes('--help')) {
  console.log('node scripts/pet.mjs --input=trace.jsonl --output=pet.jsonl [--trace=ID] [--format=jsonl|tsv] [--watch]\nnode scripts/pet.mjs --runtime=http://127.0.0.1:7878 --thread=ID --output=pet.jsonl [--segment-buckets=216000] [--resume]\nUse --demo instead of --input for synthetic telemetry. Output must not already exist unless --resume is used for live recording. A resumed recorder preserves the previous segment and starts unknown at the same path.\nLive recording rotates at 216000 buckets or 64 MiB into OUTPUT.segment-NNNNNN.jsonl and continues at the same live path. All archives are retained.\nRuntime reads only the existing local event journal. Optional authentication comes from CODEWHALE_RUNTIME_TOKEN; never put a token in the URL. No agent or provider is started.');
  process.exit(0);
}
let output, recorder, monitor, timer, runtime;
try {
  for (const a of args) if (!['--demo', '--watch', '--resume'].includes(a) && !/^--(input|output|trace|format|runtime|thread|segment-buckets)=.+/.test(a)) throw new Error('Unknown or empty option. Use --help.');
  const input = option('input'), runtimeURL = option('runtime'), path = option('output'), format = option('format') ?? 'jsonl', live = args.includes('--watch') || !!runtimeURL;
  if (!path || [!!input, args.includes('--demo'), !!runtimeURL].filter(Boolean).length !== 1
    || !['jsonl', 'tsv'].includes(format) || live && format !== 'jsonl' || args.includes('--watch') && !input
    || !!runtimeURL !== !!option('thread') || option('trace') && !input || option('segment-buckets') && !live || args.includes('--resume') && !live)
    throw new Error('Choose one input source, an unused --output path (or --resume), and JSONL for live recording. Runtime requires --thread.');
  const load = async () => {
    if (!input) return { events: petDemoEvents(), duration: 80_000 };
    if ((await stat(input)).size > 64 * 1024 * 1024) throw new Error('Input exceeds 64 MiB.');
    const traces = importTrace(await readFile(input, 'utf8'), input, { privacy: 'metadata' });
    const trace = option('trace') ? traces.find(t => t.id === option('trace')) : traces.length === 1 ? traces[0] : undefined;
    if (!trace) throw new Error('Select an existing --trace ID when input contains multiple traces.');
    return trace;
  };
  let trace = runtimeURL ? undefined : await load(), buckets = compilePetTelemetry(trace?.events ?? [], trace?.duration ?? 0);
  if (live) recorder = await createPetRecorder(path, { resume: args.includes('--resume'), maxBuckets: option('segment-buckets') === undefined ? 216_000 : Number(option('segment-buckets')), report: text => console.error(text) });
  else output = await open(path, 'wx', 0o600);
  if (runtimeURL) runtime = await followRuntime({ baseUrl: runtimeURL, threadId: option('thread'),
    token: process.env.CODEWHALE_RUNTIME_TOKEN, report: text => console.error(text) });
  if (!live) {
    await output.writeFile(format === 'tsv' ? encodePetTSV(buckets) : encodePetJSONL(buckets));
    await output.close(); output = undefined;
    console.log(`Wrote ${buckets.length} pet buckets (${args.includes('--demo') ? 'demo' : 'trace replay'}).`);
  } else {
    // The driver owns wall time. The core only sees recorded relative timestamps.
    const started = performance.now(), startedWall = Date.now();
    const origin = trace && 'originTime' in trace && trace.originTime ? Date.parse(trace.originTime) : NaN;

View on GitHub (pinned to 433685b202)