ruvnet/RuView · error · Error

brain corpus exceeds 1 MiB

Error message

brain corpus exceeds 1 MiB

What it means

loadBrain enforces a hard 1 MiB (1_048_576 byte) cap on the canonical brain corpus (core.jsonl), measured after CRLF-to-LF normalization, keeping the committed JSONL small enough to review and load. A larger file throws immediately, before any line is parsed.

Source

Thrown at harness/ruview/src/brain.js:45

  if (!EVIDENCE.has(record.evidence)) errors.push(`unsupported evidence: ${record.evidence}`);
  if (!record.source || typeof record.source.path !== 'string' || !Number.isInteger(record.source.line) || record.source.line < 1) {
    errors.push('source.path and positive source.line are required');
  } else if (isAbsolute(record.source.path) || record.source.path.split(/[\\/]/).includes('..') || /^[A-Za-z]:/.test(record.source.path)) {
    errors.push('source.path must be repository-relative without traversal');
  }
  if (!Array.isArray(record.tags) || record.tags.some((tag) => typeof tag !== 'string')) errors.push('tags must be strings');
  if ((record.content || '').length > 8192) errors.push('content exceeds 8192 characters');
  if ((record.title || '').length > 200) errors.push('title exceeds 200 characters');
  if (canonical && record.reviewed !== true) errors.push('canonical records must be reviewed');
  const combined = `${record.title || ''}\n${record.content || ''}`;
  if (SECRET.test(combined)) errors.push('record appears to contain a secret');
  if (INJECTION.test(combined)) errors.push('record contains instruction-like prompt injection');
  return errors;
}

export function loadBrain(path = CORPUS_PATH) {
  const raw = readFileSync(path, 'utf8').replace(/\r\n/g, '\n');
  if (Buffer.byteLength(raw) > 1_048_576) throw new Error('brain corpus exceeds 1 MiB');
  const records = raw.split('\n').filter(Boolean).map((line, index) => {
    if (Buffer.byteLength(line) > 16_384) throw new Error(`brain line ${index + 1}: exceeds 16 KiB`);
    let record;
    try { record = JSON.parse(line); } catch (error) { throw new Error(`brain line ${index + 1}: ${error.message}`); }
    const errors = validateBrainRecord(record, { canonical: true });
    if (errors.length) throw new Error(`brain line ${index + 1}: ${errors.join('; ')}`);
    return Object.freeze(record);
  });
  if (records.length > 1000) throw new Error('brain corpus exceeds 1000 records');
  const ids = new Set();
  for (const record of records) {
    if (ids.has(record.id)) throw new Error(`duplicate brain id: ${record.id}`);
    ids.add(record.id);
  }
  return { records, digest: sha256(raw), bytes: Buffer.byteLength(raw) };
}

function terms(value) {

View on GitHub (pinned to 4685618388)

Solutions

1. Prune stale or superseded records from core.jsonl
2. Shorten record content (each record allows at most 8192 characters; keep far below)
3. Check size before committing: wc -c harness/ruview/brain/corpus/core.jsonl

Example fix

# before: corpus is 1.1 MiB and loadBrain() throws

# after: prune superseded records until below the cap
$ wc -c harness/ruview/brain/corpus/core.jsonl
1040000 harness/ruview/brain/corpus/core.jsonl

import { statSync } from 'node:fs';
function corpusWithinBudget(path) {
  const bytes = statSync(path).size;
  return { ok: bytes <= 1_048_576, bytes };
}

try {
  const brain = loadBrain(path);
} catch (error) {
  if (error.message.startsWith('brain corpus exceeds')) {
    console.error(`corpus too large: ${error.message} — prune superseded records`);
    process.exit(1);
  }
  throw error;
}

When it happens

Trigger: harness/ruview/brain/corpus/core.jsonl grows past 1 MiB after merging records — the check is on the whole file, so one large merge or long-term accumulation trips it.

Common situations: Bulk-importing records through PRs, appending verbose content, pointing loadBrain at an aggregated corpus during local experiments.

Related errors

• brain line ${index + 1}: exceeds 16 KiB
• brain corpus exceeds 1000 records
• brain line ${index + 1}: ${error.message}
• brain line ${index + 1}: ${errors.join('; ')}
• duplicate brain id: ${record.id}

AI-assisted analysis of ruvnet/RuView@4685618388 (2026-08-16). Data as JSON: /api/errors/c03239bea847afd7. Report an issue: GitHub.