affaan-m/ECC · error · Error

Memory document ${sourcePath} is too large.

Error message

Memory document ${sourcePath} is too large.

What it means

Thrown by parseMemoryDocument() when Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES (131072 bytes = 128 KiB). This is the whole-document cap; the body-only cap (MAX_BODY_BYTES = 64 KiB) is checked separately inside normalizeBody(). The document cap bounds the parsing cost and the per-file read budget inside readMemoryFiles().

Source

Thrown at scripts/lib/memory-vault-format.js:251

    throw new Error(`Duplicate memory frontmatter field in ${sourcePath}.`);
  }
  const rawValue = line.slice(separator + 1).trim();
  try {
    return { objectKey, value: JSON.parse(rawValue) };
  } catch {
    throw new Error(`Memory frontmatter field in ${sourcePath} must use a JSON value.`);
  }
}

function parseMemoryDocument(source, sourcePath = '<memory>') {
  const openingMarker = typeof source === 'string'
    ? /^---\r?\n/.exec(source)
    : null;
  if (!openingMarker) {
    throw new Error(`Memory document ${sourcePath} must start with --- frontmatter.`);
  }
  if (Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES) {
    throw new Error(`Memory document ${sourcePath} is too large.`);
  }

  const frontmatterStart = openingMarker[0].length;
  const remainder = source.slice(frontmatterStart);
  const closingMarker = /\r?\n---(?=\r?\n|$)/.exec(remainder);
  if (!closingMarker) {
    throw new Error(`Memory document ${sourcePath} has no closing frontmatter marker.`);
  }

  const frontmatterSource = remainder.slice(0, closingMarker.index);
  const parsed = frontmatterSource.split(/\r?\n/).reduce((state, line) => {
    const next = parseFrontmatterLine(line, sourcePath, state.seen);
    return {
      values: { ...state.values, [next.objectKey]: next.value },
      seen: new Set([...state.seen, next.objectKey]),
    };
  }, { values: {}, seen: new Set() });

View on GitHub (pinned to 01e15490f0)

Solutions

1. Trim the body: serializeMemoryDocument() output is naturally small if you keep body under MAX_BODY_BYTES.
2. Split into multiple linked memories using the links field.
3. Move large payloads out of the vault and reference them by path.
4. Pre-check before parsing: if (Buffer.byteLength(source,'utf8') > 131072) skip or truncate.

Example fix

// before
const source = fs.readFileSync(hugeMdPath, 'utf8'); // 200 KiB
const mem = parseMemoryDocument(source, hugeMdPath);

// after
const MAX = 128 * 1024;
if (Buffer.byteLength(source, 'utf8') > MAX) {
  throw new Error(`file ${hugeMdPath} exceeds the 128 KiB document cap; split it`);
}
const mem = parseMemoryDocument(source, hugeMdPath);

const { MAX_DOCUMENT_BYTES } = require('./scripts/lib/memory-vault-format');
if (Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES) {
  throw new Error(`document exceeds ${MAX_DOCUMENT_BYTES} bytes; split before parsing`);
}

When it happens

Trigger: parseMemoryDocument() on a file whose frontmatter plus body exceeds 128 KiB. A memory file that grew from many appended tags or links. A copy-paste of a very large code block into body. A pre-existing markdown file repurposed as a memory document.

Common situations: Auto-append workflow that never trims. Body originally under 64 KiB but frontmatter bloat (tags with hundreds of entries despite MAX_TAGS=32) plus body pushes the document over 128 KiB. UTF-16 source re-saved as UTF-8 still producing a large file.

Related errors

• memory body is too large (maximum ${MAX_BODY_BYTES} bytes).
• memory body is too large (maximum ${maxBytes} bytes).
• Failed to write ${path.basename(filePath)}: ${error.message}
• ${label} must be a non-empty string.
• ${label} is too long (maximum ${maxChars} characters).

AI-assisted analysis of affaan-m/ECC@01e15490f0 (2026-08-13). Data as JSON: /api/errors/45acc6da6c048e23. Report an issue: GitHub.