{"record":{"id":"45acc6da6c048e23","repo":"affaan-m/ECC","slug":"memory-document-sourcepath-is-too-large","errorCode":null,"errorMessage":"Memory document ${sourcePath} is too large.","messagePattern":"Memory document (.+?) is too large\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/lib/memory-vault-format.js","lineNumber":251,"sourceCode":"    throw new Error(`Duplicate memory frontmatter field in ${sourcePath}.`);\n  }\n  const rawValue = line.slice(separator + 1).trim();\n  try {\n    return { objectKey, value: JSON.parse(rawValue) };\n  } catch {\n    throw new Error(`Memory frontmatter field in ${sourcePath} must use a JSON value.`);\n  }\n}\n\nfunction parseMemoryDocument(source, sourcePath = '<memory>') {\n  const openingMarker = typeof source === 'string'\n    ? /^---\\r?\\n/.exec(source)\n    : null;\n  if (!openingMarker) {\n    throw new Error(`Memory document ${sourcePath} must start with --- frontmatter.`);\n  }\n  if (Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES) {\n    throw new Error(`Memory document ${sourcePath} is too large.`);\n  }\n\n  const frontmatterStart = openingMarker[0].length;\n  const remainder = source.slice(frontmatterStart);\n  const closingMarker = /\\r?\\n---(?=\\r?\\n|$)/.exec(remainder);\n  if (!closingMarker) {\n    throw new Error(`Memory document ${sourcePath} has no closing frontmatter marker.`);\n  }\n\n  const frontmatterSource = remainder.slice(0, closingMarker.index);\n  const parsed = frontmatterSource.split(/\\r?\\n/).reduce((state, line) => {\n    const next = parseFrontmatterLine(line, sourcePath, state.seen);\n    return {\n      values: { ...state.values, [next.objectKey]: next.value },\n      seen: new Set([...state.seen, next.objectKey]),\n    };\n  }, { values: {}, seen: new Set() });\n","sourceCodeStart":233,"sourceCodeEnd":269,"githubUrl":"https://github.com/affaan-m/ECC/blob/01e15490f04e29cfefe3896951f43db46994d8ee/scripts/lib/memory-vault-format.js#L233-L269","documentation":"Thrown by parseMemoryDocument() when Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES (131072 bytes = 128 KiB). This is the whole-document cap; the body-only cap (MAX_BODY_BYTES = 64 KiB) is checked separately inside normalizeBody(). The document cap bounds the parsing cost and the per-file read budget inside readMemoryFiles().","triggerScenarios":"parseMemoryDocument() on a file whose frontmatter plus body exceeds 128 KiB. A memory file that grew from many appended tags or links. A copy-paste of a very large code block into body. A pre-existing markdown file repurposed as a memory document.","commonSituations":"Auto-append workflow that never trims. Body originally under 64 KiB but frontmatter bloat (tags with hundreds of entries despite MAX_TAGS=32) plus body pushes the document over 128 KiB. UTF-16 source re-saved as UTF-8 still producing a large file.","solutions":["Trim the body: serializeMemoryDocument() output is naturally small if you keep body under MAX_BODY_BYTES.","Split into multiple linked memories using the links field.","Move large payloads out of the vault and reference them by path.","Pre-check before parsing: if (Buffer.byteLength(source,'utf8') > 131072) skip or truncate."],"exampleFix":"// before\nconst source = fs.readFileSync(hugeMdPath, 'utf8'); // 200 KiB\nconst mem = parseMemoryDocument(source, hugeMdPath);\n\n// after\nconst MAX = 128 * 1024;\nif (Buffer.byteLength(source, 'utf8') > MAX) {\n  throw new Error(`file ${hugeMdPath} exceeds the 128 KiB document cap; split it`);\n}\nconst mem = parseMemoryDocument(source, hugeMdPath);","handlingStrategy":"validation","validationCode":"const { MAX_DOCUMENT_BYTES } = require('./scripts/lib/memory-vault-format');\nif (Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES) {\n  throw new Error(`document exceeds ${MAX_DOCUMENT_BYTES} bytes; split before parsing`);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep body under MAX_BODY_BYTES (64 KiB); that leaves room for frontmatter within the 128 KiB document cap.","Do not store large log/code dumps as a single memory.","Run doctorMemoryVault() periodically to spot files approaching the cap."],"tags":["limits","validation","memory-vault","size-limit"],"backgroundTag":null,"analyzedSha":"01e15490f04e29cfefe3896951f43db46994d8ee","analyzedAt":"2026-08-13T00:31:08.655Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}