{"record":{"id":"45acc6da6c048e23","repo":"affaan-m/ECC","slug":"memory-document-sourcepath-is-too-large","errorCode":null,"errorMessage":"Memory document ${sourcePath} is too large.","messagePattern":"Memory document (.+?) is too large\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/lib/memory-vault-format.js","lineNumber":251,"sourceCode":"    throw new Error(`Duplicate memory frontmatter field in ${sourcePath}.`);\n  }\n  const rawValue = line.slice(separator + 1).trim();\n  try {\n    return { objectKey, value: JSON.parse(rawValue) };\n  } catch {\n    throw new Error(`Memory frontmatter field in ${sourcePath} must use a JSON value.`);\n  }\n}\n\nfunction parseMemoryDocument(source, sourcePath = '<memory>') {\n  const openingMarker = typeof source === 'string'\n    ? /^---\\r?\\n/.exec(source)\n    : null;\n  if (!openingMarker) {\n    throw new Error(`Memory document ${sourcePath} must start with --- frontmatter.`);\n  }\n  if (Buffer.byteLength(source, 'utf8') > MAX_DOCUMENT_BYTES) {\n    throw new Error(`Memory document ${sourcePath} is too large.`);\n  }\n\n  const frontmatterStart = openingMarker[0].length;\n  const remainder = source.slice(frontmatterStart);\n  const closingMarker = /\\r?\\n---(?=\\r?\\n|$)/.exec(remainder);\n  if (!closingMarker) {\n    throw new Error(`Memory document ${sourcePath} has no closing frontmatter marker.`);\n  }\n\n  const frontmatterSource = remainder.slice(0, closingMarker.index);\n  const parsed = frontmatterSource.split(/\\r?\\n/).reduce((state, line) => {\n    const next = parseFrontmatterLine(line, sourcePath, state.seen);\n    return {\n      values: { ...state.values, [next.objectKey]: next.value },\n      seen: new Set([...state.seen, next.objectKey]),\n    };\n  }, { values: {}, seen: new Set() });\n","sourceCodeStart":233,"sourceCodeEnd":269,"githubUrl":"https://github.com/affaan-m/ECC/blob/8321021c54d670126ce3b2969d5deb880b4b0c2a/scripts/lib/memory-vault-format.js#L233-L269","documentation":"parseMemoryDocument() enforces a hard size cap (MAX_DOCUMENT_BYTES) on the UTF-8 byte length of a memory document. This prevents pathological inputs — huge pastes, binary blobs, runaway logs — from consuming memory or stalling parsing; oversized documents are rejected outright.","triggerScenarios":"Parsing a memory document whose UTF-8 encoding exceeds MAX_DOCUMENT_BYTES — e.g. a very large body pasted below the frontmatter, accumulated log content, binary data embedded in the file, or a corrupted/duplicated file.","commonSituations":"Pasting an entire article or transcript as memory body content; a script appending to the memory file on every run until it balloons; accidentally pointing the loader at a large dump file instead of a memory document.","solutions":["Trim the document body to under the MAX_DOCUMENT_BYTES limit — move large content to a separate file and reference it.","Split the memory into multiple smaller memory documents with focused titles.","Check for accidental duplication/appending in the file (e.g. a loop re-writing content) and regenerate the file.","Verify you're loading the intended file and not a large dump; check size with `ls -lh` or `wc -c`."],"exampleFix":"// before\nconst doc = fs.readFileSync('huge-memory.md', 'utf8'); // 20 MB of notes\nparseMemoryDocument(doc, 'huge-memory.md');\n\n// after\nconst doc = fs.readFileSync('huge-memory.md', 'utf8');\nif (Buffer.byteLength(doc, 'utf8') > MAX_DOCUMENT_BYTES) {\n  writeSummaryMemory('huge-memory', summarize(doc)); // store a compact summary instead\n} else {\n  parseMemoryDocument(doc, 'huge-memory.md');\n}","handlingStrategy":"validation","validationCode":"const stat = fs.statSync(path);\nif (stat.size > MAX_DOCUMENT_BYTES) {\n  throw new Error(`${path} exceeds the ${MAX_DOCUMENT_BYTES}-byte memory document limit`);\n}","typeGuard":"const isWithinSizeLimit = (s) => typeof s === 'string' && Buffer.byteLength(s, 'utf8') <= MAX_DOCUMENT_BYTES;","tryCatchPattern":"try {\n  parseMemoryDocument(source, path);\n} catch (err) {\n  if (err.message.includes('is too large')) {\n    console.error('%s exceeds MAX_DOCUMENT_BYTES; split it or store a summary.', path);\n  } else throw err;\n}","preventionTips":["Check file size (`wc -c`) before loading documents from untrusted sources.","Keep memory bodies concise; link to external files for long content.","Avoid append-only scripts that grow memory files unboundedly."],"tags":["validation","size-limit","payload-too-large"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"8321021c54d670126ce3b2969d5deb880b4b0c2a","analyzedAt":"2026-09-16T10:08:13.343Z","contentChangedAt":"2026-09-16T10:08:13.343Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}