{"record":{"id":"9c3ec5697f7b9402","repo":"affaan-m/ECC","slug":"label-is-too-large-opened-size-bytes","errorCode":null,"errorMessage":"${label} is too large (${opened.size} bytes).","messagePattern":"(.+?) is too large \\((.+?) bytes\\)\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/lib/memory-vault.js","lineNumber":170,"sourceCode":"  const descriptor = fs.openSync(filePath, flags);\n  try {\n    const opened = fs.fstatSync(descriptor, { bigint: true });\n    if (!opened.isFile()) {\n      throw new Error(`${label} must be a regular, non-symlink file.`);\n    }\n    const after = fs.lstatSync(filePath, { bigint: true });\n    if (\n      after.isSymbolicLink()\n      || !after.isFile()\n      || !sameFileIdentity(after, opened)\n    ) {\n      throw new Error(`${label} must remain a regular, non-symlink file while it is opened.`);\n    }\n    if (options.trustedRoot) {\n      assertWithinTrustedRoot(filePath, options.trustedRoot, `read ${label}`);\n    }\n    if (opened.size > BigInt(maxBytes)) {\n      throw new Error(`${label} is too large (${opened.size} bytes).`);\n    }\n\n    const chunks = [];\n    let total = 0;\n    while (total <= maxBytes) {\n      const buffer = Buffer.alloc(Math.min(64 * 1024, maxBytes + 1 - total));\n      const bytesRead = fs.readSync(descriptor, buffer, 0, buffer.length, null);\n      if (bytesRead === 0) break;\n      chunks.push(buffer.subarray(0, bytesRead));\n      total += bytesRead;\n    }\n    if (total > maxBytes) {\n      throw new Error(`${label} is too large (maximum ${maxBytes} bytes).`);\n    }\n    return decodeUtf8(Buffer.concat(chunks, total), label);\n  } finally {\n    fs.closeSync(descriptor);\n  }","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/affaan-m/ECC/blob/01e15490f04e29cfefe3896951f43db46994d8ee/scripts/lib/memory-vault.js#L152-L188","documentation":"readRegularTextFile enforces a hard size cap on any document it opens: after fstat, if opened.size exceeds maxBytes (default MAX_DOCUMENT_BYTES), it refuses to read at all. This bounds memory consumption and prevents a single oversized document from stalling a vault scan. The check uses the size at open time, before any bytes are streamed.","triggerScenarios":"A memory document (or the project .gitignore when read back by ensureProjectScopeIgnored) is larger than maxBytes. Triggered by readMemoryFiles scanning a vault containing an oversized .md file, or by an explicit readRegularTextFile call where options.maxBytes was lowered.","commonSituations":"Pasting a huge log, stack trace, or base64 blob into a memory body; a memory document that accumulated many edits over time; embedding a large image as data URI; lowering maxBytes for a specific call below the on-disk file size; a corrupted/merged document that concatenated duplicates.","solutions":["Trim the offending document's body to under MAX_DOCUMENT_BYTES (split large content into multiple linked memories).","Move oversized reference material out of the vault into an attachment and link to it, keeping only a summary in the memory body.","Locate the offending file with the diagnostics from searchMemories/doctorMemoryVault (invalidFiles list) and rewrite it via saveMemory with a fresh id.","If you genuinely need a higher cap for a trusted internal call, pass options.maxBytes explicitly — but the default is a deliberate ceiling, not a typo."],"exampleFix":"// before: memory body contains a 5MB base64 blob, read fails\n// split: save a summary memory and store the blob elsewhere\nsaveMemory({ title: 'crash log', body: log.slice(0, 4000), tags: ['summary'] });\nfs.writeFileSync('/data/attachments/full.log', fullLog);\n// after: body is well under the cap","handlingStrategy":"validation","validationCode":"const fs = require('fs');\nconst { MAX_DOCUMENT_BYTES } = require('./scripts/lib/memory-vault');\nfunction assertWithinSize(p, max = MAX_DOCUMENT_BYTES) {\n  const size = fs.statSync(p).size;\n  if (size > max) throw new Error(`${p} is too large (${size} bytes).`);\n  return size;\n}","typeGuard":null,"tryCatchPattern":"try { readRegularTextFile(filePath, { trustedRoot: root, maxBytes: MAX_DOCUMENT_BYTES }); }\ncatch (error) {\n  if (/is too large/.test(error.message)) {\n    console.error('Document exceeds the vault size cap; split or trim it:', error.message);\n    return;\n  }\n  throw error;\n}","preventionTips":["Keep memory bodies short — link to attachments for large reference material.","Before scanning, prune oversized files: find <root> -name '*.md' -size +256k.","Never embed base64 images or full logs in a memory body.","Run doctorMemoryVault to surface invalid (oversized) files in the diagnostics."],"tags":["memory-vault","validation","size-limits","filesystem"],"backgroundTag":null,"analyzedSha":"01e15490f04e29cfefe3896951f43db46994d8ee","analyzedAt":"2026-08-13T00:31:08.655Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}