{"record":{"id":"22f81c737cebda72","repo":"eyaltoledano/claude-task-master","slug":"buffer-size-exceeded","errorCode":"BUFFER_SIZE_EXCEEDED","errorMessage":"Buffer size exceeded: ${newSize} bytes > ${this.maxSize} bytes maximum","messagePattern":"Buffer size exceeded: (.+?) bytes > (.+?) bytes maximum","errorType":"exception","errorClass":"StreamingError","httpStatus":null,"severity":"error","filePath":"src/utils/stream-parser.js","lineNumber":337,"sourceCode":"\t\t}\n\t\t// If we have some items from streaming, continue with those\n\t}\n}\n\n/**\n * Buffer size validator\n */\nclass BufferSizeValidator {\n\tconstructor(maxSize) {\n\t\tthis.maxSize = maxSize;\n\t\tthis.currentSize = 0;\n\t}\n\n\tvalidateChunk(existingText, newChunk) {\n\t\tconst newSize = Buffer.byteLength(existingText + newChunk, 'utf8');\n\n\t\tif (newSize > this.maxSize) {\n\t\t\tthrow new StreamingError(\n\t\t\t\t`Buffer size exceeded: ${newSize} bytes > ${this.maxSize} bytes maximum`,\n\t\t\t\tSTREAMING_ERROR_CODES.BUFFER_SIZE_EXCEEDED\n\t\t\t);\n\t\t}\n\n\t\tthis.currentSize = newSize;\n\t}\n}\n\n/**\n * Main orchestrator for stream parsing\n */\nclass StreamParserOrchestrator {\n\tconstructor(config) {\n\t\tthis.config = new StreamParserConfig(config);\n\t\tthis.progressTracker = new ProgressTracker(this.config);\n\t\tthis.bufferValidator = new BufferSizeValidator(this.config.maxBufferSize);\n\t\tthis.jsonParser = new JSONStreamParser(this.config, this.progressTracker);","sourceCodeStart":319,"sourceCodeEnd":355,"githubUrl":"https://github.com/eyaltoledano/claude-task-master/blob/c0c98d367c55296bfe69e65680625b6db437af02/src/utils/stream-parser.js#L319-L355","documentation":"BufferValidator.validateChunk tracks the cumulative UTF-8 byte size of buffered stream text and throws a StreamingError with code BUFFER_SIZE_EXCEEDED once adding a new chunk would push the buffer past maxSize (configured as maxBufferSize). This guard prevents unbounded memory growth when a stream never terminates or produces enormous output.","triggerScenarios":"Streaming a very large AI response larger than maxBufferSize; a malformed/looping stream that never completes; setting maxBufferSize too small for the expected payload size; multi-byte characters making byte size exceed expectations.","commonSituations":"Default buffer size (often 1-10 MB) too small for large batch generations; run-away model output; forgetting to raise maxBufferSize after increasing expected output length in prompts.","solutions":["Increase maxBufferSize when constructing the parser to exceed the expected response size.","Reduce the response size: limit items per request or page the generation into multiple streams.","Verify the stream is not stuck in a loop producing endless text; add upstream timeouts/max-token limits.","If you must handle arbitrarily large inputs, process the stream in segments with separate parsers instead of one buffer."],"exampleFix":"// before\nnew StreamParser({ jsonPaths: ['$.items'], maxBufferSize: 1024 * 1024 }) // 1 MB\n// after\nnew StreamParser({ jsonPaths: ['$.items'], maxBufferSize: 10 * 1024 * 1024 }) // 10 MB","handlingStrategy":"try-catch","validationCode":"function assertBufferSizeFits(maxBufferSize, maxExpectedResponseBytes) {\n  if (maxBufferSize < maxExpectedResponseBytes) {\n    throw new RangeError(`maxBufferSize (${maxBufferSize}) is smaller than the largest expected response (${maxExpectedResponseBytes} bytes)`);\n  }\n}","typeGuard":"function fitsInBuffer(text, maxSize) {\n  return Buffer.byteLength(text, 'utf8') <= maxSize;\n}","tryCatchPattern":"try {\n  return await parser.parseStream(stream);\n} catch (err) {\n  if (err.code === 'BUFFER_SIZE_EXCEEDED') {\n    return new StreamParser({ ...options, maxBufferSize: options.maxBufferSize * 4 }).parseStream(stream);\n  }\n  throw err;\n}","preventionTips":["Size maxBufferSize above your worst-case expected response (remember UTF-8 multi-byte chars).","Cap generation size via prompt limits / max tokens instead of relying on the buffer.","Monitor for looping/stuck streams with upstream timeouts.","If payloads are inherently large, chunk the work across multiple parser runs."],"tags":["streaming","buffer","memory","limits"],"backgroundTag":"buffer-size-exceeded","analyzedSha":"c0c98d367c55296bfe69e65680625b6db437af02","analyzedAt":"2026-08-29T02:56:26.071Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}