{"record":{"id":"2e63062e5947aaa0","repo":"hiyouga/LlamaFactory","slug":"buffer-size-exceeds-max-buffer-size-self-max-buf","errorCode":null,"errorMessage":"Buffer size exceeds max buffer size {self._max_buffer_size}.","messagePattern":"Buffer size exceeds max buffer size (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/utils/objects.py","lineNumber":44,"sourceCode":"        self._buffer_size: int = 0\n        self._max_buffer_size: int = max_buffer_size\n\n    def __len__(self) -> int:\n        return len(self._buffer)\n\n    @property\n    def size(self) -> int:\n        return self._buffer_size\n\n    @property\n    def samples(self) -> list[ModelInput]:\n        return self._buffer\n\n    def put(self, samples: list[ModelInput]) -> None:\n        \"\"\"Add samples to the buffer.\"\"\"\n        num_tokens = sum(len(sample[\"input_ids\"]) for sample in samples)\n        if self._buffer_size + num_tokens > self._max_buffer_size:\n            raise ValueError(f\"Buffer size exceeds max buffer size {self._max_buffer_size}.\")\n\n        self._buffer.extend(samples)\n        self._buffer_size += num_tokens\n\n    def get(self, value: int) -> list[ModelInput]:\n        \"\"\"Get samples from the buffer and remove them.\"\"\"\n        samples = self._buffer[:value]\n        self._buffer_size -= sum(len(sample[\"input_ids\"]) for sample in samples)\n        del self._buffer[:value]\n        return samples\n\n    def clear(self) -> None:\n        \"\"\"Clear the buffer.\"\"\"\n        self._buffer = []\n        self._buffer_size = 0\n\n    def state_dict(self) -> dict:\n        \"\"\"Returns the state of the buffer.\"\"\"","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/utils/objects.py#L26-L62","documentation":"ValueError from the token buffer's put() (objects.py:44): adding the batch's tokens would push the buffered token count past _max_buffer_size. The buffer counts tokens (sum of input_ids lengths), not samples, and the check is all-or-nothing: an oversized single put() raises even on an empty buffer.","triggerScenarios":"Calling put(samples) where sum(len(sample['input_ids'])) plus the current _buffer_size exceeds max buffer size; common in generation/offline pipelines that accumulate batches of tokens, e.g. after raising batch size or sequence length.","commonSituations":"Larger cutoff_len or per-device batch size increases tokens per put; consumer falling behind so the buffer stays near capacity; a burst of long samples arriving together.","solutions":["Drain the buffer (call get()/clear()) before putting more samples so _buffer_size is near zero.","Put samples in smaller chunks so each chunk's token sum fits within the remaining capacity.","Raise the buffer's max size at construction if throughput requires larger bursts.","Compute the chunk's total tokens first and split it to fit: while tokens remain, put only what fits."],"exampleFix":"# before\nbuffer.put(all_samples)  # may exceed max_buffer_size\n\n# after\ni = 0\nwhile i < len(all_samples):\n    chunk = []\n    used = 0\n    while i < len(all_samples) and used + len(all_samples[i]['input_ids']) <= buffer.max_buffer_size - buffer.size:\n        chunk.append(all_samples[i]); used += len(all_samples[i]['input_ids']); i += 1\n    buffer.put(chunk)","handlingStrategy":"validation","validationCode":"tokens = sum(len(s[\"input_ids\"]) for s in samples)\nassert buffer.size + tokens <= buffer._max_buffer_size, f\"put of {tokens} tokens would exceed max {buffer._max_buffer_size}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Drain the buffer before large puts.","Chunk samples by token count to fit remaining capacity.","Scale max_buffer_size when increasing batch size or cutoff_len."],"tags":["buffer","memory","generation","validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}