vllm-project/vllm · error · MemoryError

Not enough space in the data buffer, try calling free_buf()

Error message

Not enough space in the data buffer, try calling free_buf() to free up space

What it means

ShmObjectStorage's ring buffer ran out of contiguous space: after wrapping the write pointer, writing `size` more bytes would overwrite the region starting at data_buffer_start. The check `occupied_size_new > data_buffer_size` fires, meaning live (not yet freed) allocations fill the whole data buffer. The message points you at free_buf()/free_unused(), which releases buffers whose in-use flags have dropped to zero.

Source

Thrown at vllm/distributed/device_communicators/shm_object_storage.py:216

        assert self.is_writer, "Only the writer can allocate buffers."
        assert size > 0, "Size must be greater than 0"
        assert self.shared_memory.buf is not None, "Buffer has been closed"
        size += self.MD_SIZE  # add metadata size to the buffer size
        # reset to beginning if the buffer does have enough contiguous space
        buffer_end_reset = self.data_buffer_end % self.data_buffer_size
        if buffer_end_reset + size > self.data_buffer_size:
            buffer_end_reset = (
                self.data_buffer_end // self.data_buffer_size + 1
            ) * self.data_buffer_size
        else:  # no reset needed
            buffer_end_reset = self.data_buffer_end

        # check if we have enough space in the data buffer
        # i.e. if the new end (self.data_buffer_end + size)
        # exceeds the start of the data buffer
        occupied_size_new = buffer_end_reset + size - self.data_buffer_start
        if occupied_size_new > self.data_buffer_size:
            raise MemoryError(
                "Not enough space in the data buffer, "
                "try calling free_buf() to free up space"
            )
        self.data_buffer_end = buffer_end_reset

        # first 4 bytes as the monotonic id
        buf_idx = self.data_buffer_end % self.data_buffer_size
        self.shared_memory.buf[buf_idx : buf_idx + self.ID_NBYTES] = self.int2byte(
            self.monotonic_id_end
        )
        # next 4 bytes as the size of the data buffer
        self.shared_memory.buf[buf_idx + self.ID_NBYTES : buf_idx + self.MD_SIZE] = (
            self.int2byte(size)
        )

        # record metadata
        self.metadata[self.monotonic_id_end % self.ID_MAX] = self.data_buffer_end
        # update buffer and monotonic id indices

View on GitHub (pinned to c794754062)

Solutions

1. Call free_unused()/free_buf() (put() itself retries once after free_unused(), so persistent failure means buffers are genuinely still in use)
2. Ensure every reader calls get() through the API that decrements the in-use flag, and that reader processes stay alive until they release references
3. Increase the shared data buffer size (storage sizing / max buffer configuration) so live objects fit
4. If the error is transient (slow readers), add reader-side consumption before more puts

Example fix

# before
storage.put(key, big_obj)  # MemoryError: Not enough space in the data buffer

# after
storage.free_unused()  # release buffers whose readers finished
storage.put(key, big_obj)

# track live bytes before a burst of puts
free = storage.ring_buffer.data_buffer_size - (
    storage.ring_buffer.data_buffer_end - storage.ring_buffer.data_buffer_start
)
if free < incoming_bytes:
    storage.free_unused()

try:
    storage.put(key, value)
except MemoryError:
    storage.free_unused()  # put() already retries once internally; do more here
    storage.put(key, value)

When it happens

Trigger: Repeated put() calls (e.g. writing encoder-cache / multimodal objects keyed by hash) without matching reader consumption, so writer_flag/id_index entries are never freed; or a single object nearly as large as the whole data_buffer_size; or readers crashing before decrementing the in-use flag.

Common situations: Long-running prefill with large multimodal inputs filling shm storage; readers falling behind or being killed so reference counts never reach zero; sizing data_buffer_size too small for the workload's live object set.

Related errors

• Insufficient space in {shm_path}: {required_bytes / mib:.0f}
• Only readers can dequeue
• `{executor_backend}` does not support async scheduling yet.
• torch_shm is known to fail without VLLM_WORKER_MULTIPROC_MET
• DeepEPv2 communicator properties query failed; networking ca

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/b6bab2e17ae18394. Report an issue: GitHub.