What causes "Attempted to free more buffers than allocated" in vllm-project/vllm?

Audit all free_buffer call sites and guarantee each acquired buffer list is freed exactly once (use try/finally with a 'freed' flag or clear the list after freeing).

vllm-project/vllm · error · RuntimeError

Attempted to free more buffers than allocated

Error message

Attempted to free more buffers than allocated

What it means

RuntimeError from the HF3FS gather/scatter buffer pool's free_buffer when the number of buffers being returned exceeds the pool's in-use count (self._inuse_count). The pool only tracks how many buffers it handed out; returning more than that means the caller is double-freeing or freeing buffers obtained elsewhere. It guards against pool accounting corruption.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/hf3fs/utils/gather_scatter_helper.py:285

            return []

        if self._inuse_count + count <= self._max_count:
            self._inuse_count += count
            result = self._free_buffers[-count:]
            del self._free_buffers[-count:]
            return result
        return None

    def free_buffer(self, buffers: list[torch.Tensor]) -> None:
        """Return buffers to the pool."""
        if not buffers:
            return

        if self._inuse_count >= len(buffers):
            self._inuse_count -= len(buffers)
            self._free_buffers.extend(buffers)
        else:
            raise RuntimeError("Attempted to free more buffers than allocated")


logger = init_logger(__name__)

View on GitHub (pinned to c794754062)

Solutions

Audit all free_buffer call sites and guarantee each acquired buffer list is freed exactly once (use try/finally with a 'freed' flag or clear the list after freeing).
Do not return buffers that were not allocated from this pool instance.
After fixing ownership, if the error persists, log _inuse_count and len(buffers) before free to find the double-free.

Example fix

# before
bufs = pool.get_buffers(n)
try:
    ...
finally:
    pool.free_buffer(bufs)
    pool.free_buffer(bufs)  # double free
# after
bufs = pool.get_buffers(n)
try:
    ...
finally:
    pool.free_buffer(bufs)
    bufs = []

Defensive patterns

Strategy: validation

Validate before calling

def can_free(pool, buffers):
    return len(buffers) <= pool._inuse_count

Try / catch

Catch RuntimeError at free sites, log pool in-use count vs freed count, and treat as a programming bug — fix the ownership path rather than swallowing.

Prevention

Free each acquired buffer list exactly once; null it after free
Centralize buffer ownership in one class
Add a debug counter asserting inuse never goes negative in tests

When it happens

Trigger: Calling free_buffer twice on the same buffer list; mixing buffers from two pools in one free_buffer call; freeing an empty-but-nonnull list after the pool was already drained; error paths that free and then the normal path frees again.

Common situations: Exception handling that returns buffers, followed by a finally block that returns them again; refactoring that changed buffer ownership without updating free sites.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/c53f6de81577a6bc. Report an issue: GitHub.