huggingface/transformers · critical · ValueError

Failed to allocate {} blocks for request {}

Error message

Failed to allocate {} blocks for request {}

What it means

Internal ValueError from Cache.allocate_blocks(): a pre-check said the allocation should succeed, but an individual group cache manager returned None from allocate_blocks, meaning it could not actually reserve the blocks. This signals a bookkeeping inconsistency between the block manager and group allocators (or a race in concurrent allocation), not normal out-of-memory — OOM is normally handled by the will_allocation_be_successful() pre-check returning False.

Source

Thrown at src/transformers/generation/continuous_batching/cache.py:335

        """Returns a boolean indicating if the allocation of (num_requested_blocks) blocks will be successful."""
        return self.blocks_needed(num_requested_blocks, allocated_blocks) <= self.get_num_free_blocks()

    def blocks_in_use(self, request_id: str) -> int:
        """Returns the total number of physical blocks currently referenced by a request across all layer groups."""
        return sum(len(cm.block_table.get(request_id, ())) for cm in self.group_cache_managers)

    def allocate_blocks(self, n_blocks: int, request_id: str, allocated_blocks: int) -> int | None:
        """Allocate cache blocks across all layer groups for a given request. Actual allocation is done by the cache
        managers, and this method only returns the maximum number of blocks actually allocated across all managers."""
        # First check allocation will be successful before starting, to avoid partial allocations
        if not self.will_allocation_be_successful(n_blocks, allocated_blocks):
            return None
        # Allocate blocks across all cache managers
        max_allocated = 0
        for cm in self.group_cache_managers:
            num_allocated_blocks = cm.allocate_blocks(n_blocks, request_id, self._block_manager)
            if num_allocated_blocks is None:
                raise ValueError(f"Failed to allocate {n_blocks} blocks for request {request_id}")
            max_allocated = max(max_allocated, num_allocated_blocks)
        return max_allocated

    def free_blocks(self, request_id: str) -> None:
        """Free all allocated cache blocks for a given request across all layer groups. Actual deallocation is done
        by the cache managers."""
        for cm in self.group_cache_managers:
            cm.free_blocks(request_id, self._block_manager)

    def get_num_free_blocks(self) -> int:
        """Get the current number of unallocated blocks available for new requests."""
        return self._block_manager.num_free_blocks

    def extend_read_and_write_indices(
        self,
        request_id: str,
        past_length: int,
        query_length: int,

View on GitHub (pinned to a597f97485)

Solutions

1. If you are a user (not modifying internals): report the issue on the transformers GitHub with a reproducer — this is an internal invariant violation
2. Reduce concurrency / max running requests so allocations never approach the block limit
3. Give the cache more memory (raise max_memory_percent or num_blocks) so the race window closes
4. If you patched the cache, audit that every allocate/free path updates both BlockManager and group allocators consistently

if cache.get_num_free_blocks() < blocks_needed:
    # queue the request instead of allocating
    enqueue(request)

try:
    cache.allocate_blocks(n_blocks, request_id, allocated)
except ValueError as e:
    if 'Failed to allocate' in str(e):
        # internal invariant break: dump state and report upstream
        log_state(cache, request_id)
        raise RuntimeError('cache allocator desynchronized; see logs') from e
    raise

When it happens

Trigger: Concurrent requests racing for the last free blocks across multiple threads/processes; bugs in custom block managers; mismatches between group allocators' capacity accounting after prefix sharing frees.

Common situations: Running the continuous-batching server under heavy concurrency; custom forks of the cache that alter free/allocate logic.

Related errors

• Memory footprint {} is more than available memory {}
• num_key_value_heads or num_attention_heads could not be foun
• head_dim or (hidden_size and num_attention_heads) could not
• Block size must be at least {}, but got {}
• Number of key value heads {} must be divisible by tensor par

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/75399c058b7579ff. Report an issue: GitHub.