huggingface/transformers · error · ValueError

flash_attn_with_kvcache_fn does not have a block_table or pa

Error message

flash_attn_with_kvcache_fn does not have a block_table or page_table argument: {}

What it means

ValueError from Cache.get_block_table_key(): the library inspects the signature of the flash-attn kernel wrapper flash_attn_with_kvcache and expects a 'block_table' (newer) or 'page_table' (older) parameter to pass paged KV indices. If neither exists, the installed flash-attn version is incompatible with transformers' paged KV cache path.

Source

Thrown at src/transformers/generation/continuous_batching/cache.py:458

            # Write new KV values to the cache (padding slots in write_index point to the trash position)
            k_cache.index_copy_(0, layer_write_index, key_states)
            v_cache.index_copy_(0, layer_write_index, value_states)

        # Return the new KV values
        return key_states_with_cache, value_states_with_cache

    def get_block_table_key(self, flash_attn_with_kvcache_fn: Any) -> str:
        """A function to get the name of the block table key for the given flash_attn_with_kvcache_fn. The function's
        signature is only inspected once. This is necessary because different version of flash have different names for
        the block table key."""
        if self._block_table_key is None:
            kwarg_names = inspect.signature(flash_attn_with_kvcache_fn).parameters.keys()
            if "block_table" in kwarg_names:
                self._block_table_key = "block_table"
            elif "page_table" in kwarg_names:
                self._block_table_key = "page_table"
            else:
                raise ValueError(
                    f"flash_attn_with_kvcache_fn does not have a block_table or page_table argument: {inspect.signature(flash_attn_with_kvcache_fn)}"
                )
        return self._block_table_key

    def search_prefix_match(self, request_id: str, prompt_ids: list[int]) -> int:
        """Searches for a prefix match in the cache for the given (prompts_ids). If one is found, we reference the
        matching blocks in the (request_id), increase the reference count of the blocks and return the number of blocks
        that match. If no prefix match is found, we return 0."""
        current_hash = None
        allocated_blocks = []
        for b in range(len(prompt_ids) // self.block_size):
            tokens = prompt_ids[b * self.block_size : (b + 1) * self.block_size]
            # Prefix sharing is only supported when there is only one full attention layer group, so group_id=0.
            current_hash = self._block_manager.compute_hash(current_hash, tokens, group_id=0)
            block_id = self._block_manager._hash_to_id.get(current_hash)
            if block_id is not None:
                allocated_blocks.append(block_id)
                self._block_manager.increase_ref_count(block_id)

View on GitHub (pinned to a597f97485)

Solutions

Upgrade flash-attn to a version whose flash_attn_with_kvcache accepts block_table (v2.3.5+ paged variants, ideally latest v2)
Verify: python -c "from flash_attn import flash_attn_with_kvcache; import inspect; print(list(inspect.signature(flash_attn_with_kvcache).parameters))"
If using a custom kernel function, add a block_table (or page_table) keyword argument to its signature

Example fix

# before: old flash-attn without paged KV
pip install flash-attn==2.3.0  # lacks block_table -> error
# after
pip install -U flash-attn --no-build-isolation

Defensive patterns

Strategy: validation

Validate before calling

import inspect
from flash_attn import flash_attn_with_kvcache
params = inspect.signature(flash_attn_with_kvcache).parameters
assert 'block_table' in params or 'page_table' in params, 'flash-attn too old for paged KV; upgrade'

Type guard

def flash_attn_supports_paged_kv() -> bool:
    import inspect
    try:
        from flash_attn import flash_attn_with_kvcache
    except ImportError:
        return False
    p = inspect.signature(flash_attn_with_kvcache).parameters
    return 'block_table' in p or 'page_table' in p

Try / catch

try:
    key = cache.get_block_table_key(flash_attn_with_kvcache)
except ValueError:
    raise RuntimeError('incompatible flash-attn; pip install -U flash-attn --no-build-isolation')

Prevention

Pin a recent flash-attn v2 in requirements to avoid silent downgrades
Run the signature check at server startup before accepting traffic

When it happens

Trigger: Using continuous batching with a very old flash-attn build that predates paged KV support, or an unusual/new fork that renamed the argument; calling code that injects a custom flash_attn_with_kvcache_fn without block/page table support.

Common situations: Environment drift: flash-attn pinned to an old version in a Docker image; installing transformers from source against a stale flash-attn; custom kernels injected for testing.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/39513f05a281a38e. Report an issue: GitHub.