{"record":{"id":"39513f05a281a38e","repo":"huggingface/transformers","slug":"flash-attn-with-kvcache-fn-does-not-have-a-block-t","errorCode":null,"errorMessage":"flash_attn_with_kvcache_fn does not have a block_table or page_table argument: {}","messagePattern":"flash_attn_with_kvcache_fn does not have a block_table or page_table argument: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/cache.py","lineNumber":458,"sourceCode":"            # Write new KV values to the cache (padding slots in write_index point to the trash position)\n            k_cache.index_copy_(0, layer_write_index, key_states)\n            v_cache.index_copy_(0, layer_write_index, value_states)\n\n        # Return the new KV values\n        return key_states_with_cache, value_states_with_cache\n\n    def get_block_table_key(self, flash_attn_with_kvcache_fn: Any) -> str:\n        \"\"\"A function to get the name of the block table key for the given flash_attn_with_kvcache_fn. The function's\n        signature is only inspected once. This is necessary because different version of flash have different names for\n        the block table key.\"\"\"\n        if self._block_table_key is None:\n            kwarg_names = inspect.signature(flash_attn_with_kvcache_fn).parameters.keys()\n            if \"block_table\" in kwarg_names:\n                self._block_table_key = \"block_table\"\n            elif \"page_table\" in kwarg_names:\n                self._block_table_key = \"page_table\"\n            else:\n                raise ValueError(\n                    f\"flash_attn_with_kvcache_fn does not have a block_table or page_table argument: {inspect.signature(flash_attn_with_kvcache_fn)}\"\n                )\n        return self._block_table_key\n\n    def search_prefix_match(self, request_id: str, prompt_ids: list[int]) -> int:\n        \"\"\"Searches for a prefix match in the cache for the given (prompts_ids). If one is found, we reference the\n        matching blocks in the (request_id), increase the reference count of the blocks and return the number of blocks\n        that match. If no prefix match is found, we return 0.\"\"\"\n        current_hash = None\n        allocated_blocks = []\n        for b in range(len(prompt_ids) // self.block_size):\n            tokens = prompt_ids[b * self.block_size : (b + 1) * self.block_size]\n            # Prefix sharing is only supported when there is only one full attention layer group, so group_id=0.\n            current_hash = self._block_manager.compute_hash(current_hash, tokens, group_id=0)\n            block_id = self._block_manager._hash_to_id.get(current_hash)\n            if block_id is not None:\n                allocated_blocks.append(block_id)\n                self._block_manager.increase_ref_count(block_id)","sourceCodeStart":440,"sourceCodeEnd":476,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/cache.py#L440-L476","documentation":"ValueError from Cache.get_block_table_key(): the library inspects the signature of the flash-attn kernel wrapper flash_attn_with_kvcache and expects a 'block_table' (newer) or 'page_table' (older) parameter to pass paged KV indices. If neither exists, the installed flash-attn version is incompatible with transformers' paged KV cache path.","triggerScenarios":"Using continuous batching with a very old flash-attn build that predates paged KV support, or an unusual/new fork that renamed the argument; calling code that injects a custom flash_attn_with_kvcache_fn without block/page table support.","commonSituations":"Environment drift: flash-attn pinned to an old version in a Docker image; installing transformers from source against a stale flash-attn; custom kernels injected for testing.","solutions":["Upgrade flash-attn to a version whose flash_attn_with_kvcache accepts block_table (v2.3.5+ paged variants, ideally latest v2)","Verify: python -c \"from flash_attn import flash_attn_with_kvcache; import inspect; print(list(inspect.signature(flash_attn_with_kvcache).parameters))\"","If using a custom kernel function, add a block_table (or page_table) keyword argument to its signature"],"exampleFix":"# before: old flash-attn without paged KV\npip install flash-attn==2.3.0  # lacks block_table -> error\n# after\npip install -U flash-attn --no-build-isolation","handlingStrategy":"validation","validationCode":"import inspect\nfrom flash_attn import flash_attn_with_kvcache\nparams = inspect.signature(flash_attn_with_kvcache).parameters\nassert 'block_table' in params or 'page_table' in params, 'flash-attn too old for paged KV; upgrade'","typeGuard":"def flash_attn_supports_paged_kv() -> bool:\n    import inspect\n    try:\n        from flash_attn import flash_attn_with_kvcache\n    except ImportError:\n        return False\n    p = inspect.signature(flash_attn_with_kvcache).parameters\n    return 'block_table' in p or 'page_table' in p","tryCatchPattern":"try:\n    key = cache.get_block_table_key(flash_attn_with_kvcache)\nexcept ValueError:\n    raise RuntimeError('incompatible flash-attn; pip install -U flash-attn --no-build-isolation')","preventionTips":["Pin a recent flash-attn v2 in requirements to avoid silent downgrades","Run the signature check at server startup before accepting traffic"],"tags":["continuous-batching","flash-attention","version-mismatch","kv-cache","environment"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}