{"record":{"id":"33b8b7e55798135a","repo":"vllm-project/vllm","slug":"hidden-states-block-size-mismatch-derived-self","errorCode":null,"errorMessage":"Hidden-states block-size mismatch: derived {self._block_size} but buffer block size is {self._kv_cache.shape[1]}; read slots would be wrong (likely a hybrid block-size resolution bug).","messagePattern":"Hidden-states block-size mismatch: derived (.+?) but buffer block size is (.+?); read slots would be wrong \\(likely a hybrid block-size resolution bug\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/example_hidden_states_connector.py","lineNumber":314,"sourceCode":"\n        from vllm.model_executor.models.extract_hidden_states import (\n            CacheOnlyAttentionLayer,\n        )\n\n        # Filter layers to only include CacheOnlyAttentionLayers\n        layers = get_layers_from_vllm_config(\n            self._vllm_config, CacheOnlyAttentionLayer, list(kv_caches.keys())\n        )\n        self.cache_layers = list(layers.keys())\n        assert len(self.cache_layers) == 1, (\n            f\"Expected 1 CacheOnlyAttentionLayer, got {len(self.cache_layers)}\"\n        )\n        self._kv_cache = kv_caches[self.cache_layers[0]]\n\n        # Block size must match the indexed buffer, else reads hit the wrong\n        # slots. Raise (not assert) so the check survives `python -O`.\n        if self._block_size != self._kv_cache.shape[1]:\n            raise ValueError(\n                f\"Hidden-states block-size mismatch: derived {self._block_size} \"\n                f\"but buffer block size is {self._kv_cache.shape[1]}; read slots \"\n                \"would be wrong (likely a hybrid block-size resolution bug).\"\n            )\n\n    @staticmethod\n    def _write_tensors(\n        tensors: dict[str, torch.Tensor],\n        event: torch.cuda.Event,\n        filename: str,\n        lock_fd: int | None,\n    ) -> None:\n        \"\"\"Thread worker: wait for async DtoH copy, write to disk, release lock.\n\n        ``lock_fd`` is an open file descriptor on the companion ``.lock``\n        file with ``LOCK_EX`` already held.  Closing it releases the lock,\n        which unblocks any client sleeping on ``LOCK_SH``.\n        \"\"\"","sourceCodeStart":296,"sourceCodeEnd":332,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/example_hidden_states_connector.py#L296-L332","documentation":"During worker-side binding, the connector derived a block size for the hidden-states buffer but the actual allocated torch buffer's second dimension differs. Because read slots are computed from the derived block size, a mismatch would silently corrupt reads, so the code raises (deliberately not assert, so it survives python -O). The message points at a hybrid block-size resolution bug in the KV cache config.","triggerScenarios":"bind_connector_metadata / kv_caches binding: kv_caches[cache_layer].shape[1] != block size derived from the kv_cache_config group spec. Happens when hybrid (multi-spec) block-size resolution computes a different block size than the allocator used for the CacheOnlyAttentionLayer buffer.","commonSituations":"Hybrid KV cache models (e.g. sliding+full attention plus a hidden-states layer) where block-size unification picked a different size for the group; bugs or version mismatches between spec derivation and buffer allocation.","solutions":["Report/investigate as a vLLM hybrid KV-cache block-size resolution bug: compare kv_cache_config group specs' block_size with the allocated buffer shape","As a workaround, force a single uniform block size for all groups (e.g. adjust --block-size or max block sizes of constituent specs so they unify)","Check you are on a version where the hybrid allocator and this connector agree; upgrade vLLM"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert derived_block_size == kv_caches[layer].shape[1], (\n    f\"block size {derived_block_size} != buffer {kv_caches[layer].shape[1]}\"\n)","typeGuard":null,"tryCatchPattern":"try:\n    connector.bind_connector_metadata(md)\nexcept ValueError as e:\n    if 'block-size mismatch' in str(e):\n        raise RuntimeError('Hybrid block-size resolution bug — file a vLLM issue') from e\n    raise","preventionTips":["Keep block sizes uniform across groups in hybrid configs","Re-run this check after any vLLM upgrade that touches the KV cache allocator"],"tags":["kv-transfer","block-size","hybrid-kv-cache","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}