{"record":{"id":"36bbba313633caed","repo":"vllm-project/vllm","slug":"moriio-kv-cache-block-size-mismatch-for-layer-lay","errorCode":null,"errorMessage":"MoRIIO KV cache block size mismatch for layer {layer_name}: {geometry.block_size} != {self.block_size}","messagePattern":"MoRIIO KV cache block size mismatch for layer (.+?): (.+?) != (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py","lineNumber":1761,"sourceCode":"                self.block_size,\n            )\n            self.block_size = first_geometry.block_size\n        # TODO(tms): self.block_len needs to be per-layer for sliding window,\n        # hybrid attn, etc\n        # block size in bytes\n        self.block_len = first_geometry.block_len\n        self.kv_cache_shape = first_kv_cache.shape\n        self.block_shape = block_shape\n        self.kv_element_size = kv_elem_size\n\n        self.dst_num_blocks[self.engine_id] = self.num_blocks\n        kv_caches_base_addr = []\n        caches_data = []\n\n        for layer_name in kv_caches:\n            geometry = self._get_layer_transfer_geometry(layer_name)\n            if geometry.block_size != self.block_size:\n                raise ValueError(\n                    \"MoRIIO KV cache block size mismatch for layer \"\n                    f\"{layer_name}: {geometry.block_size} != {self.block_size}\"\n                )\n            self.block_lens[layer_name] = geometry.block_len\n            for cache, region_len in self._iter_layer_registration_regions(layer_name):\n                base_addr = cache.data_ptr()\n                caches_data.append((base_addr, region_len, cache.device.index, \"\"))\n                kv_caches_base_addr.append(base_addr)\n\n        for layer_name, kv_cache in kv_caches.items():\n            if layer_name not in self.layer_name_to_local_kv_cache_metadata:\n                self.layer_name_to_local_kv_cache_metadata[layer_name] = []\n\n            moriio_mem_metadata = self.moriio_wrapper.register_local_tensor(kv_cache)\n            self.layer_name_to_local_kv_cache_metadata[layer_name].append(\n                moriio_mem_metadata\n            )\n","sourceCodeStart":1743,"sourceCodeEnd":1779,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py#L1743-L1779","documentation":"When registering local KV caches with MoRI-IO, the wrapper derives block_size from the first layer's transfer geometry and then requires every subsequent layer to have the same block size. A layer whose geometry.block_size differs raises ValueError, because the transfer engine assumes a uniform block size across all layers.","triggerScenarios":"Models whose layers have heterogeneous KV cache layouts: hybrid-attention models mixing full-attention and sliding-window/linear layers with different effective block sizes, or mixed KV-cache dtypes/layouts producing different geometry per layer.","commonSituations":"Serving hybrid architectures (e.g. sliding-window + full attention variants) with the moriio connector; changes to kv cache dtype or block configuration that apply unevenly across layer groups; custom models with per-layer cache shapes.","solutions":["Configure the model/cache so all layers share one block size (e.g. disable hybrid cache splitting or align sliding-window block config with full-attention config)","Use a model variant without heterogeneous per-layer cache geometry","If layer heterogeneity is required, switch to a KV connector that supports per-layer block sizes"],"exampleFix":"# before: hybrid cache with different block sizes per layer group -> ValueError\n# after: serve with a uniform cache config (same block size for all layers)","handlingStrategy":"validation","validationCode":"block_sizes = {geo.block_size for geo in (get_layer_transfer_geometry(l) for l in kv_caches)}\nif len(block_sizes) > 1:\n    raise ValueError(f\"non-uniform KV block sizes across layers: {block_sizes}; MoRI-IO requires one block size\")","typeGuard":"def has_uniform_block_sizes(kv_caches: dict) -> bool:\n    sizes = {get_layer_transfer_geometry(name).block_size for name in kv_caches}\n    return len(sizes) <= 1","tryCatchPattern":null,"preventionTips":["Avoid hybrid per-layer cache configurations with the moriio connector","Smoke-test model load with the connector before production rollout","Log per-layer geometry at registration so mismatches are visible immediately"],"tags":["kv-cache","model-config","hybrid-attention","kv-transfer"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}