vllm-project/vllm · error · ValueError

MoRIIO KV cache block size mismatch for layer {layer_name}:

Error message

MoRIIO KV cache block size mismatch for layer {layer_name}: {geometry.block_size} != {self.block_size}

What it means

When registering local KV caches with MoRI-IO, the wrapper derives block_size from the first layer's transfer geometry and then requires every subsequent layer to have the same block size. A layer whose geometry.block_size differs raises ValueError, because the transfer engine assumes a uniform block size across all layers.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:1761

                self.block_size,
            )
            self.block_size = first_geometry.block_size
        # TODO(tms): self.block_len needs to be per-layer for sliding window,
        # hybrid attn, etc
        # block size in bytes
        self.block_len = first_geometry.block_len
        self.kv_cache_shape = first_kv_cache.shape
        self.block_shape = block_shape
        self.kv_element_size = kv_elem_size

        self.dst_num_blocks[self.engine_id] = self.num_blocks
        kv_caches_base_addr = []
        caches_data = []

        for layer_name in kv_caches:
            geometry = self._get_layer_transfer_geometry(layer_name)
            if geometry.block_size != self.block_size:
                raise ValueError(
                    "MoRIIO KV cache block size mismatch for layer "
                    f"{layer_name}: {geometry.block_size} != {self.block_size}"
                )
            self.block_lens[layer_name] = geometry.block_len
            for cache, region_len in self._iter_layer_registration_regions(layer_name):
                base_addr = cache.data_ptr()
                caches_data.append((base_addr, region_len, cache.device.index, ""))
                kv_caches_base_addr.append(base_addr)

        for layer_name, kv_cache in kv_caches.items():
            if layer_name not in self.layer_name_to_local_kv_cache_metadata:
                self.layer_name_to_local_kv_cache_metadata[layer_name] = []

            moriio_mem_metadata = self.moriio_wrapper.register_local_tensor(kv_cache)
            self.layer_name_to_local_kv_cache_metadata[layer_name].append(
                moriio_mem_metadata
            )

View on GitHub (pinned to c794754062)

Solutions

1. Configure the model/cache so all layers share one block size (e.g. disable hybrid cache splitting or align sliding-window block config with full-attention config)
2. Use a model variant without heterogeneous per-layer cache geometry
3. If layer heterogeneity is required, switch to a KV connector that supports per-layer block sizes

Example fix

# before: hybrid cache with different block sizes per layer group -> ValueError
# after: serve with a uniform cache config (same block size for all layers)

block_sizes = {geo.block_size for geo in (get_layer_transfer_geometry(l) for l in kv_caches)}
if len(block_sizes) > 1:
    raise ValueError(f"non-uniform KV block sizes across layers: {block_sizes}; MoRI-IO requires one block size")

def has_uniform_block_sizes(kv_caches: dict) -> bool:
    sizes = {get_layer_transfer_geometry(name).block_size for name in kv_caches}
    return len(sizes) <= 1

When it happens

Trigger: Models whose layers have heterogeneous KV cache layouts: hybrid-attention models mixing full-attention and sliding-window/linear layers with different effective block sizes, or mixed KV-cache dtypes/layouts producing different geometry per layer.

Common situations: Serving hybrid architectures (e.g. sliding-window + full attention variants) with the moriio connector; changes to kv cache dtype or block configuration that apply unevenly across layer groups; custom models with per-layer cache shapes.

Related errors

• Hybrid KV cache manager was explicitly enabled but is not su
• No KV cache tensors were registered with Mooncake.
• MooncakeStoreConnector does not support: {unsupported}
• Partial-tail offloads for one request must share a boundary
• TP sizes and total_num_kv_heads must be positive

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/36bbba313633caed. Report an issue: GitHub.