vllm-project/vllm · error · ValueError

Could not uniquely identify the extract-hidden-states KV cac

Error message

Could not uniquely identify the extract-hidden-states KV cache group among {len(groups)} groups; the hidden-states layer must be isolated in its own group (MLA verifiers are unsupported).

What it means

The example hidden-states connector must map to the single KV cache group whose spec is HiddenStateCacheSpec (e.g. the extract-hidden-states layer). It scans kv_cache_config.kv_cache_groups; exactly one match is required, or the whole config must consist of one group only (then group 0 is assumed). Otherwise — including any MLA verifier setup — the group is ambiguous and setup aborts.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/example_hidden_states_connector.py:133

        Located by spec type so it resolves on both scheduler and worker side.
        """
        if kv_cache_config is None:
            return 0

        from vllm.v1.kv_cache_interface import HiddenStateCacheSpec

        groups = kv_cache_config.kv_cache_groups
        group_ids = [
            gid
            for gid, group in enumerate(groups)
            if isinstance(group.kv_cache_spec, HiddenStateCacheSpec)
        ]
        if len(group_ids) == 1:
            return group_ids[0]
        if not group_ids and len(groups) == 1:
            return 0
        raise ValueError(
            "Could not uniquely identify the extract-hidden-states KV cache "
            f"group among {len(groups)} groups; the hidden-states layer must be "
            "isolated in its own group (MLA verifiers are unsupported)."
        )

    @staticmethod
    def _get_cache_block_size(
        vllm_config: "VllmConfig",
        kv_cache_config: "KVCacheConfig | None",
        cache_kv_group_id: int,
    ) -> int:
        """Block size of the hidden-states group, read from its own spec.

        cache_config.block_size is bumped to a common multiple for hybrid
        verifiers; the page-aligned hidden-states group keeps a smaller one.
        """
        if kv_cache_config is None:
            return vllm_config.cache_config.block_size

View on GitHub (pinned to c794754062)

Solutions

1. Configure the model so the hidden-states layer is isolated in its own KV cache group (own kv_cache_spec/group)
2. Do not use this connector with MLA verifiers — it is unsupported by design
3. Simplify to a single kv_cache_group, in which case group 0 is used unconditionally

Example fix

# before: hidden-states layer shares a group with attention layers -> ambiguous
# (model config yields groups=[attention+hidden])

# after: isolate the hidden-states layer so groups=[attention, hidden_states]
# e.g. ensure the extract-hidden-states layer maps to its own FullAttentionSpec/HiddenStateCacheSpec group

from vllm.v1.kv_cache_interface import HiddenStateCacheSpec
groups = kv_cache_config.kv_cache_groups
ids = [i for i, g in enumerate(groups) if isinstance(g.kv_cache_spec, HiddenStateCacheSpec)]
assert len(ids) == 1 or (not ids and len(groups) == 1), "hidden-states group not unique"

When it happens

Trigger: kv_cache_config contains zero or multiple HiddenStateCacheSpec groups while also having more than one group total; typically MLA models where hidden-states cannot be isolated, or multi-group hybrid configs where the hidden-states spec appears alongside other specs in a way that is not uniquely identifiable.

Common situations: Using the hidden-states KV transfer example with an MLA model as verifier (explicitly unsupported); a hybrid model config that splits the hidden-states layer into a shared group with attention layers.

Related errors

• HTTP request failed: {0}
• JSON error: {0}
• Tokenizer error: {0}
• tokenize endpoint unavailable: {0}
• --use-replayssm is incompatible with KV connectors (P/D disa

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/3e5abc21331a0bb4. Report an issue: GitHub.