vllm-project/vllm · error · ValueError

--enable-return-routed-experts is incompatible with KV conne

Error message

--enable-return-routed-experts is incompatible with KV connectors (PD disaggregation, KV cache offload).

What it means

The final routed-experts guard rejects any KV-transfer connector instance (kv_transfer_config.is_kv_transfer_instance). It covers both PD disaggregation (kv_producer/kv_consumer — routing captured on the prefill instance can't reach the decode instance that answers the request) and single-instance KV offload/sharing (kv_both — slot_mapping semantics change when KV blocks live outside local GPU memory, breaking the slot-indexed routed_experts buffer). The in-code comment documents both failure modes explicitly.

Source

Thrown at vllm/config/vllm.py:1095

            if (
                self.parallel_config.decode_context_parallel_size > 1
                or self.parallel_config.prefill_context_parallel_size > 1
            ):
                raise ValueError(
                    "--enable-return-routed-experts is incompatible with context "
                    "parallelism (DCP > 1 or PCP > 1)."
                )

            # Incompatible with any KV connector — covers both PD disaggregation
            # (kv_producer/kv_consumer: routing captured on P can't reach D) and
            # single-instance KV offload/sharing (kv_both: slot_mapping semantics
            # change when KV blocks live outside local GPU memory, breaking the
            # slot-indexed routed_experts buffer).
            if (
                self.kv_transfer_config is not None
                and self.kv_transfer_config.is_kv_transfer_instance
            ):
                raise ValueError(
                    "--enable-return-routed-experts is incompatible with KV "
                    "connectors (PD disaggregation, KV cache offload)."
                )

        self._verify_sampling_replay_config()

        if self.lora_config is not None:
            self.lora_config.verify_with_model_config(self.model_config)

        if (
            self.mamba_config.enable_stochastic_rounding
            and self.cache_config.mamba_ssm_cache_dtype != "float16"
        ):
            raise ValueError(
                "Stochastic rounding for Mamba cache requires "
                "the SSM cache to be float16. Please set it explicitly, "
                "by specifying `--mamba-ssm-cache-dtype float16`, or disable "
                "stochastic rounding by not specifying "

View on GitHub (pinned to c794754062)

Solutions

  1. Disable --enable-return-routed-experts on any instance with a kv_transfer_config role
  2. Run a separate non-disaggregated replica without KV connectors for routing analysis
  3. Dump router logits/expert ids offline from the model forward instead of using the request-path API

Example fix

# before
vllm serve moe-model --enable-return-routed-experts \
  --kv-transfer-config '{"kv_role":"kv_producer", ...}'
# after
vllm serve moe-model --enable-return-routed-experts  # no kv-transfer-config
Defensive patterns

Strategy: validation

Validate before calling

if model_config.enable_return_routed_experts:
    assert kv_transfer_config is None or not kv_transfer_config.is_kv_transfer_instance, \
        "routed-experts return incompatible with KV connectors"

Type guard

def routed_experts_kv_ok(enabled: bool, kv_cfg) -> bool:
    return not enabled or kv_cfg is None or not kv_cfg.is_kv_transfer_instance

Prevention

When it happens

Trigger: Running --enable-return-routed-experts together with any --kv-transfer-config role (producer, consumer, or both), e.g. a P/D-disaggregated DeepSeek serving stack with routing telemetry enabled.

Common situations: Adding expert-routing observability to a disaggregated serving cluster (NIXL/Mooncake PD); KV-offload setups for sleep/wake memory management; assuming the flag is purely local and orthogonal to KV plumbing.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/6aa4346c0e6f30b4. Report an issue: GitHub.