vllm-project/vllm · error · ValueError

--enable-return-routed-experts is incompatible with pipeline

Error message

--enable-return-routed-experts is incompatible with pipeline parallelism (PP > 1).

What it means

VllmConfig rejects --enable-return-routed-experts (model_config.enable_return_routed_experts, which returns per-token expert-routing decisions for MoE analysis) when pipeline_parallel_size > 1. The routed-experts buffer is indexed by request slot across a single engine instance; with PP > 1 the request is split across pipeline stages and the routing captured on one rank cannot be assembled into the slot-indexed response on the others. The check runs in __post_init__ right after parallel-config verification.

Source

Thrown at vllm/config/vllm.py:1073

        self.instance_id = f"{time.time_ns()}"

        if self.performance_mode != "balanced":
            logger.info_once("Performance mode set to '%s'.", self.performance_mode)

        self.try_verify_and_update_config()

        if self.model_config is not None:
            self.model_config.verify_with_parallel_config(self.parallel_config)
            self.model_config.verify_dual_chunk_attention_config(self.load_config)

            self.parallel_config.is_moe_model = self.model_config.is_moe

        if (
            self.model_config is not None
            and self.model_config.enable_return_routed_experts
        ):
            if self.parallel_config.pipeline_parallel_size > 1:
                raise ValueError(
                    "--enable-return-routed-experts is incompatible with "
                    "pipeline parallelism (PP > 1)."
                )
            if (
                self.parallel_config.decode_context_parallel_size > 1
                or self.parallel_config.prefill_context_parallel_size > 1
            ):
                raise ValueError(
                    "--enable-return-routed-experts is incompatible with context "
                    "parallelism (DCP > 1 or PCP > 1)."
                )

            # Incompatible with any KV connector — covers both PD disaggregation
            # (kv_producer/kv_consumer: routing captured on P can't reach D) and
            # single-instance KV offload/sharing (kv_both: slot_mapping semantics
            # change when KV blocks live outside local GPU memory, breaking the
            # slot-indexed routed_experts buffer).
            if (

View on GitHub (pinned to c794754062)

Solutions

1. Drop pipeline parallelism (set pipeline_parallel_size=1) and use tensor/EP parallelism instead to fit the model
2. Disable --enable-return-routed-experts on PP deployments; collect routing from a separate non-PP analysis instance
3. Sample routing offline via instrumentation (hooks/dump of router logits) rather than the serving-path API

Example fix

# before
vllm serve moe-model --enable-return-routed-experts -pp 4 -tp 2
# after
vllm serve moe-model --enable-return-routed-experts -tp 8  # no PP

if model_config.enable_return_routed_experts:
    assert parallel_config.pipeline_parallel_size == 1, \
        "routed-experts return incompatible with PP > 1"

def routed_experts_pp_ok(enabled: bool, pp: int) -> bool:
    return not enabled or pp == 1

When it happens

Trigger: Launching a large MoE model sharded across pipeline stages (e.g. --pipeline-parallel-size 4) together with --enable-return-routed-experts.

Common situations: Serving a big MoE (DeepSeek/Mixtral-class) that needs PP to fit in memory while also wanting expert-routing telemetry; enabling the observability flag fleet-wide including multi-node PP deployments; research setups collecting routing statistics from production-shaped clusters.

Related errors

• --enable-return-routed-experts is incompatible with context
• Number of experts in the model must be greater than 0 when e
• Pipeline parallelism is not supported for this model. Suppor
• Async EPLB is only supported with the default policy.
• {self.communicator} communicator is incompatible with async

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/6ccefdc2467b901f. Report an issue: GitHub.