vllm-project/vllm · error · ValueError

DeepSeek V4 does not support PIECEWISE CUDA graphs with Mode

Error message

DeepSeek V4 does not support PIECEWISE CUDA graphs with Model Runner V1. Use Model Runner V2 or disable PIECEWISE CUDA graphs.

What it means

VllmConfig raises this when Model Runner V1 is active, the compilation config enables PIECEWISE CUDA graphs, and the model architecture is DeepseekV4ForCausalLM (the sole entry in MRV1_UNSUPPORTED_PIECEWISE_CUDAGRAPH_ARCHITECTURES). DeepSeek V4's piecewise-split graph is only implemented for the V2 model runner, so V1 + piecewise would crash at capture time; the config check fails fast instead. Note the guard is skipped entirely when use_v2_model_runner is true, when there is no model config, or when cudagraph_mode lacks piecewise graphs.

Source

Thrown at vllm/config/vllm.py:707

        if getattr(model_config, "is_attention_free", False):
            return False
        return is_default_v2_architecture or not model_config.is_moe

    def _validate_mrv1_piecewise_cudagraph(self) -> None:
        if self.use_v2_model_runner:
            return
        model_config = self.model_config
        if model_config is None:
            return
        if not self.compilation_config.cudagraph_mode.has_piecewise_cudagraphs():
            return
        architectures = getattr(model_config, "architectures", [])
        if any(
            arch in MRV1_UNSUPPORTED_PIECEWISE_CUDAGRAPH_ARCHITECTURES
            for arch in architectures
        ):
            raise ValueError(
                "DeepSeek V4 does not support PIECEWISE CUDA graphs with "
                "Model Runner V1. Use Model Runner V2 or disable PIECEWISE "
                "CUDA graphs."
            )

    @property
    def needs_dp_coordinator(self) -> bool:
        """
        Determine if the DPCoordinator process is needed.

        The DPCoordinator is needed in two cases:
        1. For MoE models with DP > 1: to handle wave coordination
           (even in external LB mode, since wave coordination runs in the coordinator)
        2. For non-MoE models in internal/hybrid LB mode: to collect and publish
           queue stats for load balancing across DP ranks

        Returns:
            True if DPCoordinator process is needed, False otherwise.

View on GitHub (pinned to c794754062)

Solutions

1. Enable Model Runner V2 (e.g. set the V2 model runner flag / VLLM_USE_V2_MODEL_RUNNER or the config that sets use_v2_model_runner) for DeepSeek V4
2. Or disable piecewise CUDA graphs: set cudagraph_mode to FULL/NONE (e.g. --compilation-config '{"cudagraph_mode": "FULL"}') or run with enforce_eager
3. Upgrade vLLM so DeepSeek V4 defaults to the V2 runner (it is listed in DEFAULT_V2_MODEL_RUNNER_ARCHITECTURES)

Example fix

# before
vllm serve deepseek-ai/DeepSeek-V4 --use-v1-model-runner
# after
vllm serve deepseek-ai/DeepSeek-V4  # V2 model runner is the default for this arch
# or: --compilation-config '{"cudagraph_mode": "FULL"}'

arch = model_config.architectures[0] if model_config.architectures else ""
if arch == "DeepseekV4ForCausalLM":
    assert use_v2_model_runner or not compilation_config.cudagraph_mode.has_piecewise_cudagraphs(), \
        "DeepSeek V4 + MRV1 requires disabling piecewise cudagraphs"

def deepseek_v4_runner_ok(arch: str, use_v2: bool, piecewise: bool) -> bool:
    return use_v2 or arch != "DeepseekV4ForCausalLM" or not piecewise

When it happens

Trigger: Running DeepseekV4ForCausalLM with default Model Runner V1 while compilation_config.cudagraph_mode includes PIECEWISE (the default full+piecewise mode), e.g. vllm serve <deepseek-v4> without V2 flags or with enforce_eager disabling only full graphs.

Common situations: Loading a new DeepSeek V4 checkpoint on an older engine default (V1 runner); explicitly forcing V1 via config while keeping default cudagraph_mode; an environment where DEFAULT_V2_MODEL_RUNNER_ARCHITECTURES was not applied (force-v1 overrides).

Related errors

• sampling distribution replay requires Model Runner V2
• {kind} parsing is not available for model `{model_id}`
• {kind} parsing is disabled by frontend configuration
• cannot use in-process coordinator with bootstrapped transpor
• ❌ line({node.lineno}): {message}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/f6b934336a7beb58. Report an issue: GitHub.