{"record":{"id":"411727a4f51ea1e0","repo":"vllm-project/vllm","slug":"pipeline-parallelism-is-not-supported-for-this-mod","errorCode":null,"errorMessage":"Pipeline parallelism is not supported for this model. Supported models implement the `SupportsPP` interface.","messagePattern":"Pipeline parallelism is not supported for this model\\. Supported models implement the `SupportsPP` interface\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":1381,"sourceCode":"        parallel_config: ParallelConfig,\n    ) -> None:\n        total_num_attention_heads = self.model_arch_config.total_num_attention_heads\n        tensor_parallel_size = parallel_config.tensor_parallel_size\n        if total_num_attention_heads % tensor_parallel_size != 0:\n            raise ValueError(\n                f\"Total number of attention heads ({total_num_attention_heads})\"\n                \" must be divisible by tensor parallel size \"\n                f\"({tensor_parallel_size}).\"\n            )\n\n        if parallel_config.enable_expert_parallel:\n            self._verify_with_expert_parallelism()\n\n        pipeline_parallel_size = parallel_config.pipeline_parallel_size\n        if pipeline_parallel_size > 1 and not self.registry.is_pp_supported_model(\n            self.architectures, self\n        ):\n            raise NotImplementedError(\n                \"Pipeline parallelism is not supported for this model. \"\n                \"Supported models implement the `SupportsPP` interface.\"\n            )\n\n        decode_context_parallel_size = parallel_config.decode_context_parallel_size\n        if decode_context_parallel_size > 1 and not self.use_mla:\n            total_num_kv_heads = self.get_total_num_kv_heads()\n            if tensor_parallel_size <= total_num_kv_heads:\n                raise ValueError(\n                    \"Decode context parallelism for GQA/MQA requires \"\n                    f\"`--tensor-parallel-size` ({tensor_parallel_size}) to be \"\n                    \"greater than the model's total number of KV heads \"\n                    f\"({total_num_kv_heads}). Increase `--tensor-parallel-size` \"\n                    \"or set `--decode-context-parallel-size 1`.\"\n                )\n\n            max_dcp_size = tensor_parallel_size // total_num_kv_heads\n            if decode_context_parallel_size > max_dcp_size:","sourceCodeStart":1363,"sourceCodeEnd":1399,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L1363-L1399","documentation":"Raised when pipeline_parallel_size > 1 but the model's implementation does not implement the SupportsPP interface (checked via registry.is_pp_supported_model). PP requires the model class to opt in by exposing per-layer information vLLM needs to split stages.","triggerScenarios":"Launching with --pipeline-parallel-size > 1 for an architecture whose registry entry does not implement SupportsPP; NotImplementedError raised from verify_with_parallel_config.","commonSituations":"Trying PP to fit a large dense model across nodes and discovering that architecture never got PP support; new/custom model implementations registered without SupportsPP; version differences where PP support was added or removed for a family of models.","solutions":["Set --pipeline-parallel-size 1 and use tensor parallelism (or data parallelism) for scaling instead.","Use a checkpoint/architecture that implements SupportsPP (most mainstream LLMs do; check the model class for the interface).","For a custom model, implement the SupportsPP interface on the model implementation so the registry recognizes it."],"exampleFix":"# before\nvllm serve my-unsupported-model --pipeline-parallel-size 2\n# after\nvllm serve my-unsupported-model --tensor-parallel-size 2","handlingStrategy":"validation","validationCode":"from vllm.model_executor.models.registry import _MULTIMODAL_MODELS  # pattern only\ndef pp_supported(registry, architectures, model_config) -> bool:\n    return registry.is_pp_supported_model(architectures, model_config)\n# gate --pipeline-parallel-size > 1 on this check before engine init","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check the model implementation class for the SupportsPP interface before requesting PP.","Default to TP for scaling; treat PP as opt-in per architecture.","Re-verify PP support after vLLM upgrades — model support matrices change between releases."],"tags":["pipeline-parallelism","model-support","config","startup"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}