vllm-project/vllm · error · ValueError

'tensor_parallel_size' is not a valid argument in the specul

Error message

'tensor_parallel_size' is not a valid argument in the speculative_config. Please pass 'draft_tensor_parallel_size' instead.

What it means

Raised by the SpeculativeConfig._verify_args model_validator when a 'tensor_parallel_size' key is present in the speculative_config. The field exists on the class only for internal/legacy reasons; users must express draft sharding via draft_tensor_parallel_size (the speculative_draft_* alias chain resolves to it). Passing the wrong key is rejected so silent no-ops cannot occur.

Source

Thrown at vllm/config/speculative.py:1347

            ray_workers_use_nsight=target_parallel_config.ray_workers_use_nsight,
            placement_group=target_parallel_config.placement_group,
        )

        return draft_parallel_config

    @field_validator("attention_backend", mode="before")
    @classmethod
    def _parse_attention_backend(cls, value: Any) -> Any:
        if isinstance(value, str):
            if value.lower() == "auto":
                return None
            return AttentionBackendEnum[value.upper()]
        return value

    @model_validator(mode="after")
    def _verify_args(self) -> Self:
        if self.tensor_parallel_size is not None:
            raise ValueError(
                "'tensor_parallel_size' is not a valid argument in the "
                "speculative_config. Please pass 'draft_tensor_parallel_size' instead."
            )

        if self.num_speculative_tokens is None:
            raise ValueError(
                "num_speculative_tokens must be provided with "
                "speculative model unless the draft model config contains an "
                "n_predict parameter."
            )

        if self.num_speculative_tokens <= 0:
            raise ValueError(
                "Expected num_speculative_tokens to be greater "
                f"than zero ({self.num_speculative_tokens})."
            )

        if self.rejection_sample_method == "synthetic":

View on GitHub (pinned to c794754062)

Solutions

1. Rename the key to draft_tensor_parallel_size in the speculative_config
2. Move TP control to the top-level --tensor-parallel-size flag for the target and omit it from speculative_config

Example fix

# before
speculative_config={"method": "eagle", "model": "...", "tensor_parallel_size": 4}
# after
speculative_config={"method": "eagle", "model": "...", "draft_tensor_parallel_size": 4}

if "tensor_parallel_size" in spec_cfg:
    spec_cfg["draft_tensor_parallel_size"] = spec_cfg.pop("tensor_parallel_size")

def uses_draft_tp_key(spec_cfg: dict) -> bool:
    return "tensor_parallel_size" not in spec_cfg and "draft_tensor_parallel_size" in spec_cfg or "tensor_parallel_size" not in spec_cfg

When it happens

Trigger: speculative_config={'method': 'eagle', 'model': '...', 'tensor_parallel_size': 4} — typically written by analogy with the top-level engine config or copied from an old vLLM example.

Common situations: Migrating pre-refactor speculative configs that used tensor_parallel_size; users assuming every sub-config mirrors the top-level V0/EngineArgs naming.

Related errors

• {speculative_draft_tensor_parallel_size=} cannot be other va
• Total number of attention heads ({total_num_attention_heads}
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/00e5a73221028a48. Report an issue: GitHub.