vllm-project/vllm · error · RuntimeError

shape_id='{shape_id}' requires PyTorch >= 2.11.0

Error message

shape_id='{shape_id}' requires PyTorch >= 2.11.0

What it means

With unbacked dynamic shapes, vLLM can pass a shape_id to torch._dynamo.decorators.mark_unbacked so different tensors sharing a symbolic size get the same shape id. That keyword requires PyTorch 2.11+ (tracked by the _SUPPORTS_SHAPE_ID flag); on older torch (>=2.10) where shape_id exists in vLLM's API but not in PyTorch's mark_unbacked, vLLM raises RuntimeError.

Source

Thrown at vllm/compilation/decorators.py:425

            self,
            compile_prefix=cls.__name__ if is_encoder else "",
            is_encoder=is_encoder,
        )

    cls.__init__ = __init__

    def _mark_dynamic_inputs(
        mod: type[_T], ds_type: DynamicShapesType, *args: Any, **kwargs: Any
    ) -> None:
        def mark_dynamic(
            arg: torch.Tensor, dim_shape_pairs: list[tuple[int, str | None]]
        ) -> None:
            if ds_type == DynamicShapesType.UNBACKED:
                if is_torch_equal_or_newer("2.10.0"):
                    for dim, shape_id in dim_shape_pairs:
                        if shape_id is not None:
                            if not _SUPPORTS_SHAPE_ID:
                                raise RuntimeError(
                                    f"shape_id='{shape_id}' requires PyTorch >= 2.11.0"
                                )
                            torch._dynamo.decorators.mark_unbacked(
                                arg,
                                dim,
                                hint_override=arg.size()[dim],
                                shape_id=shape_id,
                            )
                        else:
                            torch._dynamo.decorators.mark_unbacked(
                                arg,
                                dim,
                                hint_override=arg.size()[dim],
                            )
                else:
                    # For older versions, we can't use hint_override or shape_id
                    dims = [dim for dim, _ in dim_shape_pairs]
                    torch._dynamo.decorators.mark_unbacked(arg, dims)

View on GitHub (pinned to c794754062)

Solutions

1. Upgrade PyTorch to 2.11.0 or newer.
2. Or remove shape_id entries from your dynamic-shape configuration so the plain mark_unbacked path is used.

Example fix

# before
# torch==2.10.x
@support_torch_compile(dynamic_shape_id={"hidden_states": {0: "tokens"}})
class L(nn.Module): ...
# after
pip install --upgrade "torch>=2.11.0"
# or drop shape_id:
@support_torch_compile(mark_unbacked_dims={"hidden_states": 0})
class L(nn.Module): ...

from vllm.utils import is_torch_equal_or_newer

def supports_shape_id() -> bool:
    return is_torch_equal_or_newer('2.11.0')
# only configure dynamic_shape_id when supports_shape_id() is True

When it happens

Trigger: Configuring dynamic_shape_id (supplying shape ids for dims in the decorator / DynamicShapesType.UNBACKED path) while running PyTorch 2.10.x, where is_torch_equal_or_newer('2.10.0') is True but _SUPPORTS_SHAPE_ID is False.

Common situations: Newer vLLM features (shape_id-based symbolic size sharing) used on a pinned older PyTorch 2.10 wheel; CI with a frozen torch version while model configs adopt shape ids.

Related errors

• No dynamic dimensions found in the forward method of {cls}.
• Argument {k} not found in the forward method of {cls}
• Unsupported dynamic dimensions {dims} for argument {k} with
• aot_compile is not supported by the current configuration. P
• vLLM failed to compile the model. The most likely reason for

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8983ebae3a260fd7. Report an issue: GitHub.