vllm-project/vllm · error · ValueError

No dynamic dimensions found in the forward method of {cls}.

Error message

No dynamic dimensions found in the forward method of {cls}. Please provide dynamic_arg_dims explicitly.

What it means

When dynamic_arg_dims is not supplied, the decorator infers dynamic dimensions by scanning forward's parameters whose type annotation is a tensor-like type (torch.Tensor, optional tensors, IntermediateTensors) and marking dim 0 dynamic. If no parameter carries such an annotation, inference produces an empty dict and vLLM raises ValueError asking you to pass dynamic_arg_dims explicitly.

Source

Thrown at vllm/compilation/decorators.py:228

            for k, v in sig.parameters.items():
                if v.annotation in [
                    torch.Tensor,
                    torch.Tensor | None,
                    torch.FloatTensor,
                    torch.FloatTensor | None,
                    IntermediateTensors,
                    IntermediateTensors | None,
                ]:
                    inferred_dynamic_arg_dims[k] = 0

            logger.debug(
                ("Inferred dynamic dimensions for forward method of %s: %s"),
                cls,
                list(inferred_dynamic_arg_dims.keys()),
            )

        if len(inferred_dynamic_arg_dims) == 0:
            raise ValueError(
                "No dynamic dimensions found in the forward method of "
                f"{cls}. Please provide dynamic_arg_dims explicitly."
            )

        for k in inferred_dynamic_arg_dims:
            if k not in sig.parameters:
                raise ValueError(
                    f"Argument {k} not found in the forward method of {cls}"
                )

        return _support_torch_compile(
            cls,
            inferred_dynamic_arg_dims,
            mark_unbacked_dims,
            enable_if,
            is_encoder,
        )

View on GitHub (pinned to c794754062)

Solutions

1. Pass dynamic_arg_dims explicitly, e.g. @support_torch_compile(dynamic_arg_dims={"hidden_states": 0, "positions": 0}).
2. Or annotate the tensor parameters of forward with torch.Tensor / torch.Tensor | None / IntermediateTensors so inference finds them.
3. Verify each key you list matches a real parameter name in forward.

Example fix

# before
@support_torch_compile
class MyLayer(nn.Module):
    def forward(self, hidden_states, positions): ...
# after
@support_torch_compile(dynamic_arg_dims={"hidden_states": 0})
class MyLayer(nn.Module):
    def forward(self, hidden_states, positions): ...

import inspect, torch
from vllm.distributed import IntermediateTensors

TENSOR_TYPES = {torch.Tensor, torch.Tensor | None, torch.FloatTensor, torch.FloatTensor | None, IntermediateTensors, IntermediateTensors | None}

def needs_explicit_dynamic_args(cls) -> bool:
    sig = inspect.signature(cls.forward)
    return not any(p.annotation in TENSOR_TYPES for p in sig.parameters.values())
# if needs_explicit_dynamic_args(MyLayer): pass dynamic_arg_dims=...

def forward_has_tensor_params(cls) -> bool:
    return any(p.annotation in TENSOR_TYPES for p in inspect.signature(cls.forward).parameters.values())

When it happens

Trigger: Using @support_torch_compile on a class whose forward parameters are unannotated (no type hints), or annotated only with non-tensor types, without providing the dynamic_arg_dims argument to the decorator.

Common situations: Custom models written without type annotations; forward signatures using generic Any or dict annotations; encoder/multimodal wrappers whose tensor inputs are buried in kwargs.

Related errors

• Argument {k} not found in the forward method of {cls}
• decorated class should have a forward method.
• {init} received a positional argument of type {arg_type}, bu
• shape_id='{shape_id}' requires PyTorch >= 2.11.0
• Unsupported dynamic dimensions {dims} for argument {k} with

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/97fcea18962be3be. Report an issue: GitHub.