vllm-project/vllm · error · TypeError

decorated class should have a forward method.

Error message

decorated class should have a forward method.

What it means

The @support_torch_compile decorator (vllm.compilation.decorators) requires the decorated nn.Module subclass to define a forward method, because it inspects forward's signature to infer dynamic dimensions and to build the compiled callable. Decorating a class without forward is a programming error and raises TypeError immediately at decoration time.

Source

Thrown at vllm/compilation/decorators.py:205

    `is_encoder` marks this module as a portion of an multimodal encoder.
    When True, the compile range upper bound is set to MAX_INT32 instead of
    max_num_batched_tokens, since encoder input shapes are unpredictable.
    This is typically used for vision encoder sub-modules in multimodal models.

    `shape_invariants` is a function that gets compiled right before forward.
    The function should have the torch._check calls that are needed to set
    the relationships between different input sizes. For example:
            torch._check(input_ids.size()[0] == inputs_embeds.size()[0])
    This enforces constraints on the symbolic shapes without hardcoding
    specific values. It is needed for some models to avoid data dependent
    errors and maximize perf when unbacked shapes are used.
    """

    def cls_decorator_helper(cls: type[_T]) -> type[_T]:
        # helper to pass `dynamic_arg_dims` to `_support_torch_compile`
        # to avoid too much indentation for `_support_torch_compile`
        if not hasattr(cls, "forward"):
            raise TypeError("decorated class should have a forward method.")
        sig = inspect.signature(cls.forward)
        inferred_dynamic_arg_dims = dynamic_arg_dims
        if inferred_dynamic_arg_dims is None:
            inferred_dynamic_arg_dims = {}
            for k, v in sig.parameters.items():
                if v.annotation in [
                    torch.Tensor,
                    torch.Tensor | None,
                    torch.FloatTensor,
                    torch.FloatTensor | None,
                    IntermediateTensors,
                    IntermediateTensors | None,
                ]:
                    inferred_dynamic_arg_dims[k] = 0

            logger.debug(
                ("Inferred dynamic dimensions for forward method of %s: %s"),
                cls,

View on GitHub (pinned to c794754062)

Solutions

1. Define a forward method on the decorated class with the standard vLLM forward signature (hidden_states, positions, intermediate_tensors, ...).
2. If the logic is in another method, rename it to forward or have forward delegate to it.
3. Check for typos in the method name.

Example fix

# before
@support_torch_compile
class MyDecoderLayer(nn.Module):
    def __call__(self, hidden_states, positions): ...
# after
@support_torch_compile
class MyDecoderLayer(nn.Module):
    def forward(self, hidden_states, positions): ...

import inspect

def check_compile_decorated(cls) -> None:
    assert hasattr(cls, 'forward') and callable(cls.forward), (
        f'{cls.__name__} must define forward() for @support_torch_compile')
    check_compile_decorated(MyLayer)

def is_compile_ready(cls: type) -> bool:
    return callable(getattr(cls, 'forward', None))

When it happens

Trigger: Applying @support_torch_compile (directly or via a helper like support_torch_compile(dynamic_arg_dims=...)) to a class that defines __call__ but not forward, or whose forward is defined on a misspelled method (e.g. fwd, forwards).

Common situations: Porting a custom model into vLLM where inference logic lives in __call__ or a custom run() method; renaming forward during refactoring and forgetting the decorator contract; copy-paste from a non-vLLM module.

Related errors

• No dynamic dimensions found in the forward method of {cls}.
• Argument {k} not found in the forward method of {cls}
• {init} received a positional argument of type {arg_type}, bu
• vLLM failed to compile the model. The most likely reason for
• shape_id='{shape_id}' requires PyTorch >= 2.11.0

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/07c8b4721fbbc620. Report an issue: GitHub.