{"record":{"id":"07c8b4721fbbc620","repo":"vllm-project/vllm","slug":"decorated-class-should-have-a-forward-method","errorCode":null,"errorMessage":"decorated class should have a forward method.","messagePattern":"decorated class should have a forward method\\.","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/decorators.py","lineNumber":205,"sourceCode":"    `is_encoder` marks this module as a portion of an multimodal encoder.\n    When True, the compile range upper bound is set to MAX_INT32 instead of\n    max_num_batched_tokens, since encoder input shapes are unpredictable.\n    This is typically used for vision encoder sub-modules in multimodal models.\n\n    `shape_invariants` is a function that gets compiled right before forward.\n    The function should have the torch._check calls that are needed to set\n    the relationships between different input sizes. For example:\n            torch._check(input_ids.size()[0] == inputs_embeds.size()[0])\n    This enforces constraints on the symbolic shapes without hardcoding\n    specific values. It is needed for some models to avoid data dependent\n    errors and maximize perf when unbacked shapes are used.\n    \"\"\"\n\n    def cls_decorator_helper(cls: type[_T]) -> type[_T]:\n        # helper to pass `dynamic_arg_dims` to `_support_torch_compile`\n        # to avoid too much indentation for `_support_torch_compile`\n        if not hasattr(cls, \"forward\"):\n            raise TypeError(\"decorated class should have a forward method.\")\n        sig = inspect.signature(cls.forward)\n        inferred_dynamic_arg_dims = dynamic_arg_dims\n        if inferred_dynamic_arg_dims is None:\n            inferred_dynamic_arg_dims = {}\n            for k, v in sig.parameters.items():\n                if v.annotation in [\n                    torch.Tensor,\n                    torch.Tensor | None,\n                    torch.FloatTensor,\n                    torch.FloatTensor | None,\n                    IntermediateTensors,\n                    IntermediateTensors | None,\n                ]:\n                    inferred_dynamic_arg_dims[k] = 0\n\n            logger.debug(\n                (\"Inferred dynamic dimensions for forward method of %s: %s\"),\n                cls,","sourceCodeStart":187,"sourceCodeEnd":223,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/decorators.py#L187-L223","documentation":"The @support_torch_compile decorator (vllm.compilation.decorators) requires the decorated nn.Module subclass to define a forward method, because it inspects forward's signature to infer dynamic dimensions and to build the compiled callable. Decorating a class without forward is a programming error and raises TypeError immediately at decoration time.","triggerScenarios":"Applying @support_torch_compile (directly or via a helper like support_torch_compile(dynamic_arg_dims=...)) to a class that defines __call__ but not forward, or whose forward is defined on a misspelled method (e.g. fwd, forwards).","commonSituations":"Porting a custom model into vLLM where inference logic lives in __call__ or a custom run() method; renaming forward during refactoring and forgetting the decorator contract; copy-paste from a non-vLLM module.","solutions":["Define a forward method on the decorated class with the standard vLLM forward signature (hidden_states, positions, intermediate_tensors, ...).","If the logic is in another method, rename it to forward or have forward delegate to it.","Check for typos in the method name."],"exampleFix":"# before\n@support_torch_compile\nclass MyDecoderLayer(nn.Module):\n    def __call__(self, hidden_states, positions): ...\n# after\n@support_torch_compile\nclass MyDecoderLayer(nn.Module):\n    def forward(self, hidden_states, positions): ...","handlingStrategy":"validation","validationCode":"import inspect\n\ndef check_compile_decorated(cls) -> None:\n    assert hasattr(cls, 'forward') and callable(cls.forward), (\n        f'{cls.__name__} must define forward() for @support_torch_compile')\n    check_compile_decorated(MyLayer)","typeGuard":"def is_compile_ready(cls: type) -> bool:\n    return callable(getattr(cls, 'forward', None))","tryCatchPattern":null,"preventionTips":["Always define forward() on modules meant for vLLM serving","Run a smoke instantiation in unit tests before serving","Lint custom model ports for missing forward methods"],"tags":["torch-compile","decorators","vllm","model-porting"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}